From python-checkins at python.org Thu Sep 1 02:00:46 2016 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 01 Sep 2016 06:00:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Minor_beautification_=28tu?= =?utf-8?q?rn_nested-if_into_a_conjunction=29=2E?= Message-ID: <20160901060045.98042.30896.D62668D1@psf.io> https://hg.python.org/cpython/rev/ebb23744a36c changeset: 102984:ebb23744a36c user: Raymond Hettinger date: Wed Aug 31 23:00:32 2016 -0700 summary: Minor beautification (turn nested-if into a conjunction). files: Lib/random.py | 11 +++++------ 1 files changed, 5 insertions(+), 6 deletions(-) diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -119,12 +119,11 @@ x ^= len(a) a = -2 if x == -1 else x - if version == 2: - if isinstance(a, (str, bytes, bytearray)): - if isinstance(a, str): - a = a.encode() - a += _sha512(a).digest() - a = int.from_bytes(a, 'big') + if version == 2 and isinstance(a, (str, bytes, bytearray)): + if isinstance(a, str): + a = a.encode() + a += _sha512(a).digest() + a = int.from_bytes(a, 'big') super().seed(a) self.gauss_next = None -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 06:13:14 2016 From: python-checkins at python.org (python-checkins at python.org) Date: Thu, 01 Sep 2016 10:13:14 -0000 Subject: [Python-checkins] Foram realizadas 3 tentativas de entrega - - [ 372386835 ] Message-ID: An HTML attachment was scrubbed... URL: From lp_benchmark_robot at intel.com Thu Sep 1 08:40:11 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 1 Sep 2016 13:40:11 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7 2016-09-01 Message-ID: <8cd4a31c-701b-49ba-8dad-56c483902be9@irsmsx106.ger.corp.intel.com> Results for project Python 2.7, build date 2016-09-01 02:47:02 +0000 commit: bf0cb86c6219 previous commit: f478f9b88319 revision date: 2016-09-01 01:03:08 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.14% 0.44% 5.48% 3.19% :-) pybench 0.22% -0.03% 6.28% 4.15% :-( regex_v8 0.60% 0.10% -2.16% 10.48% :-) nbody 0.10% -0.12% 8.15% 2.78% :-) json_dump_v2 0.31% 0.52% 2.69% 10.23% :-| normal_startup 0.51% -0.11% -0.85% 1.85% :-) ssbench 0.14% -0.46% 2.75% 1.39% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-09-01/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Thu Sep 1 08:41:15 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 1 Sep 2016 13:41:15 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python Default 2016-09-01 Message-ID: Results for project Python default, build date 2016-09-01 02:00:19 +0000 commit: fc711879c64a previous commit: 059f9f518834 revision date: 2016-09-01 01:09:02 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.21% 0.12% 10.18% 15.43% :-) pybench 0.10% 0.26% 6.49% 7.00% :-( regex_v8 2.79% 1.15% -2.80% 8.90% :-| nbody 0.14% 0.16% -1.13% 11.65% :-( json_dump_v2 0.36% -0.23% -2.21% 12.48% :-| normal_startup 0.77% -0.36% 1.75% 6.54% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-09-01/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Thu Sep 1 13:56:00 2016 From: python-checkins at python.org (jason.coombs) Date: Thu, 01 Sep 2016 17:56:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327919=3A_Deprecat?= =?utf-8?q?e_extra=5Fpath_option_in_distutils=2E?= Message-ID: <20160901175540.22157.732.45F679A7@psf.io> https://hg.python.org/cpython/rev/94710cbcac47 changeset: 102985:94710cbcac47 user: Jason R. Coombs date: Thu Sep 01 13:55:33 2016 -0400 summary: Issue #27919: Deprecate extra_path option in distutils. files: Doc/whatsnew/3.6.rst | 6 ++++++ Lib/distutils/command/install.py | 6 ++++++ Misc/NEWS | 3 +++ 3 files changed, 15 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -782,6 +782,12 @@ now deprecated. (Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.) +* The undocumented ``extra_path`` argument to a distutils Distribution + is now considered + deprecated, will raise a warning during install if set. Support for this + parameter will be dropped in a future Python release and likely earlier + through third party tools. See :issue:`27919` for details. + Deprecated Python behavior -------------------------- diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py --- a/Lib/distutils/command/install.py +++ b/Lib/distutils/command/install.py @@ -175,6 +175,7 @@ self.compile = None self.optimize = None + # Deprecated # These two are for putting non-packagized distributions into their # own directory and creating a .pth file if it makes sense. # 'extra_path' comes from the setup file; 'install_path_file' can @@ -344,6 +345,7 @@ 'scripts', 'data', 'headers', 'userbase', 'usersite') + # Deprecated # Well, we're not actually fully completely finalized yet: we still # have to deal with 'extra_path', which is the hack for allowing # non-packagized module distributions (hello, Numerical Python!) to @@ -490,6 +492,10 @@ self.extra_path = self.distribution.extra_path if self.extra_path is not None: + log.warn( + "Distribution option extra_path is deprecated. " + "See issue27919 for details." + ) if isinstance(self.extra_path, str): self.extra_path = self.extra_path.split(',') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,9 @@ Library ------- +- Issue #27919: Deprecated ``extra_path`` distribution option in distutils + packaging. + - Issue #23229: Add new ``cmath`` constants: ``cmath.inf`` and ``cmath.nan`` to match ``math.inf`` and ``math.nan``, and also ``cmath.infj`` and ``cmath.nanj`` to match the format used by complex repr. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 14:25:47 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 01 Sep 2016 18:25:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3ODg4?= =?utf-8?q?=3A_Prevent_Windows_installer_from_displaying_console_windows_a?= =?utf-8?q?nd?= Message-ID: <20160901182546.98457.65342.EAE20F18@psf.io> https://hg.python.org/cpython/rev/e065aec0e6fa changeset: 102986:e065aec0e6fa branch: 2.7 parent: 102981:bf0cb86c6219 user: Steve Dower date: Thu Sep 01 11:21:56 2016 -0700 summary: Issue #27888: Prevent Windows installer from displaying console windows and failing when pip cannot be installed/uninstalled. files: .hgeol | 1 + Misc/NEWS | 5 +++ Tools/msi/WixCA.blob | Bin Tools/msi/msi.py | 44 +++++++++++++++++++----------- 4 files changed, 34 insertions(+), 16 deletions(-) diff --git a/.hgeol b/.hgeol --- a/.hgeol +++ b/.hgeol @@ -14,6 +14,7 @@ **.aiff = BIN **.au = BIN **.bmp = BIN +**.blob = BIN **.db = BIN **.exe = BIN **.icns = BIN diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -131,6 +131,11 @@ - Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. Also update FreedBSD version checks for the original ctype UTF-8 workaround. +Windows +------- + +- Issue #27888: Prevent Windows installer from displaying console windows and + failing when pip cannot be installed/uninstalled. What's New in Python 2.7.12? ============================ diff --git a/Tools/msi/WixCA.blob b/Tools/msi/WixCA.blob new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..171d3094effbba5f86f60da0ed00791d0cc87025 GIT binary patch [stripped] diff --git a/Tools/msi/msi.py b/Tools/msi/msi.py --- a/Tools/msi/msi.py +++ b/Tools/msi/msi.py @@ -376,6 +376,7 @@ # the installed/uninstalled state according to both the # Extensions and TclTk features. add_data(db, "Binary", [("Script", msilib.Binary("msisupport.dll"))]) + add_data(db, "Binary", [("WixCA", msilib.Binary("WixCA.blob"))]) # See "Custom Action Type 1" if msilib.Win64: CheckDir = "CheckDir" @@ -407,10 +408,11 @@ ("VerdanaRed9", "Verdana", 9, 255, 0), ]) - compileargs = r'-Wi "[TARGETDIR]Lib\compileall.py" -f -x "bad_coding|badsyntax|site-packages|py3_" "[TARGETDIR]Lib"' - lib2to3args = r'-c "import lib2to3.pygram, lib2to3.patcomp;lib2to3.patcomp.PatternCompiler()"' - updatepipargs = r'-m ensurepip -U --default-pip' - removepipargs = r'-B -m ensurepip._uninstall' + compileargs = r'"[#python.exe]" -Wi "[TARGETDIR]Lib\compileall.py" -f -x "bad_coding|badsyntax|site-packages|py3_" "[TARGETDIR]Lib"' + compileoargs = r'"[#python.exe]" -O -Wi "[TARGETDIR]Lib\compileall.py" -f -x "bad_coding|badsyntax|site-packages|py3_" "[TARGETDIR]Lib"' + lib2to3args = r'"[#python.exe]" -c "import lib2to3.pygram, lib2to3.patcomp;lib2to3.patcomp.PatternCompiler()"' + updatepipargs = r'"[#python.exe]" -m ensurepip -U --default-pip' + removepipargs = r'"[#python.exe]" -B -m ensurepip._uninstall' # See "CustomAction Table" add_data(db, "CustomAction", [ # msidbCustomActionTypeFirstSequence + msidbCustomActionTypeTextData + msidbCustomActionTypeProperty @@ -424,11 +426,16 @@ # See "Custom Action Type 18" # msidbCustomActionTypeInScript (1024); run during actual installation # msidbCustomActionTypeNoImpersonate (2048); run action in system account, not user account - ("CompilePyc", 18+1024+2048, "python.exe", compileargs), - ("CompilePyo", 18+1024+2048, "python.exe", "-O "+compileargs), - ("CompileGrammar", 18+1024+2048, "python.exe", lib2to3args), - ("UpdatePip", 18+1024+2048, "python.exe", updatepipargs), - ("RemovePip", 18+1024+2048, "python.exe", removepipargs), + ("SetCompilePycCommandLine", 51, "CompilePyc", compileargs), + ("SetCompilePyoCommandLine", 51, "CompilePyo", compileoargs), + ("SetCompileGrammarCommandLine", 51, "CompileGrammar", lib2to3args), + ("CompilePyc", 1+64+1024, "WixCA", "CAQuietExec"), + ("CompilePyo", 1+64+1024, "WixCA", "CAQuietExec"), + ("CompileGrammar", 1+64+1024, "WixCA", "CAQuietExec"), + ("SetUpdatePipCommandLine", 51, "UpdatePip", updatepipargs), + ("UpdatePip", 1+64+1024, "WixCA", "CAQuietExec"), + ("SetRemovePipCommandLine", 51, "RemovePip", removepipargs), + ("RemovePip", 1+64+1024, "WixCA", "CAQuietExec"), ]) # UI Sequences, see "InstallUISequence Table", "Using a Sequence Table" @@ -460,15 +467,20 @@ ("SetDLLDirToSystem32", 'DLLDIR="" and ' + sys32cond, 751), ("SetDLLDirToTarget", 'DLLDIR="" and not ' + sys32cond, 752), ("UpdateEditIDLE", None, 1050), + # remove pip when state changes to INSTALLSTATE_ABSENT + # run before RemoveFiles + ("SetRemovePipCommandLine", "&pip_feature=2 and !pip_feature=3", 3498), + ("RemovePip", "RemovePip", 3499), # run command if install state of pip changes to INSTALLSTATE_LOCAL # run after InstallFiles - ("UpdatePip", "&pip_feature=3", 4001), - # remove pip when state changes to INSTALLSTATE_ABSENT - # run before RemoveFiles - ("RemovePip", "&pip_feature=2", 3499), - ("CompilePyc", "COMPILEALL", 4002), - ("CompilePyo", "COMPILEALL", 4003), - ("CompileGrammar", "COMPILEALL", 4004), + ("SetUpdatePipCommandLine", "&pip_feature=3 and not !pip_feature=3", 4001), + ("UpdatePip", "UpdatePip", 4002), + ("SetCompilePycCommandLine", "COMPILEALL", 4003), + ("SetCompilePyoCommandLine", "COMPILEALL", 4004), + ("SetCompileGrammarCommandLine", "COMPILEALL", 4005), + ("CompilePyc", "CompilePyc", 4006), + ("CompilePyo", "CompilePyo", 4007), + ("CompileGrammar", "CompileGrammar", 4008), ]) add_data(db, "AdminExecuteSequence", [("InitialTargetDir", 'TARGETDIR=""', 750), -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 15:22:04 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Thu, 01 Sep 2016 19:22:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3ODgx?= =?utf-8?q?=3A_Fixed_possible_bugs_when_setting?= Message-ID: <20160901192203.52479.6600.36F3B88E@psf.io> https://hg.python.org/cpython/rev/546b1f70cbed changeset: 102987:546b1f70cbed branch: 3.5 parent: 102982:ff3a6303c5b1 user: Serhiy Storchaka date: Thu Sep 01 22:18:03 2016 +0300 summary: Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. Based on patch by Xiang Zhang. files: Lib/sqlite3/test/regression.py | 31 ++++++++- Misc/NEWS | 3 + Modules/_sqlite/connection.c | 70 ++++++++++----------- Modules/_sqlite/connection.h | 5 +- 4 files changed, 64 insertions(+), 45 deletions(-) diff --git a/Lib/sqlite3/test/regression.py b/Lib/sqlite3/test/regression.py --- a/Lib/sqlite3/test/regression.py +++ b/Lib/sqlite3/test/regression.py @@ -146,11 +146,34 @@ self.assertRaises(TypeError, sqlite.register_adapter, {}, None) def CheckSetIsolationLevel(self): - """ - See issue 3312. - """ + # See issue 27881. + class CustomStr(str): + def upper(self): + return None + def __del__(self): + con.isolation_level = "" + con = sqlite.connect(":memory:") - setattr(con, "isolation_level", "\xe9") + con.isolation_level = None + for level in "", "DEFERRED", "IMMEDIATE", "EXCLUSIVE": + with self.subTest(level=level): + con.isolation_level = level + con.isolation_level = level.lower() + con.isolation_level = level.capitalize() + con.isolation_level = CustomStr(level) + + # setting isolation_level failure should not alter previous state + con.isolation_level = None + con.isolation_level = "DEFERRED" + pairs = [ + (1, TypeError), (b'', TypeError), ("abc", ValueError), + ("IMMEDIATE\0EXCLUSIVE", ValueError), ("\xe9", ValueError), + ] + for value, exc in pairs: + with self.subTest(level=value): + with self.assertRaises(exc): + con.isolation_level = value + self.assertEqual(con.isolation_level, "DEFERRED") def CheckCursorConstructorCallCheck(self): """ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -50,6 +50,9 @@ Library ------- +- Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. + Based on patch by Xiang Zhang. + - Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory creates not a cursor. Patch by Xiang Zhang. diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -43,6 +43,14 @@ _Py_IDENTIFIER(cursor); +static const char * const begin_statements[] = { + "BEGIN ", + "BEGIN DEFERRED", + "BEGIN IMMEDIATE", + "BEGIN EXCLUSIVE", + NULL +}; + static int pysqlite_connection_set_isolation_level(pysqlite_Connection* self, PyObject* isolation_level); static void _pysqlite_drop_unused_cursor_references(pysqlite_Connection* self); @@ -258,9 +266,6 @@ Py_END_ALLOW_THREADS } - if (self->begin_statement) { - PyMem_Free(self->begin_statement); - } Py_XDECREF(self->isolation_level); Py_XDECREF(self->function_pinboard); Py_XDECREF(self->row_factory); @@ -1183,59 +1188,48 @@ static int pysqlite_connection_set_isolation_level(pysqlite_Connection* self, PyObject* isolation_level) { - PyObject* res; - PyObject* begin_statement; - static PyObject* begin_word; - - Py_XDECREF(self->isolation_level); - - if (self->begin_statement) { - PyMem_Free(self->begin_statement); - self->begin_statement = NULL; - } - if (isolation_level == Py_None) { - Py_INCREF(Py_None); - self->isolation_level = Py_None; - - res = pysqlite_connection_commit(self, NULL); + PyObject *res = pysqlite_connection_commit(self, NULL); if (!res) { return -1; } Py_DECREF(res); + self->begin_statement = NULL; self->inTransaction = 0; } else { - const char *statement; - Py_ssize_t size; + const char * const *candidate; + PyObject *uppercase_level; + _Py_IDENTIFIER(upper); - Py_INCREF(isolation_level); - self->isolation_level = isolation_level; - - if (!begin_word) { - begin_word = PyUnicode_FromString("BEGIN "); - if (!begin_word) return -1; - } - begin_statement = PyUnicode_Concat(begin_word, isolation_level); - if (!begin_statement) { + if (!PyUnicode_Check(isolation_level)) { + PyErr_Format(PyExc_TypeError, + "isolation_level must be a string or None, not %.100s", + Py_TYPE(isolation_level)->tp_name); return -1; } - statement = _PyUnicode_AsStringAndSize(begin_statement, &size); - if (!statement) { - Py_DECREF(begin_statement); + uppercase_level = _PyObject_CallMethodIdObjArgs( + (PyObject *)&PyUnicode_Type, &PyId_upper, + isolation_level, NULL); + if (!uppercase_level) { return -1; } - self->begin_statement = PyMem_Malloc(size + 2); - if (!self->begin_statement) { - Py_DECREF(begin_statement); + for (candidate = begin_statements; *candidate; candidate++) { + if (!PyUnicode_CompareWithASCIIString(uppercase_level, *candidate + 6)) + break; + } + Py_DECREF(uppercase_level); + if (!*candidate) { + PyErr_SetString(PyExc_ValueError, + "invalid value for isolation_level"); return -1; } - - strcpy(self->begin_statement, statement); - Py_DECREF(begin_statement); + self->begin_statement = *candidate; } + Py_INCREF(isolation_level); + Py_XSETREF(self->isolation_level, isolation_level); return 0; } diff --git a/Modules/_sqlite/connection.h b/Modules/_sqlite/connection.h --- a/Modules/_sqlite/connection.h +++ b/Modules/_sqlite/connection.h @@ -55,9 +55,8 @@ /* None for autocommit, otherwise a PyUnicode with the isolation level */ PyObject* isolation_level; - /* NULL for autocommit, otherwise a string with the BEGIN statement; will be - * freed in connection destructor */ - char* begin_statement; + /* NULL for autocommit, otherwise a string with the BEGIN statement */ + const char* begin_statement; /* 1 if a check should be performed for each API call if the connection is * used from the same thread it was created in */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 15:22:04 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Thu, 01 Sep 2016 19:22:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327881=3A_Fixed_possible_bugs_when_setting?= Message-ID: <20160901192204.98042.33904.DE0D7532@psf.io> https://hg.python.org/cpython/rev/96e05f1af2c8 changeset: 102988:96e05f1af2c8 parent: 102985:94710cbcac47 parent: 102987:546b1f70cbed user: Serhiy Storchaka date: Thu Sep 01 22:21:05 2016 +0300 summary: Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. Based on patch by Xiang Zhang. files: Lib/sqlite3/test/regression.py | 31 ++++++++- Misc/NEWS | 3 + Modules/_sqlite/connection.c | 70 ++++++++++----------- Modules/_sqlite/connection.h | 5 +- 4 files changed, 64 insertions(+), 45 deletions(-) diff --git a/Lib/sqlite3/test/regression.py b/Lib/sqlite3/test/regression.py --- a/Lib/sqlite3/test/regression.py +++ b/Lib/sqlite3/test/regression.py @@ -146,11 +146,34 @@ self.assertRaises(TypeError, sqlite.register_adapter, {}, None) def CheckSetIsolationLevel(self): - """ - See issue 3312. - """ + # See issue 27881. + class CustomStr(str): + def upper(self): + return None + def __del__(self): + con.isolation_level = "" + con = sqlite.connect(":memory:") - setattr(con, "isolation_level", "\xe9") + con.isolation_level = None + for level in "", "DEFERRED", "IMMEDIATE", "EXCLUSIVE": + with self.subTest(level=level): + con.isolation_level = level + con.isolation_level = level.lower() + con.isolation_level = level.capitalize() + con.isolation_level = CustomStr(level) + + # setting isolation_level failure should not alter previous state + con.isolation_level = None + con.isolation_level = "DEFERRED" + pairs = [ + (1, TypeError), (b'', TypeError), ("abc", ValueError), + ("IMMEDIATE\0EXCLUSIVE", ValueError), ("\xe9", ValueError), + ] + for value, exc in pairs: + with self.subTest(level=value): + with self.assertRaises(exc): + con.isolation_level = value + self.assertEqual(con.isolation_level, "DEFERRED") def CheckCursorConstructorCallCheck(self): """ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -70,6 +70,9 @@ - Issue #27842: The csv.DictReader now returns rows of type OrderedDict. (Contributed by Steve Holden.) +- Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. + Based on patch by Xiang Zhang. + - Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory creates not a cursor. Patch by Xiang Zhang. diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -43,6 +43,14 @@ _Py_IDENTIFIER(cursor); +static const char * const begin_statements[] = { + "BEGIN ", + "BEGIN DEFERRED", + "BEGIN IMMEDIATE", + "BEGIN EXCLUSIVE", + NULL +}; + static int pysqlite_connection_set_isolation_level(pysqlite_Connection* self, PyObject* isolation_level); static void _pysqlite_drop_unused_cursor_references(pysqlite_Connection* self); @@ -258,9 +266,6 @@ Py_END_ALLOW_THREADS } - if (self->begin_statement) { - PyMem_Free(self->begin_statement); - } Py_XDECREF(self->isolation_level); Py_XDECREF(self->function_pinboard); Py_XDECREF(self->row_factory); @@ -1175,59 +1180,48 @@ static int pysqlite_connection_set_isolation_level(pysqlite_Connection* self, PyObject* isolation_level) { - PyObject* res; - PyObject* begin_statement; - static PyObject* begin_word; - - Py_XDECREF(self->isolation_level); - - if (self->begin_statement) { - PyMem_Free(self->begin_statement); - self->begin_statement = NULL; - } - if (isolation_level == Py_None) { - Py_INCREF(Py_None); - self->isolation_level = Py_None; - - res = pysqlite_connection_commit(self, NULL); + PyObject *res = pysqlite_connection_commit(self, NULL); if (!res) { return -1; } Py_DECREF(res); + self->begin_statement = NULL; self->inTransaction = 0; } else { - const char *statement; - Py_ssize_t size; + const char * const *candidate; + PyObject *uppercase_level; + _Py_IDENTIFIER(upper); - Py_INCREF(isolation_level); - self->isolation_level = isolation_level; - - if (!begin_word) { - begin_word = PyUnicode_FromString("BEGIN "); - if (!begin_word) return -1; - } - begin_statement = PyUnicode_Concat(begin_word, isolation_level); - if (!begin_statement) { + if (!PyUnicode_Check(isolation_level)) { + PyErr_Format(PyExc_TypeError, + "isolation_level must be a string or None, not %.100s", + Py_TYPE(isolation_level)->tp_name); return -1; } - statement = _PyUnicode_AsStringAndSize(begin_statement, &size); - if (!statement) { - Py_DECREF(begin_statement); + uppercase_level = _PyObject_CallMethodIdObjArgs( + (PyObject *)&PyUnicode_Type, &PyId_upper, + isolation_level, NULL); + if (!uppercase_level) { return -1; } - self->begin_statement = PyMem_Malloc(size + 2); - if (!self->begin_statement) { - Py_DECREF(begin_statement); + for (candidate = begin_statements; *candidate; candidate++) { + if (!PyUnicode_CompareWithASCIIString(uppercase_level, *candidate + 6)) + break; + } + Py_DECREF(uppercase_level); + if (!*candidate) { + PyErr_SetString(PyExc_ValueError, + "invalid value for isolation_level"); return -1; } - - strcpy(self->begin_statement, statement); - Py_DECREF(begin_statement); + self->begin_statement = *candidate; } + Py_INCREF(isolation_level); + Py_XSETREF(self->isolation_level, isolation_level); return 0; } diff --git a/Modules/_sqlite/connection.h b/Modules/_sqlite/connection.h --- a/Modules/_sqlite/connection.h +++ b/Modules/_sqlite/connection.h @@ -55,9 +55,8 @@ /* None for autocommit, otherwise a PyUnicode with the isolation level */ PyObject* isolation_level; - /* NULL for autocommit, otherwise a string with the BEGIN statement; will be - * freed in connection destructor */ - char* begin_statement; + /* NULL for autocommit, otherwise a string with the BEGIN statement */ + const char* begin_statement; /* 1 if a check should be performed for each API call if the connection is * used from the same thread it was created in */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 16:05:29 2016 From: python-checkins at python.org (matthias.klose) Date: Thu, 01 Sep 2016 20:05:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_-_Issue_=2327917=3A_Set_pl?= =?utf-8?q?atform_triplets_for_Android_builds=2E?= Message-ID: <20160901200528.98179.93542.D812860A@psf.io> https://hg.python.org/cpython/rev/a931fdc4c4c4 changeset: 102989:a931fdc4c4c4 user: doko at ubuntu.com date: Thu Sep 01 22:05:20 2016 +0200 summary: - Issue #27917: Set platform triplets for Android builds. files: Misc/NEWS | 2 ++ configure | 26 +++++++++++++++++++++++++- configure.ac | 26 +++++++++++++++++++++++++- 3 files changed, 52 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -127,6 +127,8 @@ Build ----- +- Issue #27917: Set platform triplets for Android builds. + - Issue #25825: Update references to the $(LIBPL) installation path on AIX. This path was changed in 3.2a4. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -5279,7 +5279,31 @@ #undef powerpc #undef sparc #undef unix -#if defined(__linux__) +#if defined(__ANDROID__) +# if defined(__x86_64__) && defined(__LP64__) + x86_64-linux-android +# elif defined(__i386__) + i686-linux-android +# elif defined(__aarch64__) && defined(__AARCH64EL__) +# if defined(__ILP32__) + aarch64_ilp32-linux-android +# else + aarch64-linux-android +# endif +# elif defined(__ARM_EABI__) && defined(__ARMEL__) + arm-linux-androideabi +# elif defined(__mips_hard_float) && defined(_MIPSEL) +# if _MIPS_SIM == _ABIO32 + mipsel-linux-android +# elif _MIPS_SIM == _ABI64 + mips64el-linux-android +# else +# error unknown platform triplet +# endif +# else +# error unknown platform triplet +# endif +#elif defined(__linux__) # if defined(__x86_64__) && defined(__LP64__) x86_64-linux-gnu # elif defined(__x86_64__) && defined(__ILP32__) diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -768,7 +768,31 @@ #undef powerpc #undef sparc #undef unix -#if defined(__linux__) +#if defined(__ANDROID__) +# if defined(__x86_64__) && defined(__LP64__) + x86_64-linux-android +# elif defined(__i386__) + i686-linux-android +# elif defined(__aarch64__) && defined(__AARCH64EL__) +# if defined(__ILP32__) + aarch64_ilp32-linux-android +# else + aarch64-linux-android +# endif +# elif defined(__ARM_EABI__) && defined(__ARMEL__) + arm-linux-androideabi +# elif defined(__mips_hard_float) && defined(_MIPSEL) +# if _MIPS_SIM == _ABIO32 + mipsel-linux-android +# elif _MIPS_SIM == _ABI64 + mips64el-linux-android +# else +# error unknown platform triplet +# endif +# else +# error unknown platform triplet +# endif +#elif defined(__linux__) # if defined(__x86_64__) && defined(__LP64__) x86_64-linux-gnu # elif defined(__x86_64__) && defined(__ILP32__) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 21:17:07 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 01:17:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Add_another_te?= =?utf-8?q?st_capturing_the_basic_discovery_expectation=2E?= Message-ID: <20160902011707.27198.77599.23656F8B@psf.io> https://hg.python.org/cpython/rev/404f5fb51ffb changeset: 102992:404f5fb51ffb branch: 3.4 user: Jason R. Coombs date: Sun Aug 30 13:26:48 2015 -0400 summary: Add another test capturing the basic discovery expectation. files: Lib/distutils/tests/test_filelist.py | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -301,6 +301,17 @@ os.symlink('foo', 'bar') self.assertEqual(filelist.findall(), []) + def test_basic_discovery(self): + with test.support.temp_cwd(): + os.mkdir('foo') + file1 = os.path.join('foo', 'file1.txt') + test.support.create_empty_file(file1) + os.mkdir('bar') + file2 = os.path.join('bar', 'file2.txt') + test.support.create_empty_file(file2) + expected = [file1, file2] + self.assertEqual(filelist.findall(), expected) + if __name__ == "__main__": unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 21:17:07 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 01:17:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Use_modern_mec?= =?utf-8?q?hanism_for_test_discovery?= Message-ID: <20160902011706.81380.60669.400780C2@psf.io> https://hg.python.org/cpython/rev/cc86e9e102e8 changeset: 102990:cc86e9e102e8 branch: 3.4 parent: 102714:675e20c38fda user: Jason R. Coombs date: Sun Aug 30 13:13:11 2015 -0400 summary: Use modern mechanism for test discovery files: Lib/distutils/tests/test_filelist.py | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -7,7 +7,7 @@ from distutils.errors import DistutilsTemplateError from distutils.filelist import glob_to_re, translate_pattern, FileList -from test.support import captured_stdout, run_unittest +from test.support import captured_stdout from distutils.tests import support MANIFEST_IN = """\ @@ -292,8 +292,5 @@ self.assertWarnings() -def test_suite(): - return unittest.makeSuite(FileListTestCase) - if __name__ == "__main__": - run_unittest(test_suite()) + unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 21:17:07 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 01:17:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Add_docstring_?= =?utf-8?q?and_additional_test_revealing_nuances_of_the_implementation_as?= Message-ID: <20160902011707.2740.66459.533A2A89@psf.io> https://hg.python.org/cpython/rev/c2892a7ad148 changeset: 102993:c2892a7ad148 branch: 3.4 user: Jason R. Coombs date: Sat Sep 19 17:32:51 2015 +0200 summary: Add docstring and additional test revealing nuances of the implementation as found in setuptools. files: Lib/distutils/tests/test_filelist.py | 16 ++++++++++++++++ 1 files changed, 16 insertions(+), 0 deletions(-) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -302,6 +302,11 @@ self.assertEqual(filelist.findall(), []) def test_basic_discovery(self): + """ + When findall is called with no parameters or with + '.' as the parameter, the dot should be omitted from + the results. + """ with test.support.temp_cwd(): os.mkdir('foo') file1 = os.path.join('foo', 'file1.txt') @@ -312,6 +317,17 @@ expected = [file1, file2] self.assertEqual(filelist.findall(), expected) + def test_non_local_discovery(self): + """ + When findall is called with another path, the full + path name should be returned. + """ + with test.support.temp_dir() as temp_dir: + file1 = os.path.join(temp_dir, 'file1.txt') + test.support.create_empty_file(file1) + expected = [file1] + self.assertEqual(filelist.findall(temp_dir), expected) + if __name__ == "__main__": unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 21:17:07 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 01:17:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Sort_result_to?= =?utf-8?q?_avoid_spurious_errors_due_to_order=2E?= Message-ID: <20160902011707.57497.5561.70992754@psf.io> https://hg.python.org/cpython/rev/5d775d50224c changeset: 102994:5d775d50224c branch: 3.4 user: Jason R. Coombs date: Sun Aug 30 14:05:58 2015 -0400 summary: Sort result to avoid spurious errors due to order. files: Lib/distutils/tests/test_filelist.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -314,8 +314,8 @@ os.mkdir('bar') file2 = os.path.join('bar', 'file2.txt') test.support.create_empty_file(file2) - expected = [file1, file2] - self.assertEqual(filelist.findall(), expected) + expected = [file2, file1] + self.assertEqual(sorted(filelist.findall()), expected) def test_non_local_discovery(self): """ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 21:17:07 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 01:17:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEyMjg1?= =?utf-8?q?=3A_Add_test_capturing_failure=2E?= Message-ID: <20160902011707.6263.10609.84747B52@psf.io> https://hg.python.org/cpython/rev/941346104718 changeset: 102991:941346104718 branch: 3.4 user: Jason R. Coombs date: Sun Aug 30 13:22:56 2015 -0400 summary: Issue #12285: Add test capturing failure. files: Lib/distutils/tests/test_filelist.py | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -6,7 +6,9 @@ from distutils.log import WARN from distutils.errors import DistutilsTemplateError from distutils.filelist import glob_to_re, translate_pattern, FileList +from distutils import filelist +import test.support from test.support import captured_stdout from distutils.tests import support @@ -292,5 +294,13 @@ self.assertWarnings() +class FindAllTestCase(unittest.TestCase): + @test.support.skip_unless_symlink + def test_missing_symlink(self): + with test.support.temp_cwd(): + os.symlink('foo', 'bar') + self.assertEqual(filelist.findall(), []) + + if __name__ == "__main__": unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 21:17:07 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 01:17:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEyMjg1?= =?utf-8?q?=3A_Replace_implementation_of_findall_with_implementation_from?= Message-ID: <20160902011707.57497.42418.2CE8BE33@psf.io> https://hg.python.org/cpython/rev/56c60b3d06fb changeset: 102995:56c60b3d06fb branch: 3.4 user: Jason R. Coombs date: Sat Sep 19 18:12:15 2015 +0200 summary: Issue #12285: Replace implementation of findall with implementation from Setuptools 7ce820d524db. files: Lib/distutils/filelist.py | 48 +++++++++++--------------- 1 files changed, 21 insertions(+), 27 deletions(-) diff --git a/Lib/distutils/filelist.py b/Lib/distutils/filelist.py --- a/Lib/distutils/filelist.py +++ b/Lib/distutils/filelist.py @@ -6,6 +6,7 @@ import os, re import fnmatch +import functools from distutils.util import convert_path from distutils.errors import DistutilsTemplateError, DistutilsInternalError from distutils import log @@ -242,35 +243,28 @@ # ---------------------------------------------------------------------- # Utility functions +def _find_all_simple(path): + """ + Find all files under 'path' + """ + results = ( + os.path.join(base, file) + for base, dirs, files in os.walk(path, followlinks=True) + for file in files + ) + return filter(os.path.isfile, results) + + def findall(dir=os.curdir): - """Find all files under 'dir' and return the list of full filenames - (relative to 'dir'). """ - from stat import ST_MODE, S_ISREG, S_ISDIR, S_ISLNK - - list = [] - stack = [dir] - pop = stack.pop - push = stack.append - - while stack: - dir = pop() - names = os.listdir(dir) - - for name in names: - if dir != os.curdir: # avoid the dreaded "./" syndrome - fullname = os.path.join(dir, name) - else: - fullname = name - - # Avoid excess stat calls -- just one will do, thank you! - stat = os.stat(fullname) - mode = stat[ST_MODE] - if S_ISREG(mode): - list.append(fullname) - elif S_ISDIR(mode) and not S_ISLNK(mode): - push(fullname) - return list + Find all files under 'dir' and return the list of full filenames. + Unless dir is '.', return full filenames with dir prepended. + """ + files = _find_all_simple(dir) + if dir == os.curdir: + make_rel = functools.partial(os.path.relpath, start=dir) + files = map(make_rel, files) + return list(files) def glob_to_re(pattern): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 21:17:08 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 01:17:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEyMjg1?= =?utf-8?q?=3A_Update_NEWS?= Message-ID: <20160902011708.25383.33022.815DCEFE@psf.io> https://hg.python.org/cpython/rev/13619a3e0737 changeset: 102996:13619a3e0737 branch: 3.4 user: Jason R. Coombs date: Thu Sep 01 21:12:17 2016 -0400 summary: Issue #12285: Update NEWS files: Misc/NEWS | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,6 +13,8 @@ Library ------- +- Issue #12285: Fix error when distutils encounters symlink. + - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 21:17:08 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 01:17:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2312285=3A_Merge_with_3=2E4?= Message-ID: <20160902011708.26462.47142.D4863092@psf.io> https://hg.python.org/cpython/rev/ade53661607a changeset: 102997:ade53661607a branch: 3.5 parent: 102987:546b1f70cbed parent: 102996:13619a3e0737 user: Jason R. Coombs date: Thu Sep 01 21:15:04 2016 -0400 summary: Issue #12285: Merge with 3.4 files: Lib/distutils/filelist.py | 48 +++++++--------- Lib/distutils/tests/test_filelist.py | 42 ++++++++++++- Misc/NEWS | 2 + 3 files changed, 61 insertions(+), 31 deletions(-) diff --git a/Lib/distutils/filelist.py b/Lib/distutils/filelist.py --- a/Lib/distutils/filelist.py +++ b/Lib/distutils/filelist.py @@ -6,6 +6,7 @@ import os, re import fnmatch +import functools from distutils.util import convert_path from distutils.errors import DistutilsTemplateError, DistutilsInternalError from distutils import log @@ -242,35 +243,28 @@ # ---------------------------------------------------------------------- # Utility functions +def _find_all_simple(path): + """ + Find all files under 'path' + """ + results = ( + os.path.join(base, file) + for base, dirs, files in os.walk(path, followlinks=True) + for file in files + ) + return filter(os.path.isfile, results) + + def findall(dir=os.curdir): - """Find all files under 'dir' and return the list of full filenames - (relative to 'dir'). """ - from stat import ST_MODE, S_ISREG, S_ISDIR, S_ISLNK - - list = [] - stack = [dir] - pop = stack.pop - push = stack.append - - while stack: - dir = pop() - names = os.listdir(dir) - - for name in names: - if dir != os.curdir: # avoid the dreaded "./" syndrome - fullname = os.path.join(dir, name) - else: - fullname = name - - # Avoid excess stat calls -- just one will do, thank you! - stat = os.stat(fullname) - mode = stat[ST_MODE] - if S_ISREG(mode): - list.append(fullname) - elif S_ISDIR(mode) and not S_ISLNK(mode): - push(fullname) - return list + Find all files under 'dir' and return the list of full filenames. + Unless dir is '.', return full filenames with dir prepended. + """ + files = _find_all_simple(dir) + if dir == os.curdir: + make_rel = functools.partial(os.path.relpath, start=dir) + files = map(make_rel, files) + return list(files) def glob_to_re(pattern): diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -6,8 +6,10 @@ from distutils.log import WARN from distutils.errors import DistutilsTemplateError from distutils.filelist import glob_to_re, translate_pattern, FileList +from distutils import filelist -from test.support import captured_stdout, run_unittest +import test.support +from test.support import captured_stdout from distutils.tests import support MANIFEST_IN = """\ @@ -292,8 +294,40 @@ self.assertWarnings() -def test_suite(): - return unittest.makeSuite(FileListTestCase) +class FindAllTestCase(unittest.TestCase): + @test.support.skip_unless_symlink + def test_missing_symlink(self): + with test.support.temp_cwd(): + os.symlink('foo', 'bar') + self.assertEqual(filelist.findall(), []) + + def test_basic_discovery(self): + """ + When findall is called with no parameters or with + '.' as the parameter, the dot should be omitted from + the results. + """ + with test.support.temp_cwd(): + os.mkdir('foo') + file1 = os.path.join('foo', 'file1.txt') + test.support.create_empty_file(file1) + os.mkdir('bar') + file2 = os.path.join('bar', 'file2.txt') + test.support.create_empty_file(file2) + expected = [file2, file1] + self.assertEqual(sorted(filelist.findall()), expected) + + def test_non_local_discovery(self): + """ + When findall is called with another path, the full + path name should be returned. + """ + with test.support.temp_dir() as temp_dir: + file1 = os.path.join(temp_dir, 'file1.txt') + test.support.create_empty_file(file1) + expected = [file1] + self.assertEqual(filelist.findall(temp_dir), expected) + if __name__ == "__main__": - run_unittest(test_suite()) + unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -50,6 +50,8 @@ Library ------- +- Issue #12285: Fix error when distutils encounters symlink. + - Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. Based on patch by Xiang Zhang. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 21:17:08 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 01:17:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2312885=3A_Merge_with_3=2E5?= Message-ID: <20160902011708.90354.63141.44477244@psf.io> https://hg.python.org/cpython/rev/79422fab2ef4 changeset: 102998:79422fab2ef4 parent: 102989:a931fdc4c4c4 parent: 102997:ade53661607a user: Jason R. Coombs date: Thu Sep 01 21:16:32 2016 -0400 summary: Issue #12885: Merge with 3.5 files: Lib/distutils/filelist.py | 48 +++++++--------- Lib/distutils/tests/test_filelist.py | 42 ++++++++++++- Misc/NEWS | 2 + 3 files changed, 61 insertions(+), 31 deletions(-) diff --git a/Lib/distutils/filelist.py b/Lib/distutils/filelist.py --- a/Lib/distutils/filelist.py +++ b/Lib/distutils/filelist.py @@ -6,6 +6,7 @@ import os, re import fnmatch +import functools from distutils.util import convert_path from distutils.errors import DistutilsTemplateError, DistutilsInternalError from distutils import log @@ -242,35 +243,28 @@ # ---------------------------------------------------------------------- # Utility functions +def _find_all_simple(path): + """ + Find all files under 'path' + """ + results = ( + os.path.join(base, file) + for base, dirs, files in os.walk(path, followlinks=True) + for file in files + ) + return filter(os.path.isfile, results) + + def findall(dir=os.curdir): - """Find all files under 'dir' and return the list of full filenames - (relative to 'dir'). """ - from stat import ST_MODE, S_ISREG, S_ISDIR, S_ISLNK - - list = [] - stack = [dir] - pop = stack.pop - push = stack.append - - while stack: - dir = pop() - names = os.listdir(dir) - - for name in names: - if dir != os.curdir: # avoid the dreaded "./" syndrome - fullname = os.path.join(dir, name) - else: - fullname = name - - # Avoid excess stat calls -- just one will do, thank you! - stat = os.stat(fullname) - mode = stat[ST_MODE] - if S_ISREG(mode): - list.append(fullname) - elif S_ISDIR(mode) and not S_ISLNK(mode): - push(fullname) - return list + Find all files under 'dir' and return the list of full filenames. + Unless dir is '.', return full filenames with dir prepended. + """ + files = _find_all_simple(dir) + if dir == os.curdir: + make_rel = functools.partial(os.path.relpath, start=dir) + files = map(make_rel, files) + return list(files) def glob_to_re(pattern): diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -6,8 +6,10 @@ from distutils.log import WARN from distutils.errors import DistutilsTemplateError from distutils.filelist import glob_to_re, translate_pattern, FileList +from distutils import filelist -from test.support import captured_stdout, run_unittest +import test.support +from test.support import captured_stdout from distutils.tests import support MANIFEST_IN = """\ @@ -292,8 +294,40 @@ self.assertWarnings() -def test_suite(): - return unittest.makeSuite(FileListTestCase) +class FindAllTestCase(unittest.TestCase): + @test.support.skip_unless_symlink + def test_missing_symlink(self): + with test.support.temp_cwd(): + os.symlink('foo', 'bar') + self.assertEqual(filelist.findall(), []) + + def test_basic_discovery(self): + """ + When findall is called with no parameters or with + '.' as the parameter, the dot should be omitted from + the results. + """ + with test.support.temp_cwd(): + os.mkdir('foo') + file1 = os.path.join('foo', 'file1.txt') + test.support.create_empty_file(file1) + os.mkdir('bar') + file2 = os.path.join('bar', 'file2.txt') + test.support.create_empty_file(file2) + expected = [file2, file1] + self.assertEqual(sorted(filelist.findall()), expected) + + def test_non_local_discovery(self): + """ + When findall is called with another path, the full + path name should be returned. + """ + with test.support.temp_dir() as temp_dir: + file1 = os.path.join(temp_dir, 'file1.txt') + test.support.create_empty_file(file1) + expected = [file1] + self.assertEqual(filelist.findall(temp_dir), expected) + if __name__ == "__main__": - run_unittest(test_suite()) + unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -70,6 +70,8 @@ - Issue #27842: The csv.DictReader now returns rows of type OrderedDict. (Contributed by Steve Holden.) +- Issue #12285: Fix error when distutils encounters symlink. + - Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. Based on patch by Xiang Zhang. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 22:00:18 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 02:00:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Backed_out_cha?= =?utf-8?q?ngeset_cc86e9e102e8?= Message-ID: <20160902020018.43980.2737.63A11BD2@psf.io> https://hg.python.org/cpython/rev/a442a64c35d0 changeset: 102999:a442a64c35d0 branch: 3.4 parent: 102996:13619a3e0737 user: Jason R. Coombs date: Thu Sep 01 21:55:22 2016 -0400 summary: Backed out changeset cc86e9e102e8 files: Lib/distutils/tests/test_filelist.py | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -9,7 +9,7 @@ from distutils import filelist import test.support -from test.support import captured_stdout +from test.support import captured_stdout, run_unittest from distutils.tests import support MANIFEST_IN = """\ @@ -329,5 +329,12 @@ self.assertEqual(filelist.findall(temp_dir), expected) +def test_suite(): + return unittest.TestSuite([ + unittest.makeSuite(FileListTestCase), + unittest.makeSuite(FindAllTestCase), + ]) + + if __name__ == "__main__": - unittest.main() + run_unittest(test_suite()) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 22:00:19 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 02:00:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_backout_for_test_suite_fix?= Message-ID: <20160902020018.15915.62528.EB7EEEDC@psf.io> https://hg.python.org/cpython/rev/abc4de9b09b8 changeset: 103001:abc4de9b09b8 parent: 102998:79422fab2ef4 parent: 103000:0561fcc33211 user: Jason R. Coombs date: Thu Sep 01 22:00:03 2016 -0400 summary: Merge backout for test suite fix files: Lib/distutils/tests/test_filelist.py | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -9,7 +9,7 @@ from distutils import filelist import test.support -from test.support import captured_stdout +from test.support import captured_stdout, run_unittest from distutils.tests import support MANIFEST_IN = """\ @@ -329,5 +329,12 @@ self.assertEqual(filelist.findall(temp_dir), expected) +def test_suite(): + return unittest.TestSuite([ + unittest.makeSuite(FileListTestCase), + unittest.makeSuite(FindAllTestCase), + ]) + + if __name__ == "__main__": - unittest.main() + run_unittest(test_suite()) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 22:00:18 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 02:00:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_backout_for_test_suite_fix?= Message-ID: <20160902020018.25483.22830.D3E92142@psf.io> https://hg.python.org/cpython/rev/0561fcc33211 changeset: 103000:0561fcc33211 branch: 3.5 parent: 102997:ade53661607a parent: 102999:a442a64c35d0 user: Jason R. Coombs date: Thu Sep 01 21:59:46 2016 -0400 summary: Merge backout for test suite fix files: Lib/distutils/tests/test_filelist.py | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -9,7 +9,7 @@ from distutils import filelist import test.support -from test.support import captured_stdout +from test.support import captured_stdout, run_unittest from distutils.tests import support MANIFEST_IN = """\ @@ -329,5 +329,12 @@ self.assertEqual(filelist.findall(temp_dir), expected) +def test_suite(): + return unittest.TestSuite([ + unittest.makeSuite(FileListTestCase), + unittest.makeSuite(FindAllTestCase), + ]) + + if __name__ == "__main__": - unittest.main() + run_unittest(test_suite()) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 22:10:53 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 02:10:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2312885=3A_Merge_3=2E4?= Message-ID: <20160902021053.26663.43406.1AD40EBD@psf.io> https://hg.python.org/cpython/rev/74ccec0bd442 changeset: 103003:74ccec0bd442 branch: 3.5 parent: 103000:0561fcc33211 parent: 103002:590d0de4ff48 user: Jason R. Coombs date: Thu Sep 01 22:09:06 2016 -0400 summary: Issue #12885: Merge 3.4 files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -50,7 +50,7 @@ Library ------- -- Issue #12285: Fix error when distutils encounters symlink. +- Issue #12885: Fix error when distutils encounters symlink. - Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. Based on patch by Xiang Zhang. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 22:10:53 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 02:10:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEyODg1?= =?utf-8?q?=3A_Correct_issue_reference_in_NEWS?= Message-ID: <20160902021053.2890.1992.F689A41E@psf.io> https://hg.python.org/cpython/rev/590d0de4ff48 changeset: 103002:590d0de4ff48 branch: 3.4 parent: 102999:a442a64c35d0 user: Jason R. Coombs date: Thu Sep 01 22:08:25 2016 -0400 summary: Issue #12885: Correct issue reference in NEWS files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,7 +13,7 @@ Library ------- -- Issue #12285: Fix error when distutils encounters symlink. +- Issue #12885: Fix error when distutils encounters symlink. - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 22:10:53 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 02:10:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2312885=3A_Merge_with_3=2E5?= Message-ID: <20160902021053.90088.34876.10A1652E@psf.io> https://hg.python.org/cpython/rev/bfff89ed356b changeset: 103004:bfff89ed356b parent: 103001:abc4de9b09b8 parent: 103003:74ccec0bd442 user: Jason R. Coombs date: Thu Sep 01 22:10:09 2016 -0400 summary: Issue #12885: Merge with 3.5 files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -70,7 +70,7 @@ - Issue #27842: The csv.DictReader now returns rows of type OrderedDict. (Contributed by Steve Holden.) -- Issue #12285: Fix error when distutils encounters symlink. +- Issue #12885: Fix error when distutils encounters symlink. - Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. Based on patch by Xiang Zhang. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 23:29:41 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 03:29:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzEyODg1?= =?utf-8?q?=3A_Revert_commits_in_3=2E4_branch_which_is_security-only_fixes?= =?utf-8?q?=2E?= Message-ID: <20160902032941.57701.28265.E0BB759D@psf.io> https://hg.python.org/cpython/rev/e82b995d1a5c changeset: 103005:e82b995d1a5c branch: 3.4 parent: 103002:590d0de4ff48 user: Jason R. Coombs date: Thu Sep 01 23:27:45 2016 -0400 summary: Issue #12885: Revert commits in 3.4 branch which is security-only fixes. files: Lib/distutils/filelist.py | 46 ++++++++------ Lib/distutils/tests/test_filelist.py | 48 +--------------- Misc/NEWS | 2 - 3 files changed, 28 insertions(+), 68 deletions(-) diff --git a/Lib/distutils/filelist.py b/Lib/distutils/filelist.py --- a/Lib/distutils/filelist.py +++ b/Lib/distutils/filelist.py @@ -6,7 +6,6 @@ import os, re import fnmatch -import functools from distutils.util import convert_path from distutils.errors import DistutilsTemplateError, DistutilsInternalError from distutils import log @@ -243,28 +242,35 @@ # ---------------------------------------------------------------------- # Utility functions -def _find_all_simple(path): +def findall(dir=os.curdir): + """Find all files under 'dir' and return the list of full filenames + (relative to 'dir'). """ - Find all files under 'path' - """ - results = ( - os.path.join(base, file) - for base, dirs, files in os.walk(path, followlinks=True) - for file in files - ) - return filter(os.path.isfile, results) + from stat import ST_MODE, S_ISREG, S_ISDIR, S_ISLNK + list = [] + stack = [dir] + pop = stack.pop + push = stack.append -def findall(dir=os.curdir): - """ - Find all files under 'dir' and return the list of full filenames. - Unless dir is '.', return full filenames with dir prepended. - """ - files = _find_all_simple(dir) - if dir == os.curdir: - make_rel = functools.partial(os.path.relpath, start=dir) - files = map(make_rel, files) - return list(files) + while stack: + dir = pop() + names = os.listdir(dir) + + for name in names: + if dir != os.curdir: # avoid the dreaded "./" syndrome + fullname = os.path.join(dir, name) + else: + fullname = name + + # Avoid excess stat calls -- just one will do, thank you! + stat = os.stat(fullname) + mode = stat[ST_MODE] + if S_ISREG(mode): + list.append(fullname) + elif S_ISDIR(mode) and not S_ISLNK(mode): + push(fullname) + return list def glob_to_re(pattern): diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -6,10 +6,8 @@ from distutils.log import WARN from distutils.errors import DistutilsTemplateError from distutils.filelist import glob_to_re, translate_pattern, FileList -from distutils import filelist -import test.support -from test.support import captured_stdout, run_unittest +from test.support import captured_stdout from distutils.tests import support MANIFEST_IN = """\ @@ -294,47 +292,5 @@ self.assertWarnings() -class FindAllTestCase(unittest.TestCase): - @test.support.skip_unless_symlink - def test_missing_symlink(self): - with test.support.temp_cwd(): - os.symlink('foo', 'bar') - self.assertEqual(filelist.findall(), []) - - def test_basic_discovery(self): - """ - When findall is called with no parameters or with - '.' as the parameter, the dot should be omitted from - the results. - """ - with test.support.temp_cwd(): - os.mkdir('foo') - file1 = os.path.join('foo', 'file1.txt') - test.support.create_empty_file(file1) - os.mkdir('bar') - file2 = os.path.join('bar', 'file2.txt') - test.support.create_empty_file(file2) - expected = [file2, file1] - self.assertEqual(sorted(filelist.findall()), expected) - - def test_non_local_discovery(self): - """ - When findall is called with another path, the full - path name should be returned. - """ - with test.support.temp_dir() as temp_dir: - file1 = os.path.join(temp_dir, 'file1.txt') - test.support.create_empty_file(file1) - expected = [file1] - self.assertEqual(filelist.findall(temp_dir), expected) - - -def test_suite(): - return unittest.TestSuite([ - unittest.makeSuite(FileListTestCase), - unittest.makeSuite(FindAllTestCase), - ]) - - if __name__ == "__main__": - run_unittest(test_suite()) + unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,8 +13,6 @@ Library ------- -- Issue #12885: Fix error when distutils encounters symlink. - - In the curses module, raise an error if window.getstr() or window.instr() is passed a negative value. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 23:29:41 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 03:29:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2312885=3A_Merge_with_3=2E4=2C_retaining_commits_revert?= =?utf-8?q?ed_there=2E?= Message-ID: <20160902032941.6287.11089.76EB35AA@psf.io> https://hg.python.org/cpython/rev/a7ce98a4e9e4 changeset: 103006:a7ce98a4e9e4 branch: 3.5 parent: 103003:74ccec0bd442 parent: 103005:e82b995d1a5c user: Jason R. Coombs date: Thu Sep 01 23:29:04 2016 -0400 summary: Issue #12885: Merge with 3.4, retaining commits reverted there. files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 1 23:29:41 2016 From: python-checkins at python.org (jason.coombs) Date: Fri, 02 Sep 2016 03:29:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2312885=3A_Merge_with_3=2E5?= Message-ID: <20160902032941.2843.99064.82663B83@psf.io> https://hg.python.org/cpython/rev/fa82fabe9d65 changeset: 103007:fa82fabe9d65 parent: 103004:bfff89ed356b parent: 103006:a7ce98a4e9e4 user: Jason R. Coombs date: Thu Sep 01 23:29:28 2016 -0400 summary: Issue #12885: Merge with 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 2 02:55:37 2016 From: python-checkins at python.org (ethan.furman) Date: Fri, 02 Sep 2016 06:55:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue23591=3A_add_docs=3B_?= =?utf-8?q?code_cleanup=3B_more_tests?= Message-ID: <20160902065537.81426.35903.44C93636@psf.io> https://hg.python.org/cpython/rev/adbc7eec97f1 changeset: 103008:adbc7eec97f1 user: Ethan Furman date: Thu Sep 01 23:55:19 2016 -0700 summary: issue23591: add docs; code cleanup; more tests files: Doc/library/enum.rst | 195 +++++++++++++++++++++-------- Lib/enum.py | 8 +- Lib/test/test_enum.py | 88 ++++++++++-- 3 files changed, 213 insertions(+), 78 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -23,9 +23,9 @@ Module Contents --------------- -This module defines two enumeration classes that can be used to define unique -sets of names and values: :class:`Enum` and :class:`IntEnum`. It also defines -one decorator, :func:`unique`. +This module defines four enumeration classes that can be used to define unique +sets of names and values: :class:`Enum`, :class:`IntEnum`, and +:class:`IntFlags`. It also defines one decorator, :func:`unique`. .. class:: Enum @@ -37,10 +37,23 @@ Base class for creating enumerated constants that are also subclasses of :class:`int`. +.. class:: IntFlag + + Base class for creating enumerated constants that can be combined using + the bitwise operators without losing their :class:`IntFlag` membership. + :class:`IntFlag` members are also subclasses of :class:`int`. + +.. class:: Flag + + Base class for creating enumerated constants that can be combined using + the bitwise operations without losing their :class:`Flag` membership. + .. function:: unique Enum class decorator that ensures only one name is bound to any one value. +.. versionadded:: 3.6 ``Flag``, ``IntFlag`` + Creating an Enum ---------------- @@ -478,7 +491,7 @@ IntEnum ^^^^^^^ -A variation of :class:`Enum` is provided which is also a subclass of +The first variation of :class:`Enum` that is provided is also a subclass of :class:`int`. Members of an :class:`IntEnum` can be compared to integers; by extension, integer enumerations of different types can also be compared to each other:: @@ -521,13 +534,54 @@ >>> [i for i in range(Shape.square)] [0, 1] -For the vast majority of code, :class:`Enum` is strongly recommended, -since :class:`IntEnum` breaks some semantic promises of an enumeration (by -being comparable to integers, and thus by transitivity to other -unrelated enumerations). It should be used only in special cases where -there's no other choice; for example, when integer constants are -replaced with enumerations and backwards compatibility is required with code -that still expects integers. + +IntFlag +^^^^^^^ + +The next variation of :class:`Enum` provided, :class:`IntFlag`, is also based +on :class:`int`. The difference being :class:`IntFlag` members can be combined +using the bitwise operators (&, \|, ^, ~) and the result is still an +:class:`IntFlag` member. However, as the name implies, :class:`IntFlag` +members also subclass :class:`int` and can be used wherever an :class:`int` is. +Any operation on an :class:`IntFlag` member besides the bit-wise operations +will lose the :class:`IntFlag` membership. + + >>> from enum import IntFlag + >>> class Perm(IntFlag): + ... R = 4 + ... W = 2 + ... X = 1 + ... + >>> Perm.R | Perm.W + + >>> Perm.R + Perm.W + 6 + >>> RW = Perm.R | Perm.W + >>> Perm.R in RW + True + +.. versionadded:: 3.6 + + +Flag +^^^^ + +The last variation is :class:`Flag`. Like :class:`IntFlag`, :class:`Flag` +members can be combined using the bitwise operators (^, \|, ^, ~). Unlike +:class:`IntFlag`, they cannot be combined with, nor compared against, any +other :class:`Flag` enumeration nor :class:`int`. + +.. versionadded:: 3.6 + +.. note:: + + For the majority of new code, :class:`Enum` and :class:`Flag` are strongly + recommended, since :class:`IntEnum` and :class:`IntFlag` break some + semantic promises of an enumeration (by being comparable to integers, and + thus by transitivity to other unrelated enumerations). :class:`IntEnum` + and :class:`IntFlag` should be used only in cases where :class:`Enum` and + :class:`Flag` will not do; for example, when integer constants are replaced + with enumerations, or for interoperability with other systems. Others @@ -567,10 +621,10 @@ Interesting examples -------------------- -While :class:`Enum` and :class:`IntEnum` are expected to cover the majority of -use-cases, they cannot cover them all. Here are recipes for some different -types of enumerations that can be used directly, or as examples for creating -one's own. +While :class:`Enum`, :class:`IntEnum`, :class:`IntFlag`, and :class:`Flag` are +expected to cover the majority of use-cases, they cannot cover them all. Here +are recipes for some different types of enumerations that can be used directly, +or as examples for creating one's own. AutoNumber @@ -731,55 +785,33 @@ Finer Points ^^^^^^^^^^^^ -:class:`Enum` members are instances of an :class:`Enum` class, and even -though they are accessible as `EnumClass.member`, they should not be accessed -directly from the member as that lookup may fail or, worse, return something -besides the :class:`Enum` member you looking for:: +Supported ``__dunder__`` names +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - >>> class FieldTypes(Enum): - ... name = 0 - ... value = 1 - ... size = 2 - ... - >>> FieldTypes.value.size - - >>> FieldTypes.size.value - 2 +:attr:`__members__` is an :class:`OrderedDict` of ``member_name``:``member`` +items. It is only available on the class. -.. versionchanged:: 3.5 +:meth:`__new__`, if specified, must create and return the enum members; it is +also a very good idea to set the member's :attr:`_value_` appropriately. Once +all the members are created it is no longer used. -Boolean evaluation: Enum classes that are mixed with non-Enum types (such as -:class:`int`, :class:`str`, etc.) are evaluated according to the mixed-in -type's rules; otherwise, all members evaluate as ``True``. To make your own -Enum's boolean evaluation depend on the member's value add the following to -your class:: - def __bool__(self): - return bool(self.value) +Supported ``_sunder_`` names +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The :attr:`__members__` attribute is only available on the class. +- ``_name_`` -- name of the member +- ``_value_`` -- value of the member; can be set / modified in ``__new__`` -If you give your :class:`Enum` subclass extra methods, like the `Planet`_ -class above, those methods will show up in a :func:`dir` of the member, -but not of the class:: +- ``_missing_`` -- a lookup function used when a value is not found; may be + overridden +- ``_order_`` -- used in Python 2/3 code to ensure member order is consistent + (class attribute, removed during class creation) - >>> dir(Planet) - ['EARTH', 'JUPITER', 'MARS', 'MERCURY', 'NEPTUNE', 'SATURN', 'URANUS', 'VENUS', '__class__', '__doc__', '__members__', '__module__'] - >>> dir(Planet.EARTH) - ['__class__', '__doc__', '__module__', 'name', 'surface_gravity', 'value'] +.. versionadded:: 3.6 ``_missing_``, ``_order_`` -The :meth:`__new__` method will only be used for the creation of the -:class:`Enum` members -- after that it is replaced. Any custom :meth:`__new__` -method must create the object and set the :attr:`_value_` attribute -appropriately. - -If you wish to change how :class:`Enum` members are looked up you should either -write a helper function or a :func:`classmethod` for the :class:`Enum` -subclass. - -To help keep Python 2 / Python 3 code in sync a user-specified :attr:`_order_`, -if provided, will be checked to ensure the actual order of the enumeration -matches:: +To help keep Python 2 / Python 3 code in sync an :attr:`_order_` attribute can +be provided. It will be checked against the actual order of the enumeration +and raise an error if the two do not match:: >>> class Color(Enum): ... _order_ = 'red green blue' @@ -794,4 +826,53 @@ .. note:: In Python 2 code the :attr:`_order_` attribute is necessary as definition - order is lost during class creation. + order is lost before it can be recorded. + +``Enum`` member type +~~~~~~~~~~~~~~~~~~~~ + +:class:`Enum` members are instances of an :class:`Enum` class, and even +though they are accessible as `EnumClass.member`, they should not be accessed +directly from the member as that lookup may fail or, worse, return something +besides the ``Enum`` member you looking for:: + + >>> class FieldTypes(Enum): + ... name = 0 + ... value = 1 + ... size = 2 + ... + >>> FieldTypes.value.size + + >>> FieldTypes.size.value + 2 + +.. versionchanged:: 3.5 + + +Boolean value of ``Enum`` classes and members +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``Enum`` members that are mixed with non-Enum types (such as +:class:`int`, :class:`str`, etc.) are evaluated according to the mixed-in +type's rules; otherwise, all members evaluate as :data:`True`. To make your own +Enum's boolean evaluation depend on the member's value add the following to +your class:: + + def __bool__(self): + return bool(self.value) + +``Enum`` classes always evaluate as :data:`True`. + + +``Enum`` classes with methods +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you give your :class:`Enum` subclass extra methods, like the `Planet`_ +class above, those methods will show up in a :func:`dir` of the member, +but not of the class:: + + >>> dir(Planet) + ['EARTH', 'JUPITER', 'MARS', 'MERCURY', 'NEPTUNE', 'SATURN', 'URANUS', 'VENUS', '__class__', '__doc__', '__members__', '__module__'] + >>> dir(Planet.EARTH) + ['__class__', '__doc__', '__module__', 'name', 'surface_gravity', 'value'] + diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -10,7 +10,7 @@ from collections import OrderedDict -__all__ = ['EnumMeta', 'Enum', 'IntEnum', 'Flags', 'IntFlags', 'unique'] +__all__ = ['EnumMeta', 'Enum', 'IntEnum', 'Flag', 'IntFlag', 'unique'] def _is_descriptor(obj): @@ -104,7 +104,7 @@ enum_dict['_generate_next_value_'] = getattr(first_enum, '_generate_next_value_', None) return enum_dict - def __new__(metacls, cls, bases, classdict, **kwds): + def __new__(metacls, cls, bases, classdict): # an Enum class is final once enumeration items have been defined; it # cannot be mixed with other types (int, float, etc.) if it has an # inherited __new__ unless a new __new__ is defined (or the resulting @@ -614,7 +614,7 @@ def _reduce_ex_by_name(self, proto): return self.name -class Flags(Enum): +class Flag(Enum): """Support for flags""" @staticmethod def _generate_next_value_(name, start, count, last_value): @@ -736,7 +736,7 @@ return self.__class__(inverted) -class IntFlags(int, Flags): +class IntFlag(int, Flag): """Support for integer-based Flags""" @classmethod diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -3,7 +3,7 @@ import pydoc import unittest from collections import OrderedDict -from enum import Enum, IntEnum, EnumMeta, Flags, IntFlags, unique +from enum import Enum, IntEnum, EnumMeta, Flag, IntFlag, unique from io import StringIO from pickle import dumps, loads, PicklingError, HIGHEST_PROTOCOL from test import support @@ -33,6 +33,14 @@ except Exception as exc: FloatStooges = exc +try: + class FlagStooges(Flag): + LARRY = 1 + CURLY = 2 + MOE = 3 +except Exception as exc: + FlagStooges = exc + # for pickle test and subclass tests try: class StrEnum(str, Enum): @@ -1633,13 +1641,13 @@ verde = green -class TestFlags(unittest.TestCase): +class TestFlag(unittest.TestCase): """Tests of the Flags.""" - class Perm(Flags): + class Perm(Flag): R, W, X = 4, 2, 1 - class Open(Flags): + class Open(Flag): RO = 0 WO = 1 RW = 2 @@ -1760,7 +1768,7 @@ self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) def test_programatic_function_string(self): - Perm = Flags('Perm', 'R W X') + Perm = Flag('Perm', 'R W X') lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -1775,7 +1783,7 @@ self.assertIs(type(e), Perm) def test_programatic_function_string_with_start(self): - Perm = Flags('Perm', 'R W X', start=8) + Perm = Flag('Perm', 'R W X', start=8) lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -1790,7 +1798,7 @@ self.assertIs(type(e), Perm) def test_programatic_function_string_list(self): - Perm = Flags('Perm', ['R', 'W', 'X']) + Perm = Flag('Perm', ['R', 'W', 'X']) lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -1805,7 +1813,7 @@ self.assertIs(type(e), Perm) def test_programatic_function_iterable(self): - Perm = Flags('Perm', (('R', 2), ('W', 8), ('X', 32))) + Perm = Flag('Perm', (('R', 2), ('W', 8), ('X', 32))) lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -1820,7 +1828,7 @@ self.assertIs(type(e), Perm) def test_programatic_function_from_dict(self): - Perm = Flags('Perm', OrderedDict((('R', 2), ('W', 8), ('X', 32)))) + Perm = Flag('Perm', OrderedDict((('R', 2), ('W', 8), ('X', 32)))) lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -1834,16 +1842,42 @@ self.assertIn(e, Perm) self.assertIs(type(e), Perm) + def test_pickle(self): + if isinstance(FlagStooges, Exception): + raise FlagStooges + test_pickle_dump_load(self.assertIs, FlagStooges.CURLY|FlagStooges.MOE) + test_pickle_dump_load(self.assertIs, FlagStooges) -class TestIntFlags(unittest.TestCase): + + def test_containment(self): + Perm = self.Perm + R, W, X = Perm + RW = R | W + RX = R | X + WX = W | X + RWX = R | W | X + self.assertTrue(R in RW) + self.assertTrue(R in RX) + self.assertTrue(R in RWX) + self.assertTrue(W in RW) + self.assertTrue(W in WX) + self.assertTrue(W in RWX) + self.assertTrue(X in RX) + self.assertTrue(X in WX) + self.assertTrue(X in RWX) + self.assertFalse(R in WX) + self.assertFalse(W in RX) + self.assertFalse(X in RW) + +class TestIntFlag(unittest.TestCase): """Tests of the IntFlags.""" - class Perm(IntFlags): + class Perm(IntFlag): X = 1 << 0 W = 1 << 1 R = 1 << 2 - class Open(IntFlags): + class Open(IntFlag): RO = 0 WO = 1 RW = 2 @@ -2003,7 +2037,7 @@ self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) def test_programatic_function_string(self): - Perm = IntFlags('Perm', 'R W X') + Perm = IntFlag('Perm', 'R W X') lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -2019,7 +2053,7 @@ self.assertIs(type(e), Perm) def test_programatic_function_string_with_start(self): - Perm = IntFlags('Perm', 'R W X', start=8) + Perm = IntFlag('Perm', 'R W X', start=8) lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -2035,7 +2069,7 @@ self.assertIs(type(e), Perm) def test_programatic_function_string_list(self): - Perm = IntFlags('Perm', ['R', 'W', 'X']) + Perm = IntFlag('Perm', ['R', 'W', 'X']) lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -2051,7 +2085,7 @@ self.assertIs(type(e), Perm) def test_programatic_function_iterable(self): - Perm = IntFlags('Perm', (('R', 2), ('W', 8), ('X', 32))) + Perm = IntFlag('Perm', (('R', 2), ('W', 8), ('X', 32))) lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -2067,7 +2101,7 @@ self.assertIs(type(e), Perm) def test_programatic_function_from_dict(self): - Perm = IntFlags('Perm', OrderedDict((('R', 2), ('W', 8), ('X', 32)))) + Perm = IntFlag('Perm', OrderedDict((('R', 2), ('W', 8), ('X', 32)))) lst = list(Perm) self.assertEqual(len(lst), len(Perm)) self.assertEqual(len(Perm), 3, Perm) @@ -2083,6 +2117,26 @@ self.assertIs(type(e), Perm) + def test_containment(self): + Perm = self.Perm + R, W, X = Perm + RW = R | W + RX = R | X + WX = W | X + RWX = R | W | X + self.assertTrue(R in RW) + self.assertTrue(R in RX) + self.assertTrue(R in RWX) + self.assertTrue(W in RW) + self.assertTrue(W in WX) + self.assertTrue(W in RWX) + self.assertTrue(X in RX) + self.assertTrue(X in WX) + self.assertTrue(X in RWX) + self.assertFalse(R in WX) + self.assertFalse(W in RX) + self.assertFalse(X in RW) + class TestUnique(unittest.TestCase): def test_unique_clean(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 2 06:18:07 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 02 Sep 2016 10:18:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_PEP_7_style_for_if/else_in?= =?utf-8?q?_C?= Message-ID: <20160902101804.6307.51199.B7510509@psf.io> https://hg.python.org/cpython/rev/12ce8e0413f1 changeset: 103009:12ce8e0413f1 user: Victor Stinner date: Fri Sep 02 12:12:23 2016 +0200 summary: PEP 7 style for if/else in C Add also a newline for readability in normalize_encoding(). files: Lib/encodings/__init__.py | 1 + Objects/stringlib/codecs.h | 3 +- Objects/unicodeobject.c | 52 +++++++++++++++++--------- 3 files changed, 37 insertions(+), 19 deletions(-) diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py --- a/Lib/encodings/__init__.py +++ b/Lib/encodings/__init__.py @@ -54,6 +54,7 @@ """ if isinstance(encoding, bytes): encoding = str(encoding, "ascii") + chars = [] punct = False for c in encoding: diff --git a/Objects/stringlib/codecs.h b/Objects/stringlib/codecs.h --- a/Objects/stringlib/codecs.h +++ b/Objects/stringlib/codecs.h @@ -314,8 +314,9 @@ else if (Py_UNICODE_IS_SURROGATE(ch)) { Py_ssize_t startpos, endpos, newpos; Py_ssize_t k; - if (error_handler == _Py_ERROR_UNKNOWN) + if (error_handler == _Py_ERROR_UNKNOWN) { error_handler = get_error_handler(errors); + } startpos = i-1; endpos = startpos+1; diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -316,20 +316,27 @@ static _Py_error_handler get_error_handler(const char *errors) { - if (errors == NULL || strcmp(errors, "strict") == 0) + if (errors == NULL || strcmp(errors, "strict") == 0) { return _Py_ERROR_STRICT; - if (strcmp(errors, "surrogateescape") == 0) + } + if (strcmp(errors, "surrogateescape") == 0) { return _Py_ERROR_SURROGATEESCAPE; - if (strcmp(errors, "replace") == 0) + } + if (strcmp(errors, "replace") == 0) { return _Py_ERROR_REPLACE; - if (strcmp(errors, "ignore") == 0) + } + if (strcmp(errors, "ignore") == 0) { return _Py_ERROR_IGNORE; - if (strcmp(errors, "backslashreplace") == 0) + } + if (strcmp(errors, "backslashreplace") == 0) { return _Py_ERROR_BACKSLASHREPLACE; - if (strcmp(errors, "surrogatepass") == 0) + } + if (strcmp(errors, "surrogatepass") == 0) { return _Py_ERROR_SURROGATEPASS; - if (strcmp(errors, "xmlcharrefreplace") == 0) + } + if (strcmp(errors, "xmlcharrefreplace") == 0) { return _Py_ERROR_XMLCHARREFREPLACE; + } return _Py_ERROR_OTHER; } @@ -5636,36 +5643,45 @@ if (kind == PyUnicode_4BYTE_KIND) { const Py_UCS4 *in = (const Py_UCS4 *)data; const Py_UCS4 *end = in + len; - while (in < end) - if (*in++ >= 0x10000) + while (in < end) { + if (*in++ >= 0x10000) { pairs++; - } - if (len > PY_SSIZE_T_MAX / 2 - pairs - (byteorder == 0)) + } + } + } + if (len > PY_SSIZE_T_MAX / 2 - pairs - (byteorder == 0)) { return PyErr_NoMemory(); + } nsize = len + pairs + (byteorder == 0); v = PyBytes_FromStringAndSize(NULL, nsize * 2); - if (v == NULL) - return NULL; + if (v == NULL) { + return NULL; + } /* output buffer is 2-bytes aligned */ assert(_Py_IS_ALIGNED(PyBytes_AS_STRING(v), 2)); out = (unsigned short *)PyBytes_AS_STRING(v); - if (byteorder == 0) + if (byteorder == 0) { *out++ = 0xFEFF; - if (len == 0) + } + if (len == 0) { goto done; + } if (kind == PyUnicode_1BYTE_KIND) { ucs1lib_utf16_encode((const Py_UCS1 *)data, len, &out, native_ordering); goto done; } - if (byteorder < 0) + if (byteorder < 0) { encoding = "utf-16-le"; - else if (byteorder > 0) + } + else if (byteorder > 0) { encoding = "utf-16-be"; - else + } + else { encoding = "utf-16"; + } pos = 0; while (pos < len) { -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Fri Sep 2 11:11:56 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 2 Sep 2016 16:11:56 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python Default 2016-09-02 Message-ID: Results for project Python default, build date 2016-09-02 02:00:42 +0000 commit: 79422fab2ef4 previous commit: fc711879c64a revision date: 2016-09-02 01:16:32 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.22% -0.46% 9.76% 18.52% :-) pybench 0.11% -0.02% 6.48% 6.68% :-( regex_v8 2.77% 0.02% -2.78% 4.67% :-| nbody 0.09% -0.11% -1.25% 10.25% :-( json_dump_v2 0.40% -0.02% -2.23% 13.76% :-) normal_startup 0.85% 0.26% 2.00% 6.09% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-default-2016-09-02/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Fri Sep 2 11:11:16 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 2 Sep 2016 16:11:16 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python 2.7 2016-09-02 Message-ID: Results for project Python 2.7, build date 2016-09-02 02:46:59 +0000 commit: e065aec0e6fa previous commit: bf0cb86c6219 revision date: 2016-09-01 18:21:56 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.13% -1.27% 4.28% 6.38% :-) pybench 0.22% 0.06% 6.33% 3.97% :-( regex_v8 0.59% -0.08% -2.24% 10.96% :-) nbody 0.09% 0.05% 8.20% 1.86% :-) json_dump_v2 0.32% -0.60% 2.11% 10.47% :-| normal_startup 0.74% 0.12% -0.73% 2.28% :-) ssbench 0.21% 0.17% 2.93% 1.46% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-2-7-2016-09-02/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Sep 2 18:50:36 2016 From: python-checkins at python.org (ethan.furman) Date: Fri, 02 Sep 2016 22:50:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue23591=3A_optimize_=5F?= =?utf-8?q?high=5Fbit=28=29?= Message-ID: <20160902225036.10010.52239.6B61B44C@psf.io> https://hg.python.org/cpython/rev/31586a2f01b6 changeset: 103010:31586a2f01b6 user: Ethan Furman date: Fri Sep 02 15:50:21 2016 -0700 summary: issue23591: optimize _high_bit() files: Lib/enum.py | 9 ++------- 1 files changed, 2 insertions(+), 7 deletions(-) diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -784,13 +784,8 @@ def _high_bit(value): - """return the highest bit set in value""" - bit = 0 - while 'looking for the highest bit': - limit = 2 ** bit - if limit > value: - return bit - 1 - bit += 1 + """returns index of highest bit, or -1 if value is zero or negative""" + return value.bit_length() - 1 if value > 0 else -1 def unique(enumeration): """Class decorator for enumerations ensuring unique member values.""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 2 19:32:49 2016 From: python-checkins at python.org (ethan.furman) Date: Fri, 02 Sep 2016 23:32:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue23591=3A_bool=28empty?= =?utf-8?q?=5Fflags=29_=3D=3D_False=3B_more_docs_=26_tests?= Message-ID: <20160902233249.8560.58540.D38C2D84@psf.io> https://hg.python.org/cpython/rev/f33fc2117bb2 changeset: 103011:f33fc2117bb2 user: Ethan Furman date: Fri Sep 02 16:32:32 2016 -0700 summary: issue23591: bool(empty_flags) == False; more docs & tests files: Doc/library/enum.rst | 62 +++++++++++++++++++++++++++++- Lib/enum.py | 3 + Lib/test/test_enum.py | 16 ++++++++ 3 files changed, 78 insertions(+), 3 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -546,6 +546,10 @@ Any operation on an :class:`IntFlag` member besides the bit-wise operations will lose the :class:`IntFlag` membership. +.. versionadded:: 3.6 + +Sample :class:`IntFlag` class:: + >>> from enum import IntFlag >>> class Perm(IntFlag): ... R = 4 @@ -560,19 +564,71 @@ >>> Perm.R in RW True -.. versionadded:: 3.6 +It is also possible to name the combinations:: + + >>> class Perm(IntFlag): + ... R = 4 + ... W = 2 + ... X = 1 + ... RWX = 7 + >>> Perm.RWX + + >>> ~Perm.RWX + + +Another important difference between :class:`IntFlag` and :class:`Enum` is that +if no flags are set (the value is 0), its boolean evaluation is :data:`False`:: + + >>> Perm.R & Perm.X + + >>> bool(Perm.R & Perm.X) + False + +Because :class:`IntFlag` members are also subclasses of :class:`int` they can +be combined with them:: + + >>> Perm.X | 8 + Flag ^^^^ The last variation is :class:`Flag`. Like :class:`IntFlag`, :class:`Flag` -members can be combined using the bitwise operators (^, \|, ^, ~). Unlike +members can be combined using the bitwise operators (&, \|, ^, ~). Unlike :class:`IntFlag`, they cannot be combined with, nor compared against, any -other :class:`Flag` enumeration nor :class:`int`. +other :class:`Flag` enumeration, nor :class:`int`. .. versionadded:: 3.6 +Like :class:`IntFlag`, if a combination of :class:`Flag` members results in no +flags being set, the boolean evaluation is :data:`False`:: + + >>> from enum import Flag + >>> class Color(Flag): + ... red = 1 + ... blue = 2 + ... green = 4 + ... + >>> Color.red & Color.green + + >>> bool(Color.red & Color.green) + False + +Giving a name to the "no flags set" condition does not change its boolean +value:: + + >>> class Color(Flag): + ... black = 0 + ... red = 1 + ... blue = 2 + ... green = 4 + ... + >>> Color.black + + >>> bool(Color.black) + False + .. note:: For the majority of new code, :class:`Enum` and :class:`Flag` are strongly diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -714,6 +714,9 @@ '|'.join([str(m._name_ or m._value_) for m in members]), ) + def __bool__(self): + return bool(self._value_) + def __or__(self, other): if not isinstance(other, self.__class__): return NotImplemented diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1767,6 +1767,14 @@ self.assertIs(Open.WO & ~Open.WO, Open.RO) self.assertIs((Open.WO|Open.CE) & ~Open.WO, Open.CE) + def test_bool(self): + Perm = self.Perm + for f in Perm: + self.assertTrue(f) + Open = self.Open + for f in Open: + self.assertEqual(bool(f.value), bool(f)) + def test_programatic_function_string(self): Perm = Flag('Perm', 'R W X') lst = list(Perm) @@ -2137,6 +2145,14 @@ self.assertFalse(W in RX) self.assertFalse(X in RW) + def test_bool(self): + Perm = self.Perm + for f in Perm: + self.assertTrue(f) + Open = self.Open + for f in Open: + self.assertEqual(bool(f.value), bool(f)) + class TestUnique(unittest.TestCase): def test_unique_clean(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 05:18:43 2016 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 03 Sep 2016 09:18:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20160903085547.6696.12497.E481A2DF@psf.io> https://hg.python.org/cpython/rev/3ff76a59eb6b changeset: 103013:3ff76a59eb6b parent: 103011:f33fc2117bb2 parent: 103012:c3c4d8e4ca1a user: Raymond Hettinger date: Sat Sep 03 01:55:39 2016 -0700 summary: Merge files: Lib/test/test_builtin.py | 11 +++++++++++ Lib/test/test_long.py | 2 +- Misc/NEWS | 4 ++++ Python/bltinmodule.c | 2 +- 4 files changed, 17 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -3,6 +3,8 @@ import ast import builtins import collections +import decimal +import fractions import io import locale import os @@ -1244,6 +1246,15 @@ self.assertEqual(round(5e15+2), 5e15+2) self.assertEqual(round(5e15+3), 5e15+3) + def test_bug_27936(self): + # Verify that ndigits=None means the same as passing in no argument + for x in [1234, + 1234.56, + decimal.Decimal('1234.56'), + fractions.Fraction(123456, 100)]: + self.assertEqual(round(x, None), round(x)) + self.assertEqual(type(round(x, None)), type(round(x))) + def test_setattr(self): setattr(sys, 'spam', 1) self.assertEqual(sys.spam, 1) diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -1015,7 +1015,7 @@ self.assertIs(type(got), int) # bad second argument - bad_exponents = ('brian', 2.0, 0j, None) + bad_exponents = ('brian', 2.0, 0j) for e in bad_exponents: self.assertRaises(TypeError, round, 3, e) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -46,6 +46,10 @@ ``m_methods`` field to be used to add module level functions to instances of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang. +- Issue #27936: The round() function accepted a second None argument + for some types but not for others. Fixed the inconsistency by + accepting None for all numeric types. + - Issue #27487: Warn if a submodule argument to "python -m" or runpy.run_module() is found in sys.modules after parent packages are imported, but before the submodule is executed. diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -2043,7 +2043,7 @@ return NULL; } - if (ndigits == NULL) + if (ndigits == NULL || ndigits == Py_None) result = PyObject_CallFunctionObjArgs(round, NULL); else result = PyObject_CallFunctionObjArgs(round, ndigits, NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 05:18:43 2016 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 03 Sep 2016 09:18:43 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgMjc5MzY6?= =?utf-8?q?_Fix_inconsistent_round=28=29_behavior_between_float_and_int?= Message-ID: <20160903085547.9581.39092.B9BD2C22@psf.io> https://hg.python.org/cpython/rev/c3c4d8e4ca1a changeset: 103012:c3c4d8e4ca1a branch: 3.5 parent: 103006:a7ce98a4e9e4 user: Raymond Hettinger date: Sat Sep 03 01:55:11 2016 -0700 summary: Issue 27936: Fix inconsistent round() behavior between float and int files: Lib/test/test_builtin.py | 11 +++++++++++ Lib/test/test_long.py | 2 +- Misc/NEWS | 4 ++++ Python/bltinmodule.c | 2 +- 4 files changed, 17 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -3,6 +3,8 @@ import ast import builtins import collections +import decimal +import fractions import io import locale import os @@ -1244,6 +1246,15 @@ self.assertEqual(round(5e15+2), 5e15+2) self.assertEqual(round(5e15+3), 5e15+3) + def test_bug_27936(self): + # Verify that ndigits=None means the same as passing in no argument + for x in [1234, + 1234.56, + decimal.Decimal('1234.56'), + fractions.Fraction(123456, 100)]: + self.assertEqual(round(x, None), round(x)) + self.assertEqual(type(round(x, None)), type(round(x))) + def test_setattr(self): setattr(sys, 'spam', 1) self.assertEqual(sys.spam, 1) diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -967,7 +967,7 @@ self.assertIs(type(got), int) # bad second argument - bad_exponents = ('brian', 2.0, 0j, None) + bad_exponents = ('brian', 2.0, 0j) for e in bad_exponents: self.assertRaises(TypeError, round, 3, e) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -18,6 +18,10 @@ ``m_methods`` field to be used to add module level functions to instances of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang. +- Issue #27936: The round() function accepted a second None argument + for some types but not for others. Fixed the inconsistency by + accepting None for all numeric types. + - Issue #27487: Warn if a submodule argument to "python -m" or runpy.run_module() is found in sys.modules after parent packages are imported, but before the submodule is executed. diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -2039,7 +2039,7 @@ return NULL; } - if (ndigits == NULL) + if (ndigits == NULL || ndigits == Py_None) result = PyObject_CallFunctionObjArgs(round, NULL); else result = PyObject_CallFunctionObjArgs(round, ndigits, NULL); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 09:18:38 2016 From: python-checkins at python.org (eric.smith) Date: Sat, 03 Sep 2016 13:18:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_issue_27921=3A_Disa?= =?utf-8?q?llow_backslashes_anywhere_in_f-strings=2E_This_is_a?= Message-ID: <20160903131837.6465.56657.32C7A3CE@psf.io> https://hg.python.org/cpython/rev/d7ce127b5c0f changeset: 103014:d7ce127b5c0f user: Eric V. Smith date: Sat Sep 03 09:18:34 2016 -0400 summary: Closes issue 27921: Disallow backslashes anywhere in f-strings. This is a temporary restriction. In 3.6 beta 2, the plan is to again allow backslashes in the string parts of f-strings, but disallow them in the expression parts. files: Lib/test/libregrtest/save_env.py | 2 +- Lib/test/test_fstring.py | 142 +++++---------- Lib/test/test_tools/test_unparse.py | 4 - Lib/traceback.py | 4 +- Misc/NEWS | 5 + Python/ast.c | 10 + 6 files changed, 69 insertions(+), 98 deletions(-) diff --git a/Lib/test/libregrtest/save_env.py b/Lib/test/libregrtest/save_env.py --- a/Lib/test/libregrtest/save_env.py +++ b/Lib/test/libregrtest/save_env.py @@ -280,6 +280,6 @@ print(f"Warning -- {name} was modified by {self.testname}", file=sys.stderr, flush=True) if self.verbose > 1: - print(f" Before: {original}\n After: {current} ", + print(f" Before: {original}""\n"f" After: {current} ", file=sys.stderr, flush=True) return False diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -96,30 +96,6 @@ self.assertEqual(f'', '') self.assertEqual(f'a', 'a') self.assertEqual(f' ', ' ') - self.assertEqual(f'\N{GREEK CAPITAL LETTER DELTA}', - '\N{GREEK CAPITAL LETTER DELTA}') - self.assertEqual(f'\N{GREEK CAPITAL LETTER DELTA}', - '\u0394') - self.assertEqual(f'\N{True}', '\u22a8') - self.assertEqual(rf'\N{True}', r'\NTrue') - - def test_escape_order(self): - # note that hex(ord('{')) == 0x7b, so this - # string becomes f'a{4*10}b' - self.assertEqual(f'a\u007b4*10}b', 'a40b') - self.assertEqual(f'a\x7b4*10}b', 'a40b') - self.assertEqual(f'a\x7b4*10\N{RIGHT CURLY BRACKET}b', 'a40b') - self.assertEqual(f'{"a"!\N{LATIN SMALL LETTER R}}', "'a'") - self.assertEqual(f'{10\x3a02X}', '0A') - self.assertEqual(f'{10:02\N{LATIN CAPITAL LETTER X}}', '0A') - - self.assertAllRaise(SyntaxError, "f-string: single '}' is not allowed", - [r"""f'a{\u007b4*10}b'""", # mis-matched brackets - ]) - self.assertAllRaise(SyntaxError, 'unexpected character after line continuation character', - [r"""f'{"a"\!r}'""", - r"""f'{a\!r}'""", - ]) def test_unterminated_string(self): self.assertAllRaise(SyntaxError, 'f-string: unterminated string', @@ -285,8 +261,6 @@ "f'{ !r}'", "f'{10:{ }}'", "f' { } '", - r"f'{\n}'", - r"f'{\n \n}'", # Catch the empty expression before the # invalid conversion. @@ -328,24 +302,61 @@ ["f'{\n}'", ]) + def test_no_backslashes(self): + # See issue 27921 + + # These should work, but currently don't + self.assertAllRaise(SyntaxError, 'backslashes not allowed', + [r"f'\t'", + r"f'{2}\t'", + r"f'{2}\t{3}'", + r"f'\t{3}'", + + r"f'\N{GREEK CAPITAL LETTER DELTA}'", + r"f'{2}\N{GREEK CAPITAL LETTER DELTA}'", + r"f'{2}\N{GREEK CAPITAL LETTER DELTA}{3}'", + r"f'\N{GREEK CAPITAL LETTER DELTA}{3}'", + + r"f'\u0394'", + r"f'{2}\u0394'", + r"f'{2}\u0394{3}'", + r"f'\u0394{3}'", + + r"f'\U00000394'", + r"f'{2}\U00000394'", + r"f'{2}\U00000394{3}'", + r"f'\U00000394{3}'", + + r"f'\x20'", + r"f'{2}\x20'", + r"f'{2}\x20{3}'", + r"f'\x20{3}'", + + r"f'2\x20'", + r"f'2\x203'", + r"f'2\x203'", + ]) + + # And these don't work now, and shouldn't work in the future. + self.assertAllRaise(SyntaxError, 'backslashes not allowed', + [r"f'{\'a\'}'", + r"f'{\t3}'", + ]) + + # add this when backslashes are allowed again. see issue 27921 + # these test will be needed because unicode names will be parsed + # differently once backslashes are allowed inside expressions + ## def test_misformed_unicode_character_name(self): + ## self.assertAllRaise(SyntaxError, 'xx', + ## [r"f'\N'", + ## [r"f'\N{'", + ## [r"f'\N{GREEK CAPITAL LETTER DELTA'", + ## ]) + def test_newlines_in_expressions(self): self.assertEqual(f'{0}', '0') - self.assertEqual(f'{0\n}', '0') - self.assertEqual(f'{0\r}', '0') - self.assertEqual(f'{\n0\n}', '0') - self.assertEqual(f'{\r0\r}', '0') - self.assertEqual(f'{\n0\r}', '0') - self.assertEqual(f'{\n0}', '0') - self.assertEqual(f'{3+\n4}', '7') - self.assertEqual(f'{3+\\\n4}', '7') self.assertEqual(rf'''{3+ 4}''', '7') - self.assertEqual(f'''{3+\ -4}''', '7') - - self.assertAllRaise(SyntaxError, 'f-string: empty expression not allowed', - [r"f'{\n}'", - ]) def test_lambda(self): x = 5 @@ -380,9 +391,6 @@ def test_expressions_with_triple_quoted_strings(self): self.assertEqual(f"{'''x'''}", 'x') self.assertEqual(f"{'''eric's'''}", "eric's") - self.assertEqual(f'{"""eric\'s"""}', "eric's") - self.assertEqual(f"{'''eric\"s'''}", 'eric"s') - self.assertEqual(f'{"""eric"s"""}', 'eric"s') # Test concatenation within an expression self.assertEqual(f'{"x" """eric"s""" "y"}', 'xeric"sy') @@ -484,10 +492,6 @@ y = 5 self.assertEqual(f'{f"{0}"*3}', '000') self.assertEqual(f'{f"{y}"*3}', '555') - self.assertEqual(f'{f"{\'x\'}"*3}', 'xxx') - - self.assertEqual(f"{r'x' f'{\"s\"}'}", 'xs') - self.assertEqual(f"{r'x'rf'{\"s\"}'}", 'xs') def test_invalid_string_prefixes(self): self.assertAllRaise(SyntaxError, 'unexpected EOF while parsing', @@ -510,24 +514,14 @@ def test_leading_trailing_spaces(self): self.assertEqual(f'{ 3}', '3') self.assertEqual(f'{ 3}', '3') - self.assertEqual(f'{\t3}', '3') - self.assertEqual(f'{\t\t3}', '3') self.assertEqual(f'{3 }', '3') self.assertEqual(f'{3 }', '3') - self.assertEqual(f'{3\t}', '3') - self.assertEqual(f'{3\t\t}', '3') self.assertEqual(f'expr={ {x: y for x, y in [(1, 2), ]}}', 'expr={1: 2}') self.assertEqual(f'expr={ {x: y for x, y in [(1, 2), ]} }', 'expr={1: 2}') - def test_character_name(self): - self.assertEqual(f'{4}\N{GREEK CAPITAL LETTER DELTA}{3}', - '4\N{GREEK CAPITAL LETTER DELTA}3') - self.assertEqual(f'{{}}\N{GREEK CAPITAL LETTER DELTA}{3}', - '{}\N{GREEK CAPITAL LETTER DELTA}3') - def test_not_equal(self): # There's a special test for this because there's a special # case in the f-string parser to look for != as not ending an @@ -554,10 +548,6 @@ # Not a conversion, but show that ! is allowed in a format spec. self.assertEqual(f'{3.14:!<10.10}', '3.14!!!!!!') - self.assertEqual(f'{"\N{GREEK CAPITAL LETTER DELTA}"}', '\u0394') - self.assertEqual(f'{"\N{GREEK CAPITAL LETTER DELTA}"!r}', "'\u0394'") - self.assertEqual(f'{"\N{GREEK CAPITAL LETTER DELTA}"!a}', "'\\u0394'") - self.assertAllRaise(SyntaxError, 'f-string: invalid conversion character', ["f'{3!g}'", "f'{3!A}'", @@ -565,9 +555,7 @@ "f'{3!A}'", "f'{3!!}'", "f'{3!:}'", - "f'{3!\N{GREEK CAPITAL LETTER DELTA}}'", "f'{3! s}'", # no space before conversion char - "f'{x!\\x00:.<10}'", ]) self.assertAllRaise(SyntaxError, "f-string: expecting '}'", @@ -600,7 +588,6 @@ # Can't have { or } in a format spec. "f'{3:}>10}'", - r"f'{3:\\}>10}'", "f'{3:}}>10}'", ]) @@ -620,10 +607,6 @@ "f'{'", ]) - self.assertAllRaise(SyntaxError, 'invalid syntax', - [r"f'{3:\\{>10}'", - ]) - # But these are just normal strings. self.assertEqual(f'{"{"}', '{') self.assertEqual(f'{"}"}', '}') @@ -712,34 +695,11 @@ "'": 'squote', 'foo': 'bar', } - self.assertEqual(f'{d["\'"]}', 'squote') - self.assertEqual(f"{d['\"']}", 'dquote') - self.assertEqual(f'''{d["'"]}''', 'squote') self.assertEqual(f"""{d['"']}""", 'dquote') self.assertEqual(f'{d["foo"]}', 'bar') self.assertEqual(f"{d['foo']}", 'bar') - self.assertEqual(f'{d[\'foo\']}', 'bar') - self.assertEqual(f"{d[\"foo\"]}", 'bar') - - def test_escaped_quotes(self): - d = {'"': 'a', - "'": 'b'} - - self.assertEqual(fr"{d['\"']}", 'a') - self.assertEqual(fr'{d["\'"]}', 'b') - self.assertEqual(fr"{'\"'}", '"') - self.assertEqual(fr'{"\'"}', "'") - self.assertEqual(f'{"\\"3"}', '"3') - - self.assertAllRaise(SyntaxError, 'f-string: unterminated string', - [r'''f'{"""\\}' ''', # Backslash at end of expression - ]) - self.assertAllRaise(SyntaxError, 'unexpected character after line continuation', - [r"rf'{3\}'", - ]) - if __name__ == '__main__': unittest.main() diff --git a/Lib/test/test_tools/test_unparse.py b/Lib/test/test_tools/test_unparse.py --- a/Lib/test/test_tools/test_unparse.py +++ b/Lib/test/test_tools/test_unparse.py @@ -138,10 +138,6 @@ # See issue 25180 self.check_roundtrip(r"""f'{f"{0}"*3}'""") self.check_roundtrip(r"""f'{f"{y}"*3}'""") - self.check_roundtrip(r"""f'{f"{\'x\'}"*3}'""") - - self.check_roundtrip(r'''f"{r'x' f'{\"s\"}'}"''') - self.check_roundtrip(r'''f"{r'x'rf'{\"s\"}'}"''') def test_del_statement(self): self.check_roundtrip("del x, y, z") diff --git a/Lib/traceback.py b/Lib/traceback.py --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -402,7 +402,7 @@ count += 1 else: if count > 3: - result.append(f' [Previous line repeated {count-3} more times]\n') + result.append(f' [Previous line repeated {count-3} more times]''\n') last_file = frame.filename last_line = frame.lineno last_name = frame.name @@ -419,7 +419,7 @@ row.append(' {name} = {value}\n'.format(name=name, value=value)) result.append(''.join(row)) if count > 3: - result.append(f' [Previous line repeated {count-3} more times]\n') + result.append(f' [Previous line repeated {count-3} more times]''\n') return result diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,11 @@ Core and Builtins ----------------- +- Issue #27921: Disallow backslashes in f-strings. This is a temporary + restriction: in beta 2, backslashes will only be disallowed inside + the braces (where the expressions are). This is a breaking change + from the 3.6 alpha releases. + - Issue #27870: A left shift of zero by a large integer no longer attempts to allocate large amounts of memory. diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -4958,6 +4958,16 @@ return NULL; } } + + /* Temporary hack: if this is an f-string, no backslashes are allowed. */ + /* See issue 27921. */ + if (*fmode && strchr(s, '\\') != NULL) { + /* Syntax error. At a later date fix this so it only checks for + backslashes within the braces. */ + ast_error(c, n, "backslashes not allowed in f-strings"); + return NULL; + } + /* Avoid invoking escape decoding routines if possible. */ rawmode = rawmode || strchr(s, '\\') == NULL; if (*bytesmode) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 10:43:25 2016 From: python-checkins at python.org (eric.smith) Date: Sat, 03 Sep 2016 14:43:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27921=3A_Remove_back?= =?utf-8?q?slash_from_another_f-string=2E?= Message-ID: <20160903144325.9612.34174.C968AB8A@psf.io> https://hg.python.org/cpython/rev/899ee1e68a8d changeset: 103015:899ee1e68a8d user: Eric V. Smith date: Sat Sep 03 10:43:20 2016 -0400 summary: Issue 27921: Remove backslash from another f-string. files: Lib/http/client.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -1060,7 +1060,7 @@ if encode_chunked and self._http_vsn == 11: # chunked encoding - chunk = f'{len(chunk):X}\r\n'.encode('ascii') + chunk \ + chunk = f'{len(chunk):X}''\r\n'.encode('ascii') + chunk \ + b'\r\n' self.send(chunk) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 11:07:30 2016 From: python-checkins at python.org (eric.smith) Date: Sat, 03 Sep 2016 15:07:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27921=3A_Remove_back?= =?utf-8?q?slash_from_another_f-string=2E_I=27ll_revert_this_change?= Message-ID: <20160903150730.104299.32752.16DBE40C@psf.io> https://hg.python.org/cpython/rev/80eb6eb57537 changeset: 103016:80eb6eb57537 user: Eric V. Smith date: Sat Sep 03 11:01:53 2016 -0400 summary: Issue 27921: Remove backslash from another f-string. I'll revert this change before beta 2. files: Lib/test/test_traceback.py | 28 +++++++++++++------------- 1 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -325,13 +325,13 @@ lineno_f = f.__code__.co_firstlineno result_f = ( 'Traceback (most recent call last):\n' - f' File "{__file__}", line {lineno_f+5}, in _check_recursive_traceback_display\n' + f' File "{__file__}", line {lineno_f+5}, in _check_recursive_traceback_display''\n' ' f()\n' - f' File "{__file__}", line {lineno_f+1}, in f\n' + f' File "{__file__}", line {lineno_f+1}, in f''\n' ' f()\n' - f' File "{__file__}", line {lineno_f+1}, in f\n' + f' File "{__file__}", line {lineno_f+1}, in f''\n' ' f()\n' - f' File "{__file__}", line {lineno_f+1}, in f\n' + f' File "{__file__}", line {lineno_f+1}, in f''\n' ' f()\n' # XXX: The following line changes depending on whether the tests # are run through the interactive interpreter or with -m @@ -370,20 +370,20 @@ lineno_g = g.__code__.co_firstlineno result_g = ( - f' File "{__file__}", line {lineno_g+2}, in g\n' + f' File "{__file__}", line {lineno_g+2}, in g''\n' ' return g(count-1)\n' - f' File "{__file__}", line {lineno_g+2}, in g\n' + f' File "{__file__}", line {lineno_g+2}, in g''\n' ' return g(count-1)\n' - f' File "{__file__}", line {lineno_g+2}, in g\n' + f' File "{__file__}", line {lineno_g+2}, in g''\n' ' return g(count-1)\n' ' [Previous line repeated 6 more times]\n' - f' File "{__file__}", line {lineno_g+3}, in g\n' + f' File "{__file__}", line {lineno_g+3}, in g''\n' ' raise ValueError\n' 'ValueError\n' ) tb_line = ( 'Traceback (most recent call last):\n' - f' File "{__file__}", line {lineno_g+7}, in _check_recursive_traceback_display\n' + f' File "{__file__}", line {lineno_g+7}, in _check_recursive_traceback_display''\n' ' g()\n' ) expected = (tb_line + result_g).splitlines() @@ -407,16 +407,16 @@ lineno_h = h.__code__.co_firstlineno result_h = ( 'Traceback (most recent call last):\n' - f' File "{__file__}", line {lineno_h+7}, in _check_recursive_traceback_display\n' + f' File "{__file__}", line {lineno_h+7}, in _check_recursive_traceback_display''\n' ' h()\n' - f' File "{__file__}", line {lineno_h+2}, in h\n' + f' File "{__file__}", line {lineno_h+2}, in h''\n' ' return h(count-1)\n' - f' File "{__file__}", line {lineno_h+2}, in h\n' + f' File "{__file__}", line {lineno_h+2}, in h''\n' ' return h(count-1)\n' - f' File "{__file__}", line {lineno_h+2}, in h\n' + f' File "{__file__}", line {lineno_h+2}, in h''\n' ' return h(count-1)\n' ' [Previous line repeated 6 more times]\n' - f' File "{__file__}", line {lineno_h+3}, in h\n' + f' File "{__file__}", line {lineno_h+3}, in h''\n' ' g()\n' ) expected = (result_h + result_g).splitlines() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 11:55:44 2016 From: python-checkins at python.org (vinay.sajip) Date: Sat, 03 Sep 2016 15:55:44 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogRml4ZXMgIzI3OTM3?= =?utf-8?q?=3A_optimise_code_used_in_all_logging_calls=2E?= Message-ID: <20160903155542.8310.54344.975DF77A@psf.io> https://hg.python.org/cpython/rev/e4b6faf22e8d changeset: 103018:e4b6faf22e8d branch: 3.5 parent: 103012:c3c4d8e4ca1a user: Vinay Sajip date: Sat Sep 03 16:50:09 2016 +0100 summary: Fixes #27937: optimise code used in all logging calls. files: Lib/logging/__init__.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -1,4 +1,4 @@ -# Copyright 2001-2015 by Vinay Sajip. All Rights Reserved. +# Copyright 2001-2016 by Vinay Sajip. All Rights Reserved. # # Permission to use, copy, modify, and distribute this software and its # documentation for any purpose and without fee is hereby granted, @@ -18,7 +18,7 @@ Logging package for Python. Based on PEP 282 and comments thereto in comp.lang.python. -Copyright (C) 2001-2015 Vinay Sajip. All Rights Reserved. +Copyright (C) 2001-2016 Vinay Sajip. All Rights Reserved. To use, simply 'import logging' and log away! """ @@ -129,8 +129,9 @@ Otherwise, the string "Level %s" % level is returned. """ - # See Issue #22386 for the reason for this convoluted expression - return _levelToName.get(level, _nameToLevel.get(level, ("Level %s" % level))) + # See Issues #22386 and #27937 for why it's this way + return (_levelToName.get(level) or _nameToLevel.get(level) or + "Level %s" % level) def addLevelName(level, levelName): """ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 11:55:44 2016 From: python-checkins at python.org (vinay.sajip) Date: Sat, 03 Sep 2016 15:55:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixes_=2327937=3A_optimise?= =?utf-8?q?_code_used_in_all_logging_calls=2E?= Message-ID: <20160903155542.8560.89703.3A49BD12@psf.io> https://hg.python.org/cpython/rev/0986401d0733 changeset: 103017:0986401d0733 parent: 103015:899ee1e68a8d user: Vinay Sajip date: Sat Sep 03 15:56:07 2016 +0100 summary: Fixes #27937: optimise code used in all logging calls. files: Lib/logging/__init__.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -130,8 +130,9 @@ Otherwise, the string "Level %s" % level is returned. """ - # See Issue #22386 for the reason for this convoluted expression - return _levelToName.get(level, _nameToLevel.get(level, ("Level %s" % level))) + # See Issues #22386 and #27937 for why it's this way + return (_levelToName.get(level) or _nameToLevel.get(level) or + "Level %s" % level) def addLevelName(level, levelName): """ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 11:55:44 2016 From: python-checkins at python.org (vinay.sajip) Date: Sat, 03 Sep 2016 15:55:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327937=3A_Merge_fix_from_3=2E5=2E?= Message-ID: <20160903155542.23439.34730.71C6A5C0@psf.io> https://hg.python.org/cpython/rev/773c9401735f changeset: 103019:773c9401735f parent: 103017:0986401d0733 parent: 103018:e4b6faf22e8d user: Vinay Sajip date: Sat Sep 03 16:54:41 2016 +0100 summary: Closes #27937: Merge fix from 3.5. files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 11:55:44 2016 From: python-checkins at python.org (vinay.sajip) Date: Sat, 03 Sep 2016 15:55:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_upstream_change=2E?= Message-ID: <20160903155542.23567.83066.6EF2FD27@psf.io> https://hg.python.org/cpython/rev/0bd8975e2407 changeset: 103020:0bd8975e2407 parent: 103019:773c9401735f parent: 103016:80eb6eb57537 user: Vinay Sajip date: Sat Sep 03 16:55:35 2016 +0100 summary: Merge upstream change. files: Lib/test/test_traceback.py | 28 +++++++++++++------------- 1 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -325,13 +325,13 @@ lineno_f = f.__code__.co_firstlineno result_f = ( 'Traceback (most recent call last):\n' - f' File "{__file__}", line {lineno_f+5}, in _check_recursive_traceback_display\n' + f' File "{__file__}", line {lineno_f+5}, in _check_recursive_traceback_display''\n' ' f()\n' - f' File "{__file__}", line {lineno_f+1}, in f\n' + f' File "{__file__}", line {lineno_f+1}, in f''\n' ' f()\n' - f' File "{__file__}", line {lineno_f+1}, in f\n' + f' File "{__file__}", line {lineno_f+1}, in f''\n' ' f()\n' - f' File "{__file__}", line {lineno_f+1}, in f\n' + f' File "{__file__}", line {lineno_f+1}, in f''\n' ' f()\n' # XXX: The following line changes depending on whether the tests # are run through the interactive interpreter or with -m @@ -370,20 +370,20 @@ lineno_g = g.__code__.co_firstlineno result_g = ( - f' File "{__file__}", line {lineno_g+2}, in g\n' + f' File "{__file__}", line {lineno_g+2}, in g''\n' ' return g(count-1)\n' - f' File "{__file__}", line {lineno_g+2}, in g\n' + f' File "{__file__}", line {lineno_g+2}, in g''\n' ' return g(count-1)\n' - f' File "{__file__}", line {lineno_g+2}, in g\n' + f' File "{__file__}", line {lineno_g+2}, in g''\n' ' return g(count-1)\n' ' [Previous line repeated 6 more times]\n' - f' File "{__file__}", line {lineno_g+3}, in g\n' + f' File "{__file__}", line {lineno_g+3}, in g''\n' ' raise ValueError\n' 'ValueError\n' ) tb_line = ( 'Traceback (most recent call last):\n' - f' File "{__file__}", line {lineno_g+7}, in _check_recursive_traceback_display\n' + f' File "{__file__}", line {lineno_g+7}, in _check_recursive_traceback_display''\n' ' g()\n' ) expected = (tb_line + result_g).splitlines() @@ -407,16 +407,16 @@ lineno_h = h.__code__.co_firstlineno result_h = ( 'Traceback (most recent call last):\n' - f' File "{__file__}", line {lineno_h+7}, in _check_recursive_traceback_display\n' + f' File "{__file__}", line {lineno_h+7}, in _check_recursive_traceback_display''\n' ' h()\n' - f' File "{__file__}", line {lineno_h+2}, in h\n' + f' File "{__file__}", line {lineno_h+2}, in h''\n' ' return h(count-1)\n' - f' File "{__file__}", line {lineno_h+2}, in h\n' + f' File "{__file__}", line {lineno_h+2}, in h''\n' ' return h(count-1)\n' - f' File "{__file__}", line {lineno_h+2}, in h\n' + f' File "{__file__}", line {lineno_h+2}, in h''\n' ' return h(count-1)\n' ' [Previous line repeated 6 more times]\n' - f' File "{__file__}", line {lineno_h+3}, in h\n' + f' File "{__file__}", line {lineno_h+3}, in h''\n' ' g()\n' ) expected = (result_h + result_g).splitlines() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 12:04:43 2016 From: python-checkins at python.org (vinay.sajip) Date: Sat, 03 Sep 2016 16:04:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2327935=3A_returne?= =?utf-8?q?d_numeric_value_for_=27FATAL=27_logging_level=2E?= Message-ID: <20160903160442.68709.62675.36F61D2C@psf.io> https://hg.python.org/cpython/rev/85f9f8bf2ec8 changeset: 103021:85f9f8bf2ec8 user: Vinay Sajip date: Sat Sep 03 17:04:36 2016 +0100 summary: Closes #27935: returned numeric value for 'FATAL' logging level. files: Lib/logging/__init__.py | 1 + Lib/test/test_logging.py | 4 ++++ 2 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -108,6 +108,7 @@ } _nameToLevel = { 'CRITICAL': CRITICAL, + 'FATAL': FATAL, 'ERROR': ERROR, 'WARN': WARNING, 'WARNING': WARNING, diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -309,6 +309,10 @@ self.assertEqual(logging.getLevelName('INFO'), logging.INFO) self.assertEqual(logging.getLevelName(logging.INFO), 'INFO') + def test_issue27935(self): + fatal = logging.getLevelName('FATAL') + self.assertEqual(fatal, logging.FATAL) + class BasicFilterTest(BaseTest): """Test the bundled Filter class.""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 12:23:01 2016 From: python-checkins at python.org (mark.dickinson) Date: Sat, 03 Sep 2016 16:23:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2311734=3A_Add_supp?= =?utf-8?q?ort_for_IEEE_754_half-precision_floats_to_the_struct?= Message-ID: <20160903162301.69071.94173.1F1FABDF@psf.io> https://hg.python.org/cpython/rev/519bde9db8e0 changeset: 103022:519bde9db8e0 user: Mark Dickinson date: Sat Sep 03 17:21:29 2016 +0100 summary: Issue #11734: Add support for IEEE 754 half-precision floats to the struct module. Original patch by Eli Stevens. files: Doc/library/struct.rst | 23 +++- Include/floatobject.h | 10 +- Lib/test/test_struct.py | 107 +++++++++++++++- Misc/ACKS | 1 + Misc/NEWS | 3 + Modules/_struct.c | 76 +++++++++++- Objects/floatobject.c | 184 +++++++++++++++++++++++++++- 7 files changed, 393 insertions(+), 11 deletions(-) diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -216,6 +216,8 @@ +--------+--------------------------+--------------------+----------------+------------+ | ``N`` | :c:type:`size_t` | integer | | \(4) | +--------+--------------------------+--------------------+----------------+------------+ +| ``e`` | \(7) | float | 2 | \(5) | ++--------+--------------------------+--------------------+----------------+------------+ | ``f`` | :c:type:`float` | float | 4 | \(5) | +--------+--------------------------+--------------------+----------------+------------+ | ``d`` | :c:type:`double` | float | 8 | \(5) | @@ -257,9 +259,10 @@ fits your application. (5) - For the ``'f'`` and ``'d'`` conversion codes, the packed representation uses - the IEEE 754 binary32 (for ``'f'``) or binary64 (for ``'d'``) format, - regardless of the floating-point format used by the platform. + For the ``'f'``, ``'d'`` and ``'e'`` conversion codes, the packed + representation uses the IEEE 754 binary32, binary64 or binary16 format (for + ``'f'``, ``'d'`` or ``'e'`` respectively), regardless of the floating-point + format used by the platform. (6) The ``'P'`` format character is only available for the native byte ordering @@ -268,6 +271,16 @@ on the host system. The struct module does not interpret this as native ordering, so the ``'P'`` format is not available. +(7) + The IEEE 754 binary16 "half precision" type was introduced in the 2008 + revision of the `IEEE 754 standard `_. It has a sign + bit, a 5-bit exponent and 11-bit precision (with 10 bits explicitly stored), + and can represent numbers between approximately ``6.1e-05`` and ``6.5e+04`` + at full precision. This type is not widely supported by C compilers: on a + typical machine, an unsigned short can be used for storage, but not for math + operations. See the Wikipedia page on the `half-precision floating-point + format `_ for more information. + A format character may be preceded by an integral repeat count. For example, the format string ``'4h'`` means exactly the same as ``'hhhh'``. @@ -430,3 +443,7 @@ The calculated size of the struct (and hence of the bytes object produced by the :meth:`pack` method) corresponding to :attr:`format`. + +.. _half precision format: https://en.wikipedia.org/wiki/Half-precision_floating-point_format + +.. _ieee 754 standard: https://en.wikipedia.org/wiki/IEEE_floating_point#IEEE_754-2008 diff --git a/Include/floatobject.h b/Include/floatobject.h --- a/Include/floatobject.h +++ b/Include/floatobject.h @@ -74,9 +74,9 @@ * happens in such cases is partly accidental (alas). */ -/* The pack routines write 4 or 8 bytes, starting at p. le is a bool +/* The pack routines write 2, 4 or 8 bytes, starting at p. le is a bool * argument, true if you want the string in little-endian format (exponent - * last, at p+3 or p+7), false if you want big-endian format (exponent + * last, at p+1, p+3 or p+7), false if you want big-endian format (exponent * first, at p). * Return value: 0 if all is OK, -1 if error (and an exception is * set, most likely OverflowError). @@ -84,6 +84,7 @@ * 1): What this does is undefined if x is a NaN or infinity. * 2): -0.0 and +0.0 produce the same string. */ +PyAPI_FUNC(int) _PyFloat_Pack2(double x, unsigned char *p, int le); PyAPI_FUNC(int) _PyFloat_Pack4(double x, unsigned char *p, int le); PyAPI_FUNC(int) _PyFloat_Pack8(double x, unsigned char *p, int le); @@ -96,14 +97,15 @@ PyAPI_FUNC(int) _PyFloat_Digits(char *buf, double v, int *signum); PyAPI_FUNC(void) _PyFloat_DigitsInit(void); -/* The unpack routines read 4 or 8 bytes, starting at p. le is a bool +/* The unpack routines read 2, 4 or 8 bytes, starting at p. le is a bool * argument, true if the string is in little-endian format (exponent - * last, at p+3 or p+7), false if big-endian (exponent first, at p). + * last, at p+1, p+3 or p+7), false if big-endian (exponent first, at p). * Return value: The unpacked double. On error, this is -1.0 and * PyErr_Occurred() is true (and an exception is set, most likely * OverflowError). Note that on a non-IEEE platform this will refuse * to unpack a string that represents a NaN or infinity. */ +PyAPI_FUNC(double) _PyFloat_Unpack2(const unsigned char *p, int le); PyAPI_FUNC(double) _PyFloat_Unpack4(const unsigned char *p, int le); PyAPI_FUNC(double) _PyFloat_Unpack8(const unsigned char *p, int le); diff --git a/Lib/test/test_struct.py b/Lib/test/test_struct.py --- a/Lib/test/test_struct.py +++ b/Lib/test/test_struct.py @@ -1,5 +1,6 @@ from collections import abc import array +import math import operator import unittest import struct @@ -366,8 +367,6 @@ # SF bug 705836. "f" had a severe rounding bug, where a carry # from the low-order discarded bits could propagate into the exponent # field, causing the result to be wrong by a factor of 2. - import math - for base in range(1, 33): # smaller <- largest representable float less than base. delta = 0.5 @@ -659,6 +658,110 @@ self.assertRaises(StopIteration, next, it) self.assertRaises(StopIteration, next, it) + def test_half_float(self): + # Little-endian examples from: + # http://en.wikipedia.org/wiki/Half_precision_floating-point_format + format_bits_float__cleanRoundtrip_list = [ + (b'\x00\x3c', 1.0), + (b'\x00\xc0', -2.0), + (b'\xff\x7b', 65504.0), # (max half precision) + (b'\x00\x04', 2**-14), # ~= 6.10352 * 10**-5 (min pos normal) + (b'\x01\x00', 2**-24), # ~= 5.96046 * 10**-8 (min pos subnormal) + (b'\x00\x00', 0.0), + (b'\x00\x80', -0.0), + (b'\x00\x7c', float('+inf')), + (b'\x00\xfc', float('-inf')), + (b'\x55\x35', 0.333251953125), # ~= 1/3 + ] + + for le_bits, f in format_bits_float__cleanRoundtrip_list: + be_bits = le_bits[::-1] + self.assertEqual(f, struct.unpack('e', be_bits)[0]) + self.assertEqual(be_bits, struct.pack('>e', f)) + if sys.byteorder == 'little': + self.assertEqual(f, struct.unpack('e', le_bits)[0]) + self.assertEqual(le_bits, struct.pack('e', f)) + else: + self.assertEqual(f, struct.unpack('e', be_bits)[0]) + self.assertEqual(be_bits, struct.pack('e', f)) + + # Check for NaN handling: + format_bits__nan_list = [ + ('e', bits[::-1])[0])) + + # Check that packing produces a bit pattern representing a quiet NaN: + # all exponent bits and the msb of the fraction should all be 1. + packed = struct.pack('e', b'\x00\x01', 2.0**-25 + 2.0**-35), # Rounds to minimum subnormal + ('>e', b'\x00\x00', 2.0**-25), # Underflows to zero (nearest even mode) + ('>e', b'\x00\x00', 2.0**-26), # Underflows to zero + ('>e', b'\x03\xff', 2.0**-14 - 2.0**-24), # Largest subnormal. + ('>e', b'\x03\xff', 2.0**-14 - 2.0**-25 - 2.0**-65), + ('>e', b'\x04\x00', 2.0**-14 - 2.0**-25), + ('>e', b'\x04\x00', 2.0**-14), # Smallest normal. + ('>e', b'\x3c\x01', 1.0+2.0**-11 + 2.0**-16), # rounds to 1.0+2**(-10) + ('>e', b'\x3c\x00', 1.0+2.0**-11), # rounds to 1.0 (nearest even mode) + ('>e', b'\x3c\x00', 1.0+2.0**-12), # rounds to 1.0 + ('>e', b'\x7b\xff', 65504), # largest normal + ('>e', b'\x7b\xff', 65519), # rounds to 65504 + ('>e', b'\x80\x01', -2.0**-25 - 2.0**-35), # Rounds to minimum subnormal + ('>e', b'\x80\x00', -2.0**-25), # Underflows to zero (nearest even mode) + ('>e', b'\x80\x00', -2.0**-26), # Underflows to zero + ('>e', b'\xbc\x01', -1.0-2.0**-11 - 2.0**-16), # rounds to 1.0+2**(-10) + ('>e', b'\xbc\x00', -1.0-2.0**-11), # rounds to 1.0 (nearest even mode) + ('>e', b'\xbc\x00', -1.0-2.0**-12), # rounds to 1.0 + ('>e', b'\xfb\xff', -65519), # rounds to 65504 + ] + + for formatcode, bits, f in format_bits_float__rounding_list: + self.assertEqual(bits, struct.pack(formatcode, f)) + + # This overflows, and so raises an error + format_bits_float__roundingError_list = [ + # Values that round to infinity. + ('>e', 65520.0), + ('>e', 65536.0), + ('>e', 1e300), + ('>e', -65520.0), + ('>e', -65536.0), + ('>e', -1e300), + ('e', b'\x67\xff', 0x1ffdffffff * 2**-26), # should be 2047, if double-rounded 64>32>16, becomes 2048 + ] + + for formatcode, bits, f in format_bits_float__doubleRoundingError_list: + self.assertEqual(bits, struct.pack(formatcode, f)) + if __name__ == '__main__': unittest.main() diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1435,6 +1435,7 @@ Marek Stepniowski Baruch Sterin Chris Stern +Eli Stevens Alex Stewart Victor Stinner Richard Stoakley diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -69,6 +69,9 @@ Library ------- +- Issue #11734: Add support for IEEE 754 half-precision floats to the + struct module. Based on a patch by Eli Stevens. + - Issue #27919: Deprecated ``extra_path`` distribution option in distutils packaging. diff --git a/Modules/_struct.c b/Modules/_struct.c --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -267,6 +267,33 @@ /* Floating point helpers */ static PyObject * +unpack_halffloat(const char *p, /* start of 2-byte string */ + int le) /* true for little-endian, false for big-endian */ +{ + double x; + + x = _PyFloat_Unpack2((unsigned char *)p, le); + if (x == -1.0 && PyErr_Occurred()) { + return NULL; + } + return PyFloat_FromDouble(x); +} + +static int +pack_halffloat(char *p, /* start of 2-byte string */ + PyObject *v, /* value to pack */ + int le) /* true for little-endian, false for big-endian */ +{ + double x = PyFloat_AsDouble(v); + if (x == -1.0 && PyErr_Occurred()) { + PyErr_SetString(StructError, + "required argument is not a float"); + return -1; + } + return _PyFloat_Pack2(x, (unsigned char *)p, le); +} + +static PyObject * unpack_float(const char *p, /* start of 4-byte string */ int le) /* true for little-endian, false for big-endian */ { @@ -470,6 +497,16 @@ static PyObject * +nu_halffloat(const char *p, const formatdef *f) +{ +#if PY_LITTLE_ENDIAN + return unpack_halffloat(p, 1); +#else + return unpack_halffloat(p, 0); +#endif +} + +static PyObject * nu_float(const char *p, const formatdef *f) { float x; @@ -681,6 +718,16 @@ } static int +np_halffloat(char *p, PyObject *v, const formatdef *f) +{ +#if PY_LITTLE_ENDIAN + return pack_halffloat(p, v, 1); +#else + return pack_halffloat(p, v, 0); +#endif +} + +static int np_float(char *p, PyObject *v, const formatdef *f) { float x = (float)PyFloat_AsDouble(v); @@ -743,6 +790,7 @@ {'Q', sizeof(PY_LONG_LONG), LONG_LONG_ALIGN, nu_ulonglong,np_ulonglong}, #endif {'?', sizeof(BOOL_TYPE), BOOL_ALIGN, nu_bool, np_bool}, + {'e', sizeof(short), SHORT_ALIGN, nu_halffloat, np_halffloat}, {'f', sizeof(float), FLOAT_ALIGN, nu_float, np_float}, {'d', sizeof(double), DOUBLE_ALIGN, nu_double, np_double}, {'P', sizeof(void *), VOID_P_ALIGN, nu_void_p, np_void_p}, @@ -826,6 +874,12 @@ } static PyObject * +bu_halffloat(const char *p, const formatdef *f) +{ + return unpack_halffloat(p, 0); +} + +static PyObject * bu_float(const char *p, const formatdef *f) { return unpack_float(p, 0); @@ -922,6 +976,12 @@ } static int +bp_halffloat(char *p, PyObject *v, const formatdef *f) +{ + return pack_halffloat(p, v, 0); +} + +static int bp_float(char *p, PyObject *v, const formatdef *f) { double x = PyFloat_AsDouble(v); @@ -972,6 +1032,7 @@ {'q', 8, 0, bu_longlong, bp_longlong}, {'Q', 8, 0, bu_ulonglong, bp_ulonglong}, {'?', 1, 0, bu_bool, bp_bool}, + {'e', 2, 0, bu_halffloat, bp_halffloat}, {'f', 4, 0, bu_float, bp_float}, {'d', 8, 0, bu_double, bp_double}, {0} @@ -1054,6 +1115,12 @@ } static PyObject * +lu_halffloat(const char *p, const formatdef *f) +{ + return unpack_halffloat(p, 1); +} + +static PyObject * lu_float(const char *p, const formatdef *f) { return unpack_float(p, 1); @@ -1142,6 +1209,12 @@ } static int +lp_halffloat(char *p, PyObject *v, const formatdef *f) +{ + return pack_halffloat(p, v, 1); +} + +static int lp_float(char *p, PyObject *v, const formatdef *f) { double x = PyFloat_AsDouble(v); @@ -1182,6 +1255,7 @@ {'Q', 8, 0, lu_ulonglong, lp_ulonglong}, {'?', 1, 0, bu_bool, bp_bool}, /* Std rep not endian dep, but potentially different from native rep -- reuse bx_bool funcs. */ + {'e', 2, 0, lu_halffloat, lp_halffloat}, {'f', 4, 0, lu_float, lp_float}, {'d', 8, 0, lu_double, lp_double}, {0} @@ -2239,7 +2313,7 @@ x: pad byte (no data); c:char; b:signed byte; B:unsigned byte;\n\ ?: _Bool (requires C99; if not available, char is used instead)\n\ h:short; H:unsigned short; i:int; I:unsigned int;\n\ - l:long; L:unsigned long; f:float; d:double.\n\ + l:long; L:unsigned long; f:float; d:double; e:half-float.\n\ Special cases (preceding decimal count indicates length):\n\ s:string (array of char); p: pascal string (with count byte).\n\ Special cases (only available in native format):\n\ diff --git a/Objects/floatobject.c b/Objects/floatobject.c --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -1975,8 +1975,120 @@ /*---------------------------------------------------------------------------- - * _PyFloat_{Pack,Unpack}{4,8}. See floatobject.h. + * _PyFloat_{Pack,Unpack}{2,4,8}. See floatobject.h. + * To match the NPY_HALF_ROUND_TIES_TO_EVEN behavior in: + * https://github.com/numpy/numpy/blob/master/numpy/core/src/npymath/halffloat.c + * We use: + * bits = (unsigned short)f; Note the truncation + * if ((f - bits > 0.5) || (f - bits == 0.5 && bits % 2)) { + * bits++; + * } */ + +int +_PyFloat_Pack2(double x, unsigned char *p, int le) +{ + unsigned char sign; + int e; + double f; + unsigned short bits; + int incr = 1; + + if (x == 0.0) { + sign = (copysign(1.0, x) == -1.0); + e = 0; + bits = 0; + } + else if (Py_IS_INFINITY(x)) { + sign = (x < 0.0); + e = 0x1f; + bits = 0; + } + else if (Py_IS_NAN(x)) { + /* There are 2046 distinct half-precision NaNs (1022 signaling and + 1024 quiet), but there are only two quiet NaNs that don't arise by + quieting a signaling NaN; we get those by setting the topmost bit + of the fraction field and clearing all other fraction bits. We + choose the one with the appropriate sign. */ + sign = (copysign(1.0, x) == -1.0); + e = 0x1f; + bits = 512; + } + else { + sign = (x < 0.0); + if (sign) { + x = -x; + } + + f = frexp(x, &e); + if (f < 0.5 || f >= 1.0) { + PyErr_SetString(PyExc_SystemError, + "frexp() result out of range"); + return -1; + } + + /* Normalize f to be in the range [1.0, 2.0) */ + f *= 2.0; + e--; + + if (e >= 16) { + goto Overflow; + } + else if (e < -25) { + /* |x| < 2**-25. Underflow to zero. */ + f = 0.0; + e = 0; + } + else if (e < -14) { + /* |x| < 2**-14. Gradual underflow */ + f = ldexp(f, 14 + e); + e = 0; + } + else /* if (!(e == 0 && f == 0.0)) */ { + e += 15; + f -= 1.0; /* Get rid of leading 1 */ + } + + f *= 1024.0; /* 2**10 */ + /* Round to even */ + bits = (unsigned short)f; /* Note the truncation */ + assert(bits < 1024); + assert(e < 31); + if ((f - bits > 0.5) || ((f - bits == 0.5) && (bits % 2 == 1))) { + ++bits; + if (bits == 1024) { + /* The carry propagated out of a string of 10 1 bits. */ + bits = 0; + ++e; + if (e == 31) + goto Overflow; + } + } + } + + bits |= (e << 10) | (sign << 15); + + /* Write out result. */ + if (le) { + p += 1; + incr = -1; + } + + /* First byte */ + *p = (unsigned char)((bits >> 8) & 0xFF); + p += incr; + + /* Second byte */ + *p = (unsigned char)(bits & 0xFF); + + return 0; + + Overflow: + PyErr_SetString(PyExc_OverflowError, + "float too large to pack with e format"); + return -1; +} + int _PyFloat_Pack4(double x, unsigned char *p, int le) { @@ -2212,6 +2324,76 @@ } double +_PyFloat_Unpack2(const unsigned char *p, int le) +{ + unsigned char sign; + int e; + unsigned int f; + double x; + int incr = 1; + + if (le) { + p += 1; + incr = -1; + } + + /* First byte */ + sign = (*p >> 7) & 1; + e = (*p & 0x7C) >> 2; + f = (*p & 0x03) << 8; + p += incr; + + /* Second byte */ + f |= *p; + + if (e == 0x1f) { +#ifdef PY_NO_SHORT_FLOAT_REPR + if (f == 0) { + /* Infinity */ + return sign ? -Py_HUGE_VAL : Py_HUGE_VAL; + } + else { + /* NaN */ +#ifdef Py_NAN + return sign ? -Py_NAN : Py_NAN; +#else + PyErr_SetString( + PyExc_ValueError, + "can't unpack IEEE 754 NaN " + "on platform that does not support NaNs"); + return -1; +#endif /* #ifdef Py_NAN */ + } +#else + if (f == 0) { + /* Infinity */ + return _Py_dg_infinity(sign); + } + else { + /* NaN */ + return _Py_dg_stdnan(sign); + } +#endif /* #ifdef PY_NO_SHORT_FLOAT_REPR */ + } + + x = (double)f / 1024.0; + + if (e == 0) { + e = -14; + } + else { + x += 1.0; + e -= 15; + } + x = ldexp(x, e); + + if (sign) + x = -x; + + return x; +} + +double _PyFloat_Unpack4(const unsigned char *p, int le) { if (float_format == unknown_format) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 12:33:45 2016 From: python-checkins at python.org (eric.smith) Date: Sat, 03 Sep 2016 16:33:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27921=3A_Remove_back?= =?utf-8?q?slash_from_another_f-string=2E_I=27ll_revert_this_change?= Message-ID: <20160903163345.9646.70324.40D72471@psf.io> https://hg.python.org/cpython/rev/fd4e4fa75260 changeset: 103023:fd4e4fa75260 user: Eric V. Smith date: Sat Sep 03 12:33:38 2016 -0400 summary: Issue 27921: Remove backslash from another f-string. I'll revert this change before beta 2. I also need to look in to why test_tools/test_unparse fails with the files that are now being skipped. files: Lib/test/test_faulthandler.py | 4 ++-- Lib/test/test_tools/test_unparse.py | 10 ++++++++++ Lib/traceback.py | 4 ++-- 3 files changed, 14 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py --- a/Lib/test/test_faulthandler.py +++ b/Lib/test/test_faulthandler.py @@ -735,11 +735,11 @@ ('EXCEPTION_INT_DIVIDE_BY_ZERO', 'int divide by zero'), ('EXCEPTION_STACK_OVERFLOW', 'stack overflow'), ): - self.check_windows_exception(f""" + self.check_windows_exception(""" import faulthandler faulthandler.enable() faulthandler._raise_exception(faulthandler._{exc}) - """, + """.format(exc=exc), 3, name) diff --git a/Lib/test/test_tools/test_unparse.py b/Lib/test/test_tools/test_unparse.py --- a/Lib/test/test_tools/test_unparse.py +++ b/Lib/test/test_tools/test_unparse.py @@ -284,6 +284,16 @@ for filename in names: if test.support.verbose: print('Testing %s' % filename) + + # it's very much a hack that I'm skipping these files, but + # I can't figure out why they fail. I'll fix it when I + # address issue #27948. + if (filename.endswith('/test_fstring.py') or + filename.endswith('/test_traceback.py')): + if test.support.verbose: + print(f'Skipping {filename}: see issue 27921') + continue + source = read_pyfile(filename) self.check_roundtrip(source) diff --git a/Lib/traceback.py b/Lib/traceback.py --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -402,7 +402,7 @@ count += 1 else: if count > 3: - result.append(f' [Previous line repeated {count-3} more times]''\n') + result.append(f' [Previous line repeated {count-3} more times]'+'\n') last_file = frame.filename last_line = frame.lineno last_name = frame.name @@ -419,7 +419,7 @@ row.append(' {name} = {value}\n'.format(name=name, value=value)) result.append(''.join(row)) if count > 3: - result.append(f' [Previous line repeated {count-3} more times]''\n') + result.append(f' [Previous line repeated {count-3} more times]'+'\n') return result -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 12:45:12 2016 From: python-checkins at python.org (mark.dickinson) Date: Sat, 03 Sep 2016 16:45:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3OTM0?= =?utf-8?q?=3A_Use_float=2E=5F=5Frepr=5F=5F_instead_of_plain_repr_when_JSO?= =?utf-8?q?N-encoding_an?= Message-ID: <20160903164511.47439.56818.F58E206D@psf.io> https://hg.python.org/cpython/rev/86d66a627b77 changeset: 103024:86d66a627b77 branch: 2.7 parent: 102986:e065aec0e6fa user: Mark Dickinson date: Sat Sep 03 17:45:00 2016 +0100 summary: Issue #27934: Use float.__repr__ instead of plain repr when JSON-encoding an instance of a float subclass. Thanks Eddie James. files: Lib/json/encoder.py | 2 +- Lib/json/tests/test_float.py | 11 +++++++++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/_json.c | 4 ++-- 5 files changed, 18 insertions(+), 3 deletions(-) diff --git a/Lib/json/encoder.py b/Lib/json/encoder.py --- a/Lib/json/encoder.py +++ b/Lib/json/encoder.py @@ -28,7 +28,7 @@ #ESCAPE_DCT.setdefault(chr(i), '\\u%04x' % (i,)) INFINITY = float('inf') -FLOAT_REPR = repr +FLOAT_REPR = float.__repr__ def encode_basestring(s): """Return a JSON representation of a Python string diff --git a/Lib/json/tests/test_float.py b/Lib/json/tests/test_float.py --- a/Lib/json/tests/test_float.py +++ b/Lib/json/tests/test_float.py @@ -32,6 +32,17 @@ self.assertNotEqual(res[0], res[0]) self.assertRaises(ValueError, self.dumps, [val], allow_nan=False) + def test_float_subclasses_use_float_repr(self): + # Issue 27934. + class PeculiarFloat(float): + def __repr__(self): + return "I'm not valid JSON" + def __str__(self): + return "Neither am I" + + val = PeculiarFloat(3.2) + self.assertEqual(self.loads(self.dumps(val)), val) + class TestPyFloat(TestFloat, PyTest): pass class TestCFloat(TestFloat, CTest): pass diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -636,6 +636,7 @@ David Jacobs Kevin Jacobs Kjetil Jacobsen +Eddie James Bertrand Janin Geert Jansen Jack Jansen diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -36,6 +36,9 @@ Library ------- +- Issue #27934: Use ``float.__repr__`` instead of plain ``repr`` when JSON- + encoding an instance of a float subclass. Thanks Eddie James. + - Issue #27861: Fixed a crash in sqlite3.Connection.cursor() when a factory creates not a cursor. Patch by Xiang Zhang. diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -1960,8 +1960,8 @@ return PyString_FromString("NaN"); } } - /* Use a better float format here? */ - return PyObject_Repr(obj); + /* Make sure to use the base float class repr method */ + return PyFloat_Type.tp_repr(obj); } static PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 3 14:30:33 2016 From: python-checkins at python.org (mark.dickinson) Date: Sat, 03 Sep 2016 18:30:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326040=3A_Improve_?= =?utf-8?q?test=5Fmath_and_test=5Fcmath_coverage_and_rigour=2E_Thanks_Jeff?= Message-ID: <20160903183033.114977.1885.3CFCFFA7@psf.io> https://hg.python.org/cpython/rev/e3dbe8b7279a changeset: 103025:e3dbe8b7279a parent: 103023:fd4e4fa75260 user: Mark Dickinson date: Sat Sep 03 19:30:22 2016 +0100 summary: Issue #26040: Improve test_math and test_cmath coverage and rigour. Thanks Jeff Allen. files: Lib/test/cmath_testcases.txt | 141 ++++++++++++ Lib/test/test_math.py | 261 +++++++++++++++------- Misc/NEWS | 3 + 3 files changed, 320 insertions(+), 85 deletions(-) diff --git a/Lib/test/cmath_testcases.txt b/Lib/test/cmath_testcases.txt --- a/Lib/test/cmath_testcases.txt +++ b/Lib/test/cmath_testcases.txt @@ -53,6 +53,12 @@ -- MPFR homepage at http://www.mpfr.org for more information about the -- MPFR project. +-- A minority of the test cases were generated with the help of +-- mpmath 0.19 at 100 bit accuracy (http://mpmath.org) to improve +-- coverage of real functions with real-valued arguments. These are +-- used in test.test_math.MathTests.test_testfile, as well as in +-- test_cmath. + -------------------------- -- acos: Inverse cosine -- @@ -848,6 +854,18 @@ atan0303 atan -1e-165 1.0 -> -0.78539816339744828 190.30984376228875 atan0304 atan -9.9998886718268301e-321 -1.0 -> -0.78539816339744828 -368.76019403576692 +-- Additional real values (mpmath) +atan0400 atan 1.7976931348623157e+308 0.0 -> 1.5707963267948966192 0.0 +atan0401 atan -1.7976931348623157e+308 0.0 -> -1.5707963267948966192 0.0 +atan0402 atan 1e-17 0.0 -> 1.0000000000000000715e-17 0.0 +atan0403 atan -1e-17 0.0 -> -1.0000000000000000715e-17 0.0 +atan0404 atan 0.0001 0.0 -> 0.000099999999666666673459 0.0 +atan0405 atan -0.0001 0.0 -> -0.000099999999666666673459 0.0 +atan0406 atan 0.999999999999999 0.0 -> 0.78539816339744781002 0.0 +atan0407 atan 1.000000000000001 0.0 -> 0.78539816339744886473 0.0 +atan0408 atan 14.101419947171719 0.0 -> 1.4999999999999999969 0.0 +atan0409 atan 1255.7655915007897 0.0 -> 1.5700000000000000622 0.0 + -- special values atan1000 atan -0.0 0.0 -> -0.0 0.0 atan1001 atan nan 0.0 -> nan 0.0 @@ -1514,6 +1532,11 @@ sqrt0140 sqrt 1.6999999999999999e+308 -1.6999999999999999e+308 -> 1.4325088230154573e+154 -5.9336458271212207e+153 sqrt0141 sqrt -1.797e+308 -9.9999999999999999e+306 -> 3.7284476432057307e+152 -1.3410406899802901e+154 +-- Additional real values (mpmath) +sqrt0150 sqrt 1.7976931348623157e+308 0.0 -> 1.3407807929942596355e+154 0.0 +sqrt0151 sqrt 2.2250738585072014e-308 0.0 -> 1.4916681462400413487e-154 0.0 +sqrt0152 sqrt 5e-324 0.0 -> 2.2227587494850774834e-162 0.0 + -- special values sqrt1000 sqrt 0.0 0.0 -> 0.0 0.0 sqrt1001 sqrt -0.0 0.0 -> 0.0 0.0 @@ -1616,6 +1639,20 @@ exp0053 exp 710.0 1.6 -> -6.5231579995501372e+306 inf overflow exp0054 exp 710.0 2.8 -> -inf 7.4836177417448528e+307 overflow +-- Additional real values (mpmath) +exp0070 exp 1e-08 0.0 -> 1.00000001000000005 0.0 +exp0071 exp 0.0003 0.0 -> 1.0003000450045003375 0.0 +exp0072 exp 0.2 0.0 -> 1.2214027581601698475 0.0 +exp0073 exp 1.0 0.0 -> 2.7182818284590452354 0.0 +exp0074 exp -1e-08 0.0 -> 0.99999999000000005 0.0 +exp0075 exp -0.0003 0.0 -> 0.99970004499550033751 0.0 +exp0076 exp -1.0 0.0 -> 0.3678794411714423216 0.0 +exp0077 exp 2.220446049250313e-16 0.0 -> 1.000000000000000222 0.0 +exp0078 exp -1.1102230246251565e-16 0.0 -> 0.99999999999999988898 0.0 +exp0079 exp 2.302585092994046 0.0 -> 10.000000000000002171 0.0 +exp0080 exp -2.302585092994046 0.0 -> 0.099999999999999978292 0.0 +exp0081 exp 709.7827 0.0 -> 1.7976699566638014654e+308 0.0 + -- special values exp1000 exp 0.0 0.0 -> 1.0 0.0 exp1001 exp -0.0 0.0 -> 1.0 0.0 @@ -1708,6 +1745,23 @@ cosh0030 cosh 710.5 2.3519999999999999 -> -1.2967465239355998e+308 1.3076707908857333e+308 cosh0031 cosh -710.5 0.69999999999999996 -> 1.4085466381392499e+308 -1.1864024666450239e+308 +-- Additional real values (mpmath) +cosh0050 cosh 1e-150 0.0 -> 1.0 0.0 +cosh0051 cosh 1e-18 0.0 -> 1.0 0.0 +cosh0052 cosh 1e-09 0.0 -> 1.0000000000000000005 0.0 +cosh0053 cosh 0.0003 0.0 -> 1.0000000450000003375 0.0 +cosh0054 cosh 0.2 0.0 -> 1.0200667556190758485 0.0 +cosh0055 cosh 1.0 0.0 -> 1.5430806348152437785 0.0 +cosh0056 cosh -1e-18 0.0 -> 1.0 -0.0 +cosh0057 cosh -0.0003 0.0 -> 1.0000000450000003375 -0.0 +cosh0058 cosh -1.0 0.0 -> 1.5430806348152437785 -0.0 +cosh0059 cosh 1.3169578969248168 0.0 -> 2.0000000000000001504 0.0 +cosh0060 cosh -1.3169578969248168 0.0 -> 2.0000000000000001504 -0.0 +cosh0061 cosh 17.328679513998633 0.0 -> 16777216.000000021938 0.0 +cosh0062 cosh 18.714973875118524 0.0 -> 67108864.000000043662 0.0 +cosh0063 cosh 709.7827 0.0 -> 8.9883497833190073272e+307 0.0 +cosh0064 cosh -709.7827 0.0 -> 8.9883497833190073272e+307 -0.0 + -- special values cosh1000 cosh 0.0 0.0 -> 1.0 0.0 cosh1001 cosh 0.0 inf -> nan 0.0 invalid ignore-imag-sign @@ -1800,6 +1854,24 @@ sinh0030 sinh 710.5 -2.3999999999999999 -> -1.3579970564885919e+308 -1.24394470907798e+308 sinh0031 sinh -710.5 0.80000000000000004 -> -1.2830671601735164e+308 1.3210954193997678e+308 +-- Additional real values (mpmath) +sinh0050 sinh 1e-100 0.0 -> 1.00000000000000002e-100 0.0 +sinh0051 sinh 5e-17 0.0 -> 4.9999999999999998955e-17 0.0 +sinh0052 sinh 1e-16 0.0 -> 9.999999999999999791e-17 0.0 +sinh0053 sinh 3.7e-08 0.0 -> 3.7000000000000008885e-8 0.0 +sinh0054 sinh 0.001 0.0 -> 0.0010000001666666750208 0.0 +sinh0055 sinh 0.2 0.0 -> 0.20133600254109399895 0.0 +sinh0056 sinh 1.0 0.0 -> 1.1752011936438014569 0.0 +sinh0057 sinh -3.7e-08 0.0 -> -3.7000000000000008885e-8 0.0 +sinh0058 sinh -0.001 0.0 -> -0.0010000001666666750208 0.0 +sinh0059 sinh -1.0 0.0 -> -1.1752011936438014569 0.0 +sinh0060 sinh 1.4436354751788103 0.0 -> 1.9999999999999999078 0.0 +sinh0061 sinh -1.4436354751788103 0.0 -> -1.9999999999999999078 0.0 +sinh0062 sinh 17.328679513998633 0.0 -> 16777215.999999992136 0.0 +sinh0063 sinh 18.714973875118524 0.0 -> 67108864.000000036211 0.0 +sinh0064 sinh 709.7827 0.0 -> 8.9883497833190073272e+307 0.0 +sinh0065 sinh -709.7827 0.0 -> -8.9883497833190073272e+307 0.0 + -- special values sinh1000 sinh 0.0 0.0 -> 0.0 0.0 sinh1001 sinh 0.0 inf -> 0.0 nan invalid ignore-real-sign @@ -1897,6 +1969,24 @@ tanh0032 tanh 1000 -2.3199999999999998 -> 1.0 0.0 tanh0033 tanh -1.0000000000000001e+300 -9.6699999999999999 -> -1.0 -0.0 +-- Additional real values (mpmath) +tanh0050 tanh 1e-100 0.0 -> 1.00000000000000002e-100 0.0 +tanh0051 tanh 5e-17 0.0 -> 4.9999999999999998955e-17 0.0 +tanh0052 tanh 1e-16 0.0 -> 9.999999999999999791e-17 0.0 +tanh0053 tanh 3.7e-08 0.0 -> 3.6999999999999983559e-8 0.0 +tanh0054 tanh 0.001 0.0 -> 0.00099999966666680002076 0.0 +tanh0055 tanh 0.2 0.0 -> 0.19737532022490401141 0.0 +tanh0056 tanh 1.0 0.0 -> 0.76159415595576488812 0.0 +tanh0057 tanh -3.7e-08 0.0 -> -3.6999999999999983559e-8 0.0 +tanh0058 tanh -0.001 0.0 -> -0.00099999966666680002076 0.0 +tanh0059 tanh -1.0 0.0 -> -0.76159415595576488812 0.0 +tanh0060 tanh 0.5493061443340549 0.0 -> 0.50000000000000003402 0.0 +tanh0061 tanh -0.5493061443340549 0.0 -> -0.50000000000000003402 0.0 +tanh0062 tanh 17.328679513998633 0.0 -> 0.99999999999999822364 0.0 +tanh0063 tanh 18.714973875118524 0.0 -> 0.99999999999999988898 0.0 +tanh0064 tanh 711 0.0 -> 1.0 0.0 +tanh0065 tanh 1.797e+308 0.0 -> 1.0 0.0 + --special values tanh1000 tanh 0.0 0.0 -> 0.0 0.0 tanh1001 tanh 0.0 inf -> nan nan invalid @@ -1985,6 +2075,22 @@ cos0022 cos 7.9914515433858515 0.71659966615501436 -> -0.17375439906936566 -0.77217043527294582 cos0023 cos 0.45124351152540226 1.6992693993812158 -> 2.543477948972237 -1.1528193694875477 +-- Additional real values (mpmath) +cos0050 cos 1e-150 0.0 -> 1.0 -0.0 +cos0051 cos 1e-18 0.0 -> 1.0 -0.0 +cos0052 cos 1e-09 0.0 -> 0.9999999999999999995 -0.0 +cos0053 cos 0.0003 0.0 -> 0.9999999550000003375 -0.0 +cos0054 cos 0.2 0.0 -> 0.98006657784124162892 -0.0 +cos0055 cos 1.0 0.0 -> 0.5403023058681397174 -0.0 +cos0056 cos -1e-18 0.0 -> 1.0 0.0 +cos0057 cos -0.0003 0.0 -> 0.9999999550000003375 0.0 +cos0058 cos -1.0 0.0 -> 0.5403023058681397174 0.0 +cos0059 cos 1.0471975511965976 0.0 -> 0.50000000000000009945 -0.0 +cos0060 cos 2.5707963267948966 0.0 -> -0.84147098480789647357 -0.0 +cos0061 cos -2.5707963267948966 0.0 -> -0.84147098480789647357 0.0 +cos0062 cos 7.225663103256523 0.0 -> 0.58778525229247407559 -0.0 +cos0063 cos -8.79645943005142 0.0 -> -0.80901699437494722255 0.0 + -- special values cos1000 cos -0.0 0.0 -> 1.0 0.0 cos1001 cos -inf 0.0 -> nan 0.0 invalid ignore-imag-sign @@ -2073,6 +2179,22 @@ sin0022 sin 1.1518087354403725 4.8597235966150558 -> 58.919141989603041 26.237003403758852 sin0023 sin 0.00087773078406649192 34.792379211312095 -> 565548145569.38245 644329685822700.62 +-- Additional real values (mpmath) +sin0050 sin 1e-100 0.0 -> 1.00000000000000002e-100 0.0 +sin0051 sin 3.7e-08 0.0 -> 3.6999999999999992001e-8 0.0 +sin0052 sin 0.001 0.0 -> 0.00099999983333334168748 0.0 +sin0053 sin 0.2 0.0 -> 0.19866933079506122634 0.0 +sin0054 sin 1.0 0.0 -> 0.84147098480789650665 0.0 +sin0055 sin -3.7e-08 0.0 -> -3.6999999999999992001e-8 0.0 +sin0056 sin -0.001 0.0 -> -0.00099999983333334168748 0.0 +sin0057 sin -1.0 0.0 -> -0.84147098480789650665 0.0 +sin0058 sin 0.5235987755982989 0.0 -> 0.50000000000000004642 0.0 +sin0059 sin -0.5235987755982989 0.0 -> -0.50000000000000004642 0.0 +sin0060 sin 2.6179938779914944 0.0 -> 0.49999999999999996018 -0.0 +sin0061 sin -2.6179938779914944 0.0 -> -0.49999999999999996018 -0.0 +sin0062 sin 7.225663103256523 0.0 -> 0.80901699437494673648 0.0 +sin0063 sin -8.79645943005142 0.0 -> -0.58778525229247340658 -0.0 + -- special values sin1000 sin -0.0 0.0 -> -0.0 0.0 sin1001 sin -inf 0.0 -> nan 0.0 invalid ignore-imag-sign @@ -2161,6 +2283,25 @@ tan0022 tan 1.1615313900880577 1.7956298728647107 -> 0.041793186826390362 1.0375339546034792 tan0023 tan 0.067014779477908945 5.8517361577457097 -> 2.2088639754800034e-06 0.9999836182420061 +-- Additional real values (mpmath) +tan0050 tan 1e-100 0.0 -> 1.00000000000000002e-100 0.0 +tan0051 tan 3.7e-08 0.0 -> 3.7000000000000017328e-8 0.0 +tan0052 tan 0.001 0.0 -> 0.0010000003333334666875 0.0 +tan0053 tan 0.2 0.0 -> 0.20271003550867249488 0.0 +tan0054 tan 1.0 0.0 -> 1.5574077246549022305 0.0 +tan0055 tan -3.7e-08 0.0 -> -3.7000000000000017328e-8 0.0 +tan0056 tan -0.001 0.0 -> -0.0010000003333334666875 0.0 +tan0057 tan -1.0 0.0 -> -1.5574077246549022305 0.0 +tan0058 tan 0.4636476090008061 0.0 -> 0.49999999999999997163 0.0 +tan0059 tan -0.4636476090008061 0.0 -> -0.49999999999999997163 0.0 +tan0060 tan 1.1071487177940904 0.0 -> 1.9999999999999995298 0.0 +tan0061 tan -1.1071487177940904 0.0 -> -1.9999999999999995298 0.0 +tan0062 tan 1.5 0.0 -> 14.101419947171719388 0.0 +tan0063 tan 1.57 0.0 -> 1255.7655915007896475 0.0 +tan0064 tan 1.5707963267948961 0.0 -> 1978937966095219.0538 0.0 +tan0065 tan 7.225663103256523 0.0 -> 1.3763819204711701522 0.0 +tan0066 tan -8.79645943005142 0.0 -> 0.7265425280053614098 0.0 + -- special values tan1000 tan -0.0 0.0 -> -0.0 0.0 tan1001 tan -inf 0.0 -> nan nan invalid diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -29,6 +29,7 @@ math_testcases = os.path.join(test_dir, 'math_testcases.txt') test_file = os.path.join(test_dir, 'cmath_testcases.txt') + def to_ulps(x): """Convert a non-NaN float x to an integer, in such a way that adjacent floats are converted to adjacent integers. Then @@ -36,25 +37,39 @@ floats. The results from this function will only make sense on platforms - where C doubles are represented in IEEE 754 binary64 format. + where native doubles are represented in IEEE 754 binary64 format. + Note: 0.0 and -0.0 are converted to 0 and -1, respectively. """ n = struct.unpack(' eps: - # Use %r instead of %f so the error message - # displays full precision. Otherwise discrepancies - # in the last few bits will lead to very confusing - # error messages - self.fail('%s returned %r, expected %r' % - (name, value, expected)) + def ftest(self, name, got, expected, ulp_tol=5, abs_tol=0.0): + """Compare arguments expected and got, as floats, if either + is a float, using a tolerance expressed in multiples of + ulp(expected) or absolutely, whichever is greater. + + As a convenience, when neither argument is a float, and for + non-finite floats, exact equality is demanded. Also, nan==nan + in this function. + """ + failure = result_check(expected, got, ulp_tol, abs_tol) + if failure is not None: + self.fail("{}: {}".format(name, failure)) def testConstants(self): - self.ftest('pi', math.pi, 3.1415926) - self.ftest('e', math.e, 2.7182818) + # Ref: Abramowitz & Stegun (Dover, 1965) + self.ftest('pi', math.pi, 3.141592653589793238462643) + self.ftest('e', math.e, 2.718281828459045235360287) self.assertEqual(math.tau, 2*math.pi) def testAcos(self): @@ -378,9 +443,9 @@ def testCos(self): self.assertRaises(TypeError, math.cos) - self.ftest('cos(-pi/2)', math.cos(-math.pi/2), 0) + self.ftest('cos(-pi/2)', math.cos(-math.pi/2), 0, abs_tol=ulp(1)) self.ftest('cos(0)', math.cos(0), 1) - self.ftest('cos(pi/2)', math.cos(math.pi/2), 0) + self.ftest('cos(pi/2)', math.cos(math.pi/2), 0, abs_tol=ulp(1)) self.ftest('cos(pi)', math.cos(math.pi), -1) try: self.assertTrue(math.isnan(math.cos(INF))) @@ -970,7 +1035,8 @@ def testTanh(self): self.assertRaises(TypeError, math.tanh) self.ftest('tanh(0)', math.tanh(0), 0) - self.ftest('tanh(1)+tanh(-1)', math.tanh(1)+math.tanh(-1), 0) + self.ftest('tanh(1)+tanh(-1)', math.tanh(1)+math.tanh(-1), 0, + abs_tol=ulp(1)) self.ftest('tanh(inf)', math.tanh(INF), 1) self.ftest('tanh(-inf)', math.tanh(NINF), -1) self.assertTrue(math.isnan(math.tanh(NAN))) @@ -1084,30 +1150,48 @@ @requires_IEEE_754 def test_testfile(self): + fail_fmt = "{}: {}({!r}): {}" + + failures = [] for id, fn, ar, ai, er, ei, flags in parse_testfile(test_file): - # Skip if either the input or result is complex, or if - # flags is nonempty - if ai != 0. or ei != 0. or flags: + # Skip if either the input or result is complex + if ai != 0.0 or ei != 0.0: continue if fn in ['rect', 'polar']: # no real versions of rect, polar continue + func = getattr(math, fn) + + if 'invalid' in flags or 'divide-by-zero' in flags: + er = 'ValueError' + elif 'overflow' in flags: + er = 'OverflowError' + try: result = func(ar) - except ValueError as exc: - message = (("Unexpected ValueError: %s\n " + - "in test %s:%s(%r)\n") % (exc.args[0], id, fn, ar)) - self.fail(message) + except ValueError: + result = 'ValueError' except OverflowError: - message = ("Unexpected OverflowError in " + - "test %s:%s(%r)\n" % (id, fn, ar)) - self.fail(message) - self.ftest("%s:%s(%r)" % (id, fn, ar), result, er) + result = 'OverflowError' + + # Default tolerances + ulp_tol, abs_tol = 5, 0.0 + + failure = result_check(er, result, ulp_tol, abs_tol) + if failure is None: + continue + + msg = fail_fmt.format(id, fn, ar, failure) + failures.append(msg) + + if failures: + self.fail('Failures in test_testfile:\n ' + + '\n '.join(failures)) @requires_IEEE_754 def test_mtestfile(self): - fail_fmt = "{}:{}({!r}): expected {!r}, got {!r}" + fail_fmt = "{}: {}({!r}): {}" failures = [] for id, fn, arg, expected, flags in parse_mtestfile(math_testcases): @@ -1125,41 +1209,48 @@ except OverflowError: got = 'OverflowError' - accuracy_failure = None - if isinstance(got, float) and isinstance(expected, float): - if math.isnan(expected) and math.isnan(got): - continue - if not math.isnan(expected) and not math.isnan(got): - if fn == 'lgamma': - # we use a weaker accuracy test for lgamma; - # lgamma only achieves an absolute error of - # a few multiples of the machine accuracy, in - # general. - accuracy_failure = acc_check(expected, got, - rel_err = 5e-15, - abs_err = 5e-15) - elif fn == 'erfc': - # erfc has less-than-ideal accuracy for large - # arguments (x ~ 25 or so), mainly due to the - # error involved in computing exp(-x*x). - # - # XXX Would be better to weaken this test only - # for large x, instead of for all x. - accuracy_failure = ulps_check(expected, got, 2000) + # Default tolerances + ulp_tol, abs_tol = 5, 0.0 - else: - accuracy_failure = ulps_check(expected, got, 20) - if accuracy_failure is None: - continue + # Exceptions to the defaults + if fn == 'gamma': + # Experimental results on one platform gave + # an accuracy of <= 10 ulps across the entire float + # domain. We weaken that to require 20 ulp accuracy. + ulp_tol = 20 - if isinstance(got, str) and isinstance(expected, str): - if got == expected: - continue + elif fn == 'lgamma': + # we use a weaker accuracy test for lgamma; + # lgamma only achieves an absolute error of + # a few multiples of the machine accuracy, in + # general. + abs_tol = 1e-15 - fail_msg = fail_fmt.format(id, fn, arg, expected, got) - if accuracy_failure is not None: - fail_msg += ' ({})'.format(accuracy_failure) - failures.append(fail_msg) + elif fn == 'erfc' and arg >= 0.0: + # erfc has less-than-ideal accuracy for large + # arguments (x ~ 25 or so), mainly due to the + # error involved in computing exp(-x*x). + # + # Observed between CPython and mpmath at 25 dp: + # x < 0 : err <= 2 ulp + # 0 <= x < 1 : err <= 10 ulp + # 1 <= x < 10 : err <= 100 ulp + # 10 <= x < 20 : err <= 300 ulp + # 20 <= x : < 600 ulp + # + if arg < 1.0: + ulp_tol = 10 + elif arg < 10.0: + ulp_tol = 100 + else: + ulp_tol = 1000 + + failure = result_check(expected, got, ulp_tol, abs_tol) + if failure is None: + continue + + msg = fail_fmt.format(id, fn, arg, failure) + failures.append(msg) if failures: self.fail('Failures in test_mtestfile:\n ' + diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -135,6 +135,9 @@ Tests ----- +- Issue #26040: Improve test_math and test_cmath coverage and rigour. Patch by + Jeff Allen. + - Issue #27787: Call gc.collect() before checking each test for "dangling threads", since the dangling threads are weak references. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 4 04:58:59 2016 From: python-checkins at python.org (mark.dickinson) Date: Sun, 04 Sep 2016 08:58:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327953=3A_skip_fai?= =?utf-8?q?ling_math_and_cmath_tests_for_tan_on_OS_X_10=2E4=2E?= Message-ID: <20160904085859.9838.96566.70DD67DE@psf.io> https://hg.python.org/cpython/rev/b4d52df5595e changeset: 103026:b4d52df5595e user: Mark Dickinson date: Sun Sep 04 09:58:51 2016 +0100 summary: Issue #27953: skip failing math and cmath tests for tan on OS X 10.4. files: Lib/test/test_cmath.py | 20 ++++++++++++++++++++ Lib/test/test_math.py | 19 ++++++++++++++++++- Misc/NEWS | 3 +++ 3 files changed, 41 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_cmath.py b/Lib/test/test_cmath.py --- a/Lib/test/test_cmath.py +++ b/Lib/test/test_cmath.py @@ -4,6 +4,8 @@ import unittest import cmath, math from cmath import phase, polar, rect, pi +import platform +import sys import sysconfig INF = float('inf') @@ -332,6 +334,18 @@ @requires_IEEE_754 def test_specific_values(self): + # Some tests need to be skipped on ancient OS X versions. + # See issue #27953. + SKIP_ON_TIGER = {'tan0064'} + + osx_version = None + if sys.platform == 'darwin': + version_txt = platform.mac_ver()[0] + try: + osx_version = tuple(map(int, version_txt.split('.'))) + except ValueError: + pass + def rect_complex(z): """Wrapped version of rect that accepts a complex number instead of two float arguments.""" @@ -345,6 +359,12 @@ for id, fn, ar, ai, er, ei, flags in parse_testfile(test_file): arg = complex(ar, ai) expected = complex(er, ei) + + # Skip certain tests on OS X 10.4. + if osx_version is not None and osx_version < (10, 5): + if id in SKIP_ON_TIGER: + continue + if fn == 'rect': function = rect_complex elif fn == 'polar': diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -6,8 +6,9 @@ import unittest import math import os +import platform +import struct import sys -import struct import sysconfig eps = 1E-05 @@ -1150,6 +1151,18 @@ @requires_IEEE_754 def test_testfile(self): + # Some tests need to be skipped on ancient OS X versions. + # See issue #27953. + SKIP_ON_TIGER = {'tan0064'} + + osx_version = None + if sys.platform == 'darwin': + version_txt = platform.mac_ver()[0] + try: + osx_version = tuple(map(int, version_txt.split('.'))) + except ValueError: + pass + fail_fmt = "{}: {}({!r}): {}" failures = [] @@ -1160,6 +1173,10 @@ if fn in ['rect', 'polar']: # no real versions of rect, polar continue + # Skip certain tests on OS X 10.4. + if osx_version is not None and osx_version < (10, 5): + if id in SKIP_ON_TIGER: + continue func = getattr(math, fn) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -135,6 +135,9 @@ Tests ----- +- Issue #27953: Skip math and cmath tests that fail on OS X 10.4 due to a + poor libm implementation of tan. + - Issue #26040: Improve test_math and test_cmath coverage and rigour. Patch by Jeff Allen. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 4 07:29:27 2016 From: python-checkins at python.org (mark.dickinson) Date: Sun, 04 Sep 2016 11:29:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327427=3A_Addition?= =?utf-8?q?al_tests_for_the_math_module=2E_Thanks_Francisco_Couzo=2E?= Message-ID: <20160904112927.68817.82449.2C7EB312@psf.io> https://hg.python.org/cpython/rev/eb98f6044638 changeset: 103027:eb98f6044638 user: Mark Dickinson date: Sun Sep 04 12:29:14 2016 +0100 summary: Issue #27427: Additional tests for the math module. Thanks Francisco Couzo. files: Lib/test/test_math.py | 28 ++++++++++++++++++++++------ Misc/NEWS | 2 ++ 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -15,6 +15,7 @@ NAN = float('nan') INF = float('inf') NINF = float('-inf') +FLOAT_MAX = sys.float_info.max # detect evidence of double-rounding: fsum is not always correctly # rounded on machines that suffer from double rounding. @@ -271,6 +272,8 @@ self.ftest('acos(1)', math.acos(1), 0) self.assertRaises(ValueError, math.acos, INF) self.assertRaises(ValueError, math.acos, NINF) + self.assertRaises(ValueError, math.acos, 1 + eps) + self.assertRaises(ValueError, math.acos, -1 - eps) self.assertTrue(math.isnan(math.acos(NAN))) def testAcosh(self): @@ -290,6 +293,8 @@ self.ftest('asin(1)', math.asin(1), math.pi/2) self.assertRaises(ValueError, math.asin, INF) self.assertRaises(ValueError, math.asin, NINF) + self.assertRaises(ValueError, math.asin, 1 + eps) + self.assertRaises(ValueError, math.asin, -1 - eps) self.assertTrue(math.isnan(math.asin(NAN))) def testAsinh(self): @@ -469,6 +474,7 @@ self.ftest('degrees(pi)', math.degrees(math.pi), 180.0) self.ftest('degrees(pi/2)', math.degrees(math.pi/2), 90.0) self.ftest('degrees(-pi/4)', math.degrees(-math.pi/4), -45.0) + self.ftest('degrees(0)', math.degrees(0), 0) def testExp(self): self.assertRaises(TypeError, math.exp) @@ -478,6 +484,7 @@ self.assertEqual(math.exp(INF), INF) self.assertEqual(math.exp(NINF), 0.) self.assertTrue(math.isnan(math.exp(NAN))) + self.assertRaises(OverflowError, math.exp, 1000000) def testFabs(self): self.assertRaises(TypeError, math.fabs) @@ -720,6 +727,7 @@ self.assertEqual(math.hypot(INF, NAN), INF) self.assertEqual(math.hypot(NAN, NINF), INF) self.assertEqual(math.hypot(NINF, NAN), INF) + self.assertRaises(OverflowError, math.hypot, FLOAT_MAX, FLOAT_MAX) self.assertTrue(math.isnan(math.hypot(1.0, NAN))) self.assertTrue(math.isnan(math.hypot(NAN, -2.0))) @@ -773,8 +781,10 @@ def testLog1p(self): self.assertRaises(TypeError, math.log1p) - n= 2**90 - self.assertAlmostEqual(math.log1p(n), math.log1p(float(n))) + for n in [2, 2**90, 2**300]: + self.assertAlmostEqual(math.log1p(n), math.log1p(float(n))) + self.assertRaises(ValueError, math.log1p, -1) + self.assertEqual(math.log1p(INF), INF) @requires_IEEE_754 def testLog2(self): @@ -988,6 +998,7 @@ self.ftest('radians(180)', math.radians(180), math.pi) self.ftest('radians(90)', math.radians(90), math.pi/2) self.ftest('radians(-45)', math.radians(-45), -math.pi/4) + self.ftest('radians(0)', math.radians(0), 0) def testSin(self): self.assertRaises(TypeError, math.sin) @@ -1017,6 +1028,7 @@ self.ftest('sqrt(1)', math.sqrt(1), 1) self.ftest('sqrt(4)', math.sqrt(4), 2) self.assertEqual(math.sqrt(INF), INF) + self.assertRaises(ValueError, math.sqrt, -1) self.assertRaises(ValueError, math.sqrt, NINF) self.assertTrue(math.isnan(math.sqrt(NAN))) @@ -1087,7 +1099,8 @@ def testIsnan(self): self.assertTrue(math.isnan(float("nan"))) - self.assertTrue(math.isnan(float("inf")* 0.)) + self.assertTrue(math.isnan(float("-nan"))) + self.assertTrue(math.isnan(float("inf") * 0.)) self.assertFalse(math.isnan(float("inf"))) self.assertFalse(math.isnan(0.)) self.assertFalse(math.isnan(1.)) @@ -1380,7 +1393,8 @@ decimal_examples = [(Decimal('1.00000001'), Decimal('1.0')), (Decimal('1.00000001e-20'), Decimal('1.0e-20')), - (Decimal('1.00000001e-100'), Decimal('1.0e-100'))] + (Decimal('1.00000001e-100'), Decimal('1.0e-100')), + (Decimal('1.00000001e20'), Decimal('1.0e20'))] self.assertAllClose(decimal_examples, rel_tol=1e-8) self.assertAllNotClose(decimal_examples, rel_tol=1e-9) @@ -1388,8 +1402,10 @@ # test with Fraction values from fractions import Fraction - # could use some more examples here! - fraction_examples = [(Fraction(1, 100000000) + 1, Fraction(1))] + fraction_examples = [ + (Fraction(1, 100000000) + 1, Fraction(1)), + (Fraction(100000001), Fraction(100000000)), + (Fraction(10**8 + 1, 10**28), Fraction(1, 10**20))] self.assertAllClose(fraction_examples, rel_tol=1e-8) self.assertAllNotClose(fraction_examples, rel_tol=1e-9) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -135,6 +135,8 @@ Tests ----- +- Issue #27427: Additional tests for the math module. Patch by Francisco Couzo. + - Issue #27953: Skip math and cmath tests that fail on OS X 10.4 due to a poor libm implementation of tan. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 4 07:31:57 2016 From: python-checkins at python.org (mark.dickinson) Date: Sun, 04 Sep 2016 11:31:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_Francisco_Couzo_to_Mis?= =?utf-8?q?c/ACKS_=28for_issue_=2327427_patch=29=2E?= Message-ID: <20160904113157.104176.97692.FAD0D217@psf.io> https://hg.python.org/cpython/rev/d7c31c73b1bd changeset: 103028:d7c31c73b1bd user: Mark Dickinson date: Sun Sep 04 12:31:47 2016 +0100 summary: Add Francisco Couzo to Misc/ACKS (for issue #27427 patch). files: Misc/ACKS | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -306,6 +306,7 @@ David Cournapeau Julien Courteau Steve Cousins +Francisco Couzo Alex Coventry Matthew Dixon Cowles Ryan Coyner -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 4 14:17:55 2016 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 04 Sep 2016 18:17:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20160904181755.22162.57227.B615F69C@psf.io> https://hg.python.org/cpython/rev/0398f84b9c53 changeset: 103030:0398f84b9c53 parent: 103028:d7c31c73b1bd parent: 103029:72651343be73 user: Raymond Hettinger date: Sun Sep 04 11:17:48 2016 -0700 summary: Merge files: Doc/library/random.rst | 7 +++++-- Lib/random.py | 8 +++++--- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/Doc/library/random.rst b/Doc/library/random.rst --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -64,8 +64,11 @@ If *a* is an int, it is used directly. With version 2 (the default), a :class:`str`, :class:`bytes`, or :class:`bytearray` - object gets converted to an :class:`int` and all of its bits are used. With version 1, - the :func:`hash` of *a* is used instead. + object gets converted to an :class:`int` and all of its bits are used. + + With version 1 (provided for reproducing random sequences from older versions + of Python), the algorithm for :class:`str` and :class:`bytes` generates a + narrower range of seeds. .. versionchanged:: 3.2 Moved to the version 2 scheme which uses all of the bits in a string seed. diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -96,10 +96,12 @@ None or no argument seeds from current time or from an operating system specific randomness source if available. + If *a* is an int, all bits are used. + For version 2 (the default), all of the bits are used if *a* is a str, - bytes, or bytearray. For version 1, the hash() of *a* is used instead. - - If *a* is an int, all bits are used. + bytes, or bytearray. For version 1 (provided for reproducing random + sequences from older versions of Python), the algorithm for str and + bytes generates a narrower range of seeds. """ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 4 14:17:55 2016 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 04 Sep 2016 18:17:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Improve_docs_f?= =?utf-8?q?or_random=2Eseed=28=29?= Message-ID: <20160904181755.47340.64904.89F866EF@psf.io> https://hg.python.org/cpython/rev/72651343be73 changeset: 103029:72651343be73 branch: 3.5 parent: 103018:e4b6faf22e8d user: Raymond Hettinger date: Sun Sep 04 11:17:28 2016 -0700 summary: Improve docs for random.seed() files: Doc/library/random.rst | 7 +++++-- Lib/random.py | 8 +++++--- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/Doc/library/random.rst b/Doc/library/random.rst --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -63,8 +63,11 @@ If *a* is an int, it is used directly. With version 2 (the default), a :class:`str`, :class:`bytes`, or :class:`bytearray` - object gets converted to an :class:`int` and all of its bits are used. With version 1, - the :func:`hash` of *a* is used instead. + object gets converted to an :class:`int` and all of its bits are used. + + With version 1 (provided for reproducing random sequences from older versions + of Python), the algorithm for :class:`str` and :class:`bytes` generates a + narrower range of seeds. .. versionchanged:: 3.2 Moved to the version 2 scheme which uses all of the bits in a string seed. diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -96,10 +96,12 @@ None or no argument seeds from current time or from an operating system specific randomness source if available. + If *a* is an int, all bits are used. + For version 2 (the default), all of the bits are used if *a* is a str, - bytes, or bytearray. For version 1, the hash() of *a* is used instead. - - If *a* is an int, all bits are used. + bytes, or bytearray. For version 1 (provided for reproducing random + sequences from older versions of Python), the algorithm for str and + bytes generates a narrower range of seeds. """ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 4 14:29:21 2016 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 04 Sep 2016 18:29:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20160904182920.10049.1518.44A1AA98@psf.io> https://hg.python.org/cpython/rev/1bd1e31a3298 changeset: 103032:1bd1e31a3298 parent: 103030:0398f84b9c53 parent: 103031:7108f2a708c9 user: Raymond Hettinger date: Sun Sep 04 11:29:13 2016 -0700 summary: Merge files: Doc/library/functions.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1237,8 +1237,8 @@ .. function:: round(number[, ndigits]) Return the floating point value *number* rounded to *ndigits* digits after - the decimal point. If *ndigits* is omitted, it returns the nearest integer - to its input. Delegates to ``number.__round__(ndigits)``. + the decimal point. If *ndigits* is omitted or is ``None``, it returns the + nearest integer to its input. Delegates to ``number.__round__(ndigits)``. For the built-in types supporting :func:`round`, values are rounded to the closest multiple of 10 to the power minus *ndigits*; if two multiples are -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 4 14:29:21 2016 From: python-checkins at python.org (raymond.hettinger) Date: Sun, 04 Sep 2016 18:29:21 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgMjc5MzY6?= =?utf-8?q?_Update_doc_for_round=28=29_to_indicate_that_None_is_an_allowab?= =?utf-8?q?le?= Message-ID: <20160904182920.47323.9738.A83A41F1@psf.io> https://hg.python.org/cpython/rev/7108f2a708c9 changeset: 103031:7108f2a708c9 branch: 3.5 parent: 103029:72651343be73 user: Raymond Hettinger date: Sun Sep 04 11:28:56 2016 -0700 summary: Issue 27936: Update doc for round() to indicate that None is an allowable argument. files: Doc/library/functions.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1234,8 +1234,8 @@ .. function:: round(number[, ndigits]) Return the floating point value *number* rounded to *ndigits* digits after - the decimal point. If *ndigits* is omitted, it returns the nearest integer - to its input. Delegates to ``number.__round__(ndigits)``. + the decimal point. If *ndigits* is omitted or is ``None``, it returns the + nearest integer to its input. Delegates to ``number.__round__(ndigits)``. For the built-in types supporting :func:`round`, values are rounded to the closest multiple of 10 to the power minus *ndigits*; if two multiples are -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 4 14:39:19 2016 From: python-checkins at python.org (ethan.furman) Date: Sun, 04 Sep 2016 18:39:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue23591=3A_more_docs=3B?= =?utf-8?q?_slight_change_to_repr?= Message-ID: <20160904183918.38184.97320.788C05A2@psf.io> https://hg.python.org/cpython/rev/8e9d3a5d47d5 changeset: 103033:8e9d3a5d47d5 user: Ethan Furman date: Sun Sep 04 11:39:01 2016 -0700 summary: issue23591: more docs; slight change to repr files: Doc/library/enum.rst | 13 ++++++++++++- Lib/enum.py | 13 +++++-------- Lib/test/test_enum.py | 12 ++++++------ 3 files changed, 23 insertions(+), 15 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -574,7 +574,7 @@ >>> Perm.RWX >>> ~Perm.RWX - + Another important difference between :class:`IntFlag` and :class:`Enum` is that if no flags are set (the value is 0), its boolean evaluation is :data:`False`:: @@ -615,6 +615,17 @@ >>> bool(Color.red & Color.green) False +Individual flags should have values that are powers of two (1, 2, 4, 8, ...), +while combinations of flags won't:: + + >>> class Color(Flag): + ... red = 1 + ... blue = 2 + ... green = 4 + ... white = 7 + ... # or + ... # white = red | blue | green + Giving a name to the "no flags set" condition does not change its boolean value:: diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -692,14 +692,11 @@ if self._name_ is not None: return '<%s.%s: %r>' % (cls.__name__, self._name_, self._value_) members = self._decompose_() - if len(members) == 1 and members[0]._name_ is None: - return '<%s: %r>' % (cls.__name__, members[0]._value_) - else: - return '<%s.%s: %r>' % ( - cls.__name__, - '|'.join([str(m._name_ or m._value_) for m in members]), - self._value_, - ) + return '<%s.%s: %r>' % ( + cls.__name__, + '|'.join([str(m._name_ or m._value_) for m in members]), + self._value_, + ) def __str__(self): cls = self.__class__ diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1688,12 +1688,12 @@ self.assertEqual(repr(Perm.X), '') self.assertEqual(repr(Perm.R | Perm.W), '') self.assertEqual(repr(Perm.R | Perm.W | Perm.X), '') - self.assertEqual(repr(Perm(0)), '') + self.assertEqual(repr(Perm(0)), '') self.assertEqual(repr(~Perm.R), '') self.assertEqual(repr(~Perm.W), '') self.assertEqual(repr(~Perm.X), '') self.assertEqual(repr(~(Perm.R | Perm.W)), '') - self.assertEqual(repr(~(Perm.R | Perm.W | Perm.X)), '') + self.assertEqual(repr(~(Perm.R | Perm.W | Perm.X)), '') self.assertEqual(repr(Perm(~0)), '') Open = self.Open @@ -1933,13 +1933,13 @@ self.assertEqual(repr(Perm.R | Perm.W), '') self.assertEqual(repr(Perm.R | Perm.W | Perm.X), '') self.assertEqual(repr(Perm.R | 8), '') - self.assertEqual(repr(Perm(0)), '') - self.assertEqual(repr(Perm(8)), '') + self.assertEqual(repr(Perm(0)), '') + self.assertEqual(repr(Perm(8)), '') self.assertEqual(repr(~Perm.R), '') self.assertEqual(repr(~Perm.W), '') self.assertEqual(repr(~Perm.X), '') self.assertEqual(repr(~(Perm.R | Perm.W)), '') - self.assertEqual(repr(~(Perm.R | Perm.W | Perm.X)), '') + self.assertEqual(repr(~(Perm.R | Perm.W | Perm.X)), '') self.assertEqual(repr(~(Perm.R | 8)), '') self.assertEqual(repr(Perm(~0)), '') self.assertEqual(repr(Perm(~8)), '') @@ -1950,7 +1950,7 @@ self.assertEqual(repr(Open.AC), '') self.assertEqual(repr(Open.RO | Open.CE), '') self.assertEqual(repr(Open.WO | Open.CE), '') - self.assertEqual(repr(Open(4)), '') + self.assertEqual(repr(Open(4)), '') self.assertEqual(repr(~Open.RO), '') self.assertEqual(repr(~Open.WO), '') self.assertEqual(repr(~Open.AC), '') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 4 15:02:17 2016 From: python-checkins at python.org (terry.reedy) Date: Sun, 04 Sep 2016 19:02:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3OTIy?= =?utf-8?q?=3A_IDLE_test=5Fidlehistory_no_longer_flash_tk_widgets=2E?= Message-ID: <20160904190216.68789.51763.DB80EC3C@psf.io> https://hg.python.org/cpython/rev/6e4475894a79 changeset: 103034:6e4475894a79 branch: 2.7 parent: 103024:86d66a627b77 user: Terry Jan Reedy date: Sun Sep 04 15:02:02 2016 -0400 summary: Issue #27922: IDLE test_idlehistory no longer flash tk widgets. (Omitted for 2.7 from previous patch.) files: Lib/idlelib/idle_test/test_idlehistory.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/idle_test/test_idlehistory.py b/Lib/idlelib/idle_test/test_idlehistory.py --- a/Lib/idlelib/idle_test/test_idlehistory.py +++ b/Lib/idlelib/idle_test/test_idlehistory.py @@ -68,6 +68,7 @@ def setUpClass(cls): requires('gui') cls.root = tk.Tk() + cls.root.withdraw() def setUp(self): self.text = text = TextWrapper(self.root) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 00:02:05 2016 From: python-checkins at python.org (terry.reedy) Date: Mon, 05 Sep 2016 04:02:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3OTE4?= =?utf-8?q?=23_test=2Eresource=2Eis=5Fgui=5Favailable_no_longer_flashes_tk?= =?utf-8?q?_window=2E?= Message-ID: <20160905040205.47303.51782.7F933A9B@psf.io> https://hg.python.org/cpython/rev/de9e410e78d8 changeset: 103035:de9e410e78d8 branch: 2.7 user: Terry Jan Reedy date: Mon Sep 05 00:01:28 2016 -0400 summary: Issue #27918# test.resource.is_gui_available no longer flashes tk window. Also, don't run it if 'gui' is not requested. Patch by Xiang Zhang. files: Lib/test/test_support.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py --- a/Lib/test/test_support.py +++ b/Lib/test/test_support.py @@ -331,6 +331,7 @@ try: from Tkinter import Tk root = Tk() + root.withdraw() root.update() root.destroy() except Exception as e: @@ -355,12 +356,12 @@ def requires(resource, msg=None): """Raise ResourceDenied if the specified resource is not available.""" - if resource == 'gui' and not _is_gui_available(): - raise ResourceDenied(_is_gui_available.reason) if not is_resource_enabled(resource): if msg is None: msg = "Use of the `%s' resource not enabled" % resource raise ResourceDenied(msg) + if resource == 'gui' and not _is_gui_available(): + raise ResourceDenied(_is_gui_available.reason) def requires_mac_ver(*min_version): """Decorator raising SkipTest if the OS is Mac OS X and the OS X -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 00:02:05 2016 From: python-checkins at python.org (terry.reedy) Date: Mon, 05 Sep 2016 04:02:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTE4?= =?utf-8?q?=23_test=2Eresource=2Eis=5Fgui=5Favailable_no_longer_flashes_tk?= =?utf-8?q?_window=2E?= Message-ID: <20160905040205.8382.32000.4C16D417@psf.io> https://hg.python.org/cpython/rev/756c27efe193 changeset: 103036:756c27efe193 branch: 3.5 parent: 103031:7108f2a708c9 user: Terry Jan Reedy date: Mon Sep 05 00:01:34 2016 -0400 summary: Issue #27918# test.resource.is_gui_available no longer flashes tk window. Also, don't run it if 'gui' is not requested. Patch by Xiang Zhang. files: Lib/test/support/__init__.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -464,6 +464,7 @@ try: from tkinter import Tk root = Tk() + root.withdraw() root.update() root.destroy() except Exception as e: @@ -488,12 +489,12 @@ def requires(resource, msg=None): """Raise ResourceDenied if the specified resource is not available.""" - if resource == 'gui' and not _is_gui_available(): - raise ResourceDenied(_is_gui_available.reason) if not is_resource_enabled(resource): if msg is None: msg = "Use of the %r resource not enabled" % resource raise ResourceDenied(msg) + if resource == 'gui' and not _is_gui_available(): + raise ResourceDenied(_is_gui_available.reason) def _requires_unix_version(sysname, min_version): """Decorator raising SkipTest if the OS is `sysname` and the version is less -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 00:02:05 2016 From: python-checkins at python.org (terry.reedy) Date: Mon, 05 Sep 2016 04:02:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160905040205.22318.48285.401D240E@psf.io> https://hg.python.org/cpython/rev/ffc915a55a72 changeset: 103037:ffc915a55a72 parent: 103033:8e9d3a5d47d5 parent: 103036:756c27efe193 user: Terry Jan Reedy date: Mon Sep 05 00:01:47 2016 -0400 summary: Merge with 3.5 files: Lib/test/support/__init__.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -466,6 +466,7 @@ try: from tkinter import Tk root = Tk() + root.withdraw() root.update() root.destroy() except Exception as e: @@ -490,12 +491,12 @@ def requires(resource, msg=None): """Raise ResourceDenied if the specified resource is not available.""" - if resource == 'gui' and not _is_gui_available(): - raise ResourceDenied(_is_gui_available.reason) if not is_resource_enabled(resource): if msg is None: msg = "Use of the %r resource not enabled" % resource raise ResourceDenied(msg) + if resource == 'gui' and not _is_gui_available(): + raise ResourceDenied(_is_gui_available.reason) def _requires_unix_version(sysname, min_version): """Decorator raising SkipTest if the OS is `sysname` and the version is less -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Mon Sep 5 08:12:26 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Mon, 5 Sep 2016 13:12:26 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python Default 2016-09-05 Message-ID: Results for project Python default, build date 2016-09-05 07:22:20 +0000 commit: ffc915a55a72 previous commit: 79422fab2ef4 revision date: 2016-09-05 04:01:47 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.22% 1.28% 10.92% 17.51% :-) pybench 0.12% 0.70% 7.13% 6.07% :-( regex_v8 2.81% 0.43% -2.33% 4.21% :-) nbody 0.15% 2.07% 0.85% 9.18% :-( json_dump_v2 0.30% -0.49% -2.73% 14.26% :-| normal_startup 0.75% -0.38% 1.93% 6.32% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-default-2016-09-05/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Mon Sep 5 13:13:11 2016 From: python-checkins at python.org (steve.dower) Date: Mon, 05 Sep 2016 17:13:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Minor_improvem?= =?utf-8?q?ents_to_the_vcs_ignore_files?= Message-ID: <20160905171309.6601.5225.93976812@psf.io> https://hg.python.org/cpython/rev/5f4d5d64007d changeset: 103038:5f4d5d64007d branch: 3.5 parent: 103036:756c27efe193 user: Steve Dower date: Mon Sep 05 10:12:03 2016 -0700 summary: Minor improvements to the vcs ignore files files: .gitignore | 8 +++++--- .hgignore | 3 +++ 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,7 @@ # Two-trick pony for OSX and other case insensitive file systems: # Ignore ./python binary on Unix but still look into ./Python/ directory. /python -!/Python/** +!/Python/ *.cover *.o *.orig @@ -69,8 +69,8 @@ config.status.lineno core db_home -config.log -config.status +.hg/ +ipch/ libpython*.a libpython*.so* platform @@ -93,3 +93,5 @@ Tools/msi/obj Tools/ssl/amd64 Tools/ssl/win32 +.vs/ +.vscode/ diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -94,7 +94,10 @@ *.gcda *.gcno *.gcov +ipch/ coverage.info Tools/msi/obj Tools/ssl/amd64 Tools/ssl/win32 +.vs/ +.vscode/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 13:13:59 2016 From: python-checkins at python.org (steve.dower) Date: Mon, 05 Sep 2016 17:13:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Minor_improvements_to_the_vcs_ignore_files?= Message-ID: <20160905171310.104224.98015.02BF1F64@psf.io> https://hg.python.org/cpython/rev/9cf1cd91c56a changeset: 103039:9cf1cd91c56a parent: 103037:ffc915a55a72 parent: 103038:5f4d5d64007d user: Steve Dower date: Mon Sep 05 10:12:29 2016 -0700 summary: Minor improvements to the vcs ignore files files: .gitignore | 7 ++++--- .hgignore | 3 +++ 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,7 @@ # Two-trick pony for OSX and other case insensitive file systems: # Ignore ./python binary on Unix but still look into ./Python/ directory. /python -!/Python/** +!/Python/ *.cover *.o *.orig @@ -69,8 +69,8 @@ config.status.lineno core db_home -config.log -config.status +.hg/ +ipch/ libpython*.a libpython*.so* platform @@ -93,4 +93,5 @@ Tools/msi/obj Tools/ssl/amd64 Tools/ssl/win32 +.vs/ .vscode/ diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -95,7 +95,10 @@ *.gcda *.gcno *.gcov +ipch/ coverage.info Tools/msi/obj Tools/ssl/amd64 Tools/ssl/win32 +.vs/ +.vscode/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 13:17:53 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 17:17:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160905171750.68689.22417.97B34BE5@psf.io> https://hg.python.org/cpython/rev/125cf99907c7 changeset: 103044:125cf99907c7 parent: 103043:fbfb58f73dcf parent: 103042:4e0f56794a73 user: Benjamin Peterson date: Mon Sep 05 10:17:37 2016 -0700 summary: merge 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 13:17:53 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 17:17:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <20160905171750.68767.36122.4310231B@psf.io> https://hg.python.org/cpython/rev/fbfb58f73dcf changeset: 103043:fbfb58f73dcf parent: 103041:1b14dbe9e98b parent: 103039:9cf1cd91c56a user: Benjamin Peterson date: Mon Sep 05 10:17:30 2016 -0700 summary: merge heads files: .gitignore | 7 ++++--- .hgignore | 3 +++ 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,7 @@ # Two-trick pony for OSX and other case insensitive file systems: # Ignore ./python binary on Unix but still look into ./Python/ directory. /python -!/Python/** +!/Python/ *.cover *.o *.orig @@ -69,8 +69,8 @@ config.status.lineno core db_home -config.log -config.status +.hg/ +ipch/ libpython*.a libpython*.so* platform @@ -93,4 +93,5 @@ Tools/msi/obj Tools/ssl/amd64 Tools/ssl/win32 +.vs/ .vscode/ diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -95,7 +95,10 @@ *.gcda *.gcno *.gcov +ipch/ coverage.info Tools/msi/obj Tools/ssl/amd64 Tools/ssl/win32 +.vs/ +.vscode/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 13:18:09 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 17:18:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_do_not_allow_?= =?utf-8?q?=5FPyGen=5FFinalize_to_fail_=28closes_=2327811=29?= Message-ID: <20160905171748.8656.90419.8B5BB593@psf.io> https://hg.python.org/cpython/rev/4d531711cbc7 changeset: 103040:4d531711cbc7 branch: 3.5 parent: 103036:756c27efe193 user: Benjamin Peterson date: Mon Sep 05 10:14:54 2016 -0700 summary: do not allow _PyGen_Finalize to fail (closes #27811) Patch from Armin Rigo. files: Lib/test/test_coroutines.py | 8 +++++++ Misc/NEWS | 3 ++ Objects/genobject.c | 27 ++++++++++++------------ 3 files changed, 25 insertions(+), 13 deletions(-) diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py --- a/Lib/test/test_coroutines.py +++ b/Lib/test/test_coroutines.py @@ -1565,6 +1565,14 @@ finally: aw.close() + def test_fatal_coro_warning(self): + # Issue 27811 + async def func(): pass + with warnings.catch_warnings(): + warnings.filterwarnings("error") + func() + support.gc_collect() + class CoroAsyncIOCompatTest(unittest.TestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27811: Fix a crash when a coroutine that has not been awaited is + finalized with warnings-as-errors enabled. + - Issue #27587: Fix another issue found by PVS-Studio: Null pointer check after use of 'def' in _PyState_AddModule(). Initial patch by Christian Heimes. diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -24,18 +24,6 @@ PyObject *res; PyObject *error_type, *error_value, *error_traceback; - /* If `gen` is a coroutine, and if it was never awaited on, - issue a RuntimeWarning. */ - if (gen->gi_code != NULL - && ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE - && gen->gi_frame != NULL - && gen->gi_frame->f_lasti == -1 - && !PyErr_Occurred() - && PyErr_WarnFormat(PyExc_RuntimeWarning, 1, - "coroutine '%.50S' was never awaited", - gen->gi_qualname)) - return; - if (gen->gi_frame == NULL || gen->gi_frame->f_stacktop == NULL) /* Generator isn't paused, so no need to close */ return; @@ -43,7 +31,20 @@ /* Save the current exception, if any. */ PyErr_Fetch(&error_type, &error_value, &error_traceback); - res = gen_close(gen, NULL); + /* If `gen` is a coroutine, and if it was never awaited on, + issue a RuntimeWarning. */ + if (gen->gi_code != NULL + && ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE + && gen->gi_frame->f_lasti == -1 + && !PyErr_Occurred() + && PyErr_WarnFormat(PyExc_RuntimeWarning, 1, + "coroutine '%.50S' was never awaited", + gen->gi_qualname)) { + res = NULL; /* oops, exception */ + } + else { + res = gen_close(gen, NULL); + } if (res == NULL) PyErr_WriteUnraisable(self); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 13:18:18 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 17:18:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_3=2E5_=28closes_=2327811=29?= Message-ID: <20160905171749.8382.25815.3B3D902B@psf.io> https://hg.python.org/cpython/rev/1b14dbe9e98b changeset: 103041:1b14dbe9e98b parent: 103037:ffc915a55a72 parent: 103040:4d531711cbc7 user: Benjamin Peterson date: Mon Sep 05 10:16:31 2016 -0700 summary: merge 3.5 (closes #27811) files: Lib/test/test_coroutines.py | 8 +++++++ Misc/NEWS | 3 ++ Objects/genobject.c | 27 ++++++++++++------------ 3 files changed, 25 insertions(+), 13 deletions(-) diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py --- a/Lib/test/test_coroutines.py +++ b/Lib/test/test_coroutines.py @@ -1565,6 +1565,14 @@ finally: aw.close() + def test_fatal_coro_warning(self): + # Issue 27811 + async def func(): pass + with warnings.catch_warnings(): + warnings.filterwarnings("error") + func() + support.gc_collect() + class CoroAsyncIOCompatTest(unittest.TestCase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ - Issue #27506: Support passing the bytes/bytearray.translate() "delete" argument by keyword. +- Issue #27811: Fix a crash when a coroutine that has not been awaited is + finalized with warnings-as-errors enabled. + - Issue #27587: Fix another issue found by PVS-Studio: Null pointer check after use of 'def' in _PyState_AddModule(). Initial patch by Christian Heimes. diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -24,18 +24,6 @@ PyObject *res; PyObject *error_type, *error_value, *error_traceback; - /* If `gen` is a coroutine, and if it was never awaited on, - issue a RuntimeWarning. */ - if (gen->gi_code != NULL - && ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE - && gen->gi_frame != NULL - && gen->gi_frame->f_lasti == -1 - && !PyErr_Occurred() - && PyErr_WarnFormat(PyExc_RuntimeWarning, 1, - "coroutine '%.50S' was never awaited", - gen->gi_qualname)) - return; - if (gen->gi_frame == NULL || gen->gi_frame->f_stacktop == NULL) /* Generator isn't paused, so no need to close */ return; @@ -43,7 +31,20 @@ /* Save the current exception, if any. */ PyErr_Fetch(&error_type, &error_value, &error_traceback); - res = gen_close(gen, NULL); + /* If `gen` is a coroutine, and if it was never awaited on, + issue a RuntimeWarning. */ + if (gen->gi_code != NULL + && ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE + && gen->gi_frame->f_lasti == -1 + && !PyErr_Occurred() + && PyErr_WarnFormat(PyExc_RuntimeWarning, 1, + "coroutine '%.50S' was never awaited", + gen->gi_qualname)) { + res = NULL; /* oops, exception */ + } + else { + res = gen_close(gen, NULL); + } if (res == NULL) PyErr_WriteUnraisable(self); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 13:18:18 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 17:18:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy41IC0+IDMuNSk6?= =?utf-8?q?_merge_heads?= Message-ID: <20160905171750.30464.36176.A6BC42DF@psf.io> https://hg.python.org/cpython/rev/4e0f56794a73 changeset: 103042:4e0f56794a73 branch: 3.5 parent: 103040:4d531711cbc7 parent: 103038:5f4d5d64007d user: Benjamin Peterson date: Mon Sep 05 10:17:22 2016 -0700 summary: merge heads files: .gitignore | 8 +++++--- .hgignore | 3 +++ 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,7 @@ # Two-trick pony for OSX and other case insensitive file systems: # Ignore ./python binary on Unix but still look into ./Python/ directory. /python -!/Python/** +!/Python/ *.cover *.o *.orig @@ -69,8 +69,8 @@ config.status.lineno core db_home -config.log -config.status +.hg/ +ipch/ libpython*.a libpython*.so* platform @@ -93,3 +93,5 @@ Tools/msi/obj Tools/ssl/amd64 Tools/ssl/win32 +.vs/ +.vscode/ diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -94,7 +94,10 @@ *.gcda *.gcno *.gcov +ipch/ coverage.info Tools/msi/obj Tools/ssl/amd64 Tools/ssl/win32 +.vs/ +.vscode/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 13:41:13 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 17:41:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjc4MTIp?= Message-ID: <20160905174112.6601.28511.EC2CAE94@psf.io> https://hg.python.org/cpython/rev/1d7a938b1e47 changeset: 103046:1d7a938b1e47 parent: 103044:125cf99907c7 parent: 103045:3e4452424f9b user: Benjamin Peterson date: Mon Sep 05 10:40:34 2016 -0700 summary: merge 3.5 (#27812) files: Misc/NEWS | 3 +++ Objects/genobject.c | 5 ++++- 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,9 @@ - Issue #27506: Support passing the bytes/bytearray.translate() "delete" argument by keyword. +- Issue #27812: Properly clear out a generator's frame's backreference to the + generator to prevent crashes in frame.clear(). + - Issue #27811: Fix a crash when a coroutine that has not been awaited is finalized with warnings-as-errors enabled. diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -71,7 +71,10 @@ return; /* resurrected. :( */ _PyObject_GC_UNTRACK(self); - Py_CLEAR(gen->gi_frame); + if (gen->gi_frame != NULL) { + gen->gi_frame->f_gen = NULL; + Py_CLEAR(gen->gi_frame); + } Py_CLEAR(gen->gi_code); Py_CLEAR(gen->gi_name); Py_CLEAR(gen->gi_qualname); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 13:41:13 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 17:41:13 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogY2xlYXIgb3V0IGZf?= =?utf-8?q?gen_during_generator_finalization_=28closes_=2327812=29?= Message-ID: <20160905174112.8560.66764.D870A1F2@psf.io> https://hg.python.org/cpython/rev/3e4452424f9b changeset: 103045:3e4452424f9b branch: 3.5 parent: 103042:4e0f56794a73 user: Benjamin Peterson date: Mon Sep 05 10:39:57 2016 -0700 summary: clear out f_gen during generator finalization (closes #27812) Patch from Armin Rigo. files: Misc/NEWS | 3 +++ Objects/genobject.c | 5 ++++- 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27812: Properly clear out a generator's frame's backreference to the + generator to prevent crashes in frame.clear(). + - Issue #27811: Fix a crash when a coroutine that has not been awaited is finalized with warnings-as-errors enabled. diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -71,7 +71,10 @@ return; /* resurrected. :( */ _PyObject_GC_UNTRACK(self); - Py_CLEAR(gen->gi_frame); + if (gen->gi_frame != NULL) { + gen->gi_frame->f_gen = NULL; + Py_CLEAR(gen->gi_frame); + } Py_CLEAR(gen->gi_code); Py_CLEAR(gen->gi_name); Py_CLEAR(gen->gi_qualname); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 14:03:56 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 18:03:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NDA3?= =?utf-8?q?=3A_Make_PCbuild/prepare=5Fssl=2Epy_Python_2_compatible?= Message-ID: <20160905180356.9612.42985.16235385@psf.io> https://hg.python.org/cpython/rev/e0873191ad7d changeset: 103049:e0873191ad7d branch: 3.5 parent: 103045:3e4452424f9b user: Zachary Ware date: Mon Sep 05 12:54:08 2016 -0500 summary: Issue #27407: Make PCbuild/prepare_ssl.py Python 2 compatible files: PCbuild/prepare_ssl.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/PCbuild/prepare_ssl.py b/PCbuild/prepare_ssl.py --- a/PCbuild/prepare_ssl.py +++ b/PCbuild/prepare_ssl.py @@ -18,6 +18,8 @@ # it should configure OpenSSL such that it is ready to be built by # ssl.vcxproj on 32 or 64 bit platforms. +from __future__ import print_function + import os import re import sys @@ -89,7 +91,10 @@ def copy_includes(makefile, suffix): dir = 'include'+suffix+'\\openssl' - os.makedirs(dir, exist_ok=True) + try: + os.makedirs(dir) + except OSError: + pass copy_if_different = r'$(PERL) $(SRC_D)\util\copy-if-different.pl' with open(makefile) as fin: for line in fin: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 14:03:56 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 18:03:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NDA3?= =?utf-8?q?=3A_Make_PCbuild/prepare=5Fssl=2Epy_Python_2_compatible?= Message-ID: <20160905180356.23462.13780.998CECF5@psf.io> https://hg.python.org/cpython/rev/32552131d8cb changeset: 103048:32552131d8cb branch: 2.7 user: Zachary Ware date: Mon Sep 05 12:54:08 2016 -0500 summary: Issue #27407: Make PCbuild/prepare_ssl.py Python 2 compatible files: PCbuild/prepare_ssl.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/PCbuild/prepare_ssl.py b/PCbuild/prepare_ssl.py --- a/PCbuild/prepare_ssl.py +++ b/PCbuild/prepare_ssl.py @@ -18,6 +18,8 @@ # it should configure OpenSSL such that it is ready to be built by # ssl.vcxproj on 32 or 64 bit platforms. +from __future__ import print_function + import os import re import sys @@ -89,7 +91,10 @@ def copy_includes(makefile, suffix): dir = 'include'+suffix+'\\openssl' - os.makedirs(dir, exist_ok=True) + try: + os.makedirs(dir) + except OSError: + pass copy_if_different = r'$(PERL) $(SRC_D)\util\copy-if-different.pl' with open(makefile) as fin: for line in fin: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 14:03:56 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 18:03:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327407=3A_Merge_with_3=2E5?= Message-ID: <20160905180356.6446.58111.D1183239@psf.io> https://hg.python.org/cpython/rev/87f186cc6b80 changeset: 103050:87f186cc6b80 parent: 103046:1d7a938b1e47 parent: 103049:e0873191ad7d user: Zachary Ware date: Mon Sep 05 13:02:20 2016 -0500 summary: Closes #27407: Merge with 3.5 files: PCbuild/prepare_ssl.py | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/PCbuild/prepare_ssl.py b/PCbuild/prepare_ssl.py --- a/PCbuild/prepare_ssl.py +++ b/PCbuild/prepare_ssl.py @@ -18,6 +18,8 @@ # it should configure OpenSSL such that it is ready to be built by # ssl.vcxproj on 32 or 64 bit platforms. +from __future__ import print_function + import os import re import sys @@ -89,7 +91,10 @@ def copy_includes(makefile, suffix): dir = 'include'+suffix+'\\openssl' - os.makedirs(dir, exist_ok=True) + try: + os.makedirs(dir) + except OSError: + pass copy_if_different = r'$(PERL) $(SRC_D)\util\copy-if-different.pl' with open(makefile) as fin: for line in fin: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 14:03:57 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 18:03:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NDA3?= =?utf-8?q?=3A_Add_prepare=5Fssl=2Epy_to_2=2E7?= Message-ID: <20160905180356.22100.84680.8AB56566@psf.io> https://hg.python.org/cpython/rev/8955049a79aa changeset: 103047:8955049a79aa branch: 2.7 parent: 103035:de9e410e78d8 user: Zachary Ware date: Mon Sep 05 11:55:42 2016 -0500 summary: Issue #27407: Add prepare_ssl.py to 2.7 files: PCbuild/prepare_ssl.py | 196 +++++++++++++++++++++++++++++ 1 files changed, 196 insertions(+), 0 deletions(-) diff --git a/PCbuild/prepare_ssl.py b/PCbuild/prepare_ssl.py new file mode 100644 --- /dev/null +++ b/PCbuild/prepare_ssl.py @@ -0,0 +1,196 @@ +#! /usr/bin/env python3 +# Script for preparing OpenSSL for building on Windows. +# Uses Perl to create nmake makefiles and otherwise prepare the way +# for building on 32 or 64 bit platforms. + +# Script originally authored by Mark Hammond. +# Major revisions by: +# Martin v. L?wis +# Christian Heimes +# Zachary Ware + +# THEORETICALLY, you can: +# * Unpack the latest OpenSSL release where $(opensslDir) in +# PCbuild\pyproject.props expects it to be. +# * Install ActivePerl and ensure it is somewhere on your path. +# * Run this script with the OpenSSL source dir as the only argument. +# +# it should configure OpenSSL such that it is ready to be built by +# ssl.vcxproj on 32 or 64 bit platforms. + +import os +import re +import sys +import subprocess +from shutil import copy + +# Find all "foo.exe" files on the PATH. +def find_all_on_path(filename, extras=None): + entries = os.environ["PATH"].split(os.pathsep) + ret = [] + for p in entries: + fname = os.path.abspath(os.path.join(p, filename)) + if os.path.isfile(fname) and fname not in ret: + ret.append(fname) + if extras: + for p in extras: + fname = os.path.abspath(os.path.join(p, filename)) + if os.path.isfile(fname) and fname not in ret: + ret.append(fname) + return ret + + +# Find a suitable Perl installation for OpenSSL. +# cygwin perl does *not* work. ActivePerl does. +# Being a Perl dummy, the simplest way I can check is if the "Win32" package +# is available. +def find_working_perl(perls): + for perl in perls: + try: + subprocess.check_output([perl, "-e", "use Win32;"]) + except subprocess.CalledProcessError: + continue + else: + return perl + + if perls: + print("The following perl interpreters were found:") + for p in perls: + print(" ", p) + print(" None of these versions appear suitable for building OpenSSL") + else: + print("NO perl interpreters were found on this machine at all!") + print(" Please install ActivePerl and ensure it appears on your path") + + +def create_asms(makefile, tmp_d): + #create a custom makefile out of the provided one + asm_makefile = os.path.splitext(makefile)[0] + '.asm.mak' + with open(makefile) as fin, open(asm_makefile, 'w') as fout: + for line in fin: + # Keep everything up to the install target (it's convenient) + if line.startswith('install: all'): + break + fout.write(line) + asms = [] + for line in fin: + if '.asm' in line and line.strip().endswith('.pl'): + asms.append(line.split(':')[0]) + while line.strip(): + fout.write(line) + line = next(fin) + fout.write('\n') + + fout.write('asms: $(TMP_D) ') + fout.write(' '.join(asms)) + fout.write('\n') + os.system('nmake /f {} PERL=perl TMP_D={} asms'.format(asm_makefile, tmp_d)) + + +def copy_includes(makefile, suffix): + dir = 'include'+suffix+'\\openssl' + os.makedirs(dir, exist_ok=True) + copy_if_different = r'$(PERL) $(SRC_D)\util\copy-if-different.pl' + with open(makefile) as fin: + for line in fin: + if copy_if_different in line: + perl, script, src, dest = line.split() + if not '$(INCO_D)' in dest: + continue + # We're in the root of the source tree + src = src.replace('$(SRC_D)', '.').strip('"') + dest = dest.strip('"').replace('$(INCO_D)', dir) + print('copying', src, 'to', dest) + copy(src, dest) + + +def run_configure(configure, do_script): + print("perl Configure "+configure+" no-idea no-mdc2") + os.system("perl Configure "+configure+" no-idea no-mdc2") + print(do_script) + os.system(do_script) + + +def prep(arch): + makefile_template = "ms\\nt{}.mak" + generated_makefile = makefile_template.format('') + if arch == "x86": + configure = "VC-WIN32" + do_script = "ms\\do_nasm" + suffix = "32" + elif arch == "amd64": + configure = "VC-WIN64A" + do_script = "ms\\do_win64a" + suffix = "64" + #os.environ["VSEXTCOMP_USECL"] = "MS_OPTERON" + else: + raise ValueError('Unrecognized platform: %s' % arch) + + print("Creating the makefiles...") + sys.stdout.flush() + # run configure, copy includes, create asms + run_configure(configure, do_script) + makefile = makefile_template.format(suffix) + try: + os.unlink(makefile) + except FileNotFoundError: + pass + os.rename(generated_makefile, makefile) + copy_includes(makefile, suffix) + + print('creating asms...') + create_asms(makefile, 'tmp'+suffix) + + +def main(): + if len(sys.argv) == 1: + print("Not enough arguments: directory containing OpenSSL", + "sources must be supplied") + sys.exit(1) + + if len(sys.argv) > 2: + print("Too many arguments supplied, all we need is the directory", + "containing OpenSSL sources") + sys.exit(1) + + ssl_dir = sys.argv[1] + + if not os.path.isdir(ssl_dir): + print(ssl_dir, "is not an existing directory!") + sys.exit(1) + + # perl should be on the path, but we also look in "\perl" and "c:\\perl" + # as "well known" locations + perls = find_all_on_path("perl.exe", [r"\perl\bin", + r"C:\perl\bin", + r"\perl64\bin", + r"C:\perl64\bin", + ]) + perl = find_working_perl(perls) + if perl: + print("Found a working perl at '%s'" % (perl,)) + else: + sys.exit(1) + if not find_all_on_path('nmake.exe'): + print('Could not find nmake.exe, try running env.bat') + sys.exit(1) + if not find_all_on_path('nasm.exe'): + print('Could not find nasm.exe, please add to PATH') + sys.exit(1) + sys.stdout.flush() + + # Put our working Perl at the front of our path + os.environ["PATH"] = os.path.dirname(perl) + \ + os.pathsep + \ + os.environ["PATH"] + + old_cwd = os.getcwd() + try: + os.chdir(ssl_dir) + for arch in ['amd64', 'x86']: + prep(arch) + finally: + os.chdir(old_cwd) + +if __name__=='__main__': + main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 14:13:30 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 18:13:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_skipping_=2327921_for_?= =?utf-8?q?windows?= Message-ID: <20160905181329.32664.17155.4A28E7F5@psf.io> https://hg.python.org/cpython/rev/df5b6d1c7ea8 changeset: 103051:df5b6d1c7ea8 user: Benjamin Peterson date: Mon Sep 05 11:13:07 2016 -0700 summary: fix skipping #27921 for windows files: Lib/test/test_tools/test_unparse.py | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_tools/test_unparse.py b/Lib/test/test_tools/test_unparse.py --- a/Lib/test/test_tools/test_unparse.py +++ b/Lib/test/test_tools/test_unparse.py @@ -288,8 +288,7 @@ # it's very much a hack that I'm skipping these files, but # I can't figure out why they fail. I'll fix it when I # address issue #27948. - if (filename.endswith('/test_fstring.py') or - filename.endswith('/test_traceback.py')): + if os.path.basename(filename) in ('test_fstring.py', 'test_traceback.py'): if test.support.verbose: print(f'Skipping {filename}: see issue 27921') continue -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 14:44:47 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 05 Sep 2016 18:44:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327830=3A_Remove_u?= =?utf-8?q?nused_=5FPyStack=5FAsDict=28=29?= Message-ID: <20160905184445.47354.60095.70C55342@psf.io> https://hg.python.org/cpython/rev/119af59911a0 changeset: 103052:119af59911a0 user: Victor Stinner date: Mon Sep 05 11:43:18 2016 -0700 summary: Issue #27830: Remove unused _PyStack_AsDict() I forgot to remove this function, I made a mistake in my revert. files: Objects/abstract.c | 34 ---------------------------------- 1 files changed, 0 insertions(+), 34 deletions(-) diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2309,40 +2309,6 @@ return result; } -static PyObject * -_PyStack_AsDict(PyObject **stack, Py_ssize_t nkwargs, PyObject *func) -{ - PyObject *kwdict; - - kwdict = PyDict_New(); - if (kwdict == NULL) { - return NULL; - } - - while (--nkwargs >= 0) { - int err; - PyObject *key = *stack++; - PyObject *value = *stack++; - if (PyDict_GetItem(kwdict, key) != NULL) { - PyErr_Format(PyExc_TypeError, - "%.200s%s got multiple values " - "for keyword argument '%U'", - PyEval_GetFuncName(func), - PyEval_GetFuncDesc(func), - key); - Py_DECREF(kwdict); - return NULL; - } - - err = PyDict_SetItem(kwdict, key, value); - if (err) { - Py_DECREF(kwdict); - return NULL; - } - } - return kwdict; -} - /* Positional arguments are obj followed args. */ PyObject * _PyObject_Call_Prepend(PyObject *func, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 15:13:23 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 19:13:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_memory_indirections?= =?utf-8?q?_in_dict=5Ftraverse_=28closes_=2327956=29?= Message-ID: <20160905191322.114831.66004.C018B3AA@psf.io> https://hg.python.org/cpython/rev/027e421594b7 changeset: 103053:027e421594b7 user: Benjamin Peterson date: Mon Sep 05 12:12:59 2016 -0700 summary: remove memory indirections in dict_traverse (closes #27956) files: Objects/dictobject.c | 20 +++++++++++--------- 1 files changed, 11 insertions(+), 9 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -2519,24 +2519,26 @@ static int dict_traverse(PyObject *op, visitproc visit, void *arg) { - Py_ssize_t i, n; PyDictObject *mp = (PyDictObject *)op; - if (mp->ma_keys->dk_lookup == lookdict) { - for (i = 0; i < DK_SIZE(mp->ma_keys); i++) { - if (mp->ma_keys->dk_entries[i].me_value != NULL) { - Py_VISIT(mp->ma_keys->dk_entries[i].me_value); - Py_VISIT(mp->ma_keys->dk_entries[i].me_key); + PyDictKeysObject *keys = mp->ma_keys; + PyDictKeyEntry *entries = &keys->dk_entries[0]; + Py_ssize_t i, n = DK_SIZE(mp->ma_keys); + if (keys->dk_lookup == lookdict) { + for (i = 0; i < n; i++) { + if (entries[i].me_value != NULL) { + Py_VISIT(entries[i].me_value); + Py_VISIT(entries[i].me_key); } } } else { if (mp->ma_values != NULL) { - for (i = 0, n = DK_SIZE(mp->ma_keys); i < n; i++) { + for (i = 0; i < n; i++) { Py_VISIT(mp->ma_values[i]); } } else { - for (i = 0, n = DK_SIZE(mp->ma_keys); i < n; i++) { - Py_VISIT(mp->ma_keys->dk_entries[i].me_value); + for (i = 0; i < n; i++) { + Py_VISIT(entries[i].me_value); } } } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 15:33:11 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 19:33:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327883=3A_Update_s?= =?utf-8?q?qlite_to_3=2E14=2E1_on_Windows?= Message-ID: <20160905193311.9820.42752.102CA351@psf.io> https://hg.python.org/cpython/rev/792e7a8dddb8 changeset: 103054:792e7a8dddb8 user: Zachary Ware date: Mon Sep 05 14:32:38 2016 -0500 summary: Issue #27883: Update sqlite to 3.14.1 on Windows files: Misc/NEWS | 2 ++ PCbuild/python.props | 2 +- PCbuild/readme.txt | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -155,6 +155,8 @@ Build ----- +- Issue #27883: Update sqlite to 3.14.1.0 on Windows. + - Issue #27917: Set platform triplets for Android builds. - Issue #25825: Update references to the $(LIBPL) installation path on AIX. diff --git a/PCbuild/python.props b/PCbuild/python.props --- a/PCbuild/python.props +++ b/PCbuild/python.props @@ -43,7 +43,7 @@ $([System.IO.Path]::GetFullPath(`$(PySourcePath)externals\`)) - $(ExternalsDir)sqlite-3.8.11.0\ + $(ExternalsDir)sqlite-3.14.1.0\ $(ExternalsDir)bzip2-1.0.6\ $(ExternalsDir)xz-5.0.5\ $(ExternalsDir)openssl-1.0.2h\ diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -204,7 +204,7 @@ functionality to _ssl or _hashlib. They will not clean up their output with the normal Clean target; CleanAll should be used instead. _sqlite3 - Wraps SQLite 3.8.11.0, which is itself built by sqlite3.vcxproj + Wraps SQLite 3.14.1.0, which is itself built by sqlite3.vcxproj Homepage: http://www.sqlite.org/ _tkinter -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 15:40:46 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 19:40:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_get=5Fexternals=2Ebat?= Message-ID: <20160905194045.6425.20047.281FB484@psf.io> https://hg.python.org/cpython/rev/8c6c91983d64 changeset: 103055:8c6c91983d64 user: Zachary Ware date: Mon Sep 05 14:40:25 2016 -0500 summary: Fix get_externals.bat files: PCbuild/get_externals.bat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -55,7 +55,7 @@ set libraries=%libraries% bzip2-1.0.6 if NOT "%IncludeSSL%"=="false" set libraries=%libraries% nasm-2.11.06 if NOT "%IncludeSSL%"=="false" set libraries=%libraries% openssl-1.0.2h -set libraries=%libraries% sqlite-3.8.11.0 +set libraries=%libraries% sqlite-3.14.1.0 if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tcl-core-8.6.6.0 if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tk-8.6.6.0 if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tix-8.4.3.6 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 15:45:02 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 19:45:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_rewrite_unpack?= =?utf-8?q?=5Fadd=5Finfo=2C_so_it_has_less_memory_corruption_bugs_=28close?= =?utf-8?b?cyAjMjc5NDQp?= Message-ID: <20160905194501.9675.15074.22A4E8FD@psf.io> https://hg.python.org/cpython/rev/aefdfa167f4b changeset: 103056:aefdfa167f4b branch: 2.7 parent: 103048:32552131d8cb user: Benjamin Peterson date: Mon Sep 05 12:44:38 2016 -0700 summary: rewrite unpack_add_info, so it has less memory corruption bugs (closes #27944) files: Misc/NEWS | 3 ++ Modules/_hotshot.c | 43 ++++++++++++++++----------------- 2 files changed, 24 insertions(+), 22 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -36,6 +36,9 @@ Library ------- +- Issue #27944: Fix some memory-corruption bugs in the log reading code of the + _hotshot module. + - Issue #27934: Use ``float.__repr__`` instead of plain ``repr`` when JSON- encoding an instance of a float subclass. Thanks Eddie James. diff --git a/Modules/_hotshot.c b/Modules/_hotshot.c --- a/Modules/_hotshot.c +++ b/Modules/_hotshot.c @@ -338,34 +338,33 @@ static int unpack_add_info(LogReaderObject *self) { - PyObject *key; + PyObject *key = NULL; PyObject *value = NULL; int err; err = unpack_string(self, &key); - if (!err) { - err = unpack_string(self, &value); - if (err) - Py_DECREF(key); - else { - PyObject *list = PyDict_GetItem(self->info, key); - if (list == NULL) { - list = PyList_New(0); - if (list == NULL) { - err = ERR_EXCEPTION; - goto finally; - } - if (PyDict_SetItem(self->info, key, list)) { - Py_DECREF(list); - err = ERR_EXCEPTION; - goto finally; - } - Py_DECREF(list); - } - if (PyList_Append(list, value)) - err = ERR_EXCEPTION; + if (err) + goto finally; + err = unpack_string(self, &value); + if (err) + goto finally; + PyObject *list = PyDict_GetItem(self->info, key); + if (list == NULL) { + list = PyList_New(0); + if (list == NULL) { + err = ERR_EXCEPTION; + goto finally; } + if (PyDict_SetItem(self->info, key, list)) { + Py_DECREF(list); + err = ERR_EXCEPTION; + goto finally; + } + Py_DECREF(list); } + if (PyList_Append(list, value)) + err = ERR_EXCEPTION; + finally: Py_XDECREF(key); Py_XDECREF(value); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 16:08:16 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 20:08:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_move_declarati?= =?utf-8?q?on_to_top_of_the_function_to_appease_the_c89_gods?= Message-ID: <20160905200816.47534.91290.A7D05544@psf.io> https://hg.python.org/cpython/rev/cb2551ecf318 changeset: 103057:cb2551ecf318 branch: 2.7 user: Benjamin Peterson date: Mon Sep 05 13:07:48 2016 -0700 summary: move declaration to top of the function to appease the c89 gods files: Modules/_hotshot.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Modules/_hotshot.c b/Modules/_hotshot.c --- a/Modules/_hotshot.c +++ b/Modules/_hotshot.c @@ -340,6 +340,7 @@ { PyObject *key = NULL; PyObject *value = NULL; + PyObject *list; int err; err = unpack_string(self, &key); @@ -348,7 +349,7 @@ err = unpack_string(self, &value); if (err) goto finally; - PyObject *list = PyDict_GetItem(self->info, key); + list = PyDict_GetItem(self->info, key); if (list == NULL) { list = PyList_New(0); if (list == NULL) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 16:16:17 2016 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 05 Sep 2016 20:16:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20160905201606.68872.21581.9601BED8@psf.io> https://hg.python.org/cpython/rev/c6228efa7c41 changeset: 103059:c6228efa7c41 parent: 103055:8c6c91983d64 parent: 103058:096b712cc129 user: Raymond Hettinger date: Mon Sep 05 13:15:20 2016 -0700 summary: Merge files: Doc/library/random.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Doc/library/random.rst b/Doc/library/random.rst --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -329,6 +329,9 @@ >>> weighted_choices = [('Red', 3), ('Blue', 2), ('Yellow', 1), ('Green', 4)] >>> population = [val for val, cnt in weighted_choices for i in range(cnt)] + >>> population + ['Red', 'Red', 'Red', 'Blue', 'Blue', 'Yellow', 'Green', 'Green', 'Green', 'Green'] + >>> random.choice(population) 'Green' @@ -338,6 +341,9 @@ >>> choices, weights = zip(*weighted_choices) >>> cumdist = list(itertools.accumulate(weights)) + >>> cumdist # [3, 3+2, 3+2+1, 3+2+1+4] + [3, 5, 6, 10] + >>> x = random.random() * cumdist[-1] >>> choices[bisect.bisect(cumdist, x)] 'Blue' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 16:16:17 2016 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 05 Sep 2016 20:16:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Improve_recipe?= =?utf-8?q?_by_showing_results_of_intermediate_steps?= Message-ID: <20160905201604.104525.86420.82AF926F@psf.io> https://hg.python.org/cpython/rev/096b712cc129 changeset: 103058:096b712cc129 branch: 3.5 parent: 103049:e0873191ad7d user: Raymond Hettinger date: Mon Sep 05 13:15:02 2016 -0700 summary: Improve recipe by showing results of intermediate steps files: Doc/library/random.rst | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Doc/library/random.rst b/Doc/library/random.rst --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -328,6 +328,9 @@ >>> weighted_choices = [('Red', 3), ('Blue', 2), ('Yellow', 1), ('Green', 4)] >>> population = [val for val, cnt in weighted_choices for i in range(cnt)] + >>> population + ['Red', 'Red', 'Red', 'Blue', 'Blue', 'Yellow', 'Green', 'Green', 'Green', 'Green'] + >>> random.choice(population) 'Green' @@ -337,6 +340,9 @@ >>> choices, weights = zip(*weighted_choices) >>> cumdist = list(itertools.accumulate(weights)) + >>> cumdist # [3, 3+2, 3+2+1, 3+2+1+4] + [3, 5, 6, 10] + >>> x = random.random() * cumdist[-1] >>> choices[bisect.bisect(cumdist, x)] 'Blue' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:03:32 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 21:03:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_remove_ungramm?= =?utf-8?q?arical_apostrophe_=28closes_=2327957=29?= Message-ID: <20160905210332.114862.72790.3C8196A4@psf.io> https://hg.python.org/cpython/rev/33702360acef changeset: 103060:33702360acef branch: 3.5 parent: 103058:096b712cc129 user: Benjamin Peterson date: Mon Sep 05 14:02:59 2016 -0700 summary: remove ungrammarical apostrophe (closes #27957) files: Doc/library/importlib.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -118,7 +118,7 @@ :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path` is done. ``None`` is returned if no loader is found. - A dotted name does not have its parent's implicitly imported as that requires + A dotted name does not have its parents implicitly imported as that requires loading them and that may not be desired. To properly import a submodule you will need to import all parent packages of the submodule and use the correct argument to *path*. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:03:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 21:03:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjc5NTcp?= Message-ID: <20160905210332.6489.33814.543A2E1F@psf.io> https://hg.python.org/cpython/rev/f8d9569900a9 changeset: 103061:f8d9569900a9 parent: 103059:c6228efa7c41 parent: 103060:33702360acef user: Benjamin Peterson date: Mon Sep 05 14:03:08 2016 -0700 summary: merge 3.5 (#27957) files: Doc/library/importlib.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -118,7 +118,7 @@ :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path` is done. ``None`` is returned if no loader is found. - A dotted name does not have its parent's implicitly imported as that requires + A dotted name does not have its parents implicitly imported as that requires loading them and that may not be desired. To properly import a submodule you will need to import all parent packages of the submodule and use the correct argument to *path*. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:06:02 2016 From: python-checkins at python.org (steve.dower) Date: Mon, 05 Sep 2016 21:06:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327756=3A_Adds_new?= =?utf-8?q?_icons_for_Python_files_and_processes_on_Windows=2E_Designs?= Message-ID: <20160905210602.23482.21289.17624E21@psf.io> https://hg.python.org/cpython/rev/29815771f2ac changeset: 103062:29815771f2ac user: Steve Dower date: Mon Sep 05 14:05:17 2016 -0700 summary: Issue #27756: Adds new icons for Python files and processes on Windows. Designs by Cherry Wang. files: Misc/NEWS | 5 +++++ PC/icons/launcher.icns | Bin PC/icons/launcher.ico | Bin PC/icons/launcher.svg | 1 + PC/icons/py.icns | Bin PC/icons/py.ico | Bin PC/icons/py.svg | 1 + PC/icons/pyc.icns | Bin PC/icons/pyc.ico | Bin PC/icons/pyc.svg | 1 + PC/icons/pyd.icns | Bin PC/icons/pyd.ico | Bin PC/icons/pyd.svg | 1 + PC/icons/python.icns | Bin PC/icons/python.ico | Bin PC/icons/python.svg | 1 + PC/icons/pythonw.icns | Bin PC/icons/pythonw.ico | Bin PC/icons/pythonw.svg | 1 + PC/icons/setup.icns | Bin PC/icons/setup.ico | Bin PC/icons/setup.svg | 1 + PC/launcher.ico | Bin PC/py.ico | Bin PC/pyc.ico | Bin PC/pycon.ico | Bin PC/pylauncher.rc | 10 +++++++--- PC/python_exe.rc | 2 +- PC/pythonw_exe.rc | 6 +++--- PCbuild/pythonw.vcxproj | 2 +- Tools/msi/bundle/bundle.ico | Bin Tools/msi/bundle/bundle.wxs | 2 +- Tools/msi/common.wxs | 2 +- Tools/msi/exe/exe_files.wxs | 7 +++++-- Tools/msi/launcher/launcher_en-US.wxl | 1 + Tools/msi/launcher/launcher_reg.wxs | 8 ++++++-- 36 files changed, 38 insertions(+), 14 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -162,6 +162,11 @@ - Issue #25825: Update references to the $(LIBPL) installation path on AIX. This path was changed in 3.2a4. +Windows +------- + +- Issue #27756: Adds new icons for Python files and processes on Windows. + Designs by Cherry Wang. What's New in Python 3.6.0 alpha 4 ================================== diff --git a/PC/icons/launcher.icns b/PC/icons/launcher.icns new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..59a917f20d37ca7d86723f75fbb8e8324ce69710 GIT binary patch [stripped] diff --git a/PC/icons/launcher.ico b/PC/icons/launcher.ico new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..c4e3c693dcb4a9baa6ba26cdfe1359fc552db2f7 GIT binary patch [stripped] diff --git a/PC/icons/launcher.svg b/PC/icons/launcher.svg new file mode 100644 --- /dev/null +++ b/PC/icons/launcher.svg @@ -0,0 +1,1 @@ + \ No newline at end of file diff --git a/PC/icons/py.icns b/PC/icons/py.icns new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..2dc4e296f81e823ab751172c02ebb97d65852648 GIT binary patch [stripped] diff --git a/PC/icons/py.ico b/PC/icons/py.ico new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..1d8a79bfb310c71c30d2d59a975c8bae82775ee5 GIT binary patch [stripped] diff --git a/PC/icons/py.svg b/PC/icons/py.svg new file mode 100644 --- /dev/null +++ b/PC/icons/py.svg @@ -0,0 +1,1 @@ + \ No newline at end of file diff --git a/PC/icons/pyc.icns b/PC/icons/pyc.icns new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..50da9a161501d9566c668f5a09720f60630d9914 GIT binary patch [stripped] diff --git a/PC/icons/pyc.ico b/PC/icons/pyc.ico new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..74dde81b649eedaee58db05851f4f8c4af246501 GIT binary patch [stripped] diff --git a/PC/icons/pyc.svg b/PC/icons/pyc.svg new file mode 100644 --- /dev/null +++ b/PC/icons/pyc.svg @@ -0,0 +1,1 @@ + \ No newline at end of file diff --git a/PC/icons/pyd.icns b/PC/icons/pyd.icns new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..5d3d24ee2567946ae21327efd208c6029958cc87 GIT binary patch [stripped] diff --git a/PC/icons/pyd.ico b/PC/icons/pyd.ico new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..9f6cb601afb65ab1821af0d7418fa6dcfd83e16d GIT binary patch [stripped] diff --git a/PC/icons/pyd.svg b/PC/icons/pyd.svg new file mode 100644 --- /dev/null +++ b/PC/icons/pyd.svg @@ -0,0 +1,1 @@ + \ No newline at end of file diff --git a/PC/icons/python.icns b/PC/icons/python.icns new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..fc53e02f4a91989a80570f8fdfc288b1ade7e378 GIT binary patch [stripped] diff --git a/PC/icons/python.ico b/PC/icons/python.ico new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..b8a38ef15994ff855ce1cecd379cab146ea82dda GIT binary patch [stripped] diff --git a/PC/icons/python.svg b/PC/icons/python.svg new file mode 100644 --- /dev/null +++ b/PC/icons/python.svg @@ -0,0 +1,1 @@ + \ No newline at end of file diff --git a/PC/icons/pythonw.icns b/PC/icons/pythonw.icns new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..9354cf870a9f6a5e6154fa815fa3137aa0659457 GIT binary patch [stripped] diff --git a/PC/icons/pythonw.ico b/PC/icons/pythonw.ico new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..6195d433475e06186e5d5dc92e005c1926f20fe6 GIT binary patch [stripped] diff --git a/PC/icons/pythonw.svg b/PC/icons/pythonw.svg new file mode 100644 --- /dev/null +++ b/PC/icons/pythonw.svg @@ -0,0 +1,1 @@ + \ No newline at end of file diff --git a/PC/icons/setup.icns b/PC/icons/setup.icns new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..6f0e6b01a0da4bcde3128932ef9e2b221eeabfcc GIT binary patch [stripped] diff --git a/PC/icons/setup.ico b/PC/icons/setup.ico new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..e54364b3af11d3173c8b48d8ee32bce18f85f2ec GIT binary patch [stripped] diff --git a/PC/icons/setup.svg b/PC/icons/setup.svg new file mode 100644 --- /dev/null +++ b/PC/icons/setup.svg @@ -0,0 +1,1 @@ + \ No newline at end of file diff --git a/PC/launcher.ico b/PC/launcher.ico deleted file mode 100644 index dad7d572ce781b7b0916ed669207f1ae3b9ad83c..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/PC/py.ico b/PC/py.ico deleted file mode 100644 index 3357aef14888c501bcd7bfe02393760306a18d06..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/PC/pyc.ico b/PC/pyc.ico deleted file mode 100644 index f7bd2b1cc238c09301af80da5b8085ce13150b97..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/PC/pycon.ico b/PC/pycon.ico deleted file mode 100644 index 1ab629eff26946e1b8c0cd3223e80257b74deb88..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/PC/pylauncher.rc b/PC/pylauncher.rc --- a/PC/pylauncher.rc +++ b/PC/pylauncher.rc @@ -7,9 +7,13 @@ #include 1 RT_MANIFEST "python.manifest" -1 ICON DISCARDABLE "launcher.ico" -2 ICON DISCARDABLE "py.ico" -3 ICON DISCARDABLE "pyc.ico" +1 ICON DISCARDABLE "icons\launcher.ico" +2 ICON DISCARDABLE "icons\py.ico" +3 ICON DISCARDABLE "icons\pyc.ico" +4 ICON DISCARDABLE "icons\pyd.ico" +5 ICON DISCARDABLE "icons\python.ico" +6 ICON DISCARDABLE "icons\pythonw.ico" +7 ICON DISCARDABLE "icons\setup.ico" ///////////////////////////////////////////////////////////////////////////// // diff --git a/PC/python_exe.rc b/PC/python_exe.rc --- a/PC/python_exe.rc +++ b/PC/python_exe.rc @@ -7,7 +7,7 @@ #include 1 RT_MANIFEST "python.manifest" -1 ICON DISCARDABLE "pycon.ico" +1 ICON DISCARDABLE "icons\python.ico" ///////////////////////////////////////////////////////////////////////////// diff --git a/PC/python_exe.rc b/PC/pythonw_exe.rc copy from PC/python_exe.rc copy to PC/pythonw_exe.rc --- a/PC/python_exe.rc +++ b/PC/pythonw_exe.rc @@ -7,7 +7,7 @@ #include 1 RT_MANIFEST "python.manifest" -1 ICON DISCARDABLE "pycon.ico" +1 ICON DISCARDABLE "icons\pythonw.ico" ///////////////////////////////////////////////////////////////////////////// @@ -35,9 +35,9 @@ VALUE "CompanyName", PYTHON_COMPANY "\0" VALUE "FileDescription", "Python\0" VALUE "FileVersion", PYTHON_VERSION - VALUE "InternalName", "Python Console\0" + VALUE "InternalName", "Python Application\0" VALUE "LegalCopyright", PYTHON_COPYRIGHT "\0" - VALUE "OriginalFilename", "python" PYTHON_DEBUG_EXT ".exe\0" + VALUE "OriginalFilename", "pythonw" PYTHON_DEBUG_EXT ".exe\0" VALUE "ProductName", "Python\0" VALUE "ProductVersion", PYTHON_VERSION END diff --git a/PCbuild/pythonw.vcxproj b/PCbuild/pythonw.vcxproj --- a/PCbuild/pythonw.vcxproj +++ b/PCbuild/pythonw.vcxproj @@ -62,7 +62,7 @@ - + diff --git a/Tools/msi/bundle/bundle.ico b/Tools/msi/bundle/bundle.ico deleted file mode 100644 index 1ab629eff26946e1b8c0cd3223e80257b74deb88..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Tools/msi/bundle/bundle.wxs b/Tools/msi/bundle/bundle.wxs --- a/Tools/msi/bundle/bundle.wxs +++ b/Tools/msi/bundle/bundle.wxs @@ -4,7 +4,7 @@ - + diff --git a/Tools/msi/exe/exe_files.wxs b/Tools/msi/exe/exe_files.wxs --- a/Tools/msi/exe/exe_files.wxs +++ b/Tools/msi/exe/exe_files.wxs @@ -69,10 +69,13 @@ - + - + + + + diff --git a/Tools/msi/launcher/launcher_en-US.wxl b/Tools/msi/launcher/launcher_en-US.wxl --- a/Tools/msi/launcher/launcher_en-US.wxl +++ b/Tools/msi/launcher/launcher_en-US.wxl @@ -11,6 +11,7 @@ Python File Python File (no console) Compiled Python File + Python Extension Module Python Zip Application File Python Zip Application File (no console) diff --git a/Tools/msi/launcher/launcher_reg.wxs b/Tools/msi/launcher/launcher_reg.wxs --- a/Tools/msi/launcher/launcher_reg.wxs +++ b/Tools/msi/launcher/launcher_reg.wxs @@ -27,14 +27,18 @@ - + + + + + - + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:15:36 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 21:15:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzQ4?= =?utf-8?q?=3A_Simplify_test=5Fwinsound=2E?= Message-ID: <20160905211536.8656.74666.27B73E3E@psf.io> https://hg.python.org/cpython/rev/82467d0dbaea changeset: 103064:82467d0dbaea branch: 3.5 parent: 103060:33702360acef user: Zachary Ware date: Mon Sep 05 16:06:56 2016 -0500 summary: Issue #27748: Simplify test_winsound. The tests no longer attempt to figure out if a soundcard or particular system sounds are available. Instead, it just tries everything and accepts RuntimeError as a flavor of success. files: Lib/test/check_soundcard.vbs | 13 - Lib/test/test_winsound.py | 248 +++++----------------- 2 files changed, 60 insertions(+), 201 deletions(-) diff --git a/Lib/test/check_soundcard.vbs b/Lib/test/check_soundcard.vbs deleted file mode 100644 --- a/Lib/test/check_soundcard.vbs +++ /dev/null @@ -1,13 +0,0 @@ -rem Check for a working sound-card - exit with 0 if OK, 1 otherwise. -set wmi = GetObject("winmgmts:") -set scs = wmi.InstancesOf("win32_sounddevice") -for each sc in scs - set status = sc.Properties_("Status") - wscript.Echo(sc.Properties_("Name") + "/" + status) - if status = "OK" then - wscript.Quit 0 rem normal exit - end if -next -rem No sound card found - exit with status code of 1 -wscript.Quit 1 - diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -1,38 +1,42 @@ # Ridiculously simple test of the winsound module for Windows. -import unittest -from test import support -support.requires('audio') -import time +import functools import os import subprocess +import time +import unittest +from test import support + +support.requires('audio') winsound = support.import_module('winsound') -ctypes = support.import_module('ctypes') -import winreg -def has_sound(sound): - """Find out if a particular event is configured with a default sound""" - try: - # Ask the mixer API for the number of devices it knows about. - # When there are no devices, PlaySound will fail. - if ctypes.windll.winmm.mixerGetNumDevs() == 0: - return False - key = winreg.OpenKeyEx(winreg.HKEY_CURRENT_USER, - "AppEvents\Schemes\Apps\.Default\{0}\.Default".format(sound)) - return winreg.EnumValue(key, 0)[1] != "" - except OSError: - return False +# Unless we actually have an ear in the room, we have no idea whether a sound +# actually plays, and it's incredibly flaky trying to figure out if a sound +# even *should* play. Instead of guessing, just call the function and assume +# it either passed or raised the RuntimeError we expect in case of failure. +def sound_func(func): + @functools.wraps(func) + def wrapper(*args, **kwargs): + try: + ret = func(*args, **kwargs) + except RuntimeError as e: + if support.verbose: + print(func.__name__, 'failed:', e) + else: + if support.verbose: + print(func.__name__, 'returned') + return ret + return wrapper + + +safe_Beep = sound_func(winsound.Beep) +safe_MessageBeep = sound_func(winsound.MessageBeep) +safe_PlaySound = sound_func(winsound.PlaySound) + class BeepTest(unittest.TestCase): - # As with PlaySoundTest, incorporate the _have_soundcard() check - # into our test methods. If there's no audio device present, - # winsound.Beep returns 0 and GetLastError() returns 127, which - # is: ERROR_PROC_NOT_FOUND ("The specified procedure could not - # be found"). (FWIW, virtual/Hyper-V systems fall under this - # scenario as they have no sound devices whatsoever (not even - # a legacy Beep device).) def test_errors(self): self.assertRaises(TypeError, winsound.Beep) @@ -40,27 +44,12 @@ self.assertRaises(ValueError, winsound.Beep, 32768, 75) def test_extremes(self): - self._beep(37, 75) - self._beep(32767, 75) + safe_Beep(37, 75) + safe_Beep(32767, 75) def test_increasingfrequency(self): for i in range(100, 2000, 100): - self._beep(i, 75) - - def _beep(self, *args): - # these tests used to use _have_soundcard(), but it's quite - # possible to have a soundcard, and yet have the beep driver - # disabled. So basically, we have no way of knowing whether - # a beep should be produced or not, so currently if these - # tests fail we're ignoring them - # - # XXX the right fix for this is to define something like - # _have_enabled_beep_driver() and use that instead of the - # try/except below - try: - winsound.Beep(*args) - except RuntimeError: - pass + safe_Beep(i, 75) class MessageBeepTest(unittest.TestCase): @@ -70,22 +59,22 @@ def test_default(self): self.assertRaises(TypeError, winsound.MessageBeep, "bad") self.assertRaises(TypeError, winsound.MessageBeep, 42, 42) - winsound.MessageBeep() + safe_MessageBeep() def test_ok(self): - winsound.MessageBeep(winsound.MB_OK) + safe_MessageBeep(winsound.MB_OK) def test_asterisk(self): - winsound.MessageBeep(winsound.MB_ICONASTERISK) + safe_MessageBeep(winsound.MB_ICONASTERISK) def test_exclamation(self): - winsound.MessageBeep(winsound.MB_ICONEXCLAMATION) + safe_MessageBeep(winsound.MB_ICONEXCLAMATION) def test_hand(self): - winsound.MessageBeep(winsound.MB_ICONHAND) + safe_MessageBeep(winsound.MB_ICONHAND) def test_question(self): - winsound.MessageBeep(winsound.MB_ICONQUESTION) + safe_MessageBeep(winsound.MB_ICONQUESTION) class PlaySoundTest(unittest.TestCase): @@ -99,151 +88,34 @@ "none", winsound.SND_ASYNC | winsound.SND_MEMORY ) - @unittest.skipUnless(has_sound("SystemAsterisk"), - "No default SystemAsterisk") - def test_alias_asterisk(self): - if _have_soundcard(): - winsound.PlaySound('SystemAsterisk', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemAsterisk', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemExclamation"), - "No default SystemExclamation") - def test_alias_exclamation(self): - if _have_soundcard(): - winsound.PlaySound('SystemExclamation', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemExclamation', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemExit"), "No default SystemExit") - def test_alias_exit(self): - if _have_soundcard(): - winsound.PlaySound('SystemExit', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemExit', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemHand"), "No default SystemHand") - def test_alias_hand(self): - if _have_soundcard(): - winsound.PlaySound('SystemHand', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemHand', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemQuestion"), - "No default SystemQuestion") - def test_alias_question(self): - if _have_soundcard(): - winsound.PlaySound('SystemQuestion', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemQuestion', winsound.SND_ALIAS - ) + def test_aliases(self): + aliases = [ + "SystemAsterisk", + "SystemExclamation", + "SystemExit", + "SystemHand", + "SystemQuestion", + ] + for alias in aliases: + with self.subTest(alias=alias): + safe_PlaySound(alias, winsound.SND_ALIAS) def test_alias_fallback(self): - # In the absence of the ability to tell if a sound was actually - # played, this test has two acceptable outcomes: success (no error, - # sound was theoretically played; although as issue #19987 shows - # a box without a soundcard can "succeed") or RuntimeError. Any - # other error is a failure. - try: - winsound.PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) - except RuntimeError: - pass + safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) def test_alias_nofallback(self): - if _have_soundcard(): - # Note that this is not the same as asserting RuntimeError - # will get raised: you cannot convert this to - # self.assertRaises(...) form. The attempt may or may not - # raise RuntimeError, but it shouldn't raise anything other - # than RuntimeError, and that's all we're trying to test - # here. The MS docs aren't clear about whether the SDK - # PlaySound() with SND_ALIAS and SND_NODEFAULT will return - # True or False when the alias is unknown. On Tim's WinXP - # box today, it returns True (no exception is raised). What - # we'd really like to test is that no sound is played, but - # that requires first wiring an eardrum class into unittest - # . - try: - winsound.PlaySound( - '!"$%&/(#+*', - winsound.SND_ALIAS | winsound.SND_NODEFAULT - ) - except RuntimeError: - pass - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - '!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT - ) + safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT) def test_stopasync(self): - if _have_soundcard(): - winsound.PlaySound( - 'SystemQuestion', - winsound.SND_ALIAS | winsound.SND_ASYNC | winsound.SND_LOOP - ) - time.sleep(0.5) - try: - winsound.PlaySound( - 'SystemQuestion', - winsound.SND_ALIAS | winsound.SND_NOSTOP - ) - except RuntimeError: - pass - else: # the first sound might already be finished - pass - winsound.PlaySound(None, winsound.SND_PURGE) - else: - # Issue 8367: PlaySound(None, winsound.SND_PURGE) - # does not raise on systems without a sound card. - pass - - -def _get_cscript_path(): - """Return the full path to cscript.exe or None.""" - for dir in os.environ.get("PATH", "").split(os.pathsep): - cscript_path = os.path.join(dir, "cscript.exe") - if os.path.exists(cscript_path): - return cscript_path - -__have_soundcard_cache = None -def _have_soundcard(): - """Return True iff this computer has a soundcard.""" - global __have_soundcard_cache - if __have_soundcard_cache is None: - cscript_path = _get_cscript_path() - if cscript_path is None: - # Could not find cscript.exe to run our VBScript helper. Default - # to True: most computers these days *do* have a soundcard. - return True - - check_script = os.path.join(os.path.dirname(__file__), - "check_soundcard.vbs") - p = subprocess.Popen([cscript_path, check_script], - stdout=subprocess.PIPE) - __have_soundcard_cache = not p.wait() - p.stdout.close() - return __have_soundcard_cache + safe_PlaySound( + 'SystemQuestion', + winsound.SND_ALIAS | winsound.SND_ASYNC | winsound.SND_LOOP + ) + time.sleep(0.5) + safe_PlaySound('SystemQuestion', winsound.SND_ALIAS | winsound.SND_NOSTOP) + # Issue 8367: PlaySound(None, winsound.SND_PURGE) + # does not raise on systems without a sound card. + winsound.PlaySound(None, winsound.SND_PURGE) if __name__ == "__main__": -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:15:36 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 21:15:36 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NzQ4?= =?utf-8?q?=3A_Simplify_test=5Fwinsound=2E?= Message-ID: <20160905211536.8334.33671.7AD08DEB@psf.io> https://hg.python.org/cpython/rev/4c91651912d1 changeset: 103063:4c91651912d1 branch: 2.7 parent: 103057:cb2551ecf318 user: Zachary Ware date: Mon Sep 05 15:09:41 2016 -0500 summary: Issue #27748: Simplify test_winsound. The tests no longer attempt to figure out if a soundcard or particular system sounds are available. Instead, it just tries everything and accepts RuntimeError as a flavor of success. files: Lib/test/check_soundcard.vbs | 13 - Lib/test/test_winsound.py | 248 +++++----------------- 2 files changed, 60 insertions(+), 201 deletions(-) diff --git a/Lib/test/check_soundcard.vbs b/Lib/test/check_soundcard.vbs deleted file mode 100644 --- a/Lib/test/check_soundcard.vbs +++ /dev/null @@ -1,13 +0,0 @@ -rem Check for a working sound-card - exit with 0 if OK, 1 otherwise. -set wmi = GetObject("winmgmts:") -set scs = wmi.InstancesOf("win32_sounddevice") -for each sc in scs - set status = sc.Properties_("Status") - wscript.Echo(sc.Properties_("Name") + "/" + status) - if status = "OK" then - wscript.Quit 0 rem normal exit - end if -next -rem No sound card found - exit with status code of 1 -wscript.Quit 1 - diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -1,42 +1,41 @@ # Ridiculously simple test of the winsound module for Windows. -import unittest -from test import test_support -test_support.requires('audio') -import time +from __future__ import print_function + +import functools import os import subprocess +import time +import unittest -winsound = test_support.import_module('winsound') -ctypes = test_support.import_module('ctypes') -import _winreg +from test import test_support as support -def has_sound(sound): - """Find out if a particular event is configured with a default sound""" - try: - # Ask the mixer API for the number of devices it knows about. - # When there are no devices, PlaySound will fail. - if ctypes.windll.winmm.mixerGetNumDevs() is 0: - return False +support.requires('audio') +winsound = support.import_module('winsound') - key = _winreg.OpenKeyEx(_winreg.HKEY_CURRENT_USER, - "AppEvents\Schemes\Apps\.Default\{0}\.Default".format(sound)) - value = _winreg.EnumValue(key, 0)[1] - if value is not u"": - return True +# Unless we actually have an ear in the room, we have no idea whether a sound +# actually plays, and it's incredibly flaky trying to figure out if a sound +# even *should* play. Instead of guessing, just call the function and assume +# it either passed or raised the RuntimeError we expect in case of failure. +def sound_func(func): + @functools.wraps(func) + def wrapper(*args, **kwargs): + try: + ret = func(*args, **kwargs) + except RuntimeError as e: + if support.verbose: + print(func.__name__, 'failed:', e) else: - return False - except WindowsError: - return False + if support.verbose: + print(func.__name__, 'returned') + return ret + return wrapper + +safe_Beep = sound_func(winsound.Beep) +safe_MessageBeep = sound_func(winsound.MessageBeep) +safe_PlaySound = sound_func(winsound.PlaySound) class BeepTest(unittest.TestCase): - # As with PlaySoundTest, incorporate the _have_soundcard() check - # into our test methods. If there's no audio device present, - # winsound.Beep returns 0 and GetLastError() returns 127, which - # is: ERROR_PROC_NOT_FOUND ("The specified procedure could not - # be found"). (FWIW, virtual/Hyper-V systems fall under this - # scenario as they have no sound devices whatsoever (not even - # a legacy Beep device).) def test_errors(self): self.assertRaises(TypeError, winsound.Beep) @@ -44,27 +43,13 @@ self.assertRaises(ValueError, winsound.Beep, 32768, 75) def test_extremes(self): - self._beep(37, 75) - self._beep(32767, 75) + safe_Beep(37, 75) + safe_Beep(32767, 75) def test_increasingfrequency(self): for i in xrange(100, 2000, 100): - self._beep(i, 75) + safe_Beep(i, 75) - def _beep(self, *args): - # these tests used to use _have_soundcard(), but it's quite - # possible to have a soundcard, and yet have the beep driver - # disabled. So basically, we have no way of knowing whether - # a beep should be produced or not, so currently if these - # tests fail we're ignoring them - # - # XXX the right fix for this is to define something like - # _have_enabled_beep_driver() and use that instead of the - # try/except below - try: - winsound.Beep(*args) - except RuntimeError: - pass class MessageBeepTest(unittest.TestCase): @@ -74,22 +59,22 @@ def test_default(self): self.assertRaises(TypeError, winsound.MessageBeep, "bad") self.assertRaises(TypeError, winsound.MessageBeep, 42, 42) - winsound.MessageBeep() + safe_MessageBeep() def test_ok(self): - winsound.MessageBeep(winsound.MB_OK) + safe_MessageBeep(winsound.MB_OK) def test_asterisk(self): - winsound.MessageBeep(winsound.MB_ICONASTERISK) + safe_MessageBeep(winsound.MB_ICONASTERISK) def test_exclamation(self): - winsound.MessageBeep(winsound.MB_ICONEXCLAMATION) + safe_MessageBeep(winsound.MB_ICONEXCLAMATION) def test_hand(self): - winsound.MessageBeep(winsound.MB_ICONHAND) + safe_MessageBeep(winsound.MB_ICONHAND) def test_question(self): - winsound.MessageBeep(winsound.MB_ICONQUESTION) + safe_MessageBeep(winsound.MB_ICONQUESTION) class PlaySoundTest(unittest.TestCase): @@ -103,151 +88,38 @@ "none", winsound.SND_ASYNC | winsound.SND_MEMORY ) - @unittest.skipUnless(has_sound("SystemAsterisk"), "No default SystemAsterisk") - def test_alias_asterisk(self): - if _have_soundcard(): - winsound.PlaySound('SystemAsterisk', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemAsterisk', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemExclamation"), "No default SystemExclamation") - def test_alias_exclamation(self): - if _have_soundcard(): - winsound.PlaySound('SystemExclamation', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemExclamation', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemExit"), "No default SystemExit") - def test_alias_exit(self): - if _have_soundcard(): - winsound.PlaySound('SystemExit', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemExit', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemHand"), "No default SystemHand") - def test_alias_hand(self): - if _have_soundcard(): - winsound.PlaySound('SystemHand', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemHand', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemQuestion"), "No default SystemQuestion") - def test_alias_question(self): - if _have_soundcard(): - winsound.PlaySound('SystemQuestion', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemQuestion', winsound.SND_ALIAS - ) + def test_aliases(self): + aliases = [ + "SystemAsterisk", + "SystemExclamation", + "SystemExit", + "SystemHand", + "SystemQuestion", + ] + for alias in aliases: + safe_PlaySound(alias, winsound.SND_ALIAS) def test_alias_fallback(self): - # In the absence of the ability to tell if a sound was actually - # played, this test has two acceptable outcomes: success (no error, - # sound was theoretically played; although as issue #19987 shows - # a box without a soundcard can "succeed") or RuntimeError. Any - # other error is a failure. - try: - winsound.PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) - except RuntimeError: - pass + safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) def test_alias_nofallback(self): - if _have_soundcard(): - # Note that this is not the same as asserting RuntimeError - # will get raised: you cannot convert this to - # self.assertRaises(...) form. The attempt may or may not - # raise RuntimeError, but it shouldn't raise anything other - # than RuntimeError, and that's all we're trying to test - # here. The MS docs aren't clear about whether the SDK - # PlaySound() with SND_ALIAS and SND_NODEFAULT will return - # True or False when the alias is unknown. On Tim's WinXP - # box today, it returns True (no exception is raised). What - # we'd really like to test is that no sound is played, but - # that requires first wiring an eardrum class into unittest - # . - try: - winsound.PlaySound( - '!"$%&/(#+*', - winsound.SND_ALIAS | winsound.SND_NODEFAULT - ) - except RuntimeError: - pass - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - '!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT - ) + safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT) def test_stopasync(self): - if _have_soundcard(): - winsound.PlaySound( - 'SystemQuestion', - winsound.SND_ALIAS | winsound.SND_ASYNC | winsound.SND_LOOP - ) - time.sleep(0.5) - try: - winsound.PlaySound( - 'SystemQuestion', - winsound.SND_ALIAS | winsound.SND_NOSTOP - ) - except RuntimeError: - pass - else: # the first sound might already be finished - pass - winsound.PlaySound(None, winsound.SND_PURGE) - else: - # Issue 8367: PlaySound(None, winsound.SND_PURGE) - # does not raise on systems without a sound card. - pass - - -def _get_cscript_path(): - """Return the full path to cscript.exe or None.""" - for dir in os.environ.get("PATH", "").split(os.pathsep): - cscript_path = os.path.join(dir, "cscript.exe") - if os.path.exists(cscript_path): - return cscript_path - -__have_soundcard_cache = None -def _have_soundcard(): - """Return True iff this computer has a soundcard.""" - global __have_soundcard_cache - if __have_soundcard_cache is None: - cscript_path = _get_cscript_path() - if cscript_path is None: - # Could not find cscript.exe to run our VBScript helper. Default - # to True: most computers these days *do* have a soundcard. - return True - - check_script = os.path.join(os.path.dirname(__file__), - "check_soundcard.vbs") - p = subprocess.Popen([cscript_path, check_script], - stdout=subprocess.PIPE) - __have_soundcard_cache = not p.wait() - return __have_soundcard_cache + safe_PlaySound( + 'SystemQuestion', + winsound.SND_ALIAS | winsound.SND_ASYNC | winsound.SND_LOOP + ) + time.sleep(0.5) + safe_PlaySound('SystemQuestion', winsound.SND_ALIAS | winsound.SND_NOSTOP) + # Issue 8367: PlaySound(None, winsound.SND_PURGE) + # does not raise on systems without a sound card. + winsound.PlaySound(None, winsound.SND_PURGE) def test_main(): - test_support.run_unittest(BeepTest, MessageBeepTest, PlaySoundTest) + support.run_unittest(BeepTest, MessageBeepTest, PlaySoundTest) + if __name__=="__main__": test_main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:15:36 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 21:15:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327748=3A_Merge_with_3=2E5?= Message-ID: <20160905211536.47340.14138.34C55BBD@psf.io> https://hg.python.org/cpython/rev/27cbdd8cf67f changeset: 103065:27cbdd8cf67f parent: 103062:29815771f2ac parent: 103064:82467d0dbaea user: Zachary Ware date: Mon Sep 05 16:15:20 2016 -0500 summary: Closes #27748: Merge with 3.5 files: Lib/test/check_soundcard.vbs | 13 - Lib/test/test_winsound.py | 248 +++++----------------- 2 files changed, 60 insertions(+), 201 deletions(-) diff --git a/Lib/test/check_soundcard.vbs b/Lib/test/check_soundcard.vbs deleted file mode 100644 --- a/Lib/test/check_soundcard.vbs +++ /dev/null @@ -1,13 +0,0 @@ -rem Check for a working sound-card - exit with 0 if OK, 1 otherwise. -set wmi = GetObject("winmgmts:") -set scs = wmi.InstancesOf("win32_sounddevice") -for each sc in scs - set status = sc.Properties_("Status") - wscript.Echo(sc.Properties_("Name") + "/" + status) - if status = "OK" then - wscript.Quit 0 rem normal exit - end if -next -rem No sound card found - exit with status code of 1 -wscript.Quit 1 - diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -1,38 +1,42 @@ # Ridiculously simple test of the winsound module for Windows. -import unittest -from test import support -support.requires('audio') -import time +import functools import os import subprocess +import time +import unittest +from test import support + +support.requires('audio') winsound = support.import_module('winsound') -ctypes = support.import_module('ctypes') -import winreg -def has_sound(sound): - """Find out if a particular event is configured with a default sound""" - try: - # Ask the mixer API for the number of devices it knows about. - # When there are no devices, PlaySound will fail. - if ctypes.windll.winmm.mixerGetNumDevs() == 0: - return False - key = winreg.OpenKeyEx(winreg.HKEY_CURRENT_USER, - "AppEvents\Schemes\Apps\.Default\{0}\.Default".format(sound)) - return winreg.EnumValue(key, 0)[1] != "" - except OSError: - return False +# Unless we actually have an ear in the room, we have no idea whether a sound +# actually plays, and it's incredibly flaky trying to figure out if a sound +# even *should* play. Instead of guessing, just call the function and assume +# it either passed or raised the RuntimeError we expect in case of failure. +def sound_func(func): + @functools.wraps(func) + def wrapper(*args, **kwargs): + try: + ret = func(*args, **kwargs) + except RuntimeError as e: + if support.verbose: + print(func.__name__, 'failed:', e) + else: + if support.verbose: + print(func.__name__, 'returned') + return ret + return wrapper + + +safe_Beep = sound_func(winsound.Beep) +safe_MessageBeep = sound_func(winsound.MessageBeep) +safe_PlaySound = sound_func(winsound.PlaySound) + class BeepTest(unittest.TestCase): - # As with PlaySoundTest, incorporate the _have_soundcard() check - # into our test methods. If there's no audio device present, - # winsound.Beep returns 0 and GetLastError() returns 127, which - # is: ERROR_PROC_NOT_FOUND ("The specified procedure could not - # be found"). (FWIW, virtual/Hyper-V systems fall under this - # scenario as they have no sound devices whatsoever (not even - # a legacy Beep device).) def test_errors(self): self.assertRaises(TypeError, winsound.Beep) @@ -40,27 +44,12 @@ self.assertRaises(ValueError, winsound.Beep, 32768, 75) def test_extremes(self): - self._beep(37, 75) - self._beep(32767, 75) + safe_Beep(37, 75) + safe_Beep(32767, 75) def test_increasingfrequency(self): for i in range(100, 2000, 100): - self._beep(i, 75) - - def _beep(self, *args): - # these tests used to use _have_soundcard(), but it's quite - # possible to have a soundcard, and yet have the beep driver - # disabled. So basically, we have no way of knowing whether - # a beep should be produced or not, so currently if these - # tests fail we're ignoring them - # - # XXX the right fix for this is to define something like - # _have_enabled_beep_driver() and use that instead of the - # try/except below - try: - winsound.Beep(*args) - except RuntimeError: - pass + safe_Beep(i, 75) class MessageBeepTest(unittest.TestCase): @@ -70,22 +59,22 @@ def test_default(self): self.assertRaises(TypeError, winsound.MessageBeep, "bad") self.assertRaises(TypeError, winsound.MessageBeep, 42, 42) - winsound.MessageBeep() + safe_MessageBeep() def test_ok(self): - winsound.MessageBeep(winsound.MB_OK) + safe_MessageBeep(winsound.MB_OK) def test_asterisk(self): - winsound.MessageBeep(winsound.MB_ICONASTERISK) + safe_MessageBeep(winsound.MB_ICONASTERISK) def test_exclamation(self): - winsound.MessageBeep(winsound.MB_ICONEXCLAMATION) + safe_MessageBeep(winsound.MB_ICONEXCLAMATION) def test_hand(self): - winsound.MessageBeep(winsound.MB_ICONHAND) + safe_MessageBeep(winsound.MB_ICONHAND) def test_question(self): - winsound.MessageBeep(winsound.MB_ICONQUESTION) + safe_MessageBeep(winsound.MB_ICONQUESTION) class PlaySoundTest(unittest.TestCase): @@ -99,151 +88,34 @@ "none", winsound.SND_ASYNC | winsound.SND_MEMORY ) - @unittest.skipUnless(has_sound("SystemAsterisk"), - "No default SystemAsterisk") - def test_alias_asterisk(self): - if _have_soundcard(): - winsound.PlaySound('SystemAsterisk', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemAsterisk', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemExclamation"), - "No default SystemExclamation") - def test_alias_exclamation(self): - if _have_soundcard(): - winsound.PlaySound('SystemExclamation', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemExclamation', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemExit"), "No default SystemExit") - def test_alias_exit(self): - if _have_soundcard(): - winsound.PlaySound('SystemExit', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemExit', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemHand"), "No default SystemHand") - def test_alias_hand(self): - if _have_soundcard(): - winsound.PlaySound('SystemHand', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemHand', winsound.SND_ALIAS - ) - - @unittest.skipUnless(has_sound("SystemQuestion"), - "No default SystemQuestion") - def test_alias_question(self): - if _have_soundcard(): - winsound.PlaySound('SystemQuestion', winsound.SND_ALIAS) - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - 'SystemQuestion', winsound.SND_ALIAS - ) + def test_aliases(self): + aliases = [ + "SystemAsterisk", + "SystemExclamation", + "SystemExit", + "SystemHand", + "SystemQuestion", + ] + for alias in aliases: + with self.subTest(alias=alias): + safe_PlaySound(alias, winsound.SND_ALIAS) def test_alias_fallback(self): - # In the absence of the ability to tell if a sound was actually - # played, this test has two acceptable outcomes: success (no error, - # sound was theoretically played; although as issue #19987 shows - # a box without a soundcard can "succeed") or RuntimeError. Any - # other error is a failure. - try: - winsound.PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) - except RuntimeError: - pass + safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) def test_alias_nofallback(self): - if _have_soundcard(): - # Note that this is not the same as asserting RuntimeError - # will get raised: you cannot convert this to - # self.assertRaises(...) form. The attempt may or may not - # raise RuntimeError, but it shouldn't raise anything other - # than RuntimeError, and that's all we're trying to test - # here. The MS docs aren't clear about whether the SDK - # PlaySound() with SND_ALIAS and SND_NODEFAULT will return - # True or False when the alias is unknown. On Tim's WinXP - # box today, it returns True (no exception is raised). What - # we'd really like to test is that no sound is played, but - # that requires first wiring an eardrum class into unittest - # . - try: - winsound.PlaySound( - '!"$%&/(#+*', - winsound.SND_ALIAS | winsound.SND_NODEFAULT - ) - except RuntimeError: - pass - else: - self.assertRaises( - RuntimeError, - winsound.PlaySound, - '!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT - ) + safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT) def test_stopasync(self): - if _have_soundcard(): - winsound.PlaySound( - 'SystemQuestion', - winsound.SND_ALIAS | winsound.SND_ASYNC | winsound.SND_LOOP - ) - time.sleep(0.5) - try: - winsound.PlaySound( - 'SystemQuestion', - winsound.SND_ALIAS | winsound.SND_NOSTOP - ) - except RuntimeError: - pass - else: # the first sound might already be finished - pass - winsound.PlaySound(None, winsound.SND_PURGE) - else: - # Issue 8367: PlaySound(None, winsound.SND_PURGE) - # does not raise on systems without a sound card. - pass - - -def _get_cscript_path(): - """Return the full path to cscript.exe or None.""" - for dir in os.environ.get("PATH", "").split(os.pathsep): - cscript_path = os.path.join(dir, "cscript.exe") - if os.path.exists(cscript_path): - return cscript_path - -__have_soundcard_cache = None -def _have_soundcard(): - """Return True iff this computer has a soundcard.""" - global __have_soundcard_cache - if __have_soundcard_cache is None: - cscript_path = _get_cscript_path() - if cscript_path is None: - # Could not find cscript.exe to run our VBScript helper. Default - # to True: most computers these days *do* have a soundcard. - return True - - check_script = os.path.join(os.path.dirname(__file__), - "check_soundcard.vbs") - p = subprocess.Popen([cscript_path, check_script], - stdout=subprocess.PIPE) - __have_soundcard_cache = not p.wait() - p.stdout.close() - return __have_soundcard_cache + safe_PlaySound( + 'SystemQuestion', + winsound.SND_ALIAS | winsound.SND_ASYNC | winsound.SND_LOOP + ) + time.sleep(0.5) + safe_PlaySound('SystemQuestion', winsound.SND_ALIAS | winsound.SND_NOSTOP) + # Issue 8367: PlaySound(None, winsound.SND_PURGE) + # does not raise on systems without a sound card. + winsound.PlaySound(None, winsound.SND_PURGE) if __name__ == "__main__": -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:32:10 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 21:32:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2311620=3A_Fix_sup?= =?utf-8?q?port_for_SND=5FMEMORY_in_winsound=2EPlaySound=2E?= Message-ID: <20160905213209.9549.98216.1A28144A@psf.io> https://hg.python.org/cpython/rev/8fa615a2a896 changeset: 103066:8fa615a2a896 user: Zachary Ware date: Mon Sep 05 16:31:21 2016 -0500 summary: Closes #11620: Fix support for SND_MEMORY in winsound.PlaySound. Based on a patch by Tim Lesher. files: Doc/library/winsound.rst | 5 +- Lib/test/test_winsound.py | 16 +++++++++ Misc/NEWS | 3 + PC/clinic/winsound.c.h | 8 ++-- PC/winsound.c | 47 +++++++++++++++++++++----- 5 files changed, 63 insertions(+), 16 deletions(-) diff --git a/Doc/library/winsound.rst b/Doc/library/winsound.rst --- a/Doc/library/winsound.rst +++ b/Doc/library/winsound.rst @@ -25,7 +25,8 @@ .. function:: PlaySound(sound, flags) Call the underlying :c:func:`PlaySound` function from the Platform API. The - *sound* parameter may be a filename, audio data as a string, or ``None``. Its + *sound* parameter may be a filename, a system sound alias, audio data as a + :term:`bytes-like object`, or ``None``. Its interpretation depends on the value of *flags*, which can be a bitwise ORed combination of the constants described below. If the *sound* parameter is ``None``, any currently playing waveform sound is stopped. If the system @@ -92,7 +93,7 @@ .. data:: SND_MEMORY The *sound* parameter to :func:`PlaySound` is a memory image of a WAV file, as a - string. + :term:`bytes-like object`. .. note:: diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -87,6 +87,22 @@ winsound.PlaySound, "none", winsound.SND_ASYNC | winsound.SND_MEMORY ) + self.assertRaises(TypeError, winsound.PlaySound, b"bad", 0) + self.assertRaises(TypeError, winsound.PlaySound, "bad", + winsound.SND_MEMORY) + self.assertRaises(TypeError, winsound.PlaySound, 1, 0) + + def test_snd_memory(self): + with open(support.findfile('pluck-pcm8.wav', + subdir='audiodata'), 'rb') as f: + audio_data = f.read() + safe_PlaySound(audio_data, winsound.SND_MEMORY) + audio_data = bytearray(audio_data) + safe_PlaySound(audio_data, winsound.SND_MEMORY) + + def test_snd_filename(self): + fn = support.findfile('pluck-pcm8.wav', subdir='audiodata') + safe_PlaySound(fn, winsound.SND_FILENAME | winsound.SND_NODEFAULT) def test_aliases(self): aliases = [ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -75,6 +75,9 @@ Library ------- +- Issue #11620: Fix support for SND_MEMORY in winsound.PlaySound. Based on a + patch by Tim Lesher. + - Issue #11734: Add support for IEEE 754 half-precision floats to the struct module. Based on a patch by Eli Stevens. diff --git a/PC/clinic/winsound.c.h b/PC/clinic/winsound.c.h --- a/PC/clinic/winsound.c.h +++ b/PC/clinic/winsound.c.h @@ -17,16 +17,16 @@ {"PlaySound", (PyCFunction)winsound_PlaySound, METH_VARARGS, winsound_PlaySound__doc__}, static PyObject * -winsound_PlaySound_impl(PyObject *module, Py_UNICODE *sound, int flags); +winsound_PlaySound_impl(PyObject *module, PyObject *sound, int flags); static PyObject * winsound_PlaySound(PyObject *module, PyObject *args) { PyObject *return_value = NULL; - Py_UNICODE *sound; + PyObject *sound; int flags; - if (!PyArg_ParseTuple(args, "Zi:PlaySound", + if (!PyArg_ParseTuple(args, "Oi:PlaySound", &sound, &flags)) { goto exit; } @@ -100,4 +100,4 @@ exit: return return_value; } -/*[clinic end generated code: output=1044b2adf3c67014 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=b999334e2e444ad2 input=a9049054013a1b77]*/ diff --git a/PC/winsound.c b/PC/winsound.c --- a/PC/winsound.c +++ b/PC/winsound.c @@ -64,7 +64,7 @@ /*[clinic input] winsound.PlaySound - sound: Py_UNICODE(accept={str, NoneType}) + sound: object The sound to play; a filename, data, or None. flags: int Flag values, ored together. See module documentation. @@ -74,22 +74,49 @@ [clinic start generated code]*/ static PyObject * -winsound_PlaySound_impl(PyObject *module, Py_UNICODE *sound, int flags) -/*[clinic end generated code: output=ec24b3a2b4368378 input=3411b1b7c1f36d93]*/ +winsound_PlaySound_impl(PyObject *module, PyObject *sound, int flags) +/*[clinic end generated code: output=49a0fd16a372ebeb input=7bdf637f10201d37]*/ { int ok; + wchar_t *wsound; + Py_buffer view = {NULL, NULL}; - if (flags & SND_ASYNC && flags & SND_MEMORY) { - /* Sidestep reference counting headache; unfortunately this also - prevent SND_LOOP from memory. */ - PyErr_SetString(PyExc_RuntimeError, - "Cannot play asynchronously from memory"); - return NULL; + if (sound == Py_None) { + wsound = NULL; + } else if (flags & SND_MEMORY) { + if (flags & SND_ASYNC) { + /* Sidestep reference counting headache; unfortunately this also + prevent SND_LOOP from memory. */ + PyErr_SetString(PyExc_RuntimeError, + "Cannot play asynchronously from memory"); + return NULL; + } + if (PyObject_GetBuffer(sound, &view, PyBUF_SIMPLE) < 0) { + return NULL; + } + wsound = (wchar_t *)view.buf; + } else { + if (!PyUnicode_Check(sound)) { + PyErr_Format(PyExc_TypeError, + "'sound' must be str or None, not '%s'", + Py_TYPE(sound)->tp_name); + return NULL; + } + wsound = PyUnicode_AsWideCharString(sound, NULL); + if (wsound == NULL) { + return NULL; + } } + Py_BEGIN_ALLOW_THREADS - ok = PlaySoundW(sound, NULL, flags); + ok = PlaySoundW(wsound, NULL, flags); Py_END_ALLOW_THREADS + if (view.obj) { + PyBuffer_Release(&view); + } else if (sound != Py_None) { + PyMem_Free(wsound); + } if (!ok) { PyErr_SetString(PyExc_RuntimeError, "Failed to play sound"); return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:38:13 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 21:38:13 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2NDcw?= =?utf-8?q?=3A_Port_ssl_and_hashlib_module_to_OpenSSL_1=2E1=2E0=2E?= Message-ID: <20160905213812.6601.91435.2D2399AA@psf.io> https://hg.python.org/cpython/rev/5c75b315152b changeset: 103067:5c75b315152b branch: 3.5 parent: 103064:82467d0dbaea user: Christian Heimes date: Mon Sep 05 23:19:05 2016 +0200 summary: Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. files: Doc/library/ssl.rst | 106 +++++++++++++--- Lib/ssl.py | 18 +- Lib/test/test_ssl.py | 91 +++++++++----- Misc/NEWS | 2 + Modules/_hashopenssl.c | 165 ++++++++++++++++--------- Modules/_ssl.c | 181 +++++++++++++++++++++------- 6 files changed, 396 insertions(+), 167 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -178,7 +178,7 @@ use. Typically, the server chooses a particular protocol version, and the client must adapt to the server's choice. Most of the versions are not interoperable with the other versions. If not specified, the default is - :data:`PROTOCOL_SSLv23`; it provides the most compatibility with other + :data:`PROTOCOL_TLS`; it provides the most compatibility with other versions. Here's a table showing which versions in a client (down the side) can connect @@ -187,11 +187,11 @@ .. table:: ======================== ========= ========= ========== ========= =========== =========== - *client* / **server** **SSLv2** **SSLv3** **SSLv23** **TLSv1** **TLSv1.1** **TLSv1.2** + *client* / **server** **SSLv2** **SSLv3** **TLS** **TLSv1** **TLSv1.1** **TLSv1.2** ------------------------ --------- --------- ---------- --------- ----------- ----------- *SSLv2* yes no yes no no no *SSLv3* no yes yes no no no - *SSLv23* no yes yes yes yes yes + *TLS* (*SSLv23*) no yes yes yes yes yes *TLSv1* no no yes yes no no *TLSv1.1* no no yes no yes no *TLSv1.2* no no yes no no yes @@ -244,7 +244,7 @@ :const:`None`, this function can choose to trust the system's default CA certificates instead. - The settings are: :data:`PROTOCOL_SSLv23`, :data:`OP_NO_SSLv2`, and + The settings are: :data:`PROTOCOL_TLS`, :data:`OP_NO_SSLv2`, and :data:`OP_NO_SSLv3` with high encryption cipher suites without RC4 and without unauthenticated cipher suites. Passing :data:`~Purpose.SERVER_AUTH` as *purpose* sets :data:`~SSLContext.verify_mode` to :data:`CERT_REQUIRED` @@ -316,6 +316,11 @@ .. versionadded:: 3.3 + .. deprecated:: 3.5.3 + + OpenSSL has deprecated :func:`ssl.RAND_pseudo_bytes`, use + :func:`ssl.RAND_bytes` instead. + .. function:: RAND_status() Return ``True`` if the SSL pseudo-random number generator has been seeded @@ -334,7 +339,7 @@ See http://egd.sourceforge.net/ or http://prngd.sourceforge.net/ for sources of entropy-gathering daemons. - Availability: not available with LibreSSL. + Availability: not available with LibreSSL and OpenSSL > 1.1.0 .. function:: RAND_add(bytes, entropy) @@ -409,7 +414,7 @@ previously. Return an integer (no fractions of a second in the input format) -.. function:: get_server_certificate(addr, ssl_version=PROTOCOL_SSLv23, ca_certs=None) +.. function:: get_server_certificate(addr, ssl_version=PROTOCOL_TLS, ca_certs=None) Given the address ``addr`` of an SSL-protected server, as a (*hostname*, *port-number*) pair, fetches the server's certificate, and returns it as a @@ -425,7 +430,7 @@ .. versionchanged:: 3.5 The default *ssl_version* is changed from :data:`PROTOCOL_SSLv3` to - :data:`PROTOCOL_SSLv23` for maximum compatibility with modern servers. + :data:`PROTOCOL_TLS` for maximum compatibility with modern servers. .. function:: DER_cert_to_PEM_cert(DER_cert_bytes) @@ -451,6 +456,9 @@ * :attr:`openssl_capath_env` - OpenSSL's environment key that points to a capath, * :attr:`openssl_capath` - hard coded path to a capath directory + Availability: LibreSSL ignores the environment vars + :attr:`openssl_cafile_env` and :attr:`openssl_capath_env` + .. versionadded:: 3.4 .. function:: enum_certificates(store_name) @@ -568,11 +576,21 @@ .. versionadded:: 3.4.4 -.. data:: PROTOCOL_SSLv23 +.. data:: PROTOCOL_TLS Selects the highest protocol version that both the client and server support. Despite the name, this option can select "TLS" protocols as well as "SSL". + .. versionadded:: 3.5.3 + +.. data:: PROTOCOL_SSLv23 + + Alias for data:`PROTOCOL_TLS`. + + .. deprecated:: 3.5.3 + + Use data:`PROTOCOL_TLS` instead. + .. data:: PROTOCOL_SSLv2 Selects SSL version 2 as the channel encryption protocol. @@ -584,6 +602,10 @@ SSL version 2 is insecure. Its use is highly discouraged. + .. deprecated:: 3.5.3 + + OpenSSL has removed support for SSLv2. + .. data:: PROTOCOL_SSLv3 Selects SSL version 3 as the channel encryption protocol. @@ -595,10 +617,20 @@ SSL version 3 is insecure. Its use is highly discouraged. + .. deprecated:: 3.5.3 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol data:`PROTOCOL_TLS` with flags like data:`OP_NO_SSLv3` instead. + .. data:: PROTOCOL_TLSv1 Selects TLS version 1.0 as the channel encryption protocol. + .. deprecated:: 3.5.3 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol data:`PROTOCOL_TLS` with flags like data:`OP_NO_SSLv3` instead. + .. data:: PROTOCOL_TLSv1_1 Selects TLS version 1.1 as the channel encryption protocol. @@ -606,6 +638,11 @@ .. versionadded:: 3.4 + .. deprecated:: 3.5.3 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol data:`PROTOCOL_TLS` with flags like data:`OP_NO_SSLv3` instead. + .. data:: PROTOCOL_TLSv1_2 Selects TLS version 1.2 as the channel encryption protocol. This is the @@ -614,6 +651,11 @@ .. versionadded:: 3.4 + .. deprecated:: 3.5.3 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol data:`PROTOCOL_TLS` with flags like data:`OP_NO_SSLv3` instead. + .. data:: OP_ALL Enables workarounds for various bugs present in other SSL implementations. @@ -625,23 +667,32 @@ .. data:: OP_NO_SSLv2 Prevents an SSLv2 connection. This option is only applicable in - conjunction with :const:`PROTOCOL_SSLv23`. It prevents the peers from + conjunction with :const:`PROTOCOL_TLS`. It prevents the peers from choosing SSLv2 as the protocol version. .. versionadded:: 3.2 + .. deprecated:: 3.5.3 + + SSLv2 is deprecated + + .. data:: OP_NO_SSLv3 Prevents an SSLv3 connection. This option is only applicable in - conjunction with :const:`PROTOCOL_SSLv23`. It prevents the peers from + conjunction with :const:`PROTOCOL_TLS`. It prevents the peers from choosing SSLv3 as the protocol version. .. versionadded:: 3.2 + .. deprecated:: 3.5.3 + + SSLv3 is deprecated + .. data:: OP_NO_TLSv1 Prevents a TLSv1 connection. This option is only applicable in - conjunction with :const:`PROTOCOL_SSLv23`. It prevents the peers from + conjunction with :const:`PROTOCOL_TLS`. It prevents the peers from choosing TLSv1 as the protocol version. .. versionadded:: 3.2 @@ -649,7 +700,7 @@ .. data:: OP_NO_TLSv1_1 Prevents a TLSv1.1 connection. This option is only applicable in conjunction - with :const:`PROTOCOL_SSLv23`. It prevents the peers from choosing TLSv1.1 as + with :const:`PROTOCOL_TLS`. It prevents the peers from choosing TLSv1.1 as the protocol version. Available only with openssl version 1.0.1+. .. versionadded:: 3.4 @@ -657,7 +708,7 @@ .. data:: OP_NO_TLSv1_2 Prevents a TLSv1.2 connection. This option is only applicable in conjunction - with :const:`PROTOCOL_SSLv23`. It prevents the peers from choosing TLSv1.2 as + with :const:`PROTOCOL_TLS`. It prevents the peers from choosing TLSv1.2 as the protocol version. Available only with openssl version 1.0.1+. .. versionadded:: 3.4 @@ -1081,17 +1132,21 @@ It also manages a cache of SSL sessions for server-side sockets, in order to speed up repeated connections from the same clients. -.. class:: SSLContext(protocol) - - Create a new SSL context. You must pass *protocol* which must be one +.. class:: SSLContext(protocol=PROTOCOL_TLS) + + Create a new SSL context. You may pass *protocol* which must be one of the ``PROTOCOL_*`` constants defined in this module. - :data:`PROTOCOL_SSLv23` is currently recommended for maximum - interoperability. + :data:`PROTOCOL_TLS` is currently recommended for maximum + interoperability and default value. .. seealso:: :func:`create_default_context` lets the :mod:`ssl` module choose security settings for a given purpose. + .. versionchanged:: 3.5.3 + + :data:`PROTOCOL_TLS` is the default value. + :class:`SSLContext` objects have the following methods and attributes: @@ -1232,6 +1287,9 @@ This method will raise :exc:`NotImplementedError` if :data:`HAS_ALPN` is False. + OpenSSL 1.1.0+ will abort the handshake and raise :exc:`SSLError` when + both sides support ALPN but cannot agree on a protocol. + .. versionadded:: 3.5 .. method:: SSLContext.set_npn_protocols(protocols) @@ -1598,7 +1656,7 @@ a context from scratch (but beware that you might not get the settings right):: - >>> context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) + >>> context = ssl.SSLContext(ssl.PROTOCOL_TLS) >>> context.verify_mode = ssl.CERT_REQUIRED >>> context.check_hostname = True >>> context.load_verify_locations("/etc/ssl/certs/ca-bundle.crt") @@ -1999,15 +2057,17 @@ SSL versions 2 and 3 are considered insecure and are therefore dangerous to use. If you want maximum compatibility between clients and servers, it is -recommended to use :const:`PROTOCOL_SSLv23` as the protocol version and then +recommended to use :const:`PROTOCOL_TLS` as the protocol version and then disable SSLv2 and SSLv3 explicitly using the :data:`SSLContext.options` attribute:: - context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) + context = ssl.SSLContext(ssl.PROTOCOL_TLS) context.options |= ssl.OP_NO_SSLv2 context.options |= ssl.OP_NO_SSLv3 - -The SSL context created above will only allow TLSv1 and later (if + context.options |= ssl.OP_NO_TLSv1 + context.options |= ssl.OP_NO_TLSv1_1 + +The SSL context created above will only allow TLSv1.2 and later (if supported by your system) connections. Cipher selection diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -51,6 +51,7 @@ PROTOCOL_SSLv2 PROTOCOL_SSLv3 PROTOCOL_SSLv23 +PROTOCOL_TLS PROTOCOL_TLSv1 PROTOCOL_TLSv1_1 PROTOCOL_TLSv1_2 @@ -128,9 +129,10 @@ _IntEnum._convert( '_SSLMethod', __name__, - lambda name: name.startswith('PROTOCOL_'), + lambda name: name.startswith('PROTOCOL_') and name != 'PROTOCOL_SSLv23', source=_ssl) +PROTOCOL_SSLv23 = _SSLMethod.PROTOCOL_SSLv23 = _SSLMethod.PROTOCOL_TLS _PROTOCOL_NAMES = {value: name for name, value in _SSLMethod.__members__.items()} try: @@ -357,13 +359,13 @@ __slots__ = ('protocol', '__weakref__') _windows_cert_stores = ("CA", "ROOT") - def __new__(cls, protocol, *args, **kwargs): + def __new__(cls, protocol=PROTOCOL_TLS, *args, **kwargs): self = _SSLContext.__new__(cls, protocol) if protocol != _SSLv2_IF_EXISTS: self.set_ciphers(_DEFAULT_CIPHERS) return self - def __init__(self, protocol): + def __init__(self, protocol=PROTOCOL_TLS): self.protocol = protocol def wrap_socket(self, sock, server_side=False, @@ -438,7 +440,7 @@ if not isinstance(purpose, _ASN1Object): raise TypeError(purpose) - context = SSLContext(PROTOCOL_SSLv23) + context = SSLContext(PROTOCOL_TLS) # SSLv2 considered harmful. context.options |= OP_NO_SSLv2 @@ -475,7 +477,7 @@ context.load_default_certs(purpose) return context -def _create_unverified_context(protocol=PROTOCOL_SSLv23, *, cert_reqs=None, +def _create_unverified_context(protocol=PROTOCOL_TLS, *, cert_reqs=None, check_hostname=False, purpose=Purpose.SERVER_AUTH, certfile=None, keyfile=None, cafile=None, capath=None, cadata=None): @@ -666,7 +668,7 @@ def __init__(self, sock=None, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, - ssl_version=PROTOCOL_SSLv23, ca_certs=None, + ssl_version=PROTOCOL_TLS, ca_certs=None, do_handshake_on_connect=True, family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None, suppress_ragged_eofs=True, npn_protocols=None, ciphers=None, @@ -1056,7 +1058,7 @@ def wrap_socket(sock, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, - ssl_version=PROTOCOL_SSLv23, ca_certs=None, + ssl_version=PROTOCOL_TLS, ca_certs=None, do_handshake_on_connect=True, suppress_ragged_eofs=True, ciphers=None): @@ -1125,7 +1127,7 @@ d = pem_cert_string.strip()[len(PEM_HEADER):-len(PEM_FOOTER)] return base64.decodebytes(d.encode('ASCII', 'strict')) -def get_server_certificate(addr, ssl_version=PROTOCOL_SSLv23, ca_certs=None): +def get_server_certificate(addr, ssl_version=PROTOCOL_TLS, ca_certs=None): """Retrieve the certificate from the server at the specified address, and return it as a PEM-encoded string. If 'ca_certs' is specified, validate the server cert against it. diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -23,6 +23,9 @@ PROTOCOLS = sorted(ssl._PROTOCOL_NAMES) HOST = support.HOST +IS_LIBRESSL = ssl.OPENSSL_VERSION.startswith('LibreSSL') +IS_OPENSSL_1_1 = not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0) + def data_file(*name): return os.path.join(os.path.dirname(__file__), *name) @@ -143,8 +146,8 @@ def test_str_for_enums(self): # Make sure that the PROTOCOL_* constants have enum-like string # reprs. - proto = ssl.PROTOCOL_SSLv23 - self.assertEqual(str(proto), '_SSLMethod.PROTOCOL_SSLv23') + proto = ssl.PROTOCOL_TLS + self.assertEqual(str(proto), '_SSLMethod.PROTOCOL_TLS') ctx = ssl.SSLContext(proto) self.assertIs(ctx.protocol, proto) @@ -312,8 +315,8 @@ self.assertGreaterEqual(status, 0) self.assertLessEqual(status, 15) # Version string as returned by {Open,Libre}SSL, the format might change - if "LibreSSL" in s: - self.assertTrue(s.startswith("LibreSSL {:d}.{:d}".format(major, minor)), + if IS_LIBRESSL: + self.assertTrue(s.startswith("LibreSSL {:d}".format(major)), (s, t, hex(n))) else: self.assertTrue(s.startswith("OpenSSL {:d}.{:d}.{:d}".format(major, minor, fix)), @@ -790,7 +793,8 @@ def test_constructor(self): for protocol in PROTOCOLS: ssl.SSLContext(protocol) - self.assertRaises(TypeError, ssl.SSLContext) + ctx = ssl.SSLContext() + self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLS) self.assertRaises(ValueError, ssl.SSLContext, -1) self.assertRaises(ValueError, ssl.SSLContext, 42) @@ -811,15 +815,15 @@ def test_options(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) # OP_ALL | OP_NO_SSLv2 | OP_NO_SSLv3 is the default value - self.assertEqual(ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3, - ctx.options) + default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3) + if not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0): + default |= ssl.OP_NO_COMPRESSION + self.assertEqual(default, ctx.options) ctx.options |= ssl.OP_NO_TLSv1 - self.assertEqual(ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1, - ctx.options) + self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options) if can_clear_options(): - ctx.options = (ctx.options & ~ssl.OP_NO_SSLv2) | ssl.OP_NO_TLSv1 - self.assertEqual(ssl.OP_ALL | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv3, - ctx.options) + ctx.options = (ctx.options & ~ssl.OP_NO_TLSv1) + self.assertEqual(default, ctx.options) ctx.options = 0 # Ubuntu has OP_NO_SSLv3 forced on by default self.assertEqual(0, ctx.options & ~ssl.OP_NO_SSLv3) @@ -1155,6 +1159,7 @@ self.assertRaises(TypeError, ctx.load_default_certs, 'SERVER_AUTH') @unittest.skipIf(sys.platform == "win32", "not-Windows specific") + @unittest.skipIf(IS_LIBRESSL, "LibreSSL doesn't support env vars") def test_load_default_certs_env(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) with support.EnvironmentVarGuard() as env: @@ -1750,13 +1755,13 @@ sslobj = ctx.wrap_bio(incoming, outgoing, False, REMOTE_HOST) self.assertIs(sslobj._sslobj.owner, sslobj) self.assertIsNone(sslobj.cipher()) - self.assertIsNone(sslobj.shared_ciphers()) + self.assertIsNotNone(sslobj.shared_ciphers()) self.assertRaises(ValueError, sslobj.getpeercert) if 'tls-unique' in ssl.CHANNEL_BINDING_TYPES: self.assertIsNone(sslobj.get_channel_binding('tls-unique')) self.ssl_io_loop(sock, incoming, outgoing, sslobj.do_handshake) self.assertTrue(sslobj.cipher()) - self.assertIsNone(sslobj.shared_ciphers()) + self.assertIsNotNone(sslobj.shared_ciphers()) self.assertTrue(sslobj.getpeercert()) if 'tls-unique' in ssl.CHANNEL_BINDING_TYPES: self.assertTrue(sslobj.get_channel_binding('tls-unique')) @@ -2993,7 +2998,7 @@ with context.wrap_socket(socket.socket()) as s: self.assertIs(s.version(), None) s.connect((HOST, server.port)) - self.assertEqual(s.version(), "TLSv1") + self.assertEqual(s.version(), 'TLSv1') self.assertIs(s.version(), None) @unittest.skipUnless(ssl.HAS_ECDH, "test requires ECDH-enabled OpenSSL") @@ -3135,24 +3140,36 @@ (['http/3.0', 'http/4.0'], None) ] for client_protocols, expected in protocol_tests: - server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) server_context.load_cert_chain(CERTFILE) server_context.set_alpn_protocols(server_protocols) - client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) client_context.load_cert_chain(CERTFILE) client_context.set_alpn_protocols(client_protocols) - stats = server_params_test(client_context, server_context, - chatty=True, connectionchatty=True) - - msg = "failed trying %s (s) and %s (c).\n" \ - "was expecting %s, but got %%s from the %%s" \ - % (str(server_protocols), str(client_protocols), - str(expected)) - client_result = stats['client_alpn_protocol'] - self.assertEqual(client_result, expected, msg % (client_result, "client")) - server_result = stats['server_alpn_protocols'][-1] \ - if len(stats['server_alpn_protocols']) else 'nothing' - self.assertEqual(server_result, expected, msg % (server_result, "server")) + + try: + stats = server_params_test(client_context, + server_context, + chatty=True, + connectionchatty=True) + except ssl.SSLError as e: + stats = e + + if expected is None and IS_OPENSSL_1_1: + # OpenSSL 1.1.0 raises handshake error + self.assertIsInstance(stats, ssl.SSLError) + else: + msg = "failed trying %s (s) and %s (c).\n" \ + "was expecting %s, but got %%s from the %%s" \ + % (str(server_protocols), str(client_protocols), + str(expected)) + client_result = stats['client_alpn_protocol'] + self.assertEqual(client_result, expected, + msg % (client_result, "client")) + server_result = stats['server_alpn_protocols'][-1] \ + if len(stats['server_alpn_protocols']) else 'nothing' + self.assertEqual(server_result, expected, + msg % (server_result, "server")) def test_selected_npn_protocol(self): # selected_npn_protocol() is None unless NPN is used @@ -3300,13 +3317,23 @@ client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) client_context.verify_mode = ssl.CERT_REQUIRED client_context.load_verify_locations(SIGNING_CA) - client_context.set_ciphers("RC4") - server_context.set_ciphers("AES:RC4") + if ssl.OPENSSL_VERSION_INFO >= (1, 0, 2): + client_context.set_ciphers("AES128:AES256") + server_context.set_ciphers("AES256") + alg1 = "AES256" + alg2 = "AES-256" + else: + client_context.set_ciphers("AES:3DES") + server_context.set_ciphers("3DES") + alg1 = "3DES" + alg2 = "DES-CBC3" + stats = server_params_test(client_context, server_context) ciphers = stats['server_shared_ciphers'][0] self.assertGreater(len(ciphers), 0) for name, tls_version, bits in ciphers: - self.assertIn("RC4", name.split("-")) + if not alg1 in name.split("-") and alg2 not in name: + self.fail(name) def test_read_write_after_close_raises_valuerror(self): context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,8 @@ Library ------- +- Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. + - Issue #12885: Fix error when distutils encounters symlink. - Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -21,7 +21,6 @@ /* EVP is the preferred interface to hashing in OpenSSL */ #include -#include /* We use the object interface to discover what hashes OpenSSL supports. */ #include #include "openssl/err.h" @@ -32,11 +31,22 @@ #define HASH_OBJ_CONSTRUCTOR 0 #endif +#if (OPENSSL_VERSION_NUMBER < 0x10100000L) || defined(LIBRESSL_VERSION_NUMBER) +/* OpenSSL < 1.1.0 */ +#define EVP_MD_CTX_new EVP_MD_CTX_create +#define EVP_MD_CTX_free EVP_MD_CTX_destroy +#define HAS_FAST_PKCS5_PBKDF2_HMAC 0 +#include +#else +/* OpenSSL >= 1.1.0 */ +#define HAS_FAST_PKCS5_PBKDF2_HMAC 1 +#endif + typedef struct { PyObject_HEAD PyObject *name; /* name of this hash algorithm */ - EVP_MD_CTX ctx; /* OpenSSL message digest context */ + EVP_MD_CTX *ctx; /* OpenSSL message digest context */ #ifdef WITH_THREAD PyThread_type_lock lock; /* OpenSSL context lock */ #endif @@ -48,7 +58,6 @@ #define DEFINE_CONSTS_FOR_NEW(Name) \ static PyObject *CONST_ ## Name ## _name_obj = NULL; \ - static EVP_MD_CTX CONST_new_ ## Name ## _ctx; \ static EVP_MD_CTX *CONST_new_ ## Name ## _ctx_p = NULL; DEFINE_CONSTS_FOR_NEW(md5) @@ -59,19 +68,57 @@ DEFINE_CONSTS_FOR_NEW(sha512) +/* LCOV_EXCL_START */ +static PyObject * +_setException(PyObject *exc) +{ + unsigned long errcode; + const char *lib, *func, *reason; + + errcode = ERR_peek_last_error(); + if (!errcode) { + PyErr_SetString(exc, "unknown reasons"); + return NULL; + } + ERR_clear_error(); + + lib = ERR_lib_error_string(errcode); + func = ERR_func_error_string(errcode); + reason = ERR_reason_error_string(errcode); + + if (lib && func) { + PyErr_Format(exc, "[%s: %s] %s", lib, func, reason); + } + else if (lib) { + PyErr_Format(exc, "[%s] %s", lib, reason); + } + else { + PyErr_SetString(exc, reason); + } + return NULL; +} +/* LCOV_EXCL_STOP */ + static EVPobject * newEVPobject(PyObject *name) { EVPobject *retval = (EVPobject *)PyObject_New(EVPobject, &EVPtype); + if (retval == NULL) { + return NULL; + } + + retval->ctx = EVP_MD_CTX_new(); + if (retval->ctx == NULL) { + PyErr_NoMemory(); + return NULL; + } /* save the name for .name to return */ - if (retval != NULL) { - Py_INCREF(name); - retval->name = name; + Py_INCREF(name); + retval->name = name; #ifdef WITH_THREAD - retval->lock = NULL; + retval->lock = NULL; #endif - } return retval; } @@ -86,7 +133,7 @@ process = MUNCH_SIZE; else process = Py_SAFE_DOWNCAST(len, Py_ssize_t, unsigned int); - EVP_DigestUpdate(&self->ctx, (const void*)cp, process); + EVP_DigestUpdate(self->ctx, (const void*)cp, process); len -= process; cp += process; } @@ -101,16 +148,19 @@ if (self->lock != NULL) PyThread_free_lock(self->lock); #endif - EVP_MD_CTX_cleanup(&self->ctx); + EVP_MD_CTX_free(self->ctx); Py_XDECREF(self->name); PyObject_Del(self); } -static void locked_EVP_MD_CTX_copy(EVP_MD_CTX *new_ctx_p, EVPobject *self) +static int +locked_EVP_MD_CTX_copy(EVP_MD_CTX *new_ctx_p, EVPobject *self) { + int result; ENTER_HASHLIB(self); - EVP_MD_CTX_copy(new_ctx_p, &self->ctx); + result = EVP_MD_CTX_copy(new_ctx_p, self->ctx); LEAVE_HASHLIB(self); + return result; } /* External methods for a hash object */ @@ -126,7 +176,9 @@ if ( (newobj = newEVPobject(self->name))==NULL) return NULL; - locked_EVP_MD_CTX_copy(&newobj->ctx, self); + if (!locked_EVP_MD_CTX_copy(newobj->ctx, self)) { + return _setException(PyExc_ValueError); + } return (PyObject *)newobj; } @@ -137,16 +189,24 @@ EVP_digest(EVPobject *self, PyObject *unused) { unsigned char digest[EVP_MAX_MD_SIZE]; - EVP_MD_CTX temp_ctx; + EVP_MD_CTX *temp_ctx; PyObject *retval; unsigned int digest_size; - locked_EVP_MD_CTX_copy(&temp_ctx, self); - digest_size = EVP_MD_CTX_size(&temp_ctx); - EVP_DigestFinal(&temp_ctx, digest, NULL); + temp_ctx = EVP_MD_CTX_new(); + if (temp_ctx == NULL) { + PyErr_NoMemory(); + return NULL; + } + + if (!locked_EVP_MD_CTX_copy(temp_ctx, self)) { + return _setException(PyExc_ValueError); + } + digest_size = EVP_MD_CTX_size(temp_ctx); + EVP_DigestFinal(temp_ctx, digest, NULL); retval = PyBytes_FromStringAndSize((const char *)digest, digest_size); - EVP_MD_CTX_cleanup(&temp_ctx); + EVP_MD_CTX_free(temp_ctx); return retval; } @@ -157,15 +217,23 @@ EVP_hexdigest(EVPobject *self, PyObject *unused) { unsigned char digest[EVP_MAX_MD_SIZE]; - EVP_MD_CTX temp_ctx; + EVP_MD_CTX *temp_ctx; unsigned int digest_size; + temp_ctx = EVP_MD_CTX_new(); + if (temp_ctx == NULL) { + PyErr_NoMemory(); + return NULL; + } + /* Get the raw (binary) digest value */ - locked_EVP_MD_CTX_copy(&temp_ctx, self); - digest_size = EVP_MD_CTX_size(&temp_ctx); - EVP_DigestFinal(&temp_ctx, digest, NULL); + if (!locked_EVP_MD_CTX_copy(temp_ctx, self)) { + return _setException(PyExc_ValueError); + } + digest_size = EVP_MD_CTX_size(temp_ctx); + EVP_DigestFinal(temp_ctx, digest, NULL); - EVP_MD_CTX_cleanup(&temp_ctx); + EVP_MD_CTX_free(temp_ctx); return _Py_strhex((const char *)digest, digest_size); } @@ -219,7 +287,7 @@ EVP_get_block_size(EVPobject *self, void *closure) { long block_size; - block_size = EVP_MD_CTX_block_size(&self->ctx); + block_size = EVP_MD_CTX_block_size(self->ctx); return PyLong_FromLong(block_size); } @@ -227,7 +295,7 @@ EVP_get_digest_size(EVPobject *self, void *closure) { long size; - size = EVP_MD_CTX_size(&self->ctx); + size = EVP_MD_CTX_size(self->ctx); return PyLong_FromLong(size); } @@ -288,7 +356,7 @@ PyBuffer_Release(&view); return -1; } - EVP_DigestInit(&self->ctx, digest); + EVP_DigestInit(self->ctx, digest); self->name = name_obj; Py_INCREF(self->name); @@ -385,9 +453,9 @@ return NULL; if (initial_ctx) { - EVP_MD_CTX_copy(&self->ctx, initial_ctx); + EVP_MD_CTX_copy(self->ctx, initial_ctx); } else { - EVP_DigestInit(&self->ctx, digest); + EVP_DigestInit(self->ctx, digest); } if (cp && len) { @@ -453,6 +521,7 @@ #define PY_PBKDF2_HMAC 1 +#if !HAS_FAST_PKCS5_PBKDF2_HMAC /* Improved implementation of PKCS5_PBKDF2_HMAC() * * PKCS5_PBKDF2_HMAC_fast() hashes the password exactly one time instead of @@ -534,37 +603,8 @@ HMAC_CTX_cleanup(&hctx_tpl); return 1; } +#endif -/* LCOV_EXCL_START */ -static PyObject * -_setException(PyObject *exc) -{ - unsigned long errcode; - const char *lib, *func, *reason; - - errcode = ERR_peek_last_error(); - if (!errcode) { - PyErr_SetString(exc, "unknown reasons"); - return NULL; - } - ERR_clear_error(); - - lib = ERR_lib_error_string(errcode); - func = ERR_func_error_string(errcode); - reason = ERR_reason_error_string(errcode); - - if (lib && func) { - PyErr_Format(exc, "[%s: %s] %s", lib, func, reason); - } - else if (lib) { - PyErr_Format(exc, "[%s] %s", lib, reason); - } - else { - PyErr_SetString(exc, reason); - } - return NULL; -} -/* LCOV_EXCL_STOP */ PyDoc_STRVAR(pbkdf2_hmac__doc__, "pbkdf2_hmac(hash_name, password, salt, iterations, dklen=None) -> key\n\ @@ -646,10 +686,17 @@ key = PyBytes_AS_STRING(key_obj); Py_BEGIN_ALLOW_THREADS +#if HAS_FAST_PKCS5_PBKDF2_HMAC + retval = PKCS5_PBKDF2_HMAC((char*)password.buf, (int)password.len, + (unsigned char *)salt.buf, (int)salt.len, + iterations, digest, dklen, + (unsigned char *)key); +#else retval = PKCS5_PBKDF2_HMAC_fast((char*)password.buf, (int)password.len, (unsigned char *)salt.buf, (int)salt.len, iterations, digest, dklen, (unsigned char *)key); +#endif Py_END_ALLOW_THREADS if (!retval) { @@ -768,7 +815,7 @@ if (CONST_ ## NAME ## _name_obj == NULL) { \ CONST_ ## NAME ## _name_obj = PyUnicode_FromString(#NAME); \ if (EVP_get_digestbyname(#NAME)) { \ - CONST_new_ ## NAME ## _ctx_p = &CONST_new_ ## NAME ## _ctx; \ + CONST_new_ ## NAME ## _ctx_p = EVP_MD_CTX_new(); \ EVP_DigestInit(CONST_new_ ## NAME ## _ctx_p, EVP_get_digestbyname(#NAME)); \ } \ } \ diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -55,6 +55,14 @@ #include #endif +/* Don't warn about deprecated functions */ +#ifdef __GNUC__ +#pragma GCC diagnostic ignored "-Wdeprecated-declarations" +#endif +#ifdef __clang__ +#pragma clang diagnostic ignored "-Wdeprecated-declarations" +#endif + /* Include OpenSSL header files */ #include "openssl/rsa.h" #include "openssl/crypto.h" @@ -91,6 +99,10 @@ /* Include generated data (error codes) */ #include "_ssl_data.h" +#if (OPENSSL_VERSION_NUMBER >= 0x10100000L) && !defined(LIBRESSL_VERSION_NUMBER) +# define OPENSSL_VERSION_1_1 1 +#endif + /* Openssl comes with TLSv1.1 and TLSv1.2 between 1.0.0h and 1.0.1 http://www.openssl.org/news/changelog.html */ @@ -117,6 +129,72 @@ #define INVALID_SOCKET (-1) #endif +#ifdef OPENSSL_VERSION_1_1 +/* OpenSSL 1.1.0+ */ +#ifndef OPENSSL_NO_SSL2 +#define OPENSSL_NO_SSL2 +#endif +#else /* OpenSSL < 1.1.0 */ +#if defined(WITH_THREAD) +#define HAVE_OPENSSL_CRYPTO_LOCK +#endif + +#define TLS_method SSLv23_method + +static int X509_NAME_ENTRY_set(const X509_NAME_ENTRY *ne) +{ + return ne->set; +} + +#ifndef OPENSSL_NO_COMP +static int COMP_get_type(const COMP_METHOD *meth) +{ + return meth->type; +} + +static const char *COMP_get_name(const COMP_METHOD *meth) +{ + return meth->name; +} +#endif + +static pem_password_cb *SSL_CTX_get_default_passwd_cb(SSL_CTX *ctx) +{ + return ctx->default_passwd_callback; +} + +static void *SSL_CTX_get_default_passwd_cb_userdata(SSL_CTX *ctx) +{ + return ctx->default_passwd_callback_userdata; +} + +static int X509_OBJECT_get_type(X509_OBJECT *x) +{ + return x->type; +} + +static X509 *X509_OBJECT_get0_X509(X509_OBJECT *x) +{ + return x->data.x509; +} + +static int BIO_up_ref(BIO *b) +{ + CRYPTO_add(&b->references, 1, CRYPTO_LOCK_BIO); + return 1; +} + +static STACK_OF(X509_OBJECT) *X509_STORE_get0_objects(X509_STORE *store) { + return store->objs; +} + +static X509_VERIFY_PARAM *X509_STORE_get0_param(X509_STORE *store) +{ + return store->param; +} +#endif /* OpenSSL < 1.1.0 or LibreSSL */ + + enum py_ssl_error { /* these mirror ssl.h */ PY_SSL_ERROR_NONE, @@ -147,7 +225,7 @@ enum py_ssl_version { PY_SSL_VERSION_SSL2, PY_SSL_VERSION_SSL3=1, - PY_SSL_VERSION_SSL23, + PY_SSL_VERSION_TLS, #if HAVE_TLSv1_2 PY_SSL_VERSION_TLS1, PY_SSL_VERSION_TLS1_1, @@ -527,8 +605,8 @@ /* BIOs are reference counted and SSL_set_bio borrows our reference. * To prevent a double free in memory_bio_dealloc() we need to take an * extra reference here. */ - CRYPTO_add(&inbio->bio->references, 1, CRYPTO_LOCK_BIO); - CRYPTO_add(&outbio->bio->references, 1, CRYPTO_LOCK_BIO); + BIO_up_ref(inbio->bio); + BIO_up_ref(outbio->bio); SSL_set_bio(self->ssl, inbio->bio, outbio->bio); } mode = SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER; @@ -738,7 +816,7 @@ /* check to see if we've gotten to a new RDN */ if (rdn_level >= 0) { - if (rdn_level != entry->set) { + if (rdn_level != X509_NAME_ENTRY_set(entry)) { /* yes, new RDN */ /* add old RDN to DN */ rdnt = PyList_AsTuple(rdn); @@ -755,7 +833,7 @@ goto fail0; } } - rdn_level = entry->set; + rdn_level = X509_NAME_ENTRY_set(entry); /* now add this attribute to the current RDN */ name = X509_NAME_ENTRY_get_object(entry); @@ -853,18 +931,18 @@ goto fail; } - p = ext->value->data; + p = X509_EXTENSION_get_data(ext)->data; if (method->it) names = (GENERAL_NAMES*) (ASN1_item_d2i(NULL, &p, - ext->value->length, + X509_EXTENSION_get_data(ext)->length, ASN1_ITEM_ptr(method->it))); else names = (GENERAL_NAMES*) (method->d2i(NULL, &p, - ext->value->length)); + X509_EXTENSION_get_data(ext)->length)); for(j = 0; j < sk_GENERAL_NAME_num(names); j++) { /* get a rendering of each name in the set of names */ @@ -1075,13 +1153,11 @@ int i, j; PyObject *lst, *res = NULL; -#if OPENSSL_VERSION_NUMBER < 0x10001000L - dps = X509_get_ext_d2i(certificate, NID_crl_distribution_points, NULL, NULL); -#else +#if OPENSSL_VERSION_NUMBER >= 0x10001000L /* Calls x509v3_cache_extensions and sets up crldp */ X509_check_ca(certificate); - dps = certificate->crldp; #endif + dps = X509_get_ext_d2i(certificate, NID_crl_distribution_points, NULL, NULL); if (dps == NULL) return Py_None; @@ -1451,14 +1527,13 @@ _ssl__SSLSocket_shared_ciphers_impl(PySSLSocket *self) /*[clinic end generated code: output=3d174ead2e42c4fd input=0bfe149da8fe6306]*/ { - SSL_SESSION *sess = SSL_get_session(self->ssl); STACK_OF(SSL_CIPHER) *ciphers; int i; PyObject *res; - if (!sess || !sess->ciphers) + ciphers = SSL_get_ciphers(self->ssl); + if (!ciphers) Py_RETURN_NONE; - ciphers = sess->ciphers; res = PyList_New(sk_SSL_CIPHER_num(ciphers)); if (!res) return NULL; @@ -1567,9 +1642,9 @@ if (self->ssl == NULL) Py_RETURN_NONE; comp_method = SSL_get_current_compression(self->ssl); - if (comp_method == NULL || comp_method->type == NID_undef) + if (comp_method == NULL || COMP_get_type(comp_method) == NID_undef) Py_RETURN_NONE; - short_name = OBJ_nid2sn(comp_method->type); + short_name = COMP_get_name(comp_method); if (short_name == NULL) Py_RETURN_NONE; return PyUnicode_DecodeFSDefault(short_name); @@ -2255,8 +2330,8 @@ else if (proto_version == PY_SSL_VERSION_SSL2) ctx = SSL_CTX_new(SSLv2_method()); #endif - else if (proto_version == PY_SSL_VERSION_SSL23) - ctx = SSL_CTX_new(SSLv23_method()); + else if (proto_version == PY_SSL_VERSION_TLS) + ctx = SSL_CTX_new(TLS_method()); else proto_version = -1; PySSL_END_ALLOW_THREADS @@ -2318,8 +2393,9 @@ #ifndef OPENSSL_NO_ECDH /* Allow automatic ECDH curve selection (on OpenSSL 1.0.2+), or use prime256v1 by default. This is Apache mod_ssl's initialization - policy, so we should be safe. */ -#if defined(SSL_CTX_set_ecdh_auto) + policy, so we should be safe. OpenSSL 1.1 has it enabled by default. + */ +#if defined(SSL_CTX_set_ecdh_auto) && !defined(OPENSSL_VERSION_1_1) SSL_CTX_set_ecdh_auto(self->ctx, 1); #else { @@ -2586,10 +2662,12 @@ get_verify_flags(PySSLContext *self, void *c) { X509_STORE *store; + X509_VERIFY_PARAM *param; unsigned long flags; store = SSL_CTX_get_cert_store(self->ctx); - flags = X509_VERIFY_PARAM_get_flags(store->param); + param = X509_STORE_get0_param(store); + flags = X509_VERIFY_PARAM_get_flags(param); return PyLong_FromUnsignedLong(flags); } @@ -2597,22 +2675,24 @@ set_verify_flags(PySSLContext *self, PyObject *arg, void *c) { X509_STORE *store; + X509_VERIFY_PARAM *param; unsigned long new_flags, flags, set, clear; if (!PyArg_Parse(arg, "k", &new_flags)) return -1; store = SSL_CTX_get_cert_store(self->ctx); - flags = X509_VERIFY_PARAM_get_flags(store->param); + param = X509_STORE_get0_param(store); + flags = X509_VERIFY_PARAM_get_flags(param); clear = flags & ~new_flags; set = ~flags & new_flags; if (clear) { - if (!X509_VERIFY_PARAM_clear_flags(store->param, clear)) { + if (!X509_VERIFY_PARAM_clear_flags(param, clear)) { _setSSLError(NULL, 0, __FILE__, __LINE__); return -1; } } if (set) { - if (!X509_VERIFY_PARAM_set_flags(store->param, set)) { + if (!X509_VERIFY_PARAM_set_flags(param, set)) { _setSSLError(NULL, 0, __FILE__, __LINE__); return -1; } @@ -2789,8 +2869,8 @@ /*[clinic end generated code: output=9480bc1c380e2095 input=7cf9ac673cbee6fc]*/ { PyObject *certfile_bytes = NULL, *keyfile_bytes = NULL; - pem_password_cb *orig_passwd_cb = self->ctx->default_passwd_callback; - void *orig_passwd_userdata = self->ctx->default_passwd_callback_userdata; + pem_password_cb *orig_passwd_cb = SSL_CTX_get_default_passwd_cb(self->ctx); + void *orig_passwd_userdata = SSL_CTX_get_default_passwd_cb_userdata(self->ctx); _PySSLPasswordInfo pw_info = { NULL, NULL, NULL, 0, 0 }; int r; @@ -2917,8 +2997,9 @@ cert = d2i_X509_bio(biobuf, NULL); } else { cert = PEM_read_bio_X509(biobuf, NULL, - self->ctx->default_passwd_callback, - self->ctx->default_passwd_callback_userdata); + SSL_CTX_get_default_passwd_cb(self->ctx), + SSL_CTX_get_default_passwd_cb_userdata(self->ctx) + ); } if (cert == NULL) { break; @@ -3444,25 +3525,24 @@ /*[clinic end generated code: output=5f356f4d9cca874d input=eb40dd0f6d0e40cf]*/ { X509_STORE *store; + STACK_OF(X509_OBJECT) *objs; X509_OBJECT *obj; - int x509 = 0, crl = 0, pkey = 0, ca = 0, i; + int x509 = 0, crl = 0, ca = 0, i; store = SSL_CTX_get_cert_store(self->ctx); - for (i = 0; i < sk_X509_OBJECT_num(store->objs); i++) { - obj = sk_X509_OBJECT_value(store->objs, i); - switch (obj->type) { + objs = X509_STORE_get0_objects(store); + for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { + obj = sk_X509_OBJECT_value(objs, i); + switch (X509_OBJECT_get_type(obj)) { case X509_LU_X509: x509++; - if (X509_check_ca(obj->data.x509)) { + if (X509_check_ca(X509_OBJECT_get0_X509(obj))) { ca++; } break; case X509_LU_CRL: crl++; break; - case X509_LU_PKEY: - pkey++; - break; default: /* Ignore X509_LU_FAIL, X509_LU_RETRY, X509_LU_PKEY. * As far as I can tell they are internal states and never @@ -3492,6 +3572,7 @@ /*[clinic end generated code: output=0d58f148f37e2938 input=6887b5a09b7f9076]*/ { X509_STORE *store; + STACK_OF(X509_OBJECT) *objs; PyObject *ci = NULL, *rlist = NULL; int i; @@ -3500,17 +3581,18 @@ } store = SSL_CTX_get_cert_store(self->ctx); - for (i = 0; i < sk_X509_OBJECT_num(store->objs); i++) { + objs = X509_STORE_get0_objects(store); + for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { X509_OBJECT *obj; X509 *cert; - obj = sk_X509_OBJECT_value(store->objs, i); - if (obj->type != X509_LU_X509) { + obj = sk_X509_OBJECT_value(objs, i); + if (X509_OBJECT_get_type(obj) != X509_LU_X509) { /* not a x509 cert */ continue; } /* CA for any purpose */ - cert = obj->data.x509; + cert = X509_OBJECT_get0_X509(obj); if (!X509_check_ca(cert)) { continue; } @@ -4374,10 +4456,12 @@ }; -#ifdef WITH_THREAD +#ifdef HAVE_OPENSSL_CRYPTO_LOCK /* an implementation of OpenSSL threading operations in terms - of the Python C thread library */ + * of the Python C thread library + * Only used up to 1.0.2. OpenSSL 1.1.0+ has its own locking code. + */ static PyThread_type_lock *_ssl_locks = NULL; @@ -4458,7 +4542,7 @@ return 1; } -#endif /* def HAVE_THREAD */ +#endif /* HAVE_OPENSSL_CRYPTO_LOCK for WITH_THREAD && OpenSSL < 1.1.0 */ PyDoc_STRVAR(module_doc, "Implementation module for SSL socket operations. See the socket module\n\ @@ -4527,11 +4611,16 @@ SSL_load_error_strings(); SSL_library_init(); #ifdef WITH_THREAD +#ifdef HAVE_OPENSSL_CRYPTO_LOCK /* note that this will start threading if not already started */ if (!_setup_ssl_threads()) { return NULL; } +#elif OPENSSL_VERSION_1_1 && defined(OPENSSL_THREADS) + /* OpenSSL 1.1.0 builtin thread support is enabled */ + _ssl_locks_count++; #endif +#endif /* WITH_THREAD */ OpenSSL_add_all_algorithms(); /* Add symbols to module dict */ @@ -4678,7 +4767,9 @@ PY_SSL_VERSION_SSL3); #endif PyModule_AddIntConstant(m, "PROTOCOL_SSLv23", - PY_SSL_VERSION_SSL23); + PY_SSL_VERSION_TLS); + PyModule_AddIntConstant(m, "PROTOCOL_TLS", + PY_SSL_VERSION_TLS); PyModule_AddIntConstant(m, "PROTOCOL_TLSv1", PY_SSL_VERSION_TLS1); #if HAVE_TLSv1_2 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:38:18 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 21:38:18 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2NDcw?= =?utf-8?q?=3A_Port_ssl_and_hashlib_module_to_OpenSSL_1=2E1=2E0=2E?= Message-ID: <20160905213818.6425.7623.EA905BC5@psf.io> https://hg.python.org/cpython/rev/14b611ddaabe changeset: 103069:14b611ddaabe branch: 2.7 parent: 103063:4c91651912d1 user: Christian Heimes date: Mon Sep 05 23:37:13 2016 +0200 summary: Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. files: Doc/library/ssl.rst | 51 ++++++++- Lib/ssl.py | 16 +- Lib/test/test_ssl.py | 68 ++++++---- Misc/NEWS | 2 + Modules/_hashopenssl.c | 170 ++++++++++++++++++---------- Modules/_ssl.c | 172 ++++++++++++++++++++++------ 6 files changed, 343 insertions(+), 136 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -322,6 +322,16 @@ Random generation ^^^^^^^^^^^^^^^^^ + .. deprecated:: + + 2.7.13 OpenSSL has deprecated :func:`ssl.RAND_pseudo_bytes`, use + :func:`ssl.RAND_bytes` instead. + + .. deprecated:: + + 2.7.13 OpenSSL has deprecated :func:`ssl.RAND_pseudo_bytes`, use + :func:`ssl.RAND_bytes` instead. + .. function:: RAND_status() Return ``True`` if the SSL pseudo-random number generator has been seeded @@ -340,7 +350,7 @@ See http://egd.sourceforge.net/ or http://prngd.sourceforge.net/ for sources of entropy-gathering daemons. - Availability: not available with LibreSSL. + Availability: not available with LibreSSL and OpenSSL > 1.1.0 .. function:: RAND_add(bytes, entropy) @@ -444,6 +454,9 @@ * :attr:`openssl_capath_env` - OpenSSL's environment key that points to a capath, * :attr:`openssl_capath` - hard coded path to a capath directory + Availability: LibreSSL ignores the environment vars + :attr:`openssl_cafile_env` and :attr:`openssl_capath_env` + .. versionadded:: 2.7.9 .. function:: enum_certificates(store_name) @@ -561,11 +574,19 @@ .. versionadded:: 2.7.10 -.. data:: PROTOCOL_SSLv23 +.. data:: PROTOCOL_TLS Selects the highest protocol version that both the client and server support. Despite the name, this option can select "TLS" protocols as well as "SSL". + .. versionadded:: 2.7.13 + +.. data:: PROTOCOL_SSLv23 + + Alias for ``PROTOCOL_TLS``. + + .. deprecated:: 2.7.13 Use ``PROTOCOL_TLS`` instead. + .. data:: PROTOCOL_SSLv2 Selects SSL version 2 as the channel encryption protocol. @@ -577,6 +598,8 @@ SSL version 2 is insecure. Its use is highly discouraged. + .. deprecated:: 2.7.13 OpenSSL has removed support for SSLv2. + .. data:: PROTOCOL_SSLv3 Selects SSL version 3 as the channel encryption protocol. @@ -588,10 +611,20 @@ SSL version 3 is insecure. Its use is highly discouraged. + .. deprecated:: 2.7.13 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol with flags like ``OP_NO_SSLv3`` instead. + .. data:: PROTOCOL_TLSv1 Selects TLS version 1.0 as the channel encryption protocol. + .. deprecated:: 2.7.13 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol with flags like ``OP_NO_SSLv3`` instead. + .. data:: PROTOCOL_TLSv1_1 Selects TLS version 1.1 as the channel encryption protocol. @@ -599,6 +632,11 @@ .. versionadded:: 2.7.9 + .. deprecated:: 2.7.13 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol with flags like ``OP_NO_SSLv3`` instead. + .. data:: PROTOCOL_TLSv1_2 Selects TLS version 1.2 as the channel encryption protocol. This is the @@ -607,6 +645,12 @@ .. versionadded:: 2.7.9 + .. deprecated:: 2.7.13 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol with flags like ``OP_NO_SSLv3`` instead. + + .. data:: OP_ALL Enables workarounds for various bugs present in other SSL implementations. @@ -1112,6 +1156,9 @@ This method will raise :exc:`NotImplementedError` if :data:`HAS_ALPN` is False. + OpenSSL 1.1.0+ will abort the handshake and raise :exc:`SSLError` when + both sides support ALPN but cannot agree on a protocol. + .. versionadded:: 2.7.10 .. method:: SSLContext.set_npn_protocols(protocols) diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -51,6 +51,7 @@ PROTOCOL_SSLv2 PROTOCOL_SSLv3 PROTOCOL_SSLv23 +PROTOCOL_TLS PROTOCOL_TLSv1 PROTOCOL_TLSv1_1 PROTOCOL_TLSv1_2 @@ -126,7 +127,10 @@ from _ssl import _OPENSSL_API_VERSION -_PROTOCOL_NAMES = {value: name for name, value in globals().items() if name.startswith('PROTOCOL_')} +_PROTOCOL_NAMES = {value: name for name, value in globals().items() + if name.startswith('PROTOCOL_') + and name != 'PROTOCOL_SSLv23'} +PROTOCOL_SSLv23 = PROTOCOL_TLS try: _SSLv2_IF_EXISTS = PROTOCOL_SSLv2 @@ -408,7 +412,7 @@ if not isinstance(purpose, _ASN1Object): raise TypeError(purpose) - context = SSLContext(PROTOCOL_SSLv23) + context = SSLContext(PROTOCOL_TLS) # SSLv2 considered harmful. context.options |= OP_NO_SSLv2 @@ -445,7 +449,7 @@ context.load_default_certs(purpose) return context -def _create_unverified_context(protocol=PROTOCOL_SSLv23, cert_reqs=None, +def _create_unverified_context(protocol=PROTOCOL_TLS, cert_reqs=None, check_hostname=False, purpose=Purpose.SERVER_AUTH, certfile=None, keyfile=None, cafile=None, capath=None, cadata=None): @@ -518,7 +522,7 @@ def __init__(self, sock=None, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, - ssl_version=PROTOCOL_SSLv23, ca_certs=None, + ssl_version=PROTOCOL_TLS, ca_certs=None, do_handshake_on_connect=True, family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None, suppress_ragged_eofs=True, npn_protocols=None, ciphers=None, @@ -920,7 +924,7 @@ def wrap_socket(sock, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, - ssl_version=PROTOCOL_SSLv23, ca_certs=None, + ssl_version=PROTOCOL_TLS, ca_certs=None, do_handshake_on_connect=True, suppress_ragged_eofs=True, ciphers=None): @@ -989,7 +993,7 @@ d = pem_cert_string.strip()[len(PEM_HEADER):-len(PEM_FOOTER)] return base64.decodestring(d.encode('ASCII', 'strict')) -def get_server_certificate(addr, ssl_version=PROTOCOL_SSLv23, ca_certs=None): +def get_server_certificate(addr, ssl_version=PROTOCOL_TLS, ca_certs=None): """Retrieve the certificate from the server at the specified address, and return it as a PEM-encoded string. If 'ca_certs' is specified, validate the server cert against it. diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -26,6 +26,9 @@ PROTOCOLS = sorted(ssl._PROTOCOL_NAMES) HOST = support.HOST +IS_LIBRESSL = ssl.OPENSSL_VERSION.startswith('LibreSSL') +IS_OPENSSL_1_1 = not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0) + def data_file(*name): return os.path.join(os.path.dirname(__file__), *name) @@ -164,7 +167,6 @@ self.assertIn(ssl.HAS_SNI, {True, False}) self.assertIn(ssl.HAS_ECDH, {True, False}) - def test_random(self): v = ssl.RAND_status() if support.verbose: @@ -281,9 +283,9 @@ self.assertGreaterEqual(status, 0) self.assertLessEqual(status, 15) # Version string as returned by {Open,Libre}SSL, the format might change - if "LibreSSL" in s: - self.assertTrue(s.startswith("LibreSSL {:d}.{:d}".format(major, minor)), - (s, t)) + if IS_LIBRESSL: + self.assertTrue(s.startswith("LibreSSL {:d}".format(major)), + (s, t, hex(n))) else: self.assertTrue(s.startswith("OpenSSL {:d}.{:d}.{:d}".format(major, minor, fix)), (s, t)) @@ -742,15 +744,15 @@ def test_options(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) # OP_ALL | OP_NO_SSLv2 | OP_NO_SSLv3 is the default value - self.assertEqual(ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3, - ctx.options) + default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3) + if not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0): + default |= ssl.OP_NO_COMPRESSION + self.assertEqual(default, ctx.options) ctx.options |= ssl.OP_NO_TLSv1 - self.assertEqual(ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1, - ctx.options) + self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options) if can_clear_options(): - ctx.options = (ctx.options & ~ssl.OP_NO_SSLv2) | ssl.OP_NO_TLSv1 - self.assertEqual(ssl.OP_ALL | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv3, - ctx.options) + ctx.options = (ctx.options & ~ssl.OP_NO_TLSv1) + self.assertEqual(default, ctx.options) ctx.options = 0 self.assertEqual(0, ctx.options) else: @@ -1088,6 +1090,7 @@ self.assertRaises(TypeError, ctx.load_default_certs, 'SERVER_AUTH') @unittest.skipIf(sys.platform == "win32", "not-Windows specific") + @unittest.skipIf(IS_LIBRESSL, "LibreSSL doesn't support env vars") def test_load_default_certs_env(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) with support.EnvironmentVarGuard() as env: @@ -1534,7 +1537,6 @@ sys.stdout.write("%s\n" % x) else: self.fail("Got server certificate %s for %s:%s!" % (pem, host, port)) - pem = ssl.get_server_certificate((host, port), ca_certs=cert) if not pem: @@ -2783,7 +2785,7 @@ with closing(context.wrap_socket(socket.socket())) as s: self.assertIs(s.version(), None) s.connect((HOST, server.port)) - self.assertEqual(s.version(), "TLSv1") + self.assertEqual(s.version(), 'TLSv1') self.assertIs(s.version(), None) @unittest.skipUnless(ssl.HAS_ECDH, "test requires ECDH-enabled OpenSSL") @@ -2925,24 +2927,36 @@ (['http/3.0', 'http/4.0'], None) ] for client_protocols, expected in protocol_tests: - server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) server_context.load_cert_chain(CERTFILE) server_context.set_alpn_protocols(server_protocols) - client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) client_context.load_cert_chain(CERTFILE) client_context.set_alpn_protocols(client_protocols) - stats = server_params_test(client_context, server_context, - chatty=True, connectionchatty=True) - - msg = "failed trying %s (s) and %s (c).\n" \ - "was expecting %s, but got %%s from the %%s" \ - % (str(server_protocols), str(client_protocols), - str(expected)) - client_result = stats['client_alpn_protocol'] - self.assertEqual(client_result, expected, msg % (client_result, "client")) - server_result = stats['server_alpn_protocols'][-1] \ - if len(stats['server_alpn_protocols']) else 'nothing' - self.assertEqual(server_result, expected, msg % (server_result, "server")) + + try: + stats = server_params_test(client_context, + server_context, + chatty=True, + connectionchatty=True) + except ssl.SSLError as e: + stats = e + + if expected is None and IS_OPENSSL_1_1: + # OpenSSL 1.1.0 raises handshake error + self.assertIsInstance(stats, ssl.SSLError) + else: + msg = "failed trying %s (s) and %s (c).\n" \ + "was expecting %s, but got %%s from the %%s" \ + % (str(server_protocols), str(client_protocols), + str(expected)) + client_result = stats['client_alpn_protocol'] + self.assertEqual(client_result, expected, + msg % (client_result, "client")) + server_result = stats['server_alpn_protocols'][-1] \ + if len(stats['server_alpn_protocols']) else 'nothing' + self.assertEqual(server_result, expected, + msg % (server_result, "server")) def test_selected_npn_protocol(self): # selected_npn_protocol() is None unless NPN is used diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -36,6 +36,8 @@ Library ------- +- Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. + - Issue #27944: Fix some memory-corruption bugs in the log reading code of the _hotshot module. diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -37,8 +37,10 @@ /* EVP is the preferred interface to hashing in OpenSSL */ #include -#include #include +/* We use the object interface to discover what hashes OpenSSL supports. */ +#include +#include "openssl/err.h" #define MUNCH_SIZE INT_MAX @@ -50,15 +52,26 @@ #define HASH_OBJ_CONSTRUCTOR 0 #endif -/* Minimum OpenSSL version needed to support sha224 and higher. */ #if defined(OPENSSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x00908000) #define _OPENSSL_SUPPORTS_SHA2 #endif +#if (OPENSSL_VERSION_NUMBER < 0x10100000L) || defined(LIBRESSL_VERSION_NUMBER) +/* OpenSSL < 1.1.0 */ +#define EVP_MD_CTX_new EVP_MD_CTX_create +#define EVP_MD_CTX_free EVP_MD_CTX_destroy +#define HAS_FAST_PKCS5_PBKDF2_HMAC 0 +#include +#else +/* OpenSSL >= 1.1.0 */ +#define HAS_FAST_PKCS5_PBKDF2_HMAC 1 +#endif + + typedef struct { PyObject_HEAD PyObject *name; /* name of this hash algorithm */ - EVP_MD_CTX ctx; /* OpenSSL message digest context */ + EVP_MD_CTX *ctx; /* OpenSSL message digest context */ #ifdef WITH_THREAD PyThread_type_lock lock; /* OpenSSL context lock */ #endif @@ -70,7 +83,6 @@ #define DEFINE_CONSTS_FOR_NEW(Name) \ static PyObject *CONST_ ## Name ## _name_obj = NULL; \ - static EVP_MD_CTX CONST_new_ ## Name ## _ctx; \ static EVP_MD_CTX *CONST_new_ ## Name ## _ctx_p = NULL; DEFINE_CONSTS_FOR_NEW(md5) @@ -83,19 +95,56 @@ #endif +/* LCOV_EXCL_START */ +static PyObject * +_setException(PyObject *exc) +{ + unsigned long errcode; + const char *lib, *func, *reason; + + errcode = ERR_peek_last_error(); + if (!errcode) { + PyErr_SetString(exc, "unknown reasons"); + return NULL; + } + ERR_clear_error(); + + lib = ERR_lib_error_string(errcode); + func = ERR_func_error_string(errcode); + reason = ERR_reason_error_string(errcode); + + if (lib && func) { + PyErr_Format(exc, "[%s: %s] %s", lib, func, reason); + } + else if (lib) { + PyErr_Format(exc, "[%s] %s", lib, reason); + } + else { + PyErr_SetString(exc, reason); + } + return NULL; +} +/* LCOV_EXCL_STOP */ + static EVPobject * newEVPobject(PyObject *name) { EVPobject *retval = (EVPobject *)PyObject_New(EVPobject, &EVPtype); + if (retval == NULL) + return NULL; + + retval->ctx = EVP_MD_CTX_new(); + if (retval->ctx == NULL) { + PyErr_NoMemory(); + return NULL; + } /* save the name for .name to return */ - if (retval != NULL) { - Py_INCREF(name); - retval->name = name; + Py_INCREF(name); + retval->name = name; #ifdef WITH_THREAD - retval->lock = NULL; + retval->lock = NULL; #endif - } return retval; } @@ -111,7 +160,7 @@ process = MUNCH_SIZE; else process = Py_SAFE_DOWNCAST(len, Py_ssize_t, unsigned int); - EVP_DigestUpdate(&self->ctx, (const void*)cp, process); + EVP_DigestUpdate(self->ctx, (const void*)cp, process); len -= process; cp += process; } @@ -126,16 +175,20 @@ if (self->lock != NULL) PyThread_free_lock(self->lock); #endif - EVP_MD_CTX_cleanup(&self->ctx); + EVP_MD_CTX_free(self->ctx); Py_XDECREF(self->name); PyObject_Del(self); } -static void locked_EVP_MD_CTX_copy(EVP_MD_CTX *new_ctx_p, EVPobject *self) +static int +locked_EVP_MD_CTX_copy(EVP_MD_CTX *new_ctx_p, EVPobject *self) { + int result; ENTER_HASHLIB(self); - EVP_MD_CTX_copy(new_ctx_p, &self->ctx); + /* XXX no error reporting */ + result = EVP_MD_CTX_copy(new_ctx_p, self->ctx); LEAVE_HASHLIB(self); + return result; } /* External methods for a hash object */ @@ -151,7 +204,9 @@ if ( (newobj = newEVPobject(self->name))==NULL) return NULL; - locked_EVP_MD_CTX_copy(&newobj->ctx, self); + if (!locked_EVP_MD_CTX_copy(newobj->ctx, self)) { + return _setException(PyExc_ValueError); + } return (PyObject *)newobj; } @@ -162,16 +217,24 @@ EVP_digest(EVPobject *self, PyObject *unused) { unsigned char digest[EVP_MAX_MD_SIZE]; - EVP_MD_CTX temp_ctx; + EVP_MD_CTX *temp_ctx; PyObject *retval; unsigned int digest_size; - locked_EVP_MD_CTX_copy(&temp_ctx, self); - digest_size = EVP_MD_CTX_size(&temp_ctx); - EVP_DigestFinal(&temp_ctx, digest, NULL); + temp_ctx = EVP_MD_CTX_new(); + if (temp_ctx == NULL) { + PyErr_NoMemory(); + return NULL; + } + + if (!locked_EVP_MD_CTX_copy(temp_ctx, self)) { + return _setException(PyExc_ValueError); + } + digest_size = EVP_MD_CTX_size(temp_ctx); + EVP_DigestFinal(temp_ctx, digest, NULL); retval = PyString_FromStringAndSize((const char *)digest, digest_size); - EVP_MD_CTX_cleanup(&temp_ctx); + EVP_MD_CTX_free(temp_ctx); return retval; } @@ -182,17 +245,25 @@ EVP_hexdigest(EVPobject *self, PyObject *unused) { unsigned char digest[EVP_MAX_MD_SIZE]; - EVP_MD_CTX temp_ctx; + EVP_MD_CTX *temp_ctx; PyObject *retval; char *hex_digest; unsigned int i, j, digest_size; + temp_ctx = EVP_MD_CTX_new(); + if (temp_ctx == NULL) { + PyErr_NoMemory(); + return NULL; + } + /* Get the raw (binary) digest value */ - locked_EVP_MD_CTX_copy(&temp_ctx, self); - digest_size = EVP_MD_CTX_size(&temp_ctx); - EVP_DigestFinal(&temp_ctx, digest, NULL); + if (!locked_EVP_MD_CTX_copy(temp_ctx, self)) { + return _setException(PyExc_ValueError); + } + digest_size = EVP_MD_CTX_size(temp_ctx); + EVP_DigestFinal(temp_ctx, digest, NULL); - EVP_MD_CTX_cleanup(&temp_ctx); + EVP_MD_CTX_free(temp_ctx); /* Create a new string */ /* NOTE: not thread safe! modifying an already created string object */ @@ -266,7 +337,7 @@ EVP_get_block_size(EVPobject *self, void *closure) { long block_size; - block_size = EVP_MD_CTX_block_size(&self->ctx); + block_size = EVP_MD_CTX_block_size(self->ctx); return PyLong_FromLong(block_size); } @@ -274,7 +345,7 @@ EVP_get_digest_size(EVPobject *self, void *closure) { long size; - size = EVP_MD_CTX_size(&self->ctx); + size = EVP_MD_CTX_size(self->ctx); return PyLong_FromLong(size); } @@ -338,7 +409,7 @@ PyBuffer_Release(&view); return -1; } - EVP_DigestInit(&self->ctx, digest); + EVP_DigestInit(self->ctx, digest); self->name = name_obj; Py_INCREF(self->name); @@ -435,9 +506,9 @@ return NULL; if (initial_ctx) { - EVP_MD_CTX_copy(&self->ctx, initial_ctx); + EVP_MD_CTX_copy(self->ctx, initial_ctx); } else { - EVP_DigestInit(&self->ctx, digest); + EVP_DigestInit(self->ctx, digest); } if (cp && len) { @@ -499,6 +570,7 @@ #define PY_PBKDF2_HMAC 1 +#if !HAS_FAST_PKCS5_PBKDF2_HMAC /* Improved implementation of PKCS5_PBKDF2_HMAC() * * PKCS5_PBKDF2_HMAC_fast() hashes the password exactly one time instead of @@ -580,37 +652,8 @@ HMAC_CTX_cleanup(&hctx_tpl); return 1; } +#endif -/* LCOV_EXCL_START */ -static PyObject * -_setException(PyObject *exc) -{ - unsigned long errcode; - const char *lib, *func, *reason; - - errcode = ERR_peek_last_error(); - if (!errcode) { - PyErr_SetString(exc, "unknown reasons"); - return NULL; - } - ERR_clear_error(); - - lib = ERR_lib_error_string(errcode); - func = ERR_func_error_string(errcode); - reason = ERR_reason_error_string(errcode); - - if (lib && func) { - PyErr_Format(exc, "[%s: %s] %s", lib, func, reason); - } - else if (lib) { - PyErr_Format(exc, "[%s] %s", lib, reason); - } - else { - PyErr_SetString(exc, reason); - } - return NULL; -} -/* LCOV_EXCL_STOP */ PyDoc_STRVAR(pbkdf2_hmac__doc__, "pbkdf2_hmac(hash_name, password, salt, iterations, dklen=None) -> key\n\ @@ -692,10 +735,17 @@ key = PyBytes_AS_STRING(key_obj); Py_BEGIN_ALLOW_THREADS +#if HAS_FAST_PKCS5_PBKDF2_HMAC + retval = PKCS5_PBKDF2_HMAC((char*)password.buf, (int)password.len, + (unsigned char *)salt.buf, (int)salt.len, + iterations, digest, dklen, + (unsigned char *)key); +#else retval = PKCS5_PBKDF2_HMAC_fast((char*)password.buf, (int)password.len, (unsigned char *)salt.buf, (int)salt.len, iterations, digest, dklen, (unsigned char *)key); +#endif Py_END_ALLOW_THREADS if (!retval) { @@ -807,7 +857,7 @@ if (CONST_ ## NAME ## _name_obj == NULL) { \ CONST_ ## NAME ## _name_obj = PyString_FromString(#NAME); \ if (EVP_get_digestbyname(#NAME)) { \ - CONST_new_ ## NAME ## _ctx_p = &CONST_new_ ## NAME ## _ctx; \ + CONST_new_ ## NAME ## _ctx_p = EVP_MD_CTX_new(); \ EVP_DigestInit(CONST_new_ ## NAME ## _ctx_p, EVP_get_digestbyname(#NAME)); \ } \ } \ diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -52,6 +52,14 @@ #include #endif +/* Don't warn about deprecated functions */ +#ifdef __GNUC__ +#pragma GCC diagnostic ignored "-Wdeprecated-declarations" +#endif +#ifdef __clang__ +#pragma clang diagnostic ignored "-Wdeprecated-declarations" +#endif + /* Include OpenSSL header files */ #include "openssl/rsa.h" #include "openssl/crypto.h" @@ -87,6 +95,10 @@ /* Include generated data (error codes) */ #include "_ssl_data.h" +#if (OPENSSL_VERSION_NUMBER >= 0x10100000L) && !defined(LIBRESSL_VERSION_NUMBER) +# define OPENSSL_VERSION_1_1 1 +#endif + /* Openssl comes with TLSv1.1 and TLSv1.2 between 1.0.0h and 1.0.1 http://www.openssl.org/news/changelog.html */ @@ -110,6 +122,70 @@ # define HAVE_ALPN #endif +#ifndef INVALID_SOCKET /* MS defines this */ +#define INVALID_SOCKET (-1) +#endif + +#ifdef OPENSSL_VERSION_1_1 +/* OpenSSL 1.1.0+ */ +#ifndef OPENSSL_NO_SSL2 +#define OPENSSL_NO_SSL2 +#endif +#else /* OpenSSL < 1.1.0 */ +#if defined(WITH_THREAD) +#define HAVE_OPENSSL_CRYPTO_LOCK +#endif + +#define TLS_method SSLv23_method + +static int X509_NAME_ENTRY_set(const X509_NAME_ENTRY *ne) +{ + return ne->set; +} + +#ifndef OPENSSL_NO_COMP +static int COMP_get_type(const COMP_METHOD *meth) +{ + return meth->type; +} + +static const char *COMP_get_name(const COMP_METHOD *meth) +{ + return meth->name; +} +#endif + +static pem_password_cb *SSL_CTX_get_default_passwd_cb(SSL_CTX *ctx) +{ + return ctx->default_passwd_callback; +} + +static void *SSL_CTX_get_default_passwd_cb_userdata(SSL_CTX *ctx) +{ + return ctx->default_passwd_callback_userdata; +} + +static int X509_OBJECT_get_type(X509_OBJECT *x) +{ + return x->type; +} + +static X509 *X509_OBJECT_get0_X509(X509_OBJECT *x) +{ + return x->data.x509; +} + +static STACK_OF(X509_OBJECT) *X509_STORE_get0_objects(X509_STORE *store) { + return store->objs; +} + +static X509_VERIFY_PARAM *X509_STORE_get0_param(X509_STORE *store) +{ + return store->param; +} +#endif /* OpenSSL < 1.1.0 or LibreSSL */ + + enum py_ssl_error { /* these mirror ssl.h */ PY_SSL_ERROR_NONE, @@ -140,7 +216,7 @@ enum py_ssl_version { PY_SSL_VERSION_SSL2, PY_SSL_VERSION_SSL3=1, - PY_SSL_VERSION_SSL23, + PY_SSL_VERSION_TLS, #if HAVE_TLSv1_2 PY_SSL_VERSION_TLS1, PY_SSL_VERSION_TLS1_1, @@ -681,7 +757,7 @@ /* check to see if we've gotten to a new RDN */ if (rdn_level >= 0) { - if (rdn_level != entry->set) { + if (rdn_level != X509_NAME_ENTRY_set(entry)) { /* yes, new RDN */ /* add old RDN to DN */ rdnt = PyList_AsTuple(rdn); @@ -698,7 +774,7 @@ goto fail0; } } - rdn_level = entry->set; + rdn_level = X509_NAME_ENTRY_set(entry); /* now add this attribute to the current RDN */ name = X509_NAME_ENTRY_get_object(entry); @@ -801,18 +877,18 @@ goto fail; } - p = ext->value->data; + p = X509_EXTENSION_get_data(ext)->data; if (method->it) names = (GENERAL_NAMES*) (ASN1_item_d2i(NULL, &p, - ext->value->length, + X509_EXTENSION_get_data(ext)->length, ASN1_ITEM_ptr(method->it))); else names = (GENERAL_NAMES*) (method->d2i(NULL, &p, - ext->value->length)); + X509_EXTENSION_get_data(ext)->length)); for(j = 0; j < sk_GENERAL_NAME_num(names); j++) { /* get a rendering of each name in the set of names */ @@ -1021,13 +1097,11 @@ int i, j; PyObject *lst, *res = NULL; -#if OPENSSL_VERSION_NUMBER < 0x10001000L - dps = X509_get_ext_d2i(certificate, NID_crl_distribution_points, NULL, NULL); -#else +#if OPENSSL_VERSION_NUMBER >= 0x10001000L /* Calls x509v3_cache_extensions and sets up crldp */ X509_check_ca(certificate); - dps = certificate->crldp; #endif + dps = X509_get_ext_d2i(certificate, NID_crl_distribution_points, NULL, NULL); if (dps == NULL) return Py_None; @@ -1443,9 +1517,9 @@ if (self->ssl == NULL) Py_RETURN_NONE; comp_method = SSL_get_current_compression(self->ssl); - if (comp_method == NULL || comp_method->type == NID_undef) + if (comp_method == NULL || COMP_get_type(comp_method) == NID_undef) Py_RETURN_NONE; - short_name = OBJ_nid2sn(comp_method->type); + short_name = COMP_get_name(comp_method); if (short_name == NULL) Py_RETURN_NONE; return PyBytes_FromString(short_name); @@ -1994,7 +2068,7 @@ { char *kwlist[] = {"protocol", NULL}; PySSLContext *self; - int proto_version = PY_SSL_VERSION_SSL23; + int proto_version = PY_SSL_VERSION_TLS; long options; SSL_CTX *ctx = NULL; @@ -2020,8 +2094,8 @@ else if (proto_version == PY_SSL_VERSION_SSL2) ctx = SSL_CTX_new(SSLv2_method()); #endif - else if (proto_version == PY_SSL_VERSION_SSL23) - ctx = SSL_CTX_new(SSLv23_method()); + else if (proto_version == PY_SSL_VERSION_TLS) + ctx = SSL_CTX_new(TLS_method()); else proto_version = -1; PySSL_END_ALLOW_THREADS @@ -2067,8 +2141,9 @@ #ifndef OPENSSL_NO_ECDH /* Allow automatic ECDH curve selection (on OpenSSL 1.0.2+), or use prime256v1 by default. This is Apache mod_ssl's initialization - policy, so we should be safe. */ -#if defined(SSL_CTX_set_ecdh_auto) + policy, so we should be safe. OpenSSL 1.1 has it enabled by default. + */ +#if defined(SSL_CTX_set_ecdh_auto) && !defined(OPENSSL_VERSION_1_1) SSL_CTX_set_ecdh_auto(self->ctx, 1); #else { @@ -2336,10 +2411,12 @@ get_verify_flags(PySSLContext *self, void *c) { X509_STORE *store; + X509_VERIFY_PARAM *param; unsigned long flags; store = SSL_CTX_get_cert_store(self->ctx); - flags = X509_VERIFY_PARAM_get_flags(store->param); + param = X509_STORE_get0_param(store); + flags = X509_VERIFY_PARAM_get_flags(param); return PyLong_FromUnsignedLong(flags); } @@ -2347,22 +2424,24 @@ set_verify_flags(PySSLContext *self, PyObject *arg, void *c) { X509_STORE *store; + X509_VERIFY_PARAM *param; unsigned long new_flags, flags, set, clear; if (!PyArg_Parse(arg, "k", &new_flags)) return -1; store = SSL_CTX_get_cert_store(self->ctx); - flags = X509_VERIFY_PARAM_get_flags(store->param); + param = X509_STORE_get0_param(store); + flags = X509_VERIFY_PARAM_get_flags(param); clear = flags & ~new_flags; set = ~flags & new_flags; if (clear) { - if (!X509_VERIFY_PARAM_clear_flags(store->param, clear)) { + if (!X509_VERIFY_PARAM_clear_flags(param, clear)) { _setSSLError(NULL, 0, __FILE__, __LINE__); return -1; } } if (set) { - if (!X509_VERIFY_PARAM_set_flags(store->param, set)) { + if (!X509_VERIFY_PARAM_set_flags(param, set)) { _setSSLError(NULL, 0, __FILE__, __LINE__); return -1; } @@ -2537,8 +2616,8 @@ char *kwlist[] = {"certfile", "keyfile", "password", NULL}; PyObject *keyfile = NULL, *keyfile_bytes = NULL, *password = NULL; char *certfile_bytes = NULL; - pem_password_cb *orig_passwd_cb = self->ctx->default_passwd_callback; - void *orig_passwd_userdata = self->ctx->default_passwd_callback_userdata; + pem_password_cb *orig_passwd_cb = SSL_CTX_get_default_passwd_cb(self->ctx); + void *orig_passwd_userdata = SSL_CTX_get_default_passwd_cb_userdata(self->ctx); _PySSLPasswordInfo pw_info = { NULL, NULL, NULL, 0, 0 }; int r; @@ -2674,8 +2753,9 @@ cert = d2i_X509_bio(biobuf, NULL); } else { cert = PEM_read_bio_X509(biobuf, NULL, - self->ctx->default_passwd_callback, - self->ctx->default_passwd_callback_userdata); + SSL_CTX_get_default_passwd_cb(self->ctx), + SSL_CTX_get_default_passwd_cb_userdata(self->ctx) + ); } if (cert == NULL) { break; @@ -3160,25 +3240,24 @@ cert_store_stats(PySSLContext *self) { X509_STORE *store; + STACK_OF(X509_OBJECT) *objs; X509_OBJECT *obj; - int x509 = 0, crl = 0, pkey = 0, ca = 0, i; + int x509 = 0, crl = 0, ca = 0, i; store = SSL_CTX_get_cert_store(self->ctx); - for (i = 0; i < sk_X509_OBJECT_num(store->objs); i++) { - obj = sk_X509_OBJECT_value(store->objs, i); - switch (obj->type) { + objs = X509_STORE_get0_objects(store); + for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { + obj = sk_X509_OBJECT_value(objs, i); + switch (X509_OBJECT_get_type(obj)) { case X509_LU_X509: x509++; - if (X509_check_ca(obj->data.x509)) { + if (X509_check_ca(X509_OBJECT_get0_X509(obj))) { ca++; } break; case X509_LU_CRL: crl++; break; - case X509_LU_PKEY: - pkey++; - break; default: /* Ignore X509_LU_FAIL, X509_LU_RETRY, X509_LU_PKEY. * As far as I can tell they are internal states and never @@ -3204,6 +3283,7 @@ char *kwlist[] = {"binary_form", NULL}; X509_STORE *store; PyObject *ci = NULL, *rlist = NULL, *py_binary_mode = Py_False; + STACK_OF(X509_OBJECT) *objs; int i; int binary_mode = 0; @@ -3221,17 +3301,18 @@ } store = SSL_CTX_get_cert_store(self->ctx); - for (i = 0; i < sk_X509_OBJECT_num(store->objs); i++) { + objs = X509_STORE_get0_objects(store); + for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { X509_OBJECT *obj; X509 *cert; - obj = sk_X509_OBJECT_value(store->objs, i); - if (obj->type != X509_LU_X509) { + obj = sk_X509_OBJECT_value(objs, i); + if (X509_OBJECT_get_type(obj) != X509_LU_X509) { /* not a x509 cert */ continue; } /* CA for any purpose */ - cert = obj->data.x509; + cert = X509_OBJECT_get0_X509(obj); if (!X509_check_ca(cert)) { continue; } @@ -3842,10 +3923,12 @@ }; -#ifdef WITH_THREAD +#ifdef HAVE_OPENSSL_CRYPTO_LOCK /* an implementation of OpenSSL threading operations in terms - of the Python C thread library */ + * of the Python C thread library + * Only used up to 1.0.2. OpenSSL 1.1.0+ has its own locking code. + */ static PyThread_type_lock *_ssl_locks = NULL; @@ -3926,7 +4009,7 @@ return 1; } -#endif /* def HAVE_THREAD */ +#endif /* HAVE_OPENSSL_CRYPTO_LOCK for WITH_THREAD && OpenSSL < 1.1.0 */ PyDoc_STRVAR(module_doc, "Implementation module for SSL socket operations. See the socket module\n\ @@ -3979,11 +4062,16 @@ SSL_load_error_strings(); SSL_library_init(); #ifdef WITH_THREAD +#ifdef HAVE_OPENSSL_CRYPTO_LOCK /* note that this will start threading if not already started */ if (!_setup_ssl_threads()) { return; } +#elif OPENSSL_VERSION_1_1 && defined(OPENSSL_THREADS) + /* OpenSSL 1.1.0 builtin thread support is enabled */ + _ssl_locks_count++; #endif +#endif /* WITH_THREAD */ OpenSSL_add_all_algorithms(); /* Add symbols to module dict */ @@ -4136,7 +4224,9 @@ PY_SSL_VERSION_SSL3); #endif PyModule_AddIntConstant(m, "PROTOCOL_SSLv23", - PY_SSL_VERSION_SSL23); + PY_SSL_VERSION_TLS); + PyModule_AddIntConstant(m, "PROTOCOL_TLS", + PY_SSL_VERSION_TLS); PyModule_AddIntConstant(m, "PROTOCOL_TLSv1", PY_SSL_VERSION_TLS1); #if HAVE_TLSv1_2 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:38:18 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 21:38:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326470=3A_Port_ssl_and_hashlib_module_to_OpenSSL?= =?utf-8?b?IDEuMS4wLg==?= Message-ID: <20160905213817.104228.40882.C263DBAF@psf.io> https://hg.python.org/cpython/rev/bc5ba11973f5 changeset: 103068:bc5ba11973f5 parent: 103066:8fa615a2a896 parent: 103067:5c75b315152b user: Christian Heimes date: Mon Sep 05 23:23:24 2016 +0200 summary: Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. files: Doc/library/ssl.rst | 112 ++++++++++++++--- Lib/ssl.py | 18 +- Lib/test/test_ssl.py | 91 +++++++++----- Misc/NEWS | 2 + Modules/_hashopenssl.c | 165 ++++++++++++++++--------- Modules/_ssl.c | 181 +++++++++++++++++++++------- 6 files changed, 402 insertions(+), 167 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -49,6 +49,12 @@ helps manage settings and certificates, which can then be inherited by SSL sockets created through the :meth:`SSLContext.wrap_socket` method. +.. versionchanged:: 3.6 + + OpenSSL 0.9.8, 1.0.0 and 1.0.1 are deprecated and no longer supported. + In the future the ssl module will require at least OpenSSL 1.0.2 or + 1.1.0. + Functions, Constants, and Exceptions ------------------------------------ @@ -178,7 +184,7 @@ use. Typically, the server chooses a particular protocol version, and the client must adapt to the server's choice. Most of the versions are not interoperable with the other versions. If not specified, the default is - :data:`PROTOCOL_SSLv23`; it provides the most compatibility with other + :data:`PROTOCOL_TLS`; it provides the most compatibility with other versions. Here's a table showing which versions in a client (down the side) can connect @@ -187,11 +193,11 @@ .. table:: ======================== ========= ========= ========== ========= =========== =========== - *client* / **server** **SSLv2** **SSLv3** **SSLv23** **TLSv1** **TLSv1.1** **TLSv1.2** + *client* / **server** **SSLv2** **SSLv3** **TLS** **TLSv1** **TLSv1.1** **TLSv1.2** ------------------------ --------- --------- ---------- --------- ----------- ----------- *SSLv2* yes no yes no no no *SSLv3* no yes yes no no no - *SSLv23* no yes yes yes yes yes + *TLS* (*SSLv23*) no yes yes yes yes yes *TLSv1* no no yes yes no no *TLSv1.1* no no yes no yes no *TLSv1.2* no no yes no no yes @@ -244,7 +250,7 @@ :const:`None`, this function can choose to trust the system's default CA certificates instead. - The settings are: :data:`PROTOCOL_SSLv23`, :data:`OP_NO_SSLv2`, and + The settings are: :data:`PROTOCOL_TLS`, :data:`OP_NO_SSLv2`, and :data:`OP_NO_SSLv3` with high encryption cipher suites without RC4 and without unauthenticated cipher suites. Passing :data:`~Purpose.SERVER_AUTH` as *purpose* sets :data:`~SSLContext.verify_mode` to :data:`CERT_REQUIRED` @@ -316,6 +322,11 @@ .. versionadded:: 3.3 + .. deprecated:: 3.6 + + OpenSSL has deprecated :func:`ssl.RAND_pseudo_bytes`, use + :func:`ssl.RAND_bytes` instead. + .. function:: RAND_status() Return ``True`` if the SSL pseudo-random number generator has been seeded @@ -334,7 +345,7 @@ See http://egd.sourceforge.net/ or http://prngd.sourceforge.net/ for sources of entropy-gathering daemons. - Availability: not available with LibreSSL. + Availability: not available with LibreSSL and OpenSSL > 1.1.0 .. function:: RAND_add(bytes, entropy) @@ -409,7 +420,7 @@ previously. Return an integer (no fractions of a second in the input format) -.. function:: get_server_certificate(addr, ssl_version=PROTOCOL_SSLv23, ca_certs=None) +.. function:: get_server_certificate(addr, ssl_version=PROTOCOL_TLS, ca_certs=None) Given the address ``addr`` of an SSL-protected server, as a (*hostname*, *port-number*) pair, fetches the server's certificate, and returns it as a @@ -425,7 +436,7 @@ .. versionchanged:: 3.5 The default *ssl_version* is changed from :data:`PROTOCOL_SSLv3` to - :data:`PROTOCOL_SSLv23` for maximum compatibility with modern servers. + :data:`PROTOCOL_TLS` for maximum compatibility with modern servers. .. function:: DER_cert_to_PEM_cert(DER_cert_bytes) @@ -451,6 +462,9 @@ * :attr:`openssl_capath_env` - OpenSSL's environment key that points to a capath, * :attr:`openssl_capath` - hard coded path to a capath directory + Availability: LibreSSL ignores the environment vars + :attr:`openssl_cafile_env` and :attr:`openssl_capath_env` + .. versionadded:: 3.4 .. function:: enum_certificates(store_name) @@ -568,11 +582,21 @@ .. versionadded:: 3.4.4 -.. data:: PROTOCOL_SSLv23 +.. data:: PROTOCOL_TLS Selects the highest protocol version that both the client and server support. Despite the name, this option can select "TLS" protocols as well as "SSL". + .. versionadded:: 3.6 + +.. data:: PROTOCOL_SSLv23 + + Alias for data:`PROTOCOL_TLS`. + + .. deprecated:: 3.6 + + Use data:`PROTOCOL_TLS` instead. + .. data:: PROTOCOL_SSLv2 Selects SSL version 2 as the channel encryption protocol. @@ -584,6 +608,10 @@ SSL version 2 is insecure. Its use is highly discouraged. + .. deprecated:: 3.6 + + OpenSSL has removed support for SSLv2. + .. data:: PROTOCOL_SSLv3 Selects SSL version 3 as the channel encryption protocol. @@ -595,10 +623,20 @@ SSL version 3 is insecure. Its use is highly discouraged. + .. deprecated:: 3.6 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol data:`PROTOCOL_TLS` with flags like data:`OP_NO_SSLv3` instead. + .. data:: PROTOCOL_TLSv1 Selects TLS version 1.0 as the channel encryption protocol. + .. deprecated:: 3.6 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol data:`PROTOCOL_TLS` with flags like data:`OP_NO_SSLv3` instead. + .. data:: PROTOCOL_TLSv1_1 Selects TLS version 1.1 as the channel encryption protocol. @@ -606,6 +644,11 @@ .. versionadded:: 3.4 + .. deprecated:: 3.6 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol data:`PROTOCOL_TLS` with flags like data:`OP_NO_SSLv3` instead. + .. data:: PROTOCOL_TLSv1_2 Selects TLS version 1.2 as the channel encryption protocol. This is the @@ -614,6 +657,11 @@ .. versionadded:: 3.4 + .. deprecated:: 3.6 + + OpenSSL has deprecated all version specific protocols. Use the default + protocol data:`PROTOCOL_TLS` with flags like data:`OP_NO_SSLv3` instead. + .. data:: OP_ALL Enables workarounds for various bugs present in other SSL implementations. @@ -625,23 +673,32 @@ .. data:: OP_NO_SSLv2 Prevents an SSLv2 connection. This option is only applicable in - conjunction with :const:`PROTOCOL_SSLv23`. It prevents the peers from + conjunction with :const:`PROTOCOL_TLS`. It prevents the peers from choosing SSLv2 as the protocol version. .. versionadded:: 3.2 + .. deprecated:: 3.6 + + SSLv2 is deprecated + + .. data:: OP_NO_SSLv3 Prevents an SSLv3 connection. This option is only applicable in - conjunction with :const:`PROTOCOL_SSLv23`. It prevents the peers from + conjunction with :const:`PROTOCOL_TLS`. It prevents the peers from choosing SSLv3 as the protocol version. .. versionadded:: 3.2 + .. deprecated:: 3.6 + + SSLv3 is deprecated + .. data:: OP_NO_TLSv1 Prevents a TLSv1 connection. This option is only applicable in - conjunction with :const:`PROTOCOL_SSLv23`. It prevents the peers from + conjunction with :const:`PROTOCOL_TLS`. It prevents the peers from choosing TLSv1 as the protocol version. .. versionadded:: 3.2 @@ -649,7 +706,7 @@ .. data:: OP_NO_TLSv1_1 Prevents a TLSv1.1 connection. This option is only applicable in conjunction - with :const:`PROTOCOL_SSLv23`. It prevents the peers from choosing TLSv1.1 as + with :const:`PROTOCOL_TLS`. It prevents the peers from choosing TLSv1.1 as the protocol version. Available only with openssl version 1.0.1+. .. versionadded:: 3.4 @@ -657,7 +714,7 @@ .. data:: OP_NO_TLSv1_2 Prevents a TLSv1.2 connection. This option is only applicable in conjunction - with :const:`PROTOCOL_SSLv23`. It prevents the peers from choosing TLSv1.2 as + with :const:`PROTOCOL_TLS`. It prevents the peers from choosing TLSv1.2 as the protocol version. Available only with openssl version 1.0.1+. .. versionadded:: 3.4 @@ -1081,17 +1138,21 @@ It also manages a cache of SSL sessions for server-side sockets, in order to speed up repeated connections from the same clients. -.. class:: SSLContext(protocol) - - Create a new SSL context. You must pass *protocol* which must be one +.. class:: SSLContext(protocol=PROTOCOL_TLS) + + Create a new SSL context. You may pass *protocol* which must be one of the ``PROTOCOL_*`` constants defined in this module. - :data:`PROTOCOL_SSLv23` is currently recommended for maximum - interoperability. + :data:`PROTOCOL_TLS` is currently recommended for maximum + interoperability and default value. .. seealso:: :func:`create_default_context` lets the :mod:`ssl` module choose security settings for a given purpose. + .. versionchanged:: 3.6 + + :data:`PROTOCOL_TLS` is the default value. + :class:`SSLContext` objects have the following methods and attributes: @@ -1232,6 +1293,9 @@ This method will raise :exc:`NotImplementedError` if :data:`HAS_ALPN` is False. + OpenSSL 1.1.0+ will abort the handshake and raise :exc:`SSLError` when + both sides support ALPN but cannot agree on a protocol. + .. versionadded:: 3.5 .. method:: SSLContext.set_npn_protocols(protocols) @@ -1598,7 +1662,7 @@ a context from scratch (but beware that you might not get the settings right):: - >>> context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) + >>> context = ssl.SSLContext(ssl.PROTOCOL_TLS) >>> context.verify_mode = ssl.CERT_REQUIRED >>> context.check_hostname = True >>> context.load_verify_locations("/etc/ssl/certs/ca-bundle.crt") @@ -1999,15 +2063,17 @@ SSL versions 2 and 3 are considered insecure and are therefore dangerous to use. If you want maximum compatibility between clients and servers, it is -recommended to use :const:`PROTOCOL_SSLv23` as the protocol version and then +recommended to use :const:`PROTOCOL_TLS` as the protocol version and then disable SSLv2 and SSLv3 explicitly using the :data:`SSLContext.options` attribute:: - context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) + context = ssl.SSLContext(ssl.PROTOCOL_TLS) context.options |= ssl.OP_NO_SSLv2 context.options |= ssl.OP_NO_SSLv3 - -The SSL context created above will only allow TLSv1 and later (if + context.options |= ssl.OP_NO_TLSv1 + context.options |= ssl.OP_NO_TLSv1_1 + +The SSL context created above will only allow TLSv1.2 and later (if supported by your system) connections. Cipher selection diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -51,6 +51,7 @@ PROTOCOL_SSLv2 PROTOCOL_SSLv3 PROTOCOL_SSLv23 +PROTOCOL_TLS PROTOCOL_TLSv1 PROTOCOL_TLSv1_1 PROTOCOL_TLSv1_2 @@ -128,9 +129,10 @@ _IntEnum._convert( '_SSLMethod', __name__, - lambda name: name.startswith('PROTOCOL_'), + lambda name: name.startswith('PROTOCOL_') and name != 'PROTOCOL_SSLv23', source=_ssl) +PROTOCOL_SSLv23 = _SSLMethod.PROTOCOL_SSLv23 = _SSLMethod.PROTOCOL_TLS _PROTOCOL_NAMES = {value: name for name, value in _SSLMethod.__members__.items()} try: @@ -357,13 +359,13 @@ __slots__ = ('protocol', '__weakref__') _windows_cert_stores = ("CA", "ROOT") - def __new__(cls, protocol, *args, **kwargs): + def __new__(cls, protocol=PROTOCOL_TLS, *args, **kwargs): self = _SSLContext.__new__(cls, protocol) if protocol != _SSLv2_IF_EXISTS: self.set_ciphers(_DEFAULT_CIPHERS) return self - def __init__(self, protocol): + def __init__(self, protocol=PROTOCOL_TLS): self.protocol = protocol def wrap_socket(self, sock, server_side=False, @@ -438,7 +440,7 @@ if not isinstance(purpose, _ASN1Object): raise TypeError(purpose) - context = SSLContext(PROTOCOL_SSLv23) + context = SSLContext(PROTOCOL_TLS) # SSLv2 considered harmful. context.options |= OP_NO_SSLv2 @@ -475,7 +477,7 @@ context.load_default_certs(purpose) return context -def _create_unverified_context(protocol=PROTOCOL_SSLv23, *, cert_reqs=None, +def _create_unverified_context(protocol=PROTOCOL_TLS, *, cert_reqs=None, check_hostname=False, purpose=Purpose.SERVER_AUTH, certfile=None, keyfile=None, cafile=None, capath=None, cadata=None): @@ -666,7 +668,7 @@ def __init__(self, sock=None, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, - ssl_version=PROTOCOL_SSLv23, ca_certs=None, + ssl_version=PROTOCOL_TLS, ca_certs=None, do_handshake_on_connect=True, family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None, suppress_ragged_eofs=True, npn_protocols=None, ciphers=None, @@ -1055,7 +1057,7 @@ def wrap_socket(sock, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, - ssl_version=PROTOCOL_SSLv23, ca_certs=None, + ssl_version=PROTOCOL_TLS, ca_certs=None, do_handshake_on_connect=True, suppress_ragged_eofs=True, ciphers=None): @@ -1124,7 +1126,7 @@ d = pem_cert_string.strip()[len(PEM_HEADER):-len(PEM_FOOTER)] return base64.decodebytes(d.encode('ASCII', 'strict')) -def get_server_certificate(addr, ssl_version=PROTOCOL_SSLv23, ca_certs=None): +def get_server_certificate(addr, ssl_version=PROTOCOL_TLS, ca_certs=None): """Retrieve the certificate from the server at the specified address, and return it as a PEM-encoded string. If 'ca_certs' is specified, validate the server cert against it. diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -30,6 +30,9 @@ PROTOCOLS = sorted(ssl._PROTOCOL_NAMES) HOST = support.HOST +IS_LIBRESSL = ssl.OPENSSL_VERSION.startswith('LibreSSL') +IS_OPENSSL_1_1 = not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0) + def data_file(*name): return os.path.join(os.path.dirname(__file__), *name) @@ -150,8 +153,8 @@ def test_str_for_enums(self): # Make sure that the PROTOCOL_* constants have enum-like string # reprs. - proto = ssl.PROTOCOL_SSLv23 - self.assertEqual(str(proto), '_SSLMethod.PROTOCOL_SSLv23') + proto = ssl.PROTOCOL_TLS + self.assertEqual(str(proto), '_SSLMethod.PROTOCOL_TLS') ctx = ssl.SSLContext(proto) self.assertIs(ctx.protocol, proto) @@ -319,8 +322,8 @@ self.assertGreaterEqual(status, 0) self.assertLessEqual(status, 15) # Version string as returned by {Open,Libre}SSL, the format might change - if "LibreSSL" in s: - self.assertTrue(s.startswith("LibreSSL {:d}.{:d}".format(major, minor)), + if IS_LIBRESSL: + self.assertTrue(s.startswith("LibreSSL {:d}".format(major)), (s, t, hex(n))) else: self.assertTrue(s.startswith("OpenSSL {:d}.{:d}.{:d}".format(major, minor, fix)), @@ -813,7 +816,8 @@ def test_constructor(self): for protocol in PROTOCOLS: ssl.SSLContext(protocol) - self.assertRaises(TypeError, ssl.SSLContext) + ctx = ssl.SSLContext() + self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLS) self.assertRaises(ValueError, ssl.SSLContext, -1) self.assertRaises(ValueError, ssl.SSLContext, 42) @@ -834,15 +838,15 @@ def test_options(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) # OP_ALL | OP_NO_SSLv2 | OP_NO_SSLv3 is the default value - self.assertEqual(ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3, - ctx.options) + default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3) + if not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0): + default |= ssl.OP_NO_COMPRESSION + self.assertEqual(default, ctx.options) ctx.options |= ssl.OP_NO_TLSv1 - self.assertEqual(ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1, - ctx.options) + self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options) if can_clear_options(): - ctx.options = (ctx.options & ~ssl.OP_NO_SSLv2) | ssl.OP_NO_TLSv1 - self.assertEqual(ssl.OP_ALL | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv3, - ctx.options) + ctx.options = (ctx.options & ~ssl.OP_NO_TLSv1) + self.assertEqual(default, ctx.options) ctx.options = 0 # Ubuntu has OP_NO_SSLv3 forced on by default self.assertEqual(0, ctx.options & ~ssl.OP_NO_SSLv3) @@ -1178,6 +1182,7 @@ self.assertRaises(TypeError, ctx.load_default_certs, 'SERVER_AUTH') @unittest.skipIf(sys.platform == "win32", "not-Windows specific") + @unittest.skipIf(IS_LIBRESSL, "LibreSSL doesn't support env vars") def test_load_default_certs_env(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) with support.EnvironmentVarGuard() as env: @@ -1668,13 +1673,13 @@ sslobj = ctx.wrap_bio(incoming, outgoing, False, 'localhost') self.assertIs(sslobj._sslobj.owner, sslobj) self.assertIsNone(sslobj.cipher()) - self.assertIsNone(sslobj.shared_ciphers()) + self.assertIsNotNone(sslobj.shared_ciphers()) self.assertRaises(ValueError, sslobj.getpeercert) if 'tls-unique' in ssl.CHANNEL_BINDING_TYPES: self.assertIsNone(sslobj.get_channel_binding('tls-unique')) self.ssl_io_loop(sock, incoming, outgoing, sslobj.do_handshake) self.assertTrue(sslobj.cipher()) - self.assertIsNone(sslobj.shared_ciphers()) + self.assertIsNotNone(sslobj.shared_ciphers()) self.assertTrue(sslobj.getpeercert()) if 'tls-unique' in ssl.CHANNEL_BINDING_TYPES: self.assertTrue(sslobj.get_channel_binding('tls-unique')) @@ -2988,7 +2993,7 @@ with context.wrap_socket(socket.socket()) as s: self.assertIs(s.version(), None) s.connect((HOST, server.port)) - self.assertEqual(s.version(), "TLSv1") + self.assertEqual(s.version(), 'TLSv1') self.assertIs(s.version(), None) @unittest.skipUnless(ssl.HAS_ECDH, "test requires ECDH-enabled OpenSSL") @@ -3130,24 +3135,36 @@ (['http/3.0', 'http/4.0'], None) ] for client_protocols, expected in protocol_tests: - server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) server_context.load_cert_chain(CERTFILE) server_context.set_alpn_protocols(server_protocols) - client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) client_context.load_cert_chain(CERTFILE) client_context.set_alpn_protocols(client_protocols) - stats = server_params_test(client_context, server_context, - chatty=True, connectionchatty=True) - - msg = "failed trying %s (s) and %s (c).\n" \ - "was expecting %s, but got %%s from the %%s" \ - % (str(server_protocols), str(client_protocols), - str(expected)) - client_result = stats['client_alpn_protocol'] - self.assertEqual(client_result, expected, msg % (client_result, "client")) - server_result = stats['server_alpn_protocols'][-1] \ - if len(stats['server_alpn_protocols']) else 'nothing' - self.assertEqual(server_result, expected, msg % (server_result, "server")) + + try: + stats = server_params_test(client_context, + server_context, + chatty=True, + connectionchatty=True) + except ssl.SSLError as e: + stats = e + + if expected is None and IS_OPENSSL_1_1: + # OpenSSL 1.1.0 raises handshake error + self.assertIsInstance(stats, ssl.SSLError) + else: + msg = "failed trying %s (s) and %s (c).\n" \ + "was expecting %s, but got %%s from the %%s" \ + % (str(server_protocols), str(client_protocols), + str(expected)) + client_result = stats['client_alpn_protocol'] + self.assertEqual(client_result, expected, + msg % (client_result, "client")) + server_result = stats['server_alpn_protocols'][-1] \ + if len(stats['server_alpn_protocols']) else 'nothing' + self.assertEqual(server_result, expected, + msg % (server_result, "server")) def test_selected_npn_protocol(self): # selected_npn_protocol() is None unless NPN is used @@ -3295,13 +3312,23 @@ client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) client_context.verify_mode = ssl.CERT_REQUIRED client_context.load_verify_locations(SIGNING_CA) - client_context.set_ciphers("RC4") - server_context.set_ciphers("AES:RC4") + if ssl.OPENSSL_VERSION_INFO >= (1, 0, 2): + client_context.set_ciphers("AES128:AES256") + server_context.set_ciphers("AES256") + alg1 = "AES256" + alg2 = "AES-256" + else: + client_context.set_ciphers("AES:3DES") + server_context.set_ciphers("3DES") + alg1 = "3DES" + alg2 = "DES-CBC3" + stats = server_params_test(client_context, server_context) ciphers = stats['server_shared_ciphers'][0] self.assertGreater(len(ciphers), 0) for name, tls_version, bits in ciphers: - self.assertIn("RC4", name.split("-")) + if not alg1 in name.split("-") and alg2 not in name: + self.fail(name) def test_read_write_after_close_raises_valuerror(self): context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -75,6 +75,8 @@ Library ------- +- Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. + - Issue #11620: Fix support for SND_MEMORY in winsound.PlaySound. Based on a patch by Tim Lesher. diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -21,7 +21,6 @@ /* EVP is the preferred interface to hashing in OpenSSL */ #include -#include /* We use the object interface to discover what hashes OpenSSL supports. */ #include #include "openssl/err.h" @@ -32,11 +31,22 @@ #define HASH_OBJ_CONSTRUCTOR 0 #endif +#if (OPENSSL_VERSION_NUMBER < 0x10100000L) || defined(LIBRESSL_VERSION_NUMBER) +/* OpenSSL < 1.1.0 */ +#define EVP_MD_CTX_new EVP_MD_CTX_create +#define EVP_MD_CTX_free EVP_MD_CTX_destroy +#define HAS_FAST_PKCS5_PBKDF2_HMAC 0 +#include +#else +/* OpenSSL >= 1.1.0 */ +#define HAS_FAST_PKCS5_PBKDF2_HMAC 1 +#endif + typedef struct { PyObject_HEAD PyObject *name; /* name of this hash algorithm */ - EVP_MD_CTX ctx; /* OpenSSL message digest context */ + EVP_MD_CTX *ctx; /* OpenSSL message digest context */ #ifdef WITH_THREAD PyThread_type_lock lock; /* OpenSSL context lock */ #endif @@ -48,7 +58,6 @@ #define DEFINE_CONSTS_FOR_NEW(Name) \ static PyObject *CONST_ ## Name ## _name_obj = NULL; \ - static EVP_MD_CTX CONST_new_ ## Name ## _ctx; \ static EVP_MD_CTX *CONST_new_ ## Name ## _ctx_p = NULL; DEFINE_CONSTS_FOR_NEW(md5) @@ -59,19 +68,57 @@ DEFINE_CONSTS_FOR_NEW(sha512) +/* LCOV_EXCL_START */ +static PyObject * +_setException(PyObject *exc) +{ + unsigned long errcode; + const char *lib, *func, *reason; + + errcode = ERR_peek_last_error(); + if (!errcode) { + PyErr_SetString(exc, "unknown reasons"); + return NULL; + } + ERR_clear_error(); + + lib = ERR_lib_error_string(errcode); + func = ERR_func_error_string(errcode); + reason = ERR_reason_error_string(errcode); + + if (lib && func) { + PyErr_Format(exc, "[%s: %s] %s", lib, func, reason); + } + else if (lib) { + PyErr_Format(exc, "[%s] %s", lib, reason); + } + else { + PyErr_SetString(exc, reason); + } + return NULL; +} +/* LCOV_EXCL_STOP */ + static EVPobject * newEVPobject(PyObject *name) { EVPobject *retval = (EVPobject *)PyObject_New(EVPobject, &EVPtype); + if (retval == NULL) { + return NULL; + } + + retval->ctx = EVP_MD_CTX_new(); + if (retval->ctx == NULL) { + PyErr_NoMemory(); + return NULL; + } /* save the name for .name to return */ - if (retval != NULL) { - Py_INCREF(name); - retval->name = name; + Py_INCREF(name); + retval->name = name; #ifdef WITH_THREAD - retval->lock = NULL; + retval->lock = NULL; #endif - } return retval; } @@ -86,7 +133,7 @@ process = MUNCH_SIZE; else process = Py_SAFE_DOWNCAST(len, Py_ssize_t, unsigned int); - EVP_DigestUpdate(&self->ctx, (const void*)cp, process); + EVP_DigestUpdate(self->ctx, (const void*)cp, process); len -= process; cp += process; } @@ -101,16 +148,19 @@ if (self->lock != NULL) PyThread_free_lock(self->lock); #endif - EVP_MD_CTX_cleanup(&self->ctx); + EVP_MD_CTX_free(self->ctx); Py_XDECREF(self->name); PyObject_Del(self); } -static void locked_EVP_MD_CTX_copy(EVP_MD_CTX *new_ctx_p, EVPobject *self) +static int +locked_EVP_MD_CTX_copy(EVP_MD_CTX *new_ctx_p, EVPobject *self) { + int result; ENTER_HASHLIB(self); - EVP_MD_CTX_copy(new_ctx_p, &self->ctx); + result = EVP_MD_CTX_copy(new_ctx_p, self->ctx); LEAVE_HASHLIB(self); + return result; } /* External methods for a hash object */ @@ -126,7 +176,9 @@ if ( (newobj = newEVPobject(self->name))==NULL) return NULL; - locked_EVP_MD_CTX_copy(&newobj->ctx, self); + if (!locked_EVP_MD_CTX_copy(newobj->ctx, self)) { + return _setException(PyExc_ValueError); + } return (PyObject *)newobj; } @@ -137,16 +189,24 @@ EVP_digest(EVPobject *self, PyObject *unused) { unsigned char digest[EVP_MAX_MD_SIZE]; - EVP_MD_CTX temp_ctx; + EVP_MD_CTX *temp_ctx; PyObject *retval; unsigned int digest_size; - locked_EVP_MD_CTX_copy(&temp_ctx, self); - digest_size = EVP_MD_CTX_size(&temp_ctx); - EVP_DigestFinal(&temp_ctx, digest, NULL); + temp_ctx = EVP_MD_CTX_new(); + if (temp_ctx == NULL) { + PyErr_NoMemory(); + return NULL; + } + + if (!locked_EVP_MD_CTX_copy(temp_ctx, self)) { + return _setException(PyExc_ValueError); + } + digest_size = EVP_MD_CTX_size(temp_ctx); + EVP_DigestFinal(temp_ctx, digest, NULL); retval = PyBytes_FromStringAndSize((const char *)digest, digest_size); - EVP_MD_CTX_cleanup(&temp_ctx); + EVP_MD_CTX_free(temp_ctx); return retval; } @@ -157,15 +217,23 @@ EVP_hexdigest(EVPobject *self, PyObject *unused) { unsigned char digest[EVP_MAX_MD_SIZE]; - EVP_MD_CTX temp_ctx; + EVP_MD_CTX *temp_ctx; unsigned int digest_size; + temp_ctx = EVP_MD_CTX_new(); + if (temp_ctx == NULL) { + PyErr_NoMemory(); + return NULL; + } + /* Get the raw (binary) digest value */ - locked_EVP_MD_CTX_copy(&temp_ctx, self); - digest_size = EVP_MD_CTX_size(&temp_ctx); - EVP_DigestFinal(&temp_ctx, digest, NULL); + if (!locked_EVP_MD_CTX_copy(temp_ctx, self)) { + return _setException(PyExc_ValueError); + } + digest_size = EVP_MD_CTX_size(temp_ctx); + EVP_DigestFinal(temp_ctx, digest, NULL); - EVP_MD_CTX_cleanup(&temp_ctx); + EVP_MD_CTX_free(temp_ctx); return _Py_strhex((const char *)digest, digest_size); } @@ -219,7 +287,7 @@ EVP_get_block_size(EVPobject *self, void *closure) { long block_size; - block_size = EVP_MD_CTX_block_size(&self->ctx); + block_size = EVP_MD_CTX_block_size(self->ctx); return PyLong_FromLong(block_size); } @@ -227,7 +295,7 @@ EVP_get_digest_size(EVPobject *self, void *closure) { long size; - size = EVP_MD_CTX_size(&self->ctx); + size = EVP_MD_CTX_size(self->ctx); return PyLong_FromLong(size); } @@ -288,7 +356,7 @@ PyBuffer_Release(&view); return -1; } - EVP_DigestInit(&self->ctx, digest); + EVP_DigestInit(self->ctx, digest); self->name = name_obj; Py_INCREF(self->name); @@ -385,9 +453,9 @@ return NULL; if (initial_ctx) { - EVP_MD_CTX_copy(&self->ctx, initial_ctx); + EVP_MD_CTX_copy(self->ctx, initial_ctx); } else { - EVP_DigestInit(&self->ctx, digest); + EVP_DigestInit(self->ctx, digest); } if (cp && len) { @@ -453,6 +521,7 @@ #define PY_PBKDF2_HMAC 1 +#if !HAS_FAST_PKCS5_PBKDF2_HMAC /* Improved implementation of PKCS5_PBKDF2_HMAC() * * PKCS5_PBKDF2_HMAC_fast() hashes the password exactly one time instead of @@ -534,37 +603,8 @@ HMAC_CTX_cleanup(&hctx_tpl); return 1; } +#endif -/* LCOV_EXCL_START */ -static PyObject * -_setException(PyObject *exc) -{ - unsigned long errcode; - const char *lib, *func, *reason; - - errcode = ERR_peek_last_error(); - if (!errcode) { - PyErr_SetString(exc, "unknown reasons"); - return NULL; - } - ERR_clear_error(); - - lib = ERR_lib_error_string(errcode); - func = ERR_func_error_string(errcode); - reason = ERR_reason_error_string(errcode); - - if (lib && func) { - PyErr_Format(exc, "[%s: %s] %s", lib, func, reason); - } - else if (lib) { - PyErr_Format(exc, "[%s] %s", lib, reason); - } - else { - PyErr_SetString(exc, reason); - } - return NULL; -} -/* LCOV_EXCL_STOP */ PyDoc_STRVAR(pbkdf2_hmac__doc__, "pbkdf2_hmac(hash_name, password, salt, iterations, dklen=None) -> key\n\ @@ -646,10 +686,17 @@ key = PyBytes_AS_STRING(key_obj); Py_BEGIN_ALLOW_THREADS +#if HAS_FAST_PKCS5_PBKDF2_HMAC + retval = PKCS5_PBKDF2_HMAC((char*)password.buf, (int)password.len, + (unsigned char *)salt.buf, (int)salt.len, + iterations, digest, dklen, + (unsigned char *)key); +#else retval = PKCS5_PBKDF2_HMAC_fast((char*)password.buf, (int)password.len, (unsigned char *)salt.buf, (int)salt.len, iterations, digest, dklen, (unsigned char *)key); +#endif Py_END_ALLOW_THREADS if (!retval) { @@ -768,7 +815,7 @@ if (CONST_ ## NAME ## _name_obj == NULL) { \ CONST_ ## NAME ## _name_obj = PyUnicode_FromString(#NAME); \ if (EVP_get_digestbyname(#NAME)) { \ - CONST_new_ ## NAME ## _ctx_p = &CONST_new_ ## NAME ## _ctx; \ + CONST_new_ ## NAME ## _ctx_p = EVP_MD_CTX_new(); \ EVP_DigestInit(CONST_new_ ## NAME ## _ctx_p, EVP_get_digestbyname(#NAME)); \ } \ } \ diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -55,6 +55,14 @@ #include #endif +/* Don't warn about deprecated functions */ +#ifdef __GNUC__ +#pragma GCC diagnostic ignored "-Wdeprecated-declarations" +#endif +#ifdef __clang__ +#pragma clang diagnostic ignored "-Wdeprecated-declarations" +#endif + /* Include OpenSSL header files */ #include "openssl/rsa.h" #include "openssl/crypto.h" @@ -91,6 +99,10 @@ /* Include generated data (error codes) */ #include "_ssl_data.h" +#if (OPENSSL_VERSION_NUMBER >= 0x10100000L) && !defined(LIBRESSL_VERSION_NUMBER) +# define OPENSSL_VERSION_1_1 1 +#endif + /* Openssl comes with TLSv1.1 and TLSv1.2 between 1.0.0h and 1.0.1 http://www.openssl.org/news/changelog.html */ @@ -117,6 +129,72 @@ #define INVALID_SOCKET (-1) #endif +#ifdef OPENSSL_VERSION_1_1 +/* OpenSSL 1.1.0+ */ +#ifndef OPENSSL_NO_SSL2 +#define OPENSSL_NO_SSL2 +#endif +#else /* OpenSSL < 1.1.0 */ +#if defined(WITH_THREAD) +#define HAVE_OPENSSL_CRYPTO_LOCK +#endif + +#define TLS_method SSLv23_method + +static int X509_NAME_ENTRY_set(const X509_NAME_ENTRY *ne) +{ + return ne->set; +} + +#ifndef OPENSSL_NO_COMP +static int COMP_get_type(const COMP_METHOD *meth) +{ + return meth->type; +} + +static const char *COMP_get_name(const COMP_METHOD *meth) +{ + return meth->name; +} +#endif + +static pem_password_cb *SSL_CTX_get_default_passwd_cb(SSL_CTX *ctx) +{ + return ctx->default_passwd_callback; +} + +static void *SSL_CTX_get_default_passwd_cb_userdata(SSL_CTX *ctx) +{ + return ctx->default_passwd_callback_userdata; +} + +static int X509_OBJECT_get_type(X509_OBJECT *x) +{ + return x->type; +} + +static X509 *X509_OBJECT_get0_X509(X509_OBJECT *x) +{ + return x->data.x509; +} + +static int BIO_up_ref(BIO *b) +{ + CRYPTO_add(&b->references, 1, CRYPTO_LOCK_BIO); + return 1; +} + +static STACK_OF(X509_OBJECT) *X509_STORE_get0_objects(X509_STORE *store) { + return store->objs; +} + +static X509_VERIFY_PARAM *X509_STORE_get0_param(X509_STORE *store) +{ + return store->param; +} +#endif /* OpenSSL < 1.1.0 or LibreSSL */ + + enum py_ssl_error { /* these mirror ssl.h */ PY_SSL_ERROR_NONE, @@ -147,7 +225,7 @@ enum py_ssl_version { PY_SSL_VERSION_SSL2, PY_SSL_VERSION_SSL3=1, - PY_SSL_VERSION_SSL23, + PY_SSL_VERSION_TLS, #if HAVE_TLSv1_2 PY_SSL_VERSION_TLS1, PY_SSL_VERSION_TLS1_1, @@ -527,8 +605,8 @@ /* BIOs are reference counted and SSL_set_bio borrows our reference. * To prevent a double free in memory_bio_dealloc() we need to take an * extra reference here. */ - CRYPTO_add(&inbio->bio->references, 1, CRYPTO_LOCK_BIO); - CRYPTO_add(&outbio->bio->references, 1, CRYPTO_LOCK_BIO); + BIO_up_ref(inbio->bio); + BIO_up_ref(outbio->bio); SSL_set_bio(self->ssl, inbio->bio, outbio->bio); } mode = SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER; @@ -738,7 +816,7 @@ /* check to see if we've gotten to a new RDN */ if (rdn_level >= 0) { - if (rdn_level != entry->set) { + if (rdn_level != X509_NAME_ENTRY_set(entry)) { /* yes, new RDN */ /* add old RDN to DN */ rdnt = PyList_AsTuple(rdn); @@ -755,7 +833,7 @@ goto fail0; } } - rdn_level = entry->set; + rdn_level = X509_NAME_ENTRY_set(entry); /* now add this attribute to the current RDN */ name = X509_NAME_ENTRY_get_object(entry); @@ -853,18 +931,18 @@ goto fail; } - p = ext->value->data; + p = X509_EXTENSION_get_data(ext)->data; if (method->it) names = (GENERAL_NAMES*) (ASN1_item_d2i(NULL, &p, - ext->value->length, + X509_EXTENSION_get_data(ext)->length, ASN1_ITEM_ptr(method->it))); else names = (GENERAL_NAMES*) (method->d2i(NULL, &p, - ext->value->length)); + X509_EXTENSION_get_data(ext)->length)); for(j = 0; j < sk_GENERAL_NAME_num(names); j++) { /* get a rendering of each name in the set of names */ @@ -1075,13 +1153,11 @@ int i, j; PyObject *lst, *res = NULL; -#if OPENSSL_VERSION_NUMBER < 0x10001000L - dps = X509_get_ext_d2i(certificate, NID_crl_distribution_points, NULL, NULL); -#else +#if OPENSSL_VERSION_NUMBER >= 0x10001000L /* Calls x509v3_cache_extensions and sets up crldp */ X509_check_ca(certificate); - dps = certificate->crldp; #endif + dps = X509_get_ext_d2i(certificate, NID_crl_distribution_points, NULL, NULL); if (dps == NULL) return Py_None; @@ -1451,14 +1527,13 @@ _ssl__SSLSocket_shared_ciphers_impl(PySSLSocket *self) /*[clinic end generated code: output=3d174ead2e42c4fd input=0bfe149da8fe6306]*/ { - SSL_SESSION *sess = SSL_get_session(self->ssl); STACK_OF(SSL_CIPHER) *ciphers; int i; PyObject *res; - if (!sess || !sess->ciphers) + ciphers = SSL_get_ciphers(self->ssl); + if (!ciphers) Py_RETURN_NONE; - ciphers = sess->ciphers; res = PyList_New(sk_SSL_CIPHER_num(ciphers)); if (!res) return NULL; @@ -1567,9 +1642,9 @@ if (self->ssl == NULL) Py_RETURN_NONE; comp_method = SSL_get_current_compression(self->ssl); - if (comp_method == NULL || comp_method->type == NID_undef) + if (comp_method == NULL || COMP_get_type(comp_method) == NID_undef) Py_RETURN_NONE; - short_name = OBJ_nid2sn(comp_method->type); + short_name = COMP_get_name(comp_method); if (short_name == NULL) Py_RETURN_NONE; return PyUnicode_DecodeFSDefault(short_name); @@ -2255,8 +2330,8 @@ else if (proto_version == PY_SSL_VERSION_SSL2) ctx = SSL_CTX_new(SSLv2_method()); #endif - else if (proto_version == PY_SSL_VERSION_SSL23) - ctx = SSL_CTX_new(SSLv23_method()); + else if (proto_version == PY_SSL_VERSION_TLS) + ctx = SSL_CTX_new(TLS_method()); else proto_version = -1; PySSL_END_ALLOW_THREADS @@ -2318,8 +2393,9 @@ #ifndef OPENSSL_NO_ECDH /* Allow automatic ECDH curve selection (on OpenSSL 1.0.2+), or use prime256v1 by default. This is Apache mod_ssl's initialization - policy, so we should be safe. */ -#if defined(SSL_CTX_set_ecdh_auto) + policy, so we should be safe. OpenSSL 1.1 has it enabled by default. + */ +#if defined(SSL_CTX_set_ecdh_auto) && !defined(OPENSSL_VERSION_1_1) SSL_CTX_set_ecdh_auto(self->ctx, 1); #else { @@ -2586,10 +2662,12 @@ get_verify_flags(PySSLContext *self, void *c) { X509_STORE *store; + X509_VERIFY_PARAM *param; unsigned long flags; store = SSL_CTX_get_cert_store(self->ctx); - flags = X509_VERIFY_PARAM_get_flags(store->param); + param = X509_STORE_get0_param(store); + flags = X509_VERIFY_PARAM_get_flags(param); return PyLong_FromUnsignedLong(flags); } @@ -2597,22 +2675,24 @@ set_verify_flags(PySSLContext *self, PyObject *arg, void *c) { X509_STORE *store; + X509_VERIFY_PARAM *param; unsigned long new_flags, flags, set, clear; if (!PyArg_Parse(arg, "k", &new_flags)) return -1; store = SSL_CTX_get_cert_store(self->ctx); - flags = X509_VERIFY_PARAM_get_flags(store->param); + param = X509_STORE_get0_param(store); + flags = X509_VERIFY_PARAM_get_flags(param); clear = flags & ~new_flags; set = ~flags & new_flags; if (clear) { - if (!X509_VERIFY_PARAM_clear_flags(store->param, clear)) { + if (!X509_VERIFY_PARAM_clear_flags(param, clear)) { _setSSLError(NULL, 0, __FILE__, __LINE__); return -1; } } if (set) { - if (!X509_VERIFY_PARAM_set_flags(store->param, set)) { + if (!X509_VERIFY_PARAM_set_flags(param, set)) { _setSSLError(NULL, 0, __FILE__, __LINE__); return -1; } @@ -2789,8 +2869,8 @@ /*[clinic end generated code: output=9480bc1c380e2095 input=7cf9ac673cbee6fc]*/ { PyObject *certfile_bytes = NULL, *keyfile_bytes = NULL; - pem_password_cb *orig_passwd_cb = self->ctx->default_passwd_callback; - void *orig_passwd_userdata = self->ctx->default_passwd_callback_userdata; + pem_password_cb *orig_passwd_cb = SSL_CTX_get_default_passwd_cb(self->ctx); + void *orig_passwd_userdata = SSL_CTX_get_default_passwd_cb_userdata(self->ctx); _PySSLPasswordInfo pw_info = { NULL, NULL, NULL, 0, 0 }; int r; @@ -2917,8 +2997,9 @@ cert = d2i_X509_bio(biobuf, NULL); } else { cert = PEM_read_bio_X509(biobuf, NULL, - self->ctx->default_passwd_callback, - self->ctx->default_passwd_callback_userdata); + SSL_CTX_get_default_passwd_cb(self->ctx), + SSL_CTX_get_default_passwd_cb_userdata(self->ctx) + ); } if (cert == NULL) { break; @@ -3444,25 +3525,24 @@ /*[clinic end generated code: output=5f356f4d9cca874d input=eb40dd0f6d0e40cf]*/ { X509_STORE *store; + STACK_OF(X509_OBJECT) *objs; X509_OBJECT *obj; - int x509 = 0, crl = 0, pkey = 0, ca = 0, i; + int x509 = 0, crl = 0, ca = 0, i; store = SSL_CTX_get_cert_store(self->ctx); - for (i = 0; i < sk_X509_OBJECT_num(store->objs); i++) { - obj = sk_X509_OBJECT_value(store->objs, i); - switch (obj->type) { + objs = X509_STORE_get0_objects(store); + for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { + obj = sk_X509_OBJECT_value(objs, i); + switch (X509_OBJECT_get_type(obj)) { case X509_LU_X509: x509++; - if (X509_check_ca(obj->data.x509)) { + if (X509_check_ca(X509_OBJECT_get0_X509(obj))) { ca++; } break; case X509_LU_CRL: crl++; break; - case X509_LU_PKEY: - pkey++; - break; default: /* Ignore X509_LU_FAIL, X509_LU_RETRY, X509_LU_PKEY. * As far as I can tell they are internal states and never @@ -3492,6 +3572,7 @@ /*[clinic end generated code: output=0d58f148f37e2938 input=6887b5a09b7f9076]*/ { X509_STORE *store; + STACK_OF(X509_OBJECT) *objs; PyObject *ci = NULL, *rlist = NULL; int i; @@ -3500,17 +3581,18 @@ } store = SSL_CTX_get_cert_store(self->ctx); - for (i = 0; i < sk_X509_OBJECT_num(store->objs); i++) { + objs = X509_STORE_get0_objects(store); + for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { X509_OBJECT *obj; X509 *cert; - obj = sk_X509_OBJECT_value(store->objs, i); - if (obj->type != X509_LU_X509) { + obj = sk_X509_OBJECT_value(objs, i); + if (X509_OBJECT_get_type(obj) != X509_LU_X509) { /* not a x509 cert */ continue; } /* CA for any purpose */ - cert = obj->data.x509; + cert = X509_OBJECT_get0_X509(obj); if (!X509_check_ca(cert)) { continue; } @@ -4374,10 +4456,12 @@ }; -#ifdef WITH_THREAD +#ifdef HAVE_OPENSSL_CRYPTO_LOCK /* an implementation of OpenSSL threading operations in terms - of the Python C thread library */ + * of the Python C thread library + * Only used up to 1.0.2. OpenSSL 1.1.0+ has its own locking code. + */ static PyThread_type_lock *_ssl_locks = NULL; @@ -4458,7 +4542,7 @@ return 1; } -#endif /* def HAVE_THREAD */ +#endif /* HAVE_OPENSSL_CRYPTO_LOCK for WITH_THREAD && OpenSSL < 1.1.0 */ PyDoc_STRVAR(module_doc, "Implementation module for SSL socket operations. See the socket module\n\ @@ -4527,11 +4611,16 @@ SSL_load_error_strings(); SSL_library_init(); #ifdef WITH_THREAD +#ifdef HAVE_OPENSSL_CRYPTO_LOCK /* note that this will start threading if not already started */ if (!_setup_ssl_threads()) { return NULL; } +#elif OPENSSL_VERSION_1_1 && defined(OPENSSL_THREADS) + /* OpenSSL 1.1.0 builtin thread support is enabled */ + _ssl_locks_count++; #endif +#endif /* WITH_THREAD */ OpenSSL_add_all_algorithms(); /* Add symbols to module dict */ @@ -4678,7 +4767,9 @@ PY_SSL_VERSION_SSL3); #endif PyModule_AddIntConstant(m, "PROTOCOL_SSLv23", - PY_SSL_VERSION_SSL23); + PY_SSL_VERSION_TLS); + PyModule_AddIntConstant(m, "PROTOCOL_TLS", + PY_SSL_VERSION_TLS); PyModule_AddIntConstant(m, "PROTOCOL_TLSv1", PY_SSL_VERSION_TLS1); #if HAVE_TLSv1_2 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:52:01 2016 From: python-checkins at python.org (steve.dower) Date: Mon, 05 Sep 2016 21:52:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327756=3A_Updates_?= =?utf-8?q?installer_icons_to_be_the_console_and_launcher_icon?= Message-ID: <20160905215201.6696.6952.9FB9E410@psf.io> https://hg.python.org/cpython/rev/386aa9738c6b changeset: 103070:386aa9738c6b parent: 103068:bc5ba11973f5 user: Steve Dower date: Mon Sep 05 14:51:41 2016 -0700 summary: Issue #27756: Updates installer icons to be the console and launcher icon instead of the setup icon files: Tools/msi/common.wxs | 2 +- Tools/msi/launcher/launcher.wxs | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Tools/msi/common.wxs b/Tools/msi/common.wxs --- a/Tools/msi/common.wxs +++ b/Tools/msi/common.wxs @@ -44,7 +44,7 @@ - + diff --git a/Tools/msi/launcher/launcher.wxs b/Tools/msi/launcher/launcher.wxs --- a/Tools/msi/launcher/launcher.wxs +++ b/Tools/msi/launcher/launcher.wxs @@ -5,7 +5,10 @@ - + + + + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:53:50 2016 From: python-checkins at python.org (eric.snow) Date: Mon, 05 Sep 2016 21:53:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324254=3A_Preserve?= =?utf-8?q?_class_attribute_definition_order=2E?= Message-ID: <20160905215350.9581.16771.66C1289E@psf.io> https://hg.python.org/cpython/rev/635fd3912d4d changeset: 103071:635fd3912d4d user: Eric Snow date: Mon Sep 05 14:50:11 2016 -0700 summary: Issue #24254: Preserve class attribute definition order. files: Doc/library/inspect.rst | 367 +++++++++--------- Doc/library/types.rst | 12 + Doc/reference/compound_stmts.rst | 11 + Doc/reference/datamodel.rst | 9 +- Doc/whatsnew/3.6.rst | 26 + Include/object.h | 2 + Include/odictobject.h | 4 + Lib/test/test_builtin.py | 192 +++++++++- Lib/test/test_metaclass.py | 11 +- Lib/test/test_pydoc.py | 8 +- Lib/test/test_sys.py | 2 +- Lib/test/test_types.py | 22 + Lib/types.py | 5 +- Lib/typing.py | 1 + Misc/NEWS | 2 + Objects/odictobject.c | 15 + Objects/typeobject.c | 66 +++- Python/bltinmodule.c | 2 +- 18 files changed, 568 insertions(+), 189 deletions(-) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -34,185 +34,190 @@ They also help you determine when you can expect to find the following special attributes: -+-----------+-----------------+---------------------------+ -| Type | Attribute | Description | -+===========+=================+===========================+ -| module | __doc__ | documentation string | -+-----------+-----------------+---------------------------+ -| | __file__ | filename (missing for | -| | | built-in modules) | -+-----------+-----------------+---------------------------+ -| class | __doc__ | documentation string | -+-----------+-----------------+---------------------------+ -| | __name__ | name with which this | -| | | class was defined | -+-----------+-----------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+-----------------+---------------------------+ -| | __module__ | name of module in which | -| | | this class was defined | -+-----------+-----------------+---------------------------+ -| method | __doc__ | documentation string | -+-----------+-----------------+---------------------------+ -| | __name__ | name with which this | -| | | method was defined | -+-----------+-----------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+-----------------+---------------------------+ -| | __func__ | function object | -| | | containing implementation | -| | | of method | -+-----------+-----------------+---------------------------+ -| | __self__ | instance to which this | -| | | method is bound, or | -| | | ``None`` | -+-----------+-----------------+---------------------------+ -| function | __doc__ | documentation string | -+-----------+-----------------+---------------------------+ -| | __name__ | name with which this | -| | | function was defined | -+-----------+-----------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+-----------------+---------------------------+ -| | __code__ | code object containing | -| | | compiled function | -| | | :term:`bytecode` | -+-----------+-----------------+---------------------------+ -| | __defaults__ | tuple of any default | -| | | values for positional or | -| | | keyword parameters | -+-----------+-----------------+---------------------------+ -| | __kwdefaults__ | mapping of any default | -| | | values for keyword-only | -| | | parameters | -+-----------+-----------------+---------------------------+ -| | __globals__ | global namespace in which | -| | | this function was defined | -+-----------+-----------------+---------------------------+ -| | __annotations__ | mapping of parameters | -| | | names to annotations; | -| | | ``"return"`` key is | -| | | reserved for return | -| | | annotations. | -+-----------+-----------------+---------------------------+ -| traceback | tb_frame | frame object at this | -| | | level | -+-----------+-----------------+---------------------------+ -| | tb_lasti | index of last attempted | -| | | instruction in bytecode | -+-----------+-----------------+---------------------------+ -| | tb_lineno | current line number in | -| | | Python source code | -+-----------+-----------------+---------------------------+ -| | tb_next | next inner traceback | -| | | object (called by this | -| | | level) | -+-----------+-----------------+---------------------------+ -| frame | f_back | next outer frame object | -| | | (this frame's caller) | -+-----------+-----------------+---------------------------+ -| | f_builtins | builtins namespace seen | -| | | by this frame | -+-----------+-----------------+---------------------------+ -| | f_code | code object being | -| | | executed in this frame | -+-----------+-----------------+---------------------------+ -| | f_globals | global namespace seen by | -| | | this frame | -+-----------+-----------------+---------------------------+ -| | f_lasti | index of last attempted | -| | | instruction in bytecode | -+-----------+-----------------+---------------------------+ -| | f_lineno | current line number in | -| | | Python source code | -+-----------+-----------------+---------------------------+ -| | f_locals | local namespace seen by | -| | | this frame | -+-----------+-----------------+---------------------------+ -| | f_restricted | 0 or 1 if frame is in | -| | | restricted execution mode | -+-----------+-----------------+---------------------------+ -| | f_trace | tracing function for this | -| | | frame, or ``None`` | -+-----------+-----------------+---------------------------+ -| code | co_argcount | number of arguments (not | -| | | including \* or \*\* | -| | | args) | -+-----------+-----------------+---------------------------+ -| | co_code | string of raw compiled | -| | | bytecode | -+-----------+-----------------+---------------------------+ -| | co_consts | tuple of constants used | -| | | in the bytecode | -+-----------+-----------------+---------------------------+ -| | co_filename | name of file in which | -| | | this code object was | -| | | created | -+-----------+-----------------+---------------------------+ -| | co_firstlineno | number of first line in | -| | | Python source code | -+-----------+-----------------+---------------------------+ -| | co_flags | bitmap: 1=optimized ``|`` | -| | | 2=newlocals ``|`` 4=\*arg | -| | | ``|`` 8=\*\*arg | -+-----------+-----------------+---------------------------+ -| | co_lnotab | encoded mapping of line | -| | | numbers to bytecode | -| | | indices | -+-----------+-----------------+---------------------------+ -| | co_name | name with which this code | -| | | object was defined | -+-----------+-----------------+---------------------------+ -| | co_names | tuple of names of local | -| | | variables | -+-----------+-----------------+---------------------------+ -| | co_nlocals | number of local variables | -+-----------+-----------------+---------------------------+ -| | co_stacksize | virtual machine stack | -| | | space required | -+-----------+-----------------+---------------------------+ -| | co_varnames | tuple of names of | -| | | arguments and local | -| | | variables | -+-----------+-----------------+---------------------------+ -| generator | __name__ | name | -+-----------+-----------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+-----------------+---------------------------+ -| | gi_frame | frame | -+-----------+-----------------+---------------------------+ -| | gi_running | is the generator running? | -+-----------+-----------------+---------------------------+ -| | gi_code | code | -+-----------+-----------------+---------------------------+ -| | gi_yieldfrom | object being iterated by | -| | | ``yield from``, or | -| | | ``None`` | -+-----------+-----------------+---------------------------+ -| coroutine | __name__ | name | -+-----------+-----------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+-----------------+---------------------------+ -| | cr_await | object being awaited on, | -| | | or ``None`` | -+-----------+-----------------+---------------------------+ -| | cr_frame | frame | -+-----------+-----------------+---------------------------+ -| | cr_running | is the coroutine running? | -+-----------+-----------------+---------------------------+ -| | cr_code | code | -+-----------+-----------------+---------------------------+ -| builtin | __doc__ | documentation string | -+-----------+-----------------+---------------------------+ -| | __name__ | original name of this | -| | | function or method | -+-----------+-----------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+-----------------+---------------------------+ -| | __self__ | instance to which a | -| | | method is bound, or | -| | | ``None`` | -+-----------+-----------------+---------------------------+ ++-----------+----------------------+---------------------------+ +| Type | Attribute | Description | ++===========+======================+===========================+ +| module | __doc__ | documentation string | ++-----------+----------------------+---------------------------+ +| | __file__ | filename (missing for | +| | | built-in modules) | ++-----------+----------------------+---------------------------+ +| class | __doc__ | documentation string | ++-----------+----------------------+---------------------------+ +| | __name__ | name with which this | +| | | class was defined | ++-----------+----------------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+----------------------+---------------------------+ +| | __module__ | name of module in which | +| | | this class was defined | ++-----------+----------------------+---------------------------+ +| | __definition_order__ | the names of the class's | +| | | attributes, in the order | +| | | in which they were | +| | | defined (if known) | ++-----------+----------------------+---------------------------+ +| method | __doc__ | documentation string | ++-----------+----------------------+---------------------------+ +| | __name__ | name with which this | +| | | method was defined | ++-----------+----------------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+----------------------+---------------------------+ +| | __func__ | function object | +| | | containing implementation | +| | | of method | ++-----------+----------------------+---------------------------+ +| | __self__ | instance to which this | +| | | method is bound, or | +| | | ``None`` | ++-----------+----------------------+---------------------------+ +| function | __doc__ | documentation string | ++-----------+----------------------+---------------------------+ +| | __name__ | name with which this | +| | | function was defined | ++-----------+----------------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+----------------------+---------------------------+ +| | __code__ | code object containing | +| | | compiled function | +| | | :term:`bytecode` | ++-----------+----------------------+---------------------------+ +| | __defaults__ | tuple of any default | +| | | values for positional or | +| | | keyword parameters | ++-----------+----------------------+---------------------------+ +| | __kwdefaults__ | mapping of any default | +| | | values for keyword-only | +| | | parameters | ++-----------+----------------------+---------------------------+ +| | __globals__ | global namespace in which | +| | | this function was defined | ++-----------+----------------------+---------------------------+ +| | __annotations__ | mapping of parameters | +| | | names to annotations; | +| | | ``"return"`` key is | +| | | reserved for return | +| | | annotations. | ++-----------+----------------------+---------------------------+ +| traceback | tb_frame | frame object at this | +| | | level | ++-----------+----------------------+---------------------------+ +| | tb_lasti | index of last attempted | +| | | instruction in bytecode | ++-----------+----------------------+---------------------------+ +| | tb_lineno | current line number in | +| | | Python source code | ++-----------+----------------------+---------------------------+ +| | tb_next | next inner traceback | +| | | object (called by this | +| | | level) | ++-----------+----------------------+---------------------------+ +| frame | f_back | next outer frame object | +| | | (this frame's caller) | ++-----------+----------------------+---------------------------+ +| | f_builtins | builtins namespace seen | +| | | by this frame | ++-----------+----------------------+---------------------------+ +| | f_code | code object being | +| | | executed in this frame | ++-----------+----------------------+---------------------------+ +| | f_globals | global namespace seen by | +| | | this frame | ++-----------+----------------------+---------------------------+ +| | f_lasti | index of last attempted | +| | | instruction in bytecode | ++-----------+----------------------+---------------------------+ +| | f_lineno | current line number in | +| | | Python source code | ++-----------+----------------------+---------------------------+ +| | f_locals | local namespace seen by | +| | | this frame | ++-----------+----------------------+---------------------------+ +| | f_restricted | 0 or 1 if frame is in | +| | | restricted execution mode | ++-----------+----------------------+---------------------------+ +| | f_trace | tracing function for this | +| | | frame, or ``None`` | ++-----------+----------------------+---------------------------+ +| code | co_argcount | number of arguments (not | +| | | including \* or \*\* | +| | | args) | ++-----------+----------------------+---------------------------+ +| | co_code | string of raw compiled | +| | | bytecode | ++-----------+----------------------+---------------------------+ +| | co_consts | tuple of constants used | +| | | in the bytecode | ++-----------+----------------------+---------------------------+ +| | co_filename | name of file in which | +| | | this code object was | +| | | created | ++-----------+----------------------+---------------------------+ +| | co_firstlineno | number of first line in | +| | | Python source code | ++-----------+----------------------+---------------------------+ +| | co_flags | bitmap: 1=optimized ``|`` | +| | | 2=newlocals ``|`` 4=\*arg | +| | | ``|`` 8=\*\*arg | ++-----------+----------------------+---------------------------+ +| | co_lnotab | encoded mapping of line | +| | | numbers to bytecode | +| | | indices | ++-----------+----------------------+---------------------------+ +| | co_name | name with which this code | +| | | object was defined | ++-----------+----------------------+---------------------------+ +| | co_names | tuple of names of local | +| | | variables | ++-----------+----------------------+---------------------------+ +| | co_nlocals | number of local variables | ++-----------+----------------------+---------------------------+ +| | co_stacksize | virtual machine stack | +| | | space required | ++-----------+----------------------+---------------------------+ +| | co_varnames | tuple of names of | +| | | arguments and local | +| | | variables | ++-----------+----------------------+---------------------------+ +| generator | __name__ | name | ++-----------+----------------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+----------------------+---------------------------+ +| | gi_frame | frame | ++-----------+----------------------+---------------------------+ +| | gi_running | is the generator running? | ++-----------+----------------------+---------------------------+ +| | gi_code | code | ++-----------+----------------------+---------------------------+ +| | gi_yieldfrom | object being iterated by | +| | | ``yield from``, or | +| | | ``None`` | ++-----------+----------------------+---------------------------+ +| coroutine | __name__ | name | ++-----------+----------------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+----------------------+---------------------------+ +| | cr_await | object being awaited on, | +| | | or ``None`` | ++-----------+----------------------+---------------------------+ +| | cr_frame | frame | ++-----------+----------------------+---------------------------+ +| | cr_running | is the coroutine running? | ++-----------+----------------------+---------------------------+ +| | cr_code | code | ++-----------+----------------------+---------------------------+ +| builtin | __doc__ | documentation string | ++-----------+----------------------+---------------------------+ +| | __name__ | original name of this | +| | | function or method | ++-----------+----------------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+----------------------+---------------------------+ +| | __self__ | instance to which a | +| | | method is bound, or | +| | | ``None`` | ++-----------+----------------------+---------------------------+ .. versionchanged:: 3.5 @@ -221,6 +226,10 @@ The ``__name__`` attribute of generators is now set from the function name, instead of the code name, and it can now be modified. +.. versionchanged:: 3.6 + + Add ``__definition_order__`` to classes. + .. function:: getmembers(object[, predicate]) diff --git a/Doc/library/types.rst b/Doc/library/types.rst --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -53,8 +53,20 @@ in *kwds* argument with any ``'metaclass'`` entry removed. If no *kwds* argument is passed in, this will be an empty dict. + .. impl-detail:: + + CPython uses :class:`collections.OrderedDict` for the default + namespace. + .. versionadded:: 3.3 + .. versionchanged:: 3.6 + + The default value for the ``namespace`` element of the returned + tuple has changed from :func:`dict`. Now an insertion-order- + preserving mapping is used when the metaclass does not have a + ``__prepare__`` method, + .. seealso:: :ref:`metaclasses` diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -632,6 +632,17 @@ dictionary. The class name is bound to this class object in the original local namespace. +The order in which attributes are defined in the class body is preserved +in the ``__definition_order__`` attribute on the new class. If that order +is not known then the attribute is set to :const:`None`. The class body +may include a ``__definition_order__`` attribute. In that case it is used +directly. The value must be a tuple of identifiers or ``None``, otherwise +:exc:`TypeError` will be raised when the class statement is executed. + +.. versionchanged:: 3.6 + + Add ``__definition_order__`` to classes. + Class creation can be customized heavily using :ref:`metaclasses `. Classes can also be decorated: just like when decorating functions, :: diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1750,7 +1750,14 @@ additional keyword arguments, if any, come from the class definition). If the metaclass has no ``__prepare__`` attribute, then the class namespace -is initialised as an empty :func:`dict` instance. +is initialised as an empty ordered mapping. + +.. impl-detail:: + + In CPython the default is :class:`collections.OrderedDict`. + +.. versionchanged:: 3.6 + Defaults to :class:`collections.OrderedDict` instead of :func:`dict`. .. seealso:: diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -92,6 +92,7 @@ :pep:`4XX` - Python Virtual Environments PEP written by Carl Meyer +.. XXX PEP 520: :ref:`Preserving Class Attribute Definition Order` New Features ============ @@ -271,6 +272,31 @@ (Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.) +.. _whatsnew-deforder: + +PEP 520: Preserving Class Attribute Definition Order +---------------------------------------------------- + +Attributes in a class definition body have a natural ordering: the same +order in which the names appear in the source. This order is now +preserved in the new class's ``__definition_order__`` attribute. It is +a tuple of the attribute names, in the order in which they appear in +the class definition body. + +For types that don't have a definition (e.g. builtins), or the attribute +order could not be determined, ``__definition_order__`` is ``None``. + +Also, the effective default class *execution* namespace (returned from +``type.__prepare__()``) is now an insertion-order-preserving mapping. +For CPython, it is now ``collections.OrderedDict``. Note that the +class namespace, ``cls.__dict__``, is unchanged. + +.. seealso:: + + :pep:`520` - Preserving Class Attribute Definition Order + PEP written and implemented by Eric Snow. + + Other Language Changes ====================== diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -421,6 +421,8 @@ destructor tp_finalize; + PyObject *tp_deforder; + #ifdef COUNT_ALLOCS /* these must be last and never explicitly initialized */ Py_ssize_t tp_allocs; diff --git a/Include/odictobject.h b/Include/odictobject.h --- a/Include/odictobject.h +++ b/Include/odictobject.h @@ -28,6 +28,10 @@ PyAPI_FUNC(int) PyODict_SetItem(PyObject *od, PyObject *key, PyObject *item); PyAPI_FUNC(int) PyODict_DelItem(PyObject *od, PyObject *key); +#ifndef Py_LIMITED_API +PyAPI_FUNC(PyObject *) _PyODict_KeysAsTuple(PyObject *od); +#endif + /* wrappers around PyDict* functions */ #define PyODict_GetItem(od, key) PyDict_GetItem((PyObject *)od, key) #define PyODict_GetItemWithError(od, key) \ diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -16,8 +16,10 @@ import types import unittest import warnings +from collections import OrderedDict from operator import neg -from test.support import TESTFN, unlink, run_unittest, check_warnings +from test.support import (TESTFN, unlink, run_unittest, check_warnings, + cpython_only) from test.support.script_helper import assert_python_ok try: import pty, signal @@ -1778,6 +1780,194 @@ A.__doc__ = doc self.assertEqual(A.__doc__, doc) + def test_type_definition_order_nonempty(self): + class Spam: + b = 1 + c = 3 + a = 2 + d = 4 + eggs = 2 + e = 5 + b = 42 + + self.assertEqual(Spam.__definition_order__, + ('__module__', '__qualname__', + 'b', 'c', 'a', 'd', 'eggs', 'e')) + + def test_type_definition_order_empty(self): + class Empty: + pass + + self.assertEqual(Empty.__definition_order__, + ('__module__', '__qualname__')) + + def test_type_definition_order_on_instance(self): + class Spam: + a = 2 + b = 1 + c = 3 + with self.assertRaises(AttributeError): + Spam().__definition_order__ + + def test_type_definition_order_set_to_None(self): + class Spam: + a = 2 + b = 1 + c = 3 + Spam.__definition_order__ = None + self.assertEqual(Spam.__definition_order__, None) + + def test_type_definition_order_set_to_tuple(self): + class Spam: + a = 2 + b = 1 + c = 3 + Spam.__definition_order__ = ('x', 'y', 'z') + self.assertEqual(Spam.__definition_order__, ('x', 'y', 'z')) + + def test_type_definition_order_deleted(self): + class Spam: + a = 2 + b = 1 + c = 3 + del Spam.__definition_order__ + self.assertEqual(Spam.__definition_order__, None) + + def test_type_definition_order_set_to_bad_type(self): + class Spam: + a = 2 + b = 1 + c = 3 + Spam.__definition_order__ = 42 + self.assertEqual(Spam.__definition_order__, 42) + + def test_type_definition_order_builtins(self): + self.assertEqual(object.__definition_order__, None) + self.assertEqual(type.__definition_order__, None) + self.assertEqual(dict.__definition_order__, None) + self.assertEqual(type(None).__definition_order__, None) + + def test_type_definition_order_dunder_names_included(self): + class Dunder: + VAR = 3 + def __init__(self): + pass + + self.assertEqual(Dunder.__definition_order__, + ('__module__', '__qualname__', + 'VAR', '__init__')) + + def test_type_definition_order_only_dunder_names(self): + class DunderOnly: + __xyz__ = None + def __init__(self): + pass + + self.assertEqual(DunderOnly.__definition_order__, + ('__module__', '__qualname__', + '__xyz__', '__init__')) + + def test_type_definition_order_underscore_names(self): + class HalfDunder: + __whether_to_be = True + or_not_to_be__ = False + + self.assertEqual(HalfDunder.__definition_order__, + ('__module__', '__qualname__', + '_HalfDunder__whether_to_be', 'or_not_to_be__')) + + def test_type_definition_order_with_slots(self): + class Slots: + __slots__ = ('x', 'y') + a = 1 + b = 2 + + self.assertEqual(Slots.__definition_order__, + ('__module__', '__qualname__', + '__slots__', 'a', 'b')) + + def test_type_definition_order_overwritten_None(self): + class OverwrittenNone: + __definition_order__ = None + a = 1 + b = 2 + c = 3 + + self.assertEqual(OverwrittenNone.__definition_order__, None) + + def test_type_definition_order_overwritten_tuple(self): + class OverwrittenTuple: + __definition_order__ = ('x', 'y', 'z') + a = 1 + b = 2 + c = 3 + + self.assertEqual(OverwrittenTuple.__definition_order__, + ('x', 'y', 'z')) + + def test_type_definition_order_overwritten_bad_item(self): + with self.assertRaises(TypeError): + class PoorlyOverwritten: + __definition_order__ = ('a', 2, 'c') + a = 1 + b = 2 + c = 3 + + def test_type_definition_order_overwritten_bad_type(self): + with self.assertRaises(TypeError): + class PoorlyOverwritten: + __definition_order__ = ['a', 2, 'c'] + a = 1 + b = 2 + c = 3 + + def test_type_definition_order_metaclass(self): + class Meta(type): + SPAM = 42 + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + self.assertEqual(Meta.__definition_order__, + ('__module__', '__qualname__', + 'SPAM', '__init__')) + + def test_type_definition_order_OrderedDict(self): + class Meta(type): + def __prepare__(self, *args, **kwargs): + return OrderedDict() + + class WithODict(metaclass=Meta): + x='y' + + self.assertEqual(WithODict.__definition_order__, + ('__module__', '__qualname__', 'x')) + + class Meta(type): + def __prepare__(self, *args, **kwargs): + class ODictSub(OrderedDict): + pass + return ODictSub() + + class WithODictSub(metaclass=Meta): + x='y' + + self.assertEqual(WithODictSub.__definition_order__, + ('__module__', '__qualname__', 'x')) + + @cpython_only + def test_type_definition_order_cpython(self): + # some implementations will have an ordered-by-default dict. + + class Meta(type): + def __prepare__(self, *args, **kwargs): + return {} + + class NotOrdered(metaclass=Meta): + x='y' + + self.assertEqual(NotOrdered.__definition_order__, None) + def test_bad_args(self): with self.assertRaises(TypeError): type() diff --git a/Lib/test/test_metaclass.py b/Lib/test/test_metaclass.py --- a/Lib/test/test_metaclass.py +++ b/Lib/test/test_metaclass.py @@ -180,7 +180,7 @@ meta: C () ns: [('__module__', 'test.test_metaclass'), ('__qualname__', 'C'), ('a', 42), ('b', 24)] kw: [] - >>> type(C) is dict + >>> type(C) is types._DefaultClassNamespaceType True >>> print(sorted(C.items())) [('__module__', 'test.test_metaclass'), ('__qualname__', 'C'), ('a', 42), ('b', 24)] @@ -211,8 +211,11 @@ The default metaclass must define a __prepare__() method. - >>> type.__prepare__() - {} + >>> ns = type.__prepare__() + >>> type(ns) is types._DefaultClassNamespaceType + True + >>> list(ns) == [] + True >>> Make sure it works with subclassing. @@ -248,7 +251,9 @@ """ +from collections import OrderedDict import sys +import types # Trace function introduces __locals__ which causes various tests to fail. if hasattr(sys, 'gettrace') and sys.gettrace(): diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -427,6 +427,7 @@ expected_html = expected_html_pattern % ( (mod_url, mod_file, doc_loc) + expected_html_data_docstrings) + self.maxDiff = None self.assertEqual(result, expected_html) @unittest.skipIf(sys.flags.optimize >= 2, @@ -473,13 +474,18 @@ def test_non_str_name(self): # issue14638 # Treat illegal (non-str) name like no name + # Definition order is set to None so it looks the same in both + # cases. class A: + __definition_order__ = None __name__ = 42 class B: pass adoc = pydoc.render_doc(A()) bdoc = pydoc.render_doc(B()) - self.assertEqual(adoc.replace("A", "B"), bdoc) + self.maxDiff = None + expected = adoc.replace("A", "B") + self.assertEqual(bdoc, expected) def test_not_here(self): missing_module = "test.i_am_not_here" diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -1084,7 +1084,7 @@ check((1,2,3), vsize('') + 3*self.P) # type # static type: PyTypeObject - fmt = 'P2n15Pl4Pn9Pn11PIP' + fmt = 'P2n15Pl4Pn9Pn11PIPP' if hasattr(sys, 'getcounts'): fmt += '3n2P' s = vsize(fmt) diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -825,6 +825,28 @@ self.assertEqual(C.y, 1) self.assertEqual(C.z, 2) + def test_new_class_deforder(self): + C = types.new_class("C") + self.assertEqual(C.__definition_order__, tuple()) + + Meta = self.Meta + def func(ns): + ns["x"] = 0 + D = types.new_class("D", (), {"metaclass": Meta, "z": 2}, func) + self.assertEqual(D.__definition_order__, ('y', 'z', 'x')) + + def func(ns): + ns["__definition_order__"] = None + ns["x"] = 0 + D = types.new_class("D", (), {"metaclass": Meta, "z": 2}, func) + self.assertEqual(D.__definition_order__, None) + + def func(ns): + ns["__definition_order__"] = ('a', 'b', 'c') + ns["x"] = 0 + D = types.new_class("D", (), {"metaclass": Meta, "z": 2}, func) + self.assertEqual(D.__definition_order__, ('a', 'b', 'c')) + # Many of the following tests are derived from test_descr.py def test_prepare_class(self): # Basic test of metaclass derivation diff --git a/Lib/types.py b/Lib/types.py --- a/Lib/types.py +++ b/Lib/types.py @@ -25,8 +25,11 @@ _c.close() # Prevent ResourceWarning class _C: + _nsType = type(locals()) def _m(self): pass MethodType = type(_C()._m) +# In CPython, this should end up as OrderedDict. +_DefaultClassNamespaceType = _C._nsType BuiltinFunctionType = type(len) BuiltinMethodType = type([].append) # Same as BuiltinFunctionType @@ -85,7 +88,7 @@ if hasattr(meta, '__prepare__'): ns = meta.__prepare__(name, bases, **kwds) else: - ns = {} + ns = _DefaultClassNamespaceType() return meta, ns, kwds def _calculate_meta(meta, bases): diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -1301,6 +1301,7 @@ if (not attr.startswith('_abc_') and attr != '__abstractmethods__' and attr != '_is_protocol' and + attr != '__definition_order__' and attr != '__dict__' and attr != '__args__' and attr != '__slots__' and diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -49,6 +49,8 @@ potentially have caused off-by-one-ulp results on platforms with unreliable ldexp implementations. +- Issue #24254: Make class definition namespace ordered by default. + - Issue #27662: Fix an overflow check in ``List_New``: the original code was checking against ``Py_SIZE_MAX`` instead of the correct upper bound of ``Py_SSIZE_T_MAX``. Patch by Xiang Zhang. diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -1762,6 +1762,21 @@ return _PyDict_DelItem_KnownHash(od, key, hash); } +PyObject * +_PyODict_KeysAsTuple(PyObject *od) { + Py_ssize_t i = 0; + _ODictNode *node; + PyObject *keys = PyTuple_New(PyODict_Size(od)); + if (keys == NULL) + return NULL; + _odict_FOREACH((PyODictObject *)od, node) { + Py_INCREF(_odictnode_KEY(node)); + PyTuple_SET_ITEM(keys, i, _odictnode_KEY(node)); + i++; + } + return keys; +} + /* ------------------------------------------- * The OrderedDict views (keys/values/items) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -48,6 +48,7 @@ _Py_IDENTIFIER(__abstractmethods__); _Py_IDENTIFIER(__class__); _Py_IDENTIFIER(__delitem__); +_Py_IDENTIFIER(__definition_order__); _Py_IDENTIFIER(__dict__); _Py_IDENTIFIER(__doc__); _Py_IDENTIFIER(__getattribute__); @@ -489,6 +490,23 @@ } static PyObject * +type_deforder(PyTypeObject *type, void *context) +{ + if (type->tp_deforder == NULL) + Py_RETURN_NONE; + Py_INCREF(type->tp_deforder); + return type->tp_deforder; +} + +static int +type_set_deforder(PyTypeObject *type, PyObject *value, void *context) +{ + Py_XINCREF(value); + Py_XSETREF(type->tp_deforder, value); + return 0; +} + +static PyObject * type_abstractmethods(PyTypeObject *type, void *context) { PyObject *mod = NULL; @@ -834,6 +852,8 @@ {"__qualname__", (getter)type_qualname, (setter)type_set_qualname, NULL}, {"__bases__", (getter)type_get_bases, (setter)type_set_bases, NULL}, {"__module__", (getter)type_module, (setter)type_set_module, NULL}, + {"__definition_order__", (getter)type_deforder, + (setter)type_set_deforder, NULL}, {"__abstractmethods__", (getter)type_abstractmethods, (setter)type_set_abstractmethods, NULL}, {"__dict__", (getter)type_dict, NULL, NULL}, @@ -2351,6 +2371,7 @@ goto error; } + /* Copy the definition namespace into a new dict. */ dict = PyDict_Copy(orig_dict); if (dict == NULL) goto error; @@ -2559,6 +2580,48 @@ if (qualname != NULL && PyDict_DelItem(dict, PyId___qualname__.object) < 0) goto error; + /* Set tp_deforder to the extracted definition order, if any. */ + type->tp_deforder = _PyDict_GetItemId(dict, &PyId___definition_order__); + if (type->tp_deforder != NULL) { + Py_INCREF(type->tp_deforder); + + // Due to subclass lookup, __definition_order__ can't be in __dict__. + if (_PyDict_DelItemId(dict, &PyId___definition_order__) != 0) { + goto error; + } + + if (type->tp_deforder != Py_None) { + Py_ssize_t numnames; + + if (!PyTuple_Check(type->tp_deforder)) { + PyErr_SetString(PyExc_TypeError, + "__definition_order__ must be a tuple or None"); + goto error; + } + + // Make sure they are identifers. + numnames = PyTuple_Size(type->tp_deforder); + for (i = 0; i < numnames; i++) { + PyObject *name = PyTuple_GET_ITEM(type->tp_deforder, i); + if (name == NULL) { + goto error; + } + if (!PyUnicode_Check(name) || !PyUnicode_IsIdentifier(name)) { + PyErr_Format(PyExc_TypeError, + "__definition_order__ must " + "contain only identifiers, got '%s'", + name); + goto error; + } + } + } + } + else if (PyODict_Check(orig_dict)) { + type->tp_deforder = _PyODict_KeysAsTuple(orig_dict); + if (type->tp_deforder == NULL) + goto error; + } + /* Set tp_doc to a copy of dict['__doc__'], if the latter is there and is a string. The __doc__ accessor will first look for tp_doc; if that fails, it will still look into __dict__. @@ -3073,6 +3136,7 @@ Py_XDECREF(type->tp_mro); Py_XDECREF(type->tp_cache); Py_XDECREF(type->tp_subclasses); + Py_XDECREF(type->tp_deforder); /* A type's tp_doc is heap allocated, unlike the tp_doc slots * of most other objects. It's okay to cast it to char *. */ @@ -3115,7 +3179,7 @@ static PyObject * type_prepare(PyObject *self, PyObject *args, PyObject *kwds) { - return PyDict_New(); + return PyODict_New(); } /* diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -145,7 +145,7 @@ if (prep == NULL) { if (PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Clear(); - ns = PyDict_New(); + ns = PyODict_New(); } else { Py_DECREF(meta); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 17:55:04 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 21:55:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327744=3A_Add_AF?= =?utf-8?q?=5FALG_=28Linux_Kernel_crypto=29_to_socket_module=2E?= Message-ID: <20160905215504.6489.46116.C4553780@psf.io> https://hg.python.org/cpython/rev/74ce062a0397 changeset: 103072:74ce062a0397 user: Christian Heimes date: Mon Sep 05 23:54:41 2016 +0200 summary: Issue #27744: Add AF_ALG (Linux Kernel crypto) to socket module. files: Doc/library/socket.rst | 55 +++- Lib/test/test_socket.py | 165 ++++++++++ Misc/NEWS | 2 + Modules/socketmodule.c | 437 ++++++++++++++++++++++++--- configure | 49 ++- configure.ac | 13 + pyconfig.h.in | 3 + 7 files changed, 649 insertions(+), 75 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -131,6 +131,22 @@ string format. (ex. ``b'12:23:34:45:56:67'``) This protocol is not supported under FreeBSD. +- :const:`AF_ALG` is a Linux-only socket based interface to Kernel + cryptography. An algorithm socket is configured with a tuple of two to four + elements ``(type, name [, feat [, mask]])``, where: + + - *type* is the algorithm type as string, e.g. ``aead``, ``hash``, + ``skcipher`` or ``rng``. + + - *name* is the algorithm name and operation mode as string, e.g. + ``sha256``, ``hmac(sha256)``, ``cbc(aes)`` or ``drbg_nopr_ctr_aes256``. + + - *feat* and *mask* are unsigned 32bit integers. + + Availability Linux 2.6.38, some algorithm types require more recent Kernels. + + .. versionadded:: 3.6 + - Certain other address families (:const:`AF_PACKET`, :const:`AF_CAN`) support specific representations. @@ -350,6 +366,16 @@ TIPC related constants, matching the ones exported by the C socket API. See the TIPC documentation for more information. +.. data:: AF_ALG + SOL_ALG + ALG_* + + Constants for Linux Kernel cryptography. + + Availability: Linux >= 2.6.38. + + .. versionadded:: 3.6 + .. data:: AF_LINK Availability: BSD, OSX. @@ -1294,6 +1320,15 @@ an exception, the method now retries the system call instead of raising an :exc:`InterruptedError` exception (see :pep:`475` for the rationale). +.. method:: socket.sendmsg_afalg([msg], *, op[, iv[, assoclen[, flags]]]) + + Specialized version of :meth:`~socket.sendmsg` for :const:`AF_ALG` socket. + Set mode, IV, AEAD associated data length and flags for :const:`AF_ALG` socket. + + Availability: Linux >= 2.6.38 + + .. versionadded:: 3.6 + .. method:: socket.sendfile(file, offset=0, count=None) Send a file until EOF is reached by using high-performance @@ -1342,21 +1377,29 @@ For further information, please consult the :ref:`notes on socket timeouts `. -.. method:: socket.setsockopt(level, optname, value) +.. method:: socket.setsockopt(level, optname, value: int) +.. method:: socket.setsockopt(level, optname, value: buffer) +.. method:: socket.setsockopt(level, optname, None, optlen: int) .. index:: module: struct Set the value of the given socket option (see the Unix manual page :manpage:`setsockopt(2)`). The needed symbolic constants are defined in the - :mod:`socket` module (:const:`SO_\*` etc.). The value can be an integer or - a :term:`bytes-like object` representing a buffer. In the latter case it is - up to the caller to - ensure that the bytestring contains the proper bits (see the optional built-in - module :mod:`struct` for a way to encode C structures as bytestrings). + :mod:`socket` module (:const:`SO_\*` etc.). The value can be an integer, + None or a :term:`bytes-like object` representing a buffer. In the later + case it is up to the caller to ensure that the bytestring contains the + proper bits (see the optional built-in module :mod:`struct` for a way to + encode C structures as bytestrings). When value is set to None, + optlen argument is required. It's equivalent to call setsockopt C + function with optval=NULL and optlen=optlen. + .. versionchanged:: 3.5 Writable :term:`bytes-like object` is now accepted. + .. versionchanged:: 3.6 + setsockopt(level, optname, None, optlen: int) form added. + .. method:: socket.shutdown(how) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -5325,6 +5325,170 @@ def meth_from_sock(self, sock): return getattr(sock, "_sendfile_use_sendfile") + at unittest.skipUnless(hasattr(socket, "AF_ALG"), 'AF_ALG required') +class LinuxKernelCryptoAPI(unittest.TestCase): + # tests for AF_ALG + def create_alg(self, typ, name): + sock = socket.socket(socket.AF_ALG, socket.SOCK_SEQPACKET, 0) + sock.bind((typ, name)) + return sock + + def test_sha256(self): + expected = bytes.fromhex("ba7816bf8f01cfea414140de5dae2223b00361a396" + "177a9cb410ff61f20015ad") + with self.create_alg('hash', 'sha256') as algo: + op, _ = algo.accept() + with op: + op.sendall(b"abc") + self.assertEqual(op.recv(512), expected) + + op, _ = algo.accept() + with op: + op.send(b'a', socket.MSG_MORE) + op.send(b'b', socket.MSG_MORE) + op.send(b'c', socket.MSG_MORE) + op.send(b'') + self.assertEqual(op.recv(512), expected) + + def test_hmac_sha1(self): + expected = bytes.fromhex("effcdf6ae5eb2fa2d27416d5f184df9c259a7c79") + with self.create_alg('hash', 'hmac(sha1)') as algo: + algo.setsockopt(socket.SOL_ALG, socket.ALG_SET_KEY, b"Jefe") + op, _ = algo.accept() + with op: + op.sendall(b"what do ya want for nothing?") + self.assertEqual(op.recv(512), expected) + + def test_aes_cbc(self): + key = bytes.fromhex('06a9214036b8a15b512e03d534120006') + iv = bytes.fromhex('3dafba429d9eb430b422da802c9fac41') + msg = b"Single block msg" + ciphertext = bytes.fromhex('e353779c1079aeb82708942dbe77181a') + msglen = len(msg) + with self.create_alg('skcipher', 'cbc(aes)') as algo: + algo.setsockopt(socket.SOL_ALG, socket.ALG_SET_KEY, key) + op, _ = algo.accept() + with op: + op.sendmsg_afalg(op=socket.ALG_OP_ENCRYPT, iv=iv, + flags=socket.MSG_MORE) + op.sendall(msg) + self.assertEqual(op.recv(msglen), ciphertext) + + op, _ = algo.accept() + with op: + op.sendmsg_afalg([ciphertext], + op=socket.ALG_OP_DECRYPT, iv=iv) + self.assertEqual(op.recv(msglen), msg) + + # long message + multiplier = 1024 + longmsg = [msg] * multiplier + op, _ = algo.accept() + with op: + op.sendmsg_afalg(longmsg, + op=socket.ALG_OP_ENCRYPT, iv=iv) + enc = op.recv(msglen * multiplier) + self.assertEqual(len(enc), msglen * multiplier) + self.assertTrue(enc[:msglen], ciphertext) + + op, _ = algo.accept() + with op: + op.sendmsg_afalg([enc], + op=socket.ALG_OP_DECRYPT, iv=iv) + dec = op.recv(msglen * multiplier) + self.assertEqual(len(dec), msglen * multiplier) + self.assertEqual(dec, msg * multiplier) + + @support.requires_linux_version(3, 19) + def test_aead_aes_gcm(self): + key = bytes.fromhex('c939cc13397c1d37de6ae0e1cb7c423c') + iv = bytes.fromhex('b3d8cc017cbb89b39e0f67e2') + plain = bytes.fromhex('c3b3c41f113a31b73d9a5cd432103069') + assoc = bytes.fromhex('24825602bd12a984e0092d3e448eda5f') + expected_ct = bytes.fromhex('93fe7d9e9bfd10348a5606e5cafa7354') + expected_tag = bytes.fromhex('0032a1dc85f1c9786925a2e71d8272dd') + + taglen = len(expected_tag) + assoclen = len(assoc) + + with self.create_alg('aead', 'gcm(aes)') as algo: + algo.setsockopt(socket.SOL_ALG, socket.ALG_SET_KEY, key) + algo.setsockopt(socket.SOL_ALG, socket.ALG_SET_AEAD_AUTHSIZE, + None, taglen) + + # send assoc, plain and tag buffer in separate steps + op, _ = algo.accept() + with op: + op.sendmsg_afalg(op=socket.ALG_OP_ENCRYPT, iv=iv, + assoclen=assoclen, flags=socket.MSG_MORE) + op.sendall(assoc, socket.MSG_MORE) + op.sendall(plain, socket.MSG_MORE) + op.sendall(b'\x00' * taglen) + res = op.recv(assoclen + len(plain) + taglen) + self.assertEqual(expected_ct, res[assoclen:-taglen]) + self.assertEqual(expected_tag, res[-taglen:]) + + # now with msg + op, _ = algo.accept() + with op: + msg = assoc + plain + b'\x00' * taglen + op.sendmsg_afalg([msg], op=socket.ALG_OP_ENCRYPT, iv=iv, + assoclen=assoclen) + res = op.recv(assoclen + len(plain) + taglen) + self.assertEqual(expected_ct, res[assoclen:-taglen]) + self.assertEqual(expected_tag, res[-taglen:]) + + # create anc data manually + pack_uint32 = struct.Struct('I').pack + op, _ = algo.accept() + with op: + msg = assoc + plain + b'\x00' * taglen + op.sendmsg( + [msg], + ([socket.SOL_ALG, socket.ALG_SET_OP, pack_uint32(socket.ALG_OP_ENCRYPT)], + [socket.SOL_ALG, socket.ALG_SET_IV, pack_uint32(len(iv)) + iv], + [socket.SOL_ALG, socket.ALG_SET_AEAD_ASSOCLEN, pack_uint32(assoclen)], + ) + ) + res = op.recv(len(msg)) + self.assertEqual(expected_ct, res[assoclen:-taglen]) + self.assertEqual(expected_tag, res[-taglen:]) + + # decrypt and verify + op, _ = algo.accept() + with op: + msg = assoc + expected_ct + expected_tag + op.sendmsg_afalg([msg], op=socket.ALG_OP_DECRYPT, iv=iv, + assoclen=assoclen) + res = op.recv(len(msg)) + self.assertEqual(plain, res[assoclen:-taglen]) + + def test_drbg_pr_sha256(self): + # deterministic random bit generator, prediction resistance, sha256 + with self.create_alg('rng', 'drbg_pr_sha256') as algo: + extra_seed = os.urandom(32) + algo.setsockopt(socket.SOL_ALG, socket.ALG_SET_KEY, extra_seed) + op, _ = algo.accept() + with op: + rn = op.recv(32) + self.assertEqual(len(rn), 32) + + def test_sendmsg_afalg_args(self): + sock = socket.socket(socket.AF_ALG, socket.SOCK_SEQPACKET, 0) + with self.assertRaises(TypeError): + sock.sendmsg_afalg() + + with self.assertRaises(TypeError): + sock.sendmsg_afalg(op=None) + + with self.assertRaises(TypeError): + sock.sendmsg_afalg(1) + + with self.assertRaises(TypeError): + sock.sendmsg_afalg(op=socket.ALG_OP_ENCRYPT, assoclen=None) + + with self.assertRaises(TypeError): + sock.sendmsg_afalg(op=socket.ALG_OP_ENCRYPT, assoclen=-1) def test_main(): tests = [GeneralModuleTests, BasicTCPTest, TCPCloserTest, TCPTimeoutTest, @@ -5352,6 +5516,7 @@ tests.extend([TIPCTest, TIPCThreadableTest]) tests.extend([BasicCANTest, CANTest]) tests.extend([BasicRDSTest, RDSTest]) + tests.append(LinuxKernelCryptoAPI) tests.extend([ CmsgMacroTests, SendmsgUDPTest, diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -77,6 +77,8 @@ Library ------- +- Issue #27744: Add AF_ALG (Linux Kernel crypto) to socket module. + - Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. - Issue #11620: Fix support for SND_MEMORY in winsound.PlaySound. Based on a diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -136,7 +136,7 @@ send(data[, flags]) -- send data, may not send all of it\n\ sendto(data[, flags], addr) -- send data to a given address\n\ setblocking(0 | 1) -- set or clear the blocking I/O flag\n\ -setsockopt(level, optname, value) -- set socket options\n\ +setsockopt(level, optname, value[, optlen]) -- set socket options\n\ settimeout(None | float) -- set or clear the timeout\n\ shutdown(how) -- shut down traffic in one or both directions\n\ if_nameindex() -- return all network interface indices and names\n\ @@ -286,6 +286,36 @@ #include #endif +#ifdef HAVE_SOCKADDR_ALG +#include +#ifndef AF_ALG +#define AF_ALG 38 +#endif +#ifndef SOL_ALG +#define SOL_ALG 279 +#endif + +/* Linux 3.19 */ +#ifndef ALG_SET_AEAD_ASSOCLEN +#define ALG_SET_AEAD_ASSOCLEN 4 +#endif +#ifndef ALG_SET_AEAD_AUTHSIZE +#define ALG_SET_AEAD_AUTHSIZE 5 +#endif +/* Linux 4.8 */ +#ifndef ALG_SET_PUBKEY +#define ALG_SET_PUBKEY 6 +#endif + +#ifndef ALG_OP_SIGN +#define ALG_OP_SIGN 2 +#endif +#ifndef ALG_OP_VERIFY +#define ALG_OP_VERIFY 3 +#endif + +#endif /* HAVE_SOCKADDR_ALG */ + /* Generic socket object definitions and includes */ #define PySocket_BUILDING_SOCKET #include "socketmodule.h" @@ -1375,6 +1405,22 @@ } #endif +#ifdef HAVE_SOCKADDR_ALG + case AF_ALG: + { + struct sockaddr_alg *a = (struct sockaddr_alg *)addr; + return Py_BuildValue("s#s#HH", + a->salg_type, + strnlen((const char*)a->salg_type, + sizeof(a->salg_type)), + a->salg_name, + strnlen((const char*)a->salg_name, + sizeof(a->salg_name)), + a->salg_feat, + a->salg_mask); + } +#endif + /* More cases here... */ default: @@ -1940,6 +1986,36 @@ return 0; } #endif +#ifdef HAVE_SOCKADDR_ALG + case AF_ALG: + { + struct sockaddr_alg *sa; + char *type; + char *name; + sa = (struct sockaddr_alg *)addr_ret; + + memset(sa, 0, sizeof(*sa)); + sa->salg_family = AF_ALG; + + if (!PyArg_ParseTuple(args, "ss|HH:getsockaddrarg", + &type, &name, &sa->salg_feat, &sa->salg_mask)) + return 0; + /* sockaddr_alg has fixed-sized char arrays for type and name */ + if (strlen(type) > sizeof(sa->salg_type)) { + PyErr_SetString(PyExc_ValueError, "AF_ALG type too long."); + return 0; + } + strncpy((char *)sa->salg_type, type, sizeof(sa->salg_type)); + if (strlen(name) > sizeof(sa->salg_name)) { + PyErr_SetString(PyExc_ValueError, "AF_ALG name too long."); + return 0; + } + strncpy((char *)sa->salg_name, name, sizeof(sa->salg_name)); + + *len_ret = sizeof(*sa); + return 1; + } +#endif /* More cases here... */ @@ -2061,6 +2137,13 @@ return 0; } #endif +#ifdef HAVE_SOCKADDR_ALG + case AF_ALG: + { + *len_ret = sizeof (struct sockaddr_alg); + return 1; + } +#endif /* More cases here... */ @@ -2220,10 +2303,21 @@ sock_accept_impl(PySocketSockObject *s, void *data) { struct sock_accept *ctx = data; + struct sockaddr *addr = SAS2SA(ctx->addrbuf); + socklen_t *paddrlen = ctx->addrlen; +#ifdef HAVE_SOCKADDR_ALG + /* AF_ALG does not support accept() with addr and raises + * ECONNABORTED instead. */ + if (s->sock_family == AF_ALG) { + addr = NULL; + paddrlen = NULL; + *ctx->addrlen = 0; + } +#endif #if defined(HAVE_ACCEPT4) && defined(SOCK_CLOEXEC) if (accept4_works != 0) { - ctx->result = accept4(s->sock_fd, SAS2SA(ctx->addrbuf), ctx->addrlen, + ctx->result = accept4(s->sock_fd, addr, paddrlen, SOCK_CLOEXEC); if (ctx->result == INVALID_SOCKET && accept4_works == -1) { /* On Linux older than 2.6.28, accept4() fails with ENOSYS */ @@ -2231,9 +2325,9 @@ } } if (accept4_works == 0) - ctx->result = accept(s->sock_fd, SAS2SA(ctx->addrbuf), ctx->addrlen); + ctx->result = accept(s->sock_fd, addr, paddrlen); #else - ctx->result = accept(s->sock_fd, SAS2SA(ctx->addrbuf), ctx->addrlen); + ctx->result = accept(s->sock_fd, addr, paddrlen); #endif #ifdef MS_WINDOWS @@ -2435,9 +2529,12 @@ operations are disabled."); /* s.setsockopt() method. - With an integer third argument, sets an integer option. + With an integer third argument, sets an integer optval with optlen=4. + With None as third argument and an integer fourth argument, set + optval=NULL with unsigned int as optlen. With a string third argument, sets an option from a buffer; - use optional built-in module 'struct' to encode the string. */ + use optional built-in module 'struct' to encode the string. +*/ static PyObject * sock_setsockopt(PySocketSockObject *s, PyObject *args) @@ -2447,32 +2544,49 @@ int res; Py_buffer optval; int flag; - + unsigned int optlen; + PyObject *none; + + /* setsockopt(level, opt, flag) */ if (PyArg_ParseTuple(args, "iii:setsockopt", &level, &optname, &flag)) { res = setsockopt(s->sock_fd, level, optname, (char*)&flag, sizeof flag); - } - else { - PyErr_Clear(); - if (!PyArg_ParseTuple(args, "iiy*:setsockopt", - &level, &optname, &optval)) - return NULL; + goto done; + } + + PyErr_Clear(); + /* setsockopt(level, opt, (None, flag)) */ + if (PyArg_ParseTuple(args, "iiO!I:setsockopt", + &level, &optname, Py_TYPE(Py_None), &none, &optlen)) { + assert(sizeof(socklen_t) >= sizeof(unsigned int)); + res = setsockopt(s->sock_fd, level, optname, + NULL, (socklen_t)optlen); + goto done; + } + + PyErr_Clear(); + /* setsockopt(level, opt, buffer) */ + if (!PyArg_ParseTuple(args, "iiy*:setsockopt", + &level, &optname, &optval)) + return NULL; + #ifdef MS_WINDOWS - if (optval.len > INT_MAX) { - PyBuffer_Release(&optval); - PyErr_Format(PyExc_OverflowError, - "socket option is larger than %i bytes", - INT_MAX); - return NULL; - } - res = setsockopt(s->sock_fd, level, optname, - optval.buf, (int)optval.len); + if (optval.len > INT_MAX) { + PyBuffer_Release(&optval); + PyErr_Format(PyExc_OverflowError, + "socket option is larger than %i bytes", + INT_MAX); + return NULL; + } + res = setsockopt(s->sock_fd, level, optname, + optval.buf, (int)optval.len); #else - res = setsockopt(s->sock_fd, level, optname, optval.buf, optval.len); -#endif - PyBuffer_Release(&optval); - } + res = setsockopt(s->sock_fd, level, optname, optval.buf, optval.len); +#endif + PyBuffer_Release(&optval); + +done: if (res < 0) { return s->errorhandler(); } @@ -2481,10 +2595,13 @@ } PyDoc_STRVAR(setsockopt_doc, -"setsockopt(level, option, value)\n\ +"setsockopt(level, option, value: int)\n\ +setsockopt(level, option, value: buffer)\n\ +setsockopt(level, option, None, optlen: int)\n\ \n\ Set a socket option. See the Unix manual for level and option.\n\ -The value argument can either be an integer or a string."); +The value argument can either be an integer, a string buffer, or \n\ +None, optlen."); /* s.getsockopt() method. @@ -3774,6 +3891,51 @@ }; static int +sock_sendmsg_iovec(PySocketSockObject *s, PyObject *data_arg, + struct msghdr *msg, + Py_buffer **databufsout, Py_ssize_t *ndatabufsout) { + Py_ssize_t ndataparts, ndatabufs = 0; + int result = -1; + struct iovec *iovs = NULL; + PyObject *data_fast = NULL; + Py_buffer *databufs = NULL; + + /* Fill in an iovec for each message part, and save the Py_buffer + structs to release afterwards. */ + if ((data_fast = PySequence_Fast(data_arg, + "sendmsg() argument 1 must be an " + "iterable")) == NULL) + goto finally; + ndataparts = PySequence_Fast_GET_SIZE(data_fast); + if (ndataparts > INT_MAX) { + PyErr_SetString(PyExc_OSError, "sendmsg() argument 1 is too long"); + goto finally; + } + msg->msg_iovlen = ndataparts; + if (ndataparts > 0 && + ((msg->msg_iov = iovs = PyMem_New(struct iovec, ndataparts)) == NULL || + (databufs = PyMem_New(Py_buffer, ndataparts)) == NULL)) { + PyErr_NoMemory(); + goto finally; + } + for (; ndatabufs < ndataparts; ndatabufs++) { + if (!PyArg_Parse(PySequence_Fast_GET_ITEM(data_fast, ndatabufs), + "y*;sendmsg() argument 1 must be an iterable of " + "bytes-like objects", + &databufs[ndatabufs])) + goto finally; + iovs[ndatabufs].iov_base = databufs[ndatabufs].buf; + iovs[ndatabufs].iov_len = databufs[ndatabufs].len; + } + result = 0; + finally: + *databufsout = databufs; + *ndatabufsout = ndatabufs; + Py_XDECREF(data_fast); + return result; +} + +static int sock_sendmsg_impl(PySocketSockObject *s, void *data) { struct sock_sendmsg *ctx = data; @@ -3787,9 +3949,8 @@ static PyObject * sock_sendmsg(PySocketSockObject *s, PyObject *args) { - Py_ssize_t i, ndataparts, ndatabufs = 0, ncmsgs, ncmsgbufs = 0; + Py_ssize_t i, ndatabufs = 0, ncmsgs, ncmsgbufs = 0; Py_buffer *databufs = NULL; - struct iovec *iovs = NULL; sock_addr_t addrbuf; struct msghdr msg = {0}; struct cmsginfo { @@ -3800,7 +3961,7 @@ void *controlbuf = NULL; size_t controllen, controllen_last; int addrlen, flags = 0; - PyObject *data_arg, *cmsg_arg = NULL, *addr_arg = NULL, *data_fast = NULL, + PyObject *data_arg, *cmsg_arg = NULL, *addr_arg = NULL, *cmsg_fast = NULL, *retval = NULL; struct sock_sendmsg ctx; @@ -3818,30 +3979,8 @@ /* Fill in an iovec for each message part, and save the Py_buffer structs to release afterwards. */ - if ((data_fast = PySequence_Fast(data_arg, - "sendmsg() argument 1 must be an " - "iterable")) == NULL) + if (sock_sendmsg_iovec(s, data_arg, &msg, &databufs, &ndatabufs) == -1) { goto finally; - ndataparts = PySequence_Fast_GET_SIZE(data_fast); - if (ndataparts > INT_MAX) { - PyErr_SetString(PyExc_OSError, "sendmsg() argument 1 is too long"); - goto finally; - } - msg.msg_iovlen = ndataparts; - if (ndataparts > 0 && - ((msg.msg_iov = iovs = PyMem_New(struct iovec, ndataparts)) == NULL || - (databufs = PyMem_New(Py_buffer, ndataparts)) == NULL)) { - PyErr_NoMemory(); - goto finally; - } - for (; ndatabufs < ndataparts; ndatabufs++) { - if (!PyArg_Parse(PySequence_Fast_GET_ITEM(data_fast, ndatabufs), - "y*;sendmsg() argument 1 must be an iterable of " - "bytes-like objects", - &databufs[ndatabufs])) - goto finally; - iovs[ndatabufs].iov_base = databufs[ndatabufs].buf; - iovs[ndatabufs].iov_len = databufs[ndatabufs].len; } if (cmsg_arg == NULL) @@ -3972,8 +4111,6 @@ for (i = 0; i < ndatabufs; i++) PyBuffer_Release(&databufs[i]); PyMem_Free(databufs); - PyMem_Free(iovs); - Py_XDECREF(data_fast); return retval; } @@ -3995,6 +4132,165 @@ data sent."); #endif /* CMSG_LEN */ +#ifdef HAVE_SOCKADDR_ALG +static PyObject* +sock_sendmsg_afalg(PySocketSockObject *self, PyObject *args, PyObject *kwds) +{ + PyObject *retval = NULL; + + Py_ssize_t i, ndatabufs = 0; + Py_buffer *databufs = NULL; + PyObject *data_arg = NULL; + + Py_buffer iv = {NULL, NULL}; + + PyObject *opobj = NULL; + int op = -1; + + PyObject *assoclenobj = NULL; + int assoclen = -1; + + unsigned int *uiptr; + int flags = 0; + + struct msghdr msg; + struct cmsghdr *header = NULL; + struct af_alg_iv *alg_iv = NULL; + struct sock_sendmsg ctx; + Py_ssize_t controllen; + void *controlbuf = NULL; + static char *keywords[] = {"msg", "op", "iv", "assoclen", "flags", 0}; + + if (self->sock_family != AF_ALG) { + PyErr_SetString(PyExc_OSError, + "algset is only supported for AF_ALG"); + return NULL; + } + + if (!PyArg_ParseTupleAndKeywords(args, kwds, + "|O$O!y*O!i:sendmsg_afalg", keywords, + &data_arg, + &PyLong_Type, &opobj, &iv, + &PyLong_Type, &assoclenobj, &flags)) + return NULL; + + /* op is a required, keyword-only argument >= 0 */ + if (opobj != NULL) { + op = _PyLong_AsInt(opobj); + } + if (op < 0) { + /* override exception from _PyLong_AsInt() */ + PyErr_SetString(PyExc_TypeError, + "Invalid or missing argument 'op'"); + goto finally; + } + /* assoclen is optional but must be >= 0 */ + if (assoclenobj != NULL) { + assoclen = _PyLong_AsInt(assoclenobj); + if (assoclen == -1 && PyErr_Occurred()) { + goto finally; + } + if (assoclen < 0) { + PyErr_SetString(PyExc_TypeError, + "assoclen must be positive"); + goto finally; + } + } + + controllen = CMSG_SPACE(4); + if (iv.buf != NULL) { + controllen += CMSG_SPACE(sizeof(*alg_iv) + iv.len); + } + if (assoclen >= 0) { + controllen += CMSG_SPACE(4); + } + + controlbuf = PyMem_Malloc(controllen); + if (controlbuf == NULL) { + return PyErr_NoMemory(); + } + memset(controlbuf, 0, controllen); + + memset(&msg, 0, sizeof(msg)); + msg.msg_controllen = controllen; + msg.msg_control = controlbuf; + + /* Fill in an iovec for each message part, and save the Py_buffer + structs to release afterwards. */ + if (data_arg != NULL) { + if (sock_sendmsg_iovec(self, data_arg, &msg, &databufs, &ndatabufs) == -1) { + goto finally; + } + } + + /* set operation to encrypt or decrypt */ + header = CMSG_FIRSTHDR(&msg); + if (header == NULL) { + PyErr_SetString(PyExc_RuntimeError, + "unexpected NULL result from CMSG_FIRSTHDR"); + goto finally; + } + header->cmsg_level = SOL_ALG; + header->cmsg_type = ALG_SET_OP; + header->cmsg_len = CMSG_LEN(4); + uiptr = (void*)CMSG_DATA(header); + *uiptr = (unsigned int)op; + + /* set initialization vector */ + if (iv.buf != NULL) { + header = CMSG_NXTHDR(&msg, header); + if (header == NULL) { + PyErr_SetString(PyExc_RuntimeError, + "unexpected NULL result from CMSG_NXTHDR(iv)"); + goto finally; + } + header->cmsg_level = SOL_ALG; + header->cmsg_type = ALG_SET_IV; + header->cmsg_len = CMSG_SPACE(sizeof(*alg_iv) + iv.len); + alg_iv = (void*)CMSG_DATA(header); + alg_iv->ivlen = iv.len; + memcpy(alg_iv->iv, iv.buf, iv.len); + } + + /* set length of associated data for AEAD */ + if (assoclen >= 0) { + header = CMSG_NXTHDR(&msg, header); + if (header == NULL) { + PyErr_SetString(PyExc_RuntimeError, + "unexpected NULL result from CMSG_NXTHDR(assoc)"); + goto finally; + } + header->cmsg_level = SOL_ALG; + header->cmsg_type = ALG_SET_AEAD_ASSOCLEN; + header->cmsg_len = CMSG_LEN(4); + uiptr = (void*)CMSG_DATA(header); + *uiptr = (unsigned int)assoclen; + } + + ctx.msg = &msg; + ctx.flags = flags; + if (sock_call(self, 1, sock_sendmsg_impl, &ctx) < 0) + goto finally; + + retval = PyLong_FromSsize_t(ctx.result); + + finally: + PyMem_Free(controlbuf); + if (iv.buf != NULL) { + PyBuffer_Release(&iv); + } + for (i = 0; i < ndatabufs; i++) + PyBuffer_Release(&databufs[i]); + PyMem_Free(databufs); + return retval; +} + +PyDoc_STRVAR(sendmsg_afalg_doc, +"sendmsg_afalg([msg], *, op[, iv[, assoclen[, flags=MSG_MORE]]])\n\ +\n\ +Set operation mode, IV and length of associated data for an AF_ALG\n\ +operation socket."); +#endif /* s.shutdown(how) method */ @@ -4174,6 +4470,10 @@ {"sendmsg", (PyCFunction)sock_sendmsg, METH_VARARGS, sendmsg_doc}, #endif +#ifdef HAVE_SOCKADDR_ALG + {"sendmsg_afalg", (PyCFunction)sock_sendmsg_afalg, METH_VARARGS | METH_KEYWORDS, + sendmsg_afalg_doc}, +#endif {NULL, NULL} /* sentinel */ }; @@ -6277,6 +6577,9 @@ /* Reserved for Werner's ATM */ PyModule_AddIntMacro(m, AF_AAL5); #endif +#ifdef HAVE_SOCKADDR_ALG + PyModule_AddIntMacro(m, AF_ALG); /* Linux crypto */ +#endif #ifdef AF_X25 /* Reserved for X.25 project */ PyModule_AddIntMacro(m, AF_X25); @@ -6338,6 +6641,9 @@ #ifdef NETLINK_TAPBASE PyModule_AddIntMacro(m, NETLINK_TAPBASE); #endif +#ifdef NETLINK_CRYPTO + PyModule_AddIntMacro(m, NETLINK_CRYPTO); +#endif #endif /* AF_NETLINK */ #ifdef AF_ROUTE /* Alias to emulate 4.4BSD */ @@ -6491,6 +6797,22 @@ PyModule_AddIntMacro(m, TIPC_TOP_SRV); #endif +#ifdef HAVE_SOCKADDR_ALG + /* Socket options */ + PyModule_AddIntMacro(m, ALG_SET_KEY); + PyModule_AddIntMacro(m, ALG_SET_IV); + PyModule_AddIntMacro(m, ALG_SET_OP); + PyModule_AddIntMacro(m, ALG_SET_AEAD_ASSOCLEN); + PyModule_AddIntMacro(m, ALG_SET_AEAD_AUTHSIZE); + PyModule_AddIntMacro(m, ALG_SET_PUBKEY); + + /* Operations */ + PyModule_AddIntMacro(m, ALG_OP_DECRYPT); + PyModule_AddIntMacro(m, ALG_OP_ENCRYPT); + PyModule_AddIntMacro(m, ALG_OP_SIGN); + PyModule_AddIntMacro(m, ALG_OP_VERIFY); +#endif + /* Socket types */ PyModule_AddIntMacro(m, SOCK_STREAM); PyModule_AddIntMacro(m, SOCK_DGRAM); @@ -6761,6 +7083,9 @@ #ifdef SOL_RDS PyModule_AddIntMacro(m, SOL_RDS); #endif +#ifdef HAVE_SOCKADDR_ALG + PyModule_AddIntMacro(m, SOL_ALG); +#endif #ifdef RDS_CANCEL_SENT_TO PyModule_AddIntMacro(m, RDS_CANCEL_SENT_TO); #endif diff --git a/configure b/configure --- a/configure +++ b/configure @@ -775,7 +775,6 @@ docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir @@ -886,7 +885,6 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1139,15 +1137,6 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1285,7 +1274,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1438,7 +1427,6 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -13062,6 +13050,41 @@ fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for sockaddr_alg" >&5 +$as_echo_n "checking for sockaddr_alg... " >&6; } +if ${ac_cv_struct_sockaddr_alg+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +# include +# include +# include +int +main () +{ +struct sockaddr_alg s + ; + return 0; +} +_ACEOF +if ac_fn_c_try_compile "$LINENO"; then : + ac_cv_struct_sockaddr_alg=yes +else + ac_cv_struct_sockaddr_alg=no +fi +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext +fi + +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_struct_sockaddr_alg" >&5 +$as_echo "$ac_cv_struct_sockaddr_alg" >&6; } +if test $ac_cv_struct_sockaddr_alg = yes; then + +$as_echo "#define HAVE_SOCKADDR_ALG 1" >>confdefs.h + +fi + # checks for compiler characteristics { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether char is unsigned" >&5 diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3862,6 +3862,19 @@ AC_DEFINE(HAVE_SOCKADDR_STORAGE, 1, [struct sockaddr_storage (sys/socket.h)]) fi +AC_MSG_CHECKING(for sockaddr_alg) +AC_CACHE_VAL(ac_cv_struct_sockaddr_alg, +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +# include +# include +# include ]], [[struct sockaddr_alg s]])], + [ac_cv_struct_sockaddr_alg=yes], + [ac_cv_struct_sockaddr_alg=no])) +AC_MSG_RESULT($ac_cv_struct_sockaddr_alg) +if test $ac_cv_struct_sockaddr_alg = yes; then + AC_DEFINE(HAVE_SOCKADDR_ALG, 1, [struct sockaddr_alg (linux/if_alg.h)]) +fi + # checks for compiler characteristics AC_C_CHAR_UNSIGNED diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -892,6 +892,9 @@ /* Define to 1 if you have the `snprintf' function. */ #undef HAVE_SNPRINTF +/* struct sockaddr_alg (linux/if_alg.h) */ +#undef HAVE_SOCKADDR_ALG + /* Define if sockaddr has sa_len member */ #undef HAVE_SOCKADDR_SA_LEN -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:05:04 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 22:05:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327866=3A_Add_SSLC?= =?utf-8?q?ontext=2Eget=5Fciphers=28=29_method_to_get_a_list_of_all_enable?= =?utf-8?q?d?= Message-ID: <20160905220503.6601.20749.3450EF40@psf.io> https://hg.python.org/cpython/rev/ca8d7cb55a8e changeset: 103073:ca8d7cb55a8e user: Christian Heimes date: Tue Sep 06 00:04:45 2016 +0200 summary: Issue #27866: Add SSLContext.get_ciphers() method to get a list of all enabled ciphers. files: Doc/library/ssl.rst | 56 +++++++++++++ Lib/test/test_ssl.py | 9 ++ Misc/NEWS | 3 + Modules/_ssl.c | 117 ++++++++++++++++++++++++++++ Modules/clinic/_ssl.c.h | 27 ++++++- 5 files changed, 211 insertions(+), 1 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1259,6 +1259,62 @@ .. versionadded:: 3.4 +.. method:: SSLContext.get_ciphers() + + Get a list of enabled ciphers. The list is in order of cipher priority. + See :meth:`SSLContext.set_ciphers`. + + Example:: + + >>> ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) + >>> ctx.set_ciphers('ECDHE+AESGCM:!ECDSA') + >>> ctx.get_ciphers() # OpenSSL 1.0.x + [{'alg_bits': 256, + 'description': 'ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA ' + 'Enc=AESGCM(256) Mac=AEAD', + 'id': 50380848, + 'name': 'ECDHE-RSA-AES256-GCM-SHA384', + 'protocol': 'TLSv1/SSLv3', + 'strength_bits': 256}, + {'alg_bits': 128, + 'description': 'ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA ' + 'Enc=AESGCM(128) Mac=AEAD', + 'id': 50380847, + 'name': 'ECDHE-RSA-AES128-GCM-SHA256', + 'protocol': 'TLSv1/SSLv3', + 'strength_bits': 128}] + + On OpenSSL 1.1 and newer the cipher dict contains additional fields:: + >>> ctx.get_ciphers() # OpenSSL 1.1+ + [{'aead': True, + 'alg_bits': 256, + 'auth': 'auth-rsa', + 'description': 'ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA ' + 'Enc=AESGCM(256) Mac=AEAD', + 'digest': None, + 'id': 50380848, + 'kea': 'kx-ecdhe', + 'name': 'ECDHE-RSA-AES256-GCM-SHA384', + 'protocol': 'TLSv1.2', + 'strength_bits': 256, + 'symmetric': 'aes-256-gcm'}, + {'aead': True, + 'alg_bits': 128, + 'auth': 'auth-rsa', + 'description': 'ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA ' + 'Enc=AESGCM(128) Mac=AEAD', + 'digest': None, + 'id': 50380847, + 'kea': 'kx-ecdhe', + 'name': 'ECDHE-RSA-AES128-GCM-SHA256', + 'protocol': 'TLSv1.2', + 'strength_bits': 128, + 'symmetric': 'aes-128-gcm'}] + + Availability: OpenSSL 1.0.2+ + + .. versionadded:: 3.6 + .. method:: SSLContext.set_default_verify_paths() Load a set of default "certification authority" (CA) certificates from diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -834,6 +834,15 @@ with self.assertRaisesRegex(ssl.SSLError, "No cipher can be selected"): ctx.set_ciphers("^$:,;?*'dorothyx") + @unittest.skipIf(ssl.OPENSSL_VERSION_INFO < (1, 0, 2, 0, 0), 'OpenSSL too old') + def test_get_ciphers(self): + ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + ctx.set_ciphers('ECDHE+AESGCM:!ECDSA') + names = set(d['name'] for d in ctx.get_ciphers()) + self.assertEqual(names, + {'ECDHE-RSA-AES256-GCM-SHA384', + 'ECDHE-RSA-AES128-GCM-SHA256'}) + @skip_if_broken_ubuntu_ssl def test_options(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -77,6 +77,9 @@ Library ------- +- Issue #27866: Add SSLContext.get_ciphers() method to get a list of all + enabled ciphers. + - Issue #27744: Add AF_ALG (Linux Kernel crypto) to socket module. - Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1519,6 +1519,76 @@ return NULL; } +#if OPENSSL_VERSION_NUMBER >= 0x10002000UL +static PyObject * +cipher_to_dict(const SSL_CIPHER *cipher) +{ + const char *cipher_name, *cipher_protocol; + + unsigned long cipher_id; + int alg_bits, strength_bits, len; + char buf[512] = {0}; +#if OPENSSL_VERSION_1_1 + int aead, nid; + const char *skcipher = NULL, *digest = NULL, *kx = NULL, *auth = NULL; +#endif + PyObject *retval; + + retval = PyDict_New(); + if (retval == NULL) { + goto error; + } + + /* can be NULL */ + cipher_name = SSL_CIPHER_get_name(cipher); + cipher_protocol = SSL_CIPHER_get_version(cipher); + cipher_id = SSL_CIPHER_get_id(cipher); + SSL_CIPHER_description(cipher, buf, sizeof(buf) - 1); + len = strlen(buf); + if (len > 1 && buf[len-1] == '\n') + buf[len-1] = '\0'; + strength_bits = SSL_CIPHER_get_bits(cipher, &alg_bits); + +#if OPENSSL_VERSION_1_1 + aead = SSL_CIPHER_is_aead(cipher); + nid = SSL_CIPHER_get_cipher_nid(cipher); + skcipher = nid != NID_undef ? OBJ_nid2ln(nid) : NULL; + nid = SSL_CIPHER_get_digest_nid(cipher); + digest = nid != NID_undef ? OBJ_nid2ln(nid) : NULL; + nid = SSL_CIPHER_get_kx_nid(cipher); + kx = nid != NID_undef ? OBJ_nid2ln(nid) : NULL; + nid = SSL_CIPHER_get_auth_nid(cipher); + auth = nid != NID_undef ? OBJ_nid2ln(nid) : NULL; +#endif + + retval = Py_BuildValue( + "{sksssssssisi" +#if OPENSSL_VERSION_1_1 + "sOssssssss" +#endif + "}", + "id", cipher_id, + "name", cipher_name, + "protocol", cipher_protocol, + "description", buf, + "strength_bits", strength_bits, + "alg_bits", alg_bits +#if OPENSSL_VERSION_1_1 + ,"aead", aead ? Py_True : Py_False, + "symmetric", skcipher, + "digest", digest, + "kea", kx, + "auth", auth +#endif + ); + return retval; + + error: + Py_XDECREF(retval); + return NULL; +} +#endif + /*[clinic input] _ssl._SSLSocket.shared_ciphers [clinic start generated code]*/ @@ -2478,6 +2548,52 @@ Py_RETURN_NONE; } +#if OPENSSL_VERSION_NUMBER >= 0x10002000UL +/*[clinic input] +_ssl._SSLContext.get_ciphers +[clinic start generated code]*/ + +static PyObject * +_ssl__SSLContext_get_ciphers_impl(PySSLContext *self) +/*[clinic end generated code: output=a56e4d68a406dfc4 input=a2aadc9af89b79c5]*/ +{ + SSL *ssl = NULL; + STACK_OF(SSL_CIPHER) *sk = NULL; + SSL_CIPHER *cipher; + int i=0; + PyObject *result = NULL, *dct; + + ssl = SSL_new(self->ctx); + if (ssl == NULL) { + _setSSLError(NULL, 0, __FILE__, __LINE__); + goto exit; + } + sk = SSL_get_ciphers(ssl); + + result = PyList_New(sk_SSL_CIPHER_num(sk)); + if (result == NULL) { + goto exit; + } + + for (i = 0; i < sk_SSL_CIPHER_num(sk); i++) { + cipher = sk_SSL_CIPHER_value(sk, i); + dct = cipher_to_dict(cipher); + if (dct == NULL) { + Py_CLEAR(result); + goto exit; + } + PyList_SET_ITEM(result, i, dct); + } + + exit: + if (ssl != NULL) + SSL_free(ssl); + return result; + +} +#endif + + #ifdef OPENSSL_NPN_NEGOTIATED static int do_protocol_selection(int alpn, unsigned char **out, unsigned char *outlen, @@ -3645,6 +3761,7 @@ _SSL__SSLCONTEXT_SET_SERVERNAME_CALLBACK_METHODDEF _SSL__SSLCONTEXT_CERT_STORE_STATS_METHODDEF _SSL__SSLCONTEXT_GET_CA_CERTS_METHODDEF + _SSL__SSLCONTEXT_GET_CIPHERS_METHODDEF {NULL, NULL} /* sentinel */ }; diff --git a/Modules/clinic/_ssl.c.h b/Modules/clinic/_ssl.c.h --- a/Modules/clinic/_ssl.c.h +++ b/Modules/clinic/_ssl.c.h @@ -378,6 +378,27 @@ return return_value; } +#if (OPENSSL_VERSION_NUMBER >= 0x10002000UL) + +PyDoc_STRVAR(_ssl__SSLContext_get_ciphers__doc__, +"get_ciphers($self, /)\n" +"--\n" +"\n"); + +#define _SSL__SSLCONTEXT_GET_CIPHERS_METHODDEF \ + {"get_ciphers", (PyCFunction)_ssl__SSLContext_get_ciphers, METH_NOARGS, _ssl__SSLContext_get_ciphers__doc__}, + +static PyObject * +_ssl__SSLContext_get_ciphers_impl(PySSLContext *self); + +static PyObject * +_ssl__SSLContext_get_ciphers(PySSLContext *self, PyObject *Py_UNUSED(ignored)) +{ + return _ssl__SSLContext_get_ciphers_impl(self); +} + +#endif /* (OPENSSL_VERSION_NUMBER >= 0x10002000UL) */ + PyDoc_STRVAR(_ssl__SSLContext__set_npn_protocols__doc__, "_set_npn_protocols($self, protos, /)\n" "--\n" @@ -1128,6 +1149,10 @@ #define _SSL__SSLSOCKET_SELECTED_ALPN_PROTOCOL_METHODDEF #endif /* !defined(_SSL__SSLSOCKET_SELECTED_ALPN_PROTOCOL_METHODDEF) */ +#ifndef _SSL__SSLCONTEXT_GET_CIPHERS_METHODDEF + #define _SSL__SSLCONTEXT_GET_CIPHERS_METHODDEF +#endif /* !defined(_SSL__SSLCONTEXT_GET_CIPHERS_METHODDEF) */ + #ifndef _SSL__SSLCONTEXT_SET_ECDH_CURVE_METHODDEF #define _SSL__SSLCONTEXT_SET_ECDH_CURVE_METHODDEF #endif /* !defined(_SSL__SSLCONTEXT_SET_ECDH_CURVE_METHODDEF) */ @@ -1143,4 +1168,4 @@ #ifndef _SSL_ENUM_CRLS_METHODDEF #define _SSL_ENUM_CRLS_METHODDEF #endif /* !defined(_SSL_ENUM_CRLS_METHODDEF) */ -/*[clinic end generated code: output=6057f95343369849 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=2e7907a7d8f97ccf input=a9049054013a1b77]*/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:07:08 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 22:07:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327744=3A_correct_?= =?utf-8?q?comment_and_markup?= Message-ID: <20160905220707.6820.65796.A75756ED@psf.io> https://hg.python.org/cpython/rev/52404f9596b5 changeset: 103074:52404f9596b5 user: Christian Heimes date: Tue Sep 06 00:07:02 2016 +0200 summary: Issue #27744: correct comment and markup files: Doc/library/socket.rst | 2 +- Modules/socketmodule.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -136,7 +136,7 @@ elements ``(type, name [, feat [, mask]])``, where: - *type* is the algorithm type as string, e.g. ``aead``, ``hash``, - ``skcipher`` or ``rng``. + ``skcipher`` or ``rng``. - *name* is the algorithm name and operation mode as string, e.g. ``sha256``, ``hmac(sha256)``, ``cbc(aes)`` or ``drbg_nopr_ctr_aes256``. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -2556,7 +2556,7 @@ } PyErr_Clear(); - /* setsockopt(level, opt, (None, flag)) */ + /* setsockopt(level, opt, None, flag) */ if (PyArg_ParseTuple(args, "iiO!I:setsockopt", &level, &optname, Py_TYPE(Py_None), &none, &optlen)) { assert(sizeof(socklen_t) >= sizeof(unsigned int)); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:11:43 2016 From: python-checkins at python.org (larry.hastings) Date: Mon, 05 Sep 2016 22:11:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327355=3A_Removed_?= =?utf-8?q?support_for_Windows_CE=2E__It_was_never_finished=2C?= Message-ID: <20160905221142.47372.5721.A3D42477@psf.io> https://hg.python.org/cpython/rev/7a2eccee823b changeset: 103075:7a2eccee823b user: Larry Hastings date: Mon Sep 05 15:11:23 2016 -0700 summary: Issue #27355: Removed support for Windows CE. It was never finished, and Windows CE is no longer a relevant platform for Python. files: Doc/library/os.rst | 2 +- Doc/library/undoc.rst | 2 +- Lib/asyncore.py | 2 +- Lib/ctypes/__init__.py | 17 +- Lib/ctypes/test/test_bitfields.py | 4 +- Lib/ctypes/test/test_funcptr.py | 2 +- Lib/ctypes/test/test_loading.py | 15 +- Lib/ctypes/util.py | 10 - Lib/ntpath.py | 2 - Lib/os.py | 25 +- Lib/platform.py | 1 - Lib/tarfile.py | 2 +- Lib/test/support/__init__.py | 2 +- Misc/NEWS | 3 + Modules/_ctypes/_ctypes.c | 6 - Modules/_ctypes/libffi_arm_wince/debug.c | 59 - Modules/_ctypes/libffi_arm_wince/ffi.c | 310 --------- Modules/_ctypes/libffi_arm_wince/ffi.h | 317 ---------- Modules/_ctypes/libffi_arm_wince/ffi_common.h | 111 --- Modules/_ctypes/libffi_arm_wince/fficonfig.h | 152 ---- Modules/_ctypes/libffi_arm_wince/ffitarget.h | 49 - Modules/_ctypes/libffi_arm_wince/prep_cif.c | 175 ----- Modules/_ctypes/libffi_arm_wince/sysv.asm | 228 ------- PC/pyconfig.h | 47 - Python/sysmodule.c | 8 +- Python/thread_nt.h | 22 - Tools/freeze/freeze.py | 4 +- 27 files changed, 28 insertions(+), 1549 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -57,7 +57,7 @@ The name of the operating system dependent module imported. The following names have currently been registered: ``'posix'``, ``'nt'``, - ``'ce'``, ``'java'``. + ``'java'``. .. seealso:: :attr:`sys.platform` has a finer granularity. :func:`os.uname` gives diff --git a/Doc/library/undoc.rst b/Doc/library/undoc.rst --- a/Doc/library/undoc.rst +++ b/Doc/library/undoc.rst @@ -20,7 +20,7 @@ documented beyond this mention. There's little need to document these. :mod:`ntpath` - --- Implementation of :mod:`os.path` on Win32, Win64, and WinCE platforms. + --- Implementation of :mod:`os.path` on Win32 and Win64 platforms. :mod:`posixpath` --- Implementation of :mod:`os.path` on POSIX. diff --git a/Lib/asyncore.py b/Lib/asyncore.py --- a/Lib/asyncore.py +++ b/Lib/asyncore.py @@ -333,7 +333,7 @@ self.connecting = True err = self.socket.connect_ex(address) if err in (EINPROGRESS, EALREADY, EWOULDBLOCK) \ - or err == EINVAL and os.name in ('nt', 'ce'): + or err == EINVAL and os.name == 'nt': self.addr = address return if err in (0, EISCONN): diff --git a/Lib/ctypes/__init__.py b/Lib/ctypes/__init__.py --- a/Lib/ctypes/__init__.py +++ b/Lib/ctypes/__init__.py @@ -16,7 +16,7 @@ if __version__ != _ctypes_version: raise Exception("Version number mismatch", __version__, _ctypes_version) -if _os.name in ("nt", "ce"): +if _os.name == "nt": from _ctypes import FormatError DEFAULT_MODE = RTLD_LOCAL @@ -103,12 +103,9 @@ _c_functype_cache[(restype, argtypes, flags)] = CFunctionType return CFunctionType -if _os.name in ("nt", "ce"): +if _os.name == "nt": from _ctypes import LoadLibrary as _dlopen from _ctypes import FUNCFLAG_STDCALL as _FUNCFLAG_STDCALL - if _os.name == "ce": - # 'ce' doesn't have the stdcall calling convention - _FUNCFLAG_STDCALL = _FUNCFLAG_CDECL _win_functype_cache = {} def WINFUNCTYPE(restype, *argtypes, **kw): @@ -262,7 +259,7 @@ def _reset_cache(): _pointer_type_cache.clear() _c_functype_cache.clear() - if _os.name in ("nt", "ce"): + if _os.name == "nt": _win_functype_cache.clear() # _SimpleCData.c_wchar_p_from_param POINTER(c_wchar).from_param = c_wchar_p.from_param @@ -374,7 +371,7 @@ """ _func_flags_ = _FUNCFLAG_CDECL | _FUNCFLAG_PYTHONAPI -if _os.name in ("nt", "ce"): +if _os.name == "nt": class WinDLL(CDLL): """This class represents a dll exporting functions using the @@ -427,7 +424,7 @@ cdll = LibraryLoader(CDLL) pydll = LibraryLoader(PyDLL) -if _os.name in ("nt", "ce"): +if _os.name == "nt": pythonapi = PyDLL("python dll", None, _sys.dllhandle) elif _sys.platform == "cygwin": pythonapi = PyDLL("libpython%d.%d.dll" % _sys.version_info[:2]) @@ -435,7 +432,7 @@ pythonapi = PyDLL(None) -if _os.name in ("nt", "ce"): +if _os.name == "nt": windll = LibraryLoader(WinDLL) oledll = LibraryLoader(OleDLL) @@ -503,7 +500,7 @@ return _wstring_at(ptr, size) -if _os.name in ("nt", "ce"): # COM stuff +if _os.name == "nt": # COM stuff def DllGetClassObject(rclsid, riid, ppv): try: ccom = __import__("comtypes.server.inprocserver", globals(), locals(), ['*']) diff --git a/Lib/ctypes/test/test_bitfields.py b/Lib/ctypes/test/test_bitfields.py --- a/Lib/ctypes/test/test_bitfields.py +++ b/Lib/ctypes/test/test_bitfields.py @@ -196,7 +196,7 @@ class X(Structure): _fields_ = [("a", c_byte, 4), ("b", c_int, 4)] - if os.name in ("nt", "ce"): + if os.name == "nt": self.assertEqual(sizeof(X), sizeof(c_int)*2) else: self.assertEqual(sizeof(X), sizeof(c_int)) @@ -224,7 +224,7 @@ # MSVC does NOT combine c_short and c_int into one field, GCC # does (unless GCC is run with '-mms-bitfields' which # produces code compatible with MSVC). - if os.name in ("nt", "ce"): + if os.name == "nt": self.assertEqual(sizeof(X), sizeof(c_int) * 4) else: self.assertEqual(sizeof(X), sizeof(c_int) * 2) diff --git a/Lib/ctypes/test/test_funcptr.py b/Lib/ctypes/test/test_funcptr.py --- a/Lib/ctypes/test/test_funcptr.py +++ b/Lib/ctypes/test/test_funcptr.py @@ -39,7 +39,7 @@ # possible, as in C, to call cdecl functions with more parameters. #self.assertRaises(TypeError, c, 1, 2, 3) self.assertEqual(c(1, 2, 3, 4, 5, 6), 3) - if not WINFUNCTYPE is CFUNCTYPE and os.name != "ce": + if not WINFUNCTYPE is CFUNCTYPE: self.assertRaises(TypeError, s, 1, 2, 3) def test_structures(self): diff --git a/Lib/ctypes/test/test_loading.py b/Lib/ctypes/test/test_loading.py --- a/Lib/ctypes/test/test_loading.py +++ b/Lib/ctypes/test/test_loading.py @@ -11,8 +11,6 @@ global libc_name if os.name == "nt": libc_name = find_library("c") - elif os.name == "ce": - libc_name = "coredll" elif sys.platform == "cygwin": libc_name = "cygwin1.dll" else: @@ -49,8 +47,8 @@ cdll.LoadLibrary(lib) CDLL(lib) - @unittest.skipUnless(os.name in ("nt", "ce"), - 'test specific to Windows (NT/CE)') + @unittest.skipUnless(os.name == "nt", + 'test specific to Windows') def test_load_library(self): # CRT is no longer directly loadable. See issue23606 for the # discussion about alternative approaches. @@ -64,14 +62,9 @@ windll["kernel32"].GetModuleHandleW windll.LoadLibrary("kernel32").GetModuleHandleW WinDLL("kernel32").GetModuleHandleW - elif os.name == "ce": - windll.coredll.GetModuleHandleW - windll["coredll"].GetModuleHandleW - windll.LoadLibrary("coredll").GetModuleHandleW - WinDLL("coredll").GetModuleHandleW - @unittest.skipUnless(os.name in ("nt", "ce"), - 'test specific to Windows (NT/CE)') + @unittest.skipUnless(os.name == "nt", + 'test specific to Windows') def test_load_ordinal_functions(self): import _ctypes_test dll = WinDLL(_ctypes_test.__file__) diff --git a/Lib/ctypes/util.py b/Lib/ctypes/util.py --- a/Lib/ctypes/util.py +++ b/Lib/ctypes/util.py @@ -67,16 +67,6 @@ return fname return None -if os.name == "ce": - # search path according to MSDN: - # - absolute path specified by filename - # - The .exe launch directory - # - the Windows directory - # - ROM dll files (where are they?) - # - OEM specified search path: HKLM\Loader\SystemPath - def find_library(name): - return name - if os.name == "posix" and sys.platform == "darwin": from ctypes.macholib.dyld import dyld_find as _dyld_find def find_library(name): diff --git a/Lib/ntpath.py b/Lib/ntpath.py --- a/Lib/ntpath.py +++ b/Lib/ntpath.py @@ -28,8 +28,6 @@ pathsep = ';' altsep = '/' defpath = '.;C:\\bin' -if 'ce' in sys.builtin_module_names: - defpath = '\\Windows' devnull = 'nul' def _get_bothseps(path): diff --git a/Lib/os.py b/Lib/os.py --- a/Lib/os.py +++ b/Lib/os.py @@ -1,9 +1,9 @@ r"""OS routines for NT or Posix depending on what system we're on. This exports: - - all functions from posix, nt or ce, e.g. unlink, stat, etc. + - all functions from posix or nt, e.g. unlink, stat, etc. - os.path is either posixpath or ntpath - - os.name is either 'posix', 'nt' or 'ce'. + - os.name is either 'posix' or 'nt' - os.curdir is a string representing the current directory ('.' or ':') - os.pardir is a string representing the parent directory ('..' or '::') - os.sep is the (or a most common) pathname separator ('/' or ':' or '\\') @@ -85,27 +85,6 @@ except ImportError: pass -elif 'ce' in _names: - name = 'ce' - linesep = '\r\n' - from ce import * - try: - from ce import _exit - __all__.append('_exit') - except ImportError: - pass - # We can use the standard Windows path. - import ntpath as path - - import ce - __all__.extend(_get_exports_list(ce)) - del ce - - try: - from ce import _have_functions - except ImportError: - pass - else: raise ImportError('no os specific module found') diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -13,7 +13,6 @@ # Python bug tracker (http://bugs.python.org) and assign them to "lemburg". # # Still needed: -# * more support for WinCE # * support for MS-DOS (PythonDX ?) # * support for Amiga and other still unsupported platforms running Python # * support for additional Linux distributions diff --git a/Lib/tarfile.py b/Lib/tarfile.py --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -144,7 +144,7 @@ #--------------------------------------------------------- # initialization #--------------------------------------------------------- -if os.name in ("nt", "ce"): +if os.name == "nt": ENCODING = "utf-8" else: ENCODING = sys.getfilesystemencoding() diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -811,7 +811,7 @@ # encoded by the filesystem encoding (in strict mode). It can be None if we # cannot generate such filename. TESTFN_UNENCODABLE = None -if os.name in ('nt', 'ce'): +if os.name == 'nt': # skip win32s (0) or Windows 9x/ME (1) if sys.getwindowsversion().platform >= 2: # Different kinds of characters from various languages to minimize the diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27355: Removed support for Windows CE. It was never finished, + and Windows CE is no longer a relevant platform for Python. + - Issue #27921: Disallow backslashes in f-strings. This is a temporary restriction: in beta 2, backslashes will only be disallowed inside the braces (where the expressions are). This is a breaking change diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -111,12 +111,6 @@ #ifndef IS_INTRESOURCE #define IS_INTRESOURCE(x) (((size_t)(x) >> 16) == 0) #endif -# ifdef _WIN32_WCE -/* Unlike desktop Windows, WinCE has both W and A variants of - GetProcAddress, but the default W version is not what we want */ -# undef GetProcAddress -# define GetProcAddress GetProcAddressA -# endif #else #include "ctypes_dlfcn.h" #endif diff --git a/Modules/_ctypes/libffi_arm_wince/debug.c b/Modules/_ctypes/libffi_arm_wince/debug.c deleted file mode 100644 --- a/Modules/_ctypes/libffi_arm_wince/debug.c +++ /dev/null @@ -1,59 +0,0 @@ -/* ----------------------------------------------------------------------- - debug.c - Copyright (c) 1996 Red Hat, Inc. - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -#include -#include -#include -#include - -/* General debugging routines */ - -void ffi_stop_here(void) -{ - /* This function is only useful for debugging purposes. - Place a breakpoint on ffi_stop_here to be notified of - significant events. */ -} - -/* This function should only be called via the FFI_ASSERT() macro */ - -void ffi_assert(char *expr, char *file, int line) -{ - fprintf(stderr, "ASSERTION FAILURE: %s at %s:%d\n", expr, file, line); - ffi_stop_here(); - abort(); -} - -/* Perform a sanity check on an ffi_type structure */ - -void ffi_type_test(ffi_type *a, char *file, int line) -{ - FFI_ASSERT_AT(a != NULL, file, line); - - /*@-usedef@*/ - FFI_ASSERT_AT(a->type <= FFI_TYPE_LAST, file, line); - FFI_ASSERT_AT(a->type == FFI_TYPE_VOID || a->size > 0, file, line); - FFI_ASSERT_AT(a->type == FFI_TYPE_VOID || a->alignment > 0, file, line); - FFI_ASSERT_AT(a->type != FFI_TYPE_STRUCT || a->elements != NULL, file, line); - /*@=usedef@*/ -} diff --git a/Modules/_ctypes/libffi_arm_wince/ffi.c b/Modules/_ctypes/libffi_arm_wince/ffi.c deleted file mode 100644 --- a/Modules/_ctypes/libffi_arm_wince/ffi.c +++ /dev/null @@ -1,310 +0,0 @@ -/* ----------------------------------------------------------------------- - ffi.c - Copyright (c) 1998 Red Hat, Inc. - - ARM Foreign Function Interface - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -#include -#include - -#include - -#ifdef _WIN32_WCE -#pragma warning (disable : 4142) /* benign redefinition of type */ -#include -#endif - -/* ffi_prep_args is called by the assembly routine once stack space - has been allocated for the function's arguments */ - -/*@-exportheader@*/ -void ffi_prep_args(char *stack, extended_cif *ecif) -/*@=exportheader@*/ -{ - register unsigned int i; - register void **p_argv; - register char *argp; - register ffi_type **p_arg; - - argp = stack; - - if ( ecif->cif->rtype->type == FFI_TYPE_STRUCT ) { - *(void **) argp = ecif->rvalue; - argp += 4; - } - - p_argv = ecif->avalue; - - for (i = ecif->cif->nargs, p_arg = ecif->cif->arg_types; - (i != 0); - i--, p_arg++) - { - size_t z; - size_t argalign = (*p_arg)->alignment; - -#ifdef _WIN32_WCE - if (argalign > 4) - argalign = 4; -#endif - /* Align if necessary */ - if ((argalign - 1) & (unsigned) argp) { - argp = (char *) ALIGN(argp, argalign); - } - - z = (*p_arg)->size; - if (z < sizeof(int)) - { - z = sizeof(int); - switch ((*p_arg)->type) - { - case FFI_TYPE_SINT8: - *(signed int *) argp = (signed int)*(SINT8 *)(* p_argv); - break; - - case FFI_TYPE_UINT8: - *(unsigned int *) argp = (unsigned int)*(UINT8 *)(* p_argv); - break; - - case FFI_TYPE_SINT16: - *(signed int *) argp = (signed int)*(SINT16 *)(* p_argv); - break; - - case FFI_TYPE_UINT16: - *(unsigned int *) argp = (unsigned int)*(UINT16 *)(* p_argv); - break; - - case FFI_TYPE_STRUCT: - /* *p_argv may not be aligned for a UINT32 */ - memcpy(argp, *p_argv, z); - break; - - default: - FFI_ASSERT(0); - } - } - else if (z == sizeof(int)) - { - *(unsigned int *) argp = (unsigned int)*(UINT32 *)(* p_argv); - } - else - { - memcpy(argp, *p_argv, z); - } - p_argv++; - argp += z; - } - - return; -} - -/* Perform machine dependent cif processing */ -ffi_status ffi_prep_cif_machdep(ffi_cif *cif) -{ - /* Set the return type flag */ - switch (cif->rtype->type) - { - case FFI_TYPE_VOID: - case FFI_TYPE_STRUCT: - case FFI_TYPE_FLOAT: - case FFI_TYPE_DOUBLE: - case FFI_TYPE_SINT64: - cif->flags = (unsigned) cif->rtype->type; - break; - - case FFI_TYPE_UINT64: - cif->flags = FFI_TYPE_SINT64; - break; - - default: - cif->flags = FFI_TYPE_INT; - break; - } - - return FFI_OK; -} - -/*@-declundef@*/ -/*@-exportheader@*/ -extern void ffi_call_SYSV(void (*)(char *, extended_cif *), - /*@out@*/ extended_cif *, - unsigned, unsigned, - /*@out@*/ unsigned *, - void (*fn)()); -/*@=declundef@*/ -/*@=exportheader@*/ - -/* Return type changed from void for ctypes */ -int ffi_call(/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ void **avalue) -{ - extended_cif ecif; - - ecif.cif = cif; - ecif.avalue = avalue; - - /* If the return value is a struct and we don't have a return */ - /* value address then we need to make one */ - - if ((rvalue == NULL) && - (cif->rtype->type == FFI_TYPE_STRUCT)) - { - /*@-sysunrecog@*/ - ecif.rvalue = alloca(cif->rtype->size); - /*@=sysunrecog@*/ - } - else - ecif.rvalue = rvalue; - - - switch (cif->abi) - { - case FFI_SYSV: - /*@-usedef@*/ - ffi_call_SYSV(ffi_prep_args, &ecif, cif->bytes, - cif->flags, ecif.rvalue, fn); - /*@=usedef@*/ - break; - default: - FFI_ASSERT(0); - break; - } - /* I think calculating the real stack pointer delta is not useful - because stdcall is not supported */ - return 0; -} - -/** private members **/ - -static void ffi_prep_incoming_args_SYSV (char *stack, void **ret, - void** args, ffi_cif* cif); - -/* This function is called by ffi_closure_SYSV in sysv.asm */ - -unsigned int -ffi_closure_SYSV_inner (ffi_closure *closure, char *in_args, void *rvalue) -{ - ffi_cif *cif = closure->cif; - void **out_args; - - out_args = (void **) alloca(cif->nargs * sizeof (void *)); - - /* this call will initialize out_args, such that each - * element in that array points to the corresponding - * value on the stack; and if the function returns - * a structure, it will re-set rvalue to point to the - * structure return address. */ - - ffi_prep_incoming_args_SYSV(in_args, &rvalue, out_args, cif); - - (closure->fun)(cif, rvalue, out_args, closure->user_data); - - /* Tell ffi_closure_SYSV what the returntype is */ - return cif->flags; -} - -/*@-exportheader@*/ -static void -ffi_prep_incoming_args_SYSV(char *stack, void **rvalue, - void **avalue, ffi_cif *cif) -/*@=exportheader@*/ -{ - unsigned int i; - void **p_argv; - char *argp; - ffi_type **p_arg; - - argp = stack; - - if ( cif->rtype->type == FFI_TYPE_STRUCT ) { - *rvalue = *(void **) argp; - argp += 4; - } - - p_argv = avalue; - - for (i = cif->nargs, p_arg = cif->arg_types; (i != 0); i--, p_arg++) - { - size_t z; - size_t argalign = (*p_arg)->alignment; - -#ifdef _WIN32_WCE - if (argalign > 4) - argalign = 4; -#endif - /* Align if necessary */ - if ((argalign - 1) & (unsigned) argp) { - argp = (char *) ALIGN(argp, argalign); - } - - z = (*p_arg)->size; - if (z < sizeof(int)) - z = sizeof(int); - - *p_argv = (void*) argp; - - p_argv++; - argp += z; - } -} - -/* - add ip, pc, #-8 ; ip = address of this trampoline == address of ffi_closure - ldr pc, [pc, #-4] ; jump to __fun - DCD __fun -*/ -#define FFI_INIT_TRAMPOLINE(TRAMP,FUN) \ -{ \ - unsigned int *__tramp = (unsigned int *)(TRAMP); \ - __tramp[0] = 0xe24fc008; /* add ip, pc, #-8 */ \ - __tramp[1] = 0xe51ff004; /* ldr pc, [pc, #-4] */ \ - __tramp[2] = (unsigned int)(FUN); \ - } - -/* the cif must already be prep'ed */ - -/* defined in sysv.asm */ -void ffi_closure_SYSV(void); - -ffi_status -ffi_prep_closure (ffi_closure* closure, - ffi_cif* cif, - void (*fun)(ffi_cif*,void*,void**,void*), - void *user_data) -{ - FFI_ASSERT (cif->abi == FFI_SYSV); - - FFI_INIT_TRAMPOLINE (&closure->tramp[0], &ffi_closure_SYSV); - - closure->cif = cif; - closure->user_data = user_data; - closure->fun = fun; - -#ifdef _WIN32_WCE - /* This is important to allow calling the trampoline safely */ - FlushInstructionCache(GetCurrentProcess(), 0, 0); -#endif - - return FFI_OK; -} - diff --git a/Modules/_ctypes/libffi_arm_wince/ffi.h b/Modules/_ctypes/libffi_arm_wince/ffi.h deleted file mode 100644 --- a/Modules/_ctypes/libffi_arm_wince/ffi.h +++ /dev/null @@ -1,317 +0,0 @@ -/* -----------------------------------------------------------------*-C-*- - libffi 2.00-beta-wince - Copyright (c) 1996-2003 Red Hat, Inc. - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - - ----------------------------------------------------------------------- */ - -/* ------------------------------------------------------------------- - The basic API is described in the README file. - - The raw API is designed to bypass some of the argument packing - and unpacking on architectures for which it can be avoided. - - The closure API allows interpreted functions to be packaged up - inside a C function pointer, so that they can be called as C functions, - with no understanding on the client side that they are interpreted. - It can also be used in other cases in which it is necessary to package - up a user specified parameter and a function pointer as a single - function pointer. - - The closure API must be implemented in order to get its functionality, - e.g. for use by gij. Routines are provided to emulate the raw API - if the underlying platform doesn't allow faster implementation. - - More details on the raw and cloure API can be found in: - - http://gcc.gnu.org/ml/java/1999-q3/msg00138.html - - and - - http://gcc.gnu.org/ml/java/1999-q3/msg00174.html - -------------------------------------------------------------------- */ - -#ifndef LIBFFI_H -#define LIBFFI_H - -#ifdef __cplusplus -extern "C" { -#endif - -/* Specify which architecture libffi is configured for. */ -/*#define @TARGET@*/ - -/* ---- System configuration information --------------------------------- */ - -#include - -#ifndef LIBFFI_ASM - -#include -#include - -/* LONG_LONG_MAX is not always defined (not if STRICT_ANSI, for example). - But we can find it either under the correct ANSI name, or under GNU - C's internal name. */ -#ifdef LONG_LONG_MAX -# define FFI_LONG_LONG_MAX LONG_LONG_MAX -#else -# ifdef LLONG_MAX -# define FFI_LONG_LONG_MAX LLONG_MAX -# else -# ifdef __GNUC__ -# define FFI_LONG_LONG_MAX __LONG_LONG_MAX__ -# endif -# ifdef _MSC_VER -# define FFI_LONG_LONG_MAX _I64_MAX -# endif -# endif -#endif - -#if SCHAR_MAX == 127 -# define ffi_type_uchar ffi_type_uint8 -# define ffi_type_schar ffi_type_sint8 -#else - #error "char size not supported" -#endif - -#if SHRT_MAX == 32767 -# define ffi_type_ushort ffi_type_uint16 -# define ffi_type_sshort ffi_type_sint16 -#elif SHRT_MAX == 2147483647 -# define ffi_type_ushort ffi_type_uint32 -# define ffi_type_sshort ffi_type_sint32 -#else - #error "short size not supported" -#endif - -#if INT_MAX == 32767 -# define ffi_type_uint ffi_type_uint16 -# define ffi_type_sint ffi_type_sint16 -#elif INT_MAX == 2147483647 -# define ffi_type_uint ffi_type_uint32 -# define ffi_type_sint ffi_type_sint32 -#elif INT_MAX == 9223372036854775807 -# define ffi_type_uint ffi_type_uint64 -# define ffi_type_sint ffi_type_sint64 -#else - #error "int size not supported" -#endif - -#define ffi_type_ulong ffi_type_uint64 -#define ffi_type_slong ffi_type_sint64 -#if LONG_MAX == 2147483647 -# if FFI_LONG_LONG_MAX != 9223372036854775807 - #error "no 64-bit data type supported" -# endif -#elif LONG_MAX != 9223372036854775807 - #error "long size not supported" -#endif - -/* The closure code assumes that this works on pointers, i.e. a size_t */ -/* can hold a pointer. */ - -typedef struct _ffi_type -{ - size_t size; - unsigned short alignment; - unsigned short type; - /*@null@*/ struct _ffi_type **elements; -} ffi_type; - -/* These are defined in types.c */ -extern ffi_type ffi_type_void; -extern ffi_type ffi_type_uint8; -extern ffi_type ffi_type_sint8; -extern ffi_type ffi_type_uint16; -extern ffi_type ffi_type_sint16; -extern ffi_type ffi_type_uint32; -extern ffi_type ffi_type_sint32; -extern ffi_type ffi_type_uint64; -extern ffi_type ffi_type_sint64; -extern ffi_type ffi_type_float; -extern ffi_type ffi_type_double; -extern ffi_type ffi_type_longdouble; -extern ffi_type ffi_type_pointer; - - -typedef enum { - FFI_OK = 0, - FFI_BAD_TYPEDEF, - FFI_BAD_ABI -} ffi_status; - -typedef unsigned FFI_TYPE; - -typedef struct { - ffi_abi abi; - unsigned nargs; - /*@dependent@*/ ffi_type **arg_types; - /*@dependent@*/ ffi_type *rtype; - unsigned bytes; - unsigned flags; -#ifdef FFI_EXTRA_CIF_FIELDS - FFI_EXTRA_CIF_FIELDS; -#endif -} ffi_cif; - -/* ---- Definitions for the raw API -------------------------------------- */ - -#ifndef FFI_SIZEOF_ARG -# if LONG_MAX == 2147483647 -# define FFI_SIZEOF_ARG 4 -# elif LONG_MAX == 9223372036854775807 -# define FFI_SIZEOF_ARG 8 -# endif -#endif - -typedef union { - ffi_sarg sint; - ffi_arg uint; - float flt; - char data[FFI_SIZEOF_ARG]; - void* ptr; -} ffi_raw; - -void ffi_raw_call (/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ ffi_raw *avalue); - -void ffi_ptrarray_to_raw (ffi_cif *cif, void **args, ffi_raw *raw); -void ffi_raw_to_ptrarray (ffi_cif *cif, ffi_raw *raw, void **args); -size_t ffi_raw_size (ffi_cif *cif); - -/* This is analogous to the raw API, except it uses Java parameter */ -/* packing, even on 64-bit machines. I.e. on 64-bit machines */ -/* longs and doubles are followed by an empty 64-bit word. */ - -void ffi_java_raw_call (/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ ffi_raw *avalue); - -void ffi_java_ptrarray_to_raw (ffi_cif *cif, void **args, ffi_raw *raw); -void ffi_java_raw_to_ptrarray (ffi_cif *cif, ffi_raw *raw, void **args); -size_t ffi_java_raw_size (ffi_cif *cif); - -/* ---- Definitions for closures ----------------------------------------- */ - -#if FFI_CLOSURES - -typedef struct { - char tramp[FFI_TRAMPOLINE_SIZE]; - ffi_cif *cif; - void (*fun)(ffi_cif*,void*,void**,void*); - void *user_data; -} ffi_closure; - -ffi_status -ffi_prep_closure (ffi_closure*, - ffi_cif *, - void (*fun)(ffi_cif*,void*,void**,void*), - void *user_data); - -typedef struct { - char tramp[FFI_TRAMPOLINE_SIZE]; - - ffi_cif *cif; - -#if !FFI_NATIVE_RAW_API - - /* if this is enabled, then a raw closure has the same layout - as a regular closure. We use this to install an intermediate - handler to do the transaltion, void** -> ffi_raw*. */ - - void (*translate_args)(ffi_cif*,void*,void**,void*); - void *this_closure; - -#endif - - void (*fun)(ffi_cif*,void*,ffi_raw*,void*); - void *user_data; - -} ffi_raw_closure; - -ffi_status -ffi_prep_raw_closure (ffi_raw_closure*, - ffi_cif *cif, - void (*fun)(ffi_cif*,void*,ffi_raw*,void*), - void *user_data); - -ffi_status -ffi_prep_java_raw_closure (ffi_raw_closure*, - ffi_cif *cif, - void (*fun)(ffi_cif*,void*,ffi_raw*,void*), - void *user_data); - -#endif /* FFI_CLOSURES */ - -/* ---- Public interface definition -------------------------------------- */ - -ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, - ffi_abi abi, - unsigned int nargs, - /*@dependent@*/ /*@out@*/ /*@partial@*/ ffi_type *rtype, - /*@dependent@*/ ffi_type **atypes); - -/* Return type changed from void for ctypes */ -int ffi_call(/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ void **avalue); - -/* Useful for eliminating compiler warnings */ -#define FFI_FN(f) ((void (*)())f) - -/* ---- Definitions shared with assembly code ---------------------------- */ - -#endif - -/* If these change, update src/mips/ffitarget.h. */ -#define FFI_TYPE_VOID 0 -#define FFI_TYPE_INT 1 -#define FFI_TYPE_FLOAT 2 -#define FFI_TYPE_DOUBLE 3 -#if 0 /*@HAVE_LONG_DOUBLE@*/ -#define FFI_TYPE_LONGDOUBLE 4 -#else -#define FFI_TYPE_LONGDOUBLE FFI_TYPE_DOUBLE -#endif -#define FFI_TYPE_UINT8 5 -#define FFI_TYPE_SINT8 6 -#define FFI_TYPE_UINT16 7 -#define FFI_TYPE_SINT16 8 -#define FFI_TYPE_UINT32 9 -#define FFI_TYPE_SINT32 10 -#define FFI_TYPE_UINT64 11 -#define FFI_TYPE_SINT64 12 -#define FFI_TYPE_STRUCT 13 -#define FFI_TYPE_POINTER 14 - -/* This should always refer to the last type code (for sanity checks) */ -#define FFI_TYPE_LAST FFI_TYPE_POINTER - -#ifdef __cplusplus -} -#endif - -#endif - diff --git a/Modules/_ctypes/libffi_arm_wince/ffi_common.h b/Modules/_ctypes/libffi_arm_wince/ffi_common.h deleted file mode 100644 --- a/Modules/_ctypes/libffi_arm_wince/ffi_common.h +++ /dev/null @@ -1,111 +0,0 @@ -/* ----------------------------------------------------------------------- - ffi_common.h - Copyright (c) 1996 Red Hat, Inc. - - Common internal definitions and macros. Only necessary for building - libffi. - ----------------------------------------------------------------------- */ - -#ifndef FFI_COMMON_H -#define FFI_COMMON_H - -#ifdef __cplusplus -extern "C" { -#endif - -#include - -/* Do not move this. Some versions of AIX are very picky about where - this is positioned. */ -#ifdef __GNUC__ -# define alloca __builtin_alloca -#else -# if HAVE_ALLOCA_H -# include -# else -# ifdef _AIX - #pragma alloca -# else -# ifndef alloca /* predefined by HP cc +Olibcalls */ -char *alloca (); -# endif -# endif -# endif -#endif - -/* Check for the existence of memcpy. */ -#if STDC_HEADERS -# include -#else -# ifndef HAVE_MEMCPY -# define memcpy(d, s, n) bcopy ((s), (d), (n)) -# endif -#endif - -#if defined(FFI_DEBUG) -#include -#endif - -#ifdef FFI_DEBUG -/*@exits@*/ void ffi_assert(/*@temp@*/ char *expr, /*@temp@*/ char *file, int line); -void ffi_stop_here(void); -void ffi_type_test(/*@temp@*/ /*@out@*/ ffi_type *a, /*@temp@*/ char *file, int line); - -#define FFI_ASSERT(x) ((x) ? (void)0 : ffi_assert(#x, __FILE__,__LINE__)) -#define FFI_ASSERT_AT(x, f, l) ((x) ? 0 : ffi_assert(#x, (f), (l))) -#define FFI_ASSERT_VALID_TYPE(x) ffi_type_test (x, __FILE__, __LINE__) -#else -#define FFI_ASSERT(x) -#define FFI_ASSERT_AT(x, f, l) -#define FFI_ASSERT_VALID_TYPE(x) -#endif - -#define ALIGN(v, a) (((((size_t) (v))-1) | ((a)-1))+1) - -/* Perform machine dependent cif processing */ -ffi_status ffi_prep_cif_machdep(ffi_cif *cif); - -/* Extended cif, used in callback from assembly routine */ -typedef struct -{ - /*@dependent@*/ ffi_cif *cif; - /*@dependent@*/ void *rvalue; - /*@dependent@*/ void **avalue; -} extended_cif; - -/* Terse sized type definitions. */ -#if defined(__GNUC__) - -typedef unsigned int UINT8 __attribute__((__mode__(__QI__))); -typedef signed int SINT8 __attribute__((__mode__(__QI__))); -typedef unsigned int UINT16 __attribute__((__mode__(__HI__))); -typedef signed int SINT16 __attribute__((__mode__(__HI__))); -typedef unsigned int UINT32 __attribute__((__mode__(__SI__))); -typedef signed int SINT32 __attribute__((__mode__(__SI__))); -typedef unsigned int UINT64 __attribute__((__mode__(__DI__))); -typedef signed int SINT64 __attribute__((__mode__(__DI__))); - -#elif defined(_MSC_VER) - -typedef unsigned __int8 UINT8; -typedef signed __int8 SINT8; -typedef unsigned __int16 UINT16; -typedef signed __int16 SINT16; -typedef unsigned __int32 UINT32; -typedef signed __int32 SINT32; -typedef unsigned __int64 UINT64; -typedef signed __int64 SINT64; - -#else -#error "Need typedefs here" -#endif - -typedef float FLOAT32; - - -#ifdef __cplusplus -} -#endif - -#endif - - diff --git a/Modules/_ctypes/libffi_arm_wince/fficonfig.h b/Modules/_ctypes/libffi_arm_wince/fficonfig.h deleted file mode 100644 --- a/Modules/_ctypes/libffi_arm_wince/fficonfig.h +++ /dev/null @@ -1,152 +0,0 @@ -/* fficonfig.h created manually for Windows CE on ARM */ -/* fficonfig.h.in. Generated from configure.ac by autoheader. */ - -/* 1234 = LIL_ENDIAN, 4321 = BIGENDIAN */ -#define BYTEORDER 1234 - -/* Define to one of `_getb67', `GETB67', `getb67' for Cray-2 and Cray-YMP - systems. This function is required for `alloca.c' support on those systems. - */ -/* #undef CRAY_STACKSEG_END */ - -/* Define to 1 if using `alloca.c'. */ -/* #undef C_ALLOCA */ - -/* Define to the flags needed for the .section .eh_frame directive. */ -/* #undef EH_FRAME_FLAGS */ - -/* Define this if you want extra debugging. */ -#ifdef DEBUG /* Defined by the project settings for Debug builds */ -#define FFI_DEBUG -#else -#undef FFI_DEBUG -#endif - -/* Define this is you do not want support for the raw API. */ -/* #undef FFI_NO_RAW_API */ - -/* Define this is you do not want support for aggregate types. */ -/* #undef FFI_NO_STRUCTS */ - -/* Define to 1 if you have `alloca', as a function or macro. */ -#define HAVE_ALLOCA 1 - -/* Define to 1 if you have and it should be used (not on Ultrix). - */ -/* #undef HAVE_ALLOCA_H */ - -/* Define if your assembler supports .register. */ -/* #undef HAVE_AS_REGISTER_PSEUDO_OP */ - -/* Define if your assembler and linker support unaligned PC relative relocs. - */ -/* #undef HAVE_AS_SPARC_UA_PCREL */ - -/* Define to 1 if you have the header file. */ -/* #undef HAVE_INTTYPES_H */ - -/* Define if you have the long double type and it is bigger than a double */ -/* This differs from the MSVC build, but even there it should not be defined */ -/* #undef HAVE_LONG_DOUBLE */ - -/* Define to 1 if you have the `memcpy' function. */ -#define HAVE_MEMCPY 1 - -/* Define to 1 if you have the header file. */ -/* WinCE has this but I don't think we need to use it */ -/* #undef HAVE_MEMORY_H */ - -/* Define to 1 if you have the `mmap' function. */ -/* #undef HAVE_MMAP */ - -/* Define if mmap with MAP_ANON(YMOUS) works. */ -/* #undef HAVE_MMAP_ANON */ - -/* Define if mmap of /dev/zero works. */ -/* #undef HAVE_MMAP_DEV_ZERO */ - -/* Define if read-only mmap of a plain file works. */ -/* #undef HAVE_MMAP_FILE */ - -/* Define if .eh_frame sections should be read-only. */ -/* #undef HAVE_RO_EH_FRAME */ - -/* Define to 1 if you have the header file. */ -/* #undef HAVE_STDINT_H */ - -/* Define to 1 if you have the header file. */ -#define HAVE_STDLIB_H 1 - -/* Define to 1 if you have the header file. */ -/* #undef HAVE_STRINGS_H */ - -/* Define to 1 if you have the header file. */ -#define HAVE_STRING_H 1 - -/* Define to 1 if you have the header file. */ -/* #undef HAVE_SYS_MMAN_H */ - -/* Define to 1 if you have the header file. */ -/* #undef HAVE_SYS_STAT_H */ - -/* Define to 1 if you have the header file. */ -/* #undef HAVE_SYS_TYPES_H */ - -/* Define to 1 if you have the header file. */ -/* #undef HAVE_UNISTD_H */ - -/* Define if the host machine stores words of multi-word integers in - big-endian order. */ -/* #undef HOST_WORDS_BIG_ENDIAN */ - -/* Define to 1 if your C compiler doesn't accept -c and -o together. */ -/* #undef NO_MINUS_C_MINUS_O */ - -/* Name of package */ -/* #undef PACKAGE */ - -/* Define to the address where bug reports for this package should be sent. */ -/* #undef PACKAGE_BUGREPORT */ - -/* Define to the full name of this package. */ -/* #undef PACKAGE_NAME */ - -/* Define to the full name and version of this package. */ -/* #undef PACKAGE_STRING */ - -/* Define to the one symbol short name of this package. */ -/* #undef PACKAGE_TARNAME */ - -/* Define to the version of this package. */ -/* #undef PACKAGE_VERSION */ - -/* The number of bytes in type double */ -#define SIZEOF_DOUBLE 8 - -/* The number of bytes in type long double */ -#define SIZEOF_LONG_DOUBLE 8 - -/* If using the C implementation of alloca, define if you know the - direction of stack growth for your system; otherwise it will be - automatically deduced at run-time. - STACK_DIRECTION > 0 => grows toward higher addresses - STACK_DIRECTION < 0 => grows toward lower addresses - STACK_DIRECTION = 0 => direction of growth unknown */ -/* #undef STACK_DIRECTION */ - -/* Define to 1 if you have the ANSI C header files. */ -#define STDC_HEADERS 1 - -/* Define this if you are using Purify and want to suppress spurious messages. - */ -/* #undef USING_PURIFY */ - -/* Version number of package */ -/* #undef VERSION */ - -/* whether byteorder is bigendian */ -/* #undef WORDS_BIGENDIAN */ - -#define alloca _alloca - -#define abort() exit(999) diff --git a/Modules/_ctypes/libffi_arm_wince/ffitarget.h b/Modules/_ctypes/libffi_arm_wince/ffitarget.h deleted file mode 100644 --- a/Modules/_ctypes/libffi_arm_wince/ffitarget.h +++ /dev/null @@ -1,49 +0,0 @@ -/* -----------------------------------------------------------------*-C-*- - ffitarget.h - Copyright (c) 1996-2003 Red Hat, Inc. - Target configuration macros for ARM. - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - - ----------------------------------------------------------------------- */ - -#ifndef LIBFFI_TARGET_H -#define LIBFFI_TARGET_H - -#ifndef LIBFFI_ASM -typedef unsigned long ffi_arg; -typedef signed long ffi_sarg; - -typedef enum ffi_abi { - FFI_FIRST_ABI = 0, - FFI_SYSV, - FFI_DEFAULT_ABI = FFI_SYSV, - FFI_LAST_ABI = FFI_DEFAULT_ABI + 1 -} ffi_abi; -#endif - -/* ---- Definitions for closures ----------------------------------------- */ - -#define FFI_CLOSURES 1 -#define FFI_TRAMPOLINE_SIZE 12 - -#define FFI_NATIVE_RAW_API 0 - -#endif - diff --git a/Modules/_ctypes/libffi_arm_wince/prep_cif.c b/Modules/_ctypes/libffi_arm_wince/prep_cif.c deleted file mode 100644 --- a/Modules/_ctypes/libffi_arm_wince/prep_cif.c +++ /dev/null @@ -1,175 +0,0 @@ -/* ----------------------------------------------------------------------- - prep_cif.c - Copyright (c) 1996, 1998 Red Hat, Inc. - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -#include -#include -#include - - -/* Round up to FFI_SIZEOF_ARG. */ - -#define STACK_ARG_SIZE(x) ALIGN(x, FFI_SIZEOF_ARG) - -/* Perform machine independent initialization of aggregate type - specifications. */ - -static ffi_status initialize_aggregate(/*@out@*/ ffi_type *arg) -{ - ffi_type **ptr; - - FFI_ASSERT(arg != NULL); - - /*@-usedef@*/ - - FFI_ASSERT(arg->elements != NULL); - FFI_ASSERT(arg->size == 0); - FFI_ASSERT(arg->alignment == 0); - - ptr = &(arg->elements[0]); - - while ((*ptr) != NULL) - { - if (((*ptr)->size == 0) && (initialize_aggregate((*ptr)) != FFI_OK)) - return FFI_BAD_TYPEDEF; - - /* Perform a sanity check on the argument type */ - FFI_ASSERT_VALID_TYPE(*ptr); - - arg->size = ALIGN(arg->size, (*ptr)->alignment); - arg->size += (*ptr)->size; - - arg->alignment = (arg->alignment > (*ptr)->alignment) ? - arg->alignment : (*ptr)->alignment; - - ptr++; - } - - /* Structure size includes tail padding. This is important for - structures that fit in one register on ABIs like the PowerPC64 - Linux ABI that right justify small structs in a register. - It's also needed for nested structure layout, for example - struct A { long a; char b; }; struct B { struct A x; char y; }; - should find y at an offset of 2*sizeof(long) and result in a - total size of 3*sizeof(long). */ - arg->size = ALIGN (arg->size, arg->alignment); - - if (arg->size == 0) - return FFI_BAD_TYPEDEF; - else - return FFI_OK; - - /*@=usedef@*/ -} - -/* Perform machine independent ffi_cif preparation, then call - machine dependent routine. */ - -ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, - ffi_abi abi, unsigned int nargs, - /*@dependent@*/ /*@out@*/ /*@partial@*/ ffi_type *rtype, - /*@dependent@*/ ffi_type **atypes) -{ - unsigned bytes = 0; - unsigned int i; - ffi_type **ptr; - - FFI_ASSERT(cif != NULL); - FFI_ASSERT((abi > FFI_FIRST_ABI) && (abi <= FFI_DEFAULT_ABI)); - - cif->abi = abi; - cif->arg_types = atypes; - cif->nargs = nargs; - cif->rtype = rtype; - - cif->flags = 0; - - /* Initialize the return type if necessary */ - /*@-usedef@*/ - if ((cif->rtype->size == 0) && (initialize_aggregate(cif->rtype) != FFI_OK)) - return FFI_BAD_TYPEDEF; - /*@=usedef@*/ - - /* Perform a sanity check on the return type */ - FFI_ASSERT_VALID_TYPE(cif->rtype); - - /* x86-64 and s390 stack space allocation is handled in prep_machdep. */ -#if !defined M68K && !defined __x86_64__ && !defined S390 - /* Make space for the return structure pointer */ - if (cif->rtype->type == FFI_TYPE_STRUCT - /* MSVC returns small structures in registers. But we have a different - workaround: pretend int32 or int64 return type, and converting to - structure afterwards. */ -#ifdef SPARC - && (cif->abi != FFI_V9 || cif->rtype->size > 32) -#endif - ) - bytes = STACK_ARG_SIZE(sizeof(void*)); -#endif - - for (ptr = cif->arg_types, i = cif->nargs; i > 0; i--, ptr++) - { - - /* Initialize any uninitialized aggregate type definitions */ - if (((*ptr)->size == 0) && (initialize_aggregate((*ptr)) != FFI_OK)) - return FFI_BAD_TYPEDEF; - - /* Perform a sanity check on the argument type, do this - check after the initialization. */ - FFI_ASSERT_VALID_TYPE(*ptr); - -#if !defined __x86_64__ && !defined S390 -#ifdef SPARC - if (((*ptr)->type == FFI_TYPE_STRUCT - && ((*ptr)->size > 16 || cif->abi != FFI_V9)) - || ((*ptr)->type == FFI_TYPE_LONGDOUBLE - && cif->abi != FFI_V9)) - bytes += sizeof(void*); - else -#endif - { -#ifndef _MSC_VER - /* Don't know if this is a libffi bug or not. At least on - Windows with MSVC, function call parameters are *not* - aligned in the same way as structure fields are, they are - only aligned in integer boundaries. - - This doesn't do any harm for cdecl functions and closures, - since the caller cleans up the stack, but it is wrong for - stdcall functions where the callee cleans. - */ - - /* Add any padding if necessary */ - if (((*ptr)->alignment - 1) & bytes) - bytes = ALIGN(bytes, (*ptr)->alignment); - -#endif - bytes += STACK_ARG_SIZE((*ptr)->size); - } -#endif - } - - cif->bytes = bytes; - - /* Perform machine dependent cif processing */ - return ffi_prep_cif_machdep(cif); -} diff --git a/Modules/_ctypes/libffi_arm_wince/sysv.asm b/Modules/_ctypes/libffi_arm_wince/sysv.asm deleted file mode 100644 --- a/Modules/_ctypes/libffi_arm_wince/sysv.asm +++ /dev/null @@ -1,228 +0,0 @@ -; ----------------------------------------------------------------------- -; sysv.S - Copyright (c) 1998 Red Hat, Inc. -; -; ARM Foreign Function Interface -; -; Permission is hereby granted, free of charge, to any person obtaining -; a copy of this software and associated documentation files (the -; ``Software''), to deal in the Software without restriction, including -; without limitation the rights to use, copy, modify, merge, publish, -; distribute, sublicense, and/or sell copies of the Software, and to -; permit persons to whom the Software is furnished to do so, subject to -; the following conditions: -; -; The above copyright notice and this permission notice shall be included -; in all copies or substantial portions of the Software. -; -; THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS -; OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -; MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. -; IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR -; OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -; ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -; OTHER DEALINGS IN THE SOFTWARE. -; ----------------------------------------------------------------------- */ - -;#define LIBFFI_ASM -;#include -;#include -;#ifdef HAVE_MACHINE_ASM_H -;#include -;#else -;#ifdef __USER_LABEL_PREFIX__ -;#define CONCAT1(a, b) CONCAT2(a, b) -;#define CONCAT2(a, b) a ## b - -;/* Use the right prefix for global labels. */ -;#define CNAME(x) CONCAT1 (__USER_LABEL_PREFIX__, x) -;#else -;#define CNAME(x) x -;#endif -;#define ENTRY(x) .globl CNAME(x); .type CNAME(x),%function; CNAME(x): -;#endif - - -FFI_TYPE_VOID EQU 0 -FFI_TYPE_INT EQU 1 -FFI_TYPE_FLOAT EQU 2 -FFI_TYPE_DOUBLE EQU 3 -;FFI_TYPE_LONGDOUBLE EQU 4 -FFI_TYPE_UINT8 EQU 5 -FFI_TYPE_SINT8 EQU 6 -FFI_TYPE_UINT16 EQU 7 -FFI_TYPE_SINT16 EQU 8 -FFI_TYPE_UINT32 EQU 9 -FFI_TYPE_SINT32 EQU 10 -FFI_TYPE_UINT64 EQU 11 -FFI_TYPE_SINT64 EQU 12 -FFI_TYPE_STRUCT EQU 13 -FFI_TYPE_POINTER EQU 14 - -; WinCE always uses software floating point (I think) -__SOFTFP__ EQU {TRUE} - - - AREA |.text|, CODE, ARM ; .text - - - ; a1: ffi_prep_args - ; a2: &ecif - ; a3: cif->bytes - ; a4: fig->flags - ; sp+0: ecif.rvalue - ; sp+4: fn - - ; This assumes we are using gas. -;ENTRY(ffi_call_SYSV) - - EXPORT |ffi_call_SYSV| - -|ffi_call_SYSV| PROC - - ; Save registers - stmfd sp!, {a1-a4, fp, lr} - mov fp, sp - - ; Make room for all of the new args. - sub sp, fp, a3 - - ; Place all of the ffi_prep_args in position - mov ip, a1 - mov a1, sp - ; a2 already set - - ; And call - mov lr, pc - mov pc, ip - - ; move first 4 parameters in registers - ldr a1, [sp, #0] - ldr a2, [sp, #4] - ldr a3, [sp, #8] - ldr a4, [sp, #12] - - ; and adjust stack - ldr ip, [fp, #8] - cmp ip, #16 - movge ip, #16 - add sp, sp, ip - - ; call function - mov lr, pc - ldr pc, [fp, #28] - - ; Remove the space we pushed for the args - mov sp, fp - - ; Load a3 with the pointer to storage for the return value - ldr a3, [sp, #24] - - ; Load a4 with the return type code - ldr a4, [sp, #12] - - ; If the return value pointer is NULL, assume no return value. - cmp a3, #0 - beq call_epilogue - -; return INT - cmp a4, #FFI_TYPE_INT - streq a1, [a3] - beq call_epilogue - -; return FLOAT - cmp a4, #FFI_TYPE_FLOAT - [ __SOFTFP__ ;ifdef __SOFTFP__ - streq a1, [a3] - | ;else - stfeqs f0, [a3] - ] ;endif - beq call_epilogue - -; return DOUBLE or LONGDOUBLE - cmp a4, #FFI_TYPE_DOUBLE - [ __SOFTFP__ ;ifdef __SOFTFP__ - stmeqia a3, {a1, a2} - | ;else - stfeqd f0, [a3] - ] ;endif - beq call_epilogue - -; return SINT64 or UINT64 - cmp a4, #FFI_TYPE_SINT64 - stmeqia a3, {a1, a2} - -call_epilogue - ldmfd sp!, {a1-a4, fp, pc} - -;.ffi_call_SYSV_end: - ;.size CNAME(ffi_call_SYSV),.ffi_call_SYSV_end-CNAME(ffi_call_SYSV) - ENDP - - -RESERVE_RETURN EQU 16 - - ; This function is called by the trampoline - ; It is NOT callable from C - ; ip = pointer to struct ffi_closure - - IMPORT |ffi_closure_SYSV_inner| - - EXPORT |ffi_closure_SYSV| -|ffi_closure_SYSV| PROC - - ; Store the argument registers on the stack - stmfd sp!, {a1-a4} - ; Push the return address onto the stack - stmfd sp!, {lr} - - mov a1, ip ; first arg = address of ffi_closure - add a2, sp, #4 ; second arg = sp+4 (points to saved a1) - - ; Allocate space for a non-struct return value - sub sp, sp, #RESERVE_RETURN - mov a3, sp ; third arg = return value address - - ; static unsigned int - ; ffi_closure_SYSV_inner (ffi_closure *closure, char *in_args, void *rvalue) - bl ffi_closure_SYSV_inner - ; a1 now contains the return type code - - ; At this point the return value is on the stack - ; Transfer it to the correct registers if necessary - -; return INT - cmp a1, #FFI_TYPE_INT - ldreq a1, [sp] - beq closure_epilogue - -; return FLOAT - cmp a1, #FFI_TYPE_FLOAT - [ __SOFTFP__ ;ifdef __SOFTFP__ - ldreq a1, [sp] - | ;else - stfeqs f0, [sp] - ] ;endif - beq closure_epilogue - -; return DOUBLE or LONGDOUBLE - cmp a1, #FFI_TYPE_DOUBLE - [ __SOFTFP__ ;ifdef __SOFTFP__ - ldmeqia sp, {a1, a2} - | ;else - stfeqd f0, [sp] - ] ;endif - beq closure_epilogue - -; return SINT64 or UINT64 - cmp a1, #FFI_TYPE_SINT64 - ldmeqia sp, {a1, a2} - -closure_epilogue - add sp, sp, #RESERVE_RETURN ; remove return value buffer - ldmfd sp!, {ip} ; ip = pop return address - add sp, sp, #16 ; remove saved argument registers {a1-a4} from the stack - mov pc, ip ; return - - ENDP ; ffi_closure_SYSV - - END diff --git a/PC/pyconfig.h b/PC/pyconfig.h --- a/PC/pyconfig.h +++ b/PC/pyconfig.h @@ -14,7 +14,6 @@ MS_WIN64 - Code specific to the MS Win64 API MS_WIN32 - Code specific to the MS Win32 (and Win64) API (obsolete, this covers all supported APIs) MS_WINDOWS - Code specific to Windows, but all versions. -MS_WINCE - Code specific to Windows CE Py_ENABLE_SHARED - Code if the Python core is built as a DLL. Also note that neither "_M_IX86" or "_MSC_VER" should be used for @@ -30,10 +29,6 @@ */ -#ifdef _WIN32_WCE -#define MS_WINCE -#endif - /* Deprecated USE_DL_EXPORT macro - please use Py_BUILD_CORE */ #ifdef USE_DL_EXPORT # define Py_BUILD_CORE @@ -53,8 +48,6 @@ #define _CRT_NONSTDC_NO_DEPRECATE 1 #endif -/* Windows CE does not have these */ -#ifndef MS_WINCE #define HAVE_IO_H #define HAVE_SYS_UTIME_H #define HAVE_TEMPNAM @@ -62,11 +55,8 @@ #define HAVE_TMPNAM #define HAVE_CLOCK #define HAVE_STRERROR -#endif -#ifdef HAVE_IO_H #include -#endif #define HAVE_HYPOT #define HAVE_STRFTIME @@ -86,17 +76,6 @@ #define USE_SOCKET #endif -/* CE6 doesn't have strdup() but _strdup(). Assume the same for earlier versions. */ -#if defined(MS_WINCE) -# include -# define strdup _strdup -#endif - -#ifdef MS_WINCE -/* Windows CE does not support environment variables */ -#define getenv(v) (NULL) -#define environ (NULL) -#endif /* Compiler specific defines */ @@ -448,14 +427,10 @@ /* #define const */ /* Define to 1 if you have the header file. */ -#ifndef MS_WINCE #define HAVE_CONIO_H 1 -#endif /* Define to 1 if you have the header file. */ -#ifndef MS_WINCE #define HAVE_DIRECT_H 1 -#endif /* Define if you have dirent.h. */ /* #define DIRENT 1 */ @@ -528,9 +503,7 @@ /* #define HAVE_ALTZONE */ /* Define if you have the putenv function. */ -#ifndef MS_WINCE #define HAVE_PUTENV -#endif /* Define if your compiler supports function prototypes */ #define HAVE_PROTOTYPES @@ -558,9 +531,7 @@ #define HAVE_DYNAMIC_LOADING /* Define if you have ftime. */ -#ifndef MS_WINCE #define HAVE_FTIME -#endif /* Define if you have getpeername. */ #define HAVE_GETPEERNAME @@ -569,9 +540,7 @@ /* #undef HAVE_GETPGRP */ /* Define if you have getpid. */ -#ifndef MS_WINCE #define HAVE_GETPID -#endif /* Define if you have gettimeofday. */ /* #undef HAVE_GETTIMEOFDAY */ @@ -633,14 +602,10 @@ #endif /* Define to 1 if you have the `wcscoll' function. */ -#ifndef MS_WINCE #define HAVE_WCSCOLL 1 -#endif /* Define to 1 if you have the `wcsxfrm' function. */ -#ifndef MS_WINCE #define HAVE_WCSXFRM 1 -#endif /* Define if the zlib library has inflateCopy */ #define HAVE_ZLIB_COPY 1 @@ -649,24 +614,16 @@ /* #undef HAVE_DLFCN_H */ /* Define to 1 if you have the header file. */ -#ifndef MS_WINCE #define HAVE_ERRNO_H 1 -#endif /* Define if you have the header file. */ -#ifndef MS_WINCE #define HAVE_FCNTL_H 1 -#endif /* Define to 1 if you have the header file. */ -#ifndef MS_WINCE #define HAVE_PROCESS_H 1 -#endif /* Define to 1 if you have the header file. */ -#ifndef MS_WINCE #define HAVE_SIGNAL_H 1 -#endif /* Define if you have the prototypes. */ #define HAVE_STDARG_PROTOTYPES @@ -684,9 +641,7 @@ /* #define HAVE_SYS_SELECT_H 1 */ /* Define to 1 if you have the header file. */ -#ifndef MS_WINCE #define HAVE_SYS_STAT_H 1 -#endif /* Define if you have the header file. */ /* #define HAVE_SYS_TIME_H 1 */ @@ -695,9 +650,7 @@ /* #define HAVE_SYS_TIMES_H 1 */ /* Define to 1 if you have the header file. */ -#ifndef MS_WINCE #define HAVE_SYS_TYPES_H 1 -#endif /* Define if you have the header file. */ /* #define HAVE_SYS_UN_H 1 */ diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -2000,7 +2000,7 @@ #endif #if defined(HAVE_REALPATH) wchar_t fullpath[MAXPATHLEN]; -#elif defined(MS_WINDOWS) && !defined(MS_WINCE) +#elif defined(MS_WINDOWS) wchar_t fullpath[MAX_PATH]; #endif @@ -2039,10 +2039,8 @@ #if SEP == '\\' /* Special case for MS filename syntax */ if (_HAVE_SCRIPT_ARGUMENT(argc, argv)) { wchar_t *q; -#if defined(MS_WINDOWS) && !defined(MS_WINCE) - /* This code here replaces the first element in argv with the full - path that it represents. Under CE, there are no relative paths so - the argument must be the full path anyway. */ +#if defined(MS_WINDOWS) + /* Replace the first element in argv with the full path. */ wchar_t *ptemp; if (GetFullPathNameW(argv0, Py_ARRAY_LENGTH(fullpath), diff --git a/Python/thread_nt.h b/Python/thread_nt.h --- a/Python/thread_nt.h +++ b/Python/thread_nt.h @@ -161,11 +161,7 @@ /* thunker to call adapt between the function type used by the system's thread start function and the internally used one. */ -#if defined(MS_WINCE) -static DWORD WINAPI -#else static unsigned __stdcall -#endif bootstrap(void *call) { callobj *obj = (callobj*)call; @@ -193,32 +189,18 @@ return -1; obj->func = func; obj->arg = arg; -#if defined(MS_WINCE) - hThread = CreateThread(NULL, - Py_SAFE_DOWNCAST(_pythread_stacksize, Py_ssize_t, SIZE_T), - bootstrap, obj, 0, &threadID); -#else hThread = (HANDLE)_beginthreadex(0, Py_SAFE_DOWNCAST(_pythread_stacksize, Py_ssize_t, unsigned int), bootstrap, obj, 0, &threadID); -#endif if (hThread == 0) { -#if defined(MS_WINCE) - /* Save error in variable, to prevent PyThread_get_thread_ident - from clobbering it. */ - unsigned e = GetLastError(); - dprintf(("%ld: PyThread_start_new_thread failed, win32 error code %u\n", - PyThread_get_thread_ident(), e)); -#else /* I've seen errno == EAGAIN here, which means "there are * too many threads". */ int e = errno; dprintf(("%ld: PyThread_start_new_thread failed, errno %d\n", PyThread_get_thread_ident(), e)); -#endif threadID = (unsigned)-1; HeapFree(GetProcessHeap(), 0, obj); } @@ -249,11 +231,7 @@ dprintf(("%ld: PyThread_exit_thread called\n", PyThread_get_thread_ident())); if (!initialized) exit(0); -#if defined(MS_WINCE) - ExitThread(0); -#else _endthreadex(0); -#endif } /* diff --git a/Tools/freeze/freeze.py b/Tools/freeze/freeze.py --- a/Tools/freeze/freeze.py +++ b/Tools/freeze/freeze.py @@ -124,9 +124,7 @@ # default the exclude list for each platform if win: exclude = exclude + [ - 'dos', 'dospath', 'mac', 'macpath', 'macfs', 'MACFS', 'posix', - 'ce', - ] + 'dos', 'dospath', 'mac', 'macpath', 'macfs', 'MACFS', 'posix', ] fail_import = exclude[:] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:24:53 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 22:24:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NzQ4?= =?utf-8?q?=3A_strengthen_test=5Falias=5Fnofallback?= Message-ID: <20160905222453.47480.87399.A832082C@psf.io> https://hg.python.org/cpython/rev/6137d0ed0a15 changeset: 103076:6137d0ed0a15 branch: 2.7 parent: 103069:14b611ddaabe user: Zachary Ware date: Mon Sep 05 17:19:35 2016 -0500 summary: Issue #27748: strengthen test_alias_nofallback This test should always raise RuntimeError. files: Lib/test/test_winsound.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -103,7 +103,10 @@ safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) def test_alias_nofallback(self): - safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT) + self.assertRaises(RuntimeError, + winsound.PlaySound, + '!"$%&/(#+*', + winsound.SND_ALIAS | winsound.SND_NODEFAULT) def test_stopasync(self): safe_PlaySound( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:24:53 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 22:24:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327748=3A_Merge_with_3=2E5?= Message-ID: <20160905222453.47460.38762.4D616640@psf.io> https://hg.python.org/cpython/rev/5e8b865ffedd changeset: 103078:5e8b865ffedd parent: 103075:7a2eccee823b parent: 103077:f845e24d794e user: Zachary Ware date: Mon Sep 05 17:24:45 2016 -0500 summary: Issue #27748: Merge with 3.5 files: Lib/test/test_winsound.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -120,7 +120,10 @@ safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) def test_alias_nofallback(self): - safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT) + self.assertRaises(RuntimeError, + winsound.PlaySound, + '!"$%&/(#+*', + winsound.SND_ALIAS | winsound.SND_NODEFAULT) def test_stopasync(self): safe_PlaySound( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:24:53 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 22:24:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzQ4?= =?utf-8?q?=3A_strengthen_test=5Falias=5Fnofallback?= Message-ID: <20160905222453.6504.49945.A7330CE0@psf.io> https://hg.python.org/cpython/rev/f845e24d794e changeset: 103077:f845e24d794e branch: 3.5 parent: 103067:5c75b315152b user: Zachary Ware date: Mon Sep 05 17:19:35 2016 -0500 summary: Issue #27748: strengthen test_alias_nofallback This test should always raise RuntimeError. files: Lib/test/test_winsound.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -104,7 +104,10 @@ safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) def test_alias_nofallback(self): - safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT) + self.assertRaises(RuntimeError, + winsound.PlaySound, + '!"$%&/(#+*', + winsound.SND_ALIAS | winsound.SND_NODEFAULT) def test_stopasync(self): safe_PlaySound( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:33:06 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 22:33:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325387=3A_Check_re?= =?utf-8?q?turn_value_of_winsound=2EMessageBeep?= Message-ID: <20160905223306.37966.79181.AC26EBC9@psf.io> https://hg.python.org/cpython/rev/4e5b3dc049cc changeset: 103079:4e5b3dc049cc user: Zachary Ware date: Mon Sep 05 17:32:28 2016 -0500 summary: Issue #25387: Check return value of winsound.MessageBeep files: Doc/library/winsound.rst | 3 ++- Misc/NEWS | 2 ++ PC/winsound.c | 12 +++++++++++- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/Doc/library/winsound.rst b/Doc/library/winsound.rst --- a/Doc/library/winsound.rst +++ b/Doc/library/winsound.rst @@ -40,7 +40,8 @@ sound to play; possible values are ``-1``, ``MB_ICONASTERISK``, ``MB_ICONEXCLAMATION``, ``MB_ICONHAND``, ``MB_ICONQUESTION``, and ``MB_OK``, all described below. The value ``-1`` produces a "simple beep"; this is the final - fallback if a sound cannot be played otherwise. + fallback if a sound cannot be played otherwise. If the system indicates an + error, :exc:`RuntimeError` is raised. .. data:: SND_FILENAME diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -80,6 +80,8 @@ Library ------- +- Issue #25387: Check return value of winsound.MessageBeep. + - Issue #27866: Add SSLContext.get_ciphers() method to get a list of all enabled ciphers. diff --git a/PC/winsound.c b/PC/winsound.c --- a/PC/winsound.c +++ b/PC/winsound.c @@ -175,7 +175,17 @@ winsound_MessageBeep_impl(PyObject *module, int x) /*[clinic end generated code: output=1ad89e4d8d30a957 input=a776c8a85c9853f6]*/ { - MessageBeep(x); + BOOL ok; + + Py_BEGIN_ALLOW_THREADS + ok = MessageBeep(x); + Py_END_ALLOW_THREADS + + if (!ok) { + PyErr_SetExcFromWindowsErr(PyExc_RuntimeError, 0); + return NULL; + } + Py_RETURN_NONE; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:33:59 2016 From: python-checkins at python.org (brett.cannon) Date: Mon, 05 Sep 2016 22:33:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Implement_the_frame_evalua?= =?utf-8?q?tion_API_aspect_of_PEP_523=2E?= Message-ID: <20160905223359.104172.32563.7B253400@psf.io> https://hg.python.org/cpython/rev/584855b437d4 changeset: 103080:584855b437d4 user: Brett Cannon date: Mon Sep 05 15:33:46 2016 -0700 summary: Implement the frame evaluation API aspect of PEP 523. files: Doc/whatsnew/3.6.rst | 28 ++++++++++++++++++++++++++++ Include/ceval.h | 3 +++ Include/pystate.h | 7 +++++-- Misc/NEWS | 4 +++- Python/ceval.c | 7 +++++++ Python/pystate.c | 1 + 6 files changed, 47 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -97,6 +97,34 @@ New Features ============ +.. _pep-523: + +PEP 523: Adding a frame evaluation API to CPython +================================================= + +While Python provides extensive support to customize how code +executes, one place it has not done so is in the evaluation of frame +objects. If you wanted some way to intercept frame evaluation in +Python there really wasn't any way without directly manipulating +function pointers for defined functions. + +:pep:`523` changes this by providing an API to make frame +evaluation pluggable at the C level. This will allow for tools such +as debuggers and JITs to intercept frame evaluation before the +execution of Python code begins. This enables the use of alternative +evaluation implementations for Python code, tracking frame +evaluation, etc. + +This API is not part of the limited C API and is marked as private to +signal that usage of this API is expected to be limited and only +applicable to very select, low-level use-cases. + +.. seealso:: + + :pep:`523` - Adding a frame evaluation API to CPython + PEP written by Brett Cannon and Dino Viehland. + + .. _pep-519: PEP 519: Adding a file system path protocol diff --git a/Include/ceval.h b/Include/ceval.h --- a/Include/ceval.h +++ b/Include/ceval.h @@ -119,6 +119,9 @@ PyAPI_FUNC(PyObject *) PyEval_GetCallStats(PyObject *); PyAPI_FUNC(PyObject *) PyEval_EvalFrame(struct _frame *); PyAPI_FUNC(PyObject *) PyEval_EvalFrameEx(struct _frame *f, int exc); +#ifndef Py_LIMITED_API +PyAPI_FUNC(PyObject *) _PyEval_EvalFrameDefault(struct _frame *f, int exc); +#endif /* Interface for threads. diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -12,10 +12,13 @@ struct _ts; /* Forward */ struct _is; /* Forward */ +struct _frame; /* Forward declaration for PyFrameObject. */ #ifdef Py_LIMITED_API typedef struct _is PyInterpreterState; #else +typedef PyObject* (*_PyFrameEvalFunction)(struct _frame *, int); + typedef struct _is { struct _is *next; @@ -42,14 +45,14 @@ PyObject *builtins_copy; PyObject *import_func; + /* Initialized to PyEval_EvalFrameDefault(). */ + _PyFrameEvalFunction eval_frame; } PyInterpreterState; #endif /* State unique per thread */ -struct _frame; /* Avoid including frameobject.h */ - #ifndef Py_LIMITED_API /* Py_tracefunc return -1 when raising an exception, or 0 for success. */ typedef int (*Py_tracefunc)(PyObject *, struct _frame *, int, PyObject *); diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,7 +17,9 @@ restriction: in beta 2, backslashes will only be disallowed inside the braces (where the expressions are). This is a breaking change from the 3.6 alpha releases. - + +- Implement the frame evaluation part of PEP 523. + - Issue #27870: A left shift of zero by a large integer no longer attempts to allocate large amounts of memory. diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -797,6 +797,13 @@ PyObject * PyEval_EvalFrameEx(PyFrameObject *f, int throwflag) { + PyThreadState *tstate = PyThreadState_GET(); + return tstate->interp->eval_frame(f, throwflag); +} + +PyObject * +_PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) +{ #ifdef DXPAIRS int lastopcode = 0; #endif diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -91,6 +91,7 @@ interp->fscodec_initialized = 0; interp->importlib = NULL; interp->import_func = NULL; + interp->eval_frame = _PyEval_EvalFrameDefault; #ifdef HAVE_DLOPEN #if HAVE_DECL_RTLD_NOW interp->dlopenflags = RTLD_NOW; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:38:41 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 22:38:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27744=3A_Check_for_A?= =?utf-8?q?F=5FALG_support_in_Kernel?= Message-ID: <20160905223840.47411.6504.F941E7B3@psf.io> https://hg.python.org/cpython/rev/ee32af890e27 changeset: 103081:ee32af890e27 user: Christian Heimes date: Tue Sep 06 00:37:46 2016 +0200 summary: Issue 27744: Check for AF_ALG support in Kernel files: Lib/test/test_socket.py | 15 ++++++++++++++- 1 files changed, 14 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -65,10 +65,22 @@ s.close() return True +def _have_socket_alg(): + """Check whether AF_ALG sockets are supported on this host.""" + try: + s = socket.socket(socket.AF_ALG, socket.SOCK_SEQPACKET, 0) + except (AttributeError, OSError): + return False + else: + s.close() + return True + HAVE_SOCKET_CAN = _have_socket_can() HAVE_SOCKET_RDS = _have_socket_rds() +HAVE_SOCKET_ALG = _have_socket_alg() + # Size in bytes of the int type SIZEOF_INT = array.array("i").itemsize @@ -5325,7 +5337,8 @@ def meth_from_sock(self, sock): return getattr(sock, "_sendfile_use_sendfile") - at unittest.skipUnless(hasattr(socket, "AF_ALG"), 'AF_ALG required') + + at unittest.skipUnless(HAVE_SOCKET_ALG, 'AF_ALG required') class LinuxKernelCryptoAPI(unittest.TestCase): # tests for AF_ALG def create_alg(self, typ, name): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:39:44 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 22:39:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_do_not_pretend?= =?utf-8?q?_to_support_passing_a_fd_to_access=28=29?= Message-ID: <20160905223944.68767.14606.542ACDBE@psf.io> https://hg.python.org/cpython/rev/c5f557012ef8 changeset: 103082:c5f557012ef8 branch: 3.5 parent: 103077:f845e24d794e user: Benjamin Peterson date: Mon Sep 05 15:29:33 2016 -0700 summary: do not pretend to support passing a fd to access() files: Misc/NEWS | 3 +++ Modules/clinic/posixmodule.c.h | 6 +++--- Modules/posixmodule.c | 6 +++--- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -62,6 +62,9 @@ - Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. +- Remove support for passing a file descriptor to os.access. It never worked but + previously didn't raise. + - Issue #12885: Fix error when distutils encounters symlink. - Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -96,7 +96,7 @@ "Use the real uid/gid to test for access to a path.\n" "\n" " path\n" -" Path to be tested; can be string, bytes, or open-file-descriptor int.\n" +" Path to be tested; can be string or bytes\n" " mode\n" " Operating-system mode bitfield. Can be F_OK to test existence,\n" " or the inclusive-OR of R_OK, W_OK, and X_OK.\n" @@ -132,7 +132,7 @@ { PyObject *return_value = NULL; static char *_keywords[] = {"path", "mode", "dir_fd", "effective_ids", "follow_symlinks", NULL}; - path_t path = PATH_T_INITIALIZE("access", "path", 0, 1); + path_t path = PATH_T_INITIALIZE("access", "path", 0, 0); int mode; int dir_fd = DEFAULT_DIR_FD; int effective_ids = 0; @@ -5784,4 +5784,4 @@ #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ -/*[clinic end generated code: output=c27221987f987cf3 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=9d5f831b23145d1e input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -2517,8 +2517,8 @@ /*[clinic input] os.access -> bool - path: path_t(allow_fd=True) - Path to be tested; can be string, bytes, or open-file-descriptor int. + path: path_t + Path to be tested; can be string or bytes mode: int Operating-system mode bitfield. Can be F_OK to test existence, @@ -2556,7 +2556,7 @@ static int os_access_impl(PyObject *module, path_t *path, int mode, int dir_fd, int effective_ids, int follow_symlinks) -/*[clinic end generated code: output=cf84158bc90b1a77 input=b75a756797af45ec]*/ +/*[clinic end generated code: output=cf84158bc90b1a77 input=8e8c3a6ba791fee3]*/ { int return_value; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:39:44 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 22:39:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160905223944.37962.77780.819345B0@psf.io> https://hg.python.org/cpython/rev/664f7d3b3a00 changeset: 103083:664f7d3b3a00 parent: 103081:ee32af890e27 parent: 103082:c5f557012ef8 user: Benjamin Peterson date: Mon Sep 05 15:35:45 2016 -0700 summary: merge 3.5 files: Misc/NEWS | 3 +++ Modules/clinic/posixmodule.c.h | 6 +++--- Modules/posixmodule.c | 6 +++--- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -107,6 +107,9 @@ - Issue #27842: The csv.DictReader now returns rows of type OrderedDict. (Contributed by Steve Holden.) +- Remove support for passing a file descriptor to os.access. It never worked but + previously didn't raise. + - Issue #12885: Fix error when distutils encounters symlink. - Issue #27881: Fixed possible bugs when setting sqlite3.Connection.isolation_level. diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -100,7 +100,7 @@ "Use the real uid/gid to test for access to a path.\n" "\n" " path\n" -" Path to be tested; can be string, bytes, or open-file-descriptor int.\n" +" Path to be tested; can be string or bytes\n" " mode\n" " Operating-system mode bitfield. Can be F_OK to test existence,\n" " or the inclusive-OR of R_OK, W_OK, and X_OK.\n" @@ -137,7 +137,7 @@ PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "mode", "dir_fd", "effective_ids", "follow_symlinks", NULL}; static _PyArg_Parser _parser = {"O&i|$O&pp:access", _keywords, 0}; - path_t path = PATH_T_INITIALIZE("access", "path", 0, 1); + path_t path = PATH_T_INITIALIZE("access", "path", 0, 0); int mode; int dir_fd = DEFAULT_DIR_FD; int effective_ids = 0; @@ -6042,4 +6042,4 @@ #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ -/*[clinic end generated code: output=97180b6734421a7d input=a9049054013a1b77]*/ +/*[clinic end generated code: output=2b85bb3703a6488a input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -2595,8 +2595,8 @@ /*[clinic input] os.access -> bool - path: path_t(allow_fd=True) - Path to be tested; can be string, bytes, or open-file-descriptor int. + path: path_t + Path to be tested; can be string or bytes mode: int Operating-system mode bitfield. Can be F_OK to test existence, @@ -2634,7 +2634,7 @@ static int os_access_impl(PyObject *module, path_t *path, int mode, int dir_fd, int effective_ids, int follow_symlinks) -/*[clinic end generated code: output=cf84158bc90b1a77 input=b75a756797af45ec]*/ +/*[clinic end generated code: output=cf84158bc90b1a77 input=8e8c3a6ba791fee3]*/ { int return_value; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:41:44 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 22:41:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_os=2Eaccess_does_not_allow?= =?utf-8?q?_a_fd?= Message-ID: <20160905224144.68667.26336.64BD0000@psf.io> https://hg.python.org/cpython/rev/9482afd18bb9 changeset: 103084:9482afd18bb9 user: Benjamin Peterson date: Mon Sep 05 15:40:59 2016 -0700 summary: os.access does not allow a fd files: Lib/test/test_os.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -2826,7 +2826,7 @@ functions = [ ('stat', True, (), None), ('lstat', False, (), None), - ('access', True, (os.F_OK,), None), + ('access', False, (os.F_OK,), None), ('chflags', False, (0,), None), ('lchflags', False, (0,), None), ('open', False, (0,), getattr(os, 'close', None)), -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:48:47 2016 From: python-checkins at python.org (victor.stinner) Date: Mon, 05 Sep 2016 22:48:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327938=3A_Add_a_fa?= =?utf-8?q?st-path_for_us-ascii_encoding?= Message-ID: <20160905224846.104224.15238.389BE831@psf.io> https://hg.python.org/cpython/rev/99818330b4c0 changeset: 103085:99818330b4c0 user: Victor Stinner date: Mon Sep 05 15:40:10 2016 -0700 summary: Issue #27938: Add a fast-path for us-ascii encoding Other changes: * Rewrite _Py_normalize_encoding() as a C implementation of encodings.normalize_encoding(). For example, " utf-8 " is now normalized to "utf_8". So the fast path is now used for more name variants of the same encoding. * Avoid strcpy() when encoding is NULL: call directly the UTF-8 codec files: Objects/unicodeobject.c | 176 ++++++++++++++++++--------- 1 files changed, 115 insertions(+), 61 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3100,9 +3100,9 @@ return v; } -/* Convert encoding to lower case and replace '_' with '-' in order to - catch e.g. UTF_8. Return 0 on error (encoding is longer than lower_len-1), - 1 on success. */ +/* Normalize an encoding name: C implementation of + encodings.normalize_encoding(). Return 1 on success, or 0 on error (encoding + is longer than lower_len-1). */ int _Py_normalize_encoding(const char *encoding, char *lower, @@ -3111,30 +3111,39 @@ const char *e; char *l; char *l_end; - - if (encoding == NULL) { - /* 6 == strlen("utf-8") + 1 */ - if (lower_len < 6) - return 0; - strcpy(lower, "utf-8"); - return 1; - } + int punct; + + assert(encoding != NULL); + e = encoding; l = lower; l_end = &lower[lower_len - 1]; - while (*e) { - if (l == l_end) - return 0; - if (Py_ISUPPER(*e)) { - *l++ = Py_TOLOWER(*e++); - } - else if (*e == '_') { - *l++ = '-'; - e++; + punct = 0; + while (1) { + char c = *e; + if (c == 0) { + break; + } + + if (Py_ISALNUM(c) || c == '.') { + if (punct && l != lower) { + if (l == l_end) { + return 0; + } + *l++ = '_'; + } + punct = 0; + + if (l == l_end) { + return 0; + } + *l++ = Py_TOLOWER(c); } else { - *l++ = *e++; - } + punct = 1; + } + + e++; } *l = '\0'; return 1; @@ -3148,28 +3157,51 @@ { PyObject *buffer = NULL, *unicode; Py_buffer info; - char lower[11]; /* Enough for any encoding shortcut */ + char buflower[11]; /* strlen("iso-8859-1\0") == 11, longest shortcut */ + + if (encoding == NULL) { + return PyUnicode_DecodeUTF8Stateful(s, size, errors, NULL); + } /* Shortcuts for common default encodings */ - if (_Py_normalize_encoding(encoding, lower, sizeof(lower))) { - if ((strcmp(lower, "utf-8") == 0) || - (strcmp(lower, "utf8") == 0)) - return PyUnicode_DecodeUTF8Stateful(s, size, errors, NULL); - else if ((strcmp(lower, "latin-1") == 0) || - (strcmp(lower, "latin1") == 0) || - (strcmp(lower, "iso-8859-1") == 0) || - (strcmp(lower, "iso8859-1") == 0)) - return PyUnicode_DecodeLatin1(s, size, errors); -#ifdef HAVE_MBCS - else if (strcmp(lower, "mbcs") == 0) - return PyUnicode_DecodeMBCS(s, size, errors); -#endif - else if (strcmp(lower, "ascii") == 0) - return PyUnicode_DecodeASCII(s, size, errors); - else if (strcmp(lower, "utf-16") == 0) - return PyUnicode_DecodeUTF16(s, size, errors, 0); - else if (strcmp(lower, "utf-32") == 0) - return PyUnicode_DecodeUTF32(s, size, errors, 0); + if (_Py_normalize_encoding(encoding, buflower, sizeof(buflower))) { + char *lower = buflower; + + /* Fast paths */ + if (lower[0] == 'u' && lower[1] == 't' && lower[2] == 'f') { + lower += 3; + if (*lower == '_') { + /* Match "utf8" and "utf_8" */ + lower++; + } + + if (lower[0] == '8' && lower[1] == 0) { + return PyUnicode_DecodeUTF8Stateful(s, size, errors, NULL); + } + else if (lower[0] == '1' && lower[1] == '6' && lower[2] == 0) { + return PyUnicode_DecodeUTF16(s, size, errors, 0); + } + else if (lower[0] == '3' && lower[1] == '2' && lower[2] == 0) { + return PyUnicode_DecodeUTF32(s, size, errors, 0); + } + } + else { + if (strcmp(lower, "ascii") == 0 + || strcmp(lower, "us_ascii") == 0) { + return PyUnicode_DecodeASCII(s, size, errors); + } + #ifdef HAVE_MBCS + else if (strcmp(lower, "mbcs") == 0) { + return PyUnicode_DecodeMBCS(s, size, errors); + } + #endif + else if (strcmp(lower, "latin1") == 0 + || strcmp(lower, "latin_1") == 0 + || strcmp(lower, "iso_8859_1") == 0 + || strcmp(lower, "iso8859_1") == 0) { + return PyUnicode_DecodeLatin1(s, size, errors); + } + } } /* Decode via the codec registry */ @@ -3512,34 +3544,56 @@ const char *errors) { PyObject *v; - char lower[11]; /* Enough for any encoding shortcut */ + char buflower[11]; /* strlen("iso_8859_1\0") == 11, longest shortcut */ if (!PyUnicode_Check(unicode)) { PyErr_BadArgument(); return NULL; } + if (encoding == NULL) { + return _PyUnicode_AsUTF8String(unicode, errors); + } + /* Shortcuts for common default encodings */ - if (_Py_normalize_encoding(encoding, lower, sizeof(lower))) { - if ((strcmp(lower, "utf-8") == 0) || - (strcmp(lower, "utf8") == 0)) - { - if (errors == NULL || strcmp(errors, "strict") == 0) - return _PyUnicode_AsUTF8String(unicode, NULL); - else + if (_Py_normalize_encoding(encoding, buflower, sizeof(buflower))) { + char *lower = buflower; + + /* Fast paths */ + if (lower[0] == 'u' && lower[1] == 't' && lower[2] == 'f') { + lower += 3; + if (*lower == '_') { + /* Match "utf8" and "utf_8" */ + lower++; + } + + if (lower[0] == '8' && lower[1] == 0) { return _PyUnicode_AsUTF8String(unicode, errors); - } - else if ((strcmp(lower, "latin-1") == 0) || - (strcmp(lower, "latin1") == 0) || - (strcmp(lower, "iso-8859-1") == 0) || - (strcmp(lower, "iso8859-1") == 0)) - return _PyUnicode_AsLatin1String(unicode, errors); + } + else if (lower[0] == '1' && lower[1] == '6' && lower[2] == 0) { + return _PyUnicode_EncodeUTF16(unicode, errors, 0); + } + else if (lower[0] == '3' && lower[1] == '2' && lower[2] == 0) { + return _PyUnicode_EncodeUTF32(unicode, errors, 0); + } + } + else { + if (strcmp(lower, "ascii") == 0 + || strcmp(lower, "us_ascii") == 0) { + return _PyUnicode_AsASCIIString(unicode, errors); + } #ifdef HAVE_MBCS - else if (strcmp(lower, "mbcs") == 0) - return PyUnicode_EncodeCodePage(CP_ACP, unicode, errors); -#endif - else if (strcmp(lower, "ascii") == 0) - return _PyUnicode_AsASCIIString(unicode, errors); + else if (strcmp(lower, "mbcs") == 0) { + return PyUnicode_EncodeCodePage(CP_ACP, unicode, errors); + } +#endif + else if (strcmp(lower, "latin1") == 0 || + strcmp(lower, "latin_1") == 0 || + strcmp(lower, "iso_8859_1") == 0 || + strcmp(lower, "iso8859_1") == 0) { + return _PyUnicode_AsLatin1String(unicode, errors); + } + } } /* Encode via the codec registry */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 18:59:04 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 22:59:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27744=3A_AES-CBC_and?= =?utf-8?q?_DRBG_need_Kernel_3=2E19+?= Message-ID: <20160905225904.114781.92615.5D25F4CF@psf.io> https://hg.python.org/cpython/rev/4ebe3ade6922 changeset: 103086:4ebe3ade6922 user: Christian Heimes date: Tue Sep 06 00:58:47 2016 +0200 summary: Issue 27744: AES-CBC and DRBG need Kernel 3.19+ files: Lib/test/test_socket.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -5372,6 +5372,7 @@ op.sendall(b"what do ya want for nothing?") self.assertEqual(op.recv(512), expected) + @support.requires_linux_version(3, 19) def test_aes_cbc(self): key = bytes.fromhex('06a9214036b8a15b512e03d534120006') iv = bytes.fromhex('3dafba429d9eb430b422da802c9fac41') @@ -5476,6 +5477,7 @@ res = op.recv(len(msg)) self.assertEqual(plain, res[assoclen:-taglen]) + @support.requires_linux_version(3, 19) def test_drbg_pr_sha256(self): # deterministic random bit generator, prediction resistance, sha256 with self.create_alg('rng', 'drbg_pr_sha256') as algo: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:10:51 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 23:10:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzQ4?= =?utf-8?q?=3A_Backed_out_changeset_f845e24d794e?= Message-ID: <20160905231051.68987.91092.179E419C@psf.io> https://hg.python.org/cpython/rev/e85ce70b73b3 changeset: 103088:e85ce70b73b3 branch: 3.5 parent: 103082:c5f557012ef8 user: Zachary Ware date: Mon Sep 05 18:08:27 2016 -0500 summary: Issue #27748: Backed out changeset f845e24d794e Apparently the test succeeds on XP. files: Lib/test/test_winsound.py | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -104,10 +104,7 @@ safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) def test_alias_nofallback(self): - self.assertRaises(RuntimeError, - winsound.PlaySound, - '!"$%&/(#+*', - winsound.SND_ALIAS | winsound.SND_NODEFAULT) + safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT) def test_stopasync(self): safe_PlaySound( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:10:51 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 23:10:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327748=3A_Merge_with_3=2E5?= Message-ID: <20160905231051.47372.35937.3F521559@psf.io> https://hg.python.org/cpython/rev/29300ed6dffe changeset: 103089:29300ed6dffe parent: 103086:4ebe3ade6922 parent: 103088:e85ce70b73b3 user: Zachary Ware date: Mon Sep 05 18:10:32 2016 -0500 summary: Issue #27748: Merge with 3.5 files: Lib/test/test_winsound.py | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -120,10 +120,7 @@ safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) def test_alias_nofallback(self): - self.assertRaises(RuntimeError, - winsound.PlaySound, - '!"$%&/(#+*', - winsound.SND_ALIAS | winsound.SND_NODEFAULT) + safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT) def test_stopasync(self): safe_PlaySound( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:10:51 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 23:10:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NzQ4?= =?utf-8?q?=3A_Backed_out_changeset_6137d0ed0a15?= Message-ID: <20160905231051.47534.51573.7AB75797@psf.io> https://hg.python.org/cpython/rev/fb74947843eb changeset: 103087:fb74947843eb branch: 2.7 parent: 103076:6137d0ed0a15 user: Zachary Ware date: Mon Sep 05 18:06:06 2016 -0500 summary: Issue #27748: Backed out changeset 6137d0ed0a15 Apparently the test succeeds on XP. files: Lib/test/test_winsound.py | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -103,10 +103,7 @@ safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS) def test_alias_nofallback(self): - self.assertRaises(RuntimeError, - winsound.PlaySound, - '!"$%&/(#+*', - winsound.SND_ALIAS | winsound.SND_NODEFAULT) + safe_PlaySound('!"$%&/(#+*', winsound.SND_ALIAS | winsound.SND_NODEFAULT) def test_stopasync(self): safe_PlaySound( -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:13:54 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 23:13:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2NDcw?= =?utf-8?q?=3A_Use_short_name_rather_than_name_for_compression_name_to_fix?= Message-ID: <20160905231353.37962.56197.0F49C833@psf.io> https://hg.python.org/cpython/rev/5566732c8ac5 changeset: 103090:5566732c8ac5 branch: 3.5 parent: 103088:e85ce70b73b3 user: Christian Heimes date: Tue Sep 06 01:10:39 2016 +0200 summary: Issue #26470: Use short name rather than name for compression name to fix #27958. files: Modules/_ssl.c | 7 +------ 1 files changed, 1 insertions(+), 6 deletions(-) diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -151,11 +151,6 @@ { return meth->type; } - -static const char *COMP_get_name(const COMP_METHOD *meth) -{ - return meth->name; -} #endif static pem_password_cb *SSL_CTX_get_default_passwd_cb(SSL_CTX *ctx) @@ -1644,7 +1639,7 @@ comp_method = SSL_get_current_compression(self->ssl); if (comp_method == NULL || COMP_get_type(comp_method) == NID_undef) Py_RETURN_NONE; - short_name = COMP_get_name(comp_method); + short_name = OBJ_nid2sn(COMP_get_type(comp_method)); if (short_name == NULL) Py_RETURN_NONE; return PyUnicode_DecodeFSDefault(short_name); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:13:54 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 23:13:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2NDcw?= =?utf-8?q?=3A_Use_short_name_rather_than_name_for_compression_name_to_fix?= Message-ID: <20160905231354.104224.66536.C757C617@psf.io> https://hg.python.org/cpython/rev/2593ed9a6a62 changeset: 103091:2593ed9a6a62 branch: 2.7 parent: 103087:fb74947843eb user: Christian Heimes date: Tue Sep 06 01:10:39 2016 +0200 summary: Issue #26470: Use short name rather than name for compression name to fix #27958. files: Modules/_ssl.c | 7 +------ 1 files changed, 1 insertions(+), 6 deletions(-) diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -148,11 +148,6 @@ { return meth->type; } - -static const char *COMP_get_name(const COMP_METHOD *meth) -{ - return meth->name; -} #endif static pem_password_cb *SSL_CTX_get_default_passwd_cb(SSL_CTX *ctx) @@ -1519,7 +1514,7 @@ comp_method = SSL_get_current_compression(self->ssl); if (comp_method == NULL || COMP_get_type(comp_method) == NID_undef) Py_RETURN_NONE; - short_name = COMP_get_name(comp_method); + short_name = OBJ_nid2sn(COMP_get_type(comp_method)); if (short_name == NULL) Py_RETURN_NONE; return PyBytes_FromString(short_name); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:14:45 2016 From: python-checkins at python.org (christian.heimes) Date: Mon, 05 Sep 2016 23:14:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326470=3A_Use_short_name_rather_than_name_for_co?= =?utf-8?q?mpression_name_to_fix?= Message-ID: <20160905231444.68941.23086.ADF76D15@psf.io> https://hg.python.org/cpython/rev/d92f26a53b70 changeset: 103092:d92f26a53b70 parent: 103089:29300ed6dffe parent: 103090:5566732c8ac5 user: Christian Heimes date: Tue Sep 06 01:14:34 2016 +0200 summary: Issue #26470: Use short name rather than name for compression name to fix #27958. files: Modules/_ssl.c | 7 +------ 1 files changed, 1 insertions(+), 6 deletions(-) diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -151,11 +151,6 @@ { return meth->type; } - -static const char *COMP_get_name(const COMP_METHOD *meth) -{ - return meth->name; -} #endif static pem_password_cb *SSL_CTX_get_default_passwd_cb(SSL_CTX *ctx) @@ -1714,7 +1709,7 @@ comp_method = SSL_get_current_compression(self->ssl); if (comp_method == NULL || COMP_get_type(comp_method) == NID_undef) Py_RETURN_NONE; - short_name = COMP_get_name(comp_method); + short_name = OBJ_nid2sn(COMP_get_type(comp_method)); if (short_name == NULL) Py_RETURN_NONE; return PyUnicode_DecodeFSDefault(short_name); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:19:58 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 23:19:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Move_NEWS_entry_to_correct?= =?utf-8?q?_section=2E?= Message-ID: <20160905231958.22629.40496.5C0FCEC8@psf.io> https://hg.python.org/cpython/rev/6d1ba1eb57d7 changeset: 103093:6d1ba1eb57d7 user: Zachary Ware date: Mon Sep 05 18:17:52 2016 -0500 summary: Move NEWS entry to correct section. files: Misc/NEWS | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -177,8 +177,6 @@ Build ----- -- Issue #27883: Update sqlite to 3.14.1.0 on Windows. - - Issue #27917: Set platform triplets for Android builds. - Issue #25825: Update references to the $(LIBPL) installation path on AIX. @@ -190,6 +188,9 @@ - Issue #27756: Adds new icons for Python files and processes on Windows. Designs by Cherry Wang. +- Issue #27883: Update sqlite to 3.14.1.0 on Windows. + + What's New in Python 3.6.0 alpha 4 ================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:19:58 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 05 Sep 2016 23:19:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2320366=3A_Build_f?= =?utf-8?q?ull_text_search_support_into_SQLite_on_Windows?= Message-ID: <20160905231958.8619.13312.4C94E330@psf.io> https://hg.python.org/cpython/rev/d52f10a0f10d changeset: 103094:d52f10a0f10d user: Zachary Ware date: Mon Sep 05 18:19:13 2016 -0500 summary: Closes #20366: Build full text search support into SQLite on Windows files: Misc/NEWS | 2 ++ PCbuild/sqlite3.vcxproj | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -185,6 +185,8 @@ Windows ------- +- Issue #20366: Build full text search support into SQLite on Windows. + - Issue #27756: Adds new icons for Python files and processes on Windows. Designs by Cherry Wang. diff --git a/PCbuild/sqlite3.vcxproj b/PCbuild/sqlite3.vcxproj --- a/PCbuild/sqlite3.vcxproj +++ b/PCbuild/sqlite3.vcxproj @@ -66,7 +66,7 @@ $(sqlite3Dir);%(AdditionalIncludeDirectories) - SQLITE_API=__declspec(dllexport);%(PreprocessorDefinitions) + SQLITE_ENABLE_FTS4;SQLITE_ENABLE_FTS5;SQLITE_API=__declspec(dllexport);%(PreprocessorDefinitions) Level1 @@ -86,4 +86,4 @@ - \ No newline at end of file + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:27:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 23:27:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_remove_long_do?= =?utf-8?q?uble_from_ctypes_value_union?= Message-ID: <20160905232733.10049.28638.C4939A7E@psf.io> https://hg.python.org/cpython/rev/b5c5f16432fc changeset: 103096:b5c5f16432fc branch: 2.7 parent: 103091:2593ed9a6a62 user: Benjamin Peterson date: Mon Sep 05 16:24:52 2016 -0700 summary: remove long double from ctypes value union It is unused. It also forces a 16-byte alignment, which creates problems because Python's allocator only uses 8-byte alignment. files: Modules/_ctypes/ctypes.h | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -72,7 +72,6 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG ll; #endif - long double D; }; /* @@ -344,7 +343,6 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG q; #endif - long double D; double d; float f; void *p; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:27:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 23:27:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_remove_long_do?= =?utf-8?q?uble_from_ctypes_value_union?= Message-ID: <20160905232733.32014.45825.4634446F@psf.io> https://hg.python.org/cpython/rev/8b6be1341770 changeset: 103095:8b6be1341770 branch: 3.5 parent: 103090:5566732c8ac5 user: Benjamin Peterson date: Mon Sep 05 16:24:52 2016 -0700 summary: remove long double from ctypes value union It is unused. It also forces a 16-byte alignment, which creates problems because Python's allocator only uses 8-byte alignment. files: Modules/_ctypes/ctypes.h | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -34,7 +34,6 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG ll; #endif - long double D; }; /* @@ -306,7 +305,6 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG q; #endif - long double D; double d; float f; void *p; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 19:27:34 2016 From: python-checkins at python.org (benjamin.peterson) Date: Mon, 05 Sep 2016 23:27:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160905232734.68842.58306.079AB0A2@psf.io> https://hg.python.org/cpython/rev/58b093fd094f changeset: 103097:58b093fd094f parent: 103094:d52f10a0f10d parent: 103095:8b6be1341770 user: Benjamin Peterson date: Mon Sep 05 16:27:08 2016 -0700 summary: merge 3.5 files: Modules/_ctypes/ctypes.h | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -34,7 +34,6 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG ll; #endif - long double D; }; /* @@ -306,7 +305,6 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG q; #endif - long double D; double d; float f; void *p; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 20:17:23 2016 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 06 Sep 2016 00:17:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Update_pkgutil?= =?utf-8?q?_docs_to_reference_appropriate_finder_and_loader_object?= Message-ID: <20160906001723.32014.72211.16CE4F97@psf.io> https://hg.python.org/cpython/rev/ecbad01262c8 changeset: 103098:ecbad01262c8 branch: 3.5 parent: 103095:8b6be1341770 user: Senthil Kumaran date: Mon Sep 05 17:11:51 2016 -0700 summary: Update pkgutil docs to reference appropriate finder and loader object documentation. Initial patch contributed by Jaysinh shukla. files: Doc/library/pkgutil.rst | 28 ++++++++++++++-------------- Lib/pkgutil.py | 8 ++++---- 2 files changed, 18 insertions(+), 18 deletions(-) diff --git a/Doc/library/pkgutil.rst b/Doc/library/pkgutil.rst --- a/Doc/library/pkgutil.rst +++ b/Doc/library/pkgutil.rst @@ -46,10 +46,10 @@ .. class:: ImpImporter(dirname=None) - :pep:`302` Importer that wraps Python's "classic" import algorithm. + :pep:`302` Finder that wraps Python's "classic" import algorithm. - If *dirname* is a string, a :pep:`302` importer is created that searches that - directory. If *dirname* is ``None``, a :pep:`302` importer is created that + If *dirname* is a string, a :pep:`302` finder is created that searches that + directory. If *dirname* is ``None``, a :pep:`302` finder is created that searches the current :data:`sys.path`, plus any modules that are frozen or built-in. @@ -63,7 +63,7 @@ .. class:: ImpLoader(fullname, file, filename, etc) - :pep:`302` Loader that wraps Python's "classic" import algorithm. + :term:`Loader` that wraps Python's "classic" import algorithm. .. deprecated:: 3.3 This emulation is no longer needed, as the standard import mechanism @@ -72,7 +72,7 @@ .. function:: find_loader(fullname) - Retrieve a :pep:`302` module loader for the given *fullname*. + Retrieve a module :term:`loader` for the given *fullname*. This is a backwards compatibility wrapper around :func:`importlib.util.find_spec` that converts most failures to @@ -88,9 +88,9 @@ .. function:: get_importer(path_item) - Retrieve a :pep:`302` importer for the given *path_item*. + Retrieve a :term:`finder` for the given *path_item*. - The returned importer is cached in :data:`sys.path_importer_cache` if it was + The returned finder is cached in :data:`sys.path_importer_cache` if it was newly created by a path hook. The cache (or part of it) can be cleared manually if a rescan of @@ -103,7 +103,7 @@ .. function:: get_loader(module_or_name) - Get a :pep:`302` "loader" object for *module_or_name*. + Get a :term:`loader` object for *module_or_name*. If the module or package is accessible via the normal import mechanism, a wrapper around the relevant part of that machinery is returned. Returns @@ -121,16 +121,16 @@ .. function:: iter_importers(fullname='') - Yield :pep:`302` importers for the given module name. + Yield :term:`finder` objects for the given module name. - If fullname contains a '.', the importers will be for the package + If fullname contains a '.', the finders will be for the package containing fullname, otherwise they will be all registered top level - importers (i.e. those on both sys.meta_path and sys.path_hooks). + finders (i.e. those on both sys.meta_path and sys.path_hooks). If the named module is in a package, that package is imported as a side effect of invoking this function. - If no module name is specified, all top level importers are produced. + If no module name is specified, all top level finders are produced. .. versionchanged:: 3.3 Updated to be based directly on :mod:`importlib` rather than relying @@ -201,7 +201,7 @@ Get a resource from a package. - This is a wrapper for the :pep:`302` loader :func:`get_data` API. The + This is a wrapper for the :term:`loader` :func:`get_data` API. The *package* argument should be the name of a package, in standard module format (``foo.bar``). The *resource* argument should be in the form of a relative filename, using ``/`` as the path separator. The parent directory name @@ -216,5 +216,5 @@ d = os.path.dirname(sys.modules[package].__file__) data = open(os.path.join(d, resource), 'rb').read() - If the package cannot be located or loaded, or it uses a :pep:`302` loader + If the package cannot be located or loaded, or it uses a :term:`loader` which does not support :func:`get_data`, then ``None`` is returned. diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py --- a/Lib/pkgutil.py +++ b/Lib/pkgutil.py @@ -395,7 +395,7 @@ def get_importer(path_item): - """Retrieve a PEP 302 importer for the given path item + """Retrieve a finder for the given path item The returned importer is cached in sys.path_importer_cache if it was newly created by a path hook. @@ -419,7 +419,7 @@ def iter_importers(fullname=""): - """Yield PEP 302 importers for the given module name + """Yield finders for the given module name If fullname contains a '.', the importers will be for the package containing fullname, otherwise they will be all registered top level @@ -448,7 +448,7 @@ def get_loader(module_or_name): - """Get a PEP 302 "loader" object for module_or_name + """Get a "loader" object for module_or_name Returns None if the module cannot be found or imported. If the named module is not already imported, its containing package @@ -472,7 +472,7 @@ def find_loader(fullname): - """Find a PEP 302 "loader" object for fullname + """Find a "loader" object for fullname This is a backwards compatibility wrapper around importlib.util.find_spec that converts most failures to ImportError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 20:17:23 2016 From: python-checkins at python.org (senthil.kumaran) Date: Tue, 06 Sep 2016 00:17:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_=5Bmerge_from_3=2E5=5D_-_Update_pkgutil_docs_to_referenc?= =?utf-8?q?e_appropriate_finder_and?= Message-ID: <20160906001723.8269.91115.21F07D69@psf.io> https://hg.python.org/cpython/rev/ee58ece83391 changeset: 103099:ee58ece83391 parent: 103097:58b093fd094f parent: 103098:ecbad01262c8 user: Senthil Kumaran date: Mon Sep 05 17:16:06 2016 -0700 summary: [merge from 3.5] - Update pkgutil docs to reference appropriate finder and loader object documentation. Initial patch contributed by Jaysinh shukla. files: Doc/library/pkgutil.rst | 14 +++++++------- Lib/pkgutil.py | 8 ++++---- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/Doc/library/pkgutil.rst b/Doc/library/pkgutil.rst --- a/Doc/library/pkgutil.rst +++ b/Doc/library/pkgutil.rst @@ -63,7 +63,7 @@ .. class:: ImpLoader(fullname, file, filename, etc) - :pep:`302` Loader that wraps Python's "classic" import algorithm. + :term:`Loader` that wraps Python's "classic" import algorithm. .. deprecated:: 3.3 This emulation is no longer needed, as the standard import mechanism @@ -72,7 +72,7 @@ .. function:: find_loader(fullname) - Retrieve a :pep:`302` module loader for the given *fullname*. + Retrieve a module :term:`loader` for the given *fullname*. This is a backwards compatibility wrapper around :func:`importlib.util.find_spec` that converts most failures to @@ -88,7 +88,7 @@ .. function:: get_importer(path_item) - Retrieve a :pep:`302` finder for the given *path_item*. + Retrieve a :term:`finder` for the given *path_item*. The returned finder is cached in :data:`sys.path_importer_cache` if it was newly created by a path hook. @@ -103,7 +103,7 @@ .. function:: get_loader(module_or_name) - Get a :pep:`302` "loader" object for *module_or_name*. + Get a :term:`loader` object for *module_or_name*. If the module or package is accessible via the normal import mechanism, a wrapper around the relevant part of that machinery is returned. Returns @@ -121,7 +121,7 @@ .. function:: iter_importers(fullname='') - Yield :pep:`302` finders for the given module name. + Yield :term:`finder` objects for the given module name. If fullname contains a '.', the finders will be for the package containing fullname, otherwise they will be all registered top level @@ -201,7 +201,7 @@ Get a resource from a package. - This is a wrapper for the :pep:`302` loader :func:`get_data` API. The + This is a wrapper for the :term:`loader` :func:`get_data` API. The *package* argument should be the name of a package, in standard module format (``foo.bar``). The *resource* argument should be in the form of a relative filename, using ``/`` as the path separator. The parent directory name @@ -216,5 +216,5 @@ d = os.path.dirname(sys.modules[package].__file__) data = open(os.path.join(d, resource), 'rb').read() - If the package cannot be located or loaded, or it uses a :pep:`302` loader + If the package cannot be located or loaded, or it uses a :term:`loader` which does not support :func:`get_data`, then ``None`` is returned. diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py --- a/Lib/pkgutil.py +++ b/Lib/pkgutil.py @@ -395,7 +395,7 @@ def get_importer(path_item): - """Retrieve a PEP 302 finder for the given path item + """Retrieve a finder for the given path item The returned finder is cached in sys.path_importer_cache if it was newly created by a path hook. @@ -419,7 +419,7 @@ def iter_importers(fullname=""): - """Yield PEP 302 finders for the given module name + """Yield finders for the given module name If fullname contains a '.', the finders will be for the package containing fullname, otherwise they will be all registered top level @@ -448,7 +448,7 @@ def get_loader(module_or_name): - """Get a PEP 302 "loader" object for module_or_name + """Get a "loader" object for module_or_name Returns None if the module cannot be found or imported. If the named module is not already imported, its containing package @@ -472,7 +472,7 @@ def find_loader(fullname): - """Find a PEP 302 "loader" object for fullname + """Find a "loader" object for fullname This is a backwards compatibility wrapper around importlib.util.find_spec that converts most failures to ImportError -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 20:22:31 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 00:22:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Backed_out_cha?= =?utf-8?q?ngeset_8b6be1341770?= Message-ID: <20160906002231.47632.75581.5299A890@psf.io> https://hg.python.org/cpython/rev/65102408021e changeset: 103101:65102408021e branch: 2.7 parent: 103096:b5c5f16432fc user: Benjamin Peterson date: Mon Sep 05 17:22:09 2016 -0700 summary: Backed out changeset 8b6be1341770 files: Modules/_ctypes/ctypes.h | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -72,6 +72,7 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG ll; #endif + long double D; }; /* @@ -343,6 +344,7 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG q; #endif + long double D; double d; float f; void *p; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 20:22:31 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 00:22:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Backed_out_cha?= =?utf-8?q?ngeset_8b6be1341770?= Message-ID: <20160906002231.8382.69743.CD63D2DF@psf.io> https://hg.python.org/cpython/rev/f858d4ee1b2d changeset: 103100:f858d4ee1b2d branch: 3.5 parent: 103098:ecbad01262c8 user: Benjamin Peterson date: Mon Sep 05 17:22:09 2016 -0700 summary: Backed out changeset 8b6be1341770 files: Modules/_ctypes/ctypes.h | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -34,6 +34,7 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG ll; #endif + long double D; }; /* @@ -305,6 +306,7 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG q; #endif + long double D; double d; float f; void *p; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 20:22:36 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 00:22:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160906002236.37924.71982.011D87E6@psf.io> https://hg.python.org/cpython/rev/c259ae9ebdd5 changeset: 103102:c259ae9ebdd5 parent: 103099:ee58ece83391 parent: 103100:f858d4ee1b2d user: Benjamin Peterson date: Mon Sep 05 17:22:22 2016 -0700 summary: merge 3.5 files: Modules/_ctypes/ctypes.h | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -34,6 +34,7 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG ll; #endif + long double D; }; /* @@ -305,6 +306,7 @@ #ifdef HAVE_LONG_LONG PY_LONG_LONG q; #endif + long double D; double d; float f; void *p; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 20:25:09 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 06 Sep 2016 00:25:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Deprecate_Tix?= Message-ID: <20160906002509.8656.18751.00091C8E@psf.io> https://hg.python.org/cpython/rev/0109092fc9e9 changeset: 103103:0109092fc9e9 user: Zachary Ware date: Mon Sep 05 17:22:24 2016 -0700 summary: Deprecate Tix When building it breaks, we won't be fixing it. files: Doc/library/tkinter.tix.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/library/tkinter.tix.rst b/Doc/library/tkinter.tix.rst --- a/Doc/library/tkinter.tix.rst +++ b/Doc/library/tkinter.tix.rst @@ -10,6 +10,10 @@ .. index:: single: Tix +.. deprecated:: 3.6 + This Tk extension is unmaintained and should not be used in new code. Use + :mod:`tkinter.ttk` instead. + -------------- The :mod:`tkinter.tix` (Tk Interface Extension) module provides an additional -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 20:31:51 2016 From: python-checkins at python.org (ned.deily) Date: Tue, 06 Sep 2016 00:31:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Update_OS_X_installer_to_u?= =?utf-8?q?se_SQlite_3=2E14=2E1_and_XZ_5=2E2=2E2=2E?= Message-ID: <20160906003151.8619.62283.B2DBD97D@psf.io> https://hg.python.org/cpython/rev/31f0a38b7c53 changeset: 103104:31f0a38b7c53 user: Ned Deily date: Mon Sep 05 17:31:14 2016 -0700 summary: Update OS X installer to use SQlite 3.14.1 and XZ 5.2.2. files: Mac/BuildScript/build-installer.py | 13 +++++++------ Misc/NEWS | 2 ++ 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -299,9 +299,9 @@ if PYTHON_3: result.extend([ dict( - name="XZ 5.0.5", - url="http://tukaani.org/xz/xz-5.0.5.tar.gz", - checksum='19d924e066b6fff0bc9d1981b4e53196', + name="XZ 5.2.2", + url="http://tukaani.org/xz/xz-5.2.2.tar.gz", + checksum='7cf6a8544a7dae8e8106fdf7addfa28c', configure_pre=[ '--disable-dependency-tracking', ] @@ -344,10 +344,11 @@ ), ), dict( - name="SQLite 3.8.11", - url="https://www.sqlite.org/2015/sqlite-autoconf-3081100.tar.gz", - checksum='77b451925121028befbddbf45ea2bc49', + name="SQLite 3.14.1", + url="https://www.sqlite.org/2016/sqlite-autoconf-3140100.tar.gz", + checksum='3634a90a3f49541462bcaed3474b2684', extra_cflags=('-Os ' + '-DSQLITE_ENABLE_FTS5 ' '-DSQLITE_ENABLE_FTS4 ' '-DSQLITE_ENABLE_FTS3_PARENTHESIS ' '-DSQLITE_ENABLE_RTREE ' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -182,6 +182,8 @@ - Issue #25825: Update references to the $(LIBPL) installation path on AIX. This path was changed in 3.2a4. +- Update OS X installer to use SQLite 3.14.1 and XZ 5.2.2. + Windows ------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 20:44:57 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 00:44:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_require_a_long_long_data_t?= =?utf-8?q?ype_=28closes_=2327961=29?= Message-ID: <20160906004457.30568.58162.8C4B934F@psf.io> https://hg.python.org/cpython/rev/9206a86f7321 changeset: 103105:9206a86f7321 user: Benjamin Peterson date: Mon Sep 05 17:44:18 2016 -0700 summary: require a long long data type (closes #27961) files: Doc/c-api/unicode.rst | 5 - Include/longobject.h | 2 - Include/pyport.h | 25 +- Include/pythread.h | 5 - Include/structmember.h | 2 - Lib/test/test_struct.py | 21 +- Misc/NEWS | 3 + Modules/_ctypes/_ctypes_test.c | 3 - Modules/_ctypes/callproc.c | 4 - Modules/_ctypes/cfield.c | 14 +- Modules/_ctypes/ctypes.h | 4 - Modules/_io/_iomodule.h | 2 +- Modules/_lsprof.c | 4 - Modules/_multiprocessing/multiprocessing.h | 2 +- Modules/_sqlite/util.c | 19 - Modules/_struct.c | 43 - Modules/_testcapimodule.c | 28 +- Modules/_tkinter.c | 2 - Modules/addrinfo.h | 8 - Modules/arraymodule.c | 11 - Modules/posixmodule.c | 10 +- Modules/resource.c | 4 - Objects/longobject.c | 20 - Objects/memoryobject.c | 12 - Objects/unicodeobject.c | 6 - Python/getargs.c | 4 - Python/modsupport.c | 3 +- Python/pytime.c | 16 +- Python/structmember.c | 4 - configure | 227 ++++----- configure.ac | 128 ++--- pyconfig.h.in | 3 - 32 files changed, 179 insertions(+), 465 deletions(-) diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -525,11 +525,6 @@ copied as-is to the result string, and any extra arguments discarded. .. note:: - - The `"%lld"` and `"%llu"` format specifiers are only available - when :const:`HAVE_LONG_LONG` is defined. - - .. note:: The width formatter unit is number of characters rather than bytes. The precision formatter unit is number of bytes for ``"%s"`` and ``"%V"`` (if the ``PyObject*`` argument is NULL), and a number of diff --git a/Include/longobject.h b/Include/longobject.h --- a/Include/longobject.h +++ b/Include/longobject.h @@ -85,14 +85,12 @@ PyAPI_FUNC(PyObject *) PyLong_FromVoidPtr(void *); PyAPI_FUNC(void *) PyLong_AsVoidPtr(PyObject *); -#ifdef HAVE_LONG_LONG PyAPI_FUNC(PyObject *) PyLong_FromLongLong(PY_LONG_LONG); PyAPI_FUNC(PyObject *) PyLong_FromUnsignedLongLong(unsigned PY_LONG_LONG); PyAPI_FUNC(PY_LONG_LONG) PyLong_AsLongLong(PyObject *); PyAPI_FUNC(unsigned PY_LONG_LONG) PyLong_AsUnsignedLongLong(PyObject *); PyAPI_FUNC(unsigned PY_LONG_LONG) PyLong_AsUnsignedLongLongMask(PyObject *); PyAPI_FUNC(PY_LONG_LONG) PyLong_AsLongLongAndOverflow(PyObject *, int *); -#endif /* HAVE_LONG_LONG */ PyAPI_FUNC(PyObject *) PyLong_FromString(const char *, char **, int); #ifndef Py_LIMITED_API diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -35,10 +35,6 @@ Meaning: The C9X type uintptr_t is supported by the compiler Used in: Py_uintptr_t -HAVE_LONG_LONG -Meaning: The compiler supports the C type "long long" -Used in: PY_LONG_LONG - **************************************************************************/ /* typedefs for some C9X-defined synonyms for integral types. @@ -53,7 +49,6 @@ * integral synonyms. Only define the ones we actually need. */ -#ifdef HAVE_LONG_LONG #ifndef PY_LONG_LONG #define PY_LONG_LONG long long #if defined(LLONG_MAX) @@ -78,7 +73,6 @@ #define PY_ULLONG_MAX (PY_LLONG_MAX * Py_ULL(2) + 1) #endif /* LLONG_MAX */ #endif -#endif /* HAVE_LONG_LONG */ /* a build with 30-bit digits for Python integers needs an exact-width * 32-bit unsigned integer type to store those digits. (We could just use @@ -161,7 +155,7 @@ typedef unsigned long Py_uintptr_t; typedef long Py_intptr_t; -#elif defined(HAVE_LONG_LONG) && (SIZEOF_VOID_P <= SIZEOF_LONG_LONG) +#elif SIZEOF_VOID_P <= SIZEOF_LONG_LONG typedef unsigned PY_LONG_LONG Py_uintptr_t; typedef PY_LONG_LONG Py_intptr_t; @@ -248,19 +242,16 @@ #endif /* PY_FORMAT_LONG_LONG is analogous to PY_FORMAT_SIZE_T above, but for - * the long long type instead of the size_t type. It's only available - * when HAVE_LONG_LONG is defined. The "high level" Python format + * the long long type instead of the size_t type. The "high level" Python format * functions listed above will interpret "lld" or "llu" correctly on * all platforms. */ -#ifdef HAVE_LONG_LONG -# ifndef PY_FORMAT_LONG_LONG -# ifdef MS_WINDOWS -# define PY_FORMAT_LONG_LONG "I64" -# else -# error "This platform's pyconfig.h needs to define PY_FORMAT_LONG_LONG" -# endif -# endif +#ifndef PY_FORMAT_LONG_LONG +# ifdef MS_WINDOWS +# define PY_FORMAT_LONG_LONG "I64" +# else +# error "This platform's pyconfig.h needs to define PY_FORMAT_LONG_LONG" +# endif #endif /* Py_LOCAL can be used instead of static to get the fastest possible calling diff --git a/Include/pythread.h b/Include/pythread.h --- a/Include/pythread.h +++ b/Include/pythread.h @@ -37,13 +37,8 @@ module exposes a higher-level API, with timeouts expressed in seconds and floating-point numbers allowed. */ -#if defined(HAVE_LONG_LONG) #define PY_TIMEOUT_T PY_LONG_LONG #define PY_TIMEOUT_MAX PY_LLONG_MAX -#else -#define PY_TIMEOUT_T long -#define PY_TIMEOUT_MAX LONG_MAX -#endif /* In the NT API, the timeout is a DWORD and is expressed in milliseconds */ #if defined (NT_THREADS) diff --git a/Include/structmember.h b/Include/structmember.h --- a/Include/structmember.h +++ b/Include/structmember.h @@ -49,10 +49,8 @@ #define T_OBJECT_EX 16 /* Like T_OBJECT, but raises AttributeError when the value is NULL, instead of converting to None. */ -#ifdef HAVE_LONG_LONG #define T_LONGLONG 17 #define T_ULONGLONG 18 -#endif /* HAVE_LONG_LONG */ #define T_PYSSIZET 19 /* Py_ssize_t */ #define T_NONE 20 /* Value is always None */ diff --git a/Lib/test/test_struct.py b/Lib/test/test_struct.py --- a/Lib/test/test_struct.py +++ b/Lib/test/test_struct.py @@ -16,22 +16,10 @@ def iter_integer_formats(byteorders=byteorders): for code in integer_codes: for byteorder in byteorders: - if (byteorder in ('', '@') and code in ('q', 'Q') and - not HAVE_LONG_LONG): - continue if (byteorder not in ('', '@') and code in ('n', 'N')): continue yield code, byteorder -# Native 'q' packing isn't available on systems that don't have the C -# long long type. -try: - struct.pack('q', 5) -except struct.error: - HAVE_LONG_LONG = False -else: - HAVE_LONG_LONG = True - def string_reverse(s): return s[::-1] @@ -159,9 +147,7 @@ self.assertEqual(size, expected_size[code]) # native integer sizes - native_pairs = 'bB', 'hH', 'iI', 'lL', 'nN' - if HAVE_LONG_LONG: - native_pairs += 'qQ', + native_pairs = 'bB', 'hH', 'iI', 'lL', 'nN', 'qQ' for format_pair in native_pairs: for byteorder in '', '@': signed_size = struct.calcsize(byteorder + format_pair[0]) @@ -174,9 +160,8 @@ self.assertLessEqual(4, struct.calcsize('l')) self.assertLessEqual(struct.calcsize('h'), struct.calcsize('i')) self.assertLessEqual(struct.calcsize('i'), struct.calcsize('l')) - if HAVE_LONG_LONG: - self.assertLessEqual(8, struct.calcsize('q')) - self.assertLessEqual(struct.calcsize('l'), struct.calcsize('q')) + self.assertLessEqual(8, struct.calcsize('q')) + self.assertLessEqual(struct.calcsize('l'), struct.calcsize('q')) self.assertGreaterEqual(struct.calcsize('n'), struct.calcsize('i')) self.assertGreaterEqual(struct.calcsize('n'), struct.calcsize('P')) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27961?: Require platforms to support ``long long``. Python hasn't + compiled without ``long long`` for years, so this is basically a formality. + - Issue #27355: Removed support for Windows CE. It was never finished, and Windows CE is no longer a relevant platform for Python. diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -233,7 +233,6 @@ return (*func)(table); } -#ifdef HAVE_LONG_LONG EXPORT(PY_LONG_LONG) _testfunc_q_bhilfdq(signed char b, short h, int i, long l, float f, double d, PY_LONG_LONG q) { @@ -267,8 +266,6 @@ return sum; } -#endif - typedef struct { char *name; char *value; diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -474,7 +474,6 @@ self->tag, self->value.l); break; -#ifdef HAVE_LONG_LONG case 'q': case 'Q': sprintf(buffer, @@ -485,7 +484,6 @@ #endif self->tag, self->value.q); break; -#endif case 'd': sprintf(buffer, "", self->tag, self->value.d); @@ -593,9 +591,7 @@ short h; int i; long l; -#ifdef HAVE_LONG_LONG PY_LONG_LONG q; -#endif long double D; double d; float f; diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c --- a/Modules/_ctypes/cfield.c +++ b/Modules/_ctypes/cfield.c @@ -379,8 +379,6 @@ return 0; } -#ifdef HAVE_LONG_LONG - /* Same, but handling native long long. */ static int @@ -417,8 +415,6 @@ return 0; } -#endif - /***************************************************************** * Integer fields, with bitfield support */ @@ -888,7 +884,6 @@ return PyLong_FromUnsignedLong(val); } -#ifdef HAVE_LONG_LONG static PyObject * q_set(void *ptr, PyObject *value, Py_ssize_t size) { @@ -982,7 +977,6 @@ GET_BITFIELD(val, size); return PyLong_FromUnsignedLongLong(val); } -#endif /***************************************************************** * non-integer accessor methods, not supporting bit fields @@ -1490,9 +1484,7 @@ #if SIZEOF_VOID_P <= SIZEOF_LONG v = (void *)PyLong_AsUnsignedLongMask(value); #else -#ifndef HAVE_LONG_LONG -# error "PyLong_AsVoidPtr: sizeof(void*) > sizeof(long), but no long long" -#elif SIZEOF_LONG_LONG < SIZEOF_VOID_P +#if SIZEOF_LONG_LONG < SIZEOF_VOID_P # error "PyLong_AsVoidPtr: sizeof(PY_LONG_LONG) < sizeof(void*)" #endif v = (void *)PyLong_AsUnsignedLongLongMask(value); @@ -1538,14 +1530,12 @@ #else # error #endif -#ifdef HAVE_LONG_LONG #if SIZEOF_LONG_LONG == 8 { 'q', q_set, q_get, &ffi_type_sint64, q_set_sw, q_get_sw}, { 'Q', Q_set, Q_get, &ffi_type_uint64, Q_set_sw, Q_get_sw}, #else # error #endif -#endif { 'P', P_set, P_get, &ffi_type_pointer}, { 'z', z_set, z_get, &ffi_type_pointer}, #ifdef CTYPES_UNICODE @@ -1635,10 +1625,8 @@ #endif */ -#ifdef HAVE_LONG_LONG typedef struct { char c; PY_LONG_LONG x; } s_long_long; #define LONG_LONG_ALIGN (sizeof(s_long_long) - sizeof(PY_LONG_LONG)) -#endif /* from ffi.h: typedef struct _ffi_type diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -31,9 +31,7 @@ long l; float f; double d; -#ifdef HAVE_LONG_LONG PY_LONG_LONG ll; -#endif long double D; }; @@ -303,9 +301,7 @@ short h; int i; long l; -#ifdef HAVE_LONG_LONG PY_LONG_LONG q; -#endif long double D; double d; float f; diff --git a/Modules/_io/_iomodule.h b/Modules/_io/_iomodule.h --- a/Modules/_io/_iomodule.h +++ b/Modules/_io/_iomodule.h @@ -104,7 +104,7 @@ # define PY_OFF_T_MIN PY_SSIZE_T_MIN # define PY_OFF_T_COMPAT Py_ssize_t # define PY_PRIdOFF "zd" -#elif (HAVE_LONG_LONG && SIZEOF_OFF_T == SIZEOF_LONG_LONG) +#elif (SIZEOF_OFF_T == SIZEOF_LONG_LONG) # define PyLong_AsOff_t PyLong_AsLongLong # define PyLong_FromOff_t PyLong_FromLongLong # define PY_OFF_T_MAX PY_LLONG_MAX diff --git a/Modules/_lsprof.c b/Modules/_lsprof.c --- a/Modules/_lsprof.c +++ b/Modules/_lsprof.c @@ -2,10 +2,6 @@ #include "frameobject.h" #include "rotatingtree.h" -#if !defined(HAVE_LONG_LONG) -#error "This module requires long longs!" -#endif - /*** Selection of a high-precision timer ***/ #ifdef MS_WINDOWS diff --git a/Modules/_multiprocessing/multiprocessing.h b/Modules/_multiprocessing/multiprocessing.h --- a/Modules/_multiprocessing/multiprocessing.h +++ b/Modules/_multiprocessing/multiprocessing.h @@ -60,7 +60,7 @@ #if SIZEOF_VOID_P == SIZEOF_LONG # define F_POINTER "k" # define T_POINTER T_ULONG -#elif defined(HAVE_LONG_LONG) && (SIZEOF_VOID_P == SIZEOF_LONG_LONG) +#elif SIZEOF_VOID_P == SIZEOF_LONG_LONG # define F_POINTER "K" # define T_POINTER T_ULONGLONG #else diff --git a/Modules/_sqlite/util.c b/Modules/_sqlite/util.c --- a/Modules/_sqlite/util.c +++ b/Modules/_sqlite/util.c @@ -113,7 +113,6 @@ PyObject * _pysqlite_long_from_int64(sqlite_int64 value) { -#ifdef HAVE_LONG_LONG # if SIZEOF_LONG_LONG < 8 if (value > PY_LLONG_MAX || value < PY_LLONG_MIN) { return _PyLong_FromByteArray(&value, sizeof(value), @@ -124,14 +123,6 @@ if (value > LONG_MAX || value < LONG_MIN) return PyLong_FromLongLong(value); # endif -#else -# if SIZEOF_LONG < 8 - if (value > LONG_MAX || value < LONG_MIN) { - return _PyLong_FromByteArray(&value, sizeof(value), - IS_LITTLE_ENDIAN, 1 /* signed */); - } -# endif -#endif return PyLong_FromLong(Py_SAFE_DOWNCAST(value, sqlite_int64, long)); } @@ -139,23 +130,13 @@ _pysqlite_long_as_int64(PyObject * py_val) { int overflow; -#ifdef HAVE_LONG_LONG PY_LONG_LONG value = PyLong_AsLongLongAndOverflow(py_val, &overflow); -#else - long value = PyLong_AsLongAndOverflow(py_val, &overflow); -#endif if (value == -1 && PyErr_Occurred()) return -1; if (!overflow) { -#ifdef HAVE_LONG_LONG # if SIZEOF_LONG_LONG > 8 if (-0x8000000000000000LL <= value && value <= 0x7FFFFFFFFFFFFFFFLL) # endif -#else -# if SIZEOF_LONG > 8 - if (-0x8000000000000000L <= value && value <= 0x7FFFFFFFFFFFFFFFL) -# endif -#endif return value; } else if (sizeof(value) < sizeof(sqlite_int64)) { diff --git a/Modules/_struct.c b/Modules/_struct.c --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -71,10 +71,8 @@ /* We can't support q and Q in native mode unless the compiler does; in std mode, they're 8 bytes on all platforms. */ -#ifdef HAVE_LONG_LONG typedef struct { char c; PY_LONG_LONG x; } s_long_long; #define LONG_LONG_ALIGN (sizeof(s_long_long) - sizeof(PY_LONG_LONG)) -#endif #ifdef HAVE_C99_BOOL #define BOOL_TYPE _Bool @@ -164,8 +162,6 @@ return 0; } -#ifdef HAVE_LONG_LONG - /* Same, but handling native long long. */ static int @@ -212,8 +208,6 @@ return 0; } -#endif - /* Same, but handling Py_ssize_t */ static int @@ -463,8 +457,6 @@ /* Native mode doesn't support q or Q unless the platform C supports long long (or, on Windows, __int64). */ -#ifdef HAVE_LONG_LONG - static PyObject * nu_longlong(const char *p, const formatdef *f) { @@ -485,8 +477,6 @@ return PyLong_FromUnsignedLongLong(x); } -#endif - static PyObject * nu_bool(const char *p, const formatdef *f) { @@ -680,8 +670,6 @@ return 0; } -#ifdef HAVE_LONG_LONG - static int np_longlong(char *p, PyObject *v, const formatdef *f) { @@ -701,7 +689,6 @@ memcpy(p, (char *)&x, sizeof x); return 0; } -#endif static int @@ -785,10 +772,8 @@ {'L', sizeof(long), LONG_ALIGN, nu_ulong, np_ulong}, {'n', sizeof(size_t), SIZE_T_ALIGN, nu_ssize_t, np_ssize_t}, {'N', sizeof(size_t), SIZE_T_ALIGN, nu_size_t, np_size_t}, -#ifdef HAVE_LONG_LONG {'q', sizeof(PY_LONG_LONG), LONG_LONG_ALIGN, nu_longlong, np_longlong}, {'Q', sizeof(PY_LONG_LONG), LONG_LONG_ALIGN, nu_ulonglong,np_ulonglong}, -#endif {'?', sizeof(BOOL_TYPE), BOOL_ALIGN, nu_bool, np_bool}, {'e', sizeof(short), SHORT_ALIGN, nu_halffloat, np_halffloat}, {'f', sizeof(float), FLOAT_ALIGN, nu_float, np_float}, @@ -831,7 +816,6 @@ static PyObject * bu_longlong(const char *p, const formatdef *f) { -#ifdef HAVE_LONG_LONG PY_LONG_LONG x = 0; Py_ssize_t i = f->size; const unsigned char *bytes = (const unsigned char *)p; @@ -844,18 +828,11 @@ if (x >= LONG_MIN && x <= LONG_MAX) return PyLong_FromLong(Py_SAFE_DOWNCAST(x, PY_LONG_LONG, long)); return PyLong_FromLongLong(x); -#else - return _PyLong_FromByteArray((const unsigned char *)p, - 8, - 0, /* little-endian */ - 1 /* signed */); -#endif } static PyObject * bu_ulonglong(const char *p, const formatdef *f) { -#ifdef HAVE_LONG_LONG unsigned PY_LONG_LONG x = 0; Py_ssize_t i = f->size; const unsigned char *bytes = (const unsigned char *)p; @@ -865,12 +842,6 @@ if (x <= LONG_MAX) return PyLong_FromLong(Py_SAFE_DOWNCAST(x, unsigned PY_LONG_LONG, long)); return PyLong_FromUnsignedLongLong(x); -#else - return _PyLong_FromByteArray((const unsigned char *)p, - 8, - 0, /* little-endian */ - 0 /* signed */); -#endif } static PyObject * @@ -1072,7 +1043,6 @@ static PyObject * lu_longlong(const char *p, const formatdef *f) { -#ifdef HAVE_LONG_LONG PY_LONG_LONG x = 0; Py_ssize_t i = f->size; const unsigned char *bytes = (const unsigned char *)p; @@ -1085,18 +1055,11 @@ if (x >= LONG_MIN && x <= LONG_MAX) return PyLong_FromLong(Py_SAFE_DOWNCAST(x, PY_LONG_LONG, long)); return PyLong_FromLongLong(x); -#else - return _PyLong_FromByteArray((const unsigned char *)p, - 8, - 1, /* little-endian */ - 1 /* signed */); -#endif } static PyObject * lu_ulonglong(const char *p, const formatdef *f) { -#ifdef HAVE_LONG_LONG unsigned PY_LONG_LONG x = 0; Py_ssize_t i = f->size; const unsigned char *bytes = (const unsigned char *)p; @@ -1106,12 +1069,6 @@ if (x <= LONG_MAX) return PyLong_FromLong(Py_SAFE_DOWNCAST(x, unsigned PY_LONG_LONG, long)); return PyLong_FromUnsignedLongLong(x); -#else - return _PyLong_FromByteArray((const unsigned char *)p, - 8, - 1, /* little-endian */ - 0 /* signed */); -#endif } static PyObject * diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -60,9 +60,7 @@ CHECK_SIZEOF(SIZEOF_LONG, long); CHECK_SIZEOF(SIZEOF_VOID_P, void*); CHECK_SIZEOF(SIZEOF_TIME_T, time_t); -#ifdef HAVE_LONG_LONG CHECK_SIZEOF(SIZEOF_LONG_LONG, PY_LONG_LONG); -#endif #undef CHECK_SIZEOF @@ -357,7 +355,7 @@ } -/* Tests of PyLong_{As, From}{Unsigned,}Long(), and (#ifdef HAVE_LONG_LONG) +/* Tests of PyLong_{As, From}{Unsigned,}Long(), and PyLong_{As, From}{Unsigned,}LongLong(). Note that the meat of the test is contained in testcapi_long.h. @@ -402,8 +400,6 @@ #undef F_U_TO_PY #undef F_PY_TO_U -#ifdef HAVE_LONG_LONG - static PyObject * raise_test_longlong_error(const char* msg) { @@ -870,8 +866,6 @@ return Py_None; } -#endif /* ifdef HAVE_LONG_LONG */ - static PyObject * return_none(void *unused) { @@ -1136,7 +1130,6 @@ return PyLong_FromLong(value); } -#ifdef HAVE_LONG_LONG static PyObject * getargs_L(PyObject *self, PyObject *args) { @@ -1154,7 +1147,6 @@ return NULL; return PyLong_FromUnsignedLongLong(value); } -#endif /* This function not only tests the 'k' getargs code, but also the PyLong_AsUnsignedLongMask() and PyLong_AsUnsignedLongMask() functions. */ @@ -2279,10 +2271,8 @@ CHECK_1_FORMAT("%zu", size_t); /* "%lld" and "%llu" support added in Python 2.7. */ -#ifdef HAVE_LONG_LONG CHECK_1_FORMAT("%llu", unsigned PY_LONG_LONG); CHECK_1_FORMAT("%lld", PY_LONG_LONG); -#endif Py_RETURN_NONE; @@ -3991,14 +3981,12 @@ {"getargs_l", getargs_l, METH_VARARGS}, {"getargs_n", getargs_n, METH_VARARGS}, {"getargs_p", getargs_p, METH_VARARGS}, -#ifdef HAVE_LONG_LONG {"getargs_L", getargs_L, METH_VARARGS}, {"getargs_K", getargs_K, METH_VARARGS}, {"test_longlong_api", test_longlong_api, METH_NOARGS}, {"test_long_long_and_overflow", (PyCFunction)test_long_long_and_overflow, METH_NOARGS}, {"test_L_code", (PyCFunction)test_L_code, METH_NOARGS}, -#endif {"getargs_f", getargs_f, METH_VARARGS}, {"getargs_d", getargs_d, METH_VARARGS}, {"getargs_D", getargs_D, METH_VARARGS}, @@ -4153,10 +4141,8 @@ float float_member; double double_member; char inplace_member[6]; -#ifdef HAVE_LONG_LONG PY_LONG_LONG longlong_member; unsigned PY_LONG_LONG ulonglong_member; -#endif } all_structmembers; typedef struct { @@ -4178,10 +4164,8 @@ {"T_FLOAT", T_FLOAT, offsetof(test_structmembers, structmembers.float_member), 0, NULL}, {"T_DOUBLE", T_DOUBLE, offsetof(test_structmembers, structmembers.double_member), 0, NULL}, {"T_STRING_INPLACE", T_STRING_INPLACE, offsetof(test_structmembers, structmembers.inplace_member), 0, NULL}, -#ifdef HAVE_LONG_LONG {"T_LONGLONG", T_LONGLONG, offsetof(test_structmembers, structmembers.longlong_member), 0, NULL}, {"T_ULONGLONG", T_ULONGLONG, offsetof(test_structmembers, structmembers.ulonglong_member), 0, NULL}, -#endif {NULL} }; @@ -4193,15 +4177,9 @@ "T_BOOL", "T_BYTE", "T_UBYTE", "T_SHORT", "T_USHORT", "T_INT", "T_UINT", "T_LONG", "T_ULONG", "T_PYSSIZET", "T_FLOAT", "T_DOUBLE", "T_STRING_INPLACE", -#ifdef HAVE_LONG_LONG "T_LONGLONG", "T_ULONGLONG", -#endif NULL}; - static const char fmt[] = "|bbBhHiIlknfds#" -#ifdef HAVE_LONG_LONG - "LK" -#endif - ; + static const char fmt[] = "|bbBhHiIlknfds#LK"; test_structmembers *ob; const char *s = NULL; Py_ssize_t string_len = 0; @@ -4223,10 +4201,8 @@ &ob->structmembers.float_member, &ob->structmembers.double_member, &s, &string_len -#ifdef HAVE_LONG_LONG , &ob->structmembers.longlong_member, &ob->structmembers.ulonglong_member -#endif )) { Py_DECREF(ob); return NULL; diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -1182,10 +1182,8 @@ { Tcl_WideInt wideValue; if (Tcl_GetWideIntFromObj(Tkapp_Interp(tkapp), value, &wideValue) == TCL_OK) { -#ifdef HAVE_LONG_LONG if (sizeof(wideValue) <= SIZEOF_LONG_LONG) return PyLong_FromLongLong(wideValue); -#endif return _PyLong_FromByteArray((unsigned char *)(void *)&wideValue, sizeof(wideValue), PY_LITTLE_ENDIAN, diff --git a/Modules/addrinfo.h b/Modules/addrinfo.h --- a/Modules/addrinfo.h +++ b/Modules/addrinfo.h @@ -141,11 +141,7 @@ * RFC 2553: protocol-independent placeholder for socket addresses */ #define _SS_MAXSIZE 128 -#ifdef HAVE_LONG_LONG #define _SS_ALIGNSIZE (sizeof(PY_LONG_LONG)) -#else -#define _SS_ALIGNSIZE (sizeof(double)) -#endif /* HAVE_LONG_LONG */ #define _SS_PAD1SIZE (_SS_ALIGNSIZE - sizeof(u_char) * 2) #define _SS_PAD2SIZE (_SS_MAXSIZE - sizeof(u_char) * 2 - \ _SS_PAD1SIZE - _SS_ALIGNSIZE) @@ -158,11 +154,7 @@ unsigned short ss_family; /* address family */ #endif /* HAVE_SOCKADDR_SA_LEN */ char __ss_pad1[_SS_PAD1SIZE]; -#ifdef HAVE_LONG_LONG PY_LONG_LONG __ss_align; /* force desired structure storage alignment */ -#else - double __ss_align; /* force desired structure storage alignment */ -#endif /* HAVE_LONG_LONG */ char __ss_pad2[_SS_PAD2SIZE]; }; #endif /* !HAVE_SOCKADDR_STORAGE */ diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -418,8 +418,6 @@ return 0; } -#ifdef HAVE_LONG_LONG - static PyObject * q_getitem(arrayobject *ap, Py_ssize_t i) { @@ -469,7 +467,6 @@ ((unsigned PY_LONG_LONG *)ap->ob_item)[i] = x; return 0; } -#endif static PyObject * f_getitem(arrayobject *ap, Py_ssize_t i) @@ -521,10 +518,8 @@ {'I', sizeof(int), II_getitem, II_setitem, "I", 1, 0}, {'l', sizeof(long), l_getitem, l_setitem, "l", 1, 1}, {'L', sizeof(long), LL_getitem, LL_setitem, "L", 1, 0}, -#ifdef HAVE_LONG_LONG {'q', sizeof(PY_LONG_LONG), q_getitem, q_setitem, "q", 1, 1}, {'Q', sizeof(PY_LONG_LONG), QQ_getitem, QQ_setitem, "Q", 1, 0}, -#endif {'f', sizeof(float), f_getitem, f_setitem, "f", 0, 0}, {'d', sizeof(double), d_getitem, d_setitem, "d", 0, 0}, {'\0', 0, 0, 0, 0, 0, 0} /* Sentinel */ @@ -1814,7 +1809,6 @@ intsize = sizeof(long); is_signed = 0; break; -#if HAVE_LONG_LONG case 'q': intsize = sizeof(PY_LONG_LONG); is_signed = 1; @@ -1823,7 +1817,6 @@ intsize = sizeof(PY_LONG_LONG); is_signed = 0; break; -#endif default: return UNKNOWN_FORMAT; } @@ -2685,11 +2678,7 @@ } } PyErr_SetString(PyExc_ValueError, -#ifdef HAVE_LONG_LONG "bad typecode (must be b, B, u, h, H, i, I, l, L, q, Q, f or d)"); -#else - "bad typecode (must be b, B, u, h, H, i, I, l, L, f or d)"); -#endif return NULL; } diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -640,22 +640,14 @@ #endif /* MS_WINDOWS */ -#ifdef HAVE_LONG_LONG -# define _PyLong_FromDev PyLong_FromLongLong -#else -# define _PyLong_FromDev PyLong_FromLong -#endif +#define _PyLong_FromDev PyLong_FromLongLong #if defined(HAVE_MKNOD) && defined(HAVE_MAKEDEV) static int _Py_Dev_Converter(PyObject *obj, void *p) { -#ifdef HAVE_LONG_LONG *((dev_t *)p) = PyLong_AsUnsignedLongLong(obj); -#else - *((dev_t *)p) = PyLong_AsUnsignedLong(obj); -#endif if (PyErr_Occurred()) return 0; return 1; diff --git a/Modules/resource.c b/Modules/resource.c --- a/Modules/resource.c +++ b/Modules/resource.c @@ -135,13 +135,11 @@ static PyObject* rlimit2py(struct rlimit rl) { -#if defined(HAVE_LONG_LONG) if (sizeof(rl.rlim_cur) > sizeof(long)) { return Py_BuildValue("LL", (PY_LONG_LONG) rl.rlim_cur, (PY_LONG_LONG) rl.rlim_max); } -#endif return Py_BuildValue("ll", (long) rl.rlim_cur, (long) rl.rlim_max); } @@ -438,11 +436,9 @@ PyModule_AddIntMacro(m, RLIMIT_NPTS); #endif -#if defined(HAVE_LONG_LONG) if (sizeof(RLIM_INFINITY) > sizeof(long)) { v = PyLong_FromLongLong((PY_LONG_LONG) RLIM_INFINITY); } else -#endif { v = PyLong_FromLong((long) RLIM_INFINITY); } diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -992,9 +992,6 @@ return PyLong_FromUnsignedLong((unsigned long)(Py_uintptr_t)p); #else -#ifndef HAVE_LONG_LONG -# error "PyLong_FromVoidPtr: sizeof(void*) > sizeof(long), but no long long" -#endif #if SIZEOF_LONG_LONG < SIZEOF_VOID_P # error "PyLong_FromVoidPtr: sizeof(PY_LONG_LONG) < sizeof(void*)" #endif @@ -1017,9 +1014,6 @@ x = PyLong_AsUnsignedLong(vv); #else -#ifndef HAVE_LONG_LONG -# error "PyLong_AsVoidPtr: sizeof(void*) > sizeof(long), but no long long" -#endif #if SIZEOF_LONG_LONG < SIZEOF_VOID_P # error "PyLong_AsVoidPtr: sizeof(PY_LONG_LONG) < sizeof(void*)" #endif @@ -1037,8 +1031,6 @@ return (void *)x; } -#ifdef HAVE_LONG_LONG - /* Initial PY_LONG_LONG support by Chris Herborth (chrish at qnx.com), later * rewritten to use the newer PyLong_{As,From}ByteArray API. */ @@ -1417,8 +1409,6 @@ return res; } -#endif /* HAVE_LONG_LONG */ - #define CHECK_BINOP(v,w) \ do { \ if (!PyLong_Check(v) || !PyLong_Check(w)) \ @@ -3491,17 +3481,7 @@ /* fast path for single-digit multiplication */ if (Py_ABS(Py_SIZE(a)) <= 1 && Py_ABS(Py_SIZE(b)) <= 1) { stwodigits v = (stwodigits)(MEDIUM_VALUE(a)) * MEDIUM_VALUE(b); -#ifdef HAVE_LONG_LONG return PyLong_FromLongLong((PY_LONG_LONG)v); -#else - /* if we don't have long long then we're almost certainly - using 15-bit digits, so v will fit in a long. In the - unlikely event that we're using 30-bit digits on a platform - without long long, a large v will just cause us to fall - through to the general multiplication code below. */ - if (v >= LONG_MIN && v <= LONG_MAX) - return PyLong_FromLong((long)v); -#endif } z = k_mul(a, b); diff --git a/Objects/memoryobject.c b/Objects/memoryobject.c --- a/Objects/memoryobject.c +++ b/Objects/memoryobject.c @@ -1111,9 +1111,7 @@ case 'h': case 'H': size = sizeof(short); break; case 'i': case 'I': size = sizeof(int); break; case 'l': case 'L': size = sizeof(long); break; - #ifdef HAVE_LONG_LONG case 'q': case 'Q': size = sizeof(PY_LONG_LONG); break; - #endif case 'n': case 'N': size = sizeof(Py_ssize_t); break; case 'f': size = sizeof(float); break; case 'd': size = sizeof(double); break; @@ -1158,10 +1156,8 @@ case 'I': RETURN("I"); case 'l': RETURN("l"); case 'L': RETURN("L"); - #ifdef HAVE_LONG_LONG case 'q': RETURN("q"); case 'Q': RETURN("Q"); - #endif case 'n': RETURN("n"); case 'N': RETURN("N"); case 'f': RETURN("f"); @@ -1581,7 +1577,6 @@ return lu; } -#ifdef HAVE_LONG_LONG static PY_LONG_LONG pylong_as_lld(PyObject *item) { @@ -1611,7 +1606,6 @@ Py_DECREF(tmp); return llu; } -#endif static Py_ssize_t pylong_as_zd(PyObject *item) @@ -1691,10 +1685,8 @@ case 'L': UNPACK_SINGLE(lu, ptr, unsigned long); goto convert_lu; /* native 64-bit */ - #ifdef HAVE_LONG_LONG case 'q': UNPACK_SINGLE(lld, ptr, PY_LONG_LONG); goto convert_lld; case 'Q': UNPACK_SINGLE(llu, ptr, unsigned PY_LONG_LONG); goto convert_llu; - #endif /* ssize_t and size_t */ case 'n': UNPACK_SINGLE(zd, ptr, Py_ssize_t); goto convert_zd; @@ -1806,7 +1798,6 @@ break; /* native 64-bit */ - #ifdef HAVE_LONG_LONG case 'q': lld = pylong_as_lld(item); if (lld == -1 && PyErr_Occurred()) @@ -1819,7 +1810,6 @@ goto err_occurred; PACK_SINGLE(ptr, llu, unsigned PY_LONG_LONG); break; - #endif /* ssize_t and size_t */ case 'n': @@ -2656,10 +2646,8 @@ case 'L': CMP_SINGLE(p, q, unsigned long); return equal; /* native 64-bit */ - #ifdef HAVE_LONG_LONG case 'q': CMP_SINGLE(p, q, PY_LONG_LONG); return equal; case 'Q': CMP_SINGLE(p, q, unsigned PY_LONG_LONG); return equal; - #endif /* ssize_t and size_t */ case 'n': CMP_SINGLE(p, q, Py_ssize_t); return equal; diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -2636,13 +2636,11 @@ longflag = 1; ++f; } -#ifdef HAVE_LONG_LONG else if (f[1] == 'l' && (f[2] == 'd' || f[2] == 'u' || f[2] == 'i')) { longlongflag = 1; f += 2; } -#endif } /* handle the size_t flag. */ else if (*f == 'z' && (f[1] == 'd' || f[1] == 'u' || f[1] == 'i')) { @@ -2680,11 +2678,9 @@ if (longflag) len = sprintf(buffer, "%lu", va_arg(*vargs, unsigned long)); -#ifdef HAVE_LONG_LONG else if (longlongflag) len = sprintf(buffer, "%" PY_FORMAT_LONG_LONG "u", va_arg(*vargs, unsigned PY_LONG_LONG)); -#endif else if (size_tflag) len = sprintf(buffer, "%" PY_FORMAT_SIZE_T "u", va_arg(*vargs, size_t)); @@ -2699,11 +2695,9 @@ if (longflag) len = sprintf(buffer, "%li", va_arg(*vargs, long)); -#ifdef HAVE_LONG_LONG else if (longlongflag) len = sprintf(buffer, "%" PY_FORMAT_LONG_LONG "i", va_arg(*vargs, PY_LONG_LONG)); -#endif else if (size_tflag) len = sprintf(buffer, "%" PY_FORMAT_SIZE_T "i", va_arg(*vargs, Py_ssize_t)); diff --git a/Python/getargs.c b/Python/getargs.c --- a/Python/getargs.c +++ b/Python/getargs.c @@ -769,7 +769,6 @@ break; } -#ifdef HAVE_LONG_LONG case 'L': {/* PY_LONG_LONG */ PY_LONG_LONG *p = va_arg( *p_va, PY_LONG_LONG * ); PY_LONG_LONG ival; @@ -793,7 +792,6 @@ *p = ival; break; } -#endif case 'f': {/* float */ float *p = va_arg(*p_va, float *); @@ -2088,10 +2086,8 @@ case 'I': /* int sized bitfield */ case 'l': /* long int */ case 'k': /* long int sized bitfield */ -#ifdef HAVE_LONG_LONG case 'L': /* PY_LONG_LONG */ case 'K': /* PY_LONG_LONG sized bitfield */ -#endif case 'n': /* Py_ssize_t */ case 'f': /* float */ case 'd': /* double */ diff --git a/Python/modsupport.c b/Python/modsupport.c --- a/Python/modsupport.c +++ b/Python/modsupport.c @@ -260,13 +260,12 @@ return PyLong_FromUnsignedLong(n); } -#ifdef HAVE_LONG_LONG case 'L': return PyLong_FromLongLong((PY_LONG_LONG)va_arg(*p_va, PY_LONG_LONG)); case 'K': return PyLong_FromUnsignedLongLong((PY_LONG_LONG)va_arg(*p_va, unsigned PY_LONG_LONG)); -#endif + case 'u': { PyObject *v; diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -38,7 +38,7 @@ time_t _PyLong_AsTime_t(PyObject *obj) { -#if defined(HAVE_LONG_LONG) && SIZEOF_TIME_T == SIZEOF_LONG_LONG +#if SIZEOF_TIME_T == SIZEOF_LONG_LONG PY_LONG_LONG val; val = PyLong_AsLongLong(obj); #else @@ -57,7 +57,7 @@ PyObject * _PyLong_FromTime_t(time_t t) { -#if defined(HAVE_LONG_LONG) && SIZEOF_TIME_T == SIZEOF_LONG_LONG +#if SIZEOF_TIME_T == SIZEOF_LONG_LONG return PyLong_FromLongLong((PY_LONG_LONG)t); #else Py_BUILD_ASSERT(sizeof(time_t) <= sizeof(long)); @@ -304,17 +304,10 @@ return _PyTime_FromFloatObject(t, d, round, unit_to_ns); } else { -#ifdef HAVE_LONG_LONG PY_LONG_LONG sec; Py_BUILD_ASSERT(sizeof(PY_LONG_LONG) <= sizeof(_PyTime_t)); sec = PyLong_AsLongLong(obj); -#else - long sec; - Py_BUILD_ASSERT(sizeof(PY_LONG_LONG) <= sizeof(_PyTime_t)); - - sec = PyLong_AsLong(obj); -#endif if (sec == -1 && PyErr_Occurred()) { if (PyErr_ExceptionMatches(PyExc_OverflowError)) _PyTime_overflow(); @@ -365,13 +358,8 @@ PyObject * _PyTime_AsNanosecondsObject(_PyTime_t t) { -#ifdef HAVE_LONG_LONG Py_BUILD_ASSERT(sizeof(PY_LONG_LONG) >= sizeof(_PyTime_t)); return PyLong_FromLongLong((PY_LONG_LONG)t); -#else - Py_BUILD_ASSERT(sizeof(long) >= sizeof(_PyTime_t)); - return PyLong_FromLong((long)t); -#endif } static _PyTime_t diff --git a/Python/structmember.c b/Python/structmember.c --- a/Python/structmember.c +++ b/Python/structmember.c @@ -74,14 +74,12 @@ PyErr_SetString(PyExc_AttributeError, l->name); Py_XINCREF(v); break; -#ifdef HAVE_LONG_LONG case T_LONGLONG: v = PyLong_FromLongLong(*(PY_LONG_LONG *)addr); break; case T_ULONGLONG: v = PyLong_FromUnsignedLongLong(*(unsigned PY_LONG_LONG *)addr); break; -#endif /* HAVE_LONG_LONG */ case T_NONE: v = Py_None; Py_INCREF(v); @@ -266,7 +264,6 @@ case T_STRING_INPLACE: PyErr_SetString(PyExc_TypeError, "readonly attribute"); return -1; -#ifdef HAVE_LONG_LONG case T_LONGLONG:{ PY_LONG_LONG value; *(PY_LONG_LONG*)addr = value = PyLong_AsLongLong(v); @@ -286,7 +283,6 @@ return -1; break; } -#endif /* HAVE_LONG_LONG */ default: PyErr_Format(PyExc_SystemError, "bad memberdescr type for %s", l->name); diff --git a/configure b/configure --- a/configure +++ b/configure @@ -775,6 +775,7 @@ docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -885,6 +886,7 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1137,6 +1139,15 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1274,7 +1285,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1427,6 +1438,7 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -8294,6 +8306,39 @@ # version HP92453-01 B.11.11.23709.GP, which incorrectly rejects # declarations like `int a3[[(sizeof (unsigned char)) >= 0]];'. # This bug is HP SR number 8606223364. +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking size of long long" >&5 +$as_echo_n "checking size of long long... " >&6; } +if ${ac_cv_sizeof_long_long+:} false; then : + $as_echo_n "(cached) " >&6 +else + if ac_fn_c_compute_int "$LINENO" "(long int) (sizeof (long long))" "ac_cv_sizeof_long_long" "$ac_includes_default"; then : + +else + if test "$ac_cv_type_long_long" = yes; then + { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 +$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} +as_fn_error 77 "cannot compute sizeof (long long) +See \`config.log' for more details" "$LINENO" 5; } + else + ac_cv_sizeof_long_long=0 + fi +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sizeof_long_long" >&5 +$as_echo "$ac_cv_sizeof_long_long" >&6; } + + + +cat >>confdefs.h <<_ACEOF +#define SIZEOF_LONG_LONG $ac_cv_sizeof_long_long +_ACEOF + + +# The cast to long int works around a bug in the HP C Compiler +# version HP92453-01 B.11.11.23709.GP, which incorrectly rejects +# declarations like `int a3[[(sizeof (unsigned char)) >= 0]];'. +# This bug is HP SR number 8606223364. { $as_echo "$as_me:${as_lineno-$LINENO}: checking size of void *" >&5 $as_echo_n "checking size of void *... " >&6; } if ${ac_cv_sizeof_void_p+:} false; then : @@ -8522,67 +8567,6 @@ -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for long long support" >&5 -$as_echo_n "checking for long long support... " >&6; } -have_long_long=no -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ -long long x; x = (long long)0; - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - - -$as_echo "#define HAVE_LONG_LONG 1" >>confdefs.h - - have_long_long=yes - -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $have_long_long" >&5 -$as_echo "$have_long_long" >&6; } -if test "$have_long_long" = yes ; then -# The cast to long int works around a bug in the HP C Compiler -# version HP92453-01 B.11.11.23709.GP, which incorrectly rejects -# declarations like `int a3[[(sizeof (unsigned char)) >= 0]];'. -# This bug is HP SR number 8606223364. -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking size of long long" >&5 -$as_echo_n "checking size of long long... " >&6; } -if ${ac_cv_sizeof_long_long+:} false; then : - $as_echo_n "(cached) " >&6 -else - if ac_fn_c_compute_int "$LINENO" "(long int) (sizeof (long long))" "ac_cv_sizeof_long_long" "$ac_includes_default"; then : - -else - if test "$ac_cv_type_long_long" = yes; then - { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error 77 "cannot compute sizeof (long long) -See \`config.log' for more details" "$LINENO" 5; } - else - ac_cv_sizeof_long_long=0 - fi -fi - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sizeof_long_long" >&5 -$as_echo "$ac_cv_sizeof_long_long" >&6; } - - - -cat >>confdefs.h <<_ACEOF -#define SIZEOF_LONG_LONG $ac_cv_sizeof_long_long -_ACEOF - - -fi - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for long double support" >&5 $as_echo_n "checking for long double support... " >&6; } have_long_double=no @@ -8796,8 +8780,6 @@ { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to enable large file support" >&5 $as_echo_n "checking whether to enable large file support... " >&6; } -if test "$have_long_long" = yes -then if test "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \ "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then @@ -8809,10 +8791,6 @@ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi # The cast to long int works around a bug in the HP C Compiler # version HP92453-01 B.11.11.23709.GP, which incorrectly rejects @@ -15853,32 +15831,30 @@ fi -if test "$have_long_long" = yes -then - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for %lld and %llu printf() format support" >&5 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for %lld and %llu printf() format support" >&5 $as_echo_n "checking for %lld and %llu printf() format support... " >&6; } - if ${ac_cv_have_long_long_format+:} false; then : +if ${ac_cv_have_long_long_format+:} false; then : $as_echo_n "(cached) " >&6 else if test "$cross_compiling" = yes; then : ac_cv_have_long_long_format="cross -- assuming no" - if test x$GCC = xyes; then - save_CFLAGS=$CFLAGS - CFLAGS="$CFLAGS -Werror -Wformat" - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - - #include - #include - -int -main () -{ - - char *buffer; - sprintf(buffer, "%lld", (long long)123); - sprintf(buffer, "%lld", (long long)-123); - sprintf(buffer, "%llu", (unsigned long long)123); +if test x$GCC = xyes; then +save_CFLAGS=$CFLAGS +CFLAGS="$CFLAGS -Werror -Wformat" +cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +#include +#include + +int +main () +{ + +char *buffer; +sprintf(buffer, "%lld", (long long)123); +sprintf(buffer, "%lld", (long long)-123); +sprintf(buffer, "%llu", (unsigned long long)123); ; return 0; @@ -15889,41 +15865,41 @@ fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - CFLAGS=$save_CFLAGS - fi -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - - #include - #include - #include - - #ifdef HAVE_SYS_TYPES_H - #include - #endif - - int main() - { - char buffer[256]; - - if (sprintf(buffer, "%lld", (long long)123) < 0) - return 1; - if (strcmp(buffer, "123")) - return 1; - - if (sprintf(buffer, "%lld", (long long)-123) < 0) - return 1; - if (strcmp(buffer, "-123")) - return 1; - - if (sprintf(buffer, "%llu", (unsigned long long)123) < 0) - return 1; - if (strcmp(buffer, "123")) - return 1; - - return 0; - } +CFLAGS=$save_CFLAGS +fi +else + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +#include +#include +#include + +#ifdef HAVE_SYS_TYPES_H +#include +#endif + +int main() +{ +char buffer[256]; + +if (sprintf(buffer, "%lld", (long long)123) < 0) +return 1; +if (strcmp(buffer, "123")) +return 1; + +if (sprintf(buffer, "%lld", (long long)-123) < 0) +return 1; +if (strcmp(buffer, "-123")) +return 1; + +if (sprintf(buffer, "%llu", (unsigned long long)123) < 0) +return 1; +if (strcmp(buffer, "123")) +return 1; + +return 0; +} _ACEOF if ac_fn_c_try_run "$LINENO"; then : @@ -15938,9 +15914,8 @@ fi - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_have_long_long_format" >&5 +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_have_long_long_format" >&5 $as_echo "$ac_cv_have_long_long_format" >&6; } -fi if test "$ac_cv_have_long_long_format" = yes then diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -2092,6 +2092,7 @@ # ANSI C requires sizeof(char) == 1, so no need to check it AC_CHECK_SIZEOF(int, 4) AC_CHECK_SIZEOF(long, 4) +AC_CHECK_SIZEOF(long long, 8) AC_CHECK_SIZEOF(void *, 4) AC_CHECK_SIZEOF(short, 2) AC_CHECK_SIZEOF(float, 4) @@ -2100,17 +2101,6 @@ AC_CHECK_SIZEOF(size_t, 4) AC_CHECK_SIZEOF(pid_t, 4) -AC_MSG_CHECKING(for long long support) -have_long_long=no -AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[long long x; x = (long long)0;]])],[ - AC_DEFINE(HAVE_LONG_LONG, 1, [Define this if you have the type long long.]) - have_long_long=yes -],[]) -AC_MSG_RESULT($have_long_long) -if test "$have_long_long" = yes ; then -AC_CHECK_SIZEOF(long long, 8) -fi - AC_MSG_CHECKING(for long double support) have_long_double=no AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[long double x; x = (long double)0;]])],[ @@ -2150,8 +2140,6 @@ ]) AC_MSG_CHECKING(whether to enable large file support) -if test "$have_long_long" = yes -then if test "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \ "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then AC_DEFINE(HAVE_LARGEFILE_SUPPORT, 1, @@ -2163,9 +2151,6 @@ else AC_MSG_RESULT(no) fi -else - AC_MSG_RESULT(no) -fi AC_CHECK_SIZEOF(time_t, [], [ #ifdef HAVE_SYS_TYPES_H @@ -4925,63 +4910,60 @@ [Define to 1 if you have the /dev/ptc device file.]) fi -if test "$have_long_long" = yes -then - AC_MSG_CHECKING(for %lld and %llu printf() format support) - AC_CACHE_VAL(ac_cv_have_long_long_format, - AC_RUN_IFELSE([AC_LANG_SOURCE([[[ - #include - #include - #include - - #ifdef HAVE_SYS_TYPES_H - #include - #endif - - int main() - { - char buffer[256]; - - if (sprintf(buffer, "%lld", (long long)123) < 0) - return 1; - if (strcmp(buffer, "123")) - return 1; - - if (sprintf(buffer, "%lld", (long long)-123) < 0) - return 1; - if (strcmp(buffer, "-123")) - return 1; - - if (sprintf(buffer, "%llu", (unsigned long long)123) < 0) - return 1; - if (strcmp(buffer, "123")) - return 1; - - return 0; - } - ]]])], - [ac_cv_have_long_long_format=yes], - [ac_cv_have_long_long_format=no], - [ac_cv_have_long_long_format="cross -- assuming no" - if test x$GCC = xyes; then - save_CFLAGS=$CFLAGS - CFLAGS="$CFLAGS -Werror -Wformat" - AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ - #include - #include - ]], [[ - char *buffer; - sprintf(buffer, "%lld", (long long)123); - sprintf(buffer, "%lld", (long long)-123); - sprintf(buffer, "%llu", (unsigned long long)123); - ]])], - ac_cv_have_long_long_format=yes - ) - CFLAGS=$save_CFLAGS - fi]) - ) - AC_MSG_RESULT($ac_cv_have_long_long_format) -fi +AC_MSG_CHECKING(for %lld and %llu printf() format support) +AC_CACHE_VAL(ac_cv_have_long_long_format, +AC_RUN_IFELSE([AC_LANG_SOURCE([[[ +#include +#include +#include + +#ifdef HAVE_SYS_TYPES_H +#include +#endif + +int main() +{ +char buffer[256]; + +if (sprintf(buffer, "%lld", (long long)123) < 0) +return 1; +if (strcmp(buffer, "123")) +return 1; + +if (sprintf(buffer, "%lld", (long long)-123) < 0) +return 1; +if (strcmp(buffer, "-123")) +return 1; + +if (sprintf(buffer, "%llu", (unsigned long long)123) < 0) +return 1; +if (strcmp(buffer, "123")) +return 1; + +return 0; +} +]]])], +[ac_cv_have_long_long_format=yes], +[ac_cv_have_long_long_format=no], +[ac_cv_have_long_long_format="cross -- assuming no" +if test x$GCC = xyes; then +save_CFLAGS=$CFLAGS +CFLAGS="$CFLAGS -Werror -Wformat" +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include +#include +]], [[ +char *buffer; +sprintf(buffer, "%lld", (long long)123); +sprintf(buffer, "%lld", (long long)-123); +sprintf(buffer, "%llu", (unsigned long long)123); +]])], +ac_cv_have_long_long_format=yes +) +CFLAGS=$save_CFLAGS +fi]) +) +AC_MSG_RESULT($ac_cv_have_long_long_format) if test "$ac_cv_have_long_long_format" = yes then diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -592,9 +592,6 @@ /* Define this if you have the type long double. */ #undef HAVE_LONG_DOUBLE -/* Define this if you have the type long long. */ -#undef HAVE_LONG_LONG - /* Define to 1 if you have the `lstat' function. */ #undef HAVE_LSTAT -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 21:30:56 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 01:30:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_explicitly_cas?= =?utf-8?q?t_away_constness_to_silence_compiler_warning?= Message-ID: <20160906013056.22588.39344.6CDCC586@psf.io> https://hg.python.org/cpython/rev/fb7c89a66d23 changeset: 103106:fb7c89a66d23 branch: 3.5 parent: 103100:f858d4ee1b2d user: Benjamin Peterson date: Mon Sep 05 18:26:19 2016 -0700 summary: explicitly cast away constness to silence compiler warning files: Modules/readline.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/readline.c b/Modules/readline.c --- a/Modules/readline.c +++ b/Modules/readline.c @@ -1063,7 +1063,7 @@ Py_XDECREF(readlinestate_global->endidx); readlinestate_global->begidx = PyLong_FromLong((long) start); readlinestate_global->endidx = PyLong_FromLong((long) end); - result = completion_matches(text, *on_completion); + result = completion_matches((char *)text, *on_completion); #ifdef WITH_THREAD PyGILState_Release(gilstate); #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 21:30:56 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 01:30:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160906013056.68842.13822.02BBB0FE@psf.io> https://hg.python.org/cpython/rev/ebaeeb36eb4b changeset: 103107:ebaeeb36eb4b parent: 103105:9206a86f7321 parent: 103106:fb7c89a66d23 user: Benjamin Peterson date: Mon Sep 05 18:26:26 2016 -0700 summary: merge 3.5 files: Modules/readline.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/readline.c b/Modules/readline.c --- a/Modules/readline.c +++ b/Modules/readline.c @@ -1080,7 +1080,7 @@ Py_XDECREF(readlinestate_global->endidx); readlinestate_global->begidx = PyLong_FromLong((long) start); readlinestate_global->endidx = PyLong_FromLong((long) end); - result = completion_matches(text, *on_completion); + result = completion_matches((char *)text, *on_completion); #ifdef WITH_THREAD PyGILState_Release(gilstate); #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 22:20:09 2016 From: python-checkins at python.org (martin.panter) Date: Tue, 06 Sep 2016 02:20:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327355=3A_Import_n?= =?utf-8?q?o_longer_needed?= Message-ID: <20160906022009.30615.5800.DBEDE049@psf.io> https://hg.python.org/cpython/rev/0ce745bb7717 changeset: 103108:0ce745bb7717 user: Martin Panter date: Tue Sep 06 02:18:16 2016 +0000 summary: Issue #27355: Import no longer needed files: Lib/ctypes/test/test_funcptr.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ctypes/test/test_funcptr.py b/Lib/ctypes/test/test_funcptr.py --- a/Lib/ctypes/test/test_funcptr.py +++ b/Lib/ctypes/test/test_funcptr.py @@ -1,4 +1,4 @@ -import os, unittest +import unittest from ctypes import * try: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 22:29:55 2016 From: python-checkins at python.org (jason.coombs) Date: Tue, 06 Sep 2016 02:29:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Issue_=2327960=3A_Merge_with_3=2E4?= Message-ID: <20160906022954.47505.51159.CB95FCCA@psf.io> https://hg.python.org/cpython/rev/b442744d2d22 changeset: 103110:b442744d2d22 branch: 3.5 parent: 103106:fb7c89a66d23 parent: 103109:556a11c11edd user: Jason R. Coombs date: Mon Sep 05 22:26:19 2016 -0400 summary: Issue #27960: Merge with 3.4 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 22:29:54 2016 From: python-checkins at python.org (jason.coombs) Date: Tue, 06 Sep 2016 02:29:54 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy40KTogSXNzdWUgIzI3OTYw?= =?utf-8?q?=3A_Revert_state_to_675e20c38fdac6=2C_backing_out_all_changes_b?= =?utf-8?q?y?= Message-ID: <20160906022954.114694.71892.E0471835@psf.io> https://hg.python.org/cpython/rev/556a11c11edd changeset: 103109:556a11c11edd branch: 3.4 parent: 103005:e82b995d1a5c user: Jason R. Coombs date: Mon Sep 05 22:24:01 2016 -0400 summary: Issue #27960: Revert state to 675e20c38fdac6, backing out all changes by developed for Issue #12885. files: Lib/distutils/tests/test_filelist.py | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -7,7 +7,7 @@ from distutils.errors import DistutilsTemplateError from distutils.filelist import glob_to_re, translate_pattern, FileList -from test.support import captured_stdout +from test.support import captured_stdout, run_unittest from distutils.tests import support MANIFEST_IN = """\ @@ -292,5 +292,8 @@ self.assertWarnings() +def test_suite(): + return unittest.makeSuite(FileListTestCase) + if __name__ == "__main__": - unittest.main() + run_unittest(test_suite()) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 5 22:29:55 2016 From: python-checkins at python.org (jason.coombs) Date: Tue, 06 Sep 2016 02:29:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327960=3A_Merge_with_3=2E5?= Message-ID: <20160906022955.23419.90610.752E3C06@psf.io> https://hg.python.org/cpython/rev/7cec1a1d6950 changeset: 103111:7cec1a1d6950 parent: 103108:0ce745bb7717 parent: 103110:b442744d2d22 user: Jason R. Coombs date: Mon Sep 05 22:29:29 2016 -0400 summary: Issue #27960: Merge with 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 00:46:33 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 06 Sep 2016 04:46:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Avoid_calling_functions_wi?= =?utf-8?q?th_an_empty_string_as_format_string?= Message-ID: <20160906044633.6805.80731.F0251E2D@psf.io> https://hg.python.org/cpython/rev/8ee4ed577c03 changeset: 103113:8ee4ed577c03 user: Victor Stinner date: Mon Sep 05 18:16:01 2016 -0700 summary: Avoid calling functions with an empty string as format string Directly pass NULL rather than an empty string. files: Modules/_cursesmodule.c | 2 +- Modules/_elementtree.c | 2 +- Modules/_sqlite/connection.c | 16 ++++++++-------- Modules/_sqlite/cursor.c | 2 +- Modules/_sqlite/module.c | 2 +- Modules/_testcapimodule.c | 4 ++-- Modules/faulthandler.c | 6 +++--- Objects/genobject.c | 2 +- Objects/weakrefobject.c | 2 +- Python/bltinmodule.c | 12 ++++++------ Python/pylifecycle.c | 10 +++++----- Python/pythonrun.c | 4 ++-- Python/traceback.c | 4 ++-- 13 files changed, 34 insertions(+), 34 deletions(-) diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -2306,7 +2306,7 @@ goto error; } - data = _PyObject_CallMethodId(stream, &PyId_read, ""); + data = _PyObject_CallMethodId(stream, &PyId_read, NULL); if (data == NULL) goto error; if (!PyBytes_Check(data)) { diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3399,7 +3399,7 @@ } else if (self->handle_close) { Py_DECREF(res); - return PyObject_CallFunction(self->handle_close, ""); + return _PyObject_CallNoArg(self->handle_close); } else { return res; diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -672,7 +672,7 @@ aggregate_instance = (PyObject**)sqlite3_aggregate_context(context, sizeof(PyObject*)); if (*aggregate_instance == 0) { - *aggregate_instance = PyObject_CallFunction(aggregate_class, ""); + *aggregate_instance = PyObject_CallFunction(aggregate_class, NULL); if (PyErr_Occurred()) { *aggregate_instance = 0; @@ -744,7 +744,7 @@ PyErr_Fetch(&exception, &value, &tb); restore = 1; - function_result = _PyObject_CallMethodId(*aggregate_instance, &PyId_finalize, ""); + function_result = _PyObject_CallMethodId(*aggregate_instance, &PyId_finalize, NULL); Py_DECREF(*aggregate_instance); @@ -960,7 +960,7 @@ gilstate = PyGILState_Ensure(); #endif - ret = PyObject_CallFunction((PyObject*)user_arg, ""); + ret = PyObject_CallFunction((PyObject*)user_arg, NULL); if (!ret) { if (_enable_callback_tracebacks) { @@ -1291,7 +1291,7 @@ PyObject* result = 0; PyObject* method = 0; - cursor = _PyObject_CallMethodId((PyObject*)self, &PyId_cursor, ""); + cursor = _PyObject_CallMethodId((PyObject*)self, &PyId_cursor, NULL); if (!cursor) { goto error; } @@ -1320,7 +1320,7 @@ PyObject* result = 0; PyObject* method = 0; - cursor = _PyObject_CallMethodId((PyObject*)self, &PyId_cursor, ""); + cursor = _PyObject_CallMethodId((PyObject*)self, &PyId_cursor, NULL); if (!cursor) { goto error; } @@ -1349,7 +1349,7 @@ PyObject* result = 0; PyObject* method = 0; - cursor = _PyObject_CallMethodId((PyObject*)self, &PyId_cursor, ""); + cursor = _PyObject_CallMethodId((PyObject*)self, &PyId_cursor, NULL); if (!cursor) { goto error; } @@ -1519,7 +1519,7 @@ goto finally; } - uppercase_name = _PyObject_CallMethodId(name, &PyId_upper, ""); + uppercase_name = _PyObject_CallMethodId(name, &PyId_upper, NULL); if (!uppercase_name) { goto finally; } @@ -1611,7 +1611,7 @@ method_name = "rollback"; } - result = PyObject_CallMethod((PyObject*)self, method_name, ""); + result = PyObject_CallMethod((PyObject*)self, method_name, NULL); if (!result) { return NULL; } diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -143,7 +143,7 @@ PyObject* retval; _Py_IDENTIFIER(upper); - upcase_key = _PyObject_CallMethodId(key, &PyId_upper, ""); + upcase_key = _PyObject_CallMethodId(key, &PyId_upper, NULL); if (!upcase_key) { return NULL; } diff --git a/Modules/_sqlite/module.c b/Modules/_sqlite/module.c --- a/Modules/_sqlite/module.c +++ b/Modules/_sqlite/module.c @@ -192,7 +192,7 @@ } /* convert the name to upper case */ - name = _PyObject_CallMethodId(orig_name, &PyId_upper, ""); + name = _PyObject_CallMethodId(orig_name, &PyId_upper, NULL); if (!name) { goto error; } diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -2137,7 +2137,7 @@ PyObject *rc; int success; PyGILState_STATE s = PyGILState_Ensure(); - rc = PyObject_CallFunction((PyObject *)callable, ""); + rc = _PyObject_CallNoArg((PyObject *)callable); success = (rc != NULL); Py_XDECREF(rc); PyGILState_Release(s); @@ -3406,7 +3406,7 @@ /* Allocate a Python thread state for this thread */ state = PyGILState_Ensure(); - res = PyObject_CallFunction(test_c_thread->callback, "", NULL); + res = _PyObject_CallNoArg(test_c_thread->callback); Py_CLEAR(test_c_thread->callback); if (res == NULL) { diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -168,7 +168,7 @@ return fd; } - result = _PyObject_CallMethodId(file, &PyId_fileno, ""); + result = _PyObject_CallMethodId(file, &PyId_fileno, NULL); if (result == NULL) return -1; @@ -186,7 +186,7 @@ return -1; } - result = _PyObject_CallMethodId(file, &PyId_flush, ""); + result = _PyObject_CallMethodId(file, &PyId_flush, NULL); if (result != NULL) Py_DECREF(result); else { @@ -1290,7 +1290,7 @@ if (module == NULL) { return -1; } - res = _PyObject_CallMethodId(module, &PyId_enable, ""); + res = _PyObject_CallMethodId(module, &PyId_enable, NULL); Py_DECREF(module); if (res == NULL) return -1; diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -261,7 +261,7 @@ PyErr_WriteUnraisable(yf); PyErr_Clear(); } else { - retval = PyObject_CallFunction(meth, ""); + retval = _PyObject_CallNoArg(meth); Py_DECREF(meth); if (retval == NULL) return -1; diff --git a/Objects/weakrefobject.c b/Objects/weakrefobject.c --- a/Objects/weakrefobject.c +++ b/Objects/weakrefobject.c @@ -453,7 +453,7 @@ method(PyObject *proxy) { \ _Py_IDENTIFIER(special); \ UNWRAP(proxy); \ - return _PyObject_CallMethodId(proxy, &PyId_##special, ""); \ + return _PyObject_CallMethodId(proxy, &PyId_##special, NULL); \ } diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -1784,7 +1784,7 @@ if (do_flush == -1) return NULL; else if (do_flush) { - tmp = _PyObject_CallMethodId(file, &PyId_flush, ""); + tmp = _PyObject_CallMethodId(file, &PyId_flush, NULL); if (tmp == NULL) return NULL; else @@ -1850,7 +1850,7 @@ } /* First of all, flush stderr */ - tmp = _PyObject_CallMethodId(ferr, &PyId_flush, ""); + tmp = _PyObject_CallMethodId(ferr, &PyId_flush, NULL); if (tmp == NULL) PyErr_Clear(); else @@ -1859,7 +1859,7 @@ /* We should only use (GNU) readline if Python's sys.stdin and sys.stdout are the same as C's stdin and stdout, because we need to pass it those. */ - tmp = _PyObject_CallMethodId(fin, &PyId_fileno, ""); + tmp = _PyObject_CallMethodId(fin, &PyId_fileno, NULL); if (tmp == NULL) { PyErr_Clear(); tty = 0; @@ -1872,7 +1872,7 @@ tty = fd == fileno(stdin) && isatty(fd); } if (tty) { - tmp = _PyObject_CallMethodId(fout, &PyId_fileno, ""); + tmp = _PyObject_CallMethodId(fout, &PyId_fileno, NULL); if (tmp == NULL) { PyErr_Clear(); tty = 0; @@ -1907,7 +1907,7 @@ stdin_errors_str = _PyUnicode_AsString(stdin_errors); if (!stdin_encoding_str || !stdin_errors_str) goto _readline_errors; - tmp = _PyObject_CallMethodId(fout, &PyId_flush, ""); + tmp = _PyObject_CallMethodId(fout, &PyId_flush, NULL); if (tmp == NULL) PyErr_Clear(); else @@ -1987,7 +1987,7 @@ if (PyFile_WriteObject(prompt, fout, Py_PRINT_RAW) != 0) return NULL; } - tmp = _PyObject_CallMethodId(fout, &PyId_flush, ""); + tmp = _PyObject_CallMethodId(fout, &PyId_flush, NULL); if (tmp == NULL) PyErr_Clear(); else diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -493,7 +493,7 @@ int status = 0; if (fout != NULL && fout != Py_None && !file_is_closed(fout)) { - tmp = _PyObject_CallMethodId(fout, &PyId_flush, ""); + tmp = _PyObject_CallMethodId(fout, &PyId_flush, NULL); if (tmp == NULL) { PyErr_WriteUnraisable(fout); status = -1; @@ -503,7 +503,7 @@ } if (ferr != NULL && ferr != Py_None && !file_is_closed(ferr)) { - tmp = _PyObject_CallMethodId(ferr, &PyId_flush, ""); + tmp = _PyObject_CallMethodId(ferr, &PyId_flush, NULL); if (tmp == NULL) { PyErr_Clear(); status = -1; @@ -1072,7 +1072,7 @@ text = PyUnicode_FromString(name); if (text == NULL || _PyObject_SetAttrId(raw, &PyId_name, text) < 0) goto error; - res = _PyObject_CallMethodId(raw, &PyId_isatty, ""); + res = _PyObject_CallMethodId(raw, &PyId_isatty, NULL); if (res == NULL) goto error; isatty = PyObject_IsTrue(res); @@ -1343,7 +1343,7 @@ Py_XDECREF(tb); /* sys.stderr may be buffered: call sys.stderr.flush() */ - res = _PyObject_CallMethodId(ferr, &PyId_flush, ""); + res = _PyObject_CallMethodId(ferr, &PyId_flush, NULL); if (res == NULL) PyErr_Clear(); else @@ -1453,7 +1453,7 @@ PyErr_Clear(); return; } - result = _PyObject_CallMethodId(threading, &PyId__shutdown, ""); + result = _PyObject_CallMethodId(threading, &PyId__shutdown, NULL); if (result == NULL) { PyErr_WriteUnraisable(threading); } diff --git a/Python/pythonrun.c b/Python/pythonrun.c --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -950,7 +950,7 @@ f = _PySys_GetObjectId(&PyId_stderr); if (f != NULL) { - r = _PyObject_CallMethodId(f, &PyId_flush, ""); + r = _PyObject_CallMethodId(f, &PyId_flush, NULL); if (r) Py_DECREF(r); else @@ -958,7 +958,7 @@ } f = _PySys_GetObjectId(&PyId_stdout); if (f != NULL) { - r = _PyObject_CallMethodId(f, &PyId_flush, ""); + r = _PyObject_CallMethodId(f, &PyId_flush, NULL); if (r) Py_DECREF(r); else diff --git a/Python/traceback.c b/Python/traceback.c --- a/Python/traceback.c +++ b/Python/traceback.c @@ -314,7 +314,7 @@ if (fob == NULL) { PyErr_Clear(); - res = _PyObject_CallMethodId(binary, &PyId_close, ""); + res = _PyObject_CallMethodId(binary, &PyId_close, NULL); Py_DECREF(binary); if (res) Py_DECREF(res); @@ -334,7 +334,7 @@ break; } } - res = _PyObject_CallMethodId(fob, &PyId_close, ""); + res = _PyObject_CallMethodId(fob, &PyId_close, NULL); if (res) Py_DECREF(res); else -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 00:46:33 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 06 Sep 2016 04:46:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Avoid_inefficient_way_to_c?= =?utf-8?q?all_functions_without_argument?= Message-ID: <20160906044633.23482.12222.E3A5485E@psf.io> https://hg.python.org/cpython/rev/1ab78a8c2177 changeset: 103112:1ab78a8c2177 user: Victor Stinner date: Mon Sep 05 17:53:15 2016 -0700 summary: Avoid inefficient way to call functions without argument Don't pass "()" format to PyObject_CallXXX() to call a function without argument: pass NULL as the format string instead. It avoids to have to parse a string to produce 0 argument. files: Modules/_collectionsmodule.c | 2 +- Modules/_datetimemodule.c | 8 ++++---- Modules/_pickle.c | 2 +- Objects/typeobject.c | 8 ++++---- Python/modsupport.c | 2 +- 5 files changed, 11 insertions(+), 11 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -2009,7 +2009,7 @@ args = PyTuple_Pack(1, dd->default_factory); if (args == NULL) return NULL; - items = _PyObject_CallMethodId((PyObject *)dd, &PyId_items, "()"); + items = _PyObject_CallMethodId((PyObject *)dd, &PyId_items, NULL); if (items == NULL) { Py_DECREF(args); return NULL; diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -1378,7 +1378,7 @@ if (time != NULL) { _Py_IDENTIFIER(time); - result = _PyObject_CallMethodId(time, &PyId_time, "()"); + result = _PyObject_CallMethodId(time, &PyId_time, NULL); Py_DECREF(time); } return result; @@ -2703,7 +2703,7 @@ static PyObject * date_str(PyDateTime_Date *self) { - return _PyObject_CallMethodId((PyObject *)self, &PyId_isoformat, "()"); + return _PyObject_CallMethodId((PyObject *)self, &PyId_isoformat, NULL); } @@ -2729,7 +2729,7 @@ &format)) return NULL; - tuple = _PyObject_CallMethodId((PyObject *)self, &PyId_timetuple, "()"); + tuple = _PyObject_CallMethodId((PyObject *)self, &PyId_timetuple, NULL); if (tuple == NULL) return NULL; result = wrap_strftime((PyObject *)self, format, tuple, @@ -3675,7 +3675,7 @@ static PyObject * time_str(PyDateTime_Time *self) { - return _PyObject_CallMethodId((PyObject *)self, &PyId_isoformat, "()"); + return _PyObject_CallMethodId((PyObject *)self, &PyId_isoformat, NULL); } static PyObject * diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -2873,7 +2873,7 @@ } else { _Py_IDENTIFIER(items); - items = _PyObject_CallMethodId(obj, &PyId_items, "()"); + items = _PyObject_CallMethodId(obj, &PyId_items, NULL); if (items == NULL) goto error; iter = PyObject_GetIter(items); diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -5758,7 +5758,7 @@ FUNCNAME(PyObject *self) \ { \ _Py_static_string(id, OPSTR); \ - return call_method(self, &id, "()"); \ + return call_method(self, &id, NULL); \ } #define SLOT1(FUNCNAME, OPSTR, ARG1TYPE, ARGCODES) \ @@ -5851,7 +5851,7 @@ static Py_ssize_t slot_sq_length(PyObject *self) { - PyObject *res = call_method(self, &PyId___len__, "()"); + PyObject *res = call_method(self, &PyId___len__, NULL); Py_ssize_t len; if (res == NULL) @@ -6065,7 +6065,7 @@ slot_nb_index(PyObject *self) { _Py_IDENTIFIER(__index__); - return call_method(self, &PyId___index__, "()"); + return call_method(self, &PyId___index__, NULL); } @@ -6351,7 +6351,7 @@ slot_tp_iternext(PyObject *self) { _Py_IDENTIFIER(__next__); - return call_method(self, &PyId___next__, "()"); + return call_method(self, &PyId___next__, NULL); } static PyObject * diff --git a/Python/modsupport.c b/Python/modsupport.c --- a/Python/modsupport.c +++ b/Python/modsupport.c @@ -468,7 +468,7 @@ int n = countformat(f, '\0'); va_list lva; - Py_VA_COPY(lva, va); + Py_VA_COPY(lva, va); if (n < 0) return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 04:45:58 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 08:45:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27866=3A_relax_test_?= =?utf-8?q?case_for_set=5Fcipher=28=29_and_allow_more_cipher_suites?= Message-ID: <20160906084557.9570.7725.F961B41A@psf.io> https://hg.python.org/cpython/rev/9377ed49746b changeset: 103114:9377ed49746b user: Christian Heimes date: Tue Sep 06 10:45:44 2016 +0200 summary: Issue 27866: relax test case for set_cipher() and allow more cipher suites files: Lib/test/test_ssl.py | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -837,11 +837,10 @@ @unittest.skipIf(ssl.OPENSSL_VERSION_INFO < (1, 0, 2, 0, 0), 'OpenSSL too old') def test_get_ciphers(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) - ctx.set_ciphers('ECDHE+AESGCM:!ECDSA') + ctx.set_ciphers('AESGCM') names = set(d['name'] for d in ctx.get_ciphers()) - self.assertEqual(names, - {'ECDHE-RSA-AES256-GCM-SHA384', - 'ECDHE-RSA-AES128-GCM-SHA256'}) + self.assertIn('ECDHE-RSA-AES256-GCM-SHA384', names) + self.assertIn('ECDHE-RSA-AES128-GCM-SHA256', names) @skip_if_broken_ubuntu_ssl def test_options(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 05:14:18 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 09:14:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27744=3A_skip_test_i?= =?utf-8?q?f_AF=5FALG_socket_bind_fails?= Message-ID: <20160906091418.114862.50149.4E279585@psf.io> https://hg.python.org/cpython/rev/e3b83bfa02c5 changeset: 103115:e3b83bfa02c5 user: Christian Heimes date: Tue Sep 06 11:14:09 2016 +0200 summary: Issue 27744: skip test if AF_ALG socket bind fails files: Lib/test/test_socket.py | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -5343,7 +5343,11 @@ # tests for AF_ALG def create_alg(self, typ, name): sock = socket.socket(socket.AF_ALG, socket.SOCK_SEQPACKET, 0) - sock.bind((typ, name)) + try: + sock.bind((typ, name)) + except FileNotFoundError as e: + # type / algorithm is not available + raise unittest.SkipTest(str(e), typ, name) return sock def test_sha256(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 05:27:32 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 09:27:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27866=3A_relax_get?= =?utf-8?q?=5Fcipher=28=29_test_even_more=2E_Gentoo_buildbot_has_no_ECDHE?= Message-ID: <20160906092732.22717.94093.E1D43409@psf.io> https://hg.python.org/cpython/rev/dad4c42869f6 changeset: 103116:dad4c42869f6 user: Christian Heimes date: Tue Sep 06 11:27:25 2016 +0200 summary: Issue 27866: relax get_cipher() test even more. Gentoo buildbot has no ECDHE files: Lib/test/test_ssl.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -839,8 +839,8 @@ ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) ctx.set_ciphers('AESGCM') names = set(d['name'] for d in ctx.get_ciphers()) - self.assertIn('ECDHE-RSA-AES256-GCM-SHA384', names) - self.assertIn('ECDHE-RSA-AES128-GCM-SHA256', names) + self.assertIn('AES256-GCM-SHA384', names) + self.assertIn('AES128-GCM-SHA256', names) @skip_if_broken_ubuntu_ssl def test_options(self): -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Tue Sep 6 09:58:43 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 6 Sep 2016 14:58:43 +0100 Subject: [Python-checkins] NEUTRAL Benchmark Results for Python 2.7 2016-09-06 Message-ID: <97a55d2c-0fc0-45e7-9ceb-2a4b93efc265@irsmsx103.ger.corp.intel.com> Results for project Python 2.7, build date 2016-09-06 02:46:54 +0000 commit: 65102408021e previous commit: 6e4475894a79 revision date: 2016-09-06 00:22:09 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.21% -0.06% 4.46% 3.32% :-) pybench 0.22% -0.09% 6.30% 3.59% :-( regex_v8 0.60% -0.15% -2.24% 10.81% :-) nbody 0.08% -0.01% 8.27% 0.18% :-| json_dump_v2 0.39% -0.81% 0.54% 11.94% :-| normal_startup 0.52% 0.01% -0.74% 2.39% :-) ssbench 0.27% -0.00% 2.92% 1.22% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/neutral-benchmark-results-for-python-2-7-2016-09-06/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Tue Sep 6 10:05:39 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Tue, 6 Sep 2016 15:05:39 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python Default 2016-09-06 Message-ID: <5686c67e-2cf9-432e-b07f-c887d73a5c21@irsmsx103.ger.corp.intel.com> Results for project Python default, build date 2016-09-06 02:00:27 +0000 commit: ebaeeb36eb4b previous commit: ffc915a55a72 revision date: 2016-09-06 01:26:26 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.13% 0.07% 10.98% 17.73% :-) pybench 0.20% -0.28% 6.87% 7.26% :-( regex_v8 2.55% -1.26% -3.62% 9.23% :-| nbody 0.26% 0.33% 1.17% 7.24% :-| json_dump_v2 0.26% 1.64% -1.04% 9.95% :-) normal_startup 1.06% 0.49% 2.02% 5.52% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-09-06/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Tue Sep 6 13:02:06 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 17:02:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_fix_unused_var?= =?utf-8?q?iable_warnings_in_pysqlite_=28closes_=2327967=29?= Message-ID: <20160906170204.68689.28860.C77A4CA6@psf.io> https://hg.python.org/cpython/rev/ecaa1af8fe04 changeset: 103117:ecaa1af8fe04 branch: 2.7 parent: 103101:65102408021e user: Benjamin Peterson date: Tue Sep 06 10:01:16 2016 -0700 summary: fix unused variable warnings in pysqlite (closes #27967) files: Modules/_sqlite/cursor.c | 4 +--- Modules/_sqlite/statement.c | 4 +--- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -118,11 +118,9 @@ static void pysqlite_cursor_dealloc(pysqlite_Cursor* self) { - int rc; - /* Reset the statement if the user has not closed the cursor */ if (self->statement) { - rc = pysqlite_statement_reset(self->statement); + pysqlite_statement_reset(self->statement); Py_DECREF(self->statement); } diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -411,11 +411,9 @@ void pysqlite_statement_dealloc(pysqlite_Statement* self) { - int rc; - if (self->st) { Py_BEGIN_ALLOW_THREADS - rc = sqlite3_finalize(self->st); + sqlite3_finalize(self->st); Py_END_ALLOW_THREADS } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 13:07:05 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 17:07:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_suppress_stder?= =?utf-8?q?r_output_when_checking_gdb_=28closes_=2327969=29?= Message-ID: <20160906170704.22162.43580.5FFFF497@psf.io> https://hg.python.org/cpython/rev/5e75bf8e5526 changeset: 103118:5e75bf8e5526 branch: 2.7 user: Benjamin Peterson date: Tue Sep 06 10:06:31 2016 -0700 summary: suppress stderr output when checking gdb (closes #27969) files: Lib/test/test_gdb.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -24,6 +24,7 @@ try: proc = subprocess.Popen(["gdb", "-nx", "--version"], stdout=subprocess.PIPE, + stderr=subprocess.PIPE, universal_newlines=True) version = proc.communicate()[0] except OSError: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 13:07:46 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 17:07:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_suppress_stder?= =?utf-8?q?r_output_when_checking_gdb_=28closes_=2327969=29?= Message-ID: <20160906170704.22717.29604.15A055FD@psf.io> https://hg.python.org/cpython/rev/2c4359ff4d6d changeset: 103119:2c4359ff4d6d branch: 3.5 parent: 103110:b442744d2d22 user: Benjamin Peterson date: Tue Sep 06 10:06:31 2016 -0700 summary: suppress stderr output when checking gdb (closes #27969) files: Lib/test/test_gdb.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -25,6 +25,7 @@ try: proc = subprocess.Popen(["gdb", "-nx", "--version"], stdout=subprocess.PIPE, + stderr=subprocess.PIPE, universal_newlines=True) with proc: version = proc.communicate()[0] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 13:07:46 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 17:07:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjc5Njkp?= Message-ID: <20160906170704.6696.97349.12D846D3@psf.io> https://hg.python.org/cpython/rev/6e827e97c064 changeset: 103120:6e827e97c064 parent: 103116:dad4c42869f6 parent: 103119:2c4359ff4d6d user: Benjamin Peterson date: Tue Sep 06 10:06:51 2016 -0700 summary: merge 3.5 (#27969) files: Lib/test/test_gdb.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -24,6 +24,7 @@ try: proc = subprocess.Popen(["gdb", "-nx", "--version"], stdout=subprocess.PIPE, + stderr=subprocess.PIPE, universal_newlines=True) with proc: version = proc.communicate()[0] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 13:47:24 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 17:47:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_replace_PY=5FLONG=5FLONG_w?= =?utf-8?q?ith_long_long?= Message-ID: <20160906174714.22565.34807.2CAD15C5@psf.io> https://hg.python.org/cpython/rev/177522a8f473 changeset: 103121:177522a8f473 user: Benjamin Peterson date: Tue Sep 06 10:46:49 2016 -0700 summary: replace PY_LONG_LONG with long long files: Include/longobject.h | 12 +- Include/pyport.h | 4 +- Include/pythread.h | 2 +- Include/pytime.h | 2 +- Misc/coverity_model.c | 3 +- Modules/_datetimemodule.c | 26 ++++---- Modules/_lsprof.c | 30 ++++---- Modules/_lzmamodule.c | 16 +--- Modules/_struct.c | 52 ++++++++-------- Modules/_testcapimodule.c | 38 ++++++------ Modules/addrinfo.h | 4 +- Modules/arraymodule.c | 26 ++++---- Modules/md5module.c | 2 +- Modules/mmapmodule.c | 12 +- Modules/posixmodule.c | 22 +++--- Modules/resource.c | 6 +- Modules/sha1module.c | 2 +- Modules/sha512module.c | 12 +-- Objects/longobject.c | 82 +++++++++++++------------- Objects/memoryobject.c | 34 +++++----- Objects/rangeobject.c | 2 +- Objects/unicodeobject.c | 4 +- PC/pyconfig.h | 5 +- Python/condvar.h | 6 +- Python/getargs.c | 16 ++-- Python/modsupport.c | 4 +- Python/pytime.c | 18 ++-- Python/structmember.c | 16 ++-- Python/thread_nt.h | 2 +- 29 files changed, 224 insertions(+), 236 deletions(-) diff --git a/Include/longobject.h b/Include/longobject.h --- a/Include/longobject.h +++ b/Include/longobject.h @@ -85,12 +85,12 @@ PyAPI_FUNC(PyObject *) PyLong_FromVoidPtr(void *); PyAPI_FUNC(void *) PyLong_AsVoidPtr(PyObject *); -PyAPI_FUNC(PyObject *) PyLong_FromLongLong(PY_LONG_LONG); -PyAPI_FUNC(PyObject *) PyLong_FromUnsignedLongLong(unsigned PY_LONG_LONG); -PyAPI_FUNC(PY_LONG_LONG) PyLong_AsLongLong(PyObject *); -PyAPI_FUNC(unsigned PY_LONG_LONG) PyLong_AsUnsignedLongLong(PyObject *); -PyAPI_FUNC(unsigned PY_LONG_LONG) PyLong_AsUnsignedLongLongMask(PyObject *); -PyAPI_FUNC(PY_LONG_LONG) PyLong_AsLongLongAndOverflow(PyObject *, int *); +PyAPI_FUNC(PyObject *) PyLong_FromLongLong(long long); +PyAPI_FUNC(PyObject *) PyLong_FromUnsignedLongLong(unsigned long long); +PyAPI_FUNC(long long) PyLong_AsLongLong(PyObject *); +PyAPI_FUNC(unsigned long long) PyLong_AsUnsignedLongLong(PyObject *); +PyAPI_FUNC(unsigned long long) PyLong_AsUnsignedLongLongMask(PyObject *); +PyAPI_FUNC(long long) PyLong_AsLongLongAndOverflow(PyObject *, int *); PyAPI_FUNC(PyObject *) PyLong_FromString(const char *, char **, int); #ifndef Py_LIMITED_API diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -156,8 +156,8 @@ typedef long Py_intptr_t; #elif SIZEOF_VOID_P <= SIZEOF_LONG_LONG -typedef unsigned PY_LONG_LONG Py_uintptr_t; -typedef PY_LONG_LONG Py_intptr_t; +typedef unsigned long long Py_uintptr_t; +typedef long long Py_intptr_t; #else # error "Python needs a typedef for Py_uintptr_t in pyport.h." diff --git a/Include/pythread.h b/Include/pythread.h --- a/Include/pythread.h +++ b/Include/pythread.h @@ -37,7 +37,7 @@ module exposes a higher-level API, with timeouts expressed in seconds and floating-point numbers allowed. */ -#define PY_TIMEOUT_T PY_LONG_LONG +#define PY_TIMEOUT_T long long #define PY_TIMEOUT_MAX PY_LLONG_MAX /* In the NT API, the timeout is a DWORD and is expressed in milliseconds */ diff --git a/Include/pytime.h b/Include/pytime.h --- a/Include/pytime.h +++ b/Include/pytime.h @@ -78,7 +78,7 @@ ((_PyTime_t)(seconds) * (1000 * 1000 * 1000)) /* Create a timestamp from a number of nanoseconds. */ -PyAPI_FUNC(_PyTime_t) _PyTime_FromNanoseconds(PY_LONG_LONG ns); +PyAPI_FUNC(_PyTime_t) _PyTime_FromNanoseconds(long long ns); /* Convert a number of seconds (Python float or int) to a timetamp. Raise an exception and return -1 on error, return 0 on success. */ diff --git a/Misc/coverity_model.c b/Misc/coverity_model.c --- a/Misc/coverity_model.c +++ b/Misc/coverity_model.c @@ -22,7 +22,6 @@ #define assert(op) /* empty */ typedef int sdigit; typedef long Py_ssize_t; -typedef long long PY_LONG_LONG; typedef unsigned short wchar_t; typedef struct {} PyObject; typedef struct {} grammar; @@ -63,7 +62,7 @@ return NULL; } -PyObject *PyLong_FromLongLong(PY_LONG_LONG ival) +PyObject *PyLong_FromLongLong(long long ival) { return PyLong_FromLong((long)ival); } diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -4198,20 +4198,20 @@ /* As of version 2015f max fold in IANA database is * 23 hours at 1969-09-30 13:00:00 in Kwajalein. */ -static PY_LONG_LONG max_fold_seconds = 24 * 3600; +static long long max_fold_seconds = 24 * 3600; /* NB: date(1970,1,1).toordinal() == 719163 */ -static PY_LONG_LONG epoch = Py_LL(719163) * 24 * 60 * 60; - -static PY_LONG_LONG +static long long epoch = Py_LL(719163) * 24 * 60 * 60; + +static long long utc_to_seconds(int year, int month, int day, int hour, int minute, int second) { - PY_LONG_LONG ordinal = ymd_to_ord(year, month, day); + long long ordinal = ymd_to_ord(year, month, day); return ((ordinal * 24 + hour) * 60 + minute) * 60 + second; } -static PY_LONG_LONG -local(PY_LONG_LONG u) +static long long +local(long long u) { struct tm local_time; time_t t; @@ -4269,7 +4269,7 @@ second = Py_MIN(59, tm->tm_sec); if (tzinfo == Py_None && f == localtime) { - PY_LONG_LONG probe_seconds, result_seconds, transition; + long long probe_seconds, result_seconds, transition; result_seconds = utc_to_seconds(year, month, day, hour, minute, second); @@ -5115,14 +5115,14 @@ return local_timezone_from_timestamp(timestamp); } -static PY_LONG_LONG +static long long local_to_seconds(int year, int month, int day, int hour, int minute, int second, int fold); static PyObject * local_timezone_from_local(PyDateTime_DateTime *local_dt) { - PY_LONG_LONG seconds; + long long seconds; time_t timestamp; seconds = local_to_seconds(GET_YEAR(local_dt), GET_MONTH(local_dt), @@ -5256,11 +5256,11 @@ dstflag); } -static PY_LONG_LONG +static long long local_to_seconds(int year, int month, int day, int hour, int minute, int second, int fold) { - PY_LONG_LONG t, a, b, u1, u2, t1, t2, lt; + long long t, a, b, u1, u2, t1, t2, lt; t = utc_to_seconds(year, month, day, hour, minute, second); /* Our goal is to solve t = local(u) for u. */ lt = local(t); @@ -5320,7 +5320,7 @@ Py_DECREF(delta); } else { - PY_LONG_LONG seconds; + long long seconds; seconds = local_to_seconds(GET_YEAR(self), GET_MONTH(self), GET_DAY(self), diff --git a/Modules/_lsprof.c b/Modules/_lsprof.c --- a/Modules/_lsprof.c +++ b/Modules/_lsprof.c @@ -8,7 +8,7 @@ #include -static PY_LONG_LONG +static long long hpTimer(void) { LARGE_INTEGER li; @@ -35,11 +35,11 @@ #include #include -static PY_LONG_LONG +static long long hpTimer(void) { struct timeval tv; - PY_LONG_LONG ret; + long long ret; #ifdef GETTIMEOFDAY_NO_TZ gettimeofday(&tv); #else @@ -66,8 +66,8 @@ /* represents a function called from another function */ typedef struct _ProfilerSubEntry { rotating_node_t header; - PY_LONG_LONG tt; - PY_LONG_LONG it; + long long tt; + long long it; long callcount; long recursivecallcount; long recursionLevel; @@ -77,8 +77,8 @@ typedef struct _ProfilerEntry { rotating_node_t header; PyObject *userObj; /* PyCodeObject, or a descriptive str for builtins */ - PY_LONG_LONG tt; /* total time in this entry */ - PY_LONG_LONG it; /* inline time in this entry (not in subcalls) */ + long long tt; /* total time in this entry */ + long long it; /* inline time in this entry (not in subcalls) */ long callcount; /* how many times this was called */ long recursivecallcount; /* how many times called recursively */ long recursionLevel; @@ -86,8 +86,8 @@ } ProfilerEntry; typedef struct _ProfilerContext { - PY_LONG_LONG t0; - PY_LONG_LONG subt; + long long t0; + long long subt; struct _ProfilerContext *previous; ProfilerEntry *ctxEntry; } ProfilerContext; @@ -117,9 +117,9 @@ #define DOUBLE_TIMER_PRECISION 4294967296.0 static PyObject *empty_tuple; -static PY_LONG_LONG CallExternalTimer(ProfilerObject *pObj) +static long long CallExternalTimer(ProfilerObject *pObj) { - PY_LONG_LONG result; + long long result; PyObject *o = PyObject_Call(pObj->externalTimer, empty_tuple, NULL); if (o == NULL) { PyErr_WriteUnraisable(pObj->externalTimer); @@ -132,11 +132,11 @@ } else { /* interpret the result as a double measured in seconds. - As the profiler works with PY_LONG_LONG internally + As the profiler works with long long internally we convert it to a large integer */ double val = PyFloat_AsDouble(o); /* error handling delayed to the code below */ - result = (PY_LONG_LONG) (val * DOUBLE_TIMER_PRECISION); + result = (long long) (val * DOUBLE_TIMER_PRECISION); } Py_DECREF(o); if (PyErr_Occurred()) { @@ -338,8 +338,8 @@ static void Stop(ProfilerObject *pObj, ProfilerContext *self, ProfilerEntry *entry) { - PY_LONG_LONG tt = CALL_TIMER(pObj) - self->t0; - PY_LONG_LONG it = tt - self->subt; + long long tt = CALL_TIMER(pObj) - self->t0; + long long it = tt - self->subt; if (self->previous) self->previous->subt += tt; pObj->currentProfilerContext = self->previous; diff --git a/Modules/_lzmamodule.c b/Modules/_lzmamodule.c --- a/Modules/_lzmamodule.c +++ b/Modules/_lzmamodule.c @@ -18,12 +18,6 @@ #include - -#ifndef PY_LONG_LONG -#error "This module requires PY_LONG_LONG to be defined" -#endif - - #ifdef WITH_THREAD #define ACQUIRE_LOCK(obj) do { \ if (!PyThread_acquire_lock((obj)->lock, 0)) { \ @@ -163,7 +157,7 @@ uint32_t - the "I" (unsigned int) specifier is the right size, but silently ignores overflows on conversion. - lzma_vli - the "K" (unsigned PY_LONG_LONG) specifier is the right + lzma_vli - the "K" (unsigned long long) specifier is the right size, but like "I" it silently ignores overflows on conversion. lzma_mode and lzma_match_finder - these are enumeration types, and @@ -176,12 +170,12 @@ static int \ FUNCNAME(PyObject *obj, void *ptr) \ { \ - unsigned PY_LONG_LONG val; \ + unsigned long long val; \ \ val = PyLong_AsUnsignedLongLong(obj); \ if (PyErr_Occurred()) \ return 0; \ - if ((unsigned PY_LONG_LONG)(TYPE)val != val) { \ + if ((unsigned long long)(TYPE)val != val) { \ PyErr_SetString(PyExc_OverflowError, \ "Value too large for " #TYPE " type"); \ return 0; \ @@ -398,7 +392,7 @@ Python-level filter specifiers (represented as dicts). */ static int -spec_add_field(PyObject *spec, _Py_Identifier *key, unsigned PY_LONG_LONG value) +spec_add_field(PyObject *spec, _Py_Identifier *key, unsigned long long value) { int status; PyObject *value_object; @@ -1441,7 +1435,7 @@ /* Some of our constants are more than 32 bits wide, so PyModule_AddIntConstant would not work correctly on platforms with 32-bit longs. */ static int -module_add_int_constant(PyObject *m, const char *name, PY_LONG_LONG value) +module_add_int_constant(PyObject *m, const char *name, long long value) { PyObject *o = PyLong_FromLongLong(value); if (o == NULL) diff --git a/Modules/_struct.c b/Modules/_struct.c --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -71,8 +71,8 @@ /* We can't support q and Q in native mode unless the compiler does; in std mode, they're 8 bytes on all platforms. */ -typedef struct { char c; PY_LONG_LONG x; } s_long_long; -#define LONG_LONG_ALIGN (sizeof(s_long_long) - sizeof(PY_LONG_LONG)) +typedef struct { char c; long long x; } s_long_long; +#define LONG_LONG_ALIGN (sizeof(s_long_long) - sizeof(long long)) #ifdef HAVE_C99_BOOL #define BOOL_TYPE _Bool @@ -165,9 +165,9 @@ /* Same, but handling native long long. */ static int -get_longlong(PyObject *v, PY_LONG_LONG *p) +get_longlong(PyObject *v, long long *p) { - PY_LONG_LONG x; + long long x; v = get_pylong(v); if (v == NULL) @@ -175,7 +175,7 @@ assert(PyLong_Check(v)); x = PyLong_AsLongLong(v); Py_DECREF(v); - if (x == (PY_LONG_LONG)-1 && PyErr_Occurred()) { + if (x == (long long)-1 && PyErr_Occurred()) { if (PyErr_ExceptionMatches(PyExc_OverflowError)) PyErr_SetString(StructError, "argument out of range"); @@ -188,9 +188,9 @@ /* Same, but handling native unsigned long long. */ static int -get_ulonglong(PyObject *v, unsigned PY_LONG_LONG *p) +get_ulonglong(PyObject *v, unsigned long long *p) { - unsigned PY_LONG_LONG x; + unsigned long long x; v = get_pylong(v); if (v == NULL) @@ -198,7 +198,7 @@ assert(PyLong_Check(v)); x = PyLong_AsUnsignedLongLong(v); Py_DECREF(v); - if (x == (unsigned PY_LONG_LONG)-1 && PyErr_Occurred()) { + if (x == (unsigned long long)-1 && PyErr_Occurred()) { if (PyErr_ExceptionMatches(PyExc_OverflowError)) PyErr_SetString(StructError, "argument out of range"); @@ -460,20 +460,20 @@ static PyObject * nu_longlong(const char *p, const formatdef *f) { - PY_LONG_LONG x; + long long x; memcpy((char *)&x, p, sizeof x); if (x >= LONG_MIN && x <= LONG_MAX) - return PyLong_FromLong(Py_SAFE_DOWNCAST(x, PY_LONG_LONG, long)); + return PyLong_FromLong(Py_SAFE_DOWNCAST(x, long long, long)); return PyLong_FromLongLong(x); } static PyObject * nu_ulonglong(const char *p, const formatdef *f) { - unsigned PY_LONG_LONG x; + unsigned long long x; memcpy((char *)&x, p, sizeof x); if (x <= LONG_MAX) - return PyLong_FromLong(Py_SAFE_DOWNCAST(x, unsigned PY_LONG_LONG, long)); + return PyLong_FromLong(Py_SAFE_DOWNCAST(x, unsigned long long, long)); return PyLong_FromUnsignedLongLong(x); } @@ -673,7 +673,7 @@ static int np_longlong(char *p, PyObject *v, const formatdef *f) { - PY_LONG_LONG x; + long long x; if (get_longlong(v, &x) < 0) return -1; memcpy(p, (char *)&x, sizeof x); @@ -683,7 +683,7 @@ static int np_ulonglong(char *p, PyObject *v, const formatdef *f) { - unsigned PY_LONG_LONG x; + unsigned long long x; if (get_ulonglong(v, &x) < 0) return -1; memcpy(p, (char *)&x, sizeof x); @@ -772,8 +772,8 @@ {'L', sizeof(long), LONG_ALIGN, nu_ulong, np_ulong}, {'n', sizeof(size_t), SIZE_T_ALIGN, nu_ssize_t, np_ssize_t}, {'N', sizeof(size_t), SIZE_T_ALIGN, nu_size_t, np_size_t}, - {'q', sizeof(PY_LONG_LONG), LONG_LONG_ALIGN, nu_longlong, np_longlong}, - {'Q', sizeof(PY_LONG_LONG), LONG_LONG_ALIGN, nu_ulonglong,np_ulonglong}, + {'q', sizeof(long long), LONG_LONG_ALIGN, nu_longlong, np_longlong}, + {'Q', sizeof(long long), LONG_LONG_ALIGN, nu_ulonglong,np_ulonglong}, {'?', sizeof(BOOL_TYPE), BOOL_ALIGN, nu_bool, np_bool}, {'e', sizeof(short), SHORT_ALIGN, nu_halffloat, np_halffloat}, {'f', sizeof(float), FLOAT_ALIGN, nu_float, np_float}, @@ -816,7 +816,7 @@ static PyObject * bu_longlong(const char *p, const formatdef *f) { - PY_LONG_LONG x = 0; + long long x = 0; Py_ssize_t i = f->size; const unsigned char *bytes = (const unsigned char *)p; do { @@ -824,23 +824,23 @@ } while (--i > 0); /* Extend the sign bit. */ if (SIZEOF_LONG_LONG > f->size) - x |= -(x & ((PY_LONG_LONG)1 << ((8 * f->size) - 1))); + x |= -(x & ((long long)1 << ((8 * f->size) - 1))); if (x >= LONG_MIN && x <= LONG_MAX) - return PyLong_FromLong(Py_SAFE_DOWNCAST(x, PY_LONG_LONG, long)); + return PyLong_FromLong(Py_SAFE_DOWNCAST(x, long long, long)); return PyLong_FromLongLong(x); } static PyObject * bu_ulonglong(const char *p, const formatdef *f) { - unsigned PY_LONG_LONG x = 0; + unsigned long long x = 0; Py_ssize_t i = f->size; const unsigned char *bytes = (const unsigned char *)p; do { x = (x<<8) | *bytes++; } while (--i > 0); if (x <= LONG_MAX) - return PyLong_FromLong(Py_SAFE_DOWNCAST(x, unsigned PY_LONG_LONG, long)); + return PyLong_FromLong(Py_SAFE_DOWNCAST(x, unsigned long long, long)); return PyLong_FromUnsignedLongLong(x); } @@ -1043,7 +1043,7 @@ static PyObject * lu_longlong(const char *p, const formatdef *f) { - PY_LONG_LONG x = 0; + long long x = 0; Py_ssize_t i = f->size; const unsigned char *bytes = (const unsigned char *)p; do { @@ -1051,23 +1051,23 @@ } while (i > 0); /* Extend the sign bit. */ if (SIZEOF_LONG_LONG > f->size) - x |= -(x & ((PY_LONG_LONG)1 << ((8 * f->size) - 1))); + x |= -(x & ((long long)1 << ((8 * f->size) - 1))); if (x >= LONG_MIN && x <= LONG_MAX) - return PyLong_FromLong(Py_SAFE_DOWNCAST(x, PY_LONG_LONG, long)); + return PyLong_FromLong(Py_SAFE_DOWNCAST(x, long long, long)); return PyLong_FromLongLong(x); } static PyObject * lu_ulonglong(const char *p, const formatdef *f) { - unsigned PY_LONG_LONG x = 0; + unsigned long long x = 0; Py_ssize_t i = f->size; const unsigned char *bytes = (const unsigned char *)p; do { x = (x<<8) | bytes[--i]; } while (i > 0); if (x <= LONG_MAX) - return PyLong_FromLong(Py_SAFE_DOWNCAST(x, unsigned PY_LONG_LONG, long)); + return PyLong_FromLong(Py_SAFE_DOWNCAST(x, unsigned long long, long)); return PyLong_FromUnsignedLongLong(x); } diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -60,7 +60,7 @@ CHECK_SIZEOF(SIZEOF_LONG, long); CHECK_SIZEOF(SIZEOF_VOID_P, void*); CHECK_SIZEOF(SIZEOF_TIME_T, time_t); - CHECK_SIZEOF(SIZEOF_LONG_LONG, PY_LONG_LONG); + CHECK_SIZEOF(SIZEOF_LONG_LONG, long long); #undef CHECK_SIZEOF @@ -360,7 +360,7 @@ Note that the meat of the test is contained in testcapi_long.h. This is revolting, but delicate code duplication is worse: "almost - exactly the same" code is needed to test PY_LONG_LONG, but the ubiquitous + exactly the same" code is needed to test long long, but the ubiquitous dependence on type names makes it impossible to use a parameterized function. A giant macro would be even worse than this. A C++ template would be perfect. @@ -407,7 +407,7 @@ } #define TESTNAME test_longlong_api_inner -#define TYPENAME PY_LONG_LONG +#define TYPENAME long long #define F_S_TO_PY PyLong_FromLongLong #define F_PY_TO_S PyLong_AsLongLong #define F_U_TO_PY PyLong_FromUnsignedLongLong @@ -594,7 +594,7 @@ } /* Test the PyLong_AsLongLongAndOverflow API. General conversion to - PY_LONG_LONG is tested by test_long_api_inner. This test will + long long is tested by test_long_api_inner. This test will concentrate on proper handling of overflow. */ @@ -602,7 +602,7 @@ test_long_long_and_overflow(PyObject *self) { PyObject *num, *one, *temp; - PY_LONG_LONG value; + long long value; int overflow; /* Test that overflow is set properly for a large value. */ @@ -820,7 +820,7 @@ return Py_None; } -/* Test the L code for PyArg_ParseTuple. This should deliver a PY_LONG_LONG +/* Test the L code for PyArg_ParseTuple. This should deliver a long long for both long and int arguments. The test may leak a little memory if it fails. */ @@ -828,7 +828,7 @@ test_L_code(PyObject *self) { PyObject *tuple, *num; - PY_LONG_LONG value; + long long value; tuple = PyTuple_New(1); if (tuple == NULL) @@ -1133,7 +1133,7 @@ static PyObject * getargs_L(PyObject *self, PyObject *args) { - PY_LONG_LONG value; + long long value; if (!PyArg_ParseTuple(args, "L", &value)) return NULL; return PyLong_FromLongLong(value); @@ -1142,7 +1142,7 @@ static PyObject * getargs_K(PyObject *self, PyObject *args) { - unsigned PY_LONG_LONG value; + unsigned long long value; if (!PyArg_ParseTuple(args, "K", &value)) return NULL; return PyLong_FromUnsignedLongLong(value); @@ -2271,8 +2271,8 @@ CHECK_1_FORMAT("%zu", size_t); /* "%lld" and "%llu" support added in Python 2.7. */ - CHECK_1_FORMAT("%llu", unsigned PY_LONG_LONG); - CHECK_1_FORMAT("%lld", PY_LONG_LONG); + CHECK_1_FORMAT("%llu", unsigned long long); + CHECK_1_FORMAT("%lld", long long); Py_RETURN_NONE; @@ -3696,7 +3696,7 @@ static PyObject * test_pytime_assecondsdouble(PyObject *self, PyObject *args) { - PY_LONG_LONG ns; + long long ns; _PyTime_t ts; double d; @@ -3710,7 +3710,7 @@ static PyObject * test_PyTime_AsTimeval(PyObject *self, PyObject *args) { - PY_LONG_LONG ns; + long long ns; int round; _PyTime_t t; struct timeval tv; @@ -3724,7 +3724,7 @@ if (_PyTime_AsTimeval(t, &tv, round) < 0) return NULL; - seconds = PyLong_FromLong((PY_LONG_LONG)tv.tv_sec); + seconds = PyLong_FromLong((long long)tv.tv_sec); if (seconds == NULL) return NULL; return Py_BuildValue("Nl", seconds, tv.tv_usec); @@ -3734,7 +3734,7 @@ static PyObject * test_PyTime_AsTimespec(PyObject *self, PyObject *args) { - PY_LONG_LONG ns; + long long ns; _PyTime_t t; struct timespec ts; @@ -3750,7 +3750,7 @@ static PyObject * test_PyTime_AsMilliseconds(PyObject *self, PyObject *args) { - PY_LONG_LONG ns; + long long ns; int round; _PyTime_t t, ms; @@ -3768,7 +3768,7 @@ static PyObject * test_PyTime_AsMicroseconds(PyObject *self, PyObject *args) { - PY_LONG_LONG ns; + long long ns; int round; _PyTime_t t, ms; @@ -4141,8 +4141,8 @@ float float_member; double double_member; char inplace_member[6]; - PY_LONG_LONG longlong_member; - unsigned PY_LONG_LONG ulonglong_member; + long long longlong_member; + unsigned long long ulonglong_member; } all_structmembers; typedef struct { diff --git a/Modules/addrinfo.h b/Modules/addrinfo.h --- a/Modules/addrinfo.h +++ b/Modules/addrinfo.h @@ -141,7 +141,7 @@ * RFC 2553: protocol-independent placeholder for socket addresses */ #define _SS_MAXSIZE 128 -#define _SS_ALIGNSIZE (sizeof(PY_LONG_LONG)) +#define _SS_ALIGNSIZE (sizeof(long long)) #define _SS_PAD1SIZE (_SS_ALIGNSIZE - sizeof(u_char) * 2) #define _SS_PAD2SIZE (_SS_MAXSIZE - sizeof(u_char) * 2 - \ _SS_PAD1SIZE - _SS_ALIGNSIZE) @@ -154,7 +154,7 @@ unsigned short ss_family; /* address family */ #endif /* HAVE_SOCKADDR_SA_LEN */ char __ss_pad1[_SS_PAD1SIZE]; - PY_LONG_LONG __ss_align; /* force desired structure storage alignment */ + long long __ss_align; /* force desired structure storage alignment */ char __ss_pad2[_SS_PAD2SIZE]; }; #endif /* !HAVE_SOCKADDR_STORAGE */ diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -421,17 +421,17 @@ static PyObject * q_getitem(arrayobject *ap, Py_ssize_t i) { - return PyLong_FromLongLong(((PY_LONG_LONG *)ap->ob_item)[i]); + return PyLong_FromLongLong(((long long *)ap->ob_item)[i]); } static int q_setitem(arrayobject *ap, Py_ssize_t i, PyObject *v) { - PY_LONG_LONG x; + long long x; if (!PyArg_Parse(v, "L;array item must be integer", &x)) return -1; if (i >= 0) - ((PY_LONG_LONG *)ap->ob_item)[i] = x; + ((long long *)ap->ob_item)[i] = x; return 0; } @@ -439,20 +439,20 @@ QQ_getitem(arrayobject *ap, Py_ssize_t i) { return PyLong_FromUnsignedLongLong( - ((unsigned PY_LONG_LONG *)ap->ob_item)[i]); + ((unsigned long long *)ap->ob_item)[i]); } static int QQ_setitem(arrayobject *ap, Py_ssize_t i, PyObject *v) { - unsigned PY_LONG_LONG x; + unsigned long long x; if (PyLong_Check(v)) { x = PyLong_AsUnsignedLongLong(v); - if (x == (unsigned PY_LONG_LONG) -1 && PyErr_Occurred()) + if (x == (unsigned long long) -1 && PyErr_Occurred()) return -1; } else { - PY_LONG_LONG y; + long long y; if (!PyArg_Parse(v, "L;array item must be integer", &y)) return -1; if (y < 0) { @@ -460,11 +460,11 @@ "unsigned long long is less than minimum"); return -1; } - x = (unsigned PY_LONG_LONG)y; + x = (unsigned long long)y; } if (i >= 0) - ((unsigned PY_LONG_LONG *)ap->ob_item)[i] = x; + ((unsigned long long *)ap->ob_item)[i] = x; return 0; } @@ -518,8 +518,8 @@ {'I', sizeof(int), II_getitem, II_setitem, "I", 1, 0}, {'l', sizeof(long), l_getitem, l_setitem, "l", 1, 1}, {'L', sizeof(long), LL_getitem, LL_setitem, "L", 1, 0}, - {'q', sizeof(PY_LONG_LONG), q_getitem, q_setitem, "q", 1, 1}, - {'Q', sizeof(PY_LONG_LONG), QQ_getitem, QQ_setitem, "Q", 1, 0}, + {'q', sizeof(long long), q_getitem, q_setitem, "q", 1, 1}, + {'Q', sizeof(long long), QQ_getitem, QQ_setitem, "Q", 1, 0}, {'f', sizeof(float), f_getitem, f_setitem, "f", 0, 0}, {'d', sizeof(double), d_getitem, d_setitem, "d", 0, 0}, {'\0', 0, 0, 0, 0, 0, 0} /* Sentinel */ @@ -1810,11 +1810,11 @@ is_signed = 0; break; case 'q': - intsize = sizeof(PY_LONG_LONG); + intsize = sizeof(long long); is_signed = 1; break; case 'Q': - intsize = sizeof(PY_LONG_LONG); + intsize = sizeof(long long); is_signed = 0; break; default: diff --git a/Modules/md5module.c b/Modules/md5module.c --- a/Modules/md5module.c +++ b/Modules/md5module.c @@ -30,7 +30,7 @@ #if SIZEOF_INT == 4 typedef unsigned int MD5_INT32; /* 32-bit integer */ -typedef PY_LONG_LONG MD5_INT64; /* 64-bit integer */ +typedef long long MD5_INT64; /* 64-bit integer */ #else /* not defined. compilation will die. */ #endif diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -93,7 +93,7 @@ size_t size; size_t pos; /* relative to offset */ #ifdef MS_WINDOWS - PY_LONG_LONG offset; + long long offset; #else off_t offset; #endif @@ -446,7 +446,7 @@ #ifdef MS_WINDOWS if (self->file_handle != INVALID_HANDLE_VALUE) { DWORD low,high; - PY_LONG_LONG size; + long long size; low = GetFileSize(self->file_handle, &high); if (low == INVALID_FILE_SIZE) { /* It might be that the function appears to have failed, @@ -457,7 +457,7 @@ } if (!high && low < LONG_MAX) return PyLong_FromLong((long)low); - size = (((PY_LONG_LONG)high)<<32) + low; + size = (((long long)high)<<32) + low; return PyLong_FromLongLong(size); } else { return PyLong_FromSsize_t(self->size); @@ -1258,7 +1258,7 @@ /* A note on sizes and offsets: while the actual map size must hold in a Py_ssize_t, both the total file size and the start offset can be longer - than a Py_ssize_t, so we use PY_LONG_LONG which is always 64-bit. + than a Py_ssize_t, so we use long long which is always 64-bit. */ static PyObject * @@ -1267,7 +1267,7 @@ mmap_object *m_obj; PyObject *map_size_obj = NULL; Py_ssize_t map_size; - PY_LONG_LONG offset = 0, size; + long long offset = 0, size; DWORD off_hi; /* upper 32 bits of offset */ DWORD off_lo; /* lower 32 bits of offset */ DWORD size_hi; /* upper 32 bits of size */ @@ -1379,7 +1379,7 @@ return PyErr_SetFromWindowsErr(dwErr); } - size = (((PY_LONG_LONG) high) << 32) + low; + size = (((long long) high) << 32) + low; if (size == 0) { PyErr_SetString(PyExc_ValueError, "cannot mmap an empty file"); diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -1108,7 +1108,7 @@ } #ifdef MS_WINDOWS - typedef PY_LONG_LONG Py_off_t; + typedef long long Py_off_t; #else typedef off_t Py_off_t; #endif @@ -2073,7 +2073,7 @@ PyStructSequence_SET_ITEM(v, 0, PyLong_FromLong((long)st->st_mode)); #ifdef HAVE_LARGEFILE_SUPPORT PyStructSequence_SET_ITEM(v, 1, - PyLong_FromLongLong((PY_LONG_LONG)st->st_ino)); + PyLong_FromLongLong((long long)st->st_ino)); #else PyStructSequence_SET_ITEM(v, 1, PyLong_FromLong((long)st->st_ino)); #endif @@ -2092,7 +2092,7 @@ #endif #ifdef HAVE_LARGEFILE_SUPPORT PyStructSequence_SET_ITEM(v, 6, - PyLong_FromLongLong((PY_LONG_LONG)st->st_size)); + PyLong_FromLongLong((long long)st->st_size)); #else PyStructSequence_SET_ITEM(v, 6, PyLong_FromLong(st->st_size)); #endif @@ -9420,17 +9420,17 @@ PyStructSequence_SET_ITEM(v, 0, PyLong_FromLong((long) st.f_bsize)); PyStructSequence_SET_ITEM(v, 1, PyLong_FromLong((long) st.f_frsize)); PyStructSequence_SET_ITEM(v, 2, - PyLong_FromLongLong((PY_LONG_LONG) st.f_blocks)); + PyLong_FromLongLong((long long) st.f_blocks)); PyStructSequence_SET_ITEM(v, 3, - PyLong_FromLongLong((PY_LONG_LONG) st.f_bfree)); + PyLong_FromLongLong((long long) st.f_bfree)); PyStructSequence_SET_ITEM(v, 4, - PyLong_FromLongLong((PY_LONG_LONG) st.f_bavail)); + PyLong_FromLongLong((long long) st.f_bavail)); PyStructSequence_SET_ITEM(v, 5, - PyLong_FromLongLong((PY_LONG_LONG) st.f_files)); + PyLong_FromLongLong((long long) st.f_files)); PyStructSequence_SET_ITEM(v, 6, - PyLong_FromLongLong((PY_LONG_LONG) st.f_ffree)); + PyLong_FromLongLong((long long) st.f_ffree)); PyStructSequence_SET_ITEM(v, 7, - PyLong_FromLongLong((PY_LONG_LONG) st.f_favail)); + PyLong_FromLongLong((long long) st.f_favail)); PyStructSequence_SET_ITEM(v, 8, PyLong_FromLong((long) st.f_flag)); PyStructSequence_SET_ITEM(v, 9, PyLong_FromLong((long) st.f_namemax)); #endif @@ -11763,10 +11763,10 @@ self->win32_file_index = stat.st_ino; self->got_file_index = 1; } - return PyLong_FromLongLong((PY_LONG_LONG)self->win32_file_index); + return PyLong_FromLongLong((long long)self->win32_file_index); #else /* POSIX */ #ifdef HAVE_LARGEFILE_SUPPORT - return PyLong_FromLongLong((PY_LONG_LONG)self->d_ino); + return PyLong_FromLongLong((long long)self->d_ino); #else return PyLong_FromLong((long)self->d_ino); #endif diff --git a/Modules/resource.c b/Modules/resource.c --- a/Modules/resource.c +++ b/Modules/resource.c @@ -137,8 +137,8 @@ { if (sizeof(rl.rlim_cur) > sizeof(long)) { return Py_BuildValue("LL", - (PY_LONG_LONG) rl.rlim_cur, - (PY_LONG_LONG) rl.rlim_max); + (long long) rl.rlim_cur, + (long long) rl.rlim_max); } return Py_BuildValue("ll", (long) rl.rlim_cur, (long) rl.rlim_max); } @@ -437,7 +437,7 @@ #endif if (sizeof(RLIM_INFINITY) > sizeof(long)) { - v = PyLong_FromLongLong((PY_LONG_LONG) RLIM_INFINITY); + v = PyLong_FromLongLong((long long) RLIM_INFINITY); } else { v = PyLong_FromLong((long) RLIM_INFINITY); diff --git a/Modules/sha1module.c b/Modules/sha1module.c --- a/Modules/sha1module.c +++ b/Modules/sha1module.c @@ -30,7 +30,7 @@ #if SIZEOF_INT == 4 typedef unsigned int SHA1_INT32; /* 32-bit integer */ -typedef PY_LONG_LONG SHA1_INT64; /* 64-bit integer */ +typedef long long SHA1_INT64; /* 64-bit integer */ #else /* not defined. compilation will die. */ #endif diff --git a/Modules/sha512module.c b/Modules/sha512module.c --- a/Modules/sha512module.c +++ b/Modules/sha512module.c @@ -27,15 +27,13 @@ [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=81a3ccde92bcfe8d]*/ -#ifdef PY_LONG_LONG /* If no PY_LONG_LONG, don't compile anything! */ - /* Some useful types */ typedef unsigned char SHA_BYTE; #if SIZEOF_INT == 4 typedef unsigned int SHA_INT32; /* 32-bit integer */ -typedef unsigned PY_LONG_LONG SHA_INT64; /* 64-bit integer */ +typedef unsigned long long SHA_INT64; /* 64-bit integer */ #else /* not defined. compilation will die. */ #endif @@ -121,12 +119,12 @@ /* Various logical functions */ #define ROR64(x, y) \ - ( ((((x) & Py_ULL(0xFFFFFFFFFFFFFFFF))>>((unsigned PY_LONG_LONG)(y) & 63)) | \ - ((x)<<((unsigned PY_LONG_LONG)(64-((y) & 63))))) & Py_ULL(0xFFFFFFFFFFFFFFFF)) + ( ((((x) & Py_ULL(0xFFFFFFFFFFFFFFFF))>>((unsigned long long)(y) & 63)) | \ + ((x)<<((unsigned long long)(64-((y) & 63))))) & Py_ULL(0xFFFFFFFFFFFFFFFF)) #define Ch(x,y,z) (z ^ (x & (y ^ z))) #define Maj(x,y,z) (((x | y) & z) | (x & y)) #define S(x, n) ROR64((x),(n)) -#define R(x, n) (((x) & Py_ULL(0xFFFFFFFFFFFFFFFF)) >> ((unsigned PY_LONG_LONG)n)) +#define R(x, n) (((x) & Py_ULL(0xFFFFFFFFFFFFFFFF)) >> ((unsigned long long)n)) #define Sigma0(x) (S(x, 28) ^ S(x, 34) ^ S(x, 39)) #define Sigma1(x) (S(x, 14) ^ S(x, 18) ^ S(x, 41)) #define Gamma0(x) (S(x, 1) ^ S(x, 8) ^ R(x, 7)) @@ -803,5 +801,3 @@ PyModule_AddObject(m, "SHA512Type", (PyObject *)&SHA512type); return m; } - -#endif diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -993,9 +993,9 @@ #else #if SIZEOF_LONG_LONG < SIZEOF_VOID_P -# error "PyLong_FromVoidPtr: sizeof(PY_LONG_LONG) < sizeof(void*)" +# error "PyLong_FromVoidPtr: sizeof(long long) < sizeof(void*)" #endif - return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG)(Py_uintptr_t)p); + return PyLong_FromUnsignedLongLong((unsigned long long)(Py_uintptr_t)p); #endif /* SIZEOF_VOID_P <= SIZEOF_LONG */ } @@ -1015,9 +1015,9 @@ #else #if SIZEOF_LONG_LONG < SIZEOF_VOID_P -# error "PyLong_AsVoidPtr: sizeof(PY_LONG_LONG) < sizeof(void*)" +# error "PyLong_AsVoidPtr: sizeof(long long) < sizeof(void*)" #endif - PY_LONG_LONG x; + long long x; if (PyLong_Check(vv) && _PyLong_Sign(vv) < 0) x = PyLong_AsLongLong(vv); @@ -1031,20 +1031,20 @@ return (void *)x; } -/* Initial PY_LONG_LONG support by Chris Herborth (chrish at qnx.com), later +/* Initial long long support by Chris Herborth (chrish at qnx.com), later * rewritten to use the newer PyLong_{As,From}ByteArray API. */ -#define PY_ABS_LLONG_MIN (0-(unsigned PY_LONG_LONG)PY_LLONG_MIN) - -/* Create a new int object from a C PY_LONG_LONG int. */ +#define PY_ABS_LLONG_MIN (0-(unsigned long long)PY_LLONG_MIN) + +/* Create a new int object from a C long long int. */ PyObject * -PyLong_FromLongLong(PY_LONG_LONG ival) +PyLong_FromLongLong(long long ival) { PyLongObject *v; - unsigned PY_LONG_LONG abs_ival; - unsigned PY_LONG_LONG t; /* unsigned so >> doesn't propagate sign bit */ + unsigned long long abs_ival; + unsigned long long t; /* unsigned so >> doesn't propagate sign bit */ int ndigits = 0; int negative = 0; @@ -1052,11 +1052,11 @@ if (ival < 0) { /* avoid signed overflow on negation; see comments in PyLong_FromLong above. */ - abs_ival = (unsigned PY_LONG_LONG)(-1-ival) + 1; + abs_ival = (unsigned long long)(-1-ival) + 1; negative = 1; } else { - abs_ival = (unsigned PY_LONG_LONG)ival; + abs_ival = (unsigned long long)ival; } /* Count the number of Python digits. @@ -1081,19 +1081,19 @@ return (PyObject *)v; } -/* Create a new int object from a C unsigned PY_LONG_LONG int. */ +/* Create a new int object from a C unsigned long long int. */ PyObject * -PyLong_FromUnsignedLongLong(unsigned PY_LONG_LONG ival) +PyLong_FromUnsignedLongLong(unsigned long long ival) { PyLongObject *v; - unsigned PY_LONG_LONG t; + unsigned long long t; int ndigits = 0; if (ival < PyLong_BASE) return PyLong_FromLong((long)ival); /* Count the number of Python digits. */ - t = (unsigned PY_LONG_LONG)ival; + t = (unsigned long long)ival; while (t) { ++ndigits; t >>= PyLong_SHIFT; @@ -1182,11 +1182,11 @@ /* Get a C long long int from an int object or any object that has an __int__ method. Return -1 and set an error if overflow occurs. */ -PY_LONG_LONG +long long PyLong_AsLongLong(PyObject *vv) { PyLongObject *v; - PY_LONG_LONG bytes; + long long bytes; int res; int do_decref = 0; /* if nb_int was called */ @@ -1224,30 +1224,30 @@ Py_DECREF(v); } - /* Plan 9 can't handle PY_LONG_LONG in ? : expressions */ + /* Plan 9 can't handle long long in ? : expressions */ if (res < 0) - return (PY_LONG_LONG)-1; + return (long long)-1; else return bytes; } -/* Get a C unsigned PY_LONG_LONG int from an int object. +/* Get a C unsigned long long int from an int object. Return -1 and set an error if overflow occurs. */ -unsigned PY_LONG_LONG +unsigned long long PyLong_AsUnsignedLongLong(PyObject *vv) { PyLongObject *v; - unsigned PY_LONG_LONG bytes; + unsigned long long bytes; int res; if (vv == NULL) { PyErr_BadInternalCall(); - return (unsigned PY_LONG_LONG)-1; + return (unsigned long long)-1; } if (!PyLong_Check(vv)) { PyErr_SetString(PyExc_TypeError, "an integer is required"); - return (unsigned PY_LONG_LONG)-1; + return (unsigned long long)-1; } v = (PyLongObject*)vv; @@ -1259,9 +1259,9 @@ res = _PyLong_AsByteArray((PyLongObject *)vv, (unsigned char *)&bytes, SIZEOF_LONG_LONG, PY_LITTLE_ENDIAN, 0); - /* Plan 9 can't handle PY_LONG_LONG in ? : expressions */ + /* Plan 9 can't handle long long in ? : expressions */ if (res < 0) - return (unsigned PY_LONG_LONG)res; + return (unsigned long long)res; else return bytes; } @@ -1269,11 +1269,11 @@ /* Get a C unsigned long int from an int object, ignoring the high bits. Returns -1 and sets an error condition if an error occurs. */ -static unsigned PY_LONG_LONG +static unsigned long long _PyLong_AsUnsignedLongLongMask(PyObject *vv) { PyLongObject *v; - unsigned PY_LONG_LONG x; + unsigned long long x; Py_ssize_t i; int sign; @@ -1299,11 +1299,11 @@ return x * sign; } -unsigned PY_LONG_LONG +unsigned long long PyLong_AsUnsignedLongLongMask(PyObject *op) { PyLongObject *lo; - unsigned PY_LONG_LONG val; + unsigned long long val; if (op == NULL) { PyErr_BadInternalCall(); @@ -1316,7 +1316,7 @@ lo = _PyLong_FromNbInt(op); if (lo == NULL) - return (unsigned PY_LONG_LONG)-1; + return (unsigned long long)-1; val = _PyLong_AsUnsignedLongLongMask((PyObject *)lo); Py_DECREF(lo); @@ -1333,13 +1333,13 @@ In this case *overflow will be 0. */ -PY_LONG_LONG +long long PyLong_AsLongLongAndOverflow(PyObject *vv, int *overflow) { /* This version by Tim Peters */ PyLongObject *v; - unsigned PY_LONG_LONG x, prev; - PY_LONG_LONG res; + unsigned long long x, prev; + long long res; Py_ssize_t i; int sign; int do_decref = 0; /* if nb_int was called */ @@ -1391,8 +1391,8 @@ /* Haven't lost any bits, but casting to long requires extra * care (see comment above). */ - if (x <= (unsigned PY_LONG_LONG)PY_LLONG_MAX) { - res = (PY_LONG_LONG)x * sign; + if (x <= (unsigned long long)PY_LLONG_MAX) { + res = (long long)x * sign; } else if (sign < 0 && x == PY_ABS_LLONG_MIN) { res = PY_LLONG_MIN; @@ -3481,7 +3481,7 @@ /* fast path for single-digit multiplication */ if (Py_ABS(Py_SIZE(a)) <= 1 && Py_ABS(Py_SIZE(b)) <= 1) { stwodigits v = (stwodigits)(MEDIUM_VALUE(a)) * MEDIUM_VALUE(b); - return PyLong_FromLongLong((PY_LONG_LONG)v); + return PyLong_FromLongLong((long long)v); } z = k_mul(a, b); @@ -4650,7 +4650,7 @@ /* a fits into a long, so b must too */ x = PyLong_AsLong((PyObject *)a); y = PyLong_AsLong((PyObject *)b); -#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT +#elif defined(long long) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT x = PyLong_AsLongLong((PyObject *)a); y = PyLong_AsLongLong((PyObject *)b); #else @@ -4669,7 +4669,7 @@ } #if LONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT return PyLong_FromLong(x); -#elif defined(PY_LONG_LONG) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT +#elif defined(long long) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT return PyLong_FromLongLong(x); #else # error "_PyLong_GCD" diff --git a/Objects/memoryobject.c b/Objects/memoryobject.c --- a/Objects/memoryobject.c +++ b/Objects/memoryobject.c @@ -1111,7 +1111,7 @@ case 'h': case 'H': size = sizeof(short); break; case 'i': case 'I': size = sizeof(int); break; case 'l': case 'L': size = sizeof(long); break; - case 'q': case 'Q': size = sizeof(PY_LONG_LONG); break; + case 'q': case 'Q': size = sizeof(long long); break; case 'n': case 'N': size = sizeof(Py_ssize_t); break; case 'f': size = sizeof(float); break; case 'd': size = sizeof(double); break; @@ -1577,11 +1577,11 @@ return lu; } -static PY_LONG_LONG +static long long pylong_as_lld(PyObject *item) { PyObject *tmp; - PY_LONG_LONG lld; + long long lld; tmp = PyNumber_Index(item); if (tmp == NULL) @@ -1592,15 +1592,15 @@ return lld; } -static unsigned PY_LONG_LONG +static unsigned long long pylong_as_llu(PyObject *item) { PyObject *tmp; - unsigned PY_LONG_LONG llu; + unsigned long long llu; tmp = PyNumber_Index(item); if (tmp == NULL) - return (unsigned PY_LONG_LONG)-1; + return (unsigned long long)-1; llu = PyLong_AsUnsignedLongLong(tmp); Py_DECREF(tmp); @@ -1653,10 +1653,10 @@ Py_LOCAL_INLINE(PyObject *) unpack_single(const char *ptr, const char *fmt) { - unsigned PY_LONG_LONG llu; + unsigned long long llu; unsigned long lu; size_t zu; - PY_LONG_LONG lld; + long long lld; long ld; Py_ssize_t zd; double d; @@ -1685,8 +1685,8 @@ case 'L': UNPACK_SINGLE(lu, ptr, unsigned long); goto convert_lu; /* native 64-bit */ - case 'q': UNPACK_SINGLE(lld, ptr, PY_LONG_LONG); goto convert_lld; - case 'Q': UNPACK_SINGLE(llu, ptr, unsigned PY_LONG_LONG); goto convert_llu; + case 'q': UNPACK_SINGLE(lld, ptr, long long); goto convert_lld; + case 'Q': UNPACK_SINGLE(llu, ptr, unsigned long long); goto convert_llu; /* ssize_t and size_t */ case 'n': UNPACK_SINGLE(zd, ptr, Py_ssize_t); goto convert_zd; @@ -1747,10 +1747,10 @@ static int pack_single(char *ptr, PyObject *item, const char *fmt) { - unsigned PY_LONG_LONG llu; + unsigned long long llu; unsigned long lu; size_t zu; - PY_LONG_LONG lld; + long long lld; long ld; Py_ssize_t zd; double d; @@ -1802,13 +1802,13 @@ lld = pylong_as_lld(item); if (lld == -1 && PyErr_Occurred()) goto err_occurred; - PACK_SINGLE(ptr, lld, PY_LONG_LONG); + PACK_SINGLE(ptr, lld, long long); break; case 'Q': llu = pylong_as_llu(item); - if (llu == (unsigned PY_LONG_LONG)-1 && PyErr_Occurred()) + if (llu == (unsigned long long)-1 && PyErr_Occurred()) goto err_occurred; - PACK_SINGLE(ptr, llu, unsigned PY_LONG_LONG); + PACK_SINGLE(ptr, llu, unsigned long long); break; /* ssize_t and size_t */ @@ -2646,8 +2646,8 @@ case 'L': CMP_SINGLE(p, q, unsigned long); return equal; /* native 64-bit */ - case 'q': CMP_SINGLE(p, q, PY_LONG_LONG); return equal; - case 'Q': CMP_SINGLE(p, q, unsigned PY_LONG_LONG); return equal; + case 'q': CMP_SINGLE(p, q, long long); return equal; + case 'Q': CMP_SINGLE(p, q, unsigned long long); return equal; /* ssize_t and size_t */ case 'n': CMP_SINGLE(p, q, Py_ssize_t); return equal; diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -6,7 +6,7 @@ /* Support objects whose length is > PY_SSIZE_T_MAX. This could be sped up for small PyLongs if they fit in a Py_ssize_t. - This only matters on Win64. Though we could use PY_LONG_LONG which + This only matters on Win64. Though we could use long long which would presumably help perf. */ diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -2680,7 +2680,7 @@ va_arg(*vargs, unsigned long)); else if (longlongflag) len = sprintf(buffer, "%" PY_FORMAT_LONG_LONG "u", - va_arg(*vargs, unsigned PY_LONG_LONG)); + va_arg(*vargs, unsigned long long)); else if (size_tflag) len = sprintf(buffer, "%" PY_FORMAT_SIZE_T "u", va_arg(*vargs, size_t)); @@ -2697,7 +2697,7 @@ va_arg(*vargs, long)); else if (longlongflag) len = sprintf(buffer, "%" PY_FORMAT_LONG_LONG "i", - va_arg(*vargs, PY_LONG_LONG)); + va_arg(*vargs, long long)); else if (size_tflag) len = sprintf(buffer, "%" PY_FORMAT_SIZE_T "i", va_arg(*vargs, Py_ssize_t)); diff --git a/PC/pyconfig.h b/PC/pyconfig.h --- a/PC/pyconfig.h +++ b/PC/pyconfig.h @@ -265,7 +265,6 @@ #endif /* 64 bit ints are usually spelt __int64 unless compiler has overridden */ -#define HAVE_LONG_LONG 1 #ifndef PY_LONG_LONG # define PY_LONG_LONG __int64 # define PY_LLONG_MAX _I64_MAX @@ -379,7 +378,7 @@ #ifndef PY_UINT64_T #if SIZEOF_LONG_LONG == 8 #define HAVE_UINT64_T 1 -#define PY_UINT64_T unsigned PY_LONG_LONG +#define PY_UINT64_T unsigned long long #endif #endif @@ -396,7 +395,7 @@ #ifndef PY_INT64_T #if SIZEOF_LONG_LONG == 8 #define HAVE_INT64_T 1 -#define PY_INT64_T PY_LONG_LONG +#define PY_INT64_T long long #endif #endif diff --git a/Python/condvar.h b/Python/condvar.h --- a/Python/condvar.h +++ b/Python/condvar.h @@ -89,7 +89,7 @@ /* return 0 for success, 1 on timeout, -1 on error */ Py_LOCAL_INLINE(int) -PyCOND_TIMEDWAIT(PyCOND_T *cond, PyMUTEX_T *mut, PY_LONG_LONG us) +PyCOND_TIMEDWAIT(PyCOND_T *cond, PyMUTEX_T *mut, long long us) { int r; struct timespec ts; @@ -270,7 +270,7 @@ } Py_LOCAL_INLINE(int) -PyCOND_TIMEDWAIT(PyCOND_T *cv, PyMUTEX_T *cs, PY_LONG_LONG us) +PyCOND_TIMEDWAIT(PyCOND_T *cv, PyMUTEX_T *cs, long long us) { return _PyCOND_WAIT_MS(cv, cs, (DWORD)(us/1000)); } @@ -363,7 +363,7 @@ * 2 to indicate that we don't know. */ Py_LOCAL_INLINE(int) -PyCOND_TIMEDWAIT(PyCOND_T *cv, PyMUTEX_T *cs, PY_LONG_LONG us) +PyCOND_TIMEDWAIT(PyCOND_T *cv, PyMUTEX_T *cs, long long us) { return SleepConditionVariableSRW(cv, cs, (DWORD)(us/1000), 0) ? 2 : -1; } diff --git a/Python/getargs.c b/Python/getargs.c --- a/Python/getargs.c +++ b/Python/getargs.c @@ -769,13 +769,13 @@ break; } - case 'L': {/* PY_LONG_LONG */ - PY_LONG_LONG *p = va_arg( *p_va, PY_LONG_LONG * ); - PY_LONG_LONG ival; + case 'L': {/* long long */ + long long *p = va_arg( *p_va, long long * ); + long long ival; if (float_argument_error(arg)) RETURN_ERR_OCCURRED; ival = PyLong_AsLongLong(arg); - if (ival == (PY_LONG_LONG)-1 && PyErr_Occurred()) + if (ival == (long long)-1 && PyErr_Occurred()) RETURN_ERR_OCCURRED; else *p = ival; @@ -783,8 +783,8 @@ } case 'K': { /* long long sized bitfield */ - unsigned PY_LONG_LONG *p = va_arg(*p_va, unsigned PY_LONG_LONG *); - unsigned PY_LONG_LONG ival; + unsigned long long *p = va_arg(*p_va, unsigned long long *); + unsigned long long ival; if (PyLong_Check(arg)) ival = PyLong_AsUnsignedLongLongMask(arg); else @@ -2086,8 +2086,8 @@ case 'I': /* int sized bitfield */ case 'l': /* long int */ case 'k': /* long int sized bitfield */ - case 'L': /* PY_LONG_LONG */ - case 'K': /* PY_LONG_LONG sized bitfield */ + case 'L': /* long long */ + case 'K': /* long long sized bitfield */ case 'n': /* Py_ssize_t */ case 'f': /* float */ case 'd': /* double */ diff --git a/Python/modsupport.c b/Python/modsupport.c --- a/Python/modsupport.c +++ b/Python/modsupport.c @@ -261,10 +261,10 @@ } case 'L': - return PyLong_FromLongLong((PY_LONG_LONG)va_arg(*p_va, PY_LONG_LONG)); + return PyLong_FromLongLong((long long)va_arg(*p_va, long long)); case 'K': - return PyLong_FromUnsignedLongLong((PY_LONG_LONG)va_arg(*p_va, unsigned PY_LONG_LONG)); + return PyLong_FromUnsignedLongLong((long long)va_arg(*p_va, unsigned long long)); case 'u': { diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -39,7 +39,7 @@ _PyLong_AsTime_t(PyObject *obj) { #if SIZEOF_TIME_T == SIZEOF_LONG_LONG - PY_LONG_LONG val; + long long val; val = PyLong_AsLongLong(obj); #else long val; @@ -58,7 +58,7 @@ _PyLong_FromTime_t(time_t t) { #if SIZEOF_TIME_T == SIZEOF_LONG_LONG - return PyLong_FromLongLong((PY_LONG_LONG)t); + return PyLong_FromLongLong((long long)t); #else Py_BUILD_ASSERT(sizeof(time_t) <= sizeof(long)); return PyLong_FromLong((long)t); @@ -218,11 +218,11 @@ } _PyTime_t -_PyTime_FromNanoseconds(PY_LONG_LONG ns) +_PyTime_FromNanoseconds(long long ns) { _PyTime_t t; - Py_BUILD_ASSERT(sizeof(PY_LONG_LONG) <= sizeof(_PyTime_t)); - t = Py_SAFE_DOWNCAST(ns, PY_LONG_LONG, _PyTime_t); + Py_BUILD_ASSERT(sizeof(long long) <= sizeof(_PyTime_t)); + t = Py_SAFE_DOWNCAST(ns, long long, _PyTime_t); return t; } @@ -304,8 +304,8 @@ return _PyTime_FromFloatObject(t, d, round, unit_to_ns); } else { - PY_LONG_LONG sec; - Py_BUILD_ASSERT(sizeof(PY_LONG_LONG) <= sizeof(_PyTime_t)); + long long sec; + Py_BUILD_ASSERT(sizeof(long long) <= sizeof(_PyTime_t)); sec = PyLong_AsLongLong(obj); if (sec == -1 && PyErr_Occurred()) { @@ -358,8 +358,8 @@ PyObject * _PyTime_AsNanosecondsObject(_PyTime_t t) { - Py_BUILD_ASSERT(sizeof(PY_LONG_LONG) >= sizeof(_PyTime_t)); - return PyLong_FromLongLong((PY_LONG_LONG)t); + Py_BUILD_ASSERT(sizeof(long long) >= sizeof(_PyTime_t)); + return PyLong_FromLongLong((long long)t); } static _PyTime_t diff --git a/Python/structmember.c b/Python/structmember.c --- a/Python/structmember.c +++ b/Python/structmember.c @@ -75,10 +75,10 @@ Py_XINCREF(v); break; case T_LONGLONG: - v = PyLong_FromLongLong(*(PY_LONG_LONG *)addr); + v = PyLong_FromLongLong(*(long long *)addr); break; case T_ULONGLONG: - v = PyLong_FromUnsignedLongLong(*(unsigned PY_LONG_LONG *)addr); + v = PyLong_FromUnsignedLongLong(*(unsigned long long *)addr); break; case T_NONE: v = Py_None; @@ -265,21 +265,21 @@ PyErr_SetString(PyExc_TypeError, "readonly attribute"); return -1; case T_LONGLONG:{ - PY_LONG_LONG value; - *(PY_LONG_LONG*)addr = value = PyLong_AsLongLong(v); + long long value; + *(long long*)addr = value = PyLong_AsLongLong(v); if ((value == -1) && PyErr_Occurred()) return -1; break; } case T_ULONGLONG:{ - unsigned PY_LONG_LONG value; + unsigned long long value; /* ??? PyLong_AsLongLong accepts an int, but PyLong_AsUnsignedLongLong doesn't ??? */ if (PyLong_Check(v)) - *(unsigned PY_LONG_LONG*)addr = value = PyLong_AsUnsignedLongLong(v); + *(unsigned long long*)addr = value = PyLong_AsUnsignedLongLong(v); else - *(unsigned PY_LONG_LONG*)addr = value = PyLong_AsLong(v); - if ((value == (unsigned PY_LONG_LONG)-1) && PyErr_Occurred()) + *(unsigned long long*)addr = value = PyLong_AsLong(v); + if ((value == (unsigned long long)-1) && PyErr_Occurred()) return -1; break; } diff --git a/Python/thread_nt.h b/Python/thread_nt.h --- a/Python/thread_nt.h +++ b/Python/thread_nt.h @@ -77,7 +77,7 @@ /* wait at least until the target */ DWORD now, target = GetTickCount() + milliseconds; while (mutex->locked) { - if (PyCOND_TIMEDWAIT(&mutex->cv, &mutex->cs, (PY_LONG_LONG)milliseconds*1000) < 0) { + if (PyCOND_TIMEDWAIT(&mutex->cv, &mutex->cs, (long long)milliseconds*1000) < 0) { result = WAIT_FAILED; break; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 13:49:39 2016 From: python-checkins at python.org (steve.dower) Date: Tue, 06 Sep 2016 17:49:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Prevents_unnecessary_help_?= =?utf-8?q?text_appearing_in_doc_build=2E?= Message-ID: <20160906174938.104224.58365.D288CEB1@psf.io> https://hg.python.org/cpython/rev/f824ccaecd8b changeset: 103122:f824ccaecd8b user: Steve Dower date: Tue Sep 06 10:49:17 2016 -0700 summary: Prevents unnecessary help text appearing in doc build. files: Doc/make.bat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/make.bat b/Doc/make.bat --- a/Doc/make.bat +++ b/Doc/make.bat @@ -40,7 +40,7 @@ goto end ) -%SPHINXBUILD% 2> nul +%SPHINXBUILD% >nul 2> nul if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 13:52:25 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 17:52:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_some_silly_defined?= =?utf-8?q?=28=29_tests?= Message-ID: <20160906175209.23567.96991.98F954B0@psf.io> https://hg.python.org/cpython/rev/4c54d7ab84fa changeset: 103123:4c54d7ab84fa user: Benjamin Peterson date: Tue Sep 06 10:51:19 2016 -0700 summary: remove some silly defined() tests files: Objects/longobject.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -4650,7 +4650,7 @@ /* a fits into a long, so b must too */ x = PyLong_AsLong((PyObject *)a); y = PyLong_AsLong((PyObject *)b); -#elif defined(long long) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT +#elif PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT x = PyLong_AsLongLong((PyObject *)a); y = PyLong_AsLongLong((PyObject *)b); #else @@ -4669,7 +4669,7 @@ } #if LONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT return PyLong_FromLong(x); -#elif defined(long long) && PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT +#elif PY_LLONG_MAX >> PyLong_SHIFT >> PyLong_SHIFT return PyLong_FromLongLong(x); #else # error "_PyLong_GCD" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 14:12:03 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 18:12:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issues_=2327850_and_=2327766=3A_Remove_3DES_from_ssl_def?= =?utf-8?q?ault_cipher_list_and_add?= Message-ID: <20160906181202.104228.76207.8D87A184@psf.io> https://hg.python.org/cpython/rev/f586742e56cb changeset: 103126:f586742e56cb parent: 103123:4c54d7ab84fa parent: 103124:d2111109fd77 user: Christian Heimes date: Tue Sep 06 20:07:58 2016 +0200 summary: Issues #27850 and #27766: Remove 3DES from ssl default cipher list and add ChaCha20 Poly1305. files: Doc/library/ssl.rst | 6 +++++ Lib/ssl.py | 36 +++++++++++++++++++------------- Misc/NEWS | 6 +++++ 3 files changed, 33 insertions(+), 15 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -285,6 +285,12 @@ RC4 was dropped from the default cipher string. + .. versionchanged:: 3.6 + + ChaCha20/Poly1305 was added to the default cipher string. + + 3DES was dropped from the default cipher string. + Random generation ^^^^^^^^^^^^^^^^^ diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -157,36 +157,42 @@ else: CHANNEL_BINDING_TYPES = [] + # Disable weak or insecure ciphers by default # (OpenSSL's default setting is 'DEFAULT:!aNULL:!eNULL') # Enable a better set of ciphers by default # This list has been explicitly chosen to: # * Prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE) # * Prefer ECDHE over DHE for better performance -# * Prefer any AES-GCM over any AES-CBC for better performance and security +# * Prefer AEAD over CBC for better performance and security +# * Prefer AES-GCM over ChaCha20 because most platforms have AES-NI +# (ChaCha20 needs OpenSSL 1.1.0 or patched 1.0.2) +# * Prefer any AES-GCM and ChaCha20 over any AES-CBC for better +# performance and security # * Then Use HIGH cipher suites as a fallback -# * Then Use 3DES as fallback which is secure but slow -# * Disable NULL authentication, NULL encryption, and MD5 MACs for security -# reasons +# * Disable NULL authentication, NULL encryption, 3DES and MD5 MACs +# for security reasons _DEFAULT_CIPHERS = ( - 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:' - 'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:' - '!eNULL:!MD5' -) + 'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:' + 'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:' + '!aNULL:!eNULL:!MD5:!3DES' + ) # Restricted and more secure ciphers for the server side # This list has been explicitly chosen to: # * Prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE) # * Prefer ECDHE over DHE for better performance -# * Prefer any AES-GCM over any AES-CBC for better performance and security +# * Prefer AEAD over CBC for better performance and security +# * Prefer AES-GCM over ChaCha20 because most platforms have AES-NI +# * Prefer any AES-GCM and ChaCha20 over any AES-CBC for better +# performance and security # * Then Use HIGH cipher suites as a fallback -# * Then Use 3DES as fallback which is secure but slow -# * Disable NULL authentication, NULL encryption, MD5 MACs, DSS, and RC4 for -# security reasons +# * Disable NULL authentication, NULL encryption, MD5 MACs, DSS, RC4, and +# 3DES for security reasons _RESTRICTED_SERVER_CIPHERS = ( - 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:' - 'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:' - '!eNULL:!MD5:!DSS:!RC4' + 'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:' + 'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:' + '!aNULL:!eNULL:!MD5:!DSS:!RC4:!3DES' ) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -85,6 +85,12 @@ Library ------- +- Issue #27850: Remove 3DES from ssl module's default cipher list to counter + measure sweet32 attack (CVE-2016-2183). + +- Issue #27766: Add ChaCha20 Poly1305 to ssl module's default ciper list. + (Required OpenSSL 1.1.0 or LibreSSL). + - Issue #25387: Check return value of winsound.MessageBeep. - Issue #27866: Add SSLContext.get_ciphers() method to get a list of all -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 14:12:03 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 18:12:03 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWVzICMyNzg1?= =?utf-8?q?0_and_=2327766=3A_Remove_3DES_from_ssl_default_cipher_list_and_?= =?utf-8?q?add?= Message-ID: <20160906181202.104354.92131.05086B85@psf.io> https://hg.python.org/cpython/rev/d2111109fd77 changeset: 103124:d2111109fd77 branch: 3.5 parent: 103119:2c4359ff4d6d user: Christian Heimes date: Tue Sep 06 20:06:47 2016 +0200 summary: Issues #27850 and #27766: Remove 3DES from ssl default cipher list and add ChaCha20 Poly1305. files: Doc/library/ssl.rst | 6 +++++ Lib/ssl.py | 36 +++++++++++++++++++------------- Misc/NEWS | 6 +++++ 3 files changed, 33 insertions(+), 15 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -279,6 +279,12 @@ RC4 was dropped from the default cipher string. + .. versionchanged:: 3.5.3 + + ChaCha20/Poly1305 was added to the default cipher string. + + 3DES was dropped from the default cipher string. + Random generation ^^^^^^^^^^^^^^^^^ diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -157,36 +157,42 @@ else: CHANNEL_BINDING_TYPES = [] + # Disable weak or insecure ciphers by default # (OpenSSL's default setting is 'DEFAULT:!aNULL:!eNULL') # Enable a better set of ciphers by default # This list has been explicitly chosen to: # * Prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE) # * Prefer ECDHE over DHE for better performance -# * Prefer any AES-GCM over any AES-CBC for better performance and security +# * Prefer AEAD over CBC for better performance and security +# * Prefer AES-GCM over ChaCha20 because most platforms have AES-NI +# (ChaCha20 needs OpenSSL 1.1.0 or patched 1.0.2) +# * Prefer any AES-GCM and ChaCha20 over any AES-CBC for better +# performance and security # * Then Use HIGH cipher suites as a fallback -# * Then Use 3DES as fallback which is secure but slow -# * Disable NULL authentication, NULL encryption, and MD5 MACs for security -# reasons +# * Disable NULL authentication, NULL encryption, 3DES and MD5 MACs +# for security reasons _DEFAULT_CIPHERS = ( - 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:' - 'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:' - '!eNULL:!MD5' -) + 'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:' + 'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:' + '!aNULL:!eNULL:!MD5:!3DES' + ) # Restricted and more secure ciphers for the server side # This list has been explicitly chosen to: # * Prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE) # * Prefer ECDHE over DHE for better performance -# * Prefer any AES-GCM over any AES-CBC for better performance and security +# * Prefer AEAD over CBC for better performance and security +# * Prefer AES-GCM over ChaCha20 because most platforms have AES-NI +# * Prefer any AES-GCM and ChaCha20 over any AES-CBC for better +# performance and security # * Then Use HIGH cipher suites as a fallback -# * Then Use 3DES as fallback which is secure but slow -# * Disable NULL authentication, NULL encryption, MD5 MACs, DSS, and RC4 for -# security reasons +# * Disable NULL authentication, NULL encryption, MD5 MACs, DSS, RC4, and +# 3DES for security reasons _RESTRICTED_SERVER_CIPHERS = ( - 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:' - 'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:' - '!eNULL:!MD5:!DSS:!RC4' + 'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:' + 'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:' + '!aNULL:!eNULL:!MD5:!DSS:!RC4:!3DES' ) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,12 @@ Library ------- +- Issue #27850: Remove 3DES from ssl module's default cipher list to counter + measure sweet32 attack (CVE-2016-2183). + +- Issue #27766: Add ChaCha20 Poly1305 to ssl module's default ciper list. + (Required OpenSSL 1.1.0 or LibreSSL). + - Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. - Remove support for passing a file descriptor to os.access. It never worked but -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 14:12:09 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 18:12:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWVzICMyNzg1?= =?utf-8?q?0_and_=2327766=3A_Remove_3DES_from_ssl_default_cipher_list_and_?= =?utf-8?q?add?= Message-ID: <20160906181202.8354.8564.B366E98F@psf.io> https://hg.python.org/cpython/rev/6f4f19217d9b changeset: 103125:6f4f19217d9b branch: 2.7 parent: 103118:5e75bf8e5526 user: Christian Heimes date: Tue Sep 06 20:06:47 2016 +0200 summary: Issues #27850 and #27766: Remove 3DES from ssl default cipher list and add ChaCha20 Poly1305. files: Doc/library/ssl.rst | 6 +++++ Lib/ssl.py | 36 +++++++++++++++++++------------- Misc/NEWS | 6 +++++ 3 files changed, 33 insertions(+), 15 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -280,6 +280,12 @@ RC4 was dropped from the default cipher string. + .. versionchanged:: 2.7.13 + + ChaCha20/Poly1305 was added to the default cipher string. + + 3DES was dropped from the default cipher string. + .. function:: _https_verify_certificates(enable=True) Specifies whether or not server certificates are verified when creating diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -152,36 +152,42 @@ else: CHANNEL_BINDING_TYPES = [] + # Disable weak or insecure ciphers by default # (OpenSSL's default setting is 'DEFAULT:!aNULL:!eNULL') # Enable a better set of ciphers by default # This list has been explicitly chosen to: # * Prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE) # * Prefer ECDHE over DHE for better performance -# * Prefer any AES-GCM over any AES-CBC for better performance and security +# * Prefer AEAD over CBC for better performance and security +# * Prefer AES-GCM over ChaCha20 because most platforms have AES-NI +# (ChaCha20 needs OpenSSL 1.1.0 or patched 1.0.2) +# * Prefer any AES-GCM and ChaCha20 over any AES-CBC for better +# performance and security # * Then Use HIGH cipher suites as a fallback -# * Then Use 3DES as fallback which is secure but slow -# * Disable NULL authentication, NULL encryption, and MD5 MACs for security -# reasons +# * Disable NULL authentication, NULL encryption, 3DES and MD5 MACs +# for security reasons _DEFAULT_CIPHERS = ( - 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:' - 'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:' - '!eNULL:!MD5' -) + 'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:' + 'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:' + '!aNULL:!eNULL:!MD5:!3DES' + ) # Restricted and more secure ciphers for the server side # This list has been explicitly chosen to: # * Prefer cipher suites that offer perfect forward secrecy (DHE/ECDHE) # * Prefer ECDHE over DHE for better performance -# * Prefer any AES-GCM over any AES-CBC for better performance and security +# * Prefer AEAD over CBC for better performance and security +# * Prefer AES-GCM over ChaCha20 because most platforms have AES-NI +# * Prefer any AES-GCM and ChaCha20 over any AES-CBC for better +# performance and security # * Then Use HIGH cipher suites as a fallback -# * Then Use 3DES as fallback which is secure but slow -# * Disable NULL authentication, NULL encryption, MD5 MACs, DSS, and RC4 for -# security reasons +# * Disable NULL authentication, NULL encryption, MD5 MACs, DSS, RC4, and +# 3DES for security reasons _RESTRICTED_SERVER_CIPHERS = ( - 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:' - 'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:' - '!eNULL:!MD5:!DSS:!RC4' + 'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:' + 'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:' + '!aNULL:!eNULL:!MD5:!DSS:!RC4:!3DES' ) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -36,6 +36,12 @@ Library ------- +- Issue #27850: Remove 3DES from ssl module's default cipher list to counter + measure sweet32 attack (CVE-2016-2183). + +- Issue #27766: Add ChaCha20 Poly1305 to ssl module's default ciper list. + (Required OpenSSL 1.1.0 or LibreSSL). + - Issue #26470: Port ssl and hashlib module to OpenSSL 1.1.0. - Issue #27944: Fix some memory-corruption bugs in the log reading code of the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 14:23:55 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 18:23:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327928=3A_Add_scry?= =?utf-8?q?pt_=28password-based_key_derivation_function=29_to_hashlib?= Message-ID: <20160906182354.22113.19412.D97116C3@psf.io> https://hg.python.org/cpython/rev/d926fa1a833c changeset: 103127:d926fa1a833c user: Christian Heimes date: Tue Sep 06 20:22:28 2016 +0200 summary: Issue #27928: Add scrypt (password-based key derivation function) to hashlib module (requires OpenSSL 1.1.0). files: Doc/library/hashlib.rst | 17 ++ Lib/hashlib.py | 6 + Lib/test/test_hashlib.py | 47 +++++++ Misc/NEWS | 3 + Modules/_hashopenssl.c | 129 ++++++++++++++++++++ Modules/clinic/_hashopenssl.c.h | 60 +++++++++ 6 files changed, 262 insertions(+), 0 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -225,6 +225,23 @@ Python implementation uses an inline version of :mod:`hmac`. It is about three times slower and doesn't release the GIL. +.. function:: scrypt(password, *, salt, n, r, p, maxmem=0, dklen=64) + + The function provides scrypt password-based key derivation function as + defined in :rfc:`7914`. + + *password* and *salt* must be bytes-like objects. Applications and + libraries should limit *password* to a sensible length (e.g. 1024). *salt* + should be about 16 or more bytes from a proper source, e.g. :func:`os.urandom`. + + *n* is the CPU/Memory cost factor, *r* the block size, *p* parallelization + factor and *maxmem* limits memory (OpenSSL 1.1.0 defaults to 32 MB). + *dklen* is the length of the derived key. + + Availability: OpenSSL 1.1+ + + .. versionadded:: 3.6 + .. seealso:: diff --git a/Lib/hashlib.py b/Lib/hashlib.py --- a/Lib/hashlib.py +++ b/Lib/hashlib.py @@ -202,6 +202,12 @@ return dkey[:dklen] +try: + # OpenSSL's scrypt requires OpenSSL 1.1+ + from _hashlib import scrypt +except ImportError: + pass + for __func_name in __always_supported: # try them all, some may not work due to the OpenSSL diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py --- a/Lib/test/test_hashlib.py +++ b/Lib/test/test_hashlib.py @@ -7,6 +7,7 @@ # import array +from binascii import unhexlify import hashlib import itertools import os @@ -447,6 +448,12 @@ (b'pass\0word', b'sa\0lt', 4096, 16), ] + scrypt_test_vectors = [ + (b'', b'', 16, 1, 1, unhexlify('77d6576238657b203b19ca42c18a0497f16b4844e3074ae8dfdffa3fede21442fcd0069ded0948f8326a753a0fc81f17e8d3e0fb2e0d3628cf35e20c38d18906')), + (b'password', b'NaCl', 1024, 8, 16, unhexlify('fdbabe1c9d3472007856e7190d01e9fe7c6ad7cbc8237830e77376634b3731622eaf30d92e22a3886ff109279d9830dac727afb94a83ee6d8360cbdfa2cc0640')), + (b'pleaseletmein', b'SodiumChloride', 16384, 8, 1, unhexlify('7023bdcb3afd7348461c06cd81fd38ebfda8fbba904f8e3ea9b543f6545da1f2d5432955613f0fcf62d49705242a9af9e61e85dc0d651e40dfcf017b45575887')), + ] + pbkdf2_results = { "sha1": [ # official test vectors from RFC 6070 @@ -526,5 +533,45 @@ self._test_pbkdf2_hmac(c_hashlib.pbkdf2_hmac) + @unittest.skipUnless(hasattr(c_hashlib, 'scrypt'), + ' test requires OpenSSL > 1.1') + def test_scrypt(self): + for password, salt, n, r, p, expected in self.scrypt_test_vectors: + result = hashlib.scrypt(password, salt=salt, n=n, r=r, p=p) + self.assertEqual(result, expected) + + # this values should work + hashlib.scrypt(b'password', salt=b'salt', n=2, r=8, p=1) + # password and salt must be bytes-like + with self.assertRaises(TypeError): + hashlib.scrypt('password', salt=b'salt', n=2, r=8, p=1) + with self.assertRaises(TypeError): + hashlib.scrypt(b'password', salt='salt', n=2, r=8, p=1) + # require keyword args + with self.assertRaises(TypeError): + hashlib.scrypt(b'password') + with self.assertRaises(TypeError): + hashlib.scrypt(b'password', b'salt') + with self.assertRaises(TypeError): + hashlib.scrypt(b'password', 2, 8, 1, salt=b'salt') + for n in [-1, 0, 1, None]: + with self.assertRaises((ValueError, OverflowError, TypeError)): + hashlib.scrypt(b'password', salt=b'salt', n=n, r=8, p=1) + for r in [-1, 0, None]: + with self.assertRaises((ValueError, OverflowError, TypeError)): + hashlib.scrypt(b'password', salt=b'salt', n=2, r=r, p=1) + for p in [-1, 0, None]: + with self.assertRaises((ValueError, OverflowError, TypeError)): + hashlib.scrypt(b'password', salt=b'salt', n=2, r=8, p=p) + for maxmem in [-1, None]: + with self.assertRaises((ValueError, OverflowError, TypeError)): + hashlib.scrypt(b'password', salt=b'salt', n=2, r=8, p=1, + maxmem=maxmem) + for dklen in [-1, None]: + with self.assertRaises((ValueError, OverflowError, TypeError)): + hashlib.scrypt(b'password', salt=b'salt', n=2, r=8, p=1, + dklen=dklen) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -85,6 +85,9 @@ Library ------- +- Issue #27928: Add scrypt (password-based key derivation function) to + hashlib module (requires OpenSSL 1.1.0). + - Issue #27850: Remove 3DES from ssl module's default cipher list to counter measure sweet32 attack (CVE-2016-2183). diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -25,6 +25,12 @@ #include #include "openssl/err.h" +#include "clinic/_hashopenssl.c.h" +/*[clinic input] +module _hashlib +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=c2b4ff081bac4be1]*/ + #define MUNCH_SIZE INT_MAX #ifndef HASH_OBJ_CONSTRUCTOR @@ -713,6 +719,128 @@ #endif +#if OPENSSL_VERSION_NUMBER > 0x10100000L && !defined(OPENSSL_NO_SCRYPT) && !defined(LIBRESSL_VERSION_NUMBER) +#define PY_SCRYPT 1 + +/*[clinic input] +_hashlib.scrypt + + password: Py_buffer + * + salt: Py_buffer = None + n as n_obj: object(subclass_of='&PyLong_Type') = None + r as r_obj: object(subclass_of='&PyLong_Type') = None + p as p_obj: object(subclass_of='&PyLong_Type') = None + maxmem: long = 0 + dklen: long = 64 + + +scrypt password-based key derivation function. +[clinic start generated code]*/ + +static PyObject * +_hashlib_scrypt_impl(PyObject *module, Py_buffer *password, Py_buffer *salt, + PyObject *n_obj, PyObject *r_obj, PyObject *p_obj, + long maxmem, long dklen) +/*[clinic end generated code: output=14849e2aa2b7b46c input=48a7d63bf3f75c42]*/ +{ + PyObject *key_obj = NULL; + char *key; + int retval; + unsigned long n, r, p; + + if (password->len > INT_MAX) { + PyErr_SetString(PyExc_OverflowError, + "password is too long."); + return NULL; + } + + if (salt->buf == NULL) { + PyErr_SetString(PyExc_TypeError, + "salt is required"); + return NULL; + } + if (salt->len > INT_MAX) { + PyErr_SetString(PyExc_OverflowError, + "salt is too long."); + return NULL; + } + + n = PyLong_AsUnsignedLong(n_obj); + if (n == (unsigned long) -1 && PyErr_Occurred()) { + PyErr_SetString(PyExc_TypeError, + "n is required and must be an unsigned int"); + return NULL; + } + if (n < 2 || n & (n - 1)) { + PyErr_SetString(PyExc_ValueError, + "n must be a power of 2."); + return NULL; + } + + r = PyLong_AsUnsignedLong(r_obj); + if (r == (unsigned long) -1 && PyErr_Occurred()) { + PyErr_SetString(PyExc_TypeError, + "r is required and must be an unsigned int"); + return NULL; + } + + p = PyLong_AsUnsignedLong(p_obj); + if (p == (unsigned long) -1 && PyErr_Occurred()) { + PyErr_SetString(PyExc_TypeError, + "p is required and must be an unsigned int"); + return NULL; + } + + if (maxmem < 0 || maxmem > INT_MAX) { + /* OpenSSL 1.1.0 restricts maxmem to 32MB. It may change in the + future. The maxmem constant is private to OpenSSL. */ + PyErr_Format(PyExc_ValueError, + "maxmem must be positive and smaller than %d", + INT_MAX); + return NULL; + } + + if (dklen < 1 || dklen > INT_MAX) { + PyErr_Format(PyExc_ValueError, + "dklen must be greater than 0 and smaller than %d", + INT_MAX); + return NULL; + } + + /* let OpenSSL validate the rest */ + retval = EVP_PBE_scrypt(NULL, 0, NULL, 0, n, r, p, maxmem, NULL, 0); + if (!retval) { + /* sorry, can't do much better */ + PyErr_SetString(PyExc_ValueError, + "Invalid paramemter combination for n, r, p, maxmem."); + return NULL; + } + + key_obj = PyBytes_FromStringAndSize(NULL, dklen); + if (key_obj == NULL) { + return NULL; + } + key = PyBytes_AS_STRING(key_obj); + + Py_BEGIN_ALLOW_THREADS + retval = EVP_PBE_scrypt( + (const char*)password->buf, (size_t)password->len, + (const unsigned char *)salt->buf, (size_t)salt->len, + n, r, p, maxmem, + (unsigned char *)key, (size_t)dklen + ); + Py_END_ALLOW_THREADS + + if (!retval) { + Py_CLEAR(key_obj); + _setException(PyExc_ValueError); + return NULL; + } + return key_obj; +} +#endif + /* State for our callback function so that it can accumulate a result. */ typedef struct _internal_name_mapper_state { PyObject *set; @@ -836,6 +964,7 @@ {"pbkdf2_hmac", (PyCFunction)pbkdf2_hmac, METH_VARARGS|METH_KEYWORDS, pbkdf2_hmac__doc__}, #endif + _HASHLIB_SCRYPT_METHODDEF CONSTRUCTOR_METH_DEF(md5), CONSTRUCTOR_METH_DEF(sha1), CONSTRUCTOR_METH_DEF(sha224), diff --git a/Modules/clinic/_hashopenssl.c.h b/Modules/clinic/_hashopenssl.c.h new file mode 100644 --- /dev/null +++ b/Modules/clinic/_hashopenssl.c.h @@ -0,0 +1,60 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +#if (OPENSSL_VERSION_NUMBER > 0x10100000L && !defined(OPENSSL_NO_SCRYPT) && !defined(LIBRESSL_VERSION_NUMBER)) + +PyDoc_STRVAR(_hashlib_scrypt__doc__, +"scrypt($module, /, password, *, salt=None, n=None, r=None, p=None,\n" +" maxmem=0, dklen=64)\n" +"--\n" +"\n" +"scrypt password-based key derivation function."); + +#define _HASHLIB_SCRYPT_METHODDEF \ + {"scrypt", (PyCFunction)_hashlib_scrypt, METH_VARARGS|METH_KEYWORDS, _hashlib_scrypt__doc__}, + +static PyObject * +_hashlib_scrypt_impl(PyObject *module, Py_buffer *password, Py_buffer *salt, + PyObject *n_obj, PyObject *r_obj, PyObject *p_obj, + long maxmem, long dklen); + +static PyObject * +_hashlib_scrypt(PyObject *module, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"password", "salt", "n", "r", "p", "maxmem", "dklen", NULL}; + static _PyArg_Parser _parser = {"y*|$y*O!O!O!ll:scrypt", _keywords, 0}; + Py_buffer password = {NULL, NULL}; + Py_buffer salt = {NULL, NULL}; + PyObject *n_obj = Py_None; + PyObject *r_obj = Py_None; + PyObject *p_obj = Py_None; + long maxmem = 0; + long dklen = 64; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &password, &salt, &PyLong_Type, &n_obj, &PyLong_Type, &r_obj, &PyLong_Type, &p_obj, &maxmem, &dklen)) { + goto exit; + } + return_value = _hashlib_scrypt_impl(module, &password, &salt, n_obj, r_obj, p_obj, maxmem, dklen); + +exit: + /* Cleanup for password */ + if (password.obj) { + PyBuffer_Release(&password); + } + /* Cleanup for salt */ + if (salt.obj) { + PyBuffer_Release(&salt); + } + + return return_value; +} + +#endif /* (OPENSSL_VERSION_NUMBER > 0x10100000L && !defined(OPENSSL_NO_SCRYPT) && !defined(LIBRESSL_VERSION_NUMBER)) */ + +#ifndef _HASHLIB_SCRYPT_METHODDEF + #define _HASHLIB_SCRYPT_METHODDEF +#endif /* !defined(_HASHLIB_SCRYPT_METHODDEF) */ +/*[clinic end generated code: output=8c5386789f77430a input=a9049054013a1b77]*/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 14:58:25 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 18:58:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_require_standard_int_types?= =?utf-8?q?_to_be_defined_=28=2317884=29?= Message-ID: <20160906185824.114831.99058.E31E4E16@psf.io> https://hg.python.org/cpython/rev/ae03163b6378 changeset: 103128:ae03163b6378 user: Benjamin Peterson date: Tue Sep 06 11:58:01 2016 -0700 summary: require standard int types to be defined (#17884) files: Include/longintrepr.h | 4 - Include/pyport.h | 52 +----- Misc/NEWS | 2 + Modules/_testcapimodule.c | 8 - PC/pyconfig.h | 37 +--- Python/dtoa.c | 15 +- configure | 227 +------------------------- configure.ac | 33 +--- pyconfig.h.in | 38 ---- 9 files changed, 15 insertions(+), 401 deletions(-) diff --git a/Include/longintrepr.h b/Include/longintrepr.h --- a/Include/longintrepr.h +++ b/Include/longintrepr.h @@ -42,10 +42,6 @@ */ #if PYLONG_BITS_IN_DIGIT == 30 -#if !(defined HAVE_UINT64_T && defined HAVE_UINT32_T && \ - defined HAVE_INT64_T && defined HAVE_INT32_T) -#error "30-bit long digits requested, but the necessary types are not available on this platform" -#endif typedef PY_UINT32_T digit; typedef PY_INT32_T sdigit; /* signed variant of digit */ typedef PY_UINT64_T twodigits; diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -5,13 +5,8 @@ /* Some versions of HP-UX & Solaris need inttypes.h for int32_t, INT32_MAX, etc. */ -#ifdef HAVE_INTTYPES_H #include -#endif - -#ifdef HAVE_STDINT_H #include -#endif /************************************************************************** Symbols and macros to supply platform-independent interfaces to basic @@ -74,64 +69,19 @@ #endif /* LLONG_MAX */ #endif -/* a build with 30-bit digits for Python integers needs an exact-width - * 32-bit unsigned integer type to store those digits. (We could just use - * type 'unsigned long', but that would be wasteful on a system where longs - * are 64-bits.) On Unix systems, the autoconf macro AC_TYPE_UINT32_T defines - * uint32_t to be such a type unless stdint.h or inttypes.h defines uint32_t. - * However, it doesn't set HAVE_UINT32_T, so we do that here. - */ -#ifdef uint32_t -#define HAVE_UINT32_T 1 -#endif - -#ifdef HAVE_UINT32_T -#ifndef PY_UINT32_T #define PY_UINT32_T uint32_t -#endif -#endif - -/* Macros for a 64-bit unsigned integer type; used for type 'twodigits' in the - * integer implementation, when 30-bit digits are enabled. - */ -#ifdef uint64_t -#define HAVE_UINT64_T 1 -#endif - -#ifdef HAVE_UINT64_T -#ifndef PY_UINT64_T #define PY_UINT64_T uint64_t -#endif -#endif /* Signed variants of the above */ -#ifdef int32_t -#define HAVE_INT32_T 1 -#endif - -#ifdef HAVE_INT32_T -#ifndef PY_INT32_T #define PY_INT32_T int32_t -#endif -#endif - -#ifdef int64_t -#define HAVE_INT64_T 1 -#endif - -#ifdef HAVE_INT64_T -#ifndef PY_INT64_T #define PY_INT64_T int64_t -#endif -#endif /* If PYLONG_BITS_IN_DIGIT is not defined then we'll use 30-bit digits if all the necessary integer types are available, and we're on a 64-bit platform (as determined by SIZEOF_VOID_P); otherwise we use 15-bit digits. */ #ifndef PYLONG_BITS_IN_DIGIT -#if (defined HAVE_UINT64_T && defined HAVE_INT64_T && \ - defined HAVE_UINT32_T && defined HAVE_INT32_T && SIZEOF_VOID_P >= 8) +#if SIZEOF_VOID_P >= 8 #define PYLONG_BITS_IN_DIGIT 30 #else #define PYLONG_BITS_IN_DIGIT 15 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #17884: Python now requires systems with inttypes.h and stdint.h + - Issue #27961?: Require platforms to support ``long long``. Python hasn't compiled without ``long long`` for years, so this is basically a formality. diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -98,22 +98,14 @@ CHECK_SIGNNESS(Py_UCS1, 0); CHECK_SIGNNESS(Py_UCS2, 0); CHECK_SIGNNESS(Py_UCS4, 0); -#ifdef HAVE_INT32_T CHECK_SIZEOF(PY_INT32_T, 4); CHECK_SIGNNESS(PY_INT32_T, 1); -#endif -#ifdef HAVE_UINT32_T CHECK_SIZEOF(PY_UINT32_T, 4); CHECK_SIGNNESS(PY_UINT32_T, 0); -#endif -#ifdef HAVE_INT64_T CHECK_SIZEOF(PY_INT64_T, 8); CHECK_SIGNNESS(PY_INT64_T, 1); -#endif -#ifdef HAVE_UINT64_T CHECK_SIZEOF(PY_UINT64_T, 8); CHECK_SIGNNESS(PY_UINT64_T, 0); -#endif /* pointer/size types */ CHECK_SIZEOF(size_t, sizeof(void *)); diff --git a/PC/pyconfig.h b/PC/pyconfig.h --- a/PC/pyconfig.h +++ b/PC/pyconfig.h @@ -365,39 +365,10 @@ /* define signed and unsigned exact-width 32-bit and 64-bit types, used in the implementation of Python integers. */ -#ifndef PY_UINT32_T -#if SIZEOF_INT == 4 -#define HAVE_UINT32_T 1 -#define PY_UINT32_T unsigned int -#elif SIZEOF_LONG == 4 -#define HAVE_UINT32_T 1 -#define PY_UINT32_T unsigned long -#endif -#endif - -#ifndef PY_UINT64_T -#if SIZEOF_LONG_LONG == 8 -#define HAVE_UINT64_T 1 -#define PY_UINT64_T unsigned long long -#endif -#endif - -#ifndef PY_INT32_T -#if SIZEOF_INT == 4 -#define HAVE_INT32_T 1 -#define PY_INT32_T int -#elif SIZEOF_LONG == 4 -#define HAVE_INT32_T 1 -#define PY_INT32_T long -#endif -#endif - -#ifndef PY_INT64_T -#if SIZEOF_LONG_LONG == 8 -#define HAVE_INT64_T 1 -#define PY_INT64_T long long -#endif -#endif +#define PY_UINT32_T uint32_t +#define PY_UINT64_T uint64_t +#define PY_INT32_T int32_t +#define PY_INT64_T int64_t /* Fairly standard from here! */ diff --git a/Python/dtoa.c b/Python/dtoa.c --- a/Python/dtoa.c +++ b/Python/dtoa.c @@ -151,18 +151,9 @@ #endif -#if defined(HAVE_UINT32_T) && defined(HAVE_INT32_T) -typedef PY_UINT32_T ULong; -typedef PY_INT32_T Long; -#else -#error "Failed to find an exact-width 32-bit integer type" -#endif - -#if defined(HAVE_UINT64_T) -#define ULLong PY_UINT64_T -#else -#undef ULLong -#endif +typedef uint32_t ULong; +typedef int32_t Long; +typedef uint64_t ULLong; #undef DEBUG #ifdef Py_DEBUG diff --git a/configure b/configure --- a/configure +++ b/configure @@ -1974,136 +1974,6 @@ } # ac_fn_c_check_type -# ac_fn_c_find_uintX_t LINENO BITS VAR -# ------------------------------------ -# Finds an unsigned integer type with width BITS, setting cache variable VAR -# accordingly. -ac_fn_c_find_uintX_t () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for uint$2_t" >&5 -$as_echo_n "checking for uint$2_t... " >&6; } -if eval \${$3+:} false; then : - $as_echo_n "(cached) " >&6 -else - eval "$3=no" - # Order is important - never check a type that is potentially smaller - # than half of the expected target width. - for ac_type in uint$2_t 'unsigned int' 'unsigned long int' \ - 'unsigned long long int' 'unsigned short int' 'unsigned char'; do - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -$ac_includes_default -int -main () -{ -static int test_array [1 - 2 * !((($ac_type) -1 >> ($2 / 2 - 1)) >> ($2 / 2 - 1) == 3)]; -test_array [0] = 0; -return test_array [0]; - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - case $ac_type in #( - uint$2_t) : - eval "$3=yes" ;; #( - *) : - eval "$3=\$ac_type" ;; -esac -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - if eval test \"x\$"$3"\" = x"no"; then : - -else - break -fi - done -fi -eval ac_res=\$$3 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 -$as_echo "$ac_res" >&6; } - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - -} # ac_fn_c_find_uintX_t - -# ac_fn_c_find_intX_t LINENO BITS VAR -# ----------------------------------- -# Finds a signed integer type with width BITS, setting cache variable VAR -# accordingly. -ac_fn_c_find_intX_t () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for int$2_t" >&5 -$as_echo_n "checking for int$2_t... " >&6; } -if eval \${$3+:} false; then : - $as_echo_n "(cached) " >&6 -else - eval "$3=no" - # Order is important - never check a type that is potentially smaller - # than half of the expected target width. - for ac_type in int$2_t 'int' 'long int' \ - 'long long int' 'short int' 'signed char'; do - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -$ac_includes_default - enum { N = $2 / 2 - 1 }; -int -main () -{ -static int test_array [1 - 2 * !(0 < ($ac_type) ((((($ac_type) 1 << N) << N) - 1) * 2 + 1))]; -test_array [0] = 0; -return test_array [0]; - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -$ac_includes_default - enum { N = $2 / 2 - 1 }; -int -main () -{ -static int test_array [1 - 2 * !(($ac_type) ((((($ac_type) 1 << N) << N) - 1) * 2 + 1) - < ($ac_type) ((((($ac_type) 1 << N) << N) - 1) * 2 + 2))]; -test_array [0] = 0; -return test_array [0]; - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - -else - case $ac_type in #( - int$2_t) : - eval "$3=yes" ;; #( - *) : - eval "$3=\$ac_type" ;; -esac -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - if eval test \"x\$"$3"\" = x"no"; then : - -else - break -fi - done -fi -eval ac_res=\$$3 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 -$as_echo "$ac_res" >&6; } - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - -} # ac_fn_c_find_intX_t - # ac_fn_c_compute_int LINENO EXPR VAR INCLUDES # -------------------------------------------- # Tries to find the compile-time value of EXPR in a program that includes @@ -7582,7 +7452,7 @@ for ac_header in asm/types.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \ -sched.h shadow.h signal.h stdint.h stropts.h termios.h \ +sched.h shadow.h signal.h stropts.h termios.h \ unistd.h utime.h \ poll.h sys/devpoll.h sys/epoll.h sys/poll.h \ sys/audioio.h sys/xattr.h sys/bsdtty.h sys/event.h sys/file.h sys/ioctl.h \ @@ -8130,95 +8000,6 @@ fi -# There are two separate checks for each of the exact-width integer types we -# need. First we check whether the type is available using the usual -# AC_CHECK_TYPE macro with the default includes (which includes -# and where available). We then also use the special type checks of -# the form AC_TYPE_UINT32_T, which in the case that uint32_t is not available -# directly, #define's uint32_t to be a suitable type. - -ac_fn_c_check_type "$LINENO" "uint32_t" "ac_cv_type_uint32_t" "$ac_includes_default" -if test "x$ac_cv_type_uint32_t" = xyes; then : - -$as_echo "#define HAVE_UINT32_T 1" >>confdefs.h - -fi - -ac_fn_c_find_uintX_t "$LINENO" "32" "ac_cv_c_uint32_t" -case $ac_cv_c_uint32_t in #( - no|yes) ;; #( - *) - -$as_echo "#define _UINT32_T 1" >>confdefs.h - - -cat >>confdefs.h <<_ACEOF -#define uint32_t $ac_cv_c_uint32_t -_ACEOF -;; - esac - - -ac_fn_c_check_type "$LINENO" "uint64_t" "ac_cv_type_uint64_t" "$ac_includes_default" -if test "x$ac_cv_type_uint64_t" = xyes; then : - -$as_echo "#define HAVE_UINT64_T 1" >>confdefs.h - -fi - -ac_fn_c_find_uintX_t "$LINENO" "64" "ac_cv_c_uint64_t" -case $ac_cv_c_uint64_t in #( - no|yes) ;; #( - *) - -$as_echo "#define _UINT64_T 1" >>confdefs.h - - -cat >>confdefs.h <<_ACEOF -#define uint64_t $ac_cv_c_uint64_t -_ACEOF -;; - esac - - -ac_fn_c_check_type "$LINENO" "int32_t" "ac_cv_type_int32_t" "$ac_includes_default" -if test "x$ac_cv_type_int32_t" = xyes; then : - -$as_echo "#define HAVE_INT32_T 1" >>confdefs.h - -fi - -ac_fn_c_find_intX_t "$LINENO" "32" "ac_cv_c_int32_t" -case $ac_cv_c_int32_t in #( - no|yes) ;; #( - *) - -cat >>confdefs.h <<_ACEOF -#define int32_t $ac_cv_c_int32_t -_ACEOF -;; -esac - - -ac_fn_c_check_type "$LINENO" "int64_t" "ac_cv_type_int64_t" "$ac_includes_default" -if test "x$ac_cv_type_int64_t" = xyes; then : - -$as_echo "#define HAVE_INT64_T 1" >>confdefs.h - -fi - -ac_fn_c_find_intX_t "$LINENO" "64" "ac_cv_c_int64_t" -case $ac_cv_c_int64_t in #( - no|yes) ;; #( - *) - -cat >>confdefs.h <<_ACEOF -#define int64_t $ac_cv_c_int64_t -_ACEOF -;; -esac - - ac_fn_c_check_type "$LINENO" "ssize_t" "ac_cv_type_ssize_t" "$ac_includes_default" if test "x$ac_cv_type_ssize_t" = xyes; then : @@ -8690,12 +8471,8 @@ fi -ac_fn_c_check_type "$LINENO" "uintptr_t" "ac_cv_type_uintptr_t" "#ifdef HAVE_STDINT_H - #include - #endif - #ifdef HAVE_INTTYPES_H +ac_fn_c_check_type "$LINENO" "uintptr_t" "ac_cv_type_uintptr_t" "#include #include - #endif " if test "x$ac_cv_type_uintptr_t" = xyes; then : diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1918,7 +1918,7 @@ AC_CHECK_HEADERS(asm/types.h conio.h direct.h dlfcn.h errno.h \ fcntl.h grp.h \ ieeefp.h io.h langinfo.h libintl.h process.h pthread.h \ -sched.h shadow.h signal.h stdint.h stropts.h termios.h \ +sched.h shadow.h signal.h stropts.h termios.h \ unistd.h utime.h \ poll.h sys/devpoll.h sys/epoll.h sys/poll.h \ sys/audioio.h sys/xattr.h sys/bsdtty.h sys/event.h sys/file.h sys/ioctl.h \ @@ -2060,29 +2060,6 @@ AC_TYPE_SIZE_T AC_TYPE_UID_T -# There are two separate checks for each of the exact-width integer types we -# need. First we check whether the type is available using the usual -# AC_CHECK_TYPE macro with the default includes (which includes -# and where available). We then also use the special type checks of -# the form AC_TYPE_UINT32_T, which in the case that uint32_t is not available -# directly, #define's uint32_t to be a suitable type. - -AC_CHECK_TYPE(uint32_t, - AC_DEFINE(HAVE_UINT32_T, 1, [Define if your compiler provides uint32_t.]),,) -AC_TYPE_UINT32_T - -AC_CHECK_TYPE(uint64_t, - AC_DEFINE(HAVE_UINT64_T, 1, [Define if your compiler provides uint64_t.]),,) -AC_TYPE_UINT64_T - -AC_CHECK_TYPE(int32_t, - AC_DEFINE(HAVE_INT32_T, 1, [Define if your compiler provides int32_t.]),,) -AC_TYPE_INT32_T - -AC_CHECK_TYPE(int64_t, - AC_DEFINE(HAVE_INT64_T, 1, [Define if your compiler provides int64_t.]),,) -AC_TYPE_INT64_T - AC_CHECK_TYPE(ssize_t, AC_DEFINE(HAVE_SSIZE_T, 1, [Define if your compiler provides ssize_t]),,) AC_CHECK_TYPE(__uint128_t, @@ -2126,12 +2103,8 @@ AC_CHECK_TYPES(uintptr_t, [AC_CHECK_SIZEOF(uintptr_t, 4)], - [], [#ifdef HAVE_STDINT_H - #include - #endif - #ifdef HAVE_INTTYPES_H - #include - #endif]) + [], [#include + #include ]) AC_CHECK_SIZEOF(off_t, [], [ #ifdef HAVE_SYS_TYPES_H diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -484,12 +484,6 @@ /* Define to 1 if you have the `initgroups' function. */ #undef HAVE_INITGROUPS -/* Define if your compiler provides int32_t. */ -#undef HAVE_INT32_T - -/* Define if your compiler provides int64_t. */ -#undef HAVE_INT64_T - /* Define to 1 if you have the header file. */ #undef HAVE_INTTYPES_H @@ -1140,12 +1134,6 @@ /* Define this if you have tcl and TCL_UTF_MAX==6 */ #undef HAVE_UCS4_TCL -/* Define if your compiler provides uint32_t. */ -#undef HAVE_UINT32_T - -/* Define if your compiler provides uint64_t. */ -#undef HAVE_UINT64_T - /* Define to 1 if the system has the type `uintptr_t'. */ #undef HAVE_UINTPTR_T @@ -1479,16 +1467,6 @@ /* Define to force use of thread-safe errno, h_errno, and other functions */ #undef _REENTRANT -/* Define for Solaris 2.5.1 so the uint32_t typedef from , - , or is not used. If the typedef were allowed, the - #define below would cause a syntax error. */ -#undef _UINT32_T - -/* Define for Solaris 2.5.1 so the uint64_t typedef from , - , or is not used. If the typedef were allowed, the - #define below would cause a syntax error. */ -#undef _UINT64_T - /* Define to the level of X/Open that your system supports */ #undef _XOPEN_SOURCE @@ -1518,14 +1496,6 @@ #undef inline #endif -/* Define to the type of a signed integer type of width exactly 32 bits if - such a type exists and the standard includes do not define it. */ -#undef int32_t - -/* Define to the type of a signed integer type of width exactly 64 bits if - such a type exists and the standard includes do not define it. */ -#undef int64_t - /* Define to `int' if does not define. */ #undef mode_t @@ -1547,14 +1517,6 @@ /* Define to `int' if doesn't define. */ #undef uid_t -/* Define to the type of an unsigned integer type of width exactly 32 bits if - such a type exists and the standard includes do not define it. */ -#undef uint32_t - -/* Define to the type of an unsigned integer type of width exactly 64 bits if - such a type exists and the standard includes do not define it. */ -#undef uint64_t - /* Define to empty if the keyword does not work. */ #undef volatile -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 14:59:55 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 18:59:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_an_unanswered_quest?= =?utf-8?q?ion?= Message-ID: <20160906185954.8354.26363.CC328397@psf.io> https://hg.python.org/cpython/rev/32fbfc5a47de changeset: 103129:32fbfc5a47de user: Benjamin Peterson date: Tue Sep 06 11:59:24 2016 -0700 summary: remove an unanswered question files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -12,7 +12,7 @@ - Issue #17884: Python now requires systems with inttypes.h and stdint.h -- Issue #27961?: Require platforms to support ``long long``. Python hasn't +- Issue #27961: Require platforms to support ``long long``. Python hasn't compiled without ``long long`` for years, so this is basically a formality. - Issue #27355: Removed support for Windows CE. It was never finished, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 15:08:16 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 06 Sep 2016 19:08:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327078=3A_Added_BU?= =?utf-8?q?ILD=5FSTRING_opcode=2E__Optimized_f-strings_evaluation=2E?= Message-ID: <20160906190813.8619.32781.351C004C@psf.io> https://hg.python.org/cpython/rev/28e280915508 changeset: 103130:28e280915508 user: Serhiy Storchaka date: Tue Sep 06 22:07:53 2016 +0300 summary: Issue #27078: Added BUILD_STRING opcode. Optimized f-strings evaluation. files: Doc/library/dis.rst | 8 + Include/opcode.h | 1 + Include/unicodeobject.h | 8 + Lib/importlib/_bootstrap_external.py | 5 +- Lib/opcode.py | 1 + Misc/NEWS | 2 + Objects/abstract.c | 20 +- Objects/unicodeobject.c | 44 +- PC/launcher.c | 2 +- Python/ceval.c | 18 + Python/compile.c | 26 +- Python/importlib.h | 409 +++++++------- Python/importlib_external.h | 208 +++--- Python/opcode_targets.h | 2 +- 14 files changed, 397 insertions(+), 357 deletions(-) diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -777,6 +777,14 @@ .. versionadded:: 3.6 +.. opcode:: BUILD_STRING (count) + + Concatenates *count* strings from the stack and pushes the resulting string + onto the stack. + + .. versionadded:: 3.6 + + .. opcode:: LOAD_ATTR (namei) Replaces TOS with ``getattr(TOS, co_names[namei])``. diff --git a/Include/opcode.h b/Include/opcode.h --- a/Include/opcode.h +++ b/Include/opcode.h @@ -123,6 +123,7 @@ #define SETUP_ASYNC_WITH 154 #define FORMAT_VALUE 155 #define BUILD_CONST_KEY_MAP 156 +#define BUILD_STRING 157 /* EXCEPT_HANDLER is a special, implicit block type which is created when entering an except handler. It is not an opcode but we define it here diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -1955,6 +1955,14 @@ PyObject *seq /* Sequence object */ ); +#ifndef Py_LIMITED_API +PyAPI_FUNC(PyObject *) _PyUnicode_JoinArray( + PyObject *separator, + PyObject **items, + Py_ssize_t seqlen + ); +#endif /* Py_LIMITED_API */ + /* Return 1 if substr matches str[start:end] at the given tail end, 0 otherwise. */ diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -232,7 +232,8 @@ # Python 3.6a1 3370 (16 bit wordcode) # Python 3.6a1 3371 (add BUILD_CONST_KEY_MAP opcode #27140) # Python 3.6a1 3372 (MAKE_FUNCTION simplification, remove MAKE_CLOSURE - #27095) +# #27095) +# Python 3.6b1 3373 (add BUILD_STRING opcode #27078) # # MAGIC must change whenever the bytecode emitted by the compiler may no # longer be understood by older implementations of the eval loop (usually @@ -241,7 +242,7 @@ # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # in PC/launcher.c must also be updated. -MAGIC_NUMBER = (3372).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3373).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Lib/opcode.py b/Lib/opcode.py --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -213,5 +213,6 @@ def_op('FORMAT_VALUE', 155) def_op('BUILD_CONST_KEY_MAP', 156) +def_op('BUILD_STRING', 157) del def_op, name_op, jrel_op, jabs_op diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #27078: Added BUILD_STRING opcode. Optimized f-strings evaluation. + - Issue #17884: Python now requires systems with inttypes.h and stdint.h - Issue #27961: Require platforms to support ``long long``. Python hasn't diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -675,13 +675,31 @@ PyObject *result = NULL; _Py_IDENTIFIER(__format__); + if (format_spec != NULL && !PyUnicode_Check(format_spec)) { + PyErr_Format(PyExc_SystemError, + "Format specifier must be a string, not %.200s", + Py_TYPE(format_spec)->tp_name); + return NULL; + } + + /* Fast path for common types. */ + if (format_spec == NULL || PyUnicode_GET_LENGTH(format_spec) == 0) { + if (PyUnicode_CheckExact(obj)) { + Py_INCREF(obj); + return obj; + } + if (PyLong_CheckExact(obj)) { + return PyObject_Str(obj); + } + } + /* If no format_spec is provided, use an empty string */ if (format_spec == NULL) { empty = PyUnicode_New(0, 0); format_spec = empty; } - /* Find the (unbound!) __format__ method (a borrowed reference) */ + /* Find the (unbound!) __format__ method */ meth = _PyObject_LookupSpecial(obj, &PyId___format__); if (meth == NULL) { if (!PyErr_Occurred()) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -9892,12 +9892,33 @@ PyObject * PyUnicode_Join(PyObject *separator, PyObject *seq) { + PyObject *res; + PyObject *fseq; + Py_ssize_t seqlen; + PyObject **items; + + fseq = PySequence_Fast(seq, "can only join an iterable"); + if (fseq == NULL) { + return NULL; + } + + /* NOTE: the following code can't call back into Python code, + * so we are sure that fseq won't be mutated. + */ + + items = PySequence_Fast_ITEMS(fseq); + seqlen = PySequence_Fast_GET_SIZE(fseq); + res = _PyUnicode_JoinArray(separator, items, seqlen); + Py_DECREF(fseq); + return res; +} + +PyObject * +_PyUnicode_JoinArray(PyObject *separator, PyObject **items, Py_ssize_t seqlen) +{ + PyObject *res = NULL; /* the result */ PyObject *sep = NULL; Py_ssize_t seplen; - PyObject *res = NULL; /* the result */ - PyObject *fseq; /* PySequence_Fast(seq) */ - Py_ssize_t seqlen; /* len(fseq) -- number of items in sequence */ - PyObject **items; PyObject *item; Py_ssize_t sz, i, res_offset; Py_UCS4 maxchar; @@ -9907,30 +9928,17 @@ PyObject *last_obj; unsigned int kind = 0; - fseq = PySequence_Fast(seq, "can only join an iterable"); - if (fseq == NULL) { - return NULL; - } - - /* NOTE: the following code can't call back into Python code, - * so we are sure that fseq won't be mutated. - */ - - seqlen = PySequence_Fast_GET_SIZE(fseq); /* If empty sequence, return u"". */ if (seqlen == 0) { - Py_DECREF(fseq); _Py_RETURN_UNICODE_EMPTY(); } /* If singleton sequence with an exact Unicode, return that. */ last_obj = NULL; - items = PySequence_Fast_ITEMS(fseq); if (seqlen == 1) { if (PyUnicode_CheckExact(items[0])) { res = items[0]; Py_INCREF(res); - Py_DECREF(fseq); return res; } seplen = 0; @@ -10065,13 +10073,11 @@ assert(res_offset == PyUnicode_GET_LENGTH(res)); } - Py_DECREF(fseq); Py_XDECREF(sep); assert(_PyUnicode_CheckConsistency(res, 1)); return res; onError: - Py_DECREF(fseq); Py_XDECREF(sep); Py_XDECREF(res); return NULL; diff --git a/PC/launcher.c b/PC/launcher.c --- a/PC/launcher.c +++ b/PC/launcher.c @@ -1089,7 +1089,7 @@ { 3190, 3230, L"3.3" }, { 3250, 3310, L"3.4" }, { 3320, 3351, L"3.5" }, - { 3360, 3371, L"3.6" }, + { 3360, 3373, L"3.6" }, { 0 } }; diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2538,6 +2538,24 @@ DISPATCH(); } + TARGET(BUILD_STRING) { + PyObject *str; + PyObject *empty = PyUnicode_New(0, 0); + if (empty == NULL) { + goto error; + } + str = _PyUnicode_JoinArray(empty, stack_pointer - oparg, oparg); + Py_DECREF(empty); + if (str == NULL) + goto error; + while (--oparg >= 0) { + PyObject *item = POP(); + Py_DECREF(item); + } + PUSH(str); + DISPATCH(); + } + TARGET(BUILD_TUPLE) { PyObject *tup = PyTuple_New(oparg); if (tup == NULL) diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -970,6 +970,7 @@ case BUILD_TUPLE: case BUILD_LIST: case BUILD_SET: + case BUILD_STRING: return 1-oparg; case BUILD_LIST_UNPACK: case BUILD_TUPLE_UNPACK: @@ -3315,31 +3316,8 @@ static int compiler_joined_str(struct compiler *c, expr_ty e) { - /* Concatenate parts of a string using ''.join(parts). There are - probably better ways of doing this. - - This is used for constructs like "'x=' f'{42}'", which have to - be evaluated at compile time. */ - - static PyObject *empty_string; - static PyObject *join_string; - - if (!empty_string) { - empty_string = PyUnicode_FromString(""); - if (!empty_string) - return 0; - } - if (!join_string) { - join_string = PyUnicode_FromString("join"); - if (!join_string) - return 0; - } - - ADDOP_O(c, LOAD_CONST, empty_string, consts); - ADDOP_NAME(c, LOAD_ATTR, join_string, names); VISIT_SEQ(c, expr, e->v.JoinedStr.values); - ADDOP_I(c, BUILD_LIST, asdl_seq_LEN(e->v.JoinedStr.values)); - ADDOP_I(c, CALL_FUNCTION, 1); + ADDOP_I(c, BUILD_STRING, asdl_seq_LEN(e->v.JoinedStr.values)); return 1; } diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -242,7 +242,7 @@ 101,95,97,116,111,109,105,99,106,0,0,0,115,26,0,0, 0,0,5,16,1,6,1,26,1,2,3,14,1,20,1,16, 1,14,1,2,1,14,1,14,1,6,1,114,58,0,0,0, - 105,44,13,0,0,233,2,0,0,0,114,15,0,0,0,115, + 105,45,13,0,0,233,2,0,0,0,114,15,0,0,0,115, 2,0,0,0,13,10,90,11,95,95,112,121,99,97,99,104, 101,95,95,122,4,111,112,116,45,122,3,46,112,121,122,4, 46,112,121,99,78,41,1,218,12,111,112,116,105,109,105,122, @@ -344,7 +344,7 @@ 4,114,101,115,116,90,3,116,97,103,90,15,97,108,109,111, 115,116,95,102,105,108,101,110,97,109,101,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,17,99,97,99,104, - 101,95,102,114,111,109,95,115,111,117,114,99,101,0,1,0, + 101,95,102,114,111,109,95,115,111,117,114,99,101,1,1,0, 0,115,46,0,0,0,0,18,8,1,6,1,6,1,8,1, 4,1,8,1,12,1,12,1,16,1,8,1,8,1,8,1, 24,1,8,1,12,1,6,2,8,1,8,1,8,1,8,1, @@ -417,7 +417,7 @@ 90,9,111,112,116,95,108,101,118,101,108,90,13,98,97,115, 101,95,102,105,108,101,110,97,109,101,114,4,0,0,0,114, 4,0,0,0,114,6,0,0,0,218,17,115,111,117,114,99, - 101,95,102,114,111,109,95,99,97,99,104,101,44,1,0,0, + 101,95,102,114,111,109,95,99,97,99,104,101,45,1,0,0, 115,44,0,0,0,0,9,12,1,8,1,12,1,12,1,8, 1,6,1,10,1,10,1,8,1,6,1,10,1,8,1,16, 1,10,1,6,1,8,1,16,1,8,1,6,1,8,1,14, @@ -453,7 +453,7 @@ 110,115,105,111,110,218,11,115,111,117,114,99,101,95,112,97, 116,104,114,4,0,0,0,114,4,0,0,0,114,6,0,0, 0,218,15,95,103,101,116,95,115,111,117,114,99,101,102,105, - 108,101,77,1,0,0,115,20,0,0,0,0,7,12,1,4, + 108,101,78,1,0,0,115,20,0,0,0,0,7,12,1,4, 1,16,1,26,1,4,1,2,1,12,1,18,1,18,1,114, 94,0,0,0,99,1,0,0,0,0,0,0,0,1,0,0, 0,11,0,0,0,67,0,0,0,115,74,0,0,0,124,0, @@ -466,7 +466,7 @@ 0,0,0,114,82,0,0,0,114,69,0,0,0,114,77,0, 0,0,41,1,218,8,102,105,108,101,110,97,109,101,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,218,11,95, - 103,101,116,95,99,97,99,104,101,100,96,1,0,0,115,16, + 103,101,116,95,99,97,99,104,101,100,97,1,0,0,115,16, 0,0,0,0,1,14,1,2,1,8,1,14,1,8,1,14, 1,6,2,114,98,0,0,0,99,1,0,0,0,0,0,0, 0,2,0,0,0,11,0,0,0,67,0,0,0,115,52,0, @@ -480,7 +480,7 @@ 0,0,233,128,0,0,0,41,3,114,41,0,0,0,114,43, 0,0,0,114,42,0,0,0,41,2,114,37,0,0,0,114, 44,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,10,95,99,97,108,99,95,109,111,100,101,108, + 0,0,0,218,10,95,99,97,108,99,95,109,111,100,101,109, 1,0,0,115,12,0,0,0,0,2,2,1,14,1,14,1, 10,3,8,1,114,100,0,0,0,99,1,0,0,0,0,0, 0,0,3,0,0,0,11,0,0,0,3,0,0,0,115,68, @@ -518,7 +518,7 @@ 103,115,90,6,107,119,97,114,103,115,41,1,218,6,109,101, 116,104,111,100,114,4,0,0,0,114,6,0,0,0,218,19, 95,99,104,101,99,107,95,110,97,109,101,95,119,114,97,112, - 112,101,114,128,1,0,0,115,12,0,0,0,0,1,8,1, + 112,101,114,129,1,0,0,115,12,0,0,0,0,1,8,1, 8,1,10,1,4,1,20,1,122,40,95,99,104,101,99,107, 95,110,97,109,101,46,60,108,111,99,97,108,115,62,46,95, 99,104,101,99,107,95,110,97,109,101,95,119,114,97,112,112, @@ -538,7 +538,7 @@ 97,116,116,114,218,8,95,95,100,105,99,116,95,95,218,6, 117,112,100,97,116,101,41,3,90,3,110,101,119,90,3,111, 108,100,114,55,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,218,5,95,119,114,97,112,139,1,0, + 0,114,6,0,0,0,218,5,95,119,114,97,112,140,1,0, 0,115,8,0,0,0,0,1,10,1,10,1,22,1,122,26, 95,99,104,101,99,107,95,110,97,109,101,46,60,108,111,99, 97,108,115,62,46,95,119,114,97,112,41,1,78,41,3,218, @@ -546,7 +546,7 @@ 218,9,78,97,109,101,69,114,114,111,114,41,3,114,105,0, 0,0,114,106,0,0,0,114,116,0,0,0,114,4,0,0, 0,41,1,114,105,0,0,0,114,6,0,0,0,218,11,95, - 99,104,101,99,107,95,110,97,109,101,120,1,0,0,115,14, + 99,104,101,99,107,95,110,97,109,101,121,1,0,0,115,14, 0,0,0,0,8,14,7,2,1,10,1,14,2,14,5,10, 1,114,119,0,0,0,99,2,0,0,0,0,0,0,0,5, 0,0,0,4,0,0,0,67,0,0,0,115,60,0,0,0, @@ -574,7 +574,7 @@ 109,101,218,6,108,111,97,100,101,114,218,8,112,111,114,116, 105,111,110,115,218,3,109,115,103,114,4,0,0,0,114,4, 0,0,0,114,6,0,0,0,218,17,95,102,105,110,100,95, - 109,111,100,117,108,101,95,115,104,105,109,148,1,0,0,115, + 109,111,100,117,108,101,95,115,104,105,109,149,1,0,0,115, 10,0,0,0,0,10,14,1,16,1,4,1,22,1,114,126, 0,0,0,99,4,0,0,0,0,0,0,0,11,0,0,0, 19,0,0,0,67,0,0,0,115,128,1,0,0,105,0,125, @@ -654,7 +654,7 @@ 115,111,117,114,99,101,95,115,105,122,101,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,25,95,118,97,108, 105,100,97,116,101,95,98,121,116,101,99,111,100,101,95,104, - 101,97,100,101,114,165,1,0,0,115,76,0,0,0,0,11, + 101,97,100,101,114,166,1,0,0,115,76,0,0,0,0,11, 4,1,8,1,10,3,4,1,8,1,8,1,12,1,12,1, 12,1,8,1,12,1,12,1,12,1,12,1,10,1,12,1, 10,1,12,1,10,1,12,1,8,1,10,1,2,1,16,1, @@ -683,7 +683,7 @@ 41,5,114,56,0,0,0,114,101,0,0,0,114,92,0,0, 0,114,93,0,0,0,218,4,99,111,100,101,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,17,95,99,111, - 109,112,105,108,101,95,98,121,116,101,99,111,100,101,220,1, + 109,112,105,108,101,95,98,121,116,101,99,111,100,101,221,1, 0,0,115,16,0,0,0,0,2,10,1,10,1,12,1,8, 1,12,1,6,2,12,1,114,144,0,0,0,114,62,0,0, 0,99,3,0,0,0,0,0,0,0,4,0,0,0,3,0, @@ -702,7 +702,7 @@ 112,115,41,4,114,143,0,0,0,114,129,0,0,0,114,137, 0,0,0,114,56,0,0,0,114,4,0,0,0,114,4,0, 0,0,114,6,0,0,0,218,17,95,99,111,100,101,95,116, - 111,95,98,121,116,101,99,111,100,101,232,1,0,0,115,10, + 111,95,98,121,116,101,99,111,100,101,233,1,0,0,115,10, 0,0,0,0,3,8,1,14,1,14,1,16,1,114,147,0, 0,0,99,1,0,0,0,0,0,0,0,5,0,0,0,4, 0,0,0,67,0,0,0,115,62,0,0,0,100,1,100,2, @@ -729,7 +729,7 @@ 110,101,218,8,101,110,99,111,100,105,110,103,90,15,110,101, 119,108,105,110,101,95,100,101,99,111,100,101,114,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,13,100,101, - 99,111,100,101,95,115,111,117,114,99,101,242,1,0,0,115, + 99,111,100,101,95,115,111,117,114,99,101,243,1,0,0,115, 10,0,0,0,0,5,8,1,12,1,10,1,12,1,114,152, 0,0,0,41,2,114,123,0,0,0,218,26,115,117,98,109, 111,100,117,108,101,95,115,101,97,114,99,104,95,108,111,99, @@ -789,7 +789,7 @@ 115,117,102,102,105,120,101,115,114,156,0,0,0,90,7,100, 105,114,110,97,109,101,114,4,0,0,0,114,4,0,0,0, 114,6,0,0,0,218,23,115,112,101,99,95,102,114,111,109, - 95,102,105,108,101,95,108,111,99,97,116,105,111,110,3,2, + 95,102,105,108,101,95,108,111,99,97,116,105,111,110,4,2, 0,0,115,60,0,0,0,0,12,8,4,4,1,10,2,2, 1,14,1,14,1,6,8,18,1,6,3,8,1,16,1,14, 1,10,1,6,1,6,2,4,3,8,2,10,1,2,1,14, @@ -826,7 +826,7 @@ 67,65,76,95,77,65,67,72,73,78,69,41,2,218,3,99, 108,115,114,5,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,6,0,0,0,218,14,95,111,112,101,110,95,114,101, - 103,105,115,116,114,121,81,2,0,0,115,8,0,0,0,0, + 103,105,115,116,114,121,82,2,0,0,115,8,0,0,0,0, 2,2,1,14,1,14,1,122,36,87,105,110,100,111,119,115, 82,101,103,105,115,116,114,121,70,105,110,100,101,114,46,95, 111,112,101,110,95,114,101,103,105,115,116,114,121,99,2,0, @@ -852,7 +852,7 @@ 5,0,0,0,90,4,104,107,101,121,218,8,102,105,108,101, 112,97,116,104,114,4,0,0,0,114,4,0,0,0,114,6, 0,0,0,218,16,95,115,101,97,114,99,104,95,114,101,103, - 105,115,116,114,121,88,2,0,0,115,22,0,0,0,0,2, + 105,115,116,114,121,89,2,0,0,115,22,0,0,0,0,2, 6,1,8,2,6,1,10,1,22,1,2,1,12,1,26,1, 14,1,6,1,122,38,87,105,110,100,111,119,115,82,101,103, 105,115,116,114,121,70,105,110,100,101,114,46,95,115,101,97, @@ -874,7 +874,7 @@ 0,218,6,116,97,114,103,101,116,114,173,0,0,0,114,123, 0,0,0,114,163,0,0,0,114,161,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,9,102,105, - 110,100,95,115,112,101,99,103,2,0,0,115,26,0,0,0, + 110,100,95,115,112,101,99,104,2,0,0,115,26,0,0,0, 0,2,10,1,8,1,4,1,2,1,12,1,14,1,6,1, 16,1,14,1,6,1,10,1,8,1,122,31,87,105,110,100, 111,119,115,82,101,103,105,115,116,114,121,70,105,110,100,101, @@ -893,7 +893,7 @@ 0,114,123,0,0,0,41,4,114,167,0,0,0,114,122,0, 0,0,114,37,0,0,0,114,161,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,11,102,105,110, - 100,95,109,111,100,117,108,101,119,2,0,0,115,8,0,0, + 100,95,109,111,100,117,108,101,120,2,0,0,115,8,0,0, 0,0,7,12,1,8,1,8,2,122,33,87,105,110,100,111, 119,115,82,101,103,105,115,116,114,121,70,105,110,100,101,114, 46,102,105,110,100,95,109,111,100,117,108,101,41,2,78,78, @@ -903,7 +903,7 @@ 101,116,104,111,100,114,168,0,0,0,114,174,0,0,0,114, 177,0,0,0,114,178,0,0,0,114,4,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,165,0, - 0,0,69,2,0,0,115,20,0,0,0,8,2,4,3,4, + 0,0,70,2,0,0,115,20,0,0,0,8,2,4,3,4, 3,4,2,4,2,12,7,12,15,2,1,12,15,2,1,114, 165,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, 0,2,0,0,0,64,0,0,0,115,48,0,0,0,101,0, @@ -938,7 +938,7 @@ 97,0,0,0,90,13,102,105,108,101,110,97,109,101,95,98, 97,115,101,90,9,116,97,105,108,95,110,97,109,101,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,156,0, - 0,0,138,2,0,0,115,8,0,0,0,0,3,18,1,16, + 0,0,139,2,0,0,115,8,0,0,0,0,3,18,1,16, 1,14,1,122,24,95,76,111,97,100,101,114,66,97,115,105, 99,115,46,105,115,95,112,97,99,107,97,103,101,99,2,0, 0,0,0,0,0,0,2,0,0,0,1,0,0,0,67,0, @@ -948,7 +948,7 @@ 99,114,101,97,116,105,111,110,46,78,114,4,0,0,0,41, 2,114,103,0,0,0,114,161,0,0,0,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,13,99,114,101,97, - 116,101,95,109,111,100,117,108,101,146,2,0,0,115,0,0, + 116,101,95,109,111,100,117,108,101,147,2,0,0,115,0,0, 0,0,122,27,95,76,111,97,100,101,114,66,97,115,105,99, 115,46,99,114,101,97,116,101,95,109,111,100,117,108,101,99, 2,0,0,0,0,0,0,0,3,0,0,0,4,0,0,0, @@ -968,7 +968,7 @@ 114,114,0,0,0,41,3,114,103,0,0,0,218,6,109,111, 100,117,108,101,114,143,0,0,0,114,4,0,0,0,114,4, 0,0,0,114,6,0,0,0,218,11,101,120,101,99,95,109, - 111,100,117,108,101,149,2,0,0,115,10,0,0,0,0,2, + 111,100,117,108,101,150,2,0,0,115,10,0,0,0,0,2, 12,1,8,1,6,1,10,1,122,25,95,76,111,97,100,101, 114,66,97,115,105,99,115,46,101,120,101,99,95,109,111,100, 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, @@ -979,14 +979,14 @@ 95,108,111,97,100,95,109,111,100,117,108,101,95,115,104,105, 109,41,2,114,103,0,0,0,114,122,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,11,108,111, - 97,100,95,109,111,100,117,108,101,157,2,0,0,115,2,0, + 97,100,95,109,111,100,117,108,101,158,2,0,0,115,2,0, 0,0,0,2,122,25,95,76,111,97,100,101,114,66,97,115, 105,99,115,46,108,111,97,100,95,109,111,100,117,108,101,78, 41,8,114,108,0,0,0,114,107,0,0,0,114,109,0,0, 0,114,110,0,0,0,114,156,0,0,0,114,182,0,0,0, 114,187,0,0,0,114,189,0,0,0,114,4,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,180, - 0,0,0,133,2,0,0,115,10,0,0,0,8,3,4,2, + 0,0,0,134,2,0,0,115,10,0,0,0,8,3,4,2, 8,8,8,3,8,8,114,180,0,0,0,99,0,0,0,0, 0,0,0,0,0,0,0,0,3,0,0,0,64,0,0,0, 115,74,0,0,0,101,0,90,1,100,0,90,2,100,1,100, @@ -1011,7 +1011,7 @@ 32,32,32,32,32,32,32,78,41,1,218,7,73,79,69,114, 114,111,114,41,2,114,103,0,0,0,114,37,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,10, - 112,97,116,104,95,109,116,105,109,101,164,2,0,0,115,2, + 112,97,116,104,95,109,116,105,109,101,165,2,0,0,115,2, 0,0,0,0,6,122,23,83,111,117,114,99,101,76,111,97, 100,101,114,46,112,97,116,104,95,109,116,105,109,101,99,2, 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,67, @@ -1046,7 +1046,7 @@ 32,32,32,32,32,32,32,114,129,0,0,0,41,1,114,192, 0,0,0,41,2,114,103,0,0,0,114,37,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,10, - 112,97,116,104,95,115,116,97,116,115,172,2,0,0,115,2, + 112,97,116,104,95,115,116,97,116,115,173,2,0,0,115,2, 0,0,0,0,11,122,23,83,111,117,114,99,101,76,111,97, 100,101,114,46,112,97,116,104,95,115,116,97,116,115,99,4, 0,0,0,0,0,0,0,4,0,0,0,3,0,0,0,67, @@ -1070,7 +1070,7 @@ 93,0,0,0,90,10,99,97,99,104,101,95,112,97,116,104, 114,56,0,0,0,114,4,0,0,0,114,4,0,0,0,114, 6,0,0,0,218,15,95,99,97,99,104,101,95,98,121,116, - 101,99,111,100,101,185,2,0,0,115,2,0,0,0,0,8, + 101,99,111,100,101,186,2,0,0,115,2,0,0,0,0,8, 122,28,83,111,117,114,99,101,76,111,97,100,101,114,46,95, 99,97,99,104,101,95,98,121,116,101,99,111,100,101,99,3, 0,0,0,0,0,0,0,3,0,0,0,1,0,0,0,67, @@ -1087,7 +1087,7 @@ 32,32,32,32,32,32,78,114,4,0,0,0,41,3,114,103, 0,0,0,114,37,0,0,0,114,56,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,114,194,0,0, - 0,195,2,0,0,115,0,0,0,0,122,21,83,111,117,114, + 0,196,2,0,0,115,0,0,0,0,122,21,83,111,117,114, 99,101,76,111,97,100,101,114,46,115,101,116,95,100,97,116, 97,99,2,0,0,0,0,0,0,0,5,0,0,0,16,0, 0,0,67,0,0,0,115,84,0,0,0,124,0,106,0,124, @@ -1107,7 +1107,7 @@ 0,114,152,0,0,0,41,5,114,103,0,0,0,114,122,0, 0,0,114,37,0,0,0,114,150,0,0,0,218,3,101,120, 99,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 218,10,103,101,116,95,115,111,117,114,99,101,202,2,0,0, + 218,10,103,101,116,95,115,111,117,114,99,101,203,2,0,0, 115,14,0,0,0,0,2,10,1,2,1,14,1,16,1,6, 1,28,1,122,23,83,111,117,114,99,101,76,111,97,100,101, 114,46,103,101,116,95,115,111,117,114,99,101,114,31,0,0, @@ -1129,7 +1129,7 @@ 111,109,112,105,108,101,41,4,114,103,0,0,0,114,56,0, 0,0,114,37,0,0,0,114,199,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,14,115,111,117, - 114,99,101,95,116,111,95,99,111,100,101,212,2,0,0,115, + 114,99,101,95,116,111,95,99,111,100,101,213,2,0,0,115, 4,0,0,0,0,5,14,1,122,27,83,111,117,114,99,101, 76,111,97,100,101,114,46,115,111,117,114,99,101,95,116,111, 95,99,111,100,101,99,2,0,0,0,0,0,0,0,10,0, @@ -1186,7 +1186,7 @@ 56,0,0,0,218,10,98,121,116,101,115,95,100,97,116,97, 114,150,0,0,0,90,11,99,111,100,101,95,111,98,106,101, 99,116,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,183,0,0,0,220,2,0,0,115,78,0,0,0,0, + 0,114,183,0,0,0,221,2,0,0,115,78,0,0,0,0, 7,10,1,4,1,2,1,12,1,14,1,10,2,2,1,14, 1,14,1,6,2,12,1,2,1,14,1,14,1,6,2,2, 1,6,1,8,1,12,1,18,1,6,2,8,1,6,1,10, @@ -1198,7 +1198,7 @@ 114,193,0,0,0,114,195,0,0,0,114,194,0,0,0,114, 198,0,0,0,114,202,0,0,0,114,183,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,190,0,0,0,162,2,0,0,115,14,0,0,0, + 0,0,114,190,0,0,0,163,2,0,0,115,14,0,0,0, 8,2,8,8,8,13,8,10,8,7,8,10,14,8,114,190, 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, 4,0,0,0,0,0,0,0,115,76,0,0,0,101,0,90, @@ -1224,7 +1224,7 @@ 32,32,102,105,110,100,101,114,46,78,41,2,114,101,0,0, 0,114,37,0,0,0,41,3,114,103,0,0,0,114,122,0, 0,0,114,37,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,181,0,0,0,21,3,0,0,115, + 0,114,6,0,0,0,114,181,0,0,0,22,3,0,0,115, 4,0,0,0,0,3,6,1,122,19,70,105,108,101,76,111, 97,100,101,114,46,95,95,105,110,105,116,95,95,99,2,0, 0,0,0,0,0,0,2,0,0,0,2,0,0,0,67,0, @@ -1233,7 +1233,7 @@ 1,78,41,2,218,9,95,95,99,108,97,115,115,95,95,114, 114,0,0,0,41,2,114,103,0,0,0,218,5,111,116,104, 101,114,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,218,6,95,95,101,113,95,95,27,3,0,0,115,4,0, + 0,218,6,95,95,101,113,95,95,28,3,0,0,115,4,0, 0,0,0,1,12,1,122,17,70,105,108,101,76,111,97,100, 101,114,46,95,95,101,113,95,95,99,1,0,0,0,0,0, 0,0,1,0,0,0,3,0,0,0,67,0,0,0,115,20, @@ -1241,7 +1241,7 @@ 2,131,1,65,0,83,0,41,1,78,41,3,218,4,104,97, 115,104,114,101,0,0,0,114,37,0,0,0,41,1,114,103, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,8,95,95,104,97,115,104,95,95,31,3,0,0, + 0,0,218,8,95,95,104,97,115,104,95,95,32,3,0,0, 115,2,0,0,0,0,1,122,19,70,105,108,101,76,111,97, 100,101,114,46,95,95,104,97,115,104,95,95,99,2,0,0, 0,0,0,0,0,2,0,0,0,3,0,0,0,3,0,0, @@ -1256,7 +1256,7 @@ 218,5,115,117,112,101,114,114,206,0,0,0,114,189,0,0, 0,41,2,114,103,0,0,0,114,122,0,0,0,41,1,114, 207,0,0,0,114,4,0,0,0,114,6,0,0,0,114,189, - 0,0,0,34,3,0,0,115,2,0,0,0,0,10,122,22, + 0,0,0,35,3,0,0,115,2,0,0,0,0,10,122,22, 70,105,108,101,76,111,97,100,101,114,46,108,111,97,100,95, 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,2, 0,0,0,1,0,0,0,67,0,0,0,115,6,0,0,0, @@ -1266,7 +1266,7 @@ 102,111,117,110,100,32,98,121,32,116,104,101,32,102,105,110, 100,101,114,46,41,1,114,37,0,0,0,41,2,114,103,0, 0,0,114,122,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,154,0,0,0,46,3,0,0,115, + 0,114,6,0,0,0,114,154,0,0,0,47,3,0,0,115, 2,0,0,0,0,3,122,23,70,105,108,101,76,111,97,100, 101,114,46,103,101,116,95,102,105,108,101,110,97,109,101,99, 2,0,0,0,0,0,0,0,3,0,0,0,9,0,0,0, @@ -1278,7 +1278,7 @@ 116,101,115,46,218,1,114,78,41,3,114,52,0,0,0,114, 53,0,0,0,90,4,114,101,97,100,41,3,114,103,0,0, 0,114,37,0,0,0,114,57,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,196,0,0,0,51, + 114,4,0,0,0,114,6,0,0,0,114,196,0,0,0,52, 3,0,0,115,4,0,0,0,0,2,14,1,122,19,70,105, 108,101,76,111,97,100,101,114,46,103,101,116,95,100,97,116, 97,41,11,114,108,0,0,0,114,107,0,0,0,114,109,0, @@ -1286,7 +1286,7 @@ 0,114,211,0,0,0,114,119,0,0,0,114,189,0,0,0, 114,154,0,0,0,114,196,0,0,0,114,4,0,0,0,114, 4,0,0,0,41,1,114,207,0,0,0,114,6,0,0,0, - 114,206,0,0,0,16,3,0,0,115,14,0,0,0,8,3, + 114,206,0,0,0,17,3,0,0,115,14,0,0,0,8,3, 4,2,8,6,8,4,8,3,16,12,12,5,114,206,0,0, 0,99,0,0,0,0,0,0,0,0,0,0,0,0,3,0, 0,0,64,0,0,0,115,46,0,0,0,101,0,90,1,100, @@ -1308,7 +1308,7 @@ 109,101,90,7,115,116,95,115,105,122,101,41,3,114,103,0, 0,0,114,37,0,0,0,114,204,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,114,193,0,0,0, - 61,3,0,0,115,4,0,0,0,0,2,8,1,122,27,83, + 62,3,0,0,115,4,0,0,0,0,2,8,1,122,27,83, 111,117,114,99,101,70,105,108,101,76,111,97,100,101,114,46, 112,97,116,104,95,115,116,97,116,115,99,4,0,0,0,0, 0,0,0,5,0,0,0,5,0,0,0,67,0,0,0,115, @@ -1318,7 +1318,7 @@ 194,0,0,0,41,5,114,103,0,0,0,114,93,0,0,0, 114,92,0,0,0,114,56,0,0,0,114,44,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,195, - 0,0,0,66,3,0,0,115,4,0,0,0,0,2,8,1, + 0,0,0,67,3,0,0,115,4,0,0,0,0,2,8,1, 122,32,83,111,117,114,99,101,70,105,108,101,76,111,97,100, 101,114,46,95,99,97,99,104,101,95,98,121,116,101,99,111, 100,101,105,182,1,0,0,41,1,114,216,0,0,0,99,3, @@ -1352,7 +1352,7 @@ 114,37,0,0,0,114,56,0,0,0,114,216,0,0,0,218, 6,112,97,114,101,110,116,114,97,0,0,0,114,29,0,0, 0,114,25,0,0,0,114,197,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,194,0,0,0,71, + 114,4,0,0,0,114,6,0,0,0,114,194,0,0,0,72, 3,0,0,115,42,0,0,0,0,2,12,1,4,2,16,1, 12,1,14,2,14,1,10,1,2,1,14,1,14,2,6,1, 16,3,6,1,8,1,20,1,2,1,12,1,16,1,16,2, @@ -1361,7 +1361,7 @@ 114,108,0,0,0,114,107,0,0,0,114,109,0,0,0,114, 110,0,0,0,114,193,0,0,0,114,195,0,0,0,114,194, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,114,214,0,0,0,57,3,0,0, + 0,0,114,6,0,0,0,114,214,0,0,0,58,3,0,0, 115,8,0,0,0,8,2,4,2,8,5,8,5,114,214,0, 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,2, 0,0,0,64,0,0,0,115,32,0,0,0,101,0,90,1, @@ -1381,7 +1381,7 @@ 0,0,0,114,138,0,0,0,114,144,0,0,0,41,5,114, 103,0,0,0,114,122,0,0,0,114,37,0,0,0,114,56, 0,0,0,114,205,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,114,183,0,0,0,106,3,0,0, + 0,0,114,6,0,0,0,114,183,0,0,0,107,3,0,0, 115,8,0,0,0,0,1,10,1,10,1,18,1,122,29,83, 111,117,114,99,101,108,101,115,115,70,105,108,101,76,111,97, 100,101,114,46,103,101,116,95,99,111,100,101,99,2,0,0, @@ -1391,14 +1391,14 @@ 114,101,32,105,115,32,110,111,32,115,111,117,114,99,101,32, 99,111,100,101,46,78,114,4,0,0,0,41,2,114,103,0, 0,0,114,122,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,198,0,0,0,112,3,0,0,115, + 0,114,6,0,0,0,114,198,0,0,0,113,3,0,0,115, 2,0,0,0,0,2,122,31,83,111,117,114,99,101,108,101, 115,115,70,105,108,101,76,111,97,100,101,114,46,103,101,116, 95,115,111,117,114,99,101,78,41,6,114,108,0,0,0,114, 107,0,0,0,114,109,0,0,0,114,110,0,0,0,114,183, 0,0,0,114,198,0,0,0,114,4,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,114,219,0,0, - 0,102,3,0,0,115,6,0,0,0,8,2,4,2,8,6, + 0,103,3,0,0,115,6,0,0,0,8,2,4,2,8,6, 114,219,0,0,0,99,0,0,0,0,0,0,0,0,0,0, 0,0,3,0,0,0,64,0,0,0,115,92,0,0,0,101, 0,90,1,100,0,90,2,100,1,90,3,100,2,100,3,132, @@ -1419,7 +1419,7 @@ 0,124,2,124,0,95,1,100,0,83,0,41,1,78,41,2, 114,101,0,0,0,114,37,0,0,0,41,3,114,103,0,0, 0,114,101,0,0,0,114,37,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,181,0,0,0,129, + 114,4,0,0,0,114,6,0,0,0,114,181,0,0,0,130, 3,0,0,115,4,0,0,0,0,1,6,1,122,28,69,120, 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, 114,46,95,95,105,110,105,116,95,95,99,2,0,0,0,0, @@ -1428,7 +1428,7 @@ 124,0,106,1,124,1,106,1,107,2,83,0,41,1,78,41, 2,114,207,0,0,0,114,114,0,0,0,41,2,114,103,0, 0,0,114,208,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,209,0,0,0,133,3,0,0,115, + 0,114,6,0,0,0,114,209,0,0,0,134,3,0,0,115, 4,0,0,0,0,1,12,1,122,26,69,120,116,101,110,115, 105,111,110,70,105,108,101,76,111,97,100,101,114,46,95,95, 101,113,95,95,99,1,0,0,0,0,0,0,0,1,0,0, @@ -1437,7 +1437,7 @@ 83,0,41,1,78,41,3,114,210,0,0,0,114,101,0,0, 0,114,37,0,0,0,41,1,114,103,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,114,211,0,0, - 0,137,3,0,0,115,2,0,0,0,0,1,122,28,69,120, + 0,138,3,0,0,115,2,0,0,0,0,1,122,28,69,120, 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, 114,46,95,95,104,97,115,104,95,95,99,2,0,0,0,0, 0,0,0,3,0,0,0,4,0,0,0,67,0,0,0,115, @@ -1453,7 +1453,7 @@ 97,116,101,95,100,121,110,97,109,105,99,114,132,0,0,0, 114,101,0,0,0,114,37,0,0,0,41,3,114,103,0,0, 0,114,161,0,0,0,114,186,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,182,0,0,0,140, + 114,4,0,0,0,114,6,0,0,0,114,182,0,0,0,141, 3,0,0,115,10,0,0,0,0,2,4,1,10,1,6,1, 12,1,122,33,69,120,116,101,110,115,105,111,110,70,105,108, 101,76,111,97,100,101,114,46,99,114,101,97,116,101,95,109, @@ -1470,7 +1470,7 @@ 0,90,12,101,120,101,99,95,100,121,110,97,109,105,99,114, 132,0,0,0,114,101,0,0,0,114,37,0,0,0,41,2, 114,103,0,0,0,114,186,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,187,0,0,0,148,3, + 4,0,0,0,114,6,0,0,0,114,187,0,0,0,149,3, 0,0,115,6,0,0,0,0,2,14,1,6,1,122,31,69, 120,116,101,110,115,105,111,110,70,105,108,101,76,111,97,100, 101,114,46,101,120,101,99,95,109,111,100,117,108,101,99,2, @@ -1488,7 +1488,7 @@ 0,78,114,4,0,0,0,41,2,114,24,0,0,0,218,6, 115,117,102,102,105,120,41,1,218,9,102,105,108,101,95,110, 97,109,101,114,4,0,0,0,114,6,0,0,0,250,9,60, - 103,101,110,101,120,112,114,62,157,3,0,0,115,2,0,0, + 103,101,110,101,120,112,114,62,158,3,0,0,115,2,0,0, 0,4,1,122,49,69,120,116,101,110,115,105,111,110,70,105, 108,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, 97,103,101,46,60,108,111,99,97,108,115,62,46,60,103,101, @@ -1496,7 +1496,7 @@ 0,0,218,3,97,110,121,218,18,69,88,84,69,78,83,73, 79,78,95,83,85,70,70,73,88,69,83,41,2,114,103,0, 0,0,114,122,0,0,0,114,4,0,0,0,41,1,114,222, - 0,0,0,114,6,0,0,0,114,156,0,0,0,154,3,0, + 0,0,0,114,6,0,0,0,114,156,0,0,0,155,3,0, 0,115,6,0,0,0,0,2,14,1,12,1,122,30,69,120, 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, 114,46,105,115,95,112,97,99,107,97,103,101,99,2,0,0, @@ -1508,7 +1508,7 @@ 32,99,111,100,101,32,111,98,106,101,99,116,46,78,114,4, 0,0,0,41,2,114,103,0,0,0,114,122,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,183, - 0,0,0,160,3,0,0,115,2,0,0,0,0,2,122,28, + 0,0,0,161,3,0,0,115,2,0,0,0,0,2,122,28, 69,120,116,101,110,115,105,111,110,70,105,108,101,76,111,97, 100,101,114,46,103,101,116,95,99,111,100,101,99,2,0,0, 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, @@ -1518,7 +1518,7 @@ 97,118,101,32,110,111,32,115,111,117,114,99,101,32,99,111, 100,101,46,78,114,4,0,0,0,41,2,114,103,0,0,0, 114,122,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,198,0,0,0,164,3,0,0,115,2,0, + 6,0,0,0,114,198,0,0,0,165,3,0,0,115,2,0, 0,0,0,2,122,30,69,120,116,101,110,115,105,111,110,70, 105,108,101,76,111,97,100,101,114,46,103,101,116,95,115,111, 117,114,99,101,99,2,0,0,0,0,0,0,0,2,0,0, @@ -1529,7 +1529,7 @@ 117,110,100,32,98,121,32,116,104,101,32,102,105,110,100,101, 114,46,41,1,114,37,0,0,0,41,2,114,103,0,0,0, 114,122,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,154,0,0,0,168,3,0,0,115,2,0, + 6,0,0,0,114,154,0,0,0,169,3,0,0,115,2,0, 0,0,0,3,122,32,69,120,116,101,110,115,105,111,110,70, 105,108,101,76,111,97,100,101,114,46,103,101,116,95,102,105, 108,101,110,97,109,101,78,41,14,114,108,0,0,0,114,107, @@ -1538,7 +1538,7 @@ 0,114,187,0,0,0,114,156,0,0,0,114,183,0,0,0, 114,198,0,0,0,114,119,0,0,0,114,154,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,220,0,0,0,121,3,0,0,115,20,0,0, + 0,0,0,114,220,0,0,0,122,3,0,0,115,20,0,0, 0,8,6,4,2,8,4,8,4,8,3,8,8,8,6,8, 6,8,4,8,4,114,220,0,0,0,99,0,0,0,0,0, 0,0,0,0,0,0,0,2,0,0,0,64,0,0,0,115, @@ -1579,7 +1579,7 @@ 97,116,104,95,102,105,110,100,101,114,41,4,114,103,0,0, 0,114,101,0,0,0,114,37,0,0,0,218,11,112,97,116, 104,95,102,105,110,100,101,114,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,114,181,0,0,0,181,3,0,0, + 0,0,114,6,0,0,0,114,181,0,0,0,182,3,0,0, 115,8,0,0,0,0,1,6,1,6,1,14,1,122,23,95, 78,97,109,101,115,112,97,99,101,80,97,116,104,46,95,95, 105,110,105,116,95,95,99,1,0,0,0,0,0,0,0,4, @@ -1597,7 +1597,7 @@ 4,114,103,0,0,0,114,218,0,0,0,218,3,100,111,116, 90,2,109,101,114,4,0,0,0,114,4,0,0,0,114,6, 0,0,0,218,23,95,102,105,110,100,95,112,97,114,101,110, - 116,95,112,97,116,104,95,110,97,109,101,115,187,3,0,0, + 116,95,112,97,116,104,95,110,97,109,101,115,188,3,0,0, 115,8,0,0,0,0,2,18,1,8,2,4,3,122,38,95, 78,97,109,101,115,112,97,99,101,80,97,116,104,46,95,102, 105,110,100,95,112,97,114,101,110,116,95,112,97,116,104,95, @@ -1610,7 +1610,7 @@ 18,112,97,114,101,110,116,95,109,111,100,117,108,101,95,110, 97,109,101,90,14,112,97,116,104,95,97,116,116,114,95,110, 97,109,101,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,229,0,0,0,197,3,0,0,115,4,0,0,0, + 0,0,114,229,0,0,0,198,3,0,0,115,4,0,0,0, 0,1,12,1,122,31,95,78,97,109,101,115,112,97,99,101, 80,97,116,104,46,95,103,101,116,95,112,97,114,101,110,116, 95,112,97,116,104,99,1,0,0,0,0,0,0,0,3,0, @@ -1626,7 +1626,7 @@ 0,0,0,90,11,112,97,114,101,110,116,95,112,97,116,104, 114,161,0,0,0,114,4,0,0,0,114,4,0,0,0,114, 6,0,0,0,218,12,95,114,101,99,97,108,99,117,108,97, - 116,101,201,3,0,0,115,16,0,0,0,0,2,12,1,10, + 116,101,202,3,0,0,115,16,0,0,0,0,2,12,1,10, 1,14,3,18,1,6,1,8,1,6,1,122,27,95,78,97, 109,101,115,112,97,99,101,80,97,116,104,46,95,114,101,99, 97,108,99,117,108,97,116,101,99,1,0,0,0,0,0,0, @@ -1634,7 +1634,7 @@ 0,0,116,0,124,0,106,1,131,0,131,1,83,0,41,1, 78,41,2,218,4,105,116,101,114,114,236,0,0,0,41,1, 114,103,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,218,8,95,95,105,116,101,114,95,95,214,3, + 6,0,0,0,218,8,95,95,105,116,101,114,95,95,215,3, 0,0,115,2,0,0,0,0,1,122,23,95,78,97,109,101, 115,112,97,99,101,80,97,116,104,46,95,95,105,116,101,114, 95,95,99,3,0,0,0,0,0,0,0,3,0,0,0,3, @@ -1643,14 +1643,14 @@ 228,0,0,0,41,3,114,103,0,0,0,218,5,105,110,100, 101,120,114,37,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,6,0,0,0,218,11,95,95,115,101,116,105,116,101, - 109,95,95,217,3,0,0,115,2,0,0,0,0,1,122,26, + 109,95,95,218,3,0,0,115,2,0,0,0,0,1,122,26, 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, 95,115,101,116,105,116,101,109,95,95,99,1,0,0,0,0, 0,0,0,1,0,0,0,2,0,0,0,67,0,0,0,115, 12,0,0,0,116,0,124,0,106,1,131,0,131,1,83,0, 41,1,78,41,2,114,33,0,0,0,114,236,0,0,0,41, 1,114,103,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,218,7,95,95,108,101,110,95,95,220,3, + 114,6,0,0,0,218,7,95,95,108,101,110,95,95,221,3, 0,0,115,2,0,0,0,0,1,122,22,95,78,97,109,101, 115,112,97,99,101,80,97,116,104,46,95,95,108,101,110,95, 95,99,1,0,0,0,0,0,0,0,1,0,0,0,2,0, @@ -1659,7 +1659,7 @@ 101,115,112,97,99,101,80,97,116,104,40,123,33,114,125,41, 41,2,114,50,0,0,0,114,228,0,0,0,41,1,114,103, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,8,95,95,114,101,112,114,95,95,223,3,0,0, + 0,0,218,8,95,95,114,101,112,114,95,95,224,3,0,0, 115,2,0,0,0,0,1,122,23,95,78,97,109,101,115,112, 97,99,101,80,97,116,104,46,95,95,114,101,112,114,95,95, 99,2,0,0,0,0,0,0,0,2,0,0,0,2,0,0, @@ -1667,7 +1667,7 @@ 131,0,107,6,83,0,41,1,78,41,1,114,236,0,0,0, 41,2,114,103,0,0,0,218,4,105,116,101,109,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,12,95,95, - 99,111,110,116,97,105,110,115,95,95,226,3,0,0,115,2, + 99,111,110,116,97,105,110,115,95,95,227,3,0,0,115,2, 0,0,0,0,1,122,27,95,78,97,109,101,115,112,97,99, 101,80,97,116,104,46,95,95,99,111,110,116,97,105,110,115, 95,95,99,2,0,0,0,0,0,0,0,2,0,0,0,2, @@ -1675,7 +1675,7 @@ 106,1,124,1,131,1,1,0,100,0,83,0,41,1,78,41, 2,114,228,0,0,0,114,160,0,0,0,41,2,114,103,0, 0,0,114,243,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,160,0,0,0,229,3,0,0,115, + 0,114,6,0,0,0,114,160,0,0,0,230,3,0,0,115, 2,0,0,0,0,1,122,21,95,78,97,109,101,115,112,97, 99,101,80,97,116,104,46,97,112,112,101,110,100,78,41,14, 114,108,0,0,0,114,107,0,0,0,114,109,0,0,0,114, @@ -1683,7 +1683,7 @@ 0,0,0,114,236,0,0,0,114,238,0,0,0,114,240,0, 0,0,114,241,0,0,0,114,242,0,0,0,114,244,0,0, 0,114,160,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,226,0,0,0,174, + 114,4,0,0,0,114,6,0,0,0,114,226,0,0,0,175, 3,0,0,115,22,0,0,0,8,5,4,2,8,6,8,10, 8,4,8,13,8,3,8,3,8,3,8,3,8,3,114,226, 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, @@ -1700,7 +1700,7 @@ 41,2,114,226,0,0,0,114,228,0,0,0,41,4,114,103, 0,0,0,114,101,0,0,0,114,37,0,0,0,114,232,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,181,0,0,0,235,3,0,0,115,2,0,0,0,0, + 0,114,181,0,0,0,236,3,0,0,115,2,0,0,0,0, 1,122,25,95,78,97,109,101,115,112,97,99,101,76,111,97, 100,101,114,46,95,95,105,110,105,116,95,95,99,2,0,0, 0,0,0,0,0,2,0,0,0,2,0,0,0,67,0,0, @@ -1717,21 +1717,21 @@ 99,101,41,62,41,2,114,50,0,0,0,114,108,0,0,0, 41,2,114,167,0,0,0,114,186,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,11,109,111,100, - 117,108,101,95,114,101,112,114,238,3,0,0,115,2,0,0, + 117,108,101,95,114,101,112,114,239,3,0,0,115,2,0,0, 0,0,7,122,28,95,78,97,109,101,115,112,97,99,101,76, 111,97,100,101,114,46,109,111,100,117,108,101,95,114,101,112, 114,99,2,0,0,0,0,0,0,0,2,0,0,0,1,0, 0,0,67,0,0,0,115,4,0,0,0,100,1,83,0,41, 2,78,84,114,4,0,0,0,41,2,114,103,0,0,0,114, 122,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,156,0,0,0,247,3,0,0,115,2,0,0, + 0,0,0,114,156,0,0,0,248,3,0,0,115,2,0,0, 0,0,1,122,27,95,78,97,109,101,115,112,97,99,101,76, 111,97,100,101,114,46,105,115,95,112,97,99,107,97,103,101, 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, 78,114,32,0,0,0,114,4,0,0,0,41,2,114,103,0, 0,0,114,122,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,198,0,0,0,250,3,0,0,115, + 0,114,6,0,0,0,114,198,0,0,0,251,3,0,0,115, 2,0,0,0,0,1,122,27,95,78,97,109,101,115,112,97, 99,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, 114,99,101,99,2,0,0,0,0,0,0,0,2,0,0,0, @@ -1741,7 +1741,7 @@ 62,114,185,0,0,0,114,200,0,0,0,84,41,1,114,201, 0,0,0,41,2,114,103,0,0,0,114,122,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,183, - 0,0,0,253,3,0,0,115,2,0,0,0,0,1,122,25, + 0,0,0,254,3,0,0,115,2,0,0,0,0,1,122,25, 95,78,97,109,101,115,112,97,99,101,76,111,97,100,101,114, 46,103,101,116,95,99,111,100,101,99,2,0,0,0,0,0, 0,0,2,0,0,0,1,0,0,0,67,0,0,0,115,4, @@ -1750,14 +1750,14 @@ 32,102,111,114,32,109,111,100,117,108,101,32,99,114,101,97, 116,105,111,110,46,78,114,4,0,0,0,41,2,114,103,0, 0,0,114,161,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,182,0,0,0,0,4,0,0,115, + 0,114,6,0,0,0,114,182,0,0,0,1,4,0,0,115, 0,0,0,0,122,30,95,78,97,109,101,115,112,97,99,101, 76,111,97,100,101,114,46,99,114,101,97,116,101,95,109,111, 100,117,108,101,99,2,0,0,0,0,0,0,0,2,0,0, 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,0, 83,0,41,1,78,114,4,0,0,0,41,2,114,103,0,0, 0,114,186,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,187,0,0,0,3,4,0,0,115,2, + 114,6,0,0,0,114,187,0,0,0,4,4,0,0,115,2, 0,0,0,0,1,122,28,95,78,97,109,101,115,112,97,99, 101,76,111,97,100,101,114,46,101,120,101,99,95,109,111,100, 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, @@ -1775,7 +1775,7 @@ 32,123,33,114,125,41,4,114,117,0,0,0,114,132,0,0, 0,114,228,0,0,0,114,188,0,0,0,41,2,114,103,0, 0,0,114,122,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,189,0,0,0,6,4,0,0,115, + 0,114,6,0,0,0,114,189,0,0,0,7,4,0,0,115, 6,0,0,0,0,7,6,1,8,1,122,28,95,78,97,109, 101,115,112,97,99,101,76,111,97,100,101,114,46,108,111,97, 100,95,109,111,100,117,108,101,78,41,12,114,108,0,0,0, @@ -1784,7 +1784,7 @@ 0,0,0,114,183,0,0,0,114,182,0,0,0,114,187,0, 0,0,114,189,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,114,245,0,0,0, - 234,3,0,0,115,16,0,0,0,8,1,8,3,12,9,8, + 235,3,0,0,115,16,0,0,0,8,1,8,3,12,9,8, 3,8,3,8,3,8,3,8,3,114,245,0,0,0,99,0, 0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,64, 0,0,0,115,106,0,0,0,101,0,90,1,100,0,90,2, @@ -1817,7 +1817,7 @@ 99,97,99,104,101,218,6,118,97,108,117,101,115,114,111,0, 0,0,114,248,0,0,0,41,2,114,167,0,0,0,218,6, 102,105,110,100,101,114,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,248,0,0,0,24,4,0,0,115,6, + 114,6,0,0,0,114,248,0,0,0,25,4,0,0,115,6, 0,0,0,0,4,16,1,10,1,122,28,80,97,116,104,70, 105,110,100,101,114,46,105,110,118,97,108,105,100,97,116,101, 95,99,97,99,104,101,115,99,2,0,0,0,0,0,0,0, @@ -1837,7 +1837,7 @@ 114,121,0,0,0,114,102,0,0,0,41,3,114,167,0,0, 0,114,37,0,0,0,90,4,104,111,111,107,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,11,95,112,97, - 116,104,95,104,111,111,107,115,32,4,0,0,115,16,0,0, + 116,104,95,104,111,111,107,115,33,4,0,0,115,16,0,0, 0,0,3,18,1,12,1,12,1,2,1,8,1,14,1,12, 2,122,22,80,97,116,104,70,105,110,100,101,114,46,95,112, 97,116,104,95,104,111,111,107,115,99,2,0,0,0,0,0, @@ -1868,7 +1868,7 @@ 0,0,0,114,253,0,0,0,41,3,114,167,0,0,0,114, 37,0,0,0,114,251,0,0,0,114,4,0,0,0,114,4, 0,0,0,114,6,0,0,0,218,20,95,112,97,116,104,95, - 105,109,112,111,114,116,101,114,95,99,97,99,104,101,45,4, + 105,109,112,111,114,116,101,114,95,99,97,99,104,101,46,4, 0,0,115,22,0,0,0,0,8,8,1,2,1,12,1,14, 3,6,1,2,1,14,1,14,1,10,1,16,1,122,31,80, 97,116,104,70,105,110,100,101,114,46,95,112,97,116,104,95, @@ -1886,7 +1886,7 @@ 0,0,0,114,251,0,0,0,114,123,0,0,0,114,124,0, 0,0,114,161,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,6,0,0,0,218,16,95,108,101,103,97,99,121,95, - 103,101,116,95,115,112,101,99,67,4,0,0,115,18,0,0, + 103,101,116,95,115,112,101,99,68,4,0,0,115,18,0,0, 0,0,4,10,1,16,2,10,1,4,1,8,1,12,1,12, 1,6,1,122,27,80,97,116,104,70,105,110,100,101,114,46, 95,108,101,103,97,99,121,95,103,101,116,95,115,112,101,99, @@ -1917,7 +1917,7 @@ 110,97,109,101,115,112,97,99,101,95,112,97,116,104,90,5, 101,110,116,114,121,114,251,0,0,0,114,161,0,0,0,114, 124,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,9,95,103,101,116,95,115,112,101,99,82,4, + 0,0,0,218,9,95,103,101,116,95,115,112,101,99,83,4, 0,0,115,40,0,0,0,0,5,4,1,10,1,14,1,2, 1,10,1,8,1,10,1,14,2,12,1,8,1,2,1,10, 1,4,1,6,1,8,1,8,5,14,2,12,1,6,1,122, @@ -1945,7 +1945,7 @@ 155,0,0,0,114,226,0,0,0,41,6,114,167,0,0,0, 114,122,0,0,0,114,37,0,0,0,114,176,0,0,0,114, 161,0,0,0,114,2,1,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,177,0,0,0,114,4,0, + 0,0,0,114,6,0,0,0,114,177,0,0,0,115,4,0, 0,115,26,0,0,0,0,6,8,1,6,1,14,1,8,1, 6,1,10,1,6,1,4,3,6,1,16,1,6,2,6,2, 122,20,80,97,116,104,70,105,110,100,101,114,46,102,105,110, @@ -1967,7 +1967,7 @@ 177,0,0,0,114,123,0,0,0,41,4,114,167,0,0,0, 114,122,0,0,0,114,37,0,0,0,114,161,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,178, - 0,0,0,138,4,0,0,115,8,0,0,0,0,8,12,1, + 0,0,0,139,4,0,0,115,8,0,0,0,0,8,12,1, 8,1,4,1,122,22,80,97,116,104,70,105,110,100,101,114, 46,102,105,110,100,95,109,111,100,117,108,101,41,1,78,41, 2,78,78,41,1,78,41,12,114,108,0,0,0,114,107,0, @@ -1975,7 +1975,7 @@ 0,114,248,0,0,0,114,253,0,0,0,114,255,0,0,0, 114,0,1,0,0,114,3,1,0,0,114,177,0,0,0,114, 178,0,0,0,114,4,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,247,0,0,0,20,4,0, + 0,0,0,114,6,0,0,0,114,247,0,0,0,21,4,0, 0,115,22,0,0,0,8,2,4,2,12,8,12,13,12,22, 12,15,2,1,12,31,2,1,12,23,2,1,114,247,0,0, 0,99,0,0,0,0,0,0,0,0,0,0,0,0,3,0, @@ -2019,7 +2019,7 @@ 1,124,1,136,0,102,2,86,0,1,0,113,2,100,0,83, 0,41,1,78,114,4,0,0,0,41,2,114,24,0,0,0, 114,221,0,0,0,41,1,114,123,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,223,0,0,0,167,4,0,0,115, + 0,114,6,0,0,0,114,223,0,0,0,168,4,0,0,115, 2,0,0,0,4,0,122,38,70,105,108,101,70,105,110,100, 101,114,46,95,95,105,110,105,116,95,95,46,60,108,111,99, 97,108,115,62,46,60,103,101,110,101,120,112,114,62,114,61, @@ -2032,7 +2032,7 @@ 37,0,0,0,218,14,108,111,97,100,101,114,95,100,101,116, 97,105,108,115,90,7,108,111,97,100,101,114,115,114,163,0, 0,0,114,4,0,0,0,41,1,114,123,0,0,0,114,6, - 0,0,0,114,181,0,0,0,161,4,0,0,115,16,0,0, + 0,0,0,114,181,0,0,0,162,4,0,0,115,16,0,0, 0,0,4,4,1,14,1,28,1,6,2,10,1,6,1,8, 1,122,19,70,105,108,101,70,105,110,100,101,114,46,95,95, 105,110,105,116,95,95,99,1,0,0,0,0,0,0,0,1, @@ -2042,7 +2042,7 @@ 101,99,116,111,114,121,32,109,116,105,109,101,46,114,31,0, 0,0,78,114,90,0,0,0,41,1,114,6,1,0,0,41, 1,114,103,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,248,0,0,0,175,4,0,0,115,2, + 114,6,0,0,0,114,248,0,0,0,176,4,0,0,115,2, 0,0,0,0,2,122,28,70,105,108,101,70,105,110,100,101, 114,46,105,110,118,97,108,105,100,97,116,101,95,99,97,99, 104,101,115,99,2,0,0,0,0,0,0,0,3,0,0,0, @@ -2065,7 +2065,7 @@ 78,41,3,114,177,0,0,0,114,123,0,0,0,114,153,0, 0,0,41,3,114,103,0,0,0,114,122,0,0,0,114,161, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,120,0,0,0,181,4,0,0,115,8,0,0,0, + 0,0,114,120,0,0,0,182,4,0,0,115,8,0,0,0, 0,7,10,1,8,1,8,1,122,22,70,105,108,101,70,105, 110,100,101,114,46,102,105,110,100,95,108,111,97,100,101,114, 99,6,0,0,0,0,0,0,0,7,0,0,0,7,0,0, @@ -2076,7 +2076,7 @@ 0,0,0,114,162,0,0,0,114,122,0,0,0,114,37,0, 0,0,90,4,115,109,115,108,114,176,0,0,0,114,123,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,3,1,0,0,193,4,0,0,115,6,0,0,0,0, + 0,114,3,1,0,0,194,4,0,0,115,6,0,0,0,0, 1,10,1,12,1,122,20,70,105,108,101,70,105,110,100,101, 114,46,95,103,101,116,95,115,112,101,99,78,99,3,0,0, 0,0,0,0,0,14,0,0,0,15,0,0,0,67,0,0, @@ -2130,7 +2130,7 @@ 116,104,114,221,0,0,0,114,162,0,0,0,90,13,105,110, 105,116,95,102,105,108,101,110,97,109,101,90,9,102,117,108, 108,95,112,97,116,104,114,161,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,177,0,0,0,198, + 114,4,0,0,0,114,6,0,0,0,114,177,0,0,0,199, 4,0,0,115,70,0,0,0,0,5,4,1,14,1,2,1, 24,1,14,1,10,1,10,1,8,1,6,2,6,1,6,1, 10,2,6,1,4,2,8,1,12,1,16,1,8,1,10,1, @@ -2162,7 +2162,7 @@ 0,146,2,113,4,83,0,114,4,0,0,0,41,1,114,91, 0,0,0,41,2,114,24,0,0,0,90,2,102,110,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,250,9,60, - 115,101,116,99,111,109,112,62,19,5,0,0,115,2,0,0, + 115,101,116,99,111,109,112,62,20,5,0,0,115,2,0,0, 0,6,0,122,41,70,105,108,101,70,105,110,100,101,114,46, 95,102,105,108,108,95,99,97,99,104,101,46,60,108,111,99, 97,108,115,62,46,60,115,101,116,99,111,109,112,62,78,41, @@ -2179,7 +2179,7 @@ 111,110,116,101,110,116,115,114,243,0,0,0,114,101,0,0, 0,114,233,0,0,0,114,221,0,0,0,90,8,110,101,119, 95,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,11,1,0,0,246,4,0,0,115,34,0, + 6,0,0,0,114,11,1,0,0,247,4,0,0,115,34,0, 0,0,0,2,6,1,2,1,22,1,20,3,10,3,12,1, 12,7,6,1,10,1,16,1,4,1,18,2,4,1,14,1, 6,1,12,1,122,22,70,105,108,101,70,105,110,100,101,114, @@ -2217,7 +2217,7 @@ 1,114,37,0,0,0,41,2,114,167,0,0,0,114,10,1, 0,0,114,4,0,0,0,114,6,0,0,0,218,24,112,97, 116,104,95,104,111,111,107,95,102,111,114,95,70,105,108,101, - 70,105,110,100,101,114,31,5,0,0,115,6,0,0,0,0, + 70,105,110,100,101,114,32,5,0,0,115,6,0,0,0,0, 2,8,1,14,1,122,54,70,105,108,101,70,105,110,100,101, 114,46,112,97,116,104,95,104,111,111,107,46,60,108,111,99, 97,108,115,62,46,112,97,116,104,95,104,111,111,107,95,102, @@ -2225,7 +2225,7 @@ 0,0,41,3,114,167,0,0,0,114,10,1,0,0,114,16, 1,0,0,114,4,0,0,0,41,2,114,167,0,0,0,114, 10,1,0,0,114,6,0,0,0,218,9,112,97,116,104,95, - 104,111,111,107,21,5,0,0,115,4,0,0,0,0,10,14, + 104,111,111,107,22,5,0,0,115,4,0,0,0,0,10,14, 6,122,20,70,105,108,101,70,105,110,100,101,114,46,112,97, 116,104,95,104,111,111,107,99,1,0,0,0,0,0,0,0, 1,0,0,0,2,0,0,0,67,0,0,0,115,12,0,0, @@ -2233,7 +2233,7 @@ 122,16,70,105,108,101,70,105,110,100,101,114,40,123,33,114, 125,41,41,2,114,50,0,0,0,114,37,0,0,0,41,1, 114,103,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,242,0,0,0,39,5,0,0,115,2,0, + 6,0,0,0,114,242,0,0,0,40,5,0,0,115,2,0, 0,0,0,1,122,19,70,105,108,101,70,105,110,100,101,114, 46,95,95,114,101,112,114,95,95,41,1,78,41,15,114,108, 0,0,0,114,107,0,0,0,114,109,0,0,0,114,110,0, @@ -2242,7 +2242,7 @@ 114,177,0,0,0,114,11,1,0,0,114,179,0,0,0,114, 17,1,0,0,114,242,0,0,0,114,4,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,4,1, - 0,0,152,4,0,0,115,20,0,0,0,8,7,4,2,8, + 0,0,153,4,0,0,115,20,0,0,0,8,7,4,2,8, 14,8,4,4,2,8,12,8,5,10,48,8,31,12,18,114, 4,1,0,0,99,4,0,0,0,0,0,0,0,6,0,0, 0,11,0,0,0,67,0,0,0,115,148,0,0,0,124,0, @@ -2265,7 +2265,7 @@ 101,90,9,99,112,97,116,104,110,97,109,101,114,123,0,0, 0,114,161,0,0,0,114,4,0,0,0,114,4,0,0,0, 114,6,0,0,0,218,14,95,102,105,120,95,117,112,95,109, - 111,100,117,108,101,45,5,0,0,115,34,0,0,0,0,2, + 111,100,117,108,101,46,5,0,0,115,34,0,0,0,0,2, 10,1,10,1,4,1,4,1,8,1,8,1,12,2,10,1, 4,1,16,1,2,1,8,1,8,1,8,1,12,1,14,2, 114,22,1,0,0,99,0,0,0,0,0,0,0,0,3,0, @@ -2285,7 +2285,7 @@ 101,120,116,101,110,115,105,111,110,115,90,6,115,111,117,114, 99,101,90,8,98,121,116,101,99,111,100,101,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,114,158,0,0,0, - 68,5,0,0,115,8,0,0,0,0,5,12,1,8,1,8, + 69,5,0,0,115,8,0,0,0,0,5,12,1,8,1,8, 1,114,158,0,0,0,99,1,0,0,0,0,0,0,0,12, 0,0,0,12,0,0,0,67,0,0,0,115,188,1,0,0, 124,0,97,0,116,0,106,1,97,1,116,0,106,2,97,2, @@ -2337,7 +2337,7 @@ 2,86,0,1,0,113,2,100,1,83,0,41,2,114,31,0, 0,0,78,41,1,114,33,0,0,0,41,2,114,24,0,0, 0,114,80,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,223,0,0,0,104,5,0,0,115,2, + 114,6,0,0,0,114,223,0,0,0,105,5,0,0,115,2, 0,0,0,4,0,122,25,95,115,101,116,117,112,46,60,108, 111,99,97,108,115,62,46,60,103,101,110,101,120,112,114,62, 114,62,0,0,0,122,30,105,109,112,111,114,116,108,105,98, @@ -2368,7 +2368,7 @@ 114,101,102,95,109,111,100,117,108,101,90,13,119,105,110,114, 101,103,95,109,111,100,117,108,101,114,4,0,0,0,114,4, 0,0,0,114,6,0,0,0,218,6,95,115,101,116,117,112, - 79,5,0,0,115,82,0,0,0,0,8,4,1,6,1,6, + 80,5,0,0,115,82,0,0,0,0,8,4,1,6,1,6, 3,10,1,10,1,10,1,12,2,10,1,16,3,22,1,14, 2,22,1,8,1,10,1,10,1,4,2,2,1,10,1,6, 1,14,1,12,2,8,1,12,1,12,1,18,3,2,1,14, @@ -2392,7 +2392,7 @@ 2,114,30,1,0,0,90,17,115,117,112,112,111,114,116,101, 100,95,108,111,97,100,101,114,115,114,4,0,0,0,114,4, 0,0,0,114,6,0,0,0,218,8,95,105,110,115,116,97, - 108,108,147,5,0,0,115,16,0,0,0,0,2,8,1,6, + 108,108,148,5,0,0,115,16,0,0,0,0,2,8,1,6, 1,20,1,10,1,12,1,12,4,6,1,114,33,1,0,0, 41,1,122,3,119,105,110,41,2,114,1,0,0,0,114,2, 0,0,0,41,1,114,49,0,0,0,41,1,78,41,3,78, @@ -2426,7 +2426,7 @@ 0,114,6,0,0,0,218,8,60,109,111,100,117,108,101,62, 8,0,0,0,115,108,0,0,0,4,16,4,1,4,1,2, 1,6,3,8,17,8,5,8,5,8,6,8,12,8,10,8, - 9,8,5,8,7,10,22,10,116,16,1,12,2,4,1,4, + 9,8,5,8,7,10,22,10,117,16,1,12,2,4,1,4, 2,6,2,6,2,8,2,16,44,8,33,8,19,8,12,8, 12,8,28,8,17,10,55,10,12,10,10,8,14,6,3,4, 1,14,65,14,64,14,29,16,110,14,41,18,45,18,16,4, diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -156,7 +156,7 @@ &&TARGET_SETUP_ASYNC_WITH, &&TARGET_FORMAT_VALUE, &&TARGET_BUILD_CONST_KEY_MAP, - &&_unknown_opcode, + &&TARGET_BUILD_STRING, &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 15:08:47 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 19:08:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_do_not_need_vcstdint=2Eh_a?= =?utf-8?q?nymore?= Message-ID: <20160906190847.68747.30276.03C8AE0E@psf.io> https://hg.python.org/cpython/rev/f8950c8dfe78 changeset: 103131:f8950c8dfe78 user: Benjamin Peterson date: Tue Sep 06 12:07:53 2016 -0700 summary: do not need vcstdint.h anymore files: Modules/_decimal/libmpdec/vccompat.h | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Modules/_decimal/libmpdec/vccompat.h b/Modules/_decimal/libmpdec/vccompat.h --- a/Modules/_decimal/libmpdec/vccompat.h +++ b/Modules/_decimal/libmpdec/vccompat.h @@ -32,7 +32,6 @@ /* Visual C fixes: no stdint.h, no snprintf ... */ #ifdef _MSC_VER - #include "vcstdint.h" #undef inline #define inline __inline #undef random -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 15:35:26 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 06 Sep 2016 19:35:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325596=3A_Optimize?= =?utf-8?q?d_glob=28=29_and_iglob=28=29_functions_in_the?= Message-ID: <20160906193526.104204.7597.8151FA3A@psf.io> https://hg.python.org/cpython/rev/cb7ee9d9cddd changeset: 103132:cb7ee9d9cddd parent: 103130:28e280915508 user: Serhiy Storchaka date: Tue Sep 06 22:33:41 2016 +0300 summary: Issue #25596: Optimized glob() and iglob() functions in the glob module; they are now about 3--6 times faster. files: Doc/library/glob.rst | 2 +- Doc/whatsnew/3.6.rst | 4 + Lib/glob.py | 70 ++++++++++++++++++------------- Misc/NEWS | 3 + 4 files changed, 49 insertions(+), 30 deletions(-) diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst --- a/Doc/library/glob.rst +++ b/Doc/library/glob.rst @@ -14,7 +14,7 @@ according to the rules used by the Unix shell, although results are returned in arbitrary order. No tilde expansion is done, but ``*``, ``?``, and character ranges expressed with ``[]`` will be correctly matched. This is done by using -the :func:`os.listdir` and :func:`fnmatch.fnmatch` functions in concert, and +the :func:`os.scandir` and :func:`fnmatch.fnmatch` functions in concert, and not by actually invoking a subshell. Note that unlike :func:`fnmatch.fnmatch`, :mod:`glob` treats filenames beginning with a dot (``.``) as special cases. (For tilde and shell variable expansion, use :func:`os.path.expanduser` and diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -767,6 +767,10 @@ Argument Clinic this overhead is significantly decreased. (Contributed by Serhiy Storchaka in :issue:`27574`). +* Optimized :func:`~glob.glob` and :func:`~glob.iglob` functions in the + :mod:`glob` module; they are now about 3--6 times faster. + (Contributed by Serhiy Storchaka in :issue:`25596`). + Build and C API Changes ======================= diff --git a/Lib/glob.py b/Lib/glob.py --- a/Lib/glob.py +++ b/Lib/glob.py @@ -30,15 +30,16 @@ If recursive is true, the pattern '**' will match any files and zero or more directories and subdirectories. """ - it = _iglob(pathname, recursive) + it = _iglob(pathname, recursive, False) if recursive and _isrecursive(pathname): s = next(it) # skip empty string assert not s return it -def _iglob(pathname, recursive): +def _iglob(pathname, recursive, dironly): dirname, basename = os.path.split(pathname) if not has_magic(pathname): + assert not dironly if basename: if os.path.lexists(pathname): yield pathname @@ -49,47 +50,39 @@ return if not dirname: if recursive and _isrecursive(basename): - yield from glob2(dirname, basename) + yield from _glob2(dirname, basename, dironly) else: - yield from glob1(dirname, basename) + yield from _glob1(dirname, basename, dironly) return # `os.path.split()` returns the argument itself as a dirname if it is a # drive or UNC path. Prevent an infinite recursion if a drive or UNC path # contains magic characters (i.e. r'\\?\C:'). if dirname != pathname and has_magic(dirname): - dirs = _iglob(dirname, recursive) + dirs = _iglob(dirname, recursive, True) else: dirs = [dirname] if has_magic(basename): if recursive and _isrecursive(basename): - glob_in_dir = glob2 + glob_in_dir = _glob2 else: - glob_in_dir = glob1 + glob_in_dir = _glob1 else: - glob_in_dir = glob0 + glob_in_dir = _glob0 for dirname in dirs: - for name in glob_in_dir(dirname, basename): + for name in glob_in_dir(dirname, basename, dironly): yield os.path.join(dirname, name) # These 2 helper functions non-recursively glob inside a literal directory. -# They return a list of basenames. `glob1` accepts a pattern while `glob0` +# They return a list of basenames. _glob1 accepts a pattern while _glob0 # takes a literal basename (so it only has to check for its existence). -def glob1(dirname, pattern): - if not dirname: - if isinstance(pattern, bytes): - dirname = bytes(os.curdir, 'ASCII') - else: - dirname = os.curdir - try: - names = os.listdir(dirname) - except OSError: - return [] +def _glob1(dirname, pattern, dironly): + names = list(_iterdir(dirname, dironly)) if not _ishidden(pattern): - names = [x for x in names if not _ishidden(x)] + names = (x for x in names if not _ishidden(x)) return fnmatch.filter(names, pattern) -def glob0(dirname, basename): +def _glob0(dirname, basename, dironly): if not basename: # `os.path.split()` returns an empty basename for paths ending with a # directory separator. 'q*x/' should match only directories. @@ -100,30 +93,49 @@ return [basename] return [] +# Following functions are not public but can be used by third-party code. + +def glob0(dirname, pattern): + return _glob0(dirname, pattern, False) + +def glob1(dirname, pattern): + return _glob1(dirname, pattern, False) + # This helper function recursively yields relative pathnames inside a literal # directory. -def glob2(dirname, pattern): +def _glob2(dirname, pattern, dironly): assert _isrecursive(pattern) yield pattern[:0] - yield from _rlistdir(dirname) + yield from _rlistdir(dirname, dironly) -# Recursively yields relative pathnames inside a literal directory. -def _rlistdir(dirname): +# If dironly is false, yields all file names inside a directory. +# If dironly is true, yields only directory names. +def _iterdir(dirname, dironly): if not dirname: if isinstance(dirname, bytes): dirname = bytes(os.curdir, 'ASCII') else: dirname = os.curdir try: - names = os.listdir(dirname) - except os.error: + with os.scandir(dirname) as it: + for entry in it: + try: + if not dironly or entry.is_dir(): + yield entry.name + except OSError: + pass + except OSError: return + +# Recursively yields relative pathnames inside a literal directory. +def _rlistdir(dirname, dironly): + names = list(_iterdir(dirname, dironly)) for x in names: if not _ishidden(x): yield x path = os.path.join(dirname, x) if dirname else x - for y in _rlistdir(path): + for y in _rlistdir(path, dironly): yield os.path.join(x, y) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -89,6 +89,9 @@ Library ------- +- Issue #25596: Optimized glob() and iglob() functions in the + glob module; they are now about 3--6 times faster. + - Issue #27928: Add scrypt (password-based key derivation function) to hashlib module (requires OpenSSL 1.1.0). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 15:35:27 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 06 Sep 2016 19:35:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_heads?= Message-ID: <20160906193526.22646.71336.D40FB11C@psf.io> https://hg.python.org/cpython/rev/dc1e20228544 changeset: 103133:dc1e20228544 parent: 103132:cb7ee9d9cddd parent: 103131:f8950c8dfe78 user: Serhiy Storchaka date: Tue Sep 06 22:35:03 2016 +0300 summary: Merge heads files: Modules/_decimal/libmpdec/vccompat.h | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Modules/_decimal/libmpdec/vccompat.h b/Modules/_decimal/libmpdec/vccompat.h --- a/Modules/_decimal/libmpdec/vccompat.h +++ b/Modules/_decimal/libmpdec/vccompat.h @@ -32,7 +32,6 @@ /* Visual C fixes: no stdint.h, no snprintf ... */ #ifdef _MSC_VER - #include "vcstdint.h" #undef inline #define inline __inline #undef random -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 15:41:45 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 19:41:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_include_=28now=29_int_stan?= =?utf-8?q?dard_headers?= Message-ID: <20160906194144.104224.20848.383E40AF@psf.io> https://hg.python.org/cpython/rev/137a1c0b244a changeset: 103134:137a1c0b244a user: Benjamin Peterson date: Tue Sep 06 12:41:06 2016 -0700 summary: include (now) int standard headers files: Modules/_decimal/libmpdec/mpdecimal.h | 8 ++------ 1 files changed, 2 insertions(+), 6 deletions(-) diff --git a/Modules/_decimal/libmpdec/mpdecimal.h b/Modules/_decimal/libmpdec/mpdecimal.h --- a/Modules/_decimal/libmpdec/mpdecimal.h +++ b/Modules/_decimal/libmpdec/mpdecimal.h @@ -48,6 +48,8 @@ #include #include #include +#include +#include #ifdef _MSC_VER #include "vccompat.h" @@ -59,12 +61,6 @@ #define MPD_HIDE_SYMBOLS_END #define EXTINLINE extern inline #else - #ifdef HAVE_STDINT_H - #include - #endif - #ifdef HAVE_INTTYPES_H - #include - #endif #ifndef __GNUC_STDC_INLINE__ #define __GNUC_STDC_INLINE__ 1 #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 15:44:58 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 19:44:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_dtoa=2Ec=3A_remove_code_fo?= =?utf-8?q?r_platforms_with_64-bit_integers_=28=2317884=29?= Message-ID: <20160906194457.6601.93118.C4441092@psf.io> https://hg.python.org/cpython/rev/8b74e5528f35 changeset: 103135:8b74e5528f35 user: Benjamin Peterson date: Tue Sep 06 12:44:21 2016 -0700 summary: dtoa.c: remove code for platforms with 64-bit integers (#17884) files: Python/dtoa.c | 104 -------------------------------------- 1 files changed, 0 insertions(+), 104 deletions(-) diff --git a/Python/dtoa.c b/Python/dtoa.c --- a/Python/dtoa.c +++ b/Python/dtoa.c @@ -448,13 +448,8 @@ multadd(Bigint *b, int m, int a) /* multiply by m and add a */ { int i, wds; -#ifdef ULLong ULong *x; ULLong carry, y; -#else - ULong carry, *x, y; - ULong xi, z; -#endif Bigint *b1; wds = b->wds; @@ -462,17 +457,9 @@ i = 0; carry = a; do { -#ifdef ULLong y = *x * (ULLong)m + carry; carry = y >> 32; *x++ = (ULong)(y & FFFFFFFF); -#else - xi = *x; - y = (xi & 0xffff) * m + carry; - z = (xi >> 16) * m + (y >> 16); - carry = z >> 16; - *x++ = (z << 16) + (y & 0xffff); -#endif } while(++i < wds); if (carry) { @@ -633,12 +620,7 @@ int k, wa, wb, wc; ULong *x, *xa, *xae, *xb, *xbe, *xc, *xc0; ULong y; -#ifdef ULLong ULLong carry, z; -#else - ULong carry, z; - ULong z2; -#endif if ((!a->x[0] && a->wds == 1) || (!b->x[0] && b->wds == 1)) { c = Balloc(0); @@ -670,7 +652,6 @@ xb = b->x; xbe = xb + wb; xc0 = c->x; -#ifdef ULLong for(; xb < xbe; xc0++) { if ((y = *xb++)) { x = xa; @@ -685,39 +666,6 @@ *xc = (ULong)carry; } } -#else - for(; xb < xbe; xb++, xc0++) { - if (y = *xb & 0xffff) { - x = xa; - xc = xc0; - carry = 0; - do { - z = (*x & 0xffff) * y + (*xc & 0xffff) + carry; - carry = z >> 16; - z2 = (*x++ >> 16) * y + (*xc >> 16) + carry; - carry = z2 >> 16; - Storeinc(xc, z2, z); - } - while(x < xae); - *xc = carry; - } - if (y = *xb >> 16) { - x = xa; - xc = xc0; - carry = 0; - z2 = *xc; - do { - z = (*x & 0xffff) * y + (*xc >> 16) + carry; - carry = z >> 16; - Storeinc(xc, z, z2); - z2 = (*x++ >> 16) * y + (*xc & 0xffff) + carry; - carry = z2 >> 16; - } - while(x < xae); - *xc = z2; - } - } -#endif for(xc0 = c->x, xc = xc0 + wc; wc > 0 && !*--xc; --wc) ; c->wds = wc; return c; @@ -926,12 +874,7 @@ Bigint *c; int i, wa, wb; ULong *xa, *xae, *xb, *xbe, *xc; -#ifdef ULLong ULLong borrow, y; -#else - ULong borrow, y; - ULong z; -#endif i = cmp(a,b); if (!i) { @@ -962,7 +905,6 @@ xbe = xb + wb; xc = c->x; borrow = 0; -#ifdef ULLong do { y = (ULLong)*xa++ - *xb++ - borrow; borrow = y >> 32 & (ULong)1; @@ -974,23 +916,6 @@ borrow = y >> 32 & (ULong)1; *xc++ = (ULong)(y & FFFFFFFF); } -#else - do { - y = (*xa & 0xffff) - (*xb & 0xffff) - borrow; - borrow = (y & 0x10000) >> 16; - z = (*xa++ >> 16) - (*xb++ >> 16) - borrow; - borrow = (z & 0x10000) >> 16; - Storeinc(xc, z, y); - } - while(xb < xbe); - while(xa < xae) { - y = (*xa & 0xffff) - borrow; - borrow = (y & 0x10000) >> 16; - z = (*xa++ >> 16) - borrow; - borrow = (z & 0x10000) >> 16; - Storeinc(xc, z, y); - } -#endif while(!*--xc) wa--; c->wds = wa; @@ -1235,12 +1160,7 @@ { int n; ULong *bx, *bxe, q, *sx, *sxe; -#ifdef ULLong ULLong borrow, carry, y, ys; -#else - ULong borrow, carry, y, ys; - ULong si, z, zs; -#endif n = S->wds; #ifdef DEBUG @@ -1262,23 +1182,11 @@ borrow = 0; carry = 0; do { -#ifdef ULLong ys = *sx++ * (ULLong)q + carry; carry = ys >> 32; y = *bx - (ys & FFFFFFFF) - borrow; borrow = y >> 32 & (ULong)1; *bx++ = (ULong)(y & FFFFFFFF); -#else - si = *sx++; - ys = (si & 0xffff) * q + carry; - zs = (si >> 16) * q + (ys >> 16); - carry = zs >> 16; - y = (*bx & 0xffff) - (ys & 0xffff) - borrow; - borrow = (y & 0x10000) >> 16; - z = (*bx >> 16) - (zs & 0xffff) - borrow; - borrow = (z & 0x10000) >> 16; - Storeinc(bx, z, y); -#endif } while(sx <= sxe); if (!*bxe) { @@ -1295,23 +1203,11 @@ bx = b->x; sx = S->x; do { -#ifdef ULLong ys = *sx++ + carry; carry = ys >> 32; y = *bx - (ys & FFFFFFFF) - borrow; borrow = y >> 32 & (ULong)1; *bx++ = (ULong)(y & FFFFFFFF); -#else - si = *sx++; - ys = (si & 0xffff) + carry; - zs = (si >> 16) + (ys >> 16); - carry = zs >> 16; - y = (*bx & 0xffff) - (ys & 0xffff) - borrow; - borrow = (y & 0x10000) >> 16; - z = (*bx >> 16) - (zs & 0xffff) - borrow; - borrow = (z & 0x10000) >> 16; - Storeinc(bx, z, y); -#endif } while(sx <= sxe); bx = b->x; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 16:03:34 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 20:03:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326798=3A_Add_BLAK?= =?utf-8?q?E2_=28blake2b_and_blake2s=29_to_hashlib=2E?= Message-ID: <20160906200333.68817.76795.7D9AC5C1@psf.io> https://hg.python.org/cpython/rev/4969f6d343b1 changeset: 103136:4969f6d343b1 user: Christian Heimes date: Tue Sep 06 22:03:25 2016 +0200 summary: Issue #26798: Add BLAKE2 (blake2b and blake2s) to hashlib. files: Doc/library/crypto.rst | 1 + Doc/library/hashlib-blake2-tree.png | Bin Doc/library/hashlib-blake2.rst | 443 +++++++++ Doc/library/hashlib.rst | 14 +- Lib/hashlib.py | 38 +- Lib/test/test_hashlib.py | 201 ++++- Makefile.pre.in | 8 +- Misc/NEWS | 2 + Modules/_blake2/blake2b2s.py | 49 + Modules/_blake2/blake2b_impl.c | 460 ++++++++++ Modules/_blake2/blake2module.c | 105 ++ Modules/_blake2/blake2ns.h | 32 + Modules/_blake2/blake2s_impl.c | 460 ++++++++++ Modules/_blake2/clinic/blake2b_impl.c.h | 125 ++ Modules/_blake2/clinic/blake2s_impl.c.h | 125 ++ Modules/_blake2/impl/blake2-config.h | 74 + Modules/_blake2/impl/blake2-impl.h | 139 +++ Modules/_blake2/impl/blake2.h | 161 +++ Modules/_blake2/impl/blake2b-load-sse2.h | 70 + Modules/_blake2/impl/blake2b-load-sse41.h | 404 ++++++++ Modules/_blake2/impl/blake2b-ref.c | 416 +++++++++ Modules/_blake2/impl/blake2b-round.h | 159 +++ Modules/_blake2/impl/blake2b.c | 450 +++++++++ Modules/_blake2/impl/blake2s-load-sse2.h | 61 + Modules/_blake2/impl/blake2s-load-sse41.h | 231 +++++ Modules/_blake2/impl/blake2s-load-xop.h | 191 ++++ Modules/_blake2/impl/blake2s-ref.c | 406 ++++++++ Modules/_blake2/impl/blake2s-round.h | 90 + Modules/_blake2/impl/blake2s.c | 431 +++++++++ Modules/hashlib.h | 19 +- PCbuild/pythoncore.vcxproj | 3 + PCbuild/pythoncore.vcxproj.filters | 9 + setup.py | 16 + 33 files changed, 5364 insertions(+), 29 deletions(-) diff --git a/Doc/library/crypto.rst b/Doc/library/crypto.rst --- a/Doc/library/crypto.rst +++ b/Doc/library/crypto.rst @@ -15,5 +15,6 @@ .. toctree:: hashlib.rst + hashlib-blake2.rst hmac.rst secrets.rst diff --git a/Doc/library/hashlib-blake2-tree.png b/Doc/library/hashlib-blake2-tree.png new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..010dcbafe7c61d6dd897db31bef617da73006e66 GIT binary patch [stripped] diff --git a/Doc/library/hashlib-blake2.rst b/Doc/library/hashlib-blake2.rst new file mode 100644 --- /dev/null +++ b/Doc/library/hashlib-blake2.rst @@ -0,0 +1,443 @@ +.. _hashlib-blake2: + +:mod:`hashlib` --- BLAKE2 hash functions +======================================== + +.. module:: hashlib + :synopsis: BLAKE2 hash function for Python +.. sectionauthor:: Dmitry Chestnykh + +.. index:: + single: blake2b, blake2s + +BLAKE2_ is a cryptographic hash function, which offers highest security while +being as fast as MD5 or SHA-1, and comes in two flavors: + +* **BLAKE2b**, optimized for 64-bit platforms and produces digests of any size + between 1 and 64 bytes, + +* **BLAKE2s**, optimized for 8- to 32-bit platforms and produces digests of any + size between 1 and 32 bytes. + +BLAKE2 supports **keyed mode** (a faster and simpler replacement for HMAC_), +**salted hashing**, **personalization**, and **tree hashing**. + +Hash objects from this module follow the API of standard library's +:mod:`hashlib` objects. + + +Module +====== + +Creating hash objects +--------------------- + +New hash objects are created by calling constructor functions: + + +.. function:: blake2b(data=b'', digest_size=64, key=b'', salt=b'', \ + person=b'', fanout=1, depth=1, leaf_size=0, node_offset=0, \ + node_depth=0, inner_size=0, last_node=False) + +.. function:: blake2s(data=b'', digest_size=32, key=b'', salt=b'', \ + person=b'', fanout=1, depth=1, leaf_size=0, node_offset=0, \ + node_depth=0, inner_size=0, last_node=False) + + +These functions return the corresponding hash objects for calculating +BLAKE2b or BLAKE2s. They optionally take these general parameters: + +* *data*: initial chunk of data to hash, which must be interpretable as buffer + of bytes. + +* *digest_size*: size of output digest in bytes. + +* *key*: key for keyed hashing (up to 64 bytes for BLAKE2b, up to 32 bytes for + BLAKE2s). + +* *salt*: salt for randomized hashing (up to 16 bytes for BLAKE2b, up to 8 + bytes for BLAKE2s). + +* *person*: personalization string (up to 16 bytes for BLAKE2b, up to 8 bytes + for BLAKE2s). + +The following table shows limits for general parameters (in bytes): + +======= =========== ======== ========= =========== +Hash digest_size len(key) len(salt) len(person) +======= =========== ======== ========= =========== +BLAKE2b 64 64 16 16 +BLAKE2s 32 32 8 8 +======= =========== ======== ========= =========== + +.. note:: + + BLAKE2 specification defines constant lengths for salt and personalization + parameters, however, for convenience, this implementation accepts byte + strings of any size up to the specified length. If the length of the + parameter is less than specified, it is padded with zeros, thus, for + example, ``b'salt'`` and ``b'salt\x00'`` is the same value. (This is not + the case for *key*.) + +These sizes are available as module `constants`_ described below. + +Constructor functions also accept the following tree hashing parameters: + +* *fanout*: fanout (0 to 255, 0 if unlimited, 1 in sequential mode). + +* *depth*: maximal depth of tree (1 to 255, 255 if unlimited, 1 in + sequential mode). + +* *leaf_size*: maximal byte length of leaf (0 to 2**32-1, 0 if unlimited or in + sequential mode). + +* *node_offset*: node offset (0 to 2**64-1 for BLAKE2b, 0 to 2**48-1 for + BLAKE2s, 0 for the first, leftmost, leaf, or in sequential mode). + +* *node_depth*: node depth (0 to 255, 0 for leaves, or in sequential mode). + +* *inner_size*: inner digest size (0 to 64 for BLAKE2b, 0 to 32 for + BLAKE2s, 0 in sequential mode). + +* *last_node*: boolean indicating whether the processed node is the last + one (`False` for sequential mode). + +.. figure:: hashlib-blake2-tree.png + :alt: Explanation of tree mode parameters. + +See section 2.10 in `BLAKE2 specification +`_ for comprehensive review of tree +hashing. + + +Constants +--------- + +.. data:: blake2b.SALT_SIZE +.. data:: blake2s.SALT_SIZE + +Salt length (maximum length accepted by constructors). + + +.. data:: blake2b.PERSON_SIZE +.. data:: blake2s.PERSON_SIZE + +Personalization string length (maximum length accepted by constructors). + + +.. data:: blake2b.MAX_KEY_SIZE +.. data:: blake2s.MAX_KEY_SIZE + +Maximum key size. + + +.. data:: blake2b.MAX_DIGEST_SIZE +.. data:: blake2s.MAX_DIGEST_SIZE + +Maximum digest size that the hash function can output. + + +Examples +======== + +Simple hashing +-------------- + +To calculate hash of some data, you should first construct a hash object by +calling the appropriate constructor function (:func:`blake2b` or +:func:`blake2s`), then update it with the data by calling :meth:`update` on the +object, and, finally, get the digest out of the object by calling +:meth:`digest` (or :meth:`hexdigest` for hex-encoded string). + + >>> from hashlib import blake2b + >>> h = blake2b() + >>> h.update(b'Hello world') + >>> h.hexdigest() + '6ff843ba685842aa82031d3f53c48b66326df7639a63d128974c5c14f31a0f33343a8c65551134ed1ae0f2b0dd2bb495dc81039e3eeb0aa1bb0388bbeac29183' + + +As a shortcut, you can pass the first chunk of data to update directly to the +constructor as the first argument (or as *data* keyword argument): + + >>> from hashlib import blake2b + >>> blake2b(b'Hello world').hexdigest() + '6ff843ba685842aa82031d3f53c48b66326df7639a63d128974c5c14f31a0f33343a8c65551134ed1ae0f2b0dd2bb495dc81039e3eeb0aa1bb0388bbeac29183' + +You can call :meth:`hash.update` as many times as you need to iteratively +update the hash: + + >>> from hashlib import blake2b + >>> items = [b'Hello', b' ', b'world'] + >>> h = blake2b() + >>> for item in items: + ... h.update(item) + >>> h.hexdigest() + '6ff843ba685842aa82031d3f53c48b66326df7639a63d128974c5c14f31a0f33343a8c65551134ed1ae0f2b0dd2bb495dc81039e3eeb0aa1bb0388bbeac29183' + + +Using different digest sizes +---------------------------- + +BLAKE2 has configurable size of digests up to 64 bytes for BLAKE2b and up to 32 +bytes for BLAKE2s. For example, to replace SHA-1 with BLAKE2b without changing +the size of output, we can tell BLAKE2b to produce 20-byte digests: + + >>> from hashlib import blake2b + >>> h = blake2b(digest_size=20) + >>> h.update(b'Replacing SHA1 with the more secure function') + >>> h.hexdigest() + 'd24f26cf8de66472d58d4e1b1774b4c9158b1f4c' + >>> h.digest_size + 20 + >>> len(h.digest()) + 20 + +Hash objects with different digest sizes have completely different outputs +(shorter hashes are *not* prefixes of longer hashes); BLAKE2b and BLAKE2s +produce different outputs even if the output length is the same: + + >>> from hashlib import blake2b, blake2s + >>> blake2b(digest_size=10).hexdigest() + '6fa1d8fcfd719046d762' + >>> blake2b(digest_size=11).hexdigest() + 'eb6ec15daf9546254f0809' + >>> blake2s(digest_size=10).hexdigest() + '1bf21a98c78a1c376ae9' + >>> blake2s(digest_size=11).hexdigest() + '567004bf96e4a25773ebf4' + + +Keyed hashing +------------- + +Keyed hashing can be used for authentication as a faster and simpler +replacement for `Hash-based message authentication code +`_ (HMAC). +BLAKE2 can be securely used in prefix-MAC mode thanks to the +indifferentiability property inherited from BLAKE. + +This example shows how to get a (hex-encoded) 128-bit authentication code for +message ``b'message data'`` with key ``b'pseudorandom key'``: + + >>> from hashlib import blake2b + >>> h = blake2b(key=b'pseudorandom key', digest_size=16) + >>> h.update(b'message data') + >>> h.hexdigest() + '3d363ff7401e02026f4a4687d4863ced' + + +As a practical example, a web application can symmetrically sign cookies sent +to users and later verify them to make sure they weren't tampered with: + + >>> from hashlib import blake2b + >>> from hmac import compare_digest + >>> + >>> SECRET_KEY = b'pseudorandomly generated server secret key' + >>> AUTH_SIZE = 16 + >>> + >>> def sign(cookie): + ... h = blake2b(data=cookie, digest_size=AUTH_SIZE, key=SECRET_KEY) + ... return h.hexdigest() + >>> + >>> cookie = b'user:vatrogasac' + >>> sig = sign(cookie) + >>> print("{0},{1}".format(cookie.decode('utf-8'), sig)) + user:vatrogasac,349cf904533767ed2d755279a8df84d0 + >>> compare_digest(cookie, sig) + True + >>> compare_digest(b'user:policajac', sig) + False + >>> compare_digesty(cookie, '0102030405060708090a0b0c0d0e0f00') + False + +Even though there's a native keyed hashing mode, BLAKE2 can, of course, be used +in HMAC construction with :mod:`hmac` module: + + >>> import hmac, hashlib + >>> m = hmac.new(b'secret key', digestmod=hashlib.blake2s) + >>> m.update(b'message') + >>> m.hexdigest() + 'e3c8102868d28b5ff85fc35dda07329970d1a01e273c37481326fe0c861c8142' + + +Randomized hashing +------------------ + +By setting *salt* parameter users can introduce randomization to the hash +function. Randomized hashing is useful for protecting against collision attacks +on the hash function used in digital signatures. + + Randomized hashing is designed for situations where one party, the message + preparer, generates all or part of a message to be signed by a second + party, the message signer. If the message preparer is able to find + cryptographic hash function collisions (i.e., two messages producing the + same hash value), then she might prepare meaningful versions of the message + that would produce the same hash value and digital signature, but with + different results (e.g., transferring $1,000,000 to an account, rather than + $10). Cryptographic hash functions have been designed with collision + resistance as a major goal, but the current concentration on attacking + cryptographic hash functions may result in a given cryptographic hash + function providing less collision resistance than expected. Randomized + hashing offers the signer additional protection by reducing the likelihood + that a preparer can generate two or more messages that ultimately yield the + same hash value during the digital signature generation process ? even if + it is practical to find collisions for the hash function. However, the use + of randomized hashing may reduce the amount of security provided by a + digital signature when all portions of the message are prepared + by the signer. + + (`NIST SP-800-106 "Randomized Hashing for Digital Signatures" + `_) + +In BLAKE2 the salt is processed as a one-time input to the hash function during +initialization, rather than as an input to each compression function. + +.. warning:: + + *Salted hashing* (or just hashing) with BLAKE2 or any other general-purpose + cryptographic hash function, such as SHA-256, is not suitable for hashing + passwords. See `BLAKE2 FAQ `_ for more + information. +.. + + >>> import os + >>> from hashlib import blake2b + >>> msg = b'some message' + >>> # Calculate the first hash with a random salt. + >>> salt1 = os.urandom(blake2b.SALT_SIZE) + >>> h1 = blake2b(salt=salt1) + >>> h1.update(msg) + >>> # Calculate the second hash with a different random salt. + >>> salt2 = os.urandom(blake2b.SALT_SIZE) + >>> h2 = blake2b(salt=salt2) + >>> h2.update(msg) + >>> # The digests are different. + >>> h1.digest() != h2.digest() + True + + +Personalization +--------------- + +Sometimes it is useful to force hash function to produce different digests for +the same input for different purposes. Quoting the authors of the Skein hash +function: + + We recommend that all application designers seriously consider doing this; + we have seen many protocols where a hash that is computed in one part of + the protocol can be used in an entirely different part because two hash + computations were done on similar or related data, and the attacker can + force the application to make the hash inputs the same. Personalizing each + hash function used in the protocol summarily stops this type of attack. + + (`The Skein Hash Function Family + `_, + p. 21) + +BLAKE2 can be personalized by passing bytes to the *person* argument: + + >>> from hashlib import blake2b + >>> FILES_HASH_PERSON = b'MyApp Files Hash' + >>> BLOCK_HASH_PERSON = b'MyApp Block Hash' + >>> h = blake2b(digest_size=32, person=FILES_HASH_PERSON) + >>> h.update(b'the same content') + >>> h.hexdigest() + '20d9cd024d4fb086aae819a1432dd2466de12947831b75c5a30cf2676095d3b4' + >>> h = blake2b(digest_size=32, person=BLOCK_HASH_PERSON) + >>> h.update(b'the same content') + >>> h.hexdigest() + 'cf68fb5761b9c44e7878bfb2c4c9aea52264a80b75005e65619778de59f383a3' + +Personalization together with the keyed mode can also be used to derive different +keys from a single one. + + >>> from hashlib import blake2s + >>> from base64 import b64decode, b64encode + >>> orig_key = b64decode(b'Rm5EPJai72qcK3RGBpW3vPNfZy5OZothY+kHY6h21KM=') + >>> enc_key = blake2s(key=orig_key, person=b'kEncrypt').digest() + >>> mac_key = blake2s(key=orig_key, person=b'kMAC').digest() + >>> print(b64encode(enc_key).decode('utf-8')) + rbPb15S/Z9t+agffno5wuhB77VbRi6F9Iv2qIxU7WHw= + >>> print(b64encode(mac_key).decode('utf-8')) + G9GtHFE1YluXY1zWPlYk1e/nWfu0WSEb0KRcjhDeP/o= + +Tree mode +--------- + +Here's an example of hashing a minimal tree with two leaf nodes:: + + 10 + / \ + 00 01 + +The example uses 64-byte internal digests, and returns the 32-byte final +digest. + + >>> from hashlib import blake2b + >>> + >>> FANOUT = 2 + >>> DEPTH = 2 + >>> LEAF_SIZE = 4096 + >>> INNER_SIZE = 64 + >>> + >>> buf = bytearray(6000) + >>> + >>> # Left leaf + ... h00 = blake2b(buf[0:LEAF_SIZE], fanout=FANOUT, depth=DEPTH, + ... leaf_size=LEAF_SIZE, inner_size=INNER_SIZE, + ... node_offset=0, node_depth=0, last_node=False) + >>> # Right leaf + ... h01 = blake2b(buf[LEAF_SIZE:], fanout=FANOUT, depth=DEPTH, + ... leaf_size=LEAF_SIZE, inner_size=INNER_SIZE, + ... node_offset=1, node_depth=0, last_node=True) + >>> # Root node + ... h10 = blake2b(digest_size=32, fanout=FANOUT, depth=DEPTH, + ... leaf_size=LEAF_SIZE, inner_size=INNER_SIZE, + ... node_offset=0, node_depth=1, last_node=True) + >>> h10.update(h00.digest()) + >>> h10.update(h01.digest()) + >>> h10.hexdigest() + '3ad2a9b37c6070e374c7a8c508fe20ca86b6ed54e286e93a0318e95e881db5aa' + +Credits +======= + +BLAKE2_ was designed by *Jean-Philippe Aumasson*, *Samuel Neves*, *Zooko +Wilcox-O'Hearn*, and *Christian Winnerlein* based on SHA-3_ finalist BLAKE_ +created by *Jean-Philippe Aumasson*, *Luca Henzen*, *Willi Meier*, and +*Raphael C.-W. Phan*. + +It uses core algorithm from ChaCha_ cipher designed by *Daniel J. Bernstein*. + +The stdlib implementation is based on pyblake2_ module. It was written by +*Dmitry Chestnykh* based on C implementation written by *Samuel Neves*. The +documentation was copied from pyblake2_ and written by *Dmitry Chestnykh*. + +The C code was partly rewritten for Python by *Christian Heimes*. + +The following public domain dedication applies for both C hash function +implementation, extension code, and this documentation: + + To the extent possible under law, the author(s) have dedicated all copyright + and related and neighboring rights to this software to the public domain + worldwide. This software is distributed without any warranty. + + You should have received a copy of the CC0 Public Domain Dedication along + with this software. If not, see + http://creativecommons.org/publicdomain/zero/1.0/. + +The following people have helped with development or contributed their changes +to the project and the public domain according to the Creative Commons Public +Domain Dedication 1.0 Universal: + +* *Alexandr Sokolovskiy* + +.. seealso:: Official BLAKE2 website: https://blake2.net + +.. _BLAKE2: https://blake2.net +.. _HMAC: http://en.wikipedia.org/wiki/Hash-based_message_authentication_code +.. _BLAKE: https://131002.net/blake/ +.. _SHA-3: http://en.wikipedia.org/wiki/NIST_hash_function_competition +.. _ChaCha: http://cr.yp.to/chacha.html +.. _pyblake2: https://pythonhosted.org/pyblake2/ + diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -65,11 +65,15 @@ Constructors for hash algorithms that are always present in this module are :func:`sha1`, :func:`sha224`, :func:`sha256`, :func:`sha384`, -and :func:`sha512`. :func:`md5` is normally available as well, though it +:func:`sha512`, :func:`blake2b`, and :func:`blake2s`. +:func:`md5` is normally available as well, though it may be missing if you are using a rare "FIPS compliant" build of Python. Additional algorithms may also be available depending upon the OpenSSL library that Python uses on your platform. +.. versionadded:: 3.6 + :func:`blake2b` and :func:`blake2s` were added. + For example, to obtain the digest of the byte string ``b'Nobody inspects the spammish repetition'``:: @@ -243,6 +247,12 @@ .. versionadded:: 3.6 +BLAKE2 +------ + +BLAKE2 takes additional arguments, see :ref:`hashlib-blake2`. + + .. seealso:: Module :mod:`hmac` @@ -251,6 +261,8 @@ Module :mod:`base64` Another way to encode binary hashes for non-binary environments. + See :ref:`hashlib-blake2`. + http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf The FIPS 180-2 publication on Secure Hash Algorithms. diff --git a/Lib/hashlib.py b/Lib/hashlib.py --- a/Lib/hashlib.py +++ b/Lib/hashlib.py @@ -4,14 +4,14 @@ __doc__ = """hashlib module - A common interface to many hash functions. -new(name, data=b'') - returns a new hash object implementing the - given hash function; initializing the hash - using the given binary data. +new(name, data=b'', **kwargs) - returns a new hash object implementing the + given hash function; initializing the hash + using the given binary data. Named constructor functions are also available, these are faster than using new(name): -md5(), sha1(), sha224(), sha256(), sha384(), and sha512() +md5(), sha1(), sha224(), sha256(), sha384(), sha512(), blake2b(), and blake2s() More algorithms may be available on your platform but the above are guaranteed to exist. See the algorithms_guaranteed and algorithms_available attributes @@ -54,7 +54,8 @@ # This tuple and __get_builtin_constructor() must be modified if a new # always available algorithm is added. -__always_supported = ('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512') +__always_supported = ('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512', + 'blake2b', 'blake2s') algorithms_guaranteed = set(__always_supported) algorithms_available = set(__always_supported) @@ -85,6 +86,10 @@ import _sha512 cache['SHA384'] = cache['sha384'] = _sha512.sha384 cache['SHA512'] = cache['sha512'] = _sha512.sha512 + elif name in ('blake2b', 'blake2s'): + import _blake2 + cache['blake2b'] = _blake2.blake2b + cache['blake2s'] = _blake2.blake2s except ImportError: pass # no extension module, this hash is unsupported. @@ -107,17 +112,23 @@ return __get_builtin_constructor(name) -def __py_new(name, data=b''): +def __py_new(name, data=b'', **kwargs): + """new(name, data=b'', **kwargs) - Return a new hashing object using the + named algorithm; optionally initialized with data (which must be bytes). + """ + return __get_builtin_constructor(name)(data, **kwargs) + + +def __hash_new(name, data=b'', **kwargs): """new(name, data=b'') - Return a new hashing object using the named algorithm; optionally initialized with data (which must be bytes). """ - return __get_builtin_constructor(name)(data) - - -def __hash_new(name, data=b''): - """new(name, data=b'') - Return a new hashing object using the named algorithm; - optionally initialized with data (which must be bytes). - """ + if name in {'blake2b', 'blake2s'}: + # Prefer our blake2 implementation. + # OpenSSL 1.1.0 comes with a limited implementation of blake2b/s. + # It does neither support keyed blake2 nor advanced features like + # salt, personal, tree hashing or SSE. + return __get_builtin_constructor(name)(data, **kwargs) try: return _hashlib.new(name, data) except ValueError: @@ -218,6 +229,7 @@ import logging logging.exception('code for hash %s was not found.', __func_name) + # Cleanup locals() del __always_supported, __func_name, __get_hash del __py_new, __hash_new, __get_openssl_constructor diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py --- a/Lib/test/test_hashlib.py +++ b/Lib/test/test_hashlib.py @@ -27,6 +27,14 @@ c_hashlib = import_fresh_module('hashlib', fresh=['_hashlib']) py_hashlib = import_fresh_module('hashlib', blocked=['_hashlib']) +try: + import _blake2 +except ImportError: + _blake2 = None + +requires_blake2 = unittest.skipUnless(_blake2, 'requires _blake2') + + def hexstr(s): assert isinstance(s, bytes), repr(s) h = "0123456789abcdef" @@ -36,10 +44,24 @@ return r +URL = "https://raw.githubusercontent.com/tiran/python_vectors/master/{}.txt" + +def read_vectors(hash_name): + with support.open_urlresource(URL.format(hash_name)) as f: + for line in f: + line = line.strip() + if line.startswith('#') or not line: + continue + parts = line.split(',') + parts[0] = bytes.fromhex(parts[0]) + yield parts + + class HashLibTestCase(unittest.TestCase): supported_hash_names = ( 'md5', 'MD5', 'sha1', 'SHA1', 'sha224', 'SHA224', 'sha256', 'SHA256', - 'sha384', 'SHA384', 'sha512', 'SHA512') + 'sha384', 'SHA384', 'sha512', 'SHA512', + 'blake2b', 'blake2s') # Issue #14693: fallback modules are always compiled under POSIX _warn_on_extension_import = os.name == 'posix' or COMPILED_WITH_PYDEBUG @@ -57,6 +79,11 @@ algorithms = set() for algorithm in self.supported_hash_names: algorithms.add(algorithm.lower()) + + _blake2 = self._conditional_import_module('_blake2') + if _blake2: + algorithms.update({'blake2b', 'blake2s'}) + self.constructors_to_test = {} for algorithm in algorithms: self.constructors_to_test[algorithm] = set() @@ -65,10 +92,10 @@ # of hashlib.new given the algorithm name. for algorithm, constructors in self.constructors_to_test.items(): constructors.add(getattr(hashlib, algorithm)) - def _test_algorithm_via_hashlib_new(data=None, _alg=algorithm): + def _test_algorithm_via_hashlib_new(data=None, _alg=algorithm, **kwargs): if data is None: - return hashlib.new(_alg) - return hashlib.new(_alg, data) + return hashlib.new(_alg, **kwargs) + return hashlib.new(_alg, data, **kwargs) constructors.add(_test_algorithm_via_hashlib_new) _hashlib = self._conditional_import_module('_hashlib') @@ -100,6 +127,9 @@ if _sha512: add_builtin_constructor('sha384') add_builtin_constructor('sha512') + if _blake2: + add_builtin_constructor('blake2s') + add_builtin_constructor('blake2b') super(HashLibTestCase, self).__init__(*args, **kwargs) @@ -194,13 +224,13 @@ self.assertEqual(m1.digest(), m4_copy.digest()) self.assertEqual(m4.digest(), m4_digest) - def check(self, name, data, hexdigest): + def check(self, name, data, hexdigest, **kwargs): hexdigest = hexdigest.lower() constructors = self.constructors_to_test[name] # 2 is for hashlib.name(...) and hashlib.new(name, ...) self.assertGreaterEqual(len(constructors), 2) for hash_object_constructor in constructors: - m = hash_object_constructor(data) + m = hash_object_constructor(data, **kwargs) computed = m.hexdigest() self.assertEqual( computed, hexdigest, @@ -227,6 +257,11 @@ self.check_no_unicode('sha384') self.check_no_unicode('sha512') + @requires_blake2 + def test_no_unicode_blake2(self): + self.check_no_unicode('blake2b') + self.check_no_unicode('blake2s') + def check_blocksize_name(self, name, block_size=0, digest_size=0): constructors = self.constructors_to_test[name] for hash_object_constructor in constructors: @@ -246,6 +281,11 @@ self.check_blocksize_name('sha384', 128, 48) self.check_blocksize_name('sha512', 128, 64) + @requires_blake2 + def test_blocksize_name_blake2(self): + self.check_blocksize_name('blake2b', 128, 64) + self.check_blocksize_name('blake2s', 64, 32) + def test_case_md5_0(self): self.check('md5', b'', 'd41d8cd98f00b204e9800998ecf8427e') @@ -374,6 +414,155 @@ "e718483d0ce769644e2e42c7bc15b4638e1f98b13b2044285632a803afa973eb"+ "de0ff244877ea60a4cb0432ce577c31beb009c5c2c49aa2e4eadb217ad8cc09b") + def check_blake2(self, constructor, salt_size, person_size, key_size, + digest_size, max_offset): + self.assertEqual(constructor.SALT_SIZE, salt_size) + for i in range(salt_size + 1): + constructor(salt=b'a' * i) + salt = b'a' * (salt_size + 1) + self.assertRaises(ValueError, constructor, salt=salt) + + self.assertEqual(constructor.PERSON_SIZE, person_size) + for i in range(person_size+1): + constructor(person=b'a' * i) + person = b'a' * (person_size + 1) + self.assertRaises(ValueError, constructor, person=person) + + self.assertEqual(constructor.MAX_DIGEST_SIZE, digest_size) + for i in range(1, digest_size + 1): + constructor(digest_size=i) + self.assertRaises(ValueError, constructor, digest_size=-1) + self.assertRaises(ValueError, constructor, digest_size=0) + self.assertRaises(ValueError, constructor, digest_size=digest_size+1) + + self.assertEqual(constructor.MAX_KEY_SIZE, key_size) + for i in range(key_size+1): + constructor(key=b'a' * i) + key = b'a' * (key_size + 1) + self.assertRaises(ValueError, constructor, key=key) + self.assertEqual(constructor().hexdigest(), + constructor(key=b'').hexdigest()) + + for i in range(0, 256): + constructor(fanout=i) + self.assertRaises(ValueError, constructor, fanout=-1) + self.assertRaises(ValueError, constructor, fanout=256) + + for i in range(1, 256): + constructor(depth=i) + self.assertRaises(ValueError, constructor, depth=-1) + self.assertRaises(ValueError, constructor, depth=0) + self.assertRaises(ValueError, constructor, depth=256) + + for i in range(0, 256): + constructor(node_depth=i) + self.assertRaises(ValueError, constructor, node_depth=-1) + self.assertRaises(ValueError, constructor, node_depth=256) + + for i in range(0, digest_size + 1): + constructor(inner_size=i) + self.assertRaises(ValueError, constructor, inner_size=-1) + self.assertRaises(ValueError, constructor, inner_size=digest_size+1) + + constructor(leaf_size=0) + constructor(leaf_size=(1<<32)-1) + self.assertRaises(OverflowError, constructor, leaf_size=-1) + self.assertRaises(OverflowError, constructor, leaf_size=1<<32) + + constructor(node_offset=0) + constructor(node_offset=max_offset) + self.assertRaises(OverflowError, constructor, node_offset=-1) + self.assertRaises(OverflowError, constructor, node_offset=max_offset+1) + + constructor( + string=b'', + key=b'', + salt=b'', + person=b'', + digest_size=17, + fanout=1, + depth=1, + leaf_size=256, + node_offset=512, + node_depth=1, + inner_size=7, + last_node=True + ) + + def blake2_rfc7693(self, constructor, md_len, in_len): + def selftest_seq(length, seed): + mask = (1<<32)-1 + a = (0xDEAD4BAD * seed) & mask + b = 1 + out = bytearray(length) + for i in range(length): + t = (a + b) & mask + a, b = b, t + out[i] = (t >> 24) & 0xFF + return out + outer = constructor(digest_size=32) + for outlen in md_len: + for inlen in in_len: + indata = selftest_seq(inlen, inlen) + key = selftest_seq(outlen, outlen) + unkeyed = constructor(indata, digest_size=outlen) + outer.update(unkeyed.digest()) + keyed = constructor(indata, key=key, digest_size=outlen) + outer.update(keyed.digest()) + return outer.hexdigest() + + @requires_blake2 + def test_blake2b(self): + self.check_blake2(hashlib.blake2b, 16, 16, 64, 64, (1<<64)-1) + b2b_md_len = [20, 32, 48, 64] + b2b_in_len = [0, 3, 128, 129, 255, 1024] + self.assertEqual( + self.blake2_rfc7693(hashlib.blake2b, b2b_md_len, b2b_in_len), + "c23a7800d98123bd10f506c61e29da5603d763b8bbad2e737f5e765a7bccd475") + + @requires_blake2 + def test_case_blake2b_0(self): + self.check('blake2b', b"", + "786a02f742015903c6c6fd852552d272912f4740e15847618a86e217f71f5419"+ + "d25e1031afee585313896444934eb04b903a685b1448b755d56f701afe9be2ce") + + @requires_blake2 + def test_case_blake2b_1(self): + self.check('blake2b', b"abc", + "ba80a53f981c4d0d6a2797b69f12f6e94c212f14685ac4b74b12bb6fdbffa2d1"+ + "7d87c5392aab792dc252d5de4533cc9518d38aa8dbf1925ab92386edd4009923") + + @requires_blake2 + def test_blake2b_vectors(self): + for msg, key, md in read_vectors('blake2b'): + key = bytes.fromhex(key) + self.check('blake2b', msg, md, key=key) + + @requires_blake2 + def test_blake2s(self): + self.check_blake2(hashlib.blake2s, 8, 8, 32, 32, (1<<48)-1) + b2s_md_len = [16, 20, 28, 32] + b2s_in_len = [0, 3, 64, 65, 255, 1024] + self.assertEqual( + self.blake2_rfc7693(hashlib.blake2s, b2s_md_len, b2s_in_len), + "6a411f08ce25adcdfb02aba641451cec53c598b24f4fc787fbdc88797f4c1dfe") + + @requires_blake2 + def test_case_blake2s_0(self): + self.check('blake2s', b"", + "69217a3079908094e11121d042354a7c1f55b6482ca1a51e1b250dfd1ed0eef9") + + @requires_blake2 + def test_case_blake2s_1(self): + self.check('blake2s', b"abc", + "508c5e8c327c14e2e1a72ba34eeb452f37458b209ed63a294d999b4c86675982") + + @requires_blake2 + def test_blake2s_vectors(self): + for msg, key, md in read_vectors('blake2s'): + key = bytes.fromhex(key) + self.check('blake2s', msg, md, key=key) + def test_gil(self): # Check things work fine with an input larger than the size required # for multithreaded operation (which is hardwired to 2048). diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -541,7 +541,7 @@ # Run "Argument Clinic" over all source files # (depends on python having already been built) .PHONY=clinic -clinic: $(BUILDPYTHON) +clinic: $(BUILDPYTHON) Modules/_blake2/blake2s_impl.c $(RUNSHARED) $(PYTHON_FOR_BUILD) ./Tools/clinic/clinic.py --make # Build the interpreter @@ -571,6 +571,11 @@ Modules/_math.o: Modules/_math.c Modules/_math.h $(CC) -c $(CCSHARED) $(PY_CORE_CFLAGS) -o $@ $< +# blake2s is auto-generated from blake2b +Modules/_blake2/blake2s_impl.c: $(BUILDPYTHON) Modules/_blake2/blake2b_impl.c Modules/_blake2/blake2b2s.py + $(RUNSHARED) $(PYTHON_FOR_BUILD) Modules/_blake2/blake2b2s.py + $(RUNSHARED) $(PYTHON_FOR_BUILD) Tools/clinic/clinic.py -f $@ + # Build the shared modules # Under GNU make, MAKEFLAGS are sorted and normalized; the 's' for # -s, --silent or --quiet is always the first char. @@ -584,6 +589,7 @@ _TCLTK_INCLUDES='$(TCLTK_INCLUDES)' _TCLTK_LIBS='$(TCLTK_LIBS)' \ $(PYTHON_FOR_BUILD) $(srcdir)/setup.py $$quiet build + # Build static library # avoid long command lines, same as LIBRARY_OBJS $(LIBRARY): $(LIBRARY_OBJS) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -89,6 +89,8 @@ Library ------- +- Issue #26798: Add BLAKE2 (blake2b and blake2s) to hashlib. + - Issue #25596: Optimized glob() and iglob() functions in the glob module; they are now about 3--6 times faster. diff --git a/Modules/_blake2/blake2b2s.py b/Modules/_blake2/blake2b2s.py new file mode 100755 --- /dev/null +++ b/Modules/_blake2/blake2b2s.py @@ -0,0 +1,49 @@ +#!/usr/bin/python3 + +import os +import re + +HERE = os.path.dirname(os.path.abspath(__file__)) +BLAKE2 = os.path.join(HERE, 'impl') + +PUBLIC_SEARCH = re.compile(r'\ int (blake2[bs]p?[a-z_]*)\(') + + +def getfiles(): + for name in os.listdir(BLAKE2): + name = os.path.join(BLAKE2, name) + if os.path.isfile(name): + yield name + + +def find_public(): + public_funcs = set() + for name in getfiles(): + with open(name) as f: + for line in f: + # find public functions + mo = PUBLIC_SEARCH.search(line) + if mo: + public_funcs.add(mo.group(1)) + + for f in sorted(public_funcs): + print('#define {0:<18} PyBlake2_{0}'.format(f)) + + return public_funcs + + +def main(): + lines = [] + with open(os.path.join(HERE, 'blake2b_impl.c')) as f: + for line in f: + line = line.replace('blake2b', 'blake2s') + line = line.replace('BLAKE2b', 'BLAKE2s') + line = line.replace('BLAKE2B', 'BLAKE2S') + lines.append(line) + with open(os.path.join(HERE, 'blake2s_impl.c'), 'w') as f: + f.write(''.join(lines)) + # find_public() + + +if __name__ == '__main__': + main() diff --git a/Modules/_blake2/blake2b_impl.c b/Modules/_blake2/blake2b_impl.c new file mode 100644 --- /dev/null +++ b/Modules/_blake2/blake2b_impl.c @@ -0,0 +1,460 @@ +/* + * Written in 2013 by Dmitry Chestnykh + * Modified for CPython by Christian Heimes + * + * To the extent possible under law, the author have dedicated all + * copyright and related and neighboring rights to this software to + * the public domain worldwide. This software is distributed without + * any warranty. http://creativecommons.org/publicdomain/zero/1.0/ + */ + +/* WARNING: autogenerated file! + * + * The blake2s_impl.c is autogenerated from blake2b_impl.c. + */ + +#include "Python.h" +#include "pystrhex.h" +#ifdef WITH_THREAD +#include "pythread.h" +#endif + +#include "../hashlib.h" +#include "blake2ns.h" + +#define HAVE_BLAKE2B 1 +#define BLAKE2_LOCAL_INLINE(type) Py_LOCAL_INLINE(type) + +#include "impl/blake2.h" +#include "impl/blake2-impl.h" /* for secure_zero_memory() and store48() */ + +#ifdef BLAKE2_USE_SSE +#include "impl/blake2b.c" +#else +#include "impl/blake2b-ref.c" +#endif + + +extern PyTypeObject PyBlake2_BLAKE2bType; + +typedef struct { + PyObject_HEAD + blake2b_param param; + blake2b_state state; +#ifdef WITH_THREAD + PyThread_type_lock lock; +#endif +} BLAKE2bObject; + +#include "clinic/blake2b_impl.c.h" + +/*[clinic input] +module _blake2b +class _blake2b.blake2b "BLAKE2bObject *" "&PyBlake2_BLAKE2bType" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=6893358c6622aecf]*/ + + +static BLAKE2bObject * +new_BLAKE2bObject(PyTypeObject *type) +{ + BLAKE2bObject *self; + self = (BLAKE2bObject *)type->tp_alloc(type, 0); +#ifdef WITH_THREAD + if (self != NULL) { + self->lock = NULL; + } +#endif + return self; +} + +/*[clinic input] + at classmethod +_blake2b.blake2b.__new__ as py_blake2b_new + string as data: object = NULL + * + digest_size: int(c_default="BLAKE2B_OUTBYTES") = _blake2b.blake2b.MAX_DIGEST_SIZE + key: Py_buffer = None + salt: Py_buffer = None + person: Py_buffer = None + fanout: int = 1 + depth: int = 1 + leaf_size as leaf_size_obj: object = NULL + node_offset as node_offset_obj: object = NULL + node_depth: int = 0 + inner_size: int = 0 + last_node: bool = False + +Return a new BLAKE2b hash object. +[clinic start generated code]*/ + +static PyObject * +py_blake2b_new_impl(PyTypeObject *type, PyObject *data, int digest_size, + Py_buffer *key, Py_buffer *salt, Py_buffer *person, + int fanout, int depth, PyObject *leaf_size_obj, + PyObject *node_offset_obj, int node_depth, + int inner_size, int last_node) +/*[clinic end generated code: output=7506d8d890e5f13b input=e41548dfa0866031]*/ +{ + BLAKE2bObject *self = NULL; + Py_buffer buf; + + unsigned long leaf_size = 0; + unsigned PY_LONG_LONG node_offset = 0; + + self = new_BLAKE2bObject(type); + if (self == NULL) { + goto error; + } + + /* Zero parameter block. */ + memset(&self->param, 0, sizeof(self->param)); + + /* Set digest size. */ + if (digest_size <= 0 || digest_size > BLAKE2B_OUTBYTES) { + PyErr_Format(PyExc_ValueError, + "digest_size must be between 1 and %d bytes", + BLAKE2B_OUTBYTES); + goto error; + } + self->param.digest_length = digest_size; + + /* Set salt parameter. */ + if ((salt->obj != NULL) && salt->len) { + if (salt->len > BLAKE2B_SALTBYTES) { + PyErr_Format(PyExc_ValueError, + "maximum salt length is %d bytes", + BLAKE2B_SALTBYTES); + goto error; + } + memcpy(self->param.salt, salt->buf, salt->len); + } + + /* Set personalization parameter. */ + if ((person->obj != NULL) && person->len) { + if (person->len > BLAKE2B_PERSONALBYTES) { + PyErr_Format(PyExc_ValueError, + "maximum person length is %d bytes", + BLAKE2B_PERSONALBYTES); + goto error; + } + memcpy(self->param.personal, person->buf, person->len); + } + + /* Set tree parameters. */ + if (fanout < 0 || fanout > 255) { + PyErr_SetString(PyExc_ValueError, + "fanout must be between 0 and 255"); + goto error; + } + self->param.fanout = (uint8_t)fanout; + + if (depth <= 0 || depth > 255) { + PyErr_SetString(PyExc_ValueError, + "depth must be between 1 and 255"); + goto error; + } + self->param.depth = (uint8_t)depth; + + if (leaf_size_obj != NULL) { + leaf_size = PyLong_AsUnsignedLong(leaf_size_obj); + if (leaf_size == (unsigned long) -1 && PyErr_Occurred()) { + goto error; + } + if (leaf_size > 0xFFFFFFFFU) { + PyErr_SetString(PyExc_OverflowError, "leaf_size is too large"); + goto error; + } + } + self->param.leaf_length = (unsigned int)leaf_size; + + if (node_offset_obj != NULL) { + node_offset = PyLong_AsUnsignedLongLong(node_offset_obj); + if (node_offset == (unsigned PY_LONG_LONG) -1 && PyErr_Occurred()) { + goto error; + } + } +#ifdef HAVE_BLAKE2S + if (node_offset > 0xFFFFFFFFFFFFULL) { + /* maximum 2**48 - 1 */ + PyErr_SetString(PyExc_OverflowError, "node_offset is too large"); + goto error; + } + store48(&(self->param.node_offset), node_offset); +#else + self->param.node_offset = node_offset; +#endif + + if (node_depth < 0 || node_depth > 255) { + PyErr_SetString(PyExc_ValueError, + "node_depth must be between 0 and 255"); + goto error; + } + self->param.node_depth = node_depth; + + if (inner_size < 0 || inner_size > BLAKE2B_OUTBYTES) { + PyErr_Format(PyExc_ValueError, + "inner_size must be between 0 and is %d", + BLAKE2B_OUTBYTES); + goto error; + } + self->param.inner_length = inner_size; + + /* Set key length. */ + if ((key->obj != NULL) && key->len) { + if (key->len > BLAKE2B_KEYBYTES) { + PyErr_Format(PyExc_ValueError, + "maximum key length is %d bytes", + BLAKE2B_KEYBYTES); + goto error; + } + self->param.key_length = key->len; + } + + /* Initialize hash state. */ + if (blake2b_init_param(&self->state, &self->param) < 0) { + PyErr_SetString(PyExc_RuntimeError, + "error initializing hash state"); + goto error; + } + + /* Set last node flag (must come after initialization). */ + self->state.last_node = last_node; + + /* Process key block if any. */ + if (self->param.key_length) { + uint8_t block[BLAKE2B_BLOCKBYTES]; + memset(block, 0, sizeof(block)); + memcpy(block, key->buf, key->len); + blake2b_update(&self->state, block, sizeof(block)); + secure_zero_memory(block, sizeof(block)); + } + + /* Process initial data if any. */ + if (data != NULL) { + GET_BUFFER_VIEW_OR_ERROR(data, &buf, goto error); + + if (buf.len >= HASHLIB_GIL_MINSIZE) { + Py_BEGIN_ALLOW_THREADS + blake2b_update(&self->state, buf.buf, buf.len); + Py_END_ALLOW_THREADS + } else { + blake2b_update(&self->state, buf.buf, buf.len); + } + PyBuffer_Release(&buf); + } + + return (PyObject *)self; + + error: + if (self != NULL) { + Py_DECREF(self); + } + return NULL; +} + +/*[clinic input] +_blake2b.blake2b.copy + +Return a copy of the hash object. +[clinic start generated code]*/ + +static PyObject * +_blake2b_blake2b_copy_impl(BLAKE2bObject *self) +/*[clinic end generated code: output=c89cd33550ab1543 input=4c9c319f18f10747]*/ +{ + BLAKE2bObject *cpy; + + if ((cpy = new_BLAKE2bObject(Py_TYPE(self))) == NULL) + return NULL; + + ENTER_HASHLIB(self); + cpy->param = self->param; + cpy->state = self->state; + LEAVE_HASHLIB(self); + return (PyObject *)cpy; +} + +/*[clinic input] +_blake2b.blake2b.update + + obj: object + / + +Update this hash object's state with the provided string. +[clinic start generated code]*/ + +static PyObject * +_blake2b_blake2b_update(BLAKE2bObject *self, PyObject *obj) +/*[clinic end generated code: output=a888f07c4cddbe94 input=3ecb8c13ee4260f2]*/ +{ + Py_buffer buf; + + GET_BUFFER_VIEW_OR_ERROUT(obj, &buf); + +#ifdef WITH_THREAD + if (self->lock == NULL && buf.len >= HASHLIB_GIL_MINSIZE) + self->lock = PyThread_allocate_lock(); + + if (self->lock != NULL) { + Py_BEGIN_ALLOW_THREADS + PyThread_acquire_lock(self->lock, 1); + blake2b_update(&self->state, buf.buf, buf.len); + PyThread_release_lock(self->lock); + Py_END_ALLOW_THREADS + } else { + blake2b_update(&self->state, buf.buf, buf.len); + } +#else + blake2b_update(&self->state, buf.buf, buf.len); +#endif /* !WITH_THREAD */ + PyBuffer_Release(&buf); + + Py_INCREF(Py_None); + return Py_None; +} + +/*[clinic input] +_blake2b.blake2b.digest + +Return the digest value as a string of binary data. +[clinic start generated code]*/ + +static PyObject * +_blake2b_blake2b_digest_impl(BLAKE2bObject *self) +/*[clinic end generated code: output=b13a79360d984740 input=ac2fa462ebb1b9c7]*/ +{ + uint8_t digest[BLAKE2B_OUTBYTES]; + blake2b_state state_cpy; + + ENTER_HASHLIB(self); + state_cpy = self->state; + blake2b_final(&state_cpy, digest, self->param.digest_length); + LEAVE_HASHLIB(self); + return PyBytes_FromStringAndSize((const char *)digest, + self->param.digest_length); +} + +/*[clinic input] +_blake2b.blake2b.hexdigest + +Return the digest value as a string of hexadecimal digits. +[clinic start generated code]*/ + +static PyObject * +_blake2b_blake2b_hexdigest_impl(BLAKE2bObject *self) +/*[clinic end generated code: output=6a503611715b24bd input=d58f0b2f37812e33]*/ +{ + uint8_t digest[BLAKE2B_OUTBYTES]; + blake2b_state state_cpy; + + ENTER_HASHLIB(self); + state_cpy = self->state; + blake2b_final(&state_cpy, digest, self->param.digest_length); + LEAVE_HASHLIB(self); + return _Py_strhex((const char *)digest, self->param.digest_length); +} + + +static PyMethodDef py_blake2b_methods[] = { + _BLAKE2B_BLAKE2B_COPY_METHODDEF + _BLAKE2B_BLAKE2B_DIGEST_METHODDEF + _BLAKE2B_BLAKE2B_HEXDIGEST_METHODDEF + _BLAKE2B_BLAKE2B_UPDATE_METHODDEF + {NULL, NULL} +}; + + + +static PyObject * +py_blake2b_get_name(BLAKE2bObject *self, void *closure) +{ + return PyUnicode_FromString("blake2b"); +} + + + +static PyObject * +py_blake2b_get_block_size(BLAKE2bObject *self, void *closure) +{ + return PyLong_FromLong(BLAKE2B_BLOCKBYTES); +} + + + +static PyObject * +py_blake2b_get_digest_size(BLAKE2bObject *self, void *closure) +{ + return PyLong_FromLong(self->param.digest_length); +} + + +static PyGetSetDef py_blake2b_getsetters[] = { + {"name", (getter)py_blake2b_get_name, + NULL, NULL, NULL}, + {"block_size", (getter)py_blake2b_get_block_size, + NULL, NULL, NULL}, + {"digest_size", (getter)py_blake2b_get_digest_size, + NULL, NULL, NULL}, + {NULL} +}; + + +static void +py_blake2b_dealloc(PyObject *self) +{ + BLAKE2bObject *obj = (BLAKE2bObject *)self; + + /* Try not to leave state in memory. */ + secure_zero_memory(&obj->param, sizeof(obj->param)); + secure_zero_memory(&obj->state, sizeof(obj->state)); +#ifdef WITH_THREAD + if (obj->lock) { + PyThread_free_lock(obj->lock); + obj->lock = NULL; + } +#endif + PyObject_Del(self); +} + + +PyTypeObject PyBlake2_BLAKE2bType = { + PyVarObject_HEAD_INIT(NULL, 0) + "_blake2.blake2b", /* tp_name */ + sizeof(BLAKE2bObject), /* tp_size */ + 0, /* tp_itemsize */ + py_blake2b_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /* tp_flags */ + py_blake2b_new__doc__, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + py_blake2b_methods, /* tp_methods */ + 0, /* tp_members */ + py_blake2b_getsetters, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + py_blake2b_new, /* tp_new */ +}; diff --git a/Modules/_blake2/blake2module.c b/Modules/_blake2/blake2module.c new file mode 100644 --- /dev/null +++ b/Modules/_blake2/blake2module.c @@ -0,0 +1,105 @@ +/* + * Written in 2013 by Dmitry Chestnykh + * Modified for CPython by Christian Heimes + * + * To the extent possible under law, the author have dedicated all + * copyright and related and neighboring rights to this software to + * the public domain worldwide. This software is distributed without + * any warranty. http://creativecommons.org/publicdomain/zero/1.0/ + */ + +#include "Python.h" + +#include "impl/blake2.h" + +extern PyTypeObject PyBlake2_BLAKE2bType; +extern PyTypeObject PyBlake2_BLAKE2sType; + + +PyDoc_STRVAR(blake2mod__doc__, +"_blake2b provides BLAKE2b for hashlib\n" +); + + +static struct PyMethodDef blake2mod_functions[] = { + {NULL, NULL} +}; + +static struct PyModuleDef blake2_module = { + PyModuleDef_HEAD_INIT, + "_blake2", + blake2mod__doc__, + -1, + blake2mod_functions, + NULL, + NULL, + NULL, + NULL +}; + +#define ADD_INT(d, name, value) do { \ + PyObject *x = PyLong_FromLong(value); \ + if (!x) { \ + Py_DECREF(m); \ + return NULL; \ + } \ + if (PyDict_SetItemString(d, name, x) < 0) { \ + Py_DECREF(m); \ + return NULL; \ + } \ + Py_DECREF(x); \ +} while(0) + + +PyMODINIT_FUNC +PyInit__blake2(void) +{ + PyObject *m; + PyObject *d; + + m = PyModule_Create(&blake2_module); + if (m == NULL) + return NULL; + + /* BLAKE2b */ + Py_TYPE(&PyBlake2_BLAKE2bType) = &PyType_Type; + if (PyType_Ready(&PyBlake2_BLAKE2bType) < 0) { + return NULL; + } + + Py_INCREF(&PyBlake2_BLAKE2bType); + PyModule_AddObject(m, "blake2b", (PyObject *)&PyBlake2_BLAKE2bType); + + d = PyBlake2_BLAKE2bType.tp_dict; + ADD_INT(d, "SALT_SIZE", BLAKE2B_SALTBYTES); + ADD_INT(d, "PERSON_SIZE", BLAKE2B_PERSONALBYTES); + ADD_INT(d, "MAX_KEY_SIZE", BLAKE2B_KEYBYTES); + ADD_INT(d, "MAX_DIGEST_SIZE", BLAKE2B_OUTBYTES); + + PyModule_AddIntConstant(m, "BLAKE2B_SALT_SIZE", BLAKE2B_SALTBYTES); + PyModule_AddIntConstant(m, "BLAKE2B_PERSON_SIZE", BLAKE2B_PERSONALBYTES); + PyModule_AddIntConstant(m, "BLAKE2B_MAX_KEY_SIZE", BLAKE2B_KEYBYTES); + PyModule_AddIntConstant(m, "BLAKE2B_MAX_DIGEST_SIZE", BLAKE2B_OUTBYTES); + + /* BLAKE2s */ + Py_TYPE(&PyBlake2_BLAKE2sType) = &PyType_Type; + if (PyType_Ready(&PyBlake2_BLAKE2sType) < 0) { + return NULL; + } + + Py_INCREF(&PyBlake2_BLAKE2sType); + PyModule_AddObject(m, "blake2s", (PyObject *)&PyBlake2_BLAKE2sType); + + d = PyBlake2_BLAKE2sType.tp_dict; + ADD_INT(d, "SALT_SIZE", BLAKE2S_SALTBYTES); + ADD_INT(d, "PERSON_SIZE", BLAKE2S_PERSONALBYTES); + ADD_INT(d, "MAX_KEY_SIZE", BLAKE2S_KEYBYTES); + ADD_INT(d, "MAX_DIGEST_SIZE", BLAKE2S_OUTBYTES); + + PyModule_AddIntConstant(m, "BLAKE2S_SALT_SIZE", BLAKE2S_SALTBYTES); + PyModule_AddIntConstant(m, "BLAKE2S_PERSON_SIZE", BLAKE2S_PERSONALBYTES); + PyModule_AddIntConstant(m, "BLAKE2S_MAX_KEY_SIZE", BLAKE2S_KEYBYTES); + PyModule_AddIntConstant(m, "BLAKE2S_MAX_DIGEST_SIZE", BLAKE2S_OUTBYTES); + + return m; +} diff --git a/Modules/_blake2/blake2ns.h b/Modules/_blake2/blake2ns.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/blake2ns.h @@ -0,0 +1,32 @@ +/* Prefix all public blake2 symbols with PyBlake2_ + */ + +#ifndef Py_BLAKE2_NS +#define Py_BLAKE2_NS + +#define blake2b PyBlake2_blake2b +#define blake2b_compress PyBlake2_blake2b_compress +#define blake2b_final PyBlake2_blake2b_final +#define blake2b_init PyBlake2_blake2b_init +#define blake2b_init_key PyBlake2_blake2b_init_key +#define blake2b_init_param PyBlake2_blake2b_init_param +#define blake2b_update PyBlake2_blake2b_update +#define blake2bp PyBlake2_blake2bp +#define blake2bp_final PyBlake2_blake2bp_final +#define blake2bp_init PyBlake2_blake2bp_init +#define blake2bp_init_key PyBlake2_blake2bp_init_key +#define blake2bp_update PyBlake2_blake2bp_update +#define blake2s PyBlake2_blake2s +#define blake2s_compress PyBlake2_blake2s_compress +#define blake2s_final PyBlake2_blake2s_final +#define blake2s_init PyBlake2_blake2s_init +#define blake2s_init_key PyBlake2_blake2s_init_key +#define blake2s_init_param PyBlake2_blake2s_init_param +#define blake2s_update PyBlake2_blake2s_update +#define blake2sp PyBlake2_blake2sp +#define blake2sp_final PyBlake2_blake2sp_final +#define blake2sp_init PyBlake2_blake2sp_init +#define blake2sp_init_key PyBlake2_blake2sp_init_key +#define blake2sp_update PyBlake2_blake2sp_update + +#endif /* Py_BLAKE2_NS */ diff --git a/Modules/_blake2/blake2s_impl.c b/Modules/_blake2/blake2s_impl.c new file mode 100644 --- /dev/null +++ b/Modules/_blake2/blake2s_impl.c @@ -0,0 +1,460 @@ +/* + * Written in 2013 by Dmitry Chestnykh + * Modified for CPython by Christian Heimes + * + * To the extent possible under law, the author have dedicated all + * copyright and related and neighboring rights to this software to + * the public domain worldwide. This software is distributed without + * any warranty. http://creativecommons.org/publicdomain/zero/1.0/ + */ + +/* WARNING: autogenerated file! + * + * The blake2s_impl.c is autogenerated from blake2s_impl.c. + */ + +#include "Python.h" +#include "pystrhex.h" +#ifdef WITH_THREAD +#include "pythread.h" +#endif + +#include "../hashlib.h" +#include "blake2ns.h" + +#define HAVE_BLAKE2S 1 +#define BLAKE2_LOCAL_INLINE(type) Py_LOCAL_INLINE(type) + +#include "impl/blake2.h" +#include "impl/blake2-impl.h" /* for secure_zero_memory() and store48() */ + +#ifdef BLAKE2_USE_SSE +#include "impl/blake2s.c" +#else +#include "impl/blake2s-ref.c" +#endif + + +extern PyTypeObject PyBlake2_BLAKE2sType; + +typedef struct { + PyObject_HEAD + blake2s_param param; + blake2s_state state; +#ifdef WITH_THREAD + PyThread_type_lock lock; +#endif +} BLAKE2sObject; + +#include "clinic/blake2s_impl.c.h" + +/*[clinic input] +module _blake2s +class _blake2s.blake2s "BLAKE2sObject *" "&PyBlake2_BLAKE2sType" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=edbfcf7557a685a7]*/ + + +static BLAKE2sObject * +new_BLAKE2sObject(PyTypeObject *type) +{ + BLAKE2sObject *self; + self = (BLAKE2sObject *)type->tp_alloc(type, 0); +#ifdef WITH_THREAD + if (self != NULL) { + self->lock = NULL; + } +#endif + return self; +} + +/*[clinic input] + at classmethod +_blake2s.blake2s.__new__ as py_blake2s_new + string as data: object = NULL + * + digest_size: int(c_default="BLAKE2S_OUTBYTES") = _blake2s.blake2s.MAX_DIGEST_SIZE + key: Py_buffer = None + salt: Py_buffer = None + person: Py_buffer = None + fanout: int = 1 + depth: int = 1 + leaf_size as leaf_size_obj: object = NULL + node_offset as node_offset_obj: object = NULL + node_depth: int = 0 + inner_size: int = 0 + last_node: bool = False + +Return a new BLAKE2s hash object. +[clinic start generated code]*/ + +static PyObject * +py_blake2s_new_impl(PyTypeObject *type, PyObject *data, int digest_size, + Py_buffer *key, Py_buffer *salt, Py_buffer *person, + int fanout, int depth, PyObject *leaf_size_obj, + PyObject *node_offset_obj, int node_depth, + int inner_size, int last_node) +/*[clinic end generated code: output=fe060b258a8cbfc6 input=458cfdcb3d0d47ff]*/ +{ + BLAKE2sObject *self = NULL; + Py_buffer buf; + + unsigned long leaf_size = 0; + unsigned PY_LONG_LONG node_offset = 0; + + self = new_BLAKE2sObject(type); + if (self == NULL) { + goto error; + } + + /* Zero parameter block. */ + memset(&self->param, 0, sizeof(self->param)); + + /* Set digest size. */ + if (digest_size <= 0 || digest_size > BLAKE2S_OUTBYTES) { + PyErr_Format(PyExc_ValueError, + "digest_size must be between 1 and %d bytes", + BLAKE2S_OUTBYTES); + goto error; + } + self->param.digest_length = digest_size; + + /* Set salt parameter. */ + if ((salt->obj != NULL) && salt->len) { + if (salt->len > BLAKE2S_SALTBYTES) { + PyErr_Format(PyExc_ValueError, + "maximum salt length is %d bytes", + BLAKE2S_SALTBYTES); + goto error; + } + memcpy(self->param.salt, salt->buf, salt->len); + } + + /* Set personalization parameter. */ + if ((person->obj != NULL) && person->len) { + if (person->len > BLAKE2S_PERSONALBYTES) { + PyErr_Format(PyExc_ValueError, + "maximum person length is %d bytes", + BLAKE2S_PERSONALBYTES); + goto error; + } + memcpy(self->param.personal, person->buf, person->len); + } + + /* Set tree parameters. */ + if (fanout < 0 || fanout > 255) { + PyErr_SetString(PyExc_ValueError, + "fanout must be between 0 and 255"); + goto error; + } + self->param.fanout = (uint8_t)fanout; + + if (depth <= 0 || depth > 255) { + PyErr_SetString(PyExc_ValueError, + "depth must be between 1 and 255"); + goto error; + } + self->param.depth = (uint8_t)depth; + + if (leaf_size_obj != NULL) { + leaf_size = PyLong_AsUnsignedLong(leaf_size_obj); + if (leaf_size == (unsigned long) -1 && PyErr_Occurred()) { + goto error; + } + if (leaf_size > 0xFFFFFFFFU) { + PyErr_SetString(PyExc_OverflowError, "leaf_size is too large"); + goto error; + } + } + self->param.leaf_length = (unsigned int)leaf_size; + + if (node_offset_obj != NULL) { + node_offset = PyLong_AsUnsignedLongLong(node_offset_obj); + if (node_offset == (unsigned PY_LONG_LONG) -1 && PyErr_Occurred()) { + goto error; + } + } +#ifdef HAVE_BLAKE2S + if (node_offset > 0xFFFFFFFFFFFFULL) { + /* maximum 2**48 - 1 */ + PyErr_SetString(PyExc_OverflowError, "node_offset is too large"); + goto error; + } + store48(&(self->param.node_offset), node_offset); +#else + self->param.node_offset = node_offset; +#endif + + if (node_depth < 0 || node_depth > 255) { + PyErr_SetString(PyExc_ValueError, + "node_depth must be between 0 and 255"); + goto error; + } + self->param.node_depth = node_depth; + + if (inner_size < 0 || inner_size > BLAKE2S_OUTBYTES) { + PyErr_Format(PyExc_ValueError, + "inner_size must be between 0 and is %d", + BLAKE2S_OUTBYTES); + goto error; + } + self->param.inner_length = inner_size; + + /* Set key length. */ + if ((key->obj != NULL) && key->len) { + if (key->len > BLAKE2S_KEYBYTES) { + PyErr_Format(PyExc_ValueError, + "maximum key length is %d bytes", + BLAKE2S_KEYBYTES); + goto error; + } + self->param.key_length = key->len; + } + + /* Initialize hash state. */ + if (blake2s_init_param(&self->state, &self->param) < 0) { + PyErr_SetString(PyExc_RuntimeError, + "error initializing hash state"); + goto error; + } + + /* Set last node flag (must come after initialization). */ + self->state.last_node = last_node; + + /* Process key block if any. */ + if (self->param.key_length) { + uint8_t block[BLAKE2S_BLOCKBYTES]; + memset(block, 0, sizeof(block)); + memcpy(block, key->buf, key->len); + blake2s_update(&self->state, block, sizeof(block)); + secure_zero_memory(block, sizeof(block)); + } + + /* Process initial data if any. */ + if (data != NULL) { + GET_BUFFER_VIEW_OR_ERROR(data, &buf, goto error); + + if (buf.len >= HASHLIB_GIL_MINSIZE) { + Py_BEGIN_ALLOW_THREADS + blake2s_update(&self->state, buf.buf, buf.len); + Py_END_ALLOW_THREADS + } else { + blake2s_update(&self->state, buf.buf, buf.len); + } + PyBuffer_Release(&buf); + } + + return (PyObject *)self; + + error: + if (self != NULL) { + Py_DECREF(self); + } + return NULL; +} + +/*[clinic input] +_blake2s.blake2s.copy + +Return a copy of the hash object. +[clinic start generated code]*/ + +static PyObject * +_blake2s_blake2s_copy_impl(BLAKE2sObject *self) +/*[clinic end generated code: output=6c5bada404b7aed7 input=c8858e887ae4a07a]*/ +{ + BLAKE2sObject *cpy; + + if ((cpy = new_BLAKE2sObject(Py_TYPE(self))) == NULL) + return NULL; + + ENTER_HASHLIB(self); + cpy->param = self->param; + cpy->state = self->state; + LEAVE_HASHLIB(self); + return (PyObject *)cpy; +} + +/*[clinic input] +_blake2s.blake2s.update + + obj: object + / + +Update this hash object's state with the provided string. +[clinic start generated code]*/ + +static PyObject * +_blake2s_blake2s_update(BLAKE2sObject *self, PyObject *obj) +/*[clinic end generated code: output=fe8438a1d3cede87 input=47a408b9a3cc05c5]*/ +{ + Py_buffer buf; + + GET_BUFFER_VIEW_OR_ERROUT(obj, &buf); + +#ifdef WITH_THREAD + if (self->lock == NULL && buf.len >= HASHLIB_GIL_MINSIZE) + self->lock = PyThread_allocate_lock(); + + if (self->lock != NULL) { + Py_BEGIN_ALLOW_THREADS + PyThread_acquire_lock(self->lock, 1); + blake2s_update(&self->state, buf.buf, buf.len); + PyThread_release_lock(self->lock); + Py_END_ALLOW_THREADS + } else { + blake2s_update(&self->state, buf.buf, buf.len); + } +#else + blake2s_update(&self->state, buf.buf, buf.len); +#endif /* !WITH_THREAD */ + PyBuffer_Release(&buf); + + Py_INCREF(Py_None); + return Py_None; +} + +/*[clinic input] +_blake2s.blake2s.digest + +Return the digest value as a string of binary data. +[clinic start generated code]*/ + +static PyObject * +_blake2s_blake2s_digest_impl(BLAKE2sObject *self) +/*[clinic end generated code: output=80e81a48c6f79cf9 input=feb9a220135bdeba]*/ +{ + uint8_t digest[BLAKE2S_OUTBYTES]; + blake2s_state state_cpy; + + ENTER_HASHLIB(self); + state_cpy = self->state; + blake2s_final(&state_cpy, digest, self->param.digest_length); + LEAVE_HASHLIB(self); + return PyBytes_FromStringAndSize((const char *)digest, + self->param.digest_length); +} + +/*[clinic input] +_blake2s.blake2s.hexdigest + +Return the digest value as a string of hexadecimal digits. +[clinic start generated code]*/ + +static PyObject * +_blake2s_blake2s_hexdigest_impl(BLAKE2sObject *self) +/*[clinic end generated code: output=db6c5028c0a3c2e5 input=4e4877b8bd7aea91]*/ +{ + uint8_t digest[BLAKE2S_OUTBYTES]; + blake2s_state state_cpy; + + ENTER_HASHLIB(self); + state_cpy = self->state; + blake2s_final(&state_cpy, digest, self->param.digest_length); + LEAVE_HASHLIB(self); + return _Py_strhex((const char *)digest, self->param.digest_length); +} + + +static PyMethodDef py_blake2s_methods[] = { + _BLAKE2S_BLAKE2S_COPY_METHODDEF + _BLAKE2S_BLAKE2S_DIGEST_METHODDEF + _BLAKE2S_BLAKE2S_HEXDIGEST_METHODDEF + _BLAKE2S_BLAKE2S_UPDATE_METHODDEF + {NULL, NULL} +}; + + + +static PyObject * +py_blake2s_get_name(BLAKE2sObject *self, void *closure) +{ + return PyUnicode_FromString("blake2s"); +} + + + +static PyObject * +py_blake2s_get_block_size(BLAKE2sObject *self, void *closure) +{ + return PyLong_FromLong(BLAKE2S_BLOCKBYTES); +} + + + +static PyObject * +py_blake2s_get_digest_size(BLAKE2sObject *self, void *closure) +{ + return PyLong_FromLong(self->param.digest_length); +} + + +static PyGetSetDef py_blake2s_getsetters[] = { + {"name", (getter)py_blake2s_get_name, + NULL, NULL, NULL}, + {"block_size", (getter)py_blake2s_get_block_size, + NULL, NULL, NULL}, + {"digest_size", (getter)py_blake2s_get_digest_size, + NULL, NULL, NULL}, + {NULL} +}; + + +static void +py_blake2s_dealloc(PyObject *self) +{ + BLAKE2sObject *obj = (BLAKE2sObject *)self; + + /* Try not to leave state in memory. */ + secure_zero_memory(&obj->param, sizeof(obj->param)); + secure_zero_memory(&obj->state, sizeof(obj->state)); +#ifdef WITH_THREAD + if (obj->lock) { + PyThread_free_lock(obj->lock); + obj->lock = NULL; + } +#endif + PyObject_Del(self); +} + + +PyTypeObject PyBlake2_BLAKE2sType = { + PyVarObject_HEAD_INIT(NULL, 0) + "_blake2.blake2s", /* tp_name */ + sizeof(BLAKE2sObject), /* tp_size */ + 0, /* tp_itemsize */ + py_blake2s_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /* tp_flags */ + py_blake2s_new__doc__, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + py_blake2s_methods, /* tp_methods */ + 0, /* tp_members */ + py_blake2s_getsetters, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + py_blake2s_new, /* tp_new */ +}; diff --git a/Modules/_blake2/clinic/blake2b_impl.c.h b/Modules/_blake2/clinic/blake2b_impl.c.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/clinic/blake2b_impl.c.h @@ -0,0 +1,125 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +PyDoc_STRVAR(py_blake2b_new__doc__, +"blake2b(string=None, *, digest_size=_blake2b.blake2b.MAX_DIGEST_SIZE,\n" +" key=None, salt=None, person=None, fanout=1, depth=1,\n" +" leaf_size=None, node_offset=None, node_depth=0, inner_size=0,\n" +" last_node=False)\n" +"--\n" +"\n" +"Return a new BLAKE2b hash object."); + +static PyObject * +py_blake2b_new_impl(PyTypeObject *type, PyObject *data, int digest_size, + Py_buffer *key, Py_buffer *salt, Py_buffer *person, + int fanout, int depth, PyObject *leaf_size_obj, + PyObject *node_offset_obj, int node_depth, + int inner_size, int last_node); + +static PyObject * +py_blake2b_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"string", "digest_size", "key", "salt", "person", "fanout", "depth", "leaf_size", "node_offset", "node_depth", "inner_size", "last_node", NULL}; + static _PyArg_Parser _parser = {"|O$iy*y*y*iiOOiip:blake2b", _keywords, 0}; + PyObject *data = NULL; + int digest_size = BLAKE2B_OUTBYTES; + Py_buffer key = {NULL, NULL}; + Py_buffer salt = {NULL, NULL}; + Py_buffer person = {NULL, NULL}; + int fanout = 1; + int depth = 1; + PyObject *leaf_size_obj = NULL; + PyObject *node_offset_obj = NULL; + int node_depth = 0; + int inner_size = 0; + int last_node = 0; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &data, &digest_size, &key, &salt, &person, &fanout, &depth, &leaf_size_obj, &node_offset_obj, &node_depth, &inner_size, &last_node)) { + goto exit; + } + return_value = py_blake2b_new_impl(type, data, digest_size, &key, &salt, &person, fanout, depth, leaf_size_obj, node_offset_obj, node_depth, inner_size, last_node); + +exit: + /* Cleanup for key */ + if (key.obj) { + PyBuffer_Release(&key); + } + /* Cleanup for salt */ + if (salt.obj) { + PyBuffer_Release(&salt); + } + /* Cleanup for person */ + if (person.obj) { + PyBuffer_Release(&person); + } + + return return_value; +} + +PyDoc_STRVAR(_blake2b_blake2b_copy__doc__, +"copy($self, /)\n" +"--\n" +"\n" +"Return a copy of the hash object."); + +#define _BLAKE2B_BLAKE2B_COPY_METHODDEF \ + {"copy", (PyCFunction)_blake2b_blake2b_copy, METH_NOARGS, _blake2b_blake2b_copy__doc__}, + +static PyObject * +_blake2b_blake2b_copy_impl(BLAKE2bObject *self); + +static PyObject * +_blake2b_blake2b_copy(BLAKE2bObject *self, PyObject *Py_UNUSED(ignored)) +{ + return _blake2b_blake2b_copy_impl(self); +} + +PyDoc_STRVAR(_blake2b_blake2b_update__doc__, +"update($self, obj, /)\n" +"--\n" +"\n" +"Update this hash object\'s state with the provided string."); + +#define _BLAKE2B_BLAKE2B_UPDATE_METHODDEF \ + {"update", (PyCFunction)_blake2b_blake2b_update, METH_O, _blake2b_blake2b_update__doc__}, + +PyDoc_STRVAR(_blake2b_blake2b_digest__doc__, +"digest($self, /)\n" +"--\n" +"\n" +"Return the digest value as a string of binary data."); + +#define _BLAKE2B_BLAKE2B_DIGEST_METHODDEF \ + {"digest", (PyCFunction)_blake2b_blake2b_digest, METH_NOARGS, _blake2b_blake2b_digest__doc__}, + +static PyObject * +_blake2b_blake2b_digest_impl(BLAKE2bObject *self); + +static PyObject * +_blake2b_blake2b_digest(BLAKE2bObject *self, PyObject *Py_UNUSED(ignored)) +{ + return _blake2b_blake2b_digest_impl(self); +} + +PyDoc_STRVAR(_blake2b_blake2b_hexdigest__doc__, +"hexdigest($self, /)\n" +"--\n" +"\n" +"Return the digest value as a string of hexadecimal digits."); + +#define _BLAKE2B_BLAKE2B_HEXDIGEST_METHODDEF \ + {"hexdigest", (PyCFunction)_blake2b_blake2b_hexdigest, METH_NOARGS, _blake2b_blake2b_hexdigest__doc__}, + +static PyObject * +_blake2b_blake2b_hexdigest_impl(BLAKE2bObject *self); + +static PyObject * +_blake2b_blake2b_hexdigest(BLAKE2bObject *self, PyObject *Py_UNUSED(ignored)) +{ + return _blake2b_blake2b_hexdigest_impl(self); +} +/*[clinic end generated code: output=535a54852c98e51c input=a9049054013a1b77]*/ diff --git a/Modules/_blake2/clinic/blake2s_impl.c.h b/Modules/_blake2/clinic/blake2s_impl.c.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/clinic/blake2s_impl.c.h @@ -0,0 +1,125 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +PyDoc_STRVAR(py_blake2s_new__doc__, +"blake2s(string=None, *, digest_size=_blake2s.blake2s.MAX_DIGEST_SIZE,\n" +" key=None, salt=None, person=None, fanout=1, depth=1,\n" +" leaf_size=None, node_offset=None, node_depth=0, inner_size=0,\n" +" last_node=False)\n" +"--\n" +"\n" +"Return a new BLAKE2s hash object."); + +static PyObject * +py_blake2s_new_impl(PyTypeObject *type, PyObject *data, int digest_size, + Py_buffer *key, Py_buffer *salt, Py_buffer *person, + int fanout, int depth, PyObject *leaf_size_obj, + PyObject *node_offset_obj, int node_depth, + int inner_size, int last_node); + +static PyObject * +py_blake2s_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"string", "digest_size", "key", "salt", "person", "fanout", "depth", "leaf_size", "node_offset", "node_depth", "inner_size", "last_node", NULL}; + static _PyArg_Parser _parser = {"|O$iy*y*y*iiOOiip:blake2s", _keywords, 0}; + PyObject *data = NULL; + int digest_size = BLAKE2S_OUTBYTES; + Py_buffer key = {NULL, NULL}; + Py_buffer salt = {NULL, NULL}; + Py_buffer person = {NULL, NULL}; + int fanout = 1; + int depth = 1; + PyObject *leaf_size_obj = NULL; + PyObject *node_offset_obj = NULL; + int node_depth = 0; + int inner_size = 0; + int last_node = 0; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &data, &digest_size, &key, &salt, &person, &fanout, &depth, &leaf_size_obj, &node_offset_obj, &node_depth, &inner_size, &last_node)) { + goto exit; + } + return_value = py_blake2s_new_impl(type, data, digest_size, &key, &salt, &person, fanout, depth, leaf_size_obj, node_offset_obj, node_depth, inner_size, last_node); + +exit: + /* Cleanup for key */ + if (key.obj) { + PyBuffer_Release(&key); + } + /* Cleanup for salt */ + if (salt.obj) { + PyBuffer_Release(&salt); + } + /* Cleanup for person */ + if (person.obj) { + PyBuffer_Release(&person); + } + + return return_value; +} + +PyDoc_STRVAR(_blake2s_blake2s_copy__doc__, +"copy($self, /)\n" +"--\n" +"\n" +"Return a copy of the hash object."); + +#define _BLAKE2S_BLAKE2S_COPY_METHODDEF \ + {"copy", (PyCFunction)_blake2s_blake2s_copy, METH_NOARGS, _blake2s_blake2s_copy__doc__}, + +static PyObject * +_blake2s_blake2s_copy_impl(BLAKE2sObject *self); + +static PyObject * +_blake2s_blake2s_copy(BLAKE2sObject *self, PyObject *Py_UNUSED(ignored)) +{ + return _blake2s_blake2s_copy_impl(self); +} + +PyDoc_STRVAR(_blake2s_blake2s_update__doc__, +"update($self, obj, /)\n" +"--\n" +"\n" +"Update this hash object\'s state with the provided string."); + +#define _BLAKE2S_BLAKE2S_UPDATE_METHODDEF \ + {"update", (PyCFunction)_blake2s_blake2s_update, METH_O, _blake2s_blake2s_update__doc__}, + +PyDoc_STRVAR(_blake2s_blake2s_digest__doc__, +"digest($self, /)\n" +"--\n" +"\n" +"Return the digest value as a string of binary data."); + +#define _BLAKE2S_BLAKE2S_DIGEST_METHODDEF \ + {"digest", (PyCFunction)_blake2s_blake2s_digest, METH_NOARGS, _blake2s_blake2s_digest__doc__}, + +static PyObject * +_blake2s_blake2s_digest_impl(BLAKE2sObject *self); + +static PyObject * +_blake2s_blake2s_digest(BLAKE2sObject *self, PyObject *Py_UNUSED(ignored)) +{ + return _blake2s_blake2s_digest_impl(self); +} + +PyDoc_STRVAR(_blake2s_blake2s_hexdigest__doc__, +"hexdigest($self, /)\n" +"--\n" +"\n" +"Return the digest value as a string of hexadecimal digits."); + +#define _BLAKE2S_BLAKE2S_HEXDIGEST_METHODDEF \ + {"hexdigest", (PyCFunction)_blake2s_blake2s_hexdigest, METH_NOARGS, _blake2s_blake2s_hexdigest__doc__}, + +static PyObject * +_blake2s_blake2s_hexdigest_impl(BLAKE2sObject *self); + +static PyObject * +_blake2s_blake2s_hexdigest(BLAKE2sObject *self, PyObject *Py_UNUSED(ignored)) +{ + return _blake2s_blake2s_hexdigest_impl(self); +} +/*[clinic end generated code: output=535ea7903f9ccf76 input=a9049054013a1b77]*/ diff --git a/Modules/_blake2/impl/blake2-config.h b/Modules/_blake2/impl/blake2-config.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2-config.h @@ -0,0 +1,74 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2_CONFIG_H__ +#define __BLAKE2_CONFIG_H__ + +/* These don't work everywhere */ +#if defined(__SSE2__) || defined(__x86_64__) || defined(__amd64__) +#define HAVE_SSE2 +#endif + +#if defined(__SSSE3__) +#define HAVE_SSSE3 +#endif + +#if defined(__SSE4_1__) +#define HAVE_SSE41 +#endif + +#if defined(__AVX__) +#define HAVE_AVX +#endif + +#if defined(__XOP__) +#define HAVE_XOP +#endif + + +#ifdef HAVE_AVX2 +#ifndef HAVE_AVX +#define HAVE_AVX +#endif +#endif + +#ifdef HAVE_XOP +#ifndef HAVE_AVX +#define HAVE_AVX +#endif +#endif + +#ifdef HAVE_AVX +#ifndef HAVE_SSE41 +#define HAVE_SSE41 +#endif +#endif + +#ifdef HAVE_SSE41 +#ifndef HAVE_SSSE3 +#define HAVE_SSSE3 +#endif +#endif + +#ifdef HAVE_SSSE3 +#define HAVE_SSE2 +#endif + +#if !defined(HAVE_SSE2) +#error "This code requires at least SSE2." +#endif + +#endif + diff --git a/Modules/_blake2/impl/blake2-impl.h b/Modules/_blake2/impl/blake2-impl.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2-impl.h @@ -0,0 +1,139 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2_IMPL_H__ +#define __BLAKE2_IMPL_H__ + +#include +#include + +BLAKE2_LOCAL_INLINE(uint32_t) load32( const void *src ) +{ +#if defined(NATIVE_LITTLE_ENDIAN) + uint32_t w; + memcpy(&w, src, sizeof w); + return w; +#else + const uint8_t *p = ( const uint8_t * )src; + uint32_t w = *p++; + w |= ( uint32_t )( *p++ ) << 8; + w |= ( uint32_t )( *p++ ) << 16; + w |= ( uint32_t )( *p++ ) << 24; + return w; +#endif +} + +BLAKE2_LOCAL_INLINE(uint64_t) load64( const void *src ) +{ +#if defined(NATIVE_LITTLE_ENDIAN) + uint64_t w; + memcpy(&w, src, sizeof w); + return w; +#else + const uint8_t *p = ( const uint8_t * )src; + uint64_t w = *p++; + w |= ( uint64_t )( *p++ ) << 8; + w |= ( uint64_t )( *p++ ) << 16; + w |= ( uint64_t )( *p++ ) << 24; + w |= ( uint64_t )( *p++ ) << 32; + w |= ( uint64_t )( *p++ ) << 40; + w |= ( uint64_t )( *p++ ) << 48; + w |= ( uint64_t )( *p++ ) << 56; + return w; +#endif +} + +BLAKE2_LOCAL_INLINE(void) store32( void *dst, uint32_t w ) +{ +#if defined(NATIVE_LITTLE_ENDIAN) + memcpy(dst, &w, sizeof w); +#else + uint8_t *p = ( uint8_t * )dst; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; +#endif +} + +BLAKE2_LOCAL_INLINE(void) store64( void *dst, uint64_t w ) +{ +#if defined(NATIVE_LITTLE_ENDIAN) + memcpy(dst, &w, sizeof w); +#else + uint8_t *p = ( uint8_t * )dst; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; +#endif +} + +BLAKE2_LOCAL_INLINE(uint64_t) load48( const void *src ) +{ + const uint8_t *p = ( const uint8_t * )src; + uint64_t w = *p++; + w |= ( uint64_t )( *p++ ) << 8; + w |= ( uint64_t )( *p++ ) << 16; + w |= ( uint64_t )( *p++ ) << 24; + w |= ( uint64_t )( *p++ ) << 32; + w |= ( uint64_t )( *p++ ) << 40; + return w; +} + +BLAKE2_LOCAL_INLINE(void) store48( void *dst, uint64_t w ) +{ + uint8_t *p = ( uint8_t * )dst; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; w >>= 8; + *p++ = ( uint8_t )w; +} + +BLAKE2_LOCAL_INLINE(uint32_t) rotl32( const uint32_t w, const unsigned c ) +{ + return ( w << c ) | ( w >> ( 32 - c ) ); +} + +BLAKE2_LOCAL_INLINE(uint64_t) rotl64( const uint64_t w, const unsigned c ) +{ + return ( w << c ) | ( w >> ( 64 - c ) ); +} + +BLAKE2_LOCAL_INLINE(uint32_t) rotr32( const uint32_t w, const unsigned c ) +{ + return ( w >> c ) | ( w << ( 32 - c ) ); +} + +BLAKE2_LOCAL_INLINE(uint64_t) rotr64( const uint64_t w, const unsigned c ) +{ + return ( w >> c ) | ( w << ( 64 - c ) ); +} + +/* prevents compiler optimizing out memset() */ +BLAKE2_LOCAL_INLINE(void) secure_zero_memory(void *v, size_t n) +{ + static void *(*const volatile memset_v)(void *, int, size_t) = &memset; + memset_v(v, 0, n); +} + +#endif + diff --git a/Modules/_blake2/impl/blake2.h b/Modules/_blake2/impl/blake2.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2.h @@ -0,0 +1,161 @@ +/* + BLAKE2 reference source code package - reference C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2_H__ +#define __BLAKE2_H__ + +#include +#include + +#ifdef BLAKE2_NO_INLINE +#define BLAKE2_LOCAL_INLINE(type) static type +#endif + +#ifndef BLAKE2_LOCAL_INLINE +#define BLAKE2_LOCAL_INLINE(type) static inline type +#endif + +#if defined(__cplusplus) +extern "C" { +#endif + + enum blake2s_constant + { + BLAKE2S_BLOCKBYTES = 64, + BLAKE2S_OUTBYTES = 32, + BLAKE2S_KEYBYTES = 32, + BLAKE2S_SALTBYTES = 8, + BLAKE2S_PERSONALBYTES = 8 + }; + + enum blake2b_constant + { + BLAKE2B_BLOCKBYTES = 128, + BLAKE2B_OUTBYTES = 64, + BLAKE2B_KEYBYTES = 64, + BLAKE2B_SALTBYTES = 16, + BLAKE2B_PERSONALBYTES = 16 + }; + + typedef struct __blake2s_state + { + uint32_t h[8]; + uint32_t t[2]; + uint32_t f[2]; + uint8_t buf[2 * BLAKE2S_BLOCKBYTES]; + size_t buflen; + uint8_t last_node; + } blake2s_state; + + typedef struct __blake2b_state + { + uint64_t h[8]; + uint64_t t[2]; + uint64_t f[2]; + uint8_t buf[2 * BLAKE2B_BLOCKBYTES]; + size_t buflen; + uint8_t last_node; + } blake2b_state; + + typedef struct __blake2sp_state + { + blake2s_state S[8][1]; + blake2s_state R[1]; + uint8_t buf[8 * BLAKE2S_BLOCKBYTES]; + size_t buflen; + } blake2sp_state; + + typedef struct __blake2bp_state + { + blake2b_state S[4][1]; + blake2b_state R[1]; + uint8_t buf[4 * BLAKE2B_BLOCKBYTES]; + size_t buflen; + } blake2bp_state; + + +#pragma pack(push, 1) + typedef struct __blake2s_param + { + uint8_t digest_length; /* 1 */ + uint8_t key_length; /* 2 */ + uint8_t fanout; /* 3 */ + uint8_t depth; /* 4 */ + uint32_t leaf_length; /* 8 */ + uint8_t node_offset[6];// 14 + uint8_t node_depth; /* 15 */ + uint8_t inner_length; /* 16 */ + /* uint8_t reserved[0]; */ + uint8_t salt[BLAKE2S_SALTBYTES]; /* 24 */ + uint8_t personal[BLAKE2S_PERSONALBYTES]; /* 32 */ + } blake2s_param; + + typedef struct __blake2b_param + { + uint8_t digest_length; /* 1 */ + uint8_t key_length; /* 2 */ + uint8_t fanout; /* 3 */ + uint8_t depth; /* 4 */ + uint32_t leaf_length; /* 8 */ + uint64_t node_offset; /* 16 */ + uint8_t node_depth; /* 17 */ + uint8_t inner_length; /* 18 */ + uint8_t reserved[14]; /* 32 */ + uint8_t salt[BLAKE2B_SALTBYTES]; /* 48 */ + uint8_t personal[BLAKE2B_PERSONALBYTES]; /* 64 */ + } blake2b_param; +#pragma pack(pop) + + /* Streaming API */ + int blake2s_init( blake2s_state *S, const uint8_t outlen ); + int blake2s_init_key( blake2s_state *S, const uint8_t outlen, const void *key, const uint8_t keylen ); + int blake2s_init_param( blake2s_state *S, const blake2s_param *P ); + int blake2s_update( blake2s_state *S, const uint8_t *in, uint64_t inlen ); + int blake2s_final( blake2s_state *S, uint8_t *out, uint8_t outlen ); + + int blake2b_init( blake2b_state *S, const uint8_t outlen ); + int blake2b_init_key( blake2b_state *S, const uint8_t outlen, const void *key, const uint8_t keylen ); + int blake2b_init_param( blake2b_state *S, const blake2b_param *P ); + int blake2b_update( blake2b_state *S, const uint8_t *in, uint64_t inlen ); + int blake2b_final( blake2b_state *S, uint8_t *out, uint8_t outlen ); + + int blake2sp_init( blake2sp_state *S, const uint8_t outlen ); + int blake2sp_init_key( blake2sp_state *S, const uint8_t outlen, const void *key, const uint8_t keylen ); + int blake2sp_update( blake2sp_state *S, const uint8_t *in, uint64_t inlen ); + int blake2sp_final( blake2sp_state *S, uint8_t *out, uint8_t outlen ); + + int blake2bp_init( blake2bp_state *S, const uint8_t outlen ); + int blake2bp_init_key( blake2bp_state *S, const uint8_t outlen, const void *key, const uint8_t keylen ); + int blake2bp_update( blake2bp_state *S, const uint8_t *in, uint64_t inlen ); + int blake2bp_final( blake2bp_state *S, uint8_t *out, uint8_t outlen ); + + /* Simple API */ + int blake2s( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen ); + int blake2b( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen ); + + int blake2sp( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen ); + int blake2bp( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen ); + + static inline int blake2( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen ) + { + return blake2b( out, in, key, outlen, inlen, keylen ); + } + +#if defined(__cplusplus) +} +#endif + +#endif + diff --git a/Modules/_blake2/impl/blake2b-load-sse2.h b/Modules/_blake2/impl/blake2b-load-sse2.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2b-load-sse2.h @@ -0,0 +1,70 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2B_LOAD_SSE2_H__ +#define __BLAKE2B_LOAD_SSE2_H__ + +#define LOAD_MSG_0_1(b0, b1) b0 = _mm_set_epi64x(m2, m0); b1 = _mm_set_epi64x(m6, m4) +#define LOAD_MSG_0_2(b0, b1) b0 = _mm_set_epi64x(m3, m1); b1 = _mm_set_epi64x(m7, m5) +#define LOAD_MSG_0_3(b0, b1) b0 = _mm_set_epi64x(m10, m8); b1 = _mm_set_epi64x(m14, m12) +#define LOAD_MSG_0_4(b0, b1) b0 = _mm_set_epi64x(m11, m9); b1 = _mm_set_epi64x(m15, m13) +#define LOAD_MSG_1_1(b0, b1) b0 = _mm_set_epi64x(m4, m14); b1 = _mm_set_epi64x(m13, m9) +#define LOAD_MSG_1_2(b0, b1) b0 = _mm_set_epi64x(m8, m10); b1 = _mm_set_epi64x(m6, m15) +#define LOAD_MSG_1_3(b0, b1) b0 = _mm_set_epi64x(m0, m1); b1 = _mm_set_epi64x(m5, m11) +#define LOAD_MSG_1_4(b0, b1) b0 = _mm_set_epi64x(m2, m12); b1 = _mm_set_epi64x(m3, m7) +#define LOAD_MSG_2_1(b0, b1) b0 = _mm_set_epi64x(m12, m11); b1 = _mm_set_epi64x(m15, m5) +#define LOAD_MSG_2_2(b0, b1) b0 = _mm_set_epi64x(m0, m8); b1 = _mm_set_epi64x(m13, m2) +#define LOAD_MSG_2_3(b0, b1) b0 = _mm_set_epi64x(m3, m10); b1 = _mm_set_epi64x(m9, m7) +#define LOAD_MSG_2_4(b0, b1) b0 = _mm_set_epi64x(m6, m14); b1 = _mm_set_epi64x(m4, m1) +#define LOAD_MSG_3_1(b0, b1) b0 = _mm_set_epi64x(m3, m7); b1 = _mm_set_epi64x(m11, m13) +#define LOAD_MSG_3_2(b0, b1) b0 = _mm_set_epi64x(m1, m9); b1 = _mm_set_epi64x(m14, m12) +#define LOAD_MSG_3_3(b0, b1) b0 = _mm_set_epi64x(m5, m2); b1 = _mm_set_epi64x(m15, m4) +#define LOAD_MSG_3_4(b0, b1) b0 = _mm_set_epi64x(m10, m6); b1 = _mm_set_epi64x(m8, m0) +#define LOAD_MSG_4_1(b0, b1) b0 = _mm_set_epi64x(m5, m9); b1 = _mm_set_epi64x(m10, m2) +#define LOAD_MSG_4_2(b0, b1) b0 = _mm_set_epi64x(m7, m0); b1 = _mm_set_epi64x(m15, m4) +#define LOAD_MSG_4_3(b0, b1) b0 = _mm_set_epi64x(m11, m14); b1 = _mm_set_epi64x(m3, m6) +#define LOAD_MSG_4_4(b0, b1) b0 = _mm_set_epi64x(m12, m1); b1 = _mm_set_epi64x(m13, m8) +#define LOAD_MSG_5_1(b0, b1) b0 = _mm_set_epi64x(m6, m2); b1 = _mm_set_epi64x(m8, m0) +#define LOAD_MSG_5_2(b0, b1) b0 = _mm_set_epi64x(m10, m12); b1 = _mm_set_epi64x(m3, m11) +#define LOAD_MSG_5_3(b0, b1) b0 = _mm_set_epi64x(m7, m4); b1 = _mm_set_epi64x(m1, m15) +#define LOAD_MSG_5_4(b0, b1) b0 = _mm_set_epi64x(m5, m13); b1 = _mm_set_epi64x(m9, m14) +#define LOAD_MSG_6_1(b0, b1) b0 = _mm_set_epi64x(m1, m12); b1 = _mm_set_epi64x(m4, m14) +#define LOAD_MSG_6_2(b0, b1) b0 = _mm_set_epi64x(m15, m5); b1 = _mm_set_epi64x(m10, m13) +#define LOAD_MSG_6_3(b0, b1) b0 = _mm_set_epi64x(m6, m0); b1 = _mm_set_epi64x(m8, m9) +#define LOAD_MSG_6_4(b0, b1) b0 = _mm_set_epi64x(m3, m7); b1 = _mm_set_epi64x(m11, m2) +#define LOAD_MSG_7_1(b0, b1) b0 = _mm_set_epi64x(m7, m13); b1 = _mm_set_epi64x(m3, m12) +#define LOAD_MSG_7_2(b0, b1) b0 = _mm_set_epi64x(m14, m11); b1 = _mm_set_epi64x(m9, m1) +#define LOAD_MSG_7_3(b0, b1) b0 = _mm_set_epi64x(m15, m5); b1 = _mm_set_epi64x(m2, m8) +#define LOAD_MSG_7_4(b0, b1) b0 = _mm_set_epi64x(m4, m0); b1 = _mm_set_epi64x(m10, m6) +#define LOAD_MSG_8_1(b0, b1) b0 = _mm_set_epi64x(m14, m6); b1 = _mm_set_epi64x(m0, m11) +#define LOAD_MSG_8_2(b0, b1) b0 = _mm_set_epi64x(m9, m15); b1 = _mm_set_epi64x(m8, m3) +#define LOAD_MSG_8_3(b0, b1) b0 = _mm_set_epi64x(m13, m12); b1 = _mm_set_epi64x(m10, m1) +#define LOAD_MSG_8_4(b0, b1) b0 = _mm_set_epi64x(m7, m2); b1 = _mm_set_epi64x(m5, m4) +#define LOAD_MSG_9_1(b0, b1) b0 = _mm_set_epi64x(m8, m10); b1 = _mm_set_epi64x(m1, m7) +#define LOAD_MSG_9_2(b0, b1) b0 = _mm_set_epi64x(m4, m2); b1 = _mm_set_epi64x(m5, m6) +#define LOAD_MSG_9_3(b0, b1) b0 = _mm_set_epi64x(m9, m15); b1 = _mm_set_epi64x(m13, m3) +#define LOAD_MSG_9_4(b0, b1) b0 = _mm_set_epi64x(m14, m11); b1 = _mm_set_epi64x(m0, m12) +#define LOAD_MSG_10_1(b0, b1) b0 = _mm_set_epi64x(m2, m0); b1 = _mm_set_epi64x(m6, m4) +#define LOAD_MSG_10_2(b0, b1) b0 = _mm_set_epi64x(m3, m1); b1 = _mm_set_epi64x(m7, m5) +#define LOAD_MSG_10_3(b0, b1) b0 = _mm_set_epi64x(m10, m8); b1 = _mm_set_epi64x(m14, m12) +#define LOAD_MSG_10_4(b0, b1) b0 = _mm_set_epi64x(m11, m9); b1 = _mm_set_epi64x(m15, m13) +#define LOAD_MSG_11_1(b0, b1) b0 = _mm_set_epi64x(m4, m14); b1 = _mm_set_epi64x(m13, m9) +#define LOAD_MSG_11_2(b0, b1) b0 = _mm_set_epi64x(m8, m10); b1 = _mm_set_epi64x(m6, m15) +#define LOAD_MSG_11_3(b0, b1) b0 = _mm_set_epi64x(m0, m1); b1 = _mm_set_epi64x(m5, m11) +#define LOAD_MSG_11_4(b0, b1) b0 = _mm_set_epi64x(m2, m12); b1 = _mm_set_epi64x(m3, m7) + + +#endif + diff --git a/Modules/_blake2/impl/blake2b-load-sse41.h b/Modules/_blake2/impl/blake2b-load-sse41.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2b-load-sse41.h @@ -0,0 +1,404 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2B_LOAD_SSE41_H__ +#define __BLAKE2B_LOAD_SSE41_H__ + +#define LOAD_MSG_0_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m0, m1); \ +b1 = _mm_unpacklo_epi64(m2, m3); \ +} while(0) + + +#define LOAD_MSG_0_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m0, m1); \ +b1 = _mm_unpackhi_epi64(m2, m3); \ +} while(0) + + +#define LOAD_MSG_0_3(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m4, m5); \ +b1 = _mm_unpacklo_epi64(m6, m7); \ +} while(0) + + +#define LOAD_MSG_0_4(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m4, m5); \ +b1 = _mm_unpackhi_epi64(m6, m7); \ +} while(0) + + +#define LOAD_MSG_1_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m7, m2); \ +b1 = _mm_unpackhi_epi64(m4, m6); \ +} while(0) + + +#define LOAD_MSG_1_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m5, m4); \ +b1 = _mm_alignr_epi8(m3, m7, 8); \ +} while(0) + + +#define LOAD_MSG_1_3(b0, b1) \ +do \ +{ \ +b0 = _mm_shuffle_epi32(m0, _MM_SHUFFLE(1,0,3,2)); \ +b1 = _mm_unpackhi_epi64(m5, m2); \ +} while(0) + + +#define LOAD_MSG_1_4(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m6, m1); \ +b1 = _mm_unpackhi_epi64(m3, m1); \ +} while(0) + + +#define LOAD_MSG_2_1(b0, b1) \ +do \ +{ \ +b0 = _mm_alignr_epi8(m6, m5, 8); \ +b1 = _mm_unpackhi_epi64(m2, m7); \ +} while(0) + + +#define LOAD_MSG_2_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m4, m0); \ +b1 = _mm_blend_epi16(m1, m6, 0xF0); \ +} while(0) + + +#define LOAD_MSG_2_3(b0, b1) \ +do \ +{ \ +b0 = _mm_blend_epi16(m5, m1, 0xF0); \ +b1 = _mm_unpackhi_epi64(m3, m4); \ +} while(0) + + +#define LOAD_MSG_2_4(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m7, m3); \ +b1 = _mm_alignr_epi8(m2, m0, 8); \ +} while(0) + + +#define LOAD_MSG_3_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m3, m1); \ +b1 = _mm_unpackhi_epi64(m6, m5); \ +} while(0) + + +#define LOAD_MSG_3_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m4, m0); \ +b1 = _mm_unpacklo_epi64(m6, m7); \ +} while(0) + + +#define LOAD_MSG_3_3(b0, b1) \ +do \ +{ \ +b0 = _mm_blend_epi16(m1, m2, 0xF0); \ +b1 = _mm_blend_epi16(m2, m7, 0xF0); \ +} while(0) + + +#define LOAD_MSG_3_4(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m3, m5); \ +b1 = _mm_unpacklo_epi64(m0, m4); \ +} while(0) + + +#define LOAD_MSG_4_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m4, m2); \ +b1 = _mm_unpacklo_epi64(m1, m5); \ +} while(0) + + +#define LOAD_MSG_4_2(b0, b1) \ +do \ +{ \ +b0 = _mm_blend_epi16(m0, m3, 0xF0); \ +b1 = _mm_blend_epi16(m2, m7, 0xF0); \ +} while(0) + + +#define LOAD_MSG_4_3(b0, b1) \ +do \ +{ \ +b0 = _mm_blend_epi16(m7, m5, 0xF0); \ +b1 = _mm_blend_epi16(m3, m1, 0xF0); \ +} while(0) + + +#define LOAD_MSG_4_4(b0, b1) \ +do \ +{ \ +b0 = _mm_alignr_epi8(m6, m0, 8); \ +b1 = _mm_blend_epi16(m4, m6, 0xF0); \ +} while(0) + + +#define LOAD_MSG_5_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m1, m3); \ +b1 = _mm_unpacklo_epi64(m0, m4); \ +} while(0) + + +#define LOAD_MSG_5_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m6, m5); \ +b1 = _mm_unpackhi_epi64(m5, m1); \ +} while(0) + + +#define LOAD_MSG_5_3(b0, b1) \ +do \ +{ \ +b0 = _mm_blend_epi16(m2, m3, 0xF0); \ +b1 = _mm_unpackhi_epi64(m7, m0); \ +} while(0) + + +#define LOAD_MSG_5_4(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m6, m2); \ +b1 = _mm_blend_epi16(m7, m4, 0xF0); \ +} while(0) + + +#define LOAD_MSG_6_1(b0, b1) \ +do \ +{ \ +b0 = _mm_blend_epi16(m6, m0, 0xF0); \ +b1 = _mm_unpacklo_epi64(m7, m2); \ +} while(0) + + +#define LOAD_MSG_6_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m2, m7); \ +b1 = _mm_alignr_epi8(m5, m6, 8); \ +} while(0) + + +#define LOAD_MSG_6_3(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m0, m3); \ +b1 = _mm_shuffle_epi32(m4, _MM_SHUFFLE(1,0,3,2)); \ +} while(0) + + +#define LOAD_MSG_6_4(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m3, m1); \ +b1 = _mm_blend_epi16(m1, m5, 0xF0); \ +} while(0) + + +#define LOAD_MSG_7_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m6, m3); \ +b1 = _mm_blend_epi16(m6, m1, 0xF0); \ +} while(0) + + +#define LOAD_MSG_7_2(b0, b1) \ +do \ +{ \ +b0 = _mm_alignr_epi8(m7, m5, 8); \ +b1 = _mm_unpackhi_epi64(m0, m4); \ +} while(0) + + +#define LOAD_MSG_7_3(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m2, m7); \ +b1 = _mm_unpacklo_epi64(m4, m1); \ +} while(0) + + +#define LOAD_MSG_7_4(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m0, m2); \ +b1 = _mm_unpacklo_epi64(m3, m5); \ +} while(0) + + +#define LOAD_MSG_8_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m3, m7); \ +b1 = _mm_alignr_epi8(m0, m5, 8); \ +} while(0) + + +#define LOAD_MSG_8_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m7, m4); \ +b1 = _mm_alignr_epi8(m4, m1, 8); \ +} while(0) + + +#define LOAD_MSG_8_3(b0, b1) \ +do \ +{ \ +b0 = m6; \ +b1 = _mm_alignr_epi8(m5, m0, 8); \ +} while(0) + + +#define LOAD_MSG_8_4(b0, b1) \ +do \ +{ \ +b0 = _mm_blend_epi16(m1, m3, 0xF0); \ +b1 = m2; \ +} while(0) + + +#define LOAD_MSG_9_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m5, m4); \ +b1 = _mm_unpackhi_epi64(m3, m0); \ +} while(0) + + +#define LOAD_MSG_9_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m1, m2); \ +b1 = _mm_blend_epi16(m3, m2, 0xF0); \ +} while(0) + + +#define LOAD_MSG_9_3(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m7, m4); \ +b1 = _mm_unpackhi_epi64(m1, m6); \ +} while(0) + + +#define LOAD_MSG_9_4(b0, b1) \ +do \ +{ \ +b0 = _mm_alignr_epi8(m7, m5, 8); \ +b1 = _mm_unpacklo_epi64(m6, m0); \ +} while(0) + + +#define LOAD_MSG_10_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m0, m1); \ +b1 = _mm_unpacklo_epi64(m2, m3); \ +} while(0) + + +#define LOAD_MSG_10_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m0, m1); \ +b1 = _mm_unpackhi_epi64(m2, m3); \ +} while(0) + + +#define LOAD_MSG_10_3(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m4, m5); \ +b1 = _mm_unpacklo_epi64(m6, m7); \ +} while(0) + + +#define LOAD_MSG_10_4(b0, b1) \ +do \ +{ \ +b0 = _mm_unpackhi_epi64(m4, m5); \ +b1 = _mm_unpackhi_epi64(m6, m7); \ +} while(0) + + +#define LOAD_MSG_11_1(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m7, m2); \ +b1 = _mm_unpackhi_epi64(m4, m6); \ +} while(0) + + +#define LOAD_MSG_11_2(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m5, m4); \ +b1 = _mm_alignr_epi8(m3, m7, 8); \ +} while(0) + + +#define LOAD_MSG_11_3(b0, b1) \ +do \ +{ \ +b0 = _mm_shuffle_epi32(m0, _MM_SHUFFLE(1,0,3,2)); \ +b1 = _mm_unpackhi_epi64(m5, m2); \ +} while(0) + + +#define LOAD_MSG_11_4(b0, b1) \ +do \ +{ \ +b0 = _mm_unpacklo_epi64(m6, m1); \ +b1 = _mm_unpackhi_epi64(m3, m1); \ +} while(0) + + +#endif + diff --git a/Modules/_blake2/impl/blake2b-ref.c b/Modules/_blake2/impl/blake2b-ref.c new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2b-ref.c @@ -0,0 +1,416 @@ +/* + BLAKE2 reference source code package - reference C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ + +#include +#include +#include + +#include "blake2.h" +#include "blake2-impl.h" + +static const uint64_t blake2b_IV[8] = +{ + 0x6a09e667f3bcc908ULL, 0xbb67ae8584caa73bULL, + 0x3c6ef372fe94f82bULL, 0xa54ff53a5f1d36f1ULL, + 0x510e527fade682d1ULL, 0x9b05688c2b3e6c1fULL, + 0x1f83d9abfb41bd6bULL, 0x5be0cd19137e2179ULL +}; + +static const uint8_t blake2b_sigma[12][16] = +{ + { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } , + { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 } , + { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 } , + { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 } , + { 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 } , + { 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 } , + { 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 } , + { 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 } , + { 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 } , + { 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13 , 0 } , + { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } , + { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 } +}; + + +BLAKE2_LOCAL_INLINE(int) blake2b_set_lastnode( blake2b_state *S ) +{ + S->f[1] = -1; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_clear_lastnode( blake2b_state *S ) +{ + S->f[1] = 0; + return 0; +} + +/* Some helper functions, not necessarily useful */ +BLAKE2_LOCAL_INLINE(int) blake2b_is_lastblock( const blake2b_state *S ) +{ + return S->f[0] != 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_set_lastblock( blake2b_state *S ) +{ + if( S->last_node ) blake2b_set_lastnode( S ); + + S->f[0] = -1; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_clear_lastblock( blake2b_state *S ) +{ + if( S->last_node ) blake2b_clear_lastnode( S ); + + S->f[0] = 0; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_increment_counter( blake2b_state *S, const uint64_t inc ) +{ + S->t[0] += inc; + S->t[1] += ( S->t[0] < inc ); + return 0; +} + + + +/* Parameter-related functions */ +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_digest_length( blake2b_param *P, const uint8_t digest_length ) +{ + P->digest_length = digest_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_fanout( blake2b_param *P, const uint8_t fanout ) +{ + P->fanout = fanout; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_max_depth( blake2b_param *P, const uint8_t depth ) +{ + P->depth = depth; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_leaf_length( blake2b_param *P, const uint32_t leaf_length ) +{ + store32( &P->leaf_length, leaf_length ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_node_offset( blake2b_param *P, const uint64_t node_offset ) +{ + store64( &P->node_offset, node_offset ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_node_depth( blake2b_param *P, const uint8_t node_depth ) +{ + P->node_depth = node_depth; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_inner_length( blake2b_param *P, const uint8_t inner_length ) +{ + P->inner_length = inner_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_salt( blake2b_param *P, const uint8_t salt[BLAKE2B_SALTBYTES] ) +{ + memcpy( P->salt, salt, BLAKE2B_SALTBYTES ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_personal( blake2b_param *P, const uint8_t personal[BLAKE2B_PERSONALBYTES] ) +{ + memcpy( P->personal, personal, BLAKE2B_PERSONALBYTES ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_init0( blake2b_state *S ) +{ + memset( S, 0, sizeof( blake2b_state ) ); + + for( int i = 0; i < 8; ++i ) S->h[i] = blake2b_IV[i]; + + return 0; +} + +/* init xors IV with input parameter block */ +int blake2b_init_param( blake2b_state *S, const blake2b_param *P ) +{ + const uint8_t *p = ( const uint8_t * )( P ); + + blake2b_init0( S ); + + /* IV XOR ParamBlock */ + for( size_t i = 0; i < 8; ++i ) + S->h[i] ^= load64( p + sizeof( S->h[i] ) * i ); + + return 0; +} + + + +int blake2b_init( blake2b_state *S, const uint8_t outlen ) +{ + blake2b_param P[1]; + + if ( ( !outlen ) || ( outlen > BLAKE2B_OUTBYTES ) ) return -1; + + P->digest_length = outlen; + P->key_length = 0; + P->fanout = 1; + P->depth = 1; + store32( &P->leaf_length, 0 ); + store64( &P->node_offset, 0 ); + P->node_depth = 0; + P->inner_length = 0; + memset( P->reserved, 0, sizeof( P->reserved ) ); + memset( P->salt, 0, sizeof( P->salt ) ); + memset( P->personal, 0, sizeof( P->personal ) ); + return blake2b_init_param( S, P ); +} + + +int blake2b_init_key( blake2b_state *S, const uint8_t outlen, const void *key, const uint8_t keylen ) +{ + blake2b_param P[1]; + + if ( ( !outlen ) || ( outlen > BLAKE2B_OUTBYTES ) ) return -1; + + if ( !key || !keylen || keylen > BLAKE2B_KEYBYTES ) return -1; + + P->digest_length = outlen; + P->key_length = keylen; + P->fanout = 1; + P->depth = 1; + store32( &P->leaf_length, 0 ); + store64( &P->node_offset, 0 ); + P->node_depth = 0; + P->inner_length = 0; + memset( P->reserved, 0, sizeof( P->reserved ) ); + memset( P->salt, 0, sizeof( P->salt ) ); + memset( P->personal, 0, sizeof( P->personal ) ); + + if( blake2b_init_param( S, P ) < 0 ) return -1; + + { + uint8_t block[BLAKE2B_BLOCKBYTES]; + memset( block, 0, BLAKE2B_BLOCKBYTES ); + memcpy( block, key, keylen ); + blake2b_update( S, block, BLAKE2B_BLOCKBYTES ); + secure_zero_memory( block, BLAKE2B_BLOCKBYTES ); /* Burn the key from stack */ + } + return 0; +} + +static int blake2b_compress( blake2b_state *S, const uint8_t block[BLAKE2B_BLOCKBYTES] ) +{ + uint64_t m[16]; + uint64_t v[16]; + int i; + + for( i = 0; i < 16; ++i ) + m[i] = load64( block + i * sizeof( m[i] ) ); + + for( i = 0; i < 8; ++i ) + v[i] = S->h[i]; + + v[ 8] = blake2b_IV[0]; + v[ 9] = blake2b_IV[1]; + v[10] = blake2b_IV[2]; + v[11] = blake2b_IV[3]; + v[12] = S->t[0] ^ blake2b_IV[4]; + v[13] = S->t[1] ^ blake2b_IV[5]; + v[14] = S->f[0] ^ blake2b_IV[6]; + v[15] = S->f[1] ^ blake2b_IV[7]; +#define G(r,i,a,b,c,d) \ + do { \ + a = a + b + m[blake2b_sigma[r][2*i+0]]; \ + d = rotr64(d ^ a, 32); \ + c = c + d; \ + b = rotr64(b ^ c, 24); \ + a = a + b + m[blake2b_sigma[r][2*i+1]]; \ + d = rotr64(d ^ a, 16); \ + c = c + d; \ + b = rotr64(b ^ c, 63); \ + } while(0) +#define ROUND(r) \ + do { \ + G(r,0,v[ 0],v[ 4],v[ 8],v[12]); \ + G(r,1,v[ 1],v[ 5],v[ 9],v[13]); \ + G(r,2,v[ 2],v[ 6],v[10],v[14]); \ + G(r,3,v[ 3],v[ 7],v[11],v[15]); \ + G(r,4,v[ 0],v[ 5],v[10],v[15]); \ + G(r,5,v[ 1],v[ 6],v[11],v[12]); \ + G(r,6,v[ 2],v[ 7],v[ 8],v[13]); \ + G(r,7,v[ 3],v[ 4],v[ 9],v[14]); \ + } while(0) + ROUND( 0 ); + ROUND( 1 ); + ROUND( 2 ); + ROUND( 3 ); + ROUND( 4 ); + ROUND( 5 ); + ROUND( 6 ); + ROUND( 7 ); + ROUND( 8 ); + ROUND( 9 ); + ROUND( 10 ); + ROUND( 11 ); + + for( i = 0; i < 8; ++i ) + S->h[i] = S->h[i] ^ v[i] ^ v[i + 8]; + +#undef G +#undef ROUND + return 0; +} + +/* inlen now in bytes */ +int blake2b_update( blake2b_state *S, const uint8_t *in, uint64_t inlen ) +{ + while( inlen > 0 ) + { + size_t left = S->buflen; + size_t fill = 2 * BLAKE2B_BLOCKBYTES - left; + + if( inlen > fill ) + { + memcpy( S->buf + left, in, fill ); /* Fill buffer */ + S->buflen += fill; + blake2b_increment_counter( S, BLAKE2B_BLOCKBYTES ); + blake2b_compress( S, S->buf ); /* Compress */ + memcpy( S->buf, S->buf + BLAKE2B_BLOCKBYTES, BLAKE2B_BLOCKBYTES ); /* Shift buffer left */ + S->buflen -= BLAKE2B_BLOCKBYTES; + in += fill; + inlen -= fill; + } + else /* inlen <= fill */ + { + memcpy( S->buf + left, in, inlen ); + S->buflen += inlen; /* Be lazy, do not compress */ + in += inlen; + inlen -= inlen; + } + } + + return 0; +} + +/* Is this correct? */ +int blake2b_final( blake2b_state *S, uint8_t *out, uint8_t outlen ) +{ + uint8_t buffer[BLAKE2B_OUTBYTES] = {0}; + + if( out == NULL || outlen == 0 || outlen > BLAKE2B_OUTBYTES ) + return -1; + + if( blake2b_is_lastblock( S ) ) + return -1; + + if( S->buflen > BLAKE2B_BLOCKBYTES ) + { + blake2b_increment_counter( S, BLAKE2B_BLOCKBYTES ); + blake2b_compress( S, S->buf ); + S->buflen -= BLAKE2B_BLOCKBYTES; + memcpy( S->buf, S->buf + BLAKE2B_BLOCKBYTES, S->buflen ); + } + + blake2b_increment_counter( S, S->buflen ); + blake2b_set_lastblock( S ); + memset( S->buf + S->buflen, 0, 2 * BLAKE2B_BLOCKBYTES - S->buflen ); /* Padding */ + blake2b_compress( S, S->buf ); + + for( int i = 0; i < 8; ++i ) /* Output full hash to temp buffer */ + store64( buffer + sizeof( S->h[i] ) * i, S->h[i] ); + + memcpy( out, buffer, outlen ); + return 0; +} + +/* inlen, at least, should be uint64_t. Others can be size_t. */ +int blake2b( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen ) +{ + blake2b_state S[1]; + + /* Verify parameters */ + if ( NULL == in && inlen > 0 ) return -1; + + if ( NULL == out ) return -1; + + if( NULL == key && keylen > 0 ) return -1; + + if( !outlen || outlen > BLAKE2B_OUTBYTES ) return -1; + + if( keylen > BLAKE2B_KEYBYTES ) return -1; + + if( keylen > 0 ) + { + if( blake2b_init_key( S, outlen, key, keylen ) < 0 ) return -1; + } + else + { + if( blake2b_init( S, outlen ) < 0 ) return -1; + } + + blake2b_update( S, ( const uint8_t * )in, inlen ); + blake2b_final( S, out, outlen ); + return 0; +} + +#if defined(SUPERCOP) +int crypto_hash( unsigned char *out, unsigned char *in, unsigned long long inlen ) +{ + return blake2b( out, in, NULL, BLAKE2B_OUTBYTES, inlen, 0 ); +} +#endif + +#if defined(BLAKE2B_SELFTEST) +#include +#include "blake2-kat.h" +int main( int argc, char **argv ) +{ + uint8_t key[BLAKE2B_KEYBYTES]; + uint8_t buf[KAT_LENGTH]; + + for( size_t i = 0; i < BLAKE2B_KEYBYTES; ++i ) + key[i] = ( uint8_t )i; + + for( size_t i = 0; i < KAT_LENGTH; ++i ) + buf[i] = ( uint8_t )i; + + for( size_t i = 0; i < KAT_LENGTH; ++i ) + { + uint8_t hash[BLAKE2B_OUTBYTES]; + blake2b( hash, buf, key, BLAKE2B_OUTBYTES, i, BLAKE2B_KEYBYTES ); + + if( 0 != memcmp( hash, blake2b_keyed_kat[i], BLAKE2B_OUTBYTES ) ) + { + puts( "error" ); + return -1; + } + } + + puts( "ok" ); + return 0; +} +#endif + diff --git a/Modules/_blake2/impl/blake2b-round.h b/Modules/_blake2/impl/blake2b-round.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2b-round.h @@ -0,0 +1,159 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2B_ROUND_H__ +#define __BLAKE2B_ROUND_H__ + +#define LOADU(p) _mm_loadu_si128( (const __m128i *)(p) ) +#define STOREU(p,r) _mm_storeu_si128((__m128i *)(p), r) + +#define TOF(reg) _mm_castsi128_ps((reg)) +#define TOI(reg) _mm_castps_si128((reg)) + +#define LIKELY(x) __builtin_expect((x),1) + + +/* Microarchitecture-specific macros */ +#ifndef HAVE_XOP +#ifdef HAVE_SSSE3 +#define _mm_roti_epi64(x, c) \ + (-(c) == 32) ? _mm_shuffle_epi32((x), _MM_SHUFFLE(2,3,0,1)) \ + : (-(c) == 24) ? _mm_shuffle_epi8((x), r24) \ + : (-(c) == 16) ? _mm_shuffle_epi8((x), r16) \ + : (-(c) == 63) ? _mm_xor_si128(_mm_srli_epi64((x), -(c)), _mm_add_epi64((x), (x))) \ + : _mm_xor_si128(_mm_srli_epi64((x), -(c)), _mm_slli_epi64((x), 64-(-(c)))) +#else +#define _mm_roti_epi64(r, c) _mm_xor_si128(_mm_srli_epi64( (r), -(c) ),_mm_slli_epi64( (r), 64-(-(c)) )) +#endif +#else +/* ... */ +#endif + + + +#define G1(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1) \ + row1l = _mm_add_epi64(_mm_add_epi64(row1l, b0), row2l); \ + row1h = _mm_add_epi64(_mm_add_epi64(row1h, b1), row2h); \ + \ + row4l = _mm_xor_si128(row4l, row1l); \ + row4h = _mm_xor_si128(row4h, row1h); \ + \ + row4l = _mm_roti_epi64(row4l, -32); \ + row4h = _mm_roti_epi64(row4h, -32); \ + \ + row3l = _mm_add_epi64(row3l, row4l); \ + row3h = _mm_add_epi64(row3h, row4h); \ + \ + row2l = _mm_xor_si128(row2l, row3l); \ + row2h = _mm_xor_si128(row2h, row3h); \ + \ + row2l = _mm_roti_epi64(row2l, -24); \ + row2h = _mm_roti_epi64(row2h, -24); \ + +#define G2(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1) \ + row1l = _mm_add_epi64(_mm_add_epi64(row1l, b0), row2l); \ + row1h = _mm_add_epi64(_mm_add_epi64(row1h, b1), row2h); \ + \ + row4l = _mm_xor_si128(row4l, row1l); \ + row4h = _mm_xor_si128(row4h, row1h); \ + \ + row4l = _mm_roti_epi64(row4l, -16); \ + row4h = _mm_roti_epi64(row4h, -16); \ + \ + row3l = _mm_add_epi64(row3l, row4l); \ + row3h = _mm_add_epi64(row3h, row4h); \ + \ + row2l = _mm_xor_si128(row2l, row3l); \ + row2h = _mm_xor_si128(row2h, row3h); \ + \ + row2l = _mm_roti_epi64(row2l, -63); \ + row2h = _mm_roti_epi64(row2h, -63); \ + +#if defined(HAVE_SSSE3) +#define DIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h) \ + t0 = _mm_alignr_epi8(row2h, row2l, 8); \ + t1 = _mm_alignr_epi8(row2l, row2h, 8); \ + row2l = t0; \ + row2h = t1; \ + \ + t0 = row3l; \ + row3l = row3h; \ + row3h = t0; \ + \ + t0 = _mm_alignr_epi8(row4h, row4l, 8); \ + t1 = _mm_alignr_epi8(row4l, row4h, 8); \ + row4l = t1; \ + row4h = t0; + +#define UNDIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h) \ + t0 = _mm_alignr_epi8(row2l, row2h, 8); \ + t1 = _mm_alignr_epi8(row2h, row2l, 8); \ + row2l = t0; \ + row2h = t1; \ + \ + t0 = row3l; \ + row3l = row3h; \ + row3h = t0; \ + \ + t0 = _mm_alignr_epi8(row4l, row4h, 8); \ + t1 = _mm_alignr_epi8(row4h, row4l, 8); \ + row4l = t1; \ + row4h = t0; +#else + +#define DIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h) \ + t0 = row4l;\ + t1 = row2l;\ + row4l = row3l;\ + row3l = row3h;\ + row3h = row4l;\ + row4l = _mm_unpackhi_epi64(row4h, _mm_unpacklo_epi64(t0, t0)); \ + row4h = _mm_unpackhi_epi64(t0, _mm_unpacklo_epi64(row4h, row4h)); \ + row2l = _mm_unpackhi_epi64(row2l, _mm_unpacklo_epi64(row2h, row2h)); \ + row2h = _mm_unpackhi_epi64(row2h, _mm_unpacklo_epi64(t1, t1)) + +#define UNDIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h) \ + t0 = row3l;\ + row3l = row3h;\ + row3h = t0;\ + t0 = row2l;\ + t1 = row4l;\ + row2l = _mm_unpackhi_epi64(row2h, _mm_unpacklo_epi64(row2l, row2l)); \ + row2h = _mm_unpackhi_epi64(t0, _mm_unpacklo_epi64(row2h, row2h)); \ + row4l = _mm_unpackhi_epi64(row4l, _mm_unpacklo_epi64(row4h, row4h)); \ + row4h = _mm_unpackhi_epi64(row4h, _mm_unpacklo_epi64(t1, t1)) + +#endif + +#if defined(HAVE_SSE41) +#include "blake2b-load-sse41.h" +#else +#include "blake2b-load-sse2.h" +#endif + +#define ROUND(r) \ + LOAD_MSG_ ##r ##_1(b0, b1); \ + G1(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1); \ + LOAD_MSG_ ##r ##_2(b0, b1); \ + G2(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1); \ + DIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h); \ + LOAD_MSG_ ##r ##_3(b0, b1); \ + G1(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1); \ + LOAD_MSG_ ##r ##_4(b0, b1); \ + G2(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h,b0,b1); \ + UNDIAGONALIZE(row1l,row2l,row3l,row4l,row1h,row2h,row3h,row4h); + +#endif + diff --git a/Modules/_blake2/impl/blake2b.c b/Modules/_blake2/impl/blake2b.c new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2b.c @@ -0,0 +1,450 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ + +#include +#include +#include + +#include "blake2.h" +#include "blake2-impl.h" + +#include "blake2-config.h" + +#ifdef _MSC_VER +#include /* for _mm_set_epi64x */ +#endif +#include +#if defined(HAVE_SSSE3) +#include +#endif +#if defined(HAVE_SSE41) +#include +#endif +#if defined(HAVE_AVX) +#include +#endif +#if defined(HAVE_XOP) +#include +#endif + +#include "blake2b-round.h" + +static const uint64_t blake2b_IV[8] = +{ + 0x6a09e667f3bcc908ULL, 0xbb67ae8584caa73bULL, + 0x3c6ef372fe94f82bULL, 0xa54ff53a5f1d36f1ULL, + 0x510e527fade682d1ULL, 0x9b05688c2b3e6c1fULL, + 0x1f83d9abfb41bd6bULL, 0x5be0cd19137e2179ULL +}; + +static const uint8_t blake2b_sigma[12][16] = +{ + { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } , + { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 } , + { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 } , + { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 } , + { 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 } , + { 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 } , + { 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 } , + { 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 } , + { 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 } , + { 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13 , 0 } , + { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } , + { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 } +}; + + +/* Some helper functions, not necessarily useful */ +BLAKE2_LOCAL_INLINE(int) blake2b_set_lastnode( blake2b_state *S ) +{ + S->f[1] = -1; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_clear_lastnode( blake2b_state *S ) +{ + S->f[1] = 0; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_is_lastblock( const blake2b_state *S ) +{ + return S->f[0] != 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_set_lastblock( blake2b_state *S ) +{ + if( S->last_node ) blake2b_set_lastnode( S ); + + S->f[0] = -1; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_clear_lastblock( blake2b_state *S ) +{ + if( S->last_node ) blake2b_clear_lastnode( S ); + + S->f[0] = 0; + return 0; +} + + +BLAKE2_LOCAL_INLINE(int) blake2b_increment_counter( blake2b_state *S, const uint64_t inc ) +{ +#if __x86_64__ + /* ADD/ADC chain */ + __uint128_t t = ( ( __uint128_t )S->t[1] << 64 ) | S->t[0]; + t += inc; + S->t[0] = ( uint64_t )( t >> 0 ); + S->t[1] = ( uint64_t )( t >> 64 ); +#else + S->t[0] += inc; + S->t[1] += ( S->t[0] < inc ); +#endif + return 0; +} + + +/* Parameter-related functions */ +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_digest_length( blake2b_param *P, const uint8_t digest_length ) +{ + P->digest_length = digest_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_fanout( blake2b_param *P, const uint8_t fanout ) +{ + P->fanout = fanout; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_max_depth( blake2b_param *P, const uint8_t depth ) +{ + P->depth = depth; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_leaf_length( blake2b_param *P, const uint32_t leaf_length ) +{ + P->leaf_length = leaf_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_node_offset( blake2b_param *P, const uint64_t node_offset ) +{ + P->node_offset = node_offset; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_node_depth( blake2b_param *P, const uint8_t node_depth ) +{ + P->node_depth = node_depth; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_inner_length( blake2b_param *P, const uint8_t inner_length ) +{ + P->inner_length = inner_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_salt( blake2b_param *P, const uint8_t salt[BLAKE2B_SALTBYTES] ) +{ + memcpy( P->salt, salt, BLAKE2B_SALTBYTES ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_param_set_personal( blake2b_param *P, const uint8_t personal[BLAKE2B_PERSONALBYTES] ) +{ + memcpy( P->personal, personal, BLAKE2B_PERSONALBYTES ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_init0( blake2b_state *S ) +{ + memset( S, 0, sizeof( blake2b_state ) ); + + for( int i = 0; i < 8; ++i ) S->h[i] = blake2b_IV[i]; + + return 0; +} + +/* init xors IV with input parameter block */ +int blake2b_init_param( blake2b_state *S, const blake2b_param *P ) +{ + /*blake2b_init0( S ); */ + const uint8_t * v = ( const uint8_t * )( blake2b_IV ); + const uint8_t * p = ( const uint8_t * )( P ); + uint8_t * h = ( uint8_t * )( S->h ); + /* IV XOR ParamBlock */ + memset( S, 0, sizeof( blake2b_state ) ); + + for( int i = 0; i < BLAKE2B_OUTBYTES; ++i ) h[i] = v[i] ^ p[i]; + + return 0; +} + + +/* Some sort of default parameter block initialization, for sequential blake2b */ +int blake2b_init( blake2b_state *S, const uint8_t outlen ) +{ + const blake2b_param P = + { + outlen, + 0, + 1, + 1, + 0, + 0, + 0, + 0, + {0}, + {0}, + {0} + }; + + if ( ( !outlen ) || ( outlen > BLAKE2B_OUTBYTES ) ) return -1; + + return blake2b_init_param( S, &P ); +} + +int blake2b_init_key( blake2b_state *S, const uint8_t outlen, const void *key, const uint8_t keylen ) +{ + const blake2b_param P = + { + outlen, + keylen, + 1, + 1, + 0, + 0, + 0, + 0, + {0}, + {0}, + {0} + }; + + if ( ( !outlen ) || ( outlen > BLAKE2B_OUTBYTES ) ) return -1; + + if ( ( !keylen ) || keylen > BLAKE2B_KEYBYTES ) return -1; + + if( blake2b_init_param( S, &P ) < 0 ) + return 0; + + { + uint8_t block[BLAKE2B_BLOCKBYTES]; + memset( block, 0, BLAKE2B_BLOCKBYTES ); + memcpy( block, key, keylen ); + blake2b_update( S, block, BLAKE2B_BLOCKBYTES ); + secure_zero_memory( block, BLAKE2B_BLOCKBYTES ); /* Burn the key from stack */ + } + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2b_compress( blake2b_state *S, const uint8_t block[BLAKE2B_BLOCKBYTES] ) +{ + __m128i row1l, row1h; + __m128i row2l, row2h; + __m128i row3l, row3h; + __m128i row4l, row4h; + __m128i b0, b1; + __m128i t0, t1; +#if defined(HAVE_SSSE3) && !defined(HAVE_XOP) + const __m128i r16 = _mm_setr_epi8( 2, 3, 4, 5, 6, 7, 0, 1, 10, 11, 12, 13, 14, 15, 8, 9 ); + const __m128i r24 = _mm_setr_epi8( 3, 4, 5, 6, 7, 0, 1, 2, 11, 12, 13, 14, 15, 8, 9, 10 ); +#endif +#if defined(HAVE_SSE41) + const __m128i m0 = LOADU( block + 00 ); + const __m128i m1 = LOADU( block + 16 ); + const __m128i m2 = LOADU( block + 32 ); + const __m128i m3 = LOADU( block + 48 ); + const __m128i m4 = LOADU( block + 64 ); + const __m128i m5 = LOADU( block + 80 ); + const __m128i m6 = LOADU( block + 96 ); + const __m128i m7 = LOADU( block + 112 ); +#else + const uint64_t m0 = ( ( uint64_t * )block )[ 0]; + const uint64_t m1 = ( ( uint64_t * )block )[ 1]; + const uint64_t m2 = ( ( uint64_t * )block )[ 2]; + const uint64_t m3 = ( ( uint64_t * )block )[ 3]; + const uint64_t m4 = ( ( uint64_t * )block )[ 4]; + const uint64_t m5 = ( ( uint64_t * )block )[ 5]; + const uint64_t m6 = ( ( uint64_t * )block )[ 6]; + const uint64_t m7 = ( ( uint64_t * )block )[ 7]; + const uint64_t m8 = ( ( uint64_t * )block )[ 8]; + const uint64_t m9 = ( ( uint64_t * )block )[ 9]; + const uint64_t m10 = ( ( uint64_t * )block )[10]; + const uint64_t m11 = ( ( uint64_t * )block )[11]; + const uint64_t m12 = ( ( uint64_t * )block )[12]; + const uint64_t m13 = ( ( uint64_t * )block )[13]; + const uint64_t m14 = ( ( uint64_t * )block )[14]; + const uint64_t m15 = ( ( uint64_t * )block )[15]; +#endif + row1l = LOADU( &S->h[0] ); + row1h = LOADU( &S->h[2] ); + row2l = LOADU( &S->h[4] ); + row2h = LOADU( &S->h[6] ); + row3l = LOADU( &blake2b_IV[0] ); + row3h = LOADU( &blake2b_IV[2] ); + row4l = _mm_xor_si128( LOADU( &blake2b_IV[4] ), LOADU( &S->t[0] ) ); + row4h = _mm_xor_si128( LOADU( &blake2b_IV[6] ), LOADU( &S->f[0] ) ); + ROUND( 0 ); + ROUND( 1 ); + ROUND( 2 ); + ROUND( 3 ); + ROUND( 4 ); + ROUND( 5 ); + ROUND( 6 ); + ROUND( 7 ); + ROUND( 8 ); + ROUND( 9 ); + ROUND( 10 ); + ROUND( 11 ); + row1l = _mm_xor_si128( row3l, row1l ); + row1h = _mm_xor_si128( row3h, row1h ); + STOREU( &S->h[0], _mm_xor_si128( LOADU( &S->h[0] ), row1l ) ); + STOREU( &S->h[2], _mm_xor_si128( LOADU( &S->h[2] ), row1h ) ); + row2l = _mm_xor_si128( row4l, row2l ); + row2h = _mm_xor_si128( row4h, row2h ); + STOREU( &S->h[4], _mm_xor_si128( LOADU( &S->h[4] ), row2l ) ); + STOREU( &S->h[6], _mm_xor_si128( LOADU( &S->h[6] ), row2h ) ); + return 0; +} + + +int blake2b_update( blake2b_state *S, const uint8_t *in, uint64_t inlen ) +{ + while( inlen > 0 ) + { + size_t left = S->buflen; + size_t fill = 2 * BLAKE2B_BLOCKBYTES - left; + + if( inlen > fill ) + { + memcpy( S->buf + left, in, fill ); /* Fill buffer */ + S->buflen += fill; + blake2b_increment_counter( S, BLAKE2B_BLOCKBYTES ); + blake2b_compress( S, S->buf ); /* Compress */ + memcpy( S->buf, S->buf + BLAKE2B_BLOCKBYTES, BLAKE2B_BLOCKBYTES ); /* Shift buffer left */ + S->buflen -= BLAKE2B_BLOCKBYTES; + in += fill; + inlen -= fill; + } + else /* inlen <= fill */ + { + memcpy( S->buf + left, in, inlen ); + S->buflen += inlen; /* Be lazy, do not compress */ + in += inlen; + inlen -= inlen; + } + } + + return 0; +} + + +int blake2b_final( blake2b_state *S, uint8_t *out, uint8_t outlen ) +{ + if( outlen > BLAKE2B_OUTBYTES ) + return -1; + + if( blake2b_is_lastblock( S ) ) + return -1; + + if( S->buflen > BLAKE2B_BLOCKBYTES ) + { + blake2b_increment_counter( S, BLAKE2B_BLOCKBYTES ); + blake2b_compress( S, S->buf ); + S->buflen -= BLAKE2B_BLOCKBYTES; + memcpy( S->buf, S->buf + BLAKE2B_BLOCKBYTES, S->buflen ); + } + + blake2b_increment_counter( S, S->buflen ); + blake2b_set_lastblock( S ); + memset( S->buf + S->buflen, 0, 2 * BLAKE2B_BLOCKBYTES - S->buflen ); /* Padding */ + blake2b_compress( S, S->buf ); + memcpy( out, &S->h[0], outlen ); + return 0; +} + + +int blake2b( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen ) +{ + blake2b_state S[1]; + + /* Verify parameters */ + if ( NULL == in && inlen > 0 ) return -1; + + if ( NULL == out ) return -1; + + if( NULL == key && keylen > 0 ) return -1; + + if( !outlen || outlen > BLAKE2B_OUTBYTES ) return -1; + + if( keylen > BLAKE2B_KEYBYTES ) return -1; + + if( keylen ) + { + if( blake2b_init_key( S, outlen, key, keylen ) < 0 ) return -1; + } + else + { + if( blake2b_init( S, outlen ) < 0 ) return -1; + } + + blake2b_update( S, ( const uint8_t * )in, inlen ); + blake2b_final( S, out, outlen ); + return 0; +} + +#if defined(SUPERCOP) +int crypto_hash( unsigned char *out, unsigned char *in, unsigned long long inlen ) +{ + return blake2b( out, in, NULL, BLAKE2B_OUTBYTES, inlen, 0 ); +} +#endif + +#if defined(BLAKE2B_SELFTEST) +#include +#include "blake2-kat.h" +int main( int argc, char **argv ) +{ + uint8_t key[BLAKE2B_KEYBYTES]; + uint8_t buf[KAT_LENGTH]; + + for( size_t i = 0; i < BLAKE2B_KEYBYTES; ++i ) + key[i] = ( uint8_t )i; + + for( size_t i = 0; i < KAT_LENGTH; ++i ) + buf[i] = ( uint8_t )i; + + for( size_t i = 0; i < KAT_LENGTH; ++i ) + { + uint8_t hash[BLAKE2B_OUTBYTES]; + blake2b( hash, buf, key, BLAKE2B_OUTBYTES, i, BLAKE2B_KEYBYTES ); + + if( 0 != memcmp( hash, blake2b_keyed_kat[i], BLAKE2B_OUTBYTES ) ) + { + puts( "error" ); + return -1; + } + } + + puts( "ok" ); + return 0; +} +#endif + diff --git a/Modules/_blake2/impl/blake2s-load-sse2.h b/Modules/_blake2/impl/blake2s-load-sse2.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2s-load-sse2.h @@ -0,0 +1,61 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2S_LOAD_SSE2_H__ +#define __BLAKE2S_LOAD_SSE2_H__ + +#define LOAD_MSG_0_1(buf) buf = _mm_set_epi32(m6,m4,m2,m0) +#define LOAD_MSG_0_2(buf) buf = _mm_set_epi32(m7,m5,m3,m1) +#define LOAD_MSG_0_3(buf) buf = _mm_set_epi32(m14,m12,m10,m8) +#define LOAD_MSG_0_4(buf) buf = _mm_set_epi32(m15,m13,m11,m9) +#define LOAD_MSG_1_1(buf) buf = _mm_set_epi32(m13,m9,m4,m14) +#define LOAD_MSG_1_2(buf) buf = _mm_set_epi32(m6,m15,m8,m10) +#define LOAD_MSG_1_3(buf) buf = _mm_set_epi32(m5,m11,m0,m1) +#define LOAD_MSG_1_4(buf) buf = _mm_set_epi32(m3,m7,m2,m12) +#define LOAD_MSG_2_1(buf) buf = _mm_set_epi32(m15,m5,m12,m11) +#define LOAD_MSG_2_2(buf) buf = _mm_set_epi32(m13,m2,m0,m8) +#define LOAD_MSG_2_3(buf) buf = _mm_set_epi32(m9,m7,m3,m10) +#define LOAD_MSG_2_4(buf) buf = _mm_set_epi32(m4,m1,m6,m14) +#define LOAD_MSG_3_1(buf) buf = _mm_set_epi32(m11,m13,m3,m7) +#define LOAD_MSG_3_2(buf) buf = _mm_set_epi32(m14,m12,m1,m9) +#define LOAD_MSG_3_3(buf) buf = _mm_set_epi32(m15,m4,m5,m2) +#define LOAD_MSG_3_4(buf) buf = _mm_set_epi32(m8,m0,m10,m6) +#define LOAD_MSG_4_1(buf) buf = _mm_set_epi32(m10,m2,m5,m9) +#define LOAD_MSG_4_2(buf) buf = _mm_set_epi32(m15,m4,m7,m0) +#define LOAD_MSG_4_3(buf) buf = _mm_set_epi32(m3,m6,m11,m14) +#define LOAD_MSG_4_4(buf) buf = _mm_set_epi32(m13,m8,m12,m1) +#define LOAD_MSG_5_1(buf) buf = _mm_set_epi32(m8,m0,m6,m2) +#define LOAD_MSG_5_2(buf) buf = _mm_set_epi32(m3,m11,m10,m12) +#define LOAD_MSG_5_3(buf) buf = _mm_set_epi32(m1,m15,m7,m4) +#define LOAD_MSG_5_4(buf) buf = _mm_set_epi32(m9,m14,m5,m13) +#define LOAD_MSG_6_1(buf) buf = _mm_set_epi32(m4,m14,m1,m12) +#define LOAD_MSG_6_2(buf) buf = _mm_set_epi32(m10,m13,m15,m5) +#define LOAD_MSG_6_3(buf) buf = _mm_set_epi32(m8,m9,m6,m0) +#define LOAD_MSG_6_4(buf) buf = _mm_set_epi32(m11,m2,m3,m7) +#define LOAD_MSG_7_1(buf) buf = _mm_set_epi32(m3,m12,m7,m13) +#define LOAD_MSG_7_2(buf) buf = _mm_set_epi32(m9,m1,m14,m11) +#define LOAD_MSG_7_3(buf) buf = _mm_set_epi32(m2,m8,m15,m5) +#define LOAD_MSG_7_4(buf) buf = _mm_set_epi32(m10,m6,m4,m0) +#define LOAD_MSG_8_1(buf) buf = _mm_set_epi32(m0,m11,m14,m6) +#define LOAD_MSG_8_2(buf) buf = _mm_set_epi32(m8,m3,m9,m15) +#define LOAD_MSG_8_3(buf) buf = _mm_set_epi32(m10,m1,m13,m12) +#define LOAD_MSG_8_4(buf) buf = _mm_set_epi32(m5,m4,m7,m2) +#define LOAD_MSG_9_1(buf) buf = _mm_set_epi32(m1,m7,m8,m10) +#define LOAD_MSG_9_2(buf) buf = _mm_set_epi32(m5,m6,m4,m2) +#define LOAD_MSG_9_3(buf) buf = _mm_set_epi32(m13,m3,m9,m15) +#define LOAD_MSG_9_4(buf) buf = _mm_set_epi32(m0,m12,m14,m11) + + +#endif diff --git a/Modules/_blake2/impl/blake2s-load-sse41.h b/Modules/_blake2/impl/blake2s-load-sse41.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2s-load-sse41.h @@ -0,0 +1,231 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2S_LOAD_SSE41_H__ +#define __BLAKE2S_LOAD_SSE41_H__ + +#define LOAD_MSG_0_1(buf) \ +buf = TOI(_mm_shuffle_ps(TOF(m0), TOF(m1), _MM_SHUFFLE(2,0,2,0))); + +#define LOAD_MSG_0_2(buf) \ +buf = TOI(_mm_shuffle_ps(TOF(m0), TOF(m1), _MM_SHUFFLE(3,1,3,1))); + +#define LOAD_MSG_0_3(buf) \ +buf = TOI(_mm_shuffle_ps(TOF(m2), TOF(m3), _MM_SHUFFLE(2,0,2,0))); + +#define LOAD_MSG_0_4(buf) \ +buf = TOI(_mm_shuffle_ps(TOF(m2), TOF(m3), _MM_SHUFFLE(3,1,3,1))); + +#define LOAD_MSG_1_1(buf) \ +t0 = _mm_blend_epi16(m1, m2, 0x0C); \ +t1 = _mm_slli_si128(m3, 4); \ +t2 = _mm_blend_epi16(t0, t1, 0xF0); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(2,1,0,3)); + +#define LOAD_MSG_1_2(buf) \ +t0 = _mm_shuffle_epi32(m2,_MM_SHUFFLE(0,0,2,0)); \ +t1 = _mm_blend_epi16(m1,m3,0xC0); \ +t2 = _mm_blend_epi16(t0, t1, 0xF0); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(2,3,0,1)); + +#define LOAD_MSG_1_3(buf) \ +t0 = _mm_slli_si128(m1, 4); \ +t1 = _mm_blend_epi16(m2, t0, 0x30); \ +t2 = _mm_blend_epi16(m0, t1, 0xF0); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(2,3,0,1)); + +#define LOAD_MSG_1_4(buf) \ +t0 = _mm_unpackhi_epi32(m0,m1); \ +t1 = _mm_slli_si128(m3, 4); \ +t2 = _mm_blend_epi16(t0, t1, 0x0C); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(2,3,0,1)); + +#define LOAD_MSG_2_1(buf) \ +t0 = _mm_unpackhi_epi32(m2,m3); \ +t1 = _mm_blend_epi16(m3,m1,0x0C); \ +t2 = _mm_blend_epi16(t0, t1, 0x0F); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(3,1,0,2)); + +#define LOAD_MSG_2_2(buf) \ +t0 = _mm_unpacklo_epi32(m2,m0); \ +t1 = _mm_blend_epi16(t0, m0, 0xF0); \ +t2 = _mm_slli_si128(m3, 8); \ +buf = _mm_blend_epi16(t1, t2, 0xC0); + +#define LOAD_MSG_2_3(buf) \ +t0 = _mm_blend_epi16(m0, m2, 0x3C); \ +t1 = _mm_srli_si128(m1, 12); \ +t2 = _mm_blend_epi16(t0,t1,0x03); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(1,0,3,2)); + +#define LOAD_MSG_2_4(buf) \ +t0 = _mm_slli_si128(m3, 4); \ +t1 = _mm_blend_epi16(m0, m1, 0x33); \ +t2 = _mm_blend_epi16(t1, t0, 0xC0); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(0,1,2,3)); + +#define LOAD_MSG_3_1(buf) \ +t0 = _mm_unpackhi_epi32(m0,m1); \ +t1 = _mm_unpackhi_epi32(t0, m2); \ +t2 = _mm_blend_epi16(t1, m3, 0x0C); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(3,1,0,2)); + +#define LOAD_MSG_3_2(buf) \ +t0 = _mm_slli_si128(m2, 8); \ +t1 = _mm_blend_epi16(m3,m0,0x0C); \ +t2 = _mm_blend_epi16(t1, t0, 0xC0); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(2,0,1,3)); + +#define LOAD_MSG_3_3(buf) \ +t0 = _mm_blend_epi16(m0,m1,0x0F); \ +t1 = _mm_blend_epi16(t0, m3, 0xC0); \ +buf = _mm_shuffle_epi32(t1, _MM_SHUFFLE(3,0,1,2)); + +#define LOAD_MSG_3_4(buf) \ +t0 = _mm_unpacklo_epi32(m0,m2); \ +t1 = _mm_unpackhi_epi32(m1,m2); \ +buf = _mm_unpacklo_epi64(t1,t0); + +#define LOAD_MSG_4_1(buf) \ +t0 = _mm_unpacklo_epi64(m1,m2); \ +t1 = _mm_unpackhi_epi64(m0,m2); \ +t2 = _mm_blend_epi16(t0,t1,0x33); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(2,0,1,3)); + +#define LOAD_MSG_4_2(buf) \ +t0 = _mm_unpackhi_epi64(m1,m3); \ +t1 = _mm_unpacklo_epi64(m0,m1); \ +buf = _mm_blend_epi16(t0,t1,0x33); + +#define LOAD_MSG_4_3(buf) \ +t0 = _mm_unpackhi_epi64(m3,m1); \ +t1 = _mm_unpackhi_epi64(m2,m0); \ +buf = _mm_blend_epi16(t1,t0,0x33); + +#define LOAD_MSG_4_4(buf) \ +t0 = _mm_blend_epi16(m0,m2,0x03); \ +t1 = _mm_slli_si128(t0, 8); \ +t2 = _mm_blend_epi16(t1,m3,0x0F); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(1,2,0,3)); + +#define LOAD_MSG_5_1(buf) \ +t0 = _mm_unpackhi_epi32(m0,m1); \ +t1 = _mm_unpacklo_epi32(m0,m2); \ +buf = _mm_unpacklo_epi64(t0,t1); + +#define LOAD_MSG_5_2(buf) \ +t0 = _mm_srli_si128(m2, 4); \ +t1 = _mm_blend_epi16(m0,m3,0x03); \ +buf = _mm_blend_epi16(t1,t0,0x3C); + +#define LOAD_MSG_5_3(buf) \ +t0 = _mm_blend_epi16(m1,m0,0x0C); \ +t1 = _mm_srli_si128(m3, 4); \ +t2 = _mm_blend_epi16(t0,t1,0x30); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(1,2,3,0)); + +#define LOAD_MSG_5_4(buf) \ +t0 = _mm_unpacklo_epi64(m1,m2); \ +t1= _mm_shuffle_epi32(m3, _MM_SHUFFLE(0,2,0,1)); \ +buf = _mm_blend_epi16(t0,t1,0x33); + +#define LOAD_MSG_6_1(buf) \ +t0 = _mm_slli_si128(m1, 12); \ +t1 = _mm_blend_epi16(m0,m3,0x33); \ +buf = _mm_blend_epi16(t1,t0,0xC0); + +#define LOAD_MSG_6_2(buf) \ +t0 = _mm_blend_epi16(m3,m2,0x30); \ +t1 = _mm_srli_si128(m1, 4); \ +t2 = _mm_blend_epi16(t0,t1,0x03); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(2,1,3,0)); + +#define LOAD_MSG_6_3(buf) \ +t0 = _mm_unpacklo_epi64(m0,m2); \ +t1 = _mm_srli_si128(m1, 4); \ +buf = _mm_shuffle_epi32(_mm_blend_epi16(t0,t1,0x0C), _MM_SHUFFLE(2,3,1,0)); + +#define LOAD_MSG_6_4(buf) \ +t0 = _mm_unpackhi_epi32(m1,m2); \ +t1 = _mm_unpackhi_epi64(m0,t0); \ +buf = _mm_shuffle_epi32(t1, _MM_SHUFFLE(3,0,1,2)); + +#define LOAD_MSG_7_1(buf) \ +t0 = _mm_unpackhi_epi32(m0,m1); \ +t1 = _mm_blend_epi16(t0,m3,0x0F); \ +buf = _mm_shuffle_epi32(t1,_MM_SHUFFLE(2,0,3,1)); + +#define LOAD_MSG_7_2(buf) \ +t0 = _mm_blend_epi16(m2,m3,0x30); \ +t1 = _mm_srli_si128(m0,4); \ +t2 = _mm_blend_epi16(t0,t1,0x03); \ +buf = _mm_shuffle_epi32(t2, _MM_SHUFFLE(1,0,2,3)); + +#define LOAD_MSG_7_3(buf) \ +t0 = _mm_unpackhi_epi64(m0,m3); \ +t1 = _mm_unpacklo_epi64(m1,m2); \ +t2 = _mm_blend_epi16(t0,t1,0x3C); \ +buf = _mm_shuffle_epi32(t2,_MM_SHUFFLE(0,2,3,1)); + +#define LOAD_MSG_7_4(buf) \ +t0 = _mm_unpacklo_epi32(m0,m1); \ +t1 = _mm_unpackhi_epi32(m1,m2); \ +buf = _mm_unpacklo_epi64(t0,t1); + +#define LOAD_MSG_8_1(buf) \ +t0 = _mm_unpackhi_epi32(m1,m3); \ +t1 = _mm_unpacklo_epi64(t0,m0); \ +t2 = _mm_blend_epi16(t1,m2,0xC0); \ +buf = _mm_shufflehi_epi16(t2,_MM_SHUFFLE(1,0,3,2)); + +#define LOAD_MSG_8_2(buf) \ +t0 = _mm_unpackhi_epi32(m0,m3); \ +t1 = _mm_blend_epi16(m2,t0,0xF0); \ +buf = _mm_shuffle_epi32(t1,_MM_SHUFFLE(0,2,1,3)); + +#define LOAD_MSG_8_3(buf) \ +t0 = _mm_blend_epi16(m2,m0,0x0C); \ +t1 = _mm_slli_si128(t0,4); \ +buf = _mm_blend_epi16(t1,m3,0x0F); + +#define LOAD_MSG_8_4(buf) \ +t0 = _mm_blend_epi16(m1,m0,0x30); \ +buf = _mm_shuffle_epi32(t0,_MM_SHUFFLE(1,0,3,2)); + +#define LOAD_MSG_9_1(buf) \ +t0 = _mm_blend_epi16(m0,m2,0x03); \ +t1 = _mm_blend_epi16(m1,m2,0x30); \ +t2 = _mm_blend_epi16(t1,t0,0x0F); \ +buf = _mm_shuffle_epi32(t2,_MM_SHUFFLE(1,3,0,2)); + +#define LOAD_MSG_9_2(buf) \ +t0 = _mm_slli_si128(m0,4); \ +t1 = _mm_blend_epi16(m1,t0,0xC0); \ +buf = _mm_shuffle_epi32(t1,_MM_SHUFFLE(1,2,0,3)); + +#define LOAD_MSG_9_3(buf) \ +t0 = _mm_unpackhi_epi32(m0,m3); \ +t1 = _mm_unpacklo_epi32(m2,m3); \ +t2 = _mm_unpackhi_epi64(t0,t1); \ +buf = _mm_shuffle_epi32(t2,_MM_SHUFFLE(3,0,2,1)); + +#define LOAD_MSG_9_4(buf) \ +t0 = _mm_blend_epi16(m3,m2,0xC0); \ +t1 = _mm_unpacklo_epi32(m0,m3); \ +t2 = _mm_blend_epi16(t0,t1,0x0F); \ +buf = _mm_shuffle_epi32(t2,_MM_SHUFFLE(0,1,2,3)); + +#endif + diff --git a/Modules/_blake2/impl/blake2s-load-xop.h b/Modules/_blake2/impl/blake2s-load-xop.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2s-load-xop.h @@ -0,0 +1,191 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2S_LOAD_XOP_H__ +#define __BLAKE2S_LOAD_XOP_H__ + +#define TOB(x) ((x)*4*0x01010101 + 0x03020100) /* ..or not TOB */ + +/* Basic VPPERM emulation, for testing purposes */ +/*static __m128i _mm_perm_epi8(const __m128i src1, const __m128i src2, const __m128i sel) +{ + const __m128i sixteen = _mm_set1_epi8(16); + const __m128i t0 = _mm_shuffle_epi8(src1, sel); + const __m128i s1 = _mm_shuffle_epi8(src2, _mm_sub_epi8(sel, sixteen)); + const __m128i mask = _mm_or_si128(_mm_cmpeq_epi8(sel, sixteen), + _mm_cmpgt_epi8(sel, sixteen)); /* (>=16) = 0xff : 00 */ + return _mm_blendv_epi8(t0, s1, mask); +}*/ + +#define LOAD_MSG_0_1(buf) \ +buf = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(6),TOB(4),TOB(2),TOB(0)) ); + +#define LOAD_MSG_0_2(buf) \ +buf = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(7),TOB(5),TOB(3),TOB(1)) ); + +#define LOAD_MSG_0_3(buf) \ +buf = _mm_perm_epi8(m2, m3, _mm_set_epi32(TOB(6),TOB(4),TOB(2),TOB(0)) ); + +#define LOAD_MSG_0_4(buf) \ +buf = _mm_perm_epi8(m2, m3, _mm_set_epi32(TOB(7),TOB(5),TOB(3),TOB(1)) ); + +#define LOAD_MSG_1_1(buf) \ +t0 = _mm_perm_epi8(m1, m2, _mm_set_epi32(TOB(0),TOB(5),TOB(0),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(5),TOB(2),TOB(1),TOB(6)) ); + +#define LOAD_MSG_1_2(buf) \ +t1 = _mm_perm_epi8(m1, m2, _mm_set_epi32(TOB(2),TOB(0),TOB(4),TOB(6)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(3),TOB(7),TOB(1),TOB(0)) ); + +#define LOAD_MSG_1_3(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(5),TOB(0),TOB(0),TOB(1)) ); \ +buf = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(3),TOB(7),TOB(1),TOB(0)) ); + +#define LOAD_MSG_1_4(buf) \ +t1 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(3),TOB(7),TOB(2),TOB(0)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(3),TOB(2),TOB(1),TOB(4)) ); + +#define LOAD_MSG_2_1(buf) \ +t0 = _mm_perm_epi8(m1, m2, _mm_set_epi32(TOB(0),TOB(1),TOB(0),TOB(7)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(7),TOB(2),TOB(4),TOB(0)) ); + +#define LOAD_MSG_2_2(buf) \ +t1 = _mm_perm_epi8(m0, m2, _mm_set_epi32(TOB(0),TOB(2),TOB(0),TOB(4)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(5),TOB(2),TOB(1),TOB(0)) ); + +#define LOAD_MSG_2_3(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(7),TOB(3),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(5),TOB(2),TOB(1),TOB(6)) ); + +#define LOAD_MSG_2_4(buf) \ +t1 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(4),TOB(1),TOB(6),TOB(0)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(3),TOB(2),TOB(1),TOB(6)) ); + +#define LOAD_MSG_3_1(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(0),TOB(3),TOB(7)) ); \ +t0 = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(7),TOB(2),TOB(1),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(3),TOB(5),TOB(1),TOB(0)) ); + +#define LOAD_MSG_3_2(buf) \ +t1 = _mm_perm_epi8(m0, m2, _mm_set_epi32(TOB(0),TOB(0),TOB(1),TOB(5)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(6),TOB(4),TOB(1),TOB(0)) ); + +#define LOAD_MSG_3_3(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(4),TOB(5),TOB(2)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(7),TOB(2),TOB(1),TOB(0)) ); + +#define LOAD_MSG_3_4(buf) \ +t1 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(0),TOB(0),TOB(6)) ); \ +buf = _mm_perm_epi8(t1, m2, _mm_set_epi32(TOB(4),TOB(2),TOB(6),TOB(0)) ); + +#define LOAD_MSG_4_1(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(2),TOB(5),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(6),TOB(2),TOB(1),TOB(5)) ); + +#define LOAD_MSG_4_2(buf) \ +t1 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(4),TOB(7),TOB(0)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(7),TOB(2),TOB(1),TOB(0)) ); + +#define LOAD_MSG_4_3(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(3),TOB(6),TOB(0),TOB(0)) ); \ +t0 = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(3),TOB(2),TOB(7),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(3),TOB(2),TOB(1),TOB(6)) ); + +#define LOAD_MSG_4_4(buf) \ +t1 = _mm_perm_epi8(m0, m2, _mm_set_epi32(TOB(0),TOB(4),TOB(0),TOB(1)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(5),TOB(2),TOB(4),TOB(0)) ); + +#define LOAD_MSG_5_1(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(0),TOB(6),TOB(2)) ); \ +buf = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(4),TOB(2),TOB(1),TOB(0)) ); + +#define LOAD_MSG_5_2(buf) \ +t1 = _mm_perm_epi8(m0, m2, _mm_set_epi32(TOB(3),TOB(7),TOB(6),TOB(0)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(3),TOB(2),TOB(1),TOB(4)) ); + +#define LOAD_MSG_5_3(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(1),TOB(0),TOB(7),TOB(4)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(3),TOB(7),TOB(1),TOB(0)) ); + +#define LOAD_MSG_5_4(buf) \ +t1 = _mm_perm_epi8(m1, m2, _mm_set_epi32(TOB(5),TOB(0),TOB(1),TOB(0)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(3),TOB(6),TOB(1),TOB(5)) ); + +#define LOAD_MSG_6_1(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(4),TOB(0),TOB(1),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(3),TOB(6),TOB(1),TOB(4)) ); + +#define LOAD_MSG_6_2(buf) \ +t1 = _mm_perm_epi8(m1, m2, _mm_set_epi32(TOB(6),TOB(0),TOB(0),TOB(1)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(3),TOB(5),TOB(7),TOB(0)) ); + +#define LOAD_MSG_6_3(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(0),TOB(6),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(4),TOB(5),TOB(1),TOB(0)) ); + +#define LOAD_MSG_6_4(buf) \ +t1 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(2),TOB(3),TOB(7)) ); \ +buf = _mm_perm_epi8(t1, m2, _mm_set_epi32(TOB(7),TOB(2),TOB(1),TOB(0)) ); + +#define LOAD_MSG_7_1(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(3),TOB(0),TOB(7),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(3),TOB(4),TOB(1),TOB(5)) ); + +#define LOAD_MSG_7_2(buf) \ +t1 = _mm_perm_epi8(m0, m2, _mm_set_epi32(TOB(5),TOB(1),TOB(0),TOB(7)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(3),TOB(2),TOB(6),TOB(0)) ); + +#define LOAD_MSG_7_3(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(2),TOB(0),TOB(0),TOB(5)) ); \ +t0 = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(3),TOB(4),TOB(1),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(3),TOB(2),TOB(7),TOB(0)) ); + +#define LOAD_MSG_7_4(buf) \ +t1 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(6),TOB(4),TOB(0)) ); \ +buf = _mm_perm_epi8(t1, m2, _mm_set_epi32(TOB(6),TOB(2),TOB(1),TOB(0)) ); + +#define LOAD_MSG_8_1(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(0),TOB(0),TOB(0),TOB(6)) ); \ +t0 = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(3),TOB(7),TOB(1),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(3),TOB(2),TOB(6),TOB(0)) ); + +#define LOAD_MSG_8_2(buf) \ +t1 = _mm_perm_epi8(m0, m2, _mm_set_epi32(TOB(4),TOB(3),TOB(5),TOB(0)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(3),TOB(2),TOB(1),TOB(7)) ); + +#define LOAD_MSG_8_3(buf) \ +t0 = _mm_perm_epi8(m0, m2, _mm_set_epi32(TOB(6),TOB(1),TOB(0),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(3),TOB(2),TOB(5),TOB(4)) ); \ + +#define LOAD_MSG_8_4(buf) \ +buf = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(5),TOB(4),TOB(7),TOB(2)) ); + +#define LOAD_MSG_9_1(buf) \ +t0 = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(1),TOB(7),TOB(0),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m2, _mm_set_epi32(TOB(3),TOB(2),TOB(4),TOB(6)) ); + +#define LOAD_MSG_9_2(buf) \ +buf = _mm_perm_epi8(m0, m1, _mm_set_epi32(TOB(5),TOB(6),TOB(4),TOB(2)) ); + +#define LOAD_MSG_9_3(buf) \ +t0 = _mm_perm_epi8(m0, m2, _mm_set_epi32(TOB(0),TOB(3),TOB(5),TOB(0)) ); \ +buf = _mm_perm_epi8(t0, m3, _mm_set_epi32(TOB(5),TOB(2),TOB(1),TOB(7)) ); + +#define LOAD_MSG_9_4(buf) \ +t1 = _mm_perm_epi8(m0, m2, _mm_set_epi32(TOB(0),TOB(0),TOB(0),TOB(7)) ); \ +buf = _mm_perm_epi8(t1, m3, _mm_set_epi32(TOB(3),TOB(4),TOB(6),TOB(0)) ); + +#endif + diff --git a/Modules/_blake2/impl/blake2s-ref.c b/Modules/_blake2/impl/blake2s-ref.c new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2s-ref.c @@ -0,0 +1,406 @@ +/* + BLAKE2 reference source code package - reference C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ + +#include +#include +#include + +#include "blake2.h" +#include "blake2-impl.h" + +static const uint32_t blake2s_IV[8] = +{ + 0x6A09E667UL, 0xBB67AE85UL, 0x3C6EF372UL, 0xA54FF53AUL, + 0x510E527FUL, 0x9B05688CUL, 0x1F83D9ABUL, 0x5BE0CD19UL +}; + +static const uint8_t blake2s_sigma[10][16] = +{ + { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } , + { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 } , + { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 } , + { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 } , + { 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 } , + { 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 } , + { 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 } , + { 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 } , + { 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 } , + { 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13 , 0 } , +}; + +BLAKE2_LOCAL_INLINE(int) blake2s_set_lastnode( blake2s_state *S ) +{ + S->f[1] = -1; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_clear_lastnode( blake2s_state *S ) +{ + S->f[1] = 0; + return 0; +} + +/* Some helper functions, not necessarily useful */ +BLAKE2_LOCAL_INLINE(int) blake2s_is_lastblock( const blake2s_state *S ) +{ + return S->f[0] != 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_set_lastblock( blake2s_state *S ) +{ + if( S->last_node ) blake2s_set_lastnode( S ); + + S->f[0] = -1; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_clear_lastblock( blake2s_state *S ) +{ + if( S->last_node ) blake2s_clear_lastnode( S ); + + S->f[0] = 0; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_increment_counter( blake2s_state *S, const uint32_t inc ) +{ + S->t[0] += inc; + S->t[1] += ( S->t[0] < inc ); + return 0; +} + +/* Parameter-related functions */ +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_digest_length( blake2s_param *P, const uint8_t digest_length ) +{ + P->digest_length = digest_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_fanout( blake2s_param *P, const uint8_t fanout ) +{ + P->fanout = fanout; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_max_depth( blake2s_param *P, const uint8_t depth ) +{ + P->depth = depth; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_leaf_length( blake2s_param *P, const uint32_t leaf_length ) +{ + store32( &P->leaf_length, leaf_length ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_node_offset( blake2s_param *P, const uint64_t node_offset ) +{ + store48( P->node_offset, node_offset ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_node_depth( blake2s_param *P, const uint8_t node_depth ) +{ + P->node_depth = node_depth; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_inner_length( blake2s_param *P, const uint8_t inner_length ) +{ + P->inner_length = inner_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_salt( blake2s_param *P, const uint8_t salt[BLAKE2S_SALTBYTES] ) +{ + memcpy( P->salt, salt, BLAKE2S_SALTBYTES ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_personal( blake2s_param *P, const uint8_t personal[BLAKE2S_PERSONALBYTES] ) +{ + memcpy( P->personal, personal, BLAKE2S_PERSONALBYTES ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_init0( blake2s_state *S ) +{ + memset( S, 0, sizeof( blake2s_state ) ); + + for( int i = 0; i < 8; ++i ) S->h[i] = blake2s_IV[i]; + + return 0; +} + +/* init2 xors IV with input parameter block */ +int blake2s_init_param( blake2s_state *S, const blake2s_param *P ) +{ + const uint32_t *p = ( const uint32_t * )( P ); + + blake2s_init0( S ); + + /* IV XOR ParamBlock */ + for( size_t i = 0; i < 8; ++i ) + S->h[i] ^= load32( &p[i] ); + + return 0; +} + + +/* Sequential blake2s initialization */ +int blake2s_init( blake2s_state *S, const uint8_t outlen ) +{ + blake2s_param P[1]; + + /* Move interval verification here? */ + if ( ( !outlen ) || ( outlen > BLAKE2S_OUTBYTES ) ) return -1; + + P->digest_length = outlen; + P->key_length = 0; + P->fanout = 1; + P->depth = 1; + store32( &P->leaf_length, 0 ); + store48( &P->node_offset, 0 ); + P->node_depth = 0; + P->inner_length = 0; + /* memset(P->reserved, 0, sizeof(P->reserved) ); */ + memset( P->salt, 0, sizeof( P->salt ) ); + memset( P->personal, 0, sizeof( P->personal ) ); + return blake2s_init_param( S, P ); +} + +int blake2s_init_key( blake2s_state *S, const uint8_t outlen, const void *key, const uint8_t keylen ) +{ + blake2s_param P[1]; + + if ( ( !outlen ) || ( outlen > BLAKE2S_OUTBYTES ) ) return -1; + + if ( !key || !keylen || keylen > BLAKE2S_KEYBYTES ) return -1; + + P->digest_length = outlen; + P->key_length = keylen; + P->fanout = 1; + P->depth = 1; + store32( &P->leaf_length, 0 ); + store48( &P->node_offset, 0 ); + P->node_depth = 0; + P->inner_length = 0; + /* memset(P->reserved, 0, sizeof(P->reserved) ); */ + memset( P->salt, 0, sizeof( P->salt ) ); + memset( P->personal, 0, sizeof( P->personal ) ); + + if( blake2s_init_param( S, P ) < 0 ) return -1; + + { + uint8_t block[BLAKE2S_BLOCKBYTES]; + memset( block, 0, BLAKE2S_BLOCKBYTES ); + memcpy( block, key, keylen ); + blake2s_update( S, block, BLAKE2S_BLOCKBYTES ); + secure_zero_memory( block, BLAKE2S_BLOCKBYTES ); /* Burn the key from stack */ + } + return 0; +} + +static int blake2s_compress( blake2s_state *S, const uint8_t block[BLAKE2S_BLOCKBYTES] ) +{ + uint32_t m[16]; + uint32_t v[16]; + + for( size_t i = 0; i < 16; ++i ) + m[i] = load32( block + i * sizeof( m[i] ) ); + + for( size_t i = 0; i < 8; ++i ) + v[i] = S->h[i]; + + v[ 8] = blake2s_IV[0]; + v[ 9] = blake2s_IV[1]; + v[10] = blake2s_IV[2]; + v[11] = blake2s_IV[3]; + v[12] = S->t[0] ^ blake2s_IV[4]; + v[13] = S->t[1] ^ blake2s_IV[5]; + v[14] = S->f[0] ^ blake2s_IV[6]; + v[15] = S->f[1] ^ blake2s_IV[7]; +#define G(r,i,a,b,c,d) \ + do { \ + a = a + b + m[blake2s_sigma[r][2*i+0]]; \ + d = rotr32(d ^ a, 16); \ + c = c + d; \ + b = rotr32(b ^ c, 12); \ + a = a + b + m[blake2s_sigma[r][2*i+1]]; \ + d = rotr32(d ^ a, 8); \ + c = c + d; \ + b = rotr32(b ^ c, 7); \ + } while(0) +#define ROUND(r) \ + do { \ + G(r,0,v[ 0],v[ 4],v[ 8],v[12]); \ + G(r,1,v[ 1],v[ 5],v[ 9],v[13]); \ + G(r,2,v[ 2],v[ 6],v[10],v[14]); \ + G(r,3,v[ 3],v[ 7],v[11],v[15]); \ + G(r,4,v[ 0],v[ 5],v[10],v[15]); \ + G(r,5,v[ 1],v[ 6],v[11],v[12]); \ + G(r,6,v[ 2],v[ 7],v[ 8],v[13]); \ + G(r,7,v[ 3],v[ 4],v[ 9],v[14]); \ + } while(0) + ROUND( 0 ); + ROUND( 1 ); + ROUND( 2 ); + ROUND( 3 ); + ROUND( 4 ); + ROUND( 5 ); + ROUND( 6 ); + ROUND( 7 ); + ROUND( 8 ); + ROUND( 9 ); + + for( size_t i = 0; i < 8; ++i ) + S->h[i] = S->h[i] ^ v[i] ^ v[i + 8]; + +#undef G +#undef ROUND + return 0; +} + + +int blake2s_update( blake2s_state *S, const uint8_t *in, uint64_t inlen ) +{ + while( inlen > 0 ) + { + size_t left = S->buflen; + size_t fill = 2 * BLAKE2S_BLOCKBYTES - left; + + if( inlen > fill ) + { + memcpy( S->buf + left, in, fill ); /* Fill buffer */ + S->buflen += fill; + blake2s_increment_counter( S, BLAKE2S_BLOCKBYTES ); + blake2s_compress( S, S->buf ); /* Compress */ + memcpy( S->buf, S->buf + BLAKE2S_BLOCKBYTES, BLAKE2S_BLOCKBYTES ); /* Shift buffer left */ + S->buflen -= BLAKE2S_BLOCKBYTES; + in += fill; + inlen -= fill; + } + else /* inlen <= fill */ + { + memcpy( S->buf + left, in, inlen ); + S->buflen += inlen; /* Be lazy, do not compress */ + in += inlen; + inlen -= inlen; + } + } + + return 0; +} + +int blake2s_final( blake2s_state *S, uint8_t *out, uint8_t outlen ) +{ + uint8_t buffer[BLAKE2S_OUTBYTES] = {0}; + + if( out == NULL || outlen == 0 || outlen > BLAKE2S_OUTBYTES ) + return -1; + + if( blake2s_is_lastblock( S ) ) + return -1; + + + if( S->buflen > BLAKE2S_BLOCKBYTES ) + { + blake2s_increment_counter( S, BLAKE2S_BLOCKBYTES ); + blake2s_compress( S, S->buf ); + S->buflen -= BLAKE2S_BLOCKBYTES; + memcpy( S->buf, S->buf + BLAKE2S_BLOCKBYTES, S->buflen ); + } + + blake2s_increment_counter( S, ( uint32_t )S->buflen ); + blake2s_set_lastblock( S ); + memset( S->buf + S->buflen, 0, 2 * BLAKE2S_BLOCKBYTES - S->buflen ); /* Padding */ + blake2s_compress( S, S->buf ); + + for( int i = 0; i < 8; ++i ) /* Output full hash to temp buffer */ + store32( buffer + sizeof( S->h[i] ) * i, S->h[i] ); + + memcpy( out, buffer, outlen ); + return 0; +} + +int blake2s( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen ) +{ + blake2s_state S[1]; + + /* Verify parameters */ + if ( NULL == in && inlen > 0 ) return -1; + + if ( NULL == out ) return -1; + + if ( NULL == key && keylen > 0) return -1; + + if( !outlen || outlen > BLAKE2S_OUTBYTES ) return -1; + + if( keylen > BLAKE2S_KEYBYTES ) return -1; + + if( keylen > 0 ) + { + if( blake2s_init_key( S, outlen, key, keylen ) < 0 ) return -1; + } + else + { + if( blake2s_init( S, outlen ) < 0 ) return -1; + } + + blake2s_update( S, ( const uint8_t * )in, inlen ); + blake2s_final( S, out, outlen ); + return 0; +} + +#if defined(SUPERCOP) +int crypto_hash( unsigned char *out, unsigned char *in, unsigned long long inlen ) +{ + return blake2s( out, in, NULL, BLAKE2S_OUTBYTES, inlen, 0 ); +} +#endif + +#if defined(BLAKE2S_SELFTEST) +#include +#include "blake2-kat.h" +int main( int argc, char **argv ) +{ + uint8_t key[BLAKE2S_KEYBYTES]; + uint8_t buf[KAT_LENGTH]; + + for( size_t i = 0; i < BLAKE2S_KEYBYTES; ++i ) + key[i] = ( uint8_t )i; + + for( size_t i = 0; i < KAT_LENGTH; ++i ) + buf[i] = ( uint8_t )i; + + for( size_t i = 0; i < KAT_LENGTH; ++i ) + { + uint8_t hash[BLAKE2S_OUTBYTES]; + blake2s( hash, buf, key, BLAKE2S_OUTBYTES, i, BLAKE2S_KEYBYTES ); + + if( 0 != memcmp( hash, blake2s_keyed_kat[i], BLAKE2S_OUTBYTES ) ) + { + puts( "error" ); + return -1; + } + } + + puts( "ok" ); + return 0; +} +#endif + + diff --git a/Modules/_blake2/impl/blake2s-round.h b/Modules/_blake2/impl/blake2s-round.h new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2s-round.h @@ -0,0 +1,90 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ +#pragma once +#ifndef __BLAKE2S_ROUND_H__ +#define __BLAKE2S_ROUND_H__ + +#define LOADU(p) _mm_loadu_si128( (const __m128i *)(p) ) +#define STOREU(p,r) _mm_storeu_si128((__m128i *)(p), r) + +#define TOF(reg) _mm_castsi128_ps((reg)) +#define TOI(reg) _mm_castps_si128((reg)) + +#define LIKELY(x) __builtin_expect((x),1) + + +/* Microarchitecture-specific macros */ +#ifndef HAVE_XOP +#ifdef HAVE_SSSE3 +#define _mm_roti_epi32(r, c) ( \ + (8==-(c)) ? _mm_shuffle_epi8(r,r8) \ + : (16==-(c)) ? _mm_shuffle_epi8(r,r16) \ + : _mm_xor_si128(_mm_srli_epi32( (r), -(c) ),_mm_slli_epi32( (r), 32-(-(c)) )) ) +#else +#define _mm_roti_epi32(r, c) _mm_xor_si128(_mm_srli_epi32( (r), -(c) ),_mm_slli_epi32( (r), 32-(-(c)) )) +#endif +#else +/* ... */ +#endif + + +#define G1(row1,row2,row3,row4,buf) \ + row1 = _mm_add_epi32( _mm_add_epi32( row1, buf), row2 ); \ + row4 = _mm_xor_si128( row4, row1 ); \ + row4 = _mm_roti_epi32(row4, -16); \ + row3 = _mm_add_epi32( row3, row4 ); \ + row2 = _mm_xor_si128( row2, row3 ); \ + row2 = _mm_roti_epi32(row2, -12); + +#define G2(row1,row2,row3,row4,buf) \ + row1 = _mm_add_epi32( _mm_add_epi32( row1, buf), row2 ); \ + row4 = _mm_xor_si128( row4, row1 ); \ + row4 = _mm_roti_epi32(row4, -8); \ + row3 = _mm_add_epi32( row3, row4 ); \ + row2 = _mm_xor_si128( row2, row3 ); \ + row2 = _mm_roti_epi32(row2, -7); + +#define DIAGONALIZE(row1,row2,row3,row4) \ + row4 = _mm_shuffle_epi32( row4, _MM_SHUFFLE(2,1,0,3) ); \ + row3 = _mm_shuffle_epi32( row3, _MM_SHUFFLE(1,0,3,2) ); \ + row2 = _mm_shuffle_epi32( row2, _MM_SHUFFLE(0,3,2,1) ); + +#define UNDIAGONALIZE(row1,row2,row3,row4) \ + row4 = _mm_shuffle_epi32( row4, _MM_SHUFFLE(0,3,2,1) ); \ + row3 = _mm_shuffle_epi32( row3, _MM_SHUFFLE(1,0,3,2) ); \ + row2 = _mm_shuffle_epi32( row2, _MM_SHUFFLE(2,1,0,3) ); + +#if defined(HAVE_XOP) +#include "blake2s-load-xop.h" +#elif defined(HAVE_SSE41) +#include "blake2s-load-sse41.h" +#else +#include "blake2s-load-sse2.h" +#endif + +#define ROUND(r) \ + LOAD_MSG_ ##r ##_1(buf1); \ + G1(row1,row2,row3,row4,buf1); \ + LOAD_MSG_ ##r ##_2(buf2); \ + G2(row1,row2,row3,row4,buf2); \ + DIAGONALIZE(row1,row2,row3,row4); \ + LOAD_MSG_ ##r ##_3(buf3); \ + G1(row1,row2,row3,row4,buf3); \ + LOAD_MSG_ ##r ##_4(buf4); \ + G2(row1,row2,row3,row4,buf4); \ + UNDIAGONALIZE(row1,row2,row3,row4); \ + +#endif + diff --git a/Modules/_blake2/impl/blake2s.c b/Modules/_blake2/impl/blake2s.c new file mode 100644 --- /dev/null +++ b/Modules/_blake2/impl/blake2s.c @@ -0,0 +1,431 @@ +/* + BLAKE2 reference source code package - optimized C implementations + + Copyright 2012, Samuel Neves . You may use this under the + terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at + your option. The terms of these licenses can be found at: + + - CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0 + - OpenSSL license : https://www.openssl.org/source/license.html + - Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0 + + More information about the BLAKE2 hash function can be found at + https://blake2.net. +*/ + +#include +#include +#include + +#include "blake2.h" +#include "blake2-impl.h" + +#include "blake2-config.h" + + +#include +#if defined(HAVE_SSSE3) +#include +#endif +#if defined(HAVE_SSE41) +#include +#endif +#if defined(HAVE_AVX) +#include +#endif +#if defined(HAVE_XOP) +#include +#endif + +#include "blake2s-round.h" + +static const uint32_t blake2s_IV[8] = +{ + 0x6A09E667UL, 0xBB67AE85UL, 0x3C6EF372UL, 0xA54FF53AUL, + 0x510E527FUL, 0x9B05688CUL, 0x1F83D9ABUL, 0x5BE0CD19UL +}; + +static const uint8_t blake2s_sigma[10][16] = +{ + { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } , + { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 } , + { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 } , + { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 } , + { 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 } , + { 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 } , + { 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 } , + { 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 } , + { 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 } , + { 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13 , 0 } , +}; + + +/* Some helper functions, not necessarily useful */ +BLAKE2_LOCAL_INLINE(int) blake2s_set_lastnode( blake2s_state *S ) +{ + S->f[1] = -1; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_clear_lastnode( blake2s_state *S ) +{ + S->f[1] = 0; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_is_lastblock( const blake2s_state *S ) +{ + return S->f[0] != 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_set_lastblock( blake2s_state *S ) +{ + if( S->last_node ) blake2s_set_lastnode( S ); + + S->f[0] = -1; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_clear_lastblock( blake2s_state *S ) +{ + if( S->last_node ) blake2s_clear_lastnode( S ); + + S->f[0] = 0; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_increment_counter( blake2s_state *S, const uint32_t inc ) +{ + uint64_t t = ( ( uint64_t )S->t[1] << 32 ) | S->t[0]; + t += inc; + S->t[0] = ( uint32_t )( t >> 0 ); + S->t[1] = ( uint32_t )( t >> 32 ); + return 0; +} + + +/* Parameter-related functions */ +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_digest_length( blake2s_param *P, const uint8_t digest_length ) +{ + P->digest_length = digest_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_fanout( blake2s_param *P, const uint8_t fanout ) +{ + P->fanout = fanout; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_max_depth( blake2s_param *P, const uint8_t depth ) +{ + P->depth = depth; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_leaf_length( blake2s_param *P, const uint32_t leaf_length ) +{ + P->leaf_length = leaf_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_node_offset( blake2s_param *P, const uint64_t node_offset ) +{ + store48( P->node_offset, node_offset ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_node_depth( blake2s_param *P, const uint8_t node_depth ) +{ + P->node_depth = node_depth; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_inner_length( blake2s_param *P, const uint8_t inner_length ) +{ + P->inner_length = inner_length; + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_salt( blake2s_param *P, const uint8_t salt[BLAKE2S_SALTBYTES] ) +{ + memcpy( P->salt, salt, BLAKE2S_SALTBYTES ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_param_set_personal( blake2s_param *P, const uint8_t personal[BLAKE2S_PERSONALBYTES] ) +{ + memcpy( P->personal, personal, BLAKE2S_PERSONALBYTES ); + return 0; +} + +BLAKE2_LOCAL_INLINE(int) blake2s_init0( blake2s_state *S ) +{ + memset( S, 0, sizeof( blake2s_state ) ); + + for( int i = 0; i < 8; ++i ) S->h[i] = blake2s_IV[i]; + + return 0; +} + +/* init2 xors IV with input parameter block */ +int blake2s_init_param( blake2s_state *S, const blake2s_param *P ) +{ + /*blake2s_init0( S ); */ + const uint8_t * v = ( const uint8_t * )( blake2s_IV ); + const uint8_t * p = ( const uint8_t * )( P ); + uint8_t * h = ( uint8_t * )( S->h ); + /* IV XOR ParamBlock */ + memset( S, 0, sizeof( blake2s_state ) ); + + for( int i = 0; i < BLAKE2S_OUTBYTES; ++i ) h[i] = v[i] ^ p[i]; + + return 0; +} + + +/* Some sort of default parameter block initialization, for sequential blake2s */ +int blake2s_init( blake2s_state *S, const uint8_t outlen ) +{ + const blake2s_param P = + { + outlen, + 0, + 1, + 1, + 0, + {0}, + 0, + 0, + {0}, + {0} + }; + /* Move interval verification here? */ + if ( ( !outlen ) || ( outlen > BLAKE2S_OUTBYTES ) ) return -1; + return blake2s_init_param( S, &P ); +} + + +int blake2s_init_key( blake2s_state *S, const uint8_t outlen, const void *key, const uint8_t keylen ) +{ + const blake2s_param P = + { + outlen, + keylen, + 1, + 1, + 0, + {0}, + 0, + 0, + {0}, + {0} + }; + + /* Move interval verification here? */ + if ( ( !outlen ) || ( outlen > BLAKE2S_OUTBYTES ) ) return -1; + + if ( ( !key ) || ( !keylen ) || keylen > BLAKE2S_KEYBYTES ) return -1; + + if( blake2s_init_param( S, &P ) < 0 ) + return -1; + + { + uint8_t block[BLAKE2S_BLOCKBYTES]; + memset( block, 0, BLAKE2S_BLOCKBYTES ); + memcpy( block, key, keylen ); + blake2s_update( S, block, BLAKE2S_BLOCKBYTES ); + secure_zero_memory( block, BLAKE2S_BLOCKBYTES ); /* Burn the key from stack */ + } + return 0; +} + + +BLAKE2_LOCAL_INLINE(int) blake2s_compress( blake2s_state *S, const uint8_t block[BLAKE2S_BLOCKBYTES] ) +{ + __m128i row1, row2, row3, row4; + __m128i buf1, buf2, buf3, buf4; +#if defined(HAVE_SSE41) + __m128i t0, t1; +#if !defined(HAVE_XOP) + __m128i t2; +#endif +#endif + __m128i ff0, ff1; +#if defined(HAVE_SSSE3) && !defined(HAVE_XOP) + const __m128i r8 = _mm_set_epi8( 12, 15, 14, 13, 8, 11, 10, 9, 4, 7, 6, 5, 0, 3, 2, 1 ); + const __m128i r16 = _mm_set_epi8( 13, 12, 15, 14, 9, 8, 11, 10, 5, 4, 7, 6, 1, 0, 3, 2 ); +#endif +#if defined(HAVE_SSE41) + const __m128i m0 = LOADU( block + 00 ); + const __m128i m1 = LOADU( block + 16 ); + const __m128i m2 = LOADU( block + 32 ); + const __m128i m3 = LOADU( block + 48 ); +#else + const uint32_t m0 = ( ( uint32_t * )block )[ 0]; + const uint32_t m1 = ( ( uint32_t * )block )[ 1]; + const uint32_t m2 = ( ( uint32_t * )block )[ 2]; + const uint32_t m3 = ( ( uint32_t * )block )[ 3]; + const uint32_t m4 = ( ( uint32_t * )block )[ 4]; + const uint32_t m5 = ( ( uint32_t * )block )[ 5]; + const uint32_t m6 = ( ( uint32_t * )block )[ 6]; + const uint32_t m7 = ( ( uint32_t * )block )[ 7]; + const uint32_t m8 = ( ( uint32_t * )block )[ 8]; + const uint32_t m9 = ( ( uint32_t * )block )[ 9]; + const uint32_t m10 = ( ( uint32_t * )block )[10]; + const uint32_t m11 = ( ( uint32_t * )block )[11]; + const uint32_t m12 = ( ( uint32_t * )block )[12]; + const uint32_t m13 = ( ( uint32_t * )block )[13]; + const uint32_t m14 = ( ( uint32_t * )block )[14]; + const uint32_t m15 = ( ( uint32_t * )block )[15]; +#endif + row1 = ff0 = LOADU( &S->h[0] ); + row2 = ff1 = LOADU( &S->h[4] ); + row3 = _mm_setr_epi32( 0x6A09E667, 0xBB67AE85, 0x3C6EF372, 0xA54FF53A ); + row4 = _mm_xor_si128( _mm_setr_epi32( 0x510E527F, 0x9B05688C, 0x1F83D9AB, 0x5BE0CD19 ), LOADU( &S->t[0] ) ); + ROUND( 0 ); + ROUND( 1 ); + ROUND( 2 ); + ROUND( 3 ); + ROUND( 4 ); + ROUND( 5 ); + ROUND( 6 ); + ROUND( 7 ); + ROUND( 8 ); + ROUND( 9 ); + STOREU( &S->h[0], _mm_xor_si128( ff0, _mm_xor_si128( row1, row3 ) ) ); + STOREU( &S->h[4], _mm_xor_si128( ff1, _mm_xor_si128( row2, row4 ) ) ); + return 0; +} + +/* inlen now in bytes */ +int blake2s_update( blake2s_state *S, const uint8_t *in, uint64_t inlen ) +{ + while( inlen > 0 ) + { + size_t left = S->buflen; + size_t fill = 2 * BLAKE2S_BLOCKBYTES - left; + + if( inlen > fill ) + { + memcpy( S->buf + left, in, fill ); /* Fill buffer */ + S->buflen += fill; + blake2s_increment_counter( S, BLAKE2S_BLOCKBYTES ); + blake2s_compress( S, S->buf ); /* Compress */ + memcpy( S->buf, S->buf + BLAKE2S_BLOCKBYTES, BLAKE2S_BLOCKBYTES ); /* Shift buffer left */ + S->buflen -= BLAKE2S_BLOCKBYTES; + in += fill; + inlen -= fill; + } + else /* inlen <= fill */ + { + memcpy( S->buf + left, in, inlen ); + S->buflen += inlen; /* Be lazy, do not compress */ + in += inlen; + inlen -= inlen; + } + } + + return 0; +} + +/* Is this correct? */ +int blake2s_final( blake2s_state *S, uint8_t *out, uint8_t outlen ) +{ + uint8_t buffer[BLAKE2S_OUTBYTES] = {0}; + + if( outlen > BLAKE2S_OUTBYTES ) + return -1; + + if( blake2s_is_lastblock( S ) ) + return -1; + + if( S->buflen > BLAKE2S_BLOCKBYTES ) + { + blake2s_increment_counter( S, BLAKE2S_BLOCKBYTES ); + blake2s_compress( S, S->buf ); + S->buflen -= BLAKE2S_BLOCKBYTES; + memcpy( S->buf, S->buf + BLAKE2S_BLOCKBYTES, S->buflen ); + } + + blake2s_increment_counter( S, ( uint32_t )S->buflen ); + blake2s_set_lastblock( S ); + memset( S->buf + S->buflen, 0, 2 * BLAKE2S_BLOCKBYTES - S->buflen ); /* Padding */ + blake2s_compress( S, S->buf ); + + for( int i = 0; i < 8; ++i ) /* Output full hash to temp buffer */ + store32( buffer + sizeof( S->h[i] ) * i, S->h[i] ); + + memcpy( out, buffer, outlen ); + return 0; +} + +/* inlen, at least, should be uint64_t. Others can be size_t. */ +int blake2s( uint8_t *out, const void *in, const void *key, const uint8_t outlen, const uint64_t inlen, uint8_t keylen ) +{ + blake2s_state S[1]; + + /* Verify parameters */ + if ( NULL == in && inlen > 0 ) return -1; + + if ( NULL == out ) return -1; + + if ( NULL == key && keylen > 0) return -1; + + if( !outlen || outlen > BLAKE2S_OUTBYTES ) return -1; + + if( keylen > BLAKE2S_KEYBYTES ) return -1; + + if( keylen > 0 ) + { + if( blake2s_init_key( S, outlen, key, keylen ) < 0 ) return -1; + } + else + { + if( blake2s_init( S, outlen ) < 0 ) return -1; + } + + blake2s_update( S, ( const uint8_t * )in, inlen ); + blake2s_final( S, out, outlen ); + return 0; +} + +#if defined(SUPERCOP) +int crypto_hash( unsigned char *out, unsigned char *in, unsigned long long inlen ) +{ + return blake2s( out, in, NULL, BLAKE2S_OUTBYTES, inlen, 0 ); +} +#endif + +#if defined(BLAKE2S_SELFTEST) +#include +#include "blake2-kat.h" +int main( int argc, char **argv ) +{ + uint8_t key[BLAKE2S_KEYBYTES]; + uint8_t buf[KAT_LENGTH]; + + for( size_t i = 0; i < BLAKE2S_KEYBYTES; ++i ) + key[i] = ( uint8_t )i; + + for( size_t i = 0; i < KAT_LENGTH; ++i ) + buf[i] = ( uint8_t )i; + + for( size_t i = 0; i < KAT_LENGTH; ++i ) + { + uint8_t hash[BLAKE2S_OUTBYTES]; + + if( blake2s( hash, buf, key, BLAKE2S_OUTBYTES, i, BLAKE2S_KEYBYTES ) < 0 || + 0 != memcmp( hash, blake2s_keyed_kat[i], BLAKE2S_OUTBYTES ) ) + { + puts( "error" ); + return -1; + } + } + + puts( "ok" ); + return 0; +} +#endif + + diff --git a/Modules/hashlib.h b/Modules/hashlib.h --- a/Modules/hashlib.h +++ b/Modules/hashlib.h @@ -2,30 +2,33 @@ /* * Given a PyObject* obj, fill in the Py_buffer* viewp with the result - * of PyObject_GetBuffer. Sets an exception and issues a return NULL - * on any errors. + * of PyObject_GetBuffer. Sets an exception and issues the erraction + * on any errors, e.g. 'return NULL' or 'goto error'. */ -#define GET_BUFFER_VIEW_OR_ERROUT(obj, viewp) do { \ +#define GET_BUFFER_VIEW_OR_ERROR(obj, viewp, erraction) do { \ if (PyUnicode_Check((obj))) { \ PyErr_SetString(PyExc_TypeError, \ "Unicode-objects must be encoded before hashing");\ - return NULL; \ + erraction; \ } \ if (!PyObject_CheckBuffer((obj))) { \ PyErr_SetString(PyExc_TypeError, \ "object supporting the buffer API required"); \ - return NULL; \ + erraction; \ } \ if (PyObject_GetBuffer((obj), (viewp), PyBUF_SIMPLE) == -1) { \ - return NULL; \ + erraction; \ } \ if ((viewp)->ndim > 1) { \ PyErr_SetString(PyExc_BufferError, \ "Buffer must be single dimension"); \ PyBuffer_Release((viewp)); \ - return NULL; \ + erraction; \ } \ - } while(0); + } while(0) + +#define GET_BUFFER_VIEW_OR_ERROUT(obj, viewp) \ + GET_BUFFER_VIEW_OR_ERROR(obj, viewp, return NULL) /* * Helper code to synchronize access to the hash object when the GIL is diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -214,6 +214,9 @@ + + + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -449,6 +449,15 @@ Modules + + Modules + + + Modules + + + Modules + Modules diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -889,6 +889,22 @@ exts.append( Extension('_sha1', ['sha1module.c'], depends=['hashlib.h']) ) + blake2_deps = [os.path.join('_blake2', 'impl', name) + for name in os.listdir('Modules/_blake2/impl')] + blake2_deps.append('hashlib.h') + + blake2_macros = [] + if os.uname().machine == "x86_64": + # Every x86_64 machine has at least SSE2. + blake2_macros.append(('BLAKE2_USE_SSE', '1')) + + exts.append( Extension('_blake2', + ['_blake2/blake2module.c', + '_blake2/blake2b_impl.c', + '_blake2/blake2s_impl.c'], + define_macros=blake2_macros, + depends=blake2_deps) ) + # Modules that provide persistent dictionary-like semantics. You will # probably want to arrange for at least one of them to be available on # your machine, though none are defined by default because of library -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 16:08:46 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 20:08:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_only_include_inttypes=2Eh_?= =?utf-8?b?KCMxNzg4NCk=?= Message-ID: <20160906200845.114804.56096.0C260BDE@psf.io> https://hg.python.org/cpython/rev/c092eb31db05 changeset: 103137:c092eb31db05 user: Benjamin Peterson date: Tue Sep 06 13:05:58 2016 -0700 summary: only include inttypes.h (#17884) files: Include/pyport.h | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -3,10 +3,7 @@ #include "pyconfig.h" /* include for defines */ -/* Some versions of HP-UX & Solaris need inttypes.h for int32_t, - INT32_MAX, etc. */ #include -#include /************************************************************************** Symbols and macros to supply platform-independent interfaces to basic -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 16:24:31 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 20:24:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_replace_Python_aliases_for?= =?utf-8?q?_standard_integer_types_with_the_standard_integer?= Message-ID: <20160906202429.8503.74286.EB59244F@psf.io> https://hg.python.org/cpython/rev/be8645213517 changeset: 103138:be8645213517 user: Benjamin Peterson date: Tue Sep 06 13:24:00 2016 -0700 summary: replace Python aliases for standard integer types with the standard integer types (#17884) files: Include/longintrepr.h | 8 ++-- Include/pyhash.h | 16 +++----- Include/pytime.h | 6 +-- Modules/_randommodule.c | 44 ++++++++++------------ Modules/_testcapimodule.c | 16 ++++---- Modules/audioop.c | 8 ++-- Objects/unicodeobject.c | 8 ++-- Python/pyhash.c | 53 ++++++++++++-------------- 8 files changed, 72 insertions(+), 87 deletions(-) diff --git a/Include/longintrepr.h b/Include/longintrepr.h --- a/Include/longintrepr.h +++ b/Include/longintrepr.h @@ -42,10 +42,10 @@ */ #if PYLONG_BITS_IN_DIGIT == 30 -typedef PY_UINT32_T digit; -typedef PY_INT32_T sdigit; /* signed variant of digit */ -typedef PY_UINT64_T twodigits; -typedef PY_INT64_T stwodigits; /* signed variant of twodigits */ +typedef uint32_t digit; +typedef int32_t sdigit; /* signed variant of digit */ +typedef uint64_t twodigits; +typedef int64_t stwodigits; /* signed variant of twodigits */ #define PyLong_SHIFT 30 #define _PyLong_DECIMAL_SHIFT 9 /* max(e such that 10**e fits in a digit) */ #define _PyLong_DECIMAL_BASE ((digit)1000000000) /* 10 ** DECIMAL_SHIFT */ diff --git a/Include/pyhash.h b/Include/pyhash.h --- a/Include/pyhash.h +++ b/Include/pyhash.h @@ -36,14 +36,14 @@ * memory layout on 64 bit systems * cccccccc cccccccc cccccccc uc -- unsigned char[24] * pppppppp ssssssss ........ fnv -- two Py_hash_t - * k0k0k0k0 k1k1k1k1 ........ siphash -- two PY_UINT64_T + * k0k0k0k0 k1k1k1k1 ........ siphash -- two uint64_t * ........ ........ ssssssss djbx33a -- 16 bytes padding + one Py_hash_t * ........ ........ eeeeeeee pyexpat XML hash salt * * memory layout on 32 bit systems * cccccccc cccccccc cccccccc uc * ppppssss ........ ........ fnv -- two Py_hash_t - * k0k0k0k0 k1k1k1k1 ........ siphash -- two PY_UINT64_T (*) + * k0k0k0k0 k1k1k1k1 ........ siphash -- two uint64_t (*) * ........ ........ ssss.... djbx33a -- 16 bytes padding + one Py_hash_t * ........ ........ eeee.... pyexpat XML hash salt * @@ -59,13 +59,11 @@ Py_hash_t prefix; Py_hash_t suffix; } fnv; -#ifdef PY_UINT64_T /* two uint64 for SipHash24 */ struct { - PY_UINT64_T k0; - PY_UINT64_T k1; + uint64_t k0; + uint64_t k1; } siphash; -#endif /* a different (!) Py_hash_t for small string optimization */ struct { unsigned char padding[16]; @@ -121,8 +119,7 @@ * configure script. * * - FNV is available on all platforms and architectures. - * - SIPHASH24 only works on plaforms that provide PY_UINT64_T and doesn't - * require aligned memory for integers. + * - SIPHASH24 only works on plaforms that don't require aligned memory for integers. * - With EXTERNAL embedders can provide an alternative implementation with:: * * PyHash_FuncDef PyHash_Func = {...}; @@ -134,8 +131,7 @@ #define Py_HASH_FNV 2 #ifndef Py_HASH_ALGORITHM -# if (defined(PY_UINT64_T) && defined(PY_UINT32_T) \ - && !defined(HAVE_ALIGNED_REQUIRED)) +# ifndef HAVE_ALIGNED_REQUIRED # define Py_HASH_ALGORITHM Py_HASH_SIPHASH24 # else # define Py_HASH_ALGORITHM Py_HASH_FNV diff --git a/Include/pytime.h b/Include/pytime.h --- a/Include/pytime.h +++ b/Include/pytime.h @@ -13,16 +13,12 @@ extern "C" { #endif -#ifdef PY_INT64_T /* _PyTime_t: Python timestamp with subsecond precision. It can be used to store a duration, and so indirectly a date (related to another date, like UNIX epoch). */ -typedef PY_INT64_T _PyTime_t; +typedef int64_t _PyTime_t; #define _PyTime_MIN PY_LLONG_MIN #define _PyTime_MAX PY_LLONG_MAX -#else -# error "_PyTime_t need signed 64-bit integer type" -#endif typedef enum { /* Round towards minus infinity (-inf). diff --git a/Modules/_randommodule.c b/Modules/_randommodule.c --- a/Modules/_randommodule.c +++ b/Modules/_randommodule.c @@ -69,10 +69,6 @@ #include "Python.h" #include /* for seeding to current time */ -#ifndef PY_UINT32_T -# error "Failed to find an exact-width 32-bit integer type" -#endif - /* Period parameters -- These are all magic. Don't change. */ #define N 624 #define M 397 @@ -83,7 +79,7 @@ typedef struct { PyObject_HEAD int index; - PY_UINT32_T state[N]; + uint32_t state[N]; } RandomObject; static PyTypeObject Random_Type; @@ -95,13 +91,13 @@ /* generates a random number on [0,0xffffffff]-interval */ -static PY_UINT32_T +static uint32_t genrand_int32(RandomObject *self) { - PY_UINT32_T y; - static const PY_UINT32_T mag01[2] = {0x0U, MATRIX_A}; + uint32_t y; + static const uint32_t mag01[2] = {0x0U, MATRIX_A}; /* mag01[x] = x * MATRIX_A for x=0,1 */ - PY_UINT32_T *mt; + uint32_t *mt; mt = self->state; if (self->index >= N) { /* generate N words at one time */ @@ -141,16 +137,16 @@ static PyObject * random_random(RandomObject *self) { - PY_UINT32_T a=genrand_int32(self)>>5, b=genrand_int32(self)>>6; + uint32_t a=genrand_int32(self)>>5, b=genrand_int32(self)>>6; return PyFloat_FromDouble((a*67108864.0+b)*(1.0/9007199254740992.0)); } /* initializes mt[N] with a seed */ static void -init_genrand(RandomObject *self, PY_UINT32_T s) +init_genrand(RandomObject *self, uint32_t s) { int mti; - PY_UINT32_T *mt; + uint32_t *mt; mt = self->state; mt[0]= s; @@ -170,10 +166,10 @@ /* init_key is the array for initializing keys */ /* key_length is its length */ static PyObject * -init_by_array(RandomObject *self, PY_UINT32_T init_key[], size_t key_length) +init_by_array(RandomObject *self, uint32_t init_key[], size_t key_length) { size_t i, j, k; /* was signed in the original code. RDH 12/16/2002 */ - PY_UINT32_T *mt; + uint32_t *mt; mt = self->state; init_genrand(self, 19650218U); @@ -181,14 +177,14 @@ k = (N>key_length ? N : key_length); for (; k; k--) { mt[i] = (mt[i] ^ ((mt[i-1] ^ (mt[i-1] >> 30)) * 1664525U)) - + init_key[j] + (PY_UINT32_T)j; /* non linear */ + + init_key[j] + (uint32_t)j; /* non linear */ i++; j++; if (i>=N) { mt[0] = mt[N-1]; i=1; } if (j>=key_length) j=0; } for (k=N-1; k; k--) { mt[i] = (mt[i] ^ ((mt[i-1] ^ (mt[i-1] >> 30)) * 1566083941U)) - - (PY_UINT32_T)i; /* non linear */ + - (uint32_t)i; /* non linear */ i++; if (i>=N) { mt[0] = mt[N-1]; i=1; } } @@ -208,7 +204,7 @@ { PyObject *result = NULL; /* guilty until proved innocent */ PyObject *n = NULL; - PY_UINT32_T *key = NULL; + uint32_t *key = NULL; size_t bits, keyused; int res; PyObject *arg = NULL; @@ -220,7 +216,7 @@ time_t now; time(&now); - init_genrand(self, (PY_UINT32_T)now); + init_genrand(self, (uint32_t)now); Py_INCREF(Py_None); return Py_None; } @@ -248,7 +244,7 @@ keyused = bits == 0 ? 1 : (bits - 1) / 32 + 1; /* Convert seed to byte sequence. */ - key = (PY_UINT32_T *)PyMem_Malloc((size_t)4 * keyused); + key = (uint32_t *)PyMem_Malloc((size_t)4 * keyused); if (key == NULL) { PyErr_NoMemory(); goto Done; @@ -267,7 +263,7 @@ size_t i, j; /* Reverse an array. */ for (i = 0, j = keyused - 1; i < j; i++, j--) { - PY_UINT32_T tmp = key[i]; + uint32_t tmp = key[i]; key[i] = key[j]; key[j] = tmp; } @@ -329,7 +325,7 @@ element = PyLong_AsUnsignedLong(PyTuple_GET_ITEM(state, i)); if (element == (unsigned long)-1 && PyErr_Occurred()) return NULL; - self->state[i] = (PY_UINT32_T)element; + self->state[i] = (uint32_t)element; } index = PyLong_AsLong(PyTuple_GET_ITEM(state, i)); @@ -349,8 +345,8 @@ random_getrandbits(RandomObject *self, PyObject *args) { int k, i, words; - PY_UINT32_T r; - PY_UINT32_T *wordarray; + uint32_t r; + uint32_t *wordarray; PyObject *result; if (!PyArg_ParseTuple(args, "i:getrandbits", &k)) @@ -366,7 +362,7 @@ return PyLong_FromUnsignedLong(genrand_int32(self) >> (32 - k)); words = (k - 1) / 32 + 1; - wordarray = (PY_UINT32_T *)PyMem_Malloc(words * 4); + wordarray = (uint32_t *)PyMem_Malloc(words * 4); if (wordarray == NULL) { PyErr_NoMemory(); return NULL; diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -98,14 +98,14 @@ CHECK_SIGNNESS(Py_UCS1, 0); CHECK_SIGNNESS(Py_UCS2, 0); CHECK_SIGNNESS(Py_UCS4, 0); - CHECK_SIZEOF(PY_INT32_T, 4); - CHECK_SIGNNESS(PY_INT32_T, 1); - CHECK_SIZEOF(PY_UINT32_T, 4); - CHECK_SIGNNESS(PY_UINT32_T, 0); - CHECK_SIZEOF(PY_INT64_T, 8); - CHECK_SIGNNESS(PY_INT64_T, 1); - CHECK_SIZEOF(PY_UINT64_T, 8); - CHECK_SIGNNESS(PY_UINT64_T, 0); + CHECK_SIZEOF(int32_t, 4); + CHECK_SIGNNESS(int32_t, 1); + CHECK_SIZEOF(uint32_t, 4); + CHECK_SIGNNESS(uint32_t, 0); + CHECK_SIZEOF(int64_t, 8); + CHECK_SIGNNESS(int64_t, 1); + CHECK_SIZEOF(uint64_t, 8); + CHECK_SIGNNESS(uint64_t, 0); /* pointer/size types */ CHECK_SIZEOF(size_t, sizeof(void *)); diff --git a/Modules/audioop.c b/Modules/audioop.c --- a/Modules/audioop.c +++ b/Modules/audioop.c @@ -297,7 +297,7 @@ #define GETINT8(cp, i) GETINTX(signed char, (cp), (i)) #define GETINT16(cp, i) GETINTX(short, (cp), (i)) -#define GETINT32(cp, i) GETINTX(PY_INT32_T, (cp), (i)) +#define GETINT32(cp, i) GETINTX(int32_t, (cp), (i)) #if WORDS_BIGENDIAN #define GETINT24(cp, i) ( \ @@ -314,7 +314,7 @@ #define SETINT8(cp, i, val) SETINTX(signed char, (cp), (i), (val)) #define SETINT16(cp, i, val) SETINTX(short, (cp), (i), (val)) -#define SETINT32(cp, i, val) SETINTX(PY_INT32_T, (cp), (i), (val)) +#define SETINT32(cp, i, val) SETINTX(int32_t, (cp), (i), (val)) #if WORDS_BIGENDIAN #define SETINT24(cp, i, val) do { \ @@ -1129,7 +1129,7 @@ val = ((unsigned int)GETINT24(fragment->buf, i)) & 0xffffffu; else { assert(width == 4); - val = GETINTX(PY_UINT32_T, fragment->buf, i); + val = GETINTX(uint32_t, fragment->buf, i); } val += (unsigned int)bias; @@ -1144,7 +1144,7 @@ SETINT24(ncp, i, (int)val); else { assert(width == 4); - SETINTX(PY_UINT32_T, ncp, i, val); + SETINTX(uint32_t, ncp, i, val); } } return rv; diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -5330,7 +5330,7 @@ const void *data; Py_ssize_t len; PyObject *v; - PY_UINT32_T *out; + uint32_t *out; #if PY_LITTLE_ENDIAN int native_ordering = byteorder <= 0; #else @@ -5361,7 +5361,7 @@ /* output buffer is 4-bytes aligned */ assert(_Py_IS_ALIGNED(PyBytes_AS_STRING(v), 4)); - out = (PY_UINT32_T *)PyBytes_AS_STRING(v); + out = (uint32_t *)PyBytes_AS_STRING(v); if (byteorder == 0) *out++ = 0xFEFF; if (len == 0) @@ -5427,7 +5427,7 @@ /* four bytes are reserved for each surrogate */ if (moreunits > 1) { - Py_ssize_t outpos = out - (PY_UINT32_T*) PyBytes_AS_STRING(v); + Py_ssize_t outpos = out - (uint32_t*) PyBytes_AS_STRING(v); Py_ssize_t morebytes = 4 * (moreunits - 1); if (PyBytes_GET_SIZE(v) > PY_SSIZE_T_MAX - morebytes) { /* integer overflow */ @@ -5436,7 +5436,7 @@ } if (_PyBytes_Resize(&v, PyBytes_GET_SIZE(v) + morebytes) < 0) goto error; - out = (PY_UINT32_T*) PyBytes_AS_STRING(v) + outpos; + out = (uint32_t*) PyBytes_AS_STRING(v) + outpos; } if (PyBytes_Check(rep)) { diff --git a/Python/pyhash.c b/Python/pyhash.c --- a/Python/pyhash.c +++ b/Python/pyhash.c @@ -318,40 +318,37 @@ Modified for Python by Christian Heimes: - C89 / MSVC compatibility - - PY_UINT64_T, PY_UINT32_T and PY_UINT8_T - _rotl64() on Windows - letoh64() fallback */ -typedef unsigned char PY_UINT8_T; - /* byte swap little endian to host endian * Endian conversion not only ensures that the hash function returns the same * value on all platforms. It is also required to for a good dispersion of * the hash values' least significant bits. */ #if PY_LITTLE_ENDIAN -# define _le64toh(x) ((PY_UINT64_T)(x)) +# define _le64toh(x) ((uint64_t)(x)) #elif defined(__APPLE__) # define _le64toh(x) OSSwapLittleToHostInt64(x) #elif defined(HAVE_LETOH64) # define _le64toh(x) le64toh(x) #else -# define _le64toh(x) (((PY_UINT64_T)(x) << 56) | \ - (((PY_UINT64_T)(x) << 40) & 0xff000000000000ULL) | \ - (((PY_UINT64_T)(x) << 24) & 0xff0000000000ULL) | \ - (((PY_UINT64_T)(x) << 8) & 0xff00000000ULL) | \ - (((PY_UINT64_T)(x) >> 8) & 0xff000000ULL) | \ - (((PY_UINT64_T)(x) >> 24) & 0xff0000ULL) | \ - (((PY_UINT64_T)(x) >> 40) & 0xff00ULL) | \ - ((PY_UINT64_T)(x) >> 56)) +# define _le64toh(x) (((uint64_t)(x) << 56) | \ + (((uint64_t)(x) << 40) & 0xff000000000000ULL) | \ + (((uint64_t)(x) << 24) & 0xff0000000000ULL) | \ + (((uint64_t)(x) << 8) & 0xff00000000ULL) | \ + (((uint64_t)(x) >> 8) & 0xff000000ULL) | \ + (((uint64_t)(x) >> 24) & 0xff0000ULL) | \ + (((uint64_t)(x) >> 40) & 0xff00ULL) | \ + ((uint64_t)(x) >> 56)) #endif #ifdef _MSC_VER # define ROTATE(x, b) _rotl64(x, b) #else -# define ROTATE(x, b) (PY_UINT64_T)( ((x) << (b)) | ( (x) >> (64 - (b))) ) +# define ROTATE(x, b) (uint64_t)( ((x) << (b)) | ( (x) >> (64 - (b))) ) #endif #define HALF_ROUND(a,b,c,d,s,t) \ @@ -369,22 +366,22 @@ static Py_hash_t siphash24(const void *src, Py_ssize_t src_sz) { - PY_UINT64_T k0 = _le64toh(_Py_HashSecret.siphash.k0); - PY_UINT64_T k1 = _le64toh(_Py_HashSecret.siphash.k1); - PY_UINT64_T b = (PY_UINT64_T)src_sz << 56; - const PY_UINT64_T *in = (PY_UINT64_T*)src; + uint64_t k0 = _le64toh(_Py_HashSecret.siphash.k0); + uint64_t k1 = _le64toh(_Py_HashSecret.siphash.k1); + uint64_t b = (uint64_t)src_sz << 56; + const uint64_t *in = (uint64_t*)src; - PY_UINT64_T v0 = k0 ^ 0x736f6d6570736575ULL; - PY_UINT64_T v1 = k1 ^ 0x646f72616e646f6dULL; - PY_UINT64_T v2 = k0 ^ 0x6c7967656e657261ULL; - PY_UINT64_T v3 = k1 ^ 0x7465646279746573ULL; + uint64_t v0 = k0 ^ 0x736f6d6570736575ULL; + uint64_t v1 = k1 ^ 0x646f72616e646f6dULL; + uint64_t v2 = k0 ^ 0x6c7967656e657261ULL; + uint64_t v3 = k1 ^ 0x7465646279746573ULL; - PY_UINT64_T t; - PY_UINT8_T *pt; - PY_UINT8_T *m; + uint64_t t; + uint8_t *pt; + uint8_t *m; while (src_sz >= 8) { - PY_UINT64_T mi = _le64toh(*in); + uint64_t mi = _le64toh(*in); in += 1; src_sz -= 8; v3 ^= mi; @@ -393,13 +390,13 @@ } t = 0; - pt = (PY_UINT8_T *)&t; - m = (PY_UINT8_T *)in; + pt = (uint8_t *)&t; + m = (uint8_t *)in; switch (src_sz) { case 7: pt[6] = m[6]; case 6: pt[5] = m[5]; case 5: pt[4] = m[4]; - case 4: Py_MEMCPY(pt, m, sizeof(PY_UINT32_T)); break; + case 4: Py_MEMCPY(pt, m, sizeof(uint32_t)); break; case 3: pt[2] = m[2]; case 2: pt[1] = m[1]; case 1: pt[0] = m[0]; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 16:28:55 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 20:28:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_properly_introduce_reST_li?= =?utf-8?q?teral_blocks?= Message-ID: <20160906202855.6465.22743.C7EF8F99@psf.io> https://hg.python.org/cpython/rev/323dacd4f253 changeset: 103139:323dacd4f253 user: Benjamin Peterson date: Tue Sep 06 13:28:29 2016 -0700 summary: properly introduce reST literal blocks files: Doc/library/hashlib-blake2.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/hashlib-blake2.rst b/Doc/library/hashlib-blake2.rst --- a/Doc/library/hashlib-blake2.rst +++ b/Doc/library/hashlib-blake2.rst @@ -217,7 +217,7 @@ indifferentiability property inherited from BLAKE. This example shows how to get a (hex-encoded) 128-bit authentication code for -message ``b'message data'`` with key ``b'pseudorandom key'``: +message ``b'message data'`` with key ``b'pseudorandom key'``:: >>> from hashlib import blake2b >>> h = blake2b(key=b'pseudorandom key', digest_size=16) @@ -227,7 +227,7 @@ As a practical example, a web application can symmetrically sign cookies sent -to users and later verify them to make sure they weren't tampered with: +to users and later verify them to make sure they weren't tampered with:: >>> from hashlib import blake2b >>> from hmac import compare_digest @@ -251,7 +251,7 @@ False Even though there's a native keyed hashing mode, BLAKE2 can, of course, be used -in HMAC construction with :mod:`hmac` module: +in HMAC construction with :mod:`hmac` module:: >>> import hmac, hashlib >>> m = hmac.new(b'secret key', digestmod=hashlib.blake2s) @@ -334,7 +334,7 @@ `_, p. 21) -BLAKE2 can be personalized by passing bytes to the *person* argument: +BLAKE2 can be personalized by passing bytes to the *person* argument:: >>> from hashlib import blake2b >>> FILES_HASH_PERSON = b'MyApp Files Hash' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 16:34:20 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 20:34:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_require_uintptr=5Ft_to_exi?= =?utf-8?q?st?= Message-ID: <20160906203420.114928.81160.6BE86B9A@psf.io> https://hg.python.org/cpython/rev/83529ca38833 changeset: 103140:83529ca38833 user: Benjamin Peterson date: Tue Sep 06 13:33:56 2016 -0700 summary: require uintptr_t to exist files: Include/pyport.h | 21 ---------------- configure | 45 ------------------------------------ configure.ac | 5 ---- pyconfig.h.in | 6 ---- 4 files changed, 0 insertions(+), 77 deletions(-) diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -23,10 +23,6 @@ Meaning: Extra checks compiled in for debug mode. Used in: Py_SAFE_DOWNCAST -HAVE_UINTPTR_T -Meaning: The C9X type uintptr_t is supported by the compiler -Used in: Py_uintptr_t - **************************************************************************/ /* typedefs for some C9X-defined synonyms for integral types. @@ -90,26 +86,9 @@ * without loss of information. Similarly for intptr_t, wrt a signed * integral type. */ -#ifdef HAVE_UINTPTR_T typedef uintptr_t Py_uintptr_t; typedef intptr_t Py_intptr_t; -#elif SIZEOF_VOID_P <= SIZEOF_INT -typedef unsigned int Py_uintptr_t; -typedef int Py_intptr_t; - -#elif SIZEOF_VOID_P <= SIZEOF_LONG -typedef unsigned long Py_uintptr_t; -typedef long Py_intptr_t; - -#elif SIZEOF_VOID_P <= SIZEOF_LONG_LONG -typedef unsigned long long Py_uintptr_t; -typedef long long Py_intptr_t; - -#else -# error "Python needs a typedef for Py_uintptr_t in pyport.h." -#endif /* HAVE_UINTPTR_T */ - /* Py_ssize_t is a signed integral type such that sizeof(Py_ssize_t) == * sizeof(size_t). C99 doesn't define such a thing directly (size_t is an * unsigned integral type). See PEP 353 for details. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -8471,51 +8471,6 @@ fi -ac_fn_c_check_type "$LINENO" "uintptr_t" "ac_cv_type_uintptr_t" "#include - #include -" -if test "x$ac_cv_type_uintptr_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_UINTPTR_T 1 -_ACEOF - -# The cast to long int works around a bug in the HP C Compiler -# version HP92453-01 B.11.11.23709.GP, which incorrectly rejects -# declarations like `int a3[[(sizeof (unsigned char)) >= 0]];'. -# This bug is HP SR number 8606223364. -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking size of uintptr_t" >&5 -$as_echo_n "checking size of uintptr_t... " >&6; } -if ${ac_cv_sizeof_uintptr_t+:} false; then : - $as_echo_n "(cached) " >&6 -else - if ac_fn_c_compute_int "$LINENO" "(long int) (sizeof (uintptr_t))" "ac_cv_sizeof_uintptr_t" "$ac_includes_default"; then : - -else - if test "$ac_cv_type_uintptr_t" = yes; then - { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error 77 "cannot compute sizeof (uintptr_t) -See \`config.log' for more details" "$LINENO" 5; } - else - ac_cv_sizeof_uintptr_t=0 - fi -fi - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sizeof_uintptr_t" >&5 -$as_echo "$ac_cv_sizeof_uintptr_t" >&6; } - - - -cat >>confdefs.h <<_ACEOF -#define SIZEOF_UINTPTR_T $ac_cv_sizeof_uintptr_t -_ACEOF - - -fi - - # The cast to long int works around a bug in the HP C Compiler # version HP92453-01 B.11.11.23709.GP, which incorrectly rejects # declarations like `int a3[[(sizeof (unsigned char)) >= 0]];'. diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -2101,11 +2101,6 @@ AC_CHECK_SIZEOF(_Bool, 1) fi -AC_CHECK_TYPES(uintptr_t, - [AC_CHECK_SIZEOF(uintptr_t, 4)], - [], [#include - #include ]) - AC_CHECK_SIZEOF(off_t, [], [ #ifdef HAVE_SYS_TYPES_H #include diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -1134,9 +1134,6 @@ /* Define this if you have tcl and TCL_UTF_MAX==6 */ #undef HAVE_UCS4_TCL -/* Define to 1 if the system has the type `uintptr_t'. */ -#undef HAVE_UINTPTR_T - /* Define to 1 if you have the `uname' function. */ #undef HAVE_UNAME @@ -1311,9 +1308,6 @@ /* The size of `time_t', as computed by sizeof. */ #undef SIZEOF_TIME_T -/* The size of `uintptr_t', as computed by sizeof. */ -#undef SIZEOF_UINTPTR_T - /* The size of `void *', as computed by sizeof. */ #undef SIZEOF_VOID_P -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 16:35:29 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 20:35:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_out-of-tree_builds_for?= =?utf-8?q?_blake2?= Message-ID: <20160906203528.114719.90951.959640C7@psf.io> https://hg.python.org/cpython/rev/d40af8a3756a changeset: 103141:d40af8a3756a user: Christian Heimes date: Tue Sep 06 22:35:14 2016 +0200 summary: Fix out-of-tree builds for blake2 files: Makefile.pre.in | 8 ++++---- setup.py | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -541,7 +541,7 @@ # Run "Argument Clinic" over all source files # (depends on python having already been built) .PHONY=clinic -clinic: $(BUILDPYTHON) Modules/_blake2/blake2s_impl.c +clinic: $(BUILDPYTHON) $(srcdir)/Modules/_blake2/blake2s_impl.c $(RUNSHARED) $(PYTHON_FOR_BUILD) ./Tools/clinic/clinic.py --make # Build the interpreter @@ -572,9 +572,9 @@ $(CC) -c $(CCSHARED) $(PY_CORE_CFLAGS) -o $@ $< # blake2s is auto-generated from blake2b -Modules/_blake2/blake2s_impl.c: $(BUILDPYTHON) Modules/_blake2/blake2b_impl.c Modules/_blake2/blake2b2s.py - $(RUNSHARED) $(PYTHON_FOR_BUILD) Modules/_blake2/blake2b2s.py - $(RUNSHARED) $(PYTHON_FOR_BUILD) Tools/clinic/clinic.py -f $@ +$(srcdir)/Modules/_blake2/blake2s_impl.c: $(BUILDPYTHON) $(srcdir)/Modules/_blake2/blake2b_impl.c $(srcdir)/Modules/_blake2/blake2b2s.py + $(RUNSHARED) $(PYTHON_FOR_BUILD) $(srcdir)/Modules/_blake2/blake2b2s.py + $(RUNSHARED) $(PYTHON_FOR_BUILD) $(srcdir)/Tools/clinic/clinic.py -f $@ # Build the shared modules # Under GNU make, MAKEFLAGS are sorted and normalized; the 's' for diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -889,8 +889,8 @@ exts.append( Extension('_sha1', ['sha1module.c'], depends=['hashlib.h']) ) - blake2_deps = [os.path.join('_blake2', 'impl', name) - for name in os.listdir('Modules/_blake2/impl')] + blake2_deps = glob(os.path.join(os.getcwd(), srcdir, + 'Modules/_blake2/impl/*')) blake2_deps.append('hashlib.h') blake2_macros = [] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 16:48:21 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 20:48:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_replace_Py=5F=28u=29intptr?= =?utf-8?q?=5Ft_with_the_c99_standard_types?= Message-ID: <20160906204820.104224.75004.1A4B451B@psf.io> https://hg.python.org/cpython/rev/02c8db9c255f changeset: 103142:02c8db9c255f user: Benjamin Peterson date: Tue Sep 06 13:47:26 2016 -0700 summary: replace Py_(u)intptr_t with the c99 standard types files: Include/pyatomic.h | 4 +- Include/pymacro.h | 8 +++--- Include/pymem.h | 6 ++-- Modules/_elementtree.c | 8 +++--- Modules/_sre.c | 14 ++++++------ Modules/_testcapimodule.c | 16 +++++++------- Modules/_tracemalloc.c | 30 +++++++++++++------------- Modules/faulthandler.c | 10 ++++---- Modules/posixmodule.c | 26 +++++++++++----------- Modules/selectmodule.c | 4 +- Modules/signalmodule.c | 4 +- Modules/sre_lib.h | 2 +- Objects/descrobject.c | 2 +- Objects/longobject.c | 4 +- Objects/obmalloc.c | 2 +- PC/msvcrtmodule.c | 14 ++++++------ Parser/grammar.c | 4 +- Parser/parsetok.c | 2 +- Python/ceval_gil.h | 4 +- Python/compile.c | 6 ++-- Python/pystate.c | 2 +- 21 files changed, 86 insertions(+), 86 deletions(-) diff --git a/Include/pyatomic.h b/Include/pyatomic.h --- a/Include/pyatomic.h +++ b/Include/pyatomic.h @@ -61,7 +61,7 @@ } _Py_memory_order; typedef struct _Py_atomic_address { - Py_uintptr_t _value; + uintptr_t _value; } _Py_atomic_address; typedef struct _Py_atomic_int { @@ -98,7 +98,7 @@ } _Py_memory_order; typedef struct _Py_atomic_address { - Py_uintptr_t _value; + uintptr_t _value; } _Py_atomic_address; typedef struct _Py_atomic_int { diff --git a/Include/pymacro.h b/Include/pymacro.h --- a/Include/pymacro.h +++ b/Include/pymacro.h @@ -79,12 +79,12 @@ #define _Py_SIZE_ROUND_UP(n, a) (((size_t)(n) + \ (size_t)((a) - 1)) & ~(size_t)((a) - 1)) /* Round pointer "p" down to the closest "a"-aligned address <= "p". */ -#define _Py_ALIGN_DOWN(p, a) ((void *)((Py_uintptr_t)(p) & ~(Py_uintptr_t)((a) - 1))) +#define _Py_ALIGN_DOWN(p, a) ((void *)((uintptr_t)(p) & ~(uintptr_t)((a) - 1))) /* Round pointer "p" up to the closest "a"-aligned address >= "p". */ -#define _Py_ALIGN_UP(p, a) ((void *)(((Py_uintptr_t)(p) + \ - (Py_uintptr_t)((a) - 1)) & ~(Py_uintptr_t)((a) - 1))) +#define _Py_ALIGN_UP(p, a) ((void *)(((uintptr_t)(p) + \ + (uintptr_t)((a) - 1)) & ~(uintptr_t)((a) - 1))) /* Check if pointer "p" is aligned to "a"-bytes boundary. */ -#define _Py_IS_ALIGNED(p, a) (!((Py_uintptr_t)(p) & (Py_uintptr_t)((a) - 1))) +#define _Py_IS_ALIGNED(p, a) (!((uintptr_t)(p) & (uintptr_t)((a) - 1))) #ifdef __GNUC__ #define Py_UNUSED(name) _unused_ ## name __attribute__((unused)) diff --git a/Include/pymem.h b/Include/pymem.h --- a/Include/pymem.h +++ b/Include/pymem.h @@ -37,7 +37,7 @@ If memory block is already tracked, update the existing trace. */ PyAPI_FUNC(int) _PyTraceMalloc_Track( _PyTraceMalloc_domain_t domain, - Py_uintptr_t ptr, + uintptr_t ptr, size_t size); /* Untrack an allocated memory block in the tracemalloc module. @@ -46,7 +46,7 @@ Return -2 if tracemalloc is disabled, otherwise return 0. */ PyAPI_FUNC(int) _PyTraceMalloc_Untrack( _PyTraceMalloc_domain_t domain, - Py_uintptr_t ptr); + uintptr_t ptr); /* Get the traceback where a memory block was allocated. @@ -58,7 +58,7 @@ Raise an exception and return NULL on error. */ PyAPI_FUNC(PyObject*) _PyTraceMalloc_GetTraceback( _PyTraceMalloc_domain_t domain, - Py_uintptr_t ptr); + uintptr_t ptr); #endif /* !Py_LIMITED_API */ diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -57,9 +57,9 @@ that all use of text and tail as object pointers must be wrapped in JOIN_OBJ. see comments in the ElementObject definition for more info. */ -#define JOIN_GET(p) ((Py_uintptr_t) (p) & 1) -#define JOIN_SET(p, flag) ((void*) ((Py_uintptr_t) (JOIN_OBJ(p)) | (flag))) -#define JOIN_OBJ(p) ((PyObject*) ((Py_uintptr_t) (p) & ~(Py_uintptr_t)1)) +#define JOIN_GET(p) ((uintptr_t) (p) & 1) +#define JOIN_SET(p, flag) ((void*) ((uintptr_t) (JOIN_OBJ(p)) | (flag))) +#define JOIN_OBJ(p) ((PyObject*) ((uintptr_t) (p) & ~(uintptr_t)1)) /* Py_CLEAR for a PyObject* that uses a join flag. Pass the pointer by * reference since this function sets it to NULL. @@ -797,7 +797,7 @@ } /* add object to memo dictionary (so deepcopy won't visit it again) */ - id = PyLong_FromSsize_t((Py_uintptr_t) self); + id = PyLong_FromSsize_t((uintptr_t) self); if (!id) goto error; diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -1582,7 +1582,7 @@ skip = *code; \ VTRACE(("%lu (skip to %p)\n", \ (unsigned long)skip, code+skip)); \ - if (skip-adj > (Py_uintptr_t)(end - code)) \ + if (skip-adj > (uintptr_t)(end - code)) \ FAIL; \ code++; \ } while (0) @@ -1616,7 +1616,7 @@ case SRE_OP_CHARSET: offset = 256/SRE_CODE_BITS; /* 256-bit bitmap */ - if (offset > (Py_uintptr_t)(end - code)) + if (offset > (uintptr_t)(end - code)) FAIL; code += offset; break; @@ -1624,7 +1624,7 @@ case SRE_OP_BIGCHARSET: GET_ARG; /* Number of blocks */ offset = 256/sizeof(SRE_CODE); /* 256-byte table */ - if (offset > (Py_uintptr_t)(end - code)) + if (offset > (uintptr_t)(end - code)) FAIL; /* Make sure that each byte points to a valid block */ for (i = 0; i < 256; i++) { @@ -1633,7 +1633,7 @@ } code += offset; offset = arg * (256/SRE_CODE_BITS); /* 256-bit bitmap times arg */ - if (offset > (Py_uintptr_t)(end - code)) + if (offset > (uintptr_t)(end - code)) FAIL; code += offset; break; @@ -1784,11 +1784,11 @@ GET_ARG; prefix_len = arg; GET_ARG; /* Here comes the prefix string */ - if (prefix_len > (Py_uintptr_t)(newcode - code)) + if (prefix_len > (uintptr_t)(newcode - code)) FAIL; code += prefix_len; /* And here comes the overlap table */ - if (prefix_len > (Py_uintptr_t)(newcode - code)) + if (prefix_len > (uintptr_t)(newcode - code)) FAIL; /* Each overlap value should be < prefix_len */ for (i = 0; i < prefix_len; i++) { @@ -1917,7 +1917,7 @@ to allow arbitrary jumps anywhere in the code; so we just look for a JUMP opcode preceding our skip target. */ - if (skip >= 3 && skip-3 < (Py_uintptr_t)(end - code) && + if (skip >= 3 && skip-3 < (uintptr_t)(end - code) && code[skip-3] == SRE_OP_JUMP) { VTRACE(("both then and else parts present\n")); diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -113,10 +113,10 @@ CHECK_SIZEOF(Py_ssize_t, sizeof(void *)); CHECK_SIGNNESS(Py_ssize_t, 1); - CHECK_SIZEOF(Py_uintptr_t, sizeof(void *)); - CHECK_SIGNNESS(Py_uintptr_t, 0); - CHECK_SIZEOF(Py_intptr_t, sizeof(void *)); - CHECK_SIGNNESS(Py_intptr_t, 1); + CHECK_SIZEOF(uintptr_t, sizeof(void *)); + CHECK_SIGNNESS(uintptr_t, 0); + CHECK_SIZEOF(intptr_t, sizeof(void *)); + CHECK_SIGNNESS(intptr_t, 1); Py_INCREF(Py_None); return Py_None; @@ -3861,11 +3861,11 @@ if (release_gil) { Py_BEGIN_ALLOW_THREADS - res = _PyTraceMalloc_Track(domain, (Py_uintptr_t)ptr, size); + res = _PyTraceMalloc_Track(domain, (uintptr_t)ptr, size); Py_END_ALLOW_THREADS } else { - res = _PyTraceMalloc_Track(domain, (Py_uintptr_t)ptr, size); + res = _PyTraceMalloc_Track(domain, (uintptr_t)ptr, size); } if (res < 0) { @@ -3890,7 +3890,7 @@ if (PyErr_Occurred()) return NULL; - res = _PyTraceMalloc_Untrack(domain, (Py_uintptr_t)ptr); + res = _PyTraceMalloc_Untrack(domain, (uintptr_t)ptr); if (res < 0) { PyErr_SetString(PyExc_RuntimeError, "_PyTraceMalloc_Track error"); return NULL; @@ -3912,7 +3912,7 @@ if (PyErr_Occurred()) return NULL; - return _PyTraceMalloc_GetTraceback(domain, (Py_uintptr_t)ptr); + return _PyTraceMalloc_GetTraceback(domain, (uintptr_t)ptr); } diff --git a/Modules/_tracemalloc.c b/Modules/_tracemalloc.c --- a/Modules/_tracemalloc.c +++ b/Modules/_tracemalloc.c @@ -67,7 +67,7 @@ __attribute__((packed)) #endif { - Py_uintptr_t ptr; + uintptr_t ptr; _PyTraceMalloc_domain_t domain; } pointer_t; @@ -523,7 +523,7 @@ tracemalloc_use_domain_cb(_Py_hashtable_t *old_traces, _Py_hashtable_entry_t *entry, void *user_data) { - Py_uintptr_t ptr; + uintptr_t ptr; pointer_t key; _Py_hashtable_t *new_traces = (_Py_hashtable_t *)user_data; const void *pdata = _Py_HASHTABLE_ENTRY_PDATA(old_traces, entry); @@ -538,7 +538,7 @@ } -/* Convert tracemalloc_traces from compact key (Py_uintptr_t) to pointer_t key. +/* Convert tracemalloc_traces from compact key (uintptr_t) to pointer_t key. * Return 0 on success, -1 on error. */ static int tracemalloc_use_domain(void) @@ -572,7 +572,7 @@ static void -tracemalloc_remove_trace(_PyTraceMalloc_domain_t domain, Py_uintptr_t ptr) +tracemalloc_remove_trace(_PyTraceMalloc_domain_t domain, uintptr_t ptr) { trace_t trace; int removed; @@ -595,11 +595,11 @@ } #define REMOVE_TRACE(ptr) \ - tracemalloc_remove_trace(DEFAULT_DOMAIN, (Py_uintptr_t)(ptr)) + tracemalloc_remove_trace(DEFAULT_DOMAIN, (uintptr_t)(ptr)) static int -tracemalloc_add_trace(_PyTraceMalloc_domain_t domain, Py_uintptr_t ptr, +tracemalloc_add_trace(_PyTraceMalloc_domain_t domain, uintptr_t ptr, size_t size) { pointer_t key = {ptr, domain}; @@ -617,7 +617,7 @@ if (!tracemalloc_config.use_domain && domain != DEFAULT_DOMAIN) { /* first trace using a non-zero domain whereas traces use compact - (Py_uintptr_t) keys: switch to pointer_t keys. */ + (uintptr_t) keys: switch to pointer_t keys. */ if (tracemalloc_use_domain() < 0) { return -1; } @@ -663,7 +663,7 @@ } #define ADD_TRACE(ptr, size) \ - tracemalloc_add_trace(DEFAULT_DOMAIN, (Py_uintptr_t)(ptr), size) + tracemalloc_add_trace(DEFAULT_DOMAIN, (uintptr_t)(ptr), size) static void* @@ -1023,7 +1023,7 @@ hashtable_compare_pointer_t); } else { - tracemalloc_traces = hashtable_new(sizeof(Py_uintptr_t), + tracemalloc_traces = hashtable_new(sizeof(uintptr_t), sizeof(trace_t), _Py_hashtable_hash_ptr, _Py_hashtable_compare_direct); @@ -1414,7 +1414,7 @@ static traceback_t* -tracemalloc_get_traceback(_PyTraceMalloc_domain_t domain, Py_uintptr_t ptr) +tracemalloc_get_traceback(_PyTraceMalloc_domain_t domain, uintptr_t ptr) { trace_t trace; int found; @@ -1461,7 +1461,7 @@ else ptr = (void *)obj; - traceback = tracemalloc_get_traceback(DEFAULT_DOMAIN, (Py_uintptr_t)ptr); + traceback = tracemalloc_get_traceback(DEFAULT_DOMAIN, (uintptr_t)ptr); if (traceback == NULL) Py_RETURN_NONE; @@ -1489,7 +1489,7 @@ traceback_t *traceback; int i; - traceback = tracemalloc_get_traceback(DEFAULT_DOMAIN, (Py_uintptr_t)ptr); + traceback = tracemalloc_get_traceback(DEFAULT_DOMAIN, (uintptr_t)ptr); if (traceback == NULL) return; @@ -1762,7 +1762,7 @@ } int -_PyTraceMalloc_Track(_PyTraceMalloc_domain_t domain, Py_uintptr_t ptr, +_PyTraceMalloc_Track(_PyTraceMalloc_domain_t domain, uintptr_t ptr, size_t size) { int res; @@ -1791,7 +1791,7 @@ int -_PyTraceMalloc_Untrack(_PyTraceMalloc_domain_t domain, Py_uintptr_t ptr) +_PyTraceMalloc_Untrack(_PyTraceMalloc_domain_t domain, uintptr_t ptr) { if (!tracemalloc_config.tracing) { /* tracemalloc is not tracing: do nothing */ @@ -1807,7 +1807,7 @@ PyObject* -_PyTraceMalloc_GetTraceback(_PyTraceMalloc_domain_t domain, Py_uintptr_t ptr) +_PyTraceMalloc_GetTraceback(_PyTraceMalloc_domain_t domain, uintptr_t ptr) { traceback_t *traceback; diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -1072,12 +1072,12 @@ # pragma intel optimization_level 0 #endif static -Py_uintptr_t -stack_overflow(Py_uintptr_t min_sp, Py_uintptr_t max_sp, size_t *depth) +uintptr_t +stack_overflow(uintptr_t min_sp, uintptr_t max_sp, size_t *depth) { /* allocate 4096 bytes on the stack at each call */ unsigned char buffer[4096]; - Py_uintptr_t sp = (Py_uintptr_t)&buffer; + uintptr_t sp = (uintptr_t)&buffer; *depth += 1; if (sp < min_sp || max_sp < sp) return sp; @@ -1090,8 +1090,8 @@ faulthandler_stack_overflow(PyObject *self) { size_t depth, size; - Py_uintptr_t sp = (Py_uintptr_t)&depth; - Py_uintptr_t stop; + uintptr_t sp = (uintptr_t)&depth; + uintptr_t stop; faulthandler_suppress_crash_report(); depth = 0; diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -2492,8 +2492,8 @@ type = 'id_t' format_unit = '" _Py_PARSE_PID "' -class Py_intptr_t_converter(CConverter): - type = 'Py_intptr_t' +class intptr_t_converter(CConverter): + type = 'intptr_t' format_unit = '" _Py_PARSE_INTPTR "' class Py_off_t_converter(CConverter): @@ -5244,7 +5244,7 @@ char **argvlist; int i; Py_ssize_t argc; - Py_intptr_t spawnval; + intptr_t spawnval; PyObject *(*getitem)(PyObject *, Py_ssize_t); /* spawnv has three arguments: (mode, path, argv), where @@ -5323,7 +5323,7 @@ char **envlist; PyObject *res = NULL; Py_ssize_t argc, i, envc; - Py_intptr_t spawnval; + intptr_t spawnval; PyObject *(*getitem)(PyObject *, Py_ssize_t); Py_ssize_t lastarg = 0; @@ -7078,7 +7078,7 @@ /* MS C has a variant of waitpid() that's usable for most purposes. */ /*[clinic input] os.waitpid - pid: Py_intptr_t + pid: intptr_t options: int / @@ -7091,11 +7091,11 @@ [clinic start generated code]*/ static PyObject * -os_waitpid_impl(PyObject *module, Py_intptr_t pid, int options) +os_waitpid_impl(PyObject *module, intptr_t pid, int options) /*[clinic end generated code: output=15f1ce005a346b09 input=444c8f51cca5b862]*/ { int status; - Py_intptr_t res; + intptr_t res; int async_err = 0; do { @@ -8559,8 +8559,8 @@ Py_BEGIN_ALLOW_THREADS ok = CreatePipe(&read, &write, &attr, 0); if (ok) { - fds[0] = _open_osfhandle((Py_intptr_t)read, _O_RDONLY); - fds[1] = _open_osfhandle((Py_intptr_t)write, _O_WRONLY); + fds[0] = _open_osfhandle((intptr_t)read, _O_RDONLY); + fds[1] = _open_osfhandle((intptr_t)write, _O_WRONLY); if (fds[0] == -1 || fds[1] == -1) { CloseHandle(read); CloseHandle(write); @@ -11375,14 +11375,14 @@ #ifdef MS_WINDOWS /*[clinic input] os.get_handle_inheritable -> bool - handle: Py_intptr_t + handle: intptr_t / Get the close-on-exe flag of the specified file descriptor. [clinic start generated code]*/ static int -os_get_handle_inheritable_impl(PyObject *module, Py_intptr_t handle) +os_get_handle_inheritable_impl(PyObject *module, intptr_t handle) /*[clinic end generated code: output=9e5389b0aa0916ce input=5f7759443aae3dc5]*/ { DWORD flags; @@ -11398,7 +11398,7 @@ /*[clinic input] os.set_handle_inheritable - handle: Py_intptr_t + handle: intptr_t inheritable: bool / @@ -11406,7 +11406,7 @@ [clinic start generated code]*/ static PyObject * -os_set_handle_inheritable_impl(PyObject *module, Py_intptr_t handle, +os_set_handle_inheritable_impl(PyObject *module, intptr_t handle, int inheritable) /*[clinic end generated code: output=b1e67bfa3213d745 input=e64b2b2730469def]*/ { diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c --- a/Modules/selectmodule.c +++ b/Modules/selectmodule.c @@ -1797,7 +1797,7 @@ */ #if !defined(__OpenBSD__) # define IDENT_TYPE T_UINTPTRT -# define IDENT_CAST Py_intptr_t +# define IDENT_CAST intptr_t # define DATA_TYPE T_INTPTRT # define DATA_FMT_UNIT INTPTRT_FMT_UNIT # define IDENT_AsType PyLong_AsUintptr_t @@ -1876,7 +1876,7 @@ kqueue_event_richcompare(kqueue_event_Object *s, kqueue_event_Object *o, int op) { - Py_intptr_t result = 0; + intptr_t result = 0; if (!kqueue_event_Check(o)) { if (op == Py_EQ || op == Py_NE) { diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -198,7 +198,7 @@ report_wakeup_write_error(void *data) { int save_errno = errno; - errno = (int) (Py_intptr_t) data; + errno = (int) (intptr_t) data; PyErr_SetFromErrno(PyExc_OSError); PySys_WriteStderr("Exception ignored when trying to write to the " "signal wakeup fd:\n"); @@ -277,7 +277,7 @@ if (rc < 0) { Py_AddPendingCall(report_wakeup_write_error, - (void *)(Py_intptr_t)errno); + (void *)(intptr_t)errno); } } } diff --git a/Modules/sre_lib.h b/Modules/sre_lib.h --- a/Modules/sre_lib.h +++ b/Modules/sre_lib.h @@ -529,7 +529,7 @@ if (ctx->pattern[0] == SRE_OP_INFO) { /* optimization info block */ /* <1=skip> <2=flags> <3=min> ... */ - if (ctx->pattern[3] && (Py_uintptr_t)(end - ctx->ptr) < ctx->pattern[3]) { + if (ctx->pattern[3] && (uintptr_t)(end - ctx->ptr) < ctx->pattern[3]) { TRACE(("reject (got %" PY_FORMAT_SIZE_T "d chars, " "need %" PY_FORMAT_SIZE_T "d)\n", end - ctx->ptr, (Py_ssize_t) ctx->pattern[3])); diff --git a/Objects/descrobject.c b/Objects/descrobject.c --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -1019,7 +1019,7 @@ static PyObject * wrapper_richcompare(PyObject *a, PyObject *b, int op) { - Py_intptr_t result; + intptr_t result; PyObject *v; PyWrapperDescrObject *a_descr, *b_descr; diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -989,13 +989,13 @@ PyLong_FromVoidPtr(void *p) { #if SIZEOF_VOID_P <= SIZEOF_LONG - return PyLong_FromUnsignedLong((unsigned long)(Py_uintptr_t)p); + return PyLong_FromUnsignedLong((unsigned long)(uintptr_t)p); #else #if SIZEOF_LONG_LONG < SIZEOF_VOID_P # error "PyLong_FromVoidPtr: sizeof(long long) < sizeof(void*)" #endif - return PyLong_FromUnsignedLongLong((unsigned long long)(Py_uintptr_t)p); + return PyLong_FromUnsignedLongLong((unsigned long long)(uintptr_t)p); #endif /* SIZEOF_VOID_P <= SIZEOF_LONG */ } diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -18,7 +18,7 @@ #define uint unsigned int /* assuming >= 16 bits */ #undef uptr -#define uptr Py_uintptr_t +#define uptr uintptr_t /* Forward declaration */ static void* _PyMem_DebugRawMalloc(void *ctx, size_t size); diff --git a/PC/msvcrtmodule.c b/PC/msvcrtmodule.c --- a/PC/msvcrtmodule.c +++ b/PC/msvcrtmodule.c @@ -33,12 +33,12 @@ #endif /*[python input] -class Py_intptr_t_converter(CConverter): - type = 'Py_intptr_t' +class intptr_t_converter(CConverter): + type = 'intptr_t' format_unit = '"_Py_PARSE_INTPTR"' class handle_return_converter(long_return_converter): - type = 'Py_intptr_t' + type = 'intptr_t' cast = '(void *)' conversion_fn = 'PyLong_FromVoidPtr' @@ -148,7 +148,7 @@ /*[clinic input] msvcrt.open_osfhandle -> long - handle: Py_intptr_t + handle: intptr_t flags: int / @@ -160,7 +160,7 @@ [clinic start generated code]*/ static long -msvcrt_open_osfhandle_impl(PyObject *module, Py_intptr_t handle, int flags) +msvcrt_open_osfhandle_impl(PyObject *module, intptr_t handle, int flags) /*[clinic end generated code: output=bf65e422243a39f9 input=4d8516ed32db8f65]*/ { int fd; @@ -183,11 +183,11 @@ Raises IOError if fd is not recognized. [clinic start generated code]*/ -static Py_intptr_t +static intptr_t msvcrt_get_osfhandle_impl(PyObject *module, int fd) /*[clinic end generated code: output=eac47643338c0baa input=c7d18d02c8017ec1]*/ { - Py_intptr_t handle = -1; + intptr_t handle = -1; if (!_PyVerify_fd(fd)) { PyErr_SetFromErrno(PyExc_IOError); diff --git a/Parser/grammar.c b/Parser/grammar.c --- a/Parser/grammar.c +++ b/Parser/grammar.c @@ -63,7 +63,7 @@ s->s_upper = 0; s->s_accel = NULL; s->s_accept = 0; - return Py_SAFE_DOWNCAST(s - d->d_state, Py_intptr_t, int); + return Py_SAFE_DOWNCAST(s - d->d_state, intptr_t, int); } void @@ -105,7 +105,7 @@ if (Py_DebugFlag) printf("Label @ %8p, %d: %s\n", ll, ll->ll_nlabels, PyGrammar_LabelRepr(lb)); - return Py_SAFE_DOWNCAST(lb - ll->ll_label, Py_intptr_t, int); + return Py_SAFE_DOWNCAST(lb - ll->ll_label, intptr_t, int); } /* Same, but rather dies than adds */ diff --git a/Parser/parsetok.c b/Parser/parsetok.c --- a/Parser/parsetok.c +++ b/Parser/parsetok.c @@ -255,7 +255,7 @@ #endif if (a >= tok->line_start) col_offset = Py_SAFE_DOWNCAST(a - tok->line_start, - Py_intptr_t, int); + intptr_t, int); else col_offset = -1; diff --git a/Python/ceval_gil.h b/Python/ceval_gil.h --- a/Python/ceval_gil.h +++ b/Python/ceval_gil.h @@ -178,7 +178,7 @@ /* Sub-interpreter support: threads might have been switched under our feet using PyThreadState_Swap(). Fix the GIL last holder variable so that our heuristics work. */ - _Py_atomic_store_relaxed(&gil_last_holder, (Py_uintptr_t)tstate); + _Py_atomic_store_relaxed(&gil_last_holder, (uintptr_t)tstate); } MUTEX_LOCK(gil_mutex); @@ -240,7 +240,7 @@ _Py_ANNOTATE_RWLOCK_ACQUIRED(&gil_locked, /*is_write=*/1); if (tstate != (PyThreadState*)_Py_atomic_load_relaxed(&gil_last_holder)) { - _Py_atomic_store_relaxed(&gil_last_holder, (Py_uintptr_t)tstate); + _Py_atomic_store_relaxed(&gil_last_holder, (uintptr_t)tstate); ++gil_switch_number; } diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -476,9 +476,9 @@ { basicblock *block; for (block = u->u_blocks; block != NULL; block = block->b_list) { - assert((Py_uintptr_t)block != 0xcbcbcbcbU); - assert((Py_uintptr_t)block != 0xfbfbfbfbU); - assert((Py_uintptr_t)block != 0xdbdbdbdbU); + assert((uintptr_t)block != 0xcbcbcbcbU); + assert((uintptr_t)block != 0xfbfbfbfbU); + assert((uintptr_t)block != 0xdbdbdbdbU); if (block->b_instr != NULL) { assert(block->b_ialloc > 0); assert(block->b_iused > 0); diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -6,7 +6,7 @@ #define GET_TSTATE() \ ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)) #define SET_TSTATE(value) \ - _Py_atomic_store_relaxed(&_PyThreadState_Current, (Py_uintptr_t)(value)) + _Py_atomic_store_relaxed(&_PyThreadState_Current, (uintptr_t)(value)) #define GET_INTERP_STATE() \ (GET_TSTATE()->interp) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 16:53:38 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 20:53:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_add_back_SIZEOF=5FUINTPTR?= =?utf-8?q?=5FT?= Message-ID: <20160906205337.114736.32642.C10E9563@psf.io> https://hg.python.org/cpython/rev/b10b14fc4975 changeset: 103143:b10b14fc4975 user: Benjamin Peterson date: Tue Sep 06 13:53:14 2016 -0700 summary: add back SIZEOF_UINTPTR_T files: configure | 33 +++++++++++++++++++++++++++++++++ configure.ac | 1 + pyconfig.h.in | 3 +++ 3 files changed, 37 insertions(+), 0 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -8347,6 +8347,39 @@ _ACEOF +# The cast to long int works around a bug in the HP C Compiler +# version HP92453-01 B.11.11.23709.GP, which incorrectly rejects +# declarations like `int a3[[(sizeof (unsigned char)) >= 0]];'. +# This bug is HP SR number 8606223364. +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking size of uintptr_t" >&5 +$as_echo_n "checking size of uintptr_t... " >&6; } +if ${ac_cv_sizeof_uintptr_t+:} false; then : + $as_echo_n "(cached) " >&6 +else + if ac_fn_c_compute_int "$LINENO" "(long int) (sizeof (uintptr_t))" "ac_cv_sizeof_uintptr_t" "$ac_includes_default"; then : + +else + if test "$ac_cv_type_uintptr_t" = yes; then + { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 +$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} +as_fn_error 77 "cannot compute sizeof (uintptr_t) +See \`config.log' for more details" "$LINENO" 5; } + else + ac_cv_sizeof_uintptr_t=0 + fi +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sizeof_uintptr_t" >&5 +$as_echo "$ac_cv_sizeof_uintptr_t" >&6; } + + + +cat >>confdefs.h <<_ACEOF +#define SIZEOF_UINTPTR_T $ac_cv_sizeof_uintptr_t +_ACEOF + + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for long double support" >&5 $as_echo_n "checking for long double support... " >&6; } diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -2077,6 +2077,7 @@ AC_CHECK_SIZEOF(fpos_t, 4) AC_CHECK_SIZEOF(size_t, 4) AC_CHECK_SIZEOF(pid_t, 4) +AC_CHECK_SIZEOF(uintptr_t) AC_MSG_CHECKING(for long double support) have_long_double=no diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -1308,6 +1308,9 @@ /* The size of `time_t', as computed by sizeof. */ #undef SIZEOF_TIME_T +/* The size of `uintptr_t', as computed by sizeof. */ +#undef SIZEOF_UINTPTR_T + /* The size of `void *', as computed by sizeof. */ #undef SIZEOF_VOID_P -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 16:55:30 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Tue, 06 Sep 2016 20:55:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325761=3A_Improved?= =?utf-8?q?_error_reporting_about_truncated_pickle_data_in?= Message-ID: <20160906205530.30758.19278.647D4E2C@psf.io> https://hg.python.org/cpython/rev/231f578dfd3d changeset: 103144:231f578dfd3d user: Serhiy Storchaka date: Tue Sep 06 23:55:11 2016 +0300 summary: Issue #25761: Improved error reporting about truncated pickle data in C implementation of unpickler. UnpicklingError is now raised instead of AttributeError and ValueError in some cases. files: Lib/test/test_pickle.py | 3 +- Misc/NEWS | 4 + Modules/_pickle.c | 84 ++++++++++++++-------------- 3 files changed, 48 insertions(+), 43 deletions(-) diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py --- a/Lib/test/test_pickle.py +++ b/Lib/test/test_pickle.py @@ -139,8 +139,7 @@ class CUnpicklerTests(PyUnpicklerTests): unpickler = _pickle.Unpickler bad_stack_errors = (pickle.UnpicklingError,) - truncated_errors = (pickle.UnpicklingError, EOFError, - AttributeError, ValueError) + truncated_errors = (pickle.UnpicklingError,) class CPicklerTests(PyPicklerTests): pickler = _pickle.Pickler diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -89,6 +89,10 @@ Library ------- +- Issue #25761: Improved error reporting about truncated pickle data in + C implementation of unpickler. UnpicklingError is now raised instead of + AttributeError and ValueError in some cases. + - Issue #26798: Add BLAKE2 (blake2b and blake2s) to hashlib. - Issue #25596: Optimized glob() and iglob() functions in the diff --git a/Modules/_pickle.c b/Modules/_pickle.c --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -1092,6 +1092,14 @@ } static int +bad_readline(void) +{ + PickleState *st = _Pickle_GetGlobalState(); + PyErr_SetString(st->UnpicklingError, "pickle data was truncated"); + return -1; +} + +static int _Unpickler_SkipConsumed(UnpicklerObject *self) { Py_ssize_t consumed; @@ -1195,17 +1203,14 @@ /* This case is handled by the _Unpickler_Read() macro for efficiency */ assert(self->next_read_idx + n > self->input_len); - if (!self->read) { - PyErr_Format(PyExc_EOFError, "Ran out of input"); - return -1; - } + if (!self->read) + return bad_readline(); + num_read = _Unpickler_ReadFromFile(self, n); if (num_read < 0) return -1; - if (num_read < n) { - PyErr_Format(PyExc_EOFError, "Ran out of input"); - return -1; - } + if (num_read < n) + return bad_readline(); *s = self->input_buffer; self->next_read_idx = n; return n; @@ -1249,7 +1254,7 @@ } /* Read a line from the input stream/buffer. If we run off the end of the input - before hitting \n, return the data we found. + before hitting \n, raise an error. Returns the number of chars read, or -1 on failure. */ static Py_ssize_t @@ -1265,20 +1270,16 @@ return _Unpickler_CopyLine(self, line_start, num_read, result); } } - if (self->read) { - num_read = _Unpickler_ReadFromFile(self, READ_WHOLE_LINE); - if (num_read < 0) - return -1; - self->next_read_idx = num_read; - return _Unpickler_CopyLine(self, self->input_buffer, num_read, result); - } - - /* If we get here, we've run off the end of the input string. Return the - remaining string and let the caller figure it out. */ - *result = self->input_buffer + self->next_read_idx; - num_read = i - self->next_read_idx; - self->next_read_idx = i; - return num_read; + if (!self->read) + return bad_readline(); + + num_read = _Unpickler_ReadFromFile(self, READ_WHOLE_LINE); + if (num_read < 0) + return -1; + if (num_read == 0 || self->input_buffer[num_read - 1] != '\n') + return bad_readline(); + self->next_read_idx = num_read; + return _Unpickler_CopyLine(self, self->input_buffer, num_read, result); } /* Returns -1 (with an exception set) on failure, 0 on success. The memo array @@ -4600,14 +4601,6 @@ } static int -bad_readline(void) -{ - PickleState *st = _Pickle_GetGlobalState(); - PyErr_SetString(st->UnpicklingError, "pickle data was truncated"); - return -1; -} - -static int load_int(UnpicklerObject *self) { PyObject *value; @@ -6245,8 +6238,13 @@ case opcode: if (load_func(self, (arg)) < 0) break; continue; while (1) { - if (_Unpickler_Read(self, &s, 1) < 0) - break; + if (_Unpickler_Read(self, &s, 1) < 0) { + PickleState *st = _Pickle_GetGlobalState(); + if (PyErr_ExceptionMatches(st->UnpicklingError)) { + PyErr_Format(PyExc_EOFError, "Ran out of input"); + } + return NULL; + } switch ((enum opcode)s[0]) { OP(NONE, load_none) @@ -6318,15 +6316,19 @@ break; default: - if (s[0] == '\0') { - PyErr_SetNone(PyExc_EOFError); + { + PickleState *st = _Pickle_GetGlobalState(); + unsigned char c = (unsigned char) *s; + if (0x20 <= c && c <= 0x7e && c != '\'' && c != '\\') { + PyErr_Format(st->UnpicklingError, + "invalid load key, '%c'.", c); + } + else { + PyErr_Format(st->UnpicklingError, + "invalid load key, '\\x%02x'.", c); + } + return NULL; } - else { - PickleState *st = _Pickle_GetGlobalState(); - PyErr_Format(st->UnpicklingError, - "invalid load key, '%c'.", s[0]); - } - return NULL; } break; /* and we are done! */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 17:18:11 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 21:18:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326798=3A_for_loop?= =?utf-8?q?_initial_declarations_are_only_allowed_in_C99_or_C11_mode?= Message-ID: <20160906211811.13686.4521.79E7B1AC@psf.io> https://hg.python.org/cpython/rev/be6f3449ac13 changeset: 103145:be6f3449ac13 user: Christian Heimes date: Tue Sep 06 23:18:03 2016 +0200 summary: Issue #26798: for loop initial declarations are only allowed in C99 or C11 mode files: Modules/_blake2/impl/blake2b-ref.c | 6 ++++-- Modules/_blake2/impl/blake2b.c | 6 ++++-- Modules/_blake2/impl/blake2s-ref.c | 6 ++++-- Modules/_blake2/impl/blake2s.c | 9 ++++++--- 4 files changed, 18 insertions(+), 9 deletions(-) diff --git a/Modules/_blake2/impl/blake2b-ref.c b/Modules/_blake2/impl/blake2b-ref.c --- a/Modules/_blake2/impl/blake2b-ref.c +++ b/Modules/_blake2/impl/blake2b-ref.c @@ -145,9 +145,10 @@ BLAKE2_LOCAL_INLINE(int) blake2b_init0( blake2b_state *S ) { + int i; memset( S, 0, sizeof( blake2b_state ) ); - for( int i = 0; i < 8; ++i ) S->h[i] = blake2b_IV[i]; + for( i = 0; i < 8; ++i ) S->h[i] = blake2b_IV[i]; return 0; } @@ -319,6 +320,7 @@ int blake2b_final( blake2b_state *S, uint8_t *out, uint8_t outlen ) { uint8_t buffer[BLAKE2B_OUTBYTES] = {0}; + int i; if( out == NULL || outlen == 0 || outlen > BLAKE2B_OUTBYTES ) return -1; @@ -339,7 +341,7 @@ memset( S->buf + S->buflen, 0, 2 * BLAKE2B_BLOCKBYTES - S->buflen ); /* Padding */ blake2b_compress( S, S->buf ); - for( int i = 0; i < 8; ++i ) /* Output full hash to temp buffer */ + for( i = 0; i < 8; ++i ) /* Output full hash to temp buffer */ store64( buffer + sizeof( S->h[i] ) * i, S->h[i] ); memcpy( out, buffer, outlen ); diff --git a/Modules/_blake2/impl/blake2b.c b/Modules/_blake2/impl/blake2b.c --- a/Modules/_blake2/impl/blake2b.c +++ b/Modules/_blake2/impl/blake2b.c @@ -174,9 +174,10 @@ BLAKE2_LOCAL_INLINE(int) blake2b_init0( blake2b_state *S ) { + int i; memset( S, 0, sizeof( blake2b_state ) ); - for( int i = 0; i < 8; ++i ) S->h[i] = blake2b_IV[i]; + for( i = 0; i < 8; ++i ) S->h[i] = blake2b_IV[i]; return 0; } @@ -188,10 +189,11 @@ const uint8_t * v = ( const uint8_t * )( blake2b_IV ); const uint8_t * p = ( const uint8_t * )( P ); uint8_t * h = ( uint8_t * )( S->h ); + int i; /* IV XOR ParamBlock */ memset( S, 0, sizeof( blake2b_state ) ); - for( int i = 0; i < BLAKE2B_OUTBYTES; ++i ) h[i] = v[i] ^ p[i]; + for( i = 0; i < BLAKE2B_OUTBYTES; ++i ) h[i] = v[i] ^ p[i]; return 0; } diff --git a/Modules/_blake2/impl/blake2s-ref.c b/Modules/_blake2/impl/blake2s-ref.c --- a/Modules/_blake2/impl/blake2s-ref.c +++ b/Modules/_blake2/impl/blake2s-ref.c @@ -138,9 +138,10 @@ BLAKE2_LOCAL_INLINE(int) blake2s_init0( blake2s_state *S ) { + int i; memset( S, 0, sizeof( blake2s_state ) ); - for( int i = 0; i < 8; ++i ) S->h[i] = blake2s_IV[i]; + for( i = 0; i < 8; ++i ) S->h[i] = blake2s_IV[i]; return 0; } @@ -308,6 +309,7 @@ int blake2s_final( blake2s_state *S, uint8_t *out, uint8_t outlen ) { uint8_t buffer[BLAKE2S_OUTBYTES] = {0}; + int i; if( out == NULL || outlen == 0 || outlen > BLAKE2S_OUTBYTES ) return -1; @@ -329,7 +331,7 @@ memset( S->buf + S->buflen, 0, 2 * BLAKE2S_BLOCKBYTES - S->buflen ); /* Padding */ blake2s_compress( S, S->buf ); - for( int i = 0; i < 8; ++i ) /* Output full hash to temp buffer */ + for( i = 0; i < 8; ++i ) /* Output full hash to temp buffer */ store32( buffer + sizeof( S->h[i] ) * i, S->h[i] ); memcpy( out, buffer, outlen ); diff --git a/Modules/_blake2/impl/blake2s.c b/Modules/_blake2/impl/blake2s.c --- a/Modules/_blake2/impl/blake2s.c +++ b/Modules/_blake2/impl/blake2s.c @@ -161,9 +161,10 @@ BLAKE2_LOCAL_INLINE(int) blake2s_init0( blake2s_state *S ) { + int i; memset( S, 0, sizeof( blake2s_state ) ); - for( int i = 0; i < 8; ++i ) S->h[i] = blake2s_IV[i]; + for( i = 0; i < 8; ++i ) S->h[i] = blake2s_IV[i]; return 0; } @@ -175,10 +176,11 @@ const uint8_t * v = ( const uint8_t * )( blake2s_IV ); const uint8_t * p = ( const uint8_t * )( P ); uint8_t * h = ( uint8_t * )( S->h ); + int i; /* IV XOR ParamBlock */ memset( S, 0, sizeof( blake2s_state ) ); - for( int i = 0; i < BLAKE2S_OUTBYTES; ++i ) h[i] = v[i] ^ p[i]; + for( i = 0; i < BLAKE2S_OUTBYTES; ++i ) h[i] = v[i] ^ p[i]; return 0; } @@ -333,6 +335,7 @@ int blake2s_final( blake2s_state *S, uint8_t *out, uint8_t outlen ) { uint8_t buffer[BLAKE2S_OUTBYTES] = {0}; + int i; if( outlen > BLAKE2S_OUTBYTES ) return -1; @@ -353,7 +356,7 @@ memset( S->buf + S->buflen, 0, 2 * BLAKE2S_BLOCKBYTES - S->buflen ); /* Padding */ blake2s_compress( S, S->buf ); - for( int i = 0; i < 8; ++i ) /* Output full hash to temp buffer */ + for( i = 0; i < 8; ++i ) /* Output full hash to temp buffer */ store32( buffer + sizeof( S->h[i] ) * i, S->h[i] ); memcpy( out, buffer, outlen ); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 17:28:23 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 21:28:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3Njkx?= =?utf-8?q?=3A_Fix_ssl_module=27s_parsing_of_GEN=5FRID_subject_alternative?= =?utf-8?q?_name?= Message-ID: <20160906212823.14647.62975.7B51AEA9@psf.io> https://hg.python.org/cpython/rev/9bbf0b31da48 changeset: 103146:9bbf0b31da48 branch: 3.5 parent: 103124:d2111109fd77 user: Christian Heimes date: Tue Sep 06 23:25:35 2016 +0200 summary: Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name fields in X.509 certs. files: Lib/test/allsans.pem | 37 +++++++++++++++++++ Lib/test/make_ssl_certs.py | 49 +++++++++++++++++++++++-- Lib/test/test_ssl.py | 23 ++++++++++++ Misc/NEWS | 3 + Modules/_ssl.c | 35 ++++++++++++++++++- 5 files changed, 142 insertions(+), 5 deletions(-) diff --git a/Lib/test/allsans.pem b/Lib/test/allsans.pem new file mode 100644 --- /dev/null +++ b/Lib/test/allsans.pem @@ -0,0 +1,37 @@ +-----BEGIN PRIVATE KEY----- +MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAOoy7/QOtTjQ0niE +6uDcTwtkC0R2Tvy1AjVnXohCntZfdzbTGDoYTgXSOLsP8A697jUiJ8VCePGH50xG +Z4DKnAF3a9O3a9nr2pLXb0iY3XOMv+YEBii7CfI+3oxFYgCl0sMgHzDD2ZTVYAsm +DWgLUVsE2gHEccRwrM2tPf2EgR+FAgMBAAECgYEA3qyfyYVSeTrTYxO93x6ZaVMu +A2IZp9zSxMQL9bKiI2GRj+cV2ebSCGbg2btFnD6qBor7FWsmYz+8g6FNN/9sY4az +61rMqMtQvLBe+7L8w70FeTze4qQ4Y1oQri0qD6tBWhDVlpnbI5Py9bkZKD67yVUk +elcEA/5x4PrYXkuqsAECQQD80NjT0mDvaY0JOOaQFSEpMv6QiUA8GGX8Xli7IoKb +tAolPG8rQBa+qSpcWfDMTrWw/aWHuMEEQoP/bVDH9W4FAkEA7SYQbBAKnojZ5A3G +kOHdV7aeivRQxQk/JN8Fb8oKB9Csvpv/BsuGxPKXHdhFa6CBTTsNRtHQw/szPo4l +xMIjgQJAPoMxqibR+0EBM6+TKzteSL6oPXsCnBl4Vk/J5vPgkbmR7KUl4+7j8N8J +b2554TrxKEN/w7CGYZRE6UrRd7ATNQJAWD7Yz41sli+wfPdPU2xo1BHljyl4wMk/ +EPZYbI/PCbdyAH/F935WyQTIjNeEhZc1Zkq6FwdOWw8ns3hrv3rKgQJAHXv1BqUa +czGPIFxX2TNoqtcl6/En4vrxVB1wzsfzkkDAg98kBl7qsF+S3qujSzKikjeaVbI2 +/CyWR2P3yLtOmA== +-----END PRIVATE KEY----- +-----BEGIN CERTIFICATE----- +MIIDcjCCAtugAwIBAgIJAN5dc9TOWjB7MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTYwODA1 +MTAyMTExWhcNMjYwODAzMTAyMTExWjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO +Q2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3VuZGF0 +aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB +gQDqMu/0DrU40NJ4hOrg3E8LZAtEdk78tQI1Z16IQp7WX3c20xg6GE4F0ji7D/AO +ve41IifFQnjxh+dMRmeAypwBd2vTt2vZ69qS129ImN1zjL/mBAYouwnyPt6MRWIA +pdLDIB8ww9mU1WALJg1oC1FbBNoBxHHEcKzNrT39hIEfhQIDAQABo4IBODCCATQw +ggEwBgNVHREEggEnMIIBI4IHYWxsc2Fuc6AeBgMqAwSgFwwVc29tZSBvdGhlciBp +ZGVudGlmaWVyoDUGBisGAQUCAqArMCmgEBsOS0VSQkVST1MuUkVBTE2hFTAToAMC +AQGhDDAKGwh1c2VybmFtZYEQdXNlckBleGFtcGxlLm9yZ4IPd3d3LmV4YW1wbGUu +b3JnpGcwZTELMAkGA1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMw +IQYDVQQKDBpQeXRob24gU29mdHdhcmUgRm91bmRhdGlvbjEYMBYGA1UEAwwPZGly +bmFtZSBleGFtcGxlhhdodHRwczovL3d3dy5weXRob24ub3JnL4cEfwAAAYcQAAAA +AAAAAAAAAAAAAAAAAYgEKgMEBTANBgkqhkiG9w0BAQsFAAOBgQAy16h+F+nOmeiT +VWR0fc8F/j6FcadbLseAUaogcC15OGxCl4UYpLV88HBkABOoGCpP155qwWTwOrdG +iYPGJSusf1OnJEbvzFejZf6u078bPd9/ZL4VWLjv+FPGkjd+N+/OaqMvgj8Lu99f +3Y/C4S7YbHxxwff6C6l2Xli+q6gnuQ== +-----END CERTIFICATE----- diff --git a/Lib/test/make_ssl_certs.py b/Lib/test/make_ssl_certs.py --- a/Lib/test/make_ssl_certs.py +++ b/Lib/test/make_ssl_certs.py @@ -20,7 +20,28 @@ CN = {hostname} [req_x509_extensions] - subjectAltName = DNS:{hostname} + subjectAltName = @san + + [san] + DNS.1 = {hostname} + {extra_san} + + [dir_sect] + C = XY + L = Castle Anthrax + O = Python Software Foundation + CN = dirname example + + [princ_name] + realm = EXP:0, GeneralString:KERBEROS.REALM + principal_name = EXP:1, SEQUENCE:principal_seq + + [principal_seq] + name_type = EXP:0, INTEGER:1 + name_string = EXP:1, SEQUENCE:principals + + [principals] + princ1 = GeneralString:username [ ca ] default_ca = CA_default @@ -67,7 +88,7 @@ here = os.path.abspath(os.path.dirname(__file__)) -def make_cert_key(hostname, sign=False): +def make_cert_key(hostname, sign=False, extra_san=''): print("creating cert for " + hostname) tempnames = [] for i in range(3): @@ -75,8 +96,9 @@ tempnames.append(f.name) req_file, cert_file, key_file = tempnames try: + req = req_template.format(hostname=hostname, extra_san=extra_san) with open(req_file, 'w') as f: - f.write(req_template.format(hostname=hostname)) + f.write(req) args = ['req', '-new', '-days', '3650', '-nodes', '-newkey', 'rsa:1024', '-keyout', key_file, '-config', req_file] @@ -120,7 +142,7 @@ f.write('unique_subject = no') with tempfile.NamedTemporaryFile("w") as t: - t.write(req_template.format(hostname='our-ca-server')) + t.write(req_template.format(hostname='our-ca-server', extra_san='')) t.flush() with tempfile.NamedTemporaryFile() as f: args = ['req', '-new', '-days', '3650', '-extensions', 'v3_ca', '-nodes', @@ -171,6 +193,25 @@ f.write(key) f.write(cert) + extra_san = [ + 'otherName.1 = 1.2.3.4;UTF8:some other identifier', + 'otherName.2 = 1.3.6.1.5.2.2;SEQUENCE:princ_name', + 'email.1 = user at example.org', + 'DNS.2 = www.example.org', + # GEN_X400 + 'dirName.1 = dir_sect', + # GEN_EDIPARTY + 'URI.1 = https://www.python.org/', + 'IP.1 = 127.0.0.1', + 'IP.2 = ::1', + 'RID.1 = 1.2.3.4.5', + ] + + cert, key = make_cert_key('allsans', extra_san='\n'.join(extra_san)) + with open('allsans.pem', 'w') as f: + f.write(key) + f.write(cert) + unmake_ca() print("\n\nPlease change the values in test_ssl.py, test_parse_cert function related to notAfter,notBefore and serialNumber") check_call(['openssl','x509','-in','keycert.pem','-dates','-serial','-noout']) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -57,6 +57,8 @@ SIGNED_CERTFILE = data_file("keycert3.pem") SIGNED_CERTFILE2 = data_file("keycert4.pem") SIGNING_CA = data_file("pycacert.pem") +# cert with all kinds of subject alt names +ALLSANFILE = data_file("allsans.pem") REMOTE_HOST = "self-signed.pythontest.net" REMOTE_ROOT_CERT = data_file("selfsigned_pythontestdotnet.pem") @@ -279,6 +281,27 @@ self.assertEqual(p['subjectAltName'], san) + def test_parse_all_sans(self): + p = ssl._ssl._test_decode_cert(ALLSANFILE) + self.assertEqual(p['subjectAltName'], + ( + ('DNS', 'allsans'), + ('othername', ''), + ('othername', ''), + ('email', 'user at example.org'), + ('DNS', 'www.example.org'), + ('DirName', + ((('countryName', 'XY'),), + (('localityName', 'Castle Anthrax'),), + (('organizationName', 'Python Software Foundation'),), + (('commonName', 'dirname example'),))), + ('URI', 'https://www.python.org/'), + ('IP Address', '127.0.0.1'), + ('IP Address', '0:0:0:0:0:0:0:1\n'), + ('Registered ID', '1.2.3.4.5') + ) + ) + def test_DER_to_PEM(self): with open(CAFILE_CACERT, 'r') as f: pem = f.read() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,9 @@ Library ------- +- Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name + fields in X.509 certs. + - Issue #27850: Remove 3DES from ssl module's default cipher list to counter measure sweet32 attack (CVE-2016-2183). diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1007,6 +1007,35 @@ PyTuple_SET_ITEM(t, 1, v); break; + case GEN_RID: + t = PyTuple_New(2); + if (t == NULL) + goto fail; + + v = PyUnicode_FromString("Registered ID"); + if (v == NULL) { + Py_DECREF(t); + goto fail; + } + PyTuple_SET_ITEM(t, 0, v); + + len = i2t_ASN1_OBJECT(buf, sizeof(buf)-1, name->d.rid); + if (len < 0) { + Py_DECREF(t); + _setSSLError(NULL, 0, __FILE__, __LINE__); + goto fail; + } else if (len >= (int)sizeof(buf)) { + v = PyUnicode_FromString(""); + } else { + v = PyUnicode_FromStringAndSize(buf, len); + } + if (v == NULL) { + Py_DECREF(t); + goto fail; + } + PyTuple_SET_ITEM(t, 1, v); + break; + default: /* for everything else, we use the OpenSSL print form */ switch (gntype) { @@ -1033,8 +1062,12 @@ goto fail; } vptr = strchr(buf, ':'); - if (vptr == NULL) + if (vptr == NULL) { + PyErr_Format(PyExc_ValueError, + "Invalid value %.200s", + buf); goto fail; + } t = PyTuple_New(2); if (t == NULL) goto fail; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 17:28:23 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 21:28:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3Njkx?= =?utf-8?q?=3A_Fix_ssl_module=27s_parsing_of_GEN=5FRID_subject_alternative?= =?utf-8?q?_name?= Message-ID: <20160906212823.3577.44490.7682F11A@psf.io> https://hg.python.org/cpython/rev/74805fd9e734 changeset: 103148:74805fd9e734 branch: 2.7 parent: 103125:6f4f19217d9b user: Christian Heimes date: Tue Sep 06 23:25:35 2016 +0200 summary: Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name fields in X.509 certs. files: Lib/test/allsans.pem | 37 +++++++++++++++++++ Lib/test/make_ssl_certs.py | 49 +++++++++++++++++++++++-- Lib/test/test_ssl.py | 23 ++++++++++++ Misc/NEWS | 3 + Modules/_ssl.c | 35 ++++++++++++++++++- 5 files changed, 142 insertions(+), 5 deletions(-) diff --git a/Lib/test/allsans.pem b/Lib/test/allsans.pem new file mode 100644 --- /dev/null +++ b/Lib/test/allsans.pem @@ -0,0 +1,37 @@ +-----BEGIN PRIVATE KEY----- +MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAOoy7/QOtTjQ0niE +6uDcTwtkC0R2Tvy1AjVnXohCntZfdzbTGDoYTgXSOLsP8A697jUiJ8VCePGH50xG +Z4DKnAF3a9O3a9nr2pLXb0iY3XOMv+YEBii7CfI+3oxFYgCl0sMgHzDD2ZTVYAsm +DWgLUVsE2gHEccRwrM2tPf2EgR+FAgMBAAECgYEA3qyfyYVSeTrTYxO93x6ZaVMu +A2IZp9zSxMQL9bKiI2GRj+cV2ebSCGbg2btFnD6qBor7FWsmYz+8g6FNN/9sY4az +61rMqMtQvLBe+7L8w70FeTze4qQ4Y1oQri0qD6tBWhDVlpnbI5Py9bkZKD67yVUk +elcEA/5x4PrYXkuqsAECQQD80NjT0mDvaY0JOOaQFSEpMv6QiUA8GGX8Xli7IoKb +tAolPG8rQBa+qSpcWfDMTrWw/aWHuMEEQoP/bVDH9W4FAkEA7SYQbBAKnojZ5A3G +kOHdV7aeivRQxQk/JN8Fb8oKB9Csvpv/BsuGxPKXHdhFa6CBTTsNRtHQw/szPo4l +xMIjgQJAPoMxqibR+0EBM6+TKzteSL6oPXsCnBl4Vk/J5vPgkbmR7KUl4+7j8N8J +b2554TrxKEN/w7CGYZRE6UrRd7ATNQJAWD7Yz41sli+wfPdPU2xo1BHljyl4wMk/ +EPZYbI/PCbdyAH/F935WyQTIjNeEhZc1Zkq6FwdOWw8ns3hrv3rKgQJAHXv1BqUa +czGPIFxX2TNoqtcl6/En4vrxVB1wzsfzkkDAg98kBl7qsF+S3qujSzKikjeaVbI2 +/CyWR2P3yLtOmA== +-----END PRIVATE KEY----- +-----BEGIN CERTIFICATE----- +MIIDcjCCAtugAwIBAgIJAN5dc9TOWjB7MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTYwODA1 +MTAyMTExWhcNMjYwODAzMTAyMTExWjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO +Q2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3VuZGF0 +aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB +gQDqMu/0DrU40NJ4hOrg3E8LZAtEdk78tQI1Z16IQp7WX3c20xg6GE4F0ji7D/AO +ve41IifFQnjxh+dMRmeAypwBd2vTt2vZ69qS129ImN1zjL/mBAYouwnyPt6MRWIA +pdLDIB8ww9mU1WALJg1oC1FbBNoBxHHEcKzNrT39hIEfhQIDAQABo4IBODCCATQw +ggEwBgNVHREEggEnMIIBI4IHYWxsc2Fuc6AeBgMqAwSgFwwVc29tZSBvdGhlciBp +ZGVudGlmaWVyoDUGBisGAQUCAqArMCmgEBsOS0VSQkVST1MuUkVBTE2hFTAToAMC +AQGhDDAKGwh1c2VybmFtZYEQdXNlckBleGFtcGxlLm9yZ4IPd3d3LmV4YW1wbGUu +b3JnpGcwZTELMAkGA1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMw +IQYDVQQKDBpQeXRob24gU29mdHdhcmUgRm91bmRhdGlvbjEYMBYGA1UEAwwPZGly +bmFtZSBleGFtcGxlhhdodHRwczovL3d3dy5weXRob24ub3JnL4cEfwAAAYcQAAAA +AAAAAAAAAAAAAAAAAYgEKgMEBTANBgkqhkiG9w0BAQsFAAOBgQAy16h+F+nOmeiT +VWR0fc8F/j6FcadbLseAUaogcC15OGxCl4UYpLV88HBkABOoGCpP155qwWTwOrdG +iYPGJSusf1OnJEbvzFejZf6u078bPd9/ZL4VWLjv+FPGkjd+N+/OaqMvgj8Lu99f +3Y/C4S7YbHxxwff6C6l2Xli+q6gnuQ== +-----END CERTIFICATE----- diff --git a/Lib/test/make_ssl_certs.py b/Lib/test/make_ssl_certs.py --- a/Lib/test/make_ssl_certs.py +++ b/Lib/test/make_ssl_certs.py @@ -20,7 +20,28 @@ CN = {hostname} [req_x509_extensions] - subjectAltName = DNS:{hostname} + subjectAltName = @san + + [san] + DNS.1 = {hostname} + {extra_san} + + [dir_sect] + C = XY + L = Castle Anthrax + O = Python Software Foundation + CN = dirname example + + [princ_name] + realm = EXP:0, GeneralString:KERBEROS.REALM + principal_name = EXP:1, SEQUENCE:principal_seq + + [principal_seq] + name_type = EXP:0, INTEGER:1 + name_string = EXP:1, SEQUENCE:principals + + [principals] + princ1 = GeneralString:username [ ca ] default_ca = CA_default @@ -67,7 +88,7 @@ here = os.path.abspath(os.path.dirname(__file__)) -def make_cert_key(hostname, sign=False): +def make_cert_key(hostname, sign=False, extra_san=''): print("creating cert for " + hostname) tempnames = [] for i in range(3): @@ -75,8 +96,9 @@ tempnames.append(f.name) req_file, cert_file, key_file = tempnames try: + req = req_template.format(hostname=hostname, extra_san=extra_san) with open(req_file, 'w') as f: - f.write(req_template.format(hostname=hostname)) + f.write(req) args = ['req', '-new', '-days', '3650', '-nodes', '-newkey', 'rsa:1024', '-keyout', key_file, '-config', req_file] @@ -120,7 +142,7 @@ f.write('unique_subject = no') with tempfile.NamedTemporaryFile("w") as t: - t.write(req_template.format(hostname='our-ca-server')) + t.write(req_template.format(hostname='our-ca-server', extra_san='')) t.flush() with tempfile.NamedTemporaryFile() as f: args = ['req', '-new', '-days', '3650', '-extensions', 'v3_ca', '-nodes', @@ -171,6 +193,25 @@ f.write(key) f.write(cert) + extra_san = [ + 'otherName.1 = 1.2.3.4;UTF8:some other identifier', + 'otherName.2 = 1.3.6.1.5.2.2;SEQUENCE:princ_name', + 'email.1 = user at example.org', + 'DNS.2 = www.example.org', + # GEN_X400 + 'dirName.1 = dir_sect', + # GEN_EDIPARTY + 'URI.1 = https://www.python.org/', + 'IP.1 = 127.0.0.1', + 'IP.2 = ::1', + 'RID.1 = 1.2.3.4.5', + ] + + cert, key = make_cert_key('allsans', extra_san='\n'.join(extra_san)) + with open('allsans.pem', 'w') as f: + f.write(key) + f.write(cert) + unmake_ca() print("\n\nPlease change the values in test_ssl.py, test_parse_cert function related to notAfter,notBefore and serialNumber") check_call(['openssl','x509','-in','keycert.pem','-dates','-serial','-noout']) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -60,6 +60,8 @@ SIGNED_CERTFILE = data_file("keycert3.pem") SIGNED_CERTFILE2 = data_file("keycert4.pem") SIGNING_CA = data_file("pycacert.pem") +# cert with all kinds of subject alt names +ALLSANFILE = data_file("allsans.pem") REMOTE_HOST = "self-signed.pythontest.net" REMOTE_ROOT_CERT = data_file("selfsigned_pythontestdotnet.pem") @@ -247,6 +249,27 @@ self.assertEqual(p['subjectAltName'], san) + def test_parse_all_sans(self): + p = ssl._ssl._test_decode_cert(ALLSANFILE) + self.assertEqual(p['subjectAltName'], + ( + ('DNS', 'allsans'), + ('othername', ''), + ('othername', ''), + ('email', 'user at example.org'), + ('DNS', 'www.example.org'), + ('DirName', + ((('countryName', 'XY'),), + (('localityName', 'Castle Anthrax'),), + (('organizationName', 'Python Software Foundation'),), + (('commonName', 'dirname example'),))), + ('URI', 'https://www.python.org/'), + ('IP Address', '127.0.0.1'), + ('IP Address', '0:0:0:0:0:0:0:1\n'), + ('Registered ID', '1.2.3.4.5') + ) + ) + def test_DER_to_PEM(self): with open(CAFILE_CACERT, 'r') as f: pem = f.read() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -36,6 +36,9 @@ Library ------- +- Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name + fields in X.509 certs. + - Issue #27850: Remove 3DES from ssl module's default cipher list to counter measure sweet32 attack (CVE-2016-2183). diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -953,6 +953,35 @@ PyTuple_SET_ITEM(t, 1, v); break; + case GEN_RID: + t = PyTuple_New(2); + if (t == NULL) + goto fail; + + v = PyUnicode_FromString("Registered ID"); + if (v == NULL) { + Py_DECREF(t); + goto fail; + } + PyTuple_SET_ITEM(t, 0, v); + + len = i2t_ASN1_OBJECT(buf, sizeof(buf)-1, name->d.rid); + if (len < 0) { + Py_DECREF(t); + _setSSLError(NULL, 0, __FILE__, __LINE__); + goto fail; + } else if (len >= (int)sizeof(buf)) { + v = PyUnicode_FromString(""); + } else { + v = PyUnicode_FromStringAndSize(buf, len); + } + if (v == NULL) { + Py_DECREF(t); + goto fail; + } + PyTuple_SET_ITEM(t, 1, v); + break; + default: /* for everything else, we use the OpenSSL print form */ switch (gntype) { @@ -978,8 +1007,12 @@ goto fail; } vptr = strchr(buf, ':'); - if (vptr == NULL) + if (vptr == NULL) { + PyErr_Format(PyExc_ValueError, + "Invalid value %.200s", + buf); goto fail; + } t = PyTuple_New(2); if (t == NULL) goto fail; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 17:28:24 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 21:28:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327691=3A_Fix_ssl_module=27s_parsing_of_GEN=5FRI?= =?utf-8?q?D_subject_alternative_name?= Message-ID: <20160906212823.78392.92285.70338138@psf.io> https://hg.python.org/cpython/rev/2b9af57af3e4 changeset: 103147:2b9af57af3e4 parent: 103145:be6f3449ac13 parent: 103146:9bbf0b31da48 user: Christian Heimes date: Tue Sep 06 23:27:06 2016 +0200 summary: Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name fields in X.509 certs. files: Lib/test/allsans.pem | 37 +++++++++++++++++++ Lib/test/make_ssl_certs.py | 49 +++++++++++++++++++++++-- Lib/test/test_ssl.py | 23 ++++++++++++ Misc/NEWS | 3 + Modules/_ssl.c | 35 ++++++++++++++++++- 5 files changed, 142 insertions(+), 5 deletions(-) diff --git a/Lib/test/allsans.pem b/Lib/test/allsans.pem new file mode 100644 --- /dev/null +++ b/Lib/test/allsans.pem @@ -0,0 +1,37 @@ +-----BEGIN PRIVATE KEY----- +MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAOoy7/QOtTjQ0niE +6uDcTwtkC0R2Tvy1AjVnXohCntZfdzbTGDoYTgXSOLsP8A697jUiJ8VCePGH50xG +Z4DKnAF3a9O3a9nr2pLXb0iY3XOMv+YEBii7CfI+3oxFYgCl0sMgHzDD2ZTVYAsm +DWgLUVsE2gHEccRwrM2tPf2EgR+FAgMBAAECgYEA3qyfyYVSeTrTYxO93x6ZaVMu +A2IZp9zSxMQL9bKiI2GRj+cV2ebSCGbg2btFnD6qBor7FWsmYz+8g6FNN/9sY4az +61rMqMtQvLBe+7L8w70FeTze4qQ4Y1oQri0qD6tBWhDVlpnbI5Py9bkZKD67yVUk +elcEA/5x4PrYXkuqsAECQQD80NjT0mDvaY0JOOaQFSEpMv6QiUA8GGX8Xli7IoKb +tAolPG8rQBa+qSpcWfDMTrWw/aWHuMEEQoP/bVDH9W4FAkEA7SYQbBAKnojZ5A3G +kOHdV7aeivRQxQk/JN8Fb8oKB9Csvpv/BsuGxPKXHdhFa6CBTTsNRtHQw/szPo4l +xMIjgQJAPoMxqibR+0EBM6+TKzteSL6oPXsCnBl4Vk/J5vPgkbmR7KUl4+7j8N8J +b2554TrxKEN/w7CGYZRE6UrRd7ATNQJAWD7Yz41sli+wfPdPU2xo1BHljyl4wMk/ +EPZYbI/PCbdyAH/F935WyQTIjNeEhZc1Zkq6FwdOWw8ns3hrv3rKgQJAHXv1BqUa +czGPIFxX2TNoqtcl6/En4vrxVB1wzsfzkkDAg98kBl7qsF+S3qujSzKikjeaVbI2 +/CyWR2P3yLtOmA== +-----END PRIVATE KEY----- +-----BEGIN CERTIFICATE----- +MIIDcjCCAtugAwIBAgIJAN5dc9TOWjB7MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV +BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u +IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTYwODA1 +MTAyMTExWhcNMjYwODAzMTAyMTExWjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO +Q2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3VuZGF0 +aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB +gQDqMu/0DrU40NJ4hOrg3E8LZAtEdk78tQI1Z16IQp7WX3c20xg6GE4F0ji7D/AO +ve41IifFQnjxh+dMRmeAypwBd2vTt2vZ69qS129ImN1zjL/mBAYouwnyPt6MRWIA +pdLDIB8ww9mU1WALJg1oC1FbBNoBxHHEcKzNrT39hIEfhQIDAQABo4IBODCCATQw +ggEwBgNVHREEggEnMIIBI4IHYWxsc2Fuc6AeBgMqAwSgFwwVc29tZSBvdGhlciBp +ZGVudGlmaWVyoDUGBisGAQUCAqArMCmgEBsOS0VSQkVST1MuUkVBTE2hFTAToAMC +AQGhDDAKGwh1c2VybmFtZYEQdXNlckBleGFtcGxlLm9yZ4IPd3d3LmV4YW1wbGUu +b3JnpGcwZTELMAkGA1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMw +IQYDVQQKDBpQeXRob24gU29mdHdhcmUgRm91bmRhdGlvbjEYMBYGA1UEAwwPZGly +bmFtZSBleGFtcGxlhhdodHRwczovL3d3dy5weXRob24ub3JnL4cEfwAAAYcQAAAA +AAAAAAAAAAAAAAAAAYgEKgMEBTANBgkqhkiG9w0BAQsFAAOBgQAy16h+F+nOmeiT +VWR0fc8F/j6FcadbLseAUaogcC15OGxCl4UYpLV88HBkABOoGCpP155qwWTwOrdG +iYPGJSusf1OnJEbvzFejZf6u078bPd9/ZL4VWLjv+FPGkjd+N+/OaqMvgj8Lu99f +3Y/C4S7YbHxxwff6C6l2Xli+q6gnuQ== +-----END CERTIFICATE----- diff --git a/Lib/test/make_ssl_certs.py b/Lib/test/make_ssl_certs.py --- a/Lib/test/make_ssl_certs.py +++ b/Lib/test/make_ssl_certs.py @@ -19,7 +19,28 @@ CN = {hostname} [req_x509_extensions] - subjectAltName = DNS:{hostname} + subjectAltName = @san + + [san] + DNS.1 = {hostname} + {extra_san} + + [dir_sect] + C = XY + L = Castle Anthrax + O = Python Software Foundation + CN = dirname example + + [princ_name] + realm = EXP:0, GeneralString:KERBEROS.REALM + principal_name = EXP:1, SEQUENCE:principal_seq + + [principal_seq] + name_type = EXP:0, INTEGER:1 + name_string = EXP:1, SEQUENCE:principals + + [principals] + princ1 = GeneralString:username [ ca ] default_ca = CA_default @@ -66,7 +87,7 @@ here = os.path.abspath(os.path.dirname(__file__)) -def make_cert_key(hostname, sign=False): +def make_cert_key(hostname, sign=False, extra_san=''): print("creating cert for " + hostname) tempnames = [] for i in range(3): @@ -74,8 +95,9 @@ tempnames.append(f.name) req_file, cert_file, key_file = tempnames try: + req = req_template.format(hostname=hostname, extra_san=extra_san) with open(req_file, 'w') as f: - f.write(req_template.format(hostname=hostname)) + f.write(req) args = ['req', '-new', '-days', '3650', '-nodes', '-newkey', 'rsa:1024', '-keyout', key_file, '-config', req_file] @@ -119,7 +141,7 @@ f.write('unique_subject = no') with tempfile.NamedTemporaryFile("w") as t: - t.write(req_template.format(hostname='our-ca-server')) + t.write(req_template.format(hostname='our-ca-server', extra_san='')) t.flush() with tempfile.NamedTemporaryFile() as f: args = ['req', '-new', '-days', '3650', '-extensions', 'v3_ca', '-nodes', @@ -170,6 +192,25 @@ f.write(key) f.write(cert) + extra_san = [ + 'otherName.1 = 1.2.3.4;UTF8:some other identifier', + 'otherName.2 = 1.3.6.1.5.2.2;SEQUENCE:princ_name', + 'email.1 = user at example.org', + 'DNS.2 = www.example.org', + # GEN_X400 + 'dirName.1 = dir_sect', + # GEN_EDIPARTY + 'URI.1 = https://www.python.org/', + 'IP.1 = 127.0.0.1', + 'IP.2 = ::1', + 'RID.1 = 1.2.3.4.5', + ] + + cert, key = make_cert_key('allsans', extra_san='\n'.join(extra_san)) + with open('allsans.pem', 'w') as f: + f.write(key) + f.write(cert) + unmake_ca() print("\n\nPlease change the values in test_ssl.py, test_parse_cert function related to notAfter,notBefore and serialNumber") check_call(['openssl','x509','-in','keycert.pem','-dates','-serial','-noout']) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -65,6 +65,8 @@ SIGNED_CERTFILE2 = data_file("keycert4.pem") # Same certificate as pycacert.pem, but without extra text in file SIGNING_CA = data_file("capath", "ceff1710.0") +# cert with all kinds of subject alt names +ALLSANFILE = data_file("allsans.pem") REMOTE_HOST = "self-signed.pythontest.net" @@ -286,6 +288,27 @@ self.assertEqual(p['subjectAltName'], san) + def test_parse_all_sans(self): + p = ssl._ssl._test_decode_cert(ALLSANFILE) + self.assertEqual(p['subjectAltName'], + ( + ('DNS', 'allsans'), + ('othername', ''), + ('othername', ''), + ('email', 'user at example.org'), + ('DNS', 'www.example.org'), + ('DirName', + ((('countryName', 'XY'),), + (('localityName', 'Castle Anthrax'),), + (('organizationName', 'Python Software Foundation'),), + (('commonName', 'dirname example'),))), + ('URI', 'https://www.python.org/'), + ('IP Address', '127.0.0.1'), + ('IP Address', '0:0:0:0:0:0:0:1\n'), + ('Registered ID', '1.2.3.4.5') + ) + ) + def test_DER_to_PEM(self): with open(CAFILE_CACERT, 'r') as f: pem = f.read() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -89,6 +89,9 @@ Library ------- +- Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name + fields in X.509 certs. + - Issue #25761: Improved error reporting about truncated pickle data in C implementation of unpickler. UnpicklingError is now raised instead of AttributeError and ValueError in some cases. diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1007,6 +1007,35 @@ PyTuple_SET_ITEM(t, 1, v); break; + case GEN_RID: + t = PyTuple_New(2); + if (t == NULL) + goto fail; + + v = PyUnicode_FromString("Registered ID"); + if (v == NULL) { + Py_DECREF(t); + goto fail; + } + PyTuple_SET_ITEM(t, 0, v); + + len = i2t_ASN1_OBJECT(buf, sizeof(buf)-1, name->d.rid); + if (len < 0) { + Py_DECREF(t); + _setSSLError(NULL, 0, __FILE__, __LINE__); + goto fail; + } else if (len >= (int)sizeof(buf)) { + v = PyUnicode_FromString(""); + } else { + v = PyUnicode_FromStringAndSize(buf, len); + } + if (v == NULL) { + Py_DECREF(t); + goto fail; + } + PyTuple_SET_ITEM(t, 1, v); + break; + default: /* for everything else, we use the OpenSSL print form */ switch (gntype) { @@ -1033,8 +1062,12 @@ goto fail; } vptr = strchr(buf, ':'); - if (vptr == NULL) + if (vptr == NULL) { + PyErr_Format(PyExc_ValueError, + "Invalid value %.200s", + buf); goto fail; + } t = PyTuple_New(2); if (t == NULL) goto fail; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 17:32:55 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 21:32:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_improve_grammar?= Message-ID: <20160906213255.78508.99763.23171850@psf.io> https://hg.python.org/cpython/rev/9b8c615187f6 changeset: 103149:9b8c615187f6 parent: 103147:2b9af57af3e4 user: Benjamin Peterson date: Tue Sep 06 14:32:40 2016 -0700 summary: improve grammar files: Doc/library/hashlib-blake2.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/hashlib-blake2.rst b/Doc/library/hashlib-blake2.rst --- a/Doc/library/hashlib-blake2.rst +++ b/Doc/library/hashlib-blake2.rst @@ -370,8 +370,8 @@ / \ 00 01 -The example uses 64-byte internal digests, and returns the 32-byte final -digest. +This example uses 64-byte internal digests, and returns the 32-byte final +digest:: >>> from hashlib import blake2b >>> -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 17:35:52 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 06 Sep 2016 21:35:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2327982=3A_Allow_k?= =?utf-8?q?eyword_arguments_to_winsound_functions?= Message-ID: <20160906213551.14667.49363.964CB800@psf.io> https://hg.python.org/cpython/rev/bd4771d3e88d changeset: 103150:bd4771d3e88d user: Zachary Ware date: Tue Sep 06 16:32:43 2016 -0500 summary: Closes #27982: Allow keyword arguments to winsound functions files: Doc/whatsnew/3.6.rst | 8 +++++ Lib/test/test_winsound.py | 10 ++++++ Misc/NEWS | 3 ++ PC/clinic/winsound.c.h | 40 +++++++++++++++----------- PC/winsound.c | 17 ++++------ 5 files changed, 51 insertions(+), 27 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -692,6 +692,14 @@ (Contributed by Clement Rouault in :issue:`23026`.) +winsound +-------- + +Allowed keyword arguments to be passed to :func:`Beep `, +:func:`MessageBeep `, and :func:`PlaySound +` (:issue:`27982`). + + zipfile ------- diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -51,6 +51,10 @@ for i in range(100, 2000, 100): safe_Beep(i, 75) + def test_keyword_args(self): + safe_Beep(duration=75, frequency=2000) + + class MessageBeepTest(unittest.TestCase): def tearDown(self): @@ -76,6 +80,9 @@ def test_question(self): safe_MessageBeep(winsound.MB_ICONQUESTION) + def test_keyword_args(self): + safe_MessageBeep(type=winsound.MB_OK) + class PlaySoundTest(unittest.TestCase): @@ -92,6 +99,9 @@ winsound.SND_MEMORY) self.assertRaises(TypeError, winsound.PlaySound, 1, 0) + def test_keyword_args(self): + safe_PlaySound(flags=winsound.SND_ALIAS, sound="SystemExit") + def test_snd_memory(self): with open(support.findfile('pluck-pcm8.wav', subdir='audiodata'), 'rb') as f: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -215,6 +215,9 @@ Windows ------- +- Issue #27982: The functions of the winsound module now accept keyword + arguments. + - Issue #20366: Build full text search support into SQLite on Windows. - Issue #27756: Adds new icons for Python files and processes on Windows. diff --git a/PC/clinic/winsound.c.h b/PC/clinic/winsound.c.h --- a/PC/clinic/winsound.c.h +++ b/PC/clinic/winsound.c.h @@ -3,7 +3,7 @@ [clinic start generated code]*/ PyDoc_STRVAR(winsound_PlaySound__doc__, -"PlaySound($module, sound, flags, /)\n" +"PlaySound($module, /, sound, flags)\n" "--\n" "\n" "A wrapper around the Windows PlaySound API.\n" @@ -14,19 +14,21 @@ " Flag values, ored together. See module documentation."); #define WINSOUND_PLAYSOUND_METHODDEF \ - {"PlaySound", (PyCFunction)winsound_PlaySound, METH_VARARGS, winsound_PlaySound__doc__}, + {"PlaySound", (PyCFunction)winsound_PlaySound, METH_VARARGS|METH_KEYWORDS, winsound_PlaySound__doc__}, static PyObject * winsound_PlaySound_impl(PyObject *module, PyObject *sound, int flags); static PyObject * -winsound_PlaySound(PyObject *module, PyObject *args) +winsound_PlaySound(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"sound", "flags", NULL}; + static _PyArg_Parser _parser = {"Oi:PlaySound", _keywords, 0}; PyObject *sound; int flags; - if (!PyArg_ParseTuple(args, "Oi:PlaySound", + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &sound, &flags)) { goto exit; } @@ -37,7 +39,7 @@ } PyDoc_STRVAR(winsound_Beep__doc__, -"Beep($module, frequency, duration, /)\n" +"Beep($module, /, frequency, duration)\n" "--\n" "\n" "A wrapper around the Windows Beep API.\n" @@ -49,19 +51,21 @@ " How long the sound should play, in milliseconds."); #define WINSOUND_BEEP_METHODDEF \ - {"Beep", (PyCFunction)winsound_Beep, METH_VARARGS, winsound_Beep__doc__}, + {"Beep", (PyCFunction)winsound_Beep, METH_VARARGS|METH_KEYWORDS, winsound_Beep__doc__}, static PyObject * winsound_Beep_impl(PyObject *module, int frequency, int duration); static PyObject * -winsound_Beep(PyObject *module, PyObject *args) +winsound_Beep(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"frequency", "duration", NULL}; + static _PyArg_Parser _parser = {"ii:Beep", _keywords, 0}; int frequency; int duration; - if (!PyArg_ParseTuple(args, "ii:Beep", + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, &frequency, &duration)) { goto exit; } @@ -72,7 +76,7 @@ } PyDoc_STRVAR(winsound_MessageBeep__doc__, -"MessageBeep($module, x=MB_OK, /)\n" +"MessageBeep($module, /, type=MB_OK)\n" "--\n" "\n" "Call Windows MessageBeep(x).\n" @@ -80,24 +84,26 @@ "x defaults to MB_OK."); #define WINSOUND_MESSAGEBEEP_METHODDEF \ - {"MessageBeep", (PyCFunction)winsound_MessageBeep, METH_VARARGS, winsound_MessageBeep__doc__}, + {"MessageBeep", (PyCFunction)winsound_MessageBeep, METH_VARARGS|METH_KEYWORDS, winsound_MessageBeep__doc__}, static PyObject * -winsound_MessageBeep_impl(PyObject *module, int x); +winsound_MessageBeep_impl(PyObject *module, int type); static PyObject * -winsound_MessageBeep(PyObject *module, PyObject *args) +winsound_MessageBeep(PyObject *module, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - int x = MB_OK; + static const char * const _keywords[] = {"type", NULL}; + static _PyArg_Parser _parser = {"|i:MessageBeep", _keywords, 0}; + int type = MB_OK; - if (!PyArg_ParseTuple(args, "|i:MessageBeep", - &x)) { + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &type)) { goto exit; } - return_value = winsound_MessageBeep_impl(module, x); + return_value = winsound_MessageBeep_impl(module, type); exit: return return_value; } -/*[clinic end generated code: output=b999334e2e444ad2 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=40b3d3ef2faefb15 input=a9049054013a1b77]*/ diff --git a/PC/winsound.c b/PC/winsound.c --- a/PC/winsound.c +++ b/PC/winsound.c @@ -52,7 +52,7 @@ "SND_NOWAIT - Return immediately if the sound driver is busy\n" // Without any errors "\n" "Beep(frequency, duration) - Make a beep through the PC speaker.\n" -"MessageBeep(x) - Call Windows MessageBeep."); +"MessageBeep(type) - Call Windows MessageBeep."); /*[clinic input] module winsound @@ -68,14 +68,13 @@ The sound to play; a filename, data, or None. flags: int Flag values, ored together. See module documentation. - / A wrapper around the Windows PlaySound API. [clinic start generated code]*/ static PyObject * winsound_PlaySound_impl(PyObject *module, PyObject *sound, int flags) -/*[clinic end generated code: output=49a0fd16a372ebeb input=7bdf637f10201d37]*/ +/*[clinic end generated code: output=49a0fd16a372ebeb input=c63e1f2d848da2f2]*/ { int ok; wchar_t *wsound; @@ -132,14 +131,13 @@ Must be in the range 37 through 32,767. duration: int How long the sound should play, in milliseconds. - / A wrapper around the Windows Beep API. [clinic start generated code]*/ static PyObject * winsound_Beep_impl(PyObject *module, int frequency, int duration) -/*[clinic end generated code: output=f32382e52ee9b2fb input=628a99d2ddf73798]*/ +/*[clinic end generated code: output=f32382e52ee9b2fb input=40e360cfa00a5cf0]*/ { BOOL ok; @@ -163,8 +161,7 @@ /*[clinic input] winsound.MessageBeep - x: int(c_default="MB_OK") = MB_OK - / + type: int(c_default="MB_OK") = MB_OK Call Windows MessageBeep(x). @@ -172,13 +169,13 @@ [clinic start generated code]*/ static PyObject * -winsound_MessageBeep_impl(PyObject *module, int x) -/*[clinic end generated code: output=1ad89e4d8d30a957 input=a776c8a85c9853f6]*/ +winsound_MessageBeep_impl(PyObject *module, int type) +/*[clinic end generated code: output=120875455121121f input=db185f741ae21401]*/ { BOOL ok; Py_BEGIN_ALLOW_THREADS - ok = MessageBeep(x); + ok = MessageBeep(type); Py_END_ALLOW_THREADS if (!ok) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 17:38:20 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 21:38:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_shut_up_some_perfectly_inn?= =?utf-8?q?ocent_reST_in_hashlib-blake2?= Message-ID: <20160906213819.38893.6170.C2C4EDD4@psf.io> https://hg.python.org/cpython/rev/8e721e973383 changeset: 103151:8e721e973383 user: Benjamin Peterson date: Tue Sep 06 14:37:37 2016 -0700 summary: shut up some perfectly innocent reST in hashlib-blake2 files: Doc/tools/susp-ignored.csv | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -106,6 +106,10 @@ library/functions,,:step,a[start:stop:step] library/functions,,:stop,"a[start:stop, i]" library/functions,,:stop,a[start:stop:step] +library/hashlib-blake2,,:vatrogasac,>>> cookie = b'user:vatrogasac' +library/hashlib-blake2,,:vatrogasac,"user:vatrogasac,349cf904533767ed2d755279a8df84d0" +library/hashlib-blake2,,:policajac,">>> compare_digest(b'user:policajac', sig)" +library/hashlib-blake2,,:LEAF,"h00 = blake2b(buf[0:LEAF_SIZE], fanout=FANOUT, depth=DEPTH," library/http.client,,:port,host:port library/http.cookies,,`,!#$%&'*+-.^_`|~: library/imaplib,,:MM,"""DD-Mmm-YYYY HH:MM:SS" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 17:41:49 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 21:41:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_ssl_docume?= =?utf-8?q?ntation_and_remove_merge_accident?= Message-ID: <20160906214149.12727.58303.BCB4A472@psf.io> https://hg.python.org/cpython/rev/3e18c4dda388 changeset: 103152:3e18c4dda388 branch: 2.7 parent: 103148:74805fd9e734 user: Christian Heimes date: Tue Sep 06 23:41:37 2016 +0200 summary: Fix ssl documentation and remove merge accident files: Doc/library/ssl.rst | 8 ++------ 1 files changed, 2 insertions(+), 6 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -328,15 +328,11 @@ Random generation ^^^^^^^^^^^^^^^^^ - .. deprecated:: + .. deprecated:: 2.7.13 - 2.7.13 OpenSSL has deprecated :func:`ssl.RAND_pseudo_bytes`, use + OpenSSL has deprecated :func:`ssl.RAND_pseudo_bytes`, use :func:`ssl.RAND_bytes` instead. - .. deprecated:: - - 2.7.13 OpenSSL has deprecated :func:`ssl.RAND_pseudo_bytes`, use - :func:`ssl.RAND_bytes` instead. .. function:: RAND_status() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:06:57 2016 From: python-checkins at python.org (berker.peksag) Date: Tue, 06 Sep 2016 22:06:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_redundant_bullet_po?= =?utf-8?q?int_in_3=2E6=2Erst?= Message-ID: <20160906220653.22509.42300.0D9800F4@psf.io> https://hg.python.org/cpython/rev/09ddfa957cec changeset: 103153:09ddfa957cec parent: 103151:8e721e973383 user: Berker Peksag date: Wed Sep 07 01:07:06 2016 +0300 summary: Remove redundant bullet point in 3.6.rst files: Doc/whatsnew/3.6.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -525,8 +525,8 @@ sqlite3 ------- -* :attr:`sqlite3.Cursor.lastrowid` now supports the ``REPLACE`` statement. - (Contributed by Alex LordThorsen in :issue:`16864`.) +:attr:`sqlite3.Cursor.lastrowid` now supports the ``REPLACE`` statement. +(Contributed by Alex LordThorsen in :issue:`16864`.) socket -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:09:58 2016 From: python-checkins at python.org (ned.deily) Date: Tue, 06 Sep 2016 22:09:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2321122=3A_Fix_LTO_?= =?utf-8?q?builds_on_OS_X=2E?= Message-ID: <20160906220958.3348.18084.E183CAF7@psf.io> https://hg.python.org/cpython/rev/cc5f8179a7ba changeset: 103154:cc5f8179a7ba user: Ned Deily date: Tue Sep 06 15:09:20 2016 -0700 summary: Issue #21122: Fix LTO builds on OS X. Patch by Brett Cannon. files: Misc/NEWS | 2 ++ configure | 27 +++++++++++---------------- configure.ac | 13 ++++++++++--- 3 files changed, 23 insertions(+), 19 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -212,6 +212,8 @@ - Update OS X installer to use SQLite 3.14.1 and XZ 5.2.2. +- Issue #21122: Fix LTO builds on OS X. + Windows ------- diff --git a/configure b/configure --- a/configure +++ b/configure @@ -775,7 +775,6 @@ docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir @@ -886,7 +885,6 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1139,15 +1137,6 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1285,7 +1274,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1438,7 +1427,6 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -6484,13 +6472,20 @@ if test "$Py_LTO" = 'true' ; then case $CC in *clang*) - # Any changes made here should be reflected in the GCC+Darwin case below - LTOFLAGS="-flto" + case $ac_sys_system in + Darwin*) + # Any changes made here should be reflected in the GCC+Darwin case below + LTOFLAGS="-flto -Wl,-export_dynamic" + ;; + *) + LTOFLAGS="-flto" + ;; + esac ;; *gcc*) case $ac_sys_system in Darwin*) - LTOFLAGS="-flto" + LTOFLAGS="-flto -Wl,-export_dynamic" ;; *) LTOFLAGS="-flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1299,13 +1299,20 @@ if test "$Py_LTO" = 'true' ; then case $CC in *clang*) - # Any changes made here should be reflected in the GCC+Darwin case below - LTOFLAGS="-flto" + case $ac_sys_system in + Darwin*) + # Any changes made here should be reflected in the GCC+Darwin case below + LTOFLAGS="-flto -Wl,-export_dynamic" + ;; + *) + LTOFLAGS="-flto" + ;; + esac ;; *gcc*) case $ac_sys_system in Darwin*) - LTOFLAGS="-flto" + LTOFLAGS="-flto -Wl,-export_dynamic" ;; *) LTOFLAGS="-flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:10:20 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 22:10:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326798=3A_for_loop?= =?utf-8?q?_initial_declarations=2C_take_2?= Message-ID: <20160906221020.31332.23773.B9BAF40C@psf.io> https://hg.python.org/cpython/rev/5c31599de76a changeset: 103155:5c31599de76a user: Christian Heimes date: Wed Sep 07 00:09:22 2016 +0200 summary: Issue #26798: for loop initial declarations, take 2 files: Modules/_blake2/impl/blake2b-ref.c | 10 ++++++---- Modules/_blake2/impl/blake2b.c | 7 ++++--- Modules/_blake2/impl/blake2s-ref.c | 16 +++++++++------- Modules/_blake2/impl/blake2s.c | 7 ++++--- 4 files changed, 23 insertions(+), 17 deletions(-) diff --git a/Modules/_blake2/impl/blake2b-ref.c b/Modules/_blake2/impl/blake2b-ref.c --- a/Modules/_blake2/impl/blake2b-ref.c +++ b/Modules/_blake2/impl/blake2b-ref.c @@ -157,11 +157,12 @@ int blake2b_init_param( blake2b_state *S, const blake2b_param *P ) { const uint8_t *p = ( const uint8_t * )( P ); + size_t i; blake2b_init0( S ); /* IV XOR ParamBlock */ - for( size_t i = 0; i < 8; ++i ) + for( i = 0; i < 8; ++i ) S->h[i] ^= load64( p + sizeof( S->h[i] ) * i ); return 0; @@ -392,14 +393,15 @@ { uint8_t key[BLAKE2B_KEYBYTES]; uint8_t buf[KAT_LENGTH]; + size_t i; - for( size_t i = 0; i < BLAKE2B_KEYBYTES; ++i ) + for( i = 0; i < BLAKE2B_KEYBYTES; ++i ) key[i] = ( uint8_t )i; - for( size_t i = 0; i < KAT_LENGTH; ++i ) + for( i = 0; i < KAT_LENGTH; ++i ) buf[i] = ( uint8_t )i; - for( size_t i = 0; i < KAT_LENGTH; ++i ) + for( i = 0; i < KAT_LENGTH; ++i ) { uint8_t hash[BLAKE2B_OUTBYTES]; blake2b( hash, buf, key, BLAKE2B_OUTBYTES, i, BLAKE2B_KEYBYTES ); diff --git a/Modules/_blake2/impl/blake2b.c b/Modules/_blake2/impl/blake2b.c --- a/Modules/_blake2/impl/blake2b.c +++ b/Modules/_blake2/impl/blake2b.c @@ -426,14 +426,15 @@ { uint8_t key[BLAKE2B_KEYBYTES]; uint8_t buf[KAT_LENGTH]; + size_t i; - for( size_t i = 0; i < BLAKE2B_KEYBYTES; ++i ) + for( i = 0; i < BLAKE2B_KEYBYTES; ++i ) key[i] = ( uint8_t )i; - for( size_t i = 0; i < KAT_LENGTH; ++i ) + for( i = 0; i < KAT_LENGTH; ++i ) buf[i] = ( uint8_t )i; - for( size_t i = 0; i < KAT_LENGTH; ++i ) + for( i = 0; i < KAT_LENGTH; ++i ) { uint8_t hash[BLAKE2B_OUTBYTES]; blake2b( hash, buf, key, BLAKE2B_OUTBYTES, i, BLAKE2B_KEYBYTES ); diff --git a/Modules/_blake2/impl/blake2s-ref.c b/Modules/_blake2/impl/blake2s-ref.c --- a/Modules/_blake2/impl/blake2s-ref.c +++ b/Modules/_blake2/impl/blake2s-ref.c @@ -154,7 +154,7 @@ blake2s_init0( S ); /* IV XOR ParamBlock */ - for( size_t i = 0; i < 8; ++i ) + for( i = 0; i < 8; ++i ) S->h[i] ^= load32( &p[i] ); return 0; @@ -219,11 +219,12 @@ { uint32_t m[16]; uint32_t v[16]; + size_t i; - for( size_t i = 0; i < 16; ++i ) + for( i = 0; i < 16; ++i ) m[i] = load32( block + i * sizeof( m[i] ) ); - for( size_t i = 0; i < 8; ++i ) + for( i = 0; i < 8; ++i ) v[i] = S->h[i]; v[ 8] = blake2s_IV[0]; @@ -267,7 +268,7 @@ ROUND( 8 ); ROUND( 9 ); - for( size_t i = 0; i < 8; ++i ) + for( i = 0; i < 8; ++i ) S->h[i] = S->h[i] ^ v[i] ^ v[i + 8]; #undef G @@ -381,14 +382,15 @@ { uint8_t key[BLAKE2S_KEYBYTES]; uint8_t buf[KAT_LENGTH]; + size_t i; - for( size_t i = 0; i < BLAKE2S_KEYBYTES; ++i ) + for( i = 0; i < BLAKE2S_KEYBYTES; ++i ) key[i] = ( uint8_t )i; - for( size_t i = 0; i < KAT_LENGTH; ++i ) + for( i = 0; i < KAT_LENGTH; ++i ) buf[i] = ( uint8_t )i; - for( size_t i = 0; i < KAT_LENGTH; ++i ) + for( i = 0; i < KAT_LENGTH; ++i ) { uint8_t hash[BLAKE2S_OUTBYTES]; blake2s( hash, buf, key, BLAKE2S_OUTBYTES, i, BLAKE2S_KEYBYTES ); diff --git a/Modules/_blake2/impl/blake2s.c b/Modules/_blake2/impl/blake2s.c --- a/Modules/_blake2/impl/blake2s.c +++ b/Modules/_blake2/impl/blake2s.c @@ -407,14 +407,15 @@ { uint8_t key[BLAKE2S_KEYBYTES]; uint8_t buf[KAT_LENGTH]; + size_t i; - for( size_t i = 0; i < BLAKE2S_KEYBYTES; ++i ) + for( i = 0; i < BLAKE2S_KEYBYTES; ++i ) key[i] = ( uint8_t )i; - for( size_t i = 0; i < KAT_LENGTH; ++i ) + for( i = 0; i < KAT_LENGTH; ++i ) buf[i] = ( uint8_t )i; - for( size_t i = 0; i < KAT_LENGTH; ++i ) + for( i = 0; i < KAT_LENGTH; ++i ) { uint8_t hash[BLAKE2S_OUTBYTES]; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:32:13 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 22:32:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326798=3A_for_loop?= =?utf-8?q?_initial_declarations=2C_take_3?= Message-ID: <20160906223212.38962.3132.B13C7027@psf.io> https://hg.python.org/cpython/rev/a1e032dbcf86 changeset: 103156:a1e032dbcf86 user: Christian Heimes date: Wed Sep 07 00:32:06 2016 +0200 summary: Issue #26798: for loop initial declarations, take 3 files: Modules/_blake2/impl/blake2s-ref.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Modules/_blake2/impl/blake2s-ref.c b/Modules/_blake2/impl/blake2s-ref.c --- a/Modules/_blake2/impl/blake2s-ref.c +++ b/Modules/_blake2/impl/blake2s-ref.c @@ -150,6 +150,7 @@ int blake2s_init_param( blake2s_state *S, const blake2s_param *P ) { const uint32_t *p = ( const uint32_t * )( P ); + size_t i; blake2s_init0( S ); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:50:45 2016 From: python-checkins at python.org (brett.cannon) Date: Tue, 06 Sep 2016 22:50:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326027=3A_Support_?= =?utf-8?q?path-like_objects_in_PyUnicode-FSConverter=28=29=2E?= Message-ID: <20160906225045.3577.38766.C8235F97@psf.io> https://hg.python.org/cpython/rev/d0d9d7f55cb5 changeset: 103157:d0d9d7f55cb5 user: Brett Cannon date: Tue Sep 06 15:50:29 2016 -0700 summary: Issue #26027: Support path-like objects in PyUnicode-FSConverter(). This is to add support for os.exec*() and os.spawn*() functions. Part of PEP 519. files: Doc/c-api/unicode.rst | 5 +- Lib/test/test_os.py | 64 ++++++++++++---------------- Misc/NEWS | 5 ++ Objects/unicodeobject.c | 27 ++++++----- 4 files changed, 51 insertions(+), 50 deletions(-) diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -810,13 +810,16 @@ .. c:function:: int PyUnicode_FSConverter(PyObject* obj, void* result) - ParseTuple converter: encode :class:`str` objects to :class:`bytes` using + ParseTuple converter: encode :class:`str` objects -- obtained directly or + through the :class:`os.PathLike` interface -- to :class:`bytes` using :c:func:`PyUnicode_EncodeFSDefault`; :class:`bytes` objects are output as-is. *result* must be a :c:type:`PyBytesObject*` which must be released when it is no longer used. .. versionadded:: 3.1 + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. To decode file names during argument parsing, the ``"O&"`` converter should be used, passing :c:func:`PyUnicode_FSDecoder` as the conversion function: diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -100,6 +100,21 @@ yield +class _PathLike(os.PathLike): + + def __init__(self, path=""): + self.path = path + + def __str__(self): + return str(self.path) + + def __fspath__(self): + if isinstance(self.path, BaseException): + raise self.path + else: + return self.path + + def create_file(filename, content=b'content'): with open(filename, "xb", 0) as fp: fp.write(content) @@ -894,15 +909,7 @@ self.assertEqual(all[1], self.sub2_tree) def test_file_like_path(self): - class FileLike: - def __init__(self, path): - self._path = path - def __str__(self): - return str(self._path) - def __fspath__(self): - return self._path - - self.test_walk_prune(FileLike(self.walk_path)) + self.test_walk_prune(_PathLike(self.walk_path)) def test_walk_bottom_up(self): # Walk bottom-up. @@ -2124,7 +2131,8 @@ def test_waitpid(self): args = [sys.executable, '-c', 'pass'] - pid = os.spawnv(os.P_NOWAIT, args[0], args) + # Add an implicit test for PyUnicode_FSConverter(). + pid = os.spawnv(os.P_NOWAIT, _PathLike(args[0]), args) status = os.waitpid(pid, 0) self.assertEqual(status, (pid, 0)) @@ -2833,25 +2841,18 @@ ] def test_path_t_converter(self): - class PathLike: - def __init__(self, path): - self.path = path - - def __fspath__(self): - return self.path - str_filename = support.TESTFN if os.name == 'nt': bytes_fspath = bytes_filename = None else: bytes_filename = support.TESTFN.encode('ascii') - bytes_fspath = PathLike(bytes_filename) - fd = os.open(PathLike(str_filename), os.O_WRONLY|os.O_CREAT) + bytes_fspath = _PathLike(bytes_filename) + fd = os.open(_PathLike(str_filename), os.O_WRONLY|os.O_CREAT) self.addCleanup(support.unlink, support.TESTFN) self.addCleanup(os.close, fd) - int_fspath = PathLike(fd) - str_fspath = PathLike(str_filename) + int_fspath = _PathLike(fd) + str_fspath = _PathLike(str_filename) for name, allow_fd, extra_args, cleanup_fn in self.functions: with self.subTest(name=name): @@ -3205,15 +3206,6 @@ # if a C version is provided. fspath = staticmethod(os.fspath) - class PathLike: - def __init__(self, path=''): - self.path = path - def __fspath__(self): - if isinstance(self.path, BaseException): - raise self.path - else: - return self.path - def test_return_bytes(self): for b in b'hello', b'goodbye', b'some/path/and/file': self.assertEqual(b, self.fspath(b)) @@ -3224,16 +3216,16 @@ def test_fsencode_fsdecode(self): for p in "path/like/object", b"path/like/object": - pathlike = self.PathLike(p) + pathlike = _PathLike(p) self.assertEqual(p, self.fspath(pathlike)) self.assertEqual(b"path/like/object", os.fsencode(pathlike)) self.assertEqual("path/like/object", os.fsdecode(pathlike)) def test_pathlike(self): - self.assertEqual('#feelthegil', self.fspath(self.PathLike('#feelthegil'))) - self.assertTrue(issubclass(self.PathLike, os.PathLike)) - self.assertTrue(isinstance(self.PathLike(), os.PathLike)) + self.assertEqual('#feelthegil', self.fspath(_PathLike('#feelthegil'))) + self.assertTrue(issubclass(_PathLike, os.PathLike)) + self.assertTrue(isinstance(_PathLike(), os.PathLike)) def test_garbage_in_exception_out(self): vapor = type('blah', (), {}) @@ -3245,14 +3237,14 @@ def test_bad_pathlike(self): # __fspath__ returns a value other than str or bytes. - self.assertRaises(TypeError, self.fspath, self.PathLike(42)) + self.assertRaises(TypeError, self.fspath, _PathLike(42)) # __fspath__ attribute that is not callable. c = type('foo', (), {}) c.__fspath__ = 1 self.assertRaises(TypeError, self.fspath, c()) # __fspath__ raises an exception. self.assertRaises(ZeroDivisionError, self.fspath, - self.PathLike(ZeroDivisionError())) + _PathLike(ZeroDivisionError())) # Only test if the C version is provided, otherwise TestPEP519 already tested # the pure Python implementation. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -188,6 +188,11 @@ - Issue #27573: exit message for code.interact is now configurable. +C API +----- + +- Issue #26027: Add support for path-like objects in PyUnicode_FSConverter(). + Tests ----- diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3842,6 +3842,7 @@ int PyUnicode_FSConverter(PyObject* arg, void* addr) { + PyObject *path = NULL; PyObject *output = NULL; Py_ssize_t size; void *data; @@ -3850,22 +3851,22 @@ *(PyObject**)addr = NULL; return 1; } - if (PyBytes_Check(arg)) { - output = arg; - Py_INCREF(output); - } - else if (PyUnicode_Check(arg)) { - output = PyUnicode_EncodeFSDefault(arg); - if (!output) + path = PyOS_FSPath(arg); + if (path == NULL) { + return 0; + } + if (PyBytes_Check(path)) { + output = path; + } + else { // PyOS_FSPath() guarantees its returned value is bytes or str. + output = PyUnicode_EncodeFSDefault(path); + Py_DECREF(path); + if (!output) { return 0; + } assert(PyBytes_Check(output)); } - else { - PyErr_Format(PyExc_TypeError, - "must be str or bytes, not %.100s", - Py_TYPE(arg)->tp_name); - return 0; - } + size = PyBytes_GET_SIZE(output); data = PyBytes_AS_STRING(output); if ((size_t)size != strlen(data)) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:55:46 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 06 Sep 2016 22:55:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Run_Argument_Clinic_on_pos?= =?utf-8?q?ixmodule=2Ec?= Message-ID: <20160906225546.13951.26667.184D5E32@psf.io> https://hg.python.org/cpython/rev/1d29d3a1e0c6 changeset: 103158:1d29d3a1e0c6 user: Victor Stinner date: Tue Sep 06 15:54:20 2016 -0700 summary: Run Argument Clinic on posixmodule.c Issue #17884. files: Modules/clinic/posixmodule.c.h | 14 +++++++------- Modules/posixmodule.c | 8 ++++---- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -3011,13 +3011,13 @@ {"waitpid", (PyCFunction)os_waitpid, METH_VARARGS, os_waitpid__doc__}, static PyObject * -os_waitpid_impl(PyObject *module, Py_intptr_t pid, int options); +os_waitpid_impl(PyObject *module, intptr_t pid, int options); static PyObject * os_waitpid(PyObject *module, PyObject *args) { PyObject *return_value = NULL; - Py_intptr_t pid; + intptr_t pid; int options; if (!PyArg_ParseTuple(args, "" _Py_PARSE_INTPTR "i:waitpid", @@ -5479,13 +5479,13 @@ {"get_handle_inheritable", (PyCFunction)os_get_handle_inheritable, METH_O, os_get_handle_inheritable__doc__}, static int -os_get_handle_inheritable_impl(PyObject *module, Py_intptr_t handle); +os_get_handle_inheritable_impl(PyObject *module, intptr_t handle); static PyObject * os_get_handle_inheritable(PyObject *module, PyObject *arg) { PyObject *return_value = NULL; - Py_intptr_t handle; + intptr_t handle; int _return_value; if (!PyArg_Parse(arg, "" _Py_PARSE_INTPTR ":get_handle_inheritable", &handle)) { @@ -5515,14 +5515,14 @@ {"set_handle_inheritable", (PyCFunction)os_set_handle_inheritable, METH_VARARGS, os_set_handle_inheritable__doc__}, static PyObject * -os_set_handle_inheritable_impl(PyObject *module, Py_intptr_t handle, +os_set_handle_inheritable_impl(PyObject *module, intptr_t handle, int inheritable); static PyObject * os_set_handle_inheritable(PyObject *module, PyObject *args) { PyObject *return_value = NULL; - Py_intptr_t handle; + intptr_t handle; int inheritable; if (!PyArg_ParseTuple(args, "" _Py_PARSE_INTPTR "p:set_handle_inheritable", @@ -6042,4 +6042,4 @@ #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ -/*[clinic end generated code: output=2b85bb3703a6488a input=a9049054013a1b77]*/ +/*[clinic end generated code: output=677ce794fb126161 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -2520,7 +2520,7 @@ impl_by_reference = True; [python start generated code]*/ -/*[python end generated code: output=da39a3ee5e6b4b0d input=affe68316f160401]*/ +/*[python end generated code: output=da39a3ee5e6b4b0d input=418fce0e01144461]*/ /*[clinic input] @@ -7092,7 +7092,7 @@ static PyObject * os_waitpid_impl(PyObject *module, intptr_t pid, int options) -/*[clinic end generated code: output=15f1ce005a346b09 input=444c8f51cca5b862]*/ +/*[clinic end generated code: output=be836b221271d538 input=40f2440c515410f8]*/ { int status; intptr_t res; @@ -11383,7 +11383,7 @@ static int os_get_handle_inheritable_impl(PyObject *module, intptr_t handle) -/*[clinic end generated code: output=9e5389b0aa0916ce input=5f7759443aae3dc5]*/ +/*[clinic end generated code: output=36be5afca6ea84d8 input=cfe99f9c05c70ad1]*/ { DWORD flags; @@ -11408,7 +11408,7 @@ static PyObject * os_set_handle_inheritable_impl(PyObject *module, intptr_t handle, int inheritable) -/*[clinic end generated code: output=b1e67bfa3213d745 input=e64b2b2730469def]*/ +/*[clinic end generated code: output=021d74fe6c96baa3 input=7a7641390d8364fc]*/ { DWORD flags = inheritable ? HANDLE_FLAG_INHERIT : 0; if (!SetHandleInformation((HANDLE)handle, HANDLE_FLAG_INHERIT, flags)) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:57:59 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 22:57:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160906225758.22677.78526.9E60538C@psf.io> https://hg.python.org/cpython/rev/42673b6fc6c5 changeset: 103160:42673b6fc6c5 parent: 103156:a1e032dbcf86 parent: 103159:c90f8c994048 user: Benjamin Peterson date: Tue Sep 06 15:54:38 2016 -0700 summary: merge 3.5 files: configure | 16 +++++++++++++++- configure.ac | 2 ++ 2 files changed, 17 insertions(+), 1 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -775,6 +775,7 @@ docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -885,6 +886,7 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1137,6 +1139,15 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1274,7 +1285,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1427,6 +1438,7 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -12372,6 +12384,8 @@ break; } } + freeaddrinfo(aitop); + aitop = NULL; } if (!(inet4 == 0 || inet4 == 2)) diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3718,6 +3718,8 @@ break; } } + freeaddrinfo(aitop); + aitop = NULL; } if (!(inet4 == 0 || inet4 == 2)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:57:59 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 22:57:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <20160906225759.39004.5967.269DF060@psf.io> https://hg.python.org/cpython/rev/a882c1981029 changeset: 103162:a882c1981029 parent: 103160:42673b6fc6c5 parent: 103158:1d29d3a1e0c6 user: Benjamin Peterson date: Tue Sep 06 15:57:48 2016 -0700 summary: merge heads files: Doc/c-api/unicode.rst | 5 +- Lib/test/test_os.py | 64 +++++++++------------ Misc/NEWS | 5 + Modules/clinic/posixmodule.c.h | 14 ++-- Modules/posixmodule.c | 8 +- Objects/unicodeobject.c | 27 ++++---- 6 files changed, 62 insertions(+), 61 deletions(-) diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -810,13 +810,16 @@ .. c:function:: int PyUnicode_FSConverter(PyObject* obj, void* result) - ParseTuple converter: encode :class:`str` objects to :class:`bytes` using + ParseTuple converter: encode :class:`str` objects -- obtained directly or + through the :class:`os.PathLike` interface -- to :class:`bytes` using :c:func:`PyUnicode_EncodeFSDefault`; :class:`bytes` objects are output as-is. *result* must be a :c:type:`PyBytesObject*` which must be released when it is no longer used. .. versionadded:: 3.1 + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. To decode file names during argument parsing, the ``"O&"`` converter should be used, passing :c:func:`PyUnicode_FSDecoder` as the conversion function: diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -100,6 +100,21 @@ yield +class _PathLike(os.PathLike): + + def __init__(self, path=""): + self.path = path + + def __str__(self): + return str(self.path) + + def __fspath__(self): + if isinstance(self.path, BaseException): + raise self.path + else: + return self.path + + def create_file(filename, content=b'content'): with open(filename, "xb", 0) as fp: fp.write(content) @@ -894,15 +909,7 @@ self.assertEqual(all[1], self.sub2_tree) def test_file_like_path(self): - class FileLike: - def __init__(self, path): - self._path = path - def __str__(self): - return str(self._path) - def __fspath__(self): - return self._path - - self.test_walk_prune(FileLike(self.walk_path)) + self.test_walk_prune(_PathLike(self.walk_path)) def test_walk_bottom_up(self): # Walk bottom-up. @@ -2124,7 +2131,8 @@ def test_waitpid(self): args = [sys.executable, '-c', 'pass'] - pid = os.spawnv(os.P_NOWAIT, args[0], args) + # Add an implicit test for PyUnicode_FSConverter(). + pid = os.spawnv(os.P_NOWAIT, _PathLike(args[0]), args) status = os.waitpid(pid, 0) self.assertEqual(status, (pid, 0)) @@ -2833,25 +2841,18 @@ ] def test_path_t_converter(self): - class PathLike: - def __init__(self, path): - self.path = path - - def __fspath__(self): - return self.path - str_filename = support.TESTFN if os.name == 'nt': bytes_fspath = bytes_filename = None else: bytes_filename = support.TESTFN.encode('ascii') - bytes_fspath = PathLike(bytes_filename) - fd = os.open(PathLike(str_filename), os.O_WRONLY|os.O_CREAT) + bytes_fspath = _PathLike(bytes_filename) + fd = os.open(_PathLike(str_filename), os.O_WRONLY|os.O_CREAT) self.addCleanup(support.unlink, support.TESTFN) self.addCleanup(os.close, fd) - int_fspath = PathLike(fd) - str_fspath = PathLike(str_filename) + int_fspath = _PathLike(fd) + str_fspath = _PathLike(str_filename) for name, allow_fd, extra_args, cleanup_fn in self.functions: with self.subTest(name=name): @@ -3205,15 +3206,6 @@ # if a C version is provided. fspath = staticmethod(os.fspath) - class PathLike: - def __init__(self, path=''): - self.path = path - def __fspath__(self): - if isinstance(self.path, BaseException): - raise self.path - else: - return self.path - def test_return_bytes(self): for b in b'hello', b'goodbye', b'some/path/and/file': self.assertEqual(b, self.fspath(b)) @@ -3224,16 +3216,16 @@ def test_fsencode_fsdecode(self): for p in "path/like/object", b"path/like/object": - pathlike = self.PathLike(p) + pathlike = _PathLike(p) self.assertEqual(p, self.fspath(pathlike)) self.assertEqual(b"path/like/object", os.fsencode(pathlike)) self.assertEqual("path/like/object", os.fsdecode(pathlike)) def test_pathlike(self): - self.assertEqual('#feelthegil', self.fspath(self.PathLike('#feelthegil'))) - self.assertTrue(issubclass(self.PathLike, os.PathLike)) - self.assertTrue(isinstance(self.PathLike(), os.PathLike)) + self.assertEqual('#feelthegil', self.fspath(_PathLike('#feelthegil'))) + self.assertTrue(issubclass(_PathLike, os.PathLike)) + self.assertTrue(isinstance(_PathLike(), os.PathLike)) def test_garbage_in_exception_out(self): vapor = type('blah', (), {}) @@ -3245,14 +3237,14 @@ def test_bad_pathlike(self): # __fspath__ returns a value other than str or bytes. - self.assertRaises(TypeError, self.fspath, self.PathLike(42)) + self.assertRaises(TypeError, self.fspath, _PathLike(42)) # __fspath__ attribute that is not callable. c = type('foo', (), {}) c.__fspath__ = 1 self.assertRaises(TypeError, self.fspath, c()) # __fspath__ raises an exception. self.assertRaises(ZeroDivisionError, self.fspath, - self.PathLike(ZeroDivisionError())) + _PathLike(ZeroDivisionError())) # Only test if the C version is provided, otherwise TestPEP519 already tested # the pure Python implementation. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -188,6 +188,11 @@ - Issue #27573: exit message for code.interact is now configurable. +C API +----- + +- Issue #26027: Add support for path-like objects in PyUnicode_FSConverter(). + Tests ----- diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -3011,13 +3011,13 @@ {"waitpid", (PyCFunction)os_waitpid, METH_VARARGS, os_waitpid__doc__}, static PyObject * -os_waitpid_impl(PyObject *module, Py_intptr_t pid, int options); +os_waitpid_impl(PyObject *module, intptr_t pid, int options); static PyObject * os_waitpid(PyObject *module, PyObject *args) { PyObject *return_value = NULL; - Py_intptr_t pid; + intptr_t pid; int options; if (!PyArg_ParseTuple(args, "" _Py_PARSE_INTPTR "i:waitpid", @@ -5479,13 +5479,13 @@ {"get_handle_inheritable", (PyCFunction)os_get_handle_inheritable, METH_O, os_get_handle_inheritable__doc__}, static int -os_get_handle_inheritable_impl(PyObject *module, Py_intptr_t handle); +os_get_handle_inheritable_impl(PyObject *module, intptr_t handle); static PyObject * os_get_handle_inheritable(PyObject *module, PyObject *arg) { PyObject *return_value = NULL; - Py_intptr_t handle; + intptr_t handle; int _return_value; if (!PyArg_Parse(arg, "" _Py_PARSE_INTPTR ":get_handle_inheritable", &handle)) { @@ -5515,14 +5515,14 @@ {"set_handle_inheritable", (PyCFunction)os_set_handle_inheritable, METH_VARARGS, os_set_handle_inheritable__doc__}, static PyObject * -os_set_handle_inheritable_impl(PyObject *module, Py_intptr_t handle, +os_set_handle_inheritable_impl(PyObject *module, intptr_t handle, int inheritable); static PyObject * os_set_handle_inheritable(PyObject *module, PyObject *args) { PyObject *return_value = NULL; - Py_intptr_t handle; + intptr_t handle; int inheritable; if (!PyArg_ParseTuple(args, "" _Py_PARSE_INTPTR "p:set_handle_inheritable", @@ -6042,4 +6042,4 @@ #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ -/*[clinic end generated code: output=2b85bb3703a6488a input=a9049054013a1b77]*/ +/*[clinic end generated code: output=677ce794fb126161 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -2520,7 +2520,7 @@ impl_by_reference = True; [python start generated code]*/ -/*[python end generated code: output=da39a3ee5e6b4b0d input=affe68316f160401]*/ +/*[python end generated code: output=da39a3ee5e6b4b0d input=418fce0e01144461]*/ /*[clinic input] @@ -7092,7 +7092,7 @@ static PyObject * os_waitpid_impl(PyObject *module, intptr_t pid, int options) -/*[clinic end generated code: output=15f1ce005a346b09 input=444c8f51cca5b862]*/ +/*[clinic end generated code: output=be836b221271d538 input=40f2440c515410f8]*/ { int status; intptr_t res; @@ -11383,7 +11383,7 @@ static int os_get_handle_inheritable_impl(PyObject *module, intptr_t handle) -/*[clinic end generated code: output=9e5389b0aa0916ce input=5f7759443aae3dc5]*/ +/*[clinic end generated code: output=36be5afca6ea84d8 input=cfe99f9c05c70ad1]*/ { DWORD flags; @@ -11408,7 +11408,7 @@ static PyObject * os_set_handle_inheritable_impl(PyObject *module, intptr_t handle, int inheritable) -/*[clinic end generated code: output=b1e67bfa3213d745 input=e64b2b2730469def]*/ +/*[clinic end generated code: output=021d74fe6c96baa3 input=7a7641390d8364fc]*/ { DWORD flags = inheritable ? HANDLE_FLAG_INHERIT : 0; if (!SetHandleInformation((HANDLE)handle, HANDLE_FLAG_INHERIT, flags)) { diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3842,6 +3842,7 @@ int PyUnicode_FSConverter(PyObject* arg, void* addr) { + PyObject *path = NULL; PyObject *output = NULL; Py_ssize_t size; void *data; @@ -3850,22 +3851,22 @@ *(PyObject**)addr = NULL; return 1; } - if (PyBytes_Check(arg)) { - output = arg; - Py_INCREF(output); - } - else if (PyUnicode_Check(arg)) { - output = PyUnicode_EncodeFSDefault(arg); - if (!output) + path = PyOS_FSPath(arg); + if (path == NULL) { + return 0; + } + if (PyBytes_Check(path)) { + output = path; + } + else { // PyOS_FSPath() guarantees its returned value is bytes or str. + output = PyUnicode_EncodeFSDefault(path); + Py_DECREF(path); + if (!output) { return 0; + } assert(PyBytes_Check(output)); } - else { - PyErr_Format(PyExc_TypeError, - "must be str or bytes, not %.100s", - Py_TYPE(arg)->tp_name); - return 0; - } + size = PyBytes_GET_SIZE(output); data = PyBytes_AS_STRING(output); if ((size_t)size != strlen(data)) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:57:59 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 22:57:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_do_not_leak_ad?= =?utf-8?q?drinfo_in_configure_test?= Message-ID: <20160906225758.14547.60890.31DD9AF2@psf.io> https://hg.python.org/cpython/rev/fecbb1fa241f changeset: 103161:fecbb1fa241f branch: 2.7 parent: 103152:3e18c4dda388 user: Benjamin Peterson date: Tue Sep 06 15:54:24 2016 -0700 summary: do not leak addrinfo in configure test files: configure | 2 ++ configure.ac | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -11728,6 +11728,8 @@ break; } } + freeaddrinfo(aitop); + aitop = NULL; } if (!(inet4 == 0 || inet4 == 2)) diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3422,6 +3422,8 @@ break; } } + freeaddrinfo(aitop); + aitop = NULL; } if (!(inet4 == 0 || inet4 == 2)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:57:59 2016 From: python-checkins at python.org (benjamin.peterson) Date: Tue, 06 Sep 2016 22:57:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_do_not_leak_ad?= =?utf-8?q?drinfo_in_configure_test?= Message-ID: <20160906225758.78083.87608.B08513C6@psf.io> https://hg.python.org/cpython/rev/c90f8c994048 changeset: 103159:c90f8c994048 branch: 3.5 parent: 103146:9bbf0b31da48 user: Benjamin Peterson date: Tue Sep 06 15:54:24 2016 -0700 summary: do not leak addrinfo in configure test files: configure | 2 ++ configure.ac | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -12581,6 +12581,8 @@ break; } } + freeaddrinfo(aitop); + aitop = NULL; } if (!(inet4 == 0 || inet4 == 2)) diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3700,6 +3700,8 @@ break; } } + freeaddrinfo(aitop); + aitop = NULL; } if (!(inet4 == 0 || inet4 == 2)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:59:09 2016 From: python-checkins at python.org (brett.cannon) Date: Tue, 06 Sep 2016 22:59:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326027=2C_=2327524?= =?utf-8?q?=3A_Document_the_support_for_path-like_objects_in_os_and?= Message-ID: <20160906225909.31352.26474.131F8290@psf.io> https://hg.python.org/cpython/rev/9be0286772bf changeset: 103163:9be0286772bf parent: 103157:d0d9d7f55cb5 user: Brett Cannon date: Tue Sep 06 15:55:02 2016 -0700 summary: Issue #26027, #27524: Document the support for path-like objects in os and os.path. This completes PEP 519. files: Doc/library/functions.rst | 10 +- Doc/library/os.path.rst | 87 ++++++++++++++- Doc/library/os.rst | 158 +++++++++++++++++++++++-- 3 files changed, 233 insertions(+), 22 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -878,11 +878,11 @@ Open *file* and return a corresponding :term:`file object`. If the file cannot be opened, an :exc:`OSError` is raised. - *file* is either a string, bytes, or :class:`os.PathLike` object giving the - pathname (absolute or relative to the current working directory) of the file - to be opened or an integer file descriptor of the file to be wrapped. (If a - file descriptor is given, it is closed when the returned I/O object is - closed, unless *closefd* is set to ``False``.) + *file* is a :term:`path-like object` giving the pathname (absolute or + relative to the current working directory) of the file to be opened or an + integer file descriptor of the file to be wrapped. (If a file descriptor is + given, it is closed when the returned I/O object is closed, unless *closefd* + is set to ``False``.) *mode* is an optional string that specifies the mode in which the file is opened. It defaults to ``'r'`` which means open for reading in text mode. diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst --- a/Doc/library/os.path.rst +++ b/Doc/library/os.path.rst @@ -61,6 +61,9 @@ platforms, this is equivalent to calling the function :func:`normpath` as follows: ``normpath(join(os.getcwd(), path))``. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: basename(path) @@ -71,6 +74,9 @@ ``'/foo/bar/'`` returns ``'bar'``, the :func:`basename` function returns an empty string (``''``). + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: commonpath(paths) @@ -83,6 +89,9 @@ .. versionadded:: 3.5 + .. versionchanged:: 3.6 + Accepts a sequence of :term:`path-like objects `. + .. function:: commonprefix(list) @@ -104,12 +113,18 @@ >>> os.path.commonpath(['/usr/lib', '/usr/local/lib']) '/usr' + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: dirname(path) Return the directory name of pathname *path*. This is the first element of the pair returned by passing *path* to the function :func:`split`. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: exists(path) @@ -123,6 +138,9 @@ *path* can now be an integer: ``True`` is returned if it is an open file descriptor, ``False`` otherwise. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: lexists(path) @@ -130,6 +148,9 @@ broken symbolic links. Equivalent to :func:`exists` on platforms lacking :func:`os.lstat`. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: expanduser(path) @@ -151,6 +172,9 @@ If the expansion fails or if the path does not begin with a tilde, the path is returned unchanged. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: expandvars(path) @@ -162,6 +186,9 @@ On Windows, ``%name%`` expansions are supported in addition to ``$name`` and ``${name}``. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: getatime(path) @@ -182,6 +209,9 @@ If :func:`os.stat_float_times` returns ``True``, the result is a floating point number. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: getctime(path) @@ -191,12 +221,18 @@ the :mod:`time` module). Raise :exc:`OSError` if the file does not exist or is inaccessible. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: getsize(path) Return the size, in bytes, of *path*. Raise :exc:`OSError` if the file does not exist or is inaccessible. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: isabs(path) @@ -204,24 +240,36 @@ begins with a slash, on Windows that it begins with a (back)slash after chopping off a potential drive letter. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: isfile(path) Return ``True`` if *path* is an existing regular file. This follows symbolic links, so both :func:`islink` and :func:`isfile` can be true for the same path. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: isdir(path) Return ``True`` if *path* is an existing directory. This follows symbolic links, so both :func:`islink` and :func:`isdir` can be true for the same path. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: islink(path) Return ``True`` if *path* refers to a directory entry that is a symbolic link. Always ``False`` if symbolic links are not supported by the Python runtime. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: ismount(path) @@ -237,6 +285,9 @@ .. versionadded:: 3.4 Support for detecting non-root mount points on Windows. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: join(path, *paths) @@ -255,13 +306,20 @@ ``os.path.join("c:", "foo")`` represents a path relative to the current directory on drive :file:`C:` (:file:`c:foo`), not :file:`c:\\foo`. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` for *path* and *paths*. + .. function:: normcase(path) Normalize the case of a pathname. On Unix and Mac OS X, this returns the path unchanged; on case-insensitive filesystems, it converts the path to lowercase. On Windows, it also converts forward slashes to backward slashes. - Raise a TypeError if the type of *path* is not ``str`` or ``bytes``. + Raise a TypeError if the type of *path* is not ``str`` or ``bytes`` (directly + or indirectly through the :class:`os.PathLike` interface). + + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. .. function:: normpath(path) @@ -272,12 +330,18 @@ that contains symbolic links. On Windows, it converts forward slashes to backward slashes. To normalize case, use :func:`normcase`. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: realpath(path) Return the canonical path of the specified filename, eliminating any symbolic links encountered in the path (if they are supported by the operating system). + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: relpath(path, start=os.curdir) @@ -290,6 +354,9 @@ Availability: Unix, Windows. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: samefile(path1, path2) @@ -305,6 +372,9 @@ .. versionchanged:: 3.4 Windows now uses the same implementation as all other platforms. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: sameopenfile(fp1, fp2) @@ -315,6 +385,9 @@ .. versionchanged:: 3.2 Added Windows support. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: samestat(stat1, stat2) @@ -328,6 +401,9 @@ .. versionchanged:: 3.4 Added Windows support. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: split(path) @@ -341,6 +417,9 @@ (but the strings may differ). Also see the functions :func:`dirname` and :func:`basename`. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: splitdrive(path) @@ -359,6 +438,9 @@ and share, up to but not including the fourth separator. e.g. ``splitdrive("//host/computer/dir")`` returns ``("//host/computer", "/dir")`` + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: splitext(path) @@ -367,6 +449,9 @@ period. Leading periods on the basename are ignored; ``splitext('.cshrc')`` returns ``('.cshrc', '')``. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: splitunc(path) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -930,6 +930,9 @@ exception, the function now retries the system call instead of raising an :exc:`InterruptedError` exception (see :pep:`475` for the rationale). + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + The following constants are options for the *flags* parameter to the :func:`~os.open` function. They can be combined using the bitwise OR operator ``|``. Some of them are not available on all platforms. For descriptions of @@ -1426,6 +1429,9 @@ .. versionchanged:: 3.3 Added the *dir_fd*, *effective_ids*, and *follow_symlinks* parameters. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. data:: F_OK R_OK @@ -1450,6 +1456,9 @@ Added support for specifying *path* as a file descriptor on some platforms. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: chflags(path, flags, *, follow_symlinks=True) @@ -1476,6 +1485,9 @@ .. versionadded:: 3.3 The *follow_symlinks* argument. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: chmod(path, mode, *, dir_fd=None, follow_symlinks=True) @@ -1517,6 +1529,9 @@ Added support for specifying *path* as an open file descriptor, and the *dir_fd* and *follow_symlinks* arguments. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: chown(path, uid, gid, *, dir_fd=None, follow_symlinks=True) @@ -1536,6 +1551,9 @@ Added support for specifying an open file descriptor for *path*, and the *dir_fd* and *follow_symlinks* arguments. + .. versionchanged:: 3.6 + Supports a :term:`path-like object`. + .. function:: chroot(path) @@ -1543,6 +1561,9 @@ Availability: Unix. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: fchdir(fd) @@ -1571,6 +1592,9 @@ Availability: Unix. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: lchmod(path, mode) @@ -1581,6 +1605,8 @@ Availability: Unix. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. .. function:: lchown(path, uid, gid) @@ -1590,6 +1616,9 @@ Availability: Unix. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: link(src, dst, *, src_dir_fd=None, dst_dir_fd=None, follow_symlinks=True) @@ -1607,6 +1636,9 @@ .. versionadded:: 3.3 Added the *src_dir_fd*, *dst_dir_fd*, and *follow_symlinks* arguments. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` for *src* and *dst*. + .. function:: listdir(path='.') @@ -1614,8 +1646,9 @@ *path*. The list is in arbitrary order, and does not include the special entries ``'.'`` and ``'..'`` even if they are present in the directory. - *path* may be either of type ``str`` or of type ``bytes``. If *path* - is of type ``bytes``, the filenames returned will also be of type ``bytes``; + *path* may be a :term:`path-like object`. If *path* is of type ``bytes`` + (directly or indirectly through the :class:`PathLike` interface), + the filenames returned will also be of type ``bytes``; in all other circumstances, they will be of type ``str``. This function can also support :ref:`specifying a file descriptor @@ -1636,6 +1669,9 @@ .. versionadded:: 3.3 Added support for specifying an open file descriptor for *path*. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: lstat(path, \*, dir_fd=None) @@ -1662,6 +1698,9 @@ .. versionchanged:: 3.3 Added the *dir_fd* parameter. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` for *src* and *dst*. + .. function:: mkdir(path, mode=0o777, *, dir_fd=None) @@ -1686,6 +1725,9 @@ .. versionadded:: 3.3 The *dir_fd* argument. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: makedirs(name, mode=0o777, exist_ok=False) @@ -1719,6 +1761,9 @@ mode of the existing directory. Since this behavior was impossible to implement safely, it was removed in Python 3.4.1. See :issue:`21082`. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: mkfifo(path, mode=0o666, *, dir_fd=None) @@ -1739,6 +1784,9 @@ .. versionadded:: 3.3 The *dir_fd* argument. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: mknod(path, mode=0o600, device=0, *, dir_fd=None) @@ -1756,6 +1804,9 @@ .. versionadded:: 3.3 The *dir_fd* argument. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: major(device) @@ -1794,6 +1845,9 @@ Availability: Unix. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. data:: pathconf_names @@ -1811,9 +1865,10 @@ may be converted to an absolute pathname using ``os.path.join(os.path.dirname(path), result)``. - If the *path* is a string object, the result will also be a string object, + If the *path* is a string object (directly or indirectly through a + :class:`PathLike` interface), the result will also be a string object, and the call may raise a UnicodeDecodeError. If the *path* is a bytes - object, the result will be a bytes object. + object (direct or indirectly), the result will be a bytes object. This function can also support :ref:`paths relative to directory descriptors `. @@ -1826,6 +1881,9 @@ .. versionadded:: 3.3 The *dir_fd* argument. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: remove(path, *, dir_fd=None) @@ -1844,6 +1902,9 @@ .. versionadded:: 3.3 The *dir_fd* argument. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: removedirs(name) @@ -1858,6 +1919,9 @@ they are empty. Raises :exc:`OSError` if the leaf directory could not be successfully removed. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None) @@ -1877,6 +1941,9 @@ .. versionadded:: 3.3 The *src_dir_fd* and *dst_dir_fd* arguments. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` for *src* and *dst*. + .. function:: renames(old, new) @@ -1890,6 +1957,9 @@ This function can fail with the new directory structure made if you lack permissions needed to remove the leaf directory or file. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` for *old* and *new*. + .. function:: replace(src, dst, *, src_dir_fd=None, dst_dir_fd=None) @@ -1904,6 +1974,9 @@ .. versionadded:: 3.3 + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` for *src* and *dst*. + .. function:: rmdir(path, *, dir_fd=None) @@ -1917,6 +1990,9 @@ .. versionadded:: 3.3 The *dir_fd* parameter. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: scandir(path='.') @@ -1935,7 +2011,8 @@ always requires a system call on Unix but only requires one for symbolic links on Windows. - On Unix, *path* can be of type :class:`str` or :class:`bytes` (use + On Unix, *path* can be of type :class:`str` or :class:`bytes` (either + directly or indirectly through the :class:`PathLike` interface; use :func:`~os.fsencode` and :func:`~os.fsdecode` to encode and decode :class:`bytes` paths). On Windows, *path* must be of type :class:`str`. On both systems, the type of the :attr:`~os.DirEntry.name` and @@ -1986,6 +2063,8 @@ exhausted nor explicitly closed a :exc:`ResourceWarning` will be emitted in its destructor. + The function accepts a :term:`path-like object`. + .. class:: DirEntry @@ -2007,7 +2086,7 @@ ``os.DirEntry`` methods and handle as appropriate. To be directly usable as a :term:`path-like object`, ``os.DirEntry`` - implements the :class:`os.PathLike` interface. + implements the :class:`PathLike` interface. Attributes and methods on a ``os.DirEntry`` instance are as follows: @@ -2123,14 +2202,15 @@ .. versionadded:: 3.5 .. versionchanged:: 3.6 - Added support for the :class:`os.PathLike` interface. + Added support for the :class:`~os.PathLike` interface. .. function:: stat(path, \*, dir_fd=None, follow_symlinks=True) Get the status of a file or a file descriptor. Perform the equivalent of a :c:func:`stat` system call on the given path. *path* may be specified as - either a string or as an open file descriptor. Return a :class:`stat_result` + either a string -- directly or indirectly through the :class:`PathLike` + interface -- or as an open file descriptor. Return a :class:`stat_result` object. This function normally follows symlinks; to stat a symlink add the argument @@ -2160,6 +2240,9 @@ Added the *dir_fd* and *follow_symlinks* arguments, specifying a file descriptor instead of a path. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. class:: stat_result @@ -2381,19 +2464,22 @@ This function can support :ref:`specifying a file descriptor `. + Availability: Unix. + .. versionchanged:: 3.2 The :const:`ST_RDONLY` and :const:`ST_NOSUID` constants were added. + .. versionadded:: 3.3 + Added support for specifying an open file descriptor for *path*. + .. versionchanged:: 3.4 The :const:`ST_NODEV`, :const:`ST_NOEXEC`, :const:`ST_SYNCHRONOUS`, :const:`ST_MANDLOCK`, :const:`ST_WRITE`, :const:`ST_APPEND`, :const:`ST_IMMUTABLE`, :const:`ST_NOATIME`, :const:`ST_NODIRATIME`, and :const:`ST_RELATIME` constants were added. - Availability: Unix. - - .. versionadded:: 3.3 - Added support for specifying an open file descriptor for *path*. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. .. data:: supports_dir_fd @@ -2514,6 +2600,9 @@ Added the *dir_fd* argument, and now allow *target_is_directory* on non-Windows platforms. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` for *src* and *dst*. + .. function:: sync() @@ -2538,6 +2627,10 @@ .. versionchanged:: 3.5 Added support for Windows + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + + .. function:: unlink(path, *, dir_fd=None) Remove (delete) the file *path*. This function is semantically @@ -2548,6 +2641,9 @@ .. versionadded:: 3.3 The *dir_fd* parameter. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: utime(path, times=None, *[, ns], dir_fd=None, follow_symlinks=True) @@ -2585,6 +2681,9 @@ Added support for specifying an open file descriptor for *path*, and the *dir_fd*, *follow_symlinks*, and *ns* parameters. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: walk(top, topdown=True, onerror=None, followlinks=False) @@ -2675,6 +2774,9 @@ This function now calls :func:`os.scandir` instead of :func:`os.listdir`, making it faster by reducing the number of calls to :func:`os.stat`. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: fwalk(top='.', topdown=True, onerror=None, *, follow_symlinks=False, dir_fd=None) @@ -2731,6 +2833,9 @@ .. versionadded:: 3.3 + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + Linux extended attributes ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2742,12 +2847,16 @@ .. function:: getxattr(path, attribute, *, follow_symlinks=True) Return the value of the extended filesystem attribute *attribute* for - *path*. *attribute* can be bytes or str. If it is str, it is encoded - with the filesystem encoding. + *path*. *attribute* can be bytes or str (directly or indirectly through the + :class:`PathLike` interface). If it is str, it is encoded with the filesystem + encoding. This function can support :ref:`specifying a file descriptor ` and :ref:`not following symlinks `. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` fpr *path* and *attribute*. + .. function:: listxattr(path=None, *, follow_symlinks=True) @@ -2759,21 +2868,29 @@ This function can support :ref:`specifying a file descriptor ` and :ref:`not following symlinks `. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: removexattr(path, attribute, *, follow_symlinks=True) Removes the extended filesystem attribute *attribute* from *path*. - *attribute* should be bytes or str. If it is a string, it is encoded + *attribute* should be bytes or str (directly or indirectly through the + :class:`PathLike` interface). If it is a string, it is encoded with the filesystem encoding. This function can support :ref:`specifying a file descriptor ` and :ref:`not following symlinks `. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` for *path* and *attribute*. + .. function:: setxattr(path, attribute, value, flags=0, *, follow_symlinks=True) Set the extended filesystem attribute *attribute* on *path* to *value*. - *attribute* must be a bytes or str with no embedded NULs. If it is a str, + *attribute* must be a bytes or str with no embedded NULs (directly or + indirectly through the :class:`PathLike` interface). If it is a str, it is encoded with the filesystem encoding. *flags* may be :data:`XATTR_REPLACE` or :data:`XATTR_CREATE`. If :data:`XATTR_REPLACE` is given and the attribute does not exist, ``EEXISTS`` will be raised. @@ -2788,6 +2905,9 @@ A bug in Linux kernel versions less than 2.6.39 caused the flags argument to be ignored on some filesystems. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object` for *path* and *attribute*. + .. data:: XATTR_SIZE_MAX @@ -2889,6 +3009,9 @@ Added support for specifying an open file descriptor for *path* for :func:`execve`. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: _exit(n) Exit the process with status *n*, without calling cleanup handlers, flushing @@ -3199,6 +3322,9 @@ :func:`spawnve` are not thread-safe on Windows; we advise you to use the :mod:`subprocess` module instead. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. data:: P_NOWAIT P_NOWAITO -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:59:10 2016 From: python-checkins at python.org (brett.cannon) Date: Tue, 06 Sep 2016 22:59:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Darn_you=2C_Benjamin!?= Message-ID: <20160906225909.22677.29268.5218B328@psf.io> https://hg.python.org/cpython/rev/388d58a5de9e changeset: 103165:388d58a5de9e parent: 103164:abe3db90d8ad parent: 103162:a882c1981029 user: Brett Cannon date: Tue Sep 06 15:58:40 2016 -0700 summary: Darn you, Benjamin! files: configure | 16 +++++++++++++++- configure.ac | 2 ++ 2 files changed, 17 insertions(+), 1 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -775,6 +775,7 @@ docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -885,6 +886,7 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1137,6 +1139,15 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1274,7 +1285,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1427,6 +1438,7 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -12372,6 +12384,8 @@ break; } } + freeaddrinfo(aitop); + aitop = NULL; } if (!(inet4 == 0 || inet4 == 2)) diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3718,6 +3718,8 @@ break; } } + freeaddrinfo(aitop); + aitop = NULL; } if (!(inet4 == 0 || inet4 == 2)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 18:59:09 2016 From: python-checkins at python.org (brett.cannon) Date: Tue, 06 Sep 2016 22:59:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_for_=2327872_doc_changes?= Message-ID: <20160906225909.31427.8321.D22C2CDC@psf.io> https://hg.python.org/cpython/rev/abe3db90d8ad changeset: 103164:abe3db90d8ad parent: 103163:9be0286772bf parent: 103158:1d29d3a1e0c6 user: Brett Cannon date: Tue Sep 06 15:57:35 2016 -0700 summary: Merge for #27872 doc changes files: Modules/clinic/posixmodule.c.h | 14 +++++++------- Modules/posixmodule.c | 8 ++++---- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -3011,13 +3011,13 @@ {"waitpid", (PyCFunction)os_waitpid, METH_VARARGS, os_waitpid__doc__}, static PyObject * -os_waitpid_impl(PyObject *module, Py_intptr_t pid, int options); +os_waitpid_impl(PyObject *module, intptr_t pid, int options); static PyObject * os_waitpid(PyObject *module, PyObject *args) { PyObject *return_value = NULL; - Py_intptr_t pid; + intptr_t pid; int options; if (!PyArg_ParseTuple(args, "" _Py_PARSE_INTPTR "i:waitpid", @@ -5479,13 +5479,13 @@ {"get_handle_inheritable", (PyCFunction)os_get_handle_inheritable, METH_O, os_get_handle_inheritable__doc__}, static int -os_get_handle_inheritable_impl(PyObject *module, Py_intptr_t handle); +os_get_handle_inheritable_impl(PyObject *module, intptr_t handle); static PyObject * os_get_handle_inheritable(PyObject *module, PyObject *arg) { PyObject *return_value = NULL; - Py_intptr_t handle; + intptr_t handle; int _return_value; if (!PyArg_Parse(arg, "" _Py_PARSE_INTPTR ":get_handle_inheritable", &handle)) { @@ -5515,14 +5515,14 @@ {"set_handle_inheritable", (PyCFunction)os_set_handle_inheritable, METH_VARARGS, os_set_handle_inheritable__doc__}, static PyObject * -os_set_handle_inheritable_impl(PyObject *module, Py_intptr_t handle, +os_set_handle_inheritable_impl(PyObject *module, intptr_t handle, int inheritable); static PyObject * os_set_handle_inheritable(PyObject *module, PyObject *args) { PyObject *return_value = NULL; - Py_intptr_t handle; + intptr_t handle; int inheritable; if (!PyArg_ParseTuple(args, "" _Py_PARSE_INTPTR "p:set_handle_inheritable", @@ -6042,4 +6042,4 @@ #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ -/*[clinic end generated code: output=2b85bb3703a6488a input=a9049054013a1b77]*/ +/*[clinic end generated code: output=677ce794fb126161 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -2520,7 +2520,7 @@ impl_by_reference = True; [python start generated code]*/ -/*[python end generated code: output=da39a3ee5e6b4b0d input=affe68316f160401]*/ +/*[python end generated code: output=da39a3ee5e6b4b0d input=418fce0e01144461]*/ /*[clinic input] @@ -7092,7 +7092,7 @@ static PyObject * os_waitpid_impl(PyObject *module, intptr_t pid, int options) -/*[clinic end generated code: output=15f1ce005a346b09 input=444c8f51cca5b862]*/ +/*[clinic end generated code: output=be836b221271d538 input=40f2440c515410f8]*/ { int status; intptr_t res; @@ -11383,7 +11383,7 @@ static int os_get_handle_inheritable_impl(PyObject *module, intptr_t handle) -/*[clinic end generated code: output=9e5389b0aa0916ce input=5f7759443aae3dc5]*/ +/*[clinic end generated code: output=36be5afca6ea84d8 input=cfe99f9c05c70ad1]*/ { DWORD flags; @@ -11408,7 +11408,7 @@ static PyObject * os_set_handle_inheritable_impl(PyObject *module, intptr_t handle, int inheritable) -/*[clinic end generated code: output=b1e67bfa3213d745 input=e64b2b2730469def]*/ +/*[clinic end generated code: output=021d74fe6c96baa3 input=7a7641390d8364fc]*/ { DWORD flags = inheritable ? HANDLE_FLAG_INHERIT : 0; if (!SetHandleInformation((HANDLE)handle, HANDLE_FLAG_INHERIT, flags)) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:05:11 2016 From: python-checkins at python.org (berker.peksag) Date: Tue, 06 Sep 2016 23:05:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIxMjUw?= =?utf-8?q?=3A_Add_tests_for_SQLite=27s_ON_CONFLICT_clause?= Message-ID: <20160906230510.1548.93334.1B7860C8@psf.io> https://hg.python.org/cpython/rev/91d3022b3f03 changeset: 103166:91d3022b3f03 branch: 3.5 parent: 103159:c90f8c994048 user: Berker Peksag date: Wed Sep 07 02:04:34 2016 +0300 summary: Issue #21250: Add tests for SQLite's ON CONFLICT clause Initial patch by Alex LordThorsen. files: Lib/sqlite3/test/dbapi.py | 101 +++++++++++++++++++++++++- 1 files changed, 100 insertions(+), 1 deletions(-) diff --git a/Lib/sqlite3/test/dbapi.py b/Lib/sqlite3/test/dbapi.py --- a/Lib/sqlite3/test/dbapi.py +++ b/Lib/sqlite3/test/dbapi.py @@ -779,6 +779,100 @@ method = getattr(cur, method_name) method(*params) + +class SqliteOnConflictTests(unittest.TestCase): + """ + Tests for SQLite's "insert on conflict" feature. + + See https://www.sqlite.org/lang_conflict.html for details. + """ + + def setUp(self): + self.cx = sqlite.connect(":memory:") + self.cu = self.cx.cursor() + self.cu.execute(""" + CREATE TABLE test( + id INTEGER PRIMARY KEY, name TEXT, unique_name TEXT UNIQUE + ); + """) + + def tearDown(self): + self.cu.close() + self.cx.close() + + def CheckOnConflictRollbackWithExplicitTransaction(self): + self.cx.isolation_level = None # autocommit mode + self.cu = self.cx.cursor() + # Start an explicit transaction. + self.cu.execute("BEGIN") + self.cu.execute("INSERT INTO test(name) VALUES ('abort_test')") + self.cu.execute("INSERT OR ROLLBACK INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR ROLLBACK INTO test(unique_name) VALUES ('foo')") + # Use connection to commit. + self.cx.commit() + self.cu.execute("SELECT name, unique_name from test") + # Transaction should have rolled back and nothing should be in table. + self.assertEqual(self.cu.fetchall(), []) + + def CheckOnConflictAbortRaisesWithExplicitTransactions(self): + # Abort cancels the current sql statement but doesn't change anything + # about the current transaction. + self.cx.isolation_level = None # autocommit mode + self.cu = self.cx.cursor() + # Start an explicit transaction. + self.cu.execute("BEGIN") + self.cu.execute("INSERT INTO test(name) VALUES ('abort_test')") + self.cu.execute("INSERT OR ABORT INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR ABORT INTO test(unique_name) VALUES ('foo')") + self.cx.commit() + self.cu.execute("SELECT name, unique_name FROM test") + # Expect the first two inserts to work, third to do nothing. + self.assertEqual(self.cu.fetchall(), [('abort_test', None), (None, 'foo',)]) + + def CheckOnConflictRollbackWithoutTransaction(self): + # Start of implicit transaction + self.cu.execute("INSERT INTO test(name) VALUES ('abort_test')") + self.cu.execute("INSERT OR ROLLBACK INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR ROLLBACK INTO test(unique_name) VALUES ('foo')") + self.cu.execute("SELECT name, unique_name FROM test") + # Implicit transaction is rolled back on error. + self.assertEqual(self.cu.fetchall(), []) + + def CheckOnConflictAbortRaisesWithoutTransactions(self): + # Abort cancels the current sql statement but doesn't change anything + # about the current transaction. + self.cu.execute("INSERT INTO test(name) VALUES ('abort_test')") + self.cu.execute("INSERT OR ABORT INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR ABORT INTO test(unique_name) VALUES ('foo')") + # Make sure all other values were inserted. + self.cu.execute("SELECT name, unique_name FROM test") + self.assertEqual(self.cu.fetchall(), [('abort_test', None), (None, 'foo',)]) + + def CheckOnConflictFail(self): + self.cu.execute("INSERT OR FAIL INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR FAIL INTO test(unique_name) VALUES ('foo')") + self.assertEqual(self.cu.fetchall(), []) + + def CheckOnConflictIgnore(self): + self.cu.execute("INSERT OR IGNORE INTO test(unique_name) VALUES ('foo')") + # Nothing should happen. + self.cu.execute("INSERT OR IGNORE INTO test(unique_name) VALUES ('foo')") + self.cu.execute("SELECT unique_name FROM test") + self.assertEqual(self.cu.fetchall(), [('foo',)]) + + def CheckOnConflictReplace(self): + self.cu.execute("INSERT OR REPLACE INTO test(name, unique_name) VALUES ('Data!', 'foo')") + # There shouldn't be an IntegrityError exception. + self.cu.execute("INSERT OR REPLACE INTO test(name, unique_name) VALUES ('Very different data!', 'foo')") + self.cu.execute("SELECT name, unique_name FROM test") + self.assertEqual(self.cu.fetchall(), [('Very different data!', 'foo')]) + + def suite(): module_suite = unittest.makeSuite(ModuleTests, "Check") connection_suite = unittest.makeSuite(ConnectionTests, "Check") @@ -788,7 +882,12 @@ ext_suite = unittest.makeSuite(ExtensionTests, "Check") closed_con_suite = unittest.makeSuite(ClosedConTests, "Check") closed_cur_suite = unittest.makeSuite(ClosedCurTests, "Check") - return unittest.TestSuite((module_suite, connection_suite, cursor_suite, thread_suite, constructor_suite, ext_suite, closed_con_suite, closed_cur_suite)) + on_conflict_suite = unittest.makeSuite(SqliteOnConflictTests, "Check") + return unittest.TestSuite(( + module_suite, connection_suite, cursor_suite, thread_suite, + constructor_suite, ext_suite, closed_con_suite, closed_cur_suite, + on_conflict_suite, + )) def test(): runner = unittest.TextTestRunner() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:05:12 2016 From: python-checkins at python.org (berker.peksag) Date: Tue, 06 Sep 2016 23:05:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321250=3A_Merge_from_3=2E5?= Message-ID: <20160906230511.78171.28526.E8BC106F@psf.io> https://hg.python.org/cpython/rev/db2bedd5c34a changeset: 103167:db2bedd5c34a parent: 103165:388d58a5de9e parent: 103166:91d3022b3f03 user: Berker Peksag date: Wed Sep 07 02:05:16 2016 +0300 summary: Issue #21250: Merge from 3.5 files: Lib/sqlite3/test/dbapi.py | 101 +++++++++++++++++++++++++- 1 files changed, 100 insertions(+), 1 deletions(-) diff --git a/Lib/sqlite3/test/dbapi.py b/Lib/sqlite3/test/dbapi.py --- a/Lib/sqlite3/test/dbapi.py +++ b/Lib/sqlite3/test/dbapi.py @@ -820,6 +820,100 @@ method = getattr(cur, method_name) method(*params) + +class SqliteOnConflictTests(unittest.TestCase): + """ + Tests for SQLite's "insert on conflict" feature. + + See https://www.sqlite.org/lang_conflict.html for details. + """ + + def setUp(self): + self.cx = sqlite.connect(":memory:") + self.cu = self.cx.cursor() + self.cu.execute(""" + CREATE TABLE test( + id INTEGER PRIMARY KEY, name TEXT, unique_name TEXT UNIQUE + ); + """) + + def tearDown(self): + self.cu.close() + self.cx.close() + + def CheckOnConflictRollbackWithExplicitTransaction(self): + self.cx.isolation_level = None # autocommit mode + self.cu = self.cx.cursor() + # Start an explicit transaction. + self.cu.execute("BEGIN") + self.cu.execute("INSERT INTO test(name) VALUES ('abort_test')") + self.cu.execute("INSERT OR ROLLBACK INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR ROLLBACK INTO test(unique_name) VALUES ('foo')") + # Use connection to commit. + self.cx.commit() + self.cu.execute("SELECT name, unique_name from test") + # Transaction should have rolled back and nothing should be in table. + self.assertEqual(self.cu.fetchall(), []) + + def CheckOnConflictAbortRaisesWithExplicitTransactions(self): + # Abort cancels the current sql statement but doesn't change anything + # about the current transaction. + self.cx.isolation_level = None # autocommit mode + self.cu = self.cx.cursor() + # Start an explicit transaction. + self.cu.execute("BEGIN") + self.cu.execute("INSERT INTO test(name) VALUES ('abort_test')") + self.cu.execute("INSERT OR ABORT INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR ABORT INTO test(unique_name) VALUES ('foo')") + self.cx.commit() + self.cu.execute("SELECT name, unique_name FROM test") + # Expect the first two inserts to work, third to do nothing. + self.assertEqual(self.cu.fetchall(), [('abort_test', None), (None, 'foo',)]) + + def CheckOnConflictRollbackWithoutTransaction(self): + # Start of implicit transaction + self.cu.execute("INSERT INTO test(name) VALUES ('abort_test')") + self.cu.execute("INSERT OR ROLLBACK INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR ROLLBACK INTO test(unique_name) VALUES ('foo')") + self.cu.execute("SELECT name, unique_name FROM test") + # Implicit transaction is rolled back on error. + self.assertEqual(self.cu.fetchall(), []) + + def CheckOnConflictAbortRaisesWithoutTransactions(self): + # Abort cancels the current sql statement but doesn't change anything + # about the current transaction. + self.cu.execute("INSERT INTO test(name) VALUES ('abort_test')") + self.cu.execute("INSERT OR ABORT INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR ABORT INTO test(unique_name) VALUES ('foo')") + # Make sure all other values were inserted. + self.cu.execute("SELECT name, unique_name FROM test") + self.assertEqual(self.cu.fetchall(), [('abort_test', None), (None, 'foo',)]) + + def CheckOnConflictFail(self): + self.cu.execute("INSERT OR FAIL INTO test(unique_name) VALUES ('foo')") + with self.assertRaises(sqlite.IntegrityError): + self.cu.execute("INSERT OR FAIL INTO test(unique_name) VALUES ('foo')") + self.assertEqual(self.cu.fetchall(), []) + + def CheckOnConflictIgnore(self): + self.cu.execute("INSERT OR IGNORE INTO test(unique_name) VALUES ('foo')") + # Nothing should happen. + self.cu.execute("INSERT OR IGNORE INTO test(unique_name) VALUES ('foo')") + self.cu.execute("SELECT unique_name FROM test") + self.assertEqual(self.cu.fetchall(), [('foo',)]) + + def CheckOnConflictReplace(self): + self.cu.execute("INSERT OR REPLACE INTO test(name, unique_name) VALUES ('Data!', 'foo')") + # There shouldn't be an IntegrityError exception. + self.cu.execute("INSERT OR REPLACE INTO test(name, unique_name) VALUES ('Very different data!', 'foo')") + self.cu.execute("SELECT name, unique_name FROM test") + self.assertEqual(self.cu.fetchall(), [('Very different data!', 'foo')]) + + def suite(): module_suite = unittest.makeSuite(ModuleTests, "Check") connection_suite = unittest.makeSuite(ConnectionTests, "Check") @@ -829,7 +923,12 @@ ext_suite = unittest.makeSuite(ExtensionTests, "Check") closed_con_suite = unittest.makeSuite(ClosedConTests, "Check") closed_cur_suite = unittest.makeSuite(ClosedCurTests, "Check") - return unittest.TestSuite((module_suite, connection_suite, cursor_suite, thread_suite, constructor_suite, ext_suite, closed_con_suite, closed_cur_suite)) + on_conflict_suite = unittest.makeSuite(SqliteOnConflictTests, "Check") + return unittest.TestSuite(( + module_suite, connection_suite, cursor_suite, thread_suite, + constructor_suite, ext_suite, closed_con_suite, closed_cur_suite, + on_conflict_suite, + )) def test(): runner = unittest.TextTestRunner() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:20:05 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 06 Sep 2016 23:20:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_os=2Egetrandom=28=29?= Message-ID: <20160906232005.22892.46415.537A61F5@psf.io> https://hg.python.org/cpython/rev/27267d2fb091 changeset: 103168:27267d2fb091 user: Victor Stinner date: Tue Sep 06 16:18:52 2016 -0700 summary: Add os.getrandom() Issue #27778: Expose the Linux getrandom() syscall as a new os.getrandom() function. This change is part of the PEP 524. files: Doc/library/os.rst | 65 +++++++++++++++++---- Doc/whatsnew/3.6.rst | 4 + Lib/test/test_os.py | 32 ++++++++++ Misc/NEWS | 3 + Modules/clinic/posixmodule.c.h | 41 +++++++++++++- Modules/posixmodule.c | 66 ++++++++++++++++++++++ 6 files changed, 198 insertions(+), 13 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -3931,21 +3931,44 @@ .. versionadded:: 3.3 -.. _os-miscfunc: - -Miscellaneous Functions ------------------------ - - -.. function:: urandom(n) - - Return a string of *n* random bytes suitable for cryptographic use. + +Random numbers +-------------- + + +.. function:: getrandom(size, flags=0) + + Get up to *size* random bytes. The function can return less bytes than + requested. + + These bytes can be used to seed user-space random number generators or for + cryptographic purposes. + + ``getrandom()`` relies on entropy gathered from device drivers and other + sources of environmental noise. Unnecessarily reading large quantities of + data will have a negative impact on other users of the ``/dev/random`` and + ``/dev/urandom`` devices. + + The flags argument is a bit mask that can contain zero or more of the + following values ORed together: :py:data:`os.GRND_RANDOM` and + :py:data:`GRND_NONBLOCK`. + + See also the `Linux getrandom() manual page + `_. + + Availability: Linux 3.17 and newer. + + .. versionadded:: 3.6 + +.. function:: urandom(size) + + Return a string of *size* random bytes suitable for cryptographic use. This function returns random bytes from an OS-specific randomness source. The returned data should be unpredictable enough for cryptographic applications, though its exact quality depends on the OS implementation. - On Linux, ``getrandom()`` syscall is used if available and the urandom + On Linux, the ``getrandom()`` syscall is used if available and the urandom entropy pool is initialized (``getrandom()`` does not block). On a Unix-like system this will query ``/dev/urandom``. On Windows, it will use ``CryptGenRandom()``. If a randomness source is not found, @@ -3955,11 +3978,29 @@ provided by your platform, please see :class:`random.SystemRandom`. .. versionchanged:: 3.5.2 - On Linux, if ``getrandom()`` blocks (the urandom entropy pool is not - initialized yet), fall back on reading ``/dev/urandom``. + On Linux, if the ``getrandom()`` syscall blocks (the urandom entropy pool + is not initialized yet), fall back on reading ``/dev/urandom``. .. versionchanged:: 3.5 On Linux 3.17 and newer, the ``getrandom()`` syscall is now used when available. On OpenBSD 5.6 and newer, the C ``getentropy()`` function is now used. These functions avoid the usage of an internal file descriptor. + +.. data:: GRND_NONBLOCK + + By default, when reading from ``/dev/random``, :func:`getrandom` blocks if + no random bytes are available, and when reading from ``/dev/urandom``, it blocks + if the entropy pool has not yet been initialized. + + If the :py:data:`GRND_NONBLOCK` flag is set, then :func:`getrandom` does not + block in these cases, but instead immediately raises :exc:`BlockingIOError`. + + .. versionadded:: 3.6 + +.. data:: GRND_RANDOM + + If this bit is set, then random bytes are drawn from the + ``/dev/random`` pool instead of the ``/dev/urandom`` pool. + + .. versionadded:: 3.6 diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -484,6 +484,10 @@ will be emitted in its destructor. (Contributed by Serhiy Storchaka in :issue:`25994`.) +The Linux ``getrandom()`` syscall (get random bytes) is now exposed as the new +:func:`os.getrandom` function. +(Contributed by Victor Stinner, part of the :pep:`524`) + pickle ------ diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1248,6 +1248,7 @@ def test_urandom_value(self): data1 = os.urandom(16) + self.assertIsInstance(data1, bytes) data2 = os.urandom(16) self.assertNotEqual(data1, data2) @@ -1268,6 +1269,37 @@ self.assertNotEqual(data1, data2) + at unittest.skipUnless(hasattr(os, 'getrandom'), 'need os.getrandom()') +class GetRandomTests(unittest.TestCase): + def test_getrandom_type(self): + data = os.getrandom(16) + self.assertIsInstance(data, bytes) + self.assertEqual(len(data), 16) + + def test_getrandom0(self): + empty = os.getrandom(0) + self.assertEqual(empty, b'') + + def test_getrandom_random(self): + self.assertTrue(hasattr(os, 'GRND_RANDOM')) + + # Don't test os.getrandom(1, os.GRND_RANDOM) to not consume the rare + # resource /dev/random + + def test_getrandom_nonblock(self): + # The call must not fail. Check also that the flag exists + try: + os.getrandom(1, os.GRND_NONBLOCK) + except BlockingIOError: + # System urandom is not initialized yet + pass + + def test_getrandom_value(self): + data1 = os.getrandom(16) + data2 = os.getrandom(16) + self.assertNotEqual(data1, data2) + + # os.urandom() doesn't use a file descriptor when it is implemented with the # getentropy() function, the getrandom() function or the getrandom() syscall OS_URANDOM_DONT_USE_FD = ( diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -89,6 +89,9 @@ Library ------- +- Issue #27778: Expose the Linux ``getrandom()`` syscall as a new + :func:`os.getrandom` function. This change is part of the :pep:`524`. + - Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name fields in X.509 certs. diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -5571,6 +5571,41 @@ return return_value; } +#if defined(HAVE_GETRANDOM_SYSCALL) + +PyDoc_STRVAR(os_getrandom__doc__, +"getrandom($module, /, size, flags=0)\n" +"--\n" +"\n" +"Obtain a series of random bytes."); + +#define OS_GETRANDOM_METHODDEF \ + {"getrandom", (PyCFunction)os_getrandom, METH_VARARGS|METH_KEYWORDS, os_getrandom__doc__}, + +static PyObject * +os_getrandom_impl(PyObject *module, Py_ssize_t size, int flags); + +static PyObject * +os_getrandom(PyObject *module, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"size", "flags", NULL}; + static _PyArg_Parser _parser = {"n|i:getrandom", _keywords, 0}; + Py_ssize_t size; + int flags = 0; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &size, &flags)) { + goto exit; + } + return_value = os_getrandom_impl(module, size, flags); + +exit: + return return_value; +} + +#endif /* defined(HAVE_GETRANDOM_SYSCALL) */ + #ifndef OS_TTYNAME_METHODDEF #define OS_TTYNAME_METHODDEF #endif /* !defined(OS_TTYNAME_METHODDEF) */ @@ -6042,4 +6077,8 @@ #ifndef OS_SET_HANDLE_INHERITABLE_METHODDEF #define OS_SET_HANDLE_INHERITABLE_METHODDEF #endif /* !defined(OS_SET_HANDLE_INHERITABLE_METHODDEF) */ -/*[clinic end generated code: output=677ce794fb126161 input=a9049054013a1b77]*/ + +#ifndef OS_GETRANDOM_METHODDEF + #define OS_GETRANDOM_METHODDEF +#endif /* !defined(OS_GETRANDOM_METHODDEF) */ +/*[clinic end generated code: output=fce51c7d432662c2 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -133,6 +133,13 @@ #include #endif +#ifdef HAVE_LINUX_RANDOM_H +# include +#endif +#ifdef HAVE_GETRANDOM_SYSCALL +# include +#endif + #if defined(MS_WINDOWS) # define TERMSIZE_USE_CONIO #elif defined(HAVE_SYS_IOCTL_H) @@ -12421,6 +12428,59 @@ return PyOS_FSPath(path); } +#ifdef HAVE_GETRANDOM_SYSCALL +/*[clinic input] +os.getrandom + + size: Py_ssize_t + flags: int=0 + +Obtain a series of random bytes. +[clinic start generated code]*/ + +static PyObject * +os_getrandom_impl(PyObject *module, Py_ssize_t size, int flags) +/*[clinic end generated code: output=b3a618196a61409c input=59bafac39c594947]*/ +{ + char *buffer; + Py_ssize_t n; + PyObject *bytes; + + if (size < 0) { + errno = EINVAL; + return posix_error(); + } + + buffer = PyMem_Malloc(size); + if (buffer == NULL) { + PyErr_NoMemory(); + return NULL; + } + + while (1) { + n = syscall(SYS_getrandom, buffer, size, flags); + if (n < 0 && errno == EINTR) { + if (PyErr_CheckSignals() < 0) { + return NULL; + } + continue; + } + break; + } + + if (n < 0) { + PyMem_Free(buffer); + PyErr_SetFromErrno(PyExc_OSError); + return NULL; + } + + bytes = PyBytes_FromStringAndSize(buffer, n); + PyMem_Free(buffer); + + return bytes; +} +#endif /* HAVE_GETRANDOM_SYSCALL */ + #include "clinic/posixmodule.c.h" /*[clinic input] @@ -12621,6 +12681,7 @@ METH_VARARGS | METH_KEYWORDS, posix_scandir__doc__}, OS_FSPATH_METHODDEF + OS_GETRANDOM_METHODDEF {NULL, NULL} /* Sentinel */ }; @@ -13066,6 +13127,11 @@ if (PyModule_AddIntMacro(m, RTLD_DEEPBIND)) return -1; #endif +#ifdef HAVE_GETRANDOM_SYSCALL + if (PyModule_AddIntMacro(m, GRND_RANDOM)) return -1; + if (PyModule_AddIntMacro(m, GRND_NONBLOCK)) return -1; +#endif + return 0; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:21:12 2016 From: python-checkins at python.org (brett.cannon) Date: Tue, 06 Sep 2016 23:21:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327974=3A_Remove_i?= =?utf-8?q?mportlib=2E=5Fbootstrap=2E=5FManageReload=2E?= Message-ID: <20160906232112.31532.6179.B4C3B307@psf.io> https://hg.python.org/cpython/rev/a3db6e954f58 changeset: 103169:a3db6e954f58 user: Brett Cannon date: Tue Sep 06 16:20:46 2016 -0700 summary: Issue #27974: Remove importlib._bootstrap._ManageReload. Class was dead code. Thanks to Xiang Zhang for the patch. files: Lib/importlib/_bootstrap.py | 17 - Python/importlib.h | 3545 +++++++++++----------- 2 files changed, 1746 insertions(+), 1816 deletions(-) diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py --- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -36,23 +36,6 @@ return type(sys)(name) -class _ManageReload: - - """Manages the possible clean-up of sys.modules for load_module().""" - - def __init__(self, name): - self._name = name - - def __enter__(self): - self._is_reload = self._name in sys.modules - - def __exit__(self, *args): - if any(arg is not None for arg in args) and not self._is_reload: - try: - del sys.modules[self._name] - except KeyError: - pass - # Module-level locking ######################################################## # A dict mapping module names to weakrefs of _ModuleLock instances diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:21:42 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 23:21:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Download_hashlib_test_vect?= =?utf-8?q?ors_from_pythontest=2Enet?= Message-ID: <20160906232141.48824.82164.9E360659@psf.io> https://hg.python.org/cpython/rev/c49ff8dbf907 changeset: 103170:c49ff8dbf907 user: Christian Heimes date: Wed Sep 07 01:21:14 2016 +0200 summary: Download hashlib test vectors from pythontest.net files: Lib/test/test_hashlib.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py --- a/Lib/test/test_hashlib.py +++ b/Lib/test/test_hashlib.py @@ -44,7 +44,7 @@ return r -URL = "https://raw.githubusercontent.com/tiran/python_vectors/master/{}.txt" +URL = "http://www.pythontest.net/hashlib/{}.txt" def read_vectors(hash_name): with support.open_urlresource(URL.format(hash_name)) as f: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:36:02 2016 From: python-checkins at python.org (raymond.hettinger) Date: Tue, 06 Sep 2016 23:36:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323226=3A__Add_lin?= =?utf-8?q?space=28=29_recipe_to_the_docs?= Message-ID: <20160906233602.3387.10601.6CA3869D@psf.io> https://hg.python.org/cpython/rev/bb8fe61d78a4 changeset: 103171:bb8fe61d78a4 user: Raymond Hettinger date: Tue Sep 06 16:35:34 2016 -0700 summary: Issue #23226: Add linspace() recipe to the docs files: Doc/library/stdtypes.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1363,6 +1363,11 @@ The :attr:`~range.start`, :attr:`~range.stop` and :attr:`~range.step` attributes. +.. seealso:: + + * The `linspace recipe `_ + shows how to implement a lazy version of range that suitable for floating + point applications. .. index:: single: string; text sequence type -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:37:43 2016 From: python-checkins at python.org (victor.stinner) Date: Tue, 06 Sep 2016 23:37:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_os=2Eurandom=28=29_now_blo?= =?utf-8?q?cks_on_Linux?= Message-ID: <20160906233743.14647.41602.6C7471DB@psf.io> https://hg.python.org/cpython/rev/45fc0c83ed42 changeset: 103172:45fc0c83ed42 user: Victor Stinner date: Tue Sep 06 16:33:52 2016 -0700 summary: os.urandom() now blocks on Linux Issue #27776: The os.urandom() function does now block on Linux 3.17 and newer until the system urandom entropy pool is initialized to increase the security. This change is part of the PEP 524. files: Doc/library/os.rst | 29 ++++++++--- Doc/whatsnew/3.6.rst | 12 ++++ Include/pylifecycle.h | 3 +- Lib/random.py | 9 --- Misc/NEWS | 4 + Modules/_randommodule.c | 56 +++++++++++++++++--- Modules/posixmodule.c | 3 +- Python/random.c | 75 +++++++++++++++++----------- 8 files changed, 131 insertions(+), 60 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -3968,14 +3968,27 @@ returned data should be unpredictable enough for cryptographic applications, though its exact quality depends on the OS implementation. - On Linux, the ``getrandom()`` syscall is used if available and the urandom - entropy pool is initialized (``getrandom()`` does not block). - On a Unix-like system this will query ``/dev/urandom``. On Windows, it - will use ``CryptGenRandom()``. If a randomness source is not found, - :exc:`NotImplementedError` will be raised. - - For an easy-to-use interface to the random number generator - provided by your platform, please see :class:`random.SystemRandom`. + On Linux, if the ``getrandom()`` syscall is available, it is used in + blocking mode: block until the system urandom entropy pool is initialized + (128 bits of entropy are collected by the kernel). See the :pep:`524` for + the rationale. On Linux, the :func:`getrandom` function can be used to get + random bytes in non-blocking mode (using the :data:`GRND_NONBLOCK` flag) or + to poll until the system urandom entropy pool is initialized. + + On a Unix-like system, random bytes are read from the ``/dev/urandom`` + device. If the ``/dev/urandom`` device is not available or not readable, the + :exc:`NotImplementedError` exception is raised. + + On Windows, it will use ``CryptGenRandom()``. + + .. seealso:: + The :mod:`secrets` module provides higher level functions. For an + easy-to-use interface to the random number generator provided by your + platform, please see :class:`random.SystemRandom`. + + .. versionchanged:: 3.6.0 + On Linux, ``getrandom()`` is now used in blocking mode to increase the + security. .. versionchanged:: 3.5.2 On Linux, if the ``getrandom()`` syscall blocks (the urandom entropy pool diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -70,6 +70,12 @@ * PEP 519: :ref:`Adding a file system path protocol ` +Security improvements: + +* On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool + is initialized to increase the security. See the :pep:`524` for the + rationale. + Windows improvements: * The ``py.exe`` launcher, when used interactively, no longer prefers @@ -345,6 +351,9 @@ Improved Modules ================ +On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool +is initialized to increase the security. See the :pep:`524` for the rationale. + asyncio ------- @@ -913,6 +922,9 @@ Changes in the Python API ------------------------- +* On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool + is initialized to increase the security. + * When :meth:`importlib.abc.Loader.exec_module` is defined, :meth:`importlib.abc.Loader.create_module` must also be defined. diff --git a/Include/pylifecycle.h b/Include/pylifecycle.h --- a/Include/pylifecycle.h +++ b/Include/pylifecycle.h @@ -117,7 +117,8 @@ PyAPI_FUNC(PyOS_sighandler_t) PyOS_setsig(int, PyOS_sighandler_t); /* Random */ -PyAPI_FUNC(int) _PyOS_URandom (void *buffer, Py_ssize_t size); +PyAPI_FUNC(int) _PyOS_URandom(void *buffer, Py_ssize_t size); +PyAPI_FUNC(int) _PyOS_URandomNonblock(void *buffer, Py_ssize_t size); #ifdef __cplusplus } diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -105,15 +105,6 @@ """ - if a is None: - try: - # Seed with enough bytes to span the 19937 bit - # state space for the Mersenne Twister - a = int.from_bytes(_urandom(2500), 'big') - except NotImplementedError: - import time - a = int(time.time() * 256) # use fractional seconds - if version == 1 and isinstance(a, (str, bytes)): x = ord(a[0]) << 7 if a else 0 for c in a: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -89,6 +89,10 @@ Library ------- +- Issue #27776: The :func:`os.urandom` function does now block on Linux 3.17 + and newer until the system urandom entropy pool is initialized to increase + the security. This change is part of the :pep:`524`. + - Issue #27778: Expose the Linux ``getrandom()`` syscall as a new :func:`os.getrandom` function. This change is part of the :pep:`524`. diff --git a/Modules/_randommodule.c b/Modules/_randommodule.c --- a/Modules/_randommodule.c +++ b/Modules/_randommodule.c @@ -165,7 +165,7 @@ /* initialize by an array with array-length */ /* init_key is the array for initializing keys */ /* key_length is its length */ -static PyObject * +static void init_by_array(RandomObject *self, uint32_t init_key[], size_t key_length) { size_t i, j, k; /* was signed in the original code. RDH 12/16/2002 */ @@ -190,8 +190,6 @@ } mt[0] = 0x80000000U; /* MSB is 1; assuring non-zero initial array */ - Py_INCREF(Py_None); - return Py_None; } /* @@ -199,6 +197,37 @@ * Twister download. */ +static int +random_seed_urandom(RandomObject *self) +{ + PY_UINT32_T key[N]; + + if (_PyOS_URandomNonblock(key, sizeof(key)) < 0) { + return -1; + } + init_by_array(self, key, Py_ARRAY_LENGTH(key)); + return 0; +} + +static void +random_seed_time_pid(RandomObject *self) +{ + _PyTime_t now; + uint32_t key[5]; + + now = _PyTime_GetSystemClock(); + key[0] = (PY_UINT32_T)(now & 0xffffffffU); + key[1] = (PY_UINT32_T)(now >> 32); + + key[2] = (PY_UINT32_T)getpid(); + + now = _PyTime_GetMonotonicClock(); + key[3] = (PY_UINT32_T)(now & 0xffffffffU); + key[4] = (PY_UINT32_T)(now >> 32); + + init_by_array(self, key, Py_ARRAY_LENGTH(key)); +} + static PyObject * random_seed(RandomObject *self, PyObject *args) { @@ -212,14 +241,17 @@ if (!PyArg_UnpackTuple(args, "seed", 0, 1, &arg)) return NULL; - if (arg == NULL || arg == Py_None) { - time_t now; + if (arg == NULL || arg == Py_None) { + if (random_seed_urandom(self) >= 0) { + PyErr_Clear(); - time(&now); - init_genrand(self, (uint32_t)now); - Py_INCREF(Py_None); - return Py_None; + /* Reading system entropy failed, fall back on the worst entropy: + use the current time and process identifier. */ + random_seed_time_pid(self); + } + Py_RETURN_NONE; } + /* This algorithm relies on the number being unsigned. * So: if the arg is a PyLong, use its absolute value. * Otherwise use its hash value, cast to unsigned. @@ -269,7 +301,11 @@ } } #endif - result = init_by_array(self, key, keyused); + init_by_array(self, key, keyused); + + Py_INCREF(Py_None); + result = Py_None; + Done: Py_XDECREF(n); PyMem_Free(key); diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -11168,8 +11168,7 @@ if (bytes == NULL) return NULL; - result = _PyOS_URandom(PyBytes_AS_STRING(bytes), - PyBytes_GET_SIZE(bytes)); + result = _PyOS_URandom(PyBytes_AS_STRING(bytes), PyBytes_GET_SIZE(bytes)); if (result == -1) { Py_DECREF(bytes); return NULL; diff --git a/Python/random.c b/Python/random.c --- a/Python/random.c +++ b/Python/random.c @@ -77,7 +77,7 @@ } /* Issue #25003: Don't use getentropy() on Solaris (available since - Solaris 11.3), it is blocking whereas os.urandom() should not block. */ + * Solaris 11.3), it is blocking whereas os.urandom() should not block. */ #elif defined(HAVE_GETENTROPY) && !defined(sun) #define PY_GETENTROPY 1 @@ -121,24 +121,20 @@ /* Call getrandom() - Return 1 on success - - Return 0 if getrandom() syscall is not available (fails with ENOSYS). + - Return 0 if getrandom() syscall is not available (fails with ENOSYS) + or if getrandom(GRND_NONBLOCK) fails with EAGAIN (blocking=0 and system + urandom not initialized yet) and raise=0. - Raise an exception (if raise is non-zero) and return -1 on error: getrandom() failed with EINTR and the Python signal handler raised an exception, or getrandom() failed with a different error. */ static int -py_getrandom(void *buffer, Py_ssize_t size, int raise) +py_getrandom(void *buffer, Py_ssize_t size, int blocking, int raise) { - /* Is getrandom() supported by the running kernel? - Need Linux kernel 3.17 or newer, or Solaris 11.3 or newer */ + /* Is getrandom() supported by the running kernel? Set to 0 if getrandom() + fails with ENOSYS. Need Linux kernel 3.17 or newer, or Solaris 11.3 + or newer */ static int getrandom_works = 1; - - /* getrandom() on Linux will block if called before the kernel has - initialized the urandom entropy pool. This will cause Python - to hang on startup if called very early in the boot process - - see https://bugs.python.org/issue26839. To avoid this, use the - GRND_NONBLOCK flag. */ - const int flags = GRND_NONBLOCK; - + int flags; char *dest; long n; @@ -146,6 +142,7 @@ return 0; } + flags = blocking ? 0 : GRND_NONBLOCK; dest = buffer; while (0 < size) { #ifdef sun @@ -185,15 +182,12 @@ getrandom_works = 0; return 0; } - if (errno == EAGAIN) { - /* If we failed with EAGAIN, the entropy pool was - uninitialized. In this case, we return failure to fall - back to reading from /dev/urandom. - Note: In this case the data read will not be random so - should not be used for cryptographic purposes. Retaining - the existing semantics for practical purposes. */ - getrandom_works = 0; + /* getrandom(GRND_NONBLOCK) fails with EAGAIN if the system urandom + is not initialiazed yet. For _PyRandom_Init(), we ignore their + error and fall back on reading /dev/urandom which never blocks, + even if the system urandom is not initialized yet. */ + if (errno == EAGAIN && !raise && !blocking) { return 0; } @@ -228,13 +222,13 @@ } urandom_cache = { -1 }; -/* Read 'size' random bytes from getrandom(). Fall back on reading from +/* Read 'size' random bytes from py_getrandom(). Fall back on reading from /dev/urandom if getrandom() is not available. Return 0 on success. Raise an exception (if raise is non-zero) and return -1 on error. */ static int -dev_urandom(char *buffer, Py_ssize_t size, int raise) +dev_urandom(char *buffer, Py_ssize_t size, int blocking, int raise) { int fd; Py_ssize_t n; @@ -245,7 +239,7 @@ assert(size > 0); #ifdef PY_GETRANDOM - res = py_getrandom(buffer, size, raise); + res = py_getrandom(buffer, size, blocking, raise); if (res < 0) { return -1; } @@ -381,7 +375,7 @@ syscall) - Don't release the GIL to call syscalls. */ static int -pyurandom(void *buffer, Py_ssize_t size, int raise) +pyurandom(void *buffer, Py_ssize_t size, int blocking, int raise) { if (size < 0) { if (raise) { @@ -400,7 +394,7 @@ #elif defined(PY_GETENTROPY) return py_getentropy(buffer, size, raise); #else - return dev_urandom(buffer, size, raise); + return dev_urandom(buffer, size, blocking, raise); #endif } @@ -408,11 +402,29 @@ number generator (RNG). It is suitable for most cryptographic purposes except long living private keys for asymmetric encryption. - Return 0 on success, raise an exception and return -1 on error. */ + On Linux 3.17 and newer, the getrandom() syscall is used in blocking mode: + block until the system urandom entropy pool is initialized (128 bits are + collected by the kernel). + + Return 0 on success. Raise an exception and return -1 on error. */ int _PyOS_URandom(void *buffer, Py_ssize_t size) { - return pyurandom(buffer, size, 1); + return pyurandom(buffer, size, 1, 1); +} + +/* Fill buffer with size pseudo-random bytes from the operating system random + number generator (RNG). It is not suitable for cryptographic purpose. + + On Linux 3.17 and newer (when getrandom() syscall is used), if the system + urandom is not initialized yet, the function returns "weak" entropy read + from /dev/urandom. + + Return 0 on success. Raise an exception and return -1 on error. */ +int +_PyOS_URandomNonblock(void *buffer, Py_ssize_t size) +{ + return pyurandom(buffer, size, 0, 1); } void @@ -456,8 +468,11 @@ int res; /* _PyRandom_Init() is called very early in the Python initialization - and so exceptions cannot be used (use raise=0). */ - res = pyurandom(secret, secret_size, 0); + and so exceptions cannot be used (use raise=0). + + _PyRandom_Init() must not block Python initialization: call + pyurandom() is non-blocking mode (blocking=0): see the PEP 524. */ + res = pyurandom(secret, secret_size, 0, 0); if (res < 0) { Py_FatalError("failed to get random numbers to initialize Python"); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:45:41 2016 From: python-checkins at python.org (christian.heimes) Date: Tue, 06 Sep 2016 23:45:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326798=3A_Hello_Wi?= =?utf-8?q?nndows=2C_my_old_friend=2E_I=27ve_come_to_fix_blake2_for_you?= Message-ID: <20160906234541.77964.98485.CF3E36AF@psf.io> https://hg.python.org/cpython/rev/afa5a16456ed changeset: 103173:afa5a16456ed user: Christian Heimes date: Wed Sep 07 01:45:22 2016 +0200 summary: Issue #26798: Hello Winndows, my old friend. I've come to fix blake2 for you again. files: PC/config.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/PC/config.c b/PC/config.c --- a/PC/config.c +++ b/PC/config.c @@ -23,6 +23,7 @@ extern PyObject* PyInit__sha1(void); extern PyObject* PyInit__sha256(void); extern PyObject* PyInit__sha512(void); +extern PyObject* PyInit__blake2(void); extern PyObject* PyInit_time(void); extern PyObject* PyInit__thread(void); #ifdef WIN32 @@ -96,6 +97,7 @@ {"_sha1", PyInit__sha1}, {"_sha256", PyInit__sha256}, {"_sha512", PyInit__sha512}, + {"_blake2", PyInit__blake2}, {"time", PyInit_time}, #ifdef WITH_THREAD {"_thread", PyInit__thread}, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:48:57 2016 From: python-checkins at python.org (zach.ware) Date: Tue, 06 Sep 2016 23:48:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_libpython*=2Edylib_to_?= =?utf-8?b?LntoZyxnaXR9aWdub3Jl?= Message-ID: <20160906234857.48933.93961.C1D31638@psf.io> https://hg.python.org/cpython/rev/8b56a2c71250 changeset: 103174:8b56a2c71250 user: Zachary Ware date: Tue Sep 06 16:46:22 2016 -0700 summary: Add libpython*.dylib to .{hg,git}ignore files: .gitignore | 1 + .hgignore | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -73,6 +73,7 @@ ipch/ libpython*.a libpython*.so* +libpython*.dylib platform pybuilddir.txt pyconfig.h diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -44,6 +44,7 @@ syntax: glob libpython*.a libpython*.so* +libpython*.dylib *.swp *.o *.pyc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:58:40 2016 From: python-checkins at python.org (gregory.p.smith) Date: Tue, 06 Sep 2016 23:58:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fixes_issue263?= =?utf-8?q?07=3A_The_profile-opt_build_now_applys_PGO_to_the_built-in?= Message-ID: <20160906235840.1366.8184.E6DEFE52@psf.io> https://hg.python.org/cpython/rev/7d9cd4a0d488 changeset: 103175:7d9cd4a0d488 branch: 3.5 parent: 103166:91d3022b3f03 user: Gregory P. Smith [Google Inc.] date: Tue Sep 06 23:56:54 2016 +0000 summary: Fixes issue26307: The profile-opt build now applys PGO to the built-in modules. files: Makefile.pre.in | 2 +- Misc/NEWS | 2 ++ 2 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1595,7 +1595,7 @@ -rm -f pybuilddir.txt -rm -f Lib/lib2to3/*Grammar*.pickle -rm -f Programs/_testembed Programs/_freeze_importlib - -rm -rf build + -find build -type f -a ! -name '*.gc??' -exec rm -f {} ';' profile-removal: find . -name '*.gc??' -exec rm -f {} ';' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #26307: The profile-opt build now applys PGO to the built-in modules. + - Issue #27812: Properly clear out a generator's frame's backreference to the generator to prevent crashes in frame.clear(). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 19:58:41 2016 From: python-checkins at python.org (gregory.p.smith) Date: Tue, 06 Sep 2016 23:58:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fixes_issue26307=3A_The_profile-opt_build_now_applys_PGO?= =?utf-8?q?_to_the_built-in_modules=2E?= Message-ID: <20160906235840.612.74764.B49FE583@psf.io> https://hg.python.org/cpython/rev/bdc7292cf87e changeset: 103176:bdc7292cf87e parent: 103174:8b56a2c71250 parent: 103175:7d9cd4a0d488 user: Gregory P. Smith [Google Inc.] date: Tue Sep 06 23:58:32 2016 +0000 summary: Fixes issue26307: The profile-opt build now applys PGO to the built-in modules. files: Makefile.pre.in | 2 +- Misc/NEWS | 2 ++ 2 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1593,7 +1593,7 @@ -rm -f pybuilddir.txt -rm -f Lib/lib2to3/*Grammar*.pickle -rm -f Programs/_testembed Programs/_freeze_importlib - -rm -rf build + -find build -type f -a ! -name '*.gc??' -exec rm -f {} ';' profile-removal: find . -name '*.gc??' -exec rm -f {} ';' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #26307: The profile-opt build now applys PGO to the built-in modules. + - Issue #27078: Added BUILD_STRING opcode. Optimized f-strings evaluation. - Issue #17884: Python now requires systems with inttypes.h and stdint.h -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:03:56 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 07 Sep 2016 00:03:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=5FPyUnicodeWriter=3A_asse?= =?utf-8?q?rt_that_max_character_=3C=3D_MAX=5FUNICODE?= Message-ID: <20160907000356.48898.44017.07E02B9C@psf.io> https://hg.python.org/cpython/rev/82017ba54447 changeset: 103177:82017ba54447 user: Victor Stinner date: Tue Sep 06 16:58:36 2016 -0700 summary: _PyUnicodeWriter: assert that max character <= MAX_UNICODE files: Objects/unicodeobject.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -13348,6 +13348,8 @@ Py_ssize_t newlen; PyObject *newbuffer; + assert(maxchar <= MAX_UNICODE); + /* ensure that the _PyUnicodeWriter_Prepare macro was used */ assert((maxchar > writer->maxchar && length >= 0) || length > 0); @@ -13441,6 +13443,7 @@ Py_LOCAL_INLINE(int) _PyUnicodeWriter_WriteCharInline(_PyUnicodeWriter *writer, Py_UCS4 ch) { + assert(ch <= MAX_UNICODE); if (_PyUnicodeWriter_Prepare(writer, 1, ch) < 0) return -1; PyUnicode_WRITE(writer->kind, writer->data, writer->pos, ch); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:03:57 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 07 Sep 2016 00:03:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327776=3A_include_?= =?utf-8?q?process=2Eh_on_Windows_for_getpid=28=29?= Message-ID: <20160907000356.563.58719.6103CAAF@psf.io> https://hg.python.org/cpython/rev/ebbfc053360a changeset: 103178:ebbfc053360a user: Victor Stinner date: Tue Sep 06 17:03:03 2016 -0700 summary: Issue #27776: include process.h on Windows for getpid() files: Modules/_randommodule.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Modules/_randommodule.c b/Modules/_randommodule.c --- a/Modules/_randommodule.c +++ b/Modules/_randommodule.c @@ -68,6 +68,9 @@ #include "Python.h" #include /* for seeding to current time */ +#ifdef HAVE_PROCESS_H +# include /* needed for getpid() */ +#endif /* Period parameters -- These are all magic. Don't change. */ #define N 624 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:08:13 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 07 Sep 2016 00:08:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Optimize_unicode=5Fescape_?= =?utf-8?q?and_raw=5Funicode=5Fescape?= Message-ID: <20160907000813.12540.55271.623F5858@psf.io> https://hg.python.org/cpython/rev/ad5a28ace615 changeset: 103179:ad5a28ace615 user: Victor Stinner date: Tue Sep 06 17:04:34 2016 -0700 summary: Optimize unicode_escape and raw_unicode_escape Issue #16334. Patch written by Serhiy Storchaka. files: Objects/unicodeobject.c | 758 ++++++++++++--------------- 1 files changed, 349 insertions(+), 409 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -5846,61 +5846,6 @@ /* --- Unicode Escape Codec ----------------------------------------------- */ -/* Helper function for PyUnicode_DecodeUnicodeEscape, determines - if all the escapes in the string make it still a valid ASCII string. - Returns -1 if any escapes were found which cause the string to - pop out of ASCII range. Otherwise returns the length of the - required buffer to hold the string. - */ -static Py_ssize_t -length_of_escaped_ascii_string(const char *s, Py_ssize_t size) -{ - const unsigned char *p = (const unsigned char *)s; - const unsigned char *end = p + size; - Py_ssize_t length = 0; - - if (size < 0) - return -1; - - for (; p < end; ++p) { - if (*p > 127) { - /* Non-ASCII */ - return -1; - } - else if (*p != '\\') { - /* Normal character */ - ++length; - } - else { - /* Backslash-escape, check next char */ - ++p; - /* Escape sequence reaches till end of string or - non-ASCII follow-up. */ - if (p >= end || *p > 127) - return -1; - switch (*p) { - case '\n': - /* backslash + \n result in zero characters */ - break; - case '\\': case '\'': case '\"': - case 'b': case 'f': case 't': - case 'n': case 'r': case 'v': case 'a': - ++length; - break; - case '0': case '1': case '2': case '3': - case '4': case '5': case '6': case '7': - case 'x': case 'u': case 'U': case 'N': - /* these do not guarantee ASCII characters */ - return -1; - default: - /* count the backslash + the other character */ - length += 2; - } - } - } - return length; -} - static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL; PyObject * @@ -5909,218 +5854,212 @@ const char *errors) { const char *starts = s; - Py_ssize_t startinpos; - Py_ssize_t endinpos; _PyUnicodeWriter writer; const char *end; - char* message; - Py_UCS4 chr = 0xffffffff; /* in case 'getcode' messes up */ PyObject *errorHandler = NULL; PyObject *exc = NULL; - Py_ssize_t len; - - len = length_of_escaped_ascii_string(s, size); - if (len == 0) + + if (size == 0) { _Py_RETURN_UNICODE_EMPTY(); - - /* After length_of_escaped_ascii_string() there are two alternatives, - either the string is pure ASCII with named escapes like \n, etc. - and we determined it's exact size (common case) - or it contains \x, \u, ... escape sequences. then we create a - legacy wchar string and resize it at the end of this function. */ + } + /* Escaped strings will always be longer than the resulting + Unicode string, so we start with size here and then reduce the + length after conversion to the true value. + (but if the error callback returns a long replacement string + we'll have to allocate more space) */ _PyUnicodeWriter_Init(&writer); - if (len > 0) { - writer.min_length = len; - } - else { - /* Escaped strings will always be longer than the resulting - Unicode string, so we start with size here and then reduce the - length after conversion to the true value. - (but if the error callback returns a long replacement string - we'll have to allocate more space) */ - writer.min_length = size; - } - - if (size == 0) - return _PyUnicodeWriter_Finish(&writer); + writer.min_length = size; + if (_PyUnicodeWriter_Prepare(&writer, size, 127) < 0) { + goto onError; + } + end = s + size; - while (s < end) { - unsigned char c; - Py_UCS4 x; - int digits; + unsigned char c = (unsigned char) *s++; + Py_UCS4 ch; + int count; + Py_ssize_t startinpos; + Py_ssize_t endinpos; + const char *message; + +#define WRITE_ASCII_CHAR(ch) \ + do { \ + assert(ch <= 127); \ + assert(writer.pos < writer.size); \ + PyUnicode_WRITE(writer.kind, writer.data, writer.pos++, ch); \ + } while(0) + +#define WRITE_CHAR(ch) \ + do { \ + if (ch <= writer.maxchar) { \ + assert(writer.pos < writer.size); \ + PyUnicode_WRITE(writer.kind, writer.data, writer.pos++, ch); \ + } \ + else if (_PyUnicodeWriter_WriteCharInline(&writer, ch) < 0) { \ + goto onError; \ + } \ + } while(0) /* Non-escape characters are interpreted as Unicode ordinals */ - if (*s != '\\') { - x = (unsigned char)*s; - s++; - if (_PyUnicodeWriter_WriteCharInline(&writer, x) < 0) - goto onError; + if (c != '\\') { + WRITE_CHAR(c); continue; } - startinpos = s-starts; + startinpos = s - starts - 1; /* \ - Escapes */ - s++; - c = *s++; - if (s > end) - c = '\0'; /* Invalid after \ */ - + if (s >= end) { + message = "\\ at end of string"; + goto error; + } + c = (unsigned char) *s++; + + assert(writer.pos < writer.size); switch (c) { /* \x escapes */ -#define WRITECHAR(ch) \ - do { \ - if (_PyUnicodeWriter_WriteCharInline(&writer, (ch)) < 0) \ - goto onError; \ - } while(0) - - case '\n': break; - case '\\': WRITECHAR('\\'); break; - case '\'': WRITECHAR('\''); break; - case '\"': WRITECHAR('\"'); break; - case 'b': WRITECHAR('\b'); break; + case '\n': continue; + case '\\': WRITE_ASCII_CHAR('\\'); continue; + case '\'': WRITE_ASCII_CHAR('\''); continue; + case '\"': WRITE_ASCII_CHAR('\"'); continue; + case 'b': WRITE_ASCII_CHAR('\b'); continue; /* FF */ - case 'f': WRITECHAR('\014'); break; - case 't': WRITECHAR('\t'); break; - case 'n': WRITECHAR('\n'); break; - case 'r': WRITECHAR('\r'); break; + case 'f': WRITE_ASCII_CHAR('\014'); continue; + case 't': WRITE_ASCII_CHAR('\t'); continue; + case 'n': WRITE_ASCII_CHAR('\n'); continue; + case 'r': WRITE_ASCII_CHAR('\r'); continue; /* VT */ - case 'v': WRITECHAR('\013'); break; + case 'v': WRITE_ASCII_CHAR('\013'); continue; /* BEL, not classic C */ - case 'a': WRITECHAR('\007'); break; + case 'a': WRITE_ASCII_CHAR('\007'); continue; /* \OOO (octal) escapes */ case '0': case '1': case '2': case '3': case '4': case '5': case '6': case '7': - x = s[-1] - '0'; + ch = c - '0'; if (s < end && '0' <= *s && *s <= '7') { - x = (x<<3) + *s++ - '0'; - if (s < end && '0' <= *s && *s <= '7') - x = (x<<3) + *s++ - '0'; - } - WRITECHAR(x); - break; + ch = (ch<<3) + *s++ - '0'; + if (s < end && '0' <= *s && *s <= '7') { + ch = (ch<<3) + *s++ - '0'; + } + } + WRITE_CHAR(ch); + continue; /* hex escapes */ /* \xXX */ case 'x': - digits = 2; + count = 2; message = "truncated \\xXX escape"; goto hexescape; /* \uXXXX */ case 'u': - digits = 4; + count = 4; message = "truncated \\uXXXX escape"; goto hexescape; /* \UXXXXXXXX */ case 'U': - digits = 8; + count = 8; message = "truncated \\UXXXXXXXX escape"; hexescape: - chr = 0; - if (end - s < digits) { - /* count only hex digits */ - for (; s < end; ++s) { - c = (unsigned char)*s; - if (!Py_ISXDIGIT(c)) - goto error; + for (ch = 0; count && s < end; ++s, --count) { + c = (unsigned char)*s; + ch <<= 4; + if (c >= '0' && c <= '9') { + ch += c - '0'; } + else if (c >= 'a' && c <= 'f') { + ch += c - ('a' - 10); + } + else if (c >= 'A' && c <= 'F') { + ch += c - ('A' - 10); + } + else { + break; + } + } + if (count) { goto error; } - for (; digits--; ++s) { - c = (unsigned char)*s; - if (!Py_ISXDIGIT(c)) - goto error; - chr = (chr<<4) & ~0xF; - if (c >= '0' && c <= '9') - chr += c - '0'; - else if (c >= 'a' && c <= 'f') - chr += 10 + c - 'a'; - else - chr += 10 + c - 'A'; - } - if (chr == 0xffffffff && PyErr_Occurred()) - /* _decoding_error will have already written into the - target buffer. */ - break; - store: - /* when we get here, chr is a 32-bit unicode character */ - message = "illegal Unicode character"; - if (chr > MAX_UNICODE) + + /* when we get here, ch is a 32-bit unicode character */ + if (ch > MAX_UNICODE) { + message = "illegal Unicode character"; goto error; - WRITECHAR(chr); - break; + } + + WRITE_CHAR(ch); + continue; /* \N{name} */ case 'N': - message = "malformed \\N character escape"; if (ucnhash_CAPI == NULL) { /* load the unicode data module */ ucnhash_CAPI = (_PyUnicode_Name_CAPI *)PyCapsule_Import( PyUnicodeData_CAPSULE_NAME, 1); - if (ucnhash_CAPI == NULL) - goto ucnhashError; - } + if (ucnhash_CAPI == NULL) { + PyErr_SetString( + PyExc_UnicodeError, + "\\N escapes not supported (can't load unicodedata module)" + ); + goto onError; + } + } + + message = "malformed \\N character escape"; if (*s == '{') { - const char *start = s+1; + const char *start = ++s; + size_t namelen; /* look for the closing brace */ - while (*s != '}' && s < end) + while (s < end && *s != '}') s++; - if (s > start && s < end && *s == '}') { + namelen = s - start; + if (namelen && s < end) { /* found a name. look it up in the unicode database */ + s++; + ch = 0xffffffff; /* in case 'getcode' messes up */ + if (namelen <= INT_MAX && + ucnhash_CAPI->getcode(NULL, start, (int)namelen, + &ch, 0)) { + assert(ch <= MAX_UNICODE); + WRITE_CHAR(ch); + continue; + } message = "unknown Unicode character name"; - s++; - if (s - start - 1 <= INT_MAX && - ucnhash_CAPI->getcode(NULL, start, (int)(s-start-1), - &chr, 0)) - goto store; } } goto error; default: - if (s > end) { - message = "\\ at end of string"; - s--; - goto error; - } - else { - WRITECHAR('\\'); - WRITECHAR((unsigned char)s[-1]); - } - break; - } - continue; + WRITE_ASCII_CHAR('\\'); + WRITE_CHAR(c); + continue; + } error: endinpos = s-starts; + writer.min_length = end - s + writer.pos; if (unicode_decode_call_errorhandler_writer( errors, &errorHandler, "unicodeescape", message, &starts, &end, &startinpos, &endinpos, &exc, &s, - &writer)) + &writer)) { goto onError; - continue; - } -#undef WRITECHAR + } + if (_PyUnicodeWriter_Prepare(&writer, writer.min_length, 127) < 0) { + goto onError; + } + +#undef WRITE_ASCII_CHAR +#undef WRITE_CHAR + } Py_XDECREF(errorHandler); Py_XDECREF(exc); return _PyUnicodeWriter_Finish(&writer); - ucnhashError: - PyErr_SetString( - PyExc_UnicodeError, - "\\N escapes not supported (can't load unicodedata module)" - ); - _PyUnicodeWriter_Dealloc(&writer); - Py_XDECREF(errorHandler); - Py_XDECREF(exc); - return NULL; - onError: _PyUnicodeWriter_Dealloc(&writer); Py_XDECREF(errorHandler); @@ -6139,10 +6078,11 @@ PyUnicode_AsUnicodeEscapeString(PyObject *unicode) { Py_ssize_t i, len; + PyObject *repr; char *p; - int kind; + enum PyUnicode_Kind kind; void *data; - _PyBytesWriter writer; + Py_ssize_t expandsize; /* Initial allocation is based on the longest-possible character escape. @@ -6156,119 +6096,101 @@ PyErr_BadArgument(); return NULL; } - if (PyUnicode_READY(unicode) == -1) - return NULL; - - _PyBytesWriter_Init(&writer); + if (PyUnicode_READY(unicode) == -1) { + return NULL; + } len = PyUnicode_GET_LENGTH(unicode); + if (len == 0) { + return PyBytes_FromStringAndSize(NULL, 0); + } + kind = PyUnicode_KIND(unicode); data = PyUnicode_DATA(unicode); - - p = _PyBytesWriter_Alloc(&writer, len); - if (p == NULL) - goto error; - writer.overallocate = 1; - + /* 4 byte characters can take up 10 bytes, 2 byte characters can take up 6 + bytes, and 1 byte characters 4. */ + expandsize = kind * 2 + 2; + if (len > (PY_SSIZE_T_MAX - 2 - 1) / expandsize) { + return PyErr_NoMemory(); + } + repr = PyBytes_FromStringAndSize(NULL, 2 + expandsize * len + 1); + if (repr == NULL) { + return NULL; + } + + p = PyBytes_AS_STRING(repr); for (i = 0; i < len; i++) { Py_UCS4 ch = PyUnicode_READ(kind, data, i); - /* Escape backslashes */ - if (ch == '\\') { - /* -1: subtract 1 preallocated byte */ - p = _PyBytesWriter_Prepare(&writer, p, 2-1); - if (p == NULL) - goto error; - + /* U+0000-U+00ff range */ + if (ch < 0x100) { + if (ch >= ' ' && ch < 127) { + if (ch != '\\') { + /* Copy printable US ASCII as-is */ + *p++ = (char) ch; + } + /* Escape backslashes */ + else { + *p++ = '\\'; + *p++ = '\\'; + } + } + + /* Map special whitespace to '\t', \n', '\r' */ + else if (ch == '\t') { + *p++ = '\\'; + *p++ = 't'; + } + else if (ch == '\n') { + *p++ = '\\'; + *p++ = 'n'; + } + else if (ch == '\r') { + *p++ = '\\'; + *p++ = 'r'; + } + + /* Map non-printable US ASCII and 8-bit characters to '\xHH' */ + else { + *p++ = '\\'; + *p++ = 'x'; + *p++ = Py_hexdigits[(ch >> 4) & 0x000F]; + *p++ = Py_hexdigits[ch & 0x000F]; + } + } + /* U+0000-U+00ff range: Map 16-bit characters to '\uHHHH' */ + else if (ch < 0x10000) { + /* U+0100-U+ffff */ *p++ = '\\'; - *p++ = (char) ch; - continue; - } - - /* Map 21-bit characters to '\U00xxxxxx' */ - else if (ch >= 0x10000) { - assert(ch <= MAX_UNICODE); - - p = _PyBytesWriter_Prepare(&writer, p, 10-1); - if (p == NULL) - goto error; - + *p++ = 'u'; + *p++ = Py_hexdigits[(ch >> 12) & 0x000F]; + *p++ = Py_hexdigits[(ch >> 8) & 0x000F]; + *p++ = Py_hexdigits[(ch >> 4) & 0x000F]; + *p++ = Py_hexdigits[ch & 0x000F]; + } + /* U+010000-U+10ffff range: Map 21-bit characters to '\U00HHHHHH' */ + else { + + /* Make sure that the first two digits are zero */ + assert(ch <= MAX_UNICODE && MAX_UNICODE <= 0x10ffff); *p++ = '\\'; *p++ = 'U'; - *p++ = Py_hexdigits[(ch >> 28) & 0x0000000F]; - *p++ = Py_hexdigits[(ch >> 24) & 0x0000000F]; + *p++ = '0'; + *p++ = '0'; *p++ = Py_hexdigits[(ch >> 20) & 0x0000000F]; *p++ = Py_hexdigits[(ch >> 16) & 0x0000000F]; *p++ = Py_hexdigits[(ch >> 12) & 0x0000000F]; *p++ = Py_hexdigits[(ch >> 8) & 0x0000000F]; *p++ = Py_hexdigits[(ch >> 4) & 0x0000000F]; *p++ = Py_hexdigits[ch & 0x0000000F]; - continue; - } - - /* Map 16-bit characters to '\uxxxx' */ - if (ch >= 256) { - p = _PyBytesWriter_Prepare(&writer, p, 6-1); - if (p == NULL) - goto error; - - *p++ = '\\'; - *p++ = 'u'; - *p++ = Py_hexdigits[(ch >> 12) & 0x000F]; - *p++ = Py_hexdigits[(ch >> 8) & 0x000F]; - *p++ = Py_hexdigits[(ch >> 4) & 0x000F]; - *p++ = Py_hexdigits[ch & 0x000F]; - } - - /* Map special whitespace to '\t', \n', '\r' */ - else if (ch == '\t') { - p = _PyBytesWriter_Prepare(&writer, p, 2-1); - if (p == NULL) - goto error; - - *p++ = '\\'; - *p++ = 't'; - } - else if (ch == '\n') { - p = _PyBytesWriter_Prepare(&writer, p, 2-1); - if (p == NULL) - goto error; - - *p++ = '\\'; - *p++ = 'n'; - } - else if (ch == '\r') { - p = _PyBytesWriter_Prepare(&writer, p, 2-1); - if (p == NULL) - goto error; - - *p++ = '\\'; - *p++ = 'r'; - } - - /* Map non-printable US ASCII to '\xhh' */ - else if (ch < ' ' || ch >= 0x7F) { - /* -1: subtract 1 preallocated byte */ - p = _PyBytesWriter_Prepare(&writer, p, 4-1); - if (p == NULL) - goto error; - - *p++ = '\\'; - *p++ = 'x'; - *p++ = Py_hexdigits[(ch >> 4) & 0x000F]; - *p++ = Py_hexdigits[ch & 0x000F]; - } - - /* Copy everything else as-is */ - else - *p++ = (char) ch; - } - - return _PyBytesWriter_Finish(&writer, p); - -error: - _PyBytesWriter_Dealloc(&writer); - return NULL; + } + } + + assert(p - PyBytes_AS_STRING(repr) > 0); + if (_PyBytes_Resize(&repr, p - PyBytes_AS_STRING(repr)) < 0) { + return NULL; + } + return repr; } PyObject * @@ -6277,8 +6199,10 @@ { PyObject *result; PyObject *tmp = PyUnicode_FromUnicode(s, size); - if (tmp == NULL) - return NULL; + if (tmp == NULL) { + return NULL; + } + result = PyUnicode_AsUnicodeEscapeString(tmp); Py_DECREF(tmp); return result; @@ -6292,95 +6216,107 @@ const char *errors) { const char *starts = s; - Py_ssize_t startinpos; - Py_ssize_t endinpos; _PyUnicodeWriter writer; const char *end; - const char *bs; PyObject *errorHandler = NULL; PyObject *exc = NULL; - if (size == 0) + if (size == 0) { _Py_RETURN_UNICODE_EMPTY(); + } /* Escaped strings will always be longer than the resulting Unicode string, so we start with size here and then reduce the length after conversion to the true value. (But decoding error handler might have to resize the string) */ _PyUnicodeWriter_Init(&writer); - writer.min_length = size; + writer.min_length = size; + if (_PyUnicodeWriter_Prepare(&writer, size, 127) < 0) { + goto onError; + } end = s + size; while (s < end) { - unsigned char c; - Py_UCS4 x; - int i; + unsigned char c = (unsigned char) *s++; + Py_UCS4 ch; int count; + Py_ssize_t startinpos; + Py_ssize_t endinpos; + const char *message; + +#define WRITE_CHAR(ch) \ + do { \ + if (ch <= writer.maxchar) { \ + assert(writer.pos < writer.size); \ + PyUnicode_WRITE(writer.kind, writer.data, writer.pos++, ch); \ + } \ + else if (_PyUnicodeWriter_WriteCharInline(&writer, ch) < 0) { \ + goto onError; \ + } \ + } while(0) /* Non-escape characters are interpreted as Unicode ordinals */ - if (*s != '\\') { - x = (unsigned char)*s++; - if (_PyUnicodeWriter_WriteCharInline(&writer, x) < 0) - goto onError; + if (c != '\\' || s >= end) { + WRITE_CHAR(c); continue; } - startinpos = s-starts; - - /* \u-escapes are only interpreted iff the number of leading - backslashes if odd */ - bs = s; - for (;s < end;) { - if (*s != '\\') + + c = (unsigned char) *s++; + if (c == 'u') { + count = 4; + message = "truncated \\uXXXX escape"; + } + else if (c == 'U') { + count = 8; + message = "truncated \\UXXXXXXXX escape"; + } + else { + assert(writer.pos < writer.size); + PyUnicode_WRITE(writer.kind, writer.data, writer.pos++, '\\'); + WRITE_CHAR(c); + continue; + } + startinpos = s - starts - 2; + + /* \uHHHH with 4 hex digits, \U00HHHHHH with 8 */ + for (ch = 0; count && s < end; ++s, --count) { + c = (unsigned char)*s; + ch <<= 4; + if (c >= '0' && c <= '9') { + ch += c - '0'; + } + else if (c >= 'a' && c <= 'f') { + ch += c - ('a' - 10); + } + else if (c >= 'A' && c <= 'F') { + ch += c - ('A' - 10); + } + else { break; - x = (unsigned char)*s++; - if (_PyUnicodeWriter_WriteCharInline(&writer, x) < 0) - goto onError; - } - if (((s - bs) & 1) == 0 || - s >= end || - (*s != 'u' && *s != 'U')) { - continue; - } - writer.pos--; - count = *s=='u' ? 4 : 8; - s++; - - /* \uXXXX with 4 hex digits, \Uxxxxxxxx with 8 */ - for (x = 0, i = 0; i < count; ++i, ++s) { - c = (unsigned char)*s; - if (!Py_ISXDIGIT(c)) { - endinpos = s-starts; - if (unicode_decode_call_errorhandler_writer( - errors, &errorHandler, - "rawunicodeescape", "truncated \\uXXXX", - &starts, &end, &startinpos, &endinpos, &exc, &s, - &writer)) - goto onError; - goto nextByte; - } - x = (x<<4) & ~0xF; - if (c >= '0' && c <= '9') - x += c - '0'; - else if (c >= 'a' && c <= 'f') - x += 10 + c - 'a'; - else - x += 10 + c - 'A'; - } - if (x <= MAX_UNICODE) { - if (_PyUnicodeWriter_WriteCharInline(&writer, x) < 0) - goto onError; - } - else { - endinpos = s-starts; - if (unicode_decode_call_errorhandler_writer( - errors, &errorHandler, - "rawunicodeescape", "\\Uxxxxxxxx out of range", - &starts, &end, &startinpos, &endinpos, &exc, &s, - &writer)) - goto onError; - } - nextByte: - ; + } + } + if (!count) { + if (ch <= MAX_UNICODE) { + WRITE_CHAR(ch); + continue; + } + message = "\\Uxxxxxxxx out of range"; + } + + endinpos = s-starts; + writer.min_length = end - s + writer.pos; + if (unicode_decode_call_errorhandler_writer( + errors, &errorHandler, + "rawunicodeescape", message, + &starts, &end, &startinpos, &endinpos, &exc, &s, + &writer)) { + goto onError; + } + if (_PyUnicodeWriter_Prepare(&writer, writer.min_length, 127) < 0) { + goto onError; + } + +#undef WRITE_CHAR } Py_XDECREF(errorHandler); Py_XDECREF(exc); @@ -6391,52 +6327,73 @@ Py_XDECREF(errorHandler); Py_XDECREF(exc); return NULL; + } PyObject * PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode) { + PyObject *repr; char *p; - Py_ssize_t pos; + Py_ssize_t expandsize, pos; int kind; void *data; Py_ssize_t len; - _PyBytesWriter writer; if (!PyUnicode_Check(unicode)) { PyErr_BadArgument(); return NULL; } - if (PyUnicode_READY(unicode) == -1) - return NULL; - - _PyBytesWriter_Init(&writer); - + if (PyUnicode_READY(unicode) == -1) { + return NULL; + } kind = PyUnicode_KIND(unicode); data = PyUnicode_DATA(unicode); len = PyUnicode_GET_LENGTH(unicode); - - p = _PyBytesWriter_Alloc(&writer, len); - if (p == NULL) - goto error; - writer.overallocate = 1; - + if (kind == PyUnicode_1BYTE_KIND) { + return PyBytes_FromStringAndSize(data, len); + } + + /* 4 byte characters can take up 10 bytes, 2 byte characters can take up 6 + bytes, and 1 byte characters 4. */ + expandsize = kind * 2 + 2; + + if (len > PY_SSIZE_T_MAX / expandsize) { + return PyErr_NoMemory(); + } + repr = PyBytes_FromStringAndSize(NULL, expandsize * len); + if (repr == NULL) { + return NULL; + } + if (len == 0) { + return repr; + } + + p = PyBytes_AS_STRING(repr); for (pos = 0; pos < len; pos++) { Py_UCS4 ch = PyUnicode_READ(kind, data, pos); - /* Map 32-bit characters to '\Uxxxxxxxx' */ - if (ch >= 0x10000) { - assert(ch <= MAX_UNICODE); - - /* -1: subtract 1 preallocated byte */ - p = _PyBytesWriter_Prepare(&writer, p, 10-1); - if (p == NULL) - goto error; - + + /* U+0000-U+00ff range: Copy 8-bit characters as-is */ + if (ch < 0x100) { + *p++ = (char) ch; + } + /* U+0000-U+00ff range: Map 16-bit characters to '\uHHHH' */ + else if (ch < 0x10000) { + *p++ = '\\'; + *p++ = 'u'; + *p++ = Py_hexdigits[(ch >> 12) & 0xf]; + *p++ = Py_hexdigits[(ch >> 8) & 0xf]; + *p++ = Py_hexdigits[(ch >> 4) & 0xf]; + *p++ = Py_hexdigits[ch & 15]; + } + /* U+010000-U+10ffff range: Map 32-bit characters to '\U00HHHHHH' */ + else { + assert(ch <= MAX_UNICODE && MAX_UNICODE <= 0x10ffff); *p++ = '\\'; *p++ = 'U'; - *p++ = Py_hexdigits[(ch >> 28) & 0xf]; - *p++ = Py_hexdigits[(ch >> 24) & 0xf]; + *p++ = '0'; + *p++ = '0'; *p++ = Py_hexdigits[(ch >> 20) & 0xf]; *p++ = Py_hexdigits[(ch >> 16) & 0xf]; *p++ = Py_hexdigits[(ch >> 12) & 0xf]; @@ -6444,30 +6401,13 @@ *p++ = Py_hexdigits[(ch >> 4) & 0xf]; *p++ = Py_hexdigits[ch & 15]; } - /* Map 16-bit characters to '\uxxxx' */ - else if (ch >= 256) { - /* -1: subtract 1 preallocated byte */ - p = _PyBytesWriter_Prepare(&writer, p, 6-1); - if (p == NULL) - goto error; - - *p++ = '\\'; - *p++ = 'u'; - *p++ = Py_hexdigits[(ch >> 12) & 0xf]; - *p++ = Py_hexdigits[(ch >> 8) & 0xf]; - *p++ = Py_hexdigits[(ch >> 4) & 0xf]; - *p++ = Py_hexdigits[ch & 15]; - } - /* Copy everything else as-is */ - else - *p++ = (char) ch; - } - - return _PyBytesWriter_Finish(&writer, p); - -error: - _PyBytesWriter_Dealloc(&writer); - return NULL; + } + + assert(p > PyBytes_AS_STRING(repr)); + if (_PyBytes_Resize(&repr, p - PyBytes_AS_STRING(repr)) < 0) { + return NULL; + } + return repr; } PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:13:07 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 07 Sep 2016 00:13:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326359=3A_Add_the_?= =?utf-8?q?--with-optimizations_configure_flag=2E?= Message-ID: <20160907001307.38926.61168.494529FC@psf.io> https://hg.python.org/cpython/rev/b4b73473ecc6 changeset: 103180:b4b73473ecc6 user: Brett Cannon date: Tue Sep 06 17:12:40 2016 -0700 summary: Issue #26359: Add the --with-optimizations configure flag. The flag will activate LTO and PGO build support when available. Thanks to Alecsandur Patrascu of Intel for the original patch. files: Doc/whatsnew/3.6.rst | 4 + Makefile.pre.in | 8 +- Misc/NEWS | 3 + configure | 69 +++++++++++++++++++++++++------ configure.ac | 41 +++++++++++++++++++ 5 files changed, 108 insertions(+), 17 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -796,6 +796,10 @@ Build and C API Changes ======================= +* The ``--with-optimizations`` configure flag has been added. Turning it on + will activate LTO and PGO build support (when available). + (Original patch by Alecsandru Patrascu of Intel in :issue:`26539`.) + * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data failed (:issue:`5319`). diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -459,7 +459,7 @@ # Rules # Default target -all: build_all +all: @DEF_MAKE_ALL_RULE@ build_all: $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks Programs/_testembed python-config # Compile a binary with profile guided optimization. @@ -483,7 +483,7 @@ $(MAKE) profile-removal build_all_generate_profile: - $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" + $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" run_profile_task: : # FIXME: can't run for a cross build @@ -493,14 +493,14 @@ $(LLVM_PROF_MERGER) build_all_use_profile: - $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@" + $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@" # Compile and run with gcov .PHONY=coverage coverage-lcov coverage-report coverage: @echo "Building with support for coverage checking:" $(MAKE) clean profile-removal - $(MAKE) all CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov" + $(MAKE) @DEF_MAKE_RULE@ CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov" coverage-lcov: @echo "Creating Coverage HTML report with LCOV:" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -219,6 +219,9 @@ Build ----- +- Issue #26539: Add the --with-optimizations flag to turn on LTO and PGO build + support when available. + - Issue #27917: Set platform triplets for Android builds. - Issue #25825: Update references to the $(LIBPL) installation path on AIX. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -674,6 +674,8 @@ PGO_PROF_USE_FLAG PGO_PROF_GEN_FLAG LTOFLAGS +DEF_MAKE_RULE +DEF_MAKE_ALL_RULE ABIFLAGS LN MKDIR_P @@ -775,7 +777,6 @@ docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir @@ -809,6 +810,7 @@ enable_shared enable_profiling with_pydebug +with_optimizations with_lto with_hash_algorithm with_address_sanitizer @@ -886,7 +888,6 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1139,15 +1140,6 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1285,7 +1277,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1438,7 +1430,6 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -1501,6 +1492,8 @@ compiler --with-suffix=.exe set executable suffix --with-pydebug build with Py_DEBUG defined + --with-optimizations Enable all optimizations when available (LTO, PGO, + etc). Disabled by default. --with-lto Enable Link Time Optimization in PGO builds. Disabled by default. --with-hash-algorithm=[fnv|siphash24] @@ -6458,6 +6451,46 @@ fi +# Enable optimization flags + + +Py_OPT='false' +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-optimizations" >&5 +$as_echo_n "checking for --with-optimizations... " >&6; } + +# Check whether --with-optimizations was given. +if test "${with_optimizations+set}" = set; then : + withval=$with_optimizations; +if test "$withval" != no +then + Py_OPT='true' + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; }; +else + Py_OPT='false' + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; }; +fi +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + +if test "$Py_OPT" = 'true' ; then + Py_LTO='true' + case $ac_sys_system in + Darwin*) + # At least on macOS El Capitan, LTO does not work with PGO. + Py_LTO='false' + ;; + esac + DEF_MAKE_ALL_RULE="profile-opt" + DEF_MAKE_RULE="build_all" +else + DEF_MAKE_ALL_RULE="build_all" + DEF_MAKE_RULE="all" +fi + # Enable LTO flags { $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-lto" >&5 @@ -17581,3 +17614,13 @@ -s Modules Modules/Setup.config \ Modules/Setup.local Modules/Setup mv config.c Modules + +if test "$Py_OPT" = 'false' -a "$Py_DEBUG" != 'true'; then + echo "" >&6 + echo "" >&6 + echo "If you want a release build with all optimizations active (LTO, PGO, etc)," + echo "please run ./configure --with-optimizations" >&6 + echo "" >&6 + echo "" >&6 +fi + diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1282,6 +1282,37 @@ fi], [AC_MSG_RESULT(no)]) +# Enable optimization flags +AC_SUBST(DEF_MAKE_ALL_RULE) +AC_SUBST(DEF_MAKE_RULE) +Py_OPT='false' +AC_MSG_CHECKING(for --with-optimizations) +AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable all optimizations when available (LTO, PGO, etc). Disabled by default.]), +[ +if test "$withval" != no +then + Py_OPT='true' + AC_MSG_RESULT(yes); +else + Py_OPT='false' + AC_MSG_RESULT(no); +fi], +[AC_MSG_RESULT(no)]) +if test "$Py_OPT" = 'true' ; then + Py_LTO='true' + case $ac_sys_system in + Darwin*) + # At least on macOS El Capitan, LTO does not work with PGO. + Py_LTO='false' + ;; + esac + DEF_MAKE_ALL_RULE="profile-opt" + DEF_MAKE_RULE="build_all" +else + DEF_MAKE_ALL_RULE="build_all" + DEF_MAKE_RULE="all" +fi + # Enable LTO flags AC_SUBST(LTOFLAGS) AC_MSG_CHECKING(for --with-lto) @@ -5337,3 +5368,13 @@ -s Modules Modules/Setup.config \ Modules/Setup.local Modules/Setup mv config.c Modules + +if test "$Py_OPT" = 'false' -a "$Py_DEBUG" != 'true'; then + echo "" >&AS_MESSAGE_FD + echo "" >&AS_MESSAGE_FD + echo "If you want a release build with all optimizations active (LTO, PGO, etc)," + echo "please run ./configure --with-optimizations" >&AS_MESSAGE_FD + echo "" >&AS_MESSAGE_FD + echo "" >&AS_MESSAGE_FD +fi + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:15:59 2016 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 07 Sep 2016 00:15:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318844=3A_Add_rand?= =?utf-8?b?b20ud2VpZ2h0ZWRfY2hvaWNlcygp?= Message-ID: <20160907001555.48870.64314.9BB51604@psf.io> https://hg.python.org/cpython/rev/a5856153d942 changeset: 103181:a5856153d942 user: Raymond Hettinger date: Tue Sep 06 17:15:29 2016 -0700 summary: Issue #18844: Add random.weighted_choices() files: Doc/library/random.rst | 21 ++++++++ Lib/random.py | 28 +++++++++++- Lib/test/test_random.py | 68 +++++++++++++++++++++++++++++ Misc/NEWS | 2 + 4 files changed, 118 insertions(+), 1 deletions(-) diff --git a/Doc/library/random.rst b/Doc/library/random.rst --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -124,6 +124,27 @@ Return a random element from the non-empty sequence *seq*. If *seq* is empty, raises :exc:`IndexError`. +.. function:: weighted_choices(k, population, weights=None, *, cum_weights=None) + + Return a *k* sized list of elements chosen from the *population* with replacement. + If the *population* is empty, raises :exc:`IndexError`. + + If a *weights* sequence is specified, selections are made according to the + relative weights. Alternatively, if a *cum_weights* sequence is given, the + selections are made according to the cumulative weights. For example, the + relative weights ``[10, 5, 30, 5]`` are equivalent to the cumulative + weights ``[10, 15, 45, 50]``. Internally, the relative weights are + converted to cumulative weights before making selections, so supplying the + cumulative weights saves work. + + If neither *weights* nor *cum_weights* are specified, selections are made + with equal probability. If a weights sequence is supplied, it must be + the same length as the *population* sequence. It is a :exc:`TypeError` + to specify both *weights* and *cum_weights*. + + The *weights* or *cum_weights* can use any numeric type that interoperates + with the :class:`float` values returned by :func:`random` (that includes + integers, floats, and fractions but excludes decimals). .. function:: shuffle(x[, random]) diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -8,6 +8,7 @@ --------- pick random element pick random sample + pick weighted random sample generate random permutation distributions on the real line: @@ -43,12 +44,14 @@ from os import urandom as _urandom from _collections_abc import Set as _Set, Sequence as _Sequence from hashlib import sha512 as _sha512 +import itertools as _itertools +import bisect as _bisect __all__ = ["Random","seed","random","uniform","randint","choice","sample", "randrange","shuffle","normalvariate","lognormvariate", "expovariate","vonmisesvariate","gammavariate","triangular", "gauss","betavariate","paretovariate","weibullvariate", - "getstate","setstate", "getrandbits", + "getstate","setstate", "getrandbits", "weighted_choices", "SystemRandom"] NV_MAGICCONST = 4 * _exp(-0.5)/_sqrt(2.0) @@ -334,6 +337,28 @@ result[i] = population[j] return result + def weighted_choices(self, k, population, weights=None, *, cum_weights=None): + """Return a k sized list of population elements chosen with replacement. + + If the relative weights or cumulative weights are not specified, + the selections are made with equal probability. + + """ + if cum_weights is None: + if weights is None: + choice = self.choice + return [choice(population) for i in range(k)] + else: + cum_weights = list(_itertools.accumulate(weights)) + elif weights is not None: + raise TypeError('Cannot specify both weights and cumulative_weights') + if len(cum_weights) != len(population): + raise ValueError('The number of weights does not match the population') + bisect = _bisect.bisect + random = self.random + total = cum_weights[-1] + return [population[bisect(cum_weights, random() * total)] for i in range(k)] + ## -------------------- real-valued distributions ------------------- ## -------------------- uniform distribution ------------------- @@ -724,6 +749,7 @@ randrange = _inst.randrange sample = _inst.sample shuffle = _inst.shuffle +weighted_choices = _inst.weighted_choices normalvariate = _inst.normalvariate lognormvariate = _inst.lognormvariate expovariate = _inst.expovariate diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py --- a/Lib/test/test_random.py +++ b/Lib/test/test_random.py @@ -7,6 +7,7 @@ from functools import partial from math import log, exp, pi, fsum, sin from test import support +from fractions import Fraction class TestBasicOps: # Superclass with tests common to all generators. @@ -141,6 +142,73 @@ def test_sample_on_dicts(self): self.assertRaises(TypeError, self.gen.sample, dict.fromkeys('abcdef'), 2) + def test_weighted_choices(self): + weighted_choices = self.gen.weighted_choices + data = ['red', 'green', 'blue', 'yellow'] + str_data = 'abcd' + range_data = range(4) + set_data = set(range(4)) + + # basic functionality + for sample in [ + weighted_choices(5, data), + weighted_choices(5, data, range(4)), + weighted_choices(k=5, population=data, weights=range(4)), + weighted_choices(k=5, population=data, cum_weights=range(4)), + ]: + self.assertEqual(len(sample), 5) + self.assertEqual(type(sample), list) + self.assertTrue(set(sample) <= set(data)) + + # test argument handling + with self.assertRaises(TypeError): # missing arguments + weighted_choices(2) + + self.assertEqual(weighted_choices(0, data), []) # k == 0 + self.assertEqual(weighted_choices(-1, data), []) # negative k behaves like ``[0] * -1`` + with self.assertRaises(TypeError): + weighted_choices(2.5, data) # k is a float + + self.assertTrue(set(weighted_choices(5, str_data)) <= set(str_data)) # population is a string sequence + self.assertTrue(set(weighted_choices(5, range_data)) <= set(range_data)) # population is a range + with self.assertRaises(TypeError): + weighted_choices(2.5, set_data) # population is not a sequence + + self.assertTrue(set(weighted_choices(5, data, None)) <= set(data)) # weights is None + self.assertTrue(set(weighted_choices(5, data, weights=None)) <= set(data)) + with self.assertRaises(ValueError): + weighted_choices(5, data, [1,2]) # len(weights) != len(population) + with self.assertRaises(IndexError): + weighted_choices(5, data, [0]*4) # weights sum to zero + with self.assertRaises(TypeError): + weighted_choices(5, data, 10) # non-iterable weights + with self.assertRaises(TypeError): + weighted_choices(5, data, [None]*4) # non-numeric weights + for weights in [ + [15, 10, 25, 30], # integer weights + [15.1, 10.2, 25.2, 30.3], # float weights + [Fraction(1, 3), Fraction(2, 6), Fraction(3, 6), Fraction(4, 6)], # fractional weights + [True, False, True, False] # booleans (include / exclude) + ]: + self.assertTrue(set(weighted_choices(5, data, weights)) <= set(data)) + + with self.assertRaises(ValueError): + weighted_choices(5, data, cum_weights=[1,2]) # len(weights) != len(population) + with self.assertRaises(IndexError): + weighted_choices(5, data, cum_weights=[0]*4) # cum_weights sum to zero + with self.assertRaises(TypeError): + weighted_choices(5, data, cum_weights=10) # non-iterable cum_weights + with self.assertRaises(TypeError): + weighted_choices(5, data, cum_weights=[None]*4) # non-numeric cum_weights + with self.assertRaises(TypeError): + weighted_choices(5, data, range(4), cum_weights=range(4)) # both weights and cum_weights + for weights in [ + [15, 10, 25, 30], # integer cum_weights + [15.1, 10.2, 25.2, 30.3], # float cum_weights + [Fraction(1, 3), Fraction(2, 6), Fraction(3, 6), Fraction(4, 6)], # fractional cum_weights + ]: + self.assertTrue(set(weighted_choices(5, data, cum_weights=weights)) <= set(data)) + def test_gauss(self): # Ensure that the seed() method initializes all the hidden state. In # particular, through 2.2.1 it failed to reset a piece of state used diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -101,6 +101,8 @@ - Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name fields in X.509 certs. +- Issue #18844: Add random.weighted_choices(). + - Issue #25761: Improved error reporting about truncated pickle data in C implementation of unpickler. UnpicklingError is now raised instead of AttributeError and ValueError in some cases. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:16:59 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 07 Sep 2016 00:16:59 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MzU5?= =?utf-8?q?=3A_Add_the_--with-optimizations_flag=2E?= Message-ID: <20160907001659.78392.473.97BFAA14@psf.io> https://hg.python.org/cpython/rev/1e61cc86df03 changeset: 103182:1e61cc86df03 branch: 3.5 parent: 103175:7d9cd4a0d488 user: Brett Cannon date: Tue Sep 06 17:15:21 2016 -0700 summary: Issue #26359: Add the --with-optimizations flag. files: Makefile.pre.in | 8 ++-- Misc/NEWS | 2 + configure | 69 ++++++++++++++++++++++++++++++------ configure.ac | 41 +++++++++++++++++++++ 4 files changed, 103 insertions(+), 17 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -481,7 +481,7 @@ # Rules # Default target -all: build_all +all: @DEF_MAKE_ALL_RULE@ build_all: $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks Programs/_testembed python-config # Compile a binary with profile guided optimization. @@ -505,7 +505,7 @@ $(MAKE) profile-removal build_all_generate_profile: - $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" + $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" run_profile_task: : # FIXME: can't run for a cross build @@ -515,14 +515,14 @@ $(LLVM_PROF_MERGER) build_all_use_profile: - $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@" + $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@" # Compile and run with gcov .PHONY=coverage coverage-lcov coverage-report coverage: @echo "Building with support for coverage checking:" $(MAKE) clean profile-removal - $(MAKE) all CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov" + $(MAKE) @DEF_MAKE_RULE@ CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov" coverage-lcov: @echo "Creating Coverage HTML report with LCOV:" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -256,6 +256,8 @@ Build ----- +- Issue #26359: Add the --with-optimizations configure flag. + - Issue #27713: Suppress spurious build warnings when updating importlib's bootstrap files. Patch by Xiang Zhang diff --git a/configure b/configure --- a/configure +++ b/configure @@ -674,6 +674,8 @@ PGO_PROF_USE_FLAG PGO_PROF_GEN_FLAG LTOFLAGS +DEF_MAKE_RULE +DEF_MAKE_ALL_RULE ABIFLAGS LN MKDIR_P @@ -774,7 +776,6 @@ docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir @@ -808,6 +809,7 @@ enable_shared enable_profiling with_pydebug +with_optimizations with_lto with_hash_algorithm with_address_sanitizer @@ -885,7 +887,6 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1138,15 +1139,6 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1284,7 +1276,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1437,7 +1429,6 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -1500,6 +1491,8 @@ compiler --with-suffix=.exe set executable suffix --with-pydebug build with Py_DEBUG defined + --with-optimizations Enable all optimizations when available (LTO, PGO, + etc). Disabled by default. --with-lto Enable Link Time Optimization in PGO builds. Disabled by default. --with-hash-algorithm=[fnv|siphash24] @@ -6528,6 +6521,46 @@ fi +# Enable optimization flags + + +Py_OPT='false' +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-optimizations" >&5 +$as_echo_n "checking for --with-optimizations... " >&6; } + +# Check whether --with-optimizations was given. +if test "${with_optimizations+set}" = set; then : + withval=$with_optimizations; +if test "$withval" != no +then + Py_OPT='true' + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; }; +else + Py_OPT='false' + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; }; +fi +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + +if test "$Py_OPT" = 'true' ; then + Py_LTO='true' + case $ac_sys_system in + Darwin*) + # At least on macOS El Capitan, LTO does not work with PGO. + Py_LTO='false' + ;; + esac + DEF_MAKE_ALL_RULE="profile-opt" + DEF_MAKE_RULE="build_all" +else + DEF_MAKE_ALL_RULE="build_all" + DEF_MAKE_RULE="all" +fi + # Enable LTO flags { $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-lto" >&5 @@ -17636,3 +17669,13 @@ -s Modules Modules/Setup.config \ Modules/Setup.local Modules/Setup mv config.c Modules + +if test "$Py_OPT" = 'false' -a "$Py_DEBUG" != 'true'; then + echo "" >&6 + echo "" >&6 + echo "If you want a release build with all optimizations active (LTO, PGO, etc)," + echo "please run ./configure --with-optimizations" >&6 + echo "" >&6 + echo "" >&6 +fi + diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1230,6 +1230,37 @@ fi], [AC_MSG_RESULT(no)]) +# Enable optimization flags +AC_SUBST(DEF_MAKE_ALL_RULE) +AC_SUBST(DEF_MAKE_RULE) +Py_OPT='false' +AC_MSG_CHECKING(for --with-optimizations) +AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable all optimizations when available (LTO, PGO, etc). Disabled by default.]), +[ +if test "$withval" != no +then + Py_OPT='true' + AC_MSG_RESULT(yes); +else + Py_OPT='false' + AC_MSG_RESULT(no); +fi], +[AC_MSG_RESULT(no)]) +if test "$Py_OPT" = 'true' ; then + Py_LTO='true' + case $ac_sys_system in + Darwin*) + # At least on macOS El Capitan, LTO does not work with PGO. + Py_LTO='false' + ;; + esac + DEF_MAKE_ALL_RULE="profile-opt" + DEF_MAKE_RULE="build_all" +else + DEF_MAKE_ALL_RULE="build_all" + DEF_MAKE_RULE="all" +fi + # Enable LTO flags AC_SUBST(LTOFLAGS) AC_MSG_CHECKING(for --with-lto) @@ -5299,3 +5330,13 @@ -s Modules Modules/Setup.config \ Modules/Setup.local Modules/Setup mv config.c Modules + +if test "$Py_OPT" = 'false' -a "$Py_DEBUG" != 'true'; then + echo "" >&AS_MESSAGE_FD + echo "" >&AS_MESSAGE_FD + echo "If you want a release build with all optimizations active (LTO, PGO, etc)," + echo "please run ./configure --with-optimizations" >&AS_MESSAGE_FD + echo "" >&AS_MESSAGE_FD + echo "" >&AS_MESSAGE_FD +fi + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:16:59 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 07 Sep 2016 00:16:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20160907001659.14789.91282.C4A82790@psf.io> https://hg.python.org/cpython/rev/06d20ac57c9a changeset: 103184:06d20ac57c9a parent: 103183:f9b52b31f1b4 parent: 103181:a5856153d942 user: Brett Cannon date: Tue Sep 06 17:16:41 2016 -0700 summary: Merge files: Doc/library/random.rst | 21 ++++++++ Lib/random.py | 28 +++++++++++- Lib/test/test_random.py | 68 +++++++++++++++++++++++++++++ Misc/NEWS | 2 + 4 files changed, 118 insertions(+), 1 deletions(-) diff --git a/Doc/library/random.rst b/Doc/library/random.rst --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -124,6 +124,27 @@ Return a random element from the non-empty sequence *seq*. If *seq* is empty, raises :exc:`IndexError`. +.. function:: weighted_choices(k, population, weights=None, *, cum_weights=None) + + Return a *k* sized list of elements chosen from the *population* with replacement. + If the *population* is empty, raises :exc:`IndexError`. + + If a *weights* sequence is specified, selections are made according to the + relative weights. Alternatively, if a *cum_weights* sequence is given, the + selections are made according to the cumulative weights. For example, the + relative weights ``[10, 5, 30, 5]`` are equivalent to the cumulative + weights ``[10, 15, 45, 50]``. Internally, the relative weights are + converted to cumulative weights before making selections, so supplying the + cumulative weights saves work. + + If neither *weights* nor *cum_weights* are specified, selections are made + with equal probability. If a weights sequence is supplied, it must be + the same length as the *population* sequence. It is a :exc:`TypeError` + to specify both *weights* and *cum_weights*. + + The *weights* or *cum_weights* can use any numeric type that interoperates + with the :class:`float` values returned by :func:`random` (that includes + integers, floats, and fractions but excludes decimals). .. function:: shuffle(x[, random]) diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -8,6 +8,7 @@ --------- pick random element pick random sample + pick weighted random sample generate random permutation distributions on the real line: @@ -43,12 +44,14 @@ from os import urandom as _urandom from _collections_abc import Set as _Set, Sequence as _Sequence from hashlib import sha512 as _sha512 +import itertools as _itertools +import bisect as _bisect __all__ = ["Random","seed","random","uniform","randint","choice","sample", "randrange","shuffle","normalvariate","lognormvariate", "expovariate","vonmisesvariate","gammavariate","triangular", "gauss","betavariate","paretovariate","weibullvariate", - "getstate","setstate", "getrandbits", + "getstate","setstate", "getrandbits", "weighted_choices", "SystemRandom"] NV_MAGICCONST = 4 * _exp(-0.5)/_sqrt(2.0) @@ -334,6 +337,28 @@ result[i] = population[j] return result + def weighted_choices(self, k, population, weights=None, *, cum_weights=None): + """Return a k sized list of population elements chosen with replacement. + + If the relative weights or cumulative weights are not specified, + the selections are made with equal probability. + + """ + if cum_weights is None: + if weights is None: + choice = self.choice + return [choice(population) for i in range(k)] + else: + cum_weights = list(_itertools.accumulate(weights)) + elif weights is not None: + raise TypeError('Cannot specify both weights and cumulative_weights') + if len(cum_weights) != len(population): + raise ValueError('The number of weights does not match the population') + bisect = _bisect.bisect + random = self.random + total = cum_weights[-1] + return [population[bisect(cum_weights, random() * total)] for i in range(k)] + ## -------------------- real-valued distributions ------------------- ## -------------------- uniform distribution ------------------- @@ -724,6 +749,7 @@ randrange = _inst.randrange sample = _inst.sample shuffle = _inst.shuffle +weighted_choices = _inst.weighted_choices normalvariate = _inst.normalvariate lognormvariate = _inst.lognormvariate expovariate = _inst.expovariate diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py --- a/Lib/test/test_random.py +++ b/Lib/test/test_random.py @@ -7,6 +7,7 @@ from functools import partial from math import log, exp, pi, fsum, sin from test import support +from fractions import Fraction class TestBasicOps: # Superclass with tests common to all generators. @@ -141,6 +142,73 @@ def test_sample_on_dicts(self): self.assertRaises(TypeError, self.gen.sample, dict.fromkeys('abcdef'), 2) + def test_weighted_choices(self): + weighted_choices = self.gen.weighted_choices + data = ['red', 'green', 'blue', 'yellow'] + str_data = 'abcd' + range_data = range(4) + set_data = set(range(4)) + + # basic functionality + for sample in [ + weighted_choices(5, data), + weighted_choices(5, data, range(4)), + weighted_choices(k=5, population=data, weights=range(4)), + weighted_choices(k=5, population=data, cum_weights=range(4)), + ]: + self.assertEqual(len(sample), 5) + self.assertEqual(type(sample), list) + self.assertTrue(set(sample) <= set(data)) + + # test argument handling + with self.assertRaises(TypeError): # missing arguments + weighted_choices(2) + + self.assertEqual(weighted_choices(0, data), []) # k == 0 + self.assertEqual(weighted_choices(-1, data), []) # negative k behaves like ``[0] * -1`` + with self.assertRaises(TypeError): + weighted_choices(2.5, data) # k is a float + + self.assertTrue(set(weighted_choices(5, str_data)) <= set(str_data)) # population is a string sequence + self.assertTrue(set(weighted_choices(5, range_data)) <= set(range_data)) # population is a range + with self.assertRaises(TypeError): + weighted_choices(2.5, set_data) # population is not a sequence + + self.assertTrue(set(weighted_choices(5, data, None)) <= set(data)) # weights is None + self.assertTrue(set(weighted_choices(5, data, weights=None)) <= set(data)) + with self.assertRaises(ValueError): + weighted_choices(5, data, [1,2]) # len(weights) != len(population) + with self.assertRaises(IndexError): + weighted_choices(5, data, [0]*4) # weights sum to zero + with self.assertRaises(TypeError): + weighted_choices(5, data, 10) # non-iterable weights + with self.assertRaises(TypeError): + weighted_choices(5, data, [None]*4) # non-numeric weights + for weights in [ + [15, 10, 25, 30], # integer weights + [15.1, 10.2, 25.2, 30.3], # float weights + [Fraction(1, 3), Fraction(2, 6), Fraction(3, 6), Fraction(4, 6)], # fractional weights + [True, False, True, False] # booleans (include / exclude) + ]: + self.assertTrue(set(weighted_choices(5, data, weights)) <= set(data)) + + with self.assertRaises(ValueError): + weighted_choices(5, data, cum_weights=[1,2]) # len(weights) != len(population) + with self.assertRaises(IndexError): + weighted_choices(5, data, cum_weights=[0]*4) # cum_weights sum to zero + with self.assertRaises(TypeError): + weighted_choices(5, data, cum_weights=10) # non-iterable cum_weights + with self.assertRaises(TypeError): + weighted_choices(5, data, cum_weights=[None]*4) # non-numeric cum_weights + with self.assertRaises(TypeError): + weighted_choices(5, data, range(4), cum_weights=range(4)) # both weights and cum_weights + for weights in [ + [15, 10, 25, 30], # integer cum_weights + [15.1, 10.2, 25.2, 30.3], # float cum_weights + [Fraction(1, 3), Fraction(2, 6), Fraction(3, 6), Fraction(4, 6)], # fractional cum_weights + ]: + self.assertTrue(set(weighted_choices(5, data, cum_weights=weights)) <= set(data)) + def test_gauss(self): # Ensure that the seed() method initializes all the hidden state. In # particular, through 2.2.1 it failed to reset a piece of state used diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -101,6 +101,8 @@ - Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name fields in X.509 certs. +- Issue #18844: Add random.weighted_choices(). + - Issue #25761: Improved error reporting about truncated pickle data in C implementation of unpickler. UnpicklingError is now raised instead of AttributeError and ValueError in some cases. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:16:59 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 07 Sep 2016 00:16:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_for_=2326359?= Message-ID: <20160907001659.14766.69334.D61D79EA@psf.io> https://hg.python.org/cpython/rev/f9b52b31f1b4 changeset: 103183:f9b52b31f1b4 parent: 103180:b4b73473ecc6 parent: 103182:1e61cc86df03 user: Brett Cannon date: Tue Sep 06 17:16:11 2016 -0700 summary: Merge for #26359 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:22:31 2016 From: python-checkins at python.org (gregory.p.smith) Date: Wed, 07 Sep 2016 00:22:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Correct_a_comment_in_the_t?= =?utf-8?q?est_referencing_the_wrong_issue_number_=28issue3100?= Message-ID: <20160907002231.39027.49976.7E47AFE2@psf.io> https://hg.python.org/cpython/rev/3d8109fe6d82 changeset: 103185:3d8109fe6d82 user: Gregory P. Smith [Google Inc.] date: Wed Sep 07 00:22:22 2016 +0000 summary: Correct a comment in the test referencing the wrong issue number (issue3100 is correct, not 3110). files: Lib/test/test_weakref.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_weakref.py b/Lib/test/test_weakref.py --- a/Lib/test/test_weakref.py +++ b/Lib/test/test_weakref.py @@ -911,7 +911,7 @@ self.assertFalse(hasattr(r, "__dict__")) def test_subclass_refs_with_cycle(self): - # Bug #3110 + """Confirm https://bugs.python.org/issue3100 is fixed.""" # An instance of a weakref subclass can have attributes. # If such a weakref holds the only strong reference to the object, # deleting the weakref will delete the object. In this case, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:35:19 2016 From: python-checkins at python.org (christian.heimes) Date: Wed, 07 Sep 2016 00:35:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Bypass_=5F=5Fget=5Fopenssl?= =?utf-8?q?=5Fconstructor=28=29_and_always_use_our_own_blake2_implementati?= =?utf-8?q?on?= Message-ID: <20160907003519.78231.69378.75EEDC4C@psf.io> https://hg.python.org/cpython/rev/caad55506549 changeset: 103186:caad55506549 user: Christian Heimes date: Wed Sep 07 02:35:13 2016 +0200 summary: Bypass __get_openssl_constructor() and always use our own blake2 implementation files: Lib/hashlib.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/hashlib.py b/Lib/hashlib.py --- a/Lib/hashlib.py +++ b/Lib/hashlib.py @@ -101,6 +101,9 @@ def __get_openssl_constructor(name): + if name in {'blake2b', 'blake2s'}: + # Prefer our blake2 implementation. + return __get_builtin_constructor(name) try: f = getattr(_hashlib, 'openssl_' + name) # Allow the C module to raise ValueError. The function will be -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:49:18 2016 From: python-checkins at python.org (christian.heimes) Date: Wed, 07 Sep 2016 00:49:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Silence_two_warnings_in_bl?= =?utf-8?q?ake2=2E_key=5Flength_is_between_0_and_64_=28block_size=29=2E?= Message-ID: <20160907004918.31142.97154.9E6BB32D@psf.io> https://hg.python.org/cpython/rev/9e97d985cee8 changeset: 103187:9e97d985cee8 user: Christian Heimes date: Wed Sep 07 02:49:11 2016 +0200 summary: Silence two warnings in blake2. key_length is between 0 and 64 (block size). files: Modules/_blake2/blake2b_impl.c | 2 +- Modules/_blake2/blake2s_impl.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Modules/_blake2/blake2b_impl.c b/Modules/_blake2/blake2b_impl.c --- a/Modules/_blake2/blake2b_impl.c +++ b/Modules/_blake2/blake2b_impl.c @@ -208,7 +208,7 @@ BLAKE2B_KEYBYTES); goto error; } - self->param.key_length = key->len; + self->param.key_length = (uint8_t)key->len; } /* Initialize hash state. */ diff --git a/Modules/_blake2/blake2s_impl.c b/Modules/_blake2/blake2s_impl.c --- a/Modules/_blake2/blake2s_impl.c +++ b/Modules/_blake2/blake2s_impl.c @@ -208,7 +208,7 @@ BLAKE2S_KEYBYTES); goto error; } - self->param.key_length = key->len; + self->param.key_length = (uint8_t)key->len; } /* Initialize hash state. */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:59:10 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 00:59:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_make_sure_to_n?= =?utf-8?q?ot_call_memcpy_with_a_NULL_second_argument?= Message-ID: <20160907005910.13723.20902.59A9107A@psf.io> https://hg.python.org/cpython/rev/5f3f6f1fb73a changeset: 103188:5f3f6f1fb73a branch: 3.5 parent: 103182:1e61cc86df03 user: Benjamin Peterson date: Tue Sep 06 17:58:25 2016 -0700 summary: make sure to not call memcpy with a NULL second argument files: Objects/listobject.c | 15 +++++++++------ 1 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -634,14 +634,17 @@ item = a->ob_item; /* recycle the items that we are about to remove */ s = norig * sizeof(PyObject *); - if (s > sizeof(recycle_on_stack)) { - recycle = (PyObject **)PyMem_MALLOC(s); - if (recycle == NULL) { - PyErr_NoMemory(); - goto Error; + /* If norig == 0, item might be NULL, in which case we may not memcpy from it. */ + if (s) { + if (s > sizeof(recycle_on_stack)) { + recycle = (PyObject **)PyMem_MALLOC(s); + if (recycle == NULL) { + PyErr_NoMemory(); + goto Error; + } } + memcpy(recycle, &item[ilow], s); } - memcpy(recycle, &item[ilow], s); if (d < 0) { /* Delete -d items */ Py_ssize_t tail; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:59:11 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 00:59:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160907005911.22586.6789.A35314A8@psf.io> https://hg.python.org/cpython/rev/c2158e5456d8 changeset: 103190:c2158e5456d8 parent: 103187:9e97d985cee8 parent: 103188:5f3f6f1fb73a user: Benjamin Peterson date: Tue Sep 06 17:58:44 2016 -0700 summary: merge 3.5 files: Objects/listobject.c | 15 +++++++++------ 1 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -634,14 +634,17 @@ item = a->ob_item; /* recycle the items that we are about to remove */ s = norig * sizeof(PyObject *); - if (s > sizeof(recycle_on_stack)) { - recycle = (PyObject **)PyMem_MALLOC(s); - if (recycle == NULL) { - PyErr_NoMemory(); - goto Error; + /* If norig == 0, item might be NULL, in which case we may not memcpy from it. */ + if (s) { + if (s > sizeof(recycle_on_stack)) { + recycle = (PyObject **)PyMem_MALLOC(s); + if (recycle == NULL) { + PyErr_NoMemory(); + goto Error; + } } + memcpy(recycle, &item[ilow], s); } - memcpy(recycle, &item[ilow], s); if (d < 0) { /* Delete -d items */ Py_ssize_t tail; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 20:59:10 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 00:59:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_make_sure_to_n?= =?utf-8?q?ot_call_memcpy_with_a_NULL_second_argument?= Message-ID: <20160907005910.1556.62244.EA6B7D9B@psf.io> https://hg.python.org/cpython/rev/ec537f9f468f changeset: 103189:ec537f9f468f branch: 2.7 parent: 103161:fecbb1fa241f user: Benjamin Peterson date: Tue Sep 06 17:58:25 2016 -0700 summary: make sure to not call memcpy with a NULL second argument files: Objects/listobject.c | 15 +++++++++------ 1 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -669,14 +669,17 @@ item = a->ob_item; /* recycle the items that we are about to remove */ s = norig * sizeof(PyObject *); - if (s > sizeof(recycle_on_stack)) { - recycle = (PyObject **)PyMem_MALLOC(s); - if (recycle == NULL) { - PyErr_NoMemory(); - goto Error; + /* If norig == 0, item might be NULL, in which case we may not memcpy from it. */ + if (s) { + if (s > sizeof(recycle_on_stack)) { + recycle = (PyObject **)PyMem_MALLOC(s); + if (recycle == NULL) { + PyErr_NoMemory(); + goto Error; + } } + memcpy(recycle, &item[ilow], s); } - memcpy(recycle, &item[ilow], s); if (d < 0) { /* Delete -d items */ memmove(&item[ihigh+d], &item[ihigh], -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 21:02:54 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 07 Sep 2016 01:02:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Use_shorter_ve?= =?utf-8?q?rsion_of_Connection=2Eisolation=5Flevel_in_sqlite3=2Erst?= Message-ID: <20160907010254.13824.72471.B8409CD7@psf.io> https://hg.python.org/cpython/rev/6c19ddd10c13 changeset: 103191:6c19ddd10c13 branch: 3.5 parent: 103188:5f3f6f1fb73a user: Berker Peksag date: Wed Sep 07 04:02:41 2016 +0300 summary: Use shorter version of Connection.isolation_level in sqlite3.rst files: Doc/library/sqlite3.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -183,7 +183,7 @@ parameter is 5.0 (five seconds). For the *isolation_level* parameter, please see the - :attr:`Connection.isolation_level` property of :class:`Connection` objects. + :attr:`~Connection.isolation_level` property of :class:`Connection` objects. SQLite natively supports only the types TEXT, INTEGER, REAL, BLOB and NULL. If you want to use other types you must add support for them yourself. The -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 21:02:55 2016 From: python-checkins at python.org (berker.peksag) Date: Wed, 07 Sep 2016 01:02:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20160907010254.48870.56879.8C5D2C55@psf.io> https://hg.python.org/cpython/rev/4aa6233961e5 changeset: 103192:4aa6233961e5 parent: 103190:c2158e5456d8 parent: 103191:6c19ddd10c13 user: Berker Peksag date: Wed Sep 07 04:03:02 2016 +0300 summary: Merge from 3.5 files: Doc/library/sqlite3.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -183,7 +183,7 @@ parameter is 5.0 (five seconds). For the *isolation_level* parameter, please see the - :attr:`Connection.isolation_level` property of :class:`Connection` objects. + :attr:`~Connection.isolation_level` property of :class:`Connection` objects. SQLite natively supports only the types TEXT, INTEGER, REAL, BLOB and NULL. If you want to use other types you must add support for them yourself. The -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 21:06:19 2016 From: python-checkins at python.org (gregory.p.smith) Date: Wed, 07 Sep 2016 01:06:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fixes_issue263?= =?utf-8?q?07=3A_The_profile-opt_build_now_applys_PGO_to_the_built-in_modu?= =?utf-8?q?les=2E?= Message-ID: <20160907010618.31408.16748.55E81DA8@psf.io> https://hg.python.org/cpython/rev/75dae0b2ccb3 changeset: 103193:75dae0b2ccb3 branch: 2.7 parent: 103189:ec537f9f468f user: Gregory P. Smith [Google Inc.] date: Wed Sep 07 01:05:59 2016 +0000 summary: Fixes issue26307: The profile-opt build now applys PGO to the built-in modules. files: Makefile.pre.in | 2 +- Misc/NEWS | 2 ++ 2 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1393,7 +1393,7 @@ find build -name 'fficonfig.h' -exec rm -f {} ';' || true find build -name 'fficonfig.py' -exec rm -f {} ';' || true -rm -f Lib/lib2to3/*Grammar*.pickle - -rm -rf build + -find build -type f -a ! -name '*.gc??' -exec rm -f {} ';' profile-removal: find . -name '*.gc??' -exec rm -f {} ';' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #26307: The profile-opt build now applys PGO to the built-in modules. + - Issue #27870: A left shift of zero by a large integer no longer attempts to allocate large amounts of memory. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 22:05:53 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 02:05:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160907020553.39102.17625.B08C4522@psf.io> https://hg.python.org/cpython/rev/23b705ec6940 changeset: 103195:23b705ec6940 parent: 103192:4aa6233961e5 parent: 103194:66feda02f2a5 user: Benjamin Peterson date: Tue Sep 06 19:04:37 2016 -0700 summary: merge 3.5 files: Modules/hashtable.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Modules/hashtable.c b/Modules/hashtable.c --- a/Modules/hashtable.c +++ b/Modules/hashtable.c @@ -338,7 +338,8 @@ entry->key_hash = key_hash; Py_MEMCPY((void *)_Py_HASHTABLE_ENTRY_PKEY(entry), pkey, ht->key_size); - ENTRY_WRITE_PDATA(ht, entry, data_size, data); + if (data) + ENTRY_WRITE_PDATA(ht, entry, data_size, data); _Py_slist_prepend(&ht->buckets[index], (_Py_slist_item_t*)entry); ht->entries++; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 22:05:53 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 02:05:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_do_not_memcpy_?= =?utf-8?q?from_NULL?= Message-ID: <20160907020553.680.8321.C641475D@psf.io> https://hg.python.org/cpython/rev/66feda02f2a5 changeset: 103194:66feda02f2a5 branch: 3.5 parent: 103191:6c19ddd10c13 user: Benjamin Peterson date: Tue Sep 06 19:03:40 2016 -0700 summary: do not memcpy from NULL files: Modules/hashtable.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Modules/hashtable.c b/Modules/hashtable.c --- a/Modules/hashtable.c +++ b/Modules/hashtable.c @@ -327,7 +327,8 @@ entry->key_hash = key_hash; assert(data_size == ht->data_size); - memcpy(_Py_HASHTABLE_ENTRY_DATA(entry), data, data_size); + if (data) + memcpy(_Py_HASHTABLE_ENTRY_DATA(entry), data, data_size); _Py_slist_prepend(&ht->buckets[index], (_Py_slist_item_t*)entry); ht->entries++; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 22:09:38 2016 From: python-checkins at python.org (steve.dower) Date: Wed, 07 Sep 2016 02:09:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_some_warnings_from_MSV?= =?utf-8?q?C?= Message-ID: <20160907020937.12905.47719.37FADCF7@psf.io> https://hg.python.org/cpython/rev/68effb84fa6a changeset: 103196:68effb84fa6a user: Steve Dower date: Tue Sep 06 19:09:15 2016 -0700 summary: Fix some warnings from MSVC files: Modules/_ctypes/_ctypes.c | 2 +- Modules/_decimal/libmpdec/vccompat.h | 4 ---- Modules/audioop.c | 3 ++- Programs/_freeze_importlib.c | 2 +- 4 files changed, 4 insertions(+), 7 deletions(-) diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -3067,7 +3067,7 @@ }; #ifdef MS_WIN32 -static PPROC FindAddress(void *handle, char *name, PyObject *type) +static PPROC FindAddress(void *handle, const char *name, PyObject *type) { #ifdef MS_WIN64 /* win64 has no stdcall calling conv, so it should diff --git a/Modules/_decimal/libmpdec/vccompat.h b/Modules/_decimal/libmpdec/vccompat.h --- a/Modules/_decimal/libmpdec/vccompat.h +++ b/Modules/_decimal/libmpdec/vccompat.h @@ -48,10 +48,6 @@ #undef strtoll #define strtoll _strtoi64 #define strdup _strdup - #define PRIi64 "I64i" - #define PRIu64 "I64u" - #define PRIi32 "I32i" - #define PRIu32 "I32u" #endif diff --git a/Modules/audioop.c b/Modules/audioop.c --- a/Modules/audioop.c +++ b/Modules/audioop.c @@ -4,6 +4,7 @@ #define PY_SSIZE_T_CLEAN #include "Python.h" +#include typedef short PyInt16; @@ -448,7 +449,7 @@ int val = GETRAWSAMPLE(width, fragment->buf, i); /* Cast to unsigned before negating. Unsigned overflow is well- defined, but signed overflow is not. */ - if (val < 0) absval = -(unsigned int)val; + if (val < 0) absval = (unsigned int)-(int64_t)val; else absval = val; if (absval > max) max = absval; } diff --git a/Programs/_freeze_importlib.c b/Programs/_freeze_importlib.c --- a/Programs/_freeze_importlib.c +++ b/Programs/_freeze_importlib.c @@ -58,7 +58,7 @@ fprintf(stderr, "cannot fstat '%s'\n", inpath); goto error; } - text_size = status.st_size; + text_size = (size_t)status.st_size; text = (char *) malloc(text_size + 1); if (text == NULL) { fprintf(stderr, "could not allocate %ld bytes\n", (long) text_size); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 22:36:28 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 07 Sep 2016 02:36:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327182=3A_Add_supp?= =?utf-8?q?ort_for_path-like_objects_to_PyUnicode=5FFSDecoder=28=29=2E?= Message-ID: <20160907023628.3514.99066.67ECB438@psf.io> https://hg.python.org/cpython/rev/3417d324cbf9 changeset: 103197:3417d324cbf9 user: Brett Cannon date: Tue Sep 06 19:36:01 2016 -0700 summary: Issue #27182: Add support for path-like objects to PyUnicode_FSDecoder(). files: Doc/c-api/unicode.rst | 12 ++++-- Doc/whatsnew/3.6.rst | 18 +++++--- Lib/test/test_compile.py | 10 +++++ Misc/NEWS | 3 +- Objects/unicodeobject.c | 53 ++++++++++++++++++++------- 5 files changed, 69 insertions(+), 27 deletions(-) diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -826,13 +826,17 @@ .. c:function:: int PyUnicode_FSDecoder(PyObject* obj, void* result) - ParseTuple converter: decode :class:`bytes` objects to :class:`str` using - :c:func:`PyUnicode_DecodeFSDefaultAndSize`; :class:`str` objects are output - as-is. *result* must be a :c:type:`PyUnicodeObject*` which must be released - when it is no longer used. + ParseTuple converter: decode :class:`bytes` objects -- obtained either + directly or indirectly through the :class:`os.PathLike` interface -- to + :class:`str` using :c:func:`PyUnicode_DecodeFSDefaultAndSize`; :class:`str` + objects are output as-is. *result* must be a :c:type:`PyUnicodeObject*` which + must be released when it is no longer used. .. versionadded:: 3.2 + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. c:function:: PyObject* PyUnicode_DecodeFSDefaultAndSize(const char *s, Py_ssize_t size) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -160,14 +160,18 @@ The built-in :func:`open` function has been updated to accept :class:`os.PathLike` objects as have all relevant functions in the -:mod:`os` and :mod:`os.path` modules. The :class:`os.DirEntry` class +:mod:`os` and :mod:`os.path` modules. :c:func:`PyUnicode_FSConverter` +and :c:func:`PyUnicode_FSConverter` have been changed to accept +path-like objects. The :class:`os.DirEntry` class and relevant classes in :mod:`pathlib` have also been updated to -implement :class:`os.PathLike`. The hope is that updating the -fundamental functions for operating on file system paths will lead -to third-party code to implicitly support all -:term:`path-like objects ` without any code changes -or at least very minimal ones (e.g. calling :func:`os.fspath` at the -beginning of code before operating on a path-like object). +implement :class:`os.PathLike`. + +The hope in is that updating the fundamental functions for operating +on file system paths will lead to third-party code to implicitly +support all :term:`path-like objects ` without any +code changes or at least very minimal ones (e.g. calling +:func:`os.fspath` at the beginning of code before operating on a +path-like object). Here are some examples of how the new interface allows for :class:`pathlib.Path` to be used more easily and transparently with diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -664,6 +664,16 @@ self.assertTrue(f1(0)) self.assertTrue(f2(0.0)) + def test_path_like_objects(self): + # An implicit test for PyUnicode_FSDecoder(). + class PathLike: + def __init__(self, path): + self._path = path + def __fspath__(self): + return self._path + + compile("42", PathLike("test_compile_pathlike"), "single") + class TestStackSize(unittest.TestCase): # These tests check that the computed stack size for a code object diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -202,7 +202,8 @@ C API ----- -- Issue #26027: Add support for path-like objects in PyUnicode_FSConverter(). +- Issue #26027: Add support for path-like objects in PyUnicode_FSConverter() & + PyUnicode_FSDecoder(). Tests ----- diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3882,37 +3882,60 @@ int PyUnicode_FSDecoder(PyObject* arg, void* addr) { + int is_buffer = 0; + PyObject *path = NULL; PyObject *output = NULL; if (arg == NULL) { Py_DECREF(*(PyObject**)addr); return 1; } - if (PyUnicode_Check(arg)) { - if (PyUnicode_READY(arg) == -1) + + is_buffer = PyObject_CheckBuffer(arg); + if (!is_buffer) { + path = PyOS_FSPath(arg); + if (path == NULL) { return 0; - output = arg; - Py_INCREF(output); - } - else if (PyBytes_Check(arg) || PyObject_CheckBuffer(arg)) { - if (!PyBytes_Check(arg) && + } + } + else { + path = arg; + Py_INCREF(arg); + } + + if (PyUnicode_Check(path)) { + if (PyUnicode_READY(path) == -1) { + Py_DECREF(path); + return 0; + } + output = path; + } + else if (PyBytes_Check(path) || is_buffer) { + PyObject *path_bytes = NULL; + + if (!PyBytes_Check(path) && PyErr_WarnFormat(PyExc_DeprecationWarning, 1, - "path should be string or bytes, not %.200s", + "path should be string, bytes, or os.PathLike, not %.200s", Py_TYPE(arg)->tp_name)) { + Py_DECREF(path); return 0; } - arg = PyBytes_FromObject(arg); - if (!arg) + path_bytes = PyBytes_FromObject(path); + Py_DECREF(path); + if (!path_bytes) { return 0; - output = PyUnicode_DecodeFSDefaultAndSize(PyBytes_AS_STRING(arg), - PyBytes_GET_SIZE(arg)); - Py_DECREF(arg); - if (!output) + } + output = PyUnicode_DecodeFSDefaultAndSize(PyBytes_AS_STRING(path_bytes), + PyBytes_GET_SIZE(path_bytes)); + Py_DECREF(path_bytes); + if (!output) { return 0; + } } else { PyErr_Format(PyExc_TypeError, - "path should be string or bytes, not %.200s", + "path should be string, bytes, or os.PathLike, not %.200s", Py_TYPE(arg)->tp_name); + Py_DECREF(path); return 0; } if (PyUnicode_READY(output) == -1) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 22:38:39 2016 From: python-checkins at python.org (steve.dower) Date: Wed, 07 Sep 2016 02:38:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Adds_test=2Esupport=2EPGO_?= =?utf-8?q?and_skips_tests_that_are_not_useful_for_PGO=2E?= Message-ID: <20160907023839.14667.70289.C7C3A2D1@psf.io> https://hg.python.org/cpython/rev/bb311f81a833 changeset: 103198:bb311f81a833 user: Steve Dower date: Tue Sep 06 19:38:15 2016 -0700 summary: Adds test.support.PGO and skips tests that are not useful for PGO. files: Lib/test/libregrtest/main.py | 2 ++ Lib/test/support/__init__.py | 6 +++++- Lib/test/test_asyncore.py | 3 +++ Lib/test/test_gdb.py | 1 + Lib/test/test_multiprocessing_fork.py | 6 ++++++ Lib/test/test_multiprocessing_forkserver.py | 5 +++++ Lib/test/test_multiprocessing_main_handling.py | 3 +++ Lib/test/test_multiprocessing_spawn.py | 5 +++++ Lib/test/test_subprocess.py | 3 +++ Makefile.pre.in | 2 +- Tools/msi/buildrelease.bat | 8 +++++--- 11 files changed, 39 insertions(+), 5 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -473,6 +473,8 @@ if self.ns.wait: input("Press any key to continue...") + support.PGO = self.ns.pgo + setup_tests(self.ns) self.find_tests(tests) diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -105,7 +105,7 @@ "check_warnings", "check_no_resource_warning", "EnvironmentVarGuard", "run_with_locale", "swap_item", "swap_attr", "Matcher", "set_memlimit", "SuppressCrashReport", "sortdict", - "run_with_tz", + "run_with_tz", "PGO", ] class Error(Exception): @@ -878,6 +878,10 @@ # Save the initial cwd SAVEDCWD = os.getcwd() +# Set by libregrtest/main.py so we can skip tests that are not +# useful for PGO +PGO = False + @contextlib.contextmanager def temp_dir(path=None, quiet=False): """Return a context manager that creates a temporary directory. diff --git a/Lib/test/test_asyncore.py b/Lib/test/test_asyncore.py --- a/Lib/test/test_asyncore.py +++ b/Lib/test/test_asyncore.py @@ -11,6 +11,9 @@ from test import support from io import BytesIO +if support.PGO: + raise unittest.SkipTest("test is not helpful for PGO") + try: import threading except ImportError: diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -110,6 +110,7 @@ BREAKPOINT_FN='builtin_id' + at support.skipIf(support.PGO, "not useful for PGO") class DebuggerTests(unittest.TestCase): """Test that the debugger can debug Python.""" diff --git a/Lib/test/test_multiprocessing_fork.py b/Lib/test/test_multiprocessing_fork.py --- a/Lib/test/test_multiprocessing_fork.py +++ b/Lib/test/test_multiprocessing_fork.py @@ -1,6 +1,12 @@ import unittest import test._test_multiprocessing +from test import support + +if support.PGO: + raise unittest.SkipTest("test is not helpful for PGO") + + test._test_multiprocessing.install_tests_in_module_dict(globals(), 'fork') if __name__ == '__main__': diff --git a/Lib/test/test_multiprocessing_forkserver.py b/Lib/test/test_multiprocessing_forkserver.py --- a/Lib/test/test_multiprocessing_forkserver.py +++ b/Lib/test/test_multiprocessing_forkserver.py @@ -1,6 +1,11 @@ import unittest import test._test_multiprocessing +from test import support + +if support.PGO: + raise unittest.SkipTest("test is not helpful for PGO") + test._test_multiprocessing.install_tests_in_module_dict(globals(), 'forkserver') if __name__ == '__main__': diff --git a/Lib/test/test_multiprocessing_main_handling.py b/Lib/test/test_multiprocessing_main_handling.py --- a/Lib/test/test_multiprocessing_main_handling.py +++ b/Lib/test/test_multiprocessing_main_handling.py @@ -16,6 +16,9 @@ make_pkg, make_script, make_zip_pkg, make_zip_script, assert_python_ok) +if support.PGO: + raise unittest.SkipTest("test is not helpful for PGO") + # Look up which start methods are available to test import multiprocessing AVAILABLE_START_METHODS = set(multiprocessing.get_all_start_methods()) diff --git a/Lib/test/test_multiprocessing_spawn.py b/Lib/test/test_multiprocessing_spawn.py --- a/Lib/test/test_multiprocessing_spawn.py +++ b/Lib/test/test_multiprocessing_spawn.py @@ -1,6 +1,11 @@ import unittest import test._test_multiprocessing +from test import support + +if support.PGO: + raise unittest.SkipTest("test is not helpful for PGO") + test._test_multiprocessing.install_tests_in_module_dict(globals(), 'spawn') if __name__ == '__main__': diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -21,6 +21,9 @@ except ImportError: threading = None +if support.PGO: + raise unittest.SkipTest("test is not helpful for PGO") + mswindows = (sys.platform == "win32") # diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -235,7 +235,7 @@ # The task to run while instrumented when building the profile-opt target. # We exclude unittests with -x that take a rediculious amount of time to # run in the instrumented training build or do not provide much value. -PROFILE_TASK=-m test.regrtest --pgo -x test_asyncore test_gdb test_multiprocessing_fork test_multiprocessing_forkserver test_multiprocessing_main_handling test_multiprocessing_spawn test_subprocess +PROFILE_TASK=-m test.regrtest --pgo # report files for gcov / lcov coverage report COVERAGE_INFO= $(abs_builddir)/coverage.info diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat --- a/Tools/msi/buildrelease.bat +++ b/Tools/msi/buildrelease.bat @@ -35,7 +35,7 @@ set BUILDX64= set TARGET=Rebuild set TESTTARGETDIR= -set PGO= +set PGO=default :CheckOpts @@ -55,6 +55,7 @@ if "%1" EQU "-x86" (set BUILDX86=1) && shift && goto CheckOpts if "%1" EQU "-x64" (set BUILDX64=1) && shift && goto CheckOpts if "%1" EQU "--pgo" (set PGO=%~2) && shift && shift && goto CheckOpts +if "%1" EQU "--skip-pgo" (set PGO=) && shift && goto CheckOpts if "%1" NEQ "" echo Invalid option: "%1" && exit /B 1 @@ -195,7 +196,7 @@ :Help echo buildrelease.bat [--out DIR] [-x86] [-x64] [--certificate CERTNAME] [--build] [--skip-build] -echo [--pgo COMMAND] [--skip-doc] [--download DOWNLOAD URL] [--test TARGETDIR] +echo [--pgo COMMAND] [--skip-pgo] [--skip-doc] [--download DOWNLOAD URL] [--test TARGETDIR] echo [-h] echo. echo --out (-o) Specify an additional output directory for installers @@ -204,7 +205,8 @@ echo --build (-b) Incrementally build Python rather than rebuilding echo --skip-build (-B) Do not build Python (just do the installers) echo --skip-doc (-D) Do not build documentation -echo --pgo Build x64 installers using PGO +echo --pgo Specify PGO command for x64 installers +echo --skip-pgo Build x64 installers using PGO echo --download Specify the full download URL for MSIs echo --test Specify the test directory to run the installer tests echo -h Display this help information -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 22:42:59 2016 From: python-checkins at python.org (steve.dower) Date: Wed, 07 Sep 2016 02:42:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327959=3A_Adds_oem?= =?utf-8?q?_encoding=2C_alias_ansi_to_mbcs=2C_move_aliasmbcs_to_codec?= Message-ID: <20160907024259.22491.76805.6490CECD@psf.io> https://hg.python.org/cpython/rev/c499690f606c changeset: 103199:c499690f606c user: Steve Dower date: Tue Sep 06 19:42:27 2016 -0700 summary: Issue #27959: Adds oem encoding, alias ansi to mbcs, move aliasmbcs to codec lookup files: Include/unicodeobject.h | 2 +- Lib/encodings/__init__.py | 10 ++ Lib/encodings/aliases.py | 1 + Lib/encodings/oem.py | 41 ++++++++++ Lib/site.py | 16 --- Lib/test/test_codecs.py | 62 +++++++-------- Modules/_codecsmodule.c | 36 ++++++++ Modules/clinic/_codecsmodule.c.h | 81 +++++++++++++++++++- 8 files changed, 198 insertions(+), 51 deletions(-) diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -1663,7 +1663,7 @@ PyAPI_FUNC(PyObject*) PyUnicode_DecodeMBCS( const char *string, /* MBCS encoded string */ - Py_ssize_t length, /* size of string */ + Py_ssize_t length, /* size of string */ const char *errors /* error handling */ ); diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py --- a/Lib/encodings/__init__.py +++ b/Lib/encodings/__init__.py @@ -29,6 +29,7 @@ """#" import codecs +import sys from . import aliases _cache = {} @@ -151,3 +152,12 @@ # Register the search_function in the Python codec registry codecs.register(search_function) + +if sys.platform == 'win32': + def _alias_mbcs(encoding): + import _bootlocale + if encoding == _bootlocale.getpreferredencoding(False): + import encodings.mbcs + return encodings.mbcs.getregentry() + + codecs.register(_alias_mbcs) diff --git a/Lib/encodings/aliases.py b/Lib/encodings/aliases.py --- a/Lib/encodings/aliases.py +++ b/Lib/encodings/aliases.py @@ -458,6 +458,7 @@ 'macturkish' : 'mac_turkish', # mbcs codec + 'ansi' : 'mbcs', 'dbcs' : 'mbcs', # ptcp154 codec diff --git a/Lib/encodings/oem.py b/Lib/encodings/oem.py new file mode 100644 --- /dev/null +++ b/Lib/encodings/oem.py @@ -0,0 +1,41 @@ +""" Python 'oem' Codec for Windows + +""" +# Import them explicitly to cause an ImportError +# on non-Windows systems +from codecs import oem_encode, oem_decode +# for IncrementalDecoder, IncrementalEncoder, ... +import codecs + +### Codec APIs + +encode = oem_encode + +def decode(input, errors='strict'): + return oem_decode(input, errors, True) + +class IncrementalEncoder(codecs.IncrementalEncoder): + def encode(self, input, final=False): + return oem_encode(input, self.errors)[0] + +class IncrementalDecoder(codecs.BufferedIncrementalDecoder): + _buffer_decode = oem_decode + +class StreamWriter(codecs.StreamWriter): + encode = oem_encode + +class StreamReader(codecs.StreamReader): + decode = oem_decode + +### encodings module API + +def getregentry(): + return codecs.CodecInfo( + name='oem', + encode=encode, + decode=decode, + incrementalencoder=IncrementalEncoder, + incrementaldecoder=IncrementalDecoder, + streamreader=StreamReader, + streamwriter=StreamWriter, + ) diff --git a/Lib/site.py b/Lib/site.py --- a/Lib/site.py +++ b/Lib/site.py @@ -423,21 +423,6 @@ sys.__interactivehook__ = register_readline -def aliasmbcs(): - """On Windows, some default encodings are not provided by Python, - while they are always available as "mbcs" in each locale. Make - them usable by aliasing to "mbcs" in such a case.""" - if sys.platform == 'win32': - import _bootlocale, codecs - enc = _bootlocale.getpreferredencoding(False) - if enc.startswith('cp'): # "cp***" ? - try: - codecs.lookup(enc) - except LookupError: - import encodings - encodings._cache[enc] = encodings._unknown - encodings.aliases.aliases[enc] = 'mbcs' - CONFIG_LINE = r'^(?P(\w|[-_])+)\s*=\s*(?P.*)\s*$' def venv(known_paths): @@ -560,7 +545,6 @@ setcopyright() sethelper() enablerlcompleter() - aliasmbcs() execsitecustomize() if ENABLE_USER_SITE: execusercustomize() diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -8,11 +8,6 @@ from test import support -if sys.platform == 'win32': - VISTA_OR_LATER = (sys.getwindowsversion().major >= 6) -else: - VISTA_OR_LATER = False - try: import ctypes except ImportError: @@ -841,18 +836,13 @@ ('abc', 'strict', b'abc'), ('\xe9\u20ac', 'strict', b'\xc3\xa9\xe2\x82\xac'), ('\U0010ffff', 'strict', b'\xf4\x8f\xbf\xbf'), + ('\udc80', 'strict', None), + ('\udc80', 'ignore', b''), + ('\udc80', 'replace', b'?'), + ('\udc80', 'backslashreplace', b'\\udc80'), + ('\udc80', 'namereplace', b'\\udc80'), + ('\udc80', 'surrogatepass', b'\xed\xb2\x80'), ] - if VISTA_OR_LATER: - tests.extend(( - ('\udc80', 'strict', None), - ('\udc80', 'ignore', b''), - ('\udc80', 'replace', b'?'), - ('\udc80', 'backslashreplace', b'\\udc80'), - ('\udc80', 'namereplace', b'\\udc80'), - ('\udc80', 'surrogatepass', b'\xed\xb2\x80'), - )) - else: - tests.append(('\udc80', 'strict', b'\xed\xb2\x80')) for text, errors, expected in tests: if expected is not None: try: @@ -879,17 +869,10 @@ (b'[\xff]', 'ignore', '[]'), (b'[\xff]', 'replace', '[\ufffd]'), (b'[\xff]', 'surrogateescape', '[\udcff]'), + (b'[\xed\xb2\x80]', 'strict', None), + (b'[\xed\xb2\x80]', 'ignore', '[]'), + (b'[\xed\xb2\x80]', 'replace', '[\ufffd\ufffd\ufffd]'), ] - if VISTA_OR_LATER: - tests.extend(( - (b'[\xed\xb2\x80]', 'strict', None), - (b'[\xed\xb2\x80]', 'ignore', '[]'), - (b'[\xed\xb2\x80]', 'replace', '[\ufffd\ufffd\ufffd]'), - )) - else: - tests.extend(( - (b'[\xed\xb2\x80]', 'strict', '[\udc80]'), - )) for raw, errors, expected in tests: if expected is not None: try: @@ -904,7 +887,6 @@ self.assertRaises(UnicodeDecodeError, raw.decode, 'cp65001', errors) - @unittest.skipUnless(VISTA_OR_LATER, 'require Windows Vista or later') def test_lone_surrogates(self): self.assertRaises(UnicodeEncodeError, "\ud800".encode, "cp65001") self.assertRaises(UnicodeDecodeError, b"\xed\xa0\x80".decode, "cp65001") @@ -921,7 +903,6 @@ self.assertEqual("[\uDC80]".encode("cp65001", "replace"), b'[?]') - @unittest.skipUnless(VISTA_OR_LATER, 'require Windows Vista or later') def test_surrogatepass_handler(self): self.assertEqual("abc\ud800def".encode("cp65001", "surrogatepass"), b"abc\xed\xa0\x80def") @@ -1951,6 +1932,8 @@ if hasattr(codecs, "mbcs_encode"): all_unicode_encodings.append("mbcs") +if hasattr(codecs, "oem_encode"): + all_unicode_encodings.append("oem") # The following encoding is not tested, because it's not supposed # to work: @@ -3119,11 +3102,10 @@ (b'\xff\xf4\x8f\xbf\xbf', 'ignore', '\U0010ffff'), (b'\xff\xf4\x8f\xbf\xbf', 'replace', '\ufffd\U0010ffff'), )) - if VISTA_OR_LATER: - self.check_encode(self.CP_UTF8, ( - ('[\U0010ffff\uDC80]', 'ignore', b'[\xf4\x8f\xbf\xbf]'), - ('[\U0010ffff\uDC80]', 'replace', b'[\xf4\x8f\xbf\xbf?]'), - )) + self.check_encode(self.CP_UTF8, ( + ('[\U0010ffff\uDC80]', 'ignore', b'[\xf4\x8f\xbf\xbf]'), + ('[\U0010ffff\uDC80]', 'replace', b'[\xf4\x8f\xbf\xbf?]'), + )) def test_incremental(self): decoded = codecs.code_page_decode(932, b'\x82', 'strict', False) @@ -3144,6 +3126,20 @@ False) self.assertEqual(decoded, ('abc', 3)) + def test_mbcs_alias(self): + # Check that looking up our 'default' codepage will return + # mbcs when we don't have a more specific one available + import _bootlocale + def _get_fake_codepage(*a): + return 'cp123' + old_getpreferredencoding = _bootlocale.getpreferredencoding + _bootlocale.getpreferredencoding = _get_fake_codepage + try: + codec = codecs.lookup('cp123') + self.assertEqual(codec.name, 'mbcs') + finally: + _bootlocale.getpreferredencoding = old_getpreferredencoding + class ASCIITest(unittest.TestCase): def test_encode(self): diff --git a/Modules/_codecsmodule.c b/Modules/_codecsmodule.c --- a/Modules/_codecsmodule.c +++ b/Modules/_codecsmodule.c @@ -626,6 +626,25 @@ } /*[clinic input] +_codecs.oem_decode + data: Py_buffer + errors: str(accept={str, NoneType}) = NULL + final: int(c_default="0") = False + / +[clinic start generated code]*/ + +static PyObject * +_codecs_oem_decode_impl(PyObject *module, Py_buffer *data, + const char *errors, int final) +/*[clinic end generated code: output=da1617612f3fcad8 input=95b8a92c446b03cd]*/ +{ + Py_ssize_t consumed = data->len; + PyObject *decoded = PyUnicode_DecodeCodePageStateful(CP_OEMCP, + data->buf, data->len, errors, final ? NULL : &consumed); + return codec_tuple(decoded, consumed); +} + +/*[clinic input] _codecs.code_page_decode codepage: int data: Py_buffer @@ -971,6 +990,21 @@ } /*[clinic input] +_codecs.oem_encode + str: unicode + errors: str(accept={str, NoneType}) = NULL + / +[clinic start generated code]*/ + +static PyObject * +_codecs_oem_encode_impl(PyObject *module, PyObject *str, const char *errors) +/*[clinic end generated code: output=65d5982c737de649 input=3fc5f0028aad3cda]*/ +{ + return codec_tuple(PyUnicode_EncodeCodePage(CP_OEMCP, str, errors), + PyUnicode_GET_LENGTH(str)); +} + +/*[clinic input] _codecs.code_page_encode code_page: int str: unicode @@ -1075,6 +1109,8 @@ _CODECS_READBUFFER_ENCODE_METHODDEF _CODECS_MBCS_ENCODE_METHODDEF _CODECS_MBCS_DECODE_METHODDEF + _CODECS_OEM_ENCODE_METHODDEF + _CODECS_OEM_DECODE_METHODDEF _CODECS_CODE_PAGE_ENCODE_METHODDEF _CODECS_CODE_PAGE_DECODE_METHODDEF _CODECS_REGISTER_ERROR_METHODDEF diff --git a/Modules/clinic/_codecsmodule.c.h b/Modules/clinic/_codecsmodule.c.h --- a/Modules/clinic/_codecsmodule.c.h +++ b/Modules/clinic/_codecsmodule.c.h @@ -805,6 +805,45 @@ #if defined(HAVE_MBCS) +PyDoc_STRVAR(_codecs_oem_decode__doc__, +"oem_decode($module, data, errors=None, final=False, /)\n" +"--\n" +"\n"); + +#define _CODECS_OEM_DECODE_METHODDEF \ + {"oem_decode", (PyCFunction)_codecs_oem_decode, METH_VARARGS, _codecs_oem_decode__doc__}, + +static PyObject * +_codecs_oem_decode_impl(PyObject *module, Py_buffer *data, + const char *errors, int final); + +static PyObject * +_codecs_oem_decode(PyObject *module, PyObject *args) +{ + PyObject *return_value = NULL; + Py_buffer data = {NULL, NULL}; + const char *errors = NULL; + int final = 0; + + if (!PyArg_ParseTuple(args, "y*|zi:oem_decode", + &data, &errors, &final)) { + goto exit; + } + return_value = _codecs_oem_decode_impl(module, &data, errors, final); + +exit: + /* Cleanup for data */ + if (data.obj) { + PyBuffer_Release(&data); + } + + return return_value; +} + +#endif /* defined(HAVE_MBCS) */ + +#if defined(HAVE_MBCS) + PyDoc_STRVAR(_codecs_code_page_decode__doc__, "code_page_decode($module, codepage, data, errors=None, final=False, /)\n" "--\n" @@ -1346,6 +1385,38 @@ #if defined(HAVE_MBCS) +PyDoc_STRVAR(_codecs_oem_encode__doc__, +"oem_encode($module, str, errors=None, /)\n" +"--\n" +"\n"); + +#define _CODECS_OEM_ENCODE_METHODDEF \ + {"oem_encode", (PyCFunction)_codecs_oem_encode, METH_VARARGS, _codecs_oem_encode__doc__}, + +static PyObject * +_codecs_oem_encode_impl(PyObject *module, PyObject *str, const char *errors); + +static PyObject * +_codecs_oem_encode(PyObject *module, PyObject *args) +{ + PyObject *return_value = NULL; + PyObject *str; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "U|z:oem_encode", + &str, &errors)) { + goto exit; + } + return_value = _codecs_oem_encode_impl(module, str, errors); + +exit: + return return_value; +} + +#endif /* defined(HAVE_MBCS) */ + +#if defined(HAVE_MBCS) + PyDoc_STRVAR(_codecs_code_page_encode__doc__, "code_page_encode($module, code_page, str, errors=None, /)\n" "--\n" @@ -1446,6 +1517,10 @@ #define _CODECS_MBCS_DECODE_METHODDEF #endif /* !defined(_CODECS_MBCS_DECODE_METHODDEF) */ +#ifndef _CODECS_OEM_DECODE_METHODDEF + #define _CODECS_OEM_DECODE_METHODDEF +#endif /* !defined(_CODECS_OEM_DECODE_METHODDEF) */ + #ifndef _CODECS_CODE_PAGE_DECODE_METHODDEF #define _CODECS_CODE_PAGE_DECODE_METHODDEF #endif /* !defined(_CODECS_CODE_PAGE_DECODE_METHODDEF) */ @@ -1454,7 +1529,11 @@ #define _CODECS_MBCS_ENCODE_METHODDEF #endif /* !defined(_CODECS_MBCS_ENCODE_METHODDEF) */ +#ifndef _CODECS_OEM_ENCODE_METHODDEF + #define _CODECS_OEM_ENCODE_METHODDEF +#endif /* !defined(_CODECS_OEM_ENCODE_METHODDEF) */ + #ifndef _CODECS_CODE_PAGE_ENCODE_METHODDEF #define _CODECS_CODE_PAGE_ENCODE_METHODDEF #endif /* !defined(_CODECS_CODE_PAGE_ENCODE_METHODDEF) */ -/*[clinic end generated code: output=0221e4eece62c905 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=7874e2d559d49368 input=a9049054013a1b77]*/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 22:46:58 2016 From: python-checkins at python.org (steve.dower) Date: Wed, 07 Sep 2016 02:46:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327959=3A_Document?= =?utf-8?q?s_new_encoding_and_alias=2E?= Message-ID: <20160907024657.12705.21145.EF9BEF56@psf.io> https://hg.python.org/cpython/rev/fd0e62300fb7 changeset: 103200:fd0e62300fb7 user: Steve Dower date: Tue Sep 06 19:46:42 2016 -0700 summary: Issue #27959: Documents new encoding and alias. files: Doc/library/codecs.rst | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -1254,9 +1254,15 @@ | | | Only ``errors='strict'`` | | | | is supported. | +--------------------+---------+---------------------------+ -| mbcs | dbcs | Windows only: Encode | +| mbcs | ansi, | Windows only: Encode | +| | dbcs | operand according to the | +| | | ANSI codepage (CP_ACP) | ++--------------------+---------+---------------------------+ +| oem | | Windows only: Encode | | | | operand according to the | -| | | ANSI codepage (CP_ACP) | +| | | OEM codepage (CP_OEMCP) | +| | | | +| | | .. versionadded:: 3.6 | +--------------------+---------+---------------------------+ | palmos | | Encoding of PalmOS 3.5 | +--------------------+---------+---------------------------+ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 22:56:10 2016 From: python-checkins at python.org (steve.dower) Date: Wed, 07 Sep 2016 02:56:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327959=3A_Updates_?= =?utf-8?q?NEWS_and_whatsnew?= Message-ID: <20160907025610.48824.34473.95029A53@psf.io> https://hg.python.org/cpython/rev/786c34bdc27a changeset: 103201:786c34bdc27a user: Steve Dower date: Tue Sep 06 19:55:55 2016 -0700 summary: Issue #27959: Updates NEWS and whatsnew files: Doc/whatsnew/3.6.rst | 6 ++++++ Misc/NEWS | 3 +++ 2 files changed, 9 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -450,6 +450,12 @@ need to be adapted. See :issue:`27819` for more details. +encodings +--------- + +On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP`` and the ``'ansi'`` +alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page. + faulthandler ------------ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -237,6 +237,9 @@ Windows ------- +- Issue #27959: Adds oem encoding, alias ansi to mbcs, move aliasmbcs to + codec lookup. + - Issue #27982: The functions of the winsound module now accept keyword arguments. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 22:59:50 2016 From: python-checkins at python.org (victor.stinner) Date: Wed, 07 Sep 2016 02:59:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_test=5Fos=2EGetRandomT?= =?utf-8?b?ZXN0cygp?= Message-ID: <20160907025950.779.40561.5494EF7B@psf.io> https://hg.python.org/cpython/rev/7a243a40b421 changeset: 103202:7a243a40b421 user: Victor Stinner date: Tue Sep 06 19:57:40 2016 -0700 summary: Fix test_os.GetRandomTests() Issue #27778: Skip getrandom() tests if getrandom() fails with ENOSYS. files: Lib/test/test_os.py | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1271,6 +1271,18 @@ @unittest.skipUnless(hasattr(os, 'getrandom'), 'need os.getrandom()') class GetRandomTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + try: + os.getrandom(1) + except OSError as exc: + if exc.errno == errno.ENOSYS: + # Python compiled on a more recent Linux version + # than the current Linux kernel + raise unittest.SkipTest("getrandom() syscall fails with ENOSYS") + else: + raise + def test_getrandom_type(self): data = os.getrandom(16) self.assertIsInstance(data, bytes) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 23:17:37 2016 From: python-checkins at python.org (steve.dower) Date: Wed, 07 Sep 2016 03:17:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=236135=3A_Adds_enco?= =?utf-8?q?ding_and_errors_parameters_to_subprocess?= Message-ID: <20160907031731.12869.35054.D2B5CF6E@psf.io> https://hg.python.org/cpython/rev/720f0cf580e2 changeset: 103203:720f0cf580e2 user: Steve Dower date: Tue Sep 06 20:16:17 2016 -0700 summary: Issue #6135: Adds encoding and errors parameters to subprocess files: Doc/library/subprocess.rst | 103 ++++++++++++++--------- Doc/whatsnew/3.6.rst | 3 + Lib/subprocess.py | 87 ++++++++++--------- Lib/test/test_subprocess.py | 59 +++++++++---- Misc/NEWS | 2 + 5 files changed, 154 insertions(+), 100 deletions(-) diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -38,7 +38,8 @@ .. function:: run(args, *, stdin=None, input=None, stdout=None, stderr=None,\ - shell=False, timeout=None, check=False) + shell=False, timeout=None, check=False, \ + encoding=None, errors=None) Run the command described by *args*. Wait for command to complete, then return a :class:`CompletedProcess` instance. @@ -60,15 +61,20 @@ The *input* argument is passed to :meth:`Popen.communicate` and thus to the subprocess's stdin. If used it must be a byte sequence, or a string if - ``universal_newlines=True``. When used, the internal :class:`Popen` object - is automatically created with ``stdin=PIPE``, and the *stdin* argument may - not be used as well. + *encoding* or *errors* is specified or *universal_newlines* is True. When + used, the internal :class:`Popen` object is automatically created with + ``stdin=PIPE``, and the *stdin* argument may not be used as well. If *check* is True, and the process exits with a non-zero exit code, a :exc:`CalledProcessError` exception will be raised. Attributes of that exception hold the arguments, the exit code, and stdout and stderr if they were captured. + If *encoding* or *errors* are specified, or *universal_newlines* is True, + file objects for stdin, stdout and stderr are opened in text mode using the + specified *encoding* and *errors* or the :class:`io.TextIOWrapper` default. + Otherwise, file objects are opened in binary mode. + Examples:: >>> subprocess.run(["ls", "-l"]) # doesn't capture output @@ -85,6 +91,10 @@ .. versionadded:: 3.5 + .. versionchanged:: 3.6 + + Added *encoding* and *errors* parameters + .. class:: CompletedProcess The return value from :func:`run`, representing a process that has finished. @@ -104,8 +114,8 @@ .. attribute:: stdout Captured stdout from the child process. A bytes sequence, or a string if - :func:`run` was called with ``universal_newlines=True``. None if stdout - was not captured. + :func:`run` was called with an encoding or errors. None if stdout was not + captured. If you ran the process with ``stderr=subprocess.STDOUT``, stdout and stderr will be combined in this attribute, and :attr:`stderr` will be @@ -114,8 +124,8 @@ .. attribute:: stderr Captured stderr from the child process. A bytes sequence, or a string if - :func:`run` was called with ``universal_newlines=True``. None if stderr - was not captured. + :func:`run` was called with an encoding or errors. None if stderr was not + captured. .. method:: check_returncode() @@ -249,19 +259,22 @@ .. index:: single: universal newlines; subprocess module - If *universal_newlines* is ``False`` the file objects *stdin*, *stdout* and - *stderr* will be opened as binary streams, and no line ending conversion is - done. + If *encoding* or *errors* are specified, or *universal_newlines* is True, + the file objects *stdin*, *stdout* and *stderr* will be opened in text + mode using the *encoding* and *errors* specified in the call or the + defaults for :class:`io.TextIOWrapper`. - If *universal_newlines* is ``True``, these file objects - will be opened as text streams in :term:`universal newlines` mode - using the encoding returned by :func:`locale.getpreferredencoding(False) - `. For *stdin*, line ending characters - ``'\n'`` in the input will be converted to the default line separator - :data:`os.linesep`. For *stdout* and *stderr*, all line endings in the - output will be converted to ``'\n'``. For more information see the - documentation of the :class:`io.TextIOWrapper` class when the *newline* - argument to its constructor is ``None``. + For *stdin*, line ending characters ``'\n'`` in the input will be converted + to the default line separator :data:`os.linesep`. For *stdout* and *stderr*, + all line endings in the output will be converted to ``'\n'``. For more + information see the documentation of the :class:`io.TextIOWrapper` class + when the *newline* argument to its constructor is ``None``. + + If text mode is not used, *stdin*, *stdout* and *stderr* will be opened as + binary streams. No encoding or line ending conversion is performed. + + .. versionadded:: 3.6 + Added *encoding* and *errors* parameters. .. note:: @@ -306,7 +319,8 @@ stderr=None, preexec_fn=None, close_fds=True, shell=False, \ cwd=None, env=None, universal_newlines=False, \ startupinfo=None, creationflags=0, restore_signals=True, \ - start_new_session=False, pass_fds=()) + start_new_session=False, pass_fds=(), *, \ + encoding=None, errors=None) Execute a child program in a new process. On POSIX, the class uses :meth:`os.execvp`-like behavior to execute the child program. On Windows, @@ -482,10 +496,14 @@ .. _side-by-side assembly: https://en.wikipedia.org/wiki/Side-by-Side_Assembly - If *universal_newlines* is ``True``, the file objects *stdin*, *stdout* - and *stderr* are opened as text streams in universal newlines mode, as - described above in :ref:`frequently-used-arguments`, otherwise they are - opened as binary streams. + If *encoding* or *errors* are specified, the file objects *stdin*, *stdout* + and *stderr* are opened in text mode with the specified encoding and + *errors*, as described above in :ref:`frequently-used-arguments`. If + *universal_newlines* is ``True``, they are opened in text mode with default + encoding. Otherwise, they are opened as binary streams. + + .. versionadded:: 3.6 + *encoding* and *errors* were added. If given, *startupinfo* will be a :class:`STARTUPINFO` object, which is passed to the underlying ``CreateProcess`` function. @@ -601,11 +619,12 @@ Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate. The optional *input* argument should be data to be sent to the child process, or - ``None``, if no data should be sent to the child. The type of *input* - must be bytes or, if *universal_newlines* was ``True``, a string. + ``None``, if no data should be sent to the child. If streams were opened in + text mode, *input* must be a string. Otherwise, it must be bytes. :meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``. - The data will be bytes or, if *universal_newlines* was ``True``, strings. + The data will be strings if streams were opened in text mode; otherwise, + bytes. Note that if you want to send data to the process's stdin, you need to create the Popen object with ``stdin=PIPE``. Similarly, to get anything other than @@ -672,28 +691,30 @@ .. attribute:: Popen.stdin If the *stdin* argument was :data:`PIPE`, this attribute is a writeable - stream object as returned by :func:`open`. If the *universal_newlines* - argument was ``True``, the stream is a text stream, otherwise it is a byte - stream. If the *stdin* argument was not :data:`PIPE`, this attribute is - ``None``. + stream object as returned by :func:`open`. If the *encoding* or *errors* + arguments were specified or the *universal_newlines* argument was ``True``, + the stream is a text stream, otherwise it is a byte stream. If the *stdin* + argument was not :data:`PIPE`, this attribute is ``None``. .. attribute:: Popen.stdout If the *stdout* argument was :data:`PIPE`, this attribute is a readable stream object as returned by :func:`open`. Reading from the stream provides - output from the child process. If the *universal_newlines* argument was - ``True``, the stream is a text stream, otherwise it is a byte stream. If the - *stdout* argument was not :data:`PIPE`, this attribute is ``None``. + output from the child process. If the *encoding* or *errors* arguments were + specified or the *universal_newlines* argument was ``True``, the stream is a + text stream, otherwise it is a byte stream. If the *stdout* argument was not + :data:`PIPE`, this attribute is ``None``. .. attribute:: Popen.stderr If the *stderr* argument was :data:`PIPE`, this attribute is a readable stream object as returned by :func:`open`. Reading from the stream provides - error output from the child process. If the *universal_newlines* argument was - ``True``, the stream is a text stream, otherwise it is a byte stream. If the - *stderr* argument was not :data:`PIPE`, this attribute is ``None``. + error output from the child process. If the *encoding* or *errors* arguments + were specified or the *universal_newlines* argument was ``True``, the stream + is a text stream, otherwise it is a byte stream. If the *stderr* argument was + not :data:`PIPE`, this attribute is ``None``. .. warning:: @@ -886,7 +907,9 @@ *timeout* was added. -.. function:: check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False, timeout=None) +.. function:: check_output(args, *, stdin=None, stderr=None, shell=False, \ + encoding=None, errors=None, \ + universal_newlines=False, timeout=None) Run command with arguments and return its output. @@ -1142,7 +1165,7 @@ Return ``(status, output)`` of executing *cmd* in a shell. Execute the string *cmd* in a shell with :meth:`Popen.check_output` and - return a 2-tuple ``(status, output)``. Universal newlines mode is used; + return a 2-tuple ``(status, output)``. The locale encoding is used; see the notes on :ref:`frequently-used-arguments` for more details. A trailing newline is stripped from the output. diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -589,6 +589,9 @@ read the exit status of the child process (Contributed by Victor Stinner in :issue:`26741`). +The :class:`subprocess.Popen` constructor and all functions that pass arguments +through to it now accept *encoding* and *errors* arguments. Specifying either +of these will enable text mode for the *stdin*, *stdout* and *stderr* streams. telnetlib --------- diff --git a/Lib/subprocess.py b/Lib/subprocess.py --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -30,7 +30,8 @@ preexec_fn=None, close_fds=True, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, - restore_signals=True, start_new_session=False, pass_fds=()): + restore_signals=True, start_new_session=False, pass_fds=(), + *, encoding=None, errors=None): Arguments are: @@ -104,20 +105,13 @@ If env is not None, it defines the environment variables for the new process. -If universal_newlines is False, the file objects stdin, stdout and stderr -are opened as binary files, and no line ending conversion is done. +If encoding or errors are specified or universal_newlines is True, the file +objects stdout and stderr are opened in text mode. See io.TextIOWrapper for +the interpretation of these parameters are used. -If universal_newlines is True, the file objects stdout and stderr are -opened as a text file, but lines may be terminated by any of '\n', -the Unix end-of-line convention, '\r', the old Macintosh convention or -'\r\n', the Windows convention. All of these external representations -are seen as '\n' by the Python program. Also, the newlines attribute -of the file objects stdout, stdin and stderr are not updated by the -communicate() method. - -In either case, the process being communicated with should start up -expecting to receive bytes on its standard input and decode them with -the same encoding they are sent in. +If no encoding is specified and universal_newlines is False, the file +objects stdin, stdout and stderr are opened as binary files, and no +line ending conversion is done. The startupinfo and creationflags, if given, will be passed to the underlying CreateProcess() function. They can specify things such as @@ -234,11 +228,8 @@ and stderr, until end-of-file is reached. Wait for process to terminate. The optional input argument should be data to be sent to the child process, or None, if no data should be sent to - the child. If the Popen instance was constructed with universal_newlines - set to True, the input argument should be a string and will be encoded - using the preferred system encoding (see locale.getpreferredencoding); - if universal_newlines is False, the input argument should be a - byte string. + the child. If the Popen instance was constructed in text mode, the + input argument should be a string. Otherwise, it should be bytes. communicate() returns a tuple (stdout, stderr). @@ -808,8 +799,8 @@ """ Return (status, output) of executing cmd in a shell. Execute the string 'cmd' in a shell with 'check_output' and - return a 2-tuple (status, output). Universal newlines mode is used, - meaning that the result with be decoded to a string. + return a 2-tuple (status, output). The locale encoding is used + to decode the output and process newlines. A trailing newline is stripped from the output. The exit status for the command can be interpreted @@ -859,7 +850,7 @@ shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, - pass_fds=()): + pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been @@ -912,6 +903,8 @@ self.pid = None self.returncode = None self.universal_newlines = universal_newlines + self.encoding = encoding + self.errors = errors # Input and output objects. The general principle is like # this: @@ -944,22 +937,28 @@ if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) - if p2cwrite != -1: - self.stdin = io.open(p2cwrite, 'wb', bufsize) - if universal_newlines: - self.stdin = io.TextIOWrapper(self.stdin, write_through=True, - line_buffering=(bufsize == 1)) - if c2pread != -1: - self.stdout = io.open(c2pread, 'rb', bufsize) - if universal_newlines: - self.stdout = io.TextIOWrapper(self.stdout) - if errread != -1: - self.stderr = io.open(errread, 'rb', bufsize) - if universal_newlines: - self.stderr = io.TextIOWrapper(self.stderr) + text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False + try: + if p2cwrite != -1: + self.stdin = io.open(p2cwrite, 'wb', bufsize) + if text_mode: + self.stdin = io.TextIOWrapper(self.stdin, write_through=True, + line_buffering=(bufsize == 1), + encoding=encoding, errors=errors) + if c2pread != -1: + self.stdout = io.open(c2pread, 'rb', bufsize) + if text_mode: + self.stdout = io.TextIOWrapper(self.stdout, + encoding=encoding, errors=errors) + if errread != -1: + self.stderr = io.open(errread, 'rb', bufsize) + if text_mode: + self.stderr = io.TextIOWrapper(self.stderr, + encoding=encoding, errors=errors) + self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, @@ -993,8 +992,8 @@ raise - def _translate_newlines(self, data, encoding): - data = data.decode(encoding) + def _translate_newlines(self, data, encoding, errors): + data = data.decode(encoding, errors) return data.replace("\r\n", "\n").replace("\r", "\n") def __enter__(self): @@ -1779,13 +1778,15 @@ # Translate newlines, if requested. # This also turns bytes into strings. - if self.universal_newlines: + if self.encoding or self.errors or self.universal_newlines: if stdout is not None: stdout = self._translate_newlines(stdout, - self.stdout.encoding) + self.stdout.encoding, + self.stdout.errors) if stderr is not None: stderr = self._translate_newlines(stderr, - self.stderr.encoding) + self.stderr.encoding, + self.stderr.errors) return (stdout, stderr) @@ -1797,8 +1798,10 @@ if self.stdin and self._input is None: self._input_offset = 0 self._input = input - if self.universal_newlines and input is not None: - self._input = self._input.encode(self.stdin.encoding) + if input is not None and ( + self.encoding or self.errors or self.universal_newlines): + self._input = self._input.encode(self.stdin.encoding, + self.stdin.errors) def send_signal(self, sig): diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -894,31 +894,42 @@ # # UTF-16 and UTF-32-BE are sufficient to check both with BOM and # without, and UTF-16 and UTF-32. - import _bootlocale for encoding in ['utf-16', 'utf-32-be']: - old_getpreferredencoding = _bootlocale.getpreferredencoding - # Indirectly via io.TextIOWrapper, Popen() defaults to - # locale.getpreferredencoding(False) and earlier in Python 3.2 to - # locale.getpreferredencoding(). - def getpreferredencoding(do_setlocale=True): - return encoding code = ("import sys; " r"sys.stdout.buffer.write('1\r\n2\r3\n4'.encode('%s'))" % encoding) args = [sys.executable, '-c', code] - try: - _bootlocale.getpreferredencoding = getpreferredencoding - # We set stdin to be non-None because, as of this writing, - # a different code path is used when the number of pipes is - # zero or one. - popen = subprocess.Popen(args, universal_newlines=True, - stdin=subprocess.PIPE, - stdout=subprocess.PIPE) - stdout, stderr = popen.communicate(input='') - finally: - _bootlocale.getpreferredencoding = old_getpreferredencoding + # We set stdin to be non-None because, as of this writing, + # a different code path is used when the number of pipes is + # zero or one. + popen = subprocess.Popen(args, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + encoding=encoding) + stdout, stderr = popen.communicate(input='') self.assertEqual(stdout, '1\n2\n3\n4') + def test_communicate_errors(self): + for errors, expected in [ + ('ignore', ''), + ('replace', '\ufffd\ufffd'), + ('surrogateescape', '\udc80\udc80'), + ('backslashreplace', '\\x80\\x80'), + ]: + code = ("import sys; " + r"sys.stdout.buffer.write(b'[\x80\x80]')") + args = [sys.executable, '-c', code] + # We set stdin to be non-None because, as of this writing, + # a different code path is used when the number of pipes is + # zero or one. + popen = subprocess.Popen(args, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + encoding='utf-8', + errors=errors) + stdout, stderr = popen.communicate(input='') + self.assertEqual(stdout, '[{}]'.format(expected)) + def test_no_leaking(self): # Make sure we leak no resources if not mswindows: @@ -2539,6 +2550,18 @@ with p: self.assertIn(b"physalis", p.stdout.read()) + def test_shell_encodings(self): + # Run command through the shell (string) + for enc in ['ansi', 'oem']: + newenv = os.environ.copy() + newenv["FRUIT"] = "physalis" + p = subprocess.Popen("set", shell=1, + stdout=subprocess.PIPE, + env=newenv, + encoding=enc) + with p: + self.assertIn("physalis", p.stdout.read(), enc) + def test_call_string(self): # call() function with string argument on Windows rc = subprocess.call(sys.executable + diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -237,6 +237,8 @@ Windows ------- +- Issue #6135: Adds encoding and errors parameters to subprocess. + - Issue #27959: Adds oem encoding, alias ansi to mbcs, move aliasmbcs to codec lookup. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 23:23:11 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 03:23:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_get_skipIf_from_the_right_?= =?utf-8?q?place?= Message-ID: <20160907032311.584.3828.EF04C5F5@psf.io> https://hg.python.org/cpython/rev/b38e68ff9751 changeset: 103204:b38e68ff9751 user: Benjamin Peterson date: Tue Sep 06 20:22:41 2016 -0700 summary: get skipIf from the right place files: Lib/test/test_gdb.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -110,7 +110,7 @@ BREAKPOINT_FN='builtin_id' - at support.skipIf(support.PGO, "not useful for PGO") + at unittest.skipIf(support.PGO, "not useful for PGO") class DebuggerTests(unittest.TestCase): """Test that the debugger can debug Python.""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 23:40:25 2016 From: python-checkins at python.org (steve.dower) Date: Wed, 07 Sep 2016 03:40:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327731=3A_Opt-out_?= =?utf-8?q?of_MAX=5FPATH_on_Windows_10?= Message-ID: <20160907034025.39027.34433.7D9F1C82@psf.io> https://hg.python.org/cpython/rev/26601191b368 changeset: 103205:26601191b368 user: Steve Dower date: Tue Sep 06 20:40:11 2016 -0700 summary: Issue #27731: Opt-out of MAX_PATH on Windows 10 files: Doc/using/windows.rst | 25 +++++++++++++++++++++++++ Doc/whatsnew/3.6.rst | 5 ++++- Misc/NEWS | 2 ++ PC/python.manifest | 7 ++++++- 4 files changed, 37 insertions(+), 2 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -74,6 +74,31 @@ * If selected, the install directory will be added to the system :envvar:`PATH` * Shortcuts are available for all users +.. _max-path: + +Removing the MAX_PATH Limitation +-------------------------------- + +Windows historically has limited path lengths to 260 characters. This meant that +paths longer than this would not resolve and errors would result. + +In the latest versions of Windows, this limitation can be expanded to +approximately 32,000 characters. Your administrator will need to activate the +"Enable Win32 long paths" group policy, or set the registry value +``HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem at LongPathsEnabled`` +to ``1``. + +This allows the :func:`open` function, the :mod:`os` module and most other +path functionality to accept and return paths longer than 260 characters when +using strings. (Use of bytes as paths is deprecated on Windows, and this feature +is not available when using bytes.) + +After changing the above option, no further configuration is required. + +.. versionchanged:: 3.6 + + Support for long paths was enabled in Python. + .. _install-quiet-option: Installing Without UI diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -83,6 +83,10 @@ command line arguments or a config file). Handling of shebang lines remains unchanged - "python" refers to Python 2 in that case. +* ``python.exe`` and ``pythonw.exe`` have been marked as long-path aware, + which means that when the 260 character path limit may no longer apply. + See :ref:`removing the MAX_PATH limitation ` for details. + .. PEP-sized items next. .. _pep-4XX: @@ -507,7 +511,6 @@ :func:`os.getrandom` function. (Contributed by Victor Stinner, part of the :pep:`524`) - pickle ------ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -237,6 +237,8 @@ Windows ------- +- Issue #27731: Opt-out of MAX_PATH on Windows 10 + - Issue #6135: Adds encoding and errors parameters to subprocess. - Issue #27959: Adds oem encoding, alias ansi to mbcs, move aliasmbcs to diff --git a/PC/python.manifest b/PC/python.manifest --- a/PC/python.manifest +++ b/PC/python.manifest @@ -16,10 +16,15 @@ + + + true + + - \ No newline at end of file + -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 23:42:29 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 03:42:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_promote_some_s?= =?utf-8?q?hifts_to_unsigned=2C_so_as_not_to_invoke_undefined_behavior?= Message-ID: <20160907034228.78547.74783.F01D74AF@psf.io> https://hg.python.org/cpython/rev/2454feeb5442 changeset: 103206:2454feeb5442 branch: 3.5 parent: 103194:66feda02f2a5 user: Benjamin Peterson date: Tue Sep 06 20:40:04 2016 -0700 summary: promote some shifts to unsigned, so as not to invoke undefined behavior files: Objects/unicodeobject.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -4944,7 +4944,7 @@ mark is skipped, in all other modes, it is copied to the output stream as-is (giving a ZWNBSP character). */ if (bo == 0 && size >= 4) { - Py_UCS4 bom = (q[3] << 24) | (q[2] << 16) | (q[1] << 8) | q[0]; + Py_UCS4 bom = ((unsigned int)q[3] << 24) | (q[2] << 16) | (q[1] << 8) | q[0]; if (bom == 0x0000FEFF) { bo = -1; q += 4; @@ -4986,7 +4986,7 @@ Py_ssize_t pos = writer.pos; if (le) { do { - ch = (q[3] << 24) | (q[2] << 16) | (q[1] << 8) | q[0]; + ch = ((unsigned int)q[3] << 24) | (q[2] << 16) | (q[1] << 8) | q[0]; if (ch > maxch) break; if (kind != PyUnicode_1BYTE_KIND && @@ -4998,7 +4998,7 @@ } else { do { - ch = (q[0] << 24) | (q[1] << 16) | (q[2] << 8) | q[3]; + ch = ((unsigned int)q[0] << 24) | (q[1] << 16) | (q[2] << 8) | q[3]; if (ch > maxch) break; if (kind != PyUnicode_1BYTE_KIND && -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 23:42:29 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 03:42:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160907034229.3633.48556.27FEA259@psf.io> https://hg.python.org/cpython/rev/e588a0198b24 changeset: 103208:e588a0198b24 parent: 103205:26601191b368 parent: 103206:2454feeb5442 user: Benjamin Peterson date: Tue Sep 06 20:42:17 2016 -0700 summary: merge 3.5 files: Objects/unicodeobject.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -5229,7 +5229,7 @@ mark is skipped, in all other modes, it is copied to the output stream as-is (giving a ZWNBSP character). */ if (bo == 0 && size >= 4) { - Py_UCS4 bom = (q[3] << 24) | (q[2] << 16) | (q[1] << 8) | q[0]; + Py_UCS4 bom = ((unsigned int)q[3] << 24) | (q[2] << 16) | (q[1] << 8) | q[0]; if (bom == 0x0000FEFF) { bo = -1; q += 4; @@ -5271,7 +5271,7 @@ Py_ssize_t pos = writer.pos; if (le) { do { - ch = (q[3] << 24) | (q[2] << 16) | (q[1] << 8) | q[0]; + ch = ((unsigned int)q[3] << 24) | (q[2] << 16) | (q[1] << 8) | q[0]; if (ch > maxch) break; if (kind != PyUnicode_1BYTE_KIND && @@ -5283,7 +5283,7 @@ } else { do { - ch = (q[0] << 24) | (q[1] << 16) | (q[2] << 8) | q[3]; + ch = ((unsigned int)q[0] << 24) | (q[1] << 16) | (q[2] << 8) | q[3]; if (ch > maxch) break; if (kind != PyUnicode_1BYTE_KIND && -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Tue Sep 6 23:42:29 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 03:42:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_promote_some_s?= =?utf-8?q?hifts_to_unsigned=2C_so_as_not_to_invoke_undefined_behavior?= Message-ID: <20160907034229.12563.4183.D9724D44@psf.io> https://hg.python.org/cpython/rev/923a27028cec changeset: 103207:923a27028cec branch: 2.7 parent: 103193:75dae0b2ccb3 user: Benjamin Peterson date: Tue Sep 06 20:40:04 2016 -0700 summary: promote some shifts to unsigned, so as not to invoke undefined behavior files: Objects/unicodeobject.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -2308,7 +2308,7 @@ stream as-is (giving a ZWNBSP character). */ if (bo == 0) { if (size >= 4) { - const Py_UCS4 bom = (q[iorder[3]] << 24) | (q[iorder[2]] << 16) | + const Py_UCS4 bom = ((unsigned int)q[iorder[3]] << 24) | (q[iorder[2]] << 16) | (q[iorder[1]] << 8) | q[iorder[0]]; #ifdef BYTEORDER_IS_LITTLE_ENDIAN if (bom == 0x0000FEFF) { @@ -2378,7 +2378,7 @@ /* The remaining input chars are ignored if the callback chooses to skip the input */ } - ch = (q[iorder[3]] << 24) | (q[iorder[2]] << 16) | + ch = ((unsigned int)q[iorder[3]] << 24) | (q[iorder[2]] << 16) | (q[iorder[1]] << 8) | q[iorder[0]]; if (ch >= 0x110000) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 00:13:39 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 07 Sep 2016 04:13:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTA1?= =?utf-8?q?=3A_Docs_for_typing=2EType=5BC=5D=2C_by_Michael_Lee=2E?= Message-ID: <20160907041337.618.96021.6ACF6BC4@psf.io> https://hg.python.org/cpython/rev/9fb30d6af6a9 changeset: 103209:9fb30d6af6a9 branch: 3.5 parent: 103206:2454feeb5442 user: Guido van Rossum date: Tue Sep 06 21:12:44 2016 -0700 summary: Issue #27905: Docs for typing.Type[C], by Michael Lee. files: Doc/library/typing.rst | 39 ++++++++++++++++++++++++++++++ 1 files changed, 39 insertions(+), 0 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -502,6 +502,45 @@ except KeyError: return default +.. class:: Type + + A variable annotated with ``C`` may accept a value of type ``C``. In + contrast, a variable annotated with ``Type[C]`` may accept values that are + classes themselves -- specifically, it will accept the *class object* of + ``C``. For example:: + + a = 3 # Has type 'int' + b = int # Has type 'Type[int]' + c = type(a) # Also has type 'Type[int]' + + Note that ``Type[C]`` is covariant:: + + class User: ... + class BasicUser(User): ... + class ProUser(User): ... + class TeamUser(User): ... + + # Accepts User, BasicUser, ProUser, TeamUser, ... + def make_new_user(user_class: Type[User]) -> User: + # ... + return user_class() + + The fact that ``Type[C]`` is covariant implies that all subclasses of + ``C`` should implement the same constructor signature and class method + signatures as ``C``. The type checker should flag violations of this, + but should also allow constructor calls in subclasses that match the + constructor calls in the indicated base class. How the type checker is + required to handle this particular case may change in future revisions of + PEP 484. + + The only legal parameters for ``Type`` are classes, unions of classes, and + ``Any``. For example:: + + def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... + + ``Type[Any]`` is equivalent to ``Type`` which in turn is equivalent + to ``type``, which is the root of Python's metaclass hierarchy. + .. class:: Iterable(Generic[T_co]) A generic version of the :class:`collections.abc.Iterable`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 00:13:42 2016 From: python-checkins at python.org (guido.van.rossum) Date: Wed, 07 Sep 2016 04:13:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327905=3A_Docs_for_typing=2EType=5BC=5D=2C_by_Mi?= =?utf-8?b?Y2hhZWwgTGVlLiAoTWVyZ2UgMy41LT4zLjYp?= Message-ID: <20160907041337.1385.52308.3872F5C2@psf.io> https://hg.python.org/cpython/rev/1705cde6266d changeset: 103210:1705cde6266d parent: 103208:e588a0198b24 parent: 103209:9fb30d6af6a9 user: Guido van Rossum date: Tue Sep 06 21:13:15 2016 -0700 summary: Issue #27905: Docs for typing.Type[C], by Michael Lee. (Merge 3.5->3.6) files: Doc/library/typing.rst | 39 ++++++++++++++++++++++++++++++ 1 files changed, 39 insertions(+), 0 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -502,6 +502,45 @@ except KeyError: return default +.. class:: Type + + A variable annotated with ``C`` may accept a value of type ``C``. In + contrast, a variable annotated with ``Type[C]`` may accept values that are + classes themselves -- specifically, it will accept the *class object* of + ``C``. For example:: + + a = 3 # Has type 'int' + b = int # Has type 'Type[int]' + c = type(a) # Also has type 'Type[int]' + + Note that ``Type[C]`` is covariant:: + + class User: ... + class BasicUser(User): ... + class ProUser(User): ... + class TeamUser(User): ... + + # Accepts User, BasicUser, ProUser, TeamUser, ... + def make_new_user(user_class: Type[User]) -> User: + # ... + return user_class() + + The fact that ``Type[C]`` is covariant implies that all subclasses of + ``C`` should implement the same constructor signature and class method + signatures as ``C``. The type checker should flag violations of this, + but should also allow constructor calls in subclasses that match the + constructor calls in the indicated base class. How the type checker is + required to handle this particular case may change in future revisions of + PEP 484. + + The only legal parameters for ``Type`` are classes, unions of classes, and + ``Any``. For example:: + + def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... + + ``Type[Any]`` is equivalent to ``Type`` which in turn is equivalent + to ``type``, which is the root of Python's metaclass hierarchy. + .. class:: Iterable(Generic[T_co]) A generic version of :class:`collections.abc.Iterable`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 02:49:59 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Wed, 07 Sep 2016 06:49:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325596=3A_Falls_ba?= =?utf-8?q?ck_to_listdir_in_glob_for_bytes_paths_on_Windows=2E?= Message-ID: <20160907064959.39296.23988.B398E682@psf.io> https://hg.python.org/cpython/rev/48e573e0a610 changeset: 103211:48e573e0a610 user: Serhiy Storchaka date: Wed Sep 07 09:49:42 2016 +0300 summary: Issue #25596: Falls back to listdir in glob for bytes paths on Windows. files: Lib/glob.py | 23 ++++++++++++++++------- 1 files changed, 16 insertions(+), 7 deletions(-) diff --git a/Lib/glob.py b/Lib/glob.py --- a/Lib/glob.py +++ b/Lib/glob.py @@ -118,13 +118,22 @@ else: dirname = os.curdir try: - with os.scandir(dirname) as it: - for entry in it: - try: - if not dironly or entry.is_dir(): - yield entry.name - except OSError: - pass + if os.name == 'nt' and isinstance(dirname, bytes): + names = os.listdir(dirname) + if dironly: + for name in names: + if os.path.isdir(os.path.join(dirname, name)): + yield name + else: + yield from names + else: + with os.scandir(dirname) as it: + for entry in it: + try: + if not dironly or entry.is_dir(): + yield entry.name + except OSError: + pass except OSError: return -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 03:08:58 2016 From: python-checkins at python.org (raymond.hettinger) Date: Wed, 07 Sep 2016 07:08:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Rename_weighted=5Fchoices?= =?utf-8?q?=28=29_to_just_choices=28=29?= Message-ID: <20160907070858.12727.64078.27EF18FC@psf.io> https://hg.python.org/cpython/rev/e62ac2ed0fc4 changeset: 103212:e62ac2ed0fc4 user: Raymond Hettinger date: Wed Sep 07 00:08:44 2016 -0700 summary: Rename weighted_choices() to just choices() files: Doc/library/random.rst | 2 +- Lib/random.py | 6 +- Lib/test/test_random.py | 54 ++++++++++++++-------------- Misc/NEWS | 2 +- 4 files changed, 32 insertions(+), 32 deletions(-) diff --git a/Doc/library/random.rst b/Doc/library/random.rst --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -124,7 +124,7 @@ Return a random element from the non-empty sequence *seq*. If *seq* is empty, raises :exc:`IndexError`. -.. function:: weighted_choices(k, population, weights=None, *, cum_weights=None) +.. function:: choices(k, population, weights=None, *, cum_weights=None) Return a *k* sized list of elements chosen from the *population* with replacement. If the *population* is empty, raises :exc:`IndexError`. diff --git a/Lib/random.py b/Lib/random.py --- a/Lib/random.py +++ b/Lib/random.py @@ -51,7 +51,7 @@ "randrange","shuffle","normalvariate","lognormvariate", "expovariate","vonmisesvariate","gammavariate","triangular", "gauss","betavariate","paretovariate","weibullvariate", - "getstate","setstate", "getrandbits", "weighted_choices", + "getstate","setstate", "getrandbits", "choices", "SystemRandom"] NV_MAGICCONST = 4 * _exp(-0.5)/_sqrt(2.0) @@ -337,7 +337,7 @@ result[i] = population[j] return result - def weighted_choices(self, k, population, weights=None, *, cum_weights=None): + def choices(self, k, population, weights=None, *, cum_weights=None): """Return a k sized list of population elements chosen with replacement. If the relative weights or cumulative weights are not specified, @@ -749,7 +749,7 @@ randrange = _inst.randrange sample = _inst.sample shuffle = _inst.shuffle -weighted_choices = _inst.weighted_choices +choices = _inst.choices normalvariate = _inst.normalvariate lognormvariate = _inst.lognormvariate expovariate = _inst.expovariate diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py --- a/Lib/test/test_random.py +++ b/Lib/test/test_random.py @@ -142,8 +142,8 @@ def test_sample_on_dicts(self): self.assertRaises(TypeError, self.gen.sample, dict.fromkeys('abcdef'), 2) - def test_weighted_choices(self): - weighted_choices = self.gen.weighted_choices + def test_choices(self): + choices = self.gen.choices data = ['red', 'green', 'blue', 'yellow'] str_data = 'abcd' range_data = range(4) @@ -151,63 +151,63 @@ # basic functionality for sample in [ - weighted_choices(5, data), - weighted_choices(5, data, range(4)), - weighted_choices(k=5, population=data, weights=range(4)), - weighted_choices(k=5, population=data, cum_weights=range(4)), + choices(5, data), + choices(5, data, range(4)), + choices(k=5, population=data, weights=range(4)), + choices(k=5, population=data, cum_weights=range(4)), ]: self.assertEqual(len(sample), 5) self.assertEqual(type(sample), list) self.assertTrue(set(sample) <= set(data)) # test argument handling - with self.assertRaises(TypeError): # missing arguments - weighted_choices(2) + with self.assertRaises(TypeError): # missing arguments + choices(2) - self.assertEqual(weighted_choices(0, data), []) # k == 0 - self.assertEqual(weighted_choices(-1, data), []) # negative k behaves like ``[0] * -1`` + self.assertEqual(choices(0, data), []) # k == 0 + self.assertEqual(choices(-1, data), []) # negative k behaves like ``[0] * -1`` with self.assertRaises(TypeError): - weighted_choices(2.5, data) # k is a float + choices(2.5, data) # k is a float - self.assertTrue(set(weighted_choices(5, str_data)) <= set(str_data)) # population is a string sequence - self.assertTrue(set(weighted_choices(5, range_data)) <= set(range_data)) # population is a range + self.assertTrue(set(choices(5, str_data)) <= set(str_data)) # population is a string sequence + self.assertTrue(set(choices(5, range_data)) <= set(range_data)) # population is a range with self.assertRaises(TypeError): - weighted_choices(2.5, set_data) # population is not a sequence + choices(2.5, set_data) # population is not a sequence - self.assertTrue(set(weighted_choices(5, data, None)) <= set(data)) # weights is None - self.assertTrue(set(weighted_choices(5, data, weights=None)) <= set(data)) + self.assertTrue(set(choices(5, data, None)) <= set(data)) # weights is None + self.assertTrue(set(choices(5, data, weights=None)) <= set(data)) with self.assertRaises(ValueError): - weighted_choices(5, data, [1,2]) # len(weights) != len(population) + choices(5, data, [1,2]) # len(weights) != len(population) with self.assertRaises(IndexError): - weighted_choices(5, data, [0]*4) # weights sum to zero + choices(5, data, [0]*4) # weights sum to zero with self.assertRaises(TypeError): - weighted_choices(5, data, 10) # non-iterable weights + choices(5, data, 10) # non-iterable weights with self.assertRaises(TypeError): - weighted_choices(5, data, [None]*4) # non-numeric weights + choices(5, data, [None]*4) # non-numeric weights for weights in [ [15, 10, 25, 30], # integer weights [15.1, 10.2, 25.2, 30.3], # float weights [Fraction(1, 3), Fraction(2, 6), Fraction(3, 6), Fraction(4, 6)], # fractional weights [True, False, True, False] # booleans (include / exclude) ]: - self.assertTrue(set(weighted_choices(5, data, weights)) <= set(data)) + self.assertTrue(set(choices(5, data, weights)) <= set(data)) with self.assertRaises(ValueError): - weighted_choices(5, data, cum_weights=[1,2]) # len(weights) != len(population) + choices(5, data, cum_weights=[1,2]) # len(weights) != len(population) with self.assertRaises(IndexError): - weighted_choices(5, data, cum_weights=[0]*4) # cum_weights sum to zero + choices(5, data, cum_weights=[0]*4) # cum_weights sum to zero with self.assertRaises(TypeError): - weighted_choices(5, data, cum_weights=10) # non-iterable cum_weights + choices(5, data, cum_weights=10) # non-iterable cum_weights with self.assertRaises(TypeError): - weighted_choices(5, data, cum_weights=[None]*4) # non-numeric cum_weights + choices(5, data, cum_weights=[None]*4) # non-numeric cum_weights with self.assertRaises(TypeError): - weighted_choices(5, data, range(4), cum_weights=range(4)) # both weights and cum_weights + choices(5, data, range(4), cum_weights=range(4)) # both weights and cum_weights for weights in [ [15, 10, 25, 30], # integer cum_weights [15.1, 10.2, 25.2, 30.3], # float cum_weights [Fraction(1, 3), Fraction(2, 6), Fraction(3, 6), Fraction(4, 6)], # fractional cum_weights ]: - self.assertTrue(set(weighted_choices(5, data, cum_weights=weights)) <= set(data)) + self.assertTrue(set(choices(5, data, cum_weights=weights)) <= set(data)) def test_gauss(self): # Ensure that the seed() method initializes all the hidden state. In diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -101,7 +101,7 @@ - Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name fields in X.509 certs. -- Issue #18844: Add random.weighted_choices(). +- Issue #18844: Add random.choices(). - Issue #25761: Improved error reporting about truncated pickle data in C implementation of unpickler. UnpicklingError is now raised instead of -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 03:53:40 2016 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 07 Sep 2016 07:53:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_issue20842_-_null_merge_with_3=2E5?= Message-ID: <20160907075340.1622.66858.FEB3A20B@psf.io> https://hg.python.org/cpython/rev/a4360b1b45a8 changeset: 103214:a4360b1b45a8 parent: 103212:e62ac2ed0fc4 parent: 103213:7537ca1c2aaf user: Senthil Kumaran date: Wed Sep 07 00:53:17 2016 -0700 summary: issue20842 - null merge with 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 03:53:40 2016 From: python-checkins at python.org (senthil.kumaran) Date: Wed, 07 Sep 2016 07:53:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogW2JhY2twb3J0IHRv?= =?utf-8?q?_3=2E5=5D_-_issue26896_-_Disambiguate_uses_of_=22importer=22_wi?= =?utf-8?b?dGggImZpbmRlciIu?= Message-ID: <20160907075340.13891.33767.B4F07B99@psf.io> https://hg.python.org/cpython/rev/7537ca1c2aaf changeset: 103213:7537ca1c2aaf branch: 3.5 parent: 103209:9fb30d6af6a9 user: Senthil Kumaran date: Wed Sep 07 00:52:20 2016 -0700 summary: [backport to 3.5] - issue26896 - Disambiguate uses of "importer" with "finder". files: Doc/c-api/import.rst | 8 ++-- Lib/pkgutil.py | 18 +++++----- Lib/runpy.py | 4 +- Lib/test/test_importlib/import_/test_meta_path.py | 1 - Lib/test/test_importlib/util.py | 1 - Lib/test/test_pkgutil.py | 2 +- Misc/ACKS | 1 + Python/import.c | 7 ++- 8 files changed, 21 insertions(+), 21 deletions(-) diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -207,13 +207,13 @@ .. c:function:: PyObject* PyImport_GetImporter(PyObject *path) - Return an importer object for a :data:`sys.path`/:attr:`pkg.__path__` item + Return a finder object for a :data:`sys.path`/:attr:`pkg.__path__` item *path*, possibly by fetching it from the :data:`sys.path_importer_cache` dict. If it wasn't yet cached, traverse :data:`sys.path_hooks` until a hook is found that can handle the path item. Return ``None`` if no hook could; - this tells our caller it should fall back to the built-in import mechanism. - Cache the result in :data:`sys.path_importer_cache`. Return a new reference - to the importer object. + this tells our caller that the :term:`path based finder` could not find a + finder for this path item. Cache the result in :data:`sys.path_importer_cache`. + Return a new reference to the finder object. .. c:function:: void _PyImport_Init() diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py --- a/Lib/pkgutil.py +++ b/Lib/pkgutil.py @@ -45,7 +45,7 @@ def walk_packages(path=None, prefix='', onerror=None): - """Yields (module_loader, name, ispkg) for all modules recursively + """Yields (module_finder, name, ispkg) for all modules recursively on path, or, if path is None, all accessible modules. 'path' should be either None or a list of paths to look for @@ -102,7 +102,7 @@ def iter_modules(path=None, prefix=''): - """Yields (module_loader, name, ispkg) for all submodules on path, + """Yields (module_finder, name, ispkg) for all submodules on path, or, if path is None, all top-level modules on sys.path. 'path' should be either None or a list of paths to look for @@ -184,10 +184,10 @@ imp = importlib.import_module('imp') class ImpImporter: - """PEP 302 Importer that wraps Python's "classic" import algorithm + """PEP 302 Finder that wraps Python's "classic" import algorithm - ImpImporter(dirname) produces a PEP 302 importer that searches that - directory. ImpImporter(None) produces a PEP 302 importer that searches + ImpImporter(dirname) produces a PEP 302 finder that searches that + directory. ImpImporter(None) produces a PEP 302 finder that searches the current sys.path, plus any modules that are frozen or built-in. Note that ImpImporter does not currently support being used by placement @@ -397,7 +397,7 @@ def get_importer(path_item): """Retrieve a finder for the given path item - The returned importer is cached in sys.path_importer_cache + The returned finder is cached in sys.path_importer_cache if it was newly created by a path hook. The cache (or part of it) can be cleared manually if a @@ -421,14 +421,14 @@ def iter_importers(fullname=""): """Yield finders for the given module name - If fullname contains a '.', the importers will be for the package + If fullname contains a '.', the finders will be for the package containing fullname, otherwise they will be all registered top level - importers (i.e. those on both sys.meta_path and sys.path_hooks). + finders (i.e. those on both sys.meta_path and sys.path_hooks). If the named module is in a package, that package is imported as a side effect of invoking this function. - If no module name is specified, all top level importers are produced. + If no module name is specified, all top level finders are produced. """ if fullname.startswith('.'): msg = "Relative module name {!r} not supported".format(fullname) diff --git a/Lib/runpy.py b/Lib/runpy.py --- a/Lib/runpy.py +++ b/Lib/runpy.py @@ -98,7 +98,7 @@ # may be cleared when the temporary module goes away return mod_globals.copy() -# Helper to get the loader, code and filename for a module +# Helper to get the full name, spec and code for a module def _get_module_details(mod_name, error=ImportError): if mod_name.startswith("."): raise error("Relative module names not supported") @@ -262,7 +262,7 @@ return _run_module_code(code, init_globals, run_name, pkg_name=pkg_name, script_name=fname) else: - # Importer is defined for path, so add it to + # Finder is defined for path, so add it to # the start of sys.path sys.path.insert(0, path_name) try: diff --git a/Lib/test/test_importlib/import_/test_meta_path.py b/Lib/test/test_importlib/import_/test_meta_path.py --- a/Lib/test/test_importlib/import_/test_meta_path.py +++ b/Lib/test/test_importlib/import_/test_meta_path.py @@ -76,7 +76,6 @@ self.__import__(mod_name) assert len(log) == 1 args = log[0][0] - kwargs = log[0][1] # Assuming all arguments are positional. self.assertEqual(args[0], mod_name) self.assertIsNone(args[1]) diff --git a/Lib/test/test_importlib/util.py b/Lib/test/test_importlib/util.py --- a/Lib/test/test_importlib/util.py +++ b/Lib/test/test_importlib/util.py @@ -266,7 +266,6 @@ module = self.modules[fullname] except KeyError: return None - is_package = hasattr(module, '__path__') spec = util.spec_from_file_location( fullname, module.__file__, loader=self, submodule_search_locations=getattr(module, '__path__', None)) diff --git a/Lib/test/test_pkgutil.py b/Lib/test/test_pkgutil.py --- a/Lib/test/test_pkgutil.py +++ b/Lib/test/test_pkgutil.py @@ -205,7 +205,7 @@ del sys.meta_path[0] def test_getdata_pep302(self): - # Use a dummy importer/loader + # Use a dummy finder/loader self.assertEqual(pkgutil.get_data('foo', 'dummy'), "Hello, world!") del sys.modules['foo'] diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -990,6 +990,7 @@ Jason V. Miller Jay T. Miller Katie Miller +Oren Milman Roman Milner Julien Miotte Andrii V. Mishkovskyi diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -960,12 +960,13 @@ } -/* Return an importer object for a sys.path/pkg.__path__ item 'p', +/* Return a finder object for a sys.path/pkg.__path__ item 'p', possibly by fetching it from the path_importer_cache dict. If it wasn't yet cached, traverse path_hooks until a hook is found that can handle the path item. Return None if no hook could; - this tells our caller it should fall back to the builtin - import mechanism. Cache the result in path_importer_cache. + this tells our caller that the path based finder could not find + a finder for this path item. Cache the result in + path_importer_cache. Returns a borrowed reference. */ static PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 03:58:23 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Wed, 07 Sep 2016 07:58:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326032=3A_Optimize?= =?utf-8?q?d_globbing_in_pathlib_by_using_os=2Escandir=28=29=3B_it_is_now?= Message-ID: <20160907075822.3661.22187.C45B3B22@psf.io> https://hg.python.org/cpython/rev/927665c4aaab changeset: 103215:927665c4aaab user: Serhiy Storchaka date: Wed Sep 07 10:58:05 2016 +0300 summary: Issue #26032: Optimized globbing in pathlib by using os.scandir(); it is now about 1.5--4 times faster. files: Doc/whatsnew/3.6.rst | 3 + Lib/pathlib.py | 94 +++++++++++++------------------ Misc/NEWS | 3 + 3 files changed, 45 insertions(+), 55 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -808,6 +808,9 @@ :mod:`glob` module; they are now about 3--6 times faster. (Contributed by Serhiy Storchaka in :issue:`25596`). +* Optimized globbing in :mod:`pathlib` by using :func:`os.scandir`; + it is now about 1.5--4 times faster. + (Contributed by Serhiy Storchaka in :issue:`26032`). Build and C API Changes ======================= diff --git a/Lib/pathlib.py b/Lib/pathlib.py --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -385,6 +385,8 @@ listdir = _wrap_strfunc(os.listdir) + scandir = _wrap_strfunc(os.scandir) + chmod = _wrap_strfunc(os.chmod) if hasattr(os, "lchmod"): @@ -429,25 +431,6 @@ # Globbing helpers # - at contextmanager -def _cached(func): - try: - func.__cached__ - yield func - except AttributeError: - cache = {} - def wrapper(*args): - try: - return cache[args] - except KeyError: - value = cache[args] = func(*args) - return value - wrapper.__cached__ = True - try: - yield wrapper - finally: - cache.clear() - def _make_selector(pattern_parts): pat = pattern_parts[0] child_parts = pattern_parts[1:] @@ -473,8 +456,10 @@ self.child_parts = child_parts if child_parts: self.successor = _make_selector(child_parts) + self.dironly = True else: self.successor = _TerminatingSelector() + self.dironly = False def select_from(self, parent_path): """Iterate over all child paths of `parent_path` matched by this @@ -482,13 +467,15 @@ path_cls = type(parent_path) is_dir = path_cls.is_dir exists = path_cls.exists - listdir = parent_path._accessor.listdir - return self._select_from(parent_path, is_dir, exists, listdir) + scandir = parent_path._accessor.scandir + if not is_dir(parent_path): + return iter([]) + return self._select_from(parent_path, is_dir, exists, scandir) class _TerminatingSelector: - def _select_from(self, parent_path, is_dir, exists, listdir): + def _select_from(self, parent_path, is_dir, exists, scandir): yield parent_path @@ -498,13 +485,11 @@ self.name = name _Selector.__init__(self, child_parts) - def _select_from(self, parent_path, is_dir, exists, listdir): + def _select_from(self, parent_path, is_dir, exists, scandir): try: - if not is_dir(parent_path): - return path = parent_path._make_child_relpath(self.name) - if exists(path): - for p in self.successor._select_from(path, is_dir, exists, listdir): + if (is_dir if self.dironly else exists)(path): + for p in self.successor._select_from(path, is_dir, exists, scandir): yield p except PermissionError: return @@ -516,17 +501,18 @@ self.pat = re.compile(fnmatch.translate(pat)) _Selector.__init__(self, child_parts) - def _select_from(self, parent_path, is_dir, exists, listdir): + def _select_from(self, parent_path, is_dir, exists, scandir): try: - if not is_dir(parent_path): - return cf = parent_path._flavour.casefold - for name in listdir(parent_path): - casefolded = cf(name) - if self.pat.match(casefolded): - path = parent_path._make_child_relpath(name) - for p in self.successor._select_from(path, is_dir, exists, listdir): - yield p + entries = list(scandir(parent_path)) + for entry in entries: + if not self.dironly or entry.is_dir(): + name = entry.name + casefolded = cf(name) + if self.pat.match(casefolded): + path = parent_path._make_child_relpath(name) + for p in self.successor._select_from(path, is_dir, exists, scandir): + yield p except PermissionError: return @@ -537,32 +523,30 @@ def __init__(self, pat, child_parts): _Selector.__init__(self, child_parts) - def _iterate_directories(self, parent_path, is_dir, listdir): + def _iterate_directories(self, parent_path, is_dir, scandir): yield parent_path try: - for name in listdir(parent_path): - path = parent_path._make_child_relpath(name) - if is_dir(path) and not path.is_symlink(): - for p in self._iterate_directories(path, is_dir, listdir): + entries = list(scandir(parent_path)) + for entry in entries: + if entry.is_dir() and not entry.is_symlink(): + path = parent_path._make_child_relpath(entry.name) + for p in self._iterate_directories(path, is_dir, scandir): yield p except PermissionError: return - def _select_from(self, parent_path, is_dir, exists, listdir): + def _select_from(self, parent_path, is_dir, exists, scandir): try: - if not is_dir(parent_path): - return - with _cached(listdir) as listdir: - yielded = set() - try: - successor_select = self.successor._select_from - for starting_point in self._iterate_directories(parent_path, is_dir, listdir): - for p in successor_select(starting_point, is_dir, exists, listdir): - if p not in yielded: - yield p - yielded.add(p) - finally: - yielded.clear() + yielded = set() + try: + successor_select = self.successor._select_from + for starting_point in self._iterate_directories(parent_path, is_dir, scandir): + for p in successor_select(starting_point, is_dir, exists, scandir): + if p not in yielded: + yield p + yielded.add(p) + finally: + yielded.clear() except PermissionError: return diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -109,6 +109,9 @@ - Issue #26798: Add BLAKE2 (blake2b and blake2s) to hashlib. +- Issue #26032: Optimized globbing in pathlib by using os.scandir(); it is now + about 1.5--4 times faster. + - Issue #25596: Optimized glob() and iglob() functions in the glob module; they are now about 3--6 times faster. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 05:39:39 2016 From: python-checkins at python.org (christian.heimes) Date: Wed, 07 Sep 2016 09:39:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_blake2=3A_silence_two_more?= =?utf-8?q?_warnings_on_platforms_with_size=5Ft_=3C_uint64=5Ft=2E_Don=27t?= Message-ID: <20160907093932.3535.45217.0E2124B0@psf.io> https://hg.python.org/cpython/rev/bb3927eb5f4c changeset: 103216:bb3927eb5f4c user: Christian Heimes date: Wed Sep 07 11:39:21 2016 +0200 summary: blake2: silence two more warnings on platforms with size_t < uint64_t. Don't use SSE2 when cross-compiling files: Modules/_blake2/impl/blake2b-ref.c | 4 ++-- Modules/_blake2/impl/blake2s-ref.c | 4 ++-- setup.py | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Modules/_blake2/impl/blake2b-ref.c b/Modules/_blake2/impl/blake2b-ref.c --- a/Modules/_blake2/impl/blake2b-ref.c +++ b/Modules/_blake2/impl/blake2b-ref.c @@ -307,8 +307,8 @@ } else /* inlen <= fill */ { - memcpy( S->buf + left, in, inlen ); - S->buflen += inlen; /* Be lazy, do not compress */ + memcpy( S->buf + left, in, (size_t)inlen ); + S->buflen += (size_t)inlen; /* Be lazy, do not compress */ in += inlen; inlen -= inlen; } diff --git a/Modules/_blake2/impl/blake2s-ref.c b/Modules/_blake2/impl/blake2s-ref.c --- a/Modules/_blake2/impl/blake2s-ref.c +++ b/Modules/_blake2/impl/blake2s-ref.c @@ -298,8 +298,8 @@ } else /* inlen <= fill */ { - memcpy( S->buf + left, in, inlen ); - S->buflen += inlen; /* Be lazy, do not compress */ + memcpy( S->buf + left, in, (size_t)inlen ); + S->buflen += (size_t)inlen; /* Be lazy, do not compress */ in += inlen; inlen -= inlen; } diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -894,7 +894,7 @@ blake2_deps.append('hashlib.h') blake2_macros = [] - if os.uname().machine == "x86_64": + if not cross_compiling and os.uname().machine == "x86_64": # Every x86_64 machine has at least SSE2. blake2_macros.append(('BLAKE2_USE_SSE', '1')) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 05:58:40 2016 From: python-checkins at python.org (christian.heimes) Date: Wed, 07 Sep 2016 09:58:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2316113=3A_Add_SHA-?= =?utf-8?q?3_and_SHAKE_support_to_hashlib_module=2E?= Message-ID: <20160907095832.779.15244.80C4B89C@psf.io> https://hg.python.org/cpython/rev/f8700ee4aef0 changeset: 103217:f8700ee4aef0 user: Christian Heimes date: Wed Sep 07 11:58:24 2016 +0200 summary: Issue #16113: Add SHA-3 and SHAKE support to hashlib module. files: Doc/library/hashlib.rst | 30 +- Lib/hashlib.py | 17 +- Lib/test/test_hashlib.py | 157 +- Misc/NEWS | 2 + Modules/_sha3/README.txt | 11 + Modules/_sha3/cleanup.py | 50 + Modules/_sha3/clinic/sha3module.c.h | 148 + Modules/_sha3/kcp/KeccakHash.c | 82 + Modules/_sha3/kcp/KeccakHash.h | 114 + Modules/_sha3/kcp/KeccakP-1600-64.macros | 2208 ++++++++++ Modules/_sha3/kcp/KeccakP-1600-SnP-opt32.h | 37 + Modules/_sha3/kcp/KeccakP-1600-SnP-opt64.h | 49 + Modules/_sha3/kcp/KeccakP-1600-SnP.h | 7 + Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c | 1160 +++++ Modules/_sha3/kcp/KeccakP-1600-opt64-config.h | 3 + Modules/_sha3/kcp/KeccakP-1600-opt64.c | 474 ++ Modules/_sha3/kcp/KeccakP-1600-unrolling.macros | 185 + Modules/_sha3/kcp/KeccakSponge.c | 92 + Modules/_sha3/kcp/KeccakSponge.h | 172 + Modules/_sha3/kcp/KeccakSponge.inc | 332 + Modules/_sha3/kcp/PlSnP-Fallback.inc | 257 + Modules/_sha3/kcp/SnP-Relaned.h | 134 + Modules/_sha3/kcp/align.h | 35 + Modules/_sha3/sha3module.c | 749 +++ PC/config.c | 2 + setup.py | 7 + 26 files changed, 6495 insertions(+), 19 deletions(-) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -69,7 +69,13 @@ :func:`md5` is normally available as well, though it may be missing if you are using a rare "FIPS compliant" build of Python. Additional algorithms may also be available depending upon the OpenSSL -library that Python uses on your platform. +library that Python uses on your platform. On most platforms the +:func:`sha3_224`, :func:`sha3_256`, :func:`sha3_384`, :func:`sha3_512`, +:func:`shake_128`, :func:`shake_256` are also available. + +.. versionadded:: 3.6 + SHA3 (Keccak) and SHAKE constructors :func:`sha3_224`, :func:`sha3_256`, + :func:`sha3_384`, :func:`sha3_512`, :func:`shake_128`, :func:`shake_256`. .. versionadded:: 3.6 :func:`blake2b` and :func:`blake2s` were added. @@ -189,6 +195,28 @@ compute the digests of data sharing a common initial substring. +SHAKE variable length digests +----------------------------- + +The :func:`shake_128` and :func:`shake_256` algorithms provide variable +length digests with length_in_bits//2 up to 128 or 256 bits of security. +As such, their digest methods require a length. Maximum length is not limited +by the SHAKE algorithm. + +.. method:: shake.digest(length) + + Return the digest of the data passed to the :meth:`update` method so far. + This is a bytes object of size ``length`` which may contain bytes in + the whole range from 0 to 255. + + +.. method:: shake.hexdigest(length) + + Like :meth:`digest` except the digest is returned as a string object of + double length, containing only hexadecimal digits. This may be used to + exchange the value safely in email or other non-binary environments. + + Key derivation -------------- diff --git a/Lib/hashlib.py b/Lib/hashlib.py --- a/Lib/hashlib.py +++ b/Lib/hashlib.py @@ -11,7 +11,8 @@ Named constructor functions are also available, these are faster than using new(name): -md5(), sha1(), sha224(), sha256(), sha384(), sha512(), blake2b(), and blake2s() +md5(), sha1(), sha224(), sha256(), sha384(), sha512(), blake2b(), blake2s(), +sha3_224, sha3_256, sha3_384, sha3_512, shake_128, and shake_256. More algorithms may be available on your platform but the above are guaranteed to exist. See the algorithms_guaranteed and algorithms_available attributes @@ -55,7 +56,10 @@ # This tuple and __get_builtin_constructor() must be modified if a new # always available algorithm is added. __always_supported = ('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512', - 'blake2b', 'blake2s') + 'blake2b', 'blake2s', + 'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512', + 'shake_128', 'shake_256') + algorithms_guaranteed = set(__always_supported) algorithms_available = set(__always_supported) @@ -90,6 +94,15 @@ import _blake2 cache['blake2b'] = _blake2.blake2b cache['blake2s'] = _blake2.blake2s + elif name in {'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512', + 'shake_128', 'shake_256'}: + import _sha3 + cache['sha3_224'] = _sha3.sha3_224 + cache['sha3_256'] = _sha3.sha3_256 + cache['sha3_384'] = _sha3.sha3_384 + cache['sha3_512'] = _sha3.sha3_512 + cache['shake_128'] = _sha3.shake_128 + cache['shake_256'] = _sha3.shake_256 except ImportError: pass # no extension module, this hash is unsupported. diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py --- a/Lib/test/test_hashlib.py +++ b/Lib/test/test_hashlib.py @@ -34,6 +34,13 @@ requires_blake2 = unittest.skipUnless(_blake2, 'requires _blake2') +try: + import _sha3 +except ImportError: + _sha3 = None + +requires_sha3 = unittest.skipUnless(_sha3, 'requires _sha3') + def hexstr(s): assert isinstance(s, bytes), repr(s) @@ -61,7 +68,11 @@ supported_hash_names = ( 'md5', 'MD5', 'sha1', 'SHA1', 'sha224', 'SHA224', 'sha256', 'SHA256', 'sha384', 'SHA384', 'sha512', 'SHA512', - 'blake2b', 'blake2s') + 'blake2b', 'blake2s', + 'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512', + 'shake_128', 'shake_256') + + shakes = {'shake_128', 'shake_256'} # Issue #14693: fallback modules are always compiled under POSIX _warn_on_extension_import = os.name == 'posix' or COMPILED_WITH_PYDEBUG @@ -131,6 +142,15 @@ add_builtin_constructor('blake2s') add_builtin_constructor('blake2b') + _sha3 = self._conditional_import_module('_sha3') + if _sha3: + add_builtin_constructor('sha3_224') + add_builtin_constructor('sha3_256') + add_builtin_constructor('sha3_384') + add_builtin_constructor('sha3_512') + add_builtin_constructor('shake_128') + add_builtin_constructor('shake_256') + super(HashLibTestCase, self).__init__(*args, **kwargs) @property @@ -142,7 +162,10 @@ a = array.array("b", range(10)) for cons in self.hash_constructors: c = cons(a) - c.hexdigest() + if c.name in self.shakes: + c.hexdigest(16) + else: + c.hexdigest() def test_algorithms_guaranteed(self): self.assertEqual(hashlib.algorithms_guaranteed, @@ -186,14 +209,21 @@ def test_hexdigest(self): for cons in self.hash_constructors: h = cons() - self.assertIsInstance(h.digest(), bytes) - self.assertEqual(hexstr(h.digest()), h.hexdigest()) + if h.name in self.shakes: + self.assertIsInstance(h.digest(16), bytes) + self.assertEqual(hexstr(h.digest(16)), h.hexdigest(16)) + else: + self.assertIsInstance(h.digest(), bytes) + self.assertEqual(hexstr(h.digest()), h.hexdigest()) def test_name_attribute(self): for cons in self.hash_constructors: h = cons() self.assertIsInstance(h.name, str) - self.assertIn(h.name, self.supported_hash_names) + if h.name in self.supported_hash_names: + self.assertIn(h.name, self.supported_hash_names) + else: + self.assertNotIn(h.name, self.supported_hash_names) self.assertEqual(h.name, hashlib.new(h.name).name) def test_large_update(self): @@ -208,40 +238,46 @@ m1.update(bees) m1.update(cees) m1.update(dees) + if m1.name in self.shakes: + args = (16,) + else: + args = () m2 = cons() m2.update(aas + bees + cees + dees) - self.assertEqual(m1.digest(), m2.digest()) + self.assertEqual(m1.digest(*args), m2.digest(*args)) m3 = cons(aas + bees + cees + dees) - self.assertEqual(m1.digest(), m3.digest()) + self.assertEqual(m1.digest(*args), m3.digest(*args)) # verify copy() doesn't touch original m4 = cons(aas + bees + cees) - m4_digest = m4.digest() + m4_digest = m4.digest(*args) m4_copy = m4.copy() m4_copy.update(dees) - self.assertEqual(m1.digest(), m4_copy.digest()) - self.assertEqual(m4.digest(), m4_digest) + self.assertEqual(m1.digest(*args), m4_copy.digest(*args)) + self.assertEqual(m4.digest(*args), m4_digest) - def check(self, name, data, hexdigest, **kwargs): + def check(self, name, data, hexdigest, shake=False, **kwargs): + length = len(hexdigest)//2 hexdigest = hexdigest.lower() constructors = self.constructors_to_test[name] # 2 is for hashlib.name(...) and hashlib.new(name, ...) self.assertGreaterEqual(len(constructors), 2) for hash_object_constructor in constructors: m = hash_object_constructor(data, **kwargs) - computed = m.hexdigest() + computed = m.hexdigest() if not shake else m.hexdigest(length) self.assertEqual( computed, hexdigest, "Hash algorithm %s constructed using %s returned hexdigest" " %r for %d byte input data that should have hashed to %r." % (name, hash_object_constructor, computed, len(data), hexdigest)) - computed = m.digest() + computed = m.digest() if not shake else m.digest(length) digest = bytes.fromhex(hexdigest) self.assertEqual(computed, digest) - self.assertEqual(len(digest), m.digest_size) + if not shake: + self.assertEqual(len(digest), m.digest_size) def check_no_unicode(self, algorithm_name): # Unicode objects are not allowed as input. @@ -262,13 +298,30 @@ self.check_no_unicode('blake2b') self.check_no_unicode('blake2s') - def check_blocksize_name(self, name, block_size=0, digest_size=0): + @requires_sha3 + def test_no_unicode_sha3(self): + self.check_no_unicode('sha3_224') + self.check_no_unicode('sha3_256') + self.check_no_unicode('sha3_384') + self.check_no_unicode('sha3_512') + self.check_no_unicode('shake_128') + self.check_no_unicode('shake_256') + + def check_blocksize_name(self, name, block_size=0, digest_size=0, + digest_length=None): constructors = self.constructors_to_test[name] for hash_object_constructor in constructors: m = hash_object_constructor() self.assertEqual(m.block_size, block_size) self.assertEqual(m.digest_size, digest_size) - self.assertEqual(len(m.digest()), digest_size) + if digest_length: + self.assertEqual(len(m.digest(digest_length)), + digest_length) + self.assertEqual(len(m.hexdigest(digest_length)), + 2*digest_length) + else: + self.assertEqual(len(m.digest()), digest_size) + self.assertEqual(len(m.hexdigest()), 2*digest_size) self.assertEqual(m.name, name) # split for sha3_512 / _sha3.sha3 object self.assertIn(name.split("_")[0], repr(m)) @@ -280,6 +333,12 @@ self.check_blocksize_name('sha256', 64, 32) self.check_blocksize_name('sha384', 128, 48) self.check_blocksize_name('sha512', 128, 64) + self.check_blocksize_name('sha3_224', 144, 28) + self.check_blocksize_name('sha3_256', 136, 32) + self.check_blocksize_name('sha3_384', 104, 48) + self.check_blocksize_name('sha3_512', 72, 64) + self.check_blocksize_name('shake_128', 168, 0, 32) + self.check_blocksize_name('shake_256', 136, 0, 64) @requires_blake2 def test_blocksize_name_blake2(self): @@ -563,6 +622,72 @@ key = bytes.fromhex(key) self.check('blake2s', msg, md, key=key) + @requires_sha3 + def test_case_sha3_224_0(self): + self.check('sha3_224', b"", + "6b4e03423667dbb73b6e15454f0eb1abd4597f9a1b078e3f5b5a6bc7") + + @requires_sha3 + def test_case_sha3_224_vector(self): + for msg, md in read_vectors('sha3_224'): + self.check('sha3_224', msg, md) + + @requires_sha3 + def test_case_sha3_256_0(self): + self.check('sha3_256', b"", + "a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a") + + @requires_sha3 + def test_case_sha3_256_vector(self): + for msg, md in read_vectors('sha3_256'): + self.check('sha3_256', msg, md) + + @requires_sha3 + def test_case_sha3_384_0(self): + self.check('sha3_384', b"", + "0c63a75b845e4f7d01107d852e4c2485c51a50aaaa94fc61995e71bbee983a2a"+ + "c3713831264adb47fb6bd1e058d5f004") + + @requires_sha3 + def test_case_sha3_384_vector(self): + for msg, md in read_vectors('sha3_384'): + self.check('sha3_384', msg, md) + + @requires_sha3 + def test_case_sha3_512_0(self): + self.check('sha3_512', b"", + "a69f73cca23a9ac5c8b567dc185a756e97c982164fe25859e0d1dcc1475c80a6"+ + "15b2123af1f5f94c11e3e9402c3ac558f500199d95b6d3e301758586281dcd26") + + @requires_sha3 + def test_case_sha3_512_vector(self): + for msg, md in read_vectors('sha3_512'): + self.check('sha3_512', msg, md) + + @requires_sha3 + def test_case_shake_128_0(self): + self.check('shake_128', b"", + "7f9c2ba4e88f827d616045507605853ed73b8093f6efbc88eb1a6eacfa66ef26", + True) + self.check('shake_128', b"", "7f9c", True) + + @requires_sha3 + def test_case_shake128_vector(self): + for msg, md in read_vectors('shake_128'): + self.check('shake_128', msg, md, True) + + @requires_sha3 + def test_case_shake_256_0(self): + self.check('shake_256', b"", + "46b9dd2b0ba88d13233b3feb743eeb243fcd52ea62b81b82b50c27646ed5762f", + True) + self.check('shake_256', b"", "46b9", True) + + @requires_sha3 + def test_case_shake256_vector(self): + for msg, md in read_vectors('shake_256'): + self.check('shake_256', msg, md, True) + def test_gil(self): # Check things work fine with an input larger than the size required # for multithreaded operation (which is hardwired to 2048). diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -91,6 +91,8 @@ Library ------- +- Issue #16113: Add SHA-3 and SHAKE support to hashlib module. + - Issue #27776: The :func:`os.urandom` function does now block on Linux 3.17 and newer until the system urandom entropy pool is initialized to increase the security. This change is part of the :pep:`524`. diff --git a/Modules/_sha3/README.txt b/Modules/_sha3/README.txt new file mode 100644 --- /dev/null +++ b/Modules/_sha3/README.txt @@ -0,0 +1,11 @@ +Keccak Code Package +=================== + +The files in kcp are taken from the Keccak Code Package. They have been +slightly to be C89 compatible. The architecture specific header file +KeccakP-1600-SnP.h ha been renamed to KeccakP-1600-SnP-opt32.h or +KeccakP-1600-SnP-opt64.h. + +The 64bit files were generated with generic64lc/libkeccak.a.pack target, the +32bit files with generic32lc/libkeccak.a.pack. + diff --git a/Modules/_sha3/cleanup.py b/Modules/_sha3/cleanup.py new file mode 100755 --- /dev/null +++ b/Modules/_sha3/cleanup.py @@ -0,0 +1,50 @@ +#!/usr/bin/env python +# Copyright (C) 2012 Christian Heimes (christian at python.org) +# Licensed to PSF under a Contributor Agreement. +# +# cleanup Keccak sources + +import os +import re + +CPP1 = re.compile("^//(.*)") +CPP2 = re.compile("\ //(.*)") + +STATICS = ("void ", "int ", "HashReturn ", + "const UINT64 ", "UINT16 ", " int prefix##") + +HERE = os.path.dirname(os.path.abspath(__file__)) +KECCAK = os.path.join(HERE, "kcp") + +def getfiles(): + for name in os.listdir(KECCAK): + name = os.path.join(KECCAK, name) + if os.path.isfile(name): + yield name + +def cleanup(f): + buf = [] + for line in f: + # mark all functions and global data as static + #if line.startswith(STATICS): + # buf.append("static " + line) + # continue + # remove UINT64 typedef, we have our own + if line.startswith("typedef unsigned long long int"): + buf.append("/* %s */\n" % line.strip()) + continue + ## remove #include "brg_endian.h" + if "brg_endian.h" in line: + buf.append("/* %s */\n" % line.strip()) + continue + # transform C++ comments into ANSI C comments + line = CPP1.sub(r"/*\1 */\n", line) + line = CPP2.sub(r" /*\1 */\n", line) + buf.append(line) + return "".join(buf) + +for name in getfiles(): + with open(name) as f: + res = cleanup(f) + with open(name, "w") as f: + f.write(res) diff --git a/Modules/_sha3/clinic/sha3module.c.h b/Modules/_sha3/clinic/sha3module.c.h new file mode 100644 --- /dev/null +++ b/Modules/_sha3/clinic/sha3module.c.h @@ -0,0 +1,148 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +PyDoc_STRVAR(py_sha3_new__doc__, +"sha3_224(string=None)\n" +"--\n" +"\n" +"Return a new SHA3 hash object with a hashbit length of 28 bytes."); + +static PyObject * +py_sha3_new_impl(PyTypeObject *type, PyObject *data); + +static PyObject * +py_sha3_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static char *_keywords[] = {"string", NULL}; + PyObject *data = NULL; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:sha3_224", _keywords, + &data)) + goto exit; + return_value = py_sha3_new_impl(type, data); + +exit: + return return_value; +} + +PyDoc_STRVAR(_sha3_sha3_224_copy__doc__, +"copy($self, /)\n" +"--\n" +"\n" +"Return a copy of the hash object."); + +#define _SHA3_SHA3_224_COPY_METHODDEF \ + {"copy", (PyCFunction)_sha3_sha3_224_copy, METH_NOARGS, _sha3_sha3_224_copy__doc__}, + +static PyObject * +_sha3_sha3_224_copy_impl(SHA3object *self); + +static PyObject * +_sha3_sha3_224_copy(SHA3object *self, PyObject *Py_UNUSED(ignored)) +{ + return _sha3_sha3_224_copy_impl(self); +} + +PyDoc_STRVAR(_sha3_sha3_224_digest__doc__, +"digest($self, /)\n" +"--\n" +"\n" +"Return the digest value as a string of binary data."); + +#define _SHA3_SHA3_224_DIGEST_METHODDEF \ + {"digest", (PyCFunction)_sha3_sha3_224_digest, METH_NOARGS, _sha3_sha3_224_digest__doc__}, + +static PyObject * +_sha3_sha3_224_digest_impl(SHA3object *self); + +static PyObject * +_sha3_sha3_224_digest(SHA3object *self, PyObject *Py_UNUSED(ignored)) +{ + return _sha3_sha3_224_digest_impl(self); +} + +PyDoc_STRVAR(_sha3_sha3_224_hexdigest__doc__, +"hexdigest($self, /)\n" +"--\n" +"\n" +"Return the digest value as a string of hexadecimal digits."); + +#define _SHA3_SHA3_224_HEXDIGEST_METHODDEF \ + {"hexdigest", (PyCFunction)_sha3_sha3_224_hexdigest, METH_NOARGS, _sha3_sha3_224_hexdigest__doc__}, + +static PyObject * +_sha3_sha3_224_hexdigest_impl(SHA3object *self); + +static PyObject * +_sha3_sha3_224_hexdigest(SHA3object *self, PyObject *Py_UNUSED(ignored)) +{ + return _sha3_sha3_224_hexdigest_impl(self); +} + +PyDoc_STRVAR(_sha3_sha3_224_update__doc__, +"update($self, obj, /)\n" +"--\n" +"\n" +"Update this hash object\'s state with the provided string."); + +#define _SHA3_SHA3_224_UPDATE_METHODDEF \ + {"update", (PyCFunction)_sha3_sha3_224_update, METH_O, _sha3_sha3_224_update__doc__}, + +PyDoc_STRVAR(_sha3_shake_128_digest__doc__, +"digest($self, /, length)\n" +"--\n" +"\n" +"Return the digest value as a string of binary data."); + +#define _SHA3_SHAKE_128_DIGEST_METHODDEF \ + {"digest", (PyCFunction)_sha3_shake_128_digest, METH_VARARGS|METH_KEYWORDS, _sha3_shake_128_digest__doc__}, + +static PyObject * +_sha3_shake_128_digest_impl(SHA3object *self, unsigned long length); + +static PyObject * +_sha3_shake_128_digest(SHA3object *self, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static char *_keywords[] = {"length", NULL}; + unsigned long length; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "k:digest", _keywords, + &length)) + goto exit; + return_value = _sha3_shake_128_digest_impl(self, length); + +exit: + return return_value; +} + +PyDoc_STRVAR(_sha3_shake_128_hexdigest__doc__, +"hexdigest($self, /, length)\n" +"--\n" +"\n" +"Return the digest value as a string of hexadecimal digits."); + +#define _SHA3_SHAKE_128_HEXDIGEST_METHODDEF \ + {"hexdigest", (PyCFunction)_sha3_shake_128_hexdigest, METH_VARARGS|METH_KEYWORDS, _sha3_shake_128_hexdigest__doc__}, + +static PyObject * +_sha3_shake_128_hexdigest_impl(SHA3object *self, unsigned long length); + +static PyObject * +_sha3_shake_128_hexdigest(SHA3object *self, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static char *_keywords[] = {"length", NULL}; + unsigned long length; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "k:hexdigest", _keywords, + &length)) + goto exit; + return_value = _sha3_shake_128_hexdigest_impl(self, length); + +exit: + return return_value; +} +/*[clinic end generated code: output=2eb6db41778eeb50 input=a9049054013a1b77]*/ diff --git a/Modules/_sha3/kcp/KeccakHash.c b/Modules/_sha3/kcp/KeccakHash.c new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakHash.c @@ -0,0 +1,82 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#include +#include "KeccakHash.h" + +/* ---------------------------------------------------------------- */ + +HashReturn Keccak_HashInitialize(Keccak_HashInstance *instance, unsigned int rate, unsigned int capacity, unsigned int hashbitlen, unsigned char delimitedSuffix) +{ + HashReturn result; + + if (delimitedSuffix == 0) + return FAIL; + result = (HashReturn)KeccakWidth1600_SpongeInitialize(&instance->sponge, rate, capacity); + if (result != SUCCESS) + return result; + instance->fixedOutputLength = hashbitlen; + instance->delimitedSuffix = delimitedSuffix; + return SUCCESS; +} + +/* ---------------------------------------------------------------- */ + +HashReturn Keccak_HashUpdate(Keccak_HashInstance *instance, const BitSequence *data, DataLength databitlen) +{ + if ((databitlen % 8) == 0) + return (HashReturn)KeccakWidth1600_SpongeAbsorb(&instance->sponge, data, databitlen/8); + else { + HashReturn ret = (HashReturn)KeccakWidth1600_SpongeAbsorb(&instance->sponge, data, databitlen/8); + if (ret == SUCCESS) { + /* The last partial byte is assumed to be aligned on the least significant bits */ + + unsigned char lastByte = data[databitlen/8]; + /* Concatenate the last few bits provided here with those of the suffix */ + + unsigned short delimitedLastBytes = (unsigned short)((unsigned short)lastByte | ((unsigned short)instance->delimitedSuffix << (databitlen % 8))); + if ((delimitedLastBytes & 0xFF00) == 0x0000) { + instance->delimitedSuffix = delimitedLastBytes & 0xFF; + } + else { + unsigned char oneByte[1]; + oneByte[0] = delimitedLastBytes & 0xFF; + ret = (HashReturn)KeccakWidth1600_SpongeAbsorb(&instance->sponge, oneByte, 1); + instance->delimitedSuffix = (delimitedLastBytes >> 8) & 0xFF; + } + } + return ret; + } +} + +/* ---------------------------------------------------------------- */ + +HashReturn Keccak_HashFinal(Keccak_HashInstance *instance, BitSequence *hashval) +{ + HashReturn ret = (HashReturn)KeccakWidth1600_SpongeAbsorbLastFewBits(&instance->sponge, instance->delimitedSuffix); + if (ret == SUCCESS) + return (HashReturn)KeccakWidth1600_SpongeSqueeze(&instance->sponge, hashval, instance->fixedOutputLength/8); + else + return ret; +} + +/* ---------------------------------------------------------------- */ + +HashReturn Keccak_HashSqueeze(Keccak_HashInstance *instance, BitSequence *data, DataLength databitlen) +{ + if ((databitlen % 8) != 0) + return FAIL; + return (HashReturn)KeccakWidth1600_SpongeSqueeze(&instance->sponge, data, databitlen/8); +} diff --git a/Modules/_sha3/kcp/KeccakHash.h b/Modules/_sha3/kcp/KeccakHash.h new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakHash.h @@ -0,0 +1,114 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#ifndef _KeccakHashInterface_h_ +#define _KeccakHashInterface_h_ + +#ifndef KeccakP1600_excluded + +#include "KeccakSponge.h" +#include + +typedef unsigned char BitSequence; +typedef size_t DataLength; +typedef enum { SUCCESS = 0, FAIL = 1, BAD_HASHLEN = 2 } HashReturn; + +typedef struct { + KeccakWidth1600_SpongeInstance sponge; + unsigned int fixedOutputLength; + unsigned char delimitedSuffix; +} Keccak_HashInstance; + +/** + * Function to initialize the Keccak[r, c] sponge function instance used in sequential hashing mode. + * @param hashInstance Pointer to the hash instance to be initialized. + * @param rate The value of the rate r. + * @param capacity The value of the capacity c. + * @param hashbitlen The desired number of output bits, + * or 0 for an arbitrarily-long output. + * @param delimitedSuffix Bits that will be automatically appended to the end + * of the input message, as in domain separation. + * This is a byte containing from 0 to 7 bits + * formatted like the @a delimitedData parameter of + * the Keccak_SpongeAbsorbLastFewBits() function. + * @pre One must have r+c=1600 and the rate a multiple of 8 bits in this implementation. + * @return SUCCESS if successful, FAIL otherwise. + */ +HashReturn Keccak_HashInitialize(Keccak_HashInstance *hashInstance, unsigned int rate, unsigned int capacity, unsigned int hashbitlen, unsigned char delimitedSuffix); + +/** Macro to initialize a SHAKE128 instance as specified in the FIPS 202 standard. + */ +#define Keccak_HashInitialize_SHAKE128(hashInstance) Keccak_HashInitialize(hashInstance, 1344, 256, 0, 0x1F) + +/** Macro to initialize a SHAKE256 instance as specified in the FIPS 202 standard. + */ +#define Keccak_HashInitialize_SHAKE256(hashInstance) Keccak_HashInitialize(hashInstance, 1088, 512, 0, 0x1F) + +/** Macro to initialize a SHA3-224 instance as specified in the FIPS 202 standard. + */ +#define Keccak_HashInitialize_SHA3_224(hashInstance) Keccak_HashInitialize(hashInstance, 1152, 448, 224, 0x06) + +/** Macro to initialize a SHA3-256 instance as specified in the FIPS 202 standard. + */ +#define Keccak_HashInitialize_SHA3_256(hashInstance) Keccak_HashInitialize(hashInstance, 1088, 512, 256, 0x06) + +/** Macro to initialize a SHA3-384 instance as specified in the FIPS 202 standard. + */ +#define Keccak_HashInitialize_SHA3_384(hashInstance) Keccak_HashInitialize(hashInstance, 832, 768, 384, 0x06) + +/** Macro to initialize a SHA3-512 instance as specified in the FIPS 202 standard. + */ +#define Keccak_HashInitialize_SHA3_512(hashInstance) Keccak_HashInitialize(hashInstance, 576, 1024, 512, 0x06) + +/** + * Function to give input data to be absorbed. + * @param hashInstance Pointer to the hash instance initialized by Keccak_HashInitialize(). + * @param data Pointer to the input data. + * When @a databitLen is not a multiple of 8, the last bits of data must be + * in the least significant bits of the last byte (little-endian convention). + * @param databitLen The number of input bits provided in the input data. + * @pre In the previous call to Keccak_HashUpdate(), databitlen was a multiple of 8. + * @return SUCCESS if successful, FAIL otherwise. + */ +HashReturn Keccak_HashUpdate(Keccak_HashInstance *hashInstance, const BitSequence *data, DataLength databitlen); + +/** + * Function to call after all input blocks have been input and to get + * output bits if the length was specified when calling Keccak_HashInitialize(). + * @param hashInstance Pointer to the hash instance initialized by Keccak_HashInitialize(). + * If @a hashbitlen was not 0 in the call to Keccak_HashInitialize(), the number of + * output bits is equal to @a hashbitlen. + * If @a hashbitlen was 0 in the call to Keccak_HashInitialize(), the output bits + * must be extracted using the Keccak_HashSqueeze() function. + * @param state Pointer to the state of the sponge function initialized by Init(). + * @param hashval Pointer to the buffer where to store the output data. + * @return SUCCESS if successful, FAIL otherwise. + */ +HashReturn Keccak_HashFinal(Keccak_HashInstance *hashInstance, BitSequence *hashval); + + /** + * Function to squeeze output data. + * @param hashInstance Pointer to the hash instance initialized by Keccak_HashInitialize(). + * @param data Pointer to the buffer where to store the output data. + * @param databitlen The number of output bits desired (must be a multiple of 8). + * @pre Keccak_HashFinal() must have been already called. + * @pre @a databitlen is a multiple of 8. + * @return SUCCESS if successful, FAIL otherwise. + */ +HashReturn Keccak_HashSqueeze(Keccak_HashInstance *hashInstance, BitSequence *data, DataLength databitlen); + +#endif + +#endif diff --git a/Modules/_sha3/kcp/KeccakP-1600-64.macros b/Modules/_sha3/kcp/KeccakP-1600-64.macros new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakP-1600-64.macros @@ -0,0 +1,2208 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#define declareABCDE \ + UINT64 Aba, Abe, Abi, Abo, Abu; \ + UINT64 Aga, Age, Agi, Ago, Agu; \ + UINT64 Aka, Ake, Aki, Ako, Aku; \ + UINT64 Ama, Ame, Ami, Amo, Amu; \ + UINT64 Asa, Ase, Asi, Aso, Asu; \ + UINT64 Bba, Bbe, Bbi, Bbo, Bbu; \ + UINT64 Bga, Bge, Bgi, Bgo, Bgu; \ + UINT64 Bka, Bke, Bki, Bko, Bku; \ + UINT64 Bma, Bme, Bmi, Bmo, Bmu; \ + UINT64 Bsa, Bse, Bsi, Bso, Bsu; \ + UINT64 Ca, Ce, Ci, Co, Cu; \ + UINT64 Da, De, Di, Do, Du; \ + UINT64 Eba, Ebe, Ebi, Ebo, Ebu; \ + UINT64 Ega, Ege, Egi, Ego, Egu; \ + UINT64 Eka, Eke, Eki, Eko, Eku; \ + UINT64 Ema, Eme, Emi, Emo, Emu; \ + UINT64 Esa, Ese, Esi, Eso, Esu; \ + +#define prepareTheta \ + Ca = Aba^Aga^Aka^Ama^Asa; \ + Ce = Abe^Age^Ake^Ame^Ase; \ + Ci = Abi^Agi^Aki^Ami^Asi; \ + Co = Abo^Ago^Ako^Amo^Aso; \ + Cu = Abu^Agu^Aku^Amu^Asu; \ + +#ifdef UseBebigokimisa +/* --- Code for round, with prepare-theta (lane complementing pattern 'bebigokimisa') */ + +/* --- 64-bit lanes mapped to 64-bit words */ + +#define thetaRhoPiChiIotaPrepareTheta(i, A, E) \ + Da = Cu^ROL64(Ce, 1); \ + De = Ca^ROL64(Ci, 1); \ + Di = Ce^ROL64(Co, 1); \ + Do = Ci^ROL64(Cu, 1); \ + Du = Co^ROL64(Ca, 1); \ +\ + A##ba ^= Da; \ + Bba = A##ba; \ + A##ge ^= De; \ + Bbe = ROL64(A##ge, 44); \ + A##ki ^= Di; \ + Bbi = ROL64(A##ki, 43); \ + A##mo ^= Do; \ + Bbo = ROL64(A##mo, 21); \ + A##su ^= Du; \ + Bbu = ROL64(A##su, 14); \ + E##ba = Bba ^( Bbe | Bbi ); \ + E##ba ^= KeccakF1600RoundConstants[i]; \ + Ca = E##ba; \ + E##be = Bbe ^((~Bbi)| Bbo ); \ + Ce = E##be; \ + E##bi = Bbi ^( Bbo & Bbu ); \ + Ci = E##bi; \ + E##bo = Bbo ^( Bbu | Bba ); \ + Co = E##bo; \ + E##bu = Bbu ^( Bba & Bbe ); \ + Cu = E##bu; \ +\ + A##bo ^= Do; \ + Bga = ROL64(A##bo, 28); \ + A##gu ^= Du; \ + Bge = ROL64(A##gu, 20); \ + A##ka ^= Da; \ + Bgi = ROL64(A##ka, 3); \ + A##me ^= De; \ + Bgo = ROL64(A##me, 45); \ + A##si ^= Di; \ + Bgu = ROL64(A##si, 61); \ + E##ga = Bga ^( Bge | Bgi ); \ + Ca ^= E##ga; \ + E##ge = Bge ^( Bgi & Bgo ); \ + Ce ^= E##ge; \ + E##gi = Bgi ^( Bgo |(~Bgu)); \ + Ci ^= E##gi; \ + E##go = Bgo ^( Bgu | Bga ); \ + Co ^= E##go; \ + E##gu = Bgu ^( Bga & Bge ); \ + Cu ^= E##gu; \ +\ + A##be ^= De; \ + Bka = ROL64(A##be, 1); \ + A##gi ^= Di; \ + Bke = ROL64(A##gi, 6); \ + A##ko ^= Do; \ + Bki = ROL64(A##ko, 25); \ + A##mu ^= Du; \ + Bko = ROL64(A##mu, 8); \ + A##sa ^= Da; \ + Bku = ROL64(A##sa, 18); \ + E##ka = Bka ^( Bke | Bki ); \ + Ca ^= E##ka; \ + E##ke = Bke ^( Bki & Bko ); \ + Ce ^= E##ke; \ + E##ki = Bki ^((~Bko)& Bku ); \ + Ci ^= E##ki; \ + E##ko = (~Bko)^( Bku | Bka ); \ + Co ^= E##ko; \ + E##ku = Bku ^( Bka & Bke ); \ + Cu ^= E##ku; \ +\ + A##bu ^= Du; \ + Bma = ROL64(A##bu, 27); \ + A##ga ^= Da; \ + Bme = ROL64(A##ga, 36); \ + A##ke ^= De; \ + Bmi = ROL64(A##ke, 10); \ + A##mi ^= Di; \ + Bmo = ROL64(A##mi, 15); \ + A##so ^= Do; \ + Bmu = ROL64(A##so, 56); \ + E##ma = Bma ^( Bme & Bmi ); \ + Ca ^= E##ma; \ + E##me = Bme ^( Bmi | Bmo ); \ + Ce ^= E##me; \ + E##mi = Bmi ^((~Bmo)| Bmu ); \ + Ci ^= E##mi; \ + E##mo = (~Bmo)^( Bmu & Bma ); \ + Co ^= E##mo; \ + E##mu = Bmu ^( Bma | Bme ); \ + Cu ^= E##mu; \ +\ + A##bi ^= Di; \ + Bsa = ROL64(A##bi, 62); \ + A##go ^= Do; \ + Bse = ROL64(A##go, 55); \ + A##ku ^= Du; \ + Bsi = ROL64(A##ku, 39); \ + A##ma ^= Da; \ + Bso = ROL64(A##ma, 41); \ + A##se ^= De; \ + Bsu = ROL64(A##se, 2); \ + E##sa = Bsa ^((~Bse)& Bsi ); \ + Ca ^= E##sa; \ + E##se = (~Bse)^( Bsi | Bso ); \ + Ce ^= E##se; \ + E##si = Bsi ^( Bso & Bsu ); \ + Ci ^= E##si; \ + E##so = Bso ^( Bsu | Bsa ); \ + Co ^= E##so; \ + E##su = Bsu ^( Bsa & Bse ); \ + Cu ^= E##su; \ +\ + +/* --- Code for round (lane complementing pattern 'bebigokimisa') */ + +/* --- 64-bit lanes mapped to 64-bit words */ + +#define thetaRhoPiChiIota(i, A, E) \ + Da = Cu^ROL64(Ce, 1); \ + De = Ca^ROL64(Ci, 1); \ + Di = Ce^ROL64(Co, 1); \ + Do = Ci^ROL64(Cu, 1); \ + Du = Co^ROL64(Ca, 1); \ +\ + A##ba ^= Da; \ + Bba = A##ba; \ + A##ge ^= De; \ + Bbe = ROL64(A##ge, 44); \ + A##ki ^= Di; \ + Bbi = ROL64(A##ki, 43); \ + A##mo ^= Do; \ + Bbo = ROL64(A##mo, 21); \ + A##su ^= Du; \ + Bbu = ROL64(A##su, 14); \ + E##ba = Bba ^( Bbe | Bbi ); \ + E##ba ^= KeccakF1600RoundConstants[i]; \ + E##be = Bbe ^((~Bbi)| Bbo ); \ + E##bi = Bbi ^( Bbo & Bbu ); \ + E##bo = Bbo ^( Bbu | Bba ); \ + E##bu = Bbu ^( Bba & Bbe ); \ +\ + A##bo ^= Do; \ + Bga = ROL64(A##bo, 28); \ + A##gu ^= Du; \ + Bge = ROL64(A##gu, 20); \ + A##ka ^= Da; \ + Bgi = ROL64(A##ka, 3); \ + A##me ^= De; \ + Bgo = ROL64(A##me, 45); \ + A##si ^= Di; \ + Bgu = ROL64(A##si, 61); \ + E##ga = Bga ^( Bge | Bgi ); \ + E##ge = Bge ^( Bgi & Bgo ); \ + E##gi = Bgi ^( Bgo |(~Bgu)); \ + E##go = Bgo ^( Bgu | Bga ); \ + E##gu = Bgu ^( Bga & Bge ); \ +\ + A##be ^= De; \ + Bka = ROL64(A##be, 1); \ + A##gi ^= Di; \ + Bke = ROL64(A##gi, 6); \ + A##ko ^= Do; \ + Bki = ROL64(A##ko, 25); \ + A##mu ^= Du; \ + Bko = ROL64(A##mu, 8); \ + A##sa ^= Da; \ + Bku = ROL64(A##sa, 18); \ + E##ka = Bka ^( Bke | Bki ); \ + E##ke = Bke ^( Bki & Bko ); \ + E##ki = Bki ^((~Bko)& Bku ); \ + E##ko = (~Bko)^( Bku | Bka ); \ + E##ku = Bku ^( Bka & Bke ); \ +\ + A##bu ^= Du; \ + Bma = ROL64(A##bu, 27); \ + A##ga ^= Da; \ + Bme = ROL64(A##ga, 36); \ + A##ke ^= De; \ + Bmi = ROL64(A##ke, 10); \ + A##mi ^= Di; \ + Bmo = ROL64(A##mi, 15); \ + A##so ^= Do; \ + Bmu = ROL64(A##so, 56); \ + E##ma = Bma ^( Bme & Bmi ); \ + E##me = Bme ^( Bmi | Bmo ); \ + E##mi = Bmi ^((~Bmo)| Bmu ); \ + E##mo = (~Bmo)^( Bmu & Bma ); \ + E##mu = Bmu ^( Bma | Bme ); \ +\ + A##bi ^= Di; \ + Bsa = ROL64(A##bi, 62); \ + A##go ^= Do; \ + Bse = ROL64(A##go, 55); \ + A##ku ^= Du; \ + Bsi = ROL64(A##ku, 39); \ + A##ma ^= Da; \ + Bso = ROL64(A##ma, 41); \ + A##se ^= De; \ + Bsu = ROL64(A##se, 2); \ + E##sa = Bsa ^((~Bse)& Bsi ); \ + E##se = (~Bse)^( Bsi | Bso ); \ + E##si = Bsi ^( Bso & Bsu ); \ + E##so = Bso ^( Bsu | Bsa ); \ + E##su = Bsu ^( Bsa & Bse ); \ +\ + +#else /* UseBebigokimisa */ + +/* --- Code for round, with prepare-theta */ + +/* --- 64-bit lanes mapped to 64-bit words */ + +#define thetaRhoPiChiIotaPrepareTheta(i, A, E) \ + Da = Cu^ROL64(Ce, 1); \ + De = Ca^ROL64(Ci, 1); \ + Di = Ce^ROL64(Co, 1); \ + Do = Ci^ROL64(Cu, 1); \ + Du = Co^ROL64(Ca, 1); \ +\ + A##ba ^= Da; \ + Bba = A##ba; \ + A##ge ^= De; \ + Bbe = ROL64(A##ge, 44); \ + A##ki ^= Di; \ + Bbi = ROL64(A##ki, 43); \ + A##mo ^= Do; \ + Bbo = ROL64(A##mo, 21); \ + A##su ^= Du; \ + Bbu = ROL64(A##su, 14); \ + E##ba = Bba ^((~Bbe)& Bbi ); \ + E##ba ^= KeccakF1600RoundConstants[i]; \ + Ca = E##ba; \ + E##be = Bbe ^((~Bbi)& Bbo ); \ + Ce = E##be; \ + E##bi = Bbi ^((~Bbo)& Bbu ); \ + Ci = E##bi; \ + E##bo = Bbo ^((~Bbu)& Bba ); \ + Co = E##bo; \ + E##bu = Bbu ^((~Bba)& Bbe ); \ + Cu = E##bu; \ +\ + A##bo ^= Do; \ + Bga = ROL64(A##bo, 28); \ + A##gu ^= Du; \ + Bge = ROL64(A##gu, 20); \ + A##ka ^= Da; \ + Bgi = ROL64(A##ka, 3); \ + A##me ^= De; \ + Bgo = ROL64(A##me, 45); \ + A##si ^= Di; \ + Bgu = ROL64(A##si, 61); \ + E##ga = Bga ^((~Bge)& Bgi ); \ + Ca ^= E##ga; \ + E##ge = Bge ^((~Bgi)& Bgo ); \ + Ce ^= E##ge; \ + E##gi = Bgi ^((~Bgo)& Bgu ); \ + Ci ^= E##gi; \ + E##go = Bgo ^((~Bgu)& Bga ); \ + Co ^= E##go; \ + E##gu = Bgu ^((~Bga)& Bge ); \ + Cu ^= E##gu; \ +\ + A##be ^= De; \ + Bka = ROL64(A##be, 1); \ + A##gi ^= Di; \ + Bke = ROL64(A##gi, 6); \ + A##ko ^= Do; \ + Bki = ROL64(A##ko, 25); \ + A##mu ^= Du; \ + Bko = ROL64(A##mu, 8); \ + A##sa ^= Da; \ + Bku = ROL64(A##sa, 18); \ + E##ka = Bka ^((~Bke)& Bki ); \ + Ca ^= E##ka; \ + E##ke = Bke ^((~Bki)& Bko ); \ + Ce ^= E##ke; \ + E##ki = Bki ^((~Bko)& Bku ); \ + Ci ^= E##ki; \ + E##ko = Bko ^((~Bku)& Bka ); \ + Co ^= E##ko; \ + E##ku = Bku ^((~Bka)& Bke ); \ + Cu ^= E##ku; \ +\ + A##bu ^= Du; \ + Bma = ROL64(A##bu, 27); \ + A##ga ^= Da; \ + Bme = ROL64(A##ga, 36); \ + A##ke ^= De; \ + Bmi = ROL64(A##ke, 10); \ + A##mi ^= Di; \ + Bmo = ROL64(A##mi, 15); \ + A##so ^= Do; \ + Bmu = ROL64(A##so, 56); \ + E##ma = Bma ^((~Bme)& Bmi ); \ + Ca ^= E##ma; \ + E##me = Bme ^((~Bmi)& Bmo ); \ + Ce ^= E##me; \ + E##mi = Bmi ^((~Bmo)& Bmu ); \ + Ci ^= E##mi; \ + E##mo = Bmo ^((~Bmu)& Bma ); \ + Co ^= E##mo; \ + E##mu = Bmu ^((~Bma)& Bme ); \ + Cu ^= E##mu; \ +\ + A##bi ^= Di; \ + Bsa = ROL64(A##bi, 62); \ + A##go ^= Do; \ + Bse = ROL64(A##go, 55); \ + A##ku ^= Du; \ + Bsi = ROL64(A##ku, 39); \ + A##ma ^= Da; \ + Bso = ROL64(A##ma, 41); \ + A##se ^= De; \ + Bsu = ROL64(A##se, 2); \ + E##sa = Bsa ^((~Bse)& Bsi ); \ + Ca ^= E##sa; \ + E##se = Bse ^((~Bsi)& Bso ); \ + Ce ^= E##se; \ + E##si = Bsi ^((~Bso)& Bsu ); \ + Ci ^= E##si; \ + E##so = Bso ^((~Bsu)& Bsa ); \ + Co ^= E##so; \ + E##su = Bsu ^((~Bsa)& Bse ); \ + Cu ^= E##su; \ +\ + +/* --- Code for round */ + +/* --- 64-bit lanes mapped to 64-bit words */ + +#define thetaRhoPiChiIota(i, A, E) \ + Da = Cu^ROL64(Ce, 1); \ + De = Ca^ROL64(Ci, 1); \ + Di = Ce^ROL64(Co, 1); \ + Do = Ci^ROL64(Cu, 1); \ + Du = Co^ROL64(Ca, 1); \ +\ + A##ba ^= Da; \ + Bba = A##ba; \ + A##ge ^= De; \ + Bbe = ROL64(A##ge, 44); \ + A##ki ^= Di; \ + Bbi = ROL64(A##ki, 43); \ + A##mo ^= Do; \ + Bbo = ROL64(A##mo, 21); \ + A##su ^= Du; \ + Bbu = ROL64(A##su, 14); \ + E##ba = Bba ^((~Bbe)& Bbi ); \ + E##ba ^= KeccakF1600RoundConstants[i]; \ + E##be = Bbe ^((~Bbi)& Bbo ); \ + E##bi = Bbi ^((~Bbo)& Bbu ); \ + E##bo = Bbo ^((~Bbu)& Bba ); \ + E##bu = Bbu ^((~Bba)& Bbe ); \ +\ + A##bo ^= Do; \ + Bga = ROL64(A##bo, 28); \ + A##gu ^= Du; \ + Bge = ROL64(A##gu, 20); \ + A##ka ^= Da; \ + Bgi = ROL64(A##ka, 3); \ + A##me ^= De; \ + Bgo = ROL64(A##me, 45); \ + A##si ^= Di; \ + Bgu = ROL64(A##si, 61); \ + E##ga = Bga ^((~Bge)& Bgi ); \ + E##ge = Bge ^((~Bgi)& Bgo ); \ + E##gi = Bgi ^((~Bgo)& Bgu ); \ + E##go = Bgo ^((~Bgu)& Bga ); \ + E##gu = Bgu ^((~Bga)& Bge ); \ +\ + A##be ^= De; \ + Bka = ROL64(A##be, 1); \ + A##gi ^= Di; \ + Bke = ROL64(A##gi, 6); \ + A##ko ^= Do; \ + Bki = ROL64(A##ko, 25); \ + A##mu ^= Du; \ + Bko = ROL64(A##mu, 8); \ + A##sa ^= Da; \ + Bku = ROL64(A##sa, 18); \ + E##ka = Bka ^((~Bke)& Bki ); \ + E##ke = Bke ^((~Bki)& Bko ); \ + E##ki = Bki ^((~Bko)& Bku ); \ + E##ko = Bko ^((~Bku)& Bka ); \ + E##ku = Bku ^((~Bka)& Bke ); \ +\ + A##bu ^= Du; \ + Bma = ROL64(A##bu, 27); \ + A##ga ^= Da; \ + Bme = ROL64(A##ga, 36); \ + A##ke ^= De; \ + Bmi = ROL64(A##ke, 10); \ + A##mi ^= Di; \ + Bmo = ROL64(A##mi, 15); \ + A##so ^= Do; \ + Bmu = ROL64(A##so, 56); \ + E##ma = Bma ^((~Bme)& Bmi ); \ + E##me = Bme ^((~Bmi)& Bmo ); \ + E##mi = Bmi ^((~Bmo)& Bmu ); \ + E##mo = Bmo ^((~Bmu)& Bma ); \ + E##mu = Bmu ^((~Bma)& Bme ); \ +\ + A##bi ^= Di; \ + Bsa = ROL64(A##bi, 62); \ + A##go ^= Do; \ + Bse = ROL64(A##go, 55); \ + A##ku ^= Du; \ + Bsi = ROL64(A##ku, 39); \ + A##ma ^= Da; \ + Bso = ROL64(A##ma, 41); \ + A##se ^= De; \ + Bsu = ROL64(A##se, 2); \ + E##sa = Bsa ^((~Bse)& Bsi ); \ + E##se = Bse ^((~Bsi)& Bso ); \ + E##si = Bsi ^((~Bso)& Bsu ); \ + E##so = Bso ^((~Bsu)& Bsa ); \ + E##su = Bsu ^((~Bsa)& Bse ); \ +\ + +#endif /* UseBebigokimisa */ + + +#define copyFromState(X, state) \ + X##ba = state[ 0]; \ + X##be = state[ 1]; \ + X##bi = state[ 2]; \ + X##bo = state[ 3]; \ + X##bu = state[ 4]; \ + X##ga = state[ 5]; \ + X##ge = state[ 6]; \ + X##gi = state[ 7]; \ + X##go = state[ 8]; \ + X##gu = state[ 9]; \ + X##ka = state[10]; \ + X##ke = state[11]; \ + X##ki = state[12]; \ + X##ko = state[13]; \ + X##ku = state[14]; \ + X##ma = state[15]; \ + X##me = state[16]; \ + X##mi = state[17]; \ + X##mo = state[18]; \ + X##mu = state[19]; \ + X##sa = state[20]; \ + X##se = state[21]; \ + X##si = state[22]; \ + X##so = state[23]; \ + X##su = state[24]; \ + +#define copyToState(state, X) \ + state[ 0] = X##ba; \ + state[ 1] = X##be; \ + state[ 2] = X##bi; \ + state[ 3] = X##bo; \ + state[ 4] = X##bu; \ + state[ 5] = X##ga; \ + state[ 6] = X##ge; \ + state[ 7] = X##gi; \ + state[ 8] = X##go; \ + state[ 9] = X##gu; \ + state[10] = X##ka; \ + state[11] = X##ke; \ + state[12] = X##ki; \ + state[13] = X##ko; \ + state[14] = X##ku; \ + state[15] = X##ma; \ + state[16] = X##me; \ + state[17] = X##mi; \ + state[18] = X##mo; \ + state[19] = X##mu; \ + state[20] = X##sa; \ + state[21] = X##se; \ + state[22] = X##si; \ + state[23] = X##so; \ + state[24] = X##su; \ + +#define copyStateVariables(X, Y) \ + X##ba = Y##ba; \ + X##be = Y##be; \ + X##bi = Y##bi; \ + X##bo = Y##bo; \ + X##bu = Y##bu; \ + X##ga = Y##ga; \ + X##ge = Y##ge; \ + X##gi = Y##gi; \ + X##go = Y##go; \ + X##gu = Y##gu; \ + X##ka = Y##ka; \ + X##ke = Y##ke; \ + X##ki = Y##ki; \ + X##ko = Y##ko; \ + X##ku = Y##ku; \ + X##ma = Y##ma; \ + X##me = Y##me; \ + X##mi = Y##mi; \ + X##mo = Y##mo; \ + X##mu = Y##mu; \ + X##sa = Y##sa; \ + X##se = Y##se; \ + X##si = Y##si; \ + X##so = Y##so; \ + X##su = Y##su; \ + +#define copyFromStateAndAdd(X, state, input, laneCount) \ + if (laneCount < 16) { \ + if (laneCount < 8) { \ + if (laneCount < 4) { \ + if (laneCount < 2) { \ + if (laneCount < 1) { \ + X##ba = state[ 0]; \ + } \ + else { \ + X##ba = state[ 0]^input[ 0]; \ + } \ + X##be = state[ 1]; \ + X##bi = state[ 2]; \ + } \ + else { \ + X##ba = state[ 0]^input[ 0]; \ + X##be = state[ 1]^input[ 1]; \ + if (laneCount < 3) { \ + X##bi = state[ 2]; \ + } \ + else { \ + X##bi = state[ 2]^input[ 2]; \ + } \ + } \ + X##bo = state[ 3]; \ + X##bu = state[ 4]; \ + X##ga = state[ 5]; \ + X##ge = state[ 6]; \ + } \ + else { \ + X##ba = state[ 0]^input[ 0]; \ + X##be = state[ 1]^input[ 1]; \ + X##bi = state[ 2]^input[ 2]; \ + X##bo = state[ 3]^input[ 3]; \ + if (laneCount < 6) { \ + if (laneCount < 5) { \ + X##bu = state[ 4]; \ + } \ + else { \ + X##bu = state[ 4]^input[ 4]; \ + } \ + X##ga = state[ 5]; \ + X##ge = state[ 6]; \ + } \ + else { \ + X##bu = state[ 4]^input[ 4]; \ + X##ga = state[ 5]^input[ 5]; \ + if (laneCount < 7) { \ + X##ge = state[ 6]; \ + } \ + else { \ + X##ge = state[ 6]^input[ 6]; \ + } \ + } \ + } \ + X##gi = state[ 7]; \ + X##go = state[ 8]; \ + X##gu = state[ 9]; \ + X##ka = state[10]; \ + X##ke = state[11]; \ + X##ki = state[12]; \ + X##ko = state[13]; \ + X##ku = state[14]; \ + } \ + else { \ + X##ba = state[ 0]^input[ 0]; \ + X##be = state[ 1]^input[ 1]; \ + X##bi = state[ 2]^input[ 2]; \ + X##bo = state[ 3]^input[ 3]; \ + X##bu = state[ 4]^input[ 4]; \ + X##ga = state[ 5]^input[ 5]; \ + X##ge = state[ 6]^input[ 6]; \ + X##gi = state[ 7]^input[ 7]; \ + if (laneCount < 12) { \ + if (laneCount < 10) { \ + if (laneCount < 9) { \ + X##go = state[ 8]; \ + } \ + else { \ + X##go = state[ 8]^input[ 8]; \ + } \ + X##gu = state[ 9]; \ + X##ka = state[10]; \ + } \ + else { \ + X##go = state[ 8]^input[ 8]; \ + X##gu = state[ 9]^input[ 9]; \ + if (laneCount < 11) { \ + X##ka = state[10]; \ + } \ + else { \ + X##ka = state[10]^input[10]; \ + } \ + } \ + X##ke = state[11]; \ + X##ki = state[12]; \ + X##ko = state[13]; \ + X##ku = state[14]; \ + } \ + else { \ + X##go = state[ 8]^input[ 8]; \ + X##gu = state[ 9]^input[ 9]; \ + X##ka = state[10]^input[10]; \ + X##ke = state[11]^input[11]; \ + if (laneCount < 14) { \ + if (laneCount < 13) { \ + X##ki = state[12]; \ + } \ + else { \ + X##ki = state[12]^input[12]; \ + } \ + X##ko = state[13]; \ + X##ku = state[14]; \ + } \ + else { \ + X##ki = state[12]^input[12]; \ + X##ko = state[13]^input[13]; \ + if (laneCount < 15) { \ + X##ku = state[14]; \ + } \ + else { \ + X##ku = state[14]^input[14]; \ + } \ + } \ + } \ + } \ + X##ma = state[15]; \ + X##me = state[16]; \ + X##mi = state[17]; \ + X##mo = state[18]; \ + X##mu = state[19]; \ + X##sa = state[20]; \ + X##se = state[21]; \ + X##si = state[22]; \ + X##so = state[23]; \ + X##su = state[24]; \ + } \ + else { \ + X##ba = state[ 0]^input[ 0]; \ + X##be = state[ 1]^input[ 1]; \ + X##bi = state[ 2]^input[ 2]; \ + X##bo = state[ 3]^input[ 3]; \ + X##bu = state[ 4]^input[ 4]; \ + X##ga = state[ 5]^input[ 5]; \ + X##ge = state[ 6]^input[ 6]; \ + X##gi = state[ 7]^input[ 7]; \ + X##go = state[ 8]^input[ 8]; \ + X##gu = state[ 9]^input[ 9]; \ + X##ka = state[10]^input[10]; \ + X##ke = state[11]^input[11]; \ + X##ki = state[12]^input[12]; \ + X##ko = state[13]^input[13]; \ + X##ku = state[14]^input[14]; \ + X##ma = state[15]^input[15]; \ + if (laneCount < 24) { \ + if (laneCount < 20) { \ + if (laneCount < 18) { \ + if (laneCount < 17) { \ + X##me = state[16]; \ + } \ + else { \ + X##me = state[16]^input[16]; \ + } \ + X##mi = state[17]; \ + X##mo = state[18]; \ + } \ + else { \ + X##me = state[16]^input[16]; \ + X##mi = state[17]^input[17]; \ + if (laneCount < 19) { \ + X##mo = state[18]; \ + } \ + else { \ + X##mo = state[18]^input[18]; \ + } \ + } \ + X##mu = state[19]; \ + X##sa = state[20]; \ + X##se = state[21]; \ + X##si = state[22]; \ + } \ + else { \ + X##me = state[16]^input[16]; \ + X##mi = state[17]^input[17]; \ + X##mo = state[18]^input[18]; \ + X##mu = state[19]^input[19]; \ + if (laneCount < 22) { \ + if (laneCount < 21) { \ + X##sa = state[20]; \ + } \ + else { \ + X##sa = state[20]^input[20]; \ + } \ + X##se = state[21]; \ + X##si = state[22]; \ + } \ + else { \ + X##sa = state[20]^input[20]; \ + X##se = state[21]^input[21]; \ + if (laneCount < 23) { \ + X##si = state[22]; \ + } \ + else { \ + X##si = state[22]^input[22]; \ + } \ + } \ + } \ + X##so = state[23]; \ + X##su = state[24]; \ + } \ + else { \ + X##me = state[16]^input[16]; \ + X##mi = state[17]^input[17]; \ + X##mo = state[18]^input[18]; \ + X##mu = state[19]^input[19]; \ + X##sa = state[20]^input[20]; \ + X##se = state[21]^input[21]; \ + X##si = state[22]^input[22]; \ + X##so = state[23]^input[23]; \ + if (laneCount < 25) { \ + X##su = state[24]; \ + } \ + else { \ + X##su = state[24]^input[24]; \ + } \ + } \ + } + +#define addInput(X, input, laneCount) \ + if (laneCount == 21) { \ + X##ba ^= input[ 0]; \ + X##be ^= input[ 1]; \ + X##bi ^= input[ 2]; \ + X##bo ^= input[ 3]; \ + X##bu ^= input[ 4]; \ + X##ga ^= input[ 5]; \ + X##ge ^= input[ 6]; \ + X##gi ^= input[ 7]; \ + X##go ^= input[ 8]; \ + X##gu ^= input[ 9]; \ + X##ka ^= input[10]; \ + X##ke ^= input[11]; \ + X##ki ^= input[12]; \ + X##ko ^= input[13]; \ + X##ku ^= input[14]; \ + X##ma ^= input[15]; \ + X##me ^= input[16]; \ + X##mi ^= input[17]; \ + X##mo ^= input[18]; \ + X##mu ^= input[19]; \ + X##sa ^= input[20]; \ + } \ + else if (laneCount < 16) { \ + if (laneCount < 8) { \ + if (laneCount < 4) { \ + if (laneCount < 2) { \ + if (laneCount < 1) { \ + } \ + else { \ + X##ba ^= input[ 0]; \ + } \ + } \ + else { \ + X##ba ^= input[ 0]; \ + X##be ^= input[ 1]; \ + if (laneCount < 3) { \ + } \ + else { \ + X##bi ^= input[ 2]; \ + } \ + } \ + } \ + else { \ + X##ba ^= input[ 0]; \ + X##be ^= input[ 1]; \ + X##bi ^= input[ 2]; \ + X##bo ^= input[ 3]; \ + if (laneCount < 6) { \ + if (laneCount < 5) { \ + } \ + else { \ + X##bu ^= input[ 4]; \ + } \ + } \ + else { \ + X##bu ^= input[ 4]; \ + X##ga ^= input[ 5]; \ + if (laneCount < 7) { \ + } \ + else { \ + X##ge ^= input[ 6]; \ + } \ + } \ + } \ + } \ + else { \ + X##ba ^= input[ 0]; \ + X##be ^= input[ 1]; \ + X##bi ^= input[ 2]; \ + X##bo ^= input[ 3]; \ + X##bu ^= input[ 4]; \ + X##ga ^= input[ 5]; \ + X##ge ^= input[ 6]; \ + X##gi ^= input[ 7]; \ + if (laneCount < 12) { \ + if (laneCount < 10) { \ + if (laneCount < 9) { \ + } \ + else { \ + X##go ^= input[ 8]; \ + } \ + } \ + else { \ + X##go ^= input[ 8]; \ + X##gu ^= input[ 9]; \ + if (laneCount < 11) { \ + } \ + else { \ + X##ka ^= input[10]; \ + } \ + } \ + } \ + else { \ + X##go ^= input[ 8]; \ + X##gu ^= input[ 9]; \ + X##ka ^= input[10]; \ + X##ke ^= input[11]; \ + if (laneCount < 14) { \ + if (laneCount < 13) { \ + } \ + else { \ + X##ki ^= input[12]; \ + } \ + } \ + else { \ + X##ki ^= input[12]; \ + X##ko ^= input[13]; \ + if (laneCount < 15) { \ + } \ + else { \ + X##ku ^= input[14]; \ + } \ + } \ + } \ + } \ + } \ + else { \ + X##ba ^= input[ 0]; \ + X##be ^= input[ 1]; \ + X##bi ^= input[ 2]; \ + X##bo ^= input[ 3]; \ + X##bu ^= input[ 4]; \ + X##ga ^= input[ 5]; \ + X##ge ^= input[ 6]; \ + X##gi ^= input[ 7]; \ + X##go ^= input[ 8]; \ + X##gu ^= input[ 9]; \ + X##ka ^= input[10]; \ + X##ke ^= input[11]; \ + X##ki ^= input[12]; \ + X##ko ^= input[13]; \ + X##ku ^= input[14]; \ + X##ma ^= input[15]; \ + if (laneCount < 24) { \ + if (laneCount < 20) { \ + if (laneCount < 18) { \ + if (laneCount < 17) { \ + } \ + else { \ + X##me ^= input[16]; \ + } \ + } \ + else { \ + X##me ^= input[16]; \ + X##mi ^= input[17]; \ + if (laneCount < 19) { \ + } \ + else { \ + X##mo ^= input[18]; \ + } \ + } \ + } \ + else { \ + X##me ^= input[16]; \ + X##mi ^= input[17]; \ + X##mo ^= input[18]; \ + X##mu ^= input[19]; \ + if (laneCount < 22) { \ + if (laneCount < 21) { \ + } \ + else { \ + X##sa ^= input[20]; \ + } \ + } \ + else { \ + X##sa ^= input[20]; \ + X##se ^= input[21]; \ + if (laneCount < 23) { \ + } \ + else { \ + X##si ^= input[22]; \ + } \ + } \ + } \ + } \ + else { \ + X##me ^= input[16]; \ + X##mi ^= input[17]; \ + X##mo ^= input[18]; \ + X##mu ^= input[19]; \ + X##sa ^= input[20]; \ + X##se ^= input[21]; \ + X##si ^= input[22]; \ + X##so ^= input[23]; \ + if (laneCount < 25) { \ + } \ + else { \ + X##su ^= input[24]; \ + } \ + } \ + } + +#ifdef UseBebigokimisa + +#define copyToStateAndOutput(X, state, output, laneCount) \ + if (laneCount < 16) { \ + if (laneCount < 8) { \ + if (laneCount < 4) { \ + if (laneCount < 2) { \ + state[ 0] = X##ba; \ + if (laneCount >= 1) { \ + output[ 0] = X##ba; \ + } \ + state[ 1] = X##be; \ + state[ 2] = X##bi; \ + } \ + else { \ + state[ 0] = X##ba; \ + output[ 0] = X##ba; \ + state[ 1] = X##be; \ + output[ 1] = ~X##be; \ + state[ 2] = X##bi; \ + if (laneCount >= 3) { \ + output[ 2] = ~X##bi; \ + } \ + } \ + state[ 3] = X##bo; \ + state[ 4] = X##bu; \ + state[ 5] = X##ga; \ + state[ 6] = X##ge; \ + } \ + else { \ + state[ 0] = X##ba; \ + output[ 0] = X##ba; \ + state[ 1] = X##be; \ + output[ 1] = ~X##be; \ + state[ 2] = X##bi; \ + output[ 2] = ~X##bi; \ + state[ 3] = X##bo; \ + output[ 3] = X##bo; \ + if (laneCount < 6) { \ + state[ 4] = X##bu; \ + if (laneCount >= 5) { \ + output[ 4] = X##bu; \ + } \ + state[ 5] = X##ga; \ + state[ 6] = X##ge; \ + } \ + else { \ + state[ 4] = X##bu; \ + output[ 4] = X##bu; \ + state[ 5] = X##ga; \ + output[ 5] = X##ga; \ + state[ 6] = X##ge; \ + if (laneCount >= 7) { \ + output[ 6] = X##ge; \ + } \ + } \ + } \ + state[ 7] = X##gi; \ + state[ 8] = X##go; \ + state[ 9] = X##gu; \ + state[10] = X##ka; \ + state[11] = X##ke; \ + state[12] = X##ki; \ + state[13] = X##ko; \ + state[14] = X##ku; \ + } \ + else { \ + state[ 0] = X##ba; \ + output[ 0] = X##ba; \ + state[ 1] = X##be; \ + output[ 1] = ~X##be; \ + state[ 2] = X##bi; \ + output[ 2] = ~X##bi; \ + state[ 3] = X##bo; \ + output[ 3] = X##bo; \ + state[ 4] = X##bu; \ + output[ 4] = X##bu; \ + state[ 5] = X##ga; \ + output[ 5] = X##ga; \ + state[ 6] = X##ge; \ + output[ 6] = X##ge; \ + state[ 7] = X##gi; \ + output[ 7] = X##gi; \ + if (laneCount < 12) { \ + if (laneCount < 10) { \ + state[ 8] = X##go; \ + if (laneCount >= 9) { \ + output[ 8] = ~X##go; \ + } \ + state[ 9] = X##gu; \ + state[10] = X##ka; \ + } \ + else { \ + state[ 8] = X##go; \ + output[ 8] = ~X##go; \ + state[ 9] = X##gu; \ + output[ 9] = X##gu; \ + state[10] = X##ka; \ + if (laneCount >= 11) { \ + output[10] = X##ka; \ + } \ + } \ + state[11] = X##ke; \ + state[12] = X##ki; \ + state[13] = X##ko; \ + state[14] = X##ku; \ + } \ + else { \ + state[ 8] = X##go; \ + output[ 8] = ~X##go; \ + state[ 9] = X##gu; \ + output[ 9] = X##gu; \ + state[10] = X##ka; \ + output[10] = X##ka; \ + state[11] = X##ke; \ + output[11] = X##ke; \ + if (laneCount < 14) { \ + state[12] = X##ki; \ + if (laneCount >= 13) { \ + output[12] = ~X##ki; \ + } \ + state[13] = X##ko; \ + state[14] = X##ku; \ + } \ + else { \ + state[12] = X##ki; \ + output[12] = ~X##ki; \ + state[13] = X##ko; \ + output[13] = X##ko; \ + state[14] = X##ku; \ + if (laneCount >= 15) { \ + output[14] = X##ku; \ + } \ + } \ + } \ + } \ + state[15] = X##ma; \ + state[16] = X##me; \ + state[17] = X##mi; \ + state[18] = X##mo; \ + state[19] = X##mu; \ + state[20] = X##sa; \ + state[21] = X##se; \ + state[22] = X##si; \ + state[23] = X##so; \ + state[24] = X##su; \ + } \ + else { \ + state[ 0] = X##ba; \ + output[ 0] = X##ba; \ + state[ 1] = X##be; \ + output[ 1] = ~X##be; \ + state[ 2] = X##bi; \ + output[ 2] = ~X##bi; \ + state[ 3] = X##bo; \ + output[ 3] = X##bo; \ + state[ 4] = X##bu; \ + output[ 4] = X##bu; \ + state[ 5] = X##ga; \ + output[ 5] = X##ga; \ + state[ 6] = X##ge; \ + output[ 6] = X##ge; \ + state[ 7] = X##gi; \ + output[ 7] = X##gi; \ + state[ 8] = X##go; \ + output[ 8] = ~X##go; \ + state[ 9] = X##gu; \ + output[ 9] = X##gu; \ + state[10] = X##ka; \ + output[10] = X##ka; \ + state[11] = X##ke; \ + output[11] = X##ke; \ + state[12] = X##ki; \ + output[12] = ~X##ki; \ + state[13] = X##ko; \ + output[13] = X##ko; \ + state[14] = X##ku; \ + output[14] = X##ku; \ + state[15] = X##ma; \ + output[15] = X##ma; \ + if (laneCount < 24) { \ + if (laneCount < 20) { \ + if (laneCount < 18) { \ + state[16] = X##me; \ + if (laneCount >= 17) { \ + output[16] = X##me; \ + } \ + state[17] = X##mi; \ + state[18] = X##mo; \ + } \ + else { \ + state[16] = X##me; \ + output[16] = X##me; \ + state[17] = X##mi; \ + output[17] = ~X##mi; \ + state[18] = X##mo; \ + if (laneCount >= 19) { \ + output[18] = X##mo; \ + } \ + } \ + state[19] = X##mu; \ + state[20] = X##sa; \ + state[21] = X##se; \ + state[22] = X##si; \ + } \ + else { \ + state[16] = X##me; \ + output[16] = X##me; \ + state[17] = X##mi; \ + output[17] = ~X##mi; \ + state[18] = X##mo; \ + output[18] = X##mo; \ + state[19] = X##mu; \ + output[19] = X##mu; \ + if (laneCount < 22) { \ + state[20] = X##sa; \ + if (laneCount >= 21) { \ + output[20] = ~X##sa; \ + } \ + state[21] = X##se; \ + state[22] = X##si; \ + } \ + else { \ + state[20] = X##sa; \ + output[20] = ~X##sa; \ + state[21] = X##se; \ + output[21] = X##se; \ + state[22] = X##si; \ + if (laneCount >= 23) { \ + output[22] = X##si; \ + } \ + } \ + } \ + state[23] = X##so; \ + state[24] = X##su; \ + } \ + else { \ + state[16] = X##me; \ + output[16] = X##me; \ + state[17] = X##mi; \ + output[17] = ~X##mi; \ + state[18] = X##mo; \ + output[18] = X##mo; \ + state[19] = X##mu; \ + output[19] = X##mu; \ + state[20] = X##sa; \ + output[20] = ~X##sa; \ + state[21] = X##se; \ + output[21] = X##se; \ + state[22] = X##si; \ + output[22] = X##si; \ + state[23] = X##so; \ + output[23] = X##so; \ + state[24] = X##su; \ + if (laneCount >= 25) { \ + output[24] = X##su; \ + } \ + } \ + } + +#define output(X, output, laneCount) \ + if (laneCount < 16) { \ + if (laneCount < 8) { \ + if (laneCount < 4) { \ + if (laneCount < 2) { \ + if (laneCount >= 1) { \ + output[ 0] = X##ba; \ + } \ + } \ + else { \ + output[ 0] = X##ba; \ + output[ 1] = ~X##be; \ + if (laneCount >= 3) { \ + output[ 2] = ~X##bi; \ + } \ + } \ + } \ + else { \ + output[ 0] = X##ba; \ + output[ 1] = ~X##be; \ + output[ 2] = ~X##bi; \ + output[ 3] = X##bo; \ + if (laneCount < 6) { \ + if (laneCount >= 5) { \ + output[ 4] = X##bu; \ + } \ + } \ + else { \ + output[ 4] = X##bu; \ + output[ 5] = X##ga; \ + if (laneCount >= 7) { \ + output[ 6] = X##ge; \ + } \ + } \ + } \ + } \ + else { \ + output[ 0] = X##ba; \ + output[ 1] = ~X##be; \ + output[ 2] = ~X##bi; \ + output[ 3] = X##bo; \ + output[ 4] = X##bu; \ + output[ 5] = X##ga; \ + output[ 6] = X##ge; \ + output[ 7] = X##gi; \ + if (laneCount < 12) { \ + if (laneCount < 10) { \ + if (laneCount >= 9) { \ + output[ 8] = ~X##go; \ + } \ + } \ + else { \ + output[ 8] = ~X##go; \ + output[ 9] = X##gu; \ + if (laneCount >= 11) { \ + output[10] = X##ka; \ + } \ + } \ + } \ + else { \ + output[ 8] = ~X##go; \ + output[ 9] = X##gu; \ + output[10] = X##ka; \ + output[11] = X##ke; \ + if (laneCount < 14) { \ + if (laneCount >= 13) { \ + output[12] = ~X##ki; \ + } \ + } \ + else { \ + output[12] = ~X##ki; \ + output[13] = X##ko; \ + if (laneCount >= 15) { \ + output[14] = X##ku; \ + } \ + } \ + } \ + } \ + } \ + else { \ + output[ 0] = X##ba; \ + output[ 1] = ~X##be; \ + output[ 2] = ~X##bi; \ + output[ 3] = X##bo; \ + output[ 4] = X##bu; \ + output[ 5] = X##ga; \ + output[ 6] = X##ge; \ + output[ 7] = X##gi; \ + output[ 8] = ~X##go; \ + output[ 9] = X##gu; \ + output[10] = X##ka; \ + output[11] = X##ke; \ + output[12] = ~X##ki; \ + output[13] = X##ko; \ + output[14] = X##ku; \ + output[15] = X##ma; \ + if (laneCount < 24) { \ + if (laneCount < 20) { \ + if (laneCount < 18) { \ + if (laneCount >= 17) { \ + output[16] = X##me; \ + } \ + } \ + else { \ + output[16] = X##me; \ + output[17] = ~X##mi; \ + if (laneCount >= 19) { \ + output[18] = X##mo; \ + } \ + } \ + } \ + else { \ + output[16] = X##me; \ + output[17] = ~X##mi; \ + output[18] = X##mo; \ + output[19] = X##mu; \ + if (laneCount < 22) { \ + if (laneCount >= 21) { \ + output[20] = ~X##sa; \ + } \ + } \ + else { \ + output[20] = ~X##sa; \ + output[21] = X##se; \ + if (laneCount >= 23) { \ + output[22] = X##si; \ + } \ + } \ + } \ + } \ + else { \ + output[16] = X##me; \ + output[17] = ~X##mi; \ + output[18] = X##mo; \ + output[19] = X##mu; \ + output[20] = ~X##sa; \ + output[21] = X##se; \ + output[22] = X##si; \ + output[23] = X##so; \ + if (laneCount >= 25) { \ + output[24] = X##su; \ + } \ + } \ + } + +#define wrapOne(X, input, output, index, name) \ + X##name ^= input[index]; \ + output[index] = X##name; + +#define wrapOneInvert(X, input, output, index, name) \ + X##name ^= input[index]; \ + output[index] = ~X##name; + +#define unwrapOne(X, input, output, index, name) \ + output[index] = input[index] ^ X##name; \ + X##name ^= output[index]; + +#define unwrapOneInvert(X, input, output, index, name) \ + output[index] = ~(input[index] ^ X##name); \ + X##name ^= output[index]; \ + +#else /* UseBebigokimisa */ + + +#define copyToStateAndOutput(X, state, output, laneCount) \ + if (laneCount < 16) { \ + if (laneCount < 8) { \ + if (laneCount < 4) { \ + if (laneCount < 2) { \ + state[ 0] = X##ba; \ + if (laneCount >= 1) { \ + output[ 0] = X##ba; \ + } \ + state[ 1] = X##be; \ + state[ 2] = X##bi; \ + } \ + else { \ + state[ 0] = X##ba; \ + output[ 0] = X##ba; \ + state[ 1] = X##be; \ + output[ 1] = X##be; \ + state[ 2] = X##bi; \ + if (laneCount >= 3) { \ + output[ 2] = X##bi; \ + } \ + } \ + state[ 3] = X##bo; \ + state[ 4] = X##bu; \ + state[ 5] = X##ga; \ + state[ 6] = X##ge; \ + } \ + else { \ + state[ 0] = X##ba; \ + output[ 0] = X##ba; \ + state[ 1] = X##be; \ + output[ 1] = X##be; \ + state[ 2] = X##bi; \ + output[ 2] = X##bi; \ + state[ 3] = X##bo; \ + output[ 3] = X##bo; \ + if (laneCount < 6) { \ + state[ 4] = X##bu; \ + if (laneCount >= 5) { \ + output[ 4] = X##bu; \ + } \ + state[ 5] = X##ga; \ + state[ 6] = X##ge; \ + } \ + else { \ + state[ 4] = X##bu; \ + output[ 4] = X##bu; \ + state[ 5] = X##ga; \ + output[ 5] = X##ga; \ + state[ 6] = X##ge; \ + if (laneCount >= 7) { \ + output[ 6] = X##ge; \ + } \ + } \ + } \ + state[ 7] = X##gi; \ + state[ 8] = X##go; \ + state[ 9] = X##gu; \ + state[10] = X##ka; \ + state[11] = X##ke; \ + state[12] = X##ki; \ + state[13] = X##ko; \ + state[14] = X##ku; \ + } \ + else { \ + state[ 0] = X##ba; \ + output[ 0] = X##ba; \ + state[ 1] = X##be; \ + output[ 1] = X##be; \ + state[ 2] = X##bi; \ + output[ 2] = X##bi; \ + state[ 3] = X##bo; \ + output[ 3] = X##bo; \ + state[ 4] = X##bu; \ + output[ 4] = X##bu; \ + state[ 5] = X##ga; \ + output[ 5] = X##ga; \ + state[ 6] = X##ge; \ + output[ 6] = X##ge; \ + state[ 7] = X##gi; \ + output[ 7] = X##gi; \ + if (laneCount < 12) { \ + if (laneCount < 10) { \ + state[ 8] = X##go; \ + if (laneCount >= 9) { \ + output[ 8] = X##go; \ + } \ + state[ 9] = X##gu; \ + state[10] = X##ka; \ + } \ + else { \ + state[ 8] = X##go; \ + output[ 8] = X##go; \ + state[ 9] = X##gu; \ + output[ 9] = X##gu; \ + state[10] = X##ka; \ + if (laneCount >= 11) { \ + output[10] = X##ka; \ + } \ + } \ + state[11] = X##ke; \ + state[12] = X##ki; \ + state[13] = X##ko; \ + state[14] = X##ku; \ + } \ + else { \ + state[ 8] = X##go; \ + output[ 8] = X##go; \ + state[ 9] = X##gu; \ + output[ 9] = X##gu; \ + state[10] = X##ka; \ + output[10] = X##ka; \ + state[11] = X##ke; \ + output[11] = X##ke; \ + if (laneCount < 14) { \ + state[12] = X##ki; \ + if (laneCount >= 13) { \ + output[12]= X##ki; \ + } \ + state[13] = X##ko; \ + state[14] = X##ku; \ + } \ + else { \ + state[12] = X##ki; \ + output[12]= X##ki; \ + state[13] = X##ko; \ + output[13] = X##ko; \ + state[14] = X##ku; \ + if (laneCount >= 15) { \ + output[14] = X##ku; \ + } \ + } \ + } \ + } \ + state[15] = X##ma; \ + state[16] = X##me; \ + state[17] = X##mi; \ + state[18] = X##mo; \ + state[19] = X##mu; \ + state[20] = X##sa; \ + state[21] = X##se; \ + state[22] = X##si; \ + state[23] = X##so; \ + state[24] = X##su; \ + } \ + else { \ + state[ 0] = X##ba; \ + output[ 0] = X##ba; \ + state[ 1] = X##be; \ + output[ 1] = X##be; \ + state[ 2] = X##bi; \ + output[ 2] = X##bi; \ + state[ 3] = X##bo; \ + output[ 3] = X##bo; \ + state[ 4] = X##bu; \ + output[ 4] = X##bu; \ + state[ 5] = X##ga; \ + output[ 5] = X##ga; \ + state[ 6] = X##ge; \ + output[ 6] = X##ge; \ + state[ 7] = X##gi; \ + output[ 7] = X##gi; \ + state[ 8] = X##go; \ + output[ 8] = X##go; \ + state[ 9] = X##gu; \ + output[ 9] = X##gu; \ + state[10] = X##ka; \ + output[10] = X##ka; \ + state[11] = X##ke; \ + output[11] = X##ke; \ + state[12] = X##ki; \ + output[12]= X##ki; \ + state[13] = X##ko; \ + output[13] = X##ko; \ + state[14] = X##ku; \ + output[14] = X##ku; \ + state[15] = X##ma; \ + output[15] = X##ma; \ + if (laneCount < 24) { \ + if (laneCount < 20) { \ + if (laneCount < 18) { \ + state[16] = X##me; \ + if (laneCount >= 17) { \ + output[16] = X##me; \ + } \ + state[17] = X##mi; \ + state[18] = X##mo; \ + } \ + else { \ + state[16] = X##me; \ + output[16] = X##me; \ + state[17] = X##mi; \ + output[17] = X##mi; \ + state[18] = X##mo; \ + if (laneCount >= 19) { \ + output[18] = X##mo; \ + } \ + } \ + state[19] = X##mu; \ + state[20] = X##sa; \ + state[21] = X##se; \ + state[22] = X##si; \ + } \ + else { \ + state[16] = X##me; \ + output[16] = X##me; \ + state[17] = X##mi; \ + output[17] = X##mi; \ + state[18] = X##mo; \ + output[18] = X##mo; \ + state[19] = X##mu; \ + output[19] = X##mu; \ + if (laneCount < 22) { \ + state[20] = X##sa; \ + if (laneCount >= 21) { \ + output[20] = X##sa; \ + } \ + state[21] = X##se; \ + state[22] = X##si; \ + } \ + else { \ + state[20] = X##sa; \ + output[20] = X##sa; \ + state[21] = X##se; \ + output[21] = X##se; \ + state[22] = X##si; \ + if (laneCount >= 23) { \ + output[22] = X##si; \ + } \ + } \ + } \ + state[23] = X##so; \ + state[24] = X##su; \ + } \ + else { \ + state[16] = X##me; \ + output[16] = X##me; \ + state[17] = X##mi; \ + output[17] = X##mi; \ + state[18] = X##mo; \ + output[18] = X##mo; \ + state[19] = X##mu; \ + output[19] = X##mu; \ + state[20] = X##sa; \ + output[20] = X##sa; \ + state[21] = X##se; \ + output[21] = X##se; \ + state[22] = X##si; \ + output[22] = X##si; \ + state[23] = X##so; \ + output[23] = X##so; \ + state[24] = X##su; \ + if (laneCount >= 25) { \ + output[24] = X##su; \ + } \ + } \ + } + +#define output(X, output, laneCount) \ + if (laneCount < 16) { \ + if (laneCount < 8) { \ + if (laneCount < 4) { \ + if (laneCount < 2) { \ + if (laneCount >= 1) { \ + output[ 0] = X##ba; \ + } \ + } \ + else { \ + output[ 0] = X##ba; \ + output[ 1] = X##be; \ + if (laneCount >= 3) { \ + output[ 2] = X##bi; \ + } \ + } \ + } \ + else { \ + output[ 0] = X##ba; \ + output[ 1] = X##be; \ + output[ 2] = X##bi; \ + output[ 3] = X##bo; \ + if (laneCount < 6) { \ + if (laneCount >= 5) { \ + output[ 4] = X##bu; \ + } \ + } \ + else { \ + output[ 4] = X##bu; \ + output[ 5] = X##ga; \ + if (laneCount >= 7) { \ + output[ 6] = X##ge; \ + } \ + } \ + } \ + } \ + else { \ + output[ 0] = X##ba; \ + output[ 1] = X##be; \ + output[ 2] = X##bi; \ + output[ 3] = X##bo; \ + output[ 4] = X##bu; \ + output[ 5] = X##ga; \ + output[ 6] = X##ge; \ + output[ 7] = X##gi; \ + if (laneCount < 12) { \ + if (laneCount < 10) { \ + if (laneCount >= 9) { \ + output[ 8] = X##go; \ + } \ + } \ + else { \ + output[ 8] = X##go; \ + output[ 9] = X##gu; \ + if (laneCount >= 11) { \ + output[10] = X##ka; \ + } \ + } \ + } \ + else { \ + output[ 8] = X##go; \ + output[ 9] = X##gu; \ + output[10] = X##ka; \ + output[11] = X##ke; \ + if (laneCount < 14) { \ + if (laneCount >= 13) { \ + output[12] = X##ki; \ + } \ + } \ + else { \ + output[12] = X##ki; \ + output[13] = X##ko; \ + if (laneCount >= 15) { \ + output[14] = X##ku; \ + } \ + } \ + } \ + } \ + } \ + else { \ + output[ 0] = X##ba; \ + output[ 1] = X##be; \ + output[ 2] = X##bi; \ + output[ 3] = X##bo; \ + output[ 4] = X##bu; \ + output[ 5] = X##ga; \ + output[ 6] = X##ge; \ + output[ 7] = X##gi; \ + output[ 8] = X##go; \ + output[ 9] = X##gu; \ + output[10] = X##ka; \ + output[11] = X##ke; \ + output[12] = X##ki; \ + output[13] = X##ko; \ + output[14] = X##ku; \ + output[15] = X##ma; \ + if (laneCount < 24) { \ + if (laneCount < 20) { \ + if (laneCount < 18) { \ + if (laneCount >= 17) { \ + output[16] = X##me; \ + } \ + } \ + else { \ + output[16] = X##me; \ + output[17] = X##mi; \ + if (laneCount >= 19) { \ + output[18] = X##mo; \ + } \ + } \ + } \ + else { \ + output[16] = X##me; \ + output[17] = X##mi; \ + output[18] = X##mo; \ + output[19] = X##mu; \ + if (laneCount < 22) { \ + if (laneCount >= 21) { \ + output[20] = X##sa; \ + } \ + } \ + else { \ + output[20] = X##sa; \ + output[21] = X##se; \ + if (laneCount >= 23) { \ + output[22] = X##si; \ + } \ + } \ + } \ + } \ + else { \ + output[16] = X##me; \ + output[17] = X##mi; \ + output[18] = X##mo; \ + output[19] = X##mu; \ + output[20] = X##sa; \ + output[21] = X##se; \ + output[22] = X##si; \ + output[23] = X##so; \ + if (laneCount >= 25) { \ + output[24] = X##su; \ + } \ + } \ + } + +#define wrapOne(X, input, output, index, name) \ + X##name ^= input[index]; \ + output[index] = X##name; + +#define wrapOneInvert(X, input, output, index, name) \ + X##name ^= input[index]; \ + output[index] = X##name; + +#define unwrapOne(X, input, output, index, name) \ + output[index] = input[index] ^ X##name; \ + X##name ^= output[index]; + +#define unwrapOneInvert(X, input, output, index, name) \ + output[index] = input[index] ^ X##name; \ + X##name ^= output[index]; + +#endif + +#define wrap(X, input, output, laneCount, trailingBits) \ + if (laneCount < 16) { \ + if (laneCount < 8) { \ + if (laneCount < 4) { \ + if (laneCount < 2) { \ + if (laneCount < 1) { \ + X##ba ^= trailingBits; \ + } \ + else { \ + wrapOne(X, input, output, 0, ba) \ + X##be ^= trailingBits; \ + } \ + } \ + else { \ + wrapOne(X, input, output, 0, ba) \ + wrapOneInvert(X, input, output, 1, be) \ + if (laneCount < 3) { \ + X##bi ^= trailingBits; \ + } \ + else { \ + wrapOneInvert(X, input, output, 2, bi) \ + X##bo ^= trailingBits; \ + } \ + } \ + } \ + else { \ + wrapOne(X, input, output, 0, ba) \ + wrapOneInvert(X, input, output, 1, be) \ + wrapOneInvert(X, input, output, 2, bi) \ + wrapOne(X, input, output, 3, bo) \ + if (laneCount < 6) { \ + if (laneCount < 5) { \ + X##bu ^= trailingBits; \ + } \ + else { \ + wrapOne(X, input, output, 4, bu) \ + X##ga ^= trailingBits; \ + } \ + } \ + else { \ + wrapOne(X, input, output, 4, bu) \ + wrapOne(X, input, output, 5, ga) \ + if (laneCount < 7) { \ + X##ge ^= trailingBits; \ + } \ + else { \ + wrapOne(X, input, output, 6, ge) \ + X##gi ^= trailingBits; \ + } \ + } \ + } \ + } \ + else { \ + wrapOne(X, input, output, 0, ba) \ + wrapOneInvert(X, input, output, 1, be) \ + wrapOneInvert(X, input, output, 2, bi) \ + wrapOne(X, input, output, 3, bo) \ + wrapOne(X, input, output, 4, bu) \ + wrapOne(X, input, output, 5, ga) \ + wrapOne(X, input, output, 6, ge) \ + wrapOne(X, input, output, 7, gi) \ + if (laneCount < 12) { \ + if (laneCount < 10) { \ + if (laneCount < 9) { \ + X##go ^= trailingBits; \ + } \ + else { \ + wrapOneInvert(X, input, output, 8, go) \ + X##gu ^= trailingBits; \ + } \ + } \ + else { \ + wrapOneInvert(X, input, output, 8, go) \ + wrapOne(X, input, output, 9, gu) \ + if (laneCount < 11) { \ + X##ka ^= trailingBits; \ + } \ + else { \ + wrapOne(X, input, output, 10, ka) \ + X##ke ^= trailingBits; \ + } \ + } \ + } \ + else { \ + wrapOneInvert(X, input, output, 8, go) \ + wrapOne(X, input, output, 9, gu) \ + wrapOne(X, input, output, 10, ka) \ + wrapOne(X, input, output, 11, ke) \ + if (laneCount < 14) { \ + if (laneCount < 13) { \ + X##ki ^= trailingBits; \ + } \ + else { \ + wrapOneInvert(X, input, output, 12, ki) \ + X##ko ^= trailingBits; \ + } \ + } \ + else { \ + wrapOneInvert(X, input, output, 12, ki) \ + wrapOne(X, input, output, 13, ko) \ + if (laneCount < 15) { \ + X##ku ^= trailingBits; \ + } \ + else { \ + wrapOne(X, input, output, 14, ku) \ + X##ma ^= trailingBits; \ + } \ + } \ + } \ + } \ + } \ + else { \ + wrapOne(X, input, output, 0, ba) \ + wrapOneInvert(X, input, output, 1, be) \ + wrapOneInvert(X, input, output, 2, bi) \ + wrapOne(X, input, output, 3, bo) \ + wrapOne(X, input, output, 4, bu) \ + wrapOne(X, input, output, 5, ga) \ + wrapOne(X, input, output, 6, ge) \ + wrapOne(X, input, output, 7, gi) \ + wrapOneInvert(X, input, output, 8, go) \ + wrapOne(X, input, output, 9, gu) \ + wrapOne(X, input, output, 10, ka) \ + wrapOne(X, input, output, 11, ke) \ + wrapOneInvert(X, input, output, 12, ki) \ + wrapOne(X, input, output, 13, ko) \ + wrapOne(X, input, output, 14, ku) \ + wrapOne(X, input, output, 15, ma) \ + if (laneCount < 24) { \ + if (laneCount < 20) { \ + if (laneCount < 18) { \ + if (laneCount < 17) { \ + X##me ^= trailingBits; \ + } \ + else { \ + wrapOne(X, input, output, 16, me) \ + X##mi ^= trailingBits; \ + } \ + } \ + else { \ + wrapOne(X, input, output, 16, me) \ + wrapOneInvert(X, input, output, 17, mi) \ + if (laneCount < 19) { \ + X##mo ^= trailingBits; \ + } \ + else { \ + wrapOne(X, input, output, 18, mo) \ + X##mu ^= trailingBits; \ + } \ + } \ + } \ + else { \ + wrapOne(X, input, output, 16, me) \ + wrapOneInvert(X, input, output, 17, mi) \ + wrapOne(X, input, output, 18, mo) \ + wrapOne(X, input, output, 19, mu) \ + if (laneCount < 22) { \ + if (laneCount < 21) { \ + X##sa ^= trailingBits; \ + } \ + else { \ + wrapOneInvert(X, input, output, 20, sa) \ + X##se ^= trailingBits; \ + } \ + } \ + else { \ + wrapOneInvert(X, input, output, 20, sa) \ + wrapOne(X, input, output, 21, se) \ + if (laneCount < 23) { \ + X##si ^= trailingBits; \ + } \ + else { \ + wrapOne(X, input, output, 22, si) \ + X##so ^= trailingBits; \ + } \ + } \ + } \ + } \ + else { \ + wrapOne(X, input, output, 16, me) \ + wrapOneInvert(X, input, output, 17, mi) \ + wrapOne(X, input, output, 18, mo) \ + wrapOne(X, input, output, 19, mu) \ + wrapOneInvert(X, input, output, 20, sa) \ + wrapOne(X, input, output, 21, se) \ + wrapOne(X, input, output, 22, si) \ + wrapOne(X, input, output, 23, so) \ + if (laneCount < 25) { \ + X##su ^= trailingBits; \ + } \ + else { \ + wrapOne(X, input, output, 24, su) \ + } \ + } \ + } + +#define unwrap(X, input, output, laneCount, trailingBits) \ + if (laneCount < 16) { \ + if (laneCount < 8) { \ + if (laneCount < 4) { \ + if (laneCount < 2) { \ + if (laneCount < 1) { \ + X##ba ^= trailingBits; \ + } \ + else { \ + unwrapOne(X, input, output, 0, ba) \ + X##be ^= trailingBits; \ + } \ + } \ + else { \ + unwrapOne(X, input, output, 0, ba) \ + unwrapOneInvert(X, input, output, 1, be) \ + if (laneCount < 3) { \ + X##bi ^= trailingBits; \ + } \ + else { \ + unwrapOneInvert(X, input, output, 2, bi) \ + X##bo ^= trailingBits; \ + } \ + } \ + } \ + else { \ + unwrapOne(X, input, output, 0, ba) \ + unwrapOneInvert(X, input, output, 1, be) \ + unwrapOneInvert(X, input, output, 2, bi) \ + unwrapOne(X, input, output, 3, bo) \ + if (laneCount < 6) { \ + if (laneCount < 5) { \ + X##bu ^= trailingBits; \ + } \ + else { \ + unwrapOne(X, input, output, 4, bu) \ + X##ga ^= trailingBits; \ + } \ + } \ + else { \ + unwrapOne(X, input, output, 4, bu) \ + unwrapOne(X, input, output, 5, ga) \ + if (laneCount < 7) { \ + X##ge ^= trailingBits; \ + } \ + else { \ + unwrapOne(X, input, output, 6, ge) \ + X##gi ^= trailingBits; \ + } \ + } \ + } \ + } \ + else { \ + unwrapOne(X, input, output, 0, ba) \ + unwrapOneInvert(X, input, output, 1, be) \ + unwrapOneInvert(X, input, output, 2, bi) \ + unwrapOne(X, input, output, 3, bo) \ + unwrapOne(X, input, output, 4, bu) \ + unwrapOne(X, input, output, 5, ga) \ + unwrapOne(X, input, output, 6, ge) \ + unwrapOne(X, input, output, 7, gi) \ + if (laneCount < 12) { \ + if (laneCount < 10) { \ + if (laneCount < 9) { \ + X##go ^= trailingBits; \ + } \ + else { \ + unwrapOneInvert(X, input, output, 8, go) \ + X##gu ^= trailingBits; \ + } \ + } \ + else { \ + unwrapOneInvert(X, input, output, 8, go) \ + unwrapOne(X, input, output, 9, gu) \ + if (laneCount < 11) { \ + X##ka ^= trailingBits; \ + } \ + else { \ + unwrapOne(X, input, output, 10, ka) \ + X##ke ^= trailingBits; \ + } \ + } \ + } \ + else { \ + unwrapOneInvert(X, input, output, 8, go) \ + unwrapOne(X, input, output, 9, gu) \ + unwrapOne(X, input, output, 10, ka) \ + unwrapOne(X, input, output, 11, ke) \ + if (laneCount < 14) { \ + if (laneCount < 13) { \ + X##ki ^= trailingBits; \ + } \ + else { \ + unwrapOneInvert(X, input, output, 12, ki) \ + X##ko ^= trailingBits; \ + } \ + } \ + else { \ + unwrapOneInvert(X, input, output, 12, ki) \ + unwrapOne(X, input, output, 13, ko) \ + if (laneCount < 15) { \ + X##ku ^= trailingBits; \ + } \ + else { \ + unwrapOne(X, input, output, 14, ku) \ + X##ma ^= trailingBits; \ + } \ + } \ + } \ + } \ + } \ + else { \ + unwrapOne(X, input, output, 0, ba) \ + unwrapOneInvert(X, input, output, 1, be) \ + unwrapOneInvert(X, input, output, 2, bi) \ + unwrapOne(X, input, output, 3, bo) \ + unwrapOne(X, input, output, 4, bu) \ + unwrapOne(X, input, output, 5, ga) \ + unwrapOne(X, input, output, 6, ge) \ + unwrapOne(X, input, output, 7, gi) \ + unwrapOneInvert(X, input, output, 8, go) \ + unwrapOne(X, input, output, 9, gu) \ + unwrapOne(X, input, output, 10, ka) \ + unwrapOne(X, input, output, 11, ke) \ + unwrapOneInvert(X, input, output, 12, ki) \ + unwrapOne(X, input, output, 13, ko) \ + unwrapOne(X, input, output, 14, ku) \ + unwrapOne(X, input, output, 15, ma) \ + if (laneCount < 24) { \ + if (laneCount < 20) { \ + if (laneCount < 18) { \ + if (laneCount < 17) { \ + X##me ^= trailingBits; \ + } \ + else { \ + unwrapOne(X, input, output, 16, me) \ + X##mi ^= trailingBits; \ + } \ + } \ + else { \ + unwrapOne(X, input, output, 16, me) \ + unwrapOneInvert(X, input, output, 17, mi) \ + if (laneCount < 19) { \ + X##mo ^= trailingBits; \ + } \ + else { \ + unwrapOne(X, input, output, 18, mo) \ + X##mu ^= trailingBits; \ + } \ + } \ + } \ + else { \ + unwrapOne(X, input, output, 16, me) \ + unwrapOneInvert(X, input, output, 17, mi) \ + unwrapOne(X, input, output, 18, mo) \ + unwrapOne(X, input, output, 19, mu) \ + if (laneCount < 22) { \ + if (laneCount < 21) { \ + X##sa ^= trailingBits; \ + } \ + else { \ + unwrapOneInvert(X, input, output, 20, sa) \ + X##se ^= trailingBits; \ + } \ + } \ + else { \ + unwrapOneInvert(X, input, output, 20, sa) \ + unwrapOne(X, input, output, 21, se) \ + if (laneCount < 23) { \ + X##si ^= trailingBits; \ + } \ + else { \ + unwrapOne(X, input, output, 22, si) \ + X##so ^= trailingBits; \ + } \ + } \ + } \ + } \ + else { \ + unwrapOne(X, input, output, 16, me) \ + unwrapOneInvert(X, input, output, 17, mi) \ + unwrapOne(X, input, output, 18, mo) \ + unwrapOne(X, input, output, 19, mu) \ + unwrapOneInvert(X, input, output, 20, sa) \ + unwrapOne(X, input, output, 21, se) \ + unwrapOne(X, input, output, 22, si) \ + unwrapOne(X, input, output, 23, so) \ + if (laneCount < 25) { \ + X##su ^= trailingBits; \ + } \ + else { \ + unwrapOne(X, input, output, 24, su) \ + } \ + } \ + } diff --git a/Modules/_sha3/kcp/KeccakP-1600-SnP-opt32.h b/Modules/_sha3/kcp/KeccakP-1600-SnP-opt32.h new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakP-1600-SnP-opt32.h @@ -0,0 +1,37 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#ifndef _KeccakP_1600_SnP_h_ +#define _KeccakP_1600_SnP_h_ + +/** For the documentation, see SnP-documentation.h. + */ + +#define KeccakP1600_implementation "in-place 32-bit optimized implementation" +#define KeccakP1600_stateSizeInBytes 200 +#define KeccakP1600_stateAlignment 8 + +#define KeccakP1600_StaticInitialize() +void KeccakP1600_Initialize(void *state); +void KeccakP1600_AddByte(void *state, unsigned char data, unsigned int offset); +void KeccakP1600_AddBytes(void *state, const unsigned char *data, unsigned int offset, unsigned int length); +void KeccakP1600_OverwriteBytes(void *state, const unsigned char *data, unsigned int offset, unsigned int length); +void KeccakP1600_OverwriteWithZeroes(void *state, unsigned int byteCount); +void KeccakP1600_Permute_12rounds(void *state); +void KeccakP1600_Permute_24rounds(void *state); +void KeccakP1600_ExtractBytes(const void *state, unsigned char *data, unsigned int offset, unsigned int length); +void KeccakP1600_ExtractAndAddBytes(const void *state, const unsigned char *input, unsigned char *output, unsigned int offset, unsigned int length); + +#endif diff --git a/Modules/_sha3/kcp/KeccakP-1600-SnP-opt64.h b/Modules/_sha3/kcp/KeccakP-1600-SnP-opt64.h new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakP-1600-SnP-opt64.h @@ -0,0 +1,49 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#ifndef _KeccakP_1600_SnP_h_ +#define _KeccakP_1600_SnP_h_ + +/** For the documentation, see SnP-documentation.h. + */ + +/* #include "brg_endian.h" */ +#include "KeccakP-1600-opt64-config.h" + +#define KeccakP1600_implementation "generic 64-bit optimized implementation (" KeccakP1600_implementation_config ")" +#define KeccakP1600_stateSizeInBytes 200 +#define KeccakP1600_stateAlignment 8 +#define KeccakF1600_FastLoop_supported + +#include + +#define KeccakP1600_StaticInitialize() +void KeccakP1600_Initialize(void *state); +#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN) +#define KeccakP1600_AddByte(state, byte, offset) \ + ((unsigned char*)(state))[(offset)] ^= (byte) +#else +void KeccakP1600_AddByte(void *state, unsigned char data, unsigned int offset); +#endif +void KeccakP1600_AddBytes(void *state, const unsigned char *data, unsigned int offset, unsigned int length); +void KeccakP1600_OverwriteBytes(void *state, const unsigned char *data, unsigned int offset, unsigned int length); +void KeccakP1600_OverwriteWithZeroes(void *state, unsigned int byteCount); +void KeccakP1600_Permute_12rounds(void *state); +void KeccakP1600_Permute_24rounds(void *state); +void KeccakP1600_ExtractBytes(const void *state, unsigned char *data, unsigned int offset, unsigned int length); +void KeccakP1600_ExtractAndAddBytes(const void *state, const unsigned char *input, unsigned char *output, unsigned int offset, unsigned int length); +size_t KeccakF1600_FastLoop_Absorb(void *state, unsigned int laneCount, const unsigned char *data, size_t dataByteLen); + +#endif diff --git a/Modules/_sha3/kcp/KeccakP-1600-SnP.h b/Modules/_sha3/kcp/KeccakP-1600-SnP.h new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakP-1600-SnP.h @@ -0,0 +1,7 @@ +#if KeccakOpt == 64 + #include "KeccakP-1600-SnP-opt64.h" +#elif KeccakOpt == 32 + #include "KeccakP-1600-SnP-opt32.h" +#else + #error "No KeccakOpt" +#endif diff --git a/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c b/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c @@ -0,0 +1,1160 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#include +/* #include "brg_endian.h" */ +#include "KeccakP-1600-SnP.h" +#include "SnP-Relaned.h" + +typedef unsigned char UINT8; +typedef unsigned int UINT32; +/* WARNING: on 8-bit and 16-bit platforms, this should be replaced by: */ + +/*typedef unsigned long UINT32; */ + + +#define ROL32(a, offset) ((((UINT32)a) << (offset)) ^ (((UINT32)a) >> (32-(offset)))) + +/* Credit to Henry S. Warren, Hacker's Delight, Addison-Wesley, 2002 */ + +#define prepareToBitInterleaving(low, high, temp, temp0, temp1) \ + temp0 = (low); \ + temp = (temp0 ^ (temp0 >> 1)) & 0x22222222UL; temp0 = temp0 ^ temp ^ (temp << 1); \ + temp = (temp0 ^ (temp0 >> 2)) & 0x0C0C0C0CUL; temp0 = temp0 ^ temp ^ (temp << 2); \ + temp = (temp0 ^ (temp0 >> 4)) & 0x00F000F0UL; temp0 = temp0 ^ temp ^ (temp << 4); \ + temp = (temp0 ^ (temp0 >> 8)) & 0x0000FF00UL; temp0 = temp0 ^ temp ^ (temp << 8); \ + temp1 = (high); \ + temp = (temp1 ^ (temp1 >> 1)) & 0x22222222UL; temp1 = temp1 ^ temp ^ (temp << 1); \ + temp = (temp1 ^ (temp1 >> 2)) & 0x0C0C0C0CUL; temp1 = temp1 ^ temp ^ (temp << 2); \ + temp = (temp1 ^ (temp1 >> 4)) & 0x00F000F0UL; temp1 = temp1 ^ temp ^ (temp << 4); \ + temp = (temp1 ^ (temp1 >> 8)) & 0x0000FF00UL; temp1 = temp1 ^ temp ^ (temp << 8); + +#define toBitInterleavingAndXOR(low, high, even, odd, temp, temp0, temp1) \ + prepareToBitInterleaving(low, high, temp, temp0, temp1) \ + even ^= (temp0 & 0x0000FFFF) | (temp1 << 16); \ + odd ^= (temp0 >> 16) | (temp1 & 0xFFFF0000); + +#define toBitInterleavingAndAND(low, high, even, odd, temp, temp0, temp1) \ + prepareToBitInterleaving(low, high, temp, temp0, temp1) \ + even &= (temp0 & 0x0000FFFF) | (temp1 << 16); \ + odd &= (temp0 >> 16) | (temp1 & 0xFFFF0000); + +#define toBitInterleavingAndSet(low, high, even, odd, temp, temp0, temp1) \ + prepareToBitInterleaving(low, high, temp, temp0, temp1) \ + even = (temp0 & 0x0000FFFF) | (temp1 << 16); \ + odd = (temp0 >> 16) | (temp1 & 0xFFFF0000); + +/* Credit to Henry S. Warren, Hacker's Delight, Addison-Wesley, 2002 */ + +#define prepareFromBitInterleaving(even, odd, temp, temp0, temp1) \ + temp0 = (even); \ + temp1 = (odd); \ + temp = (temp0 & 0x0000FFFF) | (temp1 << 16); \ + temp1 = (temp0 >> 16) | (temp1 & 0xFFFF0000); \ + temp0 = temp; \ + temp = (temp0 ^ (temp0 >> 8)) & 0x0000FF00UL; temp0 = temp0 ^ temp ^ (temp << 8); \ + temp = (temp0 ^ (temp0 >> 4)) & 0x00F000F0UL; temp0 = temp0 ^ temp ^ (temp << 4); \ + temp = (temp0 ^ (temp0 >> 2)) & 0x0C0C0C0CUL; temp0 = temp0 ^ temp ^ (temp << 2); \ + temp = (temp0 ^ (temp0 >> 1)) & 0x22222222UL; temp0 = temp0 ^ temp ^ (temp << 1); \ + temp = (temp1 ^ (temp1 >> 8)) & 0x0000FF00UL; temp1 = temp1 ^ temp ^ (temp << 8); \ + temp = (temp1 ^ (temp1 >> 4)) & 0x00F000F0UL; temp1 = temp1 ^ temp ^ (temp << 4); \ + temp = (temp1 ^ (temp1 >> 2)) & 0x0C0C0C0CUL; temp1 = temp1 ^ temp ^ (temp << 2); \ + temp = (temp1 ^ (temp1 >> 1)) & 0x22222222UL; temp1 = temp1 ^ temp ^ (temp << 1); + +#define fromBitInterleaving(even, odd, low, high, temp, temp0, temp1) \ + prepareFromBitInterleaving(even, odd, temp, temp0, temp1) \ + low = temp0; \ + high = temp1; + +#define fromBitInterleavingAndXOR(even, odd, lowIn, highIn, lowOut, highOut, temp, temp0, temp1) \ + prepareFromBitInterleaving(even, odd, temp, temp0, temp1) \ + lowOut = lowIn ^ temp0; \ + highOut = highIn ^ temp1; + +void KeccakP1600_SetBytesInLaneToZero(void *state, unsigned int lanePosition, unsigned int offset, unsigned int length) +{ + UINT8 laneAsBytes[8]; + UINT32 low, high; + UINT32 temp, temp0, temp1; + UINT32 *stateAsHalfLanes = (UINT32*)state; + + memset(laneAsBytes, 0xFF, offset); + memset(laneAsBytes+offset, 0x00, length); + memset(laneAsBytes+offset+length, 0xFF, 8-offset-length); +#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN) + low = *((UINT32*)(laneAsBytes+0)); + high = *((UINT32*)(laneAsBytes+4)); +#else + low = laneAsBytes[0] + | ((UINT32)(laneAsBytes[1]) << 8) + | ((UINT32)(laneAsBytes[2]) << 16) + | ((UINT32)(laneAsBytes[3]) << 24); + high = laneAsBytes[4] + | ((UINT32)(laneAsBytes[5]) << 8) + | ((UINT32)(laneAsBytes[6]) << 16) + | ((UINT32)(laneAsBytes[7]) << 24); +#endif + toBitInterleavingAndAND(low, high, stateAsHalfLanes[lanePosition*2+0], stateAsHalfLanes[lanePosition*2+1], temp, temp0, temp1); +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_Initialize(void *state) +{ + memset(state, 0, 200); +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_AddByte(void *state, unsigned char byte, unsigned int offset) +{ + unsigned int lanePosition = offset/8; + unsigned int offsetInLane = offset%8; + UINT32 low, high; + UINT32 temp, temp0, temp1; + UINT32 *stateAsHalfLanes = (UINT32*)state; + + if (offsetInLane < 4) { + low = (UINT32)byte << (offsetInLane*8); + high = 0; + } + else { + low = 0; + high = (UINT32)byte << ((offsetInLane-4)*8); + } + toBitInterleavingAndXOR(low, high, stateAsHalfLanes[lanePosition*2+0], stateAsHalfLanes[lanePosition*2+1], temp, temp0, temp1); +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_AddBytesInLane(void *state, unsigned int lanePosition, const unsigned char *data, unsigned int offset, unsigned int length) +{ + UINT8 laneAsBytes[8]; + UINT32 low, high; + UINT32 temp, temp0, temp1; + UINT32 *stateAsHalfLanes = (UINT32*)state; + + memset(laneAsBytes, 0, 8); + memcpy(laneAsBytes+offset, data, length); +#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN) + low = *((UINT32*)(laneAsBytes+0)); + high = *((UINT32*)(laneAsBytes+4)); +#else + low = laneAsBytes[0] + | ((UINT32)(laneAsBytes[1]) << 8) + | ((UINT32)(laneAsBytes[2]) << 16) + | ((UINT32)(laneAsBytes[3]) << 24); + high = laneAsBytes[4] + | ((UINT32)(laneAsBytes[5]) << 8) + | ((UINT32)(laneAsBytes[6]) << 16) + | ((UINT32)(laneAsBytes[7]) << 24); +#endif + toBitInterleavingAndXOR(low, high, stateAsHalfLanes[lanePosition*2+0], stateAsHalfLanes[lanePosition*2+1], temp, temp0, temp1); +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_AddLanes(void *state, const unsigned char *data, unsigned int laneCount) +{ +#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN) + const UINT32 * pI = (const UINT32 *)data; + UINT32 * pS = (UINT32*)state; + UINT32 t, x0, x1; + int i; + for (i = laneCount-1; i >= 0; --i) { +#ifdef NO_MISALIGNED_ACCESSES + UINT32 low; + UINT32 high; + memcpy(&low, pI++, 4); + memcpy(&high, pI++, 4); + toBitInterleavingAndXOR(low, high, *(pS++), *(pS++), t, x0, x1); +#else + toBitInterleavingAndXOR(*(pI++), *(pI++), *(pS++), *(pS++), t, x0, x1) +#endif + } +#else + unsigned int lanePosition; + for(lanePosition=0; lanePosition= 0; --i) { +#ifdef NO_MISALIGNED_ACCESSES + UINT32 low; + UINT32 high; + memcpy(&low, pI++, 4); + memcpy(&high, pI++, 4); + toBitInterleavingAndSet(low, high, *(pS++), *(pS++), t, x0, x1); +#else + toBitInterleavingAndSet(*(pI++), *(pI++), *(pS++), *(pS++), t, x0, x1) +#endif + } +#else + unsigned int lanePosition; + for(lanePosition=0; lanePosition> 8) & 0xFF; + laneAsBytes[2] = (low >> 16) & 0xFF; + laneAsBytes[3] = (low >> 24) & 0xFF; + laneAsBytes[4] = high & 0xFF; + laneAsBytes[5] = (high >> 8) & 0xFF; + laneAsBytes[6] = (high >> 16) & 0xFF; + laneAsBytes[7] = (high >> 24) & 0xFF; +#endif + memcpy(data, laneAsBytes+offset, length); +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_ExtractLanes(const void *state, unsigned char *data, unsigned int laneCount) +{ +#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN) + UINT32 * pI = (UINT32 *)data; + const UINT32 * pS = ( const UINT32 *)state; + UINT32 t, x0, x1; + int i; + for (i = laneCount-1; i >= 0; --i) { +#ifdef NO_MISALIGNED_ACCESSES + UINT32 low; + UINT32 high; + fromBitInterleaving(*(pS++), *(pS++), low, high, t, x0, x1); + memcpy(pI++, &low, 4); + memcpy(pI++, &high, 4); +#else + fromBitInterleaving(*(pS++), *(pS++), *(pI++), *(pI++), t, x0, x1) +#endif + } +#else + unsigned int lanePosition; + for(lanePosition=0; lanePosition> 8) & 0xFF; + laneAsBytes[2] = (low >> 16) & 0xFF; + laneAsBytes[3] = (low >> 24) & 0xFF; + laneAsBytes[4] = high & 0xFF; + laneAsBytes[5] = (high >> 8) & 0xFF; + laneAsBytes[6] = (high >> 16) & 0xFF; + laneAsBytes[7] = (high >> 24) & 0xFF; + memcpy(data+lanePosition*8, laneAsBytes, 8); + } +#endif +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_ExtractBytes(const void *state, unsigned char *data, unsigned int offset, unsigned int length) +{ + SnP_ExtractBytes(state, data, offset, length, KeccakP1600_ExtractLanes, KeccakP1600_ExtractBytesInLane, 8); +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_ExtractAndAddBytesInLane(const void *state, unsigned int lanePosition, const unsigned char *input, unsigned char *output, unsigned int offset, unsigned int length) +{ + UINT32 *stateAsHalfLanes = (UINT32*)state; + UINT32 low, high, temp, temp0, temp1; + UINT8 laneAsBytes[8]; + unsigned int i; + + fromBitInterleaving(stateAsHalfLanes[lanePosition*2], stateAsHalfLanes[lanePosition*2+1], low, high, temp, temp0, temp1); +#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN) + *((UINT32*)(laneAsBytes+0)) = low; + *((UINT32*)(laneAsBytes+4)) = high; +#else + laneAsBytes[0] = low & 0xFF; + laneAsBytes[1] = (low >> 8) & 0xFF; + laneAsBytes[2] = (low >> 16) & 0xFF; + laneAsBytes[3] = (low >> 24) & 0xFF; + laneAsBytes[4] = high & 0xFF; + laneAsBytes[5] = (high >> 8) & 0xFF; + laneAsBytes[6] = (high >> 16) & 0xFF; + laneAsBytes[7] = (high >> 24) & 0xFF; +#endif + for(i=0; i= 0; --i) { +#ifdef NO_MISALIGNED_ACCESSES + UINT32 low; + UINT32 high; + fromBitInterleaving(*(pS++), *(pS++), low, high, t, x0, x1); + *(pO++) = *(pI++) ^ low; + *(pO++) = *(pI++) ^ high; +#else + fromBitInterleavingAndXOR(*(pS++), *(pS++), *(pI++), *(pI++), *(pO++), *(pO++), t, x0, x1) +#endif + } +#else + unsigned int lanePosition; + for(lanePosition=0; lanePosition> 8) & 0xFF; + laneAsBytes[2] = (low >> 16) & 0xFF; + laneAsBytes[3] = (low >> 24) & 0xFF; + laneAsBytes[4] = high & 0xFF; + laneAsBytes[5] = (high >> 8) & 0xFF; + laneAsBytes[6] = (high >> 16) & 0xFF; + laneAsBytes[7] = (high >> 24) & 0xFF; + ((UINT32*)(output+lanePosition*8))[0] = ((UINT32*)(input+lanePosition*8))[0] ^ (*(const UINT32*)(laneAsBytes+0)); + ((UINT32*)(output+lanePosition*8))[1] = ((UINT32*)(input+lanePosition*8))[0] ^ (*(const UINT32*)(laneAsBytes+4)); + } +#endif +} +/* ---------------------------------------------------------------- */ + +void KeccakP1600_ExtractAndAddBytes(const void *state, const unsigned char *input, unsigned char *output, unsigned int offset, unsigned int length) +{ + SnP_ExtractAndAddBytes(state, input, output, offset, length, KeccakP1600_ExtractAndAddLanes, KeccakP1600_ExtractAndAddBytesInLane, 8); +} + +/* ---------------------------------------------------------------- */ + +static const UINT32 KeccakF1600RoundConstants_int2[2*24+1] = +{ + 0x00000001UL, 0x00000000UL, + 0x00000000UL, 0x00000089UL, + 0x00000000UL, 0x8000008bUL, + 0x00000000UL, 0x80008080UL, + 0x00000001UL, 0x0000008bUL, + 0x00000001UL, 0x00008000UL, + 0x00000001UL, 0x80008088UL, + 0x00000001UL, 0x80000082UL, + 0x00000000UL, 0x0000000bUL, + 0x00000000UL, 0x0000000aUL, + 0x00000001UL, 0x00008082UL, + 0x00000000UL, 0x00008003UL, + 0x00000001UL, 0x0000808bUL, + 0x00000001UL, 0x8000000bUL, + 0x00000001UL, 0x8000008aUL, + 0x00000001UL, 0x80000081UL, + 0x00000000UL, 0x80000081UL, + 0x00000000UL, 0x80000008UL, + 0x00000000UL, 0x00000083UL, + 0x00000000UL, 0x80008003UL, + 0x00000001UL, 0x80008088UL, + 0x00000000UL, 0x80000088UL, + 0x00000001UL, 0x00008000UL, + 0x00000000UL, 0x80008082UL, + 0x000000FFUL +}; + +#define KeccakAtoD_round0() \ + Cx = Abu0^Agu0^Aku0^Amu0^Asu0; \ + Du1 = Abe1^Age1^Ake1^Ame1^Ase1; \ + Da0 = Cx^ROL32(Du1, 1); \ + Cz = Abu1^Agu1^Aku1^Amu1^Asu1; \ + Du0 = Abe0^Age0^Ake0^Ame0^Ase0; \ + Da1 = Cz^Du0; \ +\ + Cw = Abi0^Agi0^Aki0^Ami0^Asi0; \ + Do0 = Cw^ROL32(Cz, 1); \ + Cy = Abi1^Agi1^Aki1^Ami1^Asi1; \ + Do1 = Cy^Cx; \ +\ + Cx = Aba0^Aga0^Aka0^Ama0^Asa0; \ + De0 = Cx^ROL32(Cy, 1); \ + Cz = Aba1^Aga1^Aka1^Ama1^Asa1; \ + De1 = Cz^Cw; \ +\ + Cy = Abo1^Ago1^Ako1^Amo1^Aso1; \ + Di0 = Du0^ROL32(Cy, 1); \ + Cw = Abo0^Ago0^Ako0^Amo0^Aso0; \ + Di1 = Du1^Cw; \ +\ + Du0 = Cw^ROL32(Cz, 1); \ + Du1 = Cy^Cx; \ + +#define KeccakAtoD_round1() \ + Cx = Asu0^Agu0^Amu0^Abu1^Aku1; \ + Du1 = Age1^Ame0^Abe0^Ake1^Ase1; \ + Da0 = Cx^ROL32(Du1, 1); \ + Cz = Asu1^Agu1^Amu1^Abu0^Aku0; \ + Du0 = Age0^Ame1^Abe1^Ake0^Ase0; \ + Da1 = Cz^Du0; \ +\ + Cw = Aki1^Asi1^Agi0^Ami1^Abi0; \ + Do0 = Cw^ROL32(Cz, 1); \ + Cy = Aki0^Asi0^Agi1^Ami0^Abi1; \ + Do1 = Cy^Cx; \ +\ + Cx = Aba0^Aka1^Asa0^Aga0^Ama1; \ + De0 = Cx^ROL32(Cy, 1); \ + Cz = Aba1^Aka0^Asa1^Aga1^Ama0; \ + De1 = Cz^Cw; \ +\ + Cy = Amo0^Abo1^Ako0^Aso1^Ago0; \ + Di0 = Du0^ROL32(Cy, 1); \ + Cw = Amo1^Abo0^Ako1^Aso0^Ago1; \ + Di1 = Du1^Cw; \ +\ + Du0 = Cw^ROL32(Cz, 1); \ + Du1 = Cy^Cx; \ + +#define KeccakAtoD_round2() \ + Cx = Aku1^Agu0^Abu1^Asu1^Amu1; \ + Du1 = Ame0^Ake0^Age0^Abe0^Ase1; \ + Da0 = Cx^ROL32(Du1, 1); \ + Cz = Aku0^Agu1^Abu0^Asu0^Amu0; \ + Du0 = Ame1^Ake1^Age1^Abe1^Ase0; \ + Da1 = Cz^Du0; \ +\ + Cw = Agi1^Abi1^Asi1^Ami0^Aki1; \ + Do0 = Cw^ROL32(Cz, 1); \ + Cy = Agi0^Abi0^Asi0^Ami1^Aki0; \ + Do1 = Cy^Cx; \ +\ + Cx = Aba0^Asa1^Ama1^Aka1^Aga1; \ + De0 = Cx^ROL32(Cy, 1); \ + Cz = Aba1^Asa0^Ama0^Aka0^Aga0; \ + De1 = Cz^Cw; \ +\ + Cy = Aso0^Amo0^Ako1^Ago0^Abo0; \ + Di0 = Du0^ROL32(Cy, 1); \ + Cw = Aso1^Amo1^Ako0^Ago1^Abo1; \ + Di1 = Du1^Cw; \ +\ + Du0 = Cw^ROL32(Cz, 1); \ + Du1 = Cy^Cx; \ + +#define KeccakAtoD_round3() \ + Cx = Amu1^Agu0^Asu1^Aku0^Abu0; \ + Du1 = Ake0^Abe1^Ame1^Age0^Ase1; \ + Da0 = Cx^ROL32(Du1, 1); \ + Cz = Amu0^Agu1^Asu0^Aku1^Abu1; \ + Du0 = Ake1^Abe0^Ame0^Age1^Ase0; \ + Da1 = Cz^Du0; \ +\ + Cw = Asi0^Aki0^Abi1^Ami1^Agi1; \ + Do0 = Cw^ROL32(Cz, 1); \ + Cy = Asi1^Aki1^Abi0^Ami0^Agi0; \ + Do1 = Cy^Cx; \ +\ + Cx = Aba0^Ama0^Aga1^Asa1^Aka0; \ + De0 = Cx^ROL32(Cy, 1); \ + Cz = Aba1^Ama1^Aga0^Asa0^Aka1; \ + De1 = Cz^Cw; \ +\ + Cy = Ago1^Aso0^Ako0^Abo0^Amo1; \ + Di0 = Du0^ROL32(Cy, 1); \ + Cw = Ago0^Aso1^Ako1^Abo1^Amo0; \ + Di1 = Du1^Cw; \ +\ + Du0 = Cw^ROL32(Cz, 1); \ + Du1 = Cy^Cx; \ + +void KeccakP1600_Permute_Nrounds(void *state, unsigned int nRounds) +{ + { + UINT32 Da0, De0, Di0, Do0, Du0; + UINT32 Da1, De1, Di1, Do1, Du1; + UINT32 Ca0, Ce0, Ci0, Co0, Cu0; + UINT32 Cx, Cy, Cz, Cw; + #define Ba Ca0 + #define Be Ce0 + #define Bi Ci0 + #define Bo Co0 + #define Bu Cu0 + const UINT32 *pRoundConstants = KeccakF1600RoundConstants_int2+(24-nRounds)*2; + UINT32 *stateAsHalfLanes = (UINT32*)state; + #define Aba0 stateAsHalfLanes[ 0] + #define Aba1 stateAsHalfLanes[ 1] + #define Abe0 stateAsHalfLanes[ 2] + #define Abe1 stateAsHalfLanes[ 3] + #define Abi0 stateAsHalfLanes[ 4] + #define Abi1 stateAsHalfLanes[ 5] + #define Abo0 stateAsHalfLanes[ 6] + #define Abo1 stateAsHalfLanes[ 7] + #define Abu0 stateAsHalfLanes[ 8] + #define Abu1 stateAsHalfLanes[ 9] + #define Aga0 stateAsHalfLanes[10] + #define Aga1 stateAsHalfLanes[11] + #define Age0 stateAsHalfLanes[12] + #define Age1 stateAsHalfLanes[13] + #define Agi0 stateAsHalfLanes[14] + #define Agi1 stateAsHalfLanes[15] + #define Ago0 stateAsHalfLanes[16] + #define Ago1 stateAsHalfLanes[17] + #define Agu0 stateAsHalfLanes[18] + #define Agu1 stateAsHalfLanes[19] + #define Aka0 stateAsHalfLanes[20] + #define Aka1 stateAsHalfLanes[21] + #define Ake0 stateAsHalfLanes[22] + #define Ake1 stateAsHalfLanes[23] + #define Aki0 stateAsHalfLanes[24] + #define Aki1 stateAsHalfLanes[25] + #define Ako0 stateAsHalfLanes[26] + #define Ako1 stateAsHalfLanes[27] + #define Aku0 stateAsHalfLanes[28] + #define Aku1 stateAsHalfLanes[29] + #define Ama0 stateAsHalfLanes[30] + #define Ama1 stateAsHalfLanes[31] + #define Ame0 stateAsHalfLanes[32] + #define Ame1 stateAsHalfLanes[33] + #define Ami0 stateAsHalfLanes[34] + #define Ami1 stateAsHalfLanes[35] + #define Amo0 stateAsHalfLanes[36] + #define Amo1 stateAsHalfLanes[37] + #define Amu0 stateAsHalfLanes[38] + #define Amu1 stateAsHalfLanes[39] + #define Asa0 stateAsHalfLanes[40] + #define Asa1 stateAsHalfLanes[41] + #define Ase0 stateAsHalfLanes[42] + #define Ase1 stateAsHalfLanes[43] + #define Asi0 stateAsHalfLanes[44] + #define Asi1 stateAsHalfLanes[45] + #define Aso0 stateAsHalfLanes[46] + #define Aso1 stateAsHalfLanes[47] + #define Asu0 stateAsHalfLanes[48] + #define Asu1 stateAsHalfLanes[49] + + do + { + /* --- Code for 4 rounds */ + + /* --- using factor 2 interleaving, 64-bit lanes mapped to 32-bit words */ + + KeccakAtoD_round0(); + + Ba = (Aba0^Da0); + Be = ROL32((Age0^De0), 22); + Bi = ROL32((Aki1^Di1), 22); + Bo = ROL32((Amo1^Do1), 11); + Bu = ROL32((Asu0^Du0), 7); + Aba0 = Ba ^((~Be)& Bi ); + Aba0 ^= *(pRoundConstants++); + Age0 = Be ^((~Bi)& Bo ); + Aki1 = Bi ^((~Bo)& Bu ); + Amo1 = Bo ^((~Bu)& Ba ); + Asu0 = Bu ^((~Ba)& Be ); + + Ba = (Aba1^Da1); + Be = ROL32((Age1^De1), 22); + Bi = ROL32((Aki0^Di0), 21); + Bo = ROL32((Amo0^Do0), 10); + Bu = ROL32((Asu1^Du1), 7); + Aba1 = Ba ^((~Be)& Bi ); + Aba1 ^= *(pRoundConstants++); + Age1 = Be ^((~Bi)& Bo ); + Aki0 = Bi ^((~Bo)& Bu ); + Amo0 = Bo ^((~Bu)& Ba ); + Asu1 = Bu ^((~Ba)& Be ); + + Bi = ROL32((Aka1^Da1), 2); + Bo = ROL32((Ame1^De1), 23); + Bu = ROL32((Asi1^Di1), 31); + Ba = ROL32((Abo0^Do0), 14); + Be = ROL32((Agu0^Du0), 10); + Aka1 = Ba ^((~Be)& Bi ); + Ame1 = Be ^((~Bi)& Bo ); + Asi1 = Bi ^((~Bo)& Bu ); + Abo0 = Bo ^((~Bu)& Ba ); + Agu0 = Bu ^((~Ba)& Be ); + + Bi = ROL32((Aka0^Da0), 1); + Bo = ROL32((Ame0^De0), 22); + Bu = ROL32((Asi0^Di0), 30); + Ba = ROL32((Abo1^Do1), 14); + Be = ROL32((Agu1^Du1), 10); + Aka0 = Ba ^((~Be)& Bi ); + Ame0 = Be ^((~Bi)& Bo ); + Asi0 = Bi ^((~Bo)& Bu ); + Abo1 = Bo ^((~Bu)& Ba ); + Agu1 = Bu ^((~Ba)& Be ); + + Bu = ROL32((Asa0^Da0), 9); + Ba = ROL32((Abe1^De1), 1); + Be = ROL32((Agi0^Di0), 3); + Bi = ROL32((Ako1^Do1), 13); + Bo = ROL32((Amu0^Du0), 4); + Asa0 = Ba ^((~Be)& Bi ); + Abe1 = Be ^((~Bi)& Bo ); + Agi0 = Bi ^((~Bo)& Bu ); + Ako1 = Bo ^((~Bu)& Ba ); + Amu0 = Bu ^((~Ba)& Be ); + + Bu = ROL32((Asa1^Da1), 9); + Ba = (Abe0^De0); + Be = ROL32((Agi1^Di1), 3); + Bi = ROL32((Ako0^Do0), 12); + Bo = ROL32((Amu1^Du1), 4); + Asa1 = Ba ^((~Be)& Bi ); + Abe0 = Be ^((~Bi)& Bo ); + Agi1 = Bi ^((~Bo)& Bu ); + Ako0 = Bo ^((~Bu)& Ba ); + Amu1 = Bu ^((~Ba)& Be ); + + Be = ROL32((Aga0^Da0), 18); + Bi = ROL32((Ake0^De0), 5); + Bo = ROL32((Ami1^Di1), 8); + Bu = ROL32((Aso0^Do0), 28); + Ba = ROL32((Abu1^Du1), 14); + Aga0 = Ba ^((~Be)& Bi ); + Ake0 = Be ^((~Bi)& Bo ); + Ami1 = Bi ^((~Bo)& Bu ); + Aso0 = Bo ^((~Bu)& Ba ); + Abu1 = Bu ^((~Ba)& Be ); + + Be = ROL32((Aga1^Da1), 18); + Bi = ROL32((Ake1^De1), 5); + Bo = ROL32((Ami0^Di0), 7); + Bu = ROL32((Aso1^Do1), 28); + Ba = ROL32((Abu0^Du0), 13); + Aga1 = Ba ^((~Be)& Bi ); + Ake1 = Be ^((~Bi)& Bo ); + Ami0 = Bi ^((~Bo)& Bu ); + Aso1 = Bo ^((~Bu)& Ba ); + Abu0 = Bu ^((~Ba)& Be ); + + Bo = ROL32((Ama1^Da1), 21); + Bu = ROL32((Ase0^De0), 1); + Ba = ROL32((Abi0^Di0), 31); + Be = ROL32((Ago1^Do1), 28); + Bi = ROL32((Aku1^Du1), 20); + Ama1 = Ba ^((~Be)& Bi ); + Ase0 = Be ^((~Bi)& Bo ); + Abi0 = Bi ^((~Bo)& Bu ); + Ago1 = Bo ^((~Bu)& Ba ); + Aku1 = Bu ^((~Ba)& Be ); + + Bo = ROL32((Ama0^Da0), 20); + Bu = ROL32((Ase1^De1), 1); + Ba = ROL32((Abi1^Di1), 31); + Be = ROL32((Ago0^Do0), 27); + Bi = ROL32((Aku0^Du0), 19); + Ama0 = Ba ^((~Be)& Bi ); + Ase1 = Be ^((~Bi)& Bo ); + Abi1 = Bi ^((~Bo)& Bu ); + Ago0 = Bo ^((~Bu)& Ba ); + Aku0 = Bu ^((~Ba)& Be ); + + KeccakAtoD_round1(); + + Ba = (Aba0^Da0); + Be = ROL32((Ame1^De0), 22); + Bi = ROL32((Agi1^Di1), 22); + Bo = ROL32((Aso1^Do1), 11); + Bu = ROL32((Aku1^Du0), 7); + Aba0 = Ba ^((~Be)& Bi ); + Aba0 ^= *(pRoundConstants++); + Ame1 = Be ^((~Bi)& Bo ); + Agi1 = Bi ^((~Bo)& Bu ); + Aso1 = Bo ^((~Bu)& Ba ); + Aku1 = Bu ^((~Ba)& Be ); + + Ba = (Aba1^Da1); + Be = ROL32((Ame0^De1), 22); + Bi = ROL32((Agi0^Di0), 21); + Bo = ROL32((Aso0^Do0), 10); + Bu = ROL32((Aku0^Du1), 7); + Aba1 = Ba ^((~Be)& Bi ); + Aba1 ^= *(pRoundConstants++); + Ame0 = Be ^((~Bi)& Bo ); + Agi0 = Bi ^((~Bo)& Bu ); + Aso0 = Bo ^((~Bu)& Ba ); + Aku0 = Bu ^((~Ba)& Be ); + + Bi = ROL32((Asa1^Da1), 2); + Bo = ROL32((Ake1^De1), 23); + Bu = ROL32((Abi1^Di1), 31); + Ba = ROL32((Amo1^Do0), 14); + Be = ROL32((Agu0^Du0), 10); + Asa1 = Ba ^((~Be)& Bi ); + Ake1 = Be ^((~Bi)& Bo ); + Abi1 = Bi ^((~Bo)& Bu ); + Amo1 = Bo ^((~Bu)& Ba ); + Agu0 = Bu ^((~Ba)& Be ); + + Bi = ROL32((Asa0^Da0), 1); + Bo = ROL32((Ake0^De0), 22); + Bu = ROL32((Abi0^Di0), 30); + Ba = ROL32((Amo0^Do1), 14); + Be = ROL32((Agu1^Du1), 10); + Asa0 = Ba ^((~Be)& Bi ); + Ake0 = Be ^((~Bi)& Bo ); + Abi0 = Bi ^((~Bo)& Bu ); + Amo0 = Bo ^((~Bu)& Ba ); + Agu1 = Bu ^((~Ba)& Be ); + + Bu = ROL32((Ama1^Da0), 9); + Ba = ROL32((Age1^De1), 1); + Be = ROL32((Asi1^Di0), 3); + Bi = ROL32((Ako0^Do1), 13); + Bo = ROL32((Abu1^Du0), 4); + Ama1 = Ba ^((~Be)& Bi ); + Age1 = Be ^((~Bi)& Bo ); + Asi1 = Bi ^((~Bo)& Bu ); + Ako0 = Bo ^((~Bu)& Ba ); + Abu1 = Bu ^((~Ba)& Be ); + + Bu = ROL32((Ama0^Da1), 9); + Ba = (Age0^De0); + Be = ROL32((Asi0^Di1), 3); + Bi = ROL32((Ako1^Do0), 12); + Bo = ROL32((Abu0^Du1), 4); + Ama0 = Ba ^((~Be)& Bi ); + Age0 = Be ^((~Bi)& Bo ); + Asi0 = Bi ^((~Bo)& Bu ); + Ako1 = Bo ^((~Bu)& Ba ); + Abu0 = Bu ^((~Ba)& Be ); + + Be = ROL32((Aka1^Da0), 18); + Bi = ROL32((Abe1^De0), 5); + Bo = ROL32((Ami0^Di1), 8); + Bu = ROL32((Ago1^Do0), 28); + Ba = ROL32((Asu1^Du1), 14); + Aka1 = Ba ^((~Be)& Bi ); + Abe1 = Be ^((~Bi)& Bo ); + Ami0 = Bi ^((~Bo)& Bu ); + Ago1 = Bo ^((~Bu)& Ba ); + Asu1 = Bu ^((~Ba)& Be ); + + Be = ROL32((Aka0^Da1), 18); + Bi = ROL32((Abe0^De1), 5); + Bo = ROL32((Ami1^Di0), 7); + Bu = ROL32((Ago0^Do1), 28); + Ba = ROL32((Asu0^Du0), 13); + Aka0 = Ba ^((~Be)& Bi ); + Abe0 = Be ^((~Bi)& Bo ); + Ami1 = Bi ^((~Bo)& Bu ); + Ago0 = Bo ^((~Bu)& Ba ); + Asu0 = Bu ^((~Ba)& Be ); + + Bo = ROL32((Aga1^Da1), 21); + Bu = ROL32((Ase0^De0), 1); + Ba = ROL32((Aki1^Di0), 31); + Be = ROL32((Abo1^Do1), 28); + Bi = ROL32((Amu1^Du1), 20); + Aga1 = Ba ^((~Be)& Bi ); + Ase0 = Be ^((~Bi)& Bo ); + Aki1 = Bi ^((~Bo)& Bu ); + Abo1 = Bo ^((~Bu)& Ba ); + Amu1 = Bu ^((~Ba)& Be ); + + Bo = ROL32((Aga0^Da0), 20); + Bu = ROL32((Ase1^De1), 1); + Ba = ROL32((Aki0^Di1), 31); + Be = ROL32((Abo0^Do0), 27); + Bi = ROL32((Amu0^Du0), 19); + Aga0 = Ba ^((~Be)& Bi ); + Ase1 = Be ^((~Bi)& Bo ); + Aki0 = Bi ^((~Bo)& Bu ); + Abo0 = Bo ^((~Bu)& Ba ); + Amu0 = Bu ^((~Ba)& Be ); + + KeccakAtoD_round2(); + + Ba = (Aba0^Da0); + Be = ROL32((Ake1^De0), 22); + Bi = ROL32((Asi0^Di1), 22); + Bo = ROL32((Ago0^Do1), 11); + Bu = ROL32((Amu1^Du0), 7); + Aba0 = Ba ^((~Be)& Bi ); + Aba0 ^= *(pRoundConstants++); + Ake1 = Be ^((~Bi)& Bo ); + Asi0 = Bi ^((~Bo)& Bu ); + Ago0 = Bo ^((~Bu)& Ba ); + Amu1 = Bu ^((~Ba)& Be ); + + Ba = (Aba1^Da1); + Be = ROL32((Ake0^De1), 22); + Bi = ROL32((Asi1^Di0), 21); + Bo = ROL32((Ago1^Do0), 10); + Bu = ROL32((Amu0^Du1), 7); + Aba1 = Ba ^((~Be)& Bi ); + Aba1 ^= *(pRoundConstants++); + Ake0 = Be ^((~Bi)& Bo ); + Asi1 = Bi ^((~Bo)& Bu ); + Ago1 = Bo ^((~Bu)& Ba ); + Amu0 = Bu ^((~Ba)& Be ); + + Bi = ROL32((Ama0^Da1), 2); + Bo = ROL32((Abe0^De1), 23); + Bu = ROL32((Aki0^Di1), 31); + Ba = ROL32((Aso1^Do0), 14); + Be = ROL32((Agu0^Du0), 10); + Ama0 = Ba ^((~Be)& Bi ); + Abe0 = Be ^((~Bi)& Bo ); + Aki0 = Bi ^((~Bo)& Bu ); + Aso1 = Bo ^((~Bu)& Ba ); + Agu0 = Bu ^((~Ba)& Be ); + + Bi = ROL32((Ama1^Da0), 1); + Bo = ROL32((Abe1^De0), 22); + Bu = ROL32((Aki1^Di0), 30); + Ba = ROL32((Aso0^Do1), 14); + Be = ROL32((Agu1^Du1), 10); + Ama1 = Ba ^((~Be)& Bi ); + Abe1 = Be ^((~Bi)& Bo ); + Aki1 = Bi ^((~Bo)& Bu ); + Aso0 = Bo ^((~Bu)& Ba ); + Agu1 = Bu ^((~Ba)& Be ); + + Bu = ROL32((Aga1^Da0), 9); + Ba = ROL32((Ame0^De1), 1); + Be = ROL32((Abi1^Di0), 3); + Bi = ROL32((Ako1^Do1), 13); + Bo = ROL32((Asu1^Du0), 4); + Aga1 = Ba ^((~Be)& Bi ); + Ame0 = Be ^((~Bi)& Bo ); + Abi1 = Bi ^((~Bo)& Bu ); + Ako1 = Bo ^((~Bu)& Ba ); + Asu1 = Bu ^((~Ba)& Be ); + + Bu = ROL32((Aga0^Da1), 9); + Ba = (Ame1^De0); + Be = ROL32((Abi0^Di1), 3); + Bi = ROL32((Ako0^Do0), 12); + Bo = ROL32((Asu0^Du1), 4); + Aga0 = Ba ^((~Be)& Bi ); + Ame1 = Be ^((~Bi)& Bo ); + Abi0 = Bi ^((~Bo)& Bu ); + Ako0 = Bo ^((~Bu)& Ba ); + Asu0 = Bu ^((~Ba)& Be ); + + Be = ROL32((Asa1^Da0), 18); + Bi = ROL32((Age1^De0), 5); + Bo = ROL32((Ami1^Di1), 8); + Bu = ROL32((Abo1^Do0), 28); + Ba = ROL32((Aku0^Du1), 14); + Asa1 = Ba ^((~Be)& Bi ); + Age1 = Be ^((~Bi)& Bo ); + Ami1 = Bi ^((~Bo)& Bu ); + Abo1 = Bo ^((~Bu)& Ba ); + Aku0 = Bu ^((~Ba)& Be ); + + Be = ROL32((Asa0^Da1), 18); + Bi = ROL32((Age0^De1), 5); + Bo = ROL32((Ami0^Di0), 7); + Bu = ROL32((Abo0^Do1), 28); + Ba = ROL32((Aku1^Du0), 13); + Asa0 = Ba ^((~Be)& Bi ); + Age0 = Be ^((~Bi)& Bo ); + Ami0 = Bi ^((~Bo)& Bu ); + Abo0 = Bo ^((~Bu)& Ba ); + Aku1 = Bu ^((~Ba)& Be ); + + Bo = ROL32((Aka0^Da1), 21); + Bu = ROL32((Ase0^De0), 1); + Ba = ROL32((Agi1^Di0), 31); + Be = ROL32((Amo0^Do1), 28); + Bi = ROL32((Abu0^Du1), 20); + Aka0 = Ba ^((~Be)& Bi ); + Ase0 = Be ^((~Bi)& Bo ); + Agi1 = Bi ^((~Bo)& Bu ); + Amo0 = Bo ^((~Bu)& Ba ); + Abu0 = Bu ^((~Ba)& Be ); + + Bo = ROL32((Aka1^Da0), 20); + Bu = ROL32((Ase1^De1), 1); + Ba = ROL32((Agi0^Di1), 31); + Be = ROL32((Amo1^Do0), 27); + Bi = ROL32((Abu1^Du0), 19); + Aka1 = Ba ^((~Be)& Bi ); + Ase1 = Be ^((~Bi)& Bo ); + Agi0 = Bi ^((~Bo)& Bu ); + Amo1 = Bo ^((~Bu)& Ba ); + Abu1 = Bu ^((~Ba)& Be ); + + KeccakAtoD_round3(); + + Ba = (Aba0^Da0); + Be = ROL32((Abe0^De0), 22); + Bi = ROL32((Abi0^Di1), 22); + Bo = ROL32((Abo0^Do1), 11); + Bu = ROL32((Abu0^Du0), 7); + Aba0 = Ba ^((~Be)& Bi ); + Aba0 ^= *(pRoundConstants++); + Abe0 = Be ^((~Bi)& Bo ); + Abi0 = Bi ^((~Bo)& Bu ); + Abo0 = Bo ^((~Bu)& Ba ); + Abu0 = Bu ^((~Ba)& Be ); + + Ba = (Aba1^Da1); + Be = ROL32((Abe1^De1), 22); + Bi = ROL32((Abi1^Di0), 21); + Bo = ROL32((Abo1^Do0), 10); + Bu = ROL32((Abu1^Du1), 7); + Aba1 = Ba ^((~Be)& Bi ); + Aba1 ^= *(pRoundConstants++); + Abe1 = Be ^((~Bi)& Bo ); + Abi1 = Bi ^((~Bo)& Bu ); + Abo1 = Bo ^((~Bu)& Ba ); + Abu1 = Bu ^((~Ba)& Be ); + + Bi = ROL32((Aga0^Da1), 2); + Bo = ROL32((Age0^De1), 23); + Bu = ROL32((Agi0^Di1), 31); + Ba = ROL32((Ago0^Do0), 14); + Be = ROL32((Agu0^Du0), 10); + Aga0 = Ba ^((~Be)& Bi ); + Age0 = Be ^((~Bi)& Bo ); + Agi0 = Bi ^((~Bo)& Bu ); + Ago0 = Bo ^((~Bu)& Ba ); + Agu0 = Bu ^((~Ba)& Be ); + + Bi = ROL32((Aga1^Da0), 1); + Bo = ROL32((Age1^De0), 22); + Bu = ROL32((Agi1^Di0), 30); + Ba = ROL32((Ago1^Do1), 14); + Be = ROL32((Agu1^Du1), 10); + Aga1 = Ba ^((~Be)& Bi ); + Age1 = Be ^((~Bi)& Bo ); + Agi1 = Bi ^((~Bo)& Bu ); + Ago1 = Bo ^((~Bu)& Ba ); + Agu1 = Bu ^((~Ba)& Be ); + + Bu = ROL32((Aka0^Da0), 9); + Ba = ROL32((Ake0^De1), 1); + Be = ROL32((Aki0^Di0), 3); + Bi = ROL32((Ako0^Do1), 13); + Bo = ROL32((Aku0^Du0), 4); + Aka0 = Ba ^((~Be)& Bi ); + Ake0 = Be ^((~Bi)& Bo ); + Aki0 = Bi ^((~Bo)& Bu ); + Ako0 = Bo ^((~Bu)& Ba ); + Aku0 = Bu ^((~Ba)& Be ); + + Bu = ROL32((Aka1^Da1), 9); + Ba = (Ake1^De0); + Be = ROL32((Aki1^Di1), 3); + Bi = ROL32((Ako1^Do0), 12); + Bo = ROL32((Aku1^Du1), 4); + Aka1 = Ba ^((~Be)& Bi ); + Ake1 = Be ^((~Bi)& Bo ); + Aki1 = Bi ^((~Bo)& Bu ); + Ako1 = Bo ^((~Bu)& Ba ); + Aku1 = Bu ^((~Ba)& Be ); + + Be = ROL32((Ama0^Da0), 18); + Bi = ROL32((Ame0^De0), 5); + Bo = ROL32((Ami0^Di1), 8); + Bu = ROL32((Amo0^Do0), 28); + Ba = ROL32((Amu0^Du1), 14); + Ama0 = Ba ^((~Be)& Bi ); + Ame0 = Be ^((~Bi)& Bo ); + Ami0 = Bi ^((~Bo)& Bu ); + Amo0 = Bo ^((~Bu)& Ba ); + Amu0 = Bu ^((~Ba)& Be ); + + Be = ROL32((Ama1^Da1), 18); + Bi = ROL32((Ame1^De1), 5); + Bo = ROL32((Ami1^Di0), 7); + Bu = ROL32((Amo1^Do1), 28); + Ba = ROL32((Amu1^Du0), 13); + Ama1 = Ba ^((~Be)& Bi ); + Ame1 = Be ^((~Bi)& Bo ); + Ami1 = Bi ^((~Bo)& Bu ); + Amo1 = Bo ^((~Bu)& Ba ); + Amu1 = Bu ^((~Ba)& Be ); + + Bo = ROL32((Asa0^Da1), 21); + Bu = ROL32((Ase0^De0), 1); + Ba = ROL32((Asi0^Di0), 31); + Be = ROL32((Aso0^Do1), 28); + Bi = ROL32((Asu0^Du1), 20); + Asa0 = Ba ^((~Be)& Bi ); + Ase0 = Be ^((~Bi)& Bo ); + Asi0 = Bi ^((~Bo)& Bu ); + Aso0 = Bo ^((~Bu)& Ba ); + Asu0 = Bu ^((~Ba)& Be ); + + Bo = ROL32((Asa1^Da0), 20); + Bu = ROL32((Ase1^De1), 1); + Ba = ROL32((Asi1^Di1), 31); + Be = ROL32((Aso1^Do0), 27); + Bi = ROL32((Asu1^Du0), 19); + Asa1 = Ba ^((~Be)& Bi ); + Ase1 = Be ^((~Bi)& Bo ); + Asi1 = Bi ^((~Bo)& Bu ); + Aso1 = Bo ^((~Bu)& Ba ); + Asu1 = Bu ^((~Ba)& Be ); + } + while ( *pRoundConstants != 0xFF ); + + #undef Aba0 + #undef Aba1 + #undef Abe0 + #undef Abe1 + #undef Abi0 + #undef Abi1 + #undef Abo0 + #undef Abo1 + #undef Abu0 + #undef Abu1 + #undef Aga0 + #undef Aga1 + #undef Age0 + #undef Age1 + #undef Agi0 + #undef Agi1 + #undef Ago0 + #undef Ago1 + #undef Agu0 + #undef Agu1 + #undef Aka0 + #undef Aka1 + #undef Ake0 + #undef Ake1 + #undef Aki0 + #undef Aki1 + #undef Ako0 + #undef Ako1 + #undef Aku0 + #undef Aku1 + #undef Ama0 + #undef Ama1 + #undef Ame0 + #undef Ame1 + #undef Ami0 + #undef Ami1 + #undef Amo0 + #undef Amo1 + #undef Amu0 + #undef Amu1 + #undef Asa0 + #undef Asa1 + #undef Ase0 + #undef Ase1 + #undef Asi0 + #undef Asi1 + #undef Aso0 + #undef Aso1 + #undef Asu0 + #undef Asu1 + } +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_Permute_12rounds(void *state) +{ + KeccakP1600_Permute_Nrounds(state, 12); +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_Permute_24rounds(void *state) +{ + KeccakP1600_Permute_Nrounds(state, 24); +} diff --git a/Modules/_sha3/kcp/KeccakP-1600-opt64-config.h b/Modules/_sha3/kcp/KeccakP-1600-opt64-config.h new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakP-1600-opt64-config.h @@ -0,0 +1,3 @@ +#define KeccakP1600_implementation_config "lane complementing, all rounds unrolled" +#define KeccakP1600_fullUnrolling +#define KeccakP1600_useLaneComplementing diff --git a/Modules/_sha3/kcp/KeccakP-1600-opt64.c b/Modules/_sha3/kcp/KeccakP-1600-opt64.c new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakP-1600-opt64.c @@ -0,0 +1,474 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#include +#include +/* #include "brg_endian.h" */ +#include "KeccakP-1600-opt64-config.h" + +#if NOT_PYTHON +typedef unsigned char UINT8; +/* typedef unsigned long long int UINT64; */ +#endif + +#if defined(KeccakP1600_useLaneComplementing) +#define UseBebigokimisa +#endif + +#if defined(_MSC_VER) +#define ROL64(a, offset) _rotl64(a, offset) +#elif defined(KeccakP1600_useSHLD) + #define ROL64(x,N) ({ \ + register UINT64 __out; \ + register UINT64 __in = x; \ + __asm__ ("shld %2,%0,%0" : "=r"(__out) : "0"(__in), "i"(N)); \ + __out; \ + }) +#else +#define ROL64(a, offset) ((((UINT64)a) << offset) ^ (((UINT64)a) >> (64-offset))) +#endif + +#include "KeccakP-1600-64.macros" +#ifdef KeccakP1600_fullUnrolling +#define FullUnrolling +#else +#define Unrolling KeccakP1600_unrolling +#endif +#include "KeccakP-1600-unrolling.macros" +#include "SnP-Relaned.h" + +static const UINT64 KeccakF1600RoundConstants[24] = { + 0x0000000000000001ULL, + 0x0000000000008082ULL, + 0x800000000000808aULL, + 0x8000000080008000ULL, + 0x000000000000808bULL, + 0x0000000080000001ULL, + 0x8000000080008081ULL, + 0x8000000000008009ULL, + 0x000000000000008aULL, + 0x0000000000000088ULL, + 0x0000000080008009ULL, + 0x000000008000000aULL, + 0x000000008000808bULL, + 0x800000000000008bULL, + 0x8000000000008089ULL, + 0x8000000000008003ULL, + 0x8000000000008002ULL, + 0x8000000000000080ULL, + 0x000000000000800aULL, + 0x800000008000000aULL, + 0x8000000080008081ULL, + 0x8000000000008080ULL, + 0x0000000080000001ULL, + 0x8000000080008008ULL }; + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_Initialize(void *state) +{ + memset(state, 0, 200); +#ifdef KeccakP1600_useLaneComplementing + ((UINT64*)state)[ 1] = ~(UINT64)0; + ((UINT64*)state)[ 2] = ~(UINT64)0; + ((UINT64*)state)[ 8] = ~(UINT64)0; + ((UINT64*)state)[12] = ~(UINT64)0; + ((UINT64*)state)[17] = ~(UINT64)0; + ((UINT64*)state)[20] = ~(UINT64)0; +#endif +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_AddBytesInLane(void *state, unsigned int lanePosition, const unsigned char *data, unsigned int offset, unsigned int length) +{ +#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN) + UINT64 lane; + if (length == 0) + return; + if (length == 1) + lane = data[0]; + else { + lane = 0; + memcpy(&lane, data, length); + } + lane <<= offset*8; +#else + UINT64 lane = 0; + unsigned int i; + for(i=0; i>= offset*8; + for(i=0; i>= 8; + } +#endif +} + +/* ---------------------------------------------------------------- */ + +#if (PLATFORM_BYTE_ORDER != IS_LITTLE_ENDIAN) +void fromWordToBytes(UINT8 *bytes, const UINT64 word) +{ + unsigned int i; + + for(i=0; i<(64/8); i++) + bytes[i] = (word >> (8*i)) & 0xFF; +} +#endif + +void KeccakP1600_ExtractLanes(const void *state, unsigned char *data, unsigned int laneCount) +{ +#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN) + memcpy(data, state, laneCount*8); +#else + unsigned int i; + + for(i=0; i 1) { + ((UINT64*)data)[ 1] = ~((UINT64*)data)[ 1]; + if (laneCount > 2) { + ((UINT64*)data)[ 2] = ~((UINT64*)data)[ 2]; + if (laneCount > 8) { + ((UINT64*)data)[ 8] = ~((UINT64*)data)[ 8]; + if (laneCount > 12) { + ((UINT64*)data)[12] = ~((UINT64*)data)[12]; + if (laneCount > 17) { + ((UINT64*)data)[17] = ~((UINT64*)data)[17]; + if (laneCount > 20) { + ((UINT64*)data)[20] = ~((UINT64*)data)[20]; + } + } + } + } + } + } +#endif +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_ExtractBytes(const void *state, unsigned char *data, unsigned int offset, unsigned int length) +{ + SnP_ExtractBytes(state, data, offset, length, KeccakP1600_ExtractLanes, KeccakP1600_ExtractBytesInLane, 8); +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_ExtractAndAddBytesInLane(const void *state, unsigned int lanePosition, const unsigned char *input, unsigned char *output, unsigned int offset, unsigned int length) +{ + UINT64 lane = ((UINT64*)state)[lanePosition]; +#ifdef KeccakP1600_useLaneComplementing + if ((lanePosition == 1) || (lanePosition == 2) || (lanePosition == 8) || (lanePosition == 12) || (lanePosition == 17) || (lanePosition == 20)) + lane = ~lane; +#endif +#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN) + { + unsigned int i; + UINT64 lane1[1]; + lane1[0] = lane; + for(i=0; i>= offset*8; + for(i=0; i>= 8; + } +#endif +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_ExtractAndAddLanes(const void *state, const unsigned char *input, unsigned char *output, unsigned int laneCount) +{ + unsigned int i; +#if (PLATFORM_BYTE_ORDER != IS_LITTLE_ENDIAN) + unsigned char temp[8]; + unsigned int j; +#endif + + for(i=0; i 1) { + ((UINT64*)output)[ 1] = ~((UINT64*)output)[ 1]; + if (laneCount > 2) { + ((UINT64*)output)[ 2] = ~((UINT64*)output)[ 2]; + if (laneCount > 8) { + ((UINT64*)output)[ 8] = ~((UINT64*)output)[ 8]; + if (laneCount > 12) { + ((UINT64*)output)[12] = ~((UINT64*)output)[12]; + if (laneCount > 17) { + ((UINT64*)output)[17] = ~((UINT64*)output)[17]; + if (laneCount > 20) { + ((UINT64*)output)[20] = ~((UINT64*)output)[20]; + } + } + } + } + } + } +#endif +} + +/* ---------------------------------------------------------------- */ + +void KeccakP1600_ExtractAndAddBytes(const void *state, const unsigned char *input, unsigned char *output, unsigned int offset, unsigned int length) +{ + SnP_ExtractAndAddBytes(state, input, output, offset, length, KeccakP1600_ExtractAndAddLanes, KeccakP1600_ExtractAndAddBytesInLane, 8); +} + +/* ---------------------------------------------------------------- */ + +size_t KeccakF1600_FastLoop_Absorb(void *state, unsigned int laneCount, const unsigned char *data, size_t dataByteLen) +{ + size_t originalDataByteLen = dataByteLen; + declareABCDE + #ifndef KeccakP1600_fullUnrolling + unsigned int i; + #endif + UINT64 *stateAsLanes = (UINT64*)state; + UINT64 *inDataAsLanes = (UINT64*)data; + + copyFromState(A, stateAsLanes) + while(dataByteLen >= laneCount*8) { + addInput(A, inDataAsLanes, laneCount) + rounds24 + inDataAsLanes += laneCount; + dataByteLen -= laneCount*8; + } + copyToState(stateAsLanes, A) + return originalDataByteLen - dataByteLen; +} diff --git a/Modules/_sha3/kcp/KeccakP-1600-unrolling.macros b/Modules/_sha3/kcp/KeccakP-1600-unrolling.macros new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakP-1600-unrolling.macros @@ -0,0 +1,185 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#if (defined(FullUnrolling)) +#define rounds24 \ + prepareTheta \ + thetaRhoPiChiIotaPrepareTheta( 0, A, E) \ + thetaRhoPiChiIotaPrepareTheta( 1, E, A) \ + thetaRhoPiChiIotaPrepareTheta( 2, A, E) \ + thetaRhoPiChiIotaPrepareTheta( 3, E, A) \ + thetaRhoPiChiIotaPrepareTheta( 4, A, E) \ + thetaRhoPiChiIotaPrepareTheta( 5, E, A) \ + thetaRhoPiChiIotaPrepareTheta( 6, A, E) \ + thetaRhoPiChiIotaPrepareTheta( 7, E, A) \ + thetaRhoPiChiIotaPrepareTheta( 8, A, E) \ + thetaRhoPiChiIotaPrepareTheta( 9, E, A) \ + thetaRhoPiChiIotaPrepareTheta(10, A, E) \ + thetaRhoPiChiIotaPrepareTheta(11, E, A) \ + thetaRhoPiChiIotaPrepareTheta(12, A, E) \ + thetaRhoPiChiIotaPrepareTheta(13, E, A) \ + thetaRhoPiChiIotaPrepareTheta(14, A, E) \ + thetaRhoPiChiIotaPrepareTheta(15, E, A) \ + thetaRhoPiChiIotaPrepareTheta(16, A, E) \ + thetaRhoPiChiIotaPrepareTheta(17, E, A) \ + thetaRhoPiChiIotaPrepareTheta(18, A, E) \ + thetaRhoPiChiIotaPrepareTheta(19, E, A) \ + thetaRhoPiChiIotaPrepareTheta(20, A, E) \ + thetaRhoPiChiIotaPrepareTheta(21, E, A) \ + thetaRhoPiChiIotaPrepareTheta(22, A, E) \ + thetaRhoPiChiIota(23, E, A) \ + +#define rounds12 \ + prepareTheta \ + thetaRhoPiChiIotaPrepareTheta(12, A, E) \ + thetaRhoPiChiIotaPrepareTheta(13, E, A) \ + thetaRhoPiChiIotaPrepareTheta(14, A, E) \ + thetaRhoPiChiIotaPrepareTheta(15, E, A) \ + thetaRhoPiChiIotaPrepareTheta(16, A, E) \ + thetaRhoPiChiIotaPrepareTheta(17, E, A) \ + thetaRhoPiChiIotaPrepareTheta(18, A, E) \ + thetaRhoPiChiIotaPrepareTheta(19, E, A) \ + thetaRhoPiChiIotaPrepareTheta(20, A, E) \ + thetaRhoPiChiIotaPrepareTheta(21, E, A) \ + thetaRhoPiChiIotaPrepareTheta(22, A, E) \ + thetaRhoPiChiIota(23, E, A) \ + +#elif (Unrolling == 12) +#define rounds24 \ + prepareTheta \ + for(i=0; i<24; i+=12) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+ 1, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+ 2, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+ 3, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+ 4, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+ 5, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+ 6, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+ 7, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+ 8, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+ 9, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+10, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+11, E, A) \ + } \ + +#define rounds12 \ + prepareTheta \ + thetaRhoPiChiIotaPrepareTheta(12, A, E) \ + thetaRhoPiChiIotaPrepareTheta(13, E, A) \ + thetaRhoPiChiIotaPrepareTheta(14, A, E) \ + thetaRhoPiChiIotaPrepareTheta(15, E, A) \ + thetaRhoPiChiIotaPrepareTheta(16, A, E) \ + thetaRhoPiChiIotaPrepareTheta(17, E, A) \ + thetaRhoPiChiIotaPrepareTheta(18, A, E) \ + thetaRhoPiChiIotaPrepareTheta(19, E, A) \ + thetaRhoPiChiIotaPrepareTheta(20, A, E) \ + thetaRhoPiChiIotaPrepareTheta(21, E, A) \ + thetaRhoPiChiIotaPrepareTheta(22, A, E) \ + thetaRhoPiChiIota(23, E, A) \ + +#elif (Unrolling == 6) +#define rounds24 \ + prepareTheta \ + for(i=0; i<24; i+=6) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+1, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+2, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+3, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+4, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+5, E, A) \ + } \ + +#define rounds12 \ + prepareTheta \ + for(i=12; i<24; i+=6) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+1, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+2, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+3, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+4, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+5, E, A) \ + } \ + +#elif (Unrolling == 4) +#define rounds24 \ + prepareTheta \ + for(i=0; i<24; i+=4) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+1, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+2, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+3, E, A) \ + } \ + +#define rounds12 \ + prepareTheta \ + for(i=12; i<24; i+=4) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+1, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+2, A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+3, E, A) \ + } \ + +#elif (Unrolling == 3) +#define rounds24 \ + prepareTheta \ + for(i=0; i<24; i+=3) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+1, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+2, A, E) \ + copyStateVariables(A, E) \ + } \ + +#define rounds12 \ + prepareTheta \ + for(i=12; i<24; i+=3) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+1, E, A) \ + thetaRhoPiChiIotaPrepareTheta(i+2, A, E) \ + copyStateVariables(A, E) \ + } \ + +#elif (Unrolling == 2) +#define rounds24 \ + prepareTheta \ + for(i=0; i<24; i+=2) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+1, E, A) \ + } \ + +#define rounds12 \ + prepareTheta \ + for(i=12; i<24; i+=2) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + thetaRhoPiChiIotaPrepareTheta(i+1, E, A) \ + } \ + +#elif (Unrolling == 1) +#define rounds24 \ + prepareTheta \ + for(i=0; i<24; i++) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + copyStateVariables(A, E) \ + } \ + +#define rounds12 \ + prepareTheta \ + for(i=12; i<24; i++) { \ + thetaRhoPiChiIotaPrepareTheta(i , A, E) \ + copyStateVariables(A, E) \ + } \ + +#else +#error "Unrolling is not correctly specified!" +#endif diff --git a/Modules/_sha3/kcp/KeccakSponge.c b/Modules/_sha3/kcp/KeccakSponge.c new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakSponge.c @@ -0,0 +1,92 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#include "KeccakSponge.h" + +#ifdef KeccakReference + #include "displayIntermediateValues.h" +#endif + +#ifndef KeccakP200_excluded + #include "KeccakP-200-SnP.h" + + #define prefix KeccakWidth200 + #define SnP KeccakP200 + #define SnP_width 200 + #define SnP_Permute KeccakP200_Permute_18rounds + #if defined(KeccakF200_FastLoop_supported) + #define SnP_FastLoop_Absorb KeccakF200_FastLoop_Absorb + #endif + #include "KeccakSponge.inc" + #undef prefix + #undef SnP + #undef SnP_width + #undef SnP_Permute + #undef SnP_FastLoop_Absorb +#endif + +#ifndef KeccakP400_excluded + #include "KeccakP-400-SnP.h" + + #define prefix KeccakWidth400 + #define SnP KeccakP400 + #define SnP_width 400 + #define SnP_Permute KeccakP400_Permute_20rounds + #if defined(KeccakF400_FastLoop_supported) + #define SnP_FastLoop_Absorb KeccakF400_FastLoop_Absorb + #endif + #include "KeccakSponge.inc" + #undef prefix + #undef SnP + #undef SnP_width + #undef SnP_Permute + #undef SnP_FastLoop_Absorb +#endif + +#ifndef KeccakP800_excluded + #include "KeccakP-800-SnP.h" + + #define prefix KeccakWidth800 + #define SnP KeccakP800 + #define SnP_width 800 + #define SnP_Permute KeccakP800_Permute_22rounds + #if defined(KeccakF800_FastLoop_supported) + #define SnP_FastLoop_Absorb KeccakF800_FastLoop_Absorb + #endif + #include "KeccakSponge.inc" + #undef prefix + #undef SnP + #undef SnP_width + #undef SnP_Permute + #undef SnP_FastLoop_Absorb +#endif + +#ifndef KeccakP1600_excluded + #include "KeccakP-1600-SnP.h" + + #define prefix KeccakWidth1600 + #define SnP KeccakP1600 + #define SnP_width 1600 + #define SnP_Permute KeccakP1600_Permute_24rounds + #if defined(KeccakF1600_FastLoop_supported) + #define SnP_FastLoop_Absorb KeccakF1600_FastLoop_Absorb + #endif + #include "KeccakSponge.inc" + #undef prefix + #undef SnP + #undef SnP_width + #undef SnP_Permute + #undef SnP_FastLoop_Absorb +#endif diff --git a/Modules/_sha3/kcp/KeccakSponge.h b/Modules/_sha3/kcp/KeccakSponge.h new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakSponge.h @@ -0,0 +1,172 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#ifndef _KeccakSponge_h_ +#define _KeccakSponge_h_ + +/** General information + * + * The following type and functions are not actually implemented. Their + * documentation is generic, with the prefix Prefix replaced by + * - KeccakWidth200 for a sponge function based on Keccak-f[200] + * - KeccakWidth400 for a sponge function based on Keccak-f[400] + * - KeccakWidth800 for a sponge function based on Keccak-f[800] + * - KeccakWidth1600 for a sponge function based on Keccak-f[1600] + * + * In all these functions, the rate and capacity must sum to the width of the + * chosen permutation. For instance, to use the sponge function + * Keccak[r=1344, c=256], one must use KeccakWidth1600_Sponge() or a combination + * of KeccakWidth1600_SpongeInitialize(), KeccakWidth1600_SpongeAbsorb(), + * KeccakWidth1600_SpongeAbsorbLastFewBits() and + * KeccakWidth1600_SpongeSqueeze(). + * + * The Prefix_SpongeInstance contains the sponge instance attributes for use + * with the Prefix_Sponge* functions. + * It gathers the state processed by the permutation as well as the rate, + * the position of input/output bytes in the state and the phase + * (absorbing or squeezing). + */ + +#ifdef DontReallyInclude_DocumentationOnly +/** Function to evaluate the sponge function Keccak[r, c] in a single call. + * @param rate The value of the rate r. + * @param capacity The value of the capacity c. + * @param input Pointer to the input message (before the suffix). + * @param inputByteLen The length of the input message in bytes. + * @param suffix Byte containing from 0 to 7 suffix bits + * that must be absorbed after @a input. + * These n bits must be in the least significant bit positions. + * These bits must be delimited with a bit 1 at position n + * (counting from 0=LSB to 7=MSB) and followed by bits 0 + * from position n+1 to position 7. + * Some examples: + * - If no bits are to be absorbed, then @a suffix must be 0x01. + * - If the 2-bit sequence 0,0 is to be absorbed, @a suffix must be 0x04. + * - If the 5-bit sequence 0,1,0,0,1 is to be absorbed, @a suffix must be 0x32. + * - If the 7-bit sequence 1,1,0,1,0,0,0 is to be absorbed, @a suffix must be 0x8B. + * . + * @param output Pointer to the output buffer. + * @param outputByteLen The desired number of output bytes. + * @pre One must have r+c equal to the supported width of this implementation + * and the rate a multiple of 8 bits (one byte) in this implementation. + * @pre @a suffix ? 0x00 + * @return Zero if successful, 1 otherwise. + */ +int Prefix_Sponge(unsigned int rate, unsigned int capacity, const unsigned char *input, size_t inputByteLen, unsigned char suffix, unsigned char *output, size_t outputByteLen); + +/** + * Function to initialize the state of the Keccak[r, c] sponge function. + * The phase of the sponge function is set to absorbing. + * @param spongeInstance Pointer to the sponge instance to be initialized. + * @param rate The value of the rate r. + * @param capacity The value of the capacity c. + * @pre One must have r+c equal to the supported width of this implementation + * and the rate a multiple of 8 bits (one byte) in this implementation. + * @return Zero if successful, 1 otherwise. + */ +int Prefix_SpongeInitialize(Prefix_SpongeInstance *spongeInstance, unsigned int rate, unsigned int capacity); + +/** + * Function to give input data bytes for the sponge function to absorb. + * @param spongeInstance Pointer to the sponge instance initialized by Prefix_SpongeInitialize(). + * @param data Pointer to the input data. + * @param dataByteLen The number of input bytes provided in the input data. + * @pre The sponge function must be in the absorbing phase, + * i.e., Prefix_SpongeSqueeze() or Prefix_SpongeAbsorbLastFewBits() + * must not have been called before. + * @return Zero if successful, 1 otherwise. + */ +int Prefix_SpongeAbsorb(Prefix_SpongeInstance *spongeInstance, const unsigned char *data, size_t dataByteLen); + +/** + * Function to give input data bits for the sponge function to absorb + * and then to switch to the squeezing phase. + * @param spongeInstance Pointer to the sponge instance initialized by Prefix_SpongeInitialize(). + * @param delimitedData Byte containing from 0 to 7 trailing bits + * that must be absorbed. + * These n bits must be in the least significant bit positions. + * These bits must be delimited with a bit 1 at position n + * (counting from 0=LSB to 7=MSB) and followed by bits 0 + * from position n+1 to position 7. + * Some examples: + * - If no bits are to be absorbed, then @a delimitedData must be 0x01. + * - If the 2-bit sequence 0,0 is to be absorbed, @a delimitedData must be 0x04. + * - If the 5-bit sequence 0,1,0,0,1 is to be absorbed, @a delimitedData must be 0x32. + * - If the 7-bit sequence 1,1,0,1,0,0,0 is to be absorbed, @a delimitedData must be 0x8B. + * . + * @pre The sponge function must be in the absorbing phase, + * i.e., Prefix_SpongeSqueeze() or Prefix_SpongeAbsorbLastFewBits() + * must not have been called before. + * @pre @a delimitedData ? 0x00 + * @return Zero if successful, 1 otherwise. + */ +int Prefix_SpongeAbsorbLastFewBits(Prefix_SpongeInstance *spongeInstance, unsigned char delimitedData); + +/** + * Function to squeeze output data from the sponge function. + * If the sponge function was in the absorbing phase, this function + * switches it to the squeezing phase + * as if Prefix_SpongeAbsorbLastFewBits(spongeInstance, 0x01) was called. + * @param spongeInstance Pointer to the sponge instance initialized by Prefix_SpongeInitialize(). + * @param data Pointer to the buffer where to store the output data. + * @param dataByteLen The number of output bytes desired. + * @return Zero if successful, 1 otherwise. + */ +int Prefix_SpongeSqueeze(Prefix_SpongeInstance *spongeInstance, unsigned char *data, size_t dataByteLen); +#endif + +#include +#include "align.h" + +#define KCP_DeclareSpongeStructure(prefix, size, alignment) \ + ALIGN(alignment) typedef struct prefix##_SpongeInstanceStruct { \ + unsigned char state[size]; \ + unsigned int rate; \ + unsigned int byteIOIndex; \ + int squeezing; \ + } prefix##_SpongeInstance; + +#define KCP_DeclareSpongeFunctions(prefix) \ + int prefix##_Sponge(unsigned int rate, unsigned int capacity, const unsigned char *input, size_t inputByteLen, unsigned char suffix, unsigned char *output, size_t outputByteLen); \ + int prefix##_SpongeInitialize(prefix##_SpongeInstance *spongeInstance, unsigned int rate, unsigned int capacity); \ + int prefix##_SpongeAbsorb(prefix##_SpongeInstance *spongeInstance, const unsigned char *data, size_t dataByteLen); \ + int prefix##_SpongeAbsorbLastFewBits(prefix##_SpongeInstance *spongeInstance, unsigned char delimitedData); \ + int prefix##_SpongeSqueeze(prefix##_SpongeInstance *spongeInstance, unsigned char *data, size_t dataByteLen); + +#ifndef KeccakP200_excluded + #include "KeccakP-200-SnP.h" + KCP_DeclareSpongeStructure(KeccakWidth200, KeccakP200_stateSizeInBytes, KeccakP200_stateAlignment) + KCP_DeclareSpongeFunctions(KeccakWidth200) +#endif + +#ifndef KeccakP400_excluded + #include "KeccakP-400-SnP.h" + KCP_DeclareSpongeStructure(KeccakWidth400, KeccakP400_stateSizeInBytes, KeccakP400_stateAlignment) + KCP_DeclareSpongeFunctions(KeccakWidth400) +#endif + +#ifndef KeccakP800_excluded + #include "KeccakP-800-SnP.h" + KCP_DeclareSpongeStructure(KeccakWidth800, KeccakP800_stateSizeInBytes, KeccakP800_stateAlignment) + KCP_DeclareSpongeFunctions(KeccakWidth800) +#endif + +#ifndef KeccakP1600_excluded + #include "KeccakP-1600-SnP.h" + KCP_DeclareSpongeStructure(KeccakWidth1600, KeccakP1600_stateSizeInBytes, KeccakP1600_stateAlignment) + KCP_DeclareSpongeFunctions(KeccakWidth1600) +#endif + +#endif diff --git a/Modules/_sha3/kcp/KeccakSponge.inc b/Modules/_sha3/kcp/KeccakSponge.inc new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/KeccakSponge.inc @@ -0,0 +1,332 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#define JOIN0(a, b) a ## b +#define JOIN(a, b) JOIN0(a, b) + +#define Sponge JOIN(prefix, _Sponge) +#define SpongeInstance JOIN(prefix, _SpongeInstance) +#define SpongeInitialize JOIN(prefix, _SpongeInitialize) +#define SpongeAbsorb JOIN(prefix, _SpongeAbsorb) +#define SpongeAbsorbLastFewBits JOIN(prefix, _SpongeAbsorbLastFewBits) +#define SpongeSqueeze JOIN(prefix, _SpongeSqueeze) + +#define SnP_stateSizeInBytes JOIN(SnP, _stateSizeInBytes) +#define SnP_stateAlignment JOIN(SnP, _stateAlignment) +#define SnP_StaticInitialize JOIN(SnP, _StaticInitialize) +#define SnP_Initialize JOIN(SnP, _Initialize) +#define SnP_AddByte JOIN(SnP, _AddByte) +#define SnP_AddBytes JOIN(SnP, _AddBytes) +#define SnP_ExtractBytes JOIN(SnP, _ExtractBytes) + +int Sponge(unsigned int rate, unsigned int capacity, const unsigned char *input, size_t inputByteLen, unsigned char suffix, unsigned char *output, size_t outputByteLen) +{ + ALIGN(SnP_stateAlignment) unsigned char state[SnP_stateSizeInBytes]; + unsigned int partialBlock; + const unsigned char *curInput = input; + unsigned char *curOutput = output; + unsigned int rateInBytes = rate/8; + + if (rate+capacity != SnP_width) + return 1; + if ((rate <= 0) || (rate > SnP_width) || ((rate % 8) != 0)) + return 1; + if (suffix == 0) + return 1; + + /* Initialize the state */ + + SnP_StaticInitialize(); + SnP_Initialize(state); + + /* First, absorb whole blocks */ + +#ifdef SnP_FastLoop_Absorb + if (((rateInBytes % (SnP_width/200)) == 0) && (inputByteLen >= rateInBytes)) { + /* fast lane: whole lane rate */ + + size_t j; + j = SnP_FastLoop_Absorb(state, rateInBytes/(SnP_width/200), curInput, inputByteLen); + curInput += j; + inputByteLen -= j; + } +#endif + while(inputByteLen >= (size_t)rateInBytes) { + #ifdef KeccakReference + displayBytes(1, "Block to be absorbed", curInput, rateInBytes); + #endif + SnP_AddBytes(state, curInput, 0, rateInBytes); + SnP_Permute(state); + curInput += rateInBytes; + inputByteLen -= rateInBytes; + } + + /* Then, absorb what remains */ + + partialBlock = (unsigned int)inputByteLen; + #ifdef KeccakReference + displayBytes(1, "Block to be absorbed (part)", curInput, partialBlock); + #endif + SnP_AddBytes(state, curInput, 0, partialBlock); + + /* Finally, absorb the suffix */ + + #ifdef KeccakReference + { + unsigned char delimitedData1[1]; + delimitedData1[0] = suffix; + displayBytes(1, "Block to be absorbed (last few bits + first bit of padding)", delimitedData1, 1); + } + #endif + /* Last few bits, whose delimiter coincides with first bit of padding */ + + SnP_AddByte(state, suffix, partialBlock); + /* If the first bit of padding is at position rate-1, we need a whole new block for the second bit of padding */ + + if ((suffix >= 0x80) && (partialBlock == (rateInBytes-1))) + SnP_Permute(state); + /* Second bit of padding */ + + SnP_AddByte(state, 0x80, rateInBytes-1); + #ifdef KeccakReference + { + unsigned char block[SnP_width/8]; + memset(block, 0, SnP_width/8); + block[rateInBytes-1] = 0x80; + displayBytes(1, "Second bit of padding", block, rateInBytes); + } + #endif + SnP_Permute(state); + #ifdef KeccakReference + displayText(1, "--- Switching to squeezing phase ---"); + #endif + + /* First, output whole blocks */ + + while(outputByteLen > (size_t)rateInBytes) { + SnP_ExtractBytes(state, curOutput, 0, rateInBytes); + SnP_Permute(state); + #ifdef KeccakReference + displayBytes(1, "Squeezed block", curOutput, rateInBytes); + #endif + curOutput += rateInBytes; + outputByteLen -= rateInBytes; + } + + /* Finally, output what remains */ + + partialBlock = (unsigned int)outputByteLen; + SnP_ExtractBytes(state, curOutput, 0, partialBlock); + #ifdef KeccakReference + displayBytes(1, "Squeezed block (part)", curOutput, partialBlock); + #endif + + return 0; +} + +/* ---------------------------------------------------------------- */ +/* ---------------------------------------------------------------- */ +/* ---------------------------------------------------------------- */ + +int SpongeInitialize(SpongeInstance *instance, unsigned int rate, unsigned int capacity) +{ + if (rate+capacity != SnP_width) + return 1; + if ((rate <= 0) || (rate > SnP_width) || ((rate % 8) != 0)) + return 1; + SnP_StaticInitialize(); + SnP_Initialize(instance->state); + instance->rate = rate; + instance->byteIOIndex = 0; + instance->squeezing = 0; + + return 0; +} + +/* ---------------------------------------------------------------- */ + +int SpongeAbsorb(SpongeInstance *instance, const unsigned char *data, size_t dataByteLen) +{ + size_t i, j; + unsigned int partialBlock; + const unsigned char *curData; + unsigned int rateInBytes = instance->rate/8; + + if (instance->squeezing) + return 1; /* Too late for additional input */ + + + i = 0; + curData = data; + while(i < dataByteLen) { + if ((instance->byteIOIndex == 0) && (dataByteLen >= (i + rateInBytes))) { +#ifdef SnP_FastLoop_Absorb + /* processing full blocks first */ + + if ((rateInBytes % (SnP_width/200)) == 0) { + /* fast lane: whole lane rate */ + + j = SnP_FastLoop_Absorb(instance->state, rateInBytes/(SnP_width/200), curData, dataByteLen - i); + i += j; + curData += j; + } + else { +#endif + for(j=dataByteLen-i; j>=rateInBytes; j-=rateInBytes) { + #ifdef KeccakReference + displayBytes(1, "Block to be absorbed", curData, rateInBytes); + #endif + SnP_AddBytes(instance->state, curData, 0, rateInBytes); + SnP_Permute(instance->state); + curData+=rateInBytes; + } + i = dataByteLen - j; +#ifdef SnP_FastLoop_Absorb + } +#endif + } + else { + /* normal lane: using the message queue */ + + partialBlock = (unsigned int)(dataByteLen - i); + if (partialBlock+instance->byteIOIndex > rateInBytes) + partialBlock = rateInBytes-instance->byteIOIndex; + #ifdef KeccakReference + displayBytes(1, "Block to be absorbed (part)", curData, partialBlock); + #endif + i += partialBlock; + + SnP_AddBytes(instance->state, curData, instance->byteIOIndex, partialBlock); + curData += partialBlock; + instance->byteIOIndex += partialBlock; + if (instance->byteIOIndex == rateInBytes) { + SnP_Permute(instance->state); + instance->byteIOIndex = 0; + } + } + } + return 0; +} + +/* ---------------------------------------------------------------- */ + +int SpongeAbsorbLastFewBits(SpongeInstance *instance, unsigned char delimitedData) +{ + unsigned int rateInBytes = instance->rate/8; + + if (delimitedData == 0) + return 1; + if (instance->squeezing) + return 1; /* Too late for additional input */ + + + #ifdef KeccakReference + { + unsigned char delimitedData1[1]; + delimitedData1[0] = delimitedData; + displayBytes(1, "Block to be absorbed (last few bits + first bit of padding)", delimitedData1, 1); + } + #endif + /* Last few bits, whose delimiter coincides with first bit of padding */ + + SnP_AddByte(instance->state, delimitedData, instance->byteIOIndex); + /* If the first bit of padding is at position rate-1, we need a whole new block for the second bit of padding */ + + if ((delimitedData >= 0x80) && (instance->byteIOIndex == (rateInBytes-1))) + SnP_Permute(instance->state); + /* Second bit of padding */ + + SnP_AddByte(instance->state, 0x80, rateInBytes-1); + #ifdef KeccakReference + { + unsigned char block[SnP_width/8]; + memset(block, 0, SnP_width/8); + block[rateInBytes-1] = 0x80; + displayBytes(1, "Second bit of padding", block, rateInBytes); + } + #endif + SnP_Permute(instance->state); + instance->byteIOIndex = 0; + instance->squeezing = 1; + #ifdef KeccakReference + displayText(1, "--- Switching to squeezing phase ---"); + #endif + return 0; +} + +/* ---------------------------------------------------------------- */ + +int SpongeSqueeze(SpongeInstance *instance, unsigned char *data, size_t dataByteLen) +{ + size_t i, j; + unsigned int partialBlock; + unsigned int rateInBytes = instance->rate/8; + unsigned char *curData; + + if (!instance->squeezing) + SpongeAbsorbLastFewBits(instance, 0x01); + + i = 0; + curData = data; + while(i < dataByteLen) { + if ((instance->byteIOIndex == rateInBytes) && (dataByteLen >= (i + rateInBytes))) { + for(j=dataByteLen-i; j>=rateInBytes; j-=rateInBytes) { + SnP_Permute(instance->state); + SnP_ExtractBytes(instance->state, curData, 0, rateInBytes); + #ifdef KeccakReference + displayBytes(1, "Squeezed block", curData, rateInBytes); + #endif + curData+=rateInBytes; + } + i = dataByteLen - j; + } + else { + /* normal lane: using the message queue */ + + if (instance->byteIOIndex == rateInBytes) { + SnP_Permute(instance->state); + instance->byteIOIndex = 0; + } + partialBlock = (unsigned int)(dataByteLen - i); + if (partialBlock+instance->byteIOIndex > rateInBytes) + partialBlock = rateInBytes-instance->byteIOIndex; + i += partialBlock; + + SnP_ExtractBytes(instance->state, curData, instance->byteIOIndex, partialBlock); + #ifdef KeccakReference + displayBytes(1, "Squeezed block (part)", curData, partialBlock); + #endif + curData += partialBlock; + instance->byteIOIndex += partialBlock; + } + } + return 0; +} + +/* ---------------------------------------------------------------- */ + +#undef Sponge +#undef SpongeInstance +#undef SpongeInitialize +#undef SpongeAbsorb +#undef SpongeAbsorbLastFewBits +#undef SpongeSqueeze +#undef SnP_stateSizeInBytes +#undef SnP_stateAlignment +#undef SnP_StaticInitialize +#undef SnP_Initialize +#undef SnP_AddByte +#undef SnP_AddBytes +#undef SnP_ExtractBytes diff --git a/Modules/_sha3/kcp/PlSnP-Fallback.inc b/Modules/_sha3/kcp/PlSnP-Fallback.inc new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/PlSnP-Fallback.inc @@ -0,0 +1,257 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +/* expect PlSnP_baseParallelism, PlSnP_targetParallelism */ + +/* expect SnP_stateSizeInBytes, SnP_stateAlignment */ + +/* expect prefix */ + +/* expect SnP_* */ + + +#define JOIN0(a, b) a ## b +#define JOIN(a, b) JOIN0(a, b) + +#define PlSnP_StaticInitialize JOIN(prefix, _StaticInitialize) +#define PlSnP_InitializeAll JOIN(prefix, _InitializeAll) +#define PlSnP_AddByte JOIN(prefix, _AddByte) +#define PlSnP_AddBytes JOIN(prefix, _AddBytes) +#define PlSnP_AddLanesAll JOIN(prefix, _AddLanesAll) +#define PlSnP_OverwriteBytes JOIN(prefix, _OverwriteBytes) +#define PlSnP_OverwriteLanesAll JOIN(prefix, _OverwriteLanesAll) +#define PlSnP_OverwriteWithZeroes JOIN(prefix, _OverwriteWithZeroes) +#define PlSnP_ExtractBytes JOIN(prefix, _ExtractBytes) +#define PlSnP_ExtractLanesAll JOIN(prefix, _ExtractLanesAll) +#define PlSnP_ExtractAndAddBytes JOIN(prefix, _ExtractAndAddBytes) +#define PlSnP_ExtractAndAddLanesAll JOIN(prefix, _ExtractAndAddLanesAll) + +#if (PlSnP_baseParallelism == 1) + #define SnP_stateSizeInBytes JOIN(SnP, _stateSizeInBytes) + #define SnP_stateAlignment JOIN(SnP, _stateAlignment) +#else + #define SnP_stateSizeInBytes JOIN(SnP, _statesSizeInBytes) + #define SnP_stateAlignment JOIN(SnP, _statesAlignment) +#endif +#define PlSnP_factor ((PlSnP_targetParallelism)/(PlSnP_baseParallelism)) +#define SnP_stateOffset (((SnP_stateSizeInBytes+(SnP_stateAlignment-1))/SnP_stateAlignment)*SnP_stateAlignment) +#define stateWithIndex(i) ((unsigned char *)states+((i)*SnP_stateOffset)) + +#define SnP_StaticInitialize JOIN(SnP, _StaticInitialize) +#define SnP_Initialize JOIN(SnP, _Initialize) +#define SnP_InitializeAll JOIN(SnP, _InitializeAll) +#define SnP_AddByte JOIN(SnP, _AddByte) +#define SnP_AddBytes JOIN(SnP, _AddBytes) +#define SnP_AddLanesAll JOIN(SnP, _AddLanesAll) +#define SnP_OverwriteBytes JOIN(SnP, _OverwriteBytes) +#define SnP_OverwriteLanesAll JOIN(SnP, _OverwriteLanesAll) +#define SnP_OverwriteWithZeroes JOIN(SnP, _OverwriteWithZeroes) +#define SnP_ExtractBytes JOIN(SnP, _ExtractBytes) +#define SnP_ExtractLanesAll JOIN(SnP, _ExtractLanesAll) +#define SnP_ExtractAndAddBytes JOIN(SnP, _ExtractAndAddBytes) +#define SnP_ExtractAndAddLanesAll JOIN(SnP, _ExtractAndAddLanesAll) + +void PlSnP_StaticInitialize( void ) +{ + SnP_StaticInitialize(); +} + +void PlSnP_InitializeAll(void *states) +{ + unsigned int i; + + for(i=0; i 0) { \ + unsigned int _bytesInLane = SnP_laneLengthInBytes - _offsetInLane; \ + if (_bytesInLane > _sizeLeft) \ + _bytesInLane = _sizeLeft; \ + SnP_AddBytesInLane(state, _lanePosition, _curData, _offsetInLane, _bytesInLane); \ + _sizeLeft -= _bytesInLane; \ + _lanePosition++; \ + _offsetInLane = 0; \ + _curData += _bytesInLane; \ + } \ + } \ + } + +#define SnP_OverwriteBytes(state, data, offset, length, SnP_OverwriteLanes, SnP_OverwriteBytesInLane, SnP_laneLengthInBytes) \ + { \ + if ((offset) == 0) { \ + SnP_OverwriteLanes(state, data, (length)/SnP_laneLengthInBytes); \ + SnP_OverwriteBytesInLane(state, \ + (length)/SnP_laneLengthInBytes, \ + (data)+((length)/SnP_laneLengthInBytes)*SnP_laneLengthInBytes, \ + 0, \ + (length)%SnP_laneLengthInBytes); \ + } \ + else { \ + unsigned int _sizeLeft = (length); \ + unsigned int _lanePosition = (offset)/SnP_laneLengthInBytes; \ + unsigned int _offsetInLane = (offset)%SnP_laneLengthInBytes; \ + const unsigned char *_curData = (data); \ + while(_sizeLeft > 0) { \ + unsigned int _bytesInLane = SnP_laneLengthInBytes - _offsetInLane; \ + if (_bytesInLane > _sizeLeft) \ + _bytesInLane = _sizeLeft; \ + SnP_OverwriteBytesInLane(state, _lanePosition, _curData, _offsetInLane, _bytesInLane); \ + _sizeLeft -= _bytesInLane; \ + _lanePosition++; \ + _offsetInLane = 0; \ + _curData += _bytesInLane; \ + } \ + } \ + } + +#define SnP_ExtractBytes(state, data, offset, length, SnP_ExtractLanes, SnP_ExtractBytesInLane, SnP_laneLengthInBytes) \ + { \ + if ((offset) == 0) { \ + SnP_ExtractLanes(state, data, (length)/SnP_laneLengthInBytes); \ + SnP_ExtractBytesInLane(state, \ + (length)/SnP_laneLengthInBytes, \ + (data)+((length)/SnP_laneLengthInBytes)*SnP_laneLengthInBytes, \ + 0, \ + (length)%SnP_laneLengthInBytes); \ + } \ + else { \ + unsigned int _sizeLeft = (length); \ + unsigned int _lanePosition = (offset)/SnP_laneLengthInBytes; \ + unsigned int _offsetInLane = (offset)%SnP_laneLengthInBytes; \ + unsigned char *_curData = (data); \ + while(_sizeLeft > 0) { \ + unsigned int _bytesInLane = SnP_laneLengthInBytes - _offsetInLane; \ + if (_bytesInLane > _sizeLeft) \ + _bytesInLane = _sizeLeft; \ + SnP_ExtractBytesInLane(state, _lanePosition, _curData, _offsetInLane, _bytesInLane); \ + _sizeLeft -= _bytesInLane; \ + _lanePosition++; \ + _offsetInLane = 0; \ + _curData += _bytesInLane; \ + } \ + } \ + } + +#define SnP_ExtractAndAddBytes(state, input, output, offset, length, SnP_ExtractAndAddLanes, SnP_ExtractAndAddBytesInLane, SnP_laneLengthInBytes) \ + { \ + if ((offset) == 0) { \ + SnP_ExtractAndAddLanes(state, input, output, (length)/SnP_laneLengthInBytes); \ + SnP_ExtractAndAddBytesInLane(state, \ + (length)/SnP_laneLengthInBytes, \ + (input)+((length)/SnP_laneLengthInBytes)*SnP_laneLengthInBytes, \ + (output)+((length)/SnP_laneLengthInBytes)*SnP_laneLengthInBytes, \ + 0, \ + (length)%SnP_laneLengthInBytes); \ + } \ + else { \ + unsigned int _sizeLeft = (length); \ + unsigned int _lanePosition = (offset)/SnP_laneLengthInBytes; \ + unsigned int _offsetInLane = (offset)%SnP_laneLengthInBytes; \ + const unsigned char *_curInput = (input); \ + unsigned char *_curOutput = (output); \ + while(_sizeLeft > 0) { \ + unsigned int _bytesInLane = SnP_laneLengthInBytes - _offsetInLane; \ + if (_bytesInLane > _sizeLeft) \ + _bytesInLane = _sizeLeft; \ + SnP_ExtractAndAddBytesInLane(state, _lanePosition, _curInput, _curOutput, _offsetInLane, _bytesInLane); \ + _sizeLeft -= _bytesInLane; \ + _lanePosition++; \ + _offsetInLane = 0; \ + _curInput += _bytesInLane; \ + _curOutput += _bytesInLane; \ + } \ + } \ + } + +#endif diff --git a/Modules/_sha3/kcp/align.h b/Modules/_sha3/kcp/align.h new file mode 100644 --- /dev/null +++ b/Modules/_sha3/kcp/align.h @@ -0,0 +1,35 @@ +/* +Implementation by the Keccak, Keyak and Ketje Teams, namely, Guido Bertoni, +Joan Daemen, Micha?l Peeters, Gilles Van Assche and Ronny Van Keer, hereby +denoted as "the implementer". + +For more information, feedback or questions, please refer to our websites: +http://keccak.noekeon.org/ +http://keyak.noekeon.org/ +http://ketje.noekeon.org/ + +To the extent possible under law, the implementer has waived all copyright +and related or neighboring rights to the source code in this file. +http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#ifndef _align_h_ +#define _align_h_ + +/* on Mac OS-X and possibly others, ALIGN(x) is defined in param.h, and -Werror chokes on the redef. */ + +#ifdef ALIGN +#undef ALIGN +#endif + +#if defined(__GNUC__) +#define ALIGN(x) __attribute__ ((aligned(x))) +#elif defined(_MSC_VER) +#define ALIGN(x) __declspec(align(x)) +#elif defined(__ARMCC_VERSION) +#define ALIGN(x) __align(x) +#else +#define ALIGN(x) +#endif + +#endif diff --git a/Modules/_sha3/sha3module.c b/Modules/_sha3/sha3module.c new file mode 100644 --- /dev/null +++ b/Modules/_sha3/sha3module.c @@ -0,0 +1,749 @@ +/* SHA3 module + * + * This module provides an interface to the SHA3 algorithm + * + * See below for information about the original code this module was + * based upon. Additional work performed by: + * + * Andrew Kuchling (amk at amk.ca) + * Greg Stein (gstein at lyra.org) + * Trevor Perrin (trevp at trevp.net) + * Gregory P. Smith (greg at krypto.org) + * + * Copyright (C) 2012-2016 Christian Heimes (christian at python.org) + * Licensed to PSF under a Contributor Agreement. + * + */ + +#include "Python.h" +#include "pystrhex.h" +#include "../hashlib.h" + +/* ************************************************************************** + * SHA-3 (Keccak) and SHAKE + * + * The code is based on KeccakCodePackage from 2016-04-23 + * commit 647f93079afc4ada3d23737477a6e52511ca41fd + * + * The reference implementation is altered in this points: + * - C++ comments are converted to ANSI C comments. + * - all function names are mangled + * - typedef for UINT64 is commented out. + * - brg_endian.h is removed + * + * *************************************************************************/ + +#ifdef __sparc + /* opt64 uses un-aligned memory access that causes a BUS error with msg + * 'invalid address alignment' on SPARC. */ + #define KeccakOpt 32 +#elif SIZEOF_VOID_P == 8 && defined(PY_UINT64_T) + /* opt64 works only for 64bit platforms with unsigned int64 */ + #define KeccakOpt 64 +#else + /* opt32 is used for the remaining 32 and 64bit platforms */ + #define KeccakOpt 32 +#endif + +#if KeccakOpt == 64 && defined(PY_UINT64_T) + /* 64bit platforms with unsigned int64 */ + typedef PY_UINT64_T UINT64; + typedef unsigned char UINT8; +#endif + +/* replacement for brg_endian.h */ +#define IS_LITTLE_ENDIAN 1234 +#define IS_BIG_ENDIAN 4321 +#if PY_LITTLE_ENDIAN +#define PLATFORM_BYTE_ORDER IS_LITTLE_ENDIAN +#endif +#if PY_BIG_ENDIAN +#define PLATFORM_BYTE_ORDER IS_BIG_ENDIAN +#endif + +/* mangle names */ +#define KeccakF1600_FastLoop_Absorb _PySHA3_KeccakF1600_FastLoop_Absorb +#define Keccak_HashFinal _PySHA3_Keccak_HashFinal +#define Keccak_HashInitialize _PySHA3_Keccak_HashInitialize +#define Keccak_HashSqueeze _PySHA3_Keccak_HashSqueeze +#define Keccak_HashUpdate _PySHA3_Keccak_HashUpdate +#define KeccakP1600_AddBytes _PySHA3_KeccakP1600_AddBytes +#define KeccakP1600_AddBytesInLane _PySHA3_KeccakP1600_AddBytesInLane +#define KeccakP1600_AddLanes _PySHA3_KeccakP1600_AddLanes +#define KeccakP1600_ExtractAndAddBytes _PySHA3_KeccakP1600_ExtractAndAddBytes +#define KeccakP1600_ExtractAndAddBytesInLane _PySHA3_KeccakP1600_ExtractAndAddBytesInLane +#define KeccakP1600_ExtractAndAddLanes _PySHA3_KeccakP1600_ExtractAndAddLanes +#define KeccakP1600_ExtractBytes _PySHA3_KeccakP1600_ExtractBytes +#define KeccakP1600_ExtractBytesInLane _PySHA3_KeccakP1600_ExtractBytesInLane +#define KeccakP1600_ExtractLanes _PySHA3_KeccakP1600_ExtractLanes +#define KeccakP1600_Initialize _PySHA3_KeccakP1600_Initialize +#define KeccakP1600_OverwriteBytes _PySHA3_KeccakP1600_OverwriteBytes +#define KeccakP1600_OverwriteBytesInLane _PySHA3_KeccakP1600_OverwriteBytesInLane +#define KeccakP1600_OverwriteLanes _PySHA3_KeccakP1600_OverwriteLanes +#define KeccakP1600_OverwriteWithZeroes _PySHA3_KeccakP1600_OverwriteWithZeroes +#define KeccakP1600_Permute_12rounds _PySHA3_KeccakP1600_Permute_12rounds +#define KeccakP1600_Permute_24rounds _PySHA3_KeccakP1600_Permute_24rounds +#define KeccakWidth1600_Sponge _PySHA3_KeccakWidth1600_Sponge +#define KeccakWidth1600_SpongeAbsorb _PySHA3_KeccakWidth1600_SpongeAbsorb +#define KeccakWidth1600_SpongeAbsorbLastFewBits _PySHA3_KeccakWidth1600_SpongeAbsorbLastFewBits +#define KeccakWidth1600_SpongeInitialize _PySHA3_KeccakWidth1600_SpongeInitialize +#define KeccakWidth1600_SpongeSqueeze _PySHA3_KeccakWidth1600_SpongeSqueeze +#if KeccakOpt == 32 +#define KeccakP1600_AddByte _PySHA3_KeccakP1600_AddByte +#define KeccakP1600_Permute_Nrounds _PySHA3_KeccakP1600_Permute_Nrounds +#define KeccakP1600_SetBytesInLaneToZero _PySHA3_KeccakP1600_SetBytesInLaneToZero +#endif + +/* we are only interested in KeccakP1600 */ +#define KeccakP200_excluded 1 +#define KeccakP400_excluded 1 +#define KeccakP800_excluded 1 + +/* inline all Keccak dependencies */ +#include "kcp/KeccakHash.h" +#include "kcp/KeccakSponge.h" +#include "kcp/KeccakHash.c" +#include "kcp/KeccakSponge.c" +#if KeccakOpt == 64 + #include "kcp/KeccakP-1600-opt64.c" +#elif KeccakOpt == 32 + #include "kcp/KeccakP-1600-inplace32BI.c" +#endif + +#define SHA3_MAX_DIGESTSIZE 64 /* 64 Bytes (512 Bits) for 224 to 512 */ +#define SHA3_state Keccak_HashInstance +#define SHA3_init Keccak_HashInitialize +#define SHA3_process Keccak_HashUpdate +#define SHA3_done Keccak_HashFinal +#define SHA3_squeeze Keccak_HashSqueeze +#define SHA3_copystate(dest, src) memcpy(&(dest), &(src), sizeof(SHA3_state)) + + +/*[clinic input] +module _sha3 +class _sha3.sha3_224 "SHA3object *" "&SHA3_224typ" +class _sha3.sha3_256 "SHA3object *" "&SHA3_256typ" +class _sha3.sha3_384 "SHA3object *" "&SHA3_384typ" +class _sha3.sha3_512 "SHA3object *" "&SHA3_512typ" +class _sha3.shake_128 "SHA3object *" "&SHAKE128type" +class _sha3.shake_256 "SHA3object *" "&SHAKE256type" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=b8a53680f370285a]*/ + +/* The structure for storing SHA3 info */ + +#define PY_WITH_KECCAK 0 + +typedef struct { + PyObject_HEAD + SHA3_state hash_state; +#ifdef WITH_THREAD + PyThread_type_lock lock; +#endif +} SHA3object; + +static PyTypeObject SHA3_224type; +static PyTypeObject SHA3_256type; +static PyTypeObject SHA3_384type; +static PyTypeObject SHA3_512type; +#ifdef PY_WITH_KECCAK +static PyTypeObject Keccak_224type; +static PyTypeObject Keccak_256type; +static PyTypeObject Keccak_384type; +static PyTypeObject Keccak_512type; +#endif +static PyTypeObject SHAKE128type; +static PyTypeObject SHAKE256type; + +#include "clinic/sha3module.c.h" + +static SHA3object * +newSHA3object(PyTypeObject *type) +{ + SHA3object *newobj; + newobj = (SHA3object *)PyObject_New(SHA3object, type); + if (newobj == NULL) { + return NULL; + } +#ifdef WITH_THREAD + newobj->lock = NULL; +#endif + return newobj; +} + + +/*[clinic input] + at classmethod +_sha3.sha3_224.__new__ as py_sha3_new + string as data: object = NULL + +Return a new SHA3 hash object with a hashbit length of 28 bytes. +[clinic start generated code]*/ + +static PyObject * +py_sha3_new_impl(PyTypeObject *type, PyObject *data) +/*[clinic end generated code: output=8d5c34279e69bf09 input=d7c582b950a858b6]*/ +{ + SHA3object *self = NULL; + Py_buffer buf = {NULL, NULL}; + HashReturn res; + + self = newSHA3object(type); + if (self == NULL) { + goto error; + } + + if (type == &SHA3_224type) { + res = Keccak_HashInitialize_SHA3_224(&self->hash_state); + } else if (type == &SHA3_256type) { + res = Keccak_HashInitialize_SHA3_256(&self->hash_state); + } else if (type == &SHA3_384type) { + res = Keccak_HashInitialize_SHA3_384(&self->hash_state); + } else if (type == &SHA3_512type) { + res = Keccak_HashInitialize_SHA3_512(&self->hash_state); +#ifdef PY_WITH_KECCAK + } else if (type == &Keccak_224type) { + res = Keccak_HashInitialize(&self->hash_state, 1152, 448, 224, 0x01); + } else if (type == &Keccak_256type) { + res = Keccak_HashInitialize(&self->hash_state, 1088, 512, 256, 0x01); + } else if (type == &Keccak_384type) { + res = Keccak_HashInitialize(&self->hash_state, 832, 768, 384, 0x01); + } else if (type == &Keccak_512type) { + res = Keccak_HashInitialize(&self->hash_state, 576, 1024, 512, 0x01); +#endif + } else if (type == &SHAKE128type) { + res = Keccak_HashInitialize_SHAKE128(&self->hash_state); + } else if (type == &SHAKE256type) { + res = Keccak_HashInitialize_SHAKE256(&self->hash_state); + } else { + PyErr_BadInternalCall(); + goto error; + } + + if (data) { + GET_BUFFER_VIEW_OR_ERROR(data, &buf, goto error); +#ifdef WITH_THREAD + if (buf.len >= HASHLIB_GIL_MINSIZE) { + /* invariant: New objects can't be accessed by other code yet, + * thus it's safe to release the GIL without locking the object. + */ + Py_BEGIN_ALLOW_THREADS + res = SHA3_process(&self->hash_state, buf.buf, buf.len * 8); + Py_END_ALLOW_THREADS + } + else { + res = SHA3_process(&self->hash_state, buf.buf, buf.len * 8); + } +#else + res = SHA3_process(&self->hash_state, buf.buf, buf.len * 8); +#endif + if (res != SUCCESS) { + PyErr_SetString(PyExc_RuntimeError, + "internal error in SHA3 Update()"); + goto error; + } + PyBuffer_Release(&buf); + } + + return (PyObject *)self; + + error: + if (self) { + Py_DECREF(self); + } + if (data && buf.obj) { + PyBuffer_Release(&buf); + } + return NULL; +} + + +/* Internal methods for a hash object */ + +static void +SHA3_dealloc(SHA3object *self) +{ +#ifdef WITH_THREAD + if (self->lock) { + PyThread_free_lock(self->lock); + } +#endif + PyObject_Del(self); +} + + +/* External methods for a hash object */ + + +/*[clinic input] +_sha3.sha3_224.copy + +Return a copy of the hash object. +[clinic start generated code]*/ + +static PyObject * +_sha3_sha3_224_copy_impl(SHA3object *self) +/*[clinic end generated code: output=6c537411ecdcda4c input=93a44aaebea51ba8]*/ +{ + SHA3object *newobj; + + if ((newobj = newSHA3object(Py_TYPE(self))) == NULL) { + return NULL; + } + ENTER_HASHLIB(self); + SHA3_copystate(newobj->hash_state, self->hash_state); + LEAVE_HASHLIB(self); + return (PyObject *)newobj; +} + + +/*[clinic input] +_sha3.sha3_224.digest + +Return the digest value as a string of binary data. +[clinic start generated code]*/ + +static PyObject * +_sha3_sha3_224_digest_impl(SHA3object *self) +/*[clinic end generated code: output=fd531842e20b2d5b input=a5807917d219b30e]*/ +{ + unsigned char digest[SHA3_MAX_DIGESTSIZE]; + SHA3_state temp; + HashReturn res; + + ENTER_HASHLIB(self); + SHA3_copystate(temp, self->hash_state); + LEAVE_HASHLIB(self); + res = SHA3_done(&temp, digest); + if (res != SUCCESS) { + PyErr_SetString(PyExc_RuntimeError, "internal error in SHA3 Final()"); + return NULL; + } + return PyBytes_FromStringAndSize((const char *)digest, + self->hash_state.fixedOutputLength / 8); +} + + +/*[clinic input] +_sha3.sha3_224.hexdigest + +Return the digest value as a string of hexadecimal digits. +[clinic start generated code]*/ + +static PyObject * +_sha3_sha3_224_hexdigest_impl(SHA3object *self) +/*[clinic end generated code: output=75ad03257906918d input=2d91bb6e0d114ee3]*/ +{ + unsigned char digest[SHA3_MAX_DIGESTSIZE]; + SHA3_state temp; + HashReturn res; + + /* Get the raw (binary) digest value */ + ENTER_HASHLIB(self); + SHA3_copystate(temp, self->hash_state); + LEAVE_HASHLIB(self); + res = SHA3_done(&temp, digest); + if (res != SUCCESS) { + PyErr_SetString(PyExc_RuntimeError, "internal error in SHA3 Final()"); + return NULL; + } + return _Py_strhex((const char *)digest, + self->hash_state.fixedOutputLength / 8); +} + + +/*[clinic input] +_sha3.sha3_224.update + + obj: object + / + +Update this hash object's state with the provided string. +[clinic start generated code]*/ + +static PyObject * +_sha3_sha3_224_update(SHA3object *self, PyObject *obj) +/*[clinic end generated code: output=06721d55b483e0af input=be44bf0d1c279791]*/ +{ + Py_buffer buf; + HashReturn res; + + GET_BUFFER_VIEW_OR_ERROUT(obj, &buf); + + /* add new data, the function takes the length in bits not bytes */ +#ifdef WITH_THREAD + if (self->lock == NULL && buf.len >= HASHLIB_GIL_MINSIZE) { + self->lock = PyThread_allocate_lock(); + } + /* Once a lock exists all code paths must be synchronized. We have to + * release the GIL even for small buffers as acquiring the lock may take + * an unlimited amount of time when another thread updates this object + * with lots of data. */ + if (self->lock) { + Py_BEGIN_ALLOW_THREADS + PyThread_acquire_lock(self->lock, 1); + res = SHA3_process(&self->hash_state, buf.buf, buf.len * 8); + PyThread_release_lock(self->lock); + Py_END_ALLOW_THREADS + } + else { + res = SHA3_process(&self->hash_state, buf.buf, buf.len * 8); + } +#else + res = SHA3_process(&self->hash_state, buf.buf, buf.len * 8); +#endif + + if (res != SUCCESS) { + PyBuffer_Release(&buf); + PyErr_SetString(PyExc_RuntimeError, + "internal error in SHA3 Update()"); + return NULL; + } + + PyBuffer_Release(&buf); + Py_INCREF(Py_None); + return Py_None; +} + + +static PyMethodDef SHA3_methods[] = { + _SHA3_SHA3_224_COPY_METHODDEF + _SHA3_SHA3_224_DIGEST_METHODDEF + _SHA3_SHA3_224_HEXDIGEST_METHODDEF + _SHA3_SHA3_224_UPDATE_METHODDEF + {NULL, NULL} /* sentinel */ +}; + + +static PyObject * +SHA3_get_block_size(SHA3object *self, void *closure) +{ + int rate = self->hash_state.sponge.rate; + return PyLong_FromLong(rate / 8); +} + + +static PyObject * +SHA3_get_name(SHA3object *self, void *closure) +{ + PyTypeObject *type = Py_TYPE(self); + if (type == &SHA3_224type) { + return PyUnicode_FromString("sha3_224"); + } else if (type == &SHA3_256type) { + return PyUnicode_FromString("sha3_256"); + } else if (type == &SHA3_384type) { + return PyUnicode_FromString("sha3_384"); + } else if (type == &SHA3_512type) { + return PyUnicode_FromString("sha3_512"); +#ifdef PY_WITH_KECCAK + } else if (type == &Keccak_224type) { + return PyUnicode_FromString("keccak_224"); + } else if (type == &Keccak_256type) { + return PyUnicode_FromString("keccak_256"); + } else if (type == &Keccak_384type) { + return PyUnicode_FromString("keccak_384"); + } else if (type == &Keccak_512type) { + return PyUnicode_FromString("keccak_512"); +#endif + } else if (type == &SHAKE128type) { + return PyUnicode_FromString("shake_128"); + } else if (type == &SHAKE256type) { + return PyUnicode_FromString("shake_256"); + } else { + PyErr_BadInternalCall(); + return NULL; + } +} + + +static PyObject * +SHA3_get_digest_size(SHA3object *self, void *closure) +{ + return PyLong_FromLong(self->hash_state.fixedOutputLength / 8); +} + + +static PyObject * +SHA3_get_capacity_bits(SHA3object *self, void *closure) +{ + int capacity = 1600 - self->hash_state.sponge.rate; + return PyLong_FromLong(capacity); +} + + +static PyObject * +SHA3_get_rate_bits(SHA3object *self, void *closure) +{ + unsigned int rate = self->hash_state.sponge.rate; + return PyLong_FromLong(rate); +} + +static PyObject * +SHA3_get_suffix(SHA3object *self, void *closure) +{ + unsigned char suffix[2]; + suffix[0] = self->hash_state.delimitedSuffix; + suffix[1] = 0; + return PyBytes_FromStringAndSize((const char *)suffix, 1); +} + + +static PyGetSetDef SHA3_getseters[] = { + {"block_size", (getter)SHA3_get_block_size, NULL, NULL, NULL}, + {"name", (getter)SHA3_get_name, NULL, NULL, NULL}, + {"digest_size", (getter)SHA3_get_digest_size, NULL, NULL, NULL}, + {"_capacity_bits", (getter)SHA3_get_capacity_bits, NULL, NULL, NULL}, + {"_rate_bits", (getter)SHA3_get_rate_bits, NULL, NULL, NULL}, + {"_suffix", (getter)SHA3_get_suffix, NULL, NULL, NULL}, + {NULL} /* Sentinel */ +}; + + +#define SHA3_TYPE(type_obj, type_name, type_doc, type_methods) \ + static PyTypeObject type_obj = { \ + PyVarObject_HEAD_INIT(NULL, 0) \ + type_name, /* tp_name */ \ + sizeof(SHA3object), /* tp_size */ \ + 0, /* tp_itemsize */ \ + /* methods */ \ + (destructor)SHA3_dealloc, /* tp_dealloc */ \ + 0, /* tp_print */ \ + 0, /* tp_getattr */ \ + 0, /* tp_setattr */ \ + 0, /* tp_reserved */ \ + 0, /* tp_repr */ \ + 0, /* tp_as_number */ \ + 0, /* tp_as_sequence */ \ + 0, /* tp_as_mapping */ \ + 0, /* tp_hash */ \ + 0, /* tp_call */ \ + 0, /* tp_str */ \ + 0, /* tp_getattro */ \ + 0, /* tp_setattro */ \ + 0, /* tp_as_buffer */ \ + Py_TPFLAGS_DEFAULT, /* tp_flags */ \ + type_doc, /* tp_doc */ \ + 0, /* tp_traverse */ \ + 0, /* tp_clear */ \ + 0, /* tp_richcompare */ \ + 0, /* tp_weaklistoffset */ \ + 0, /* tp_iter */ \ + 0, /* tp_iternext */ \ + type_methods, /* tp_methods */ \ + NULL, /* tp_members */ \ + SHA3_getseters, /* tp_getset */ \ + 0, /* tp_base */ \ + 0, /* tp_dict */ \ + 0, /* tp_descr_get */ \ + 0, /* tp_descr_set */ \ + 0, /* tp_dictoffset */ \ + 0, /* tp_init */ \ + 0, /* tp_alloc */ \ + py_sha3_new, /* tp_new */ \ + } + +PyDoc_STRVAR(sha3_256__doc__, +"sha3_256([string]) -> SHA3 object\n\ +\n\ +Return a new SHA3 hash object with a hashbit length of 32 bytes."); + +PyDoc_STRVAR(sha3_384__doc__, +"sha3_384([string]) -> SHA3 object\n\ +\n\ +Return a new SHA3 hash object with a hashbit length of 48 bytes."); + +PyDoc_STRVAR(sha3_512__doc__, +"sha3_512([string]) -> SHA3 object\n\ +\n\ +Return a new SHA3 hash object with a hashbit length of 64 bytes."); + +SHA3_TYPE(SHA3_224type, "_sha3.sha3_224", py_sha3_new__doc__, SHA3_methods); +SHA3_TYPE(SHA3_256type, "_sha3.sha3_256", sha3_256__doc__, SHA3_methods); +SHA3_TYPE(SHA3_384type, "_sha3.sha3_384", sha3_384__doc__, SHA3_methods); +SHA3_TYPE(SHA3_512type, "_sha3.sha3_512", sha3_512__doc__, SHA3_methods); + +#ifdef PY_WITH_KECCAK +PyDoc_STRVAR(keccak_224__doc__, +"keccak_224([string]) -> Keccak object\n\ +\n\ +Return a new Keccak hash object with a hashbit length of 28 bytes."); + +PyDoc_STRVAR(keccak_256__doc__, +"keccak_256([string]) -> Keccak object\n\ +\n\ +Return a new Keccak hash object with a hashbit length of 32 bytes."); + +PyDoc_STRVAR(keccak_384__doc__, +"keccak_384([string]) -> Keccak object\n\ +\n\ +Return a new Keccak hash object with a hashbit length of 48 bytes."); + +PyDoc_STRVAR(keccak_512__doc__, +"keccak_512([string]) -> Keccak object\n\ +\n\ +Return a new Keccak hash object with a hashbit length of 64 bytes."); + +SHA3_TYPE(Keccak_224type, "_sha3.keccak_224", keccak_224__doc__, SHA3_methods); +SHA3_TYPE(Keccak_256type, "_sha3.keccak_256", keccak_256__doc__, SHA3_methods); +SHA3_TYPE(Keccak_384type, "_sha3.keccak_384", keccak_384__doc__, SHA3_methods); +SHA3_TYPE(Keccak_512type, "_sha3.keccak_512", keccak_512__doc__, SHA3_methods); +#endif + + +static PyObject * +_SHAKE_digest(SHA3object *self, unsigned long digestlen, int hex) +{ + unsigned char *digest = NULL; + SHA3_state temp; + int res; + PyObject *result = NULL; + + if ((digest = (unsigned char*)PyMem_Malloc(digestlen)) == NULL) { + return PyErr_NoMemory(); + } + + /* Get the raw (binary) digest value */ + ENTER_HASHLIB(self); + SHA3_copystate(temp, self->hash_state); + LEAVE_HASHLIB(self); + res = SHA3_done(&temp, NULL); + if (res != SUCCESS) { + PyErr_SetString(PyExc_RuntimeError, "internal error in SHA3 done()"); + goto error; + } + res = SHA3_squeeze(&temp, digest, digestlen * 8); + if (res != SUCCESS) { + PyErr_SetString(PyExc_RuntimeError, "internal error in SHA3 Squeeze()"); + return NULL; + } + if (hex) { + result = _Py_strhex((const char *)digest, digestlen); + } else { + result = PyBytes_FromStringAndSize((const char *)digest, + digestlen); + } + error: + if (digest != NULL) { + PyMem_Free(digest); + } + return result; +} + + +/*[clinic input] +_sha3.shake_128.digest + + length: unsigned_long(bitwise=True) + \ + +Return the digest value as a string of binary data. +[clinic start generated code]*/ + +static PyObject * +_sha3_shake_128_digest_impl(SHA3object *self, unsigned long length) +/*[clinic end generated code: output=2313605e2f87bb8f input=608c8ca80ae9d115]*/ +{ + return _SHAKE_digest(self, length, 0); +} + + +/*[clinic input] +_sha3.shake_128.hexdigest + + length: unsigned_long(bitwise=True) + \ + +Return the digest value as a string of hexadecimal digits. +[clinic start generated code]*/ + +static PyObject * +_sha3_shake_128_hexdigest_impl(SHA3object *self, unsigned long length) +/*[clinic end generated code: output=bf8e2f1e490944a8 input=64e56b4760db4573]*/ +{ + return _SHAKE_digest(self, length, 1); +} + + +static PyMethodDef SHAKE_methods[] = { + _SHA3_SHA3_224_COPY_METHODDEF + _SHA3_SHAKE_128_DIGEST_METHODDEF + _SHA3_SHAKE_128_HEXDIGEST_METHODDEF + _SHA3_SHA3_224_UPDATE_METHODDEF + {NULL, NULL} /* sentinel */ +}; + +PyDoc_STRVAR(shake_128__doc__, +"shake_128([string]) -> SHAKE object\n\ +\n\ +Return a new SHAKE hash object."); + +PyDoc_STRVAR(shake_256__doc__, +"shake_256([string]) -> SHAKE object\n\ +\n\ +Return a new SHAKE hash object."); + +SHA3_TYPE(SHAKE128type, "_sha3.shake_128", shake_128__doc__, SHAKE_methods); +SHA3_TYPE(SHAKE256type, "_sha3.shake_256", shake_256__doc__, SHAKE_methods); + + +/* Initialize this module. */ +static struct PyModuleDef _SHA3module = { + PyModuleDef_HEAD_INIT, + "_sha3", + NULL, + -1, + NULL, + NULL, + NULL, + NULL, + NULL +}; + + +PyMODINIT_FUNC +PyInit__sha3(void) +{ + PyObject *m = NULL; + + m = PyModule_Create(&_SHA3module); + +#define init_sha3type(name, type) \ + do { \ + Py_TYPE(type) = &PyType_Type; \ + if (PyType_Ready(type) < 0) { \ + goto error; \ + } \ + Py_INCREF((PyObject *)type); \ + if (PyModule_AddObject(m, name, (PyObject *)type) < 0) { \ + goto error; \ + } \ + } while(0) + + init_sha3type("sha3_224", &SHA3_224type); + init_sha3type("sha3_256", &SHA3_256type); + init_sha3type("sha3_384", &SHA3_384type); + init_sha3type("sha3_512", &SHA3_512type); +#ifdef PY_WITH_KECCAK + init_sha3type("keccak_224", &Keccak_224type); + init_sha3type("keccak_256", &Keccak_256type); + init_sha3type("keccak_384", &Keccak_384type); + init_sha3type("keccak_512", &Keccak_512type); +#endif + init_sha3type("shake_128", &SHAKE128type); + init_sha3type("shake_256", &SHAKE256type); + +#undef init_sha3type + + if (PyModule_AddIntConstant(m, "keccakopt", KeccakOpt) < 0) { + goto error; + } + if (PyModule_AddStringConstant(m, "implementation", + KeccakP1600_implementation) < 0) { + goto error; + } + + return m; + error: + Py_DECREF(m); + return NULL; +} diff --git a/PC/config.c b/PC/config.c --- a/PC/config.c +++ b/PC/config.c @@ -23,6 +23,7 @@ extern PyObject* PyInit__sha1(void); extern PyObject* PyInit__sha256(void); extern PyObject* PyInit__sha512(void); +extern PyObject* PyInit__sha3(void); extern PyObject* PyInit__blake2(void); extern PyObject* PyInit_time(void); extern PyObject* PyInit__thread(void); @@ -97,6 +98,7 @@ {"_sha1", PyInit__sha1}, {"_sha256", PyInit__sha256}, {"_sha512", PyInit__sha512}, + {"_sha3", PyInit__sha3}, {"_blake2", PyInit__blake2}, {"time", PyInit_time}, #ifdef WITH_THREAD diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -905,6 +905,13 @@ define_macros=blake2_macros, depends=blake2_deps) ) + sha3_deps = glob(os.path.join(os.getcwd(), srcdir, + 'Modules/_sha3/kcp/*')) + sha3_deps.append('hashlib.h') + exts.append( Extension('_sha3', + ['_sha3/sha3module.c'], + depends=sha3_deps)) + # Modules that provide persistent dictionary-like semantics. You will # probably want to arrange for at least one of them to be available on # your machine, though none are defined by default because of library -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 06:43:01 2016 From: python-checkins at python.org (christian.heimes) Date: Wed, 07 Sep 2016 10:43:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2316113=3A_KeccakP-?= =?utf-8?q?1600-opt64_does_not_support_big_endian_platforms_yet=2E?= Message-ID: <20160907104253.3610.27201.76BC0961@psf.io> https://hg.python.org/cpython/rev/4971ca2960c7 changeset: 103218:4971ca2960c7 user: Christian Heimes date: Wed Sep 07 12:42:47 2016 +0200 summary: Issue #16113: KeccakP-1600-opt64 does not support big endian platforms yet. files: Modules/_sha3/sha3module.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Modules/_sha3/sha3module.c b/Modules/_sha3/sha3module.c --- a/Modules/_sha3/sha3module.c +++ b/Modules/_sha3/sha3module.c @@ -37,8 +37,11 @@ /* opt64 uses un-aligned memory access that causes a BUS error with msg * 'invalid address alignment' on SPARC. */ #define KeccakOpt 32 +#elif PY_BIG_ENDIAN + /* opt64 is not yet supported on big endian platforms */ + #define KeccakOpt 32 #elif SIZEOF_VOID_P == 8 && defined(PY_UINT64_T) - /* opt64 works only for 64bit platforms with unsigned int64 */ + /* opt64 works only on little-endian 64bit platforms with unsigned int64 */ #define KeccakOpt 64 #else /* opt32 is used for the remaining 32 and 64bit platforms */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 06:52:07 2016 From: python-checkins at python.org (christian.heimes) Date: Wed, 07 Sep 2016 10:52:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2316113=3A_compile_?= =?utf-8?q?the_module_on_Windows=2C_too=2E?= Message-ID: <20160907105202.78171.60042.E3642597@psf.io> https://hg.python.org/cpython/rev/e8884dcace9f changeset: 103219:e8884dcace9f user: Christian Heimes date: Wed Sep 07 12:51:56 2016 +0200 summary: Issue #16113: compile the module on Windows, too. files: PCbuild/pythoncore.vcxproj | 1 + PCbuild/pythoncore.vcxproj.filters | 3 +++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -228,6 +228,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -491,6 +491,9 @@ Modules + + Modules + Modules -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 07:01:27 2016 From: python-checkins at python.org (christian.heimes) Date: Wed, 07 Sep 2016 11:01:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2316113=3A_take_2_o?= =?utf-8?q?n_big_endian_machines=2E?= Message-ID: <20160907110122.39296.41255.69709C79@psf.io> https://hg.python.org/cpython/rev/68df416e94ba changeset: 103220:68df416e94ba user: Christian Heimes date: Wed Sep 07 13:01:15 2016 +0200 summary: Issue #16113: take 2 on big endian machines. files: Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c | 20 +++++---- 1 files changed, 11 insertions(+), 9 deletions(-) diff --git a/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c b/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c --- a/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c +++ b/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c @@ -188,17 +188,18 @@ unsigned int lanePosition; for(lanePosition=0; lanePosition> 8) & 0xFF; laneAsBytes[2] = (low >> 16) & 0xFF; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 07:18:49 2016 From: python-checkins at python.org (christian.heimes) Date: Wed, 07 Sep 2016 11:18:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2316113=3A_one_more?= =?utf-8?q?_C90_violation_in_big_endian_code=2E?= Message-ID: <20160907111846.78083.87115.143D5EB7@psf.io> https://hg.python.org/cpython/rev/ddc95a9bc2e0 changeset: 103221:ddc95a9bc2e0 user: Christian Heimes date: Wed Sep 07 13:18:40 2016 +0200 summary: Issue #16113: one more C90 violation in big endian code. files: Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c b/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c --- a/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c +++ b/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c @@ -333,8 +333,8 @@ for(lanePosition=0; lanePosition> 8) & 0xFF; laneAsBytes[2] = (low >> 16) & 0xFF; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 11:17:51 2016 From: python-checkins at python.org (ethan.furman) Date: Wed, 07 Sep 2016 15:17:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_add_recipes_for_pseudo-val?= =?utf-8?q?ueless_enums?= Message-ID: <20160907151732.48781.72883.2099A2C7@psf.io> https://hg.python.org/cpython/rev/871bdb06c1cf changeset: 103222:871bdb06c1cf user: Ethan Furman date: Wed Sep 07 08:17:15 2016 -0700 summary: add recipes for pseudo-valueless enums files: Doc/library/enum.rst | 79 +++++++++++++++++++++++++++---- Misc/ACKS | 1 + 2 files changed, 69 insertions(+), 11 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -694,12 +694,66 @@ or as examples for creating one's own. -AutoNumber -^^^^^^^^^^ +Omitting values +^^^^^^^^^^^^^^^ -Avoids having to specify the value for each enumeration member:: +In many use-cases one doesn't care what the actual value of an enumeration +is. There are several ways to define this type of simple enumeration: - >>> class AutoNumber(Enum): +- use instances of :class:`object` as the value +- use a descriptive string as the value +- use a tuple as the value and a custom :meth:`__new__` to replace the + tuple with an :class:`int` value + +Using any of these methods signifies to the user that these values are not +important, and also enables one to add, remove, or reorder members without +having to renumber the remaining members. + +Whichever method you choose, you should provide a :meth:`repr` that also hides +the (unimportant) value:: + + >>> class NoValue(Enum): + ... def __repr__(self): + ... return '<%s.%s>' % (self.__class__.__name__, self.name) + ... + + +Using :class:`object` +""""""""""""""""""""" + +Using :class:`object` would look like:: + + >>> class Color(NoValue): + ... red = object() + ... green = object() + ... blue = object() + ... + >>> Color.green + + + +Using a descriptive string +"""""""""""""""""""""""""" + +Using a string as the value would look like:: + + >>> class Color(NoValue): + ... red = 'stop' + ... green = 'go' + ... blue = 'too fast!' + ... + >>> Color.green + + >>> Color.green.value + 'go' + + +Using a custom :meth:`__new__` +"""""""""""""""""""""""""""""" + +Using an auto-numbering :meth:`__new__` would look like:: + + >>> class AutoNumber(NoValue): ... def __new__(cls): ... value = len(cls.__members__) + 1 ... obj = object.__new__(cls) @@ -711,8 +765,11 @@ ... green = () ... blue = () ... - >>> Color.green.value == 2 - True + >>> Color.green + + >>> Color.green.value + 2 + .. note:: @@ -853,7 +910,7 @@ ^^^^^^^^^^^^ Supported ``__dunder__`` names -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +"""""""""""""""""""""""""""""" :attr:`__members__` is an :class:`OrderedDict` of ``member_name``:``member`` items. It is only available on the class. @@ -864,7 +921,7 @@ Supported ``_sunder_`` names -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +"""""""""""""""""""""""""""" - ``_name_`` -- name of the member - ``_value_`` -- value of the member; can be set / modified in ``__new__`` @@ -896,7 +953,7 @@ order is lost before it can be recorded. ``Enum`` member type -~~~~~~~~~~~~~~~~~~~~ +"""""""""""""""""""" :class:`Enum` members are instances of an :class:`Enum` class, and even though they are accessible as `EnumClass.member`, they should not be accessed @@ -917,7 +974,7 @@ Boolean value of ``Enum`` classes and members -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +""""""""""""""""""""""""""""""""""""""""""""" ``Enum`` members that are mixed with non-Enum types (such as :class:`int`, :class:`str`, etc.) are evaluated according to the mixed-in @@ -932,7 +989,7 @@ ``Enum`` classes with methods -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +""""""""""""""""""""""""""""" If you give your :class:`Enum` subclass extra methods, like the `Planet`_ class above, those methods will show up in a :func:`dir` of the member, diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -549,6 +549,7 @@ Joseph Hackman Barry Haddow Philipp Hagemeister +John Hagen Paul ten Hagen Rasmus Hahn Peter Haight -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 11:47:51 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 15:47:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjc5Njgp?= Message-ID: <20160907154742.13824.48735.AAA5E5FF@psf.io> https://hg.python.org/cpython/rev/f48b00b1ec4f changeset: 103224:f48b00b1ec4f parent: 103222:871bdb06c1cf parent: 103223:234f758449f8 user: Benjamin Peterson date: Wed Sep 07 08:47:18 2016 -0700 summary: merge 3.5 (#27968) files: Objects/genobject.c | 27 +++++++++++++++------------ 1 files changed, 15 insertions(+), 12 deletions(-) diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -21,7 +21,7 @@ _PyGen_Finalize(PyObject *self) { PyGenObject *gen = (PyGenObject *)self; - PyObject *res; + PyObject *res = NULL; PyObject *error_type, *error_value, *error_traceback; if (gen->gi_frame == NULL || gen->gi_frame->f_stacktop == NULL) @@ -33,23 +33,26 @@ /* If `gen` is a coroutine, and if it was never awaited on, issue a RuntimeWarning. */ - if (gen->gi_code != NULL - && ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE - && gen->gi_frame->f_lasti == -1 - && !PyErr_Occurred() - && PyErr_WarnFormat(PyExc_RuntimeWarning, 1, - "coroutine '%.50S' was never awaited", - gen->gi_qualname)) { - res = NULL; /* oops, exception */ + if (gen->gi_code != NULL && + ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE && + gen->gi_frame->f_lasti == -1) { + if (!error_value) { + PyErr_WarnFormat(PyExc_RuntimeWarning, 1, + "coroutine '%.50S' was never awaited", + gen->gi_qualname); + } } else { res = gen_close(gen, NULL); } - if (res == NULL) - PyErr_WriteUnraisable(self); - else + if (res == NULL) { + if (PyErr_Occurred()) + PyErr_WriteUnraisable(self); + } + else { Py_DECREF(res); + } /* Restore the saved exception. */ PyErr_Restore(error_type, error_value, error_traceback); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 11:47:51 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 15:47:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_supress_corout?= =?utf-8?q?ine_warning_when_an_exception_is_pending_=28=2327968=29?= Message-ID: <20160907154742.13824.97647.1274D2C5@psf.io> https://hg.python.org/cpython/rev/234f758449f8 changeset: 103223:234f758449f8 branch: 3.5 parent: 103213:7537ca1c2aaf user: Benjamin Peterson date: Wed Sep 07 08:46:59 2016 -0700 summary: supress coroutine warning when an exception is pending (#27968) files: Objects/genobject.c | 27 +++++++++++++++------------ 1 files changed, 15 insertions(+), 12 deletions(-) diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -21,7 +21,7 @@ _PyGen_Finalize(PyObject *self) { PyGenObject *gen = (PyGenObject *)self; - PyObject *res; + PyObject *res = NULL; PyObject *error_type, *error_value, *error_traceback; if (gen->gi_frame == NULL || gen->gi_frame->f_stacktop == NULL) @@ -33,23 +33,26 @@ /* If `gen` is a coroutine, and if it was never awaited on, issue a RuntimeWarning. */ - if (gen->gi_code != NULL - && ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE - && gen->gi_frame->f_lasti == -1 - && !PyErr_Occurred() - && PyErr_WarnFormat(PyExc_RuntimeWarning, 1, - "coroutine '%.50S' was never awaited", - gen->gi_qualname)) { - res = NULL; /* oops, exception */ + if (gen->gi_code != NULL && + ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE && + gen->gi_frame->f_lasti == -1) { + if (!error_value) { + PyErr_WarnFormat(PyExc_RuntimeWarning, 1, + "coroutine '%.50S' was never awaited", + gen->gi_qualname); + } } else { res = gen_close(gen, NULL); } - if (res == NULL) - PyErr_WriteUnraisable(self); - else + if (res == NULL) { + if (PyErr_Occurred()) + PyErr_WriteUnraisable(self); + } + else { Py_DECREF(res); + } /* Restore the saved exception. */ PyErr_Restore(error_type, error_value, error_traceback); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 11:54:59 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 15:54:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_new_and_exciting_shutdown_?= =?utf-8?q?error_on_windows?= Message-ID: <20160907155457.31408.72074.CE0CD866@psf.io> https://hg.python.org/cpython/rev/1d8545fc0d63 changeset: 103225:1d8545fc0d63 user: Benjamin Peterson date: Wed Sep 07 08:54:35 2016 -0700 summary: new and exciting shutdown error on windows files: Lib/test/test_io.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -3230,7 +3230,8 @@ class CTextIOWrapperTest(TextIOWrapperTest): io = io - shutdown_error = "RuntimeError: could not find io module state" + shutdown_error = ("ImportError: sys.meta_path is None" + if os.name == "nt" else "RuntimeError: could not find io module state") def test_initialization(self): r = self.BytesIO(b"\xc3\xa9\n\n") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 12:01:10 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 16:01:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_3=2E5_=28closes_=2327968=29?= Message-ID: <20160907160104.14547.65675.E55578C1@psf.io> https://hg.python.org/cpython/rev/b43923ce86f0 changeset: 103227:b43923ce86f0 parent: 103225:1d8545fc0d63 parent: 103226:096dfac57e44 user: Benjamin Peterson date: Wed Sep 07 09:00:57 2016 -0700 summary: merge 3.5 (closes #27968) files: Lib/test/test_coroutines.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py --- a/Lib/test/test_coroutines.py +++ b/Lib/test/test_coroutines.py @@ -1568,10 +1568,11 @@ def test_fatal_coro_warning(self): # Issue 27811 async def func(): pass - with warnings.catch_warnings(): + with warnings.catch_warnings(), support.captured_stderr() as stderr: warnings.filterwarnings("error") func() support.gc_collect() + self.assertIn("was never awaited", stderr.getvalue()) class CoroAsyncIOCompatTest(unittest.TestCase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 12:01:10 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 16:01:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_capture_stderr?= =?utf-8?q?_to_silence_output_during_test=5Fcoroutines_=28closes_=2327968?= =?utf-8?q?=29?= Message-ID: <20160907160104.1306.20519.68D28C3A@psf.io> https://hg.python.org/cpython/rev/096dfac57e44 changeset: 103226:096dfac57e44 branch: 3.5 parent: 103223:234f758449f8 user: Benjamin Peterson date: Wed Sep 07 09:00:48 2016 -0700 summary: capture stderr to silence output during test_coroutines (closes #27968) files: Lib/test/test_coroutines.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py --- a/Lib/test/test_coroutines.py +++ b/Lib/test/test_coroutines.py @@ -1568,10 +1568,11 @@ def test_fatal_coro_warning(self): # Issue 27811 async def func(): pass - with warnings.catch_warnings(): + with warnings.catch_warnings(), support.captured_stderr() as stderr: warnings.filterwarnings("error") func() support.gc_collect() + self.assertIn("was never awaited", stderr.getvalue()) class CoroAsyncIOCompatTest(unittest.TestCase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 12:28:06 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 16:28:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_replace_PY=5FSIZE=5FMAX_wi?= =?utf-8?q?th_SIZE=5FMAX?= Message-ID: <20160907162800.1522.60701.344C9CBF@psf.io> https://hg.python.org/cpython/rev/f6412378821c changeset: 103228:f6412378821c user: Benjamin Peterson date: Wed Sep 07 09:26:18 2016 -0700 summary: replace PY_SIZE_MAX with SIZE_MAX files: Include/pyport.h | 10 +--------- Modules/_elementtree.c | 2 +- Modules/_tkinter.c | 2 +- Modules/_tracemalloc.c | 4 ++-- Modules/audioop.c | 2 +- Objects/listobject.c | 4 ++-- Objects/longobject.c | 2 +- Objects/obmalloc.c | 2 +- Parser/node.c | 2 +- Python/asdl.c | 8 ++++---- Python/ast.c | 2 +- Python/compile.c | 4 ++-- 12 files changed, 18 insertions(+), 26 deletions(-) diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -115,16 +115,8 @@ typedef int Py_ssize_clean_t; #endif -/* Largest possible value of size_t. - SIZE_MAX is part of C99, so it might be defined on some - platforms. If it is not defined, (size_t)-1 is a portable - definition for C89, due to the way signed->unsigned - conversion is defined. */ -#ifdef SIZE_MAX +/* Largest possible value of size_t. */ #define PY_SIZE_MAX SIZE_MAX -#else -#define PY_SIZE_MAX ((size_t)-1) -#endif /* Largest positive value of type Py_ssize_t. */ #define PY_SSIZE_T_MAX ((Py_ssize_t)(((size_t)-1)>>1)) diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -1787,7 +1787,7 @@ step = -step; } - assert((size_t)slicelen <= PY_SIZE_MAX / sizeof(PyObject *)); + assert((size_t)slicelen <= SIZE_MAX / sizeof(PyObject *)); /* recycle is a list that will contain all the children * scheduled for removal. diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -976,7 +976,7 @@ }; -#if PY_SIZE_MAX > INT_MAX +#if SIZE_MAX > INT_MAX #define CHECK_STRING_LENGTH(s) do { \ if (s != NULL && strlen(s) >= INT_MAX) { \ PyErr_SetString(PyExc_OverflowError, "string is too long"); \ diff --git a/Modules/_tracemalloc.c b/Modules/_tracemalloc.c --- a/Modules/_tracemalloc.c +++ b/Modules/_tracemalloc.c @@ -655,7 +655,7 @@ } } - assert(tracemalloc_traced_memory <= PY_SIZE_MAX - size); + assert(tracemalloc_traced_memory <= SIZE_MAX - size); tracemalloc_traced_memory += size; if (tracemalloc_traced_memory > tracemalloc_peak_traced_memory) tracemalloc_peak_traced_memory = tracemalloc_traced_memory; @@ -672,7 +672,7 @@ PyMemAllocatorEx *alloc = (PyMemAllocatorEx *)ctx; void *ptr; - assert(elsize == 0 || nelem <= PY_SIZE_MAX / elsize); + assert(elsize == 0 || nelem <= SIZE_MAX / elsize); if (use_calloc) ptr = alloc->calloc(alloc->ctx, nelem, elsize); diff --git a/Modules/audioop.c b/Modules/audioop.c --- a/Modules/audioop.c +++ b/Modules/audioop.c @@ -1337,7 +1337,7 @@ weightA /= d; weightB /= d; - if ((size_t)nchannels > PY_SIZE_MAX/sizeof(int)) { + if ((size_t)nchannels > SIZE_MAX/sizeof(int)) { PyErr_SetString(PyExc_MemoryError, "not enough memory for output buffer"); return NULL; diff --git a/Objects/listobject.c b/Objects/listobject.c --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -49,7 +49,7 @@ new_allocated = (newsize >> 3) + (newsize < 9 ? 3 : 6); /* check for integer overflow */ - if (new_allocated > PY_SIZE_MAX - newsize) { + if (new_allocated > SIZE_MAX - newsize) { PyErr_NoMemory(); return -1; } else { @@ -59,7 +59,7 @@ if (newsize == 0) new_allocated = 0; items = self->ob_item; - if (new_allocated <= (PY_SIZE_MAX / sizeof(PyObject *))) + if (new_allocated <= (SIZE_MAX / sizeof(PyObject *))) PyMem_RESIZE(items, PyObject *, new_allocated); else items = NULL; diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -721,7 +721,7 @@ assert(ndigits == 0 || v->ob_digit[ndigits - 1] != 0); if (ndigits > 0) { digit msd = v->ob_digit[ndigits - 1]; - if ((size_t)(ndigits - 1) > PY_SIZE_MAX / (size_t)PyLong_SHIFT) + if ((size_t)(ndigits - 1) > SIZE_MAX / (size_t)PyLong_SHIFT) goto Overflow; result = (size_t)(ndigits - 1) * (size_t)PyLong_SHIFT; do { diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -1057,7 +1057,7 @@ if (numarenas <= maxarenas) return NULL; /* overflow */ #if SIZEOF_SIZE_T <= SIZEOF_INT - if (numarenas > PY_SIZE_MAX / sizeof(*arenas)) + if (numarenas > SIZE_MAX / sizeof(*arenas)) return NULL; /* overflow */ #endif nbytes = numarenas * sizeof(*arenas); diff --git a/Parser/node.c b/Parser/node.c --- a/Parser/node.c +++ b/Parser/node.c @@ -91,7 +91,7 @@ if (current_capacity < 0 || required_capacity < 0) return E_OVERFLOW; if (current_capacity < required_capacity) { - if ((size_t)required_capacity > PY_SIZE_MAX / sizeof(node)) { + if ((size_t)required_capacity > SIZE_MAX / sizeof(node)) { return E_NOMEM; } n = n1->n_child; diff --git a/Python/asdl.c b/Python/asdl.c --- a/Python/asdl.c +++ b/Python/asdl.c @@ -9,14 +9,14 @@ /* check size is sane */ if (size < 0 || - (size && (((size_t)size - 1) > (PY_SIZE_MAX / sizeof(void *))))) { + (size && (((size_t)size - 1) > (SIZE_MAX / sizeof(void *))))) { PyErr_NoMemory(); return NULL; } n = (size ? (sizeof(void *) * (size - 1)) : 0); /* check if size can be added safely */ - if (n > PY_SIZE_MAX - sizeof(asdl_seq)) { + if (n > SIZE_MAX - sizeof(asdl_seq)) { PyErr_NoMemory(); return NULL; } @@ -40,14 +40,14 @@ /* check size is sane */ if (size < 0 || - (size && (((size_t)size - 1) > (PY_SIZE_MAX / sizeof(void *))))) { + (size && (((size_t)size - 1) > (SIZE_MAX / sizeof(void *))))) { PyErr_NoMemory(); return NULL; } n = (size ? (sizeof(void *) * (size - 1)) : 0); /* check if size can be added safely */ - if (n > PY_SIZE_MAX - sizeof(asdl_seq)) { + if (n > SIZE_MAX - sizeof(asdl_seq)) { PyErr_NoMemory(); return NULL; } diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -3985,7 +3985,7 @@ const char *end; /* check for integer overflow */ - if (len > PY_SIZE_MAX / 6) + if (len > SIZE_MAX / 6) return NULL; /* "?" (2 bytes) may become "\U000000E4" (10 bytes), or 1:5 "\?" (3 bytes) may become "\u005c\U000000E4" (16 bytes), or ~1:6 */ diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -804,7 +804,7 @@ oldsize = b->b_ialloc * sizeof(struct instr); newsize = oldsize << 1; - if (oldsize > (PY_SIZE_MAX >> 1)) { + if (oldsize > (SIZE_MAX >> 1)) { PyErr_NoMemory(); return -1; } @@ -4520,7 +4520,7 @@ a->a_lnotab = PyBytes_FromStringAndSize(NULL, DEFAULT_LNOTAB_SIZE); if (!a->a_lnotab) return 0; - if ((size_t)nblocks > PY_SIZE_MAX / sizeof(basicblock *)) { + if ((size_t)nblocks > SIZE_MAX / sizeof(basicblock *)) { PyErr_NoMemory(); return 0; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 12:32:17 2016 From: python-checkins at python.org (steve.dower) Date: Wed, 07 Sep 2016 16:32:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327959=3A_Prevent_?= =?utf-8?q?ImportError_from_escaping_codec_search_function?= Message-ID: <20160907163202.13723.85187.1F5450C4@psf.io> https://hg.python.org/cpython/rev/47b4dbd451f5 changeset: 103229:47b4dbd451f5 user: Steve Dower date: Wed Sep 07 09:31:52 2016 -0700 summary: Issue #27959: Prevent ImportError from escaping codec search function files: Lib/encodings/__init__.py | 12 ++++++++---- Lib/test/test_io.py | 3 +-- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py --- a/Lib/encodings/__init__.py +++ b/Lib/encodings/__init__.py @@ -155,9 +155,13 @@ if sys.platform == 'win32': def _alias_mbcs(encoding): - import _bootlocale - if encoding == _bootlocale.getpreferredencoding(False): - import encodings.mbcs - return encodings.mbcs.getregentry() + try: + import _bootlocale + if encoding == _bootlocale.getpreferredencoding(False): + import encodings.mbcs + return encodings.mbcs.getregentry() + except ImportError: + # Imports may fail while we are shutting down + pass codecs.register(_alias_mbcs) diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -3230,8 +3230,7 @@ class CTextIOWrapperTest(TextIOWrapperTest): io = io - shutdown_error = ("ImportError: sys.meta_path is None" - if os.name == "nt" else "RuntimeError: could not find io module state") + shutdown_error = "RuntimeError: could not find io module state" def test_initialization(self): r = self.BytesIO(b"\xc3\xa9\n\n") -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Wed Sep 7 13:20:13 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 7 Sep 2016 18:20:13 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python 2.7 2016-09-07 Message-ID: <23b37340-c64b-4a68-bc0e-274bab3dd3fc@irsmsx153.ger.corp.intel.com> Results for project Python 2.7, build date 2016-09-07 02:47:12 +0000 commit: 75dae0b2ccb3 previous commit: 65102408021e revision date: 2016-09-07 01:05:59 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.24% -1.79% 2.75% 5.31% :-) pybench 0.19% -0.66% 5.67% 4.93% :-( regex_v8 0.58% 0.08% -2.16% 10.48% :-) nbody 0.06% -1.40% 6.98% 4.22% :-| json_dump_v2 0.29% 1.09% 1.62% 10.39% :-| normal_startup 0.46% -0.22% -0.97% 2.06% :-) ssbench 0.15% -0.33% 2.58% 1.04% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-2-7-2016-09-07/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Wed Sep 7 13:21:16 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Wed, 7 Sep 2016 18:21:16 +0100 Subject: [Python-checkins] UGLY Benchmark Results for Python Default 2016-09-07 Message-ID: <3813f6d5-61e7-4d85-8bed-3ff2c6b330bf@irsmsx153.ger.corp.intel.com> Results for project Python default, build date 2016-09-07 02:00:49 +0000 commit: 4aa6233961e5 previous commit: ebaeeb36eb4b revision date: 2016-09-07 01:03:02 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.29% -0.72% 10.35% 17.46% :-) pybench 0.22% -0.18% 6.70% 7.30% :-( regex_v8 2.63% 0.05% -3.57% 4.26% :-) nbody 0.14% 1.12% 2.28% 5.16% :-( json_dump_v2 0.43% -1.63% -2.69% 13.74% :-| normal_startup 1.08% -0.54% -1.64% 6.33% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-09-07/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Wed Sep 7 13:33:53 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 17:33:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_make_=5FPy=5Fstatic=5Fstri?= =?utf-8?q?ng=5Finit_use_a_designated_initializer?= Message-ID: <20160907173353.78547.37776.9D372EA9@psf.io> https://hg.python.org/cpython/rev/1b1abe815db0 changeset: 103230:1b1abe815db0 user: Benjamin Peterson date: Wed Sep 07 10:33:28 2016 -0700 summary: make _Py_static_string_init use a designated initializer files: Include/object.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -144,7 +144,7 @@ PyObject *object; } _Py_Identifier; -#define _Py_static_string_init(value) { 0, value, 0 } +#define _Py_static_string_init(value) { .next = NULL, .string = value, .object = NULL } #define _Py_static_string(varname, value) static _Py_Identifier varname = _Py_static_string_init(value) #define _Py_IDENTIFIER(varname) _Py_static_string(PyId_##varname, #varname) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 13:41:27 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 07 Sep 2016 17:41:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_27988=3A_Make_sure_iter=5Fattachments_does_not_?= =?utf-8?q?mutate_the_payload_list=2E?= Message-ID: <20160907174126.22532.14117.55214741@psf.io> https://hg.python.org/cpython/rev/69da5242aae3 changeset: 103232:69da5242aae3 parent: 103230:1b1abe815db0 parent: 103231:3bf2f6818719 user: R David Murray date: Wed Sep 07 13:41:06 2016 -0400 summary: Merge: 27988: Make sure iter_attachments does not mutate the payload list. files: Lib/email/message.py | 2 +- Lib/test/test_email/test_message.py | 10 ++++++++++ Misc/NEWS | 2 ++ 3 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Lib/email/message.py b/Lib/email/message.py --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -1022,7 +1022,7 @@ maintype, subtype = self.get_content_type().split('/') if maintype != 'multipart' or subtype == 'alternative': return - parts = self.get_payload() + parts = self.get_payload().copy() if maintype == 'multipart' and subtype == 'related': # For related, we treat everything but the root as an attachment. # The root may be indicated by 'start'; if there's no start or we diff --git a/Lib/test/test_email/test_message.py b/Lib/test/test_email/test_message.py --- a/Lib/test/test_email/test_message.py +++ b/Lib/test/test_email/test_message.py @@ -732,6 +732,16 @@ m.set_param('filename', 'abc.png', 'Content-Disposition') self.assertTrue(m.is_attachment()) + def test_iter_attachments_mutation(self): + # We had a bug where iter_attachments was mutating the list. + m = self._make_message() + m.set_content('arbitrary text as main part') + m.add_related('more text as a related part') + m.add_related('yet more text as a second "attachment"') + orig = m.get_payload().copy() + self.assertEqual(len(list(m.iter_attachments())), 2) + self.assertEqual(m.get_payload(), orig) + class TestEmailMessage(TestEmailMessageBase, TestEmailBase): message = EmailMessage diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -91,6 +91,8 @@ Library ------- +- Issue 27988: Fix email iter_attachments incorrect mutation of payload list. + - Issue #16113: Add SHA-3 and SHAKE support to hashlib module. - Issue #27776: The :func:`os.urandom` function does now block on Linux 3.17 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 13:41:40 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 07 Sep 2016 17:41:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogMjc5ODg6IE1ha2Ug?= =?utf-8?q?sure_iter=5Fattachments_does_not_mutate_the_payload_list=2E?= Message-ID: <20160907174126.13824.48118.AD25A54A@psf.io> https://hg.python.org/cpython/rev/3bf2f6818719 changeset: 103231:3bf2f6818719 branch: 3.5 parent: 103226:096dfac57e44 user: R David Murray date: Wed Sep 07 13:39:36 2016 -0400 summary: 27988: Make sure iter_attachments does not mutate the payload list. files: Lib/email/message.py | 2 +- Lib/test/test_email/test_message.py | 10 ++++++++++ Misc/NEWS | 2 ++ 3 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Lib/email/message.py b/Lib/email/message.py --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -1022,7 +1022,7 @@ maintype, subtype = self.get_content_type().split('/') if maintype != 'multipart' or subtype == 'alternative': return - parts = self.get_payload() + parts = self.get_payload().copy() if maintype == 'multipart' and subtype == 'related': # For related, we treat everything but the root as an attachment. # The root may be indicated by 'start'; if there's no start or we diff --git a/Lib/test/test_email/test_message.py b/Lib/test/test_email/test_message.py --- a/Lib/test/test_email/test_message.py +++ b/Lib/test/test_email/test_message.py @@ -732,6 +732,16 @@ m.set_param('filename', 'abc.png', 'Content-Disposition') self.assertTrue(m.is_attachment()) + def test_iter_attachments_mutation(self): + # We had a bug where iter_attachments was mutating the list. + m = self._make_message() + m.set_content('arbitrary text as main part') + m.add_related('more text as a related part') + m.add_related('yet more text as a second "attachment"') + orig = m.get_payload().copy() + self.assertEqual(len(list(m.iter_attachments())), 2) + self.assertEqual(m.get_payload(), orig) + class TestEmailMessage(TestEmailMessageBase, TestEmailBase): message = EmailMessage diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -62,6 +62,8 @@ Library ------- +- Issue 27988: Fix email iter_attachments incorrect mutation of payload list. + - Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name fields in X.509 certs. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 14:02:21 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 07 Sep 2016 18:02:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2326209=3A_Clarify_type_of_*localaddr*/*remote?= =?utf-8?q?adr*_in_smtpd_docs=2E?= Message-ID: <20160907180220.12752.55594.59518F13@psf.io> https://hg.python.org/cpython/rev/fe2ca2216334 changeset: 103234:fe2ca2216334 parent: 103232:69da5242aae3 parent: 103233:1ed37f6e91bb user: R David Murray date: Wed Sep 07 14:02:11 2016 -0400 summary: Merge: #26209: Clarify type of *localaddr*/*remoteadr* in smtpd docs. files: Doc/library/smtpd.rst | 7 ++++--- Doc/library/socket.rst | 2 ++ 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Doc/library/smtpd.rst b/Doc/library/smtpd.rst --- a/Doc/library/smtpd.rst +++ b/Doc/library/smtpd.rst @@ -32,9 +32,10 @@ map=None, enable_SMTPUTF8=False, decode_data=False) Create a new :class:`SMTPServer` object, which binds to local address - *localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. It - inherits from :class:`asyncore.dispatcher`, and so will insert itself into - :mod:`asyncore`'s event loop on instantiation. + *localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. Both + *localaddr* and *remoteaddr* should be a :ref:`(host, port) ` + tuple. The object inherits from :class:`asyncore.dispatcher`, and so will + insert itself into :mod:`asyncore`'s event loop on instantiation. *data_size_limit* specifies the maximum number of bytes that will be accepted in a ``DATA`` command. A value of ``None`` or ``0`` means no diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -63,6 +63,8 @@ .. versionchanged:: 3.5 Writable :term:`bytes-like object` is now accepted. +.. _host_port: + - A pair ``(host, port)`` is used for the :const:`AF_INET` address family, where *host* is a string representing either a hostname in Internet domain notation like ``'daring.cwi.nl'`` or an IPv4 address like ``'100.50.200.5'``, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 14:02:26 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 07 Sep 2016 18:02:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI2MjA5OiBDbGFy?= =?utf-8?q?ify_type_of_*localaddr*/*remoteadr*_in_smtpd_docs=2E?= Message-ID: <20160907180220.3670.94417.62F9D751@psf.io> https://hg.python.org/cpython/rev/1ed37f6e91bb changeset: 103233:1ed37f6e91bb branch: 3.5 parent: 103231:3bf2f6818719 user: R David Murray date: Wed Sep 07 14:01:23 2016 -0400 summary: #26209: Clarify type of *localaddr*/*remoteadr* in smtpd docs. files: Doc/library/smtpd.rst | 7 ++++--- Doc/library/socket.rst | 2 ++ 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Doc/library/smtpd.rst b/Doc/library/smtpd.rst --- a/Doc/library/smtpd.rst +++ b/Doc/library/smtpd.rst @@ -32,9 +32,10 @@ map=None, enable_SMTPUTF8=False, decode_data=True) Create a new :class:`SMTPServer` object, which binds to local address - *localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. It - inherits from :class:`asyncore.dispatcher`, and so will insert itself into - :mod:`asyncore`'s event loop on instantiation. + *localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. Both + *localaddr* and *remoteaddr* should be a :ref:`(host, port) ` + tuple. The object inherits from :class:`asyncore.dispatcher`, and so will + insert itself into :mod:`asyncore`'s event loop on instantiation. *data_size_limit* specifies the maximum number of bytes that will be accepted in a ``DATA`` command. A value of ``None`` or ``0`` means no diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -63,6 +63,8 @@ .. versionchanged:: 3.5 Writable :term:`bytes-like object` is now accepted. +.. _host_port: + - A pair ``(host, port)`` is used for the :const:`AF_INET` address family, where *host* is a string representing either a hostname in Internet domain notation like ``'daring.cwi.nl'`` or an IPv4 address like ``'100.50.200.5'``, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 14:06:57 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 18:06:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_require_C99_bool?= Message-ID: <20160907180657.78083.66923.95117770@psf.io> https://hg.python.org/cpython/rev/7f4c9cfae20c changeset: 103235:7f4c9cfae20c user: Benjamin Peterson date: Wed Sep 07 11:06:17 2016 -0700 summary: require C99 bool files: Modules/_ctypes/cfield.c | 14 ++------- Modules/_struct.c | 17 +++-------- Objects/memoryobject.c | 20 ------------- configure | 42 ++++++++------------------- configure.ac | 10 ------ pyconfig.h.in | 3 -- 6 files changed, 21 insertions(+), 85 deletions(-) diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c --- a/Modules/_ctypes/cfield.c +++ b/Modules/_ctypes/cfield.c @@ -711,14 +711,6 @@ } #endif -#ifdef HAVE_C99_BOOL -#define BOOL_TYPE _Bool -#else -#define BOOL_TYPE char -#undef SIZEOF__BOOL -#define SIZEOF__BOOL 1 -#endif - static PyObject * bool_set(void *ptr, PyObject *value, Py_ssize_t size) { @@ -726,10 +718,10 @@ case -1: return NULL; case 0: - *(BOOL_TYPE *)ptr = 0; + *(_Bool *)ptr = 0; _RET(value); default: - *(BOOL_TYPE *)ptr = 1; + *(_Bool *)ptr = 1; _RET(value); } } @@ -737,7 +729,7 @@ static PyObject * bool_get(void *ptr, Py_ssize_t size) { - return PyBool_FromLong((long)*(BOOL_TYPE *)ptr); + return PyBool_FromLong((long)*(_Bool *)ptr); } static PyObject * diff --git a/Modules/_struct.c b/Modules/_struct.c --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -60,6 +60,7 @@ typedef struct { char c; double x; } st_double; typedef struct { char c; void *x; } st_void_p; typedef struct { char c; size_t x; } st_size_t; +typedef struct { char c; _Bool x; } st_bool; #define SHORT_ALIGN (sizeof(st_short) - sizeof(short)) #define INT_ALIGN (sizeof(st_int) - sizeof(int)) @@ -68,21 +69,13 @@ #define DOUBLE_ALIGN (sizeof(st_double) - sizeof(double)) #define VOID_P_ALIGN (sizeof(st_void_p) - sizeof(void *)) #define SIZE_T_ALIGN (sizeof(st_size_t) - sizeof(size_t)) +#define BOOL_ALIGN (sizeof(st_bool) - sizeof(_Bool)) /* We can't support q and Q in native mode unless the compiler does; in std mode, they're 8 bytes on all platforms. */ typedef struct { char c; long long x; } s_long_long; #define LONG_LONG_ALIGN (sizeof(s_long_long) - sizeof(long long)) -#ifdef HAVE_C99_BOOL -#define BOOL_TYPE _Bool -typedef struct { char c; _Bool x; } s_bool; -#define BOOL_ALIGN (sizeof(s_bool) - sizeof(BOOL_TYPE)) -#else -#define BOOL_TYPE char -#define BOOL_ALIGN 0 -#endif - #ifdef __powerc #pragma options align=reset #endif @@ -480,7 +473,7 @@ static PyObject * nu_bool(const char *p, const formatdef *f) { - BOOL_TYPE x; + _Bool x; memcpy((char *)&x, p, sizeof x); return PyBool_FromLong(x != 0); } @@ -695,7 +688,7 @@ np_bool(char *p, PyObject *v, const formatdef *f) { int y; - BOOL_TYPE x; + _Bool x; y = PyObject_IsTrue(v); if (y < 0) return -1; @@ -774,7 +767,7 @@ {'N', sizeof(size_t), SIZE_T_ALIGN, nu_size_t, np_size_t}, {'q', sizeof(long long), LONG_LONG_ALIGN, nu_longlong, np_longlong}, {'Q', sizeof(long long), LONG_LONG_ALIGN, nu_ulonglong,np_ulonglong}, - {'?', sizeof(BOOL_TYPE), BOOL_ALIGN, nu_bool, np_bool}, + {'?', sizeof(_Bool), BOOL_ALIGN, nu_bool, np_bool}, {'e', sizeof(short), SHORT_ALIGN, nu_halffloat, np_halffloat}, {'f', sizeof(float), FLOAT_ALIGN, nu_float, np_float}, {'d', sizeof(double), DOUBLE_ALIGN, nu_double, np_double}, diff --git a/Objects/memoryobject.c b/Objects/memoryobject.c --- a/Objects/memoryobject.c +++ b/Objects/memoryobject.c @@ -1115,11 +1115,7 @@ case 'n': case 'N': size = sizeof(Py_ssize_t); break; case 'f': size = sizeof(float); break; case 'd': size = sizeof(double); break; - #ifdef HAVE_C99_BOOL case '?': size = sizeof(_Bool); break; - #else - case '?': size = sizeof(char); break; - #endif case 'P': size = sizeof(void *); break; } @@ -1162,11 +1158,7 @@ case 'N': RETURN("N"); case 'f': RETURN("f"); case 'd': RETURN("d"); - #ifdef HAVE_C99_BOOL case '?': RETURN("?"); - #else - case '?': RETURN("?"); - #endif case 'P': RETURN("P"); } @@ -1673,11 +1665,7 @@ case 'l': UNPACK_SINGLE(ld, ptr, long); goto convert_ld; /* boolean */ - #ifdef HAVE_C99_BOOL case '?': UNPACK_SINGLE(ld, ptr, _Bool); goto convert_bool; - #else - case '?': UNPACK_SINGLE(ld, ptr, char); goto convert_bool; - #endif /* unsigned integers */ case 'H': UNPACK_SINGLE(lu, ptr, unsigned short); goto convert_lu; @@ -1843,11 +1831,7 @@ ld = PyObject_IsTrue(item); if (ld < 0) return -1; /* preserve original error */ - #ifdef HAVE_C99_BOOL PACK_SINGLE(ptr, ld, _Bool); - #else - PACK_SINGLE(ptr, ld, char); - #endif break; /* bytes object */ @@ -2634,11 +2618,7 @@ case 'l': CMP_SINGLE(p, q, long); return equal; /* boolean */ - #ifdef HAVE_C99_BOOL case '?': CMP_SINGLE(p, q, _Bool); return equal; - #else - case '?': CMP_SINGLE(p, q, char); return equal; - #endif /* unsigned integers */ case 'H': CMP_SINGLE(p, q, unsigned short); return equal; diff --git a/configure b/configure --- a/configure +++ b/configure @@ -777,6 +777,7 @@ docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -888,6 +889,7 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1140,6 +1142,15 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1277,7 +1288,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1430,6 +1441,7 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -8482,33 +8494,6 @@ fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for _Bool support" >&5 -$as_echo_n "checking for _Bool support... " >&6; } -have_c99_bool=no -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ -_Bool x; x = (_Bool)0; - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - - -$as_echo "#define HAVE_C99_BOOL 1" >>confdefs.h - - have_c99_bool=yes - -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $have_c99_bool" >&5 -$as_echo "$have_c99_bool" >&6; } -if test "$have_c99_bool" = yes ; then # The cast to long int works around a bug in the HP C Compiler # version HP92453-01 B.11.11.23709.GP, which incorrectly rejects # declarations like `int a3[[(sizeof (unsigned char)) >= 0]];'. @@ -8542,7 +8527,6 @@ _ACEOF -fi # The cast to long int works around a bug in the HP C Compiler # version HP92453-01 B.11.11.23709.GP, which incorrectly rejects diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -2128,17 +2128,7 @@ AC_CHECK_SIZEOF(long double, 16) fi - -AC_MSG_CHECKING(for _Bool support) -have_c99_bool=no -AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[_Bool x; x = (_Bool)0;]])],[ - AC_DEFINE(HAVE_C99_BOOL, 1, [Define this if you have the type _Bool.]) - have_c99_bool=yes -],[]) -AC_MSG_RESULT($have_c99_bool) -if test "$have_c99_bool" = yes ; then AC_CHECK_SIZEOF(_Bool, 1) -fi AC_CHECK_SIZEOF(off_t, [], [ #ifdef HAVE_SYS_TYPES_H diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -107,9 +107,6 @@ /* Has builtin atomics */ #undef HAVE_BUILTIN_ATOMIC -/* Define this if you have the type _Bool. */ -#undef HAVE_C99_BOOL - /* Define to 1 if you have the 'chflags' function. */ #undef HAVE_CHFLAGS -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 14:10:12 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 07 Sep 2016 18:10:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogIzI2MjA5OiBDbGFy?= =?utf-8?q?ify_type_of_*localaddr*/*remoteadr*_in_smtpd_docs=2E?= Message-ID: <20160907181011.48928.99144.7160A610@psf.io> https://hg.python.org/cpython/rev/7aaf8cff23e5 changeset: 103236:7aaf8cff23e5 branch: 2.7 parent: 103207:923a27028cec user: R David Murray date: Wed Sep 07 14:09:51 2016 -0400 summary: #26209: Clarify type of *localaddr*/*remoteadr* in smtpd docs. files: Doc/library/smtpd.rst | 7 ++++--- Doc/library/socket.rst | 2 ++ 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Doc/library/smtpd.rst b/Doc/library/smtpd.rst --- a/Doc/library/smtpd.rst +++ b/Doc/library/smtpd.rst @@ -23,9 +23,10 @@ .. class:: SMTPServer(localaddr, remoteaddr) Create a new :class:`SMTPServer` object, which binds to local address - *localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. It - inherits from :class:`asyncore.dispatcher`, and so will insert itself into - :mod:`asyncore`'s event loop on instantiation. + *localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. Both + *localaddr* and *remoteaddr* should be a :ref:`(host, port) ` + tuple. The object inherits from :class:`asyncore.dispatcher`, and so will + insert itself into :mod:`asyncore`'s event loop on instantiation. .. method:: process_message(peer, mailfrom, rcpttos, data) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -34,6 +34,8 @@ files, buffer allocation on receive operations is automatic, and buffer length is implicit on send operations. +.. _host_port: + Socket addresses are represented as follows: A single string is used for the :const:`AF_UNIX` address family. A pair ``(host, port)`` is used for the :const:`AF_INET` address family, where *host* is a string representing either a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 14:16:55 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 07 Sep 2016 18:16:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_the_co=5Fextra_field_a?= =?utf-8?q?nd_accompanying_APIs_to_code_objects=2E?= Message-ID: <20160907181653.846.62178.B1BB0CF3@psf.io> https://hg.python.org/cpython/rev/07dde8ba266d changeset: 103237:07dde8ba266d parent: 103235:7f4c9cfae20c user: Brett Cannon date: Wed Sep 07 11:16:41 2016 -0700 summary: Add the co_extra field and accompanying APIs to code objects. This completes PEP 523. files: Doc/whatsnew/3.6.rst | 3 +- Include/ceval.h | 4 + Include/code.h | 20 ++++++- Include/pystate.h | 7 ++ Misc/NEWS | 2 +- Objects/codeobject.c | 91 ++++++++++++++++++++++++++++++++ Python/ceval.c | 14 ++++ Python/pystate.c | 1 + 8 files changed, 139 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -127,7 +127,8 @@ This API is not part of the limited C API and is marked as private to signal that usage of this API is expected to be limited and only -applicable to very select, low-level use-cases. +applicable to very select, low-level use-cases. Semantics of the +API will change with Python as necessary. .. seealso:: diff --git a/Include/ceval.h b/Include/ceval.h --- a/Include/ceval.h +++ b/Include/ceval.h @@ -187,6 +187,10 @@ PyAPI_FUNC(unsigned long) _PyEval_GetSwitchInterval(void); #endif +#ifndef Py_LIMITED_API +PyAPI_FUNC(Py_ssize_t) _PyEval_RequestCodeExtraIndex(freefunc); +#endif + #define Py_BEGIN_ALLOW_THREADS { \ PyThreadState *_save; \ _save = PyEval_SaveThread(); diff --git a/Include/code.h b/Include/code.h --- a/Include/code.h +++ b/Include/code.h @@ -7,6 +7,14 @@ extern "C" { #endif + +/* Holder for co_extra information */ +typedef struct { + Py_ssize_t ce_size; + void **ce_extras; +} _PyCodeObjectExtra; + + /* Bytecode object */ typedef struct { PyObject_HEAD @@ -15,6 +23,7 @@ int co_nlocals; /* #local variables */ int co_stacksize; /* #entries needed for evaluation stack */ int co_flags; /* CO_..., see below */ + int co_firstlineno; /* first source line number */ PyObject *co_code; /* instruction opcodes */ PyObject *co_consts; /* list (constants used) */ PyObject *co_names; /* list of strings (names used) */ @@ -30,11 +39,12 @@ unsigned char *co_cell2arg; /* Maps cell vars which are arguments. */ PyObject *co_filename; /* unicode (where it was loaded from) */ PyObject *co_name; /* unicode (name, for reference) */ - int co_firstlineno; /* first source line number */ PyObject *co_lnotab; /* string (encoding addr<->lineno mapping) See Objects/lnotab_notes.txt for details. */ void *co_zombieframe; /* for optimization only (see frameobject.c) */ PyObject *co_weakreflist; /* to support weakrefs to code objects */ + /* Scratch space for extra data relating to the code object */ + _PyCodeObjectExtra *co_extra; } PyCodeObject; /* Masks for co_flags above */ @@ -128,6 +138,14 @@ PyAPI_FUNC(PyObject*) PyCode_Optimize(PyObject *code, PyObject* consts, PyObject *names, PyObject *lnotab); + +#ifndef Py_LIMITED_API +PyAPI_FUNC(int) _PyCode_GetExtra(PyObject *code, Py_ssize_t index, + void **extra); +PyAPI_FUNC(int) _PyCode_SetExtra(PyObject *code, Py_ssize_t index, + void *extra); +#endif + #ifdef __cplusplus } #endif diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -8,6 +8,10 @@ extern "C" { #endif +/* This limitation is for performance and simplicity. If needed it can be +removed (with effort). */ +#define MAX_CO_EXTRA_USERS 255 + /* State shared between threads */ struct _ts; /* Forward */ @@ -141,6 +145,9 @@ PyObject *coroutine_wrapper; int in_coroutine_wrapper; + Py_ssize_t co_extra_user_count; + freefunc co_extra_freefuncs[MAX_CO_EXTRA_USERS]; + /* XXX signal handlers should also be here */ } PyThreadState; diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,7 +27,7 @@ the braces (where the expressions are). This is a breaking change from the 3.6 alpha releases. -- Implement the frame evaluation part of PEP 523. +- Implement PEP 523. - Issue #27870: A left shift of zero by a large integer no longer attempts to allocate large amounts of memory. diff --git a/Objects/codeobject.c b/Objects/codeobject.c --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -152,6 +152,7 @@ co->co_lnotab = lnotab; co->co_zombieframe = NULL; co->co_weakreflist = NULL; + co->co_extra = NULL; return co; } @@ -361,6 +362,20 @@ static void code_dealloc(PyCodeObject *co) { + if (co->co_extra != NULL) { + PyThreadState *tstate = PyThreadState_Get(); + + for (Py_ssize_t i = 0; i < co->co_extra->ce_size; i++) { + freefunc free_extra = tstate->co_extra_freefuncs[i]; + + if (free_extra != NULL) { + free_extra(co->co_extra->ce_extras[i]); + } + } + + PyMem_FREE(co->co_extra); + } + Py_XDECREF(co->co_code); Py_XDECREF(co->co_consts); Py_XDECREF(co->co_names); @@ -752,3 +767,79 @@ return line; } + + +int +_PyCode_GetExtra(PyObject *code, Py_ssize_t index, void **extra) +{ + PyCodeObject *o; + + assert(*extra == NULL); + + if (!PyCode_Check(code)) { + PyErr_BadInternalCall(); + return 1; + } + + o = (PyCodeObject*) code; + + if (o->co_extra == NULL || o->co_extra->ce_size <= index) { + return 0; + } + + *extra = o->co_extra->ce_extras[index]; + return 0; +} + + +int +_PyCode_SetExtra(PyObject *code, Py_ssize_t index, void *extra) +{ + PyCodeObject *o; + PyThreadState *tstate = PyThreadState_Get(); + + if (!PyCode_Check(code) || index < 0 || + index >= tstate->co_extra_user_count) { + PyErr_BadInternalCall(); + return 1; + } + + o = (PyCodeObject*) code; + + if (o->co_extra == NULL) { + o->co_extra = (_PyCodeObjectExtra*) PyMem_Malloc( + sizeof(_PyCodeObjectExtra)); + if (o->co_extra == NULL) { + return 1; + } + + o->co_extra->ce_extras = PyMem_Malloc( + tstate->co_extra_user_count * sizeof(void*)); + if (o->co_extra->ce_extras == NULL) { + return 1; + } + + o->co_extra->ce_size = tstate->co_extra_user_count; + + for (Py_ssize_t i = 0; i < o->co_extra->ce_size; i++) { + o->co_extra->ce_extras[i] = NULL; + } + } + else if (o->co_extra->ce_size <= index) { + o->co_extra->ce_extras = PyMem_Realloc( + o->co_extra->ce_extras, tstate->co_extra_user_count * sizeof(void*)); + + if (o->co_extra->ce_extras == NULL) { + return 1; + } + + o->co_extra->ce_size = tstate->co_extra_user_count; + + for (Py_ssize_t i = o->co_extra->ce_size; i < o->co_extra->ce_size; i++) { + o->co_extra->ce_extras[i] = NULL; + } + } + + o->co_extra->ce_extras[index] = extra; + return 0; +} diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -5608,3 +5608,17 @@ } #endif + +Py_ssize_t +_PyEval_RequestCodeExtraIndex(freefunc free) +{ + PyThreadState *tstate = PyThreadState_Get(); + Py_ssize_t new_index; + + if (tstate->co_extra_user_count == MAX_CO_EXTRA_USERS - 1) { + return -1; + } + new_index = tstate->co_extra_user_count++; + tstate->co_extra_freefuncs[new_index] = free; + return new_index; +} diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -227,6 +227,7 @@ tstate->coroutine_wrapper = NULL; tstate->in_coroutine_wrapper = 0; + tstate->co_extra_user_count = 0; if (init) _PyThreadState_Init(tstate); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 14:29:19 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 18:29:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_use_a_the_bool_type_for_a_?= =?utf-8?q?boolean_variable?= Message-ID: <20160907182913.14789.49412.D673BCEB@psf.io> https://hg.python.org/cpython/rev/1ef9a3021da9 changeset: 103238:1ef9a3021da9 user: Benjamin Peterson date: Wed Sep 07 11:28:35 2016 -0700 summary: use a the bool type for a boolean variable files: Objects/codeobject.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Objects/codeobject.c b/Objects/codeobject.c --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -1,3 +1,5 @@ +#include + #include "Python.h" #include "code.h" #include "structmember.h" @@ -96,7 +98,7 @@ Py_ssize_t total_args = argcount + kwonlyargcount + ((flags & CO_VARARGS) != 0) + ((flags & CO_VARKEYWORDS) != 0); Py_ssize_t alloc_size = sizeof(unsigned char) * n_cellvars; - int used_cell2arg = 0; + bool used_cell2arg = false; cell2arg = PyMem_MALLOC(alloc_size); if (cell2arg == NULL) return NULL; @@ -109,7 +111,7 @@ PyObject *arg = PyTuple_GET_ITEM(varnames, j); if (!PyUnicode_Compare(cell, arg)) { cell2arg[i] = j; - used_cell2arg = 1; + used_cell2arg = true; break; } } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 14:40:22 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 18:40:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_hardcode_sizeof=28=5FBool?= =?utf-8?q?=29_on_windows?= Message-ID: <20160907184008.14718.31527.962564E6@psf.io> https://hg.python.org/cpython/rev/aac00f62273c changeset: 103239:aac00f62273c user: Benjamin Peterson date: Wed Sep 07 11:39:46 2016 -0700 summary: hardcode sizeof(_Bool) on windows files: PC/pyconfig.h | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/PC/pyconfig.h b/PC/pyconfig.h --- a/PC/pyconfig.h +++ b/PC/pyconfig.h @@ -643,6 +643,9 @@ /* The size of `wchar_t', as computed by sizeof. */ #define SIZEOF_WCHAR_T 2 +/* The size of `_Bool', as computed by sizeof. */ +#define SIZEOF__BOOL 1 + /* The size of `pid_t', as computed by sizeof. */ #define SIZEOF_PID_T SIZEOF_INT -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 14:43:52 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 18:43:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_permit_intermingled_declar?= =?utf-8?q?ations?= Message-ID: <20160907184349.49065.68929.D204B914@psf.io> https://hg.python.org/cpython/rev/26dbaa9c11b2 changeset: 103240:26dbaa9c11b2 user: Benjamin Peterson date: Wed Sep 07 11:43:22 2016 -0700 summary: permit intermingled declarations files: configure | 43 ---------------------------------------- configure.ac | 22 -------------------- 2 files changed, 0 insertions(+), 65 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -6834,49 +6834,6 @@ BASECFLAGS="$BASECFLAGS -Wno-unused-result" fi - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -Werror=declaration-after-statement" >&5 -$as_echo_n "checking for -Werror=declaration-after-statement... " >&6; } - ac_save_cc="$CC" - CC="$CC -Werror=declaration-after-statement" - save_CFLAGS="$CFLAGS" - if ${ac_cv_declaration_after_statement_warning+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - - -int -main () -{ - - ; - return 0; -} - -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - - ac_cv_declaration_after_statement_warning=yes - -else - - ac_cv_declaration_after_statement_warning=no - -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -fi - - CFLAGS="$save_CFLAGS" - CC="$ac_save_cc" - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_declaration_after_statement_warning" >&5 -$as_echo "$ac_cv_declaration_after_statement_warning" >&6; } - - if test $ac_cv_declaration_after_statement_warning = yes - then - CFLAGS_NODIST="$CFLAGS_NODIST -Werror=declaration-after-statement" - fi - { $as_echo "$as_me:${as_lineno-$LINENO}: checking if we can turn on $CC mixed sign comparison warning" >&5 $as_echo_n "checking if we can turn on $CC mixed sign comparison warning... " >&6; } ac_save_cc="$CC" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1542,28 +1542,6 @@ BASECFLAGS="$BASECFLAGS -Wno-unused-result" fi - AC_MSG_CHECKING(for -Werror=declaration-after-statement) - ac_save_cc="$CC" - CC="$CC -Werror=declaration-after-statement" - save_CFLAGS="$CFLAGS" - AC_CACHE_VAL(ac_cv_declaration_after_statement_warning, - AC_COMPILE_IFELSE( - [ - AC_LANG_PROGRAM([[]], [[]]) - ],[ - ac_cv_declaration_after_statement_warning=yes - ],[ - ac_cv_declaration_after_statement_warning=no - ])) - CFLAGS="$save_CFLAGS" - CC="$ac_save_cc" - AC_MSG_RESULT($ac_cv_declaration_after_statement_warning) - - if test $ac_cv_declaration_after_statement_warning = yes - then - CFLAGS_NODIST="$CFLAGS_NODIST -Werror=declaration-after-statement" - fi - AC_MSG_CHECKING(if we can turn on $CC mixed sign comparison warning) ac_save_cc="$CC" CC="$CC -Wsign-compare" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 14:56:20 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 18:56:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_compile_with_-std=3Dc99?= Message-ID: <20160907185607.31371.45295.0D1BAD06@psf.io> https://hg.python.org/cpython/rev/2fc6b04a89a3 changeset: 103241:2fc6b04a89a3 user: Benjamin Peterson date: Wed Sep 07 11:53:55 2016 -0700 summary: compile with -std=c99 files: configure | 2 ++ configure.ac | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -6694,6 +6694,8 @@ SCO_SV*) OPT="$OPT -m486 -DSCO5" ;; esac + + OPT="$OPT -std=c99" ;; *) diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1458,6 +1458,8 @@ SCO_SV*) OPT="$OPT -m486 -DSCO5" ;; esac + + OPT="$OPT -std=c99" ;; *) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 15:00:46 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 19:00:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_put_-std=3Dc99_in_CFLAGS?= =?utf-8?q?=5FNODIST?= Message-ID: <20160907190044.48726.5565.21FBE42F@psf.io> https://hg.python.org/cpython/rev/b3bc33667b00 changeset: 103242:b3bc33667b00 user: Benjamin Peterson date: Wed Sep 07 12:00:06 2016 -0700 summary: put -std=c99 in CFLAGS_NODIST files: configure | 4 ++-- configure.ac | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -6694,8 +6694,6 @@ SCO_SV*) OPT="$OPT -m486 -DSCO5" ;; esac - - OPT="$OPT -std=c99" ;; *) @@ -6714,6 +6712,8 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) + CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" + # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable # -fno-strict-aliasing on versions of GCC that support but produce diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1458,8 +1458,6 @@ SCO_SV*) OPT="$OPT -m486 -DSCO5" ;; esac - - OPT="$OPT -std=c99" ;; *) @@ -1478,6 +1476,8 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) + CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" + # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable # -fno-strict-aliasing on versions of GCC that support but produce -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 15:51:22 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 07 Sep 2016 19:51:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Change_error_return_value_?= =?utf-8?q?to_be_more_consistent_with_the_rest_of_Python?= Message-ID: <20160907195120.12481.11164.E4FA2487@psf.io> https://hg.python.org/cpython/rev/be272696991e changeset: 103243:be272696991e user: Brett Cannon date: Wed Sep 07 12:51:08 2016 -0700 summary: Change error return value to be more consistent with the rest of Python files: Objects/codeobject.c | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Objects/codeobject.c b/Objects/codeobject.c --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -780,7 +780,7 @@ if (!PyCode_Check(code)) { PyErr_BadInternalCall(); - return 1; + return -1; } o = (PyCodeObject*) code; @@ -803,7 +803,7 @@ if (!PyCode_Check(code) || index < 0 || index >= tstate->co_extra_user_count) { PyErr_BadInternalCall(); - return 1; + return -1; } o = (PyCodeObject*) code; @@ -812,13 +812,13 @@ o->co_extra = (_PyCodeObjectExtra*) PyMem_Malloc( sizeof(_PyCodeObjectExtra)); if (o->co_extra == NULL) { - return 1; + return -1; } o->co_extra->ce_extras = PyMem_Malloc( tstate->co_extra_user_count * sizeof(void*)); if (o->co_extra->ce_extras == NULL) { - return 1; + return -1; } o->co_extra->ce_size = tstate->co_extra_user_count; @@ -832,7 +832,7 @@ o->co_extra->ce_extras, tstate->co_extra_user_count * sizeof(void*)); if (o->co_extra->ce_extras == NULL) { - return 1; + return -1; } o->co_extra->ce_size = tstate->co_extra_user_count; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 16:48:59 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 07 Sep 2016 20:48:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2327331=3A_add_policy_key?= =?utf-8?q?word_argument_to_all_MIME_subclasses=2E?= Message-ID: <20160907204856.1421.96089.55D8EAB0@psf.io> https://hg.python.org/cpython/rev/481d14cb7595 changeset: 103244:481d14cb7595 user: R David Murray date: Wed Sep 07 16:48:35 2016 -0400 summary: #27331: add policy keyword argument to all MIME subclasses. Patch by Berker Peksag. files: Doc/library/email.mime.rst | 53 ++++++++++++++++-- Doc/whatsnew/3.6.rst | 7 ++ Lib/email/mime/application.py | 5 +- Lib/email/mime/audio.py | 5 +- Lib/email/mime/base.py | 8 ++- Lib/email/mime/image.py | 5 +- Lib/email/mime/message.py | 4 +- Lib/email/mime/multipart.py | 3 +- Lib/email/mime/text.py | 4 +- Lib/test/test_email/test_email.py | 43 +++++++++++++++- Misc/NEWS | 2 + 11 files changed, 118 insertions(+), 21 deletions(-) diff --git a/Doc/library/email.mime.rst b/Doc/library/email.mime.rst --- a/Doc/library/email.mime.rst +++ b/Doc/library/email.mime.rst @@ -25,7 +25,7 @@ .. currentmodule:: email.mime.base -.. class:: MIMEBase(_maintype, _subtype, **_params) +.. class:: MIMEBase(_maintype, _subtype, *, policy=compat32, **_params) Module: :mod:`email.mime.base` @@ -41,10 +41,17 @@ key/value dictionary and is passed directly to :meth:`Message.add_header `. + If *policy* is specified, (defaults to the + :class:`compat32 ` policy) it will be passed to + :class:`~email.message.Message`. + The :class:`MIMEBase` class always adds a :mailheader:`Content-Type` header (based on *_maintype*, *_subtype*, and *_params*), and a :mailheader:`MIME-Version` header (always set to ``1.0``). + .. versionchanged:: 3.6 + Added *policy* keyword-only parameter. + .. currentmodule:: email.mime.nonmultipart @@ -62,7 +69,8 @@ .. currentmodule:: email.mime.multipart -.. class:: MIMEMultipart(_subtype='mixed', boundary=None, _subparts=None, **_params) +.. class:: MIMEMultipart(_subtype='mixed', boundary=None, _subparts=None, \ + *, policy=compat32, **_params) Module: :mod:`email.mime.multipart` @@ -82,14 +90,20 @@ to the message by using the :meth:`Message.attach ` method. + Optional *policy* argument defaults to :class:`compat32 `. + Additional parameters for the :mailheader:`Content-Type` header are taken from the keyword arguments, or passed into the *_params* argument, which is a keyword dictionary. + .. versionchanged:: 3.6 + Added *policy* keyword-only parameter. .. currentmodule:: email.mime.application -.. class:: MIMEApplication(_data, _subtype='octet-stream', _encoder=email.encoders.encode_base64, **_params) +.. class:: MIMEApplication(_data, _subtype='octet-stream', \ + _encoder=email.encoders.encode_base64, \ + *, policy=compat32, **_params) Module: :mod:`email.mime.application` @@ -109,12 +123,18 @@ object as necessary. The default encoding is base64. See the :mod:`email.encoders` module for a list of the built-in encoders. + Optional *policy* argument defaults to :class:`compat32 `. + *_params* are passed straight through to the base class constructor. + .. versionchanged:: 3.6 + Added *policy* keyword-only parameter. .. currentmodule:: email.mime.audio -.. class:: MIMEAudio(_audiodata, _subtype=None, _encoder=email.encoders.encode_base64, **_params) +.. class:: MIMEAudio(_audiodata, _subtype=None, \ + _encoder=email.encoders.encode_base64, \ + *, policy=compat32, **_params) Module: :mod:`email.mime.audio` @@ -137,12 +157,18 @@ object as necessary. The default encoding is base64. See the :mod:`email.encoders` module for a list of the built-in encoders. + Optional *policy* argument defaults to :class:`compat32 `. + *_params* are passed straight through to the base class constructor. + .. versionchanged:: 3.6 + Added *policy* keyword-only parameter. .. currentmodule:: email.mime.image -.. class:: MIMEImage(_imagedata, _subtype=None, _encoder=email.encoders.encode_base64, **_params) +.. class:: MIMEImage(_imagedata, _subtype=None, \ + _encoder=email.encoders.encode_base64, \ + *, policy=compat32, **_params) Module: :mod:`email.mime.image` @@ -165,13 +191,17 @@ object as necessary. The default encoding is base64. See the :mod:`email.encoders` module for a list of the built-in encoders. + Optional *policy* argument defaults to :class:`compat32 `. + *_params* are passed straight through to the :class:`~email.mime.base.MIMEBase` constructor. + .. versionchanged:: 3.6 + Added *policy* keyword-only parameter. .. currentmodule:: email.mime.message -.. class:: MIMEMessage(_msg, _subtype='rfc822') +.. class:: MIMEMessage(_msg, _subtype='rfc822', *, policy=compat32) Module: :mod:`email.mime.message` @@ -184,10 +214,14 @@ Optional *_subtype* sets the subtype of the message; it defaults to :mimetype:`rfc822`. + Optional *policy* argument defaults to :class:`compat32 `. + + .. versionchanged:: 3.6 + Added *policy* keyword-only parameter. .. currentmodule:: email.mime.text -.. class:: MIMEText(_text, _subtype='plain', _charset=None) +.. class:: MIMEText(_text, _subtype='plain', _charset=None, *, policy=compat32) Module: :mod:`email.mime.text` @@ -211,5 +245,10 @@ will automatically encode the new payload (and add a new :mailheader:`Content-Transfer-Encoding` header). + Optional *policy* argument defaults to :class:`compat32 `. + .. versionchanged:: 3.5 *_charset* also accepts :class:`~email.charset.Charset` instances. + + .. versionchanged:: 3.6 + Added *policy* keyword-only parameter. diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -455,6 +455,13 @@ need to be adapted. See :issue:`27819` for more details. +email +----- + +The :mod:`email.mime` classes now all accept an optional *policy* keyword. +(Contributed by Berker Peksag in :issue:`27331`.) + + encodings --------- diff --git a/Lib/email/mime/application.py b/Lib/email/mime/application.py --- a/Lib/email/mime/application.py +++ b/Lib/email/mime/application.py @@ -14,7 +14,7 @@ """Class for generating application/* MIME documents.""" def __init__(self, _data, _subtype='octet-stream', - _encoder=encoders.encode_base64, **_params): + _encoder=encoders.encode_base64, *, policy=None, **_params): """Create an application/* type MIME document. _data is a string containing the raw application data. @@ -31,6 +31,7 @@ """ if _subtype is None: raise TypeError('Invalid application MIME subtype') - MIMENonMultipart.__init__(self, 'application', _subtype, **_params) + MIMENonMultipart.__init__(self, 'application', _subtype, policy=policy, + **_params) self.set_payload(_data) _encoder(self) diff --git a/Lib/email/mime/audio.py b/Lib/email/mime/audio.py --- a/Lib/email/mime/audio.py +++ b/Lib/email/mime/audio.py @@ -43,7 +43,7 @@ """Class for generating audio/* MIME documents.""" def __init__(self, _audiodata, _subtype=None, - _encoder=encoders.encode_base64, **_params): + _encoder=encoders.encode_base64, *, policy=None, **_params): """Create an audio/* type MIME document. _audiodata is a string containing the raw audio data. If this data @@ -68,6 +68,7 @@ _subtype = _whatsnd(_audiodata) if _subtype is None: raise TypeError('Could not find audio MIME subtype') - MIMENonMultipart.__init__(self, 'audio', _subtype, **_params) + MIMENonMultipart.__init__(self, 'audio', _subtype, policy=policy, + **_params) self.set_payload(_audiodata) _encoder(self) diff --git a/Lib/email/mime/base.py b/Lib/email/mime/base.py --- a/Lib/email/mime/base.py +++ b/Lib/email/mime/base.py @@ -6,6 +6,8 @@ __all__ = ['MIMEBase'] +import email.policy + from email import message @@ -13,14 +15,16 @@ class MIMEBase(message.Message): """Base class for MIME specializations.""" - def __init__(self, _maintype, _subtype, **_params): + def __init__(self, _maintype, _subtype, *, policy=None, **_params): """This constructor adds a Content-Type: and a MIME-Version: header. The Content-Type: header is taken from the _maintype and _subtype arguments. Additional parameters for this header are taken from the keyword arguments. """ - message.Message.__init__(self) + if policy is None: + policy = email.policy.compat32 + message.Message.__init__(self, policy=policy) ctype = '%s/%s' % (_maintype, _subtype) self.add_header('Content-Type', ctype, **_params) self['MIME-Version'] = '1.0' diff --git a/Lib/email/mime/image.py b/Lib/email/mime/image.py --- a/Lib/email/mime/image.py +++ b/Lib/email/mime/image.py @@ -17,7 +17,7 @@ """Class for generating image/* type MIME documents.""" def __init__(self, _imagedata, _subtype=None, - _encoder=encoders.encode_base64, **_params): + _encoder=encoders.encode_base64, *, policy=None, **_params): """Create an image/* type MIME document. _imagedata is a string containing the raw image data. If this data @@ -41,6 +41,7 @@ _subtype = imghdr.what(None, _imagedata) if _subtype is None: raise TypeError('Could not guess image MIME subtype') - MIMENonMultipart.__init__(self, 'image', _subtype, **_params) + MIMENonMultipart.__init__(self, 'image', _subtype, policy=policy, + **_params) self.set_payload(_imagedata) _encoder(self) diff --git a/Lib/email/mime/message.py b/Lib/email/mime/message.py --- a/Lib/email/mime/message.py +++ b/Lib/email/mime/message.py @@ -14,7 +14,7 @@ class MIMEMessage(MIMENonMultipart): """Class representing message/* MIME documents.""" - def __init__(self, _msg, _subtype='rfc822'): + def __init__(self, _msg, _subtype='rfc822', *, policy=None): """Create a message/* type MIME document. _msg is a message object and must be an instance of Message, or a @@ -24,7 +24,7 @@ default is "rfc822" (this is defined by the MIME standard, even though the term "rfc822" is technically outdated by RFC 2822). """ - MIMENonMultipart.__init__(self, 'message', _subtype) + MIMENonMultipart.__init__(self, 'message', _subtype, policy=policy) if not isinstance(_msg, message.Message): raise TypeError('Argument is not an instance of Message') # It's convenient to use this base class method. We need to do it diff --git a/Lib/email/mime/multipart.py b/Lib/email/mime/multipart.py --- a/Lib/email/mime/multipart.py +++ b/Lib/email/mime/multipart.py @@ -14,6 +14,7 @@ """Base class for MIME multipart/* type messages.""" def __init__(self, _subtype='mixed', boundary=None, _subparts=None, + *, policy=None, **_params): """Creates a multipart/* type message. @@ -33,7 +34,7 @@ Additional parameters for the Content-Type header are taken from the keyword arguments (or passed into the _params argument). """ - MIMEBase.__init__(self, 'multipart', _subtype, **_params) + MIMEBase.__init__(self, 'multipart', _subtype, policy=policy, **_params) # Initialise _payload to an empty list as the Message superclass's # implementation of is_multipart assumes that _payload is a list for diff --git a/Lib/email/mime/text.py b/Lib/email/mime/text.py --- a/Lib/email/mime/text.py +++ b/Lib/email/mime/text.py @@ -14,7 +14,7 @@ class MIMEText(MIMENonMultipart): """Class for generating text/* type MIME documents.""" - def __init__(self, _text, _subtype='plain', _charset=None): + def __init__(self, _text, _subtype='plain', _charset=None, *, policy=None): """Create a text/* type MIME document. _text is the string for this message object. @@ -38,7 +38,7 @@ if isinstance(_charset, Charset): _charset = str(_charset) - MIMENonMultipart.__init__(self, 'text', _subtype, + MIMENonMultipart.__init__(self, 'text', _subtype, policy=policy, **{'charset': _charset}) self.set_payload(_text, _charset) diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -31,6 +31,7 @@ from email.mime.base import MIMEBase from email.mime.message import MIMEMessage from email.mime.multipart import MIMEMultipart +from email.mime.nonmultipart import MIMENonMultipart from email import utils from email import errors from email import encoders @@ -2062,7 +2063,13 @@ --===============0012394164==--""") self.assertEqual(m.get_payload(0).get_payload(), 'YXNkZg==') - + def test_mimebase_default_policy(self): + m = MIMEBase('multipart', 'mixed') + self.assertIs(m.policy, email.policy.compat32) + + def test_mimebase_custom_policy(self): + m = MIMEBase('multipart', 'mixed', policy=email.policy.default) + self.assertIs(m.policy, email.policy.default) # Test some badly formatted messages class TestNonConformant(TestEmailBase): @@ -2664,6 +2671,19 @@ msg = MIMEMultipart() self.assertTrue(msg.is_multipart()) + def test_multipart_default_policy(self): + msg = MIMEMultipart() + msg['To'] = 'a at b.com' + msg['To'] = 'c at d.com' + self.assertEqual(msg.get_all('to'), ['a at b.com', 'c at d.com']) + + def test_multipart_custom_policy(self): + msg = MIMEMultipart(policy=email.policy.default) + msg['To'] = 'a at b.com' + with self.assertRaises(ValueError) as cm: + msg['To'] = 'c at d.com' + self.assertEqual(str(cm.exception), + 'There may be at most 1 To headers in a message') # A general test of parser->model->generator idempotency. IOW, read a message # in, parse it into a message object tree, then without touching the tree, @@ -3313,6 +3333,27 @@ g.flatten(msg, linesep='\r\n') self.assertEqual(s.getvalue(), msgtxt) + def test_mime_classes_policy_argument(self): + with openfile('audiotest.au', 'rb') as fp: + audiodata = fp.read() + with openfile('PyBanner048.gif', 'rb') as fp: + bindata = fp.read() + classes = [ + (MIMEApplication, ('',)), + (MIMEAudio, (audiodata,)), + (MIMEImage, (bindata,)), + (MIMEMessage, (Message(),)), + (MIMENonMultipart, ('multipart', 'mixed')), + (MIMEText, ('',)), + ] + for cls, constructor in classes: + with self.subTest(cls=cls.__name__, policy='compat32'): + m = cls(*constructor) + self.assertIs(m.policy, email.policy.compat32) + with self.subTest(cls=cls.__name__, policy='default'): + m = cls(*constructor, policy=email.policy.default) + self.assertIs(m.policy, email.policy.default) + # Test the iterator/generators class TestIterators(TestEmailBase): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -91,6 +91,8 @@ Library ------- +- Issue 27331: The email.mime classes now all accept an optional policy keyword. + - Issue 27988: Fix email iter_attachments incorrect mutation of payload list. - Issue #16113: Add SHA-3 and SHAKE support to hashlib module. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 17:07:45 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 07 Sep 2016 21:07:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Eliminate_a_tautological-p?= =?utf-8?q?ointer-compare_warning_found_by_Clang=2E?= Message-ID: <20160907210742.14624.32239.9D24DE73@psf.io> https://hg.python.org/cpython/rev/bac7899c55fa changeset: 103245:bac7899c55fa user: Brett Cannon date: Wed Sep 07 14:07:16 2016 -0700 summary: Eliminate a tautological-pointer-compare warning found by Clang. files: Misc/NEWS | 2 ++ Modules/_scproxy.c | 14 +++++--------- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -97,6 +97,8 @@ - Issue #16113: Add SHA-3 and SHAKE support to hashlib module. +- Eliminate a tautological-pointer-compare warning in _scproxy.c. + - Issue #27776: The :func:`os.urandom` function does now block on Linux 3.17 and newer until the system urandom entropy pool is initialized to increase the security. This change is part of the :pep:`524`. diff --git a/Modules/_scproxy.c b/Modules/_scproxy.c --- a/Modules/_scproxy.c +++ b/Modules/_scproxy.c @@ -71,16 +71,12 @@ result = PyDict_New(); if (result == NULL) goto error; - if (&kSCPropNetProxiesExcludeSimpleHostnames != NULL) { - aNum = CFDictionaryGetValue(proxyDict, - kSCPropNetProxiesExcludeSimpleHostnames); - if (aNum == NULL) { - v = PyBool_FromLong(0); - } else { - v = PyBool_FromLong(cfnum_to_int32(aNum)); - } - } else { + aNum = CFDictionaryGetValue(proxyDict, + kSCPropNetProxiesExcludeSimpleHostnames); + if (aNum == NULL) { v = PyBool_FromLong(0); + } else { + v = PyBool_FromLong(cfnum_to_int32(aNum)); } if (v == NULL) goto error; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 17:09:09 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 21:09:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogdXNlIHRoZSAnX19saW51eF9f?= =?utf-8?q?=27_instead_=27linux=27_preprocessor_define?= Message-ID: <20160907210908.49065.16127.A5F69751@psf.io> https://hg.python.org/cpython/rev/577ef39f514b changeset: 103246:577ef39f514b user: Benjamin Peterson date: Wed Sep 07 14:08:34 2016 -0700 summary: use the '__linux__' instead 'linux' preprocessor define files: Modules/socketmodule.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -153,7 +153,7 @@ On the other hand, not all Linux versions agree, so there the settings computed by the configure script are needed! */ -#ifndef linux +#ifndef __linux__ # undef HAVE_GETHOSTBYNAME_R_3_ARG # undef HAVE_GETHOSTBYNAME_R_5_ARG # undef HAVE_GETHOSTBYNAME_R_6_ARG @@ -176,7 +176,7 @@ # define HAVE_GETHOSTBYNAME_R_3_ARG # elif defined(__sun) || defined(__sgi) # define HAVE_GETHOSTBYNAME_R_5_ARG -# elif defined(linux) +# elif defined(__linux__) /* Rely on the configure script */ # else # undef HAVE_GETHOSTBYNAME_R @@ -1214,7 +1214,7 @@ case AF_UNIX: { struct sockaddr_un *a = (struct sockaddr_un *) addr; -#ifdef linux +#ifdef __linux__ if (a->sun_path[0] == 0) { /* Linux abstract namespace */ addrlen -= offsetof(struct sockaddr_un, sun_path); return PyBytes_FromStringAndSize(a->sun_path, addrlen); @@ -1529,7 +1529,7 @@ assert(path.len >= 0); addr = (struct sockaddr_un*)addr_ret; -#ifdef linux +#ifdef __linux__ if (path.len > 0 && *(const char *)path.buf == 0) { /* Linux abstract namespace extension */ if ((size_t)path.len > sizeof addr->sun_path) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 17:12:58 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 21:12:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_use_c++_style_comments?= Message-ID: <20160907211258.13913.34709.B1B3D3AD@psf.io> https://hg.python.org/cpython/rev/da509382764c changeset: 103247:da509382764c user: Benjamin Peterson date: Wed Sep 07 14:12:36 2016 -0700 summary: use c++ style comments files: Objects/namespaceobject.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Objects/namespaceobject.c b/Objects/namespaceobject.c --- a/Objects/namespaceobject.c +++ b/Objects/namespaceobject.c @@ -1,4 +1,4 @@ -/* namespace object implementation */ +// namespace object implementation #include "Python.h" #include "structmember.h" @@ -16,7 +16,7 @@ }; -/* Methods */ +// Methods static PyObject * namespace_new(PyTypeObject *type, PyObject *args, PyObject *kwds) @@ -40,7 +40,7 @@ static int namespace_init(_PyNamespaceObject *ns, PyObject *args, PyObject *kwds) { - /* ignore args if it's NULL or empty */ + // ignore args if it's NULL or empty if (args != NULL) { Py_ssize_t argcount = PyObject_Size(args); if (argcount < 0) @@ -191,7 +191,7 @@ static PyMethodDef namespace_methods[] = { {"__reduce__", (PyCFunction)namespace_reduce, METH_NOARGS, namespace_reduce__doc__}, - {NULL, NULL} /* sentinel */ + {NULL, NULL} // sentinel }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 17:32:06 2016 From: python-checkins at python.org (brett.cannon) Date: Wed, 07 Sep 2016 21:32:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Make_PyCodeObject=2Eco=5Fe?= =?utf-8?q?xtra_even_more_private_to_force_users_through_the_proper?= Message-ID: <20160907213201.31352.95373.63B46FE5@psf.io> https://hg.python.org/cpython/rev/33150309db74 changeset: 103248:33150309db74 user: Brett Cannon date: Wed Sep 07 14:30:39 2016 -0700 summary: Make PyCodeObject.co_extra even more private to force users through the proper API. files: Include/code.h | 14 ++----- Objects/codeobject.c | 54 ++++++++++++++++++------------- 2 files changed, 35 insertions(+), 33 deletions(-) diff --git a/Include/code.h b/Include/code.h --- a/Include/code.h +++ b/Include/code.h @@ -7,14 +7,6 @@ extern "C" { #endif - -/* Holder for co_extra information */ -typedef struct { - Py_ssize_t ce_size; - void **ce_extras; -} _PyCodeObjectExtra; - - /* Bytecode object */ typedef struct { PyObject_HEAD @@ -43,8 +35,10 @@ Objects/lnotab_notes.txt for details. */ void *co_zombieframe; /* for optimization only (see frameobject.c) */ PyObject *co_weakreflist; /* to support weakrefs to code objects */ - /* Scratch space for extra data relating to the code object */ - _PyCodeObjectExtra *co_extra; + /* Scratch space for extra data relating to the code object.__icc_nan + Type is a void* to keep the format private in codeobject.c to force + people to go through the proper APIs. */ + void *co_extra; } PyCodeObject; /* Masks for co_flags above */ diff --git a/Objects/codeobject.c b/Objects/codeobject.c --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -7,6 +7,12 @@ #define NAME_CHARS \ "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz" +/* Holder for co_extra information */ +typedef struct { + Py_ssize_t ce_size; + void **ce_extras; +} _PyCodeObjectExtra; + /* all_name_chars(s): true iff all chars in s are valid NAME_CHARS */ static int @@ -366,12 +372,13 @@ { if (co->co_extra != NULL) { PyThreadState *tstate = PyThreadState_Get(); + _PyCodeObjectExtra *co_extra = co->co_extra; - for (Py_ssize_t i = 0; i < co->co_extra->ce_size; i++) { + for (Py_ssize_t i = 0; i < co_extra->ce_size; i++) { freefunc free_extra = tstate->co_extra_freefuncs[i]; if (free_extra != NULL) { - free_extra(co->co_extra->ce_extras[i]); + free_extra(co_extra->ce_extras[i]); } } @@ -774,8 +781,6 @@ int _PyCode_GetExtra(PyObject *code, Py_ssize_t index, void **extra) { - PyCodeObject *o; - assert(*extra == NULL); if (!PyCode_Check(code)) { @@ -783,13 +788,15 @@ return -1; } - o = (PyCodeObject*) code; + PyCodeObject *o = (PyCodeObject*) code; + _PyCodeObjectExtra *co_extra = (_PyCodeObjectExtra*) o->co_extra; - if (o->co_extra == NULL || o->co_extra->ce_size <= index) { + + if (co_extra == NULL || co_extra->ce_size <= index) { return 0; } - *extra = o->co_extra->ce_extras[index]; + *extra = co_extra->ce_extras[index]; return 0; } @@ -797,7 +804,6 @@ int _PyCode_SetExtra(PyObject *code, Py_ssize_t index, void *extra) { - PyCodeObject *o; PyThreadState *tstate = PyThreadState_Get(); if (!PyCode_Check(code) || index < 0 || @@ -806,42 +812,44 @@ return -1; } - o = (PyCodeObject*) code; + PyCodeObject *o = (PyCodeObject*) code; + _PyCodeObjectExtra *co_extra = (_PyCodeObjectExtra *) o->co_extra; - if (o->co_extra == NULL) { + if (co_extra == NULL) { o->co_extra = (_PyCodeObjectExtra*) PyMem_Malloc( sizeof(_PyCodeObjectExtra)); if (o->co_extra == NULL) { return -1; } + co_extra = (_PyCodeObjectExtra *) o->co_extra; - o->co_extra->ce_extras = PyMem_Malloc( + co_extra->ce_extras = PyMem_Malloc( tstate->co_extra_user_count * sizeof(void*)); - if (o->co_extra->ce_extras == NULL) { + if (co_extra->ce_extras == NULL) { return -1; } - o->co_extra->ce_size = tstate->co_extra_user_count; + co_extra->ce_size = tstate->co_extra_user_count; - for (Py_ssize_t i = 0; i < o->co_extra->ce_size; i++) { - o->co_extra->ce_extras[i] = NULL; + for (Py_ssize_t i = 0; i < co_extra->ce_size; i++) { + co_extra->ce_extras[i] = NULL; } } - else if (o->co_extra->ce_size <= index) { - o->co_extra->ce_extras = PyMem_Realloc( - o->co_extra->ce_extras, tstate->co_extra_user_count * sizeof(void*)); + else if (co_extra->ce_size <= index) { + co_extra->ce_extras = PyMem_Realloc( + co_extra->ce_extras, tstate->co_extra_user_count * sizeof(void*)); - if (o->co_extra->ce_extras == NULL) { + if (co_extra->ce_extras == NULL) { return -1; } - o->co_extra->ce_size = tstate->co_extra_user_count; + co_extra->ce_size = tstate->co_extra_user_count; - for (Py_ssize_t i = o->co_extra->ce_size; i < o->co_extra->ce_size; i++) { - o->co_extra->ce_extras[i] = NULL; + for (Py_ssize_t i = co_extra->ce_size; i < co_extra->ce_size; i++) { + co_extra->ce_extras[i] = NULL; } } - o->co_extra->ce_extras[index] = extra; + co_extra->ce_extras[index] = extra; return 0; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 17:47:45 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 07 Sep 2016 21:47:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzIyMjMzOiBPbmx5?= =?utf-8?q?_split_headers_on_=5Cr_and/or_=5Cn=2C_per_email_RFCs=2E?= Message-ID: <20160907214744.14871.6243.C3DF1677@psf.io> https://hg.python.org/cpython/rev/69900c5992c5 changeset: 103249:69900c5992c5 branch: 3.5 parent: 103233:1ed37f6e91bb user: R David Murray date: Wed Sep 07 17:44:34 2016 -0400 summary: #22233: Only split headers on \r and/or \n, per email RFCs. Original patch by Martin Panter, new policy fixes by me. files: Lib/email/feedparser.py | 33 +++++++------- Lib/email/policy.py | 9 +++- Lib/test/test_email/test_email.py | 6 +- Lib/test/test_email/test_parser.py | 41 ++++++++++++++++++ Lib/test/test_httplib.py | 30 +++++++++++++ Misc/NEWS | 4 + 6 files changed, 104 insertions(+), 19 deletions(-) diff --git a/Lib/email/feedparser.py b/Lib/email/feedparser.py --- a/Lib/email/feedparser.py +++ b/Lib/email/feedparser.py @@ -27,6 +27,7 @@ from email import message from email._policybase import compat32 from collections import deque +from io import StringIO NLCRE = re.compile('\r\n|\r|\n') NLCRE_bol = re.compile('(\r\n|\r|\n)') @@ -51,8 +52,9 @@ simple abstraction -- it parses until EOF closes the current message. """ def __init__(self): - # Chunks of the last partial line pushed into this object. - self._partial = [] + # Text stream of the last partial line pushed into this object. + # See issue 22233 for why this is a text stream and not a list. + self._partial = StringIO(newline='') # A deque of full, pushed lines self._lines = deque() # The stack of false-EOF checking predicates. @@ -68,8 +70,10 @@ def close(self): # Don't forget any trailing partial line. - self.pushlines(''.join(self._partial).splitlines(True)) - self._partial = [] + self._partial.seek(0) + self.pushlines(self._partial.readlines()) + self._partial.seek(0) + self._partial.truncate() self._closed = True def readline(self): @@ -97,26 +101,23 @@ def push(self, data): """Push some new data into this object.""" - # Crack into lines, but preserve the linesep characters on the end of each - parts = data.splitlines(True) - - if not parts or not parts[0].endswith(('\n', '\r')): - # No new complete lines, so just accumulate partials - self._partial += parts + self._partial.write(data) + if '\n' not in data and '\r' not in data: + # No new complete lines, wait for more. return - if self._partial: - # If there are previous leftovers, complete them now - self._partial.append(parts[0]) - parts[0:1] = ''.join(self._partial).splitlines(True) - del self._partial[:] + # Crack into lines, preserving the linesep characters. + self._partial.seek(0) + parts = self._partial.readlines() + self._partial.seek(0) + self._partial.truncate() # If the last element of the list does not end in a newline, then treat # it as a partial line. We only check for '\n' here because a line # ending with '\r' might be a line that was split in the middle of a # '\r\n' sequence (see bugs 1555570 and 1721862). if not parts[-1].endswith('\n'): - self._partial = [parts.pop()] + self._partial.write(parts.pop()) self.pushlines(parts) def pushlines(self, lines): diff --git a/Lib/email/policy.py b/Lib/email/policy.py --- a/Lib/email/policy.py +++ b/Lib/email/policy.py @@ -2,6 +2,7 @@ code that adds all the email6 features. """ +import re from email._policybase import Policy, Compat32, compat32, _extend_docstrings from email.utils import _has_surrogates from email.headerregistry import HeaderRegistry as HeaderRegistry @@ -18,6 +19,8 @@ 'HTTP', ] +linesep_splitter = re.compile(r'\n|\r') + @_extend_docstrings class EmailPolicy(Policy): @@ -135,6 +138,8 @@ if hasattr(value, 'name') and value.name.lower() == name.lower(): return (name, value) if isinstance(value, str) and len(value.splitlines())>1: + # XXX this error message isn't quite right when we use splitlines + # (see issue 22233), but I'm not sure what should happen here. raise ValueError("Header values may not contain linefeed " "or carriage return characters") return (name, self.header_factory(name, value)) @@ -150,7 +155,9 @@ """ if hasattr(value, 'name'): return value - return self.header_factory(name, ''.join(value.splitlines())) + # We can't use splitlines here because it splits on more than \r and \n. + value = ''.join(linesep_splitter.split(value)) + return self.header_factory(name, value) def fold(self, name, value): """+ diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3444,10 +3444,12 @@ self.assertEqual(m.keys(), ['a', 'b']) m = self.parse(['a:\r', '\nb:\n']) self.assertEqual(m.keys(), ['a', 'b']) + + # Only CR and LF should break header fields m = self.parse(['a:\x85b:\u2028c:\n']) - self.assertEqual(m.items(), [('a', '\x85'), ('b', '\u2028'), ('c', '')]) + self.assertEqual(m.items(), [('a', '\x85b:\u2028c:')]) m = self.parse(['a:\r', 'b:\x85', 'c:\n']) - self.assertEqual(m.items(), [('a', ''), ('b', '\x85'), ('c', '')]) + self.assertEqual(m.items(), [('a', ''), ('b', '\x85c:')]) def test_long_lines(self): # Expected peak memory use on 32-bit platform: 6*N*M bytes. diff --git a/Lib/test/test_email/test_parser.py b/Lib/test/test_email/test_parser.py --- a/Lib/test/test_email/test_parser.py +++ b/Lib/test/test_email/test_parser.py @@ -2,6 +2,7 @@ import email import unittest from email.message import Message +from email.policy import default from test.test_email import TestEmailBase @@ -32,5 +33,45 @@ # XXX add tests for other functions that take Message arg. +class TestParserBase: + + def test_only_split_on_cr_lf(self): + # The unicode line splitter splits on unicode linebreaks, which are + # more numerous than allowed by the email RFCs; make sure we are only + # splitting on those two. + msg = self.parser( + "Next-Line: not\x85broken\r\n" + "Null: not\x00broken\r\n" + "Vertical-Tab: not\vbroken\r\n" + "Form-Feed: not\fbroken\r\n" + "File-Separator: not\x1Cbroken\r\n" + "Group-Separator: not\x1Dbroken\r\n" + "Record-Separator: not\x1Ebroken\r\n" + "Line-Separator: not\u2028broken\r\n" + "Paragraph-Separator: not\u2029broken\r\n" + "\r\n", + policy=default, + ) + self.assertEqual(msg.items(), [ + ("Next-Line", "not\x85broken"), + ("Null", "not\x00broken"), + ("Vertical-Tab", "not\vbroken"), + ("Form-Feed", "not\fbroken"), + ("File-Separator", "not\x1Cbroken"), + ("Group-Separator", "not\x1Dbroken"), + ("Record-Separator", "not\x1Ebroken"), + ("Line-Separator", "not\u2028broken"), + ("Paragraph-Separator", "not\u2029broken"), + ]) + self.assertEqual(msg.get_payload(), "") + +class TestParser(TestParserBase, TestEmailBase): + parser = staticmethod(email.message_from_string) + +class TestBytesParser(TestParserBase, TestEmailBase): + def parser(self, s, *args, **kw): + return email.message_from_bytes(s.encode(), *args, **kw) + + if __name__ == '__main__': unittest.main() diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -283,6 +283,36 @@ self.assertEqual(resp.getheader('First'), 'val') self.assertEqual(resp.getheader('Second'), 'val') + def test_parse_all_octets(self): + # Ensure no valid header field octet breaks the parser + body = ( + b'HTTP/1.1 200 OK\r\n' + b"!#$%&'*+-.^_`|~: value\r\n" # Special token characters + b'VCHAR: ' + bytes(range(0x21, 0x7E + 1)) + b'\r\n' + b'obs-text: ' + bytes(range(0x80, 0xFF + 1)) + b'\r\n' + b'obs-fold: text\r\n' + b' folded with space\r\n' + b'\tfolded with tab\r\n' + b'Content-Length: 0\r\n' + b'\r\n' + ) + sock = FakeSocket(body) + resp = client.HTTPResponse(sock) + resp.begin() + self.assertEqual(resp.getheader('Content-Length'), '0') + self.assertEqual(resp.msg['Content-Length'], '0') + self.assertEqual(resp.getheader("!#$%&'*+-.^_`|~"), 'value') + self.assertEqual(resp.msg["!#$%&'*+-.^_`|~"], 'value') + vchar = ''.join(map(chr, range(0x21, 0x7E + 1))) + self.assertEqual(resp.getheader('VCHAR'), vchar) + self.assertEqual(resp.msg['VCHAR'], vchar) + self.assertIsNotNone(resp.getheader('obs-text')) + self.assertIn('obs-text', resp.msg) + for folded in (resp.getheader('obs-fold'), resp.msg['obs-fold']): + self.assertTrue(folded.startswith('text')) + self.assertIn(' folded with space', folded) + self.assertTrue(folded.endswith('folded with tab')) + def test_invalid_headers(self): conn = client.HTTPConnection('example.com') conn.sock = FakeSocket('') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -62,6 +62,10 @@ Library ------- +- Issue #22233: Break email header lines *only* on the RFC specified CR and LF + characters, not on arbitrary unicode line breaks. This also fixes a bug in + HTTP header parsing. + - Issue 27988: Fix email iter_attachments incorrect mutation of payload list. - Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 17:47:45 2016 From: python-checkins at python.org (r.david.murray) Date: Wed, 07 Sep 2016 21:47:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2322233=3A_Only_split_headers_on_=5Cr_and/or_?= =?utf-8?q?=5Cn=2C_per_email_RFCs=2E?= Message-ID: <20160907214744.31427.40529.0069CDDC@psf.io> https://hg.python.org/cpython/rev/4d2369b901be changeset: 103250:4d2369b901be parent: 103248:33150309db74 parent: 103249:69900c5992c5 user: R David Murray date: Wed Sep 07 17:46:55 2016 -0400 summary: Merge: #22233: Only split headers on \r and/or \n, per email RFCs. files: Lib/email/feedparser.py | 33 +++++++------- Lib/email/policy.py | 9 +++- Lib/test/test_email/test_email.py | 6 +- Lib/test/test_email/test_parser.py | 41 ++++++++++++++++++ Lib/test/test_httplib.py | 30 +++++++++++++ Misc/NEWS | 4 + 6 files changed, 104 insertions(+), 19 deletions(-) diff --git a/Lib/email/feedparser.py b/Lib/email/feedparser.py --- a/Lib/email/feedparser.py +++ b/Lib/email/feedparser.py @@ -27,6 +27,7 @@ from email import message from email._policybase import compat32 from collections import deque +from io import StringIO NLCRE = re.compile('\r\n|\r|\n') NLCRE_bol = re.compile('(\r\n|\r|\n)') @@ -51,8 +52,9 @@ simple abstraction -- it parses until EOF closes the current message. """ def __init__(self): - # Chunks of the last partial line pushed into this object. - self._partial = [] + # Text stream of the last partial line pushed into this object. + # See issue 22233 for why this is a text stream and not a list. + self._partial = StringIO(newline='') # A deque of full, pushed lines self._lines = deque() # The stack of false-EOF checking predicates. @@ -68,8 +70,10 @@ def close(self): # Don't forget any trailing partial line. - self.pushlines(''.join(self._partial).splitlines(True)) - self._partial = [] + self._partial.seek(0) + self.pushlines(self._partial.readlines()) + self._partial.seek(0) + self._partial.truncate() self._closed = True def readline(self): @@ -97,26 +101,23 @@ def push(self, data): """Push some new data into this object.""" - # Crack into lines, but preserve the linesep characters on the end of each - parts = data.splitlines(True) - - if not parts or not parts[0].endswith(('\n', '\r')): - # No new complete lines, so just accumulate partials - self._partial += parts + self._partial.write(data) + if '\n' not in data and '\r' not in data: + # No new complete lines, wait for more. return - if self._partial: - # If there are previous leftovers, complete them now - self._partial.append(parts[0]) - parts[0:1] = ''.join(self._partial).splitlines(True) - del self._partial[:] + # Crack into lines, preserving the linesep characters. + self._partial.seek(0) + parts = self._partial.readlines() + self._partial.seek(0) + self._partial.truncate() # If the last element of the list does not end in a newline, then treat # it as a partial line. We only check for '\n' here because a line # ending with '\r' might be a line that was split in the middle of a # '\r\n' sequence (see bugs 1555570 and 1721862). if not parts[-1].endswith('\n'): - self._partial = [parts.pop()] + self._partial.write(parts.pop()) self.pushlines(parts) def pushlines(self, lines): diff --git a/Lib/email/policy.py b/Lib/email/policy.py --- a/Lib/email/policy.py +++ b/Lib/email/policy.py @@ -2,6 +2,7 @@ code that adds all the email6 features. """ +import re from email._policybase import Policy, Compat32, compat32, _extend_docstrings from email.utils import _has_surrogates from email.headerregistry import HeaderRegistry as HeaderRegistry @@ -18,6 +19,8 @@ 'HTTP', ] +linesep_splitter = re.compile(r'\n|\r') + @_extend_docstrings class EmailPolicy(Policy): @@ -135,6 +138,8 @@ if hasattr(value, 'name') and value.name.lower() == name.lower(): return (name, value) if isinstance(value, str) and len(value.splitlines())>1: + # XXX this error message isn't quite right when we use splitlines + # (see issue 22233), but I'm not sure what should happen here. raise ValueError("Header values may not contain linefeed " "or carriage return characters") return (name, self.header_factory(name, value)) @@ -150,7 +155,9 @@ """ if hasattr(value, 'name'): return value - return self.header_factory(name, ''.join(value.splitlines())) + # We can't use splitlines here because it splits on more than \r and \n. + value = ''.join(linesep_splitter.split(value)) + return self.header_factory(name, value) def fold(self, name, value): """+ diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3484,10 +3484,12 @@ self.assertEqual(m.keys(), ['a', 'b']) m = self.parse(['a:\r', '\nb:\n']) self.assertEqual(m.keys(), ['a', 'b']) + + # Only CR and LF should break header fields m = self.parse(['a:\x85b:\u2028c:\n']) - self.assertEqual(m.items(), [('a', '\x85'), ('b', '\u2028'), ('c', '')]) + self.assertEqual(m.items(), [('a', '\x85b:\u2028c:')]) m = self.parse(['a:\r', 'b:\x85', 'c:\n']) - self.assertEqual(m.items(), [('a', ''), ('b', '\x85'), ('c', '')]) + self.assertEqual(m.items(), [('a', ''), ('b', '\x85c:')]) def test_long_lines(self): # Expected peak memory use on 32-bit platform: 6*N*M bytes. diff --git a/Lib/test/test_email/test_parser.py b/Lib/test/test_email/test_parser.py --- a/Lib/test/test_email/test_parser.py +++ b/Lib/test/test_email/test_parser.py @@ -2,6 +2,7 @@ import email import unittest from email.message import Message +from email.policy import default from test.test_email import TestEmailBase @@ -32,5 +33,45 @@ # XXX add tests for other functions that take Message arg. +class TestParserBase: + + def test_only_split_on_cr_lf(self): + # The unicode line splitter splits on unicode linebreaks, which are + # more numerous than allowed by the email RFCs; make sure we are only + # splitting on those two. + msg = self.parser( + "Next-Line: not\x85broken\r\n" + "Null: not\x00broken\r\n" + "Vertical-Tab: not\vbroken\r\n" + "Form-Feed: not\fbroken\r\n" + "File-Separator: not\x1Cbroken\r\n" + "Group-Separator: not\x1Dbroken\r\n" + "Record-Separator: not\x1Ebroken\r\n" + "Line-Separator: not\u2028broken\r\n" + "Paragraph-Separator: not\u2029broken\r\n" + "\r\n", + policy=default, + ) + self.assertEqual(msg.items(), [ + ("Next-Line", "not\x85broken"), + ("Null", "not\x00broken"), + ("Vertical-Tab", "not\vbroken"), + ("Form-Feed", "not\fbroken"), + ("File-Separator", "not\x1Cbroken"), + ("Group-Separator", "not\x1Dbroken"), + ("Record-Separator", "not\x1Ebroken"), + ("Line-Separator", "not\u2028broken"), + ("Paragraph-Separator", "not\u2029broken"), + ]) + self.assertEqual(msg.get_payload(), "") + +class TestParser(TestParserBase, TestEmailBase): + parser = staticmethod(email.message_from_string) + +class TestBytesParser(TestParserBase, TestEmailBase): + def parser(self, s, *args, **kw): + return email.message_from_bytes(s.encode(), *args, **kw) + + if __name__ == '__main__': unittest.main() diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -283,6 +283,36 @@ self.assertEqual(resp.getheader('First'), 'val') self.assertEqual(resp.getheader('Second'), 'val') + def test_parse_all_octets(self): + # Ensure no valid header field octet breaks the parser + body = ( + b'HTTP/1.1 200 OK\r\n' + b"!#$%&'*+-.^_`|~: value\r\n" # Special token characters + b'VCHAR: ' + bytes(range(0x21, 0x7E + 1)) + b'\r\n' + b'obs-text: ' + bytes(range(0x80, 0xFF + 1)) + b'\r\n' + b'obs-fold: text\r\n' + b' folded with space\r\n' + b'\tfolded with tab\r\n' + b'Content-Length: 0\r\n' + b'\r\n' + ) + sock = FakeSocket(body) + resp = client.HTTPResponse(sock) + resp.begin() + self.assertEqual(resp.getheader('Content-Length'), '0') + self.assertEqual(resp.msg['Content-Length'], '0') + self.assertEqual(resp.getheader("!#$%&'*+-.^_`|~"), 'value') + self.assertEqual(resp.msg["!#$%&'*+-.^_`|~"], 'value') + vchar = ''.join(map(chr, range(0x21, 0x7E + 1))) + self.assertEqual(resp.getheader('VCHAR'), vchar) + self.assertEqual(resp.msg['VCHAR'], vchar) + self.assertIsNotNone(resp.getheader('obs-text')) + self.assertIn('obs-text', resp.msg) + for folded in (resp.getheader('obs-fold'), resp.msg['obs-fold']): + self.assertTrue(folded.startswith('text')) + self.assertIn(' folded with space', folded) + self.assertTrue(folded.endswith('folded with tab')) + def test_invalid_headers(self): conn = client.HTTPConnection('example.com') conn.sock = FakeSocket('') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -91,6 +91,10 @@ Library ------- +- Issue #22233: Break email header lines *only* on the RFC specified CR and LF + characters, not on arbitrary unicode line breaks. This also fixes a bug in + HTTP header parsing. + - Issue 27331: The email.mime classes now all accept an optional policy keyword. - Issue 27988: Fix email iter_attachments incorrect mutation of payload list. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 17:48:43 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 21:48:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_more_linux_-=3E_=5F=5Flinu?= =?utf-8?b?eF9f?= Message-ID: <20160907214829.3474.44970.DA5A08D1@psf.io> https://hg.python.org/cpython/rev/33c398a9a6fb changeset: 103251:33c398a9a6fb user: Benjamin Peterson date: Wed Sep 07 14:45:10 2016 -0700 summary: more linux -> __linux__ files: Modules/_ctypes/libffi/src/dlmalloc.c | 2 +- Modules/ossaudiodev.c | 2 +- Modules/posixmodule.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Modules/_ctypes/libffi/src/dlmalloc.c b/Modules/_ctypes/libffi/src/dlmalloc.c --- a/Modules/_ctypes/libffi/src/dlmalloc.c +++ b/Modules/_ctypes/libffi/src/dlmalloc.c @@ -525,7 +525,7 @@ #define MMAP_CLEARS 1 #endif /* MMAP_CLEARS */ #ifndef HAVE_MREMAP -#ifdef linux +#ifdef __linux__ #define HAVE_MREMAP 1 #else /* linux */ #define HAVE_MREMAP 0 diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c --- a/Modules/ossaudiodev.c +++ b/Modules/ossaudiodev.c @@ -37,7 +37,7 @@ #include #endif -#if defined(linux) +#ifdef __linux__ #ifndef HAVE_STDINT_H typedef unsigned long uint32_t; diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -8446,7 +8446,7 @@ if (!PyArg_ParseTupleAndKeywords(args, kwdict, "iiOn:sendfile", keywords, &out, &in, &offobj, &count)) return NULL; -#ifdef linux +#ifdef __linux__ if (offobj == Py_None) { do { Py_BEGIN_ALLOW_THREADS -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 17:56:40 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 21:56:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_expected_layout_of_cod?= =?utf-8?q?e_objects?= Message-ID: <20160907215637.1306.92988.6B6F8E58@psf.io> https://hg.python.org/cpython/rev/c2212d98ef13 changeset: 103252:c2212d98ef13 user: Benjamin Peterson date: Wed Sep 07 14:56:15 2016 -0700 summary: fix expected layout of code objects files: Lib/test/test_sys.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -912,13 +912,13 @@ return inner check(get_cell().__closure__[0], size('P')) # code - check(get_cell().__code__, size('5i9Pi3P')) - check(get_cell.__code__, size('5i9Pi3P')) + check(get_cell().__code__, size('6i13P')) + check(get_cell.__code__, size('6i13P')) def get_cell2(x): def inner(): return x return inner - check(get_cell2.__code__, size('5i9Pi3P') + 1) + check(get_cell2.__code__, size('6i13P') + 1) # complex check(complex(0,1), size('2d')) # method_descriptor (descriptor object) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 18:34:11 2016 From: python-checkins at python.org (benjamin.peterson) Date: Wed, 07 Sep 2016 22:34:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_replace_some_Py=5FLOCAL=5F?= =?utf-8?q?INLINE_with_the_inline_keyword?= Message-ID: <20160907223410.12582.11829.9A267178@psf.io> https://hg.python.org/cpython/rev/da03060d281c changeset: 103253:da03060d281c user: Benjamin Peterson date: Wed Sep 07 15:33:32 2016 -0700 summary: replace some Py_LOCAL_INLINE with the inline keyword files: Objects/unicodeobject.c | 19 ++++++++++--------- 1 files changed, 10 insertions(+), 9 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -204,7 +204,7 @@ } while (0) /* Forward declaration */ -Py_LOCAL_INLINE(int) +static inline int _PyUnicodeWriter_WriteCharInline(_PyUnicodeWriter *writer, Py_UCS4 ch); /* List of static strings. */ @@ -720,7 +720,7 @@ ((ch) < 128U ? ascii_linebreak[(ch)] : \ (BLOOM(bloom_linebreak, (ch)) && Py_UNICODE_ISLINEBREAK(ch))) -Py_LOCAL_INLINE(BLOOM_MASK) +static inline BLOOM_MASK make_bloom_mask(int kind, void* ptr, Py_ssize_t len) { #define BLOOM_UPDATE(TYPE, MASK, PTR, LEN) \ @@ -826,9 +826,10 @@ static PyObject * fixup(PyObject *self, Py_UCS4 (*fixfct)(PyObject *s)); -Py_LOCAL_INLINE(Py_ssize_t) findchar(const void *s, int kind, - Py_ssize_t size, Py_UCS4 ch, - int direction) +static inline Py_ssize_t +findchar(const void *s, int kind, + Py_ssize_t size, Py_UCS4 ch, + int direction) { switch (kind) { case PyUnicode_1BYTE_KIND: @@ -2138,7 +2139,7 @@ } } -Py_LOCAL_INLINE(Py_UCS4) +static inline Py_UCS4 align_maxchar(Py_UCS4 maxchar) { if (maxchar <= 127) @@ -11290,7 +11291,7 @@ first argument is a unicode object. */ -Py_LOCAL_INLINE(int) +static inline int parse_args_finds_unicode(const char * function_name, PyObject *args, PyObject **substring, Py_ssize_t *start, Py_ssize_t *end) @@ -13267,7 +13268,7 @@ return PyBool_FromLong(result); } -Py_LOCAL_INLINE(void) +static inline void _PyUnicodeWriter_Update(_PyUnicodeWriter *writer) { writer->maxchar = PyUnicode_MAX_CHAR_VALUE(writer->buffer); @@ -13403,7 +13404,7 @@ return _PyUnicodeWriter_PrepareInternal(writer, 0, maxchar); } -Py_LOCAL_INLINE(int) +static inline int _PyUnicodeWriter_WriteCharInline(_PyUnicodeWriter *writer, Py_UCS4 ch) { assert(ch <= MAX_UNICODE); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:11:23 2016 From: python-checkins at python.org (gregory.p.smith) Date: Wed, 07 Sep 2016 23:11:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fixes_Issue_=2327983=3A_Cause_lack_of_llvm-profdata_tool?= =?utf-8?q?_when_using_clang_as?= Message-ID: <20160907231113.20970.3392.9A6E40E7@psf.io> https://hg.python.org/cpython/rev/9f2467e13c98 changeset: 103255:9f2467e13c98 parent: 103253:da03060d281c parent: 103254:279450d88fb0 user: Gregory P. Smith date: Wed Sep 07 16:11:08 2016 -0700 summary: Fixes Issue #27983: Cause lack of llvm-profdata tool when using clang as required for PGO linking to be a configure time error rather than make time when --with-optimizations is enabled. Also improve our ability to find the llvm-profdata tool on MacOS and some Linuxes. files: Misc/NEWS | 5 + configure | 202 +++++++++++++++++++++++++++++++++----- configure.ac | 55 +++++++++- 3 files changed, 230 insertions(+), 32 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,11 @@ Core and Builtins ----------------- +- Issue #27983: Cause lack of llvm-profdata tool when using clang as + required for PGO linking to be a configure time error rather than + make time when --with-optimizations is enabled. Also improve our + ability to find the llvm-profdata tool on MacOS and some Linuxes. + - Issue #26307: The profile-opt build now applys PGO to the built-in modules. - Issue #27078: Added BUILD_STRING opcode. Optimized f-strings evaluation. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -668,6 +668,11 @@ BASECFLAGS OPT LLVM_PROF_FOUND +target_os +target_vendor +target_cpu +target +LLVM_PROFDATA LLVM_PROF_ERR LLVM_PROF_FILE LLVM_PROF_MERGER @@ -1462,6 +1467,7 @@ System types: --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] + --target=TARGET configure for building compilers for TARGET [HOST] _ACEOF fi @@ -6497,9 +6503,11 @@ ;; esac DEF_MAKE_ALL_RULE="profile-opt" + REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" else DEF_MAKE_ALL_RULE="build_all" + REQUIRE_PGO="no" DEF_MAKE_RULE="all" fi @@ -6558,25 +6566,84 @@ - -# Extract the first word of "llvm-profdata", so it can be a program name with args. -set dummy llvm-profdata; ac_word=$2 +# Make this work on systems where llvm tools are not installed with their +# normal names in the default $PATH (ie: Ubuntu). They exist under the +# non-suffixed name in their versioned llvm directory. +llvm_bin_dir='' +llvm_path="${PATH}" +if test "${CC}" = "clang" +then + clang_bin=`which clang` + # Some systems install clang elsewhere as a symlink to the real path + # which is where the related llvm tools are located. + if test -L "${clang_bin}" + then + clang_dir=`dirname "${clang_bin}"` + clang_bin=`readlink "${clang_bin}"` + llvm_bin_dir="${clang_dir}/"`dirname "${clang_bin}"` + llvm_path="${llvm_path}${PATH_SEPARATOR}${llvm_bin_dir}" + fi +fi + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking target system type" >&5 +$as_echo_n "checking target system type... " >&6; } +if ${ac_cv_target+:} false; then : + $as_echo_n "(cached) " >&6 +else + if test "x$target_alias" = x; then + ac_cv_target=$ac_cv_host +else + ac_cv_target=`$SHELL "$ac_aux_dir/config.sub" $target_alias` || + as_fn_error $? "$SHELL $ac_aux_dir/config.sub $target_alias failed" "$LINENO" 5 +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_target" >&5 +$as_echo "$ac_cv_target" >&6; } +case $ac_cv_target in +*-*-*) ;; +*) as_fn_error $? "invalid value of canonical target" "$LINENO" 5;; +esac +target=$ac_cv_target +ac_save_IFS=$IFS; IFS='-' +set x $ac_cv_target +shift +target_cpu=$1 +target_vendor=$2 +shift; shift +# Remember, the first character of IFS is used to create $*, +# except with old shells: +target_os=$* +IFS=$ac_save_IFS +case $target_os in *\ *) target_os=`echo "$target_os" | sed 's/ /-/g'`;; esac + + +# The aliases save the names the user supplied, while $host etc. +# will get canonicalized. +test -n "$target_alias" && + test "$program_prefix$program_suffix$program_transform_name" = \ + NONENONEs,x,x, && + program_prefix=${target_alias}- +# Extract the first word of "$target_alias-llvm-profdata", so it can be a program name with args. +set dummy $target_alias-llvm-profdata; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_LLVM_PROF_FOUND+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$LLVM_PROF_FOUND"; then - ac_cv_prog_LLVM_PROF_FOUND="$LLVM_PROF_FOUND" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH +if ${ac_cv_path_LLVM_PROFDATA+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $LLVM_PROFDATA in + [\\/]* | ?:[\\/]*) + ac_cv_path_LLVM_PROFDATA="$LLVM_PROFDATA" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in ${llvm_path} do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_LLVM_PROF_FOUND="found" + ac_cv_path_LLVM_PROFDATA="$as_dir/$ac_word$ac_exec_ext" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi @@ -6584,30 +6651,105 @@ done IFS=$as_save_IFS - test -z "$ac_cv_prog_LLVM_PROF_FOUND" && ac_cv_prog_LLVM_PROF_FOUND="not-found" -fi -fi -LLVM_PROF_FOUND=$ac_cv_prog_LLVM_PROF_FOUND -if test -n "$LLVM_PROF_FOUND"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LLVM_PROF_FOUND" >&5 -$as_echo "$LLVM_PROF_FOUND" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - + ;; +esac +fi +LLVM_PROFDATA=$ac_cv_path_LLVM_PROFDATA +if test -n "$LLVM_PROFDATA"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LLVM_PROFDATA" >&5 +$as_echo "$LLVM_PROFDATA" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +if test -z "$ac_cv_path_LLVM_PROFDATA"; then + if test "$build" = "$target"; then + ac_pt_LLVM_PROFDATA=$LLVM_PROFDATA + # Extract the first word of "llvm-profdata", so it can be a program name with args. +set dummy llvm-profdata; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_LLVM_PROFDATA+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_LLVM_PROFDATA in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_LLVM_PROFDATA="$ac_pt_LLVM_PROFDATA" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in ${llvm_path} +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_LLVM_PROFDATA="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + test -z "$ac_cv_path_ac_pt_LLVM_PROFDATA" && ac_cv_path_ac_pt_LLVM_PROFDATA="''" + ;; +esac +fi +ac_pt_LLVM_PROFDATA=$ac_cv_path_ac_pt_LLVM_PROFDATA +if test -n "$ac_pt_LLVM_PROFDATA"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_LLVM_PROFDATA" >&5 +$as_echo "$ac_pt_LLVM_PROFDATA" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + LLVM_PROFDATA=$ac_pt_LLVM_PROFDATA + else + LLVM_PROFDATA="''" + fi +else + LLVM_PROFDATA="$ac_cv_path_LLVM_PROFDATA" +fi + + +if test -n "${LLVM_PROFDATA}" -a -x "${LLVM_PROFDATA}" +then + LLVM_PROF_FOUND="found" +else + LLVM_PROF_FOUND="not-found" +fi +if test "$ac_sys_system" = "Darwin" -a "${LLVM_PROF_FOUND}" = "not-found" +then + found_llvm_profdata=`/usr/bin/xcrun -find llvm-profdata 2>/dev/null` + if test -n "${found_llvm_profdata}" + then + # llvm-profdata isn't directly in $PATH in some cases. + # https://apple.stackexchange.com/questions/197053/ + LLVM_PROFDATA='/usr/bin/xcrun llvm-profdata' + LLVM_PROF_FOUND=found + { $as_echo "$as_me:${as_lineno-$LINENO}: llvm-profdata found via xcrun: ${LLVM_PROFDATA}" >&5 +$as_echo "$as_me: llvm-profdata found via xcrun: ${LLVM_PROFDATA}" >&6;} + fi +fi LLVM_PROF_ERR=no case $CC in *clang*) # Any changes made here should be reflected in the GCC+Darwin case below PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" if test $LLVM_PROF_FOUND = not-found then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5 + fi fi ;; *gcc*) @@ -6615,11 +6757,15 @@ Darwin*) PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" - if test $LLVM_PROF_FOUND = not-found + if test "${LLVM_PROF_FOUND}" = "not-found" then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5 + fi fi ;; *) diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1307,9 +1307,11 @@ ;; esac DEF_MAKE_ALL_RULE="profile-opt" + REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" else DEF_MAKE_ALL_RULE="build_all" + REQUIRE_PGO="no" DEF_MAKE_RULE="all" fi @@ -1359,19 +1361,60 @@ AC_SUBST(LLVM_PROF_MERGER) AC_SUBST(LLVM_PROF_FILE) AC_SUBST(LLVM_PROF_ERR) +# Make this work on systems where llvm tools are not installed with their +# normal names in the default $PATH (ie: Ubuntu). They exist under the +# non-suffixed name in their versioned llvm directory. +llvm_bin_dir='' +llvm_path="${PATH}" +if test "${CC}" = "clang" +then + clang_bin=`which clang` + # Some systems install clang elsewhere as a symlink to the real path + # which is where the related llvm tools are located. + if test -L "${clang_bin}" + then + clang_dir=`dirname "${clang_bin}"` + clang_bin=`readlink "${clang_bin}"` + llvm_bin_dir="${clang_dir}/"`dirname "${clang_bin}"` + llvm_path="${llvm_path}${PATH_SEPARATOR}${llvm_bin_dir}" + fi +fi +AC_SUBST(LLVM_PROFDATA) +AC_PATH_TARGET_TOOL(LLVM_PROFDATA, llvm-profdata, '', ${llvm_path}) AC_SUBST(LLVM_PROF_FOUND) -AC_CHECK_PROG(LLVM_PROF_FOUND, llvm-profdata, found, not-found) +if test -n "${LLVM_PROFDATA}" -a -x "${LLVM_PROFDATA}" +then + LLVM_PROF_FOUND="found" +else + LLVM_PROF_FOUND="not-found" +fi +if test "$ac_sys_system" = "Darwin" -a "${LLVM_PROF_FOUND}" = "not-found" +then + found_llvm_profdata=`/usr/bin/xcrun -find llvm-profdata 2>/dev/null` + if test -n "${found_llvm_profdata}" + then + # llvm-profdata isn't directly in $PATH in some cases. + # https://apple.stackexchange.com/questions/197053/ + LLVM_PROFDATA='/usr/bin/xcrun llvm-profdata' + LLVM_PROF_FOUND=found + AC_MSG_NOTICE([llvm-profdata found via xcrun: ${LLVM_PROFDATA}]) + fi +fi LLVM_PROF_ERR=no case $CC in *clang*) # Any changes made here should be reflected in the GCC+Darwin case below PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" if test $LLVM_PROF_FOUND = not-found then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.]) + fi fi ;; *gcc*) @@ -1379,11 +1422,15 @@ Darwin*) PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" - if test $LLVM_PROF_FOUND = not-found + if test "${LLVM_PROF_FOUND}" = "not-found" then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.]) + fi fi ;; *) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:11:22 2016 From: python-checkins at python.org (gregory.p.smith) Date: Wed, 07 Sep 2016 23:11:22 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogRml4ZXMgaXNzdWUj?= =?utf-8?q?_27983=3A_Cause_lack_of_llvm-profdata_tool_when_using_clang_-?= Message-ID: <20160907231113.12852.79952.0E6254CE@psf.io> https://hg.python.org/cpython/rev/279450d88fb0 changeset: 103254:279450d88fb0 branch: 3.5 parent: 103249:69900c5992c5 user: Gregory P. Smith date: Wed Sep 07 16:10:00 2016 -0700 summary: Fixes issue# 27983: Cause lack of llvm-profdata tool when using clang - required for PGO linking - to be a configure time error rather than make time when --with-optimizations is enabled. Also improve our ability to find the llvm-profdata tool on MacOS and some Linuxes. files: Misc/NEWS | 5 + configure | 216 +++++++++++++++++++++++++++++++++----- configure.ac | 55 +++++++++- 3 files changed, 243 insertions(+), 33 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,11 @@ Core and Builtins ----------------- +- Issue #27983: Cause lack of llvm-profdata tool when using clang as + required for PGO linking to be a configure time error rather than + make time when --with-optimizations is enabled. Also improve our + ability to find the llvm-profdata tool on MacOS and some Linuxes. + - Issue #26307: The profile-opt build now applys PGO to the built-in modules. - Issue #27812: Properly clear out a generator's frame's backreference to the diff --git a/configure b/configure --- a/configure +++ b/configure @@ -668,6 +668,11 @@ BASECFLAGS OPT LLVM_PROF_FOUND +target_os +target_vendor +target_cpu +target +LLVM_PROFDATA LLVM_PROF_ERR LLVM_PROF_FILE LLVM_PROF_MERGER @@ -776,6 +781,7 @@ docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -887,6 +893,7 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1139,6 +1146,15 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1276,7 +1292,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1429,6 +1445,7 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -1449,6 +1466,7 @@ System types: --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] + --target=TARGET configure for building compilers for TARGET [HOST] _ACEOF fi @@ -6555,9 +6573,11 @@ ;; esac DEF_MAKE_ALL_RULE="profile-opt" + REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" else DEF_MAKE_ALL_RULE="build_all" + REQUIRE_PGO="no" DEF_MAKE_RULE="all" fi @@ -6609,25 +6629,84 @@ - -# Extract the first word of "llvm-profdata", so it can be a program name with args. -set dummy llvm-profdata; ac_word=$2 +# Make this work on systems where llvm tools are not installed with their +# normal names in the default $PATH (ie: Ubuntu). They exist under the +# non-suffixed name in their versioned llvm directory. +llvm_bin_dir='' +llvm_path="${PATH}" +if test "${CC}" = "clang" +then + clang_bin=`which clang` + # Some systems install clang elsewhere as a symlink to the real path + # which is where the related llvm tools are located. + if test -L "${clang_bin}" + then + clang_dir=`dirname "${clang_bin}"` + clang_bin=`readlink "${clang_bin}"` + llvm_bin_dir="${clang_dir}/"`dirname "${clang_bin}"` + llvm_path="${llvm_path}${PATH_SEPARATOR}${llvm_bin_dir}" + fi +fi + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking target system type" >&5 +$as_echo_n "checking target system type... " >&6; } +if ${ac_cv_target+:} false; then : + $as_echo_n "(cached) " >&6 +else + if test "x$target_alias" = x; then + ac_cv_target=$ac_cv_host +else + ac_cv_target=`$SHELL "$ac_aux_dir/config.sub" $target_alias` || + as_fn_error $? "$SHELL $ac_aux_dir/config.sub $target_alias failed" "$LINENO" 5 +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_target" >&5 +$as_echo "$ac_cv_target" >&6; } +case $ac_cv_target in +*-*-*) ;; +*) as_fn_error $? "invalid value of canonical target" "$LINENO" 5;; +esac +target=$ac_cv_target +ac_save_IFS=$IFS; IFS='-' +set x $ac_cv_target +shift +target_cpu=$1 +target_vendor=$2 +shift; shift +# Remember, the first character of IFS is used to create $*, +# except with old shells: +target_os=$* +IFS=$ac_save_IFS +case $target_os in *\ *) target_os=`echo "$target_os" | sed 's/ /-/g'`;; esac + + +# The aliases save the names the user supplied, while $host etc. +# will get canonicalized. +test -n "$target_alias" && + test "$program_prefix$program_suffix$program_transform_name" = \ + NONENONEs,x,x, && + program_prefix=${target_alias}- +# Extract the first word of "$target_alias-llvm-profdata", so it can be a program name with args. +set dummy $target_alias-llvm-profdata; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_LLVM_PROF_FOUND+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$LLVM_PROF_FOUND"; then - ac_cv_prog_LLVM_PROF_FOUND="$LLVM_PROF_FOUND" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH +if ${ac_cv_path_LLVM_PROFDATA+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $LLVM_PROFDATA in + [\\/]* | ?:[\\/]*) + ac_cv_path_LLVM_PROFDATA="$LLVM_PROFDATA" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in ${llvm_path} do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_LLVM_PROF_FOUND="found" + ac_cv_path_LLVM_PROFDATA="$as_dir/$ac_word$ac_exec_ext" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi @@ -6635,30 +6714,105 @@ done IFS=$as_save_IFS - test -z "$ac_cv_prog_LLVM_PROF_FOUND" && ac_cv_prog_LLVM_PROF_FOUND="not-found" -fi -fi -LLVM_PROF_FOUND=$ac_cv_prog_LLVM_PROF_FOUND -if test -n "$LLVM_PROF_FOUND"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LLVM_PROF_FOUND" >&5 -$as_echo "$LLVM_PROF_FOUND" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - + ;; +esac +fi +LLVM_PROFDATA=$ac_cv_path_LLVM_PROFDATA +if test -n "$LLVM_PROFDATA"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LLVM_PROFDATA" >&5 +$as_echo "$LLVM_PROFDATA" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +if test -z "$ac_cv_path_LLVM_PROFDATA"; then + if test "$build" = "$target"; then + ac_pt_LLVM_PROFDATA=$LLVM_PROFDATA + # Extract the first word of "llvm-profdata", so it can be a program name with args. +set dummy llvm-profdata; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_LLVM_PROFDATA+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_LLVM_PROFDATA in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_LLVM_PROFDATA="$ac_pt_LLVM_PROFDATA" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in ${llvm_path} +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_LLVM_PROFDATA="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + test -z "$ac_cv_path_ac_pt_LLVM_PROFDATA" && ac_cv_path_ac_pt_LLVM_PROFDATA="''" + ;; +esac +fi +ac_pt_LLVM_PROFDATA=$ac_cv_path_ac_pt_LLVM_PROFDATA +if test -n "$ac_pt_LLVM_PROFDATA"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_LLVM_PROFDATA" >&5 +$as_echo "$ac_pt_LLVM_PROFDATA" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + LLVM_PROFDATA=$ac_pt_LLVM_PROFDATA + else + LLVM_PROFDATA="''" + fi +else + LLVM_PROFDATA="$ac_cv_path_LLVM_PROFDATA" +fi + + +if test -n "${LLVM_PROFDATA}" -a -x "${LLVM_PROFDATA}" +then + LLVM_PROF_FOUND="found" +else + LLVM_PROF_FOUND="not-found" +fi +if test "$ac_sys_system" = "Darwin" -a "${LLVM_PROF_FOUND}" = "not-found" +then + found_llvm_profdata=`/usr/bin/xcrun -find llvm-profdata 2>/dev/null` + if test -n "${found_llvm_profdata}" + then + # llvm-profdata isn't directly in $PATH in some cases. + # https://apple.stackexchange.com/questions/197053/ + LLVM_PROFDATA='/usr/bin/xcrun llvm-profdata' + LLVM_PROF_FOUND=found + { $as_echo "$as_me:${as_lineno-$LINENO}: llvm-profdata found via xcrun: ${LLVM_PROFDATA}" >&5 +$as_echo "$as_me: llvm-profdata found via xcrun: ${LLVM_PROFDATA}" >&6;} + fi +fi LLVM_PROF_ERR=no case $CC in *clang*) # Any changes made here should be reflected in the GCC+Darwin case below PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" if test $LLVM_PROF_FOUND = not-found then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5 + fi fi ;; *gcc*) @@ -6666,11 +6820,15 @@ Darwin*) PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" - if test $LLVM_PROF_FOUND = not-found + if test "${LLVM_PROF_FOUND}" = "not-found" then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5 + fi fi ;; *) diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1255,9 +1255,11 @@ ;; esac DEF_MAKE_ALL_RULE="profile-opt" + REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" else DEF_MAKE_ALL_RULE="build_all" + REQUIRE_PGO="no" DEF_MAKE_RULE="all" fi @@ -1300,19 +1302,60 @@ AC_SUBST(LLVM_PROF_MERGER) AC_SUBST(LLVM_PROF_FILE) AC_SUBST(LLVM_PROF_ERR) +# Make this work on systems where llvm tools are not installed with their +# normal names in the default $PATH (ie: Ubuntu). They exist under the +# non-suffixed name in their versioned llvm directory. +llvm_bin_dir='' +llvm_path="${PATH}" +if test "${CC}" = "clang" +then + clang_bin=`which clang` + # Some systems install clang elsewhere as a symlink to the real path + # which is where the related llvm tools are located. + if test -L "${clang_bin}" + then + clang_dir=`dirname "${clang_bin}"` + clang_bin=`readlink "${clang_bin}"` + llvm_bin_dir="${clang_dir}/"`dirname "${clang_bin}"` + llvm_path="${llvm_path}${PATH_SEPARATOR}${llvm_bin_dir}" + fi +fi +AC_SUBST(LLVM_PROFDATA) +AC_PATH_TARGET_TOOL(LLVM_PROFDATA, llvm-profdata, '', ${llvm_path}) AC_SUBST(LLVM_PROF_FOUND) -AC_CHECK_PROG(LLVM_PROF_FOUND, llvm-profdata, found, not-found) +if test -n "${LLVM_PROFDATA}" -a -x "${LLVM_PROFDATA}" +then + LLVM_PROF_FOUND="found" +else + LLVM_PROF_FOUND="not-found" +fi +if test "$ac_sys_system" = "Darwin" -a "${LLVM_PROF_FOUND}" = "not-found" +then + found_llvm_profdata=`/usr/bin/xcrun -find llvm-profdata 2>/dev/null` + if test -n "${found_llvm_profdata}" + then + # llvm-profdata isn't directly in $PATH in some cases. + # https://apple.stackexchange.com/questions/197053/ + LLVM_PROFDATA='/usr/bin/xcrun llvm-profdata' + LLVM_PROF_FOUND=found + AC_MSG_NOTICE([llvm-profdata found via xcrun: ${LLVM_PROFDATA}]) + fi +fi LLVM_PROF_ERR=no case $CC in *clang*) # Any changes made here should be reflected in the GCC+Darwin case below PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" if test $LLVM_PROF_FOUND = not-found then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.]) + fi fi ;; *gcc*) @@ -1320,11 +1363,15 @@ Darwin*) PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" - if test $LLVM_PROF_FOUND = not-found + if test "${LLVM_PROF_FOUND}" = "not-found" then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.]) + fi fi ;; *) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:42:47 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 07 Sep 2016 23:42:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NTcw?= =?utf-8?q?=3A_Avoid_zero-length_memcpy=28=29_calls_with_null_source_point?= =?utf-8?q?ers?= Message-ID: <20160907234247.48584.54122.57E0D459@psf.io> https://hg.python.org/cpython/rev/e231dcad3a9b changeset: 103256:e231dcad3a9b branch: 3.5 parent: 103254:279450d88fb0 user: Martin Panter date: Wed Sep 07 11:04:41 2016 +0000 summary: Issue #27570: Avoid zero-length memcpy() calls with null source pointers files: Lib/test/test_array.py | 14 ++++++++++++-- Misc/NEWS | 3 +++ Modules/_ctypes/_ctypes.c | 6 ++++-- Modules/_ctypes/stgdict.c | 8 +++++--- Modules/arraymodule.c | 24 ++++++++++++++++-------- 5 files changed, 40 insertions(+), 15 deletions(-) diff --git a/Lib/test/test_array.py b/Lib/test/test_array.py --- a/Lib/test/test_array.py +++ b/Lib/test/test_array.py @@ -38,14 +38,24 @@ if have_long_long: typecodes += 'qQ' -class BadConstructorTest(unittest.TestCase): +class MiscTest(unittest.TestCase): - def test_constructor(self): + def test_bad_constructor(self): self.assertRaises(TypeError, array.array) self.assertRaises(TypeError, array.array, spam=42) self.assertRaises(TypeError, array.array, 'xx') self.assertRaises(ValueError, array.array, 'x') + def test_empty(self): + # Exercise code for handling zero-length arrays + a = array.array('B') + a[:] = a + self.assertEqual(len(a), 0) + self.assertEqual(len(a + a), 0) + self.assertEqual(len(a * 3), 0) + a += a + self.assertEqual(len(a), 0) + # Machine format codes. # diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -67,6 +67,9 @@ Library ------- +- Issue #27570: Avoid zero-length memcpy() etc calls with null source + pointers in the "ctypes" and "array" modules. + - Issue #22233: Break email header lines *only* on the RFC specified CR and LF characters, not on arbitrary unicode line breaks. This also fixes a bug in HTTP header parsing. diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -1381,8 +1381,10 @@ goto error; } stgdict->shape[0] = length; - memmove(&stgdict->shape[1], itemdict->shape, - sizeof(Py_ssize_t) * (stgdict->ndim - 1)); + if (stgdict->ndim > 1) { + memmove(&stgdict->shape[1], itemdict->shape, + sizeof(Py_ssize_t) * (stgdict->ndim - 1)); + } itemsize = itemdict->size; if (length * itemsize < 0) { diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c --- a/Modules/_ctypes/stgdict.c +++ b/Modules/_ctypes/stgdict.c @@ -391,9 +391,11 @@ } memset(stgdict->ffi_type_pointer.elements, 0, sizeof(ffi_type *) * (basedict->length + len + 1)); - memcpy(stgdict->ffi_type_pointer.elements, - basedict->ffi_type_pointer.elements, - sizeof(ffi_type *) * (basedict->length)); + if (basedict->length > 0) { + memcpy(stgdict->ffi_type_pointer.elements, + basedict->ffi_type_pointer.elements, + sizeof(ffi_type *) * (basedict->length)); + } ffi_ofs = basedict->length; } else { offset = 0; diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -745,8 +745,10 @@ np = (arrayobject *) newarrayobject(&Arraytype, ihigh - ilow, a->ob_descr); if (np == NULL) return NULL; - memcpy(np->ob_item, a->ob_item + ilow * a->ob_descr->itemsize, - (ihigh-ilow) * a->ob_descr->itemsize); + if (ihigh > ilow) { + memcpy(np->ob_item, a->ob_item + ilow * a->ob_descr->itemsize, + (ihigh-ilow) * a->ob_descr->itemsize); + } return (PyObject *)np; } @@ -804,9 +806,13 @@ if (np == NULL) { return NULL; } - memcpy(np->ob_item, a->ob_item, Py_SIZE(a)*a->ob_descr->itemsize); - memcpy(np->ob_item + Py_SIZE(a)*a->ob_descr->itemsize, - b->ob_item, Py_SIZE(b)*b->ob_descr->itemsize); + if (Py_SIZE(a) > 0) { + memcpy(np->ob_item, a->ob_item, Py_SIZE(a)*a->ob_descr->itemsize); + } + if (Py_SIZE(b) > 0) { + memcpy(np->ob_item + Py_SIZE(a)*a->ob_descr->itemsize, + b->ob_item, Py_SIZE(b)*b->ob_descr->itemsize); + } return (PyObject *)np; #undef b } @@ -826,7 +832,7 @@ np = (arrayobject *) newarrayobject(&Arraytype, size, a->ob_descr); if (np == NULL) return NULL; - if (n == 0) + if (size == 0) return (PyObject *)np; oldbytes = Py_SIZE(a) * a->ob_descr->itemsize; newbytes = oldbytes * n; @@ -985,8 +991,10 @@ size = oldsize + Py_SIZE(b); if (array_resize(self, size) == -1) return -1; - memcpy(self->ob_item + oldsize * self->ob_descr->itemsize, - b->ob_item, bbsize * b->ob_descr->itemsize); + if (bbsize > 0) { + memcpy(self->ob_item + oldsize * self->ob_descr->itemsize, + b->ob_item, bbsize * b->ob_descr->itemsize); + } return 0; #undef b -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:42:48 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 07 Sep 2016 23:42:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTkz?= =?utf-8?q?=3A_Fix_problems_with_plural_objects_in_docs_and_comments?= Message-ID: <20160907234248.13046.12333.6B814943@psf.io> https://hg.python.org/cpython/rev/8f02a1a6b647 changeset: 103258:8f02a1a6b647 branch: 3.5 user: Martin Panter date: Wed Sep 07 11:04:41 2016 +0000 summary: Issue #27993: Fix problems with plural objects in docs and comments files: Doc/library/argparse.rst | 4 ++-- Doc/library/weakref.rst | 2 +- Doc/library/xml.dom.rst | 2 +- Lib/inspect.py | 5 ++--- Lib/test/test_json/test_decode.py | 2 +- Misc/HISTORY | 2 +- Misc/NEWS | 4 ++-- 7 files changed, 10 insertions(+), 11 deletions(-) diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -188,7 +188,7 @@ prog ^^^^ -By default, :class:`ArgumentParser` objects uses ``sys.argv[0]`` to determine +By default, :class:`ArgumentParser` objects use ``sys.argv[0]`` to determine how to display the name of the program in help messages. This default is almost always desirable because it will make the help messages match how the program was invoked on the command line. For example, consider a file named @@ -558,7 +558,7 @@ ^^^^^^^^^^^^^^^^ :class:`ArgumentParser` objects do not allow two actions with the same option -string. By default, :class:`ArgumentParser` objects raises an exception if an +string. By default, :class:`ArgumentParser` objects raise an exception if an attempt is made to create an argument with an option string that is already in use:: diff --git a/Doc/library/weakref.rst b/Doc/library/weakref.rst --- a/Doc/library/weakref.rst +++ b/Doc/library/weakref.rst @@ -414,7 +414,7 @@ Example ------- -This simple example shows how an application can use objects IDs to retrieve +This simple example shows how an application can use object IDs to retrieve objects that it has seen before. The IDs of the objects can then be used in other data structures without forcing the objects to remain alive, but the objects can still be retrieved by ID if they do. diff --git a/Doc/library/xml.dom.rst b/Doc/library/xml.dom.rst --- a/Doc/library/xml.dom.rst +++ b/Doc/library/xml.dom.rst @@ -404,7 +404,7 @@ ^^^^^^^^^^^^^^^^ A :class:`NodeList` represents a sequence of nodes. These objects are used in -two ways in the DOM Core recommendation: the :class:`Element` objects provides +two ways in the DOM Core recommendation: an :class:`Element` object provides one as its list of child nodes, and the :meth:`getElementsByTagName` and :meth:`getElementsByTagNameNS` methods of :class:`Node` return objects with this interface to represent query results. diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -171,9 +171,8 @@ def isgeneratorfunction(object): """Return true if the object is a user-defined generator function. - Generator function objects provides same attributes as functions. - - See help(isfunction) for attributes listing.""" + Generator function objects provide the same attributes as functions. + See help(isfunction) for a list of attributes.""" return bool((isfunction(object) or ismethod(object)) and object.__code__.co_flags & CO_GENERATOR) diff --git a/Lib/test/test_json/test_decode.py b/Lib/test/test_json/test_decode.py --- a/Lib/test/test_json/test_decode.py +++ b/Lib/test/test_json/test_decode.py @@ -35,7 +35,7 @@ self.assertEqual(self.loads(s, object_pairs_hook=OrderedDict, object_hook=lambda x: None), OrderedDict(p)) - # check that empty objects literals work (see #17368) + # check that empty object literals work (see #17368) self.assertEqual(self.loads('{}', object_pairs_hook=OrderedDict), OrderedDict()) self.assertEqual(self.loads('{"empty": {}}', diff --git a/Misc/HISTORY b/Misc/HISTORY --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -26057,7 +26057,7 @@ module implementing deepcopy and normal (shallow) copy operations. See the library reference manual. -- Documentation strings for many objects types are accessible through +- Documentation strings for many object types are accessible through the __doc__ attribute. Modules, classes and functions support special syntax to initialize the __doc__ attribute: if the first statement consists of just a string literal, that string literal becomes the diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -4771,7 +4771,7 @@ - Issue #20588: Make Python-ast.c C89 compliant. -- Issue #20437: Fixed 22 potential bugs when deleting objects references. +- Issue #20437: Fixed 22 potential bugs when deleting object references. - Issue #20500: Displaying an exception at interpreter shutdown no longer risks triggering an assertion failure in PyObject_Str. @@ -5676,7 +5676,7 @@ when the creation of the replacement exception won't lose any information. - Issue #19466: Clear the frames of daemon threads earlier during the - Python shutdown to call objects destructors. So "unclosed file" resource + Python shutdown to call object destructors. So "unclosed file" resource warnings are now correctly emitted for daemon threads. - Issue #19514: Deduplicate some _Py_IDENTIFIER declarations. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:42:48 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 07 Sep 2016 23:42:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327570=3A_Merge_null_pointer_fixes_from_3=2E5?= Message-ID: <20160907234248.2885.5854.B7A3CB90@psf.io> https://hg.python.org/cpython/rev/2d0fb659372c changeset: 103259:2d0fb659372c parent: 103255:9f2467e13c98 parent: 103256:e231dcad3a9b user: Martin Panter date: Wed Sep 07 23:31:39 2016 +0000 summary: Issue #27570: Merge null pointer fixes from 3.5 files: Lib/test/test_array.py | 14 ++++++++++++-- Misc/NEWS | 3 +++ Modules/_ctypes/_ctypes.c | 6 ++++-- Modules/_ctypes/stgdict.c | 8 +++++--- Modules/arraymodule.c | 24 ++++++++++++++++-------- 5 files changed, 40 insertions(+), 15 deletions(-) diff --git a/Lib/test/test_array.py b/Lib/test/test_array.py --- a/Lib/test/test_array.py +++ b/Lib/test/test_array.py @@ -36,14 +36,24 @@ if have_long_long: typecodes += 'qQ' -class BadConstructorTest(unittest.TestCase): +class MiscTest(unittest.TestCase): - def test_constructor(self): + def test_bad_constructor(self): self.assertRaises(TypeError, array.array) self.assertRaises(TypeError, array.array, spam=42) self.assertRaises(TypeError, array.array, 'xx') self.assertRaises(ValueError, array.array, 'x') + def test_empty(self): + # Exercise code for handling zero-length arrays + a = array.array('B') + a[:] = a + self.assertEqual(len(a), 0) + self.assertEqual(len(a + a), 0) + self.assertEqual(len(a * 3), 0) + a += a + self.assertEqual(len(a), 0) + # Machine format codes. # diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -96,6 +96,9 @@ Library ------- +- Issue #27570: Avoid zero-length memcpy() etc calls with null source + pointers in the "ctypes" and "array" modules. + - Issue #22233: Break email header lines *only* on the RFC specified CR and LF characters, not on arbitrary unicode line breaks. This also fixes a bug in HTTP header parsing. diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -1375,8 +1375,10 @@ goto error; } stgdict->shape[0] = length; - memmove(&stgdict->shape[1], itemdict->shape, - sizeof(Py_ssize_t) * (stgdict->ndim - 1)); + if (stgdict->ndim > 1) { + memmove(&stgdict->shape[1], itemdict->shape, + sizeof(Py_ssize_t) * (stgdict->ndim - 1)); + } itemsize = itemdict->size; if (length * itemsize < 0) { diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c --- a/Modules/_ctypes/stgdict.c +++ b/Modules/_ctypes/stgdict.c @@ -391,9 +391,11 @@ } memset(stgdict->ffi_type_pointer.elements, 0, sizeof(ffi_type *) * (basedict->length + len + 1)); - memcpy(stgdict->ffi_type_pointer.elements, - basedict->ffi_type_pointer.elements, - sizeof(ffi_type *) * (basedict->length)); + if (basedict->length > 0) { + memcpy(stgdict->ffi_type_pointer.elements, + basedict->ffi_type_pointer.elements, + sizeof(ffi_type *) * (basedict->length)); + } ffi_ofs = basedict->length; } else { offset = 0; diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -740,8 +740,10 @@ np = (arrayobject *) newarrayobject(&Arraytype, ihigh - ilow, a->ob_descr); if (np == NULL) return NULL; - memcpy(np->ob_item, a->ob_item + ilow * a->ob_descr->itemsize, - (ihigh-ilow) * a->ob_descr->itemsize); + if (ihigh > ilow) { + memcpy(np->ob_item, a->ob_item + ilow * a->ob_descr->itemsize, + (ihigh-ilow) * a->ob_descr->itemsize); + } return (PyObject *)np; } @@ -799,9 +801,13 @@ if (np == NULL) { return NULL; } - memcpy(np->ob_item, a->ob_item, Py_SIZE(a)*a->ob_descr->itemsize); - memcpy(np->ob_item + Py_SIZE(a)*a->ob_descr->itemsize, - b->ob_item, Py_SIZE(b)*b->ob_descr->itemsize); + if (Py_SIZE(a) > 0) { + memcpy(np->ob_item, a->ob_item, Py_SIZE(a)*a->ob_descr->itemsize); + } + if (Py_SIZE(b) > 0) { + memcpy(np->ob_item + Py_SIZE(a)*a->ob_descr->itemsize, + b->ob_item, Py_SIZE(b)*b->ob_descr->itemsize); + } return (PyObject *)np; #undef b } @@ -821,7 +827,7 @@ np = (arrayobject *) newarrayobject(&Arraytype, size, a->ob_descr); if (np == NULL) return NULL; - if (n == 0) + if (size == 0) return (PyObject *)np; oldbytes = Py_SIZE(a) * a->ob_descr->itemsize; newbytes = oldbytes * n; @@ -942,8 +948,10 @@ size = oldsize + Py_SIZE(b); if (array_resize(self, size) == -1) return -1; - memcpy(self->ob_item + oldsize * self->ob_descr->itemsize, - b->ob_item, bbsize * b->ob_descr->itemsize); + if (bbsize > 0) { + memcpy(self->ob_item + oldsize * self->ob_descr->itemsize, + b->ob_item, bbsize * b->ob_descr->itemsize); + } return 0; #undef b -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:42:48 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 07 Sep 2016 23:42:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3ODk1?= =?utf-8?q?=3A__Spelling_fixes_=28Contributed_by_Ville_Skytt=C3=A4=29=2E?= Message-ID: <20160907234248.36557.38343.A99C37B8@psf.io> https://hg.python.org/cpython/rev/ff3df2b1ac7f changeset: 103257:ff3df2b1ac7f branch: 3.5 user: Martin Panter date: Wed Sep 07 12:03:06 2016 +0000 summary: Issue #27895: Spelling fixes (Contributed by Ville Skytt?). files: Doc/library/email.contentmanager.rst | 2 +- Doc/library/smtpd.rst | 4 +- Doc/whatsnew/3.3.rst | 2 +- Include/abstract.h | 2 +- Include/pymath.h | 2 +- Lib/asyncio/streams.py | 2 +- Lib/concurrent/futures/process.py | 2 +- Lib/concurrent/futures/thread.py | 2 +- Lib/distutils/tests/test_msvc9compiler.py | 2 +- Lib/email/message.py | 2 +- Lib/http/client.py | 2 +- Lib/idlelib/idle_test/test_formatparagraph.py | 2 +- Lib/shutil.py | 2 +- Lib/test/_test_multiprocessing.py | 2 +- Lib/test/datetimetester.py | 2 +- Lib/test/test_asyncio/test_locks.py | 4 +- Lib/test/test_concurrent_futures.py | 2 +- Lib/test/test_descr.py | 2 +- Lib/test/test_email/test_email.py | 4 +- Lib/test/test_importlib/test_util.py | 2 +- Lib/test/test_ipaddress.py | 6 +- Lib/test/test_pep247.py | 2 +- Lib/test/test_subprocess.py | 2 +- Lib/test/test_urllib.py | 2 +- Lib/test/test_winreg.py | 2 +- Lib/unittest/test/test_discovery.py | 2 +- Lib/unittest/test/testmock/testcallable.py | 2 +- Lib/venv/scripts/posix/activate | 2 +- Lib/venv/scripts/posix/activate.csh | 2 +- Lib/venv/scripts/posix/activate.fish | 2 +- Mac/PythonLauncher/MyAppDelegate.m | 2 +- Misc/HISTORY | 22 +++++----- Misc/NEWS | 10 ++-- Modules/_ctypes/ctypes.h | 2 +- Modules/_hashopenssl.c | 2 +- Modules/_io/iobase.c | 2 +- Modules/_testcapimodule.c | 2 +- Modules/_threadmodule.c | 2 +- Modules/_tracemalloc.c | 2 +- Modules/mathmodule.c | 2 +- Modules/socketmodule.c | 2 +- Modules/zipimport.c | 2 +- Objects/bytearrayobject.c | 2 +- Objects/codeobject.c | 2 +- Objects/listsort.txt | 2 +- Objects/longobject.c | 2 +- Objects/typeobject.c | 2 +- Python/condvar.h | 2 +- Python/formatter_unicode.c | 2 +- README | 2 +- configure | 2 +- configure.ac | 2 +- 52 files changed, 71 insertions(+), 71 deletions(-) diff --git a/Doc/library/email.contentmanager.rst b/Doc/library/email.contentmanager.rst --- a/Doc/library/email.contentmanager.rst +++ b/Doc/library/email.contentmanager.rst @@ -433,5 +433,5 @@ If *headers* is specified and is a list of strings of the form ``headername: headervalue`` or a list of ``header`` objects - (distinguised from strings by having a ``name`` attribute), add the + (distinguished from strings by having a ``name`` attribute), add the headers to *msg*. diff --git a/Doc/library/smtpd.rst b/Doc/library/smtpd.rst --- a/Doc/library/smtpd.rst +++ b/Doc/library/smtpd.rst @@ -45,7 +45,7 @@ dictionary is a suitable value). If not specified the :mod:`asyncore` global socket map is used. - *enable_SMTPUTF8* determins whether the ``SMTPUTF8`` extension (as defined + *enable_SMTPUTF8* determines whether the ``SMTPUTF8`` extension (as defined in :RFC:`6531`) should be enabled. The default is ``False``. If set to ``True``, *decode_data* must be ``False`` (otherwise an error is raised). When ``True``, ``SMTPUTF8`` is accepted as a parameter to the ``MAIL`` @@ -162,7 +162,7 @@ accepted in a ``DATA`` command. A value of ``None`` or ``0`` means no limit. - *enable_SMTPUTF8* determins whether the ``SMTPUTF8`` extension (as defined + *enable_SMTPUTF8* determines whether the ``SMTPUTF8`` extension (as defined in :RFC:`6531`) should be enabled. The default is ``False``. A :exc:`ValueError` is raised if both *enable_SMTPUTF8* and *decode_data* are set to ``True`` at the same time. diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -1954,7 +1954,7 @@ :attr:`~ssl.OP_NO_COMPRESSION` can be used to disable compression. (Contributed by Antoine Pitrou in :issue:`13634`.) -* Support has been added for the Next Procotol Negotiation extension using +* Support has been added for the Next Protocol Negotiation extension using the :meth:`ssl.SSLContext.set_npn_protocols` method. (Contributed by Colin Marc in :issue:`14204`.) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -458,7 +458,7 @@ /* old buffer API FIXME: usage of these should all be replaced in Python itself but for backwards compatibility we will implement them. - Their usage without a corresponding "unlock" mechansim + Their usage without a corresponding "unlock" mechanism may create issues (but they would already be there). */ PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj, diff --git a/Include/pymath.h b/Include/pymath.h --- a/Include/pymath.h +++ b/Include/pymath.h @@ -37,7 +37,7 @@ #endif /* __STDC__ */ #endif /* _MSC_VER */ -/* High precision defintion of pi and e (Euler) +/* High precision definition of pi and e (Euler) * The values are taken from libc6's math.h. */ #ifndef Py_MATH_PIl diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -590,7 +590,7 @@ bytes. If the EOF was received and the internal buffer is empty, return an empty bytes object. - If n is zero, return empty bytes object immediatelly. + If n is zero, return empty bytes object immediately. If n is positive, this function try to read `n` bytes, and may return less or equal bytes than requested, but at least one byte. If EOF was diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py --- a/Lib/concurrent/futures/process.py +++ b/Lib/concurrent/futures/process.py @@ -63,7 +63,7 @@ # interpreter to exit when there are still idle processes in a # ProcessPoolExecutor's process pool (i.e. shutdown() was not called). However, # allowing workers to die with the interpreter has two undesirable properties: -# - The workers would still be running during interpretor shutdown, +# - The workers would still be running during interpreter shutdown, # meaning that they would fail in unpredictable ways. # - The workers could be killed while evaluating a work item, which could # be bad if the callable being evaluated has external side-effects e.g. diff --git a/Lib/concurrent/futures/thread.py b/Lib/concurrent/futures/thread.py --- a/Lib/concurrent/futures/thread.py +++ b/Lib/concurrent/futures/thread.py @@ -16,7 +16,7 @@ # to exit when there are still idle threads in a ThreadPoolExecutor's thread # pool (i.e. shutdown() was not called). However, allowing workers to die with # the interpreter has two undesirable properties: -# - The workers would still be running during interpretor shutdown, +# - The workers would still be running during interpreter shutdown, # meaning that they would fail in unpredictable ways. # - The workers could be killed while evaluating a work item, which could # be bad if the callable being evaluated has external side-effects e.g. diff --git a/Lib/distutils/tests/test_msvc9compiler.py b/Lib/distutils/tests/test_msvc9compiler.py --- a/Lib/distutils/tests/test_msvc9compiler.py +++ b/Lib/distutils/tests/test_msvc9compiler.py @@ -125,7 +125,7 @@ self.assertRaises(KeyError, Reg.get_value, 'xxx', 'xxx') # looking for values that should exist on all - # windows registeries versions. + # windows registry versions. path = r'Control Panel\Desktop' v = Reg.get_value(path, 'dragfullwindows') self.assertIn(v, ('0', '1', '2')) diff --git a/Lib/email/message.py b/Lib/email/message.py --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -1043,7 +1043,7 @@ yield from parts return # Otherwise we more or less invert the remaining logic in get_body. - # This only really works in edge cases (ex: non-text relateds or + # This only really works in edge cases (ex: non-text related or # alternatives) if the sending agent sets content-disposition. seen = [] # Only skip the first example of each candidate type. for part in parts: diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -136,7 +136,7 @@ # # VCHAR defined in http://tools.ietf.org/html/rfc5234#appendix-B.1 -# the patterns for both name and value are more leniant than RFC +# the patterns for both name and value are more lenient than RFC # definitions to allow for backwards compatibility _is_legal_header_name = re.compile(rb'[^:\s][^:\r\n]*').fullmatch _is_illegal_header_value = re.compile(rb'\n(?![ \t])|\r(?![ \t\n])').search diff --git a/Lib/idlelib/idle_test/test_formatparagraph.py b/Lib/idlelib/idle_test/test_formatparagraph.py --- a/Lib/idlelib/idle_test/test_formatparagraph.py +++ b/Lib/idlelib/idle_test/test_formatparagraph.py @@ -159,7 +159,7 @@ class ReformatFunctionTest(unittest.TestCase): """Test the reformat_paragraph function without the editor window.""" - def test_reformat_paragrah(self): + def test_reformat_paragraph(self): Equal = self.assertEqual reform = fp.reformat_paragraph hw = "O hello world" diff --git a/Lib/shutil.py b/Lib/shutil.py --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -64,7 +64,7 @@ class RegistryError(Exception): """Raised when a registry operation with the archiving - and unpacking registeries fails""" + and unpacking registries fails""" def copyfileobj(fsrc, fdst, length=16*1024): diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -26,7 +26,7 @@ _multiprocessing = test.support.import_module('_multiprocessing') # Skip tests if sem_open implementation is broken. test.support.import_module('multiprocessing.synchronize') -# import threading after _multiprocessing to raise a more revelant error +# import threading after _multiprocessing to raise a more relevant error # message: "No module named _multiprocessing". _multiprocessing is not compiled # without thread support. import threading diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -3884,7 +3884,7 @@ self.assertRaises(TypeError, lambda: as_date >= as_datetime) self.assertRaises(TypeError, lambda: as_datetime >= as_date) - # Neverthelss, comparison should work with the base-class (date) + # Nevertheless, comparison should work with the base-class (date) # projection if use of a date method is forced. self.assertEqual(as_date.__eq__(as_datetime), True) different_day = (as_date.day + 1) % 20 + 1 diff --git a/Lib/test/test_asyncio/test_locks.py b/Lib/test/test_asyncio/test_locks.py --- a/Lib/test/test_asyncio/test_locks.py +++ b/Lib/test/test_asyncio/test_locks.py @@ -130,8 +130,8 @@ def test_cancel_race(self): # Several tasks: # - A acquires the lock - # - B is blocked in aqcuire() - # - C is blocked in aqcuire() + # - B is blocked in acquire() + # - C is blocked in acquire() # # Now, concurrently: # - B is cancelled diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py --- a/Lib/test/test_concurrent_futures.py +++ b/Lib/test/test_concurrent_futures.py @@ -4,7 +4,7 @@ test.support.import_module('_multiprocessing') # Skip tests if sem_open implementation is broken. test.support.import_module('multiprocessing.synchronize') -# import threading after _multiprocessing to raise a more revelant error +# import threading after _multiprocessing to raise a more relevant error # message: "No module named _multiprocessing". _multiprocessing is not compiled # without thread support. test.support.import_module('threading') diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -876,7 +876,7 @@ self.assertEqual(Frag().__int__(), 42) self.assertEqual(int(Frag()), 42) - def test_diamond_inheritence(self): + def test_diamond_inheritance(self): # Testing multiple inheritance special cases... class A(object): def spam(self): return "A" diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -723,12 +723,12 @@ # Issue 5871: reject an attempt to embed a header inside a header value # (header injection attack). - def test_embeded_header_via_Header_rejected(self): + def test_embedded_header_via_Header_rejected(self): msg = Message() msg['Dummy'] = Header('dummy\nX-Injected-Header: test') self.assertRaises(errors.HeaderParseError, msg.as_string) - def test_embeded_header_via_string_rejected(self): + def test_embedded_header_via_string_rejected(self): msg = Message() msg['Dummy'] = 'dummy\nX-Injected-Header: test' self.assertRaises(errors.HeaderParseError, msg.as_string) diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py --- a/Lib/test/test_importlib/test_util.py +++ b/Lib/test/test_importlib/test_util.py @@ -372,7 +372,7 @@ # bacon self.assertEqual('bacon', self.util.resolve_name('bacon', None)) - def test_aboslute_within_package(self): + def test_absolute_within_package(self): # bacon in spam self.assertEqual('bacon', self.util.resolve_name('bacon', 'spam')) diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py --- a/Lib/test/test_ipaddress.py +++ b/Lib/test/test_ipaddress.py @@ -1262,7 +1262,7 @@ ip4 = ipaddress.IPv4Address('1.1.1.3') ip5 = ipaddress.IPv4Address('1.1.1.4') ip6 = ipaddress.IPv4Address('1.1.1.0') - # check that addreses are subsumed properly. + # check that addresses are subsumed properly. collapsed = ipaddress.collapse_addresses( [ip1, ip2, ip3, ip4, ip5, ip6]) self.assertEqual(list(collapsed), @@ -1276,7 +1276,7 @@ ip4 = ipaddress.IPv4Address('1.1.1.3') #ip5 = ipaddress.IPv4Interface('1.1.1.4/30') #ip6 = ipaddress.IPv4Interface('1.1.1.4/30') - # check that addreses are subsumed properly. + # check that addresses are subsumed properly. collapsed = ipaddress.collapse_addresses([ip1, ip2, ip3, ip4]) self.assertEqual(list(collapsed), [ipaddress.IPv4Network('1.1.1.0/30')]) @@ -1290,7 +1290,7 @@ # stored in no particular order b/c we want CollapseAddr to call # [].sort ip6 = ipaddress.IPv4Network('1.1.0.0/22') - # check that addreses are subsumed properly. + # check that addresses are subsumed properly. collapsed = ipaddress.collapse_addresses([ip1, ip2, ip3, ip4, ip5, ip6]) self.assertEqual(list(collapsed), diff --git a/Lib/test/test_pep247.py b/Lib/test/test_pep247.py --- a/Lib/test/test_pep247.py +++ b/Lib/test/test_pep247.py @@ -1,5 +1,5 @@ """ -Test suite to check compilance with PEP 247, the standard API +Test suite to check compliance with PEP 247, the standard API for hashing algorithms """ diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -686,7 +686,7 @@ self.assertEqual(stdout, "banana") self.assertStderrEqual(stderr.encode(), b"pineapple\npear\n") - def test_communicate_timeout_large_ouput(self): + def test_communicate_timeout_large_output(self): # Test an expiring timeout while the child is outputting lots of data. p = subprocess.Popen([sys.executable, "-c", 'import sys,os,time;' diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py --- a/Lib/test/test_urllib.py +++ b/Lib/test/test_urllib.py @@ -1,4 +1,4 @@ -"""Regresssion tests for what was in Python 2's "urllib" module""" +"""Regression tests for what was in Python 2's "urllib" module""" import urllib.parse import urllib.request diff --git a/Lib/test/test_winreg.py b/Lib/test/test_winreg.py --- a/Lib/test/test_winreg.py +++ b/Lib/test/test_winreg.py @@ -168,7 +168,7 @@ DeleteKey(key, subkeystr) try: - # Shouldnt be able to delete it twice! + # Shouldn't be able to delete it twice! DeleteKey(key, subkeystr) self.fail("Deleting the key twice succeeded") except OSError: diff --git a/Lib/unittest/test/test_discovery.py b/Lib/unittest/test/test_discovery.py --- a/Lib/unittest/test/test_discovery.py +++ b/Lib/unittest/test/test_discovery.py @@ -349,7 +349,7 @@ suite = list(loader._find_tests(abspath('/foo'), 'test*.py')) # We should have loaded tests from both my_package and - # my_pacakge.test_module, and also run the load_tests hook in both. + # my_package.test_module, and also run the load_tests hook in both. # (normally this would be nested TestSuites.) self.assertEqual(suite, [['my_package load_tests', [], diff --git a/Lib/unittest/test/testmock/testcallable.py b/Lib/unittest/test/testmock/testcallable.py --- a/Lib/unittest/test/testmock/testcallable.py +++ b/Lib/unittest/test/testmock/testcallable.py @@ -27,7 +27,7 @@ self.assertIn(mock.__class__.__name__, repr(mock)) - def test_heirarchy(self): + def test_hierarchy(self): self.assertTrue(issubclass(MagicMock, Mock)) self.assertTrue(issubclass(NonCallableMagicMock, NonCallableMock)) diff --git a/Lib/venv/scripts/posix/activate b/Lib/venv/scripts/posix/activate --- a/Lib/venv/scripts/posix/activate +++ b/Lib/venv/scripts/posix/activate @@ -34,7 +34,7 @@ fi } -# unset irrelavent variables +# unset irrelevant variables deactivate nondestructive VIRTUAL_ENV="__VENV_DIR__" diff --git a/Lib/venv/scripts/posix/activate.csh b/Lib/venv/scripts/posix/activate.csh --- a/Lib/venv/scripts/posix/activate.csh +++ b/Lib/venv/scripts/posix/activate.csh @@ -5,7 +5,7 @@ alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PATH" && unset _OLD_VIRTUAL_PATH; rehash; test $?_OLD_VIRTUAL_PROMPT != 0 && set prompt="$_OLD_VIRTUAL_PROMPT" && unset _OLD_VIRTUAL_PROMPT; unsetenv VIRTUAL_ENV; test "\!:*" != "nondestructive" && unalias deactivate' -# Unset irrelavent variables. +# Unset irrelevant variables. deactivate nondestructive setenv VIRTUAL_ENV "__VENV_DIR__" diff --git a/Lib/venv/scripts/posix/activate.fish b/Lib/venv/scripts/posix/activate.fish --- a/Lib/venv/scripts/posix/activate.fish +++ b/Lib/venv/scripts/posix/activate.fish @@ -29,7 +29,7 @@ end end -# unset irrelavent variables +# unset irrelevant variables deactivate nondestructive set -gx VIRTUAL_ENV "__VENV_DIR__" diff --git a/Mac/PythonLauncher/MyAppDelegate.m b/Mac/PythonLauncher/MyAppDelegate.m --- a/Mac/PythonLauncher/MyAppDelegate.m +++ b/Mac/PythonLauncher/MyAppDelegate.m @@ -34,7 +34,7 @@ - (BOOL)shouldShowUI { // if this call comes before applicationDidFinishLaunching: we - // should terminate immedeately after starting the script. + // should terminate immediately after starting the script. if (!initial_action_done) should_terminate = YES; initial_action_done = YES; diff --git a/Misc/HISTORY b/Misc/HISTORY --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -1131,7 +1131,7 @@ and http.client. Patch by EungJun Yi. - Issue #14777: tkinter may return undecoded UTF-8 bytes as a string when - accessing the Tk clipboard. Modify clipboad_get() to first request type + accessing the Tk clipboard. Modify clipboard_get() to first request type UTF8_STRING when no specific type is requested in an X11 windowing environment, falling back to the current default type STRING if that fails. Original patch by Thomas Kluyver. @@ -5693,7 +5693,7 @@ for reading). - hashlib has two new constant attributes: algorithms_guaranteed and - algorithms_avaiable that respectively list the names of hash algorithms + algorithms_available that respectively list the names of hash algorithms guaranteed to exist in all Python implementations and the names of hash algorithms available in the current process. @@ -7344,7 +7344,7 @@ - Issue #2846: Add support for gzip.GzipFile reading zero-padded files. Patch by Brian Curtin. -- Issue #7681: Use floor division in appropiate places in the wave module. +- Issue #7681: Use floor division in appropriate places in the wave module. - Issue #5372: Drop the reuse of .o files in Distutils' ccompiler (since Extension extra options may change the output without changing the .c @@ -10921,7 +10921,7 @@ - Support for BeOS and AtheOS was removed (according to PEP 11). -- Support for RiscOS, Irix, Tru64 was removed (alledgedly). +- Support for RiscOS, Irix, Tru64 was removed (allegedly). Tools/Demos ----------- @@ -12912,7 +12912,7 @@ - Bug #947906: An object oriented interface has been added to the calendar module. It's possible to generate HTML calendar now and the module can be called as a script (e.g. via ``python -mcalendar``). Localized month and - weekday names can be ouput (even if an exotic encoding is used) using + weekday names can be output (even if an exotic encoding is used) using special classes that use unicode. Build @@ -13295,7 +13295,7 @@ ``True`` for ``!=``, and raises ``TypeError`` for other comparison operators. Because datetime is a subclass of date, comparing only the base class (date) members can still be done, if that's desired, by - forcing using of the approprate date method; e.g., + forcing using of the appropriate date method; e.g., ``a_date.__eq__(a_datetime)`` is true if and only if the year, month and day members of ``a_date`` and ``a_datetime`` are equal. @@ -23770,7 +23770,7 @@ - copy.py: Make sure the objects returned by __getinitargs__() are kept alive (in the memo) to avoid a certain kind of nasty crash. (Not -easily reproducable because it requires a later call to +easily reproducible because it requires a later call to __getinitargs__() to return a tuple that happens to be allocated at the same address.) @@ -27402,7 +27402,7 @@ There is now a script to patch Makefile and config.c to add a new optional built-in module: Addmodule.sh. Read the script before using! -Useing Addmodule.sh, all optional modules can now be configured at +Using Addmodule.sh, all optional modules can now be configured at compile time using Configure.py, so there are no modules left that require dynamic loading. @@ -27833,9 +27833,9 @@ SUNAUDIODEV: symbolic constant definitions for sunaudiodef (sun only) -SV: symbolic constat definitions for sv (sgi only) - -CD: symbolic constat definitions for cd (sgi only) +SV: symbolic constant definitions for sv (sgi only) + +CD: symbolic constant definitions for cd (sgi only) New demos diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -4577,7 +4577,7 @@ - Issue #22120: For functions using an unsigned integer return converter, Argument Clinic now generates a cast to that type for the comparison - to -1 in the generated code. (This supresses a compilation warning.) + to -1 in the generated code. (This suppresses a compilation warning.) - Issue #18974: Tools/scripts/diff.py now uses argparse instead of optparse. @@ -5677,7 +5677,7 @@ - Issue #19466: Clear the frames of daemon threads earlier during the Python shutdown to call objects destructors. So "unclosed file" resource - warnings are now corretly emitted for daemon threads. + warnings are now correctly emitted for daemon threads. - Issue #19514: Deduplicate some _Py_IDENTIFIER declarations. Patch by Andrei Dorian Duma. @@ -6607,7 +6607,7 @@ - Issue #18709: Fix CVE-2013-4238. The SSL module now handles NULL bytes inside subjectAltName correctly. Formerly the module has used OpenSSL's - GENERAL_NAME_print() function to get the string represention of ASN.1 + GENERAL_NAME_print() function to get the string representation of ASN.1 strings for ``rfc822Name`` (email), ``dNSName`` (DNS) and ``uniformResourceIdentifier`` (URI). @@ -6700,7 +6700,7 @@ Documentation ------------- -- Issue #18743: Fix references to non-existant "StringIO" module. +- Issue #18743: Fix references to non-existent "StringIO" module. - Issue #18783: Removed existing mentions of Python long type in docstrings, error messages and comments. @@ -7639,7 +7639,7 @@ specifically addresses a stack misalignment issue on x86 and issues on some more recent platforms. -- Issue #8862: Fixed curses cleanup when getkey is interrputed by a signal. +- Issue #8862: Fixed curses cleanup when getkey is interrupted by a signal. - Issue #17443: imaplib.IMAP4_stream was using the default unbuffered IO in subprocess, but the imap code assumes buffered IO. In Python2 this diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -238,7 +238,7 @@ StgDictObject function to a generic one. Currently, PyCFuncPtr types have 'converters' and 'checker' entries in their - type dict. They are only used to cache attributes from other entries, whihc + type dict. They are only used to cache attributes from other entries, which is wrong. One use case is the .value attribute that all simple types have. But some diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -771,7 +771,7 @@ /* * This macro generates constructor function definitions for specific * hash algorithms. These constructors are much faster than calling - * the generic one passing it a python string and are noticably + * the generic one passing it a python string and are noticeably * faster than calling a python new() wrapper. Thats important for * code that wants to make hashes of a bunch of small strings. */ diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -90,7 +90,7 @@ return NULL; } -/* Positionning */ +/* Positioning */ PyDoc_STRVAR(iobase_seek_doc, "Change stream position.\n" diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -3781,7 +3781,7 @@ { PyThreadState *tstate = PyThreadState_GET(); - /* substract one to ignore the frame of the get_recursion_depth() call */ + /* subtract one to ignore the frame of the get_recursion_depth() call */ return PyLong_FromLong(tstate->recursion_depth - 1); } diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -45,7 +45,7 @@ /* Helper to acquire an interruptible lock with a timeout. If the lock acquire * is interrupted, signal handlers are run, and if they raise an exception, * PY_LOCK_INTR is returned. Otherwise, PY_LOCK_ACQUIRED or PY_LOCK_FAILURE - * are returned, depending on whether the lock can be acquired withing the + * are returned, depending on whether the lock can be acquired within the * timeout. */ static PyLockStatus diff --git a/Modules/_tracemalloc.c b/Modules/_tracemalloc.c --- a/Modules/_tracemalloc.c +++ b/Modules/_tracemalloc.c @@ -516,7 +516,7 @@ if (tracemalloc_add_trace(ptr2, new_size) < 0) { /* Memory allocation failed. The error cannot be reported to - the caller, because realloc() may already have shrinked the + the caller, because realloc() may already have shrunk the memory block and so removed bytes. This case is very unlikely: a hash entry has just been diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -1274,7 +1274,7 @@ /* Divide-and-conquer factorial algorithm * - * Based on the formula and psuedo-code provided at: + * Based on the formula and pseudo-code provided at: * http://www.luschny.de/math/factorial/binarysplitfact.html * * Faster algorithms exist, but they're more complicated and depend on diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -6538,7 +6538,7 @@ PyModule_AddIntConstant(m, "SOMAXCONN", 5); /* Common value */ #endif - /* Ancilliary message types */ + /* Ancillary message types */ #ifdef SCM_RIGHTS PyModule_AddIntMacro(m, SCM_RIGHTS); #endif diff --git a/Modules/zipimport.c b/Modules/zipimport.c --- a/Modules/zipimport.c +++ b/Modules/zipimport.c @@ -1315,7 +1315,7 @@ return code; } -/* Replace any occurances of "\r\n?" in the input string with "\n". +/* Replace any occurrences of "\r\n?" in the input string with "\n". This converts DOS and Mac line endings to Unix line endings. Also append a trailing "\n" to be compatible with PyParser_SimpleParseFile(). Returns a new reference. */ diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -506,7 +506,7 @@ If growth < 0 and lo != 0, the operation is completed, but a MemoryError is still raised and the memory block is not - shrinked. Otherwise, the bytearray is restored in its previous + shrunk. Otherwise, the bytearray is restored in its previous state and a MemoryError is raised. */ if (lo == 0) { self->ob_start += growth; diff --git a/Objects/codeobject.c b/Objects/codeobject.c --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -718,7 +718,7 @@ /* possible optimization: if f->f_lasti == instr_ub (likely to be a common case) then we already know instr_lb -- if we stored the matching value of p - somwhere we could skip the first while loop. */ + somewhere we could skip the first while loop. */ /* See lnotab_notes.txt for the description of co_lnotab. A point to remember: increments to p diff --git a/Objects/listsort.txt b/Objects/listsort.txt --- a/Objects/listsort.txt +++ b/Objects/listsort.txt @@ -694,7 +694,7 @@ But in CPython's case, comparisons are extraordinarily expensive compared to moving data, and the details matter. Moving objects is just copying -pointers. Comparisons can be arbitrarily expensive (can invoke arbitary +pointers. Comparisons can be arbitrarily expensive (can invoke arbitrary user-supplied Python code), but even in simple cases (like 3 < 4) _all_ decisions are made at runtime: what's the type of the left comparand? the type of the right? do they need to be coerced to a common type? where's the diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -368,7 +368,7 @@ /* Checking for overflow in PyLong_AsLong is a PITA since C doesn't define * anything about what happens when a signed integer operation overflows, * and some compilers think they're doing you a favor by being "clever" - * then. The bit pattern for the largest postive signed long is + * then. The bit pattern for the largest positive signed long is * (unsigned long)LONG_MAX, and for the smallest negative signed long * it is abs(LONG_MIN), which we could write -(unsigned long)LONG_MIN. * However, some other compilers warn about applying unary minus to an diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3759,7 +3759,7 @@ /* Try to fetch cached copy of copyreg from sys.modules first in an attempt to avoid the import overhead. Previously this was implemented by storing a reference to the cached module in a static variable, but - this broke when multiple embeded interpreters were in use (see issue + this broke when multiple embedded interpreters were in use (see issue #17408 and #19088). */ copyreg_module = PyDict_GetItemWithError(interp->modules, copyreg_str); if (copyreg_module != NULL) { diff --git a/Python/condvar.h b/Python/condvar.h --- a/Python/condvar.h +++ b/Python/condvar.h @@ -238,7 +238,7 @@ cv->waiting++; PyMUTEX_UNLOCK(cs); /* "lost wakeup bug" would occur if the caller were interrupted here, - * but we are safe because we are using a semaphore wich has an internal + * but we are safe because we are using a semaphore which has an internal * count. */ wait = WaitForSingleObjectEx(cv->sem, ms, FALSE); diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -118,7 +118,7 @@ } InternalFormatSpec; #if 0 -/* Occassionally useful for debugging. Should normally be commented out. */ +/* Occasionally useful for debugging. Should normally be commented out. */ static void DEBUG_PRINT_FORMAT_SPEC(InternalFormatSpec *format) { diff --git a/README b/README --- a/README +++ b/README @@ -68,7 +68,7 @@ After this instrumented version of the interpreter is built, the Makefile will automatically run a training workload. This is necessary in order to profile the interpreter execution. Note also that any output, both stdout -and stderr, that may appear at this step is supressed. +and stderr, that may appear at this step is suppressed. Finally, the last step is to rebuild the interpreter, using the information collected in the previous one. The end result will be a Python binary diff --git a/configure b/configure --- a/configure +++ b/configure @@ -7267,7 +7267,7 @@ # Calculate an appropriate deployment target for this build: # The deployment target value is used explicitly to enable certain # features are enabled (such as builtin libedit support for readline) - # through the use of Apple's Availabiliy Macros and is used as a + # through the use of Apple's Availability Macros and is used as a # component of the string returned by distutils.get_platform(). # # Use the value from: diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1689,7 +1689,7 @@ # Calculate an appropriate deployment target for this build: # The deployment target value is used explicitly to enable certain # features are enabled (such as builtin libedit support for readline) - # through the use of Apple's Availabiliy Macros and is used as a + # through the use of Apple's Availability Macros and is used as a # component of the string returned by distutils.get_platform(). # # Use the value from: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:42:48 2016 From: python-checkins at python.org (martin.panter) Date: Wed, 07 Sep 2016 23:42:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327993=3A_Merge_plural_fixes_from_3=2E5?= Message-ID: <20160907234248.13065.77726.B1BB6518@psf.io> https://hg.python.org/cpython/rev/bb2a7134d82b changeset: 103260:bb2a7134d82b parent: 103259:2d0fb659372c parent: 103258:8f02a1a6b647 user: Martin Panter date: Wed Sep 07 23:36:43 2016 +0000 summary: Issue #27993: Merge plural fixes from 3.5 files: Doc/library/argparse.rst | 4 ++-- Doc/library/weakref.rst | 2 +- Doc/library/xml.dom.rst | 2 +- Lib/inspect.py | 5 ++--- Lib/test/test_json/test_decode.py | 2 +- Misc/HISTORY | 2 +- Misc/NEWS | 4 ++-- 7 files changed, 10 insertions(+), 11 deletions(-) diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -188,7 +188,7 @@ prog ^^^^ -By default, :class:`ArgumentParser` objects uses ``sys.argv[0]`` to determine +By default, :class:`ArgumentParser` objects use ``sys.argv[0]`` to determine how to display the name of the program in help messages. This default is almost always desirable because it will make the help messages match how the program was invoked on the command line. For example, consider a file named @@ -558,7 +558,7 @@ ^^^^^^^^^^^^^^^^ :class:`ArgumentParser` objects do not allow two actions with the same option -string. By default, :class:`ArgumentParser` objects raises an exception if an +string. By default, :class:`ArgumentParser` objects raise an exception if an attempt is made to create an argument with an option string that is already in use:: diff --git a/Doc/library/weakref.rst b/Doc/library/weakref.rst --- a/Doc/library/weakref.rst +++ b/Doc/library/weakref.rst @@ -414,7 +414,7 @@ Example ------- -This simple example shows how an application can use objects IDs to retrieve +This simple example shows how an application can use object IDs to retrieve objects that it has seen before. The IDs of the objects can then be used in other data structures without forcing the objects to remain alive, but the objects can still be retrieved by ID if they do. diff --git a/Doc/library/xml.dom.rst b/Doc/library/xml.dom.rst --- a/Doc/library/xml.dom.rst +++ b/Doc/library/xml.dom.rst @@ -404,7 +404,7 @@ ^^^^^^^^^^^^^^^^ A :class:`NodeList` represents a sequence of nodes. These objects are used in -two ways in the DOM Core recommendation: the :class:`Element` objects provides +two ways in the DOM Core recommendation: an :class:`Element` object provides one as its list of child nodes, and the :meth:`getElementsByTagName` and :meth:`getElementsByTagNameNS` methods of :class:`Node` return objects with this interface to represent query results. diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -171,9 +171,8 @@ def isgeneratorfunction(object): """Return true if the object is a user-defined generator function. - Generator function objects provides same attributes as functions. - - See help(isfunction) for attributes listing.""" + Generator function objects provide the same attributes as functions. + See help(isfunction) for a list of attributes.""" return bool((isfunction(object) or ismethod(object)) and object.__code__.co_flags & CO_GENERATOR) diff --git a/Lib/test/test_json/test_decode.py b/Lib/test/test_json/test_decode.py --- a/Lib/test/test_json/test_decode.py +++ b/Lib/test/test_json/test_decode.py @@ -35,7 +35,7 @@ self.assertEqual(self.loads(s, object_pairs_hook=OrderedDict, object_hook=lambda x: None), OrderedDict(p)) - # check that empty objects literals work (see #17368) + # check that empty object literals work (see #17368) self.assertEqual(self.loads('{}', object_pairs_hook=OrderedDict), OrderedDict()) self.assertEqual(self.loads('{"empty": {}}', diff --git a/Misc/HISTORY b/Misc/HISTORY --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -26057,7 +26057,7 @@ module implementing deepcopy and normal (shallow) copy operations. See the library reference manual. -- Documentation strings for many objects types are accessible through +- Documentation strings for many object types are accessible through the __doc__ attribute. Modules, classes and functions support special syntax to initialize the __doc__ attribute: if the first statement consists of just a string literal, that string literal becomes the diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -6017,7 +6017,7 @@ - Issue #20588: Make Python-ast.c C89 compliant. -- Issue #20437: Fixed 22 potential bugs when deleting objects references. +- Issue #20437: Fixed 22 potential bugs when deleting object references. - Issue #20500: Displaying an exception at interpreter shutdown no longer risks triggering an assertion failure in PyObject_Str. @@ -6922,7 +6922,7 @@ when the creation of the replacement exception won't lose any information. - Issue #19466: Clear the frames of daemon threads earlier during the - Python shutdown to call objects destructors. So "unclosed file" resource + Python shutdown to call object destructors. So "unclosed file" resource warnings are now correctly emitted for daemon threads. - Issue #19514: Deduplicate some _Py_IDENTIFIER declarations. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:53:25 2016 From: python-checkins at python.org (davin.potts) Date: Wed, 07 Sep 2016 23:53:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixes_issue_=236766=3A_Upd?= =?utf-8?q?ated_multiprocessing_Proxy_Objects_to_support_nesting?= Message-ID: <20160907235325.36420.6374.46D2F153@psf.io> https://hg.python.org/cpython/rev/39e7307f9aee changeset: 103261:39e7307f9aee user: Davin Potts date: Wed Sep 07 18:48:01 2016 -0500 summary: Fixes issue #6766: Updated multiprocessing Proxy Objects to support nesting files: Doc/library/multiprocessing.rst | 85 +++++++++++------ Lib/multiprocessing/managers.py | 95 ++++++++++++++----- Lib/test/_test_multiprocessing.py | 70 +++++++++++++- 3 files changed, 193 insertions(+), 57 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1682,7 +1682,9 @@ of processes. Objects of this type are returned by :func:`multiprocessing.Manager`. - It also supports creation of shared lists and dictionaries. + Its methods create and return :ref:`multiprocessing-proxy_objects` for a + number of commonly used data types to be synchronized across processes. + This notably includes shared lists and dictionaries. .. method:: Barrier(parties[, action[, timeout]]) @@ -1745,31 +1747,17 @@ dict(mapping) dict(sequence) - Create a shared ``dict`` object and return a proxy for it. + Create a shared :class:`dict` object and return a proxy for it. .. method:: list() list(sequence) - Create a shared ``list`` object and return a proxy for it. - - .. note:: - - Modifications to mutable values or items in dict and list proxies will not - be propagated through the manager, because the proxy has no way of knowing - when its values or items are modified. To modify such an item, you can - re-assign the modified object to the container proxy:: - - # create a list proxy and append a mutable object (a dictionary) - lproxy = manager.list() - lproxy.append({}) - # now mutate the dictionary - d = lproxy[0] - d['a'] = 1 - d['b'] = 2 - # at this point, the changes to d are not yet synced, but by - # reassigning the dictionary, the proxy is notified of the change - lproxy[0] = d - + Create a shared :class:`list` object and return a proxy for it. + + .. versionchanged:: 3.6 + Shared objects are capable of being nested. For example, a shared + container object such as a shared list can contain other shared objects + which will all be managed and synchronized by the :class:`SyncManager`. .. class:: Namespace @@ -1881,6 +1869,8 @@ >>> s = m.get_server() >>> s.serve_forever() +.. _multiprocessing-proxy_objects: + Proxy Objects ~~~~~~~~~~~~~ @@ -1890,8 +1880,7 @@ A proxy object has methods which invoke corresponding methods of its referent (although not every method of the referent will necessarily be available through -the proxy). A proxy can usually be used in most of the same ways that its -referent can: +the proxy). In this way, a proxy can be used just like its referent can: .. doctest:: @@ -1912,9 +1901,9 @@ the proxy. An important feature of proxy objects is that they are picklable so they can be -passed between processes. Note, however, that if a proxy is sent to the -corresponding manager's process then unpickling it will produce the referent -itself. This means, for example, that one shared object can contain a second: +passed between processes. As such, a referent can contain +:ref:`multiprocessing-proxy_objects`. This permits nesting of these managed +lists, dicts, and other :ref:`multiprocessing-proxy_objects`: .. doctest:: @@ -1922,10 +1911,46 @@ >>> b = manager.list() >>> a.append(b) # referent of a now contains referent of b >>> print(a, b) - [[]] [] + [] [] >>> b.append('hello') - >>> print(a, b) - [['hello']] ['hello'] + >>> print(a[0], b) + ['hello'] ['hello'] + +Similarly, dict and list proxies may be nested inside one another:: + + >>> l_outer = manager.list([ manager.dict() for i in range(2) ]) + >>> d_first_inner = l_outer[0] + >>> d_first_inner['a'] = 1 + >>> d_first_inner['b'] = 2 + >>> l_outer[1]['c'] = 3 + >>> l_outer[1]['z'] = 26 + >>> print(l_outer[0]) + {'a': 1, 'b': 2} + >>> print(l_outer[1]) + {'c': 3, 'z': 26} + +If standard (non-proxy) :class:`list` or :class:`dict` objects are contained +in a referent, modifications to those mutable values will not be propagated +through the manager because the proxy has no way of knowing when the values +contained within are modified. However, storing a value in a container proxy +(which triggers a ``__setitem__`` on the proxy object) does propagate through +the manager and so to effectively modify such an item, one could re-assign the +modified value to the container proxy:: + + # create a list proxy and append a mutable object (a dictionary) + lproxy = manager.list() + lproxy.append({}) + # now mutate the dictionary + d = lproxy[0] + d['a'] = 1 + d['b'] = 2 + # at this point, the changes to d are not yet synced, but by + # updating the dictionary, the proxy is notified of the change + lproxy[0] = d + +This approach is perhaps less convenient than employing nested +:ref:`multiprocessing-proxy_objects` for most use cases but also +demonstrates a level of control over the synchronization. .. note:: diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -142,7 +142,8 @@ self.id_to_obj = {'0': (None, ())} self.id_to_refcount = {} - self.mutex = threading.RLock() + self.id_to_local_proxy_obj = {} + self.mutex = threading.Lock() def serve_forever(self): ''' @@ -227,7 +228,14 @@ methodname = obj = None request = recv() ident, methodname, args, kwds = request - obj, exposed, gettypeid = id_to_obj[ident] + try: + obj, exposed, gettypeid = id_to_obj[ident] + except KeyError as ke: + try: + obj, exposed, gettypeid = \ + self.id_to_local_proxy_obj[ident] + except KeyError as second_ke: + raise ke if methodname not in exposed: raise AttributeError( @@ -308,7 +316,7 @@ ''' with self.mutex: result = [] - keys = list(self.id_to_obj.keys()) + keys = list(self.id_to_refcount.keys()) keys.sort() for ident in keys: if ident != '0': @@ -321,7 +329,8 @@ ''' Number of shared objects ''' - return len(self.id_to_obj) - 1 # don't count ident='0' + # Doesn't use (len(self.id_to_obj) - 1) as we shouldn't count ident='0' + return len(self.id_to_refcount) def shutdown(self, c): ''' @@ -363,13 +372,9 @@ self.id_to_obj[ident] = (obj, set(exposed), method_to_typeid) if ident not in self.id_to_refcount: self.id_to_refcount[ident] = 0 - # increment the reference count immediately, to avoid - # this object being garbage collected before a Proxy - # object for it can be created. The caller of create() - # is responsible for doing a decref once the Proxy object - # has been created. - self.incref(c, ident) - return ident, tuple(exposed) + + self.incref(c, ident) + return ident, tuple(exposed) def get_methods(self, c, token): ''' @@ -387,15 +392,45 @@ def incref(self, c, ident): with self.mutex: - self.id_to_refcount[ident] += 1 + try: + self.id_to_refcount[ident] += 1 + except KeyError as ke: + # If no external references exist but an internal (to the + # manager) still does and a new external reference is created + # from it, restore the manager's tracking of it from the + # previously stashed internal ref. + if ident in self.id_to_local_proxy_obj: + self.id_to_refcount[ident] = 1 + self.id_to_obj[ident] = \ + self.id_to_local_proxy_obj[ident] + obj, exposed, gettypeid = self.id_to_obj[ident] + util.debug('Server re-enabled tracking & INCREF %r', ident) + else: + raise ke def decref(self, c, ident): + if ident not in self.id_to_refcount and \ + ident in self.id_to_local_proxy_obj: + util.debug('Server DECREF skipping %r', ident) + return + with self.mutex: assert self.id_to_refcount[ident] >= 1 self.id_to_refcount[ident] -= 1 if self.id_to_refcount[ident] == 0: - del self.id_to_obj[ident], self.id_to_refcount[ident] - util.debug('disposing of obj with id %r', ident) + del self.id_to_refcount[ident] + + if ident not in self.id_to_refcount: + # Two-step process in case the object turns out to contain other + # proxy objects (e.g. a managed list of managed lists). + # Otherwise, deleting self.id_to_obj[ident] would trigger the + # deleting of the stored value (another managed object) which would + # in turn attempt to acquire the mutex that is already held here. + self.id_to_obj[ident] = (None, (), None) # thread-safe + util.debug('disposing of obj with id %r', ident) + with self.mutex: + del self.id_to_obj[ident] + # # Class to represent state of a manager @@ -658,7 +693,7 @@ _mutex = util.ForkAwareThreadLock() def __init__(self, token, serializer, manager=None, - authkey=None, exposed=None, incref=True): + authkey=None, exposed=None, incref=True, manager_owned=False): with BaseProxy._mutex: tls_idset = BaseProxy._address_to_local.get(token.address, None) if tls_idset is None: @@ -680,6 +715,12 @@ self._serializer = serializer self._Client = listener_client[serializer][1] + # Should be set to True only when a proxy object is being created + # on the manager server; primary use case: nested proxy objects. + # RebuildProxy detects when a proxy is being created on the manager + # and sets this value appropriately. + self._owned_by_manager = manager_owned + if authkey is not None: self._authkey = process.AuthenticationString(authkey) elif self._manager is not None: @@ -738,6 +779,10 @@ return self._callmethod('#GETVALUE') def _incref(self): + if self._owned_by_manager: + util.debug('owned_by_manager skipped INCREF of %r', self._token.id) + return + conn = self._Client(self._token.address, authkey=self._authkey) dispatch(conn, None, 'incref', (self._id,)) util.debug('INCREF %r', self._token.id) @@ -822,19 +867,19 @@ def RebuildProxy(func, token, serializer, kwds): ''' Function used for unpickling proxy objects. - - If possible the shared object is returned, or otherwise a proxy for it. ''' server = getattr(process.current_process(), '_manager_server', None) - if server and server.address == token.address: - return server.id_to_obj[token.id][0] - else: - incref = ( - kwds.pop('incref', True) and - not getattr(process.current_process(), '_inheriting', False) - ) - return func(token, serializer, incref=incref, **kwds) + util.debug('Rebuild a proxy owned by manager, token=%r', token) + kwds['manager_owned'] = True + if token.id not in server.id_to_local_proxy_obj: + server.id_to_local_proxy_obj[token.id] = \ + server.id_to_obj[token.id] + incref = ( + kwds.pop('incref', True) and + not getattr(process.current_process(), '_inheriting', False) + ) + return func(token, serializer, incref=incref, **kwds) # # Functions to create proxies and proxy types diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -1628,13 +1628,33 @@ d = [a, b] e = self.list(d) self.assertEqual( - e[:], + [element[:] for element in e], [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 0, 1, 2, 3, 4]] ) f = self.list([a]) a.append('hello') - self.assertEqual(f[:], [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 'hello']]) + self.assertEqual(f[0][:], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 'hello']) + + def test_list_proxy_in_list(self): + a = self.list([self.list(range(3)) for _i in range(3)]) + self.assertEqual([inner[:] for inner in a], [[0, 1, 2]] * 3) + + a[0][-1] = 55 + self.assertEqual(a[0][:], [0, 1, 55]) + for i in range(1, 3): + self.assertEqual(a[i][:], [0, 1, 2]) + + self.assertEqual(a[1].pop(), 2) + self.assertEqual(len(a[1]), 2) + for i in range(0, 3, 2): + self.assertEqual(len(a[i]), 3) + + del a + + b = self.list() + b.append(b) + del b def test_dict(self): d = self.dict() @@ -1646,6 +1666,52 @@ self.assertEqual(sorted(d.values()), [chr(i) for i in indices]) self.assertEqual(sorted(d.items()), [(i, chr(i)) for i in indices]) + def test_dict_proxy_nested(self): + pets = self.dict(ferrets=2, hamsters=4) + supplies = self.dict(water=10, feed=3) + d = self.dict(pets=pets, supplies=supplies) + + self.assertEqual(supplies['water'], 10) + self.assertEqual(d['supplies']['water'], 10) + + d['supplies']['blankets'] = 5 + self.assertEqual(supplies['blankets'], 5) + self.assertEqual(d['supplies']['blankets'], 5) + + d['supplies']['water'] = 7 + self.assertEqual(supplies['water'], 7) + self.assertEqual(d['supplies']['water'], 7) + + del pets + del supplies + self.assertEqual(d['pets']['ferrets'], 2) + d['supplies']['blankets'] = 11 + self.assertEqual(d['supplies']['blankets'], 11) + + pets = d['pets'] + supplies = d['supplies'] + supplies['water'] = 7 + self.assertEqual(supplies['water'], 7) + self.assertEqual(d['supplies']['water'], 7) + + d.clear() + self.assertEqual(len(d), 0) + self.assertEqual(supplies['water'], 7) + self.assertEqual(pets['hamsters'], 4) + + l = self.list([pets, supplies]) + l[0]['marmots'] = 1 + self.assertEqual(pets['marmots'], 1) + self.assertEqual(l[0]['marmots'], 1) + + del pets + del supplies + self.assertEqual(l[0]['marmots'], 1) + + outer = self.list([[88, 99], l]) + self.assertIsInstance(outer[0], list) # Not a ListProxy + self.assertEqual(outer[-1][-1]['feed'], 3) + def test_namespace(self): n = self.Namespace() n.name = 'Bob' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:56:37 2016 From: python-checkins at python.org (eric.snow) Date: Wed, 07 Sep 2016 23:56:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2315767=3A_Add_Modu?= =?utf-8?q?leNotFoundError=2E?= Message-ID: <20160907235637.87446.4276.7F67A7A3@psf.io> https://hg.python.org/cpython/rev/90549c3c609c changeset: 103262:90549c3c609c user: Eric Snow date: Wed Sep 07 15:42:32 2016 -0700 summary: Issue #15767: Add ModuleNotFoundError. files: Doc/c-api/exceptions.rst | 2 ++ Doc/library/exceptions.rst | 13 +++++++++++-- Include/pyerrors.h | 1 + Lib/_compat_pickle.py | 7 +++++++ Lib/test/exception_hierarchy.txt | 1 + Lib/test/test_pickle.py | 8 ++++++++ Misc/NEWS | 3 +++ Objects/exceptions.c | 9 +++++++++ 8 files changed, 42 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -782,6 +782,8 @@ +-----------------------------------------+---------------------------------+----------+ | :c:data:`PyExc_ImportError` | :exc:`ImportError` | | +-----------------------------------------+---------------------------------+----------+ +| :c:data:`PyExc_ModuleNotFoundError` | :exc:`ModuleNotFoundError` | | ++-----------------------------------------+---------------------------------+----------+ | :c:data:`PyExc_IndexError` | :exc:`IndexError` | | +-----------------------------------------+---------------------------------+----------+ | :c:data:`PyExc_InterruptedError` | :exc:`InterruptedError` | | diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -170,8 +170,9 @@ .. exception:: ImportError - Raised when an :keyword:`import` statement fails to find the module definition - or when a ``from ... import`` fails to find a name that is to be imported. + Raised when the :keyword:`import` statement has troubles trying to + load a module. Also raised when the "from list" in ``from ... import`` + has a name that cannot be found. The :attr:`name` and :attr:`path` attributes can be set using keyword-only arguments to the constructor. When set they represent the name of the module @@ -181,6 +182,14 @@ .. versionchanged:: 3.3 Added the :attr:`name` and :attr:`path` attributes. +.. exception:: ModuleNotFoundError + + A subclass of :exc:`ImportError` which is raised by :keyword:`import` + when a module could not be located. It is also raised when ``None`` + is found in :data:`sys.modules`. + + .. versionadded:: 3.6 + .. exception:: IndexError diff --git a/Include/pyerrors.h b/Include/pyerrors.h --- a/Include/pyerrors.h +++ b/Include/pyerrors.h @@ -160,6 +160,7 @@ PyAPI_DATA(PyObject *) PyExc_FloatingPointError; PyAPI_DATA(PyObject *) PyExc_OSError; PyAPI_DATA(PyObject *) PyExc_ImportError; +PyAPI_DATA(PyObject *) PyExc_ModuleNotFoundError; PyAPI_DATA(PyObject *) PyExc_IndexError; PyAPI_DATA(PyObject *) PyExc_KeyError; PyAPI_DATA(PyObject *) PyExc_KeyboardInterrupt; diff --git a/Lib/_compat_pickle.py b/Lib/_compat_pickle.py --- a/Lib/_compat_pickle.py +++ b/Lib/_compat_pickle.py @@ -242,3 +242,10 @@ for excname in PYTHON3_OSERROR_EXCEPTIONS: REVERSE_NAME_MAPPING[('builtins', excname)] = ('exceptions', 'OSError') + +PYTHON3_IMPORTERROR_EXCEPTIONS = ( + 'ModuleNotFoundError', +) + +for excname in PYTHON3_IMPORTERROR_EXCEPTIONS: + REVERSE_NAME_MAPPING[('builtins', excname)] = ('exceptions', 'ImportError') diff --git a/Lib/test/exception_hierarchy.txt b/Lib/test/exception_hierarchy.txt --- a/Lib/test/exception_hierarchy.txt +++ b/Lib/test/exception_hierarchy.txt @@ -14,6 +14,7 @@ +-- BufferError +-- EOFError +-- ImportError + +-- ModuleNotFoundError +-- LookupError | +-- IndexError | +-- KeyError diff --git a/Lib/test/test_pickle.py b/Lib/test/test_pickle.py --- a/Lib/test/test_pickle.py +++ b/Lib/test/test_pickle.py @@ -335,6 +335,9 @@ if (module2, name2) == ('exceptions', 'OSError'): attr = getattribute(module3, name3) self.assertTrue(issubclass(attr, OSError)) + elif (module2, name2) == ('exceptions', 'ImportError'): + attr = getattribute(module3, name3) + self.assertTrue(issubclass(attr, ImportError)) else: module, name = mapping(module2, name2) if module3[:1] != '_': @@ -401,6 +404,11 @@ if exc is not OSError and issubclass(exc, OSError): self.assertEqual(reverse_mapping('builtins', name), ('exceptions', 'OSError')) + elif exc is not ImportError and issubclass(exc, ImportError): + self.assertEqual(reverse_mapping('builtins', name), + ('exceptions', 'ImportError')) + self.assertEqual(mapping('exceptions', name), + ('exceptions', name)) else: self.assertEqual(reverse_mapping('builtins', name), ('exceptions', name)) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -8043,6 +8043,9 @@ - Issue #18137: Detect integer overflow on precision in float.__format__() and complex.__format__(). +- Issue #15767: Introduce ModuleNotFoundError which is raised when a module + could not be found. + - Issue #18183: Fix various unicode operations on strings with large unicode codepoints. diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -706,6 +706,13 @@ "module."); /* + * ModuleNotFoundError extends ImportError + */ + +MiddlingExtendsException(PyExc_ImportError, ModuleNotFoundError, ImportError, + "Module not found."); + +/* * OSError extends Exception */ @@ -2469,6 +2476,7 @@ PRE_INIT(SystemExit) PRE_INIT(KeyboardInterrupt) PRE_INIT(ImportError) + PRE_INIT(ModuleNotFoundError) PRE_INIT(OSError) PRE_INIT(EOFError) PRE_INIT(RuntimeError) @@ -2541,6 +2549,7 @@ POST_INIT(SystemExit) POST_INIT(KeyboardInterrupt) POST_INIT(ImportError) + POST_INIT(ModuleNotFoundError) POST_INIT(OSError) INIT_ALIAS(EnvironmentError, OSError) INIT_ALIAS(IOError, OSError) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 19:56:37 2016 From: python-checkins at python.org (eric.snow) Date: Wed, 07 Sep 2016 23:56:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2315767=3A_Use_Modu?= =?utf-8?q?leNotFoundError=2E?= Message-ID: <20160907235637.18971.43286.8C935D2E@psf.io> https://hg.python.org/cpython/rev/5fdb8c897023 changeset: 103263:5fdb8c897023 user: Eric Snow date: Wed Sep 07 16:56:15 2016 -0700 summary: Issue #15767: Use ModuleNotFoundError. files: Doc/c-api/exceptions.rst | 7 + Doc/reference/import.rst | 20 +- Doc/whatsnew/3.6.rst | 7 + Include/pyerrors.h | 3 + Lib/importlib/_bootstrap.py | 14 +- Lib/pydoc.py | 2 +- Lib/test/test_cmd_line_script.py | 2 +- Lib/test/test_import/__init__.py | 12 + Lib/test/test_importlib/import_/test_api.py | 4 + Lib/test/test_importlib/import_/test_fromlist.py | 10 +- Lib/test/test_pydoc.py | 2 +- Lib/test/test_site.py | 2 +- Misc/NEWS | 4 + Python/errors.c | 38 +- Python/import.c | 3 +- Python/importlib.h | 501 ++++----- 16 files changed, 342 insertions(+), 289 deletions(-) diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -306,6 +306,13 @@ :mod:`warnings` module and the :option:`-W` option in the command line documentation. There is no C API for warning control. +.. c:function:: PyObject* PyErr_SetImportErrorSubclass(PyObject *msg, PyObject *name, PyObject *path) + + Much like :c:func:`PyErr_SetImportError` but this function allows for + specifying a subclass of :exc:`ImportError` to raise. + + .. versionadded:: 3.4 + .. c:function:: int PyErr_WarnExplicitObject(PyObject *category, PyObject *message, PyObject *filename, int lineno, PyObject *module, PyObject *registry) diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@ -36,7 +36,7 @@ When a module is first imported, Python searches for the module and if found, it creates a module object [#fnmo]_, initializing it. If the named module -cannot be found, an :exc:`ImportError` is raised. Python implements various +cannot be found, an :exc:`ModuleNotFoundError` is raised. Python implements various strategies to search for the named module when the import machinery is invoked. These strategies can be modified and extended by using various hooks described in the sections below. @@ -167,7 +167,7 @@ This name will be used in various phases of the import search, and it may be the dotted path to a submodule, e.g. ``foo.bar.baz``. In this case, Python first tries to import ``foo``, then ``foo.bar``, and finally ``foo.bar.baz``. -If any of the intermediate imports fail, an :exc:`ImportError` is raised. +If any of the intermediate imports fail, an :exc:`ModuleNotFoundError` is raised. The module cache @@ -186,7 +186,7 @@ During import, the module name is looked up in :data:`sys.modules` and if present, the associated value is the module satisfying the import, and the process completes. However, if the value is ``None``, then an -:exc:`ImportError` is raised. If the module name is missing, Python will +:exc:`ModuleNotFoundError` is raised. If the module name is missing, Python will continue searching for the module. :data:`sys.modules` is writable. Deleting a key may not destroy the @@ -194,7 +194,7 @@ but it will invalidate the cache entry for the named module, causing Python to search anew for the named module upon its next import. The key can also be assigned to ``None``, forcing the next import -of the module to result in an :exc:`ImportError`. +of the module to result in an :exc:`ModuleNotFoundError`. Beware though, as if you keep a reference to the module object, invalidate its cache entry in :data:`sys.modules`, and then re-import the @@ -288,8 +288,8 @@ If the meta path finder knows how to handle the named module, it returns a spec object. If it cannot handle the named module, it returns ``None``. If :data:`sys.meta_path` processing reaches the end of its list without returning -a spec, then an :exc:`ImportError` is raised. Any other exceptions raised -are simply propagated up, aborting the import process. +a spec, then a :exc:`ModuleNotFoundError` is raised. Any other exceptions +raised are simply propagated up, aborting the import process. The :meth:`~importlib.abc.MetaPathFinder.find_spec()` method of meta path finders is called with two or three arguments. The first is the fully @@ -298,9 +298,9 @@ top-level modules, the second argument is ``None``, but for submodules or subpackages, the second argument is the value of the parent package's ``__path__`` attribute. If the appropriate ``__path__`` attribute cannot -be accessed, an :exc:`ImportError` is raised. The third argument is an -existing module object that will be the target of loading later. The -import system passes in a target module only during reload. +be accessed, an :exc:`ModuleNotFoundError` is raised. The third argument +is an existing module object that will be the target of loading later. +The import system passes in a target module only during reload. The meta path may be traversed multiple times for a single import request. For example, assuming none of the modules involved has already been cached, @@ -887,7 +887,7 @@ To selectively prevent import of some modules from a hook early on the meta path (rather than disabling the standard import system entirely), -it is sufficient to raise :exc:`ImportError` directly from +it is sufficient to raise :exc:`ModuleNoFoundError` directly from :meth:`~importlib.abc.MetaPathFinder.find_spec` instead of returning ``None``. The latter indicates that the meta path search should continue, while raising an exception terminates it immediately. diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -350,6 +350,10 @@ :ref:`py36-traceback` for an example). (Contributed by Emanuel Barry in :issue:`26823`.) +* Import now raises the new exception :exc:`ModuleNotFoundError` + (subclass of :exc:`ImportError`) when it cannot find a module. Code + that current checks for ImportError (in try-except) will still work. + New Modules =========== @@ -959,6 +963,9 @@ * When :meth:`importlib.abc.Loader.exec_module` is defined, :meth:`importlib.abc.Loader.create_module` must also be defined. +* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg** + argument is not set. Previously only ``NULL`` was returned. + * The format of the ``co_lnotab`` attribute of code objects changed to support negative line number delta. By default, Python does not emit bytecode with negative line number delta. Functions using ``frame.f_lineno``, diff --git a/Include/pyerrors.h b/Include/pyerrors.h --- a/Include/pyerrors.h +++ b/Include/pyerrors.h @@ -284,6 +284,9 @@ PyAPI_FUNC(PyObject *) PyErr_SetExcWithArgsKwargs(PyObject *, PyObject *, PyObject *); + +PyAPI_FUNC(PyObject *) PyErr_SetImportErrorSubclass(PyObject *, PyObject *, + PyObject *, PyObject *); PyAPI_FUNC(PyObject *) PyErr_SetImportError(PyObject *, PyObject *, PyObject *); diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py --- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -943,10 +943,10 @@ path = parent_module.__path__ except AttributeError: msg = (_ERR_MSG + '; {!r} is not a package').format(name, parent) - raise ImportError(msg, name=name) from None + raise ModuleNotFoundError(msg, name=name) from None spec = _find_spec(name, path) if spec is None: - raise ImportError(_ERR_MSG.format(name), name=name) + raise ModuleNotFoundError(_ERR_MSG.format(name), name=name) else: module = _load_unlocked(spec) if parent: @@ -982,10 +982,11 @@ _imp.release_lock() message = ('import of {} halted; ' 'None in sys.modules'.format(name)) - raise ImportError(message, name=name) + raise ModuleNotFoundError(message, name=name) _lock_unlock_module(name) return module + def _handle_fromlist(module, fromlist, import_): """Figure out what __import__ should return. @@ -1007,13 +1008,12 @@ from_name = '{}.{}'.format(module.__name__, x) try: _call_with_frames_removed(import_, from_name) - except ImportError as exc: + except ModuleNotFoundError as exc: # Backwards-compatibility dictates we ignore failed # imports triggered by fromlist for modules that don't # exist. - if str(exc).startswith(_ERR_MSG_PREFIX): - if exc.name == from_name: - continue + if exc.name == from_name: + continue raise return module diff --git a/Lib/pydoc.py b/Lib/pydoc.py --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -350,7 +350,7 @@ elif exc is SyntaxError: # A SyntaxError occurred before we could execute the module. raise ErrorDuringImport(value.filename, info) - elif exc is ImportError and value.name == path: + elif issubclass(exc, ImportError) and value.name == path: # No such module in the path. return None else: diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -428,7 +428,7 @@ ('builtins.x', br'Error while finding module specification.*' br'AttributeError'), ('builtins.x.y', br'Error while finding module specification.*' - br'ImportError.*No module named.*not a package'), + br'ModuleNotFoundError.*No module named.*not a package'), ('os.path', br'loader.*cannot handle'), ('importlib', br'No module named.*' br'is a package and cannot be directly executed'), diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py --- a/Lib/test/test_import/__init__.py +++ b/Lib/test/test_import/__init__.py @@ -69,6 +69,18 @@ def tearDown(self): unload(TESTFN) + def test_import_raises_ModuleNotFoundError(self): + with self.assertRaises(ModuleNotFoundError): + import something_that_should_not_exist_anywhere + + def test_from_import_missing_module_raises_ModuleNotFoundError(self): + with self.assertRaises(ModuleNotFoundError): + from something_that_should_not_exist_anywhere import blah + + def test_from_import_missing_attr_raises_ImportError(self): + with self.assertRaises(ImportError): + from importlib import something_that_should_not_exist_anywhere + def test_case_sensitivity(self): # Brief digression to test that import is case-sensitive: if we got # this far, we know for sure that "random" exists. diff --git a/Lib/test/test_importlib/import_/test_api.py b/Lib/test/test_importlib/import_/test_api.py --- a/Lib/test/test_importlib/import_/test_api.py +++ b/Lib/test/test_importlib/import_/test_api.py @@ -43,6 +43,10 @@ """Test API-specific details for __import__ (e.g. raising the right exception when passing in an int for the module name).""" + def test_raises_ModuleNotFoundError(self): + with self.assertRaises(ModuleNotFoundError): + util.import_importlib('some module that does not exist') + def test_name_requires_rparition(self): # Raise TypeError if a non-string is passed in for the module name. with self.assertRaises(TypeError): diff --git a/Lib/test/test_importlib/import_/test_fromlist.py b/Lib/test/test_importlib/import_/test_fromlist.py --- a/Lib/test/test_importlib/import_/test_fromlist.py +++ b/Lib/test/test_importlib/import_/test_fromlist.py @@ -73,16 +73,16 @@ self.assertTrue(hasattr(module, 'module')) self.assertEqual(module.module.__name__, 'pkg.module') - def test_module_from_package_triggers_ImportError(self): - # If a submodule causes an ImportError because it tries to import - # a module which doesn't exist, that should let the ImportError - # propagate. + def test_module_from_package_triggers_ModuleNotFoundError(self): + # If a submodule causes an ModuleNotFoundError because it tries + # to import a module which doesn't exist, that should let the + # ModuleNotFoundError propagate. def module_code(): import i_do_not_exist with util.mock_modules('pkg.__init__', 'pkg.mod', module_code={'pkg.mod': module_code}) as importer: with util.import_state(meta_path=[importer]): - with self.assertRaises(ImportError) as exc: + with self.assertRaises(ModuleNotFoundError) as exc: self.__import__('pkg', fromlist=['mod']) self.assertEqual('i_do_not_exist', exc.exception.name) diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -263,7 +263,7 @@ Use help(str) for help on the str class.'''.replace('\n', os.linesep) # output pattern for module with bad imports -badimport_pattern = "problem in %s - ImportError: No module named %r" +badimport_pattern = "problem in %s - ModuleNotFoundError: No module named %r" expected_dynamicattribute_pattern = """ Help on class DA in module %s: diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -138,7 +138,7 @@ re.escape(os.path.join(pth_dir, pth_fn))) # XXX: ditto previous XXX comment. self.assertRegex(err_out.getvalue(), 'Traceback') - self.assertRegex(err_out.getvalue(), 'ImportError') + self.assertRegex(err_out.getvalue(), 'ModuleNotFoundError') @unittest.skipIf(sys.platform == "win32", "Windows does not raise an " "error for file paths containing null characters") diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -9966,6 +9966,10 @@ PyImport_ExecCodeModuleWithPathnames() (and thus by extension PyImport_ExecCodeModule() and PyImport_ExecCodeModuleEx()). +- Issue #15767: Added PyErr_SetImportErrorSubclass(). + +- PyErr_SetImportError() now sets TypeError when its msg argument is set. + - Issue #9369: The types of `char*` arguments of PyObject_CallFunction() and PyObject_CallMethod() now changed to `const char*`. Based on patches by J?rg M?ller and Lars Buitinck. diff --git a/Python/errors.c b/Python/errors.c --- a/Python/errors.c +++ b/Python/errors.c @@ -697,27 +697,37 @@ #endif /* MS_WINDOWS */ PyObject * -PyErr_SetImportError(PyObject *msg, PyObject *name, PyObject *path) +PyErr_SetImportErrorSubclass(PyObject *exception, PyObject *msg, + PyObject *name, PyObject *path) { + int issubclass; PyObject *kwargs, *error; + issubclass = PyObject_IsSubclass(exception, PyExc_ImportError); + if (issubclass < 0) { + return NULL; + } + else if (!issubclass) { + PyErr_SetString(PyExc_TypeError, "expected a subclass of ImportError"); + return NULL; + } + if (msg == NULL) { + PyErr_SetString(PyExc_TypeError, "expected a message argument"); return NULL; } + if (name == NULL) { + name = Py_None; + } + if (path == NULL) { + path = Py_None; + } + kwargs = PyDict_New(); if (kwargs == NULL) { return NULL; } - - if (name == NULL) { - name = Py_None; - } - - if (path == NULL) { - path = Py_None; - } - if (PyDict_SetItemString(kwargs, "name", name) < 0) { goto done; } @@ -725,7 +735,7 @@ goto done; } - error = _PyObject_FastCallDict(PyExc_ImportError, &msg, 1, kwargs); + error = _PyObject_FastCallDict(exception, &msg, 1, kwargs); if (error != NULL) { PyErr_SetObject((PyObject *)Py_TYPE(error), error); Py_DECREF(error); @@ -736,6 +746,12 @@ return NULL; } +PyObject * +PyErr_SetImportError(PyObject *msg, PyObject *name, PyObject *path) +{ + return PyErr_SetImportErrorSubclass(PyExc_ImportError, msg, name, path); +} + void _PyErr_BadInternalCall(const char *filename, int lineno) { diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -1539,7 +1539,8 @@ PyObject *msg = PyUnicode_FromFormat("import of %R halted; " "None in sys.modules", abs_name); if (msg != NULL) { - PyErr_SetImportError(msg, abs_name, NULL); + PyErr_SetImportErrorSubclass(PyExc_ModuleNotFoundError, msg, + abs_name, NULL); Py_DECREF(msg); } mod = NULL; diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:01:10 2016 From: python-checkins at python.org (brett.cannon) Date: Thu, 08 Sep 2016 00:01:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327911=3A_Remove_s?= =?utf-8?q?ome_unnecessary_error_checks_in_import=2Ec=2E?= Message-ID: <20160908000110.2824.47812.E6775387@psf.io> https://hg.python.org/cpython/rev/96c6816825dc changeset: 103264:96c6816825dc user: Brett Cannon date: Wed Sep 07 17:00:43 2016 -0700 summary: Issue #27911: Remove some unnecessary error checks in import.c. Thanks to Xiang Zhang for the patch. files: Misc/NEWS | 3 +++ Python/import.c | 8 ++------ 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27911: Remove unnecessary error checks in + import.c:exec_builtin_or_dynamic(). + - Issue #27983: Cause lack of llvm-profdata tool when using clang as required for PGO linking to be a configure time error rather than make time when --with-optimizations is enabled. Also improve our diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -1942,19 +1942,15 @@ def = PyModule_GetDef(mod); if (def == NULL) { - if (PyErr_Occurred()) { - return -1; - } return 0; } + state = PyModule_GetState(mod); - if (PyErr_Occurred()) { - return -1; - } if (state) { /* Already initialized; skip reload */ return 0; } + return PyModule_ExecDef(mod, def); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:25:16 2016 From: python-checkins at python.org (vinay.sajip) Date: Thu, 08 Sep 2016 00:25:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixes_=2327930=3A_improved?= =?utf-8?q?_QueueListener_behaviour=2E?= Message-ID: <20160908002516.69908.17563.FDA5402B@psf.io> https://hg.python.org/cpython/rev/121b5b41c9e8 changeset: 103265:121b5b41c9e8 parent: 103263:5fdb8c897023 user: Vinay Sajip date: Thu Sep 08 01:00:54 2016 +0100 summary: Fixes #27930: improved QueueListener behaviour. files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:25:16 2016 From: python-checkins at python.org (vinay.sajip) Date: Thu, 08 Sep 2016 00:25:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogRml4ZXMgIzI3OTMw?= =?utf-8?q?=3A_improved_QueueListener_behaviour=2E?= Message-ID: <20160908002516.48681.22481.B37EC390@psf.io> https://hg.python.org/cpython/rev/b2ea0ede3753 changeset: 103266:b2ea0ede3753 branch: 3.5 parent: 103258:8f02a1a6b647 user: Vinay Sajip date: Thu Sep 08 01:13:39 2016 +0100 summary: Fixes #27930: improved QueueListener behaviour. files: Lib/logging/handlers.py | 21 +----- Lib/test/test_logging.py | 90 ++++++++++++++++++++++++++- Misc/NEWS | 3 + 3 files changed, 93 insertions(+), 21 deletions(-) diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -1,4 +1,4 @@ -# Copyright 2001-2015 by Vinay Sajip. All Rights Reserved. +# Copyright 2001-2016 by Vinay Sajip. All Rights Reserved. # # Permission to use, copy, modify, and distribute this software and its # documentation for any purpose and without fee is hereby granted, @@ -18,7 +18,7 @@ Additional handlers for the logging package for Python. The core package is based on PEP 282 and comments thereto in comp.lang.python. -Copyright (C) 2001-2015 Vinay Sajip. All Rights Reserved. +Copyright (C) 2001-2016 Vinay Sajip. All Rights Reserved. To use, simply 'import logging.handlers' and log away! """ @@ -1366,7 +1366,6 @@ """ self.queue = queue self.handlers = handlers - self._stop = threading.Event() self._thread = None self.respect_handler_level = respect_handler_level @@ -1387,7 +1386,7 @@ LogRecords to process. """ self._thread = t = threading.Thread(target=self._monitor) - t.setDaemon(True) + t.daemon = True t.start() def prepare(self , record): @@ -1426,7 +1425,7 @@ """ q = self.queue has_task_done = hasattr(q, 'task_done') - while not self._stop.isSet(): + while True: try: record = self.dequeue(True) if record is self._sentinel: @@ -1435,17 +1434,6 @@ if has_task_done: q.task_done() except queue.Empty: - pass - # There might still be records in the queue. - while True: - try: - record = self.dequeue(False) - if record is self._sentinel: - break - self.handle(record) - if has_task_done: - q.task_done() - except queue.Empty: break def enqueue_sentinel(self): @@ -1466,7 +1454,6 @@ Note that if you don't call this before your application exits, there may be some records still left on the queue, which won't be processed. """ - self._stop.set() self.enqueue_sentinel() self._thread.join() self._thread = None diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -1,4 +1,4 @@ -# Copyright 2001-2014 by Vinay Sajip. All Rights Reserved. +# Copyright 2001-2016 by Vinay Sajip. All Rights Reserved. # # Permission to use, copy, modify, and distribute this software and its # documentation for any purpose and without fee is hereby granted, @@ -16,7 +16,7 @@ """Test harness for the logging module. Run all tests. -Copyright (C) 2001-2014 Vinay Sajip. All Rights Reserved. +Copyright (C) 2001-2016 Vinay Sajip. All Rights Reserved. """ import logging @@ -3022,6 +3022,84 @@ self.assertFalse(handler.matches(levelno=logging.ERROR, message='5')) self.assertTrue(handler.matches(levelno=logging.CRITICAL, message='6')) +if hasattr(logging.handlers, 'QueueListener'): + import multiprocessing + from unittest.mock import patch + + class QueueListenerTest(BaseTest): + """ + Tests based on patch submitted for issue #27930. Ensure that + QueueListener handles all log messages. + """ + + repeat = 20 + + @staticmethod + def setup_and_log(log_queue, ident): + """ + Creates a logger with a QueueHandler that logs to a queue read by a + QueueListener. Starts the listener, logs five messages, and stops + the listener. + """ + logger = logging.getLogger('test_logger_with_id_%s' % ident) + logger.setLevel(logging.DEBUG) + handler = logging.handlers.QueueHandler(log_queue) + logger.addHandler(handler) + listener = logging.handlers.QueueListener(log_queue) + listener.start() + + logger.info('one') + logger.info('two') + logger.info('three') + logger.info('four') + logger.info('five') + + listener.stop() + logger.removeHandler(handler) + handler.close() + + @patch.object(logging.handlers.QueueListener, 'handle') + def test_handle_called_with_queue_queue(self, mock_handle): + for i in range(self.repeat): + log_queue = queue.Queue() + self.setup_and_log(log_queue, '%s_%s' % (self.id(), i)) + self.assertEqual(mock_handle.call_count, 5 * self.repeat, + 'correct number of handled log messages') + + @patch.object(logging.handlers.QueueListener, 'handle') + def test_handle_called_with_mp_queue(self, mock_handle): + for i in range(self.repeat): + log_queue = multiprocessing.Queue() + self.setup_and_log(log_queue, '%s_%s' % (self.id(), i)) + self.assertEqual(mock_handle.call_count, 5 * self.repeat, + 'correct number of handled log messages') + + @staticmethod + def get_all_from_queue(log_queue): + try: + while True: + yield log_queue.get_nowait() + except queue.Empty: + return [] + + def test_no_messages_in_queue_after_stop(self): + """ + Five messages are logged then the QueueListener is stopped. This + test then gets everything off the queue. Failure of this test + indicates that messages were not registered on the queue until + _after_ the QueueListener stopped. + """ + for i in range(self.repeat): + queue = multiprocessing.Queue() + self.setup_and_log(queue, '%s_%s' %(self.id(), i)) + # time.sleep(1) + items = list(self.get_all_from_queue(queue)) + expected = [[], [logging.handlers.QueueListener._sentinel]] + self.assertIn(items, expected, + 'Found unexpected messages in queue: %s' % ( + [m.msg if isinstance(m, logging.LogRecord) + else m for m in items])) + ZERO = datetime.timedelta(0) @@ -4166,7 +4244,7 @@ # first and restore it at the end. @support.run_with_locale('LC_ALL', '') def test_main(): - support.run_unittest( + tests = [ BuiltinLevelsTest, BasicFilterTest, CustomLevelsAndFiltersTest, HandlerTest, MemoryHandlerTest, ConfigFileTest, SocketHandlerTest, DatagramHandlerTest, MemoryTest, EncodingTest, WarningsTest, @@ -4177,7 +4255,11 @@ RotatingFileHandlerTest, LastResortTest, LogRecordTest, ExceptionTest, SysLogHandlerTest, HTTPHandlerTest, NTEventLogHandlerTest, TimedRotatingFileHandlerTest, - UnixSocketHandlerTest, UnixDatagramHandlerTest, UnixSysLogHandlerTest) + UnixSocketHandlerTest, UnixDatagramHandlerTest, UnixSysLogHandlerTest, + ] + if hasattr(logging.handlers, 'QueueListener'): + tests.append(QueueListenerTest) + support.run_unittest(*tests) if __name__ == "__main__": test_main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -218,6 +218,9 @@ - Issue #27392: Add loop.connect_accepted_socket(). Patch by Jim Fulton. +- Issue #27930: Improved behaviour of logging.handlers.QueueListener. + Thanks to Paulo Andrade and Petr Viktorin for the analysis and patch. + IDLE ---- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:25:17 2016 From: python-checkins at python.org (vinay.sajip) Date: Thu, 08 Sep 2016 00:25:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2327930=3A_Merged_fix_from_3=2E5=2E?= Message-ID: <20160908002516.69808.85651.DA501150@psf.io> https://hg.python.org/cpython/rev/89b185372b2c changeset: 103267:89b185372b2c parent: 103264:96c6816825dc parent: 103266:b2ea0ede3753 user: Vinay Sajip date: Thu Sep 08 01:24:12 2016 +0100 summary: Closes #27930: Merged fix from 3.5. files: Lib/logging/handlers.py | 17 +-- Lib/test/test_logging.py | 160 ++++++++++++++------------ Misc/NEWS | 3 + 3 files changed, 91 insertions(+), 89 deletions(-) diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -1388,7 +1388,6 @@ """ self.queue = queue self.handlers = handlers - self._stop = threading.Event() self._thread = None self.respect_handler_level = respect_handler_level @@ -1409,7 +1408,7 @@ LogRecords to process. """ self._thread = t = threading.Thread(target=self._monitor) - t.setDaemon(True) + t.daemon = True t.start() def prepare(self , record): @@ -1448,7 +1447,7 @@ """ q = self.queue has_task_done = hasattr(q, 'task_done') - while not self._stop.isSet(): + while True: try: record = self.dequeue(True) if record is self._sentinel: @@ -1457,17 +1456,6 @@ if has_task_done: q.task_done() except queue.Empty: - pass - # There might still be records in the queue. - while True: - try: - record = self.dequeue(False) - if record is self._sentinel: - break - self.handle(record) - if has_task_done: - q.task_done() - except queue.Empty: break def enqueue_sentinel(self): @@ -1488,7 +1476,6 @@ Note that if you don't call this before your application exits, there may be some records still left on the queue, which won't be processed. """ - self._stop.set() self.enqueue_sentinel() self._thread.join() self._thread = None diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -1,4 +1,4 @@ -# Copyright 2001-2014 by Vinay Sajip. All Rights Reserved. +# Copyright 2001-2016 by Vinay Sajip. All Rights Reserved. # # Permission to use, copy, modify, and distribute this software and its # documentation for any purpose and without fee is hereby granted, @@ -16,7 +16,7 @@ """Test harness for the logging module. Run all tests. -Copyright (C) 2001-2014 Vinay Sajip. All Rights Reserved. +Copyright (C) 2001-2016 Vinay Sajip. All Rights Reserved. """ import logging @@ -26,7 +26,6 @@ import codecs import configparser import datetime -import pathlib import pickle import io import gc @@ -309,10 +308,6 @@ self.assertEqual(logging.getLevelName('INFO'), logging.INFO) self.assertEqual(logging.getLevelName(logging.INFO), 'INFO') - def test_issue27935(self): - fatal = logging.getLevelName('FATAL') - self.assertEqual(fatal, logging.FATAL) - class BasicFilterTest(BaseTest): """Test the bundled Filter class.""" @@ -580,29 +575,6 @@ self.assertFalse(h.shouldFlush(r)) h.close() - def test_path_objects(self): - """ - Test that Path objects are accepted as filename arguments to handlers. - - See Issue #27493. - """ - fd, fn = tempfile.mkstemp() - os.close(fd) - os.unlink(fn) - pfn = pathlib.Path(fn) - cases = ( - (logging.FileHandler, (pfn, 'w')), - (logging.handlers.RotatingFileHandler, (pfn, 'a')), - (logging.handlers.TimedRotatingFileHandler, (pfn, 'h')), - ) - if sys.platform in ('linux', 'darwin'): - cases += ((logging.handlers.WatchedFileHandler, (pfn, 'w')),) - for cls, args in cases: - h = cls(*args) - self.assertTrue(os.path.exists(fn)) - h.close() - os.unlink(fn) - @unittest.skipIf(os.name == 'nt', 'WatchedFileHandler not appropriate for Windows.') @unittest.skipUnless(threading, 'Threading required for this test.') def test_race(self): @@ -986,7 +958,7 @@ def setUp(self): BaseTest.setUp(self) self.mem_hdlr = logging.handlers.MemoryHandler(10, logging.WARNING, - self.root_hdlr) + self.root_hdlr) self.mem_logger = logging.getLogger('mem') self.mem_logger.propagate = 0 self.mem_logger.addHandler(self.mem_hdlr) @@ -1023,36 +995,6 @@ self.mem_logger.debug(self.next_message()) self.assert_log_lines(lines) - def test_flush_on_close(self): - """ - Test that the flush-on-close configuration works as expected. - """ - self.mem_logger.debug(self.next_message()) - self.assert_log_lines([]) - self.mem_logger.info(self.next_message()) - self.assert_log_lines([]) - self.mem_logger.removeHandler(self.mem_hdlr) - # Default behaviour is to flush on close. Check that it happens. - self.mem_hdlr.close() - lines = [ - ('DEBUG', '1'), - ('INFO', '2'), - ] - self.assert_log_lines(lines) - # Now configure for flushing not to be done on close. - self.mem_hdlr = logging.handlers.MemoryHandler(10, logging.WARNING, - self.root_hdlr, - False) - self.mem_logger.addHandler(self.mem_hdlr) - self.mem_logger.debug(self.next_message()) - self.assert_log_lines(lines) # no change - self.mem_logger.info(self.next_message()) - self.assert_log_lines(lines) # no change - self.mem_logger.removeHandler(self.mem_hdlr) - self.mem_hdlr.close() - # assert that no new lines have been added - self.assert_log_lines(lines) # no change - class ExceptionFormatter(logging.Formatter): """A special exception formatter.""" @@ -3080,6 +3022,84 @@ self.assertFalse(handler.matches(levelno=logging.ERROR, message='5')) self.assertTrue(handler.matches(levelno=logging.CRITICAL, message='6')) +if hasattr(logging.handlers, 'QueueListener'): + import multiprocessing + from unittest.mock import patch + + class QueueListenerTest(BaseTest): + """ + Tests based on patch submitted for issue #27930. Ensure that + QueueListener handles all log messages. + """ + + repeat = 20 + + @staticmethod + def setup_and_log(log_queue, ident): + """ + Creates a logger with a QueueHandler that logs to a queue read by a + QueueListener. Starts the listener, logs five messages, and stops + the listener. + """ + logger = logging.getLogger('test_logger_with_id_%s' % ident) + logger.setLevel(logging.DEBUG) + handler = logging.handlers.QueueHandler(log_queue) + logger.addHandler(handler) + listener = logging.handlers.QueueListener(log_queue) + listener.start() + + logger.info('one') + logger.info('two') + logger.info('three') + logger.info('four') + logger.info('five') + + listener.stop() + logger.removeHandler(handler) + handler.close() + + @patch.object(logging.handlers.QueueListener, 'handle') + def test_handle_called_with_queue_queue(self, mock_handle): + for i in range(self.repeat): + log_queue = queue.Queue() + self.setup_and_log(log_queue, '%s_%s' % (self.id(), i)) + self.assertEqual(mock_handle.call_count, 5 * self.repeat, + 'correct number of handled log messages') + + @patch.object(logging.handlers.QueueListener, 'handle') + def test_handle_called_with_mp_queue(self, mock_handle): + for i in range(self.repeat): + log_queue = multiprocessing.Queue() + self.setup_and_log(log_queue, '%s_%s' % (self.id(), i)) + self.assertEqual(mock_handle.call_count, 5 * self.repeat, + 'correct number of handled log messages') + + @staticmethod + def get_all_from_queue(log_queue): + try: + while True: + yield log_queue.get_nowait() + except queue.Empty: + return [] + + def test_no_messages_in_queue_after_stop(self): + """ + Five messages are logged then the QueueListener is stopped. This + test then gets everything off the queue. Failure of this test + indicates that messages were not registered on the queue until + _after_ the QueueListener stopped. + """ + for i in range(self.repeat): + queue = multiprocessing.Queue() + self.setup_and_log(queue, '%s_%s' %(self.id(), i)) + # time.sleep(1) + items = list(self.get_all_from_queue(queue)) + expected = [[], [logging.handlers.QueueListener._sentinel]] + self.assertIn(items, expected, + 'Found unexpected messages in queue: %s' % ( + [m.msg if isinstance(m, logging.LogRecord) + else m for m in items])) + ZERO = datetime.timedelta(0) @@ -4219,23 +4239,12 @@ msg = 'Record not found in event log, went back %d records' % GO_BACK self.assertTrue(found, msg=msg) - -class MiscTestCase(unittest.TestCase): - def test__all__(self): - blacklist = {'logThreads', 'logMultiprocessing', - 'logProcesses', 'currentframe', - 'PercentStyle', 'StrFormatStyle', 'StringTemplateStyle', - 'Filterer', 'PlaceHolder', 'Manager', 'RootLogger', - 'root'} - support.check__all__(self, logging, blacklist=blacklist) - - # Set the locale to the platform-dependent default. I have no idea # why the test does this, but in any case we save the current locale # first and restore it at the end. @support.run_with_locale('LC_ALL', '') def test_main(): - support.run_unittest( + tests = [ BuiltinLevelsTest, BasicFilterTest, CustomLevelsAndFiltersTest, HandlerTest, MemoryHandlerTest, ConfigFileTest, SocketHandlerTest, DatagramHandlerTest, MemoryTest, EncodingTest, WarningsTest, @@ -4247,7 +4256,10 @@ ExceptionTest, SysLogHandlerTest, HTTPHandlerTest, NTEventLogHandlerTest, TimedRotatingFileHandlerTest, UnixSocketHandlerTest, UnixDatagramHandlerTest, UnixSysLogHandlerTest, - MiscTestCase) + ] + if hasattr(logging.handlers, 'QueueListener'): + tests.append(QueueListenerTest) + support.run_unittest(*tests) if __name__ == "__main__": test_main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -225,6 +225,9 @@ - Issue #27573: exit message for code.interact is now configurable. +- Issue #27930: Improved behaviour of logging.handlers.QueueListener. + Thanks to Paulo Andrade and Petr Viktorin for the analysis and patch. + C API ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:25:17 2016 From: python-checkins at python.org (vinay.sajip) Date: Thu, 08 Sep 2016 00:25:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Null_merge=2E?= Message-ID: <20160908002517.87545.16018.45DA1FDF@psf.io> https://hg.python.org/cpython/rev/15d79fcaf98c changeset: 103268:15d79fcaf98c parent: 103267:89b185372b2c parent: 103265:121b5b41c9e8 user: Vinay Sajip date: Thu Sep 08 01:25:04 2016 +0100 summary: Null merge. files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:27:48 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 00:27:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328005=3A_Allow_Im?= =?utf-8?q?portErrors_in_encoding_implementation_to_propagate=2E?= Message-ID: <20160908002747.2824.2204.17127D80@psf.io> https://hg.python.org/cpython/rev/2b2e7b39bb68 changeset: 103269:2b2e7b39bb68 user: Steve Dower date: Wed Sep 07 17:27:33 2016 -0700 summary: Issue #28005: Allow ImportErrors in encoding implementation to propagate. files: Lib/encodings/__init__.py | 5 +++-- Misc/NEWS | 2 ++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py --- a/Lib/encodings/__init__.py +++ b/Lib/encodings/__init__.py @@ -98,8 +98,9 @@ # module with side-effects that is not in the 'encodings' package. mod = __import__('encodings.' + modname, fromlist=_import_tail, level=0) - except ImportError: - pass + except ModuleNotFoundError as ex: + if ex.name != 'encodings.' + modname: + raise else: break else: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -99,6 +99,8 @@ Library ------- +- Issue #28005: Allow ImportErrors in encoding implementation to propagate. + - Issue #27570: Avoid zero-length memcpy() etc calls with null source pointers in the "ctypes" and "array" modules. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:37:09 2016 From: python-checkins at python.org (vinay.sajip) Date: Thu, 08 Sep 2016 00:37:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Added_back_test_code_lost_?= =?utf-8?q?during_merge=2E?= Message-ID: <20160908003708.8490.68565.394BD0D8@psf.io> https://hg.python.org/cpython/rev/fffa689ba0c4 changeset: 103270:fffa689ba0c4 user: Vinay Sajip date: Thu Sep 08 01:37:03 2016 +0100 summary: Added back test code lost during merge. files: Lib/test/test_logging.py | 72 +++++++++++++++++++++++++++- 1 files changed, 71 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -26,6 +26,7 @@ import codecs import configparser import datetime +import pathlib import pickle import io import gc @@ -308,6 +309,10 @@ self.assertEqual(logging.getLevelName('INFO'), logging.INFO) self.assertEqual(logging.getLevelName(logging.INFO), 'INFO') + def test_issue27935(self): + fatal = logging.getLevelName('FATAL') + self.assertEqual(fatal, logging.FATAL) + class BasicFilterTest(BaseTest): """Test the bundled Filter class.""" @@ -575,6 +580,29 @@ self.assertFalse(h.shouldFlush(r)) h.close() + def test_path_objects(self): + """ + Test that Path objects are accepted as filename arguments to handlers. + + See Issue #27493. + """ + fd, fn = tempfile.mkstemp() + os.close(fd) + os.unlink(fn) + pfn = pathlib.Path(fn) + cases = ( + (logging.FileHandler, (pfn, 'w')), + (logging.handlers.RotatingFileHandler, (pfn, 'a')), + (logging.handlers.TimedRotatingFileHandler, (pfn, 'h')), + ) + if sys.platform in ('linux', 'darwin'): + cases += ((logging.handlers.WatchedFileHandler, (pfn, 'w')),) + for cls, args in cases: + h = cls(*args) + self.assertTrue(os.path.exists(fn)) + h.close() + os.unlink(fn) + @unittest.skipIf(os.name == 'nt', 'WatchedFileHandler not appropriate for Windows.') @unittest.skipUnless(threading, 'Threading required for this test.') def test_race(self): @@ -958,7 +986,7 @@ def setUp(self): BaseTest.setUp(self) self.mem_hdlr = logging.handlers.MemoryHandler(10, logging.WARNING, - self.root_hdlr) + self.root_hdlr) self.mem_logger = logging.getLogger('mem') self.mem_logger.propagate = 0 self.mem_logger.addHandler(self.mem_hdlr) @@ -995,6 +1023,36 @@ self.mem_logger.debug(self.next_message()) self.assert_log_lines(lines) + def test_flush_on_close(self): + """ + Test that the flush-on-close configuration works as expected. + """ + self.mem_logger.debug(self.next_message()) + self.assert_log_lines([]) + self.mem_logger.info(self.next_message()) + self.assert_log_lines([]) + self.mem_logger.removeHandler(self.mem_hdlr) + # Default behaviour is to flush on close. Check that it happens. + self.mem_hdlr.close() + lines = [ + ('DEBUG', '1'), + ('INFO', '2'), + ] + self.assert_log_lines(lines) + # Now configure for flushing not to be done on close. + self.mem_hdlr = logging.handlers.MemoryHandler(10, logging.WARNING, + self.root_hdlr, + False) + self.mem_logger.addHandler(self.mem_hdlr) + self.mem_logger.debug(self.next_message()) + self.assert_log_lines(lines) # no change + self.mem_logger.info(self.next_message()) + self.assert_log_lines(lines) # no change + self.mem_logger.removeHandler(self.mem_hdlr) + self.mem_hdlr.close() + # assert that no new lines have been added + self.assert_log_lines(lines) # no change + class ExceptionFormatter(logging.Formatter): """A special exception formatter.""" @@ -4239,6 +4297,17 @@ msg = 'Record not found in event log, went back %d records' % GO_BACK self.assertTrue(found, msg=msg) + +class MiscTestCase(unittest.TestCase): + def test__all__(self): + blacklist = {'logThreads', 'logMultiprocessing', + 'logProcesses', 'currentframe', + 'PercentStyle', 'StrFormatStyle', 'StringTemplateStyle', + 'Filterer', 'PlaceHolder', 'Manager', 'RootLogger', + 'root'} + support.check__all__(self, logging, blacklist=blacklist) + + # Set the locale to the platform-dependent default. I have no idea # why the test does this, but in any case we save the current locale # first and restore it at the end. @@ -4256,6 +4325,7 @@ ExceptionTest, SysLogHandlerTest, HTTPHandlerTest, NTEventLogHandlerTest, TimedRotatingFileHandlerTest, UnixSocketHandlerTest, UnixDatagramHandlerTest, UnixSysLogHandlerTest, + MiscTestCase ] if hasattr(logging.handlers, 'QueueListener'): tests.append(QueueListenerTest) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:48:31 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 00:48:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogRml4IGluZGVudGF0aW9uIChyZWluZGVudC5weSku?= Message-ID: <20160908004831.12375.38565.8AF39326@psf.io> https://hg.python.org/cpython/rev/56bd6ff4c98e changeset: 103275:56bd6ff4c98e parent: 103273:7b9ef58da99d parent: 103274:99ce1713ae15 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 00:48:22 2016 +0000 summary: Fix indentation (reindent.py). files: Lib/lib2to3/pgen2/grammar.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Lib/lib2to3/pgen2/grammar.py b/Lib/lib2to3/pgen2/grammar.py --- a/Lib/lib2to3/pgen2/grammar.py +++ b/Lib/lib2to3/pgen2/grammar.py @@ -139,12 +139,12 @@ def _make_deterministic(top): if isinstance(top, dict): - return collections.OrderedDict( - sorted(((k, _make_deterministic(v)) for k, v in top.items()))) + return collections.OrderedDict( + sorted(((k, _make_deterministic(v)) for k, v in top.items()))) if isinstance(top, list): - return [_make_deterministic(e) for e in top] + return [_make_deterministic(e) for e in top] if isinstance(top, tuple): - return tuple(_make_deterministic(e) for e in top) + return tuple(_make_deterministic(e) for e in top) return top -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:48:31 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 00:48:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <20160908004831.36224.2799.1486B435@psf.io> https://hg.python.org/cpython/rev/7b9ef58da99d changeset: 103273:7b9ef58da99d parent: 103272:e4faff120be2 parent: 103270:fffa689ba0c4 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 00:46:58 2016 +0000 summary: merge heads files: Lib/test/test_logging.py | 72 +++++++++++++++++++++++++++- 1 files changed, 71 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -26,6 +26,7 @@ import codecs import configparser import datetime +import pathlib import pickle import io import gc @@ -308,6 +309,10 @@ self.assertEqual(logging.getLevelName('INFO'), logging.INFO) self.assertEqual(logging.getLevelName(logging.INFO), 'INFO') + def test_issue27935(self): + fatal = logging.getLevelName('FATAL') + self.assertEqual(fatal, logging.FATAL) + class BasicFilterTest(BaseTest): """Test the bundled Filter class.""" @@ -575,6 +580,29 @@ self.assertFalse(h.shouldFlush(r)) h.close() + def test_path_objects(self): + """ + Test that Path objects are accepted as filename arguments to handlers. + + See Issue #27493. + """ + fd, fn = tempfile.mkstemp() + os.close(fd) + os.unlink(fn) + pfn = pathlib.Path(fn) + cases = ( + (logging.FileHandler, (pfn, 'w')), + (logging.handlers.RotatingFileHandler, (pfn, 'a')), + (logging.handlers.TimedRotatingFileHandler, (pfn, 'h')), + ) + if sys.platform in ('linux', 'darwin'): + cases += ((logging.handlers.WatchedFileHandler, (pfn, 'w')),) + for cls, args in cases: + h = cls(*args) + self.assertTrue(os.path.exists(fn)) + h.close() + os.unlink(fn) + @unittest.skipIf(os.name == 'nt', 'WatchedFileHandler not appropriate for Windows.') @unittest.skipUnless(threading, 'Threading required for this test.') def test_race(self): @@ -958,7 +986,7 @@ def setUp(self): BaseTest.setUp(self) self.mem_hdlr = logging.handlers.MemoryHandler(10, logging.WARNING, - self.root_hdlr) + self.root_hdlr) self.mem_logger = logging.getLogger('mem') self.mem_logger.propagate = 0 self.mem_logger.addHandler(self.mem_hdlr) @@ -995,6 +1023,36 @@ self.mem_logger.debug(self.next_message()) self.assert_log_lines(lines) + def test_flush_on_close(self): + """ + Test that the flush-on-close configuration works as expected. + """ + self.mem_logger.debug(self.next_message()) + self.assert_log_lines([]) + self.mem_logger.info(self.next_message()) + self.assert_log_lines([]) + self.mem_logger.removeHandler(self.mem_hdlr) + # Default behaviour is to flush on close. Check that it happens. + self.mem_hdlr.close() + lines = [ + ('DEBUG', '1'), + ('INFO', '2'), + ] + self.assert_log_lines(lines) + # Now configure for flushing not to be done on close. + self.mem_hdlr = logging.handlers.MemoryHandler(10, logging.WARNING, + self.root_hdlr, + False) + self.mem_logger.addHandler(self.mem_hdlr) + self.mem_logger.debug(self.next_message()) + self.assert_log_lines(lines) # no change + self.mem_logger.info(self.next_message()) + self.assert_log_lines(lines) # no change + self.mem_logger.removeHandler(self.mem_hdlr) + self.mem_hdlr.close() + # assert that no new lines have been added + self.assert_log_lines(lines) # no change + class ExceptionFormatter(logging.Formatter): """A special exception formatter.""" @@ -4239,6 +4297,17 @@ msg = 'Record not found in event log, went back %d records' % GO_BACK self.assertTrue(found, msg=msg) + +class MiscTestCase(unittest.TestCase): + def test__all__(self): + blacklist = {'logThreads', 'logMultiprocessing', + 'logProcesses', 'currentframe', + 'PercentStyle', 'StrFormatStyle', 'StringTemplateStyle', + 'Filterer', 'PlaceHolder', 'Manager', 'RootLogger', + 'root'} + support.check__all__(self, logging, blacklist=blacklist) + + # Set the locale to the platform-dependent default. I have no idea # why the test does this, but in any case we save the current locale # first and restore it at the end. @@ -4256,6 +4325,7 @@ ExceptionTest, SysLogHandlerTest, HTTPHandlerTest, NTEventLogHandlerTest, TimedRotatingFileHandlerTest, UnixSocketHandlerTest, UnixDatagramHandlerTest, UnixSysLogHandlerTest, + MiscTestCase ] if hasattr(logging.handlers, 'QueueListener'): tests.append(QueueListenerTest) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:48:30 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 00:48:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogbGliMnRvMy5wZ2Vu?= =?utf-8?q?3=2Edriver=2Eload=5Fgrammar=28=29_now_creates_a_stable_cache_fi?= =?utf-8?q?le?= Message-ID: <20160908004830.48696.40520.DA3CC8A0@psf.io> https://hg.python.org/cpython/rev/186bb8dc5540 changeset: 103271:186bb8dc5540 branch: 3.5 parent: 103266:b2ea0ede3753 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 00:40:07 2016 +0000 summary: lib2to3.pgen3.driver.load_grammar() now creates a stable cache file between runs given the same Grammar.txt input regardless of the hash randomization setting. files: Lib/lib2to3/pgen2/driver.py | 15 ++- Lib/lib2to3/pgen2/grammar.py | 28 +++++++- Lib/lib2to3/pgen2/pgen.py | 8 +- Lib/lib2to3/tests/support.py | 6 +- Lib/lib2to3/tests/test_parser.py | 72 +++++++++++++++++++- Misc/NEWS | 4 + 6 files changed, 115 insertions(+), 18 deletions(-) diff --git a/Lib/lib2to3/pgen2/driver.py b/Lib/lib2to3/pgen2/driver.py --- a/Lib/lib2to3/pgen2/driver.py +++ b/Lib/lib2to3/pgen2/driver.py @@ -106,16 +106,19 @@ return self.parse_tokens(tokens, debug) +def _generate_pickle_name(gt): + head, tail = os.path.splitext(gt) + if tail == ".txt": + tail = "" + return head + tail + ".".join(map(str, sys.version_info)) + ".pickle" + + def load_grammar(gt="Grammar.txt", gp=None, save=True, force=False, logger=None): """Load the grammar (maybe from a pickle).""" if logger is None: logger = logging.getLogger() - if gp is None: - head, tail = os.path.splitext(gt) - if tail == ".txt": - tail = "" - gp = head + tail + ".".join(map(str, sys.version_info)) + ".pickle" + gp = _generate_pickle_name(gt) if gp is None else gp if force or not _newer(gp, gt): logger.info("Generating grammar tables from %s", gt) g = pgen.generate_grammar(gt) @@ -124,7 +127,7 @@ try: g.dump(gp) except OSError as e: - logger.info("Writing failed:"+str(e)) + logger.info("Writing failed: %s", e) else: g = grammar.Grammar() g.load(gp) diff --git a/Lib/lib2to3/pgen2/grammar.py b/Lib/lib2to3/pgen2/grammar.py --- a/Lib/lib2to3/pgen2/grammar.py +++ b/Lib/lib2to3/pgen2/grammar.py @@ -13,6 +13,7 @@ """ # Python imports +import collections import pickle # Local imports @@ -85,9 +86,21 @@ self.start = 256 def dump(self, filename): - """Dump the grammar tables to a pickle file.""" + """Dump the grammar tables to a pickle file. + + dump() recursively changes all dict to OrderedDict, so the pickled file + is not exactly the same as what was passed in to dump(). load() uses the + pickled file to create the tables, but only changes OrderedDict to dict + at the top level; it does not recursively change OrderedDict to dict. + So, the loaded tables are different from the original tables that were + passed to load() in that some of the OrderedDict (from the pickled file) + are not changed back to dict. For parsing, this has no effect on + performance because OrderedDict uses dict's __getitem__ with nothing in + between. + """ with open(filename, "wb") as f: - pickle.dump(self.__dict__, f, 2) + d = _make_deterministic(self.__dict__) + pickle.dump(d, f, 2) def load(self, filename): """Load the grammar tables from a pickle file.""" @@ -124,6 +137,17 @@ print("start", self.start) +def _make_deterministic(top): + if isinstance(top, dict): + return collections.OrderedDict( + sorted(((k, _make_deterministic(v)) for k, v in top.items()))) + if isinstance(top, list): + return [_make_deterministic(e) for e in top] + if isinstance(top, tuple): + return tuple(_make_deterministic(e) for e in top) + return top + + # Map from operator to number (since tokenize doesn't do this) opmap_raw = """ diff --git a/Lib/lib2to3/pgen2/pgen.py b/Lib/lib2to3/pgen2/pgen.py --- a/Lib/lib2to3/pgen2/pgen.py +++ b/Lib/lib2to3/pgen2/pgen.py @@ -39,7 +39,7 @@ states = [] for state in dfa: arcs = [] - for label, next in state.arcs.items(): + for label, next in sorted(state.arcs.items()): arcs.append((self.make_label(c, label), dfa.index(next))) if state.isfinal: arcs.append((0, dfa.index(state))) @@ -52,7 +52,7 @@ def make_first(self, c, name): rawfirst = self.first[name] first = {} - for label in rawfirst: + for label in sorted(rawfirst): ilabel = self.make_label(c, label) ##assert ilabel not in first # XXX failed on <> ... != first[ilabel] = 1 @@ -192,7 +192,7 @@ for label, next in nfastate.arcs: if label is not None: addclosure(next, arcs.setdefault(label, {})) - for label, nfaset in arcs.items(): + for label, nfaset in sorted(arcs.items()): for st in states: if st.nfaset == nfaset: break @@ -222,7 +222,7 @@ print("Dump of DFA for", name) for i, state in enumerate(dfa): print(" State", i, state.isfinal and "(final)" or "") - for label, next in state.arcs.items(): + for label, next in sorted(state.arcs.items()): print(" %s -> %d" % (label, dfa.index(next))) def simplify_dfa(self, dfa): diff --git a/Lib/lib2to3/tests/support.py b/Lib/lib2to3/tests/support.py --- a/Lib/lib2to3/tests/support.py +++ b/Lib/lib2to3/tests/support.py @@ -11,13 +11,13 @@ # Local imports from lib2to3 import pytree, refactor -from lib2to3.pgen2 import driver +from lib2to3.pgen2 import driver as pgen2_driver test_dir = os.path.dirname(__file__) proj_dir = os.path.normpath(os.path.join(test_dir, "..")) grammar_path = os.path.join(test_dir, "..", "Grammar.txt") -grammar = driver.load_grammar(grammar_path) -driver = driver.Driver(grammar, convert=pytree.convert) +grammar = pgen2_driver.load_grammar(grammar_path) +driver = pgen2_driver.Driver(grammar, convert=pytree.convert) def parse_string(string): return driver.parse_string(reformat(string), debug=True) diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -6,8 +6,6 @@ test_grammar.py files from both Python 2 and Python 3. """ -from __future__ import with_statement - # Testing imports from . import support from .support import driver, test_dir @@ -15,12 +13,15 @@ # Python imports import os +import shutil +import subprocess import sys +import tempfile import unittest import warnings -import subprocess # Local imports +from lib2to3.pgen2 import driver as pgen2_driver from lib2to3.pgen2 import tokenize from ..pgen2.parse import ParseError from lib2to3.pygram import python_symbols as syms @@ -35,6 +36,71 @@ self.assertEqual(t.children[1].children[0].type, syms.print_stmt) +class TestPgen2Caching(support.TestCase): + def test_load_grammar_from_txt_file(self): + pgen2_driver.load_grammar(support.grammar_path, save=False, force=True) + + def test_load_grammar_from_pickle(self): + # Make a copy of the grammar file in a temp directory we are + # guaranteed to be able to write to. + tmpdir = tempfile.mkdtemp() + try: + grammar_copy = os.path.join( + tmpdir, os.path.basename(support.grammar_path)) + shutil.copy(support.grammar_path, grammar_copy) + pickle_name = pgen2_driver._generate_pickle_name(grammar_copy) + + pgen2_driver.load_grammar(grammar_copy, save=True, force=True) + self.assertTrue(os.path.exists(pickle_name)) + + os.unlink(grammar_copy) # Only the pickle remains... + pgen2_driver.load_grammar(grammar_copy, save=False, force=False) + finally: + shutil.rmtree(tmpdir) + + @unittest.skipIf(sys.executable is None, 'sys.executable required') + def test_load_grammar_from_subprocess(self): + tmpdir = tempfile.mkdtemp() + tmpsubdir = os.path.join(tmpdir, 'subdir') + try: + os.mkdir(tmpsubdir) + grammar_base = os.path.basename(support.grammar_path) + grammar_copy = os.path.join(tmpdir, grammar_base) + grammar_sub_copy = os.path.join(tmpsubdir, grammar_base) + shutil.copy(support.grammar_path, grammar_copy) + shutil.copy(support.grammar_path, grammar_sub_copy) + pickle_name = pgen2_driver._generate_pickle_name(grammar_copy) + pickle_sub_name = pgen2_driver._generate_pickle_name( + grammar_sub_copy) + self.assertNotEqual(pickle_name, pickle_sub_name) + + # Generate a pickle file from this process. + pgen2_driver.load_grammar(grammar_copy, save=True, force=True) + self.assertTrue(os.path.exists(pickle_name)) + + # Generate a new pickle file in a subprocess with a most likely + # different hash randomization seed. + sub_env = dict(os.environ) + sub_env['PYTHONHASHSEED'] = 'random' + subprocess.check_call( + [sys.executable, '-c', """ +from lib2to3.pgen2 import driver as pgen2_driver +pgen2_driver.load_grammar(%r, save=True, force=True) + """ % (grammar_sub_copy,)], + env=sub_env) + self.assertTrue(os.path.exists(pickle_sub_name)) + + with open(pickle_name, 'rb') as pickle_f_1, \ + open(pickle_sub_name, 'rb') as pickle_f_2: + self.assertEqual( + pickle_f_1.read(), pickle_f_2.read(), + msg='Grammar caches generated using different hash seeds' + ' were not identical.') + finally: + shutil.rmtree(tmpdir) + + + class GrammarTest(support.TestCase): def validate(self, code): support.parse_string(code) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -67,6 +67,10 @@ Library ------- +- lib2to3.pgen3.driver.load_grammar() now creates a stable cache file + between runs given the same Grammar.txt input regardless of the hash + randomization setting. + - Issue #27570: Avoid zero-length memcpy() etc calls with null source pointers in the "ctypes" and "array" modules. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:48:31 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 00:48:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_lib2to3=2Epgen3=2Edriver=2Eload=5Fgrammar=28=29_now_crea?= =?utf-8?q?tes_a_stable_cache_file?= Message-ID: <20160908004830.19067.53128.4AA597A0@psf.io> https://hg.python.org/cpython/rev/e4faff120be2 changeset: 103272:e4faff120be2 parent: 103269:2b2e7b39bb68 parent: 103271:186bb8dc5540 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 00:46:26 2016 +0000 summary: lib2to3.pgen3.driver.load_grammar() now creates a stable cache file between runs given the same Grammar.txt input regardless of the hash randomization setting. files: Lib/lib2to3/pgen2/driver.py | 15 ++- Lib/lib2to3/pgen2/grammar.py | 28 +++++++- Lib/lib2to3/pgen2/pgen.py | 8 +- Lib/lib2to3/tests/support.py | 6 +- Lib/lib2to3/tests/test_parser.py | 71 +++++++++++++++++++- Misc/NEWS | 4 + 6 files changed, 116 insertions(+), 16 deletions(-) diff --git a/Lib/lib2to3/pgen2/driver.py b/Lib/lib2to3/pgen2/driver.py --- a/Lib/lib2to3/pgen2/driver.py +++ b/Lib/lib2to3/pgen2/driver.py @@ -106,16 +106,19 @@ return self.parse_tokens(tokens, debug) +def _generate_pickle_name(gt): + head, tail = os.path.splitext(gt) + if tail == ".txt": + tail = "" + return head + tail + ".".join(map(str, sys.version_info)) + ".pickle" + + def load_grammar(gt="Grammar.txt", gp=None, save=True, force=False, logger=None): """Load the grammar (maybe from a pickle).""" if logger is None: logger = logging.getLogger() - if gp is None: - head, tail = os.path.splitext(gt) - if tail == ".txt": - tail = "" - gp = head + tail + ".".join(map(str, sys.version_info)) + ".pickle" + gp = _generate_pickle_name(gt) if gp is None else gp if force or not _newer(gp, gt): logger.info("Generating grammar tables from %s", gt) g = pgen.generate_grammar(gt) @@ -124,7 +127,7 @@ try: g.dump(gp) except OSError as e: - logger.info("Writing failed:"+str(e)) + logger.info("Writing failed: %s", e) else: g = grammar.Grammar() g.load(gp) diff --git a/Lib/lib2to3/pgen2/grammar.py b/Lib/lib2to3/pgen2/grammar.py --- a/Lib/lib2to3/pgen2/grammar.py +++ b/Lib/lib2to3/pgen2/grammar.py @@ -13,6 +13,7 @@ """ # Python imports +import collections import pickle # Local imports @@ -85,9 +86,21 @@ self.start = 256 def dump(self, filename): - """Dump the grammar tables to a pickle file.""" + """Dump the grammar tables to a pickle file. + + dump() recursively changes all dict to OrderedDict, so the pickled file + is not exactly the same as what was passed in to dump(). load() uses the + pickled file to create the tables, but only changes OrderedDict to dict + at the top level; it does not recursively change OrderedDict to dict. + So, the loaded tables are different from the original tables that were + passed to load() in that some of the OrderedDict (from the pickled file) + are not changed back to dict. For parsing, this has no effect on + performance because OrderedDict uses dict's __getitem__ with nothing in + between. + """ with open(filename, "wb") as f: - pickle.dump(self.__dict__, f, 2) + d = _make_deterministic(self.__dict__) + pickle.dump(d, f, 2) def load(self, filename): """Load the grammar tables from a pickle file.""" @@ -124,6 +137,17 @@ print("start", self.start) +def _make_deterministic(top): + if isinstance(top, dict): + return collections.OrderedDict( + sorted(((k, _make_deterministic(v)) for k, v in top.items()))) + if isinstance(top, list): + return [_make_deterministic(e) for e in top] + if isinstance(top, tuple): + return tuple(_make_deterministic(e) for e in top) + return top + + # Map from operator to number (since tokenize doesn't do this) opmap_raw = """ diff --git a/Lib/lib2to3/pgen2/pgen.py b/Lib/lib2to3/pgen2/pgen.py --- a/Lib/lib2to3/pgen2/pgen.py +++ b/Lib/lib2to3/pgen2/pgen.py @@ -39,7 +39,7 @@ states = [] for state in dfa: arcs = [] - for label, next in state.arcs.items(): + for label, next in sorted(state.arcs.items()): arcs.append((self.make_label(c, label), dfa.index(next))) if state.isfinal: arcs.append((0, dfa.index(state))) @@ -52,7 +52,7 @@ def make_first(self, c, name): rawfirst = self.first[name] first = {} - for label in rawfirst: + for label in sorted(rawfirst): ilabel = self.make_label(c, label) ##assert ilabel not in first # XXX failed on <> ... != first[ilabel] = 1 @@ -192,7 +192,7 @@ for label, next in nfastate.arcs: if label is not None: addclosure(next, arcs.setdefault(label, {})) - for label, nfaset in arcs.items(): + for label, nfaset in sorted(arcs.items()): for st in states: if st.nfaset == nfaset: break @@ -222,7 +222,7 @@ print("Dump of DFA for", name) for i, state in enumerate(dfa): print(" State", i, state.isfinal and "(final)" or "") - for label, next in state.arcs.items(): + for label, next in sorted(state.arcs.items()): print(" %s -> %d" % (label, dfa.index(next))) def simplify_dfa(self, dfa): diff --git a/Lib/lib2to3/tests/support.py b/Lib/lib2to3/tests/support.py --- a/Lib/lib2to3/tests/support.py +++ b/Lib/lib2to3/tests/support.py @@ -9,13 +9,13 @@ # Local imports from lib2to3 import pytree, refactor -from lib2to3.pgen2 import driver +from lib2to3.pgen2 import driver as pgen2_driver test_dir = os.path.dirname(__file__) proj_dir = os.path.normpath(os.path.join(test_dir, "..")) grammar_path = os.path.join(test_dir, "..", "Grammar.txt") -grammar = driver.load_grammar(grammar_path) -driver = driver.Driver(grammar, convert=pytree.convert) +grammar = pgen2_driver.load_grammar(grammar_path) +driver = pgen2_driver.Driver(grammar, convert=pytree.convert) def parse_string(string): return driver.parse_string(reformat(string), debug=True) diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -15,11 +15,15 @@ # Python imports import os +import shutil +import subprocess +import sys +import tempfile import unittest import warnings -import subprocess # Local imports +from lib2to3.pgen2 import driver as pgen2_driver from lib2to3.pgen2 import tokenize from ..pgen2.parse import ParseError from lib2to3.pygram import python_symbols as syms @@ -34,6 +38,71 @@ self.assertEqual(t.children[1].children[0].type, syms.print_stmt) +class TestPgen2Caching(support.TestCase): + def test_load_grammar_from_txt_file(self): + pgen2_driver.load_grammar(support.grammar_path, save=False, force=True) + + def test_load_grammar_from_pickle(self): + # Make a copy of the grammar file in a temp directory we are + # guaranteed to be able to write to. + tmpdir = tempfile.mkdtemp() + try: + grammar_copy = os.path.join( + tmpdir, os.path.basename(support.grammar_path)) + shutil.copy(support.grammar_path, grammar_copy) + pickle_name = pgen2_driver._generate_pickle_name(grammar_copy) + + pgen2_driver.load_grammar(grammar_copy, save=True, force=True) + self.assertTrue(os.path.exists(pickle_name)) + + os.unlink(grammar_copy) # Only the pickle remains... + pgen2_driver.load_grammar(grammar_copy, save=False, force=False) + finally: + shutil.rmtree(tmpdir) + + @unittest.skipIf(sys.executable is None, 'sys.executable required') + def test_load_grammar_from_subprocess(self): + tmpdir = tempfile.mkdtemp() + tmpsubdir = os.path.join(tmpdir, 'subdir') + try: + os.mkdir(tmpsubdir) + grammar_base = os.path.basename(support.grammar_path) + grammar_copy = os.path.join(tmpdir, grammar_base) + grammar_sub_copy = os.path.join(tmpsubdir, grammar_base) + shutil.copy(support.grammar_path, grammar_copy) + shutil.copy(support.grammar_path, grammar_sub_copy) + pickle_name = pgen2_driver._generate_pickle_name(grammar_copy) + pickle_sub_name = pgen2_driver._generate_pickle_name( + grammar_sub_copy) + self.assertNotEqual(pickle_name, pickle_sub_name) + + # Generate a pickle file from this process. + pgen2_driver.load_grammar(grammar_copy, save=True, force=True) + self.assertTrue(os.path.exists(pickle_name)) + + # Generate a new pickle file in a subprocess with a most likely + # different hash randomization seed. + sub_env = dict(os.environ) + sub_env['PYTHONHASHSEED'] = 'random' + subprocess.check_call( + [sys.executable, '-c', """ +from lib2to3.pgen2 import driver as pgen2_driver +pgen2_driver.load_grammar(%r, save=True, force=True) + """ % (grammar_sub_copy,)], + env=sub_env) + self.assertTrue(os.path.exists(pickle_sub_name)) + + with open(pickle_name, 'rb') as pickle_f_1, \ + open(pickle_sub_name, 'rb') as pickle_f_2: + self.assertEqual( + pickle_f_1.read(), pickle_f_2.read(), + msg='Grammar caches generated using different hash seeds' + ' were not identical.') + finally: + shutil.rmtree(tmpdir) + + + class GrammarTest(support.TestCase): def validate(self, code): support.parse_string(code) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -99,6 +99,10 @@ Library ------- +- lib2to3.pgen3.driver.load_grammar() now creates a stable cache file + between runs given the same Grammar.txt input regardless of the hash + randomization setting. + - Issue #28005: Allow ImportErrors in encoding implementation to propagate. - Issue #27570: Avoid zero-length memcpy() etc calls with null source -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:48:31 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 00:48:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_indentatio?= =?utf-8?q?n_=28reindent=2Epy=29=2E?= Message-ID: <20160908004831.59475.10382.5A483BBB@psf.io> https://hg.python.org/cpython/rev/99ce1713ae15 changeset: 103274:99ce1713ae15 branch: 3.5 parent: 103271:186bb8dc5540 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 00:48:07 2016 +0000 summary: Fix indentation (reindent.py). files: Lib/lib2to3/pgen2/grammar.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Lib/lib2to3/pgen2/grammar.py b/Lib/lib2to3/pgen2/grammar.py --- a/Lib/lib2to3/pgen2/grammar.py +++ b/Lib/lib2to3/pgen2/grammar.py @@ -139,12 +139,12 @@ def _make_deterministic(top): if isinstance(top, dict): - return collections.OrderedDict( - sorted(((k, _make_deterministic(v)) for k, v in top.items()))) + return collections.OrderedDict( + sorted(((k, _make_deterministic(v)) for k, v in top.items()))) if isinstance(top, list): - return [_make_deterministic(e) for e in top] + return [_make_deterministic(e) for e in top] if isinstance(top, tuple): - return tuple(_make_deterministic(e) for e in top) + return tuple(_make_deterministic(e) for e in top) return top -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 20:51:47 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 00:51:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_expected_error_message?= =?utf-8?q?_in_PyTextIOWrapperTest?= Message-ID: <20160908005146.8512.6715.1B2DE01A@psf.io> https://hg.python.org/cpython/rev/318dad874200 changeset: 103276:318dad874200 user: Steve Dower date: Wed Sep 07 17:51:30 2016 -0700 summary: Fix expected error message in PyTextIOWrapperTest files: Lib/test/test_io.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -3276,7 +3276,7 @@ class PyTextIOWrapperTest(TextIOWrapperTest): io = pyio - shutdown_error = "LookupError: unknown encoding: ascii" + shutdown_error = "ImportError: sys.meta_path is None, Python is likely shutting down" class IncrementalNewlineDecoderTest(unittest.TestCase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:00:49 2016 From: python-checkins at python.org (davin.potts) Date: Thu, 08 Sep 2016 01:00:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Updated_Misc/NEWS?= Message-ID: <20160908010049.21363.37037.E91D85AB@psf.io> https://hg.python.org/cpython/rev/f1013d826fde changeset: 103277:f1013d826fde user: Davin Potts date: Wed Sep 07 20:00:33 2016 -0500 summary: Updated Misc/NEWS files: Misc/NEWS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -234,6 +234,9 @@ - Issue #27930: Improved behaviour of logging.handlers.QueueListener. Thanks to Paulo Andrade and Petr Viktorin for the analysis and patch. +- Issue #6766: Distributed reference counting added to multiprocessing + to support nesting of shared values / proxy objects. + C API ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:04:56 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 01:04:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogbGliMnRvMy5wZ2Vu?= =?utf-8?q?3=2Edriver=2Eload=5Fgrammar=28=29_now_creates_a_stable_cache_fi?= =?utf-8?q?le?= Message-ID: <20160908010453.21010.89004.5F41B3A1@psf.io> https://hg.python.org/cpython/rev/26397c1ea557 changeset: 103278:26397c1ea557 branch: 2.7 parent: 103236:7aaf8cff23e5 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 01:04:37 2016 +0000 summary: lib2to3.pgen3.driver.load_grammar() now creates a stable cache file between runs given the same Grammar.txt input regardless of the hash randomization setting. Backport of 186bb8dc5540 from 3.5. Done in 2.7 per the lib2to3 exemption. files: Lib/lib2to3/pgen2/driver.py | 17 ++- Lib/lib2to3/pgen2/grammar.py | 31 +++++++- Lib/lib2to3/pgen2/pgen.py | 8 +- Lib/lib2to3/tests/support.py | 6 +- Lib/lib2to3/tests/test_parser.py | 72 +++++++++++++++++++- Misc/NEWS | 4 + 6 files changed, 118 insertions(+), 20 deletions(-) diff --git a/Lib/lib2to3/pgen2/driver.py b/Lib/lib2to3/pgen2/driver.py --- a/Lib/lib2to3/pgen2/driver.py +++ b/Lib/lib2to3/pgen2/driver.py @@ -106,16 +106,19 @@ return self.parse_tokens(tokens, debug) +def _generate_pickle_name(gt): + head, tail = os.path.splitext(gt) + if tail == ".txt": + tail = "" + return head + tail + ".".join(map(str, sys.version_info)) + ".pickle" + + def load_grammar(gt="Grammar.txt", gp=None, save=True, force=False, logger=None): """Load the grammar (maybe from a pickle).""" if logger is None: logger = logging.getLogger() - if gp is None: - head, tail = os.path.splitext(gt) - if tail == ".txt": - tail = "" - gp = head + tail + ".".join(map(str, sys.version_info)) + ".pickle" + gp = _generate_pickle_name(gt) if gp is None else gp if force or not _newer(gp, gt): logger.info("Generating grammar tables from %s", gt) g = pgen.generate_grammar(gt) @@ -123,8 +126,8 @@ logger.info("Writing grammar tables to %s", gp) try: g.dump(gp) - except IOError, e: - logger.info("Writing failed:"+str(e)) + except IOError as e: + logger.info("Writing failed: %s", e) else: g = grammar.Grammar() g.load(gp) diff --git a/Lib/lib2to3/pgen2/grammar.py b/Lib/lib2to3/pgen2/grammar.py --- a/Lib/lib2to3/pgen2/grammar.py +++ b/Lib/lib2to3/pgen2/grammar.py @@ -13,6 +13,7 @@ """ # Python imports +import collections import pickle # Local imports @@ -85,10 +86,21 @@ self.start = 256 def dump(self, filename): - """Dump the grammar tables to a pickle file.""" - f = open(filename, "wb") - pickle.dump(self.__dict__, f, 2) - f.close() + """Dump the grammar tables to a pickle file. + + dump() recursively changes all dict to OrderedDict, so the pickled file + is not exactly the same as what was passed in to dump(). load() uses the + pickled file to create the tables, but only changes OrderedDict to dict + at the top level; it does not recursively change OrderedDict to dict. + So, the loaded tables are different from the original tables that were + passed to load() in that some of the OrderedDict (from the pickled file) + are not changed back to dict. For parsing, this has no effect on + performance because OrderedDict uses dict's __getitem__ with nothing in + between. + """ + with open(filename, "wb") as f: + d = _make_deterministic(self.__dict__) + pickle.dump(d, f, 2) def load(self, filename): """Load the grammar tables from a pickle file.""" @@ -126,6 +138,17 @@ print "start", self.start +def _make_deterministic(top): + if isinstance(top, dict): + return collections.OrderedDict( + sorted(((k, _make_deterministic(v)) for k, v in top.iteritems()))) + if isinstance(top, list): + return [_make_deterministic(e) for e in top] + if isinstance(top, tuple): + return tuple(_make_deterministic(e) for e in top) + return top + + # Map from operator to number (since tokenize doesn't do this) opmap_raw = """ diff --git a/Lib/lib2to3/pgen2/pgen.py b/Lib/lib2to3/pgen2/pgen.py --- a/Lib/lib2to3/pgen2/pgen.py +++ b/Lib/lib2to3/pgen2/pgen.py @@ -39,7 +39,7 @@ states = [] for state in dfa: arcs = [] - for label, next in state.arcs.iteritems(): + for label, next in sorted(state.arcs.iteritems()): arcs.append((self.make_label(c, label), dfa.index(next))) if state.isfinal: arcs.append((0, dfa.index(state))) @@ -52,7 +52,7 @@ def make_first(self, c, name): rawfirst = self.first[name] first = {} - for label in rawfirst: + for label in sorted(rawfirst): ilabel = self.make_label(c, label) ##assert ilabel not in first # XXX failed on <> ... != first[ilabel] = 1 @@ -192,7 +192,7 @@ for label, next in nfastate.arcs: if label is not None: addclosure(next, arcs.setdefault(label, {})) - for label, nfaset in arcs.iteritems(): + for label, nfaset in sorted(arcs.iteritems()): for st in states: if st.nfaset == nfaset: break @@ -222,7 +222,7 @@ print "Dump of DFA for", name for i, state in enumerate(dfa): print " State", i, state.isfinal and "(final)" or "" - for label, next in state.arcs.iteritems(): + for label, next in sorted(state.arcs.iteritems()): print " %s -> %d" % (label, dfa.index(next)) def simplify_dfa(self, dfa): diff --git a/Lib/lib2to3/tests/support.py b/Lib/lib2to3/tests/support.py --- a/Lib/lib2to3/tests/support.py +++ b/Lib/lib2to3/tests/support.py @@ -11,13 +11,13 @@ # Local imports from lib2to3 import pytree, refactor -from lib2to3.pgen2 import driver +from lib2to3.pgen2 import driver as pgen2_driver test_dir = os.path.dirname(__file__) proj_dir = os.path.normpath(os.path.join(test_dir, "..")) grammar_path = os.path.join(test_dir, "..", "Grammar.txt") -grammar = driver.load_grammar(grammar_path) -driver = driver.Driver(grammar, convert=pytree.convert) +grammar = pgen2_driver.load_grammar(grammar_path) +driver = pgen2_driver.Driver(grammar, convert=pytree.convert) def parse_string(string): return driver.parse_string(reformat(string), debug=True) diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -6,17 +6,20 @@ test_grammar.py files from both Python 2 and Python 3. """ -from __future__ import with_statement - # Testing imports from . import support from .support import driver, test_dir # Python imports import os +import shutil +import subprocess import sys +import tempfile +import unittest # Local imports +from lib2to3.pgen2 import driver as pgen2_driver from lib2to3.pgen2 import tokenize from ..pgen2.parse import ParseError from lib2to3.pygram import python_symbols as syms @@ -31,6 +34,71 @@ self.assertEqual(t.children[1].children[0].type, syms.print_stmt) +class TestPgen2Caching(support.TestCase): + def test_load_grammar_from_txt_file(self): + pgen2_driver.load_grammar(support.grammar_path, save=False, force=True) + + def test_load_grammar_from_pickle(self): + # Make a copy of the grammar file in a temp directory we are + # guaranteed to be able to write to. + tmpdir = tempfile.mkdtemp() + try: + grammar_copy = os.path.join( + tmpdir, os.path.basename(support.grammar_path)) + shutil.copy(support.grammar_path, grammar_copy) + pickle_name = pgen2_driver._generate_pickle_name(grammar_copy) + + pgen2_driver.load_grammar(grammar_copy, save=True, force=True) + self.assertTrue(os.path.exists(pickle_name)) + + os.unlink(grammar_copy) # Only the pickle remains... + pgen2_driver.load_grammar(grammar_copy, save=False, force=False) + finally: + shutil.rmtree(tmpdir) + + @unittest.skipIf(sys.executable is None, 'sys.executable required') + def test_load_grammar_from_subprocess(self): + tmpdir = tempfile.mkdtemp() + tmpsubdir = os.path.join(tmpdir, 'subdir') + try: + os.mkdir(tmpsubdir) + grammar_base = os.path.basename(support.grammar_path) + grammar_copy = os.path.join(tmpdir, grammar_base) + grammar_sub_copy = os.path.join(tmpsubdir, grammar_base) + shutil.copy(support.grammar_path, grammar_copy) + shutil.copy(support.grammar_path, grammar_sub_copy) + pickle_name = pgen2_driver._generate_pickle_name(grammar_copy) + pickle_sub_name = pgen2_driver._generate_pickle_name( + grammar_sub_copy) + self.assertNotEqual(pickle_name, pickle_sub_name) + + # Generate a pickle file from this process. + pgen2_driver.load_grammar(grammar_copy, save=True, force=True) + self.assertTrue(os.path.exists(pickle_name)) + + # Generate a new pickle file in a subprocess with a most likely + # different hash randomization seed. + sub_env = dict(os.environ) + sub_env['PYTHONHASHSEED'] = 'random' + subprocess.check_call( + [sys.executable, '-c', """ +from lib2to3.pgen2 import driver as pgen2_driver +pgen2_driver.load_grammar(%r, save=True, force=True) + """ % (grammar_sub_copy,)], + env=sub_env) + self.assertTrue(os.path.exists(pickle_sub_name)) + + with open(pickle_name, 'rb') as pickle_f_1, \ + open(pickle_sub_name, 'rb') as pickle_f_2: + self.assertEqual( + pickle_f_1.read(), pickle_f_2.read(), + msg='Grammar caches generated using different hash seeds' + ' were not identical.') + finally: + shutil.rmtree(tmpdir) + + + class GrammarTest(support.TestCase): def validate(self, code): support.parse_string(code) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -38,6 +38,10 @@ Library ------- +- lib2to3.pgen3.driver.load_grammar() now creates a stable cache file + between runs given the same Grammar.txt input regardless of the hash + randomization setting. + - Issue #27691: Fix ssl module's parsing of GEN_RID subject alternative name fields in X.509 certs. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:09:45 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 01:09:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_make_sure_expected_values_?= =?utf-8?q?are_interpreted_as_doubles?= Message-ID: <20160908010945.2686.56387.FB391165@psf.io> https://hg.python.org/cpython/rev/a6be890d4c88 changeset: 103279:a6be890d4c88 parent: 103277:f1013d826fde user: Benjamin Peterson date: Wed Sep 07 18:09:22 2016 -0700 summary: make sure expected values are interpreted as doubles files: Modules/_testcapimodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -2309,7 +2309,7 @@ result = PyOS_string_to_double(STR, NULL, NULL); \ if (result == -1.0 && PyErr_Occurred()) \ return NULL; \ - if (result != expected) { \ + if (result != (double)expected) { \ msg = "conversion of " STR " to float failed"; \ goto fail; \ } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:11:12 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 01:11:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_reST?= Message-ID: <20160908011112.21076.14814.6AD0052E@psf.io> https://hg.python.org/cpython/rev/cfc3a99b28a6 changeset: 103280:cfc3a99b28a6 user: Benjamin Peterson date: Wed Sep 07 18:10:50 2016 -0700 summary: fix reST files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -11,7 +11,7 @@ ----------------- - Issue #27911: Remove unnecessary error checks in - import.c:exec_builtin_or_dynamic(). + ``exec_builtin_or_dynamic()``. - Issue #27983: Cause lack of llvm-profdata tool when using clang as required for PGO linking to be a configure time error rather than -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:16:13 2016 From: python-checkins at python.org (r.david.murray) Date: Thu, 08 Sep 2016 01:16:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2324277=3A_The_new_email_?= =?utf-8?q?API_is_no_longer_provisional=2E?= Message-ID: <20160908011611.59554.92330.7528AA9C@psf.io> https://hg.python.org/cpython/rev/b97118691363 changeset: 103281:b97118691363 user: R David Murray date: Wed Sep 07 21:15:59 2016 -0400 summary: #24277: The new email API is no longer provisional. This is a wholesale reorganization and editing of the email documentation to make the new API the standard one, and the old API the 'legacy' one. The default is still the compat32 policy, for backward compatibility. We will change that eventually. files: Doc/includes/email-alternative-new-api.py | 56 - Doc/includes/email-alternative.py | 70 +- Doc/includes/email-dir.py | 55 +- Doc/includes/email-headers.py | 22 +- Doc/includes/email-mime.py | 29 +- Doc/includes/email-read-alternative.py | 0 Doc/includes/email-simple.py | 8 +- Doc/includes/email-unpack.py | 8 +- Doc/library/email.charset.rst | 5 + Doc/library/email.compat32-message.rst | 754 ++++++++++ Doc/library/email.contentmanager.rst | 298 +--- Doc/library/email.encoders.rst | 6 + Doc/library/email.errors.rst | 38 +- Doc/library/email.examples.rst | 33 +- Doc/library/email.generator.rst | 384 ++-- Doc/library/email.header.rst | 8 + Doc/library/email.headerregistry.rst | 43 +- Doc/library/email.message.rst | 694 ++++---- Doc/library/email.mime.rst | 5 + Doc/library/email.parser.rst | 380 ++-- Doc/library/email.policy.rst | 265 ++- Doc/library/email.rst | 451 +---- Doc/library/email.util.rst | 68 +- Lib/email/message.py | 20 + Lib/test/test_email/test_message.py | 20 + 25 files changed, 2064 insertions(+), 1656 deletions(-) diff --git a/Doc/includes/email-alternative-new-api.py b/Doc/includes/email-alternative-new-api.py deleted file mode 100644 --- a/Doc/includes/email-alternative-new-api.py +++ /dev/null @@ -1,56 +0,0 @@ -#!/usr/bin/env python3 - -import smtplib - -from email.message import EmailMessage -from email.headerregistry import Address -from email.utils import make_msgid - -# Create the base text message. -msg = EmailMessage() -msg['Subject'] = "Ayons asperges pour le d?jeuner" -msg['From'] = Address("Pep? Le Pew", "pepe", "example.com") -msg['To'] = (Address("Penelope Pussycat", "penelope", "example.com"), - Address("Fabrette Pussycat", "fabrette", "example.com")) -msg.set_content("""\ -Salut! - -Cela ressemble ? un excellent recipie[1] d?jeuner. - -[1] http://www.yummly.com/recipe/Roasted-Asparagus-Epicurious-203718 - ---Pep? -""") - -# Add the html version. This converts the message into a multipart/alternative -# container, with the original text message as the first part and the new html -# message as the second part. -asparagus_cid = make_msgid() -msg.add_alternative("""\ - - - -

Salut!<\p> -

Cela ressemble ? un excellent - - - -""".format(asparagus_cid=asparagus_cid[1:-1]), subtype='html') -# note that we needed to peel the <> off the msgid for use in the html. - -# Now add the related image to the html part. -with open("roasted-asparagus.jpg", 'rb') as img: - msg.get_payload()[1].add_related(img.read(), 'image', 'jpeg', - cid=asparagus_cid) - -# Make a local copy of what we are going to send. -with open('outgoing.msg', 'wb') as f: - f.write(bytes(msg)) - -# Send the message via local SMTP server. -with smtplib.SMTP('localhost') as s: - s.send_message(msg) diff --git a/Doc/includes/email-alternative.py b/Doc/includes/email-alternative.py old mode 100755 new mode 100644 --- a/Doc/includes/email-alternative.py +++ b/Doc/includes/email-alternative.py @@ -2,47 +2,55 @@ import smtplib -from email.mime.multipart import MIMEMultipart -from email.mime.text import MIMEText +from email.message import EmailMessage +from email.headerregistry import Address +from email.utils import make_msgid -# me == my email address -# you == recipient's email address -me = "my at email.com" -you = "your at email.com" +# Create the base text message. +msg = EmailMessage() +msg['Subject'] = "Ayons asperges pour le d?jeuner" +msg['From'] = Address("Pep? Le Pew", "pepe", "example.com") +msg['To'] = (Address("Penelope Pussycat", "penelope", "example.com"), + Address("Fabrette Pussycat", "fabrette", "example.com")) +msg.set_content("""\ +Salut! -# Create message container - the correct MIME type is multipart/alternative. -msg = MIMEMultipart('alternative') -msg['Subject'] = "Link" -msg['From'] = me -msg['To'] = you +Cela ressemble ? un excellent recipie[1] d?jeuner. -# Create the body of the message (a plain-text and an HTML version). -text = "Hi!\nHow are you?\nHere is the link you wanted:\nhttps://www.python.org" -html = """\ +[1] http://www.yummly.com/recipe/Roasted-Asparagus-Epicurious-203718 + +--Pep? +""") + +# Add the html version. This converts the message into a multipart/alternative +# container, with the original text message as the first part and the new html +# message as the second part. +asparagus_cid = make_msgid() +msg.add_alternative("""\ -

Hi!
- How are you?
- Here is the
link you wanted. +

Salut!<\p> +

Cela ressemble ? un excellent + -""" +""".format(asparagus_cid=asparagus_cid[1:-1]), subtype='html') +# note that we needed to peel the <> off the msgid for use in the html. -# Record the MIME types of both parts - text/plain and text/html. -part1 = MIMEText(text, 'plain') -part2 = MIMEText(html, 'html') +# Now add the related image to the html part. +with open("roasted-asparagus.jpg", 'rb') as img: + msg.get_payload()[1].add_related(img.read(), 'image', 'jpeg', + cid=asparagus_cid) -# Attach parts into message container. -# According to RFC 2046, the last part of a multipart message, in this case -# the HTML message, is best and preferred. -msg.attach(part1) -msg.attach(part2) +# Make a local copy of what we are going to send. +with open('outgoing.msg', 'wb') as f: + f.write(bytes(msg)) # Send the message via local SMTP server. -s = smtplib.SMTP('localhost') -# sendmail function takes 3 arguments: sender's address, recipient's address -# and message to send - here it is sent as one string. -s.sendmail(me, you, msg.as_string()) -s.quit() +with smtplib.SMTP('localhost') as s: + s.send_message(msg) diff --git a/Doc/includes/email-dir.py b/Doc/includes/email-dir.py --- a/Doc/includes/email-dir.py +++ b/Doc/includes/email-dir.py @@ -3,22 +3,14 @@ """Send the contents of a directory as a MIME message.""" import os -import sys import smtplib # For guessing MIME type based on file name extension import mimetypes from argparse import ArgumentParser -from email import encoders -from email.message import Message -from email.mime.audio import MIMEAudio -from email.mime.base import MIMEBase -from email.mime.image import MIMEImage -from email.mime.multipart import MIMEMultipart -from email.mime.text import MIMEText - -COMMASPACE = ', ' +from email.message import EmailMessage +from email.policy import SMTP def main(): @@ -47,12 +39,12 @@ directory = args.directory if not directory: directory = '.' - # Create the enclosing (outer) message - outer = MIMEMultipart() - outer['Subject'] = 'Contents of directory %s' % os.path.abspath(directory) - outer['To'] = COMMASPACE.join(args.recipients) - outer['From'] = args.sender - outer.preamble = 'You will not see this in a MIME-aware mail reader.\n' + # Create the message + msg = EmailMessage() + msg['Subject'] = 'Contents of directory %s' % os.path.abspath(directory) + msg['To'] = ', '.join(args.recipients) + msg['From'] = args.sender + msg.preamble = 'You will not see this in a MIME-aware mail reader.\n' for filename in os.listdir(directory): path = os.path.join(directory, filename) @@ -67,33 +59,18 @@ # use a generic bag-of-bits type. ctype = 'application/octet-stream' maintype, subtype = ctype.split('/', 1) - if maintype == 'text': - with open(path) as fp: - # Note: we should handle calculating the charset - msg = MIMEText(fp.read(), _subtype=subtype) - elif maintype == 'image': - with open(path, 'rb') as fp: - msg = MIMEImage(fp.read(), _subtype=subtype) - elif maintype == 'audio': - with open(path, 'rb') as fp: - msg = MIMEAudio(fp.read(), _subtype=subtype) - else: - with open(path, 'rb') as fp: - msg = MIMEBase(maintype, subtype) - msg.set_payload(fp.read()) - # Encode the payload using Base64 - encoders.encode_base64(msg) - # Set the filename parameter - msg.add_header('Content-Disposition', 'attachment', filename=filename) - outer.attach(msg) + with open(path, 'rb') as fp: + msg.add_attachment(fp.read(), + maintype=maintype, + subtype=subtype, + filename=filename) # Now send or store the message - composed = outer.as_string() if args.output: - with open(args.output, 'w') as fp: - fp.write(composed) + with open(args.output, 'wb') as fp: + fp.write(msg.as_bytes(policy=SMTP)) else: with smtplib.SMTP('localhost') as s: - s.sendmail(args.sender, args.recipients, composed) + s.send_message(msg) if __name__ == '__main__': diff --git a/Doc/includes/email-headers.py b/Doc/includes/email-headers.py --- a/Doc/includes/email-headers.py +++ b/Doc/includes/email-headers.py @@ -1,18 +1,24 @@ # Import the email modules we'll need -from email.parser import Parser +from email.parser import BytesParser, Parser +from email.policy import default # If the e-mail headers are in a file, uncomment these two lines: -# with open(messagefile) as fp: -# headers = Parser().parse(fp) +# with open(messagefile, 'rb') as fp: +# headers = BytesParser(policy=default).parse(fp) -# Or for parsing headers in a string, use: -headers = Parser().parsestr('From: \n' +# Or for parsing headers in a string (this is an uncommon operation), use: +headers = Parser(policy=default).parsestr( + 'From: Foo Bar \n' 'To: \n' 'Subject: Test message\n' '\n' 'Body would go here\n') # Now the header items can be accessed as a dictionary: -print('To: %s' % headers['to']) -print('From: %s' % headers['from']) -print('Subject: %s' % headers['subject']) +print('To: {}'.format(headers['to'])) +print('From: {}'.format(headers['from'])) +print('Subject: {}'.format(headers['subject'])) + +# You can also access the parts of the addresses: +print('Recipient username: {}'.format(headers['to'].addresses[0].username)) +print('Sender name: {}'.format(headers['from'].addresses[0].display_name)) diff --git a/Doc/includes/email-mime.py b/Doc/includes/email-mime.py --- a/Doc/includes/email-mime.py +++ b/Doc/includes/email-mime.py @@ -1,30 +1,29 @@ # Import smtplib for the actual sending function import smtplib +# And imghdr to find the types of our images +import imghdr + # Here are the email package modules we'll need -from email.mime.image import MIMEImage -from email.mime.multipart import MIMEMultipart +from email.message import EmailMessage -COMMASPACE = ', ' - -# Create the container (outer) email message. -msg = MIMEMultipart() +# Create the container email message. +msg = EmailMessage() msg['Subject'] = 'Our family reunion' # me == the sender's email address # family = the list of all recipients' email addresses msg['From'] = me -msg['To'] = COMMASPACE.join(family) +msg['To'] = ', '.join(family) msg.preamble = 'Our family reunion' -# Assume we know that the image files are all in PNG format +# Open the files in binary mode. Use imghdr to figure out the +# MIME subtype for each specific image. for file in pngfiles: - # Open the files in binary mode. Let the MIMEImage class automatically - # guess the specific image type. with open(file, 'rb') as fp: - img = MIMEImage(fp.read()) - msg.attach(img) + img_data = fp.read() + msg.add_attachment(img_data, maintype='image', + subtype=imghdr.what(None, img_data)) # Send the email via our own SMTP server. -s = smtplib.SMTP('localhost') -s.send_message(msg) -s.quit() +with smtplib.SMTP('localhost') as s: + s.send_message(msg) diff --git a/Doc/includes/email-read-alternative-new-api.py b/Doc/includes/email-read-alternative.py rename from Doc/includes/email-read-alternative-new-api.py rename to Doc/includes/email-read-alternative.py diff --git a/Doc/includes/email-simple.py b/Doc/includes/email-simple.py --- a/Doc/includes/email-simple.py +++ b/Doc/includes/email-simple.py @@ -2,13 +2,13 @@ import smtplib # Import the email modules we'll need -from email.mime.text import MIMEText +from email.message import EmailMessage -# Open a plain text file for reading. For this example, assume that -# the text file contains only ASCII characters. +# Open the plain text file whose name is in textfile for reading. with open(textfile) as fp: # Create a text/plain message - msg = MIMEText(fp.read()) + msg = EmailMessage() + msg.set_content(fp.read()) # me == the sender's email address # you == the recipient's email address diff --git a/Doc/includes/email-unpack.py b/Doc/includes/email-unpack.py --- a/Doc/includes/email-unpack.py +++ b/Doc/includes/email-unpack.py @@ -3,11 +3,11 @@ """Unpack a MIME message into a directory of files.""" import os -import sys import email -import errno import mimetypes +from email.policy import default + from argparse import ArgumentParser @@ -22,8 +22,8 @@ parser.add_argument('msgfile') args = parser.parse_args() - with open(args.msgfile) as fp: - msg = email.message_from_file(fp) + with open(args.msgfile, 'rb') as fp: + msg = email.message_from_binary_file(fp, policy=default) try: os.mkdir(args.directory) diff --git a/Doc/library/email.charset.rst b/Doc/library/email.charset.rst --- a/Doc/library/email.charset.rst +++ b/Doc/library/email.charset.rst @@ -8,6 +8,11 @@ -------------- +This module is part of the legacy (``Compat32``) email API. In the new +API only the aliases table is used. + +The remaining text in this section is the original documentation of the module. + This module provides a class :class:`Charset` for representing character sets and character set conversions in email messages, as well as a character set registry and several convenience methods for manipulating this registry. diff --git a/Doc/library/email.compat32-message.rst b/Doc/library/email.compat32-message.rst new file mode 100644 --- /dev/null +++ b/Doc/library/email.compat32-message.rst @@ -0,0 +1,754 @@ +.. _compat32_message: + +:mod:`email.message.Message`: Representing an email message using the :data:`~email.policy.compat32` API +-------------------------------------------------------------------------------------------------------- + +.. module:: email.message + :synopsis: The base class representing email messages in a fashion + backward compatible with python3.2 + + +The :class:`Message` class is very similar to the +:class:`~email.message.EmailMessage` class, without the methods added by that +class, and with the default behavior of certain other methods being slightly +different. We also document here some methods that, while supported by the +:class:`~email.message.EmailMessage` class, are not recommended unless you are +dealing with legacy code. + +The philosophy and structure of the two classes is otherwise the same. + +This document describes the behavior under the default (for :class:`Message`) +policy :attr:`~email.policy.Compat32`. If you are going to use another policy, +you should be using the :class:`~email.message.EmailMessage` class instead. + +An email message consists of *headers* and a *payload*. Headers must be +:rfc:`5233` style names and values, where the field name and value are +separated by a colon. The colon is not part of either the field name or the +field value. The payload may be a simple text message, or a binary object, or +a structured sequence of sub-messages each with their own set of headers and +their own payload. The latter type of payload is indicated by the message +having a MIME type such as :mimetype:`multipart/\*` or +:mimetype:`message/rfc822`. + +The conceptual model provided by a :class:`Message` object is that of an +ordered dictionary of headers with additional methods for accessing both +specialized information from the headers, for accessing the payload, for +generating a serialized version of the mssage, and for recursively walking over +the object tree. Note that duplicate headers are supported but special methods +must be used to access them. + +The :class:`Message` psuedo-dictionary is indexed by the header names, which +must be ASCII values. The values of the dictionary are strings that are +supposed to contain only ASCII characters; there is some special handling for +non-ASCII input, but it doesn't always produce the correct results. Headers +are stored and returned in case-preserving form, but field names are matched +case-insensitively. There may also be a single envelope header, also known as +the *Unix-From* header or the ``From_`` header. The *payload* is either a +string or bytes, in the case of simple message objects, or a list of +:class:`Message` objects, for MIME container documents (e.g. +:mimetype:`multipart/\*` and :mimetype:`message/rfc822`). + +Here are the methods of the :class:`Message` class: + + +.. class:: Message(policy=compat32) + + If *policy* is specified (it must be an instance of a :mod:`~email.policy` + class) use the rules it specifies to update and serialize the representation + of the message. If *policy* is not set, use the :class:`compat32 + ` policy, which maintains backward compatibility with + the Python 3.2 version of the email package. For more information see the + :mod:`~email.policy` documentation. + + .. versionchanged:: 3.3 The *policy* keyword argument was added. + + + .. method:: as_string(unixfrom=False, maxheaderlen=0, policy=None) + + Return the entire message flattened as a string. When optional *unixfrom* + is true, the envelope header is included in the returned string. + *unixfrom* defaults to ``False``. For backward compabitility reasons, + *maxheaderlen* defaults to ``0``, so if you want a different value you + must override it explicitly (the value specified for *max_line_length* in + the policy will be ignored by this method). The *policy* argument may be + used to override the default policy obtained from the message instance. + This can be used to control some of the formatting produced by the + method, since the specified *policy* will be passed to the ``Generator``. + + Flattening the message may trigger changes to the :class:`Message` if + defaults need to be filled in to complete the transformation to a string + (for example, MIME boundaries may be generated or modified). + + Note that this method is provided as a convenience and may not always + format the message the way you want. For example, by default it does + not do the mangling of lines that begin with ``From`` that is + required by the unix mbox format. For more flexibility, instantiate a + :class:`~email.generator.Generator` instance and use its + :meth:`~email.generator.Generator.flatten` method directly. For example:: + + from io import StringIO + from email.generator import Generator + fp = StringIO() + g = Generator(fp, mangle_from_=True, maxheaderlen=60) + g.flatten(msg) + text = fp.getvalue() + + If the message object contains binary data that is not encoded according + to RFC standards, the non-compliant data will be replaced by unicode + "unknown character" code points. (See also :meth:`.as_bytes` and + :class:`~email.generator.BytesGenerator`.) + + .. versionchanged:: 3.4 the *policy* keyword argument was added. + + + .. method:: __str__() + + Equivalent to :meth:`.as_string()`. Allows ``str(msg)`` to produce a + string containing the formatted message. + + + .. method:: as_bytes(unixfrom=False, policy=None) + + Return the entire message flattened as a bytes object. When optional + *unixfrom* is true, the envelope header is included in the returned + string. *unixfrom* defaults to ``False``. The *policy* argument may be + used to override the default policy obtained from the message instance. + This can be used to control some of the formatting produced by the + method, since the specified *policy* will be passed to the + ``BytesGenerator``. + + Flattening the message may trigger changes to the :class:`Message` if + defaults need to be filled in to complete the transformation to a string + (for example, MIME boundaries may be generated or modified). + + Note that this method is provided as a convenience and may not always + format the message the way you want. For example, by default it does + not do the mangling of lines that begin with ``From`` that is + required by the unix mbox format. For more flexibility, instantiate a + :class:`~email.generator.BytesGenerator` instance and use its + :meth:`~email.generator.BytesGenerator.flatten` method directly. + For example:: + + from io import BytesIO + from email.generator import BytesGenerator + fp = BytesIO() + g = BytesGenerator(fp, mangle_from_=True, maxheaderlen=60) + g.flatten(msg) + text = fp.getvalue() + + .. versionadded:: 3.4 + + + .. method:: __bytes__() + + Equivalent to :meth:`.as_bytes()`. Allows ``bytes(msg)`` to produce a + bytes object containing the formatted message. + + .. versionadded:: 3.4 + + + .. method:: is_multipart() + + Return ``True`` if the message's payload is a list of sub-\ + :class:`Message` objects, otherwise return ``False``. When + :meth:`is_multipart` returns ``False``, the payload should be a string + object (which might be a CTE encoded binary payload. (Note that + :meth:`is_multipart` returning ``True`` does not necessarily mean that + "msg.get_content_maintype() == 'multipart'" will return the ``True``. + For example, ``is_multipart`` will return ``True`` when the + :class:`Message` is of type ``message/rfc822``.) + + + .. method:: set_unixfrom(unixfrom) + + Set the message's envelope header to *unixfrom*, which should be a string. + + + .. method:: get_unixfrom() + + Return the message's envelope header. Defaults to ``None`` if the + envelope header was never set. + + + .. method:: attach(payload) + + Add the given *payload* to the current payload, which must be ``None`` or + a list of :class:`Message` objects before the call. After the call, the + payload will always be a list of :class:`Message` objects. If you want to + set the payload to a scalar object (e.g. a string), use + :meth:`set_payload` instead. + + This is a legacy method. On the + :class:`~email.emailmessage.EmailMessage` class its functionality is + replaced by :meth:`~email.message.EmailMessage.set_content` and the + realted ``make`` and ``add`` methods. + + + .. method:: get_payload(i=None, decode=False) + + Return the current payload, which will be a list of + :class:`Message` objects when :meth:`is_multipart` is ``True``, or a + string when :meth:`is_multipart` is ``False``. If the payload is a list + and you mutate the list object, you modify the message's payload in place. + + With optional argument *i*, :meth:`get_payload` will return the *i*-th + element of the payload, counting from zero, if :meth:`is_multipart` is + ``True``. An :exc:`IndexError` will be raised if *i* is less than 0 or + greater than or equal to the number of items in the payload. If the + payload is a string (i.e. :meth:`is_multipart` is ``False``) and *i* is + given, a :exc:`TypeError` is raised. + + Optional *decode* is a flag indicating whether the payload should be + decoded or not, according to the :mailheader:`Content-Transfer-Encoding` + header. When ``True`` and the message is not a multipart, the payload will + be decoded if this header's value is ``quoted-printable`` or ``base64``. + If some other encoding is used, or :mailheader:`Content-Transfer-Encoding` + header is missing, the payload is + returned as-is (undecoded). In all cases the returned value is binary + data. If the message is a multipart and the *decode* flag is ``True``, + then ``None`` is returned. If the payload is base64 and it was not + perfectly formed (missing padding, characters outside the base64 + alphabet), then an appropriate defect will be added to the message's + defect property (:class:`~email.errors.InvalidBase64PaddingDefect` or + :class:`~email.errors.InvalidBase64CharactersDefect`, respectively). + + When *decode* is ``False`` (the default) the body is returned as a string + without decoding the :mailheader:`Content-Transfer-Encoding`. However, + for a :mailheader:`Content-Transfer-Encoding` of 8bit, an attempt is made + to decode the original bytes using the ``charset`` specified by the + :mailheader:`Content-Type` header, using the ``replace`` error handler. + If no ``charset`` is specified, or if the ``charset`` given is not + recognized by the email package, the body is decoded using the default + ASCII charset. + + This is a legacy method. On the + :class:`~email.emailmessage.EmailMessage` class its functionality is + replaced by :meth:`~email.message.EmailMessage.get_content` and + :meth:`~email.message.EmailMessage.iter_parts`. + + + .. method:: set_payload(payload, charset=None) + + Set the entire message object's payload to *payload*. It is the client's + responsibility to ensure the payload invariants. Optional *charset* sets + the message's default character set; see :meth:`set_charset` for details. + + This is a legacy method. On the + :class:`~email.emailmessage.EmailMessage` class its functionality is + replaced by :meth:`~email.message.EmailMessage.set_content`. + + + .. method:: set_charset(charset) + + Set the character set of the payload to *charset*, which can either be a + :class:`~email.charset.Charset` instance (see :mod:`email.charset`), a + string naming a character set, or ``None``. If it is a string, it will + be converted to a :class:`~email.charset.Charset` instance. If *charset* + is ``None``, the ``charset`` parameter will be removed from the + :mailheader:`Content-Type` header (the message will not be otherwise + modified). Anything else will generate a :exc:`TypeError`. + + If there is no existing :mailheader:`MIME-Version` header one will be + added. If there is no existing :mailheader:`Content-Type` header, one + will be added with a value of :mimetype:`text/plain`. Whether the + :mailheader:`Content-Type` header already exists or not, its ``charset`` + parameter will be set to *charset.output_charset*. If + *charset.input_charset* and *charset.output_charset* differ, the payload + will be re-encoded to the *output_charset*. If there is no existing + :mailheader:`Content-Transfer-Encoding` header, then the payload will be + transfer-encoded, if needed, using the specified + :class:`~email.charset.Charset`, and a header with the appropriate value + will be added. If a :mailheader:`Content-Transfer-Encoding` header + already exists, the payload is assumed to already be correctly encoded + using that :mailheader:`Content-Transfer-Encoding` and is not modified. + + This is a legacy method. On the + :class:`~email.emailmessage.EmailMessage` class its functionality is + replaced by the *charset* parameter of the + :meth:`email.emailmessage.EmailMessage.set_content` method. + + + .. method:: get_charset() + + Return the :class:`~email.charset.Charset` instance associated with the + message's payload. + + This is a legacy method. On the + :class:`~email.emailmessage.EmailMessage` class it always returns + ``None``. + + + The following methods implement a mapping-like interface for accessing the + message's :rfc:`2822` headers. Note that there are some semantic differences + between these methods and a normal mapping (i.e. dictionary) interface. For + example, in a dictionary there are no duplicate keys, but here there may be + duplicate message headers. Also, in dictionaries there is no guaranteed + order to the keys returned by :meth:`keys`, but in a :class:`Message` object, + headers are always returned in the order they appeared in the original + message, or were added to the message later. Any header deleted and then + re-added are always appended to the end of the header list. + + These semantic differences are intentional and are biased toward maximal + convenience. + + Note that in all cases, any envelope header present in the message is not + included in the mapping interface. + + In a model generated from bytes, any header values that (in contravention of + the RFCs) contain non-ASCII bytes will, when retrieved through this + interface, be represented as :class:`~email.header.Header` objects with + a charset of `unknown-8bit`. + + + .. method:: __len__() + + Return the total number of headers, including duplicates. + + + .. method:: __contains__(name) + + Return true if the message object has a field named *name*. Matching is + done case-insensitively and *name* should not include the trailing colon. + Used for the ``in`` operator, e.g.:: + + if 'message-id' in myMessage: + print('Message-ID:', myMessage['message-id']) + + + .. method:: __getitem__(name) + + Return the value of the named header field. *name* should not include the + colon field separator. If the header is missing, ``None`` is returned; a + :exc:`KeyError` is never raised. + + Note that if the named field appears more than once in the message's + headers, exactly which of those field values will be returned is + undefined. Use the :meth:`get_all` method to get the values of all the + extant named headers. + + + .. method:: __setitem__(name, val) + + Add a header to the message with field name *name* and value *val*. The + field is appended to the end of the message's existing fields. + + Note that this does *not* overwrite or delete any existing header with the same + name. If you want to ensure that the new header is the only one present in the + message with field name *name*, delete the field first, e.g.:: + + del msg['subject'] + msg['subject'] = 'Python roolz!' + + + .. method:: __delitem__(name) + + Delete all occurrences of the field with name *name* from the message's + headers. No exception is raised if the named field isn't present in the + headers. + + + .. method:: keys() + + Return a list of all the message's header field names. + + + .. method:: values() + + Return a list of all the message's field values. + + + .. method:: items() + + Return a list of 2-tuples containing all the message's field headers and + values. + + + .. method:: get(name, failobj=None) + + Return the value of the named header field. This is identical to + :meth:`__getitem__` except that optional *failobj* is returned if the + named header is missing (defaults to ``None``). + + Here are some additional useful methods: + + + .. method:: get_all(name, failobj=None) + + Return a list of all the values for the field named *name*. If there are + no such named headers in the message, *failobj* is returned (defaults to + ``None``). + + + .. method:: add_header(_name, _value, **_params) + + Extended header setting. This method is similar to :meth:`__setitem__` + except that additional header parameters can be provided as keyword + arguments. *_name* is the header field to add and *_value* is the + *primary* value for the header. + + For each item in the keyword argument dictionary *_params*, the key is + taken as the parameter name, with underscores converted to dashes (since + dashes are illegal in Python identifiers). Normally, the parameter will + be added as ``key="value"`` unless the value is ``None``, in which case + only the key will be added. If the value contains non-ASCII characters, + it can be specified as a three tuple in the format + ``(CHARSET, LANGUAGE, VALUE)``, where ``CHARSET`` is a string naming the + charset to be used to encode the value, ``LANGUAGE`` can usually be set + to ``None`` or the empty string (see :rfc:`2231` for other possibilities), + and ``VALUE`` is the string value containing non-ASCII code points. If + a three tuple is not passed and the value contains non-ASCII characters, + it is automatically encoded in :rfc:`2231` format using a ``CHARSET`` + of ``utf-8`` and a ``LANGUAGE`` of ``None``. + + Here's an example:: + + msg.add_header('Content-Disposition', 'attachment', filename='bud.gif') + + This will add a header that looks like :: + + Content-Disposition: attachment; filename="bud.gif" + + An example with non-ASCII characters:: + + msg.add_header('Content-Disposition', 'attachment', + filename=('iso-8859-1', '', 'Fu?baller.ppt')) + + Which produces :: + + Content-Disposition: attachment; filename*="iso-8859-1''Fu%DFballer.ppt" + + + .. method:: replace_header(_name, _value) + + Replace a header. Replace the first header found in the message that + matches *_name*, retaining header order and field name case. If no + matching header was found, a :exc:`KeyError` is raised. + + + .. method:: get_content_type() + + Return the message's content type. The returned string is coerced to + lower case of the form :mimetype:`maintype/subtype`. If there was no + :mailheader:`Content-Type` header in the message the default type as given + by :meth:`get_default_type` will be returned. Since according to + :rfc:`2045`, messages always have a default type, :meth:`get_content_type` + will always return a value. + + :rfc:`2045` defines a message's default type to be :mimetype:`text/plain` + unless it appears inside a :mimetype:`multipart/digest` container, in + which case it would be :mimetype:`message/rfc822`. If the + :mailheader:`Content-Type` header has an invalid type specification, + :rfc:`2045` mandates that the default type be :mimetype:`text/plain`. + + + .. method:: get_content_maintype() + + Return the message's main content type. This is the :mimetype:`maintype` + part of the string returned by :meth:`get_content_type`. + + + .. method:: get_content_subtype() + + Return the message's sub-content type. This is the :mimetype:`subtype` + part of the string returned by :meth:`get_content_type`. + + + .. method:: get_default_type() + + Return the default content type. Most messages have a default content + type of :mimetype:`text/plain`, except for messages that are subparts of + :mimetype:`multipart/digest` containers. Such subparts have a default + content type of :mimetype:`message/rfc822`. + + + .. method:: set_default_type(ctype) + + Set the default content type. *ctype* should either be + :mimetype:`text/plain` or :mimetype:`message/rfc822`, although this is not + enforced. The default content type is not stored in the + :mailheader:`Content-Type` header. + + + .. method:: get_params(failobj=None, header='content-type', unquote=True) + + Return the message's :mailheader:`Content-Type` parameters, as a list. + The elements of the returned list are 2-tuples of key/value pairs, as + split on the ``'='`` sign. The left hand side of the ``'='`` is the key, + while the right hand side is the value. If there is no ``'='`` sign in + the parameter the value is the empty string, otherwise the value is as + described in :meth:`get_param` and is unquoted if optional *unquote* is + ``True`` (the default). + + Optional *failobj* is the object to return if there is no + :mailheader:`Content-Type` header. Optional *header* is the header to + search instead of :mailheader:`Content-Type`. + + This is a legacy method. On the + :class:`~email.emailmessage.EmailMessage` class its functionality is + replaced by the *params* property of the individual header objects + returned by the header access methods. + + + .. method:: get_param(param, failobj=None, header='content-type', unquote=True) + + Return the value of the :mailheader:`Content-Type` header's parameter + *param* as a string. If the message has no :mailheader:`Content-Type` + header or if there is no such parameter, then *failobj* is returned + (defaults to ``None``). + + Optional *header* if given, specifies the message header to use instead of + :mailheader:`Content-Type`. + + Parameter keys are always compared case insensitively. The return value + can either be a string, or a 3-tuple if the parameter was :rfc:`2231` + encoded. When it's a 3-tuple, the elements of the value are of the form + ``(CHARSET, LANGUAGE, VALUE)``. Note that both ``CHARSET`` and + ``LANGUAGE`` can be ``None``, in which case you should consider ``VALUE`` + to be encoded in the ``us-ascii`` charset. You can usually ignore + ``LANGUAGE``. + + If your application doesn't care whether the parameter was encoded as in + :rfc:`2231`, you can collapse the parameter value by calling + :func:`email.utils.collapse_rfc2231_value`, passing in the return value + from :meth:`get_param`. This will return a suitably decoded Unicode + string when the value is a tuple, or the original string unquoted if it + isn't. For example:: + + rawparam = msg.get_param('foo') + param = email.utils.collapse_rfc2231_value(rawparam) + + In any case, the parameter value (either the returned string, or the + ``VALUE`` item in the 3-tuple) is always unquoted, unless *unquote* is set + to ``False``. + + This is a legacy method. On the + :class:`~email.emailmessage.EmailMessage` class its functionality is + replaced by the *params* property of the individual header objects + returned by the header access methods. + + + .. method:: set_param(param, value, header='Content-Type', requote=True, \ + charset=None, language='', replace=False) + + Set a parameter in the :mailheader:`Content-Type` header. If the + parameter already exists in the header, its value will be replaced with + *value*. If the :mailheader:`Content-Type` header as not yet been defined + for this message, it will be set to :mimetype:`text/plain` and the new + parameter value will be appended as per :rfc:`2045`. + + Optional *header* specifies an alternative header to + :mailheader:`Content-Type`, and all parameters will be quoted as necessary + unless optional *requote* is ``False`` (the default is ``True``). + + If optional *charset* is specified, the parameter will be encoded + according to :rfc:`2231`. Optional *language* specifies the RFC 2231 + language, defaulting to the empty string. Both *charset* and *language* + should be strings. + + If *replace* is ``False`` (the default) the header is moved to the + end of the list of headers. If *replace* is ``True``, the header + will be updated in place. + + .. versionchanged:: 3.4 ``replace`` keyword was added. + + + .. method:: del_param(param, header='content-type', requote=True) + + Remove the given parameter completely from the :mailheader:`Content-Type` + header. The header will be re-written in place without the parameter or + its value. All values will be quoted as necessary unless *requote* is + ``False`` (the default is ``True``). Optional *header* specifies an + alternative to :mailheader:`Content-Type`. + + + .. method:: set_type(type, header='Content-Type', requote=True) + + Set the main type and subtype for the :mailheader:`Content-Type` + header. *type* must be a string in the form :mimetype:`maintype/subtype`, + otherwise a :exc:`ValueError` is raised. + + This method replaces the :mailheader:`Content-Type` header, keeping all + the parameters in place. If *requote* is ``False``, this leaves the + existing header's quoting as is, otherwise the parameters will be quoted + (the default). + + An alternative header can be specified in the *header* argument. When the + :mailheader:`Content-Type` header is set a :mailheader:`MIME-Version` + header is also added. + + This is a legacy method. On the + :class:`~email.emailmessage.EmailMessage` class its functionality is + replaced by the ``make_`` and ``add_`` methods. + + + .. method:: get_filename(failobj=None) + + Return the value of the ``filename`` parameter of the + :mailheader:`Content-Disposition` header of the message. If the header + does not have a ``filename`` parameter, this method falls back to looking + for the ``name`` parameter on the :mailheader:`Content-Type` header. If + neither is found, or the header is missing, then *failobj* is returned. + The returned string will always be unquoted as per + :func:`email.utils.unquote`. + + + .. method:: get_boundary(failobj=None) + + Return the value of the ``boundary`` parameter of the + :mailheader:`Content-Type` header of the message, or *failobj* if either + the header is missing, or has no ``boundary`` parameter. The returned + string will always be unquoted as per :func:`email.utils.unquote`. + + + .. method:: set_boundary(boundary) + + Set the ``boundary`` parameter of the :mailheader:`Content-Type` header to + *boundary*. :meth:`set_boundary` will always quote *boundary* if + necessary. A :exc:`~email.errors.HeaderParseError` is raised if the + message object has no :mailheader:`Content-Type` header. + + Note that using this method is subtly different than deleting the old + :mailheader:`Content-Type` header and adding a new one with the new + boundary via :meth:`add_header`, because :meth:`set_boundary` preserves + the order of the :mailheader:`Content-Type` header in the list of + headers. However, it does *not* preserve any continuation lines which may + have been present in the original :mailheader:`Content-Type` header. + + + .. method:: get_content_charset(failobj=None) + + Return the ``charset`` parameter of the :mailheader:`Content-Type` header, + coerced to lower case. If there is no :mailheader:`Content-Type` header, or if + that header has no ``charset`` parameter, *failobj* is returned. + + Note that this method differs from :meth:`get_charset` which returns the + :class:`~email.charset.Charset` instance for the default encoding of the message body. + + + .. method:: get_charsets(failobj=None) + + Return a list containing the character set names in the message. If the + message is a :mimetype:`multipart`, then the list will contain one element + for each subpart in the payload, otherwise, it will be a list of length 1. + + Each item in the list will be a string which is the value of the + ``charset`` parameter in the :mailheader:`Content-Type` header for the + represented subpart. However, if the subpart has no + :mailheader:`Content-Type` header, no ``charset`` parameter, or is not of + the :mimetype:`text` main MIME type, then that item in the returned list + will be *failobj*. + + + .. method:: get_content_disposition() + + Return the lowercased value (without parameters) of the message's + :mailheader:`Content-Disposition` header if it has one, or ``None``. The + possible values for this method are *inline*, *attachment* or ``None`` + if the message follows :rfc:`2183`. + + .. versionadded:: 3.5 + + .. method:: walk() + + The :meth:`walk` method is an all-purpose generator which can be used to + iterate over all the parts and subparts of a message object tree, in + depth-first traversal order. You will typically use :meth:`walk` as the + iterator in a ``for`` loop; each iteration returns the next subpart. + + Here's an example that prints the MIME type of every part of a multipart + message structure: + + .. testsetup:: + + >>> from email import message_from_binary_file + >>> with open('Lib/test/test_email/data/msg_16.txt', 'rb') as f: + ... msg = message_from_binary_file(f) + >>> from email.iterators import _structure + + .. doctest:: + + >>> for part in msg.walk(): + ... print(part.get_content_type()) + multipart/report + text/plain + message/delivery-status + text/plain + text/plain + message/rfc822 + text/plain + + ``walk`` iterates over the subparts of any part where + :meth:`is_multipart` returns ``True``, even though + ``msg.get_content_maintype() == 'multipart'`` may return ``False``. We + can see this in our example by making use of the ``_structure`` debug + helper function: + + .. doctest:: + + >>> for part in msg.walk(): + ... print(part.get_content_maintype() == 'multipart'), + ... part.is_multipart()) + True True + False False + False True + False False + False False + False True + False False + >>> _structure(msg) + multipart/report + text/plain + message/delivery-status + text/plain + text/plain + message/rfc822 + text/plain + + Here the ``message`` parts are not ``multiparts``, but they do contain + subparts. ``is_multipart()`` returns ``True`` and ``walk`` descends + into the subparts. + + + :class:`Message` objects can also optionally contain two instance attributes, + which can be used when generating the plain text of a MIME message. + + + .. attribute:: preamble + + The format of a MIME document allows for some text between the blank line + following the headers, and the first multipart boundary string. Normally, + this text is never visible in a MIME-aware mail reader because it falls + outside the standard MIME armor. However, when viewing the raw text of + the message, or when viewing the message in a non-MIME aware reader, this + text can become visible. + + The *preamble* attribute contains this leading extra-armor text for MIME + documents. When the :class:`~email.parser.Parser` discovers some text + after the headers but before the first boundary string, it assigns this + text to the message's *preamble* attribute. When the + :class:`~email.generator.Generator` is writing out the plain text + representation of a MIME message, and it finds the + message has a *preamble* attribute, it will write this text in the area + between the headers and the first boundary. See :mod:`email.parser` and + :mod:`email.generator` for details. + + Note that if the message object has no preamble, the *preamble* attribute + will be ``None``. + + + .. attribute:: epilogue + + The *epilogue* attribute acts the same way as the *preamble* attribute, + except that it contains text that appears between the last boundary and + the end of the message. + + You do not need to set the epilogue to the empty string in order for the + :class:`~email.generator.Generator` to print a newline at the end of the + file. + + + .. attribute:: defects + + The *defects* attribute contains a list of all the problems found when + parsing this message. See :mod:`email.errors` for a detailed description + of the possible parsing defects. diff --git a/Doc/library/email.contentmanager.rst b/Doc/library/email.contentmanager.rst --- a/Doc/library/email.contentmanager.rst +++ b/Doc/library/email.contentmanager.rst @@ -7,251 +7,14 @@ .. moduleauthor:: R. David Murray .. sectionauthor:: R. David Murray -.. versionadded:: 3.4 - as a :term:`provisional module `. - **Source code:** :source:`Lib/email/contentmanager.py` -.. note:: +------------ - The contentmanager module has been included in the standard library on a - :term:`provisional basis `. Backwards incompatible - changes (up to and including removal of the module) may occur if deemed - necessary by the core developers. +.. versionadded:: 3.4 as a :term:`provisional module `. --------------- +.. versionchanged:: 3.6 provisional status removed. -The :mod:`~email.message` module provides a class that can represent an -arbitrary email message. That basic message model has a useful and flexible -API, but it provides only a lower-level API for interacting with the generic -parts of a message (the headers, generic header parameters, and the payload, -which may be a list of sub-parts). This module provides classes and tools -that provide an enhanced and extensible API for dealing with various specific -types of content, including the ability to retrieve the content of the message -as a specialized object type rather than as a simple bytes object. The module -automatically takes care of the RFC-specified MIME details (required headers -and parameters, etc.) for the certain common content types content properties, -and support for additional types can be added by an application using the -extension mechanisms. - -This module defines the eponymous "Content Manager" classes. The base -:class:`.ContentManager` class defines an API for registering content -management functions which extract data from ``Message`` objects or insert data -and headers into ``Message`` objects, thus providing a way of converting -between ``Message`` objects containing data and other representations of that -data (Python data types, specialized Python objects, external files, etc). The -module also defines one concrete content manager: :data:`raw_data_manager` -converts between MIME content types and ``str`` or ``bytes`` data. It also -provides a convenient API for managing the MIME parameters when inserting -content into ``Message``\ s. It also handles inserting and extracting -``Message`` objects when dealing with the ``message/rfc822`` content type. - -Another part of the enhanced interface is subclasses of -:class:`~email.message.Message` that provide new convenience API functions, -including convenience methods for calling the Content Managers derived from -this module. - -.. note:: - - Although :class:`.EmailMessage` and :class:`.MIMEPart` are currently - documented in this module because of the provisional nature of the code, the - implementation lives in the :mod:`email.message` module. - -.. currentmodule:: email.message - -.. class:: EmailMessage(policy=default) - - If *policy* is specified (it must be an instance of a :mod:`~email.policy` - class) use the rules it specifies to udpate and serialize the representation - of the message. If *policy* is not set, use the - :class:`~email.policy.default` policy, which follows the rules of the email - RFCs except for line endings (instead of the RFC mandated ``\r\n``, it uses - the Python standard ``\n`` line endings). For more information see the - :mod:`~email.policy` documentation. - - This class is a subclass of :class:`~email.message.Message`. It adds - the following methods: - - - .. method:: is_attachment - - Return ``True`` if there is a :mailheader:`Content-Disposition` header - and its (case insensitive) value is ``attachment``, ``False`` otherwise. - - .. versionchanged:: 3.4.2 - is_attachment is now a method instead of a property, for consistency - with :meth:`~email.message.Message.is_multipart`. - - - .. method:: get_body(preferencelist=('related', 'html', 'plain')) - - Return the MIME part that is the best candidate to be the "body" of the - message. - - *preferencelist* must be a sequence of strings from the set ``related``, - ``html``, and ``plain``, and indicates the order of preference for the - content type of the part returned. - - Start looking for candidate matches with the object on which the - ``get_body`` method is called. - - If ``related`` is not included in *preferencelist*, consider the root - part (or subpart of the root part) of any related encountered as a - candidate if the (sub-)part matches a preference. - - When encountering a ``multipart/related``, check the ``start`` parameter - and if a part with a matching :mailheader:`Content-ID` is found, consider - only it when looking for candidate matches. Otherwise consider only the - first (default root) part of the ``multipart/related``. - - If a part has a :mailheader:`Content-Disposition` header, only consider - the part a candidate match if the value of the header is ``inline``. - - If none of the candidates matches any of the preferences in - *preferneclist*, return ``None``. - - Notes: (1) For most applications the only *preferencelist* combinations - that really make sense are ``('plain',)``, ``('html', 'plain')``, and the - default, ``('related', 'html', 'plain')``. (2) Because matching starts - with the object on which ``get_body`` is called, calling ``get_body`` on - a ``multipart/related`` will return the object itself unless - *preferencelist* has a non-default value. (3) Messages (or message parts) - that do not specify a :mailheader:`Content-Type` or whose - :mailheader:`Content-Type` header is invalid will be treated as if they - are of type ``text/plain``, which may occasionally cause ``get_body`` to - return unexpected results. - - - .. method:: iter_attachments() - - Return an iterator over all of the parts of the message that are not - candidate "body" parts. That is, skip the first occurrence of each of - ``text/plain``, ``text/html``, ``multipart/related``, or - ``multipart/alternative`` (unless they are explicitly marked as - attachments via :mailheader:`Content-Disposition: attachment`), and - return all remaining parts. When applied directly to a - ``multipart/related``, return an iterator over the all the related parts - except the root part (ie: the part pointed to by the ``start`` parameter, - or the first part if there is no ``start`` parameter or the ``start`` - parameter doesn't match the :mailheader:`Content-ID` of any of the - parts). When applied directly to a ``multipart/alternative`` or a - non-``multipart``, return an empty iterator. - - - .. method:: iter_parts() - - Return an iterator over all of the immediate sub-parts of the message, - which will be empty for a non-``multipart``. (See also - :meth:`~email.message.walk`.) - - - .. method:: get_content(*args, content_manager=None, **kw) - - Call the ``get_content`` method of the *content_manager*, passing self - as the message object, and passing along any other arguments or keywords - as additional arguments. If *content_manager* is not specified, use - the ``content_manager`` specified by the current :mod:`~email.policy`. - - - .. method:: set_content(*args, content_manager=None, **kw) - - Call the ``set_content`` method of the *content_manager*, passing self - as the message object, and passing along any other arguments or keywords - as additional arguments. If *content_manager* is not specified, use - the ``content_manager`` specified by the current :mod:`~email.policy`. - - - .. method:: make_related(boundary=None) - - Convert a non-``multipart`` message into a ``multipart/related`` message, - moving any existing :mailheader:`Content-` headers and payload into a - (new) first part of the ``multipart``. If *boundary* is specified, use - it as the boundary string in the multipart, otherwise leave the boundary - to be automatically created when it is needed (for example, when the - message is serialized). - - - .. method:: make_alternative(boundary=None) - - Convert a non-``multipart`` or a ``multipart/related`` into a - ``multipart/alternative``, moving any existing :mailheader:`Content-` - headers and payload into a (new) first part of the ``multipart``. If - *boundary* is specified, use it as the boundary string in the multipart, - otherwise leave the boundary to be automatically created when it is - needed (for example, when the message is serialized). - - - .. method:: make_mixed(boundary=None) - - Convert a non-``multipart``, a ``multipart/related``, or a - ``multipart-alternative`` into a ``multipart/mixed``, moving any existing - :mailheader:`Content-` headers and payload into a (new) first part of the - ``multipart``. If *boundary* is specified, use it as the boundary string - in the multipart, otherwise leave the boundary to be automatically - created when it is needed (for example, when the message is serialized). - - - .. method:: add_related(*args, content_manager=None, **kw) - - If the message is a ``multipart/related``, create a new message - object, pass all of the arguments to its :meth:`set_content` method, - and :meth:`~email.message.Message.attach` it to the ``multipart``. If - the message is a non-``multipart``, call :meth:`make_related` and then - proceed as above. If the message is any other type of ``multipart``, - raise a :exc:`TypeError`. If *content_manager* is not specified, use - the ``content_manager`` specified by the current :mod:`~email.policy`. - If the added part has no :mailheader:`Content-Disposition` header, - add one with the value ``inline``. - - - .. method:: add_alternative(*args, content_manager=None, **kw) - - If the message is a ``multipart/alternative``, create a new message - object, pass all of the arguments to its :meth:`set_content` method, and - :meth:`~email.message.Message.attach` it to the ``multipart``. If the - message is a non-``multipart`` or ``multipart/related``, call - :meth:`make_alternative` and then proceed as above. If the message is - any other type of ``multipart``, raise a :exc:`TypeError`. If - *content_manager* is not specified, use the ``content_manager`` specified - by the current :mod:`~email.policy`. - - - .. method:: add_attachment(*args, content_manager=None, **kw) - - If the message is a ``multipart/mixed``, create a new message object, - pass all of the arguments to its :meth:`set_content` method, and - :meth:`~email.message.Message.attach` it to the ``multipart``. If the - message is a non-``multipart``, ``multipart/related``, or - ``multipart/alternative``, call :meth:`make_mixed` and then proceed as - above. If *content_manager* is not specified, use the ``content_manager`` - specified by the current :mod:`~email.policy`. If the added part - has no :mailheader:`Content-Disposition` header, add one with the value - ``attachment``. This method can be used both for explicit attachments - (:mailheader:`Content-Disposition: attachment` and ``inline`` attachments - (:mailheader:`Content-Disposition: inline`), by passing appropriate - options to the ``content_manager``. - - - .. method:: clear() - - Remove the payload and all of the headers. - - - .. method:: clear_content() - - Remove the payload and all of the :exc:`Content-` headers, leaving - all other headers intact and in their original order. - - -.. class:: MIMEPart(policy=default) - - This class represents a subpart of a MIME message. It is identical to - :class:`EmailMessage`, except that no :mailheader:`MIME-Version` headers are - added when :meth:`~EmailMessage.set_content` is called, since sub-parts do - not need their own :mailheader:`MIME-Version` headers. - - -.. currentmodule:: email.contentmanager .. class:: ContentManager() @@ -362,7 +125,7 @@ set_content(msg, <'bytes'>, maintype, subtype, cte="base64", \ disposition=None, filename=None, cid=None, \ params=None, headers=None) - set_content(msg, <'Message'>, cte=None, \ + set_content(msg, <'EmailMessage'>, cte=None, \ disposition=None, filename=None, cid=None, \ params=None, headers=None) set_content(msg, <'list'>, subtype='mixed', \ @@ -378,14 +141,14 @@ subtype to *subtype* if it is specified, or ``plain`` if it is not. * For ``bytes``, use the specified *maintype* and *subtype*, or raise a :exc:`TypeError` if they are not specified. - * For :class:`~email.message.Message` objects, set the maintype to - ``message``, and set the subtype to *subtype* if it is specified - or ``rfc822`` if it is not. If *subtype* is ``partial``, raise an - error (``bytes`` objects must be used to construct - ``message/partial`` parts). + * For :class:`~email.message.EmailMessage` objects, set the maintype + to ``message``, and set the subtype to *subtype* if it is + specified or ``rfc822`` if it is not. If *subtype* is + ``partial``, raise an error (``bytes`` objects must be used to + construct ``message/partial`` parts). * For *<'list'>*, which should be a list of - :class:`~email.message.Message` objects, set the ``maintype`` to - ``multipart``, and the ``subtype`` to *subtype* if it is + :class:`~email.message.EmailMessage` objects, set the ``maintype`` + to ``multipart``, and the ``subtype`` to *subtype* if it is specified, and ``mixed`` if it is not. If the message parts in the *<'list'>* have :mailheader:`MIME-Version` headers, remove them. @@ -397,32 +160,35 @@ If *cte* is set, encode the payload using the specified content transfer encoding, and set the :mailheader:`Content-Transfer-Endcoding` header to - that value. For ``str`` objects, if it is not set use heuristics to - determine the most compact encoding. Possible values for *cte* are - ``quoted-printable``, ``base64``, ``7bit``, ``8bit``, and ``binary``. - If the input cannot be encoded in the specified encoding (eg: ``7bit``), - raise a :exc:`ValueError`. For :class:`~email.message.Message`, per - :rfc:`2046`, raise an error if a *cte* of ``quoted-printable`` or - ``base64`` is requested for *subtype* ``rfc822``, and for any *cte* - other than ``7bit`` for *subtype* ``external-body``. For - ``message/rfc822``, use ``8bit`` if *cte* is not specified. For all - other values of *subtype*, use ``7bit``. + that value. Possible values for *cte* are ``quoted-printable``, + ``base64``, ``7bit``, ``8bit``, and ``binary``. If the input cannot be + encoded in the specified encoding (for example, specifying a *cte* of + ``7bit`` for an input that contains non-ASCII values), raise a + :exc:`ValueError`. + + * For ``str`` objects, if *cte* is not set use heuristics to + determine the most compact encoding. + * For :class:`~email.message.EmailMessage`, per :rfc:`2046`, raise + an error if a *cte* of ``quoted-printable`` or ``base64`` is + requested for *subtype* ``rfc822``, and for any *cte* other than + ``7bit`` for *subtype* ``external-body``. For + ``message/rfc822``, use ``8bit`` if *cte* is not specified. For + all other values of *subtype*, use ``7bit``. .. note:: A *cte* of ``binary`` does not actually work correctly yet. - The ``Message`` object as modified by ``set_content`` is correct, but - :class:`~email.generator.BytesGenerator` does not serialize it - correctly. + The ``EmailMessage`` object as modified by ``set_content`` is + correct, but :class:`~email.generator.BytesGenerator` does not + serialize it correctly. If *disposition* is set, use it as the value of the :mailheader:`Content-Disposition` header. If not specified, and *filename* is specified, add the header with the value ``attachment``. - If it is not specified and *filename* is also not specified, do not add - the header. The only valid values for *disposition* are ``attachment`` - and ``inline``. + If *disposition* is not specified and *filename* is also not specified, + do not add the header. The only valid values for *disposition* are + ``attachment`` and ``inline``. If *filename* is specified, use it as the value of the ``filename`` - parameter of the :mailheader:`Content-Disposition` header. There is no - default. + parameter of the :mailheader:`Content-Disposition` header. If *cid* is specified, add a :mailheader:`Content-ID` header with *cid* as its value. diff --git a/Doc/library/email.encoders.rst b/Doc/library/email.encoders.rst --- a/Doc/library/email.encoders.rst +++ b/Doc/library/email.encoders.rst @@ -8,6 +8,12 @@ -------------- +This module is part of the legacy (``Compat32``) email API. In the +new API the functionality is provided by the *cte* parameter of +the :meth:`~email.message.EmailMessage.set_content` method. + +The remaining text in this section is the original documentation of the module. + When creating :class:`~email.message.Message` objects from scratch, you often need to encode the payloads for transport through compliant mail servers. This is especially true for :mimetype:`image/\*` and :mimetype:`text/\*` type messages diff --git a/Doc/library/email.errors.rst b/Doc/library/email.errors.rst --- a/Doc/library/email.errors.rst +++ b/Doc/library/email.errors.rst @@ -20,33 +20,27 @@ .. exception:: MessageParseError() - This is the base class for exceptions raised by the :class:`~email.parser.Parser` - class. It is derived from :exc:`MessageError`. + This is the base class for exceptions raised by the + :class:`~email.parser.Parser` class. It is derived from + :exc:`MessageError`. This class is also used internally by the parser used + by :mod:`~email.headerregistry`. .. exception:: HeaderParseError() - Raised under some error conditions when parsing the :rfc:`2822` headers of a - message, this class is derived from :exc:`MessageParseError`. It can be raised - from the :meth:`Parser.parse ` or - :meth:`Parser.parsestr ` methods. - - Situations where it can be raised include finding an envelope header after the - first :rfc:`2822` header of the message, finding a continuation line before the - first :rfc:`2822` header is found, or finding a line in the headers which is - neither a header or a continuation line. + Raised under some error conditions when parsing the :rfc:`5322` headers of a + message, this class is derived from :exc:`MessageParseError`. The + :meth:`~email.message.EmailMessage.set_boundary` method will raise this + error if the content type is unknown when the method is called. + :class:`~email.header.Header` may raise this error for certain base64 + decoding errors, and when an attempt is made to create a header that appears + to contain an embedded header (that is, there is what is supposed to be a + continuation line that has no leading whitespace and looks like a header). .. exception:: BoundaryError() - Raised under some error conditions when parsing the :rfc:`2822` headers of a - message, this class is derived from :exc:`MessageParseError`. It can be raised - from the :meth:`Parser.parse ` or - :meth:`Parser.parsestr ` methods. - - Situations where it can be raised include not being able to find the starting or - terminating boundary in a :mimetype:`multipart/\*` message when strict parsing - is used. + Deprecated and no longer used. .. exception:: MultipartConversionError() @@ -64,14 +58,14 @@ :class:`~email.mime.nonmultipart.MIMENonMultipart` (e.g. :class:`~email.mime.image.MIMEImage`). -Here's the list of the defects that the :class:`~email.parser.FeedParser` + +Here is the list of the defects that the :class:`~email.parser.FeedParser` can find while parsing messages. Note that the defects are added to the message where the problem was found, so for example, if a message nested inside a :mimetype:`multipart/alternative` had a malformed header, that nested message object would have a defect, but the containing messages would not. -All defect classes are subclassed from :class:`email.errors.MessageDefect`, but -this class is *not* an exception! +All defect classes are subclassed from :class:`email.errors.MessageDefect`. * :class:`NoBoundaryInMultipartDefect` -- A message claimed to be a multipart, but had no :mimetype:`boundary` parameter. diff --git a/Doc/library/email-examples.rst b/Doc/library/email.examples.rst rename from Doc/library/email-examples.rst rename to Doc/library/email.examples.rst --- a/Doc/library/email-examples.rst +++ b/Doc/library/email.examples.rst @@ -6,13 +6,14 @@ Here are a few examples of how to use the :mod:`email` package to read, write, and send simple email messages, as well as more complex MIME messages. -First, let's see how to create and send a simple text message: +First, let's see how to create and send a simple text message (both the +text content and the addresses may contain unicode characters): .. literalinclude:: ../includes/email-simple.py -And parsing RFC822 headers can easily be done by the parse(filename) or -parsestr(message_as_string) methods of the Parser() class: +Parsing RFC822 headers can easily be done by the using the classes +from the :mod:`~email.parser` module: .. literalinclude:: ../includes/email-headers.py @@ -34,30 +35,19 @@ .. literalinclude:: ../includes/email-unpack.py + Here's an example of how to create an HTML message with an alternative plain -text version: [2]_ +text version. To make things a bit more interesting, we include a related +image in the html part, and we save a copy of what we are going to send to +disk, as well as sending it. .. literalinclude:: ../includes/email-alternative.py -.. _email-contentmanager-api-examples: +If we were sent the message from the last example, here is one way we could +process it: -Examples using the Provisional API -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Here is a reworking of the last example using the provisional API. To make -things a bit more interesting, we include a related image in the html part, and -we save a copy of what we are going to send to disk, as well as sending it. - -This example also shows how easy it is to include non-ASCII, and simplifies the -sending of the message using the :meth:`.send_message` method of the -:mod:`smtplib` module. - -.. literalinclude:: ../includes/email-alternative-new-api.py - -If we were instead sent the message from the last example, here is one -way we could process it: - -.. literalinclude:: ../includes/email-read-alternative-new-api.py +.. literalinclude:: ../includes/email-read-alternative.py Up to the prompt, the output from the above is: @@ -75,4 +65,3 @@ .. rubric:: Footnotes .. [1] Thanks to Matthew Dixon Cowles for the original inspiration and examples. -.. [2] Contributed by Martin Matejek. diff --git a/Doc/library/email.generator.rst b/Doc/library/email.generator.rst --- a/Doc/library/email.generator.rst +++ b/Doc/library/email.generator.rst @@ -8,210 +8,243 @@ -------------- -One of the most common tasks is to generate the flat text of the email message -represented by a message object structure. You will need to do this if you want -to send your message via the :mod:`smtplib` module or the :mod:`nntplib` module, -or print the message on the console. Taking a message object structure and -producing a flat text document is the job of the :class:`Generator` class. +One of the most common tasks is to generate the flat (serialized) version of +the email message represented by a message object structure. You will need to +do this if you want to send your message via :meth:`smtplib.SMTP.sendmail` or +the :mod:`nntplib` module, or print the message on the console. Taking a +message object structure and producing a serialized representation is the job +of the generator classes. -Again, as with the :mod:`email.parser` module, you aren't limited to the -functionality of the bundled generator; you could write one from scratch -yourself. However the bundled generator knows how to generate most email in a -standards-compliant way, should handle MIME and non-MIME email messages just -fine, and is designed so that the transformation from flat text, to a message -structure via the :class:`~email.parser.Parser` class, and back to flat text, -is idempotent (the input is identical to the output) [#]_. On the other hand, -using the Generator on a :class:`~email.message.Message` constructed by program -may result in changes to the :class:`~email.message.Message` object as defaults -are filled in. +As with the :mod:`email.parser` module, you aren't limited to the functionality +of the bundled generator; you could write one from scratch yourself. However +the bundled generator knows how to generate most email in a standards-compliant +way, should handle MIME and non-MIME email messages just fine, and is designed +so that the bytes-oriented parsing and generation operations are inverses, +assuming the same non-transforming :mod:`~email.policy` is used for both. That +is, parsing the serialized byte stream via the +:class:`~email.parser.BytesParser` class and then regenerating the serialized +byte stream using :class:`BytesGenerator` should produce output identical to +the input [#]_. (On the other hand, using the generator on an +:class:`~email.message.EmailMessage` constructed by program may result in +changes to the :class:`~email.message.EmailMessage` object as defaults are +filled in.) -:class:`bytes` output can be generated using the :class:`BytesGenerator` class. -If the message object structure contains non-ASCII bytes, this generator's -:meth:`~BytesGenerator.flatten` method will emit the original bytes. Parsing a -binary message and then flattening it with :class:`BytesGenerator` should be -idempotent for standards compliant messages. +The :class:`Generator` class can be used to flatten a message into a text (as +opposed to binary) serialized representation, but since Unicode cannot +represent binary data directly, the message is of necessity transformed into +something that contains only ASCII characters, using the standard email RFC +Content Transfer Encoding techniques for encoding email messages for transport +over channels that are not "8 bit clean". -Here are the public methods of the :class:`Generator` class, imported from the -:mod:`email.generator` module: +.. class:: BytesGenerator(outfp, mangle_from_=None, maxheaderlen=None, *, \ + policy=None) -.. class:: Generator(outfp, mangle_from_=True, maxheaderlen=78, *, policy=None) + Return a :class:`BytesGenerator` object that will write any message provided + to the :meth:`flatten` method, or any surrogateescape encoded text provided + to the :meth:`write` method, to the :term:`file-like object` *outfp*. + *outfp* must support a ``write`` method that accepts binary data. - The constructor for the :class:`Generator` class takes a :term:`file-like object` - called *outfp* for an argument. *outfp* must support the :meth:`write` method - and be usable as the output file for the :func:`print` function. + If optional *mangle_from_* is ``True``, put a ``>`` character in front of + any line in the body that starts with the exact string ``"From "``, that is + ``From`` followed by a space at the beginning of a line. *mangle_from_* + defaults to the value of the :attr:`~email.policy.Policy.mangle_from_` + setting of the *policy* (which is ``True`` for the + :data:`~email.policy.compat32` policy and ``False`` for all others). + *mangle_from_* is intended for use when messages are stored in unix mbox + format (see :mod:`mailbox` and `WHY THE CONTENT-LENGTH FORMAT IS BAD + `_). - Optional *mangle_from_* is a flag that, when ``True``, puts a ``>`` character in - front of any line in the body that starts exactly as ``From``, i.e. ``From`` - followed by a space at the beginning of the line. This is the only guaranteed - portable way to avoid having such lines be mistaken for a Unix mailbox format - envelope header separator (see `WHY THE CONTENT-LENGTH FORMAT IS BAD - `_ for details). *mangle_from_* - defaults to ``True``, but you might want to set this to ``False`` if you are not - writing Unix mailbox format files. + If *maxheaderlen* is not ``None``, refold any header lines that are longer + than *maxheaderlen*, or if ``0``, do not rewrap any headers. If + *manheaderlen* is ``None`` (the default), wrap headers and other message + lines according to the *policy* settings. - Optional *maxheaderlen* specifies the longest length for a non-continued header. - When a header line is longer than *maxheaderlen* (in characters, with tabs - expanded to 8 spaces), the header will be split as defined in the - :class:`~email.header.Header` class. Set to zero to disable header wrapping. - The default is 78, as recommended (but not required) by :rfc:`2822`. + If *policy* is specified, use that policy to control message generation. If + *policy* is ``None`` (the default), use the policy associated with the + :class:`~email.message.Message` or :class:`~email.message.EmailMessage` + object passed to ``flatten`` to control the message generation. See + :mod:`email.policy` for details on what *policy* controls. - The *policy* keyword specifies a :mod:`~email.policy` object that controls a - number of aspects of the generator's operation. If no *policy* is specified, - then the *policy* attached to the message object passed to :attr:`flatten` - is used. + .. versionadded:: 3.2 .. versionchanged:: 3.3 Added the *policy* keyword. - The other public :class:`Generator` methods are: + .. versionchanged:: 3.6 The default behavior of the *mangle_from_* + and *maxheaderlen* parameters is to follow the policy. .. method:: flatten(msg, unixfrom=False, linesep=None) - Print the textual representation of the message object structure rooted at - *msg* to the output file specified when the :class:`Generator` instance - was created. Subparts are visited depth-first and the resulting text will - be properly MIME encoded. + Print the textual representation of the message object structure rooted + at *msg* to the output file specified when the :class:`BytesGenerator` + instance was created. - Optional *unixfrom* is a flag that forces the printing of the envelope - header delimiter before the first :rfc:`2822` header of the root message - object. If the root object has no envelope header, a standard one is - crafted. By default, this is set to ``False`` to inhibit the printing of - the envelope delimiter. + If the :mod:`~email.policy` option :attr:`~email.policy.Policy.cte_type` + is ``8bit`` (the default), copy any headers in the original parsed + message that have not been modified to the output with any bytes with the + high bit set reproduced as in the original, and preserve the non-ASCII + :mailheader:`Content-Transfer-Encoding` of any body parts that have them. + If ``cte_type`` is ``7bit``, convert the bytes with the high bit set as + needed using an ASCII-compatible :mailheader:`Content-Transfer-Encoding`. + That is, transform parts with non-ASCII + :mailheader:`Cotnent-Transfer-Encoding` + (:mailheader:`Content-Transfer-Encoding: 8bit`) to an ASCII compatibile + :mailheader:`Content-Transfer-Encoding`, and encode RFC-invalid non-ASCII + bytes in headers using the MIME ``unknown-8bit`` character set, thus + rendering them RFC-compliant. + .. XXX: There should be an option that just does the RFC + compliance transformation on headers but leaves CTE 8bit parts alone. + + If *unixfrom* is ``True``, print the envelope header delimiter used by + the Unix mailbox format (see :mod:`mailbox`) before the first of the + :rfc:`5322` headers of the root message object. If the root object has + no envelope header, craft a standard one. The default is ``False``. Note that for subparts, no envelope header is ever printed. - Optional *linesep* specifies the line separator character used to - terminate lines in the output. If specified it overrides the value - specified by the *msg*\'s or ``Generator``\'s ``policy``. + If *linesep* is not ``None``, use it as the separator character between + all the lines of the flattened message. If *linesep* is ``None`` (the + default), use the value specified in the *policy*. - Because strings cannot represent non-ASCII bytes, if the policy that - applies when ``flatten`` is run has :attr:`~email.policy.Policy.cte_type` - set to ``8bit``, ``Generator`` will operate as if it were set to - ``7bit``. This means that messages parsed with a Bytes parser that have - a :mailheader:`Content-Transfer-Encoding` of ``8bit`` will be converted - to a use a ``7bit`` Content-Transfer-Encoding. Non-ASCII bytes in the - headers will be :rfc:`2047` encoded with a charset of ``unknown-8bit``. + .. XXX: flatten should take a *policy* keyword. + + + .. method:: clone(fp) + + Return an independent clone of this :class:`BytesGenerator` instance with + the exact same option settings, and *fp* as the new *outfp*. + + + .. method:: write(s) + + Encode *s* using the ``ASCII`` codec and the ``surrogateescape`` error + handler, and pass it to the *write* method of the *outfp* passed to the + :class:`BytesGenerator`'s constructor. + + +As a convenience, :class:`~email.message.EmailMessage` provides the methods +:meth:`~email.message.EmailMessage.as_bytes` and ``bytes(aMessage)`` (a.k.a. +:meth:`~email.message.EmailMessage.__bytes__`), which simplify the generation of +a serialized binary representation of a message object. For more detail, see +:mod:`email.message`. + + +Because strings cannot represent binary data, the :class:`Generator` class must +convert any binary data in any message it flattens to an ASCII compatible +format, by converting them to an ASCII compatible +:mailheader:`Content-Transfer_Encoding`. Using the terminology of the email +RFCs, you can think of this as :class:`Generator` serializing to an I/O stream +that is not "8 bit clean". In other words, most applications will want +to be using :class:`BytesGenerator`, and not :class:`Generator`. + +.. class:: Generator(outfp, mangle_from_=None, maxheaderlen=None, *, \ + policy=None) + + Return a :class:`Generator` object that will write any message provided + to the :meth:`flatten` method, or any text provided to the :meth:`write` + method, to the :term:`file-like object` *outfp*. *outfp* must support a + ``write`` method that accepts string data. + + If optional *mangle_from_* is ``True``, put a ``>`` character in front of + any line in the body that starts with the exact string ``"From "``, that is + ``From`` followed by a space at the beginning of a line. *mangle_from_* + defaults to the value of the :attr:`~email.policy.Policy.mangle_from_` + setting of the *policy* (which is ``True`` for the + :data:`~email.policy.compat32` policy and ``False`` for all others). + *mangle_from_* is intended for use when messages are stored in unix mbox + format (see :mod:`mailbox` and `WHY THE CONTENT-LENGTH FORMAT IS BAD + `_). + + If *maxheaderlen* is not ``None``, refold any header lines that are longer + than *maxheaderlen*, or if ``0``, do not rewrap any headers. If + *manheaderlen* is ``None`` (the default), wrap headers and other message + lines according to the *policy* settings. + + If *policy* is specified, use that policy to control message generation. If + *policy* is ``None`` (the default), use the policy associated with the + :class:`~email.message.Message` or :class:`~email.message.EmailMessage` + object passed to ``flatten`` to control the message generation. See + :mod:`email.policy` for details on what *policy* controls. + + .. versionchanged:: 3.3 Added the *policy* keyword. + + .. versionchanged:: 3.6 The default behavior of the *mangle_from_* + and *maxheaderlen* parameters is to follow the policy. + + + .. method:: flatten(msg, unixfrom=False, linesep=None) + + Print the textual representation of the message object structure rooted + at *msg* to the output file specified when the :class:`Generator` + instance was created. + + If the :mod:`~email.policy` option :attr:`~email.policy.Policy.cte_type` + is ``8bit``, generate the message as if the option were set to ``7bit``. + (This is required because strings cannot represent non-ASCII bytes.) + Convert any bytes with the high bit set as needed using an + ASCII-compatible :mailheader:`Content-Transfer-Encoding`. That is, + transform parts with non-ASCII :mailheader:`Cotnent-Transfer-Encoding` + (:mailheader:`Content-Transfer-Encoding: 8bit`) to an ASCII compatibile + :mailheader:`Content-Transfer-Encoding`, and encode RFC-invalid non-ASCII + bytes in headers using the MIME ``unknown-8bit`` character set, thus + rendering them RFC-compliant. + + If *unixfrom* is ``True``, print the envelope header delimiter used by + the Unix mailbox format (see :mod:`mailbox`) before the first of the + :rfc:`5322` headers of the root message object. If the root object has + no envelope header, craft a standard one. The default is ``False``. + Note that for subparts, no envelope header is ever printed. + + If *linesep* is not ``None``, use it as the separator character between + all the lines of the flattened message. If *linesep* is ``None`` (the + default), use the value specified in the *policy*. + + .. XXX: flatten should take a *policy* keyword. .. versionchanged:: 3.2 Added support for re-encoding ``8bit`` message bodies, and the *linesep* argument. + .. method:: clone(fp) Return an independent clone of this :class:`Generator` instance with the - exact same options. + exact same options, and *fp* as the new *outfp*. + .. method:: write(s) - Write the string *s* to the underlying file object, i.e. *outfp* passed to - :class:`Generator`'s constructor. This provides just enough file-like API - for :class:`Generator` instances to be used in the :func:`print` function. + Write *s* to the *write* method of the *outfp* passed to the + :class:`Generator`'s constructor. This provides just enough file-like + API for :class:`Generator` instances to be used in the :func:`print` + function. -As a convenience, see the :class:`~email.message.Message` methods -:meth:`~email.message.Message.as_string` and ``str(aMessage)``, a.k.a. -:meth:`~email.message.Message.__str__`, which simplify the generation of a -formatted string representation of a message object. For more detail, see + +As a convenience, :class:`~email.message.EmailMessage` provides the methods +:meth:`~email.message.EmailMessage.as_string` and ``str(aMessage)`` (a.k.a. +:meth:`~email.message.EmailMessage.__str__`), which simplify the generation of +a formatted string representation of a message object. For more detail, see :mod:`email.message`. -.. class:: BytesGenerator(outfp, mangle_from_=True, maxheaderlen=78, *, \ - policy=None) - The constructor for the :class:`BytesGenerator` class takes a binary - :term:`file-like object` called *outfp* for an argument. *outfp* must - support a :meth:`write` method that accepts binary data. +The :mod:`email.generator` module also provides a derived class, +:class:`DecodedGenerator`, which is like the :class:`Generator` base class, +except that non-\ :mimetype:`text` parts are not serialized, but are instead +represented in the output stream by a string derived from a template filled +in with information about the part. - Optional *mangle_from_* is a flag that, when ``True``, puts a ``>`` - character in front of any line in the body that starts exactly as ``From``, - i.e. ``From`` followed by a space at the beginning of the line. This is the - only guaranteed portable way to avoid having such lines be mistaken for a - Unix mailbox format envelope header separator (see `WHY THE CONTENT-LENGTH - FORMAT IS BAD `_ for details). - *mangle_from_* defaults to ``True``, but you might want to set this to - ``False`` if you are not writing Unix mailbox format files. +.. class:: DecodedGenerator(outfp, mangle_from_=None, maxheaderlen=78, fmt=None) - Optional *maxheaderlen* specifies the longest length for a non-continued - header. When a header line is longer than *maxheaderlen* (in characters, - with tabs expanded to 8 spaces), the header will be split as defined in the - :class:`~email.header.Header` class. Set to zero to disable header - wrapping. The default is 78, as recommended (but not required) by - :rfc:`2822`. + Act like :class:`Generator`, except that for any subpart of the message + passed to :meth:`Generator.flatten`, if the subpart is of main type + :mimetype:`text`, print the decoded payload of the subpart, and if the main + type is not :mimetype:`text`, instead of printing it fill in the string + *fmt* using information from the part and print the resulting + filled-in string. - - The *policy* keyword specifies a :mod:`~email.policy` object that controls a - number of aspects of the generator's operation. If no *policy* is specified, - then the *policy* attached to the message object passed to :attr:`flatten` - is used. - - .. versionchanged:: 3.3 Added the *policy* keyword. - - The other public :class:`BytesGenerator` methods are: - - - .. method:: flatten(msg, unixfrom=False, linesep=None) - - Print the textual representation of the message object structure rooted - at *msg* to the output file specified when the :class:`BytesGenerator` - instance was created. Subparts are visited depth-first and the resulting - text will be properly MIME encoded. If the :mod:`~email.policy` option - :attr:`~email.policy.Policy.cte_type` is ``8bit`` (the default), - then any bytes with the high bit set in the original parsed message that - have not been modified will be copied faithfully to the output. If - ``cte_type`` is ``7bit``, the bytes will be converted as needed - using an ASCII-compatible Content-Transfer-Encoding. In particular, - RFC-invalid non-ASCII bytes in headers will be encoded using the MIME - ``unknown-8bit`` character set, thus rendering them RFC-compliant. - - .. XXX: There should be a complementary option that just does the RFC - compliance transformation but leaves CTE 8bit parts alone. - - Messages parsed with a Bytes parser that have a - :mailheader:`Content-Transfer-Encoding` of 8bit will be reconstructed - as 8bit if they have not been modified. - - Optional *unixfrom* is a flag that forces the printing of the envelope - header delimiter before the first :rfc:`2822` header of the root message - object. If the root object has no envelope header, a standard one is - crafted. By default, this is set to ``False`` to inhibit the printing of - the envelope delimiter. - - Note that for subparts, no envelope header is ever printed. - - Optional *linesep* specifies the line separator character used to - terminate lines in the output. If specified it overrides the value - specified by the ``Generator``\ or *msg*\ 's ``policy``. - - .. method:: clone(fp) - - Return an independent clone of this :class:`BytesGenerator` instance with - the exact same options. - - .. method:: write(s) - - Write the string *s* to the underlying file object. *s* is encoded using - the ``ASCII`` codec and written to the *write* method of the *outfp* - *outfp* passed to the :class:`BytesGenerator`'s constructor. This - provides just enough file-like API for :class:`BytesGenerator` instances - to be used in the :func:`print` function. - - .. versionadded:: 3.2 - -The :mod:`email.generator` module also provides a derived class, called -:class:`DecodedGenerator` which is like the :class:`Generator` base class, -except that non-\ :mimetype:`text` parts are substituted with a format string -representing the part. - - -.. class:: DecodedGenerator(outfp, mangle_from_=True, maxheaderlen=78, fmt=None) - - This class, derived from :class:`Generator` walks through all the subparts of a - message. If the subpart is of main type :mimetype:`text`, then it prints the - decoded payload of the subpart. Optional *_mangle_from_* and *maxheaderlen* are - as with the :class:`Generator` base class. - - If the subpart is not of main type :mimetype:`text`, optional *fmt* is a format - string that is used instead of the message payload. *fmt* is expanded with the - following keywords, ``%(keyword)s`` format: + To fill in *fmt*, execute ``fmt % part_info``, where ``part_info`` + is a dictionary composed of the following keys and values: * ``type`` -- Full MIME type of the non-\ :mimetype:`text` part @@ -225,15 +258,22 @@ * ``encoding`` -- Content transfer encoding of the non-\ :mimetype:`text` part - The default value for *fmt* is ``None``, meaning :: + If *fmt* is ``None``, use the following default *fmt*: - [Non-text (%(type)s) part of message omitted, filename %(filename)s] + "[Non-text (%(type)s) part of message omitted, filename %(filename)s]" + + Optional *_mangle_from_* and *maxheaderlen* are as with the + :class:`Generator` base class, except that the default value for + *maxheaderlen* is ``78`` (the RFC standard default header length). .. rubric:: Footnotes -.. [#] This statement assumes that you use the appropriate setting for the - ``unixfrom`` argument, and that you set maxheaderlen=0 (which will - preserve whatever the input line lengths were). It is also not strictly - true, since in many cases runs of whitespace in headers are collapsed - into single blanks. The latter is a bug that will eventually be fixed. +.. [#] This statement assumes that you use the appropriate setting for + ``unixfrom``, and that there are no :mod:`policy` settings calling for + automatic adjustments (for example, + :attr:`~email.policy.Policy.refold_source` must be ``none``, which is + *not* the default). It is also not 100% true, since if the message + does not conform to the RFC standards occasionally information about the + exact original text is lost during parsing error recovery. It is a goal + to fix these latter edge cases when possible. diff --git a/Doc/library/email.header.rst b/Doc/library/email.header.rst --- a/Doc/library/email.header.rst +++ b/Doc/library/email.header.rst @@ -8,6 +8,14 @@ -------------- +This module is part of the legacy (``Compat32``) email API. In the current API +encoding and decoding of headers is handled transparently by the +dictionary-like API of the :class:`~email.message.EmailMessage` class. In +addition to uses in legacy code, this module can be useful in applications that +need to completely control the character sets used when encoding headers. + +The remaining text in this section is the original documentation of the module. + :rfc:`2822` is the base standard that describes the format of email messages. It derives from the older :rfc:`822` standard which came into widespread use at a time when most email was composed of ASCII characters only. :rfc:`2822` is a diff --git a/Doc/library/email.headerregistry.rst b/Doc/library/email.headerregistry.rst --- a/Doc/library/email.headerregistry.rst +++ b/Doc/library/email.headerregistry.rst @@ -7,19 +7,13 @@ .. moduleauthor:: R. David Murray .. sectionauthor:: R. David Murray -.. versionadded:: 3.3 - as a :term:`provisional module `. - **Source code:** :source:`Lib/email/headerregistry.py` -.. note:: +-------------- - The headerregistry module has been included in the standard library on a - :term:`provisional basis `. Backwards incompatible - changes (up to and including removal of the module) may occur if deemed - necessary by the core developers. +.. versionadded:: 3.3 as a :term:`provisional module `. --------------- +.. versionchanged:: 3.6 provisonal status removed. Headers are represented by customized subclasses of :class:`str`. The particular class used to represent a given header is determined by the @@ -86,10 +80,11 @@ .. method:: fold(*, policy) Return a string containing :attr:`~email.policy.Policy.linesep` - characters as required to correctly fold the header according - to *policy*. A :attr:`~email.policy.Policy.cte_type` of - ``8bit`` will be treated as if it were ``7bit``, since strings - may not contain binary data. + characters as required to correctly fold the header according to + *policy*. A :attr:`~email.policy.Policy.cte_type` of ``8bit`` will be + treated as if it were ``7bit``, since headers may not contain arbitrary + binary data. If :attr:`~email.policy.EmailPolicy.utf8` is ``False``, + non-ASCII data will be :rfc:`2047` encoded. ``BaseHeader`` by itself cannot be used to create a header object. It @@ -106,7 +101,7 @@ values for at least the keys ``decoded`` and ``defects``. ``decoded`` should be the string value for the header (that is, the header value fully decoded to unicode). The parse method should assume that *string* may - contain transport encoded parts, but should correctly handle all valid + contain content-transfer-encoded parts, but should correctly handle all valid unicode characters as well so that it can parse un-encoded header values. ``BaseHeader``'s ``__new__`` then creates the header instance, and calls its @@ -135,11 +130,10 @@ mechanism for encoding non-ASCII text as ASCII characters within a header value. When a *value* containing encoded words is passed to the constructor, the ``UnstructuredHeader`` parser converts such encoded words - back in to the original unicode, following the :rfc:`2047` rules for - unstructured text. The parser uses heuristics to attempt to decode certain - non-compliant encoded words. Defects are registered in such cases, as well - as defects for issues such as invalid characters within the encoded words or - the non-encoded text. + into unicode, following the :rfc:`2047` rules for unstructured text. The + parser uses heuristics to attempt to decode certain non-compliant encoded + words. Defects are registered in such cases, as well as defects for issues + such as invalid characters within the encoded words or the non-encoded text. This header type provides no additional attributes. @@ -213,15 +207,16 @@ the list of addresses is "flattened" into a one dimensional list). The ``decoded`` value of the header will have all encoded words decoded to - unicode. :class:`~encodings.idna` encoded domain names are also decoded to unicode. The - ``decoded`` value is set by :attr:`~str.join`\ ing the :class:`str` value of - the elements of the ``groups`` attribute with ``', '``. + unicode. :class:`~encodings.idna` encoded domain names are also decoded to + unicode. The ``decoded`` value is set by :attr:`~str.join`\ ing the + :class:`str` value of the elements of the ``groups`` attribute with ``', + '``. A list of :class:`.Address` and :class:`.Group` objects in any combination may be used to set the value of an address header. ``Group`` objects whose ``display_name`` is ``None`` will be interpreted as single addresses, which allows an address list to be copied with groups intact by using the list - obtained ``groups`` attribute of the source header. + obtained from the ``groups`` attribute of the source header. .. class:: SingleAddressHeader @@ -267,7 +262,7 @@ .. class:: ParameterizedMIMEHeader - MOME headers all start with the prefix 'Content-'. Each specific header has + MIME headers all start with the prefix 'Content-'. Each specific header has a certain value, described under the class for that header. Some can also take a list of supplemental parameters, which have a common format. This class serves as a base for all the MIME headers that take parameters. diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -3,91 +3,108 @@ .. module:: email.message :synopsis: The base class representing email messages. +.. moduleauthor:: R. David Murray +.. sectionauthor:: R. David Murray , + Barry A. Warsaw **Source code:** :source:`Lib/email/message.py` -------------- -The central class in the :mod:`email` package is the :class:`Message` class, -imported from the :mod:`email.message` module. It is the base class for the -:mod:`email` object model. :class:`Message` provides the core functionality for -setting and querying header fields, and for accessing message bodies. +.. versionadded:: 3.4 + the classes documented here were added :term:`provisionaly `. -Conceptually, a :class:`Message` object consists of *headers* and *payloads*. -Headers are :rfc:`2822` style field names and values where the field name and -value are separated by a colon. The colon is not part of either the field name -or the field value. +.. versionchanged:: 3.6 + provisional status removed, docs for legacy message class moved + to :ref:`compat32_message`. -Headers are stored and returned in case-preserving form but are matched -case-insensitively. There may also be a single envelope header, also known as -the *Unix-From* header or the ``From_`` header. The payload is either a string -in the case of simple message objects or a list of :class:`Message` objects for -MIME container documents (e.g. :mimetype:`multipart/\*` and -:mimetype:`message/rfc822`). +The central class in the :mod:`email` package is the :class:`EmailMessage` +class, imported from the :mod:`email.message` module. It is the base class for +the :mod:`email` object model. :class:`EmailMessage` provides the core +functionality for setting and querying header fields, for accessing message +bodies, and for creating or modifying structured messages. -:class:`Message` objects provide a mapping style interface for accessing the -message headers, and an explicit interface for accessing both the headers and -the payload. It provides convenience methods for generating a flat text -representation of the message object tree, for accessing commonly used header -parameters, and for recursively walking over the object tree. +An email message consists of *headers* and a *payload* (which is also referred +to as the *content*). Headers are :rfc:`5322` or :rfc:`6532` style field names +and values, where the field name and value are separated by a colon. The colon +is not part of either the field name or the field value. The payload may be a +simple text message, or a binary object, or a structured sequence of +sub-messages each with their own set of headers and their own payload. The +latter type of payload is indicated by the message having a MIME type such as +:mimetype:`multipart/\*` or :mimetype:`message/rfc822`. -Here are the methods of the :class:`Message` class: +The conceptual model provided by an :class:`EmailMessage` object is that of an +ordered dictionary of headers coupled with a *payload* that represents the +:rfc:`5322` body of the message, which might be a list of sub-``EmailMessage`` +objects. In addition to the normal dictionary methods for accessing the header +names and values, there are methods for accessing specialized information from +the headers (for example the MIME content type), for operating on the payload, +for generating a serialized version of the message, and for recursively walking +over the object tree. +The :class:`EmailMessage` dictionary-like interface is indexed by the header +names, which must be ASCII values. The values of the dictionary are strings +with some extra methods. Headers are stored and returned in case-preserving +form, but field names are matched case-insensitively. Unlike a real dict, +there is an ordering to the keys, and there can be duplicate keys. Additional +methods are provided for working with headers that have duplicate keys. -.. class:: Message(policy=compat32) +The *payload* is either a string or bytes object, in the case of simple message +objects, or a list of :class:`EmailMessage` objects, for MIME container +documents such as :mimetype:`multipart/\*` and :mimetype:`message/rfc822` +message objects. - If *policy* is specified (it must be an instance of a :mod:`~email.policy` - class) use the rules it specifies to update and serialize the representation - of the message. If *policy* is not set, use the :class:`compat32 - ` policy, which maintains backward compatibility with - the Python 3.2 version of the email package. For more information see the + +.. class:: EmailMessage(policy=default) + + If *policy* is specified use the rules it specifies to udpate and serialize + the representation of the message. If *policy* is not set, use the + :class:`~email.policy.default` policy, which follows the rules of the email + RFCs except for line endings (instead of the RFC mandated ``\r\n``, it uses + the Python standard ``\n`` line endings). For more information see the :mod:`~email.policy` documentation. - .. versionchanged:: 3.3 The *policy* keyword argument was added. + .. method:: as_string(unixfrom=False, maxheaderlen=None, policy=None) + Return the entire message flattened as a string. When optional + *unixfrom* is true, the envelope header is included in the returned + string. *unixfrom* defaults to ``False``. For backward compabitility + with the base :class:`~email.message.Message` class *maxheaderlen* is + accepted, but defaults to ``None``, which means that by default the line + length is controlled by the + :attr:`~email.policy.EmailPolicy.max_line_length` of the policy. The + *policy* argument may be used to override the default policy obtained + from the message instance. This can be used to control some of the + formatting produced by the method, since the specified *policy* will be + passed to the :class:`~email.generator.Generator`. - .. method:: as_string(unixfrom=False, maxheaderlen=0, policy=None) + Flattening the message may trigger changes to the :class:`EmailMessage` + if defaults need to be filled in to complete the transformation to a + string (for example, MIME boundaries may be generated or modified). - Return the entire message flattened as a string. When optional *unixfrom* - is true, the envelope header is included in the returned string. - *unixfrom* defaults to ``False``. For backward compabitility reasons, - *maxheaderlen* defaults to ``0``, so if you want a different value you - must override it explicitly (the value specified for *max_line_length* in - the policy will be ignored by this method). The *policy* argument may be - used to override the default policy obtained from the message instance. - This can be used to control some of the formatting produced by the - method, since the specified *policy* will be passed to the ``Generator``. + Note that this method is provided as a convenience and may not be the + most useful way to serialize messages in your application, especially if + you are dealing with multiple messages. See + :class:`email.generator.Generator` for a more flexible API for + serializing messages. Note also that this method is restricted to + producing messages serialized as "7 bit clean" when + :attr:`~email.policy.EmailPolicy.utf8` is ``False``, which is the default. - Flattening the message may trigger changes to the :class:`Message` if - defaults need to be filled in to complete the transformation to a string - (for example, MIME boundaries may be generated or modified). - - Note that this method is provided as a convenience and may not always - format the message the way you want. For example, by default it does - not do the mangling of lines that begin with ``From`` that is - required by the unix mbox format. For more flexibility, instantiate a - :class:`~email.generator.Generator` instance and use its - :meth:`~email.generator.Generator.flatten` method directly. For example:: - - from io import StringIO - from email.generator import Generator - fp = StringIO() - g = Generator(fp, mangle_from_=True, maxheaderlen=60) - g.flatten(msg) - text = fp.getvalue() - - If the message object contains binary data that is not encoded according - to RFC standards, the non-compliant data will be replaced by unicode - "unknown character" code points. (See also :meth:`.as_bytes` and - :class:`~email.generator.BytesGenerator`.) - - .. versionchanged:: 3.4 the *policy* keyword argument was added. + .. versionchanged:: 3.6 the default behavior when *maxheaderlen* + is not specified was changed from defaulting to 0 to defaulting + to the value of *max_line_length* from the policy. .. method:: __str__() - Equivalent to :meth:`.as_string()`. Allows ``str(msg)`` to produce a - string containing the formatted message. + Equivalent to `as_string(policy=self.policy.clone(utf8=True)`. Allows + ``str(msg)`` to produce a string containing the serialized message in a + readable format. + + .. versionchanged:: 3.4 the method was changed to use ``utf8=True``, + thus producing an :rfc:`6531`-like message representation, instead of + being a direct alias for :meth:`as_string`. .. method:: as_bytes(unixfrom=False, policy=None) @@ -98,52 +115,42 @@ used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified *policy* will be passed to the - ``BytesGenerator``. + :class:`~email.generator.BytesGenerator`. - Flattening the message may trigger changes to the :class:`Message` if - defaults need to be filled in to complete the transformation to a string - (for example, MIME boundaries may be generated or modified). + Flattening the message may trigger changes to the :class:`EmailMessage` + if defaults need to be filled in to complete the transformation to a + string (for example, MIME boundaries may be generated or modified). - Note that this method is provided as a convenience and may not always - format the message the way you want. For example, by default it does - not do the mangling of lines that begin with ``From`` that is - required by the unix mbox format. For more flexibility, instantiate a - :class:`~email.generator.BytesGenerator` instance and use its - :meth:`~email.generator.BytesGenerator.flatten` method directly. - For example:: - - from io import BytesIO - from email.generator import BytesGenerator - fp = BytesIO() - g = BytesGenerator(fp, mangle_from_=True, maxheaderlen=60) - g.flatten(msg) - text = fp.getvalue() - - .. versionadded:: 3.4 + Note that this method is provided as a convenience and may not be the + most useful way to serialize messages in your application, especially if + you are dealing with multiple messages. See + :class:`email.generator.BytesGenerator` for a more flexible API for + serializing messages. .. method:: __bytes__() Equivalent to :meth:`.as_bytes()`. Allows ``bytes(msg)`` to produce a - bytes object containing the formatted message. - - .. versionadded:: 3.4 + bytes object containing the serialized message. .. method:: is_multipart() Return ``True`` if the message's payload is a list of sub-\ - :class:`Message` objects, otherwise return ``False``. When + :class:`EmailMessage` objects, otherwise return ``False``. When :meth:`is_multipart` returns ``False``, the payload should be a string - object. (Note that :meth:`is_multipart` returning ``True`` does not - necessarily mean that "msg.get_content_maintype() == 'multipart'" will - return the ``True``. For example, ``is_multipart`` will return ``True`` - when the :class:`Message` is of type ``message/rfc822``.) + object (which might be a CTE encoded binary payload). Note that + :meth:`is_multipart` returning ``True`` does not necessarily mean that + "msg.get_content_maintype() == 'multipart'" will return the ``True``. + For example, ``is_multipart`` will return ``True`` when the + :class:`EmailMessage` is of type ``message/rfc822``. .. method:: set_unixfrom(unixfrom) - Set the message's envelope header to *unixfrom*, which should be a string. + Set the message's envelope header to *unixfrom*, which should be a + string. (See :class:`~mailbox.mboxMessage` for a brief description of + this header.) .. method:: get_unixfrom() @@ -152,109 +159,23 @@ envelope header was never set. - .. method:: attach(payload) - - Add the given *payload* to the current payload, which must be ``None`` or - a list of :class:`Message` objects before the call. After the call, the - payload will always be a list of :class:`Message` objects. If you want to - set the payload to a scalar object (e.g. a string), use - :meth:`set_payload` instead. - - - .. method:: get_payload(i=None, decode=False) - - Return the current payload, which will be a list of - :class:`Message` objects when :meth:`is_multipart` is ``True``, or a - string when :meth:`is_multipart` is ``False``. If the payload is a list - and you mutate the list object, you modify the message's payload in place. - - With optional argument *i*, :meth:`get_payload` will return the *i*-th - element of the payload, counting from zero, if :meth:`is_multipart` is - ``True``. An :exc:`IndexError` will be raised if *i* is less than 0 or - greater than or equal to the number of items in the payload. If the - payload is a string (i.e. :meth:`is_multipart` is ``False``) and *i* is - given, a :exc:`TypeError` is raised. - - Optional *decode* is a flag indicating whether the payload should be - decoded or not, according to the :mailheader:`Content-Transfer-Encoding` - header. When ``True`` and the message is not a multipart, the payload will - be decoded if this header's value is ``quoted-printable`` or ``base64``. - If some other encoding is used, or :mailheader:`Content-Transfer-Encoding` - header is missing, the payload is - returned as-is (undecoded). In all cases the returned value is binary - data. If the message is a multipart and the *decode* flag is ``True``, - then ``None`` is returned. If the payload is base64 and it was not - perfectly formed (missing padding, characters outside the base64 - alphabet), then an appropriate defect will be added to the message's - defect property (:class:`~email.errors.InvalidBase64PaddingDefect` or - :class:`~email.errors.InvalidBase64CharactersDefect`, respectively). - - When *decode* is ``False`` (the default) the body is returned as a string - without decoding the :mailheader:`Content-Transfer-Encoding`. However, - for a :mailheader:`Content-Transfer-Encoding` of 8bit, an attempt is made - to decode the original bytes using the ``charset`` specified by the - :mailheader:`Content-Type` header, using the ``replace`` error handler. - If no ``charset`` is specified, or if the ``charset`` given is not - recognized by the email package, the body is decoded using the default - ASCII charset. - - - .. method:: set_payload(payload, charset=None) - - Set the entire message object's payload to *payload*. It is the client's - responsibility to ensure the payload invariants. Optional *charset* sets - the message's default character set; see :meth:`set_charset` for details. - - .. method:: set_charset(charset) - - Set the character set of the payload to *charset*, which can either be a - :class:`~email.charset.Charset` instance (see :mod:`email.charset`), a - string naming a character set, or ``None``. If it is a string, it will - be converted to a :class:`~email.charset.Charset` instance. If *charset* - is ``None``, the ``charset`` parameter will be removed from the - :mailheader:`Content-Type` header (the message will not be otherwise - modified). Anything else will generate a :exc:`TypeError`. - - If there is no existing :mailheader:`MIME-Version` header one will be - added. If there is no existing :mailheader:`Content-Type` header, one - will be added with a value of :mimetype:`text/plain`. Whether the - :mailheader:`Content-Type` header already exists or not, its ``charset`` - parameter will be set to *charset.output_charset*. If - *charset.input_charset* and *charset.output_charset* differ, the payload - will be re-encoded to the *output_charset*. If there is no existing - :mailheader:`Content-Transfer-Encoding` header, then the payload will be - transfer-encoded, if needed, using the specified - :class:`~email.charset.Charset`, and a header with the appropriate value - will be added. If a :mailheader:`Content-Transfer-Encoding` header - already exists, the payload is assumed to already be correctly encoded - using that :mailheader:`Content-Transfer-Encoding` and is not modified. - - .. method:: get_charset() - - Return the :class:`~email.charset.Charset` instance associated with the - message's payload. - - The following methods implement a mapping-like interface for accessing the - message's :rfc:`2822` headers. Note that there are some semantic differences + The following methods implement the mapping-like interface for accessing the + message's headers. Note that there are some semantic differences between these methods and a normal mapping (i.e. dictionary) interface. For example, in a dictionary there are no duplicate keys, but here there may be duplicate message headers. Also, in dictionaries there is no guaranteed - order to the keys returned by :meth:`keys`, but in a :class:`Message` object, - headers are always returned in the order they appeared in the original - message, or were added to the message later. Any header deleted and then - re-added are always appended to the end of the header list. + order to the keys returned by :meth:`keys`, but in an :class:`EmailMessage` + object, headers are always returned in the order they appeared in the + original message, or in which they were added to the message later. Any + header deleted and then re-added is always appended to the end of the + header list. - These semantic differences are intentional and are biased toward maximal - convenience. + These semantic differences are intentional and are biased toward + convenience in the most common use cases. Note that in all cases, any envelope header present in the message is not included in the mapping interface. - In a model generated from bytes, any header values that (in contravention of - the RFCs) contain non-ASCII bytes will, when retrieved through this - interface, be represented as :class:`~email.header.Header` objects with - a charset of `unknown-8bit`. - .. method:: __len__() @@ -264,8 +185,8 @@ .. method:: __contains__(name) Return true if the message object has a field named *name*. Matching is - done case-insensitively and *name* should not include the trailing colon. - Used for the ``in`` operator, e.g.:: + done without regard to case and *name* does not include the trailing + colon. Used for the ``in`` operator. For example:: if 'message-id' in myMessage: print('Message-ID:', myMessage['message-id']) @@ -273,20 +194,23 @@ .. method:: __getitem__(name) - Return the value of the named header field. *name* should not include the + Return the value of the named header field. *name* does not include the colon field separator. If the header is missing, ``None`` is returned; a :exc:`KeyError` is never raised. Note that if the named field appears more than once in the message's headers, exactly which of those field values will be returned is undefined. Use the :meth:`get_all` method to get the values of all the - extant named headers. + extant headers named *name*. + + Using the standard (non-``compat32``) policies, the returned value is an + instance of a subclass of :class:`email.headerregistry.BaseHeader`. .. method:: __setitem__(name, val) Add a header to the message with field name *name* and value *val*. The - field is appended to the end of the message's existing fields. + field is appended to the end of the message's existing headers. Note that this does *not* overwrite or delete any existing header with the same name. If you want to ensure that the new header is the only one present in the @@ -295,6 +219,13 @@ del msg['subject'] msg['subject'] = 'Python roolz!' + If the :mod:`policy` defines certain haders to be unique (as the standard + policies do), this method may raise a :exc:`ValueError` when an attempt + is made to assign a value to such a header when one already exists. This + behavior is intentional for consistency's sake, but do not depend on it + as we may choose to make such assignments do an automatic deletion of the + existing header in the future. + .. method:: __delitem__(name) @@ -323,9 +254,10 @@ Return the value of the named header field. This is identical to :meth:`__getitem__` except that optional *failobj* is returned if the - named header is missing (defaults to ``None``). + named header is missing (*failobj* defaults to ``None``). - Here are some additional useful methods: + + Here are some additional useful header related methods: .. method:: get_all(name, failobj=None) @@ -346,17 +278,19 @@ taken as the parameter name, with underscores converted to dashes (since dashes are illegal in Python identifiers). Normally, the parameter will be added as ``key="value"`` unless the value is ``None``, in which case - only the key will be added. If the value contains non-ASCII characters, - it can be specified as a three tuple in the format - ``(CHARSET, LANGUAGE, VALUE)``, where ``CHARSET`` is a string naming the - charset to be used to encode the value, ``LANGUAGE`` can usually be set - to ``None`` or the empty string (see :rfc:`2231` for other possibilities), - and ``VALUE`` is the string value containing non-ASCII code points. If - a three tuple is not passed and the value contains non-ASCII characters, - it is automatically encoded in :rfc:`2231` format using a ``CHARSET`` - of ``utf-8`` and a ``LANGUAGE`` of ``None``. + only the key will be added. - Here's an example:: + If the value contains non-ASCII characters, the charset and language may + be explicitly controlled by specifing the value as a three tuple in the + format ``(CHARSET, LANGUAGE, VALUE)``, where ``CHARSET`` is a string + naming the charset to be used to encode the value, ``LANGUAGE`` can + usually be set to ``None`` or the empty string (see :rfc:`2231` for other + possibilities), and ``VALUE`` is the string value containing non-ASCII + code points. If a three tuple is not passed and the value contains + non-ASCII characters, it is automatically encoded in :rfc:`2231` format + using a ``CHARSET`` of ``utf-8`` and a ``LANGUAGE`` of ``None``. + + Here is an example:: msg.add_header('Content-Disposition', 'attachment', filename='bud.gif') @@ -364,37 +298,35 @@ Content-Disposition: attachment; filename="bud.gif" - An example with non-ASCII characters:: + An example of the extended interface with non-ASCII characters:: msg.add_header('Content-Disposition', 'attachment', filename=('iso-8859-1', '', 'Fu?baller.ppt')) - Which produces :: - - Content-Disposition: attachment; filename*="iso-8859-1''Fu%DFballer.ppt" - .. method:: replace_header(_name, _value) Replace a header. Replace the first header found in the message that - matches *_name*, retaining header order and field name case. If no - matching header was found, a :exc:`KeyError` is raised. + matches *_name*, retaining header order and field name case of the + original header. If no matching header is found, raise a + :exc:`KeyError`. .. method:: get_content_type() - Return the message's content type. The returned string is coerced to - lower case of the form :mimetype:`maintype/subtype`. If there was no - :mailheader:`Content-Type` header in the message the default type as given - by :meth:`get_default_type` will be returned. Since according to - :rfc:`2045`, messages always have a default type, :meth:`get_content_type` - will always return a value. + Return the message's content type, coerced to lower case of the form + :mimetype:`maintype/subtype`. If there is no :mailheader:`Content-Type` + header in the message return the value returned by + :meth:`get_default_type`. If the :mailheader:`Content-Type` header is + invalid, return ``text/plain``. - :rfc:`2045` defines a message's default type to be :mimetype:`text/plain` - unless it appears inside a :mimetype:`multipart/digest` container, in - which case it would be :mimetype:`message/rfc822`. If the - :mailheader:`Content-Type` header has an invalid type specification, - :rfc:`2045` mandates that the default type be :mimetype:`text/plain`. + (According to :rfc:`2045`, messages always have a default type, + :meth:`get_content_type` will always return a value. :rfc:`2045` defines + a message's default type to be :mimetype:`text/plain` unless it appears + inside a :mimetype:`multipart/digest` container, in which case it would + be :mimetype:`message/rfc822`. If the :mailheader:`Content-Type` header + has an invalid type specification, :rfc:`2045` mandates that the default + type be :mimetype:`text/plain`.) .. method:: get_content_maintype() @@ -420,81 +352,41 @@ .. method:: set_default_type(ctype) Set the default content type. *ctype* should either be - :mimetype:`text/plain` or :mimetype:`message/rfc822`, although this is not - enforced. The default content type is not stored in the - :mailheader:`Content-Type` header. - - - .. method:: get_params(failobj=None, header='content-type', unquote=True) - - Return the message's :mailheader:`Content-Type` parameters, as a list. - The elements of the returned list are 2-tuples of key/value pairs, as - split on the ``'='`` sign. The left hand side of the ``'='`` is the key, - while the right hand side is the value. If there is no ``'='`` sign in - the parameter the value is the empty string, otherwise the value is as - described in :meth:`get_param` and is unquoted if optional *unquote* is - ``True`` (the default). - - Optional *failobj* is the object to return if there is no - :mailheader:`Content-Type` header. Optional *header* is the header to - search instead of :mailheader:`Content-Type`. - - - .. method:: get_param(param, failobj=None, header='content-type', unquote=True) - - Return the value of the :mailheader:`Content-Type` header's parameter - *param* as a string. If the message has no :mailheader:`Content-Type` - header or if there is no such parameter, then *failobj* is returned - (defaults to ``None``). - - Optional *header* if given, specifies the message header to use instead of - :mailheader:`Content-Type`. - - Parameter keys are always compared case insensitively. The return value - can either be a string, or a 3-tuple if the parameter was :rfc:`2231` - encoded. When it's a 3-tuple, the elements of the value are of the form - ``(CHARSET, LANGUAGE, VALUE)``. Note that both ``CHARSET`` and - ``LANGUAGE`` can be ``None``, in which case you should consider ``VALUE`` - to be encoded in the ``us-ascii`` charset. You can usually ignore - ``LANGUAGE``. - - If your application doesn't care whether the parameter was encoded as in - :rfc:`2231`, you can collapse the parameter value by calling - :func:`email.utils.collapse_rfc2231_value`, passing in the return value - from :meth:`get_param`. This will return a suitably decoded Unicode - string when the value is a tuple, or the original string unquoted if it - isn't. For example:: - - rawparam = msg.get_param('foo') - param = email.utils.collapse_rfc2231_value(rawparam) - - In any case, the parameter value (either the returned string, or the - ``VALUE`` item in the 3-tuple) is always unquoted, unless *unquote* is set - to ``False``. + :mimetype:`text/plain` or :mimetype:`message/rfc822`, although this is + not enforced. The default content type is not stored in the + :mailheader:`Content-Type` header, so it only affects the return value of + the ``get_content_type`` methods when no :mailheader:`Content-Type` + header is present in the message. .. method:: set_param(param, value, header='Content-Type', requote=True, \ charset=None, language='', replace=False) Set a parameter in the :mailheader:`Content-Type` header. If the - parameter already exists in the header, its value will be replaced with - *value*. If the :mailheader:`Content-Type` header as not yet been defined - for this message, it will be set to :mimetype:`text/plain` and the new - parameter value will be appended as per :rfc:`2045`. + parameter already exists in the header, replace its value with *value*. + When *header* is ``Content-Type`` (the default) and the header does not + yet exist in the message, add it, set its value to + :mimetype:`text/plain`, and append the new parameter value. Optional + *header* specifies an alternative header to :mailheader:`Content-Type`. - Optional *header* specifies an alternative header to - :mailheader:`Content-Type`, and all parameters will be quoted as necessary - unless optional *requote* is ``False`` (the default is ``True``). - - If optional *charset* is specified, the parameter will be encoded - according to :rfc:`2231`. Optional *language* specifies the RFC 2231 - language, defaulting to the empty string. Both *charset* and *language* - should be strings. + If the value contains non-ASCII characters, the charset and language may + be explicity specified using the optional *charset* and *language* + parameters. Optional *language* specifies the :rfc:`2231` language, + defaulting to the empty string. Both *charset* and *language* should be + strings. The default is to use the ``utf8`` *charset* and ``None`` for + the *language*. If *replace* is ``False`` (the default) the header is moved to the end of the list of headers. If *replace* is ``True``, the header will be updated in place. + Use of the *requote* parameter with :class:`EmailMessage` objects is + deprecated. + + Note that existing parameter values of headers may be accessed through + the :attr:`~email.headerregistry.BaseHeader.params` attribute of the + header value (for example, ``msg['Content-Type'].params['charset']``. + .. versionchanged:: 3.4 ``replace`` keyword was added. @@ -502,25 +394,11 @@ Remove the given parameter completely from the :mailheader:`Content-Type` header. The header will be re-written in place without the parameter or - its value. All values will be quoted as necessary unless *requote* is - ``False`` (the default is ``True``). Optional *header* specifies an - alternative to :mailheader:`Content-Type`. + its value. Optional *header* specifies an alternative to + :mailheader:`Content-Type`. - - .. method:: set_type(type, header='Content-Type', requote=True) - - Set the main type and subtype for the :mailheader:`Content-Type` - header. *type* must be a string in the form :mimetype:`maintype/subtype`, - otherwise a :exc:`ValueError` is raised. - - This method replaces the :mailheader:`Content-Type` header, keeping all - the parameters in place. If *requote* is ``False``, this leaves the - existing header's quoting as is, otherwise the parameters will be quoted - (the default). - - An alternative header can be specified in the *header* argument. When the - :mailheader:`Content-Type` header is set a :mailheader:`MIME-Version` - header is also added. + Use of the *requote* parameter with :class:`EmailMessage` objects is + deprecated. .. method:: get_filename(failobj=None) @@ -549,12 +427,11 @@ necessary. A :exc:`~email.errors.HeaderParseError` is raised if the message object has no :mailheader:`Content-Type` header. - Note that using this method is subtly different than deleting the old + Note that using this method is subtly different from deleting the old :mailheader:`Content-Type` header and adding a new one with the new boundary via :meth:`add_header`, because :meth:`set_boundary` preserves the order of the :mailheader:`Content-Type` header in the list of - headers. However, it does *not* preserve any continuation lines which may - have been present in the original :mailheader:`Content-Type` header. + headers. .. method:: get_content_charset(failobj=None) @@ -563,9 +440,6 @@ coerced to lower case. If there is no :mailheader:`Content-Type` header, or if that header has no ``charset`` parameter, *failobj* is returned. - Note that this method differs from :meth:`get_charset` which returns the - :class:`~email.charset.Charset` instance for the default encoding of the message body. - .. method:: get_charsets(failobj=None) @@ -575,10 +449,19 @@ Each item in the list will be a string which is the value of the ``charset`` parameter in the :mailheader:`Content-Type` header for the - represented subpart. However, if the subpart has no - :mailheader:`Content-Type` header, no ``charset`` parameter, or is not of - the :mimetype:`text` main MIME type, then that item in the returned list - will be *failobj*. + represented subpart. If the subpart has no :mailheader:`Content-Type` + header, no ``charset`` parameter, or is not of the :mimetype:`text` main + MIME type, then that item in the returned list will be *failobj*. + + + .. method:: is_attachment + + Return ``True`` if there is a :mailheader:`Content-Disposition` header + and its (case insensitive) value is ``attachment``, ``False`` otherwise. + + .. versionchanged:: 3.4.2 + is_attachment is now a method instead of a property, for consistency + with :meth:`~email.message.Message.is_multipart`. .. method:: get_content_disposition() @@ -590,6 +473,11 @@ .. versionadded:: 3.5 + + The following methods relate to interrogating and manipulating the content + (payload) of the message. + + .. method:: walk() The :meth:`walk` method is an all-purpose generator which can be used to @@ -651,8 +539,169 @@ into the subparts. - :class:`Message` objects can also optionally contain two instance attributes, - which can be used when generating the plain text of a MIME message. + .. method:: get_body(preferencelist=('related', 'html', 'plain')) + + Return the MIME part that is the best candidate to be the "body" of the + message. + + *preferencelist* must be a sequence of strings from the set ``related``, + ``html``, and ``plain``, and indicates the order of preference for the + content type of the part returned. + + Start looking for candidate matches with the object on which the + ``get_body`` method is called. + + If ``related`` is not included in *preferencelist*, consider the root + part (or subpart of the root part) of any related encountered as a + candidate if the (sub-)part matches a preference. + + When encountering a ``multipart/related``, check the ``start`` parameter + and if a part with a matching :mailheader:`Content-ID` is found, consider + only it when looking for candidate matches. Otherwise consider only the + first (default root) part of the ``multipart/related``. + + If a part has a :mailheader:`Content-Disposition` header, only consider + the part a candidate match if the value of the header is ``inline``. + + If none of the candidates matches any of the preferences in + *preferneclist*, return ``None``. + + Notes: (1) For most applications the only *preferencelist* combinations + that really make sense are ``('plain',)``, ``('html', 'plain')``, and the + default ``('related', 'html', 'plain')``. (2) Because matching starts + with the object on which ``get_body`` is called, calling ``get_body`` on + a ``multipart/related`` will return the object itself unless + *preferencelist* has a non-default value. (3) Messages (or message parts) + that do not specify a :mailheader:`Content-Type` or whose + :mailheader:`Content-Type` header is invalid will be treated as if they + are of type ``text/plain``, which may occasionally cause ``get_body`` to + return unexpected results. + + + .. method:: iter_attachments() + + Return an iterator over all of the immediate sub-parts of the message + that are not candidate "body" parts. That is, skip the first occurrence + of each of ``text/plain``, ``text/html``, ``multipart/related``, or + ``multipart/alternative`` (unless they are explicitly marked as + attachments via :mailheader:`Content-Disposition: attachment`), and + return all remaining parts. When applied directly to a + ``multipart/related``, return an iterator over the all the related parts + except the root part (ie: the part pointed to by the ``start`` parameter, + or the first part if there is no ``start`` parameter or the ``start`` + parameter doesn't match the :mailheader:`Content-ID` of any of the + parts). When applied directly to a ``multipart/alternative`` or a + non-``multipart``, return an empty iterator. + + + .. method:: iter_parts() + + Return an iterator over all of the immediate sub-parts of the message, + which will be empty for a non-``multipart``. (See also + :meth:`~email.message.EmailMessage.walk`.) + + + .. method:: get_content(*args, content_manager=None, **kw) + + Call the :meth:`~email.contentmanager.ContentManager.get_content` method + of the *content_manager*, passing self as the message object, and passing + along any other arguments or keywords as additional arguments. If + *content_manager* is not specified, use the ``content_manager`` specified + by the current :mod:`~email.policy`. + + + .. method:: set_content(*args, content_manager=None, **kw) + + Call the :meth:`~email.contentmanager.ContentManager.set_content` method + of the *content_manager*, passing self as the message object, and passing + along any other arguments or keywords as additional arguments. If + *content_manager* is not specified, use the ``content_manager`` specified + by the current :mod:`~email.policy`. + + + .. method:: make_related(boundary=None) + + Convert a non-``multipart`` message into a ``multipart/related`` message, + moving any existing :mailheader:`Content-` headers and payload into a + (new) first part of the ``multipart``. If *boundary* is specified, use + it as the boundary string in the multipart, otherwise leave the boundary + to be automatically created when it is needed (for example, when the + message is serialized). + + + .. method:: make_alternative(boundary=None) + + Convert a non-``multipart`` or a ``multipart/related`` into a + ``multipart/alternative``, moving any existing :mailheader:`Content-` + headers and payload into a (new) first part of the ``multipart``. If + *boundary* is specified, use it as the boundary string in the multipart, + otherwise leave the boundary to be automatically created when it is + needed (for example, when the message is serialized). + + + .. method:: make_mixed(boundary=None) + + Convert a non-``multipart``, a ``multipart/related``, or a + ``multipart-alternative`` into a ``multipart/mixed``, moving any existing + :mailheader:`Content-` headers and payload into a (new) first part of the + ``multipart``. If *boundary* is specified, use it as the boundary string + in the multipart, otherwise leave the boundary to be automatically + created when it is needed (for example, when the message is serialized). + + + .. method:: add_related(*args, content_manager=None, **kw) + + If the message is a ``multipart/related``, create a new message + object, pass all of the arguments to its :meth:`set_content` method, + and :meth:`~email.message.Message.attach` it to the ``multipart``. If + the message is a non-``multipart``, call :meth:`make_related` and then + proceed as above. If the message is any other type of ``multipart``, + raise a :exc:`TypeError`. If *content_manager* is not specified, use + the ``content_manager`` specified by the current :mod:`~email.policy`. + If the added part has no :mailheader:`Content-Disposition` header, + add one with the value ``inline``. + + + .. method:: add_alternative(*args, content_manager=None, **kw) + + If the message is a ``multipart/alternative``, create a new message + object, pass all of the arguments to its :meth:`set_content` method, and + :meth:`~email.message.Message.attach` it to the ``multipart``. If the + message is a non-``multipart`` or ``multipart/related``, call + :meth:`make_alternative` and then proceed as above. If the message is + any other type of ``multipart``, raise a :exc:`TypeError`. If + *content_manager* is not specified, use the ``content_manager`` specified + by the current :mod:`~email.policy`. + + + .. method:: add_attachment(*args, content_manager=None, **kw) + + If the message is a ``multipart/mixed``, create a new message object, + pass all of the arguments to its :meth:`set_content` method, and + :meth:`~email.message.Message.attach` it to the ``multipart``. If the + message is a non-``multipart``, ``multipart/related``, or + ``multipart/alternative``, call :meth:`make_mixed` and then proceed as + above. If *content_manager* is not specified, use the ``content_manager`` + specified by the current :mod:`~email.policy`. If the added part + has no :mailheader:`Content-Disposition` header, add one with the value + ``attachment``. This method can be used both for explicit attachments + (:mailheader:`Content-Disposition: attachment` and ``inline`` attachments + (:mailheader:`Content-Disposition: inline`), by passing appropriate + options to the ``content_manager``. + + + .. method:: clear() + + Remove the payload and all of the headers. + + + .. method:: clear_content() + + Remove the payload and all of the :exc:`Content-` headers, leaving + all other headers intact and in their original order. + + + :class:`EmailMessage` objects have the following instance attributes: .. attribute:: preamble @@ -682,11 +731,8 @@ The *epilogue* attribute acts the same way as the *preamble* attribute, except that it contains text that appears between the last boundary and - the end of the message. - - You do not need to set the epilogue to the empty string in order for the - :class:`~email.generator.Generator` to print a newline at the end of the - file. + the end of the message. As with the :attr:`~EmailMessage.preamble`, + if there is no epilog text this attribute will be ``None``. .. attribute:: defects @@ -694,3 +740,11 @@ The *defects* attribute contains a list of all the problems found when parsing this message. See :mod:`email.errors` for a detailed description of the possible parsing defects. + + +.. class:: MIMEPart(policy=default) + + This class represents a subpart of a MIME message. It is identical to + :class:`EmailMessage`, except that no :mailheader:`MIME-Version` headers are + added when :meth:`~EmailMessage.set_content` is called, since sub-parts do + not need their own :mailheader:`MIME-Version` headers. diff --git a/Doc/library/email.mime.rst b/Doc/library/email.mime.rst --- a/Doc/library/email.mime.rst +++ b/Doc/library/email.mime.rst @@ -8,6 +8,11 @@ -------------- +This module is part of the legacy (``Compat32``) email API. Its functionality +is partially replaced by the :mod:`~email.contentmanager` in the new API, but +in certain applications these classes may still be useful, even in non-legacy +code. + Ordinarily, you get a message object structure by passing a file or some text to a parser, which parses the text and returns the root message object. However you can also build a complete message structure from scratch, or even individual diff --git a/Doc/library/email.parser.rst b/Doc/library/email.parser.rst --- a/Doc/library/email.parser.rst +++ b/Doc/library/email.parser.rst @@ -8,190 +8,155 @@ -------------- -Message object structures can be created in one of two ways: they can be created -from whole cloth by instantiating :class:`~email.message.Message` objects and -stringing them together via :meth:`~email.message.Message.attach` and -:meth:`~email.message.Message.set_payload` calls, or they -can be created by parsing a flat text representation of the email message. +Message object structures can be created in one of two ways: they can be +created from whole cloth by creating an :class:`~email.message.EmailMessage` +object, adding headers using the dictionary interface, and adding payload(s) +using :meth:`~email.message.EmailMessage.set_content` and related methods, or +they can be created by parsing a serialized representation of the email +message. The :mod:`email` package provides a standard parser that understands most email -document structures, including MIME documents. You can pass the parser a string -or a file object, and the parser will return to you the root -:class:`~email.message.Message` instance of the object structure. For simple, -non-MIME messages the payload of this root object will likely be a string -containing the text of the message. For MIME messages, the root object will -return ``True`` from its :meth:`~email.message.Message.is_multipart` method, and -the subparts can be accessed via the :meth:`~email.message.Message.get_payload` -and :meth:`~email.message.Message.walk` methods. +document structures, including MIME documents. You can pass the parser a +bytes, string or file object, and the parser will return to you the root +:class:`~email.message.EmailMessage` instance of the object structure. For +simple, non-MIME messages the payload of this root object will likely be a +string containing the text of the message. For MIME messages, the root object +will return ``True`` from its :meth:`~email.message.EmailMessage.is_multipart` +method, and the subparts can be accessed via the payload manipulation methods, +such as :meth:`~email.message.EmailMessage.get_body`, +:meth:`~email.message.EmailMessage.iter_parts`, and +:meth:`~email.message.EmailMessage.walk`. -There are actually two parser interfaces available for use, the classic -:class:`Parser` API and the incremental :class:`FeedParser` API. The classic -:class:`Parser` API is fine if you have the entire text of the message in memory -as a string, or if the entire message lives in a file on the file system. -:class:`FeedParser` is more appropriate for when you're reading the message from -a stream which might block waiting for more input (e.g. reading an email message -from a socket). The :class:`FeedParser` can consume and parse the message -incrementally, and only returns the root object when you close the parser [#]_. +There are actually two parser interfaces available for use, the :class:`Parser` +API and the incremental :class:`FeedParser` API. The :class:`Parser` API is +most useful if you have the entire text of the message in memory, or if the +entire message lives in a file on the file system. :class:`FeedParser` is more +appropriate when you are reading the message from a stream which might block +waiting for more input (such as reading an email message from a socket). The +:class:`FeedParser` can consume and parse the message incrementally, and only +returns the root object when you close the parser. Note that the parser can be extended in limited ways, and of course you can -implement your own parser completely from scratch. There is no magical -connection between the :mod:`email` package's bundled parser and the -:class:`~email.message.Message` class, so your custom parser can create message -object trees any way it finds necessary. +implement your own parser completely from scratch. All of the logic that +connects the :mod:`email` package's bundled parser and the +:class:`~email.message.EmailMessage` class is embodied in the :mod:`policy` +class, so a custom parser can create message object trees any way it finds +necessary by implementing custom versions of the appropriate :mod:`policy` +methods. FeedParser API ^^^^^^^^^^^^^^ -The :class:`FeedParser`, imported from the :mod:`email.feedparser` module, -provides an API that is conducive to incremental parsing of email messages, such -as would be necessary when reading the text of an email message from a source -that can block (e.g. a socket). The :class:`FeedParser` can of course be used -to parse an email message fully contained in a string or a file, but the classic -:class:`Parser` API may be more convenient for such use cases. The semantics -and results of the two parser APIs are identical. +The :class:`BytesFeedParser`, imported from the :mod:`email.feedparser` module, +provides an API that is conducive to incremental parsing of email messages, +such as would be necessary when reading the text of an email message from a +source that can block (such as a socket). The :class:`BytesFeedParser` can of +course be used to parse an email message fully contained in a :term:`bytes-like +object`, string, or file, but the :class:`BytesParser` API may be more +convenient for such use cases. The semantics and results of the two parser +APIs are identical. -The :class:`FeedParser`'s API is simple; you create an instance, feed it a bunch -of text until there's no more to feed it, then close the parser to retrieve the -root message object. The :class:`FeedParser` is extremely accurate when parsing -standards-compliant messages, and it does a very good job of parsing -non-compliant messages, providing information about how a message was deemed -broken. It will populate a message object's *defects* attribute with a list of -any problems it found in a message. See the :mod:`email.errors` module for the +The :class:`BytesFeedParser`'s API is simple; you create an instance, feed it a +bunch of bytes until there's no more to feed it, then close the parser to +retrieve the root message object. The :class:`BytesFeedParser` is extremely +accurate when parsing standards-compliant messages, and it does a very good job +of parsing non-compliant messages, providing information about how a message +was deemed broken. It will populate a message object's +:attr:`~email.message.EmailMessage.defects` attribute with a list of any +problems it found in a message. See the :mod:`email.errors` module for the list of defects that it can find. -Here is the API for the :class:`FeedParser`: +Here is the API for the :class:`BytesFeedParser`: -.. class:: FeedParser(_factory=email.message.Message, *, policy=policy.compat32) +.. class:: BytesFeedParser(_factory=None, *, policy=policy.compat32) - Create a :class:`FeedParser` instance. Optional *_factory* is a no-argument - callable that will be called whenever a new message object is needed. It - defaults to the :class:`email.message.Message` class. + Create a :class:`BytesFeedParser` instance. Optional *_factory* is a + no-argument callable; if not specified determine the default based on the + *policy*. Call *_factory* whenever a new message object is needed. - If *policy* is specified (it must be an instance of a :mod:`~email.policy` - class) use the rules it specifies to update the representation of the - message. If *policy* is not set, use the :class:`compat32 - ` policy, which maintains backward compatibility with - the Python 3.2 version of the email package. For more information see the + If *policy* is specified use the rules it specifies to update the + representation of the message. If *policy* is not set, use the + :class:`compat32 ` policy, which maintains backward + compatibility with the Python 3.2 version of the email package and provides + :class:`~email.message.Message` as the default factory. All other policies + provide :class:`~email.message.EmailMessage` as the default *_factory*. For + more information on what else *policy* controls, see the :mod:`~email.policy` documentation. + Note: **The policy keyword should always be specified**; The default will + change to :data:`email.policy.default` in a future version of Python. + + .. versionadded:: 3.2 + .. versionchanged:: 3.3 Added the *policy* keyword. + .. method:: feed(data) - Feed the :class:`FeedParser` some more data. *data* should be a string - containing one or more lines. The lines can be partial and the - :class:`FeedParser` will stitch such partial lines together properly. The - lines in the string can have any of the common three line endings, - carriage return, newline, or carriage return and newline (they can even be - mixed). + Feed the parser some more data. *data* should be a :term:`bytes-like + object` containing one or more lines. The lines can be partial and the + parser will stitch such partial lines together properly. The lines can + have any of the three common line endings: carriage return, newline, or + carriage return and newline (they can even be mixed). + .. method:: close() - Closing a :class:`FeedParser` completes the parsing of all previously fed - data, and returns the root message object. It is undefined what happens - if you feed more data to a closed :class:`FeedParser`. + Complete the parsing of all previously fed data and return the root + message object. It is undefined what happens if :meth:`~feed` is called + after this method has been called. -.. class:: BytesFeedParser(_factory=email.message.Message) +.. class:: FeedParser(_factory=None, *, policy=policy.compat32) - Works exactly like :class:`FeedParser` except that the input to the - :meth:`~FeedParser.feed` method must be bytes and not string. + Works like :class:`BytesFeedParser` except that the input to the + :meth:`~BytesFeedParser.feed` method must be a string. This is of limited + utility, since the only way for such a message to be valid is for it to + contain only ASCII text or, if :attr:`~email.policy.Policy.utf8` is + ``True``, no binary attachments. - .. versionadded:: 3.2 + .. versionchanged:: 3.3 Added the *policy* keyword. -Parser class API -^^^^^^^^^^^^^^^^ +Parser API +^^^^^^^^^^ -The :class:`Parser` class, imported from the :mod:`email.parser` module, +The :class:`BytesParser` class, imported from the :mod:`email.parser` module, provides an API that can be used to parse a message when the complete contents -of the message are available in a string or file. The :mod:`email.parser` -module also provides header-only parsers, called :class:`HeaderParser` and -:class:`BytesHeaderParser`, which can be used if you're only interested in the -headers of the message. :class:`HeaderParser` and :class:`BytesHeaderParser` +of the message are available in a :term:`bytes-like object` or file. The +:mod:`email.parser` module also provides :class:`Parser` for parsing strings, +and header-only parsers, :class:`BytesHeaderParser` and +:class:`HeaderParser`, which can be used if you're only interested in the +headers of the message. :class:`BytesHeaderParser` and :class:`HeaderParser` can be much faster in these situations, since they do not attempt to parse the -message body, instead setting the payload to the raw body as a string. They -have the same API as the :class:`Parser` and :class:`BytesParser` classes. +message body, instead setting the payload to the raw body. -.. versionadded:: 3.3 - The BytesHeaderParser class. +.. class:: BytesParser(_class=None, *, policy=policy.compat32) -.. class:: Parser(_class=email.message.Message, *, policy=policy.compat32) + Create a :class:`BytesParser` instance. The *_class* and *policy* + arguments have the same meaning and sematnics as the *_factory* + and *policy* arguments of :class:`BytesFeedParser`. - The constructor for the :class:`Parser` class takes an optional argument - *_class*. This must be a callable factory (such as a function or a class), and - it is used whenever a sub-message object needs to be created. It defaults to - :class:`~email.message.Message` (see :mod:`email.message`). The factory will - be called without arguments. - - If *policy* is specified (it must be an instance of a :mod:`~email.policy` - class) use the rules it specifies to update the representation of the - message. If *policy* is not set, use the :class:`compat32 - ` policy, which maintains backward compatibility with - the Python 3.2 version of the email package. For more information see the - :mod:`~email.policy` documentation. + Note: **The policy keyword should always be specified**; The default will + change to :data:`email.policy.default` in a future version of Python. .. versionchanged:: 3.3 Removed the *strict* argument that was deprecated in 2.4. Added the *policy* keyword. - The other public :class:`Parser` methods are: - - - .. method:: parse(fp, headersonly=False) - - Read all the data from the file-like object *fp*, parse the resulting - text, and return the root message object. *fp* must support both the - :meth:`~io.TextIOBase.readline` and the :meth:`~io.TextIOBase.read` - methods on file-like objects. - - The text contained in *fp* must be formatted as a block of :rfc:`2822` - style headers and header continuation lines, optionally preceded by an - envelope header. The header block is terminated either by the end of the - data or by a blank line. Following the header block is the body of the - message (which may contain MIME-encoded subparts). - - Optional *headersonly* is a flag specifying whether to stop parsing after - reading the headers or not. The default is ``False``, meaning it parses - the entire contents of the file. - - .. method:: parsestr(text, headersonly=False) - - Similar to the :meth:`parse` method, except it takes a string object - instead of a file-like object. Calling this method on a string is exactly - equivalent to wrapping *text* in a :class:`~io.StringIO` instance first and - calling :meth:`parse`. - - Optional *headersonly* is as with the :meth:`parse` method. - - -.. class:: BytesParser(_class=email.message.Message, *, policy=policy.compat32) - - This class is exactly parallel to :class:`Parser`, but handles bytes input. - The *_class* and *strict* arguments are interpreted in the same way as for - the :class:`Parser` constructor. - - If *policy* is specified (it must be an instance of a :mod:`~email.policy` - class) use the rules it specifies to update the representation of the - message. If *policy* is not set, use the :class:`compat32 - ` policy, which maintains backward compatibility with - the Python 3.2 version of the email package. For more information see the - :mod:`~email.policy` documentation. - - .. versionchanged:: 3.3 - Removed the *strict* argument. Added the *policy* keyword. .. method:: parse(fp, headersonly=False) Read all the data from the binary file-like object *fp*, parse the resulting bytes, and return the message object. *fp* must support both the :meth:`~io.IOBase.readline` and the :meth:`~io.IOBase.read` - methods on file-like objects. + methods. - The bytes contained in *fp* must be formatted as a block of :rfc:`2822` + The bytes contained in *fp* must be formatted as a block of :rfc:`5322` + (or, if :attr:`~email.policy.Policy.utf8` is ``True``, :rfc:`6532`) style headers and header continuation lines, optionally preceded by an envelope header. The header block is terminated either by the end of the data or by a blank line. Following the header block is the body of the @@ -202,73 +167,122 @@ reading the headers or not. The default is ``False``, meaning it parses the entire contents of the file. - .. method:: parsebytes(text, headersonly=False) + + .. method:: parsebytes(bytes, headersonly=False) Similar to the :meth:`parse` method, except it takes a :term:`bytes-like - object` instead of a file-like object. Calling this method is equivalent - to wrapping *text* in a :class:`~io.BytesIO` instance first and calling - :meth:`parse`. + object` instead of a file-like object. Calling this method on a + :term:`bytes-like object` is equivalent to wrapping *bytes* in a + :class:`~io.BytesIO` instance first and calling :meth:`parse`. Optional *headersonly* is as with the :meth:`parse` method. .. versionadded:: 3.2 +.. class:: BytesHeaderParser(_class=None, *, policy=policy.compat32) + + Exactly like :class:`BytesParser`, except that *headersonly* + defaults to ``True``. + + .. versionadded:: 3.3 + + +.. class:: Parser(_class=None, *, policy=policy.compat32) + + This class is parallel to :class:`BytesParser`, but handles string input. + + .. versionchanged:: 3.3 + Removed the *strict* argument. Added the *policy* keyword. + + + .. method:: parse(fp, headersonly=False) + + Read all the data from the text-mode file-like object *fp*, parse the + resulting text, and return the root message object. *fp* must support + both the :meth:`~io.TextIOBase.readline` and the + :meth:`~io.TextIOBase.read` methods on file-like objects. + + Other than the text mode requirement, this method operates like + :meth:`BytesParser.parse`. + + + .. method:: parsestr(text, headersonly=False) + + Similar to the :meth:`parse` method, except it takes a string object + instead of a file-like object. Calling this method on a string is + equivalent to wrapping *text* in a :class:`~io.StringIO` instance first + and calling :meth:`parse`. + + Optional *headersonly* is as with the :meth:`parse` method. + + +.. class:: HeaderParser(_class=None, *, policy=policy.compat32) + + Exactly like :class:`Parser`, except that *headersonly* + defaults to ``True``. + + Since creating a message object structure from a string or a file object is such a common task, four functions are provided as a convenience. They are available in the top-level :mod:`email` package namespace. .. currentmodule:: email -.. function:: message_from_string(s, _class=email.message.Message, *, \ + +.. function:: message_from_bytes(s, _class=None, *, \ + policy=policy.compat32) + + Return a message object structure from a :term:`bytes-like object`. This is + equivalent to ``BytesParser().parsebytes(s)``. Optional *_class* and + *strict* are interpreted as with the :class:`~email.parser.BytesParser` class + constructor. + + .. versionadded:: 3.2 + .. versionchanged:: 3.3 + Removed the *strict* argument. Added the *policy* keyword. + + +.. function:: message_from_binary_file(fp, _class=None, *, \ + policy=policy.compat32) + + Return a message object structure tree from an open binary :term:`file + object`. This is equivalent to ``BytesParser().parse(fp)``. *_class* and + *policy* are interpreted as with the :class:`~email.parser.BytesParser` class + constructor. + + .. versionadded:: 3.2 + .. versionchanged:: 3.3 + Removed the *strict* argument. Added the *policy* keyword. + + +.. function:: message_from_string(s, _class=None, *, \ policy=policy.compat32) - Return a message object structure from a string. This is exactly equivalent to + Return a message object structure from a string. This is equivalent to ``Parser().parsestr(s)``. *_class* and *policy* are interpreted as with the :class:`~email.parser.Parser` class constructor. .. versionchanged:: 3.3 Removed the *strict* argument. Added the *policy* keyword. -.. function:: message_from_bytes(s, _class=email.message.Message, *, \ - policy=policy.compat32) - Return a message object structure from a :term:`bytes-like object`. This is exactly - equivalent to ``BytesParser().parsebytes(s)``. Optional *_class* and - *strict* are interpreted as with the :class:`~email.parser.Parser` class - constructor. - - .. versionadded:: 3.2 - .. versionchanged:: 3.3 - Removed the *strict* argument. Added the *policy* keyword. - -.. function:: message_from_file(fp, _class=email.message.Message, *, \ +.. function:: message_from_file(fp, _class=None, *, \ policy=policy.compat32) Return a message object structure tree from an open :term:`file object`. - This is exactly equivalent to ``Parser().parse(fp)``. *_class* - and *policy* are interpreted as with the :class:`~email.parser.Parser` class - constructor. + This is equivalent to ``Parser().parse(fp)``. *_class* and *policy* are + interpreted as with the :class:`~email.parser.Parser` class constructor. .. versionchanged:: 3.3 Removed the *strict* argument. Added the *policy* keyword. -.. function:: message_from_binary_file(fp, _class=email.message.Message, *, \ - policy=policy.compat32) - Return a message object structure tree from an open binary :term:`file - object`. This is exactly equivalent to ``BytesParser().parse(fp)``. - *_class* and *policy* are interpreted as with the - :class:`~email.parser.Parser` class constructor. - - .. versionadded:: 3.2 - .. versionchanged:: 3.3 - Removed the *strict* argument. Added the *policy* keyword. - -Here's an example of how you might use this at an interactive Python prompt:: +Here's an example of how you might use :func:`message_from_bytes` at an +interactive Python prompt:: >>> import email - >>> msg = email.message_from_string(myString) # doctest: +SKIP + >>> msg = email.message_from_bytes(myBytes) # doctest: +SKIP Additional notes @@ -278,35 +292,27 @@ * Most non-\ :mimetype:`multipart` type messages are parsed as a single message object with a string payload. These objects will return ``False`` for - :meth:`~email.message.Message.is_multipart`. Their - :meth:`~email.message.Message.get_payload` method will return a string object. + :meth:`~email.message.EmailMessage.is_multipart`, and + :meth:`~email.message.EmailMessage.iter_parts` will yield an empty list. * All :mimetype:`multipart` type messages will be parsed as a container message object with a list of sub-message objects for their payload. The outer container message will return ``True`` for - :meth:`~email.message.Message.is_multipart` and their - :meth:`~email.message.Message.get_payload` method will return the list of - :class:`~email.message.Message` subparts. + :meth:`~email.message.EmailMessage.is_multipart`, and + :meth:`~email.message.EmailMessage.iter_parts` will yield a list of subparts. -* Most messages with a content type of :mimetype:`message/\*` (e.g. - :mimetype:`message/delivery-status` and :mimetype:`message/rfc822`) will also be - parsed as container object containing a list payload of length 1. Their - :meth:`~email.message.Message.is_multipart` method will return ``True``. - The single element in the list payload will be a sub-message object. +* Most messages with a content type of :mimetype:`message/\*` (such as + :mimetype:`message/delivery-status` and :mimetype:`message/rfc822`) will also + be parsed as container object containing a list payload of length 1. Their + :meth:`~email.message.EmailMessage.is_multipart` method will return ``True``. + The single element yielded by :meth:`~email.message.EmailMessage.iter_parts` + will be a sub-message object. -* Some non-standards compliant messages may not be internally consistent about +* Some non-standards-compliant messages may not be internally consistent about their :mimetype:`multipart`\ -edness. Such messages may have a :mailheader:`Content-Type` header of type :mimetype:`multipart`, but their - :meth:`~email.message.Message.is_multipart` method may return ``False``. + :meth:`~email.message.EmailMessage.is_multipart` method may return ``False``. If such messages were parsed with the :class:`~email.parser.FeedParser`, they will have an instance of the :class:`~email.errors.MultipartInvariantViolationDefect` class in their *defects* attribute list. See :mod:`email.errors` for details. - -.. rubric:: Footnotes - -.. [#] As of email package version 3.0, introduced in Python 2.4, the classic - :class:`~email.parser.Parser` was re-implemented in terms of the - :class:`~email.parser.FeedParser`, so the semantics and results are - identical between the two parsers. - diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -18,9 +18,12 @@ email messages (a block of header fields each consisting of a name followed by a colon followed by a value, the whole block followed by a blank line and an arbitrary 'body'), is a format that has found utility outside of the realm of -email. Some of these uses conform fairly closely to the main RFCs, some do -not. And even when working with email, there are times when it is desirable to -break strict compliance with the RFCs. +email. Some of these uses conform fairly closely to the main email RFCs, some +do not. Even when working with email, there are times when it is desirable to +break strict compliance with the RFCs, such as generating emails that +interoperate with email servers that do not themselves follow the standards, or +that implement extensions you want to use in ways that violate the +standards. Policy objects give the email package the flexibility to handle all these disparate use cases. @@ -31,27 +34,40 @@ email package to alter the default behavior. The settable values and their defaults are described below. -There is a default policy used by all classes in the email package. This -policy is named :class:`Compat32`, with a corresponding pre-defined instance -named :const:`compat32`. It provides for complete backward compatibility (in -some cases, including bug compatibility) with the pre-Python3.3 version of the -email package. +There is a default policy used by all classes in the email package. For all of +the :mod:`~email.parser` classes and the related convenience functions, and for +the :class:`~email.message.Message` class, this is the :class:`Compat32` +policy, via its corresponding pre-defined instance :const:`compat32`. This +policy provides for complete backward compatibility (in some cases, including +bug compatibility) with the pre-Python3.3 version of the email package. + +This default value for the *policy* keyword to +:class:`~email.message.EmailMessage` is the :class:`EmailPolicy` policy, via +its pre-defined instance :data:`~default`. + +When a :class:`~email.message.Message` or :class:`~email.message.EmailMessage` +object is created, it acquires a policy. If the message is created by a +:mod:`~email.parser`, a policy passed to the parser will be the policy used by +the message it creates. If the message is created by the program, then the +policy can be specified when it is created. When a message is passed to a +:mod:`~email.generator`, the generator uses the policy from the message by +default, but you can also pass a specific policy to the generator that will +override the one stored on the message object. + +The default value for the *policy* keyword for the :mod:`email.parser` classes +and the parser convenience functions **will be changing** in a future version of +Python. Therefore you should **always specify explicitly which policy you want +to use** when calling any of the classes and functions described in the +:mod:`~email.parser` module. The first part of this documentation covers the features of :class:`Policy`, an -:term:`abstract base class` that defines the features that are common to all +:term:`abstract base class` that defines the features that are common to all policy objects, including :const:`compat32`. This includes certain hook methods that are called internally by the email package, which a custom policy -could override to obtain different behavior. - -When a :class:`~email.message.Message` object is created, it acquires a policy. -By default this will be :const:`compat32`, but a different policy can be -specified. If the ``Message`` is created by a :mod:`~email.parser`, a policy -passed to the parser will be the policy used by the ``Message`` it creates. If -the ``Message`` is created by the program, then the policy can be specified -when it is created. When a ``Message`` is passed to a :mod:`~email.generator`, -the generator uses the policy from the ``Message`` by default, but you can also -pass a specific policy to the generator that will override the one stored on -the ``Message`` object. +could override to obtain different behavior. The second part describes the +concrete classes :class:`EmailPolicy` and :class:`Compat32`, which implement +the hooks that provide the standard behavior and the backward compatible +behavior and features, respectively. :class:`Policy` instances are immutable, but they can be cloned, accepting the same keyword arguments as the class constructor and returning a new @@ -147,6 +163,7 @@ This class defines the following properties, and thus values for the following may be passed in the constructor of any policy class: + .. attribute:: max_line_length The maximum length of any line in the serialized output, not counting the @@ -154,12 +171,14 @@ ``0`` or :const:`None` indicates that no line wrapping should be done at all. + .. attribute:: linesep The string to be used to terminate lines in serialized output. The default is ``\n`` because that's the internal end-of-line discipline used by Python, though ``\r\n`` is required by the RFCs. + .. attribute:: cte_type Controls the type of Content Transfer Encodings that may be or are @@ -174,8 +193,8 @@ ``8bit`` data is not constrained to be 7 bit clean. Data in headers is still required to be ASCII-only and so will be encoded (see - 'binary_fold' below for an exception), but body parts may use - the ``8bit`` CTE. + :meth:`fold_binary` and :attr:`~EmailPolicy.utf8` below for + exceptions), but body parts may use the ``8bit`` CTE. ======== =============================================================== A ``cte_type`` value of ``8bit`` only works with ``BytesGenerator``, not @@ -183,6 +202,7 @@ ``Generator`` is operating under a policy that specifies ``cte_type=8bit``, it will act as if ``cte_type`` is ``7bit``. + .. attribute:: raise_on_defect If :const:`True`, any defects encountered will be raised as errors. If @@ -190,7 +210,6 @@ :meth:`register_defect` method. - .. attribute:: mangle_from\_ If :const:`True`, lines starting with *"From "* in the body are @@ -201,19 +220,23 @@ .. versionadded:: 3.5 The *mangle_from_* parameter. + The following :class:`Policy` method is intended to be called by code using the email library to create policy instances with custom settings: + .. method:: clone(**kw) Return a new :class:`Policy` instance whose attributes have the same values as the current instance, except where those attributes are given new values by the keyword arguments. + The remaining :class:`Policy` methods are called by the email package code, and are not intended to be called by an application using the email package. A custom policy must implement all of these methods. + .. method:: handle_defect(obj, defect) Handle a *defect* found on *obj*. When the email package calls this @@ -224,6 +247,7 @@ it is ``True``, *defect* is raised as an exception. If it is ``False`` (the default), *obj* and *defect* are passed to :meth:`register_defect`. + .. method:: register_defect(obj, defect) Register a *defect* on *obj*. In the email package, *defect* will always @@ -236,14 +260,16 @@ custom ``Message`` objects) should also provide such an attribute, otherwise defects in parsed messages will raise unexpected errors. + .. method:: header_max_count(name) Return the maximum allowed number of headers named *name*. - Called when a header is added to a :class:`~email.message.Message` - object. If the returned value is not ``0`` or ``None``, and there are - already a number of headers with the name *name* equal to the value - returned, a :exc:`ValueError` is raised. + Called when a header is added to an :class:`~email.message.EmailMessage` + or :class:`~email.message.Message` object. If the returned value is not + ``0`` or ``None``, and there are already a number of headers with the + name *name* greather than or equal to the value returned, a + :exc:`ValueError` is raised. Because the default behavior of ``Message.__setitem__`` is to append the value to the list of headers, it is easy to create duplicate headers @@ -255,6 +281,7 @@ The default implementation returns ``None`` for all header names. + .. method:: header_source_parse(sourcelines) The email package calls this method with a list of strings, each string @@ -274,6 +301,7 @@ There is no default implementation + .. method:: header_store_parse(name, value) The email package calls this method with the name and value provided by @@ -289,6 +317,7 @@ There is no default implementation + .. method:: header_fetch_parse(name, value) The email package calls this method with the *name* and *value* currently @@ -304,6 +333,7 @@ There is no default implementation + .. method:: fold(name, value) The email package calls this method with the *name* and *value* currently @@ -316,6 +346,7 @@ *value* may contain surrogateescaped binary data. There should be no surrogateescaped binary data in the string returned by the method. + .. method:: fold_binary(name, value) The same as :meth:`fold`, except that the returned value should be a @@ -325,73 +356,6 @@ converted back into binary data in the returned bytes object. -.. class:: Compat32(**kw) - - This concrete :class:`Policy` is the backward compatibility policy. It - replicates the behavior of the email package in Python 3.2. The - :mod:`~email.policy` module also defines an instance of this class, - :const:`compat32`, that is used as the default policy. Thus the default - behavior of the email package is to maintain compatibility with Python 3.2. - - The following attributes have values that are different from the - :class:`Policy` default: - - .. attribute:: mangle_from_ - - The default is ``True``. - - The class provides the following concrete implementations of the - abstract methods of :class:`Policy`: - - .. method:: header_source_parse(sourcelines) - - The name is parsed as everything up to the '``:``' and returned - unmodified. The value is determined by stripping leading whitespace off - the remainder of the first line, joining all subsequent lines together, - and stripping any trailing carriage return or linefeed characters. - - .. method:: header_store_parse(name, value) - - The name and value are returned unmodified. - - .. method:: header_fetch_parse(name, value) - - If the value contains binary data, it is converted into a - :class:`~email.header.Header` object using the ``unknown-8bit`` charset. - Otherwise it is returned unmodified. - - .. method:: fold(name, value) - - Headers are folded using the :class:`~email.header.Header` folding - algorithm, which preserves existing line breaks in the value, and wraps - each resulting line to the ``max_line_length``. Non-ASCII binary data are - CTE encoded using the ``unknown-8bit`` charset. - - .. method:: fold_binary(name, value) - - Headers are folded using the :class:`~email.header.Header` folding - algorithm, which preserves existing line breaks in the value, and wraps - each resulting line to the ``max_line_length``. If ``cte_type`` is - ``7bit``, non-ascii binary data is CTE encoded using the ``unknown-8bit`` - charset. Otherwise the original source header is used, with its existing - line breaks and any (RFC invalid) binary data it may contain. - - -An instance of :class:`Compat32` is provided as a module constant: - -.. data:: compat32 - - An instance of :class:`Compat32`, providing backward compatibility with the - behavior of the email package in Python 3.2. - - -.. note:: - - The documentation below describes new policies that are included in the - standard library on a :term:`provisional basis `. - Backwards incompatible changes (up to and including removal of the feature) - may occur if deemed necessary by the core developers. - .. class:: EmailPolicy(**kw) @@ -407,6 +371,11 @@ In addition to the settable attributes listed above that apply to all policies, this policy adds the following additional attributes: + .. versionadded:: 3.3 as a :term:`provisional feature `. + + .. versionchanged:: 3.6 provisional status removed. + + .. attribute:: utf8 If ``False``, follow :rfc:`5322`, supporting non-ASCII characters in @@ -415,13 +384,14 @@ formatted in this way may be passed to SMTP servers that support the ``SMTPUTF8`` extension (:rfc:`6531`). + .. attribute:: refold_source If the value for a header in the ``Message`` object originated from a :mod:`~email.parser` (as opposed to being set by a program), this attribute indicates whether or not a generator should refold that value - when transforming the message back into stream form. The possible values - are: + when transforming the message back into serialized form. The possible + values are: ======== =============================================================== ``none`` all source values use original folding @@ -434,23 +404,24 @@ The default is ``long``. + .. attribute:: header_factory A callable that takes two arguments, ``name`` and ``value``, where ``name`` is a header field name and ``value`` is an unfolded header field value, and returns a string subclass that represents that header. A default ``header_factory`` (see :mod:`~email.headerregistry`) is provided - that understands some of the :RFC:`5322` header field types. (Currently - address fields and date fields have special treatment, while all other - fields are treated as unstructured. This list will be completed before - the extension is marked stable.) + that supports custom parsing for the various address and date :RFC:`5322` + header field types, and the major MIME header field stypes. Support for + additional custom parsing will be added in the future. + .. attribute:: content_manager An object with at least two methods: get_content and set_content. When - the :meth:`~email.message.Message.get_content` or - :meth:`~email.message.Message.set_content` method of a - :class:`~email.message.Message` object is called, it calls the + the :meth:`~email.message.EmailMessage.get_content` or + :meth:`~email.message.EmailMessage.set_content` method of an + :class:`~email.message.EmailMessage` object is called, it calls the corresponding method of this object, passing it the message object as its first argument, and any arguments or keywords that were passed to it as additional arguments. By default ``content_manager`` is set to @@ -462,16 +433,22 @@ The class provides the following concrete implementations of the abstract methods of :class:`Policy`: + .. method:: header_max_count(name) Returns the value of the :attr:`~email.headerregistry.BaseHeader.max_count` attribute of the specialized class used to represent the header with the given name. + .. method:: header_source_parse(sourcelines) - The implementation of this method is the same as that for the - :class:`Compat32` policy. + + The name is parsed as everything up to the '``:``' and returned + unmodified. The value is determined by stripping leading whitespace off + the remainder of the first line, joining all subsequent lines together, + and stripping any trailing carriage return or linefeed characters. + .. method:: header_store_parse(name, value) @@ -482,6 +459,7 @@ the value. In this case a ``ValueError`` is raised if the input value contains CR or LF characters. + .. method:: header_fetch_parse(name, value) If the value has a ``name`` attribute, it is returned to unmodified. @@ -490,6 +468,7 @@ header object is returned. Any surrogateescaped bytes get turned into the unicode unknown-character glyph. + .. method:: fold(name, value) Header folding is controlled by the :attr:`refold_source` policy setting. @@ -508,6 +487,7 @@ regardless of the ``refold_source`` setting, which causes the binary data to be CTE encoded using the ``unknown-8bit`` charset. + .. method:: fold_binary(name, value) The same as :meth:`fold` if :attr:`~Policy.cte_type` is ``7bit``, except @@ -519,23 +499,27 @@ ``refold_header`` setting, since there is no way to know whether the binary data consists of single byte characters or multibyte characters. + The following instances of :class:`EmailPolicy` provide defaults suitable for specific application domains. Note that in the future the behavior of these instances (in particular the ``HTTP`` instance) may be adjusted to conform even more closely to the RFCs relevant to their domains. + .. data:: default An instance of ``EmailPolicy`` with all defaults unchanged. This policy uses the standard Python ``\n`` line endings rather than the RFC-correct ``\r\n``. + .. data:: SMTP Suitable for serializing messages in conformance with the email RFCs. Like ``default``, but with ``linesep`` set to ``\r\n``, which is RFC compliant. + .. data:: SMTPUTF8 The same as ``SMTP`` except that :attr:`~EmailPolicy.utf8` is ``True``. @@ -544,11 +528,13 @@ sender or recipient addresses have non-ASCII characters (the :meth:`smtplib.SMTP.send_message` method handles this automatically). + .. data:: HTTP Suitable for serializing headers with for use in HTTP traffic. Like ``SMTP`` except that ``max_line_length`` is set to ``None`` (unlimited). + .. data:: strict Convenience instance. The same as ``default`` except that @@ -557,6 +543,7 @@ somepolicy + policy.strict + With all of these :class:`EmailPolicies <.EmailPolicy>`, the effective API of the email package is changed from the Python 3.2 API in the following ways: @@ -573,7 +560,7 @@ and allowed. From the application view, this means that any header obtained through the -:class:`~email.message.Message` is a header object with extra +:class:`~email.message.EmailMessage` is a header object with extra attributes, whose string value is the fully decoded unicode value of the header. Likewise, a header may be assigned a new value, or a new header created, using a unicode string, and the policy will take care of converting @@ -581,3 +568,69 @@ The header objects and their attributes are described in :mod:`~email.headerregistry`. + + + +.. class:: Compat32(**kw) + + This concrete :class:`Policy` is the backward compatibility policy. It + replicates the behavior of the email package in Python 3.2. The + :mod:`~email.policy` module also defines an instance of this class, + :const:`compat32`, that is used as the default policy. Thus the default + behavior of the email package is to maintain compatibility with Python 3.2. + + The following attributes have values that are different from the + :class:`Policy` default: + + + .. attribute:: mangle_from_ + + The default is ``True``. + + + The class provides the following concrete implementations of the + abstract methods of :class:`Policy`: + + + .. method:: header_source_parse(sourcelines) + + The name is parsed as everything up to the '``:``' and returned + unmodified. The value is determined by stripping leading whitespace off + the remainder of the first line, joining all subsequent lines together, + and stripping any trailing carriage return or linefeed characters. + + + .. method:: header_store_parse(name, value) + + The name and value are returned unmodified. + + + .. method:: header_fetch_parse(name, value) + + If the value contains binary data, it is converted into a + :class:`~email.header.Header` object using the ``unknown-8bit`` charset. + Otherwise it is returned unmodified. + + + .. method:: fold(name, value) + + Headers are folded using the :class:`~email.header.Header` folding + algorithm, which preserves existing line breaks in the value, and wraps + each resulting line to the ``max_line_length``. Non-ASCII binary data are + CTE encoded using the ``unknown-8bit`` charset. + + + .. method:: fold_binary(name, value) + + Headers are folded using the :class:`~email.header.Header` folding + algorithm, which preserves existing line breaks in the value, and wraps + each resulting line to the ``max_line_length``. If ``cte_type`` is + ``7bit``, non-ascii binary data is CTE encoded using the ``unknown-8bit`` + charset. Otherwise the original source header is used, with its existing + line breaks and any (RFC invalid) binary data it may contain. + + +.. data:: compat32 + + An instance of :class:`Compat32`, providing backward compatibility with the + behavior of the email package in Python 3.2. diff --git a/Doc/library/email.rst b/Doc/library/email.rst --- a/Doc/library/email.rst +++ b/Doc/library/email.rst @@ -3,50 +3,99 @@ .. module:: email :synopsis: Package supporting the parsing, manipulating, and generating - email messages, including MIME documents. - -.. moduleauthor:: Barry A. Warsaw -.. sectionauthor:: Barry A. Warsaw -.. Copyright (C) 2001-2010 Python Software Foundation + email messages. +.. moduleauthor:: Barry A. Warsaw , + R. David Murray +.. sectionauthor:: R. David Murray **Source code:** :source:`Lib/email/__init__.py` -------------- -The :mod:`email` package is a library for managing email messages, including -MIME and other :rfc:`2822`\ -based message documents. It is specifically *not* -designed to do any sending of email messages to SMTP (:rfc:`2821`), NNTP, or -other servers; those are functions of modules such as :mod:`smtplib` and -:mod:`nntplib`. The :mod:`email` package attempts to be as RFC-compliant as -possible, supporting in addition to :rfc:`2822`, such MIME-related RFCs as -:rfc:`2045`, :rfc:`2046`, :rfc:`2047`, and :rfc:`2231`. +The :mod:`email` package is a library for managing email messages. It is +specifically *not* designed to do any sending of email messages to SMTP +(:rfc:`2821`), NNTP, or other servers; those are functions of modules such as +:mod:`smtplib` and :mod:`nntplib`. The :mod:`email` package attempts to be as +RFC-compliant as possible, supporting :rfc:`5233` and :rfc:`6532`, as well as +such MIME-related RFCs as :rfc:`2045`, :rfc:`2046`, :rfc:`2047`, :rfc:`2183`, +and :rfc:`2231`. -The primary distinguishing feature of the :mod:`email` package is that it splits -the parsing and generating of email messages from the internal *object model* -representation of email. Applications using the :mod:`email` package deal -primarily with objects; you can add sub-objects to messages, remove sub-objects -from messages, completely re-arrange the contents, etc. There is a separate -parser and a separate generator which handles the transformation from flat text -to the object model, and then back to flat text again. There are also handy -subclasses for some common MIME object types, and a few miscellaneous utilities -that help with such common tasks as extracting and parsing message field values, -creating RFC-compliant dates, etc. +The overall structure of the email package can be divided into three major +components, plus a fourth component that controls the behavior of the other +components. + +The central component of the package is an "object model" that represents email +messages. An application interacts with the package primarily through the +object model interface defined in the :mod:`~email.message` sub-module. The +application can use this API to ask questions about an existing email, to +construct a new email, or to add or remove email subcomponents that themselves +use the same object model interface. That is, following the nature of email +messages and their MIME subcomponents, the email object model is a tree +structure of objects that all provide the :class:`~email.message.EmailMessage` +API. + +The other two major components of the package are the :mod:`~email.parser` and +the :mod:`~email.generator`. The parser takes the serialized version of an +email message (a stream of bytes) and converts it into a tree of +:class:`~email.message.EmailMessage` objects. The generator takes an +:class:`~email.message.EmailMessage` and turns it back into a serialized byte +stream. (The parser and generator also handle streams of text characters, but +this usage is discouraged as it is too easy to end up with messages that are +not valid in one way or another.) + +The control component is the :mod:`~email.policy` module. Every +:class:`~email.message.EmailMessage`, every :mod:`~email.generator`, and every +:mod:`~email.parser` has an associated :mod:`~email.policy` object that +controls its behavior. Usually an application only needs to specify the policy +when an :class:`~email.message.EmailMessage` is created, either by directly +instantiating an :class:`~email.message.EmailMessage` to create a new email, +or by parsing an input stream using a :mod:`~email.parser`. But the policy can +be changed when the message is serialized using a :mod:`~email.generator`. +This allows, for example, a generic email message to be parsed from disk, but +to serialize it using standard SMTP settings when sending it to an email +server. + +The email package does its best to hide the details of the various governing +RFCs from the application. Conceptually the application should be able to +treat the email message as a structured tree of unicode text and binary +attachments, without having to worry about how these are represented when +serialized. In practice, however, it is often necessary to be aware of at +least some of the rules governing MIME messages and their structure, +specifically the names and nature of the MIME "content types" and how they +identify multipart documents. For the most part this knowledge should only be +required for more complex applications, and even then it should only be the +high level structure in question, and not the details of how those structures +are represented. Since MIME content types are used widely in modern internet +software (not just email), this will be a familiar concept to many programmers. The following sections describe the functionality of the :mod:`email` package. -The ordering follows a progression that should be common in applications: an -email message is read as flat text from a file or other source, the text is -parsed to produce the object structure of the email message, this structure is -manipulated, and finally, the object tree is rendered back into flat text. +We start with the :mod:`~email.message` object model, which is the primary +interface an application will use, and follow that with the +:mod:`~email.parser` and :mod:`~email.generator` components. Then we cover the +:mod:`~email.policy` controls, which completes the treatment of the main +components of the library. -It is perfectly feasible to create the object structure out of whole cloth --- -i.e. completely from scratch. From there, a similar progression can be taken as -above. +The next three sections cover the exceptions the package may raise and the +defects (non-compliance with the RFCs) that the :mod:`~email.parser` may +detect. Then we cover the :mod:`~email.headerregistry` and the +:mod:`~email.contentmanager` sub-components, which provide tools for doing more +detailed manipulation of headers and payloads, respectively. Both of these +components contain features relevant to consuming and producing non-trivial +messages, but also document their extensibility APIs, which will be of interest +to advanced applications. -Also included are detailed specifications of all the classes and modules that -the :mod:`email` package provides, the exception classes you might encounter -while using the :mod:`email` package, some auxiliary utilities, and a few -examples. For users of the older :mod:`mimelib` package, or previous versions -of the :mod:`email` package, a section on differences and porting is provided. +Following those is a set of examples of using the fundamental parts of the APIs +covered in the preceding sections. + +The forgoing represent the modern (unicode friendly) API of the email package. +The remaining sections, starting with the :class:`~email.message.Message` +class, cover the legacy :data:`~email.policy.compat32` API that deals much more +directly with the details of how email messages are represented. The +:data:`~email.policy.compat32` API does *not* hide the details of the RFCs from +the application, but for applications that need to operate at that level, they +can be useful tools. This documentation is also relevant for applications that +are still using the :mod:`~email.policy.compat32` API for backward +compatibility reasons. Contents of the :mod:`email` package documentation: @@ -56,335 +105,39 @@ email.parser.rst email.generator.rst email.policy.rst + + email.errors.rst email.headerregistry.rst email.contentmanager.rst + + email.examples.rst + + email.compat32-message.rst email.mime.rst email.header.rst email.charset.rst email.encoders.rst - email.errors.rst email.util.rst email.iterators.rst - email-examples.rst .. seealso:: Module :mod:`smtplib` - SMTP protocol client + SMTP (Simple Mail Transport Protcol) client + + Module :mod:`poplib` + POP (Post Office Protocol) client + + Module :mod:`imaplib` + IMAP (Internet Message Access Protocol) client Module :mod:`nntplib` - NNTP protocol client + NNTP (Net News Transport Protocol) client + Module :mod:`mailbox` + Tools for creating, reading, and managing collections of messages on disk + using a variety standard formats. -.. _email-pkg-history: - -Package History ---------------- - -This table describes the release history of the email package, corresponding to -the version of Python that the package was released with. For purposes of this -document, when you see a note about change or added versions, these refer to the -Python version the change was made in, *not* the email package version. This -table also describes the Python compatibility of each version of the package. - -+---------------+------------------------------+-----------------------+ -| email version | distributed with | compatible with | -+===============+==============================+=======================+ -| :const:`1.x` | Python 2.2.0 to Python 2.2.1 | *no longer supported* | -+---------------+------------------------------+-----------------------+ -| :const:`2.5` | Python 2.2.2+ and Python 2.3 | Python 2.1 to 2.5 | -+---------------+------------------------------+-----------------------+ -| :const:`3.0` | Python 2.4 and Python 2.5 | Python 2.3 to 2.6 | -+---------------+------------------------------+-----------------------+ -| :const:`4.0` | Python 2.5 to Python 2.7 | Python 2.3 to 2.7 | -+---------------+------------------------------+-----------------------+ -| :const:`5.0` | Python 3.0 and Python 3.1 | Python 3.0 to 3.2 | -+---------------+------------------------------+-----------------------+ -| :const:`5.1` | Python 3.2 | Python 3.2 | -+---------------+------------------------------+-----------------------+ - -After Version 5.1 (Python 3.2), the email package no longer has a version that -is separate from the Python version. (See the :ref:`whatsnew-index` documents -for the respective Python versions for details on changes.) - -Here are the major differences between :mod:`email` version 5.1 and -version 5.0: - -* It is once again possible to parse messages containing non-ASCII bytes, - and to reproduce such messages if the data containing the non-ASCII - bytes is not modified. - -* New functions :func:`message_from_bytes` and :func:`message_from_binary_file`, - and new classes :class:`~email.parser.BytesFeedParser` and - :class:`~email.parser.BytesParser` allow binary message data to be parsed - into model objects. - -* Given bytes input to the model, :meth:`~email.message.Message.get_payload` - will by default decode a message body that has a - :mailheader:`Content-Transfer-Encoding` of ``8bit`` using the charset - specified in the MIME headers and return the resulting string. - -* Given bytes input to the model, :class:`~email.generator.Generator` will - convert message bodies that have a :mailheader:`Content-Transfer-Encoding` of - 8bit to instead have a 7bit Content-Transfer-Encoding. - -* New class :class:`~email.generator.BytesGenerator` produces bytes - as output, preserving any unchanged non-ASCII data that was - present in the input used to build the model, including message bodies - with a :mailheader:`Content-Transfer-Encoding` of 8bit. - -Here are the major differences between :mod:`email` version 5.0 and version 4: - -* All operations are on unicode strings. Text inputs must be strings, - text outputs are strings. Outputs are limited to the ASCII character - set and so can be encoded to ASCII for transmission. Inputs are also - limited to ASCII; this is an acknowledged limitation of email 5.0 and - means it can only be used to parse email that is 7bit clean. - -Here are the major differences between :mod:`email` version 4 and version 3: - -* All modules have been renamed according to :pep:`8` standards. For example, - the version 3 module :mod:`email.Message` was renamed to :mod:`email.message` in - version 4. - -* A new subpackage :mod:`email.mime` was added and all the version 3 - :mod:`email.MIME\*` modules were renamed and situated into the :mod:`email.mime` - subpackage. For example, the version 3 module :mod:`email.MIMEText` was renamed - to :mod:`email.mime.text`. - - *Note that the version 3 names will continue to work until Python 2.6*. - -* The :mod:`email.mime.application` module was added, which contains the - :class:`~email.mime.application.MIMEApplication` class. - -* Methods that were deprecated in version 3 have been removed. These include - :meth:`Generator.__call__`, :meth:`Message.get_type`, - :meth:`Message.get_main_type`, :meth:`Message.get_subtype`. - -* Fixes have been added for :rfc:`2231` support which can change some of the - return types for :func:`Message.get_param ` - and friends. Under some - circumstances, values which used to return a 3-tuple now return simple strings - (specifically, if all extended parameter segments were unencoded, there is no - language and charset designation expected, so the return type is now a simple - string). Also, %-decoding used to be done for both encoded and unencoded - segments; this decoding is now done only for encoded segments. - -Here are the major differences between :mod:`email` version 3 and version 2: - -* The :class:`~email.parser.FeedParser` class was introduced, and the - :class:`~email.parser.Parser` class was implemented in terms of the - :class:`~email.parser.FeedParser`. All parsing therefore is - non-strict, and parsing will make a best effort never to raise an exception. - Problems found while parsing messages are stored in the message's *defect* - attribute. - -* All aspects of the API which raised :exc:`DeprecationWarning`\ s in version 2 - have been removed. These include the *_encoder* argument to the - :class:`~email.mime.text.MIMEText` constructor, the - :meth:`Message.add_payload` method, the :func:`Utils.dump_address_pair` - function, and the functions :func:`Utils.decode` and :func:`Utils.encode`. - -* New :exc:`DeprecationWarning`\ s have been added to: - :meth:`Generator.__call__`, :meth:`Message.get_type`, - :meth:`Message.get_main_type`, :meth:`Message.get_subtype`, and the *strict* - argument to the :class:`~email.parser.Parser` class. These are expected to - be removed in future versions. - -* Support for Pythons earlier than 2.3 has been removed. - -Here are the differences between :mod:`email` version 2 and version 1: - -* The :mod:`email.Header` and :mod:`email.Charset` modules have been added. - -* The pickle format for :class:`~email.message.Message` instances has changed. - Since this was never (and still isn't) formally defined, this isn't - considered a backward incompatibility. However if your application pickles - and unpickles :class:`~email.message.Message` instances, be aware that in - :mod:`email` version 2, :class:`~email.message.Message` instances now have - private variables *_charset* and *_default_type*. - -* Several methods in the :class:`~email.message.Message` class have been - deprecated, or their signatures changed. Also, many new methods have been - added. See the documentation for the :class:`~email.message.Message` class - for details. The changes should be completely backward compatible. - -* The object structure has changed in the face of :mimetype:`message/rfc822` - content types. In :mod:`email` version 1, such a type would be represented - by a scalar payload, i.e. the container message's - :meth:`~email.message.Message.is_multipart` returned false, - :meth:`~email.message.Message.get_payload` was not a list object, but a - single :class:`~email.message.Message` instance. - - This structure was inconsistent with the rest of the package, so the object - representation for :mimetype:`message/rfc822` content types was changed. In - :mod:`email` version 2, the container *does* return ``True`` from - :meth:`~email.message.Message.is_multipart`, and - :meth:`~email.message.Message.get_payload` returns a list containing a single - :class:`~email.message.Message` item. - - Note that this is one place that backward compatibility could not be - completely maintained. However, if you're already testing the return type of - :meth:`~email.message.Message.get_payload`, you should be fine. You just need - to make sure your code doesn't do a :meth:`~email.message.Message.set_payload` - with a :class:`~email.message.Message` instance on a container with a content - type of :mimetype:`message/rfc822`. - -* The :class:`~email.parser.Parser` constructor's *strict* argument was added, - and its :meth:`~email.parser.Parser.parse` and - :meth:`~email.parser.Parser.parsestr` methods grew a *headersonly* argument. - The *strict* flag was also added to functions :func:`email.message_from_file` - and :func:`email.message_from_string`. - -* :meth:`Generator.__call__` is deprecated; use :meth:`Generator.flatten - ` instead. The - :class:`~email.generator.Generator` class has also grown the - :meth:`~email.generator.Generator.clone` method. - -* The :class:`~email.generator.DecodedGenerator` class in the - :mod:`email.generator` module was added. - -* The intermediate base classes - :class:`~email.mime.nonmultipart.MIMENonMultipart` and - :class:`~email.mime.multipart.MIMEMultipart` have been added, and interposed - in the class hierarchy for most of the other MIME-related derived classes. - -* The *_encoder* argument to the :class:`~email.mime.text.MIMEText` constructor - has been deprecated. Encoding now happens implicitly based on the - *_charset* argument. - -* The following functions in the :mod:`email.Utils` module have been deprecated: - :func:`dump_address_pairs`, :func:`decode`, and :func:`encode`. The following - functions have been added to the module: :func:`make_msgid`, - :func:`decode_rfc2231`, :func:`encode_rfc2231`, and :func:`decode_params`. - -* The non-public function :func:`email.Iterators._structure` was added. - - -Differences from :mod:`mimelib` -------------------------------- - -The :mod:`email` package was originally prototyped as a separate library called -`mimelib `_. Changes have been made so that method names -are more consistent, and some methods or modules have either been added or -removed. The semantics of some of the methods have also changed. For the most -part, any functionality available in :mod:`mimelib` is still available in the -:mod:`email` package, albeit often in a different way. Backward compatibility -between the :mod:`mimelib` package and the :mod:`email` package was not a -priority. - -Here is a brief description of the differences between the :mod:`mimelib` and -the :mod:`email` packages, along with hints on how to port your applications. - -Of course, the most visible difference between the two packages is that the -package name has been changed to :mod:`email`. In addition, the top-level -package has the following differences: - -* :func:`messageFromString` has been renamed to :func:`message_from_string`. - -* :func:`messageFromFile` has been renamed to :func:`message_from_file`. - -The :class:`~email.message.Message` class has the following differences: - -* The method :meth:`asString` was renamed to - :meth:`~email.message.Message.as_string`. - -* The method :meth:`ismultipart` was renamed to - :meth:`~email.message.Message.is_multipart`. - -* The :meth:`~email.message.Message.get_payload` method has grown a *decode* - optional argument. - -* The method :meth:`getall` was renamed to - :meth:`~email.message.Message.get_all`. - -* The method :meth:`addheader` was renamed to - :meth:`~email.message.Message.add_header`. - -* The method :meth:`gettype` was renamed to :meth:`get_type`. - -* The method :meth:`getmaintype` was renamed to :meth:`get_main_type`. - -* The method :meth:`getsubtype` was renamed to :meth:`get_subtype`. - -* The method :meth:`getparams` was renamed to - :meth:`~email.message.Message.get_params`. Also, whereas :meth:`getparams` - returned a list of strings, :meth:`~email.message.Message.get_params` returns - a list of 2-tuples, effectively the key/value pairs of the parameters, split - on the ``'='`` sign. - -* The method :meth:`getparam` was renamed to - :meth:`~email.message.Message.get_param`. - -* The method :meth:`getcharsets` was renamed to - :meth:`~email.message.Message.get_charsets`. - -* The method :meth:`getfilename` was renamed to - :meth:`~email.message.Message.get_filename`. - -* The method :meth:`getboundary` was renamed to - :meth:`~email.message.Message.get_boundary`. - -* The method :meth:`setboundary` was renamed to - :meth:`~email.message.Message.set_boundary`. - -* The method :meth:`getdecodedpayload` was removed. To get similar - functionality, pass the value 1 to the *decode* flag of the - :meth:`~email.message.Message.get_payload` method. - -* The method :meth:`getpayloadastext` was removed. Similar functionality is - supported by the :class:`~email.generator.DecodedGenerator` class in the - :mod:`email.generator` module. - -* The method :meth:`getbodyastext` was removed. You can get similar - functionality by creating an iterator with - :func:`~email.iterators.typed_subpart_iterator` in the :mod:`email.iterators` - module. - -The :class:`~email.parser.Parser` class has no differences in its public -interface. It does have some additional smarts to recognize -:mimetype:`message/delivery-status` type messages, which it represents as a -:class:`~email.message.Message` instance containing separate -:class:`~email.message.Message` subparts for each header block in the delivery -status notification [#]_. - -The :class:`~email.generator.Generator` class has no differences in its public -interface. There is a new class in the :mod:`email.generator` module though, -called :class:`~email.generator.DecodedGenerator` which provides most of the -functionality previously available in the :meth:`Message.getpayloadastext` -method. - -The following modules and classes have been changed: - -* The :class:`~email.mime.base.MIMEBase` class constructor arguments *_major* - and *_minor* have changed to *_maintype* and *_subtype* respectively. - -* The ``Image`` class/module has been renamed to ``MIMEImage``. The *_minor* - argument has been renamed to *_subtype*. - -* The ``Text`` class/module has been renamed to ``MIMEText``. The *_minor* - argument has been renamed to *_subtype*. - -* The ``MessageRFC822`` class/module has been renamed to ``MIMEMessage``. Note - that an earlier version of :mod:`mimelib` called this class/module ``RFC822``, - but that clashed with the Python standard library module :mod:`rfc822` on some - case-insensitive file systems. - - Also, the :class:`~email.mime.message.MIMEMessage` class now represents any - kind of MIME message - with main type :mimetype:`message`. It takes an optional argument *_subtype* - which is used to set the MIME subtype. *_subtype* defaults to - :mimetype:`rfc822`. - -:mod:`mimelib` provided some utility functions in its :mod:`address` and -:mod:`date` modules. All of these functions have been moved to the -:mod:`email.utils` module. - -The ``MsgReader`` class/module has been removed. Its functionality is most -closely supported in the :func:`~email.iterators.body_line_iterator` function -in the :mod:`email.iterators` module. - -.. rubric:: Footnotes - -.. [#] Delivery Status Notifications (DSN) are defined in :rfc:`1894`. + Module :mod:`smtpd` + SMTP server framework (primarily useful for testing) diff --git a/Doc/library/email.util.rst b/Doc/library/email.util.rst --- a/Doc/library/email.util.rst +++ b/Doc/library/email.util.rst @@ -8,7 +8,43 @@ -------------- -There are several useful utilities provided in the :mod:`email.utils` module: +There are a couple of useful utilities provided in the :mod:`email.utils` +module: + +.. function:: localtime(dt=None) + + Return local time as an aware datetime object. If called without + arguments, return current time. Otherwise *dt* argument should be a + :class:`~datetime.datetime` instance, and it is converted to the local time + zone according to the system time zone database. If *dt* is naive (that + is, ``dt.tzinfo`` is ``None``), it is assumed to be in local time. In this + case, a positive or zero value for *isdst* causes ``localtime`` to presume + initially that summer time (for example, Daylight Saving Time) is or is not + (respectively) in effect for the specified time. A negative value for + *isdst* causes the ``localtime`` to attempt to divine whether summer time + is in effect for the specified time. + + .. versionadded:: 3.3 + + +.. function:: make_msgid(idstring=None, domain=None) + + Returns a string suitable for an :rfc:`2822`\ -compliant + :mailheader:`Message-ID` header. Optional *idstring* if given, is a string + used to strengthen the uniqueness of the message id. Optional *domain* if + given provides the portion of the msgid after the '@'. The default is the + local hostname. It is not normally necessary to override this default, but + may be useful certain cases, such as a constructing distributed system that + uses a consistent domain name across multiple hosts. + + .. versionchanged:: 3.2 + Added the *domain* keyword. + + +The remaining functions are part of the legacy (``Compat32``) email API. There +is no need to directly use these with the new API, since the parsing and +formatting they provide is done automatically by the header parsing machinery +of the new API. .. function:: quote(str) @@ -141,36 +177,6 @@ .. versionadded:: 3.3 -.. function:: localtime(dt=None) - - Return local time as an aware datetime object. If called without - arguments, return current time. Otherwise *dt* argument should be a - :class:`~datetime.datetime` instance, and it is converted to the local time - zone according to the system time zone database. If *dt* is naive (that - is, ``dt.tzinfo`` is ``None``), it is assumed to be in local time. In this - case, a positive or zero value for *isdst* causes ``localtime`` to presume - initially that summer time (for example, Daylight Saving Time) is or is not - (respectively) in effect for the specified time. A negative value for - *isdst* causes the ``localtime`` to attempt to divine whether summer time - is in effect for the specified time. - - .. versionadded:: 3.3 - - -.. function:: make_msgid(idstring=None, domain=None) - - Returns a string suitable for an :rfc:`2822`\ -compliant - :mailheader:`Message-ID` header. Optional *idstring* if given, is a string - used to strengthen the uniqueness of the message id. Optional *domain* if - given provides the portion of the msgid after the '@'. The default is the - local hostname. It is not normally necessary to override this default, but - may be useful certain cases, such as a constructing distributed system that - uses a consistent domain name across multiple hosts. - - .. versionchanged:: 3.2 - Added the *domain* keyword. - - .. function:: decode_rfc2231(s) Decode the string *s* according to :rfc:`2231`. diff --git a/Lib/email/message.py b/Lib/email/message.py --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -951,6 +951,26 @@ policy = default Message.__init__(self, policy) + + def as_string(self, unixfrom=False, maxheaderlen=None, policy=None): + """Return the entire formatted message as a string. + + Optional 'unixfrom', when true, means include the Unix From_ envelope + header. maxheaderlen is retained for backward compatibility with the + base Message class, but defaults to None, meaning that the policy value + for max_line_length controls the header maximum length. 'policy' is + passed to the Generator instance used to serialize the mesasge; if it + is not specified the policy associated with the message instance is + used. + """ + policy = self.policy if policy is None else policy + if maxheaderlen is None: + maxheaderlen = policy.max_line_length + return super().as_string(maxheaderlen=maxheaderlen, policy=policy) + + def __str__(self): + return self.as_string(policy=self.policy.clone(utf8=True)) + def is_attachment(self): c_d = self.get('content-disposition') return False if c_d is None else c_d.content_disposition == 'attachment' diff --git a/Lib/test/test_email/test_message.py b/Lib/test/test_email/test_message.py --- a/Lib/test/test_email/test_message.py +++ b/Lib/test/test_email/test_message.py @@ -764,6 +764,26 @@ m.set_content(content_manager=cm) self.assertEqual(m['MIME-Version'], '1.0') + def test_as_string_uses_max_header_length_by_default(self): + m = self._str_msg('Subject: long line' + ' ab'*50 + '\n\n') + self.assertEqual(len(m.as_string().strip().splitlines()), 3) + + def test_as_string_allows_maxheaderlen(self): + m = self._str_msg('Subject: long line' + ' ab'*50 + '\n\n') + self.assertEqual(len(m.as_string(maxheaderlen=0).strip().splitlines()), + 1) + self.assertEqual(len(m.as_string(maxheaderlen=34).strip().splitlines()), + 6) + + def test_str_defaults_to_policy_max_line_length(self): + m = self._str_msg('Subject: long line' + ' ab'*50 + '\n\n') + self.assertEqual(len(str(m).strip().splitlines()), 3) + + def test_str_defaults_to_utf8(self): + m = EmailMessage() + m['Subject'] = 'unic?de' + self.assertEqual(str(m), 'Subject: unic?de\n\n') + class TestMIMEPart(TestEmailMessageBase, TestEmailBase): # Doing the full test run here may seem a bit redundant, since the two -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:22:08 2016 From: python-checkins at python.org (r.david.murray) Date: Thu, 08 Sep 2016 01:22:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2324277=3A_What=27s_New_a?= =?utf-8?q?nd_news_entries_for_previous_commit=2E?= Message-ID: <20160908012207.2532.96020.C99A7AF1@psf.io> https://hg.python.org/cpython/rev/bd5b38e3db1a changeset: 103282:bd5b38e3db1a user: R David Murray date: Wed Sep 07 21:21:58 2016 -0400 summary: #24277: What's New and news entries for previous commit. files: Doc/whatsnew/3.6.rst | 5 +++++ Misc/NEWS | 3 +++ 2 files changed, 8 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -462,6 +462,11 @@ email ----- +The new email API, enabled via the *policy* keyword to various constructors, is +no longer provisional. The :mod:`email` documentation has been reorganized and +rewritten to focus on the new API, while retaining the old documentation for +the legacy API. (Contributed by R. David Murray in :issue:`24277`.) + The :mod:`email.mime` classes now all accept an optional *policy* keyword. (Contributed by Berker Peksag in :issue:`27331`.) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -99,6 +99,9 @@ Library ------- +- Issue #24277: The new email API is no longer provisional, and the docs + have been reorganized and rewritten to emphasize the new API. + - lib2to3.pgen3.driver.load_grammar() now creates a stable cache file between runs given the same Grammar.txt input regardless of the hash randomization setting. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:38:22 2016 From: python-checkins at python.org (eric.snow) Date: Thu, 08 Sep 2016 01:38:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317211=3A_Yield_a_?= =?utf-8?q?namedtuple_in_pkgutil=2E?= Message-ID: <20160908013821.69549.81412.D2899E26@psf.io> https://hg.python.org/cpython/rev/4f023e60564b changeset: 103283:4f023e60564b user: Eric Snow date: Wed Sep 07 18:37:17 2016 -0700 summary: Issue #17211: Yield a namedtuple in pkgutil. Patch by Ramchandra Apte. files: Doc/library/pkgutil.rst | 8 +++++- Lib/pkgutil.py | 29 ++++++++++++++++----------- Lib/test/test_pkgutil.py | 5 ++- Lib/test/test_runpy.py | 11 +++++---- Misc/NEWS | 3 ++ 5 files changed, 35 insertions(+), 21 deletions(-) diff --git a/Doc/library/pkgutil.rst b/Doc/library/pkgutil.rst --- a/Doc/library/pkgutil.rst +++ b/Doc/library/pkgutil.rst @@ -11,6 +11,10 @@ This module provides utilities for the import system, in particular package support. +.. class:: ModuleInfo(module_finder, name, ispkg) + + A namedtuple that holds a brief summary of a module's info. + .. function:: extend_path(path, name) @@ -139,7 +143,7 @@ .. function:: iter_modules(path=None, prefix='') - Yields ``(module_finder, name, ispkg)`` for all submodules on *path*, or, if + Yields :class:`ModuleInfo` for all submodules on *path*, or, if *path* is ``None``, all top-level modules on ``sys.path``. *path* should be either ``None`` or a list of paths to look for modules in. @@ -160,7 +164,7 @@ .. function:: walk_packages(path=None, prefix='', onerror=None) - Yields ``(module_finder, name, ispkg)`` for all modules recursively on + Yields :class:`ModuleInfo` for all modules recursively on *path*, or, if *path* is ``None``, all accessible modules. *path* should be either ``None`` or a list of paths to look for modules in. diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py --- a/Lib/pkgutil.py +++ b/Lib/pkgutil.py @@ -1,5 +1,6 @@ """Utilities to support packages.""" +from collections import namedtuple from functools import singledispatch as simplegeneric import importlib import importlib.util @@ -14,9 +15,14 @@ 'get_importer', 'iter_importers', 'get_loader', 'find_loader', 'walk_packages', 'iter_modules', 'get_data', 'ImpImporter', 'ImpLoader', 'read_code', 'extend_path', + 'ModuleInfo', ] +ModuleInfo = namedtuple('ModuleInfo', 'module_finder name ispkg') +ModuleInfo.__doc__ = 'A namedtuple with minimal info about a module.' + + def _get_spec(finder, name): """Return the finder-specific module spec.""" # Works with legacy finders. @@ -45,7 +51,7 @@ def walk_packages(path=None, prefix='', onerror=None): - """Yields (module_finder, name, ispkg) for all modules recursively + """Yields ModuleInfo for all modules recursively on path, or, if path is None, all accessible modules. 'path' should be either None or a list of paths to look for @@ -78,31 +84,31 @@ return True m[p] = True - for importer, name, ispkg in iter_modules(path, prefix): - yield importer, name, ispkg + for info in iter_modules(path, prefix): + yield info - if ispkg: + if info.ispkg: try: - __import__(name) + __import__(info.name) except ImportError: if onerror is not None: - onerror(name) + onerror(info.name) except Exception: if onerror is not None: - onerror(name) + onerror(info.name) else: raise else: - path = getattr(sys.modules[name], '__path__', None) or [] + path = getattr(sys.modules[info.name], '__path__', None) or [] # don't traverse path items we've seen before path = [p for p in path if not seen(p)] - yield from walk_packages(path, name+'.', onerror) + yield from walk_packages(path, info.name+'.', onerror) def iter_modules(path=None, prefix=''): - """Yields (module_finder, name, ispkg) for all submodules on path, + """Yields ModuleInfo for all submodules on path, or, if path is None, all top-level modules on sys.path. 'path' should be either None or a list of paths to look for @@ -111,7 +117,6 @@ 'prefix' is a string to output on the front of every module name on output. """ - if path is None: importers = iter_importers() else: @@ -122,7 +127,7 @@ for name, ispkg in iter_importer_modules(i, prefix): if name not in yielded: yielded[name] = 1 - yield i, name, ispkg + yield ModuleInfo(i, name, ispkg) @simplegeneric diff --git a/Lib/test/test_pkgutil.py b/Lib/test/test_pkgutil.py --- a/Lib/test/test_pkgutil.py +++ b/Lib/test/test_pkgutil.py @@ -81,8 +81,9 @@ self.assertEqual(res2, RESOURCE_DATA) names = [] - for loader, name, ispkg in pkgutil.iter_modules([zip_file]): - names.append(name) + for moduleinfo in pkgutil.iter_modules([zip_file]): + self.assertIsInstance(moduleinfo, pkgutil.ModuleInfo) + names.append(moduleinfo.name) self.assertEqual(names, ['test_getdata_zipfile']) del sys.path[0] diff --git a/Lib/test/test_runpy.py b/Lib/test/test_runpy.py --- a/Lib/test/test_runpy.py +++ b/Lib/test/test_runpy.py @@ -577,13 +577,14 @@ self.addCleanup(self._del_pkg, pkg_dir) for depth in range(2, max_depth+1): self._add_relative_modules(pkg_dir, "", depth) - for finder, mod_name, ispkg in pkgutil.walk_packages([pkg_dir]): - self.assertIsInstance(finder, + for moduleinfo in pkgutil.walk_packages([pkg_dir]): + self.assertIsInstance(moduleinfo, pkgutil.ModuleInfo) + self.assertIsInstance(moduleinfo.module_finder, importlib.machinery.FileFinder) - if ispkg: - expected_packages.remove(mod_name) + if moduleinfo.ispkg: + expected_packages.remove(moduleinfo.name) else: - expected_modules.remove(mod_name) + expected_modules.remove(moduleinfo.name) self.assertEqual(len(expected_packages), 0, expected_packages) self.assertEqual(len(expected_modules), 0, expected_modules) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -7845,6 +7845,9 @@ - Issue #16809: Tkinter's splitlist() and split() methods now accept Tcl_Obj argument. +- Issue #17211: Yield a namedtuple in pkgutil. + Patch by Ramchandra Apte. + - Issue #18324: set_payload now correctly handles binary input. This also supersedes the previous fixes for #14360, #1717, and #16564. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:40:38 2016 From: python-checkins at python.org (brett.cannon) Date: Thu, 08 Sep 2016 01:40:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326667=3A_Add_path?= =?utf-8?q?-like_object_support_to_importlib=2Eutil=2E?= Message-ID: <20160908014038.1702.58370.88D3331F@psf.io> https://hg.python.org/cpython/rev/1a36cf6389d8 changeset: 103284:1a36cf6389d8 user: Brett Cannon date: Wed Sep 07 18:39:18 2016 -0700 summary: Issue #26667: Add path-like object support to importlib.util. files: Doc/library/importlib.rst | 9 + Doc/whatsnew/3.6.rst | 12 +- Lib/importlib/_bootstrap_external.py | 4 + Lib/test/test_importlib/test_spec.py | 6 + Lib/test/test_importlib/test_util.py | 19 + Misc/NEWS | 4 +- Python/importlib_external.h | 4354 +++++++------ 7 files changed, 2230 insertions(+), 2178 deletions(-) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -1149,6 +1149,9 @@ The *optimization* parameter was added and the *debug_override* parameter was deprecated. + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: source_from_cache(path) @@ -1162,6 +1165,9 @@ .. versionadded:: 3.4 + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. function:: decode_source(source_bytes) Decode the given bytes representing source code and return it as a string @@ -1298,6 +1304,9 @@ .. versionadded:: 3.4 + .. versionchanged:: 3.6 + Accepts a :term:`path-like object`. + .. class:: LazyLoader(loader) A class which postpones the execution of the loader of a module until the diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -68,8 +68,6 @@ Standard library improvements: -* PEP 519: :ref:`Adding a file system path protocol ` - Security improvements: * On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool @@ -513,6 +511,11 @@ :class:`importlib.machinery.ExtensionFileLoader` couldn't be used with :class:`importlib.util.LazyLoader`. +:func:`importlib.util.cache_from_source`, +:func:`importlib.util.source_from_cache`, and +:func:`importlib.util.spec_from_file_location` now accept a +:term:`path-like object`. + os -- @@ -528,6 +531,11 @@ :func:`os.getrandom` function. (Contributed by Victor Stinner, part of the :pep:`524`) +See the summary for :ref:`PEP 519 ` for details on how the +:mod:`os` and :mod:`os.path` modules now support +:term:`path-like objects `. + + pickle ------ diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -279,6 +279,7 @@ message = 'debug_override or optimization must be set to None' raise TypeError(message) optimization = '' if debug_override else 1 + path = _os.fspath(path) head, tail = _path_split(path) base, sep, rest = tail.rpartition('.') tag = sys.implementation.cache_tag @@ -309,6 +310,7 @@ """ if sys.implementation.cache_tag is None: raise NotImplementedError('sys.implementation.cache_tag is None') + path = _os.fspath(path) head, pycache_filename = _path_split(path) head, pycache = _path_split(head) if pycache != _PYCACHE: @@ -536,6 +538,8 @@ location = loader.get_filename(name) except ImportError: pass + else: + location = _os.fspath(location) # If the location is on the filesystem, but doesn't actually exist, # we could return None here, indicating that the location is not diff --git a/Lib/test/test_importlib/test_spec.py b/Lib/test/test_importlib/test_spec.py --- a/Lib/test/test_importlib/test_spec.py +++ b/Lib/test/test_importlib/test_spec.py @@ -5,6 +5,7 @@ util = test_util.import_importlib('importlib.util') import os.path +import pathlib from test.support import CleanImport import unittest import sys @@ -659,6 +660,11 @@ self.assertEqual(spec.cached, self.cached) self.assertTrue(spec.has_location) + def test_spec_from_file_location_path_like_arg(self): + spec = self.util.spec_from_file_location(self.name, + pathlib.PurePath(self.path)) + self.assertEqual(spec.origin, self.path) + def test_spec_from_file_location_default_without_location(self): spec = self.util.spec_from_file_location(self.name) diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py --- a/Lib/test/test_importlib/test_util.py +++ b/Lib/test/test_importlib/test_util.py @@ -5,6 +5,7 @@ importlib_util = util.import_importlib('importlib.util') import os +import pathlib import string import sys from test import support @@ -677,6 +678,15 @@ '\\foo\\bar\\baz\\__pycache__\\qux.{}.pyc'.format(self.tag)) @unittest.skipUnless(sys.implementation.cache_tag is not None, + 'requires sys.implementation.cache_tag not be None') + def test_source_from_cache_path_like_arg(self): + path = pathlib.PurePath('foo', 'bar', 'baz', 'qux.py') + expect = os.path.join('foo', 'bar', 'baz', '__pycache__', + 'qux.{}.pyc'.format(self.tag)) + self.assertEqual(self.util.cache_from_source(path, optimization=''), + expect) + + @unittest.skipUnless(sys.implementation.cache_tag is not None, 'requires sys.implementation.cache_tag to not be ' 'None') def test_source_from_cache(self): @@ -738,6 +748,15 @@ with self.assertRaises(ValueError): self.util.source_from_cache(path) + @unittest.skipUnless(sys.implementation.cache_tag is not None, + 'requires sys.implementation.cache_tag to not be ' + 'None') + def test_source_from_cache_path_like_arg(self): + path = pathlib.PurePath('foo', 'bar', 'baz', '__pycache__', + 'qux.{}.pyc'.format(self.tag)) + expect = os.path.join('foo', 'bar', 'baz', 'qux.py') + self.assertEqual(self.util.source_from_cache(path), expect) + (Frozen_PEP3147Tests, Source_PEP3147Tests diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -108,6 +108,8 @@ - Issue #28005: Allow ImportErrors in encoding implementation to propagate. +- Issue #26667: Support path-like objects in importlib.util. + - Issue #27570: Avoid zero-length memcpy() etc calls with null source pointers in the "ctypes" and "array" modules. @@ -237,7 +239,7 @@ - Issue #27930: Improved behaviour of logging.handlers.QueueListener. Thanks to Paulo Andrade and Petr Viktorin for the analysis and patch. -- Issue #6766: Distributed reference counting added to multiprocessing +- Issue #6766: Distributed reference counting added to multiprocessing to support nesting of shared values / proxy objects. C API diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -247,2189 +247,2193 @@ 101,95,95,122,4,111,112,116,45,122,3,46,112,121,122,4, 46,112,121,99,78,41,1,218,12,111,112,116,105,109,105,122, 97,116,105,111,110,99,2,0,0,0,1,0,0,0,11,0, - 0,0,6,0,0,0,67,0,0,0,115,234,0,0,0,124, + 0,0,6,0,0,0,67,0,0,0,115,244,0,0,0,124, 1,100,1,107,9,114,52,116,0,106,1,100,2,116,2,131, 2,1,0,124,2,100,1,107,9,114,40,100,3,125,3,116, 3,124,3,131,1,130,1,124,1,114,48,100,4,110,2,100, - 5,125,2,116,4,124,0,131,1,92,2,125,4,125,5,124, - 5,106,5,100,6,131,1,92,3,125,6,125,7,125,8,116, - 6,106,7,106,8,125,9,124,9,100,1,107,8,114,104,116, - 9,100,7,131,1,130,1,100,4,106,10,124,6,114,116,124, - 6,110,2,124,8,124,7,124,9,103,3,131,1,125,10,124, - 2,100,1,107,8,114,162,116,6,106,11,106,12,100,8,107, - 2,114,154,100,4,125,2,110,8,116,6,106,11,106,12,125, - 2,116,13,124,2,131,1,125,2,124,2,100,4,107,3,114, - 214,124,2,106,14,131,0,115,200,116,15,100,9,106,16,124, - 2,131,1,131,1,130,1,100,10,106,16,124,10,116,17,124, - 2,131,3,125,10,116,18,124,4,116,19,124,10,116,20,100, - 8,25,0,23,0,131,3,83,0,41,11,97,254,2,0,0, - 71,105,118,101,110,32,116,104,101,32,112,97,116,104,32,116, - 111,32,97,32,46,112,121,32,102,105,108,101,44,32,114,101, - 116,117,114,110,32,116,104,101,32,112,97,116,104,32,116,111, - 32,105,116,115,32,46,112,121,99,32,102,105,108,101,46,10, - 10,32,32,32,32,84,104,101,32,46,112,121,32,102,105,108, - 101,32,100,111,101,115,32,110,111,116,32,110,101,101,100,32, - 116,111,32,101,120,105,115,116,59,32,116,104,105,115,32,115, - 105,109,112,108,121,32,114,101,116,117,114,110,115,32,116,104, - 101,32,112,97,116,104,32,116,111,32,116,104,101,10,32,32, - 32,32,46,112,121,99,32,102,105,108,101,32,99,97,108,99, - 117,108,97,116,101,100,32,97,115,32,105,102,32,116,104,101, - 32,46,112,121,32,102,105,108,101,32,119,101,114,101,32,105, - 109,112,111,114,116,101,100,46,10,10,32,32,32,32,84,104, - 101,32,39,111,112,116,105,109,105,122,97,116,105,111,110,39, - 32,112,97,114,97,109,101,116,101,114,32,99,111,110,116,114, - 111,108,115,32,116,104,101,32,112,114,101,115,117,109,101,100, - 32,111,112,116,105,109,105,122,97,116,105,111,110,32,108,101, - 118,101,108,32,111,102,10,32,32,32,32,116,104,101,32,98, - 121,116,101,99,111,100,101,32,102,105,108,101,46,32,73,102, - 32,39,111,112,116,105,109,105,122,97,116,105,111,110,39,32, - 105,115,32,110,111,116,32,78,111,110,101,44,32,116,104,101, - 32,115,116,114,105,110,103,32,114,101,112,114,101,115,101,110, - 116,97,116,105,111,110,10,32,32,32,32,111,102,32,116,104, - 101,32,97,114,103,117,109,101,110,116,32,105,115,32,116,97, - 107,101,110,32,97,110,100,32,118,101,114,105,102,105,101,100, - 32,116,111,32,98,101,32,97,108,112,104,97,110,117,109,101, - 114,105,99,32,40,101,108,115,101,32,86,97,108,117,101,69, - 114,114,111,114,10,32,32,32,32,105,115,32,114,97,105,115, - 101,100,41,46,10,10,32,32,32,32,84,104,101,32,100,101, - 98,117,103,95,111,118,101,114,114,105,100,101,32,112,97,114, - 97,109,101,116,101,114,32,105,115,32,100,101,112,114,101,99, - 97,116,101,100,46,32,73,102,32,100,101,98,117,103,95,111, - 118,101,114,114,105,100,101,32,105,115,32,110,111,116,32,78, - 111,110,101,44,10,32,32,32,32,97,32,84,114,117,101,32, - 118,97,108,117,101,32,105,115,32,116,104,101,32,115,97,109, - 101,32,97,115,32,115,101,116,116,105,110,103,32,39,111,112, - 116,105,109,105,122,97,116,105,111,110,39,32,116,111,32,116, - 104,101,32,101,109,112,116,121,32,115,116,114,105,110,103,10, - 32,32,32,32,119,104,105,108,101,32,97,32,70,97,108,115, - 101,32,118,97,108,117,101,32,105,115,32,101,113,117,105,118, - 97,108,101,110,116,32,116,111,32,115,101,116,116,105,110,103, - 32,39,111,112,116,105,109,105,122,97,116,105,111,110,39,32, - 116,111,32,39,49,39,46,10,10,32,32,32,32,73,102,32, - 115,121,115,46,105,109,112,108,101,109,101,110,116,97,116,105, - 111,110,46,99,97,99,104,101,95,116,97,103,32,105,115,32, - 78,111,110,101,32,116,104,101,110,32,78,111,116,73,109,112, - 108,101,109,101,110,116,101,100,69,114,114,111,114,32,105,115, - 32,114,97,105,115,101,100,46,10,10,32,32,32,32,78,122, - 70,116,104,101,32,100,101,98,117,103,95,111,118,101,114,114, - 105,100,101,32,112,97,114,97,109,101,116,101,114,32,105,115, - 32,100,101,112,114,101,99,97,116,101,100,59,32,117,115,101, - 32,39,111,112,116,105,109,105,122,97,116,105,111,110,39,32, - 105,110,115,116,101,97,100,122,50,100,101,98,117,103,95,111, - 118,101,114,114,105,100,101,32,111,114,32,111,112,116,105,109, - 105,122,97,116,105,111,110,32,109,117,115,116,32,98,101,32, - 115,101,116,32,116,111,32,78,111,110,101,114,32,0,0,0, - 114,31,0,0,0,218,1,46,122,36,115,121,115,46,105,109, + 5,125,2,116,4,106,5,124,0,131,1,125,0,116,6,124, + 0,131,1,92,2,125,4,125,5,124,5,106,7,100,6,131, + 1,92,3,125,6,125,7,125,8,116,8,106,9,106,10,125, + 9,124,9,100,1,107,8,114,114,116,11,100,7,131,1,130, + 1,100,4,106,12,124,6,114,126,124,6,110,2,124,8,124, + 7,124,9,103,3,131,1,125,10,124,2,100,1,107,8,114, + 172,116,8,106,13,106,14,100,8,107,2,114,164,100,4,125, + 2,110,8,116,8,106,13,106,14,125,2,116,15,124,2,131, + 1,125,2,124,2,100,4,107,3,114,224,124,2,106,16,131, + 0,115,210,116,17,100,9,106,18,124,2,131,1,131,1,130, + 1,100,10,106,18,124,10,116,19,124,2,131,3,125,10,116, + 20,124,4,116,21,124,10,116,22,100,8,25,0,23,0,131, + 3,83,0,41,11,97,254,2,0,0,71,105,118,101,110,32, + 116,104,101,32,112,97,116,104,32,116,111,32,97,32,46,112, + 121,32,102,105,108,101,44,32,114,101,116,117,114,110,32,116, + 104,101,32,112,97,116,104,32,116,111,32,105,116,115,32,46, + 112,121,99,32,102,105,108,101,46,10,10,32,32,32,32,84, + 104,101,32,46,112,121,32,102,105,108,101,32,100,111,101,115, + 32,110,111,116,32,110,101,101,100,32,116,111,32,101,120,105, + 115,116,59,32,116,104,105,115,32,115,105,109,112,108,121,32, + 114,101,116,117,114,110,115,32,116,104,101,32,112,97,116,104, + 32,116,111,32,116,104,101,10,32,32,32,32,46,112,121,99, + 32,102,105,108,101,32,99,97,108,99,117,108,97,116,101,100, + 32,97,115,32,105,102,32,116,104,101,32,46,112,121,32,102, + 105,108,101,32,119,101,114,101,32,105,109,112,111,114,116,101, + 100,46,10,10,32,32,32,32,84,104,101,32,39,111,112,116, + 105,109,105,122,97,116,105,111,110,39,32,112,97,114,97,109, + 101,116,101,114,32,99,111,110,116,114,111,108,115,32,116,104, + 101,32,112,114,101,115,117,109,101,100,32,111,112,116,105,109, + 105,122,97,116,105,111,110,32,108,101,118,101,108,32,111,102, + 10,32,32,32,32,116,104,101,32,98,121,116,101,99,111,100, + 101,32,102,105,108,101,46,32,73,102,32,39,111,112,116,105, + 109,105,122,97,116,105,111,110,39,32,105,115,32,110,111,116, + 32,78,111,110,101,44,32,116,104,101,32,115,116,114,105,110, + 103,32,114,101,112,114,101,115,101,110,116,97,116,105,111,110, + 10,32,32,32,32,111,102,32,116,104,101,32,97,114,103,117, + 109,101,110,116,32,105,115,32,116,97,107,101,110,32,97,110, + 100,32,118,101,114,105,102,105,101,100,32,116,111,32,98,101, + 32,97,108,112,104,97,110,117,109,101,114,105,99,32,40,101, + 108,115,101,32,86,97,108,117,101,69,114,114,111,114,10,32, + 32,32,32,105,115,32,114,97,105,115,101,100,41,46,10,10, + 32,32,32,32,84,104,101,32,100,101,98,117,103,95,111,118, + 101,114,114,105,100,101,32,112,97,114,97,109,101,116,101,114, + 32,105,115,32,100,101,112,114,101,99,97,116,101,100,46,32, + 73,102,32,100,101,98,117,103,95,111,118,101,114,114,105,100, + 101,32,105,115,32,110,111,116,32,78,111,110,101,44,10,32, + 32,32,32,97,32,84,114,117,101,32,118,97,108,117,101,32, + 105,115,32,116,104,101,32,115,97,109,101,32,97,115,32,115, + 101,116,116,105,110,103,32,39,111,112,116,105,109,105,122,97, + 116,105,111,110,39,32,116,111,32,116,104,101,32,101,109,112, + 116,121,32,115,116,114,105,110,103,10,32,32,32,32,119,104, + 105,108,101,32,97,32,70,97,108,115,101,32,118,97,108,117, + 101,32,105,115,32,101,113,117,105,118,97,108,101,110,116,32, + 116,111,32,115,101,116,116,105,110,103,32,39,111,112,116,105, + 109,105,122,97,116,105,111,110,39,32,116,111,32,39,49,39, + 46,10,10,32,32,32,32,73,102,32,115,121,115,46,105,109, 112,108,101,109,101,110,116,97,116,105,111,110,46,99,97,99, - 104,101,95,116,97,103,32,105,115,32,78,111,110,101,233,0, - 0,0,0,122,24,123,33,114,125,32,105,115,32,110,111,116, - 32,97,108,112,104,97,110,117,109,101,114,105,99,122,7,123, - 125,46,123,125,123,125,41,21,218,9,95,119,97,114,110,105, - 110,103,115,218,4,119,97,114,110,218,18,68,101,112,114,101, - 99,97,116,105,111,110,87,97,114,110,105,110,103,218,9,84, - 121,112,101,69,114,114,111,114,114,40,0,0,0,114,34,0, - 0,0,114,8,0,0,0,218,14,105,109,112,108,101,109,101, - 110,116,97,116,105,111,110,218,9,99,97,99,104,101,95,116, - 97,103,218,19,78,111,116,73,109,112,108,101,109,101,110,116, - 101,100,69,114,114,111,114,114,28,0,0,0,218,5,102,108, - 97,103,115,218,8,111,112,116,105,109,105,122,101,218,3,115, - 116,114,218,7,105,115,97,108,110,117,109,218,10,86,97,108, - 117,101,69,114,114,111,114,114,50,0,0,0,218,4,95,79, - 80,84,114,30,0,0,0,218,8,95,80,89,67,65,67,72, - 69,218,17,66,89,84,69,67,79,68,69,95,83,85,70,70, - 73,88,69,83,41,11,114,37,0,0,0,90,14,100,101,98, - 117,103,95,111,118,101,114,114,105,100,101,114,60,0,0,0, - 218,7,109,101,115,115,97,103,101,218,4,104,101,97,100,114, - 39,0,0,0,90,4,98,97,115,101,218,3,115,101,112,218, - 4,114,101,115,116,90,3,116,97,103,90,15,97,108,109,111, - 115,116,95,102,105,108,101,110,97,109,101,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,17,99,97,99,104, - 101,95,102,114,111,109,95,115,111,117,114,99,101,1,1,0, - 0,115,46,0,0,0,0,18,8,1,6,1,6,1,8,1, - 4,1,8,1,12,1,12,1,16,1,8,1,8,1,8,1, - 24,1,8,1,12,1,6,2,8,1,8,1,8,1,8,1, - 14,1,14,1,114,82,0,0,0,99,1,0,0,0,0,0, - 0,0,8,0,0,0,5,0,0,0,67,0,0,0,115,220, - 0,0,0,116,0,106,1,106,2,100,1,107,8,114,20,116, - 3,100,2,131,1,130,1,116,4,124,0,131,1,92,2,125, - 1,125,2,116,4,124,1,131,1,92,2,125,1,125,3,124, - 3,116,5,107,3,114,68,116,6,100,3,106,7,116,5,124, - 0,131,2,131,1,130,1,124,2,106,8,100,4,131,1,125, - 4,124,4,100,11,107,7,114,102,116,6,100,7,106,7,124, - 2,131,1,131,1,130,1,110,86,124,4,100,6,107,2,114, - 188,124,2,106,9,100,4,100,5,131,2,100,12,25,0,125, - 5,124,5,106,10,116,11,131,1,115,150,116,6,100,8,106, - 7,116,11,131,1,131,1,130,1,124,5,116,12,116,11,131, - 1,100,1,133,2,25,0,125,6,124,6,106,13,131,0,115, - 188,116,6,100,9,106,7,124,5,131,1,131,1,130,1,124, - 2,106,14,100,4,131,1,100,10,25,0,125,7,116,15,124, - 1,124,7,116,16,100,10,25,0,23,0,131,2,83,0,41, - 13,97,110,1,0,0,71,105,118,101,110,32,116,104,101,32, - 112,97,116,104,32,116,111,32,97,32,46,112,121,99,46,32, - 102,105,108,101,44,32,114,101,116,117,114,110,32,116,104,101, - 32,112,97,116,104,32,116,111,32,105,116,115,32,46,112,121, - 32,102,105,108,101,46,10,10,32,32,32,32,84,104,101,32, - 46,112,121,99,32,102,105,108,101,32,100,111,101,115,32,110, - 111,116,32,110,101,101,100,32,116,111,32,101,120,105,115,116, - 59,32,116,104,105,115,32,115,105,109,112,108,121,32,114,101, - 116,117,114,110,115,32,116,104,101,32,112,97,116,104,32,116, - 111,10,32,32,32,32,116,104,101,32,46,112,121,32,102,105, - 108,101,32,99,97,108,99,117,108,97,116,101,100,32,116,111, - 32,99,111,114,114,101,115,112,111,110,100,32,116,111,32,116, - 104,101,32,46,112,121,99,32,102,105,108,101,46,32,32,73, - 102,32,112,97,116,104,32,100,111,101,115,10,32,32,32,32, - 110,111,116,32,99,111,110,102,111,114,109,32,116,111,32,80, - 69,80,32,51,49,52,55,47,52,56,56,32,102,111,114,109, - 97,116,44,32,86,97,108,117,101,69,114,114,111,114,32,119, - 105,108,108,32,98,101,32,114,97,105,115,101,100,46,32,73, - 102,10,32,32,32,32,115,121,115,46,105,109,112,108,101,109, - 101,110,116,97,116,105,111,110,46,99,97,99,104,101,95,116, - 97,103,32,105,115,32,78,111,110,101,32,116,104,101,110,32, - 78,111,116,73,109,112,108,101,109,101,110,116,101,100,69,114, - 114,111,114,32,105,115,32,114,97,105,115,101,100,46,10,10, - 32,32,32,32,78,122,36,115,121,115,46,105,109,112,108,101, - 109,101,110,116,97,116,105,111,110,46,99,97,99,104,101,95, - 116,97,103,32,105,115,32,78,111,110,101,122,37,123,125,32, - 110,111,116,32,98,111,116,116,111,109,45,108,101,118,101,108, - 32,100,105,114,101,99,116,111,114,121,32,105,110,32,123,33, - 114,125,114,61,0,0,0,114,59,0,0,0,233,3,0,0, - 0,122,33,101,120,112,101,99,116,101,100,32,111,110,108,121, - 32,50,32,111,114,32,51,32,100,111,116,115,32,105,110,32, - 123,33,114,125,122,57,111,112,116,105,109,105,122,97,116,105, - 111,110,32,112,111,114,116,105,111,110,32,111,102,32,102,105, - 108,101,110,97,109,101,32,100,111,101,115,32,110,111,116,32, - 115,116,97,114,116,32,119,105,116,104,32,123,33,114,125,122, - 52,111,112,116,105,109,105,122,97,116,105,111,110,32,108,101, - 118,101,108,32,123,33,114,125,32,105,115,32,110,111,116,32, - 97,110,32,97,108,112,104,97,110,117,109,101,114,105,99,32, - 118,97,108,117,101,114,62,0,0,0,62,2,0,0,0,114, - 59,0,0,0,114,83,0,0,0,233,254,255,255,255,41,17, - 114,8,0,0,0,114,67,0,0,0,114,68,0,0,0,114, - 69,0,0,0,114,40,0,0,0,114,76,0,0,0,114,74, - 0,0,0,114,50,0,0,0,218,5,99,111,117,110,116,114, - 36,0,0,0,114,10,0,0,0,114,75,0,0,0,114,33, - 0,0,0,114,73,0,0,0,218,9,112,97,114,116,105,116, - 105,111,110,114,30,0,0,0,218,15,83,79,85,82,67,69, - 95,83,85,70,70,73,88,69,83,41,8,114,37,0,0,0, - 114,79,0,0,0,90,16,112,121,99,97,99,104,101,95,102, - 105,108,101,110,97,109,101,90,7,112,121,99,97,99,104,101, - 90,9,100,111,116,95,99,111,117,110,116,114,60,0,0,0, - 90,9,111,112,116,95,108,101,118,101,108,90,13,98,97,115, - 101,95,102,105,108,101,110,97,109,101,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,218,17,115,111,117,114,99, - 101,95,102,114,111,109,95,99,97,99,104,101,45,1,0,0, - 115,44,0,0,0,0,9,12,1,8,1,12,1,12,1,8, - 1,6,1,10,1,10,1,8,1,6,1,10,1,8,1,16, - 1,10,1,6,1,8,1,16,1,8,1,6,1,8,1,14, - 1,114,88,0,0,0,99,1,0,0,0,0,0,0,0,5, - 0,0,0,12,0,0,0,67,0,0,0,115,128,0,0,0, - 116,0,124,0,131,1,100,1,107,2,114,16,100,2,83,0, - 124,0,106,1,100,3,131,1,92,3,125,1,125,2,125,3, - 124,1,12,0,115,58,124,3,106,2,131,0,100,7,100,8, - 133,2,25,0,100,6,107,3,114,62,124,0,83,0,121,12, - 116,3,124,0,131,1,125,4,87,0,110,36,4,0,116,4, - 116,5,102,2,107,10,114,110,1,0,1,0,1,0,124,0, - 100,2,100,9,133,2,25,0,125,4,89,0,110,2,88,0, - 116,6,124,4,131,1,114,124,124,4,83,0,124,0,83,0, - 41,10,122,188,67,111,110,118,101,114,116,32,97,32,98,121, - 116,101,99,111,100,101,32,102,105,108,101,32,112,97,116,104, - 32,116,111,32,97,32,115,111,117,114,99,101,32,112,97,116, - 104,32,40,105,102,32,112,111,115,115,105,98,108,101,41,46, - 10,10,32,32,32,32,84,104,105,115,32,102,117,110,99,116, - 105,111,110,32,101,120,105,115,116,115,32,112,117,114,101,108, - 121,32,102,111,114,32,98,97,99,107,119,97,114,100,115,45, - 99,111,109,112,97,116,105,98,105,108,105,116,121,32,102,111, - 114,10,32,32,32,32,80,121,73,109,112,111,114,116,95,69, - 120,101,99,67,111,100,101,77,111,100,117,108,101,87,105,116, - 104,70,105,108,101,110,97,109,101,115,40,41,32,105,110,32, - 116,104,101,32,67,32,65,80,73,46,10,10,32,32,32,32, - 114,62,0,0,0,78,114,61,0,0,0,114,83,0,0,0, - 114,31,0,0,0,90,2,112,121,233,253,255,255,255,233,255, - 255,255,255,114,90,0,0,0,41,7,114,33,0,0,0,114, - 34,0,0,0,218,5,108,111,119,101,114,114,88,0,0,0, - 114,69,0,0,0,114,74,0,0,0,114,46,0,0,0,41, - 5,218,13,98,121,116,101,99,111,100,101,95,112,97,116,104, - 114,81,0,0,0,114,38,0,0,0,90,9,101,120,116,101, - 110,115,105,111,110,218,11,115,111,117,114,99,101,95,112,97, - 116,104,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,218,15,95,103,101,116,95,115,111,117,114,99,101,102,105, - 108,101,78,1,0,0,115,20,0,0,0,0,7,12,1,4, - 1,16,1,26,1,4,1,2,1,12,1,18,1,18,1,114, - 94,0,0,0,99,1,0,0,0,0,0,0,0,1,0,0, - 0,11,0,0,0,67,0,0,0,115,74,0,0,0,124,0, - 106,0,116,1,116,2,131,1,131,1,114,46,121,8,116,3, - 124,0,131,1,83,0,4,0,116,4,107,10,114,42,1,0, - 1,0,1,0,89,0,113,70,88,0,110,24,124,0,106,0, - 116,1,116,5,131,1,131,1,114,66,124,0,83,0,110,4, - 100,0,83,0,100,0,83,0,41,1,78,41,6,218,8,101, - 110,100,115,119,105,116,104,218,5,116,117,112,108,101,114,87, - 0,0,0,114,82,0,0,0,114,69,0,0,0,114,77,0, - 0,0,41,1,218,8,102,105,108,101,110,97,109,101,114,4, + 104,101,95,116,97,103,32,105,115,32,78,111,110,101,32,116, + 104,101,110,32,78,111,116,73,109,112,108,101,109,101,110,116, + 101,100,69,114,114,111,114,32,105,115,32,114,97,105,115,101, + 100,46,10,10,32,32,32,32,78,122,70,116,104,101,32,100, + 101,98,117,103,95,111,118,101,114,114,105,100,101,32,112,97, + 114,97,109,101,116,101,114,32,105,115,32,100,101,112,114,101, + 99,97,116,101,100,59,32,117,115,101,32,39,111,112,116,105, + 109,105,122,97,116,105,111,110,39,32,105,110,115,116,101,97, + 100,122,50,100,101,98,117,103,95,111,118,101,114,114,105,100, + 101,32,111,114,32,111,112,116,105,109,105,122,97,116,105,111, + 110,32,109,117,115,116,32,98,101,32,115,101,116,32,116,111, + 32,78,111,110,101,114,32,0,0,0,114,31,0,0,0,218, + 1,46,122,36,115,121,115,46,105,109,112,108,101,109,101,110, + 116,97,116,105,111,110,46,99,97,99,104,101,95,116,97,103, + 32,105,115,32,78,111,110,101,233,0,0,0,0,122,24,123, + 33,114,125,32,105,115,32,110,111,116,32,97,108,112,104,97, + 110,117,109,101,114,105,99,122,7,123,125,46,123,125,123,125, + 41,23,218,9,95,119,97,114,110,105,110,103,115,218,4,119, + 97,114,110,218,18,68,101,112,114,101,99,97,116,105,111,110, + 87,97,114,110,105,110,103,218,9,84,121,112,101,69,114,114, + 111,114,114,3,0,0,0,218,6,102,115,112,97,116,104,114, + 40,0,0,0,114,34,0,0,0,114,8,0,0,0,218,14, + 105,109,112,108,101,109,101,110,116,97,116,105,111,110,218,9, + 99,97,99,104,101,95,116,97,103,218,19,78,111,116,73,109, + 112,108,101,109,101,110,116,101,100,69,114,114,111,114,114,28, + 0,0,0,218,5,102,108,97,103,115,218,8,111,112,116,105, + 109,105,122,101,218,3,115,116,114,218,7,105,115,97,108,110, + 117,109,218,10,86,97,108,117,101,69,114,114,111,114,114,50, + 0,0,0,218,4,95,79,80,84,114,30,0,0,0,218,8, + 95,80,89,67,65,67,72,69,218,17,66,89,84,69,67,79, + 68,69,95,83,85,70,70,73,88,69,83,41,11,114,37,0, + 0,0,90,14,100,101,98,117,103,95,111,118,101,114,114,105, + 100,101,114,60,0,0,0,218,7,109,101,115,115,97,103,101, + 218,4,104,101,97,100,114,39,0,0,0,90,4,98,97,115, + 101,218,3,115,101,112,218,4,114,101,115,116,90,3,116,97, + 103,90,15,97,108,109,111,115,116,95,102,105,108,101,110,97, + 109,101,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,218,17,99,97,99,104,101,95,102,114,111,109,95,115,111, + 117,114,99,101,1,1,0,0,115,48,0,0,0,0,18,8, + 1,6,1,6,1,8,1,4,1,8,1,12,1,10,1,12, + 1,16,1,8,1,8,1,8,1,24,1,8,1,12,1,6, + 2,8,1,8,1,8,1,8,1,14,1,14,1,114,83,0, + 0,0,99,1,0,0,0,0,0,0,0,8,0,0,0,5, + 0,0,0,67,0,0,0,115,230,0,0,0,116,0,106,1, + 106,2,100,1,107,8,114,20,116,3,100,2,131,1,130,1, + 116,4,106,5,124,0,131,1,125,0,116,6,124,0,131,1, + 92,2,125,1,125,2,116,6,124,1,131,1,92,2,125,1, + 125,3,124,3,116,7,107,3,114,78,116,8,100,3,106,9, + 116,7,124,0,131,2,131,1,130,1,124,2,106,10,100,4, + 131,1,125,4,124,4,100,11,107,7,114,112,116,8,100,7, + 106,9,124,2,131,1,131,1,130,1,110,86,124,4,100,6, + 107,2,114,198,124,2,106,11,100,4,100,5,131,2,100,12, + 25,0,125,5,124,5,106,12,116,13,131,1,115,160,116,8, + 100,8,106,9,116,13,131,1,131,1,130,1,124,5,116,14, + 116,13,131,1,100,1,133,2,25,0,125,6,124,6,106,15, + 131,0,115,198,116,8,100,9,106,9,124,5,131,1,131,1, + 130,1,124,2,106,16,100,4,131,1,100,10,25,0,125,7, + 116,17,124,1,124,7,116,18,100,10,25,0,23,0,131,2, + 83,0,41,13,97,110,1,0,0,71,105,118,101,110,32,116, + 104,101,32,112,97,116,104,32,116,111,32,97,32,46,112,121, + 99,46,32,102,105,108,101,44,32,114,101,116,117,114,110,32, + 116,104,101,32,112,97,116,104,32,116,111,32,105,116,115,32, + 46,112,121,32,102,105,108,101,46,10,10,32,32,32,32,84, + 104,101,32,46,112,121,99,32,102,105,108,101,32,100,111,101, + 115,32,110,111,116,32,110,101,101,100,32,116,111,32,101,120, + 105,115,116,59,32,116,104,105,115,32,115,105,109,112,108,121, + 32,114,101,116,117,114,110,115,32,116,104,101,32,112,97,116, + 104,32,116,111,10,32,32,32,32,116,104,101,32,46,112,121, + 32,102,105,108,101,32,99,97,108,99,117,108,97,116,101,100, + 32,116,111,32,99,111,114,114,101,115,112,111,110,100,32,116, + 111,32,116,104,101,32,46,112,121,99,32,102,105,108,101,46, + 32,32,73,102,32,112,97,116,104,32,100,111,101,115,10,32, + 32,32,32,110,111,116,32,99,111,110,102,111,114,109,32,116, + 111,32,80,69,80,32,51,49,52,55,47,52,56,56,32,102, + 111,114,109,97,116,44,32,86,97,108,117,101,69,114,114,111, + 114,32,119,105,108,108,32,98,101,32,114,97,105,115,101,100, + 46,32,73,102,10,32,32,32,32,115,121,115,46,105,109,112, + 108,101,109,101,110,116,97,116,105,111,110,46,99,97,99,104, + 101,95,116,97,103,32,105,115,32,78,111,110,101,32,116,104, + 101,110,32,78,111,116,73,109,112,108,101,109,101,110,116,101, + 100,69,114,114,111,114,32,105,115,32,114,97,105,115,101,100, + 46,10,10,32,32,32,32,78,122,36,115,121,115,46,105,109, + 112,108,101,109,101,110,116,97,116,105,111,110,46,99,97,99, + 104,101,95,116,97,103,32,105,115,32,78,111,110,101,122,37, + 123,125,32,110,111,116,32,98,111,116,116,111,109,45,108,101, + 118,101,108,32,100,105,114,101,99,116,111,114,121,32,105,110, + 32,123,33,114,125,114,61,0,0,0,114,59,0,0,0,233, + 3,0,0,0,122,33,101,120,112,101,99,116,101,100,32,111, + 110,108,121,32,50,32,111,114,32,51,32,100,111,116,115,32, + 105,110,32,123,33,114,125,122,57,111,112,116,105,109,105,122, + 97,116,105,111,110,32,112,111,114,116,105,111,110,32,111,102, + 32,102,105,108,101,110,97,109,101,32,100,111,101,115,32,110, + 111,116,32,115,116,97,114,116,32,119,105,116,104,32,123,33, + 114,125,122,52,111,112,116,105,109,105,122,97,116,105,111,110, + 32,108,101,118,101,108,32,123,33,114,125,32,105,115,32,110, + 111,116,32,97,110,32,97,108,112,104,97,110,117,109,101,114, + 105,99,32,118,97,108,117,101,114,62,0,0,0,62,2,0, + 0,0,114,59,0,0,0,114,84,0,0,0,233,254,255,255, + 255,41,19,114,8,0,0,0,114,68,0,0,0,114,69,0, + 0,0,114,70,0,0,0,114,3,0,0,0,114,67,0,0, + 0,114,40,0,0,0,114,77,0,0,0,114,75,0,0,0, + 114,50,0,0,0,218,5,99,111,117,110,116,114,36,0,0, + 0,114,10,0,0,0,114,76,0,0,0,114,33,0,0,0, + 114,74,0,0,0,218,9,112,97,114,116,105,116,105,111,110, + 114,30,0,0,0,218,15,83,79,85,82,67,69,95,83,85, + 70,70,73,88,69,83,41,8,114,37,0,0,0,114,80,0, + 0,0,90,16,112,121,99,97,99,104,101,95,102,105,108,101, + 110,97,109,101,90,7,112,121,99,97,99,104,101,90,9,100, + 111,116,95,99,111,117,110,116,114,60,0,0,0,90,9,111, + 112,116,95,108,101,118,101,108,90,13,98,97,115,101,95,102, + 105,108,101,110,97,109,101,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,218,17,115,111,117,114,99,101,95,102, + 114,111,109,95,99,97,99,104,101,46,1,0,0,115,46,0, + 0,0,0,9,12,1,8,1,10,1,12,1,12,1,8,1, + 6,1,10,1,10,1,8,1,6,1,10,1,8,1,16,1, + 10,1,6,1,8,1,16,1,8,1,6,1,8,1,14,1, + 114,89,0,0,0,99,1,0,0,0,0,0,0,0,5,0, + 0,0,12,0,0,0,67,0,0,0,115,128,0,0,0,116, + 0,124,0,131,1,100,1,107,2,114,16,100,2,83,0,124, + 0,106,1,100,3,131,1,92,3,125,1,125,2,125,3,124, + 1,12,0,115,58,124,3,106,2,131,0,100,7,100,8,133, + 2,25,0,100,6,107,3,114,62,124,0,83,0,121,12,116, + 3,124,0,131,1,125,4,87,0,110,36,4,0,116,4,116, + 5,102,2,107,10,114,110,1,0,1,0,1,0,124,0,100, + 2,100,9,133,2,25,0,125,4,89,0,110,2,88,0,116, + 6,124,4,131,1,114,124,124,4,83,0,124,0,83,0,41, + 10,122,188,67,111,110,118,101,114,116,32,97,32,98,121,116, + 101,99,111,100,101,32,102,105,108,101,32,112,97,116,104,32, + 116,111,32,97,32,115,111,117,114,99,101,32,112,97,116,104, + 32,40,105,102,32,112,111,115,115,105,98,108,101,41,46,10, + 10,32,32,32,32,84,104,105,115,32,102,117,110,99,116,105, + 111,110,32,101,120,105,115,116,115,32,112,117,114,101,108,121, + 32,102,111,114,32,98,97,99,107,119,97,114,100,115,45,99, + 111,109,112,97,116,105,98,105,108,105,116,121,32,102,111,114, + 10,32,32,32,32,80,121,73,109,112,111,114,116,95,69,120, + 101,99,67,111,100,101,77,111,100,117,108,101,87,105,116,104, + 70,105,108,101,110,97,109,101,115,40,41,32,105,110,32,116, + 104,101,32,67,32,65,80,73,46,10,10,32,32,32,32,114, + 62,0,0,0,78,114,61,0,0,0,114,84,0,0,0,114, + 31,0,0,0,90,2,112,121,233,253,255,255,255,233,255,255, + 255,255,114,91,0,0,0,41,7,114,33,0,0,0,114,34, + 0,0,0,218,5,108,111,119,101,114,114,89,0,0,0,114, + 70,0,0,0,114,75,0,0,0,114,46,0,0,0,41,5, + 218,13,98,121,116,101,99,111,100,101,95,112,97,116,104,114, + 82,0,0,0,114,38,0,0,0,90,9,101,120,116,101,110, + 115,105,111,110,218,11,115,111,117,114,99,101,95,112,97,116, + 104,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, + 218,15,95,103,101,116,95,115,111,117,114,99,101,102,105,108, + 101,80,1,0,0,115,20,0,0,0,0,7,12,1,4,1, + 16,1,26,1,4,1,2,1,12,1,18,1,18,1,114,95, + 0,0,0,99,1,0,0,0,0,0,0,0,1,0,0,0, + 11,0,0,0,67,0,0,0,115,74,0,0,0,124,0,106, + 0,116,1,116,2,131,1,131,1,114,46,121,8,116,3,124, + 0,131,1,83,0,4,0,116,4,107,10,114,42,1,0,1, + 0,1,0,89,0,113,70,88,0,110,24,124,0,106,0,116, + 1,116,5,131,1,131,1,114,66,124,0,83,0,110,4,100, + 0,83,0,100,0,83,0,41,1,78,41,6,218,8,101,110, + 100,115,119,105,116,104,218,5,116,117,112,108,101,114,88,0, + 0,0,114,83,0,0,0,114,70,0,0,0,114,78,0,0, + 0,41,1,218,8,102,105,108,101,110,97,109,101,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,218,11,95,103, + 101,116,95,99,97,99,104,101,100,99,1,0,0,115,16,0, + 0,0,0,1,14,1,2,1,8,1,14,1,8,1,14,1, + 6,2,114,99,0,0,0,99,1,0,0,0,0,0,0,0, + 2,0,0,0,11,0,0,0,67,0,0,0,115,52,0,0, + 0,121,14,116,0,124,0,131,1,106,1,125,1,87,0,110, + 24,4,0,116,2,107,10,114,38,1,0,1,0,1,0,100, + 1,125,1,89,0,110,2,88,0,124,1,100,2,79,0,125, + 1,124,1,83,0,41,3,122,51,67,97,108,99,117,108,97, + 116,101,32,116,104,101,32,109,111,100,101,32,112,101,114,109, + 105,115,115,105,111,110,115,32,102,111,114,32,97,32,98,121, + 116,101,99,111,100,101,32,102,105,108,101,46,105,182,1,0, + 0,233,128,0,0,0,41,3,114,41,0,0,0,114,43,0, + 0,0,114,42,0,0,0,41,2,114,37,0,0,0,114,44, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,10,95,99,97,108,99,95,109,111,100,101,111,1, + 0,0,115,12,0,0,0,0,2,2,1,14,1,14,1,10, + 3,8,1,114,101,0,0,0,99,1,0,0,0,0,0,0, + 0,3,0,0,0,11,0,0,0,3,0,0,0,115,68,0, + 0,0,100,6,135,0,102,1,100,2,100,3,132,9,125,1, + 121,10,116,0,106,1,125,2,87,0,110,28,4,0,116,2, + 107,10,114,52,1,0,1,0,1,0,100,4,100,5,132,0, + 125,2,89,0,110,2,88,0,124,2,124,1,136,0,131,2, + 1,0,124,1,83,0,41,7,122,252,68,101,99,111,114,97, + 116,111,114,32,116,111,32,118,101,114,105,102,121,32,116,104, + 97,116,32,116,104,101,32,109,111,100,117,108,101,32,98,101, + 105,110,103,32,114,101,113,117,101,115,116,101,100,32,109,97, + 116,99,104,101,115,32,116,104,101,32,111,110,101,32,116,104, + 101,10,32,32,32,32,108,111,97,100,101,114,32,99,97,110, + 32,104,97,110,100,108,101,46,10,10,32,32,32,32,84,104, + 101,32,102,105,114,115,116,32,97,114,103,117,109,101,110,116, + 32,40,115,101,108,102,41,32,109,117,115,116,32,100,101,102, + 105,110,101,32,95,110,97,109,101,32,119,104,105,99,104,32, + 116,104,101,32,115,101,99,111,110,100,32,97,114,103,117,109, + 101,110,116,32,105,115,10,32,32,32,32,99,111,109,112,97, + 114,101,100,32,97,103,97,105,110,115,116,46,32,73,102,32, + 116,104,101,32,99,111,109,112,97,114,105,115,111,110,32,102, + 97,105,108,115,32,116,104,101,110,32,73,109,112,111,114,116, + 69,114,114,111,114,32,105,115,32,114,97,105,115,101,100,46, + 10,10,32,32,32,32,78,99,2,0,0,0,0,0,0,0, + 4,0,0,0,5,0,0,0,31,0,0,0,115,64,0,0, + 0,124,1,100,0,107,8,114,16,124,0,106,0,125,1,110, + 34,124,0,106,0,124,1,107,3,114,50,116,1,100,1,124, + 0,106,0,124,1,102,2,22,0,100,2,124,1,144,1,131, + 1,130,1,136,0,124,0,124,1,124,2,124,3,142,2,83, + 0,41,3,78,122,30,108,111,97,100,101,114,32,102,111,114, + 32,37,115,32,99,97,110,110,111,116,32,104,97,110,100,108, + 101,32,37,115,218,4,110,97,109,101,41,2,114,102,0,0, + 0,218,11,73,109,112,111,114,116,69,114,114,111,114,41,4, + 218,4,115,101,108,102,114,102,0,0,0,218,4,97,114,103, + 115,90,6,107,119,97,114,103,115,41,1,218,6,109,101,116, + 104,111,100,114,4,0,0,0,114,6,0,0,0,218,19,95, + 99,104,101,99,107,95,110,97,109,101,95,119,114,97,112,112, + 101,114,131,1,0,0,115,12,0,0,0,0,1,8,1,8, + 1,10,1,4,1,20,1,122,40,95,99,104,101,99,107,95, + 110,97,109,101,46,60,108,111,99,97,108,115,62,46,95,99, + 104,101,99,107,95,110,97,109,101,95,119,114,97,112,112,101, + 114,99,2,0,0,0,0,0,0,0,3,0,0,0,7,0, + 0,0,83,0,0,0,115,60,0,0,0,120,40,100,5,68, + 0,93,32,125,2,116,0,124,1,124,2,131,2,114,6,116, + 1,124,0,124,2,116,2,124,1,124,2,131,2,131,3,1, + 0,113,6,87,0,124,0,106,3,106,4,124,1,106,3,131, + 1,1,0,100,0,83,0,41,6,78,218,10,95,95,109,111, + 100,117,108,101,95,95,218,8,95,95,110,97,109,101,95,95, + 218,12,95,95,113,117,97,108,110,97,109,101,95,95,218,7, + 95,95,100,111,99,95,95,41,4,122,10,95,95,109,111,100, + 117,108,101,95,95,122,8,95,95,110,97,109,101,95,95,122, + 12,95,95,113,117,97,108,110,97,109,101,95,95,122,7,95, + 95,100,111,99,95,95,41,5,218,7,104,97,115,97,116,116, + 114,218,7,115,101,116,97,116,116,114,218,7,103,101,116,97, + 116,116,114,218,8,95,95,100,105,99,116,95,95,218,6,117, + 112,100,97,116,101,41,3,90,3,110,101,119,90,3,111,108, + 100,114,55,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,218,5,95,119,114,97,112,142,1,0,0, + 115,8,0,0,0,0,1,10,1,10,1,22,1,122,26,95, + 99,104,101,99,107,95,110,97,109,101,46,60,108,111,99,97, + 108,115,62,46,95,119,114,97,112,41,1,78,41,3,218,10, + 95,98,111,111,116,115,116,114,97,112,114,117,0,0,0,218, + 9,78,97,109,101,69,114,114,111,114,41,3,114,106,0,0, + 0,114,107,0,0,0,114,117,0,0,0,114,4,0,0,0, + 41,1,114,106,0,0,0,114,6,0,0,0,218,11,95,99, + 104,101,99,107,95,110,97,109,101,123,1,0,0,115,14,0, + 0,0,0,8,14,7,2,1,10,1,14,2,14,5,10,1, + 114,120,0,0,0,99,2,0,0,0,0,0,0,0,5,0, + 0,0,4,0,0,0,67,0,0,0,115,60,0,0,0,124, + 0,106,0,124,1,131,1,92,2,125,2,125,3,124,2,100, + 1,107,8,114,56,116,1,124,3,131,1,114,56,100,2,125, + 4,116,2,106,3,124,4,106,4,124,3,100,3,25,0,131, + 1,116,5,131,2,1,0,124,2,83,0,41,4,122,155,84, + 114,121,32,116,111,32,102,105,110,100,32,97,32,108,111,97, + 100,101,114,32,102,111,114,32,116,104,101,32,115,112,101,99, + 105,102,105,101,100,32,109,111,100,117,108,101,32,98,121,32, + 100,101,108,101,103,97,116,105,110,103,32,116,111,10,32,32, + 32,32,115,101,108,102,46,102,105,110,100,95,108,111,97,100, + 101,114,40,41,46,10,10,32,32,32,32,84,104,105,115,32, + 109,101,116,104,111,100,32,105,115,32,100,101,112,114,101,99, + 97,116,101,100,32,105,110,32,102,97,118,111,114,32,111,102, + 32,102,105,110,100,101,114,46,102,105,110,100,95,115,112,101, + 99,40,41,46,10,10,32,32,32,32,78,122,44,78,111,116, + 32,105,109,112,111,114,116,105,110,103,32,100,105,114,101,99, + 116,111,114,121,32,123,125,58,32,109,105,115,115,105,110,103, + 32,95,95,105,110,105,116,95,95,114,62,0,0,0,41,6, + 218,11,102,105,110,100,95,108,111,97,100,101,114,114,33,0, + 0,0,114,63,0,0,0,114,64,0,0,0,114,50,0,0, + 0,218,13,73,109,112,111,114,116,87,97,114,110,105,110,103, + 41,5,114,104,0,0,0,218,8,102,117,108,108,110,97,109, + 101,218,6,108,111,97,100,101,114,218,8,112,111,114,116,105, + 111,110,115,218,3,109,115,103,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,218,17,95,102,105,110,100,95,109, + 111,100,117,108,101,95,115,104,105,109,151,1,0,0,115,10, + 0,0,0,0,10,14,1,16,1,4,1,22,1,114,127,0, + 0,0,99,4,0,0,0,0,0,0,0,11,0,0,0,19, + 0,0,0,67,0,0,0,115,128,1,0,0,105,0,125,4, + 124,2,100,1,107,9,114,22,124,2,124,4,100,2,60,0, + 110,4,100,3,125,2,124,3,100,1,107,9,114,42,124,3, + 124,4,100,4,60,0,124,0,100,1,100,5,133,2,25,0, + 125,5,124,0,100,5,100,6,133,2,25,0,125,6,124,0, + 100,6,100,7,133,2,25,0,125,7,124,5,116,0,107,3, + 114,122,100,8,106,1,124,2,124,5,131,2,125,8,116,2, + 106,3,100,9,124,8,131,2,1,0,116,4,124,8,124,4, + 141,1,130,1,110,86,116,5,124,6,131,1,100,5,107,3, + 114,166,100,10,106,1,124,2,131,1,125,8,116,2,106,3, + 100,9,124,8,131,2,1,0,116,6,124,8,131,1,130,1, + 110,42,116,5,124,7,131,1,100,5,107,3,114,208,100,11, + 106,1,124,2,131,1,125,8,116,2,106,3,100,9,124,8, + 131,2,1,0,116,6,124,8,131,1,130,1,124,1,100,1, + 107,9,144,1,114,116,121,16,116,7,124,1,100,12,25,0, + 131,1,125,9,87,0,110,20,4,0,116,8,107,10,114,254, + 1,0,1,0,1,0,89,0,110,48,88,0,116,9,124,6, + 131,1,124,9,107,3,144,1,114,46,100,13,106,1,124,2, + 131,1,125,8,116,2,106,3,100,9,124,8,131,2,1,0, + 116,4,124,8,124,4,141,1,130,1,121,16,124,1,100,14, + 25,0,100,15,64,0,125,10,87,0,110,22,4,0,116,8, + 107,10,144,1,114,84,1,0,1,0,1,0,89,0,110,32, + 88,0,116,9,124,7,131,1,124,10,107,3,144,1,114,116, + 116,4,100,13,106,1,124,2,131,1,124,4,141,1,130,1, + 124,0,100,7,100,1,133,2,25,0,83,0,41,16,97,122, + 1,0,0,86,97,108,105,100,97,116,101,32,116,104,101,32, + 104,101,97,100,101,114,32,111,102,32,116,104,101,32,112,97, + 115,115,101,100,45,105,110,32,98,121,116,101,99,111,100,101, + 32,97,103,97,105,110,115,116,32,115,111,117,114,99,101,95, + 115,116,97,116,115,32,40,105,102,10,32,32,32,32,103,105, + 118,101,110,41,32,97,110,100,32,114,101,116,117,114,110,105, + 110,103,32,116,104,101,32,98,121,116,101,99,111,100,101,32, + 116,104,97,116,32,99,97,110,32,98,101,32,99,111,109,112, + 105,108,101,100,32,98,121,32,99,111,109,112,105,108,101,40, + 41,46,10,10,32,32,32,32,65,108,108,32,111,116,104,101, + 114,32,97,114,103,117,109,101,110,116,115,32,97,114,101,32, + 117,115,101,100,32,116,111,32,101,110,104,97,110,99,101,32, + 101,114,114,111,114,32,114,101,112,111,114,116,105,110,103,46, + 10,10,32,32,32,32,73,109,112,111,114,116,69,114,114,111, + 114,32,105,115,32,114,97,105,115,101,100,32,119,104,101,110, + 32,116,104,101,32,109,97,103,105,99,32,110,117,109,98,101, + 114,32,105,115,32,105,110,99,111,114,114,101,99,116,32,111, + 114,32,116,104,101,32,98,121,116,101,99,111,100,101,32,105, + 115,10,32,32,32,32,102,111,117,110,100,32,116,111,32,98, + 101,32,115,116,97,108,101,46,32,69,79,70,69,114,114,111, + 114,32,105,115,32,114,97,105,115,101,100,32,119,104,101,110, + 32,116,104,101,32,100,97,116,97,32,105,115,32,102,111,117, + 110,100,32,116,111,32,98,101,10,32,32,32,32,116,114,117, + 110,99,97,116,101,100,46,10,10,32,32,32,32,78,114,102, + 0,0,0,122,10,60,98,121,116,101,99,111,100,101,62,114, + 37,0,0,0,114,14,0,0,0,233,8,0,0,0,233,12, + 0,0,0,122,30,98,97,100,32,109,97,103,105,99,32,110, + 117,109,98,101,114,32,105,110,32,123,33,114,125,58,32,123, + 33,114,125,122,2,123,125,122,43,114,101,97,99,104,101,100, + 32,69,79,70,32,119,104,105,108,101,32,114,101,97,100,105, + 110,103,32,116,105,109,101,115,116,97,109,112,32,105,110,32, + 123,33,114,125,122,48,114,101,97,99,104,101,100,32,69,79, + 70,32,119,104,105,108,101,32,114,101,97,100,105,110,103,32, + 115,105,122,101,32,111,102,32,115,111,117,114,99,101,32,105, + 110,32,123,33,114,125,218,5,109,116,105,109,101,122,26,98, + 121,116,101,99,111,100,101,32,105,115,32,115,116,97,108,101, + 32,102,111,114,32,123,33,114,125,218,4,115,105,122,101,108, + 3,0,0,0,255,127,255,127,3,0,41,10,218,12,77,65, + 71,73,67,95,78,85,77,66,69,82,114,50,0,0,0,114, + 118,0,0,0,218,16,95,118,101,114,98,111,115,101,95,109, + 101,115,115,97,103,101,114,103,0,0,0,114,33,0,0,0, + 218,8,69,79,70,69,114,114,111,114,114,16,0,0,0,218, + 8,75,101,121,69,114,114,111,114,114,21,0,0,0,41,11, + 114,56,0,0,0,218,12,115,111,117,114,99,101,95,115,116, + 97,116,115,114,102,0,0,0,114,37,0,0,0,90,11,101, + 120,99,95,100,101,116,97,105,108,115,90,5,109,97,103,105, + 99,90,13,114,97,119,95,116,105,109,101,115,116,97,109,112, + 90,8,114,97,119,95,115,105,122,101,114,79,0,0,0,218, + 12,115,111,117,114,99,101,95,109,116,105,109,101,218,11,115, + 111,117,114,99,101,95,115,105,122,101,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,218,25,95,118,97,108,105, + 100,97,116,101,95,98,121,116,101,99,111,100,101,95,104,101, + 97,100,101,114,168,1,0,0,115,76,0,0,0,0,11,4, + 1,8,1,10,3,4,1,8,1,8,1,12,1,12,1,12, + 1,8,1,12,1,12,1,12,1,12,1,10,1,12,1,10, + 1,12,1,10,1,12,1,8,1,10,1,2,1,16,1,14, + 1,6,2,14,1,10,1,12,1,10,1,2,1,16,1,16, + 1,6,2,14,1,10,1,6,1,114,139,0,0,0,99,4, + 0,0,0,0,0,0,0,5,0,0,0,6,0,0,0,67, + 0,0,0,115,86,0,0,0,116,0,106,1,124,0,131,1, + 125,4,116,2,124,4,116,3,131,2,114,58,116,4,106,5, + 100,1,124,2,131,2,1,0,124,3,100,2,107,9,114,52, + 116,6,106,7,124,4,124,3,131,2,1,0,124,4,83,0, + 110,24,116,8,100,3,106,9,124,2,131,1,100,4,124,1, + 100,5,124,2,144,2,131,1,130,1,100,2,83,0,41,6, + 122,60,67,111,109,112,105,108,101,32,98,121,116,101,99,111, + 100,101,32,97,115,32,114,101,116,117,114,110,101,100,32,98, + 121,32,95,118,97,108,105,100,97,116,101,95,98,121,116,101, + 99,111,100,101,95,104,101,97,100,101,114,40,41,46,122,21, + 99,111,100,101,32,111,98,106,101,99,116,32,102,114,111,109, + 32,123,33,114,125,78,122,23,78,111,110,45,99,111,100,101, + 32,111,98,106,101,99,116,32,105,110,32,123,33,114,125,114, + 102,0,0,0,114,37,0,0,0,41,10,218,7,109,97,114, + 115,104,97,108,90,5,108,111,97,100,115,218,10,105,115,105, + 110,115,116,97,110,99,101,218,10,95,99,111,100,101,95,116, + 121,112,101,114,118,0,0,0,114,133,0,0,0,218,4,95, + 105,109,112,90,16,95,102,105,120,95,99,111,95,102,105,108, + 101,110,97,109,101,114,103,0,0,0,114,50,0,0,0,41, + 5,114,56,0,0,0,114,102,0,0,0,114,93,0,0,0, + 114,94,0,0,0,218,4,99,111,100,101,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,17,95,99,111,109, + 112,105,108,101,95,98,121,116,101,99,111,100,101,223,1,0, + 0,115,16,0,0,0,0,2,10,1,10,1,12,1,8,1, + 12,1,6,2,12,1,114,145,0,0,0,114,62,0,0,0, + 99,3,0,0,0,0,0,0,0,4,0,0,0,3,0,0, + 0,67,0,0,0,115,56,0,0,0,116,0,116,1,131,1, + 125,3,124,3,106,2,116,3,124,1,131,1,131,1,1,0, + 124,3,106,2,116,3,124,2,131,1,131,1,1,0,124,3, + 106,2,116,4,106,5,124,0,131,1,131,1,1,0,124,3, + 83,0,41,1,122,80,67,111,109,112,105,108,101,32,97,32, + 99,111,100,101,32,111,98,106,101,99,116,32,105,110,116,111, + 32,98,121,116,101,99,111,100,101,32,102,111,114,32,119,114, + 105,116,105,110,103,32,111,117,116,32,116,111,32,97,32,98, + 121,116,101,45,99,111,109,112,105,108,101,100,10,32,32,32, + 32,102,105,108,101,46,41,6,218,9,98,121,116,101,97,114, + 114,97,121,114,132,0,0,0,218,6,101,120,116,101,110,100, + 114,19,0,0,0,114,140,0,0,0,90,5,100,117,109,112, + 115,41,4,114,144,0,0,0,114,130,0,0,0,114,138,0, + 0,0,114,56,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,218,17,95,99,111,100,101,95,116,111, + 95,98,121,116,101,99,111,100,101,235,1,0,0,115,10,0, + 0,0,0,3,8,1,14,1,14,1,16,1,114,148,0,0, + 0,99,1,0,0,0,0,0,0,0,5,0,0,0,4,0, + 0,0,67,0,0,0,115,62,0,0,0,100,1,100,2,108, + 0,125,1,116,1,106,2,124,0,131,1,106,3,125,2,124, + 1,106,4,124,2,131,1,125,3,116,1,106,5,100,2,100, + 3,131,2,125,4,124,4,106,6,124,0,106,6,124,3,100, + 1,25,0,131,1,131,1,83,0,41,4,122,121,68,101,99, + 111,100,101,32,98,121,116,101,115,32,114,101,112,114,101,115, + 101,110,116,105,110,103,32,115,111,117,114,99,101,32,99,111, + 100,101,32,97,110,100,32,114,101,116,117,114,110,32,116,104, + 101,32,115,116,114,105,110,103,46,10,10,32,32,32,32,85, + 110,105,118,101,114,115,97,108,32,110,101,119,108,105,110,101, + 32,115,117,112,112,111,114,116,32,105,115,32,117,115,101,100, + 32,105,110,32,116,104,101,32,100,101,99,111,100,105,110,103, + 46,10,32,32,32,32,114,62,0,0,0,78,84,41,7,218, + 8,116,111,107,101,110,105,122,101,114,52,0,0,0,90,7, + 66,121,116,101,115,73,79,90,8,114,101,97,100,108,105,110, + 101,90,15,100,101,116,101,99,116,95,101,110,99,111,100,105, + 110,103,90,25,73,110,99,114,101,109,101,110,116,97,108,78, + 101,119,108,105,110,101,68,101,99,111,100,101,114,218,6,100, + 101,99,111,100,101,41,5,218,12,115,111,117,114,99,101,95, + 98,121,116,101,115,114,149,0,0,0,90,21,115,111,117,114, + 99,101,95,98,121,116,101,115,95,114,101,97,100,108,105,110, + 101,218,8,101,110,99,111,100,105,110,103,90,15,110,101,119, + 108,105,110,101,95,100,101,99,111,100,101,114,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,13,100,101,99, + 111,100,101,95,115,111,117,114,99,101,245,1,0,0,115,10, + 0,0,0,0,5,8,1,12,1,10,1,12,1,114,153,0, + 0,0,41,2,114,124,0,0,0,218,26,115,117,98,109,111, + 100,117,108,101,95,115,101,97,114,99,104,95,108,111,99,97, + 116,105,111,110,115,99,2,0,0,0,2,0,0,0,9,0, + 0,0,19,0,0,0,67,0,0,0,115,20,1,0,0,124, + 1,100,1,107,8,114,60,100,2,125,1,116,0,124,2,100, + 3,131,2,114,70,121,14,124,2,106,1,124,0,131,1,125, + 1,87,0,113,70,4,0,116,2,107,10,114,56,1,0,1, + 0,1,0,89,0,113,70,88,0,110,10,116,3,106,4,124, + 1,131,1,125,1,116,5,106,6,124,0,124,2,100,4,124, + 1,144,1,131,2,125,4,100,5,124,4,95,7,124,2,100, + 1,107,8,114,158,120,54,116,8,131,0,68,0,93,40,92, + 2,125,5,125,6,124,1,106,9,116,10,124,6,131,1,131, + 1,114,110,124,5,124,0,124,1,131,2,125,2,124,2,124, + 4,95,11,80,0,113,110,87,0,100,1,83,0,124,3,116, + 12,107,8,114,224,116,0,124,2,100,6,131,2,114,230,121, + 14,124,2,106,13,124,0,131,1,125,7,87,0,110,20,4, + 0,116,2,107,10,114,210,1,0,1,0,1,0,89,0,113, + 230,88,0,124,7,114,230,103,0,124,4,95,14,110,6,124, + 3,124,4,95,14,124,4,106,14,103,0,107,2,144,1,114, + 16,124,1,144,1,114,16,116,15,124,1,131,1,100,7,25, + 0,125,8,124,4,106,14,106,16,124,8,131,1,1,0,124, + 4,83,0,41,8,97,61,1,0,0,82,101,116,117,114,110, + 32,97,32,109,111,100,117,108,101,32,115,112,101,99,32,98, + 97,115,101,100,32,111,110,32,97,32,102,105,108,101,32,108, + 111,99,97,116,105,111,110,46,10,10,32,32,32,32,84,111, + 32,105,110,100,105,99,97,116,101,32,116,104,97,116,32,116, + 104,101,32,109,111,100,117,108,101,32,105,115,32,97,32,112, + 97,99,107,97,103,101,44,32,115,101,116,10,32,32,32,32, + 115,117,98,109,111,100,117,108,101,95,115,101,97,114,99,104, + 95,108,111,99,97,116,105,111,110,115,32,116,111,32,97,32, + 108,105,115,116,32,111,102,32,100,105,114,101,99,116,111,114, + 121,32,112,97,116,104,115,46,32,32,65,110,10,32,32,32, + 32,101,109,112,116,121,32,108,105,115,116,32,105,115,32,115, + 117,102,102,105,99,105,101,110,116,44,32,116,104,111,117,103, + 104,32,105,116,115,32,110,111,116,32,111,116,104,101,114,119, + 105,115,101,32,117,115,101,102,117,108,32,116,111,32,116,104, + 101,10,32,32,32,32,105,109,112,111,114,116,32,115,121,115, + 116,101,109,46,10,10,32,32,32,32,84,104,101,32,108,111, + 97,100,101,114,32,109,117,115,116,32,116,97,107,101,32,97, + 32,115,112,101,99,32,97,115,32,105,116,115,32,111,110,108, + 121,32,95,95,105,110,105,116,95,95,40,41,32,97,114,103, + 46,10,10,32,32,32,32,78,122,9,60,117,110,107,110,111, + 119,110,62,218,12,103,101,116,95,102,105,108,101,110,97,109, + 101,218,6,111,114,105,103,105,110,84,218,10,105,115,95,112, + 97,99,107,97,103,101,114,62,0,0,0,41,17,114,112,0, + 0,0,114,155,0,0,0,114,103,0,0,0,114,3,0,0, + 0,114,67,0,0,0,114,118,0,0,0,218,10,77,111,100, + 117,108,101,83,112,101,99,90,13,95,115,101,116,95,102,105, + 108,101,97,116,116,114,218,27,95,103,101,116,95,115,117,112, + 112,111,114,116,101,100,95,102,105,108,101,95,108,111,97,100, + 101,114,115,114,96,0,0,0,114,97,0,0,0,114,124,0, + 0,0,218,9,95,80,79,80,85,76,65,84,69,114,157,0, + 0,0,114,154,0,0,0,114,40,0,0,0,218,6,97,112, + 112,101,110,100,41,9,114,102,0,0,0,90,8,108,111,99, + 97,116,105,111,110,114,124,0,0,0,114,154,0,0,0,218, + 4,115,112,101,99,218,12,108,111,97,100,101,114,95,99,108, + 97,115,115,218,8,115,117,102,102,105,120,101,115,114,157,0, + 0,0,90,7,100,105,114,110,97,109,101,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,23,115,112,101,99, + 95,102,114,111,109,95,102,105,108,101,95,108,111,99,97,116, + 105,111,110,6,2,0,0,115,62,0,0,0,0,12,8,4, + 4,1,10,2,2,1,14,1,14,1,8,2,10,8,18,1, + 6,3,8,1,16,1,14,1,10,1,6,1,6,2,4,3, + 8,2,10,1,2,1,14,1,14,1,6,2,4,1,8,2, + 6,1,12,1,6,1,12,1,12,2,114,165,0,0,0,99, + 0,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0, + 64,0,0,0,115,80,0,0,0,101,0,90,1,100,0,90, + 2,100,1,90,3,100,2,90,4,100,3,90,5,100,4,90, + 6,101,7,100,5,100,6,132,0,131,1,90,8,101,7,100, + 7,100,8,132,0,131,1,90,9,101,7,100,14,100,10,100, + 11,132,1,131,1,90,10,101,7,100,15,100,12,100,13,132, + 1,131,1,90,11,100,9,83,0,41,16,218,21,87,105,110, + 100,111,119,115,82,101,103,105,115,116,114,121,70,105,110,100, + 101,114,122,62,77,101,116,97,32,112,97,116,104,32,102,105, + 110,100,101,114,32,102,111,114,32,109,111,100,117,108,101,115, + 32,100,101,99,108,97,114,101,100,32,105,110,32,116,104,101, + 32,87,105,110,100,111,119,115,32,114,101,103,105,115,116,114, + 121,46,122,59,83,111,102,116,119,97,114,101,92,80,121,116, + 104,111,110,92,80,121,116,104,111,110,67,111,114,101,92,123, + 115,121,115,95,118,101,114,115,105,111,110,125,92,77,111,100, + 117,108,101,115,92,123,102,117,108,108,110,97,109,101,125,122, + 65,83,111,102,116,119,97,114,101,92,80,121,116,104,111,110, + 92,80,121,116,104,111,110,67,111,114,101,92,123,115,121,115, + 95,118,101,114,115,105,111,110,125,92,77,111,100,117,108,101, + 115,92,123,102,117,108,108,110,97,109,101,125,92,68,101,98, + 117,103,70,99,2,0,0,0,0,0,0,0,2,0,0,0, + 11,0,0,0,67,0,0,0,115,50,0,0,0,121,14,116, + 0,106,1,116,0,106,2,124,1,131,2,83,0,4,0,116, + 3,107,10,114,44,1,0,1,0,1,0,116,0,106,1,116, + 0,106,4,124,1,131,2,83,0,88,0,100,0,83,0,41, + 1,78,41,5,218,7,95,119,105,110,114,101,103,90,7,79, + 112,101,110,75,101,121,90,17,72,75,69,89,95,67,85,82, + 82,69,78,84,95,85,83,69,82,114,42,0,0,0,90,18, + 72,75,69,89,95,76,79,67,65,76,95,77,65,67,72,73, + 78,69,41,2,218,3,99,108,115,114,5,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,14,95, + 111,112,101,110,95,114,101,103,105,115,116,114,121,86,2,0, + 0,115,8,0,0,0,0,2,2,1,14,1,14,1,122,36, + 87,105,110,100,111,119,115,82,101,103,105,115,116,114,121,70, + 105,110,100,101,114,46,95,111,112,101,110,95,114,101,103,105, + 115,116,114,121,99,2,0,0,0,0,0,0,0,6,0,0, + 0,16,0,0,0,67,0,0,0,115,116,0,0,0,124,0, + 106,0,114,14,124,0,106,1,125,2,110,6,124,0,106,2, + 125,2,124,2,106,3,100,1,124,1,100,2,100,3,116,4, + 106,5,100,0,100,4,133,2,25,0,22,0,144,2,131,0, + 125,3,121,38,124,0,106,6,124,3,131,1,143,18,125,4, + 116,7,106,8,124,4,100,5,131,2,125,5,87,0,100,0, + 81,0,82,0,88,0,87,0,110,20,4,0,116,9,107,10, + 114,110,1,0,1,0,1,0,100,0,83,0,88,0,124,5, + 83,0,41,6,78,114,123,0,0,0,90,11,115,121,115,95, + 118,101,114,115,105,111,110,122,5,37,100,46,37,100,114,59, + 0,0,0,114,32,0,0,0,41,10,218,11,68,69,66,85, + 71,95,66,85,73,76,68,218,18,82,69,71,73,83,84,82, + 89,95,75,69,89,95,68,69,66,85,71,218,12,82,69,71, + 73,83,84,82,89,95,75,69,89,114,50,0,0,0,114,8, + 0,0,0,218,12,118,101,114,115,105,111,110,95,105,110,102, + 111,114,169,0,0,0,114,167,0,0,0,90,10,81,117,101, + 114,121,86,97,108,117,101,114,42,0,0,0,41,6,114,168, + 0,0,0,114,123,0,0,0,90,12,114,101,103,105,115,116, + 114,121,95,107,101,121,114,5,0,0,0,90,4,104,107,101, + 121,218,8,102,105,108,101,112,97,116,104,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,16,95,115,101,97, + 114,99,104,95,114,101,103,105,115,116,114,121,93,2,0,0, + 115,22,0,0,0,0,2,6,1,8,2,6,1,10,1,22, + 1,2,1,12,1,26,1,14,1,6,1,122,38,87,105,110, + 100,111,119,115,82,101,103,105,115,116,114,121,70,105,110,100, + 101,114,46,95,115,101,97,114,99,104,95,114,101,103,105,115, + 116,114,121,78,99,4,0,0,0,0,0,0,0,8,0,0, + 0,14,0,0,0,67,0,0,0,115,122,0,0,0,124,0, + 106,0,124,1,131,1,125,4,124,4,100,0,107,8,114,22, + 100,0,83,0,121,12,116,1,124,4,131,1,1,0,87,0, + 110,20,4,0,116,2,107,10,114,54,1,0,1,0,1,0, + 100,0,83,0,88,0,120,60,116,3,131,0,68,0,93,50, + 92,2,125,5,125,6,124,4,106,4,116,5,124,6,131,1, + 131,1,114,64,116,6,106,7,124,1,124,5,124,1,124,4, + 131,2,100,1,124,4,144,1,131,2,125,7,124,7,83,0, + 113,64,87,0,100,0,83,0,41,2,78,114,156,0,0,0, + 41,8,114,175,0,0,0,114,41,0,0,0,114,42,0,0, + 0,114,159,0,0,0,114,96,0,0,0,114,97,0,0,0, + 114,118,0,0,0,218,16,115,112,101,99,95,102,114,111,109, + 95,108,111,97,100,101,114,41,8,114,168,0,0,0,114,123, + 0,0,0,114,37,0,0,0,218,6,116,97,114,103,101,116, + 114,174,0,0,0,114,124,0,0,0,114,164,0,0,0,114, + 162,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,218,9,102,105,110,100,95,115,112,101,99,108,2, + 0,0,115,26,0,0,0,0,2,10,1,8,1,4,1,2, + 1,12,1,14,1,6,1,16,1,14,1,6,1,10,1,8, + 1,122,31,87,105,110,100,111,119,115,82,101,103,105,115,116, + 114,121,70,105,110,100,101,114,46,102,105,110,100,95,115,112, + 101,99,99,3,0,0,0,0,0,0,0,4,0,0,0,3, + 0,0,0,67,0,0,0,115,36,0,0,0,124,0,106,0, + 124,1,124,2,131,2,125,3,124,3,100,1,107,9,114,28, + 124,3,106,1,83,0,110,4,100,1,83,0,100,1,83,0, + 41,2,122,108,70,105,110,100,32,109,111,100,117,108,101,32, + 110,97,109,101,100,32,105,110,32,116,104,101,32,114,101,103, + 105,115,116,114,121,46,10,10,32,32,32,32,32,32,32,32, + 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, + 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, + 101,120,101,99,95,109,111,100,117,108,101,40,41,32,105,110, + 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, + 78,41,2,114,178,0,0,0,114,124,0,0,0,41,4,114, + 168,0,0,0,114,123,0,0,0,114,37,0,0,0,114,162, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,11,102,105,110,100,95,109,111,100,117,108,101,124, + 2,0,0,115,8,0,0,0,0,7,12,1,8,1,8,2, + 122,33,87,105,110,100,111,119,115,82,101,103,105,115,116,114, + 121,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, + 117,108,101,41,2,78,78,41,1,78,41,12,114,109,0,0, + 0,114,108,0,0,0,114,110,0,0,0,114,111,0,0,0, + 114,172,0,0,0,114,171,0,0,0,114,170,0,0,0,218, + 11,99,108,97,115,115,109,101,116,104,111,100,114,169,0,0, + 0,114,175,0,0,0,114,178,0,0,0,114,179,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,166,0,0,0,74,2,0,0,115,20,0, + 0,0,8,2,4,3,4,3,4,2,4,2,12,7,12,15, + 2,1,12,15,2,1,114,166,0,0,0,99,0,0,0,0, + 0,0,0,0,0,0,0,0,2,0,0,0,64,0,0,0, + 115,48,0,0,0,101,0,90,1,100,0,90,2,100,1,90, + 3,100,2,100,3,132,0,90,4,100,4,100,5,132,0,90, + 5,100,6,100,7,132,0,90,6,100,8,100,9,132,0,90, + 7,100,10,83,0,41,11,218,13,95,76,111,97,100,101,114, + 66,97,115,105,99,115,122,83,66,97,115,101,32,99,108,97, + 115,115,32,111,102,32,99,111,109,109,111,110,32,99,111,100, + 101,32,110,101,101,100,101,100,32,98,121,32,98,111,116,104, + 32,83,111,117,114,99,101,76,111,97,100,101,114,32,97,110, + 100,10,32,32,32,32,83,111,117,114,99,101,108,101,115,115, + 70,105,108,101,76,111,97,100,101,114,46,99,2,0,0,0, + 0,0,0,0,5,0,0,0,3,0,0,0,67,0,0,0, + 115,64,0,0,0,116,0,124,0,106,1,124,1,131,1,131, + 1,100,1,25,0,125,2,124,2,106,2,100,2,100,1,131, + 2,100,3,25,0,125,3,124,1,106,3,100,2,131,1,100, + 4,25,0,125,4,124,3,100,5,107,2,111,62,124,4,100, + 5,107,3,83,0,41,6,122,141,67,111,110,99,114,101,116, + 101,32,105,109,112,108,101,109,101,110,116,97,116,105,111,110, + 32,111,102,32,73,110,115,112,101,99,116,76,111,97,100,101, + 114,46,105,115,95,112,97,99,107,97,103,101,32,98,121,32, + 99,104,101,99,107,105,110,103,32,105,102,10,32,32,32,32, + 32,32,32,32,116,104,101,32,112,97,116,104,32,114,101,116, + 117,114,110,101,100,32,98,121,32,103,101,116,95,102,105,108, + 101,110,97,109,101,32,104,97,115,32,97,32,102,105,108,101, + 110,97,109,101,32,111,102,32,39,95,95,105,110,105,116,95, + 95,46,112,121,39,46,114,31,0,0,0,114,61,0,0,0, + 114,62,0,0,0,114,59,0,0,0,218,8,95,95,105,110, + 105,116,95,95,41,4,114,40,0,0,0,114,155,0,0,0, + 114,36,0,0,0,114,34,0,0,0,41,5,114,104,0,0, + 0,114,123,0,0,0,114,98,0,0,0,90,13,102,105,108, + 101,110,97,109,101,95,98,97,115,101,90,9,116,97,105,108, + 95,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,157,0,0,0,143,2,0,0,115,8,0, + 0,0,0,3,18,1,16,1,14,1,122,24,95,76,111,97, + 100,101,114,66,97,115,105,99,115,46,105,115,95,112,97,99, + 107,97,103,101,99,2,0,0,0,0,0,0,0,2,0,0, + 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, + 83,0,41,2,122,42,85,115,101,32,100,101,102,97,117,108, + 116,32,115,101,109,97,110,116,105,99,115,32,102,111,114,32, + 109,111,100,117,108,101,32,99,114,101,97,116,105,111,110,46, + 78,114,4,0,0,0,41,2,114,104,0,0,0,114,162,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,218,13,99,114,101,97,116,101,95,109,111,100,117,108,101, + 151,2,0,0,115,0,0,0,0,122,27,95,76,111,97,100, + 101,114,66,97,115,105,99,115,46,99,114,101,97,116,101,95, + 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,3, + 0,0,0,4,0,0,0,67,0,0,0,115,56,0,0,0, + 124,0,106,0,124,1,106,1,131,1,125,2,124,2,100,1, + 107,8,114,36,116,2,100,2,106,3,124,1,106,1,131,1, + 131,1,130,1,116,4,106,5,116,6,124,2,124,1,106,7, + 131,3,1,0,100,1,83,0,41,3,122,19,69,120,101,99, + 117,116,101,32,116,104,101,32,109,111,100,117,108,101,46,78, + 122,52,99,97,110,110,111,116,32,108,111,97,100,32,109,111, + 100,117,108,101,32,123,33,114,125,32,119,104,101,110,32,103, + 101,116,95,99,111,100,101,40,41,32,114,101,116,117,114,110, + 115,32,78,111,110,101,41,8,218,8,103,101,116,95,99,111, + 100,101,114,109,0,0,0,114,103,0,0,0,114,50,0,0, + 0,114,118,0,0,0,218,25,95,99,97,108,108,95,119,105, + 116,104,95,102,114,97,109,101,115,95,114,101,109,111,118,101, + 100,218,4,101,120,101,99,114,115,0,0,0,41,3,114,104, + 0,0,0,218,6,109,111,100,117,108,101,114,144,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, + 11,101,120,101,99,95,109,111,100,117,108,101,154,2,0,0, + 115,10,0,0,0,0,2,12,1,8,1,6,1,10,1,122, + 25,95,76,111,97,100,101,114,66,97,115,105,99,115,46,101, + 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, + 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, + 12,0,0,0,116,0,106,1,124,0,124,1,131,2,83,0, + 41,1,122,26,84,104,105,115,32,109,111,100,117,108,101,32, + 105,115,32,100,101,112,114,101,99,97,116,101,100,46,41,2, + 114,118,0,0,0,218,17,95,108,111,97,100,95,109,111,100, + 117,108,101,95,115,104,105,109,41,2,114,104,0,0,0,114, + 123,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,218,11,108,111,97,100,95,109,111,100,117,108,101, + 162,2,0,0,115,2,0,0,0,0,2,122,25,95,76,111, + 97,100,101,114,66,97,115,105,99,115,46,108,111,97,100,95, + 109,111,100,117,108,101,78,41,8,114,109,0,0,0,114,108, + 0,0,0,114,110,0,0,0,114,111,0,0,0,114,157,0, + 0,0,114,183,0,0,0,114,188,0,0,0,114,190,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,181,0,0,0,138,2,0,0,115,10, + 0,0,0,8,3,4,2,8,8,8,3,8,8,114,181,0, + 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,3, + 0,0,0,64,0,0,0,115,74,0,0,0,101,0,90,1, + 100,0,90,2,100,1,100,2,132,0,90,3,100,3,100,4, + 132,0,90,4,100,5,100,6,132,0,90,5,100,7,100,8, + 132,0,90,6,100,9,100,10,132,0,90,7,100,18,100,12, + 156,1,100,13,100,14,132,2,90,8,100,15,100,16,132,0, + 90,9,100,17,83,0,41,19,218,12,83,111,117,114,99,101, + 76,111,97,100,101,114,99,2,0,0,0,0,0,0,0,2, + 0,0,0,1,0,0,0,67,0,0,0,115,8,0,0,0, + 116,0,130,1,100,1,83,0,41,2,122,178,79,112,116,105, + 111,110,97,108,32,109,101,116,104,111,100,32,116,104,97,116, + 32,114,101,116,117,114,110,115,32,116,104,101,32,109,111,100, + 105,102,105,99,97,116,105,111,110,32,116,105,109,101,32,40, + 97,110,32,105,110,116,41,32,102,111,114,32,116,104,101,10, + 32,32,32,32,32,32,32,32,115,112,101,99,105,102,105,101, + 100,32,112,97,116,104,44,32,119,104,101,114,101,32,112,97, + 116,104,32,105,115,32,97,32,115,116,114,46,10,10,32,32, + 32,32,32,32,32,32,82,97,105,115,101,115,32,73,79,69, + 114,114,111,114,32,119,104,101,110,32,116,104,101,32,112,97, + 116,104,32,99,97,110,110,111,116,32,98,101,32,104,97,110, + 100,108,101,100,46,10,32,32,32,32,32,32,32,32,78,41, + 1,218,7,73,79,69,114,114,111,114,41,2,114,104,0,0, + 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,218,10,112,97,116,104,95,109,116,105,109, + 101,169,2,0,0,115,2,0,0,0,0,6,122,23,83,111, + 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, + 109,116,105,109,101,99,2,0,0,0,0,0,0,0,2,0, + 0,0,3,0,0,0,67,0,0,0,115,14,0,0,0,100, + 1,124,0,106,0,124,1,131,1,105,1,83,0,41,2,97, + 170,1,0,0,79,112,116,105,111,110,97,108,32,109,101,116, + 104,111,100,32,114,101,116,117,114,110,105,110,103,32,97,32, + 109,101,116,97,100,97,116,97,32,100,105,99,116,32,102,111, + 114,32,116,104,101,32,115,112,101,99,105,102,105,101,100,32, + 112,97,116,104,10,32,32,32,32,32,32,32,32,116,111,32, + 98,121,32,116,104,101,32,112,97,116,104,32,40,115,116,114, + 41,46,10,32,32,32,32,32,32,32,32,80,111,115,115,105, + 98,108,101,32,107,101,121,115,58,10,32,32,32,32,32,32, + 32,32,45,32,39,109,116,105,109,101,39,32,40,109,97,110, + 100,97,116,111,114,121,41,32,105,115,32,116,104,101,32,110, + 117,109,101,114,105,99,32,116,105,109,101,115,116,97,109,112, + 32,111,102,32,108,97,115,116,32,115,111,117,114,99,101,10, + 32,32,32,32,32,32,32,32,32,32,99,111,100,101,32,109, + 111,100,105,102,105,99,97,116,105,111,110,59,10,32,32,32, + 32,32,32,32,32,45,32,39,115,105,122,101,39,32,40,111, + 112,116,105,111,110,97,108,41,32,105,115,32,116,104,101,32, + 115,105,122,101,32,105,110,32,98,121,116,101,115,32,111,102, + 32,116,104,101,32,115,111,117,114,99,101,32,99,111,100,101, + 46,10,10,32,32,32,32,32,32,32,32,73,109,112,108,101, + 109,101,110,116,105,110,103,32,116,104,105,115,32,109,101,116, + 104,111,100,32,97,108,108,111,119,115,32,116,104,101,32,108, + 111,97,100,101,114,32,116,111,32,114,101,97,100,32,98,121, + 116,101,99,111,100,101,32,102,105,108,101,115,46,10,32,32, + 32,32,32,32,32,32,82,97,105,115,101,115,32,73,79,69, + 114,114,111,114,32,119,104,101,110,32,116,104,101,32,112,97, + 116,104,32,99,97,110,110,111,116,32,98,101,32,104,97,110, + 100,108,101,100,46,10,32,32,32,32,32,32,32,32,114,130, + 0,0,0,41,1,114,193,0,0,0,41,2,114,104,0,0, + 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,218,10,112,97,116,104,95,115,116,97,116, + 115,177,2,0,0,115,2,0,0,0,0,11,122,23,83,111, + 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, + 115,116,97,116,115,99,4,0,0,0,0,0,0,0,4,0, + 0,0,3,0,0,0,67,0,0,0,115,12,0,0,0,124, + 0,106,0,124,2,124,3,131,2,83,0,41,1,122,228,79, + 112,116,105,111,110,97,108,32,109,101,116,104,111,100,32,119, + 104,105,99,104,32,119,114,105,116,101,115,32,100,97,116,97, + 32,40,98,121,116,101,115,41,32,116,111,32,97,32,102,105, + 108,101,32,112,97,116,104,32,40,97,32,115,116,114,41,46, + 10,10,32,32,32,32,32,32,32,32,73,109,112,108,101,109, + 101,110,116,105,110,103,32,116,104,105,115,32,109,101,116,104, + 111,100,32,97,108,108,111,119,115,32,102,111,114,32,116,104, + 101,32,119,114,105,116,105,110,103,32,111,102,32,98,121,116, + 101,99,111,100,101,32,102,105,108,101,115,46,10,10,32,32, + 32,32,32,32,32,32,84,104,101,32,115,111,117,114,99,101, + 32,112,97,116,104,32,105,115,32,110,101,101,100,101,100,32, + 105,110,32,111,114,100,101,114,32,116,111,32,99,111,114,114, + 101,99,116,108,121,32,116,114,97,110,115,102,101,114,32,112, + 101,114,109,105,115,115,105,111,110,115,10,32,32,32,32,32, + 32,32,32,41,1,218,8,115,101,116,95,100,97,116,97,41, + 4,114,104,0,0,0,114,94,0,0,0,90,10,99,97,99, + 104,101,95,112,97,116,104,114,56,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,15,95,99,97, + 99,104,101,95,98,121,116,101,99,111,100,101,190,2,0,0, + 115,2,0,0,0,0,8,122,28,83,111,117,114,99,101,76, + 111,97,100,101,114,46,95,99,97,99,104,101,95,98,121,116, + 101,99,111,100,101,99,3,0,0,0,0,0,0,0,3,0, + 0,0,1,0,0,0,67,0,0,0,115,4,0,0,0,100, + 1,83,0,41,2,122,150,79,112,116,105,111,110,97,108,32, + 109,101,116,104,111,100,32,119,104,105,99,104,32,119,114,105, + 116,101,115,32,100,97,116,97,32,40,98,121,116,101,115,41, + 32,116,111,32,97,32,102,105,108,101,32,112,97,116,104,32, + 40,97,32,115,116,114,41,46,10,10,32,32,32,32,32,32, + 32,32,73,109,112,108,101,109,101,110,116,105,110,103,32,116, + 104,105,115,32,109,101,116,104,111,100,32,97,108,108,111,119, + 115,32,102,111,114,32,116,104,101,32,119,114,105,116,105,110, + 103,32,111,102,32,98,121,116,101,99,111,100,101,32,102,105, + 108,101,115,46,10,32,32,32,32,32,32,32,32,78,114,4, + 0,0,0,41,3,114,104,0,0,0,114,37,0,0,0,114, + 56,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,114,195,0,0,0,200,2,0,0,115,0,0,0, + 0,122,21,83,111,117,114,99,101,76,111,97,100,101,114,46, + 115,101,116,95,100,97,116,97,99,2,0,0,0,0,0,0, + 0,5,0,0,0,16,0,0,0,67,0,0,0,115,84,0, + 0,0,124,0,106,0,124,1,131,1,125,2,121,14,124,0, + 106,1,124,2,131,1,125,3,87,0,110,50,4,0,116,2, + 107,10,114,74,1,0,125,4,1,0,122,22,116,3,100,1, + 100,2,124,1,144,1,131,1,124,4,130,2,87,0,89,0, + 100,3,100,3,125,4,126,4,88,0,110,2,88,0,116,4, + 124,3,131,1,83,0,41,4,122,52,67,111,110,99,114,101, + 116,101,32,105,109,112,108,101,109,101,110,116,97,116,105,111, + 110,32,111,102,32,73,110,115,112,101,99,116,76,111,97,100, + 101,114,46,103,101,116,95,115,111,117,114,99,101,46,122,39, + 115,111,117,114,99,101,32,110,111,116,32,97,118,97,105,108, + 97,98,108,101,32,116,104,114,111,117,103,104,32,103,101,116, + 95,100,97,116,97,40,41,114,102,0,0,0,78,41,5,114, + 155,0,0,0,218,8,103,101,116,95,100,97,116,97,114,42, + 0,0,0,114,103,0,0,0,114,153,0,0,0,41,5,114, + 104,0,0,0,114,123,0,0,0,114,37,0,0,0,114,151, + 0,0,0,218,3,101,120,99,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,218,10,103,101,116,95,115,111,117, + 114,99,101,207,2,0,0,115,14,0,0,0,0,2,10,1, + 2,1,14,1,16,1,6,1,28,1,122,23,83,111,117,114, + 99,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, + 114,99,101,114,31,0,0,0,41,1,218,9,95,111,112,116, + 105,109,105,122,101,99,3,0,0,0,1,0,0,0,4,0, + 0,0,9,0,0,0,67,0,0,0,115,26,0,0,0,116, + 0,106,1,116,2,124,1,124,2,100,1,100,2,100,3,100, + 4,124,3,144,2,131,4,83,0,41,5,122,130,82,101,116, + 117,114,110,32,116,104,101,32,99,111,100,101,32,111,98,106, + 101,99,116,32,99,111,109,112,105,108,101,100,32,102,114,111, + 109,32,115,111,117,114,99,101,46,10,10,32,32,32,32,32, + 32,32,32,84,104,101,32,39,100,97,116,97,39,32,97,114, + 103,117,109,101,110,116,32,99,97,110,32,98,101,32,97,110, + 121,32,111,98,106,101,99,116,32,116,121,112,101,32,116,104, + 97,116,32,99,111,109,112,105,108,101,40,41,32,115,117,112, + 112,111,114,116,115,46,10,32,32,32,32,32,32,32,32,114, + 186,0,0,0,218,12,100,111,110,116,95,105,110,104,101,114, + 105,116,84,114,72,0,0,0,41,3,114,118,0,0,0,114, + 185,0,0,0,218,7,99,111,109,112,105,108,101,41,4,114, + 104,0,0,0,114,56,0,0,0,114,37,0,0,0,114,200, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,14,115,111,117,114,99,101,95,116,111,95,99,111, + 100,101,217,2,0,0,115,4,0,0,0,0,5,14,1,122, + 27,83,111,117,114,99,101,76,111,97,100,101,114,46,115,111, + 117,114,99,101,95,116,111,95,99,111,100,101,99,2,0,0, + 0,0,0,0,0,10,0,0,0,43,0,0,0,67,0,0, + 0,115,106,1,0,0,124,0,106,0,124,1,131,1,125,2, + 100,1,125,3,121,12,116,1,124,2,131,1,125,4,87,0, + 110,24,4,0,116,2,107,10,114,50,1,0,1,0,1,0, + 100,1,125,4,89,0,110,174,88,0,121,14,124,0,106,3, + 124,2,131,1,125,5,87,0,110,20,4,0,116,4,107,10, + 114,86,1,0,1,0,1,0,89,0,110,138,88,0,116,5, + 124,5,100,2,25,0,131,1,125,3,121,14,124,0,106,6, + 124,4,131,1,125,6,87,0,110,20,4,0,116,7,107,10, + 114,134,1,0,1,0,1,0,89,0,110,90,88,0,121,26, + 116,8,124,6,100,3,124,5,100,4,124,1,100,5,124,4, + 144,3,131,1,125,7,87,0,110,24,4,0,116,9,116,10, + 102,2,107,10,114,186,1,0,1,0,1,0,89,0,110,38, + 88,0,116,11,106,12,100,6,124,4,124,2,131,3,1,0, + 116,13,124,7,100,4,124,1,100,7,124,4,100,8,124,2, + 144,3,131,1,83,0,124,0,106,6,124,2,131,1,125,8, + 124,0,106,14,124,8,124,2,131,2,125,9,116,11,106,12, + 100,9,124,2,131,2,1,0,116,15,106,16,12,0,144,1, + 114,102,124,4,100,1,107,9,144,1,114,102,124,3,100,1, + 107,9,144,1,114,102,116,17,124,9,124,3,116,18,124,8, + 131,1,131,3,125,6,121,30,124,0,106,19,124,2,124,4, + 124,6,131,3,1,0,116,11,106,12,100,10,124,4,131,2, + 1,0,87,0,110,22,4,0,116,2,107,10,144,1,114,100, + 1,0,1,0,1,0,89,0,110,2,88,0,124,9,83,0, + 41,11,122,190,67,111,110,99,114,101,116,101,32,105,109,112, + 108,101,109,101,110,116,97,116,105,111,110,32,111,102,32,73, + 110,115,112,101,99,116,76,111,97,100,101,114,46,103,101,116, + 95,99,111,100,101,46,10,10,32,32,32,32,32,32,32,32, + 82,101,97,100,105,110,103,32,111,102,32,98,121,116,101,99, + 111,100,101,32,114,101,113,117,105,114,101,115,32,112,97,116, + 104,95,115,116,97,116,115,32,116,111,32,98,101,32,105,109, + 112,108,101,109,101,110,116,101,100,46,32,84,111,32,119,114, + 105,116,101,10,32,32,32,32,32,32,32,32,98,121,116,101, + 99,111,100,101,44,32,115,101,116,95,100,97,116,97,32,109, + 117,115,116,32,97,108,115,111,32,98,101,32,105,109,112,108, + 101,109,101,110,116,101,100,46,10,10,32,32,32,32,32,32, + 32,32,78,114,130,0,0,0,114,136,0,0,0,114,102,0, + 0,0,114,37,0,0,0,122,13,123,125,32,109,97,116,99, + 104,101,115,32,123,125,114,93,0,0,0,114,94,0,0,0, + 122,19,99,111,100,101,32,111,98,106,101,99,116,32,102,114, + 111,109,32,123,125,122,10,119,114,111,116,101,32,123,33,114, + 125,41,20,114,155,0,0,0,114,83,0,0,0,114,70,0, + 0,0,114,194,0,0,0,114,192,0,0,0,114,16,0,0, + 0,114,197,0,0,0,114,42,0,0,0,114,139,0,0,0, + 114,103,0,0,0,114,134,0,0,0,114,118,0,0,0,114, + 133,0,0,0,114,145,0,0,0,114,203,0,0,0,114,8, + 0,0,0,218,19,100,111,110,116,95,119,114,105,116,101,95, + 98,121,116,101,99,111,100,101,114,148,0,0,0,114,33,0, + 0,0,114,196,0,0,0,41,10,114,104,0,0,0,114,123, + 0,0,0,114,94,0,0,0,114,137,0,0,0,114,93,0, + 0,0,218,2,115,116,114,56,0,0,0,218,10,98,121,116, + 101,115,95,100,97,116,97,114,151,0,0,0,90,11,99,111, + 100,101,95,111,98,106,101,99,116,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,184,0,0,0,225,2,0, + 0,115,78,0,0,0,0,7,10,1,4,1,2,1,12,1, + 14,1,10,2,2,1,14,1,14,1,6,2,12,1,2,1, + 14,1,14,1,6,2,2,1,6,1,8,1,12,1,18,1, + 6,2,8,1,6,1,10,1,4,1,8,1,10,1,12,1, + 12,1,20,1,10,1,6,1,10,1,2,1,14,1,16,1, + 16,1,6,1,122,21,83,111,117,114,99,101,76,111,97,100, + 101,114,46,103,101,116,95,99,111,100,101,78,114,91,0,0, + 0,41,10,114,109,0,0,0,114,108,0,0,0,114,110,0, + 0,0,114,193,0,0,0,114,194,0,0,0,114,196,0,0, + 0,114,195,0,0,0,114,199,0,0,0,114,203,0,0,0, + 114,184,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,191,0,0,0,167,2, + 0,0,115,14,0,0,0,8,2,8,8,8,13,8,10,8, + 7,8,10,14,8,114,191,0,0,0,99,0,0,0,0,0, + 0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,115, + 76,0,0,0,101,0,90,1,100,0,90,2,100,1,90,3, + 100,2,100,3,132,0,90,4,100,4,100,5,132,0,90,5, + 100,6,100,7,132,0,90,6,101,7,135,0,102,1,100,8, + 100,9,132,8,131,1,90,8,101,7,100,10,100,11,132,0, + 131,1,90,9,100,12,100,13,132,0,90,10,135,0,83,0, + 41,14,218,10,70,105,108,101,76,111,97,100,101,114,122,103, + 66,97,115,101,32,102,105,108,101,32,108,111,97,100,101,114, + 32,99,108,97,115,115,32,119,104,105,99,104,32,105,109,112, + 108,101,109,101,110,116,115,32,116,104,101,32,108,111,97,100, + 101,114,32,112,114,111,116,111,99,111,108,32,109,101,116,104, + 111,100,115,32,116,104,97,116,10,32,32,32,32,114,101,113, + 117,105,114,101,32,102,105,108,101,32,115,121,115,116,101,109, + 32,117,115,97,103,101,46,99,3,0,0,0,0,0,0,0, + 3,0,0,0,2,0,0,0,67,0,0,0,115,16,0,0, + 0,124,1,124,0,95,0,124,2,124,0,95,1,100,1,83, + 0,41,2,122,75,67,97,99,104,101,32,116,104,101,32,109, + 111,100,117,108,101,32,110,97,109,101,32,97,110,100,32,116, + 104,101,32,112,97,116,104,32,116,111,32,116,104,101,32,102, + 105,108,101,32,102,111,117,110,100,32,98,121,32,116,104,101, + 10,32,32,32,32,32,32,32,32,102,105,110,100,101,114,46, + 78,41,2,114,102,0,0,0,114,37,0,0,0,41,3,114, + 104,0,0,0,114,123,0,0,0,114,37,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,182,0, + 0,0,26,3,0,0,115,4,0,0,0,0,3,6,1,122, + 19,70,105,108,101,76,111,97,100,101,114,46,95,95,105,110, + 105,116,95,95,99,2,0,0,0,0,0,0,0,2,0,0, + 0,2,0,0,0,67,0,0,0,115,24,0,0,0,124,0, + 106,0,124,1,106,0,107,2,111,22,124,0,106,1,124,1, + 106,1,107,2,83,0,41,1,78,41,2,218,9,95,95,99, + 108,97,115,115,95,95,114,115,0,0,0,41,2,114,104,0, + 0,0,218,5,111,116,104,101,114,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,218,6,95,95,101,113,95,95, + 32,3,0,0,115,4,0,0,0,0,1,12,1,122,17,70, + 105,108,101,76,111,97,100,101,114,46,95,95,101,113,95,95, + 99,1,0,0,0,0,0,0,0,1,0,0,0,3,0,0, + 0,67,0,0,0,115,20,0,0,0,116,0,124,0,106,1, + 131,1,116,0,124,0,106,2,131,1,65,0,83,0,41,1, + 78,41,3,218,4,104,97,115,104,114,102,0,0,0,114,37, + 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,218,8,95,95,104,97,115, + 104,95,95,36,3,0,0,115,2,0,0,0,0,1,122,19, + 70,105,108,101,76,111,97,100,101,114,46,95,95,104,97,115, + 104,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, + 3,0,0,0,3,0,0,0,115,16,0,0,0,116,0,116, + 1,124,0,131,2,106,2,124,1,131,1,83,0,41,1,122, + 100,76,111,97,100,32,97,32,109,111,100,117,108,101,32,102, + 114,111,109,32,97,32,102,105,108,101,46,10,10,32,32,32, + 32,32,32,32,32,84,104,105,115,32,109,101,116,104,111,100, + 32,105,115,32,100,101,112,114,101,99,97,116,101,100,46,32, + 32,85,115,101,32,101,120,101,99,95,109,111,100,117,108,101, + 40,41,32,105,110,115,116,101,97,100,46,10,10,32,32,32, + 32,32,32,32,32,41,3,218,5,115,117,112,101,114,114,207, + 0,0,0,114,190,0,0,0,41,2,114,104,0,0,0,114, + 123,0,0,0,41,1,114,208,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,190,0,0,0,39,3,0,0,115,2, + 0,0,0,0,10,122,22,70,105,108,101,76,111,97,100,101, + 114,46,108,111,97,100,95,109,111,100,117,108,101,99,2,0, + 0,0,0,0,0,0,2,0,0,0,1,0,0,0,67,0, + 0,0,115,6,0,0,0,124,0,106,0,83,0,41,1,122, + 58,82,101,116,117,114,110,32,116,104,101,32,112,97,116,104, + 32,116,111,32,116,104,101,32,115,111,117,114,99,101,32,102, + 105,108,101,32,97,115,32,102,111,117,110,100,32,98,121,32, + 116,104,101,32,102,105,110,100,101,114,46,41,1,114,37,0, + 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,155,0, + 0,0,51,3,0,0,115,2,0,0,0,0,3,122,23,70, + 105,108,101,76,111,97,100,101,114,46,103,101,116,95,102,105, + 108,101,110,97,109,101,99,2,0,0,0,0,0,0,0,3, + 0,0,0,9,0,0,0,67,0,0,0,115,32,0,0,0, + 116,0,106,1,124,1,100,1,131,2,143,10,125,2,124,2, + 106,2,131,0,83,0,81,0,82,0,88,0,100,2,83,0, + 41,3,122,39,82,101,116,117,114,110,32,116,104,101,32,100, + 97,116,97,32,102,114,111,109,32,112,97,116,104,32,97,115, + 32,114,97,119,32,98,121,116,101,115,46,218,1,114,78,41, + 3,114,52,0,0,0,114,53,0,0,0,90,4,114,101,97, + 100,41,3,114,104,0,0,0,114,37,0,0,0,114,57,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,197,0,0,0,56,3,0,0,115,4,0,0,0,0, + 2,14,1,122,19,70,105,108,101,76,111,97,100,101,114,46, + 103,101,116,95,100,97,116,97,41,11,114,109,0,0,0,114, + 108,0,0,0,114,110,0,0,0,114,111,0,0,0,114,182, + 0,0,0,114,210,0,0,0,114,212,0,0,0,114,120,0, + 0,0,114,190,0,0,0,114,155,0,0,0,114,197,0,0, + 0,114,4,0,0,0,114,4,0,0,0,41,1,114,208,0, + 0,0,114,6,0,0,0,114,207,0,0,0,21,3,0,0, + 115,14,0,0,0,8,3,4,2,8,6,8,4,8,3,16, + 12,12,5,114,207,0,0,0,99,0,0,0,0,0,0,0, + 0,0,0,0,0,3,0,0,0,64,0,0,0,115,46,0, + 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, + 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, + 100,7,156,1,100,8,100,9,132,2,90,6,100,10,83,0, + 41,11,218,16,83,111,117,114,99,101,70,105,108,101,76,111, + 97,100,101,114,122,62,67,111,110,99,114,101,116,101,32,105, + 109,112,108,101,109,101,110,116,97,116,105,111,110,32,111,102, + 32,83,111,117,114,99,101,76,111,97,100,101,114,32,117,115, + 105,110,103,32,116,104,101,32,102,105,108,101,32,115,121,115, + 116,101,109,46,99,2,0,0,0,0,0,0,0,3,0,0, + 0,3,0,0,0,67,0,0,0,115,22,0,0,0,116,0, + 124,1,131,1,125,2,124,2,106,1,124,2,106,2,100,1, + 156,2,83,0,41,2,122,33,82,101,116,117,114,110,32,116, + 104,101,32,109,101,116,97,100,97,116,97,32,102,111,114,32, + 116,104,101,32,112,97,116,104,46,41,2,122,5,109,116,105, + 109,101,122,4,115,105,122,101,41,3,114,41,0,0,0,218, + 8,115,116,95,109,116,105,109,101,90,7,115,116,95,115,105, + 122,101,41,3,114,104,0,0,0,114,37,0,0,0,114,205, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,114,194,0,0,0,66,3,0,0,115,4,0,0,0, + 0,2,8,1,122,27,83,111,117,114,99,101,70,105,108,101, + 76,111,97,100,101,114,46,112,97,116,104,95,115,116,97,116, + 115,99,4,0,0,0,0,0,0,0,5,0,0,0,5,0, + 0,0,67,0,0,0,115,26,0,0,0,116,0,124,1,131, + 1,125,4,124,0,106,1,124,2,124,3,100,1,124,4,144, + 1,131,2,83,0,41,2,78,218,5,95,109,111,100,101,41, + 2,114,101,0,0,0,114,195,0,0,0,41,5,114,104,0, + 0,0,114,94,0,0,0,114,93,0,0,0,114,56,0,0, + 0,114,44,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,196,0,0,0,71,3,0,0,115,4, + 0,0,0,0,2,8,1,122,32,83,111,117,114,99,101,70, + 105,108,101,76,111,97,100,101,114,46,95,99,97,99,104,101, + 95,98,121,116,101,99,111,100,101,105,182,1,0,0,41,1, + 114,217,0,0,0,99,3,0,0,0,1,0,0,0,9,0, + 0,0,17,0,0,0,67,0,0,0,115,250,0,0,0,116, + 0,124,1,131,1,92,2,125,4,125,5,103,0,125,6,120, + 40,124,4,114,56,116,1,124,4,131,1,12,0,114,56,116, + 0,124,4,131,1,92,2,125,4,125,7,124,6,106,2,124, + 7,131,1,1,0,113,18,87,0,120,108,116,3,124,6,131, + 1,68,0,93,96,125,7,116,4,124,4,124,7,131,2,125, + 4,121,14,116,5,106,6,124,4,131,1,1,0,87,0,113, + 68,4,0,116,7,107,10,114,118,1,0,1,0,1,0,119, + 68,89,0,113,68,4,0,116,8,107,10,114,162,1,0,125, + 8,1,0,122,18,116,9,106,10,100,1,124,4,124,8,131, + 3,1,0,100,2,83,0,100,2,125,8,126,8,88,0,113, + 68,88,0,113,68,87,0,121,28,116,11,124,1,124,2,124, + 3,131,3,1,0,116,9,106,10,100,3,124,1,131,2,1, + 0,87,0,110,48,4,0,116,8,107,10,114,244,1,0,125, + 8,1,0,122,20,116,9,106,10,100,1,124,1,124,8,131, + 3,1,0,87,0,89,0,100,2,100,2,125,8,126,8,88, + 0,110,2,88,0,100,2,83,0,41,4,122,27,87,114,105, + 116,101,32,98,121,116,101,115,32,100,97,116,97,32,116,111, + 32,97,32,102,105,108,101,46,122,27,99,111,117,108,100,32, + 110,111,116,32,99,114,101,97,116,101,32,123,33,114,125,58, + 32,123,33,114,125,78,122,12,99,114,101,97,116,101,100,32, + 123,33,114,125,41,12,114,40,0,0,0,114,48,0,0,0, + 114,161,0,0,0,114,35,0,0,0,114,30,0,0,0,114, + 3,0,0,0,90,5,109,107,100,105,114,218,15,70,105,108, + 101,69,120,105,115,116,115,69,114,114,111,114,114,42,0,0, + 0,114,118,0,0,0,114,133,0,0,0,114,58,0,0,0, + 41,9,114,104,0,0,0,114,37,0,0,0,114,56,0,0, + 0,114,217,0,0,0,218,6,112,97,114,101,110,116,114,98, + 0,0,0,114,29,0,0,0,114,25,0,0,0,114,198,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,195,0,0,0,76,3,0,0,115,42,0,0,0,0, + 2,12,1,4,2,16,1,12,1,14,2,14,1,10,1,2, + 1,14,1,14,2,6,1,16,3,6,1,8,1,20,1,2, + 1,12,1,16,1,16,2,8,1,122,25,83,111,117,114,99, + 101,70,105,108,101,76,111,97,100,101,114,46,115,101,116,95, + 100,97,116,97,78,41,7,114,109,0,0,0,114,108,0,0, + 0,114,110,0,0,0,114,111,0,0,0,114,194,0,0,0, + 114,196,0,0,0,114,195,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,215, + 0,0,0,62,3,0,0,115,8,0,0,0,8,2,4,2, + 8,5,8,5,114,215,0,0,0,99,0,0,0,0,0,0, + 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,32, + 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, + 2,100,3,132,0,90,4,100,4,100,5,132,0,90,5,100, + 6,83,0,41,7,218,20,83,111,117,114,99,101,108,101,115, + 115,70,105,108,101,76,111,97,100,101,114,122,45,76,111,97, + 100,101,114,32,119,104,105,99,104,32,104,97,110,100,108,101, + 115,32,115,111,117,114,99,101,108,101,115,115,32,102,105,108, + 101,32,105,109,112,111,114,116,115,46,99,2,0,0,0,0, + 0,0,0,5,0,0,0,6,0,0,0,67,0,0,0,115, + 56,0,0,0,124,0,106,0,124,1,131,1,125,2,124,0, + 106,1,124,2,131,1,125,3,116,2,124,3,100,1,124,1, + 100,2,124,2,144,2,131,1,125,4,116,3,124,4,100,1, + 124,1,100,3,124,2,144,2,131,1,83,0,41,4,78,114, + 102,0,0,0,114,37,0,0,0,114,93,0,0,0,41,4, + 114,155,0,0,0,114,197,0,0,0,114,139,0,0,0,114, + 145,0,0,0,41,5,114,104,0,0,0,114,123,0,0,0, + 114,37,0,0,0,114,56,0,0,0,114,206,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,184, + 0,0,0,111,3,0,0,115,8,0,0,0,0,1,10,1, + 10,1,18,1,122,29,83,111,117,114,99,101,108,101,115,115, + 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, + 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, + 0,41,2,122,39,82,101,116,117,114,110,32,78,111,110,101, + 32,97,115,32,116,104,101,114,101,32,105,115,32,110,111,32, + 115,111,117,114,99,101,32,99,111,100,101,46,78,114,4,0, + 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, + 0,0,117,3,0,0,115,2,0,0,0,0,2,122,31,83, + 111,117,114,99,101,108,101,115,115,70,105,108,101,76,111,97, + 100,101,114,46,103,101,116,95,115,111,117,114,99,101,78,41, + 6,114,109,0,0,0,114,108,0,0,0,114,110,0,0,0, + 114,111,0,0,0,114,184,0,0,0,114,199,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,114,220,0,0,0,107,3,0,0,115,6,0,0, + 0,8,2,4,2,8,6,114,220,0,0,0,99,0,0,0, + 0,0,0,0,0,0,0,0,0,3,0,0,0,64,0,0, + 0,115,92,0,0,0,101,0,90,1,100,0,90,2,100,1, + 90,3,100,2,100,3,132,0,90,4,100,4,100,5,132,0, + 90,5,100,6,100,7,132,0,90,6,100,8,100,9,132,0, + 90,7,100,10,100,11,132,0,90,8,100,12,100,13,132,0, + 90,9,100,14,100,15,132,0,90,10,100,16,100,17,132,0, + 90,11,101,12,100,18,100,19,132,0,131,1,90,13,100,20, + 83,0,41,21,218,19,69,120,116,101,110,115,105,111,110,70, + 105,108,101,76,111,97,100,101,114,122,93,76,111,97,100,101, + 114,32,102,111,114,32,101,120,116,101,110,115,105,111,110,32, + 109,111,100,117,108,101,115,46,10,10,32,32,32,32,84,104, + 101,32,99,111,110,115,116,114,117,99,116,111,114,32,105,115, + 32,100,101,115,105,103,110,101,100,32,116,111,32,119,111,114, + 107,32,119,105,116,104,32,70,105,108,101,70,105,110,100,101, + 114,46,10,10,32,32,32,32,99,3,0,0,0,0,0,0, + 0,3,0,0,0,2,0,0,0,67,0,0,0,115,16,0, + 0,0,124,1,124,0,95,0,124,2,124,0,95,1,100,0, + 83,0,41,1,78,41,2,114,102,0,0,0,114,37,0,0, + 0,41,3,114,104,0,0,0,114,102,0,0,0,114,37,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,182,0,0,0,134,3,0,0,115,4,0,0,0,0, + 1,6,1,122,28,69,120,116,101,110,115,105,111,110,70,105, + 108,101,76,111,97,100,101,114,46,95,95,105,110,105,116,95, + 95,99,2,0,0,0,0,0,0,0,2,0,0,0,2,0, + 0,0,67,0,0,0,115,24,0,0,0,124,0,106,0,124, + 1,106,0,107,2,111,22,124,0,106,1,124,1,106,1,107, + 2,83,0,41,1,78,41,2,114,208,0,0,0,114,115,0, + 0,0,41,2,114,104,0,0,0,114,209,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,210,0, + 0,0,138,3,0,0,115,4,0,0,0,0,1,12,1,122, + 26,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, + 97,100,101,114,46,95,95,101,113,95,95,99,1,0,0,0, + 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, + 115,20,0,0,0,116,0,124,0,106,1,131,1,116,0,124, + 0,106,2,131,1,65,0,83,0,41,1,78,41,3,114,211, + 0,0,0,114,102,0,0,0,114,37,0,0,0,41,1,114, + 104,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,114,212,0,0,0,142,3,0,0,115,2,0,0, + 0,0,1,122,28,69,120,116,101,110,115,105,111,110,70,105, + 108,101,76,111,97,100,101,114,46,95,95,104,97,115,104,95, + 95,99,2,0,0,0,0,0,0,0,3,0,0,0,4,0, + 0,0,67,0,0,0,115,36,0,0,0,116,0,106,1,116, + 2,106,3,124,1,131,2,125,2,116,0,106,4,100,1,124, + 1,106,5,124,0,106,6,131,3,1,0,124,2,83,0,41, + 2,122,38,67,114,101,97,116,101,32,97,110,32,117,110,105, + 116,105,97,108,105,122,101,100,32,101,120,116,101,110,115,105, + 111,110,32,109,111,100,117,108,101,122,38,101,120,116,101,110, + 115,105,111,110,32,109,111,100,117,108,101,32,123,33,114,125, + 32,108,111,97,100,101,100,32,102,114,111,109,32,123,33,114, + 125,41,7,114,118,0,0,0,114,185,0,0,0,114,143,0, + 0,0,90,14,99,114,101,97,116,101,95,100,121,110,97,109, + 105,99,114,133,0,0,0,114,102,0,0,0,114,37,0,0, + 0,41,3,114,104,0,0,0,114,162,0,0,0,114,187,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,183,0,0,0,145,3,0,0,115,10,0,0,0,0, + 2,4,1,10,1,6,1,12,1,122,33,69,120,116,101,110, + 115,105,111,110,70,105,108,101,76,111,97,100,101,114,46,99, + 114,101,97,116,101,95,109,111,100,117,108,101,99,2,0,0, + 0,0,0,0,0,2,0,0,0,4,0,0,0,67,0,0, + 0,115,36,0,0,0,116,0,106,1,116,2,106,3,124,1, + 131,2,1,0,116,0,106,4,100,1,124,0,106,5,124,0, + 106,6,131,3,1,0,100,2,83,0,41,3,122,30,73,110, + 105,116,105,97,108,105,122,101,32,97,110,32,101,120,116,101, + 110,115,105,111,110,32,109,111,100,117,108,101,122,40,101,120, + 116,101,110,115,105,111,110,32,109,111,100,117,108,101,32,123, + 33,114,125,32,101,120,101,99,117,116,101,100,32,102,114,111, + 109,32,123,33,114,125,78,41,7,114,118,0,0,0,114,185, + 0,0,0,114,143,0,0,0,90,12,101,120,101,99,95,100, + 121,110,97,109,105,99,114,133,0,0,0,114,102,0,0,0, + 114,37,0,0,0,41,2,114,104,0,0,0,114,187,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, + 114,188,0,0,0,153,3,0,0,115,6,0,0,0,0,2, + 14,1,6,1,122,31,69,120,116,101,110,115,105,111,110,70, + 105,108,101,76,111,97,100,101,114,46,101,120,101,99,95,109, + 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, + 0,0,4,0,0,0,3,0,0,0,115,36,0,0,0,116, + 0,124,0,106,1,131,1,100,1,25,0,137,0,116,2,135, + 0,102,1,100,2,100,3,132,8,116,3,68,0,131,1,131, + 1,83,0,41,4,122,49,82,101,116,117,114,110,32,84,114, + 117,101,32,105,102,32,116,104,101,32,101,120,116,101,110,115, + 105,111,110,32,109,111,100,117,108,101,32,105,115,32,97,32, + 112,97,99,107,97,103,101,46,114,31,0,0,0,99,1,0, + 0,0,0,0,0,0,2,0,0,0,4,0,0,0,51,0, + 0,0,115,26,0,0,0,124,0,93,18,125,1,136,0,100, + 0,124,1,23,0,107,2,86,0,1,0,113,2,100,1,83, + 0,41,2,114,182,0,0,0,78,114,4,0,0,0,41,2, + 114,24,0,0,0,218,6,115,117,102,102,105,120,41,1,218, + 9,102,105,108,101,95,110,97,109,101,114,4,0,0,0,114, + 6,0,0,0,250,9,60,103,101,110,101,120,112,114,62,162, + 3,0,0,115,2,0,0,0,4,1,122,49,69,120,116,101, + 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, + 105,115,95,112,97,99,107,97,103,101,46,60,108,111,99,97, + 108,115,62,46,60,103,101,110,101,120,112,114,62,41,4,114, + 40,0,0,0,114,37,0,0,0,218,3,97,110,121,218,18, + 69,88,84,69,78,83,73,79,78,95,83,85,70,70,73,88, + 69,83,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,41,1,114,223,0,0,0,114,6,0,0,0,114, + 157,0,0,0,159,3,0,0,115,6,0,0,0,0,2,14, + 1,12,1,122,30,69,120,116,101,110,115,105,111,110,70,105, + 108,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, + 97,103,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, + 0,41,2,122,63,82,101,116,117,114,110,32,78,111,110,101, + 32,97,115,32,97,110,32,101,120,116,101,110,115,105,111,110, + 32,109,111,100,117,108,101,32,99,97,110,110,111,116,32,99, + 114,101,97,116,101,32,97,32,99,111,100,101,32,111,98,106, + 101,99,116,46,78,114,4,0,0,0,41,2,114,104,0,0, + 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,184,0,0,0,165,3,0,0,115,2, + 0,0,0,0,2,122,28,69,120,116,101,110,115,105,111,110, + 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, + 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, + 0,41,2,122,53,82,101,116,117,114,110,32,78,111,110,101, + 32,97,115,32,101,120,116,101,110,115,105,111,110,32,109,111, + 100,117,108,101,115,32,104,97,118,101,32,110,111,32,115,111, + 117,114,99,101,32,99,111,100,101,46,78,114,4,0,0,0, + 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,199,0,0,0, + 169,3,0,0,115,2,0,0,0,0,2,122,30,69,120,116, + 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, + 46,103,101,116,95,115,111,117,114,99,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, + 115,6,0,0,0,124,0,106,0,83,0,41,1,122,58,82, + 101,116,117,114,110,32,116,104,101,32,112,97,116,104,32,116, + 111,32,116,104,101,32,115,111,117,114,99,101,32,102,105,108, + 101,32,97,115,32,102,111,117,110,100,32,98,121,32,116,104, + 101,32,102,105,110,100,101,114,46,41,1,114,37,0,0,0, + 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,155,0,0,0, + 173,3,0,0,115,2,0,0,0,0,3,122,32,69,120,116, + 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, + 46,103,101,116,95,102,105,108,101,110,97,109,101,78,41,14, + 114,109,0,0,0,114,108,0,0,0,114,110,0,0,0,114, + 111,0,0,0,114,182,0,0,0,114,210,0,0,0,114,212, + 0,0,0,114,183,0,0,0,114,188,0,0,0,114,157,0, + 0,0,114,184,0,0,0,114,199,0,0,0,114,120,0,0, + 0,114,155,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,114,221,0,0,0,126, + 3,0,0,115,20,0,0,0,8,6,4,2,8,4,8,4, + 8,3,8,8,8,6,8,6,8,4,8,4,114,221,0,0, + 0,99,0,0,0,0,0,0,0,0,0,0,0,0,2,0, + 0,0,64,0,0,0,115,96,0,0,0,101,0,90,1,100, + 0,90,2,100,1,90,3,100,2,100,3,132,0,90,4,100, + 4,100,5,132,0,90,5,100,6,100,7,132,0,90,6,100, + 8,100,9,132,0,90,7,100,10,100,11,132,0,90,8,100, + 12,100,13,132,0,90,9,100,14,100,15,132,0,90,10,100, + 16,100,17,132,0,90,11,100,18,100,19,132,0,90,12,100, + 20,100,21,132,0,90,13,100,22,83,0,41,23,218,14,95, + 78,97,109,101,115,112,97,99,101,80,97,116,104,97,38,1, + 0,0,82,101,112,114,101,115,101,110,116,115,32,97,32,110, + 97,109,101,115,112,97,99,101,32,112,97,99,107,97,103,101, + 39,115,32,112,97,116,104,46,32,32,73,116,32,117,115,101, + 115,32,116,104,101,32,109,111,100,117,108,101,32,110,97,109, + 101,10,32,32,32,32,116,111,32,102,105,110,100,32,105,116, + 115,32,112,97,114,101,110,116,32,109,111,100,117,108,101,44, + 32,97,110,100,32,102,114,111,109,32,116,104,101,114,101,32, + 105,116,32,108,111,111,107,115,32,117,112,32,116,104,101,32, + 112,97,114,101,110,116,39,115,10,32,32,32,32,95,95,112, + 97,116,104,95,95,46,32,32,87,104,101,110,32,116,104,105, + 115,32,99,104,97,110,103,101,115,44,32,116,104,101,32,109, + 111,100,117,108,101,39,115,32,111,119,110,32,112,97,116,104, + 32,105,115,32,114,101,99,111,109,112,117,116,101,100,44,10, + 32,32,32,32,117,115,105,110,103,32,112,97,116,104,95,102, + 105,110,100,101,114,46,32,32,70,111,114,32,116,111,112,45, + 108,101,118,101,108,32,109,111,100,117,108,101,115,44,32,116, + 104,101,32,112,97,114,101,110,116,32,109,111,100,117,108,101, + 39,115,32,112,97,116,104,10,32,32,32,32,105,115,32,115, + 121,115,46,112,97,116,104,46,99,4,0,0,0,0,0,0, + 0,4,0,0,0,2,0,0,0,67,0,0,0,115,36,0, + 0,0,124,1,124,0,95,0,124,2,124,0,95,1,116,2, + 124,0,106,3,131,0,131,1,124,0,95,4,124,3,124,0, + 95,5,100,0,83,0,41,1,78,41,6,218,5,95,110,97, + 109,101,218,5,95,112,97,116,104,114,97,0,0,0,218,16, + 95,103,101,116,95,112,97,114,101,110,116,95,112,97,116,104, + 218,17,95,108,97,115,116,95,112,97,114,101,110,116,95,112, + 97,116,104,218,12,95,112,97,116,104,95,102,105,110,100,101, + 114,41,4,114,104,0,0,0,114,102,0,0,0,114,37,0, + 0,0,218,11,112,97,116,104,95,102,105,110,100,101,114,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,182, + 0,0,0,186,3,0,0,115,8,0,0,0,0,1,6,1, + 6,1,14,1,122,23,95,78,97,109,101,115,112,97,99,101, + 80,97,116,104,46,95,95,105,110,105,116,95,95,99,1,0, + 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, + 0,0,115,38,0,0,0,124,0,106,0,106,1,100,1,131, + 1,92,3,125,1,125,2,125,3,124,2,100,2,107,2,114, + 30,100,6,83,0,124,1,100,5,102,2,83,0,41,7,122, + 62,82,101,116,117,114,110,115,32,97,32,116,117,112,108,101, + 32,111,102,32,40,112,97,114,101,110,116,45,109,111,100,117, + 108,101,45,110,97,109,101,44,32,112,97,114,101,110,116,45, + 112,97,116,104,45,97,116,116,114,45,110,97,109,101,41,114, + 61,0,0,0,114,32,0,0,0,114,8,0,0,0,114,37, + 0,0,0,90,8,95,95,112,97,116,104,95,95,41,2,122, + 3,115,121,115,122,4,112,97,116,104,41,2,114,228,0,0, + 0,114,34,0,0,0,41,4,114,104,0,0,0,114,219,0, + 0,0,218,3,100,111,116,90,2,109,101,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,23,95,102,105,110, + 100,95,112,97,114,101,110,116,95,112,97,116,104,95,110,97, + 109,101,115,192,3,0,0,115,8,0,0,0,0,2,18,1, + 8,2,4,3,122,38,95,78,97,109,101,115,112,97,99,101, + 80,97,116,104,46,95,102,105,110,100,95,112,97,114,101,110, + 116,95,112,97,116,104,95,110,97,109,101,115,99,1,0,0, + 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, + 0,115,28,0,0,0,124,0,106,0,131,0,92,2,125,1, + 125,2,116,1,116,2,106,3,124,1,25,0,124,2,131,2, + 83,0,41,1,78,41,4,114,235,0,0,0,114,114,0,0, + 0,114,8,0,0,0,218,7,109,111,100,117,108,101,115,41, + 3,114,104,0,0,0,90,18,112,97,114,101,110,116,95,109, + 111,100,117,108,101,95,110,97,109,101,90,14,112,97,116,104, + 95,97,116,116,114,95,110,97,109,101,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,230,0,0,0,202,3, + 0,0,115,4,0,0,0,0,1,12,1,122,31,95,78,97, + 109,101,115,112,97,99,101,80,97,116,104,46,95,103,101,116, + 95,112,97,114,101,110,116,95,112,97,116,104,99,1,0,0, + 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, + 0,115,80,0,0,0,116,0,124,0,106,1,131,0,131,1, + 125,1,124,1,124,0,106,2,107,3,114,74,124,0,106,3, + 124,0,106,4,124,1,131,2,125,2,124,2,100,0,107,9, + 114,68,124,2,106,5,100,0,107,8,114,68,124,2,106,6, + 114,68,124,2,106,6,124,0,95,7,124,1,124,0,95,2, + 124,0,106,7,83,0,41,1,78,41,8,114,97,0,0,0, + 114,230,0,0,0,114,231,0,0,0,114,232,0,0,0,114, + 228,0,0,0,114,124,0,0,0,114,154,0,0,0,114,229, + 0,0,0,41,3,114,104,0,0,0,90,11,112,97,114,101, + 110,116,95,112,97,116,104,114,162,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,12,95,114,101, + 99,97,108,99,117,108,97,116,101,206,3,0,0,115,16,0, + 0,0,0,2,12,1,10,1,14,3,18,1,6,1,8,1, + 6,1,122,27,95,78,97,109,101,115,112,97,99,101,80,97, + 116,104,46,95,114,101,99,97,108,99,117,108,97,116,101,99, + 1,0,0,0,0,0,0,0,1,0,0,0,2,0,0,0, + 67,0,0,0,115,12,0,0,0,116,0,124,0,106,1,131, + 0,131,1,83,0,41,1,78,41,2,218,4,105,116,101,114, + 114,237,0,0,0,41,1,114,104,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,8,95,95,105, + 116,101,114,95,95,219,3,0,0,115,2,0,0,0,0,1, + 122,23,95,78,97,109,101,115,112,97,99,101,80,97,116,104, + 46,95,95,105,116,101,114,95,95,99,3,0,0,0,0,0, + 0,0,3,0,0,0,3,0,0,0,67,0,0,0,115,14, + 0,0,0,124,2,124,0,106,0,124,1,60,0,100,0,83, + 0,41,1,78,41,1,114,229,0,0,0,41,3,114,104,0, + 0,0,218,5,105,110,100,101,120,114,37,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,218,11,95, - 103,101,116,95,99,97,99,104,101,100,97,1,0,0,115,16, - 0,0,0,0,1,14,1,2,1,8,1,14,1,8,1,14, - 1,6,2,114,98,0,0,0,99,1,0,0,0,0,0,0, - 0,2,0,0,0,11,0,0,0,67,0,0,0,115,52,0, - 0,0,121,14,116,0,124,0,131,1,106,1,125,1,87,0, - 110,24,4,0,116,2,107,10,114,38,1,0,1,0,1,0, - 100,1,125,1,89,0,110,2,88,0,124,1,100,2,79,0, - 125,1,124,1,83,0,41,3,122,51,67,97,108,99,117,108, - 97,116,101,32,116,104,101,32,109,111,100,101,32,112,101,114, - 109,105,115,115,105,111,110,115,32,102,111,114,32,97,32,98, - 121,116,101,99,111,100,101,32,102,105,108,101,46,105,182,1, - 0,0,233,128,0,0,0,41,3,114,41,0,0,0,114,43, - 0,0,0,114,42,0,0,0,41,2,114,37,0,0,0,114, - 44,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,10,95,99,97,108,99,95,109,111,100,101,109, - 1,0,0,115,12,0,0,0,0,2,2,1,14,1,14,1, - 10,3,8,1,114,100,0,0,0,99,1,0,0,0,0,0, - 0,0,3,0,0,0,11,0,0,0,3,0,0,0,115,68, - 0,0,0,100,6,135,0,102,1,100,2,100,3,132,9,125, - 1,121,10,116,0,106,1,125,2,87,0,110,28,4,0,116, - 2,107,10,114,52,1,0,1,0,1,0,100,4,100,5,132, - 0,125,2,89,0,110,2,88,0,124,2,124,1,136,0,131, - 2,1,0,124,1,83,0,41,7,122,252,68,101,99,111,114, - 97,116,111,114,32,116,111,32,118,101,114,105,102,121,32,116, - 104,97,116,32,116,104,101,32,109,111,100,117,108,101,32,98, - 101,105,110,103,32,114,101,113,117,101,115,116,101,100,32,109, - 97,116,99,104,101,115,32,116,104,101,32,111,110,101,32,116, - 104,101,10,32,32,32,32,108,111,97,100,101,114,32,99,97, - 110,32,104,97,110,100,108,101,46,10,10,32,32,32,32,84, - 104,101,32,102,105,114,115,116,32,97,114,103,117,109,101,110, - 116,32,40,115,101,108,102,41,32,109,117,115,116,32,100,101, - 102,105,110,101,32,95,110,97,109,101,32,119,104,105,99,104, - 32,116,104,101,32,115,101,99,111,110,100,32,97,114,103,117, - 109,101,110,116,32,105,115,10,32,32,32,32,99,111,109,112, - 97,114,101,100,32,97,103,97,105,110,115,116,46,32,73,102, - 32,116,104,101,32,99,111,109,112,97,114,105,115,111,110,32, - 102,97,105,108,115,32,116,104,101,110,32,73,109,112,111,114, - 116,69,114,114,111,114,32,105,115,32,114,97,105,115,101,100, - 46,10,10,32,32,32,32,78,99,2,0,0,0,0,0,0, - 0,4,0,0,0,5,0,0,0,31,0,0,0,115,64,0, - 0,0,124,1,100,0,107,8,114,16,124,0,106,0,125,1, - 110,34,124,0,106,0,124,1,107,3,114,50,116,1,100,1, - 124,0,106,0,124,1,102,2,22,0,100,2,124,1,144,1, - 131,1,130,1,136,0,124,0,124,1,124,2,124,3,142,2, - 83,0,41,3,78,122,30,108,111,97,100,101,114,32,102,111, - 114,32,37,115,32,99,97,110,110,111,116,32,104,97,110,100, - 108,101,32,37,115,218,4,110,97,109,101,41,2,114,101,0, - 0,0,218,11,73,109,112,111,114,116,69,114,114,111,114,41, - 4,218,4,115,101,108,102,114,101,0,0,0,218,4,97,114, - 103,115,90,6,107,119,97,114,103,115,41,1,218,6,109,101, - 116,104,111,100,114,4,0,0,0,114,6,0,0,0,218,19, - 95,99,104,101,99,107,95,110,97,109,101,95,119,114,97,112, - 112,101,114,129,1,0,0,115,12,0,0,0,0,1,8,1, - 8,1,10,1,4,1,20,1,122,40,95,99,104,101,99,107, - 95,110,97,109,101,46,60,108,111,99,97,108,115,62,46,95, - 99,104,101,99,107,95,110,97,109,101,95,119,114,97,112,112, - 101,114,99,2,0,0,0,0,0,0,0,3,0,0,0,7, - 0,0,0,83,0,0,0,115,60,0,0,0,120,40,100,5, - 68,0,93,32,125,2,116,0,124,1,124,2,131,2,114,6, - 116,1,124,0,124,2,116,2,124,1,124,2,131,2,131,3, - 1,0,113,6,87,0,124,0,106,3,106,4,124,1,106,3, - 131,1,1,0,100,0,83,0,41,6,78,218,10,95,95,109, - 111,100,117,108,101,95,95,218,8,95,95,110,97,109,101,95, - 95,218,12,95,95,113,117,97,108,110,97,109,101,95,95,218, - 7,95,95,100,111,99,95,95,41,4,122,10,95,95,109,111, - 100,117,108,101,95,95,122,8,95,95,110,97,109,101,95,95, - 122,12,95,95,113,117,97,108,110,97,109,101,95,95,122,7, - 95,95,100,111,99,95,95,41,5,218,7,104,97,115,97,116, - 116,114,218,7,115,101,116,97,116,116,114,218,7,103,101,116, - 97,116,116,114,218,8,95,95,100,105,99,116,95,95,218,6, - 117,112,100,97,116,101,41,3,90,3,110,101,119,90,3,111, - 108,100,114,55,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,218,5,95,119,114,97,112,140,1,0, - 0,115,8,0,0,0,0,1,10,1,10,1,22,1,122,26, - 95,99,104,101,99,107,95,110,97,109,101,46,60,108,111,99, - 97,108,115,62,46,95,119,114,97,112,41,1,78,41,3,218, - 10,95,98,111,111,116,115,116,114,97,112,114,116,0,0,0, - 218,9,78,97,109,101,69,114,114,111,114,41,3,114,105,0, - 0,0,114,106,0,0,0,114,116,0,0,0,114,4,0,0, - 0,41,1,114,105,0,0,0,114,6,0,0,0,218,11,95, - 99,104,101,99,107,95,110,97,109,101,121,1,0,0,115,14, - 0,0,0,0,8,14,7,2,1,10,1,14,2,14,5,10, - 1,114,119,0,0,0,99,2,0,0,0,0,0,0,0,5, - 0,0,0,4,0,0,0,67,0,0,0,115,60,0,0,0, - 124,0,106,0,124,1,131,1,92,2,125,2,125,3,124,2, - 100,1,107,8,114,56,116,1,124,3,131,1,114,56,100,2, - 125,4,116,2,106,3,124,4,106,4,124,3,100,3,25,0, - 131,1,116,5,131,2,1,0,124,2,83,0,41,4,122,155, - 84,114,121,32,116,111,32,102,105,110,100,32,97,32,108,111, - 97,100,101,114,32,102,111,114,32,116,104,101,32,115,112,101, - 99,105,102,105,101,100,32,109,111,100,117,108,101,32,98,121, - 32,100,101,108,101,103,97,116,105,110,103,32,116,111,10,32, - 32,32,32,115,101,108,102,46,102,105,110,100,95,108,111,97, - 100,101,114,40,41,46,10,10,32,32,32,32,84,104,105,115, - 32,109,101,116,104,111,100,32,105,115,32,100,101,112,114,101, - 99,97,116,101,100,32,105,110,32,102,97,118,111,114,32,111, - 102,32,102,105,110,100,101,114,46,102,105,110,100,95,115,112, - 101,99,40,41,46,10,10,32,32,32,32,78,122,44,78,111, - 116,32,105,109,112,111,114,116,105,110,103,32,100,105,114,101, - 99,116,111,114,121,32,123,125,58,32,109,105,115,115,105,110, - 103,32,95,95,105,110,105,116,95,95,114,62,0,0,0,41, - 6,218,11,102,105,110,100,95,108,111,97,100,101,114,114,33, - 0,0,0,114,63,0,0,0,114,64,0,0,0,114,50,0, - 0,0,218,13,73,109,112,111,114,116,87,97,114,110,105,110, - 103,41,5,114,103,0,0,0,218,8,102,117,108,108,110,97, - 109,101,218,6,108,111,97,100,101,114,218,8,112,111,114,116, - 105,111,110,115,218,3,109,115,103,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,17,95,102,105,110,100,95, - 109,111,100,117,108,101,95,115,104,105,109,149,1,0,0,115, - 10,0,0,0,0,10,14,1,16,1,4,1,22,1,114,126, - 0,0,0,99,4,0,0,0,0,0,0,0,11,0,0,0, - 19,0,0,0,67,0,0,0,115,128,1,0,0,105,0,125, - 4,124,2,100,1,107,9,114,22,124,2,124,4,100,2,60, - 0,110,4,100,3,125,2,124,3,100,1,107,9,114,42,124, - 3,124,4,100,4,60,0,124,0,100,1,100,5,133,2,25, - 0,125,5,124,0,100,5,100,6,133,2,25,0,125,6,124, - 0,100,6,100,7,133,2,25,0,125,7,124,5,116,0,107, - 3,114,122,100,8,106,1,124,2,124,5,131,2,125,8,116, - 2,106,3,100,9,124,8,131,2,1,0,116,4,124,8,124, - 4,141,1,130,1,110,86,116,5,124,6,131,1,100,5,107, - 3,114,166,100,10,106,1,124,2,131,1,125,8,116,2,106, - 3,100,9,124,8,131,2,1,0,116,6,124,8,131,1,130, - 1,110,42,116,5,124,7,131,1,100,5,107,3,114,208,100, - 11,106,1,124,2,131,1,125,8,116,2,106,3,100,9,124, - 8,131,2,1,0,116,6,124,8,131,1,130,1,124,1,100, - 1,107,9,144,1,114,116,121,16,116,7,124,1,100,12,25, - 0,131,1,125,9,87,0,110,20,4,0,116,8,107,10,114, - 254,1,0,1,0,1,0,89,0,110,48,88,0,116,9,124, - 6,131,1,124,9,107,3,144,1,114,46,100,13,106,1,124, - 2,131,1,125,8,116,2,106,3,100,9,124,8,131,2,1, - 0,116,4,124,8,124,4,141,1,130,1,121,16,124,1,100, - 14,25,0,100,15,64,0,125,10,87,0,110,22,4,0,116, - 8,107,10,144,1,114,84,1,0,1,0,1,0,89,0,110, - 32,88,0,116,9,124,7,131,1,124,10,107,3,144,1,114, - 116,116,4,100,13,106,1,124,2,131,1,124,4,141,1,130, - 1,124,0,100,7,100,1,133,2,25,0,83,0,41,16,97, - 122,1,0,0,86,97,108,105,100,97,116,101,32,116,104,101, - 32,104,101,97,100,101,114,32,111,102,32,116,104,101,32,112, - 97,115,115,101,100,45,105,110,32,98,121,116,101,99,111,100, - 101,32,97,103,97,105,110,115,116,32,115,111,117,114,99,101, - 95,115,116,97,116,115,32,40,105,102,10,32,32,32,32,103, - 105,118,101,110,41,32,97,110,100,32,114,101,116,117,114,110, - 105,110,103,32,116,104,101,32,98,121,116,101,99,111,100,101, - 32,116,104,97,116,32,99,97,110,32,98,101,32,99,111,109, - 112,105,108,101,100,32,98,121,32,99,111,109,112,105,108,101, - 40,41,46,10,10,32,32,32,32,65,108,108,32,111,116,104, - 101,114,32,97,114,103,117,109,101,110,116,115,32,97,114,101, - 32,117,115,101,100,32,116,111,32,101,110,104,97,110,99,101, - 32,101,114,114,111,114,32,114,101,112,111,114,116,105,110,103, - 46,10,10,32,32,32,32,73,109,112,111,114,116,69,114,114, - 111,114,32,105,115,32,114,97,105,115,101,100,32,119,104,101, - 110,32,116,104,101,32,109,97,103,105,99,32,110,117,109,98, - 101,114,32,105,115,32,105,110,99,111,114,114,101,99,116,32, - 111,114,32,116,104,101,32,98,121,116,101,99,111,100,101,32, - 105,115,10,32,32,32,32,102,111,117,110,100,32,116,111,32, - 98,101,32,115,116,97,108,101,46,32,69,79,70,69,114,114, - 111,114,32,105,115,32,114,97,105,115,101,100,32,119,104,101, - 110,32,116,104,101,32,100,97,116,97,32,105,115,32,102,111, - 117,110,100,32,116,111,32,98,101,10,32,32,32,32,116,114, - 117,110,99,97,116,101,100,46,10,10,32,32,32,32,78,114, - 101,0,0,0,122,10,60,98,121,116,101,99,111,100,101,62, - 114,37,0,0,0,114,14,0,0,0,233,8,0,0,0,233, - 12,0,0,0,122,30,98,97,100,32,109,97,103,105,99,32, - 110,117,109,98,101,114,32,105,110,32,123,33,114,125,58,32, - 123,33,114,125,122,2,123,125,122,43,114,101,97,99,104,101, - 100,32,69,79,70,32,119,104,105,108,101,32,114,101,97,100, - 105,110,103,32,116,105,109,101,115,116,97,109,112,32,105,110, - 32,123,33,114,125,122,48,114,101,97,99,104,101,100,32,69, - 79,70,32,119,104,105,108,101,32,114,101,97,100,105,110,103, - 32,115,105,122,101,32,111,102,32,115,111,117,114,99,101,32, - 105,110,32,123,33,114,125,218,5,109,116,105,109,101,122,26, - 98,121,116,101,99,111,100,101,32,105,115,32,115,116,97,108, - 101,32,102,111,114,32,123,33,114,125,218,4,115,105,122,101, - 108,3,0,0,0,255,127,255,127,3,0,41,10,218,12,77, - 65,71,73,67,95,78,85,77,66,69,82,114,50,0,0,0, - 114,117,0,0,0,218,16,95,118,101,114,98,111,115,101,95, - 109,101,115,115,97,103,101,114,102,0,0,0,114,33,0,0, - 0,218,8,69,79,70,69,114,114,111,114,114,16,0,0,0, - 218,8,75,101,121,69,114,114,111,114,114,21,0,0,0,41, - 11,114,56,0,0,0,218,12,115,111,117,114,99,101,95,115, - 116,97,116,115,114,101,0,0,0,114,37,0,0,0,90,11, - 101,120,99,95,100,101,116,97,105,108,115,90,5,109,97,103, - 105,99,90,13,114,97,119,95,116,105,109,101,115,116,97,109, - 112,90,8,114,97,119,95,115,105,122,101,114,78,0,0,0, - 218,12,115,111,117,114,99,101,95,109,116,105,109,101,218,11, - 115,111,117,114,99,101,95,115,105,122,101,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,25,95,118,97,108, - 105,100,97,116,101,95,98,121,116,101,99,111,100,101,95,104, - 101,97,100,101,114,166,1,0,0,115,76,0,0,0,0,11, - 4,1,8,1,10,3,4,1,8,1,8,1,12,1,12,1, - 12,1,8,1,12,1,12,1,12,1,12,1,10,1,12,1, - 10,1,12,1,10,1,12,1,8,1,10,1,2,1,16,1, - 14,1,6,2,14,1,10,1,12,1,10,1,2,1,16,1, - 16,1,6,2,14,1,10,1,6,1,114,138,0,0,0,99, - 4,0,0,0,0,0,0,0,5,0,0,0,6,0,0,0, - 67,0,0,0,115,86,0,0,0,116,0,106,1,124,0,131, - 1,125,4,116,2,124,4,116,3,131,2,114,58,116,4,106, - 5,100,1,124,2,131,2,1,0,124,3,100,2,107,9,114, - 52,116,6,106,7,124,4,124,3,131,2,1,0,124,4,83, - 0,110,24,116,8,100,3,106,9,124,2,131,1,100,4,124, - 1,100,5,124,2,144,2,131,1,130,1,100,2,83,0,41, - 6,122,60,67,111,109,112,105,108,101,32,98,121,116,101,99, - 111,100,101,32,97,115,32,114,101,116,117,114,110,101,100,32, - 98,121,32,95,118,97,108,105,100,97,116,101,95,98,121,116, - 101,99,111,100,101,95,104,101,97,100,101,114,40,41,46,122, - 21,99,111,100,101,32,111,98,106,101,99,116,32,102,114,111, - 109,32,123,33,114,125,78,122,23,78,111,110,45,99,111,100, - 101,32,111,98,106,101,99,116,32,105,110,32,123,33,114,125, - 114,101,0,0,0,114,37,0,0,0,41,10,218,7,109,97, - 114,115,104,97,108,90,5,108,111,97,100,115,218,10,105,115, - 105,110,115,116,97,110,99,101,218,10,95,99,111,100,101,95, - 116,121,112,101,114,117,0,0,0,114,132,0,0,0,218,4, - 95,105,109,112,90,16,95,102,105,120,95,99,111,95,102,105, - 108,101,110,97,109,101,114,102,0,0,0,114,50,0,0,0, - 41,5,114,56,0,0,0,114,101,0,0,0,114,92,0,0, - 0,114,93,0,0,0,218,4,99,111,100,101,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,17,95,99,111, - 109,112,105,108,101,95,98,121,116,101,99,111,100,101,221,1, - 0,0,115,16,0,0,0,0,2,10,1,10,1,12,1,8, - 1,12,1,6,2,12,1,114,144,0,0,0,114,62,0,0, - 0,99,3,0,0,0,0,0,0,0,4,0,0,0,3,0, - 0,0,67,0,0,0,115,56,0,0,0,116,0,116,1,131, - 1,125,3,124,3,106,2,116,3,124,1,131,1,131,1,1, - 0,124,3,106,2,116,3,124,2,131,1,131,1,1,0,124, - 3,106,2,116,4,106,5,124,0,131,1,131,1,1,0,124, - 3,83,0,41,1,122,80,67,111,109,112,105,108,101,32,97, - 32,99,111,100,101,32,111,98,106,101,99,116,32,105,110,116, - 111,32,98,121,116,101,99,111,100,101,32,102,111,114,32,119, - 114,105,116,105,110,103,32,111,117,116,32,116,111,32,97,32, - 98,121,116,101,45,99,111,109,112,105,108,101,100,10,32,32, - 32,32,102,105,108,101,46,41,6,218,9,98,121,116,101,97, - 114,114,97,121,114,131,0,0,0,218,6,101,120,116,101,110, - 100,114,19,0,0,0,114,139,0,0,0,90,5,100,117,109, - 112,115,41,4,114,143,0,0,0,114,129,0,0,0,114,137, - 0,0,0,114,56,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,218,17,95,99,111,100,101,95,116, - 111,95,98,121,116,101,99,111,100,101,233,1,0,0,115,10, - 0,0,0,0,3,8,1,14,1,14,1,16,1,114,147,0, - 0,0,99,1,0,0,0,0,0,0,0,5,0,0,0,4, - 0,0,0,67,0,0,0,115,62,0,0,0,100,1,100,2, - 108,0,125,1,116,1,106,2,124,0,131,1,106,3,125,2, - 124,1,106,4,124,2,131,1,125,3,116,1,106,5,100,2, - 100,3,131,2,125,4,124,4,106,6,124,0,106,6,124,3, - 100,1,25,0,131,1,131,1,83,0,41,4,122,121,68,101, - 99,111,100,101,32,98,121,116,101,115,32,114,101,112,114,101, - 115,101,110,116,105,110,103,32,115,111,117,114,99,101,32,99, - 111,100,101,32,97,110,100,32,114,101,116,117,114,110,32,116, - 104,101,32,115,116,114,105,110,103,46,10,10,32,32,32,32, - 85,110,105,118,101,114,115,97,108,32,110,101,119,108,105,110, - 101,32,115,117,112,112,111,114,116,32,105,115,32,117,115,101, - 100,32,105,110,32,116,104,101,32,100,101,99,111,100,105,110, - 103,46,10,32,32,32,32,114,62,0,0,0,78,84,41,7, - 218,8,116,111,107,101,110,105,122,101,114,52,0,0,0,90, - 7,66,121,116,101,115,73,79,90,8,114,101,97,100,108,105, - 110,101,90,15,100,101,116,101,99,116,95,101,110,99,111,100, - 105,110,103,90,25,73,110,99,114,101,109,101,110,116,97,108, - 78,101,119,108,105,110,101,68,101,99,111,100,101,114,218,6, - 100,101,99,111,100,101,41,5,218,12,115,111,117,114,99,101, - 95,98,121,116,101,115,114,148,0,0,0,90,21,115,111,117, - 114,99,101,95,98,121,116,101,115,95,114,101,97,100,108,105, - 110,101,218,8,101,110,99,111,100,105,110,103,90,15,110,101, - 119,108,105,110,101,95,100,101,99,111,100,101,114,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,13,100,101, - 99,111,100,101,95,115,111,117,114,99,101,243,1,0,0,115, - 10,0,0,0,0,5,8,1,12,1,10,1,12,1,114,152, - 0,0,0,41,2,114,123,0,0,0,218,26,115,117,98,109, - 111,100,117,108,101,95,115,101,97,114,99,104,95,108,111,99, - 97,116,105,111,110,115,99,2,0,0,0,2,0,0,0,9, - 0,0,0,19,0,0,0,67,0,0,0,115,8,1,0,0, - 124,1,100,1,107,8,114,58,100,2,125,1,116,0,124,2, - 100,3,131,2,114,58,121,14,124,2,106,1,124,0,131,1, - 125,1,87,0,110,20,4,0,116,2,107,10,114,56,1,0, - 1,0,1,0,89,0,110,2,88,0,116,3,106,4,124,0, - 124,2,100,4,124,1,144,1,131,2,125,4,100,5,124,4, - 95,5,124,2,100,1,107,8,114,146,120,54,116,6,131,0, - 68,0,93,40,92,2,125,5,125,6,124,1,106,7,116,8, - 124,6,131,1,131,1,114,98,124,5,124,0,124,1,131,2, - 125,2,124,2,124,4,95,9,80,0,113,98,87,0,100,1, - 83,0,124,3,116,10,107,8,114,212,116,0,124,2,100,6, - 131,2,114,218,121,14,124,2,106,11,124,0,131,1,125,7, - 87,0,110,20,4,0,116,2,107,10,114,198,1,0,1,0, - 1,0,89,0,113,218,88,0,124,7,114,218,103,0,124,4, - 95,12,110,6,124,3,124,4,95,12,124,4,106,12,103,0, - 107,2,144,1,114,4,124,1,144,1,114,4,116,13,124,1, - 131,1,100,7,25,0,125,8,124,4,106,12,106,14,124,8, - 131,1,1,0,124,4,83,0,41,8,97,61,1,0,0,82, - 101,116,117,114,110,32,97,32,109,111,100,117,108,101,32,115, - 112,101,99,32,98,97,115,101,100,32,111,110,32,97,32,102, - 105,108,101,32,108,111,99,97,116,105,111,110,46,10,10,32, - 32,32,32,84,111,32,105,110,100,105,99,97,116,101,32,116, - 104,97,116,32,116,104,101,32,109,111,100,117,108,101,32,105, - 115,32,97,32,112,97,99,107,97,103,101,44,32,115,101,116, - 10,32,32,32,32,115,117,98,109,111,100,117,108,101,95,115, - 101,97,114,99,104,95,108,111,99,97,116,105,111,110,115,32, - 116,111,32,97,32,108,105,115,116,32,111,102,32,100,105,114, - 101,99,116,111,114,121,32,112,97,116,104,115,46,32,32,65, - 110,10,32,32,32,32,101,109,112,116,121,32,108,105,115,116, - 32,105,115,32,115,117,102,102,105,99,105,101,110,116,44,32, - 116,104,111,117,103,104,32,105,116,115,32,110,111,116,32,111, - 116,104,101,114,119,105,115,101,32,117,115,101,102,117,108,32, - 116,111,32,116,104,101,10,32,32,32,32,105,109,112,111,114, - 116,32,115,121,115,116,101,109,46,10,10,32,32,32,32,84, - 104,101,32,108,111,97,100,101,114,32,109,117,115,116,32,116, - 97,107,101,32,97,32,115,112,101,99,32,97,115,32,105,116, - 115,32,111,110,108,121,32,95,95,105,110,105,116,95,95,40, - 41,32,97,114,103,46,10,10,32,32,32,32,78,122,9,60, - 117,110,107,110,111,119,110,62,218,12,103,101,116,95,102,105, - 108,101,110,97,109,101,218,6,111,114,105,103,105,110,84,218, - 10,105,115,95,112,97,99,107,97,103,101,114,62,0,0,0, - 41,15,114,111,0,0,0,114,154,0,0,0,114,102,0,0, - 0,114,117,0,0,0,218,10,77,111,100,117,108,101,83,112, - 101,99,90,13,95,115,101,116,95,102,105,108,101,97,116,116, - 114,218,27,95,103,101,116,95,115,117,112,112,111,114,116,101, - 100,95,102,105,108,101,95,108,111,97,100,101,114,115,114,95, - 0,0,0,114,96,0,0,0,114,123,0,0,0,218,9,95, - 80,79,80,85,76,65,84,69,114,156,0,0,0,114,153,0, - 0,0,114,40,0,0,0,218,6,97,112,112,101,110,100,41, - 9,114,101,0,0,0,90,8,108,111,99,97,116,105,111,110, - 114,123,0,0,0,114,153,0,0,0,218,4,115,112,101,99, - 218,12,108,111,97,100,101,114,95,99,108,97,115,115,218,8, - 115,117,102,102,105,120,101,115,114,156,0,0,0,90,7,100, - 105,114,110,97,109,101,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,218,23,115,112,101,99,95,102,114,111,109, - 95,102,105,108,101,95,108,111,99,97,116,105,111,110,4,2, - 0,0,115,60,0,0,0,0,12,8,4,4,1,10,2,2, - 1,14,1,14,1,6,8,18,1,6,3,8,1,16,1,14, - 1,10,1,6,1,6,2,4,3,8,2,10,1,2,1,14, - 1,14,1,6,2,4,1,8,2,6,1,12,1,6,1,12, - 1,12,2,114,164,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,4,0,0,0,64,0,0,0,115,80,0, + 95,115,101,116,105,116,101,109,95,95,222,3,0,0,115,2, + 0,0,0,0,1,122,26,95,78,97,109,101,115,112,97,99, + 101,80,97,116,104,46,95,95,115,101,116,105,116,101,109,95, + 95,99,1,0,0,0,0,0,0,0,1,0,0,0,2,0, + 0,0,67,0,0,0,115,12,0,0,0,116,0,124,0,106, + 1,131,0,131,1,83,0,41,1,78,41,2,114,33,0,0, + 0,114,237,0,0,0,41,1,114,104,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,218,7,95,95, + 108,101,110,95,95,225,3,0,0,115,2,0,0,0,0,1, + 122,22,95,78,97,109,101,115,112,97,99,101,80,97,116,104, + 46,95,95,108,101,110,95,95,99,1,0,0,0,0,0,0, + 0,1,0,0,0,2,0,0,0,67,0,0,0,115,12,0, + 0,0,100,1,106,0,124,0,106,1,131,1,83,0,41,2, + 78,122,20,95,78,97,109,101,115,112,97,99,101,80,97,116, + 104,40,123,33,114,125,41,41,2,114,50,0,0,0,114,229, + 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,218,8,95,95,114,101,112, + 114,95,95,228,3,0,0,115,2,0,0,0,0,1,122,23, + 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, + 95,114,101,112,114,95,95,99,2,0,0,0,0,0,0,0, + 2,0,0,0,2,0,0,0,67,0,0,0,115,12,0,0, + 0,124,1,124,0,106,0,131,0,107,6,83,0,41,1,78, + 41,1,114,237,0,0,0,41,2,114,104,0,0,0,218,4, + 105,116,101,109,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,218,12,95,95,99,111,110,116,97,105,110,115,95, + 95,231,3,0,0,115,2,0,0,0,0,1,122,27,95,78, + 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,99, + 111,110,116,97,105,110,115,95,95,99,2,0,0,0,0,0, + 0,0,2,0,0,0,2,0,0,0,67,0,0,0,115,16, + 0,0,0,124,0,106,0,106,1,124,1,131,1,1,0,100, + 0,83,0,41,1,78,41,2,114,229,0,0,0,114,161,0, + 0,0,41,2,114,104,0,0,0,114,244,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,161,0, + 0,0,234,3,0,0,115,2,0,0,0,0,1,122,21,95, + 78,97,109,101,115,112,97,99,101,80,97,116,104,46,97,112, + 112,101,110,100,78,41,14,114,109,0,0,0,114,108,0,0, + 0,114,110,0,0,0,114,111,0,0,0,114,182,0,0,0, + 114,235,0,0,0,114,230,0,0,0,114,237,0,0,0,114, + 239,0,0,0,114,241,0,0,0,114,242,0,0,0,114,243, + 0,0,0,114,245,0,0,0,114,161,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,227,0,0,0,179,3,0,0,115,22,0,0,0,8, + 5,4,2,8,6,8,10,8,4,8,13,8,3,8,3,8, + 3,8,3,8,3,114,227,0,0,0,99,0,0,0,0,0, + 0,0,0,0,0,0,0,3,0,0,0,64,0,0,0,115, + 80,0,0,0,101,0,90,1,100,0,90,2,100,1,100,2, + 132,0,90,3,101,4,100,3,100,4,132,0,131,1,90,5, + 100,5,100,6,132,0,90,6,100,7,100,8,132,0,90,7, + 100,9,100,10,132,0,90,8,100,11,100,12,132,0,90,9, + 100,13,100,14,132,0,90,10,100,15,100,16,132,0,90,11, + 100,17,83,0,41,18,218,16,95,78,97,109,101,115,112,97, + 99,101,76,111,97,100,101,114,99,4,0,0,0,0,0,0, + 0,4,0,0,0,4,0,0,0,67,0,0,0,115,18,0, + 0,0,116,0,124,1,124,2,124,3,131,3,124,0,95,1, + 100,0,83,0,41,1,78,41,2,114,227,0,0,0,114,229, + 0,0,0,41,4,114,104,0,0,0,114,102,0,0,0,114, + 37,0,0,0,114,233,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,182,0,0,0,240,3,0, + 0,115,2,0,0,0,0,1,122,25,95,78,97,109,101,115, + 112,97,99,101,76,111,97,100,101,114,46,95,95,105,110,105, + 116,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, + 2,0,0,0,67,0,0,0,115,12,0,0,0,100,1,106, + 0,124,1,106,1,131,1,83,0,41,2,122,115,82,101,116, + 117,114,110,32,114,101,112,114,32,102,111,114,32,116,104,101, + 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, + 32,32,84,104,101,32,109,101,116,104,111,100,32,105,115,32, + 100,101,112,114,101,99,97,116,101,100,46,32,32,84,104,101, + 32,105,109,112,111,114,116,32,109,97,99,104,105,110,101,114, + 121,32,100,111,101,115,32,116,104,101,32,106,111,98,32,105, + 116,115,101,108,102,46,10,10,32,32,32,32,32,32,32,32, + 122,25,60,109,111,100,117,108,101,32,123,33,114,125,32,40, + 110,97,109,101,115,112,97,99,101,41,62,41,2,114,50,0, + 0,0,114,109,0,0,0,41,2,114,168,0,0,0,114,187, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,11,109,111,100,117,108,101,95,114,101,112,114,243, + 3,0,0,115,2,0,0,0,0,7,122,28,95,78,97,109, + 101,115,112,97,99,101,76,111,97,100,101,114,46,109,111,100, + 117,108,101,95,114,101,112,114,99,2,0,0,0,0,0,0, + 0,2,0,0,0,1,0,0,0,67,0,0,0,115,4,0, + 0,0,100,1,83,0,41,2,78,84,114,4,0,0,0,41, + 2,114,104,0,0,0,114,123,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,114,157,0,0,0,252, + 3,0,0,115,2,0,0,0,0,1,122,27,95,78,97,109, + 101,115,112,97,99,101,76,111,97,100,101,114,46,105,115,95, + 112,97,99,107,97,103,101,99,2,0,0,0,0,0,0,0, + 2,0,0,0,1,0,0,0,67,0,0,0,115,4,0,0, + 0,100,1,83,0,41,2,78,114,32,0,0,0,114,4,0, + 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, + 0,0,255,3,0,0,115,2,0,0,0,0,1,122,27,95, + 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, + 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, + 0,0,0,2,0,0,0,6,0,0,0,67,0,0,0,115, + 18,0,0,0,116,0,100,1,100,2,100,3,100,4,100,5, + 144,1,131,3,83,0,41,6,78,114,32,0,0,0,122,8, + 60,115,116,114,105,110,103,62,114,186,0,0,0,114,201,0, + 0,0,84,41,1,114,202,0,0,0,41,2,114,104,0,0, + 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,184,0,0,0,2,4,0,0,115,2, + 0,0,0,0,1,122,25,95,78,97,109,101,115,112,97,99, + 101,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, + 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, + 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, + 122,42,85,115,101,32,100,101,102,97,117,108,116,32,115,101, + 109,97,110,116,105,99,115,32,102,111,114,32,109,111,100,117, + 108,101,32,99,114,101,97,116,105,111,110,46,78,114,4,0, + 0,0,41,2,114,104,0,0,0,114,162,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,183,0, + 0,0,5,4,0,0,115,0,0,0,0,122,30,95,78,97, + 109,101,115,112,97,99,101,76,111,97,100,101,114,46,99,114, + 101,97,116,101,95,109,111,100,117,108,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, + 115,4,0,0,0,100,0,83,0,41,1,78,114,4,0,0, + 0,41,2,114,104,0,0,0,114,187,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,188,0,0, + 0,8,4,0,0,115,2,0,0,0,0,1,122,28,95,78, + 97,109,101,115,112,97,99,101,76,111,97,100,101,114,46,101, + 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, + 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, + 26,0,0,0,116,0,106,1,100,1,124,0,106,2,131,2, + 1,0,116,0,106,3,124,0,124,1,131,2,83,0,41,2, + 122,98,76,111,97,100,32,97,32,110,97,109,101,115,112,97, + 99,101,32,109,111,100,117,108,101,46,10,10,32,32,32,32, + 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, + 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, + 85,115,101,32,101,120,101,99,95,109,111,100,117,108,101,40, + 41,32,105,110,115,116,101,97,100,46,10,10,32,32,32,32, + 32,32,32,32,122,38,110,97,109,101,115,112,97,99,101,32, + 109,111,100,117,108,101,32,108,111,97,100,101,100,32,119,105, + 116,104,32,112,97,116,104,32,123,33,114,125,41,4,114,118, + 0,0,0,114,133,0,0,0,114,229,0,0,0,114,189,0, + 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,190,0, + 0,0,11,4,0,0,115,6,0,0,0,0,7,6,1,8, + 1,122,28,95,78,97,109,101,115,112,97,99,101,76,111,97, + 100,101,114,46,108,111,97,100,95,109,111,100,117,108,101,78, + 41,12,114,109,0,0,0,114,108,0,0,0,114,110,0,0, + 0,114,182,0,0,0,114,180,0,0,0,114,247,0,0,0, + 114,157,0,0,0,114,199,0,0,0,114,184,0,0,0,114, + 183,0,0,0,114,188,0,0,0,114,190,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,114,246,0,0,0,239,3,0,0,115,16,0,0,0, + 8,1,8,3,12,9,8,3,8,3,8,3,8,3,8,3, + 114,246,0,0,0,99,0,0,0,0,0,0,0,0,0,0, + 0,0,4,0,0,0,64,0,0,0,115,106,0,0,0,101, + 0,90,1,100,0,90,2,100,1,90,3,101,4,100,2,100, + 3,132,0,131,1,90,5,101,4,100,4,100,5,132,0,131, + 1,90,6,101,4,100,6,100,7,132,0,131,1,90,7,101, + 4,100,8,100,9,132,0,131,1,90,8,101,4,100,17,100, + 11,100,12,132,1,131,1,90,9,101,4,100,18,100,13,100, + 14,132,1,131,1,90,10,101,4,100,19,100,15,100,16,132, + 1,131,1,90,11,100,10,83,0,41,20,218,10,80,97,116, + 104,70,105,110,100,101,114,122,62,77,101,116,97,32,112,97, + 116,104,32,102,105,110,100,101,114,32,102,111,114,32,115,121, + 115,46,112,97,116,104,32,97,110,100,32,112,97,99,107,97, + 103,101,32,95,95,112,97,116,104,95,95,32,97,116,116,114, + 105,98,117,116,101,115,46,99,1,0,0,0,0,0,0,0, + 2,0,0,0,4,0,0,0,67,0,0,0,115,42,0,0, + 0,120,36,116,0,106,1,106,2,131,0,68,0,93,22,125, + 1,116,3,124,1,100,1,131,2,114,12,124,1,106,4,131, + 0,1,0,113,12,87,0,100,2,83,0,41,3,122,125,67, + 97,108,108,32,116,104,101,32,105,110,118,97,108,105,100,97, + 116,101,95,99,97,99,104,101,115,40,41,32,109,101,116,104, + 111,100,32,111,110,32,97,108,108,32,112,97,116,104,32,101, + 110,116,114,121,32,102,105,110,100,101,114,115,10,32,32,32, + 32,32,32,32,32,115,116,111,114,101,100,32,105,110,32,115, + 121,115,46,112,97,116,104,95,105,109,112,111,114,116,101,114, + 95,99,97,99,104,101,115,32,40,119,104,101,114,101,32,105, + 109,112,108,101,109,101,110,116,101,100,41,46,218,17,105,110, + 118,97,108,105,100,97,116,101,95,99,97,99,104,101,115,78, + 41,5,114,8,0,0,0,218,19,112,97,116,104,95,105,109, + 112,111,114,116,101,114,95,99,97,99,104,101,218,6,118,97, + 108,117,101,115,114,112,0,0,0,114,249,0,0,0,41,2, + 114,168,0,0,0,218,6,102,105,110,100,101,114,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, + 0,29,4,0,0,115,6,0,0,0,0,4,16,1,10,1, + 122,28,80,97,116,104,70,105,110,100,101,114,46,105,110,118, + 97,108,105,100,97,116,101,95,99,97,99,104,101,115,99,2, + 0,0,0,0,0,0,0,3,0,0,0,12,0,0,0,67, + 0,0,0,115,86,0,0,0,116,0,106,1,100,1,107,9, + 114,30,116,0,106,1,12,0,114,30,116,2,106,3,100,2, + 116,4,131,2,1,0,120,50,116,0,106,1,68,0,93,36, + 125,2,121,8,124,2,124,1,131,1,83,0,4,0,116,5, + 107,10,114,72,1,0,1,0,1,0,119,38,89,0,113,38, + 88,0,113,38,87,0,100,1,83,0,100,1,83,0,41,3, + 122,46,83,101,97,114,99,104,32,115,121,115,46,112,97,116, + 104,95,104,111,111,107,115,32,102,111,114,32,97,32,102,105, + 110,100,101,114,32,102,111,114,32,39,112,97,116,104,39,46, + 78,122,23,115,121,115,46,112,97,116,104,95,104,111,111,107, + 115,32,105,115,32,101,109,112,116,121,41,6,114,8,0,0, + 0,218,10,112,97,116,104,95,104,111,111,107,115,114,63,0, + 0,0,114,64,0,0,0,114,122,0,0,0,114,103,0,0, + 0,41,3,114,168,0,0,0,114,37,0,0,0,90,4,104, + 111,111,107,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,11,95,112,97,116,104,95,104,111,111,107,115,37, + 4,0,0,115,16,0,0,0,0,3,18,1,12,1,12,1, + 2,1,8,1,14,1,12,2,122,22,80,97,116,104,70,105, + 110,100,101,114,46,95,112,97,116,104,95,104,111,111,107,115, + 99,2,0,0,0,0,0,0,0,3,0,0,0,19,0,0, + 0,67,0,0,0,115,102,0,0,0,124,1,100,1,107,2, + 114,42,121,12,116,0,106,1,131,0,125,1,87,0,110,20, + 4,0,116,2,107,10,114,40,1,0,1,0,1,0,100,2, + 83,0,88,0,121,14,116,3,106,4,124,1,25,0,125,2, + 87,0,110,40,4,0,116,5,107,10,114,96,1,0,1,0, + 1,0,124,0,106,6,124,1,131,1,125,2,124,2,116,3, + 106,4,124,1,60,0,89,0,110,2,88,0,124,2,83,0, + 41,3,122,210,71,101,116,32,116,104,101,32,102,105,110,100, + 101,114,32,102,111,114,32,116,104,101,32,112,97,116,104,32, + 101,110,116,114,121,32,102,114,111,109,32,115,121,115,46,112, + 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, + 104,101,46,10,10,32,32,32,32,32,32,32,32,73,102,32, + 116,104,101,32,112,97,116,104,32,101,110,116,114,121,32,105, + 115,32,110,111,116,32,105,110,32,116,104,101,32,99,97,99, + 104,101,44,32,102,105,110,100,32,116,104,101,32,97,112,112, + 114,111,112,114,105,97,116,101,32,102,105,110,100,101,114,10, + 32,32,32,32,32,32,32,32,97,110,100,32,99,97,99,104, + 101,32,105,116,46,32,73,102,32,110,111,32,102,105,110,100, + 101,114,32,105,115,32,97,118,97,105,108,97,98,108,101,44, + 32,115,116,111,114,101,32,78,111,110,101,46,10,10,32,32, + 32,32,32,32,32,32,114,32,0,0,0,78,41,7,114,3, + 0,0,0,114,47,0,0,0,218,17,70,105,108,101,78,111, + 116,70,111,117,110,100,69,114,114,111,114,114,8,0,0,0, + 114,250,0,0,0,114,135,0,0,0,114,254,0,0,0,41, + 3,114,168,0,0,0,114,37,0,0,0,114,252,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, + 20,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, + 99,97,99,104,101,50,4,0,0,115,22,0,0,0,0,8, + 8,1,2,1,12,1,14,3,6,1,2,1,14,1,14,1, + 10,1,16,1,122,31,80,97,116,104,70,105,110,100,101,114, + 46,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, + 99,97,99,104,101,99,3,0,0,0,0,0,0,0,6,0, + 0,0,3,0,0,0,67,0,0,0,115,82,0,0,0,116, + 0,124,2,100,1,131,2,114,26,124,2,106,1,124,1,131, + 1,92,2,125,3,125,4,110,14,124,2,106,2,124,1,131, + 1,125,3,103,0,125,4,124,3,100,0,107,9,114,60,116, + 3,106,4,124,1,124,3,131,2,83,0,116,3,106,5,124, + 1,100,0,131,2,125,5,124,4,124,5,95,6,124,5,83, + 0,41,2,78,114,121,0,0,0,41,7,114,112,0,0,0, + 114,121,0,0,0,114,179,0,0,0,114,118,0,0,0,114, + 176,0,0,0,114,158,0,0,0,114,154,0,0,0,41,6, + 114,168,0,0,0,114,123,0,0,0,114,252,0,0,0,114, + 124,0,0,0,114,125,0,0,0,114,162,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,16,95, + 108,101,103,97,99,121,95,103,101,116,95,115,112,101,99,72, + 4,0,0,115,18,0,0,0,0,4,10,1,16,2,10,1, + 4,1,8,1,12,1,12,1,6,1,122,27,80,97,116,104, + 70,105,110,100,101,114,46,95,108,101,103,97,99,121,95,103, + 101,116,95,115,112,101,99,78,99,4,0,0,0,0,0,0, + 0,9,0,0,0,5,0,0,0,67,0,0,0,115,170,0, + 0,0,103,0,125,4,120,160,124,2,68,0,93,130,125,5, + 116,0,124,5,116,1,116,2,102,2,131,2,115,30,113,10, + 124,0,106,3,124,5,131,1,125,6,124,6,100,1,107,9, + 114,10,116,4,124,6,100,2,131,2,114,72,124,6,106,5, + 124,1,124,3,131,2,125,7,110,12,124,0,106,6,124,1, + 124,6,131,2,125,7,124,7,100,1,107,8,114,94,113,10, + 124,7,106,7,100,1,107,9,114,108,124,7,83,0,124,7, + 106,8,125,8,124,8,100,1,107,8,114,130,116,9,100,3, + 131,1,130,1,124,4,106,10,124,8,131,1,1,0,113,10, + 87,0,116,11,106,12,124,1,100,1,131,2,125,7,124,4, + 124,7,95,8,124,7,83,0,100,1,83,0,41,4,122,63, + 70,105,110,100,32,116,104,101,32,108,111,97,100,101,114,32, + 111,114,32,110,97,109,101,115,112,97,99,101,95,112,97,116, + 104,32,102,111,114,32,116,104,105,115,32,109,111,100,117,108, + 101,47,112,97,99,107,97,103,101,32,110,97,109,101,46,78, + 114,178,0,0,0,122,19,115,112,101,99,32,109,105,115,115, + 105,110,103,32,108,111,97,100,101,114,41,13,114,141,0,0, + 0,114,73,0,0,0,218,5,98,121,116,101,115,114,0,1, + 0,0,114,112,0,0,0,114,178,0,0,0,114,1,1,0, + 0,114,124,0,0,0,114,154,0,0,0,114,103,0,0,0, + 114,147,0,0,0,114,118,0,0,0,114,158,0,0,0,41, + 9,114,168,0,0,0,114,123,0,0,0,114,37,0,0,0, + 114,177,0,0,0,218,14,110,97,109,101,115,112,97,99,101, + 95,112,97,116,104,90,5,101,110,116,114,121,114,252,0,0, + 0,114,162,0,0,0,114,125,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,9,95,103,101,116, + 95,115,112,101,99,87,4,0,0,115,40,0,0,0,0,5, + 4,1,10,1,14,1,2,1,10,1,8,1,10,1,14,2, + 12,1,8,1,2,1,10,1,4,1,6,1,8,1,8,5, + 14,2,12,1,6,1,122,20,80,97,116,104,70,105,110,100, + 101,114,46,95,103,101,116,95,115,112,101,99,99,4,0,0, + 0,0,0,0,0,6,0,0,0,4,0,0,0,67,0,0, + 0,115,104,0,0,0,124,2,100,1,107,8,114,14,116,0, + 106,1,125,2,124,0,106,2,124,1,124,2,124,3,131,3, + 125,4,124,4,100,1,107,8,114,42,100,1,83,0,110,58, + 124,4,106,3,100,1,107,8,114,96,124,4,106,4,125,5, + 124,5,114,90,100,2,124,4,95,5,116,6,124,1,124,5, + 124,0,106,2,131,3,124,4,95,4,124,4,83,0,113,100, + 100,1,83,0,110,4,124,4,83,0,100,1,83,0,41,3, + 122,141,84,114,121,32,116,111,32,102,105,110,100,32,97,32, + 115,112,101,99,32,102,111,114,32,39,102,117,108,108,110,97, + 109,101,39,32,111,110,32,115,121,115,46,112,97,116,104,32, + 111,114,32,39,112,97,116,104,39,46,10,10,32,32,32,32, + 32,32,32,32,84,104,101,32,115,101,97,114,99,104,32,105, + 115,32,98,97,115,101,100,32,111,110,32,115,121,115,46,112, + 97,116,104,95,104,111,111,107,115,32,97,110,100,32,115,121, + 115,46,112,97,116,104,95,105,109,112,111,114,116,101,114,95, + 99,97,99,104,101,46,10,32,32,32,32,32,32,32,32,78, + 90,9,110,97,109,101,115,112,97,99,101,41,7,114,8,0, + 0,0,114,37,0,0,0,114,4,1,0,0,114,124,0,0, + 0,114,154,0,0,0,114,156,0,0,0,114,227,0,0,0, + 41,6,114,168,0,0,0,114,123,0,0,0,114,37,0,0, + 0,114,177,0,0,0,114,162,0,0,0,114,3,1,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, + 178,0,0,0,119,4,0,0,115,26,0,0,0,0,6,8, + 1,6,1,14,1,8,1,6,1,10,1,6,1,4,3,6, + 1,16,1,6,2,6,2,122,20,80,97,116,104,70,105,110, + 100,101,114,46,102,105,110,100,95,115,112,101,99,99,3,0, + 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, + 0,0,115,30,0,0,0,124,0,106,0,124,1,124,2,131, + 2,125,3,124,3,100,1,107,8,114,24,100,1,83,0,124, + 3,106,1,83,0,41,2,122,170,102,105,110,100,32,116,104, + 101,32,109,111,100,117,108,101,32,111,110,32,115,121,115,46, + 112,97,116,104,32,111,114,32,39,112,97,116,104,39,32,98, + 97,115,101,100,32,111,110,32,115,121,115,46,112,97,116,104, + 95,104,111,111,107,115,32,97,110,100,10,32,32,32,32,32, + 32,32,32,115,121,115,46,112,97,116,104,95,105,109,112,111, + 114,116,101,114,95,99,97,99,104,101,46,10,10,32,32,32, + 32,32,32,32,32,84,104,105,115,32,109,101,116,104,111,100, + 32,105,115,32,100,101,112,114,101,99,97,116,101,100,46,32, + 32,85,115,101,32,102,105,110,100,95,115,112,101,99,40,41, + 32,105,110,115,116,101,97,100,46,10,10,32,32,32,32,32, + 32,32,32,78,41,2,114,178,0,0,0,114,124,0,0,0, + 41,4,114,168,0,0,0,114,123,0,0,0,114,37,0,0, + 0,114,162,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,179,0,0,0,143,4,0,0,115,8, + 0,0,0,0,8,12,1,8,1,4,1,122,22,80,97,116, + 104,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, + 117,108,101,41,1,78,41,2,78,78,41,1,78,41,12,114, + 109,0,0,0,114,108,0,0,0,114,110,0,0,0,114,111, + 0,0,0,114,180,0,0,0,114,249,0,0,0,114,254,0, + 0,0,114,0,1,0,0,114,1,1,0,0,114,4,1,0, + 0,114,178,0,0,0,114,179,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, + 248,0,0,0,25,4,0,0,115,22,0,0,0,8,2,4, + 2,12,8,12,13,12,22,12,15,2,1,12,31,2,1,12, + 23,2,1,114,248,0,0,0,99,0,0,0,0,0,0,0, + 0,0,0,0,0,3,0,0,0,64,0,0,0,115,90,0, 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 90,4,100,3,90,5,100,4,90,6,101,7,100,5,100,6, - 132,0,131,1,90,8,101,7,100,7,100,8,132,0,131,1, - 90,9,101,7,100,14,100,10,100,11,132,1,131,1,90,10, - 101,7,100,15,100,12,100,13,132,1,131,1,90,11,100,9, - 83,0,41,16,218,21,87,105,110,100,111,119,115,82,101,103, - 105,115,116,114,121,70,105,110,100,101,114,122,62,77,101,116, - 97,32,112,97,116,104,32,102,105,110,100,101,114,32,102,111, - 114,32,109,111,100,117,108,101,115,32,100,101,99,108,97,114, - 101,100,32,105,110,32,116,104,101,32,87,105,110,100,111,119, - 115,32,114,101,103,105,115,116,114,121,46,122,59,83,111,102, - 116,119,97,114,101,92,80,121,116,104,111,110,92,80,121,116, - 104,111,110,67,111,114,101,92,123,115,121,115,95,118,101,114, - 115,105,111,110,125,92,77,111,100,117,108,101,115,92,123,102, - 117,108,108,110,97,109,101,125,122,65,83,111,102,116,119,97, - 114,101,92,80,121,116,104,111,110,92,80,121,116,104,111,110, - 67,111,114,101,92,123,115,121,115,95,118,101,114,115,105,111, - 110,125,92,77,111,100,117,108,101,115,92,123,102,117,108,108, - 110,97,109,101,125,92,68,101,98,117,103,70,99,2,0,0, - 0,0,0,0,0,2,0,0,0,11,0,0,0,67,0,0, - 0,115,50,0,0,0,121,14,116,0,106,1,116,0,106,2, - 124,1,131,2,83,0,4,0,116,3,107,10,114,44,1,0, - 1,0,1,0,116,0,106,1,116,0,106,4,124,1,131,2, - 83,0,88,0,100,0,83,0,41,1,78,41,5,218,7,95, - 119,105,110,114,101,103,90,7,79,112,101,110,75,101,121,90, - 17,72,75,69,89,95,67,85,82,82,69,78,84,95,85,83, - 69,82,114,42,0,0,0,90,18,72,75,69,89,95,76,79, - 67,65,76,95,77,65,67,72,73,78,69,41,2,218,3,99, - 108,115,114,5,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,218,14,95,111,112,101,110,95,114,101, - 103,105,115,116,114,121,82,2,0,0,115,8,0,0,0,0, - 2,2,1,14,1,14,1,122,36,87,105,110,100,111,119,115, - 82,101,103,105,115,116,114,121,70,105,110,100,101,114,46,95, - 111,112,101,110,95,114,101,103,105,115,116,114,121,99,2,0, - 0,0,0,0,0,0,6,0,0,0,16,0,0,0,67,0, - 0,0,115,116,0,0,0,124,0,106,0,114,14,124,0,106, - 1,125,2,110,6,124,0,106,2,125,2,124,2,106,3,100, - 1,124,1,100,2,100,3,116,4,106,5,100,0,100,4,133, - 2,25,0,22,0,144,2,131,0,125,3,121,38,124,0,106, - 6,124,3,131,1,143,18,125,4,116,7,106,8,124,4,100, - 5,131,2,125,5,87,0,100,0,81,0,82,0,88,0,87, - 0,110,20,4,0,116,9,107,10,114,110,1,0,1,0,1, - 0,100,0,83,0,88,0,124,5,83,0,41,6,78,114,122, - 0,0,0,90,11,115,121,115,95,118,101,114,115,105,111,110, - 122,5,37,100,46,37,100,114,59,0,0,0,114,32,0,0, - 0,41,10,218,11,68,69,66,85,71,95,66,85,73,76,68, - 218,18,82,69,71,73,83,84,82,89,95,75,69,89,95,68, - 69,66,85,71,218,12,82,69,71,73,83,84,82,89,95,75, - 69,89,114,50,0,0,0,114,8,0,0,0,218,12,118,101, - 114,115,105,111,110,95,105,110,102,111,114,168,0,0,0,114, - 166,0,0,0,90,10,81,117,101,114,121,86,97,108,117,101, - 114,42,0,0,0,41,6,114,167,0,0,0,114,122,0,0, - 0,90,12,114,101,103,105,115,116,114,121,95,107,101,121,114, - 5,0,0,0,90,4,104,107,101,121,218,8,102,105,108,101, - 112,97,116,104,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,16,95,115,101,97,114,99,104,95,114,101,103, - 105,115,116,114,121,89,2,0,0,115,22,0,0,0,0,2, - 6,1,8,2,6,1,10,1,22,1,2,1,12,1,26,1, - 14,1,6,1,122,38,87,105,110,100,111,119,115,82,101,103, - 105,115,116,114,121,70,105,110,100,101,114,46,95,115,101,97, - 114,99,104,95,114,101,103,105,115,116,114,121,78,99,4,0, - 0,0,0,0,0,0,8,0,0,0,14,0,0,0,67,0, - 0,0,115,122,0,0,0,124,0,106,0,124,1,131,1,125, - 4,124,4,100,0,107,8,114,22,100,0,83,0,121,12,116, - 1,124,4,131,1,1,0,87,0,110,20,4,0,116,2,107, - 10,114,54,1,0,1,0,1,0,100,0,83,0,88,0,120, - 60,116,3,131,0,68,0,93,50,92,2,125,5,125,6,124, - 4,106,4,116,5,124,6,131,1,131,1,114,64,116,6,106, - 7,124,1,124,5,124,1,124,4,131,2,100,1,124,4,144, - 1,131,2,125,7,124,7,83,0,113,64,87,0,100,0,83, - 0,41,2,78,114,155,0,0,0,41,8,114,174,0,0,0, - 114,41,0,0,0,114,42,0,0,0,114,158,0,0,0,114, - 95,0,0,0,114,96,0,0,0,114,117,0,0,0,218,16, - 115,112,101,99,95,102,114,111,109,95,108,111,97,100,101,114, - 41,8,114,167,0,0,0,114,122,0,0,0,114,37,0,0, - 0,218,6,116,97,114,103,101,116,114,173,0,0,0,114,123, - 0,0,0,114,163,0,0,0,114,161,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,9,102,105, - 110,100,95,115,112,101,99,104,2,0,0,115,26,0,0,0, - 0,2,10,1,8,1,4,1,2,1,12,1,14,1,6,1, - 16,1,14,1,6,1,10,1,8,1,122,31,87,105,110,100, - 111,119,115,82,101,103,105,115,116,114,121,70,105,110,100,101, - 114,46,102,105,110,100,95,115,112,101,99,99,3,0,0,0, - 0,0,0,0,4,0,0,0,3,0,0,0,67,0,0,0, - 115,36,0,0,0,124,0,106,0,124,1,124,2,131,2,125, - 3,124,3,100,1,107,9,114,28,124,3,106,1,83,0,110, - 4,100,1,83,0,100,1,83,0,41,2,122,108,70,105,110, - 100,32,109,111,100,117,108,101,32,110,97,109,101,100,32,105, - 110,32,116,104,101,32,114,101,103,105,115,116,114,121,46,10, + 100,3,132,0,90,4,100,4,100,5,132,0,90,5,101,6, + 90,7,100,6,100,7,132,0,90,8,100,8,100,9,132,0, + 90,9,100,19,100,11,100,12,132,1,90,10,100,13,100,14, + 132,0,90,11,101,12,100,15,100,16,132,0,131,1,90,13, + 100,17,100,18,132,0,90,14,100,10,83,0,41,20,218,10, + 70,105,108,101,70,105,110,100,101,114,122,172,70,105,108,101, + 45,98,97,115,101,100,32,102,105,110,100,101,114,46,10,10, + 32,32,32,32,73,110,116,101,114,97,99,116,105,111,110,115, + 32,119,105,116,104,32,116,104,101,32,102,105,108,101,32,115, + 121,115,116,101,109,32,97,114,101,32,99,97,99,104,101,100, + 32,102,111,114,32,112,101,114,102,111,114,109,97,110,99,101, + 44,32,98,101,105,110,103,10,32,32,32,32,114,101,102,114, + 101,115,104,101,100,32,119,104,101,110,32,116,104,101,32,100, + 105,114,101,99,116,111,114,121,32,116,104,101,32,102,105,110, + 100,101,114,32,105,115,32,104,97,110,100,108,105,110,103,32, + 104,97,115,32,98,101,101,110,32,109,111,100,105,102,105,101, + 100,46,10,10,32,32,32,32,99,2,0,0,0,0,0,0, + 0,5,0,0,0,5,0,0,0,7,0,0,0,115,88,0, + 0,0,103,0,125,3,120,40,124,2,68,0,93,32,92,2, + 137,0,125,4,124,3,106,0,135,0,102,1,100,1,100,2, + 132,8,124,4,68,0,131,1,131,1,1,0,113,10,87,0, + 124,3,124,0,95,1,124,1,112,58,100,3,124,0,95,2, + 100,6,124,0,95,3,116,4,131,0,124,0,95,5,116,4, + 131,0,124,0,95,6,100,5,83,0,41,7,122,154,73,110, + 105,116,105,97,108,105,122,101,32,119,105,116,104,32,116,104, + 101,32,112,97,116,104,32,116,111,32,115,101,97,114,99,104, + 32,111,110,32,97,110,100,32,97,32,118,97,114,105,97,98, + 108,101,32,110,117,109,98,101,114,32,111,102,10,32,32,32, + 32,32,32,32,32,50,45,116,117,112,108,101,115,32,99,111, + 110,116,97,105,110,105,110,103,32,116,104,101,32,108,111,97, + 100,101,114,32,97,110,100,32,116,104,101,32,102,105,108,101, + 32,115,117,102,102,105,120,101,115,32,116,104,101,32,108,111, + 97,100,101,114,10,32,32,32,32,32,32,32,32,114,101,99, + 111,103,110,105,122,101,115,46,99,1,0,0,0,0,0,0, + 0,2,0,0,0,3,0,0,0,51,0,0,0,115,22,0, + 0,0,124,0,93,14,125,1,124,1,136,0,102,2,86,0, + 1,0,113,2,100,0,83,0,41,1,78,114,4,0,0,0, + 41,2,114,24,0,0,0,114,222,0,0,0,41,1,114,124, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,224,0, + 0,0,172,4,0,0,115,2,0,0,0,4,0,122,38,70, + 105,108,101,70,105,110,100,101,114,46,95,95,105,110,105,116, + 95,95,46,60,108,111,99,97,108,115,62,46,60,103,101,110, + 101,120,112,114,62,114,61,0,0,0,114,31,0,0,0,78, + 114,91,0,0,0,41,7,114,147,0,0,0,218,8,95,108, + 111,97,100,101,114,115,114,37,0,0,0,218,11,95,112,97, + 116,104,95,109,116,105,109,101,218,3,115,101,116,218,11,95, + 112,97,116,104,95,99,97,99,104,101,218,19,95,114,101,108, + 97,120,101,100,95,112,97,116,104,95,99,97,99,104,101,41, + 5,114,104,0,0,0,114,37,0,0,0,218,14,108,111,97, + 100,101,114,95,100,101,116,97,105,108,115,90,7,108,111,97, + 100,101,114,115,114,164,0,0,0,114,4,0,0,0,41,1, + 114,124,0,0,0,114,6,0,0,0,114,182,0,0,0,166, + 4,0,0,115,16,0,0,0,0,4,4,1,14,1,28,1, + 6,2,10,1,6,1,8,1,122,19,70,105,108,101,70,105, + 110,100,101,114,46,95,95,105,110,105,116,95,95,99,1,0, + 0,0,0,0,0,0,1,0,0,0,2,0,0,0,67,0, + 0,0,115,10,0,0,0,100,3,124,0,95,0,100,2,83, + 0,41,4,122,31,73,110,118,97,108,105,100,97,116,101,32, + 116,104,101,32,100,105,114,101,99,116,111,114,121,32,109,116, + 105,109,101,46,114,31,0,0,0,78,114,91,0,0,0,41, + 1,114,7,1,0,0,41,1,114,104,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, + 0,180,4,0,0,115,2,0,0,0,0,2,122,28,70,105, + 108,101,70,105,110,100,101,114,46,105,110,118,97,108,105,100, + 97,116,101,95,99,97,99,104,101,115,99,2,0,0,0,0, + 0,0,0,3,0,0,0,2,0,0,0,67,0,0,0,115, + 42,0,0,0,124,0,106,0,124,1,131,1,125,2,124,2, + 100,1,107,8,114,26,100,1,103,0,102,2,83,0,124,2, + 106,1,124,2,106,2,112,38,103,0,102,2,83,0,41,2, + 122,197,84,114,121,32,116,111,32,102,105,110,100,32,97,32, + 108,111,97,100,101,114,32,102,111,114,32,116,104,101,32,115, + 112,101,99,105,102,105,101,100,32,109,111,100,117,108,101,44, + 32,111,114,32,116,104,101,32,110,97,109,101,115,112,97,99, + 101,10,32,32,32,32,32,32,32,32,112,97,99,107,97,103, + 101,32,112,111,114,116,105,111,110,115,46,32,82,101,116,117, + 114,110,115,32,40,108,111,97,100,101,114,44,32,108,105,115, + 116,45,111,102,45,112,111,114,116,105,111,110,115,41,46,10, 10,32,32,32,32,32,32,32,32,84,104,105,115,32,109,101, 116,104,111,100,32,105,115,32,100,101,112,114,101,99,97,116, - 101,100,46,32,32,85,115,101,32,101,120,101,99,95,109,111, - 100,117,108,101,40,41,32,105,110,115,116,101,97,100,46,10, - 10,32,32,32,32,32,32,32,32,78,41,2,114,177,0,0, - 0,114,123,0,0,0,41,4,114,167,0,0,0,114,122,0, - 0,0,114,37,0,0,0,114,161,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,11,102,105,110, - 100,95,109,111,100,117,108,101,120,2,0,0,115,8,0,0, - 0,0,7,12,1,8,1,8,2,122,33,87,105,110,100,111, - 119,115,82,101,103,105,115,116,114,121,70,105,110,100,101,114, - 46,102,105,110,100,95,109,111,100,117,108,101,41,2,78,78, - 41,1,78,41,12,114,108,0,0,0,114,107,0,0,0,114, - 109,0,0,0,114,110,0,0,0,114,171,0,0,0,114,170, - 0,0,0,114,169,0,0,0,218,11,99,108,97,115,115,109, - 101,116,104,111,100,114,168,0,0,0,114,174,0,0,0,114, - 177,0,0,0,114,178,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,165,0, - 0,0,70,2,0,0,115,20,0,0,0,8,2,4,3,4, - 3,4,2,4,2,12,7,12,15,2,1,12,15,2,1,114, - 165,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, - 0,2,0,0,0,64,0,0,0,115,48,0,0,0,101,0, - 90,1,100,0,90,2,100,1,90,3,100,2,100,3,132,0, - 90,4,100,4,100,5,132,0,90,5,100,6,100,7,132,0, - 90,6,100,8,100,9,132,0,90,7,100,10,83,0,41,11, - 218,13,95,76,111,97,100,101,114,66,97,115,105,99,115,122, - 83,66,97,115,101,32,99,108,97,115,115,32,111,102,32,99, - 111,109,109,111,110,32,99,111,100,101,32,110,101,101,100,101, - 100,32,98,121,32,98,111,116,104,32,83,111,117,114,99,101, - 76,111,97,100,101,114,32,97,110,100,10,32,32,32,32,83, - 111,117,114,99,101,108,101,115,115,70,105,108,101,76,111,97, - 100,101,114,46,99,2,0,0,0,0,0,0,0,5,0,0, - 0,3,0,0,0,67,0,0,0,115,64,0,0,0,116,0, - 124,0,106,1,124,1,131,1,131,1,100,1,25,0,125,2, - 124,2,106,2,100,2,100,1,131,2,100,3,25,0,125,3, - 124,1,106,3,100,2,131,1,100,4,25,0,125,4,124,3, - 100,5,107,2,111,62,124,4,100,5,107,3,83,0,41,6, - 122,141,67,111,110,99,114,101,116,101,32,105,109,112,108,101, - 109,101,110,116,97,116,105,111,110,32,111,102,32,73,110,115, - 112,101,99,116,76,111,97,100,101,114,46,105,115,95,112,97, - 99,107,97,103,101,32,98,121,32,99,104,101,99,107,105,110, - 103,32,105,102,10,32,32,32,32,32,32,32,32,116,104,101, - 32,112,97,116,104,32,114,101,116,117,114,110,101,100,32,98, - 121,32,103,101,116,95,102,105,108,101,110,97,109,101,32,104, - 97,115,32,97,32,102,105,108,101,110,97,109,101,32,111,102, - 32,39,95,95,105,110,105,116,95,95,46,112,121,39,46,114, - 31,0,0,0,114,61,0,0,0,114,62,0,0,0,114,59, - 0,0,0,218,8,95,95,105,110,105,116,95,95,41,4,114, - 40,0,0,0,114,154,0,0,0,114,36,0,0,0,114,34, - 0,0,0,41,5,114,103,0,0,0,114,122,0,0,0,114, - 97,0,0,0,90,13,102,105,108,101,110,97,109,101,95,98, - 97,115,101,90,9,116,97,105,108,95,110,97,109,101,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,156,0, - 0,0,139,2,0,0,115,8,0,0,0,0,3,18,1,16, - 1,14,1,122,24,95,76,111,97,100,101,114,66,97,115,105, - 99,115,46,105,115,95,112,97,99,107,97,103,101,99,2,0, - 0,0,0,0,0,0,2,0,0,0,1,0,0,0,67,0, - 0,0,115,4,0,0,0,100,1,83,0,41,2,122,42,85, - 115,101,32,100,101,102,97,117,108,116,32,115,101,109,97,110, - 116,105,99,115,32,102,111,114,32,109,111,100,117,108,101,32, - 99,114,101,97,116,105,111,110,46,78,114,4,0,0,0,41, - 2,114,103,0,0,0,114,161,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,13,99,114,101,97, - 116,101,95,109,111,100,117,108,101,147,2,0,0,115,0,0, - 0,0,122,27,95,76,111,97,100,101,114,66,97,115,105,99, - 115,46,99,114,101,97,116,101,95,109,111,100,117,108,101,99, - 2,0,0,0,0,0,0,0,3,0,0,0,4,0,0,0, - 67,0,0,0,115,56,0,0,0,124,0,106,0,124,1,106, - 1,131,1,125,2,124,2,100,1,107,8,114,36,116,2,100, - 2,106,3,124,1,106,1,131,1,131,1,130,1,116,4,106, - 5,116,6,124,2,124,1,106,7,131,3,1,0,100,1,83, - 0,41,3,122,19,69,120,101,99,117,116,101,32,116,104,101, - 32,109,111,100,117,108,101,46,78,122,52,99,97,110,110,111, - 116,32,108,111,97,100,32,109,111,100,117,108,101,32,123,33, - 114,125,32,119,104,101,110,32,103,101,116,95,99,111,100,101, - 40,41,32,114,101,116,117,114,110,115,32,78,111,110,101,41, - 8,218,8,103,101,116,95,99,111,100,101,114,108,0,0,0, - 114,102,0,0,0,114,50,0,0,0,114,117,0,0,0,218, - 25,95,99,97,108,108,95,119,105,116,104,95,102,114,97,109, - 101,115,95,114,101,109,111,118,101,100,218,4,101,120,101,99, - 114,114,0,0,0,41,3,114,103,0,0,0,218,6,109,111, - 100,117,108,101,114,143,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,11,101,120,101,99,95,109, - 111,100,117,108,101,150,2,0,0,115,10,0,0,0,0,2, - 12,1,8,1,6,1,10,1,122,25,95,76,111,97,100,101, - 114,66,97,115,105,99,115,46,101,120,101,99,95,109,111,100, - 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 3,0,0,0,67,0,0,0,115,12,0,0,0,116,0,106, - 1,124,0,124,1,131,2,83,0,41,1,122,26,84,104,105, - 115,32,109,111,100,117,108,101,32,105,115,32,100,101,112,114, - 101,99,97,116,101,100,46,41,2,114,117,0,0,0,218,17, - 95,108,111,97,100,95,109,111,100,117,108,101,95,115,104,105, - 109,41,2,114,103,0,0,0,114,122,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,11,108,111, - 97,100,95,109,111,100,117,108,101,158,2,0,0,115,2,0, - 0,0,0,2,122,25,95,76,111,97,100,101,114,66,97,115, - 105,99,115,46,108,111,97,100,95,109,111,100,117,108,101,78, - 41,8,114,108,0,0,0,114,107,0,0,0,114,109,0,0, - 0,114,110,0,0,0,114,156,0,0,0,114,182,0,0,0, - 114,187,0,0,0,114,189,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,180, - 0,0,0,134,2,0,0,115,10,0,0,0,8,3,4,2, - 8,8,8,3,8,8,114,180,0,0,0,99,0,0,0,0, - 0,0,0,0,0,0,0,0,3,0,0,0,64,0,0,0, - 115,74,0,0,0,101,0,90,1,100,0,90,2,100,1,100, - 2,132,0,90,3,100,3,100,4,132,0,90,4,100,5,100, - 6,132,0,90,5,100,7,100,8,132,0,90,6,100,9,100, - 10,132,0,90,7,100,18,100,12,156,1,100,13,100,14,132, - 2,90,8,100,15,100,16,132,0,90,9,100,17,83,0,41, - 19,218,12,83,111,117,114,99,101,76,111,97,100,101,114,99, - 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, - 67,0,0,0,115,8,0,0,0,116,0,130,1,100,1,83, - 0,41,2,122,178,79,112,116,105,111,110,97,108,32,109,101, - 116,104,111,100,32,116,104,97,116,32,114,101,116,117,114,110, - 115,32,116,104,101,32,109,111,100,105,102,105,99,97,116,105, - 111,110,32,116,105,109,101,32,40,97,110,32,105,110,116,41, - 32,102,111,114,32,116,104,101,10,32,32,32,32,32,32,32, - 32,115,112,101,99,105,102,105,101,100,32,112,97,116,104,44, - 32,119,104,101,114,101,32,112,97,116,104,32,105,115,32,97, - 32,115,116,114,46,10,10,32,32,32,32,32,32,32,32,82, - 97,105,115,101,115,32,73,79,69,114,114,111,114,32,119,104, - 101,110,32,116,104,101,32,112,97,116,104,32,99,97,110,110, - 111,116,32,98,101,32,104,97,110,100,108,101,100,46,10,32, - 32,32,32,32,32,32,32,78,41,1,218,7,73,79,69,114, - 114,111,114,41,2,114,103,0,0,0,114,37,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,10, - 112,97,116,104,95,109,116,105,109,101,165,2,0,0,115,2, - 0,0,0,0,6,122,23,83,111,117,114,99,101,76,111,97, - 100,101,114,46,112,97,116,104,95,109,116,105,109,101,99,2, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,67, - 0,0,0,115,14,0,0,0,100,1,124,0,106,0,124,1, - 131,1,105,1,83,0,41,2,97,170,1,0,0,79,112,116, - 105,111,110,97,108,32,109,101,116,104,111,100,32,114,101,116, - 117,114,110,105,110,103,32,97,32,109,101,116,97,100,97,116, - 97,32,100,105,99,116,32,102,111,114,32,116,104,101,32,115, - 112,101,99,105,102,105,101,100,32,112,97,116,104,10,32,32, - 32,32,32,32,32,32,116,111,32,98,121,32,116,104,101,32, - 112,97,116,104,32,40,115,116,114,41,46,10,32,32,32,32, - 32,32,32,32,80,111,115,115,105,98,108,101,32,107,101,121, - 115,58,10,32,32,32,32,32,32,32,32,45,32,39,109,116, - 105,109,101,39,32,40,109,97,110,100,97,116,111,114,121,41, - 32,105,115,32,116,104,101,32,110,117,109,101,114,105,99,32, - 116,105,109,101,115,116,97,109,112,32,111,102,32,108,97,115, - 116,32,115,111,117,114,99,101,10,32,32,32,32,32,32,32, - 32,32,32,99,111,100,101,32,109,111,100,105,102,105,99,97, - 116,105,111,110,59,10,32,32,32,32,32,32,32,32,45,32, - 39,115,105,122,101,39,32,40,111,112,116,105,111,110,97,108, - 41,32,105,115,32,116,104,101,32,115,105,122,101,32,105,110, - 32,98,121,116,101,115,32,111,102,32,116,104,101,32,115,111, - 117,114,99,101,32,99,111,100,101,46,10,10,32,32,32,32, - 32,32,32,32,73,109,112,108,101,109,101,110,116,105,110,103, - 32,116,104,105,115,32,109,101,116,104,111,100,32,97,108,108, - 111,119,115,32,116,104,101,32,108,111,97,100,101,114,32,116, - 111,32,114,101,97,100,32,98,121,116,101,99,111,100,101,32, - 102,105,108,101,115,46,10,32,32,32,32,32,32,32,32,82, - 97,105,115,101,115,32,73,79,69,114,114,111,114,32,119,104, - 101,110,32,116,104,101,32,112,97,116,104,32,99,97,110,110, - 111,116,32,98,101,32,104,97,110,100,108,101,100,46,10,32, - 32,32,32,32,32,32,32,114,129,0,0,0,41,1,114,192, - 0,0,0,41,2,114,103,0,0,0,114,37,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,10, - 112,97,116,104,95,115,116,97,116,115,173,2,0,0,115,2, - 0,0,0,0,11,122,23,83,111,117,114,99,101,76,111,97, - 100,101,114,46,112,97,116,104,95,115,116,97,116,115,99,4, - 0,0,0,0,0,0,0,4,0,0,0,3,0,0,0,67, - 0,0,0,115,12,0,0,0,124,0,106,0,124,2,124,3, - 131,2,83,0,41,1,122,228,79,112,116,105,111,110,97,108, - 32,109,101,116,104,111,100,32,119,104,105,99,104,32,119,114, - 105,116,101,115,32,100,97,116,97,32,40,98,121,116,101,115, - 41,32,116,111,32,97,32,102,105,108,101,32,112,97,116,104, - 32,40,97,32,115,116,114,41,46,10,10,32,32,32,32,32, - 32,32,32,73,109,112,108,101,109,101,110,116,105,110,103,32, - 116,104,105,115,32,109,101,116,104,111,100,32,97,108,108,111, - 119,115,32,102,111,114,32,116,104,101,32,119,114,105,116,105, - 110,103,32,111,102,32,98,121,116,101,99,111,100,101,32,102, - 105,108,101,115,46,10,10,32,32,32,32,32,32,32,32,84, - 104,101,32,115,111,117,114,99,101,32,112,97,116,104,32,105, - 115,32,110,101,101,100,101,100,32,105,110,32,111,114,100,101, - 114,32,116,111,32,99,111,114,114,101,99,116,108,121,32,116, - 114,97,110,115,102,101,114,32,112,101,114,109,105,115,115,105, - 111,110,115,10,32,32,32,32,32,32,32,32,41,1,218,8, - 115,101,116,95,100,97,116,97,41,4,114,103,0,0,0,114, - 93,0,0,0,90,10,99,97,99,104,101,95,112,97,116,104, - 114,56,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,218,15,95,99,97,99,104,101,95,98,121,116, - 101,99,111,100,101,186,2,0,0,115,2,0,0,0,0,8, - 122,28,83,111,117,114,99,101,76,111,97,100,101,114,46,95, - 99,97,99,104,101,95,98,121,116,101,99,111,100,101,99,3, - 0,0,0,0,0,0,0,3,0,0,0,1,0,0,0,67, - 0,0,0,115,4,0,0,0,100,1,83,0,41,2,122,150, - 79,112,116,105,111,110,97,108,32,109,101,116,104,111,100,32, - 119,104,105,99,104,32,119,114,105,116,101,115,32,100,97,116, - 97,32,40,98,121,116,101,115,41,32,116,111,32,97,32,102, - 105,108,101,32,112,97,116,104,32,40,97,32,115,116,114,41, - 46,10,10,32,32,32,32,32,32,32,32,73,109,112,108,101, - 109,101,110,116,105,110,103,32,116,104,105,115,32,109,101,116, - 104,111,100,32,97,108,108,111,119,115,32,102,111,114,32,116, - 104,101,32,119,114,105,116,105,110,103,32,111,102,32,98,121, - 116,101,99,111,100,101,32,102,105,108,101,115,46,10,32,32, - 32,32,32,32,32,32,78,114,4,0,0,0,41,3,114,103, - 0,0,0,114,37,0,0,0,114,56,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,194,0,0, - 0,196,2,0,0,115,0,0,0,0,122,21,83,111,117,114, - 99,101,76,111,97,100,101,114,46,115,101,116,95,100,97,116, - 97,99,2,0,0,0,0,0,0,0,5,0,0,0,16,0, - 0,0,67,0,0,0,115,84,0,0,0,124,0,106,0,124, - 1,131,1,125,2,121,14,124,0,106,1,124,2,131,1,125, - 3,87,0,110,50,4,0,116,2,107,10,114,74,1,0,125, - 4,1,0,122,22,116,3,100,1,100,2,124,1,144,1,131, - 1,124,4,130,2,87,0,89,0,100,3,100,3,125,4,126, - 4,88,0,110,2,88,0,116,4,124,3,131,1,83,0,41, - 4,122,52,67,111,110,99,114,101,116,101,32,105,109,112,108, - 101,109,101,110,116,97,116,105,111,110,32,111,102,32,73,110, - 115,112,101,99,116,76,111,97,100,101,114,46,103,101,116,95, - 115,111,117,114,99,101,46,122,39,115,111,117,114,99,101,32, - 110,111,116,32,97,118,97,105,108,97,98,108,101,32,116,104, - 114,111,117,103,104,32,103,101,116,95,100,97,116,97,40,41, - 114,101,0,0,0,78,41,5,114,154,0,0,0,218,8,103, - 101,116,95,100,97,116,97,114,42,0,0,0,114,102,0,0, - 0,114,152,0,0,0,41,5,114,103,0,0,0,114,122,0, - 0,0,114,37,0,0,0,114,150,0,0,0,218,3,101,120, - 99,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 218,10,103,101,116,95,115,111,117,114,99,101,203,2,0,0, - 115,14,0,0,0,0,2,10,1,2,1,14,1,16,1,6, - 1,28,1,122,23,83,111,117,114,99,101,76,111,97,100,101, - 114,46,103,101,116,95,115,111,117,114,99,101,114,31,0,0, - 0,41,1,218,9,95,111,112,116,105,109,105,122,101,99,3, - 0,0,0,1,0,0,0,4,0,0,0,9,0,0,0,67, - 0,0,0,115,26,0,0,0,116,0,106,1,116,2,124,1, - 124,2,100,1,100,2,100,3,100,4,124,3,144,2,131,4, - 83,0,41,5,122,130,82,101,116,117,114,110,32,116,104,101, - 32,99,111,100,101,32,111,98,106,101,99,116,32,99,111,109, - 112,105,108,101,100,32,102,114,111,109,32,115,111,117,114,99, - 101,46,10,10,32,32,32,32,32,32,32,32,84,104,101,32, - 39,100,97,116,97,39,32,97,114,103,117,109,101,110,116,32, - 99,97,110,32,98,101,32,97,110,121,32,111,98,106,101,99, - 116,32,116,121,112,101,32,116,104,97,116,32,99,111,109,112, - 105,108,101,40,41,32,115,117,112,112,111,114,116,115,46,10, - 32,32,32,32,32,32,32,32,114,185,0,0,0,218,12,100, - 111,110,116,95,105,110,104,101,114,105,116,84,114,71,0,0, - 0,41,3,114,117,0,0,0,114,184,0,0,0,218,7,99, - 111,109,112,105,108,101,41,4,114,103,0,0,0,114,56,0, - 0,0,114,37,0,0,0,114,199,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,14,115,111,117, - 114,99,101,95,116,111,95,99,111,100,101,213,2,0,0,115, - 4,0,0,0,0,5,14,1,122,27,83,111,117,114,99,101, - 76,111,97,100,101,114,46,115,111,117,114,99,101,95,116,111, - 95,99,111,100,101,99,2,0,0,0,0,0,0,0,10,0, - 0,0,43,0,0,0,67,0,0,0,115,106,1,0,0,124, - 0,106,0,124,1,131,1,125,2,100,1,125,3,121,12,116, - 1,124,2,131,1,125,4,87,0,110,24,4,0,116,2,107, - 10,114,50,1,0,1,0,1,0,100,1,125,4,89,0,110, - 174,88,0,121,14,124,0,106,3,124,2,131,1,125,5,87, - 0,110,20,4,0,116,4,107,10,114,86,1,0,1,0,1, - 0,89,0,110,138,88,0,116,5,124,5,100,2,25,0,131, - 1,125,3,121,14,124,0,106,6,124,4,131,1,125,6,87, - 0,110,20,4,0,116,7,107,10,114,134,1,0,1,0,1, - 0,89,0,110,90,88,0,121,26,116,8,124,6,100,3,124, - 5,100,4,124,1,100,5,124,4,144,3,131,1,125,7,87, - 0,110,24,4,0,116,9,116,10,102,2,107,10,114,186,1, - 0,1,0,1,0,89,0,110,38,88,0,116,11,106,12,100, - 6,124,4,124,2,131,3,1,0,116,13,124,7,100,4,124, - 1,100,7,124,4,100,8,124,2,144,3,131,1,83,0,124, - 0,106,6,124,2,131,1,125,8,124,0,106,14,124,8,124, - 2,131,2,125,9,116,11,106,12,100,9,124,2,131,2,1, - 0,116,15,106,16,12,0,144,1,114,102,124,4,100,1,107, - 9,144,1,114,102,124,3,100,1,107,9,144,1,114,102,116, - 17,124,9,124,3,116,18,124,8,131,1,131,3,125,6,121, - 30,124,0,106,19,124,2,124,4,124,6,131,3,1,0,116, - 11,106,12,100,10,124,4,131,2,1,0,87,0,110,22,4, - 0,116,2,107,10,144,1,114,100,1,0,1,0,1,0,89, - 0,110,2,88,0,124,9,83,0,41,11,122,190,67,111,110, - 99,114,101,116,101,32,105,109,112,108,101,109,101,110,116,97, - 116,105,111,110,32,111,102,32,73,110,115,112,101,99,116,76, - 111,97,100,101,114,46,103,101,116,95,99,111,100,101,46,10, - 10,32,32,32,32,32,32,32,32,82,101,97,100,105,110,103, - 32,111,102,32,98,121,116,101,99,111,100,101,32,114,101,113, - 117,105,114,101,115,32,112,97,116,104,95,115,116,97,116,115, - 32,116,111,32,98,101,32,105,109,112,108,101,109,101,110,116, - 101,100,46,32,84,111,32,119,114,105,116,101,10,32,32,32, - 32,32,32,32,32,98,121,116,101,99,111,100,101,44,32,115, - 101,116,95,100,97,116,97,32,109,117,115,116,32,97,108,115, - 111,32,98,101,32,105,109,112,108,101,109,101,110,116,101,100, - 46,10,10,32,32,32,32,32,32,32,32,78,114,129,0,0, - 0,114,135,0,0,0,114,101,0,0,0,114,37,0,0,0, - 122,13,123,125,32,109,97,116,99,104,101,115,32,123,125,114, - 92,0,0,0,114,93,0,0,0,122,19,99,111,100,101,32, - 111,98,106,101,99,116,32,102,114,111,109,32,123,125,122,10, - 119,114,111,116,101,32,123,33,114,125,41,20,114,154,0,0, - 0,114,82,0,0,0,114,69,0,0,0,114,193,0,0,0, - 114,191,0,0,0,114,16,0,0,0,114,196,0,0,0,114, - 42,0,0,0,114,138,0,0,0,114,102,0,0,0,114,133, - 0,0,0,114,117,0,0,0,114,132,0,0,0,114,144,0, - 0,0,114,202,0,0,0,114,8,0,0,0,218,19,100,111, - 110,116,95,119,114,105,116,101,95,98,121,116,101,99,111,100, - 101,114,147,0,0,0,114,33,0,0,0,114,195,0,0,0, - 41,10,114,103,0,0,0,114,122,0,0,0,114,93,0,0, - 0,114,136,0,0,0,114,92,0,0,0,218,2,115,116,114, - 56,0,0,0,218,10,98,121,116,101,115,95,100,97,116,97, - 114,150,0,0,0,90,11,99,111,100,101,95,111,98,106,101, - 99,116,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,183,0,0,0,221,2,0,0,115,78,0,0,0,0, - 7,10,1,4,1,2,1,12,1,14,1,10,2,2,1,14, - 1,14,1,6,2,12,1,2,1,14,1,14,1,6,2,2, - 1,6,1,8,1,12,1,18,1,6,2,8,1,6,1,10, - 1,4,1,8,1,10,1,12,1,12,1,20,1,10,1,6, - 1,10,1,2,1,14,1,16,1,16,1,6,1,122,21,83, - 111,117,114,99,101,76,111,97,100,101,114,46,103,101,116,95, - 99,111,100,101,78,114,90,0,0,0,41,10,114,108,0,0, - 0,114,107,0,0,0,114,109,0,0,0,114,192,0,0,0, - 114,193,0,0,0,114,195,0,0,0,114,194,0,0,0,114, - 198,0,0,0,114,202,0,0,0,114,183,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,190,0,0,0,163,2,0,0,115,14,0,0,0, - 8,2,8,8,8,13,8,10,8,7,8,10,14,8,114,190, - 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, - 4,0,0,0,0,0,0,0,115,76,0,0,0,101,0,90, - 1,100,0,90,2,100,1,90,3,100,2,100,3,132,0,90, - 4,100,4,100,5,132,0,90,5,100,6,100,7,132,0,90, - 6,101,7,135,0,102,1,100,8,100,9,132,8,131,1,90, - 8,101,7,100,10,100,11,132,0,131,1,90,9,100,12,100, - 13,132,0,90,10,135,0,83,0,41,14,218,10,70,105,108, - 101,76,111,97,100,101,114,122,103,66,97,115,101,32,102,105, - 108,101,32,108,111,97,100,101,114,32,99,108,97,115,115,32, - 119,104,105,99,104,32,105,109,112,108,101,109,101,110,116,115, - 32,116,104,101,32,108,111,97,100,101,114,32,112,114,111,116, - 111,99,111,108,32,109,101,116,104,111,100,115,32,116,104,97, - 116,10,32,32,32,32,114,101,113,117,105,114,101,32,102,105, - 108,101,32,115,121,115,116,101,109,32,117,115,97,103,101,46, - 99,3,0,0,0,0,0,0,0,3,0,0,0,2,0,0, - 0,67,0,0,0,115,16,0,0,0,124,1,124,0,95,0, - 124,2,124,0,95,1,100,1,83,0,41,2,122,75,67,97, - 99,104,101,32,116,104,101,32,109,111,100,117,108,101,32,110, - 97,109,101,32,97,110,100,32,116,104,101,32,112,97,116,104, - 32,116,111,32,116,104,101,32,102,105,108,101,32,102,111,117, - 110,100,32,98,121,32,116,104,101,10,32,32,32,32,32,32, - 32,32,102,105,110,100,101,114,46,78,41,2,114,101,0,0, - 0,114,37,0,0,0,41,3,114,103,0,0,0,114,122,0, - 0,0,114,37,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,181,0,0,0,22,3,0,0,115, - 4,0,0,0,0,3,6,1,122,19,70,105,108,101,76,111, - 97,100,101,114,46,95,95,105,110,105,116,95,95,99,2,0, - 0,0,0,0,0,0,2,0,0,0,2,0,0,0,67,0, - 0,0,115,24,0,0,0,124,0,106,0,124,1,106,0,107, - 2,111,22,124,0,106,1,124,1,106,1,107,2,83,0,41, - 1,78,41,2,218,9,95,95,99,108,97,115,115,95,95,114, - 114,0,0,0,41,2,114,103,0,0,0,218,5,111,116,104, - 101,114,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,218,6,95,95,101,113,95,95,28,3,0,0,115,4,0, - 0,0,0,1,12,1,122,17,70,105,108,101,76,111,97,100, - 101,114,46,95,95,101,113,95,95,99,1,0,0,0,0,0, - 0,0,1,0,0,0,3,0,0,0,67,0,0,0,115,20, - 0,0,0,116,0,124,0,106,1,131,1,116,0,124,0,106, - 2,131,1,65,0,83,0,41,1,78,41,3,218,4,104,97, - 115,104,114,101,0,0,0,114,37,0,0,0,41,1,114,103, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,8,95,95,104,97,115,104,95,95,32,3,0,0, - 115,2,0,0,0,0,1,122,19,70,105,108,101,76,111,97, - 100,101,114,46,95,95,104,97,115,104,95,95,99,2,0,0, - 0,0,0,0,0,2,0,0,0,3,0,0,0,3,0,0, - 0,115,16,0,0,0,116,0,116,1,124,0,131,2,106,2, - 124,1,131,1,83,0,41,1,122,100,76,111,97,100,32,97, - 32,109,111,100,117,108,101,32,102,114,111,109,32,97,32,102, - 105,108,101,46,10,10,32,32,32,32,32,32,32,32,84,104, - 105,115,32,109,101,116,104,111,100,32,105,115,32,100,101,112, - 114,101,99,97,116,101,100,46,32,32,85,115,101,32,101,120, - 101,99,95,109,111,100,117,108,101,40,41,32,105,110,115,116, - 101,97,100,46,10,10,32,32,32,32,32,32,32,32,41,3, - 218,5,115,117,112,101,114,114,206,0,0,0,114,189,0,0, - 0,41,2,114,103,0,0,0,114,122,0,0,0,41,1,114, - 207,0,0,0,114,4,0,0,0,114,6,0,0,0,114,189, - 0,0,0,35,3,0,0,115,2,0,0,0,0,10,122,22, - 70,105,108,101,76,111,97,100,101,114,46,108,111,97,100,95, - 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,2, - 0,0,0,1,0,0,0,67,0,0,0,115,6,0,0,0, - 124,0,106,0,83,0,41,1,122,58,82,101,116,117,114,110, - 32,116,104,101,32,112,97,116,104,32,116,111,32,116,104,101, - 32,115,111,117,114,99,101,32,102,105,108,101,32,97,115,32, - 102,111,117,110,100,32,98,121,32,116,104,101,32,102,105,110, - 100,101,114,46,41,1,114,37,0,0,0,41,2,114,103,0, - 0,0,114,122,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,154,0,0,0,47,3,0,0,115, - 2,0,0,0,0,3,122,23,70,105,108,101,76,111,97,100, - 101,114,46,103,101,116,95,102,105,108,101,110,97,109,101,99, - 2,0,0,0,0,0,0,0,3,0,0,0,9,0,0,0, - 67,0,0,0,115,32,0,0,0,116,0,106,1,124,1,100, - 1,131,2,143,10,125,2,124,2,106,2,131,0,83,0,81, - 0,82,0,88,0,100,2,83,0,41,3,122,39,82,101,116, - 117,114,110,32,116,104,101,32,100,97,116,97,32,102,114,111, - 109,32,112,97,116,104,32,97,115,32,114,97,119,32,98,121, - 116,101,115,46,218,1,114,78,41,3,114,52,0,0,0,114, - 53,0,0,0,90,4,114,101,97,100,41,3,114,103,0,0, - 0,114,37,0,0,0,114,57,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,196,0,0,0,52, - 3,0,0,115,4,0,0,0,0,2,14,1,122,19,70,105, - 108,101,76,111,97,100,101,114,46,103,101,116,95,100,97,116, - 97,41,11,114,108,0,0,0,114,107,0,0,0,114,109,0, - 0,0,114,110,0,0,0,114,181,0,0,0,114,209,0,0, - 0,114,211,0,0,0,114,119,0,0,0,114,189,0,0,0, - 114,154,0,0,0,114,196,0,0,0,114,4,0,0,0,114, - 4,0,0,0,41,1,114,207,0,0,0,114,6,0,0,0, - 114,206,0,0,0,17,3,0,0,115,14,0,0,0,8,3, - 4,2,8,6,8,4,8,3,16,12,12,5,114,206,0,0, - 0,99,0,0,0,0,0,0,0,0,0,0,0,0,3,0, - 0,0,64,0,0,0,115,46,0,0,0,101,0,90,1,100, - 0,90,2,100,1,90,3,100,2,100,3,132,0,90,4,100, - 4,100,5,132,0,90,5,100,6,100,7,156,1,100,8,100, - 9,132,2,90,6,100,10,83,0,41,11,218,16,83,111,117, - 114,99,101,70,105,108,101,76,111,97,100,101,114,122,62,67, - 111,110,99,114,101,116,101,32,105,109,112,108,101,109,101,110, - 116,97,116,105,111,110,32,111,102,32,83,111,117,114,99,101, - 76,111,97,100,101,114,32,117,115,105,110,103,32,116,104,101, - 32,102,105,108,101,32,115,121,115,116,101,109,46,99,2,0, - 0,0,0,0,0,0,3,0,0,0,3,0,0,0,67,0, - 0,0,115,22,0,0,0,116,0,124,1,131,1,125,2,124, - 2,106,1,124,2,106,2,100,1,156,2,83,0,41,2,122, - 33,82,101,116,117,114,110,32,116,104,101,32,109,101,116,97, - 100,97,116,97,32,102,111,114,32,116,104,101,32,112,97,116, - 104,46,41,2,122,5,109,116,105,109,101,122,4,115,105,122, - 101,41,3,114,41,0,0,0,218,8,115,116,95,109,116,105, - 109,101,90,7,115,116,95,115,105,122,101,41,3,114,103,0, - 0,0,114,37,0,0,0,114,204,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,193,0,0,0, - 62,3,0,0,115,4,0,0,0,0,2,8,1,122,27,83, - 111,117,114,99,101,70,105,108,101,76,111,97,100,101,114,46, - 112,97,116,104,95,115,116,97,116,115,99,4,0,0,0,0, - 0,0,0,5,0,0,0,5,0,0,0,67,0,0,0,115, - 26,0,0,0,116,0,124,1,131,1,125,4,124,0,106,1, - 124,2,124,3,100,1,124,4,144,1,131,2,83,0,41,2, - 78,218,5,95,109,111,100,101,41,2,114,100,0,0,0,114, - 194,0,0,0,41,5,114,103,0,0,0,114,93,0,0,0, - 114,92,0,0,0,114,56,0,0,0,114,44,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,195, - 0,0,0,67,3,0,0,115,4,0,0,0,0,2,8,1, - 122,32,83,111,117,114,99,101,70,105,108,101,76,111,97,100, - 101,114,46,95,99,97,99,104,101,95,98,121,116,101,99,111, - 100,101,105,182,1,0,0,41,1,114,216,0,0,0,99,3, - 0,0,0,1,0,0,0,9,0,0,0,17,0,0,0,67, - 0,0,0,115,250,0,0,0,116,0,124,1,131,1,92,2, - 125,4,125,5,103,0,125,6,120,40,124,4,114,56,116,1, - 124,4,131,1,12,0,114,56,116,0,124,4,131,1,92,2, - 125,4,125,7,124,6,106,2,124,7,131,1,1,0,113,18, - 87,0,120,108,116,3,124,6,131,1,68,0,93,96,125,7, - 116,4,124,4,124,7,131,2,125,4,121,14,116,5,106,6, - 124,4,131,1,1,0,87,0,113,68,4,0,116,7,107,10, - 114,118,1,0,1,0,1,0,119,68,89,0,113,68,4,0, - 116,8,107,10,114,162,1,0,125,8,1,0,122,18,116,9, - 106,10,100,1,124,4,124,8,131,3,1,0,100,2,83,0, - 100,2,125,8,126,8,88,0,113,68,88,0,113,68,87,0, - 121,28,116,11,124,1,124,2,124,3,131,3,1,0,116,9, - 106,10,100,3,124,1,131,2,1,0,87,0,110,48,4,0, - 116,8,107,10,114,244,1,0,125,8,1,0,122,20,116,9, - 106,10,100,1,124,1,124,8,131,3,1,0,87,0,89,0, - 100,2,100,2,125,8,126,8,88,0,110,2,88,0,100,2, - 83,0,41,4,122,27,87,114,105,116,101,32,98,121,116,101, - 115,32,100,97,116,97,32,116,111,32,97,32,102,105,108,101, - 46,122,27,99,111,117,108,100,32,110,111,116,32,99,114,101, - 97,116,101,32,123,33,114,125,58,32,123,33,114,125,78,122, - 12,99,114,101,97,116,101,100,32,123,33,114,125,41,12,114, - 40,0,0,0,114,48,0,0,0,114,160,0,0,0,114,35, - 0,0,0,114,30,0,0,0,114,3,0,0,0,90,5,109, - 107,100,105,114,218,15,70,105,108,101,69,120,105,115,116,115, - 69,114,114,111,114,114,42,0,0,0,114,117,0,0,0,114, - 132,0,0,0,114,58,0,0,0,41,9,114,103,0,0,0, - 114,37,0,0,0,114,56,0,0,0,114,216,0,0,0,218, - 6,112,97,114,101,110,116,114,97,0,0,0,114,29,0,0, - 0,114,25,0,0,0,114,197,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,194,0,0,0,72, - 3,0,0,115,42,0,0,0,0,2,12,1,4,2,16,1, - 12,1,14,2,14,1,10,1,2,1,14,1,14,2,6,1, - 16,3,6,1,8,1,20,1,2,1,12,1,16,1,16,2, - 8,1,122,25,83,111,117,114,99,101,70,105,108,101,76,111, - 97,100,101,114,46,115,101,116,95,100,97,116,97,78,41,7, - 114,108,0,0,0,114,107,0,0,0,114,109,0,0,0,114, - 110,0,0,0,114,193,0,0,0,114,195,0,0,0,114,194, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,114,214,0,0,0,58,3,0,0, - 115,8,0,0,0,8,2,4,2,8,5,8,5,114,214,0, - 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,2, - 0,0,0,64,0,0,0,115,32,0,0,0,101,0,90,1, - 100,0,90,2,100,1,90,3,100,2,100,3,132,0,90,4, - 100,4,100,5,132,0,90,5,100,6,83,0,41,7,218,20, - 83,111,117,114,99,101,108,101,115,115,70,105,108,101,76,111, - 97,100,101,114,122,45,76,111,97,100,101,114,32,119,104,105, - 99,104,32,104,97,110,100,108,101,115,32,115,111,117,114,99, - 101,108,101,115,115,32,102,105,108,101,32,105,109,112,111,114, - 116,115,46,99,2,0,0,0,0,0,0,0,5,0,0,0, - 6,0,0,0,67,0,0,0,115,56,0,0,0,124,0,106, - 0,124,1,131,1,125,2,124,0,106,1,124,2,131,1,125, - 3,116,2,124,3,100,1,124,1,100,2,124,2,144,2,131, - 1,125,4,116,3,124,4,100,1,124,1,100,3,124,2,144, - 2,131,1,83,0,41,4,78,114,101,0,0,0,114,37,0, - 0,0,114,92,0,0,0,41,4,114,154,0,0,0,114,196, - 0,0,0,114,138,0,0,0,114,144,0,0,0,41,5,114, - 103,0,0,0,114,122,0,0,0,114,37,0,0,0,114,56, - 0,0,0,114,205,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,114,183,0,0,0,107,3,0,0, - 115,8,0,0,0,0,1,10,1,10,1,18,1,122,29,83, - 111,117,114,99,101,108,101,115,115,70,105,108,101,76,111,97, - 100,101,114,46,103,101,116,95,99,111,100,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, - 0,115,4,0,0,0,100,1,83,0,41,2,122,39,82,101, - 116,117,114,110,32,78,111,110,101,32,97,115,32,116,104,101, - 114,101,32,105,115,32,110,111,32,115,111,117,114,99,101,32, - 99,111,100,101,46,78,114,4,0,0,0,41,2,114,103,0, - 0,0,114,122,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,198,0,0,0,113,3,0,0,115, - 2,0,0,0,0,2,122,31,83,111,117,114,99,101,108,101, - 115,115,70,105,108,101,76,111,97,100,101,114,46,103,101,116, - 95,115,111,117,114,99,101,78,41,6,114,108,0,0,0,114, - 107,0,0,0,114,109,0,0,0,114,110,0,0,0,114,183, - 0,0,0,114,198,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,219,0,0, - 0,103,3,0,0,115,6,0,0,0,8,2,4,2,8,6, - 114,219,0,0,0,99,0,0,0,0,0,0,0,0,0,0, - 0,0,3,0,0,0,64,0,0,0,115,92,0,0,0,101, - 0,90,1,100,0,90,2,100,1,90,3,100,2,100,3,132, - 0,90,4,100,4,100,5,132,0,90,5,100,6,100,7,132, - 0,90,6,100,8,100,9,132,0,90,7,100,10,100,11,132, - 0,90,8,100,12,100,13,132,0,90,9,100,14,100,15,132, - 0,90,10,100,16,100,17,132,0,90,11,101,12,100,18,100, - 19,132,0,131,1,90,13,100,20,83,0,41,21,218,19,69, - 120,116,101,110,115,105,111,110,70,105,108,101,76,111,97,100, - 101,114,122,93,76,111,97,100,101,114,32,102,111,114,32,101, - 120,116,101,110,115,105,111,110,32,109,111,100,117,108,101,115, - 46,10,10,32,32,32,32,84,104,101,32,99,111,110,115,116, - 114,117,99,116,111,114,32,105,115,32,100,101,115,105,103,110, - 101,100,32,116,111,32,119,111,114,107,32,119,105,116,104,32, - 70,105,108,101,70,105,110,100,101,114,46,10,10,32,32,32, - 32,99,3,0,0,0,0,0,0,0,3,0,0,0,2,0, - 0,0,67,0,0,0,115,16,0,0,0,124,1,124,0,95, - 0,124,2,124,0,95,1,100,0,83,0,41,1,78,41,2, - 114,101,0,0,0,114,37,0,0,0,41,3,114,103,0,0, - 0,114,101,0,0,0,114,37,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,181,0,0,0,130, - 3,0,0,115,4,0,0,0,0,1,6,1,122,28,69,120, - 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, - 114,46,95,95,105,110,105,116,95,95,99,2,0,0,0,0, - 0,0,0,2,0,0,0,2,0,0,0,67,0,0,0,115, - 24,0,0,0,124,0,106,0,124,1,106,0,107,2,111,22, - 124,0,106,1,124,1,106,1,107,2,83,0,41,1,78,41, - 2,114,207,0,0,0,114,114,0,0,0,41,2,114,103,0, - 0,0,114,208,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,209,0,0,0,134,3,0,0,115, - 4,0,0,0,0,1,12,1,122,26,69,120,116,101,110,115, - 105,111,110,70,105,108,101,76,111,97,100,101,114,46,95,95, - 101,113,95,95,99,1,0,0,0,0,0,0,0,1,0,0, - 0,3,0,0,0,67,0,0,0,115,20,0,0,0,116,0, - 124,0,106,1,131,1,116,0,124,0,106,2,131,1,65,0, - 83,0,41,1,78,41,3,114,210,0,0,0,114,101,0,0, - 0,114,37,0,0,0,41,1,114,103,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,211,0,0, - 0,138,3,0,0,115,2,0,0,0,0,1,122,28,69,120, - 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, - 114,46,95,95,104,97,115,104,95,95,99,2,0,0,0,0, - 0,0,0,3,0,0,0,4,0,0,0,67,0,0,0,115, - 36,0,0,0,116,0,106,1,116,2,106,3,124,1,131,2, - 125,2,116,0,106,4,100,1,124,1,106,5,124,0,106,6, - 131,3,1,0,124,2,83,0,41,2,122,38,67,114,101,97, - 116,101,32,97,110,32,117,110,105,116,105,97,108,105,122,101, - 100,32,101,120,116,101,110,115,105,111,110,32,109,111,100,117, - 108,101,122,38,101,120,116,101,110,115,105,111,110,32,109,111, - 100,117,108,101,32,123,33,114,125,32,108,111,97,100,101,100, - 32,102,114,111,109,32,123,33,114,125,41,7,114,117,0,0, - 0,114,184,0,0,0,114,142,0,0,0,90,14,99,114,101, - 97,116,101,95,100,121,110,97,109,105,99,114,132,0,0,0, - 114,101,0,0,0,114,37,0,0,0,41,3,114,103,0,0, - 0,114,161,0,0,0,114,186,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,182,0,0,0,141, - 3,0,0,115,10,0,0,0,0,2,4,1,10,1,6,1, - 12,1,122,33,69,120,116,101,110,115,105,111,110,70,105,108, - 101,76,111,97,100,101,114,46,99,114,101,97,116,101,95,109, - 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, - 0,0,4,0,0,0,67,0,0,0,115,36,0,0,0,116, - 0,106,1,116,2,106,3,124,1,131,2,1,0,116,0,106, - 4,100,1,124,0,106,5,124,0,106,6,131,3,1,0,100, - 2,83,0,41,3,122,30,73,110,105,116,105,97,108,105,122, - 101,32,97,110,32,101,120,116,101,110,115,105,111,110,32,109, - 111,100,117,108,101,122,40,101,120,116,101,110,115,105,111,110, - 32,109,111,100,117,108,101,32,123,33,114,125,32,101,120,101, - 99,117,116,101,100,32,102,114,111,109,32,123,33,114,125,78, - 41,7,114,117,0,0,0,114,184,0,0,0,114,142,0,0, - 0,90,12,101,120,101,99,95,100,121,110,97,109,105,99,114, - 132,0,0,0,114,101,0,0,0,114,37,0,0,0,41,2, - 114,103,0,0,0,114,186,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,187,0,0,0,149,3, - 0,0,115,6,0,0,0,0,2,14,1,6,1,122,31,69, - 120,116,101,110,115,105,111,110,70,105,108,101,76,111,97,100, - 101,114,46,101,120,101,99,95,109,111,100,117,108,101,99,2, - 0,0,0,0,0,0,0,2,0,0,0,4,0,0,0,3, - 0,0,0,115,36,0,0,0,116,0,124,0,106,1,131,1, - 100,1,25,0,137,0,116,2,135,0,102,1,100,2,100,3, - 132,8,116,3,68,0,131,1,131,1,83,0,41,4,122,49, - 82,101,116,117,114,110,32,84,114,117,101,32,105,102,32,116, - 104,101,32,101,120,116,101,110,115,105,111,110,32,109,111,100, - 117,108,101,32,105,115,32,97,32,112,97,99,107,97,103,101, - 46,114,31,0,0,0,99,1,0,0,0,0,0,0,0,2, - 0,0,0,4,0,0,0,51,0,0,0,115,26,0,0,0, - 124,0,93,18,125,1,136,0,100,0,124,1,23,0,107,2, - 86,0,1,0,113,2,100,1,83,0,41,2,114,181,0,0, - 0,78,114,4,0,0,0,41,2,114,24,0,0,0,218,6, - 115,117,102,102,105,120,41,1,218,9,102,105,108,101,95,110, - 97,109,101,114,4,0,0,0,114,6,0,0,0,250,9,60, - 103,101,110,101,120,112,114,62,158,3,0,0,115,2,0,0, - 0,4,1,122,49,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, - 97,103,101,46,60,108,111,99,97,108,115,62,46,60,103,101, - 110,101,120,112,114,62,41,4,114,40,0,0,0,114,37,0, - 0,0,218,3,97,110,121,218,18,69,88,84,69,78,83,73, - 79,78,95,83,85,70,70,73,88,69,83,41,2,114,103,0, - 0,0,114,122,0,0,0,114,4,0,0,0,41,1,114,222, - 0,0,0,114,6,0,0,0,114,156,0,0,0,155,3,0, - 0,115,6,0,0,0,0,2,14,1,12,1,122,30,69,120, - 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, - 114,46,105,115,95,112,97,99,107,97,103,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, - 0,115,4,0,0,0,100,1,83,0,41,2,122,63,82,101, - 116,117,114,110,32,78,111,110,101,32,97,115,32,97,110,32, - 101,120,116,101,110,115,105,111,110,32,109,111,100,117,108,101, - 32,99,97,110,110,111,116,32,99,114,101,97,116,101,32,97, - 32,99,111,100,101,32,111,98,106,101,99,116,46,78,114,4, - 0,0,0,41,2,114,103,0,0,0,114,122,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,183, - 0,0,0,161,3,0,0,115,2,0,0,0,0,2,122,28, - 69,120,116,101,110,115,105,111,110,70,105,108,101,76,111,97, - 100,101,114,46,103,101,116,95,99,111,100,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, - 0,115,4,0,0,0,100,1,83,0,41,2,122,53,82,101, - 116,117,114,110,32,78,111,110,101,32,97,115,32,101,120,116, - 101,110,115,105,111,110,32,109,111,100,117,108,101,115,32,104, - 97,118,101,32,110,111,32,115,111,117,114,99,101,32,99,111, - 100,101,46,78,114,4,0,0,0,41,2,114,103,0,0,0, - 114,122,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,198,0,0,0,165,3,0,0,115,2,0, - 0,0,0,2,122,30,69,120,116,101,110,115,105,111,110,70, - 105,108,101,76,111,97,100,101,114,46,103,101,116,95,115,111, - 117,114,99,101,99,2,0,0,0,0,0,0,0,2,0,0, - 0,1,0,0,0,67,0,0,0,115,6,0,0,0,124,0, - 106,0,83,0,41,1,122,58,82,101,116,117,114,110,32,116, - 104,101,32,112,97,116,104,32,116,111,32,116,104,101,32,115, - 111,117,114,99,101,32,102,105,108,101,32,97,115,32,102,111, - 117,110,100,32,98,121,32,116,104,101,32,102,105,110,100,101, - 114,46,41,1,114,37,0,0,0,41,2,114,103,0,0,0, - 114,122,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,154,0,0,0,169,3,0,0,115,2,0, - 0,0,0,3,122,32,69,120,116,101,110,115,105,111,110,70, - 105,108,101,76,111,97,100,101,114,46,103,101,116,95,102,105, - 108,101,110,97,109,101,78,41,14,114,108,0,0,0,114,107, - 0,0,0,114,109,0,0,0,114,110,0,0,0,114,181,0, - 0,0,114,209,0,0,0,114,211,0,0,0,114,182,0,0, - 0,114,187,0,0,0,114,156,0,0,0,114,183,0,0,0, - 114,198,0,0,0,114,119,0,0,0,114,154,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,220,0,0,0,122,3,0,0,115,20,0,0, - 0,8,6,4,2,8,4,8,4,8,3,8,8,8,6,8, - 6,8,4,8,4,114,220,0,0,0,99,0,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,64,0,0,0,115, - 96,0,0,0,101,0,90,1,100,0,90,2,100,1,90,3, - 100,2,100,3,132,0,90,4,100,4,100,5,132,0,90,5, - 100,6,100,7,132,0,90,6,100,8,100,9,132,0,90,7, - 100,10,100,11,132,0,90,8,100,12,100,13,132,0,90,9, - 100,14,100,15,132,0,90,10,100,16,100,17,132,0,90,11, - 100,18,100,19,132,0,90,12,100,20,100,21,132,0,90,13, - 100,22,83,0,41,23,218,14,95,78,97,109,101,115,112,97, - 99,101,80,97,116,104,97,38,1,0,0,82,101,112,114,101, - 115,101,110,116,115,32,97,32,110,97,109,101,115,112,97,99, - 101,32,112,97,99,107,97,103,101,39,115,32,112,97,116,104, - 46,32,32,73,116,32,117,115,101,115,32,116,104,101,32,109, - 111,100,117,108,101,32,110,97,109,101,10,32,32,32,32,116, - 111,32,102,105,110,100,32,105,116,115,32,112,97,114,101,110, - 116,32,109,111,100,117,108,101,44,32,97,110,100,32,102,114, - 111,109,32,116,104,101,114,101,32,105,116,32,108,111,111,107, - 115,32,117,112,32,116,104,101,32,112,97,114,101,110,116,39, - 115,10,32,32,32,32,95,95,112,97,116,104,95,95,46,32, - 32,87,104,101,110,32,116,104,105,115,32,99,104,97,110,103, - 101,115,44,32,116,104,101,32,109,111,100,117,108,101,39,115, - 32,111,119,110,32,112,97,116,104,32,105,115,32,114,101,99, - 111,109,112,117,116,101,100,44,10,32,32,32,32,117,115,105, - 110,103,32,112,97,116,104,95,102,105,110,100,101,114,46,32, - 32,70,111,114,32,116,111,112,45,108,101,118,101,108,32,109, - 111,100,117,108,101,115,44,32,116,104,101,32,112,97,114,101, - 110,116,32,109,111,100,117,108,101,39,115,32,112,97,116,104, - 10,32,32,32,32,105,115,32,115,121,115,46,112,97,116,104, - 46,99,4,0,0,0,0,0,0,0,4,0,0,0,2,0, - 0,0,67,0,0,0,115,36,0,0,0,124,1,124,0,95, - 0,124,2,124,0,95,1,116,2,124,0,106,3,131,0,131, - 1,124,0,95,4,124,3,124,0,95,5,100,0,83,0,41, - 1,78,41,6,218,5,95,110,97,109,101,218,5,95,112,97, - 116,104,114,96,0,0,0,218,16,95,103,101,116,95,112,97, - 114,101,110,116,95,112,97,116,104,218,17,95,108,97,115,116, - 95,112,97,114,101,110,116,95,112,97,116,104,218,12,95,112, - 97,116,104,95,102,105,110,100,101,114,41,4,114,103,0,0, - 0,114,101,0,0,0,114,37,0,0,0,218,11,112,97,116, - 104,95,102,105,110,100,101,114,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,114,181,0,0,0,182,3,0,0, - 115,8,0,0,0,0,1,6,1,6,1,14,1,122,23,95, - 78,97,109,101,115,112,97,99,101,80,97,116,104,46,95,95, - 105,110,105,116,95,95,99,1,0,0,0,0,0,0,0,4, - 0,0,0,3,0,0,0,67,0,0,0,115,38,0,0,0, - 124,0,106,0,106,1,100,1,131,1,92,3,125,1,125,2, - 125,3,124,2,100,2,107,2,114,30,100,6,83,0,124,1, - 100,5,102,2,83,0,41,7,122,62,82,101,116,117,114,110, - 115,32,97,32,116,117,112,108,101,32,111,102,32,40,112,97, - 114,101,110,116,45,109,111,100,117,108,101,45,110,97,109,101, - 44,32,112,97,114,101,110,116,45,112,97,116,104,45,97,116, - 116,114,45,110,97,109,101,41,114,61,0,0,0,114,32,0, - 0,0,114,8,0,0,0,114,37,0,0,0,90,8,95,95, - 112,97,116,104,95,95,41,2,122,3,115,121,115,122,4,112, - 97,116,104,41,2,114,227,0,0,0,114,34,0,0,0,41, - 4,114,103,0,0,0,114,218,0,0,0,218,3,100,111,116, - 90,2,109,101,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,23,95,102,105,110,100,95,112,97,114,101,110, - 116,95,112,97,116,104,95,110,97,109,101,115,188,3,0,0, - 115,8,0,0,0,0,2,18,1,8,2,4,3,122,38,95, - 78,97,109,101,115,112,97,99,101,80,97,116,104,46,95,102, - 105,110,100,95,112,97,114,101,110,116,95,112,97,116,104,95, - 110,97,109,101,115,99,1,0,0,0,0,0,0,0,3,0, - 0,0,3,0,0,0,67,0,0,0,115,28,0,0,0,124, - 0,106,0,131,0,92,2,125,1,125,2,116,1,116,2,106, - 3,124,1,25,0,124,2,131,2,83,0,41,1,78,41,4, - 114,234,0,0,0,114,113,0,0,0,114,8,0,0,0,218, - 7,109,111,100,117,108,101,115,41,3,114,103,0,0,0,90, - 18,112,97,114,101,110,116,95,109,111,100,117,108,101,95,110, - 97,109,101,90,14,112,97,116,104,95,97,116,116,114,95,110, - 97,109,101,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,229,0,0,0,198,3,0,0,115,4,0,0,0, - 0,1,12,1,122,31,95,78,97,109,101,115,112,97,99,101, - 80,97,116,104,46,95,103,101,116,95,112,97,114,101,110,116, - 95,112,97,116,104,99,1,0,0,0,0,0,0,0,3,0, - 0,0,3,0,0,0,67,0,0,0,115,80,0,0,0,116, - 0,124,0,106,1,131,0,131,1,125,1,124,1,124,0,106, - 2,107,3,114,74,124,0,106,3,124,0,106,4,124,1,131, - 2,125,2,124,2,100,0,107,9,114,68,124,2,106,5,100, - 0,107,8,114,68,124,2,106,6,114,68,124,2,106,6,124, - 0,95,7,124,1,124,0,95,2,124,0,106,7,83,0,41, - 1,78,41,8,114,96,0,0,0,114,229,0,0,0,114,230, - 0,0,0,114,231,0,0,0,114,227,0,0,0,114,123,0, - 0,0,114,153,0,0,0,114,228,0,0,0,41,3,114,103, - 0,0,0,90,11,112,97,114,101,110,116,95,112,97,116,104, - 114,161,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,218,12,95,114,101,99,97,108,99,117,108,97, - 116,101,202,3,0,0,115,16,0,0,0,0,2,12,1,10, - 1,14,3,18,1,6,1,8,1,6,1,122,27,95,78,97, - 109,101,115,112,97,99,101,80,97,116,104,46,95,114,101,99, - 97,108,99,117,108,97,116,101,99,1,0,0,0,0,0,0, - 0,1,0,0,0,2,0,0,0,67,0,0,0,115,12,0, - 0,0,116,0,124,0,106,1,131,0,131,1,83,0,41,1, - 78,41,2,218,4,105,116,101,114,114,236,0,0,0,41,1, - 114,103,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,218,8,95,95,105,116,101,114,95,95,215,3, - 0,0,115,2,0,0,0,0,1,122,23,95,78,97,109,101, - 115,112,97,99,101,80,97,116,104,46,95,95,105,116,101,114, - 95,95,99,3,0,0,0,0,0,0,0,3,0,0,0,3, - 0,0,0,67,0,0,0,115,14,0,0,0,124,2,124,0, - 106,0,124,1,60,0,100,0,83,0,41,1,78,41,1,114, - 228,0,0,0,41,3,114,103,0,0,0,218,5,105,110,100, - 101,120,114,37,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,218,11,95,95,115,101,116,105,116,101, - 109,95,95,218,3,0,0,115,2,0,0,0,0,1,122,26, - 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, - 95,115,101,116,105,116,101,109,95,95,99,1,0,0,0,0, - 0,0,0,1,0,0,0,2,0,0,0,67,0,0,0,115, - 12,0,0,0,116,0,124,0,106,1,131,0,131,1,83,0, - 41,1,78,41,2,114,33,0,0,0,114,236,0,0,0,41, - 1,114,103,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,218,7,95,95,108,101,110,95,95,221,3, - 0,0,115,2,0,0,0,0,1,122,22,95,78,97,109,101, - 115,112,97,99,101,80,97,116,104,46,95,95,108,101,110,95, - 95,99,1,0,0,0,0,0,0,0,1,0,0,0,2,0, - 0,0,67,0,0,0,115,12,0,0,0,100,1,106,0,124, - 0,106,1,131,1,83,0,41,2,78,122,20,95,78,97,109, - 101,115,112,97,99,101,80,97,116,104,40,123,33,114,125,41, - 41,2,114,50,0,0,0,114,228,0,0,0,41,1,114,103, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,8,95,95,114,101,112,114,95,95,224,3,0,0, - 115,2,0,0,0,0,1,122,23,95,78,97,109,101,115,112, - 97,99,101,80,97,116,104,46,95,95,114,101,112,114,95,95, - 99,2,0,0,0,0,0,0,0,2,0,0,0,2,0,0, - 0,67,0,0,0,115,12,0,0,0,124,1,124,0,106,0, - 131,0,107,6,83,0,41,1,78,41,1,114,236,0,0,0, - 41,2,114,103,0,0,0,218,4,105,116,101,109,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,12,95,95, - 99,111,110,116,97,105,110,115,95,95,227,3,0,0,115,2, - 0,0,0,0,1,122,27,95,78,97,109,101,115,112,97,99, - 101,80,97,116,104,46,95,95,99,111,110,116,97,105,110,115, - 95,95,99,2,0,0,0,0,0,0,0,2,0,0,0,2, - 0,0,0,67,0,0,0,115,16,0,0,0,124,0,106,0, - 106,1,124,1,131,1,1,0,100,0,83,0,41,1,78,41, - 2,114,228,0,0,0,114,160,0,0,0,41,2,114,103,0, - 0,0,114,243,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,160,0,0,0,230,3,0,0,115, - 2,0,0,0,0,1,122,21,95,78,97,109,101,115,112,97, - 99,101,80,97,116,104,46,97,112,112,101,110,100,78,41,14, - 114,108,0,0,0,114,107,0,0,0,114,109,0,0,0,114, - 110,0,0,0,114,181,0,0,0,114,234,0,0,0,114,229, - 0,0,0,114,236,0,0,0,114,238,0,0,0,114,240,0, - 0,0,114,241,0,0,0,114,242,0,0,0,114,244,0,0, - 0,114,160,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,226,0,0,0,175, - 3,0,0,115,22,0,0,0,8,5,4,2,8,6,8,10, - 8,4,8,13,8,3,8,3,8,3,8,3,8,3,114,226, - 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, - 3,0,0,0,64,0,0,0,115,80,0,0,0,101,0,90, - 1,100,0,90,2,100,1,100,2,132,0,90,3,101,4,100, - 3,100,4,132,0,131,1,90,5,100,5,100,6,132,0,90, - 6,100,7,100,8,132,0,90,7,100,9,100,10,132,0,90, - 8,100,11,100,12,132,0,90,9,100,13,100,14,132,0,90, - 10,100,15,100,16,132,0,90,11,100,17,83,0,41,18,218, - 16,95,78,97,109,101,115,112,97,99,101,76,111,97,100,101, - 114,99,4,0,0,0,0,0,0,0,4,0,0,0,4,0, - 0,0,67,0,0,0,115,18,0,0,0,116,0,124,1,124, - 2,124,3,131,3,124,0,95,1,100,0,83,0,41,1,78, - 41,2,114,226,0,0,0,114,228,0,0,0,41,4,114,103, - 0,0,0,114,101,0,0,0,114,37,0,0,0,114,232,0, + 101,100,46,32,32,85,115,101,32,102,105,110,100,95,115,112, + 101,99,40,41,32,105,110,115,116,101,97,100,46,10,10,32, + 32,32,32,32,32,32,32,78,41,3,114,178,0,0,0,114, + 124,0,0,0,114,154,0,0,0,41,3,114,104,0,0,0, + 114,123,0,0,0,114,162,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,121,0,0,0,186,4, + 0,0,115,8,0,0,0,0,7,10,1,8,1,8,1,122, + 22,70,105,108,101,70,105,110,100,101,114,46,102,105,110,100, + 95,108,111,97,100,101,114,99,6,0,0,0,0,0,0,0, + 7,0,0,0,7,0,0,0,67,0,0,0,115,30,0,0, + 0,124,1,124,2,124,3,131,2,125,6,116,0,124,2,124, + 3,100,1,124,6,100,2,124,4,144,2,131,2,83,0,41, + 3,78,114,124,0,0,0,114,154,0,0,0,41,1,114,165, + 0,0,0,41,7,114,104,0,0,0,114,163,0,0,0,114, + 123,0,0,0,114,37,0,0,0,90,4,115,109,115,108,114, + 177,0,0,0,114,124,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,4,1,0,0,198,4,0, + 0,115,6,0,0,0,0,1,10,1,12,1,122,20,70,105, + 108,101,70,105,110,100,101,114,46,95,103,101,116,95,115,112, + 101,99,78,99,3,0,0,0,0,0,0,0,14,0,0,0, + 15,0,0,0,67,0,0,0,115,100,1,0,0,100,1,125, + 3,124,1,106,0,100,2,131,1,100,3,25,0,125,4,121, + 24,116,1,124,0,106,2,112,34,116,3,106,4,131,0,131, + 1,106,5,125,5,87,0,110,24,4,0,116,6,107,10,114, + 66,1,0,1,0,1,0,100,10,125,5,89,0,110,2,88, + 0,124,5,124,0,106,7,107,3,114,92,124,0,106,8,131, + 0,1,0,124,5,124,0,95,7,116,9,131,0,114,114,124, + 0,106,10,125,6,124,4,106,11,131,0,125,7,110,10,124, + 0,106,12,125,6,124,4,125,7,124,7,124,6,107,6,114, + 218,116,13,124,0,106,2,124,4,131,2,125,8,120,72,124, + 0,106,14,68,0,93,54,92,2,125,9,125,10,100,5,124, + 9,23,0,125,11,116,13,124,8,124,11,131,2,125,12,116, + 15,124,12,131,1,114,152,124,0,106,16,124,10,124,1,124, + 12,124,8,103,1,124,2,131,5,83,0,113,152,87,0,116, + 17,124,8,131,1,125,3,120,90,124,0,106,14,68,0,93, + 80,92,2,125,9,125,10,116,13,124,0,106,2,124,4,124, + 9,23,0,131,2,125,12,116,18,106,19,100,6,124,12,100, + 7,100,3,144,1,131,2,1,0,124,7,124,9,23,0,124, + 6,107,6,114,226,116,15,124,12,131,1,114,226,124,0,106, + 16,124,10,124,1,124,12,100,8,124,2,131,5,83,0,113, + 226,87,0,124,3,144,1,114,96,116,18,106,19,100,9,124, + 8,131,2,1,0,116,18,106,20,124,1,100,8,131,2,125, + 13,124,8,103,1,124,13,95,21,124,13,83,0,100,8,83, + 0,41,11,122,111,84,114,121,32,116,111,32,102,105,110,100, + 32,97,32,115,112,101,99,32,102,111,114,32,116,104,101,32, + 115,112,101,99,105,102,105,101,100,32,109,111,100,117,108,101, + 46,10,10,32,32,32,32,32,32,32,32,82,101,116,117,114, + 110,115,32,116,104,101,32,109,97,116,99,104,105,110,103,32, + 115,112,101,99,44,32,111,114,32,78,111,110,101,32,105,102, + 32,110,111,116,32,102,111,117,110,100,46,10,32,32,32,32, + 32,32,32,32,70,114,61,0,0,0,114,59,0,0,0,114, + 31,0,0,0,114,182,0,0,0,122,9,116,114,121,105,110, + 103,32,123,125,90,9,118,101,114,98,111,115,105,116,121,78, + 122,25,112,111,115,115,105,98,108,101,32,110,97,109,101,115, + 112,97,99,101,32,102,111,114,32,123,125,114,91,0,0,0, + 41,22,114,34,0,0,0,114,41,0,0,0,114,37,0,0, + 0,114,3,0,0,0,114,47,0,0,0,114,216,0,0,0, + 114,42,0,0,0,114,7,1,0,0,218,11,95,102,105,108, + 108,95,99,97,99,104,101,114,7,0,0,0,114,10,1,0, + 0,114,92,0,0,0,114,9,1,0,0,114,30,0,0,0, + 114,6,1,0,0,114,46,0,0,0,114,4,1,0,0,114, + 48,0,0,0,114,118,0,0,0,114,133,0,0,0,114,158, + 0,0,0,114,154,0,0,0,41,14,114,104,0,0,0,114, + 123,0,0,0,114,177,0,0,0,90,12,105,115,95,110,97, + 109,101,115,112,97,99,101,90,11,116,97,105,108,95,109,111, + 100,117,108,101,114,130,0,0,0,90,5,99,97,99,104,101, + 90,12,99,97,99,104,101,95,109,111,100,117,108,101,90,9, + 98,97,115,101,95,112,97,116,104,114,222,0,0,0,114,163, + 0,0,0,90,13,105,110,105,116,95,102,105,108,101,110,97, + 109,101,90,9,102,117,108,108,95,112,97,116,104,114,162,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,181,0,0,0,236,3,0,0,115,2,0,0,0,0, - 1,122,25,95,78,97,109,101,115,112,97,99,101,76,111,97, - 100,101,114,46,95,95,105,110,105,116,95,95,99,2,0,0, - 0,0,0,0,0,2,0,0,0,2,0,0,0,67,0,0, - 0,115,12,0,0,0,100,1,106,0,124,1,106,1,131,1, - 83,0,41,2,122,115,82,101,116,117,114,110,32,114,101,112, - 114,32,102,111,114,32,116,104,101,32,109,111,100,117,108,101, - 46,10,10,32,32,32,32,32,32,32,32,84,104,101,32,109, - 101,116,104,111,100,32,105,115,32,100,101,112,114,101,99,97, - 116,101,100,46,32,32,84,104,101,32,105,109,112,111,114,116, - 32,109,97,99,104,105,110,101,114,121,32,100,111,101,115,32, - 116,104,101,32,106,111,98,32,105,116,115,101,108,102,46,10, - 10,32,32,32,32,32,32,32,32,122,25,60,109,111,100,117, - 108,101,32,123,33,114,125,32,40,110,97,109,101,115,112,97, - 99,101,41,62,41,2,114,50,0,0,0,114,108,0,0,0, - 41,2,114,167,0,0,0,114,186,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,11,109,111,100, - 117,108,101,95,114,101,112,114,239,3,0,0,115,2,0,0, - 0,0,7,122,28,95,78,97,109,101,115,112,97,99,101,76, - 111,97,100,101,114,46,109,111,100,117,108,101,95,114,101,112, - 114,99,2,0,0,0,0,0,0,0,2,0,0,0,1,0, - 0,0,67,0,0,0,115,4,0,0,0,100,1,83,0,41, - 2,78,84,114,4,0,0,0,41,2,114,103,0,0,0,114, - 122,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,156,0,0,0,248,3,0,0,115,2,0,0, - 0,0,1,122,27,95,78,97,109,101,115,112,97,99,101,76, - 111,97,100,101,114,46,105,115,95,112,97,99,107,97,103,101, - 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, - 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, - 78,114,32,0,0,0,114,4,0,0,0,41,2,114,103,0, - 0,0,114,122,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,198,0,0,0,251,3,0,0,115, - 2,0,0,0,0,1,122,27,95,78,97,109,101,115,112,97, - 99,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, - 114,99,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 6,0,0,0,67,0,0,0,115,18,0,0,0,116,0,100, - 1,100,2,100,3,100,4,100,5,144,1,131,3,83,0,41, - 6,78,114,32,0,0,0,122,8,60,115,116,114,105,110,103, - 62,114,185,0,0,0,114,200,0,0,0,84,41,1,114,201, - 0,0,0,41,2,114,103,0,0,0,114,122,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,183, - 0,0,0,254,3,0,0,115,2,0,0,0,0,1,122,25, - 95,78,97,109,101,115,112,97,99,101,76,111,97,100,101,114, - 46,103,101,116,95,99,111,100,101,99,2,0,0,0,0,0, - 0,0,2,0,0,0,1,0,0,0,67,0,0,0,115,4, - 0,0,0,100,1,83,0,41,2,122,42,85,115,101,32,100, - 101,102,97,117,108,116,32,115,101,109,97,110,116,105,99,115, - 32,102,111,114,32,109,111,100,117,108,101,32,99,114,101,97, - 116,105,111,110,46,78,114,4,0,0,0,41,2,114,103,0, - 0,0,114,161,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,182,0,0,0,1,4,0,0,115, - 0,0,0,0,122,30,95,78,97,109,101,115,112,97,99,101, - 76,111,97,100,101,114,46,99,114,101,97,116,101,95,109,111, - 100,117,108,101,99,2,0,0,0,0,0,0,0,2,0,0, - 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,0, - 83,0,41,1,78,114,4,0,0,0,41,2,114,103,0,0, - 0,114,186,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,187,0,0,0,4,4,0,0,115,2, - 0,0,0,0,1,122,28,95,78,97,109,101,115,112,97,99, - 101,76,111,97,100,101,114,46,101,120,101,99,95,109,111,100, - 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 3,0,0,0,67,0,0,0,115,26,0,0,0,116,0,106, - 1,100,1,124,0,106,2,131,2,1,0,116,0,106,3,124, - 0,124,1,131,2,83,0,41,2,122,98,76,111,97,100,32, - 97,32,110,97,109,101,115,112,97,99,101,32,109,111,100,117, - 108,101,46,10,10,32,32,32,32,32,32,32,32,84,104,105, - 115,32,109,101,116,104,111,100,32,105,115,32,100,101,112,114, - 101,99,97,116,101,100,46,32,32,85,115,101,32,101,120,101, - 99,95,109,111,100,117,108,101,40,41,32,105,110,115,116,101, - 97,100,46,10,10,32,32,32,32,32,32,32,32,122,38,110, - 97,109,101,115,112,97,99,101,32,109,111,100,117,108,101,32, - 108,111,97,100,101,100,32,119,105,116,104,32,112,97,116,104, - 32,123,33,114,125,41,4,114,117,0,0,0,114,132,0,0, - 0,114,228,0,0,0,114,188,0,0,0,41,2,114,103,0, - 0,0,114,122,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,189,0,0,0,7,4,0,0,115, - 6,0,0,0,0,7,6,1,8,1,122,28,95,78,97,109, - 101,115,112,97,99,101,76,111,97,100,101,114,46,108,111,97, - 100,95,109,111,100,117,108,101,78,41,12,114,108,0,0,0, - 114,107,0,0,0,114,109,0,0,0,114,181,0,0,0,114, - 179,0,0,0,114,246,0,0,0,114,156,0,0,0,114,198, - 0,0,0,114,183,0,0,0,114,182,0,0,0,114,187,0, - 0,0,114,189,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,245,0,0,0, - 235,3,0,0,115,16,0,0,0,8,1,8,3,12,9,8, - 3,8,3,8,3,8,3,8,3,114,245,0,0,0,99,0, - 0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,64, - 0,0,0,115,106,0,0,0,101,0,90,1,100,0,90,2, - 100,1,90,3,101,4,100,2,100,3,132,0,131,1,90,5, - 101,4,100,4,100,5,132,0,131,1,90,6,101,4,100,6, - 100,7,132,0,131,1,90,7,101,4,100,8,100,9,132,0, - 131,1,90,8,101,4,100,17,100,11,100,12,132,1,131,1, - 90,9,101,4,100,18,100,13,100,14,132,1,131,1,90,10, - 101,4,100,19,100,15,100,16,132,1,131,1,90,11,100,10, - 83,0,41,20,218,10,80,97,116,104,70,105,110,100,101,114, - 122,62,77,101,116,97,32,112,97,116,104,32,102,105,110,100, - 101,114,32,102,111,114,32,115,121,115,46,112,97,116,104,32, - 97,110,100,32,112,97,99,107,97,103,101,32,95,95,112,97, - 116,104,95,95,32,97,116,116,114,105,98,117,116,101,115,46, - 99,1,0,0,0,0,0,0,0,2,0,0,0,4,0,0, - 0,67,0,0,0,115,42,0,0,0,120,36,116,0,106,1, - 106,2,131,0,68,0,93,22,125,1,116,3,124,1,100,1, - 131,2,114,12,124,1,106,4,131,0,1,0,113,12,87,0, - 100,2,83,0,41,3,122,125,67,97,108,108,32,116,104,101, - 32,105,110,118,97,108,105,100,97,116,101,95,99,97,99,104, - 101,115,40,41,32,109,101,116,104,111,100,32,111,110,32,97, - 108,108,32,112,97,116,104,32,101,110,116,114,121,32,102,105, - 110,100,101,114,115,10,32,32,32,32,32,32,32,32,115,116, - 111,114,101,100,32,105,110,32,115,121,115,46,112,97,116,104, - 95,105,109,112,111,114,116,101,114,95,99,97,99,104,101,115, - 32,40,119,104,101,114,101,32,105,109,112,108,101,109,101,110, - 116,101,100,41,46,218,17,105,110,118,97,108,105,100,97,116, - 101,95,99,97,99,104,101,115,78,41,5,114,8,0,0,0, - 218,19,112,97,116,104,95,105,109,112,111,114,116,101,114,95, - 99,97,99,104,101,218,6,118,97,108,117,101,115,114,111,0, - 0,0,114,248,0,0,0,41,2,114,167,0,0,0,218,6, - 102,105,110,100,101,114,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,248,0,0,0,25,4,0,0,115,6, - 0,0,0,0,4,16,1,10,1,122,28,80,97,116,104,70, - 105,110,100,101,114,46,105,110,118,97,108,105,100,97,116,101, - 95,99,97,99,104,101,115,99,2,0,0,0,0,0,0,0, - 3,0,0,0,12,0,0,0,67,0,0,0,115,86,0,0, - 0,116,0,106,1,100,1,107,9,114,30,116,0,106,1,12, - 0,114,30,116,2,106,3,100,2,116,4,131,2,1,0,120, - 50,116,0,106,1,68,0,93,36,125,2,121,8,124,2,124, - 1,131,1,83,0,4,0,116,5,107,10,114,72,1,0,1, - 0,1,0,119,38,89,0,113,38,88,0,113,38,87,0,100, - 1,83,0,100,1,83,0,41,3,122,46,83,101,97,114,99, - 104,32,115,121,115,46,112,97,116,104,95,104,111,111,107,115, - 32,102,111,114,32,97,32,102,105,110,100,101,114,32,102,111, - 114,32,39,112,97,116,104,39,46,78,122,23,115,121,115,46, - 112,97,116,104,95,104,111,111,107,115,32,105,115,32,101,109, - 112,116,121,41,6,114,8,0,0,0,218,10,112,97,116,104, - 95,104,111,111,107,115,114,63,0,0,0,114,64,0,0,0, - 114,121,0,0,0,114,102,0,0,0,41,3,114,167,0,0, - 0,114,37,0,0,0,90,4,104,111,111,107,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,11,95,112,97, - 116,104,95,104,111,111,107,115,33,4,0,0,115,16,0,0, - 0,0,3,18,1,12,1,12,1,2,1,8,1,14,1,12, - 2,122,22,80,97,116,104,70,105,110,100,101,114,46,95,112, - 97,116,104,95,104,111,111,107,115,99,2,0,0,0,0,0, - 0,0,3,0,0,0,19,0,0,0,67,0,0,0,115,102, - 0,0,0,124,1,100,1,107,2,114,42,121,12,116,0,106, - 1,131,0,125,1,87,0,110,20,4,0,116,2,107,10,114, - 40,1,0,1,0,1,0,100,2,83,0,88,0,121,14,116, - 3,106,4,124,1,25,0,125,2,87,0,110,40,4,0,116, - 5,107,10,114,96,1,0,1,0,1,0,124,0,106,6,124, - 1,131,1,125,2,124,2,116,3,106,4,124,1,60,0,89, - 0,110,2,88,0,124,2,83,0,41,3,122,210,71,101,116, - 32,116,104,101,32,102,105,110,100,101,114,32,102,111,114,32, - 116,104,101,32,112,97,116,104,32,101,110,116,114,121,32,102, - 114,111,109,32,115,121,115,46,112,97,116,104,95,105,109,112, - 111,114,116,101,114,95,99,97,99,104,101,46,10,10,32,32, - 32,32,32,32,32,32,73,102,32,116,104,101,32,112,97,116, - 104,32,101,110,116,114,121,32,105,115,32,110,111,116,32,105, - 110,32,116,104,101,32,99,97,99,104,101,44,32,102,105,110, - 100,32,116,104,101,32,97,112,112,114,111,112,114,105,97,116, - 101,32,102,105,110,100,101,114,10,32,32,32,32,32,32,32, - 32,97,110,100,32,99,97,99,104,101,32,105,116,46,32,73, - 102,32,110,111,32,102,105,110,100,101,114,32,105,115,32,97, - 118,97,105,108,97,98,108,101,44,32,115,116,111,114,101,32, - 78,111,110,101,46,10,10,32,32,32,32,32,32,32,32,114, - 32,0,0,0,78,41,7,114,3,0,0,0,114,47,0,0, - 0,218,17,70,105,108,101,78,111,116,70,111,117,110,100,69, - 114,114,111,114,114,8,0,0,0,114,249,0,0,0,114,134, - 0,0,0,114,253,0,0,0,41,3,114,167,0,0,0,114, - 37,0,0,0,114,251,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,20,95,112,97,116,104,95, - 105,109,112,111,114,116,101,114,95,99,97,99,104,101,46,4, - 0,0,115,22,0,0,0,0,8,8,1,2,1,12,1,14, - 3,6,1,2,1,14,1,14,1,10,1,16,1,122,31,80, - 97,116,104,70,105,110,100,101,114,46,95,112,97,116,104,95, - 105,109,112,111,114,116,101,114,95,99,97,99,104,101,99,3, - 0,0,0,0,0,0,0,6,0,0,0,3,0,0,0,67, - 0,0,0,115,82,0,0,0,116,0,124,2,100,1,131,2, - 114,26,124,2,106,1,124,1,131,1,92,2,125,3,125,4, - 110,14,124,2,106,2,124,1,131,1,125,3,103,0,125,4, - 124,3,100,0,107,9,114,60,116,3,106,4,124,1,124,3, - 131,2,83,0,116,3,106,5,124,1,100,0,131,2,125,5, - 124,4,124,5,95,6,124,5,83,0,41,2,78,114,120,0, - 0,0,41,7,114,111,0,0,0,114,120,0,0,0,114,178, - 0,0,0,114,117,0,0,0,114,175,0,0,0,114,157,0, - 0,0,114,153,0,0,0,41,6,114,167,0,0,0,114,122, - 0,0,0,114,251,0,0,0,114,123,0,0,0,114,124,0, - 0,0,114,161,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,218,16,95,108,101,103,97,99,121,95, - 103,101,116,95,115,112,101,99,68,4,0,0,115,18,0,0, - 0,0,4,10,1,16,2,10,1,4,1,8,1,12,1,12, - 1,6,1,122,27,80,97,116,104,70,105,110,100,101,114,46, - 95,108,101,103,97,99,121,95,103,101,116,95,115,112,101,99, - 78,99,4,0,0,0,0,0,0,0,9,0,0,0,5,0, - 0,0,67,0,0,0,115,170,0,0,0,103,0,125,4,120, - 160,124,2,68,0,93,130,125,5,116,0,124,5,116,1,116, - 2,102,2,131,2,115,30,113,10,124,0,106,3,124,5,131, - 1,125,6,124,6,100,1,107,9,114,10,116,4,124,6,100, - 2,131,2,114,72,124,6,106,5,124,1,124,3,131,2,125, - 7,110,12,124,0,106,6,124,1,124,6,131,2,125,7,124, - 7,100,1,107,8,114,94,113,10,124,7,106,7,100,1,107, - 9,114,108,124,7,83,0,124,7,106,8,125,8,124,8,100, - 1,107,8,114,130,116,9,100,3,131,1,130,1,124,4,106, - 10,124,8,131,1,1,0,113,10,87,0,116,11,106,12,124, - 1,100,1,131,2,125,7,124,4,124,7,95,8,124,7,83, - 0,100,1,83,0,41,4,122,63,70,105,110,100,32,116,104, - 101,32,108,111,97,100,101,114,32,111,114,32,110,97,109,101, - 115,112,97,99,101,95,112,97,116,104,32,102,111,114,32,116, - 104,105,115,32,109,111,100,117,108,101,47,112,97,99,107,97, - 103,101,32,110,97,109,101,46,78,114,177,0,0,0,122,19, - 115,112,101,99,32,109,105,115,115,105,110,103,32,108,111,97, - 100,101,114,41,13,114,140,0,0,0,114,72,0,0,0,218, - 5,98,121,116,101,115,114,255,0,0,0,114,111,0,0,0, - 114,177,0,0,0,114,0,1,0,0,114,123,0,0,0,114, - 153,0,0,0,114,102,0,0,0,114,146,0,0,0,114,117, - 0,0,0,114,157,0,0,0,41,9,114,167,0,0,0,114, - 122,0,0,0,114,37,0,0,0,114,176,0,0,0,218,14, - 110,97,109,101,115,112,97,99,101,95,112,97,116,104,90,5, - 101,110,116,114,121,114,251,0,0,0,114,161,0,0,0,114, - 124,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,9,95,103,101,116,95,115,112,101,99,83,4, - 0,0,115,40,0,0,0,0,5,4,1,10,1,14,1,2, - 1,10,1,8,1,10,1,14,2,12,1,8,1,2,1,10, - 1,4,1,6,1,8,1,8,5,14,2,12,1,6,1,122, - 20,80,97,116,104,70,105,110,100,101,114,46,95,103,101,116, - 95,115,112,101,99,99,4,0,0,0,0,0,0,0,6,0, - 0,0,4,0,0,0,67,0,0,0,115,104,0,0,0,124, - 2,100,1,107,8,114,14,116,0,106,1,125,2,124,0,106, - 2,124,1,124,2,124,3,131,3,125,4,124,4,100,1,107, - 8,114,42,100,1,83,0,110,58,124,4,106,3,100,1,107, - 8,114,96,124,4,106,4,125,5,124,5,114,90,100,2,124, - 4,95,5,116,6,124,1,124,5,124,0,106,2,131,3,124, - 4,95,4,124,4,83,0,113,100,100,1,83,0,110,4,124, - 4,83,0,100,1,83,0,41,3,122,141,84,114,121,32,116, - 111,32,102,105,110,100,32,97,32,115,112,101,99,32,102,111, - 114,32,39,102,117,108,108,110,97,109,101,39,32,111,110,32, - 115,121,115,46,112,97,116,104,32,111,114,32,39,112,97,116, - 104,39,46,10,10,32,32,32,32,32,32,32,32,84,104,101, - 32,115,101,97,114,99,104,32,105,115,32,98,97,115,101,100, - 32,111,110,32,115,121,115,46,112,97,116,104,95,104,111,111, - 107,115,32,97,110,100,32,115,121,115,46,112,97,116,104,95, - 105,109,112,111,114,116,101,114,95,99,97,99,104,101,46,10, - 32,32,32,32,32,32,32,32,78,90,9,110,97,109,101,115, - 112,97,99,101,41,7,114,8,0,0,0,114,37,0,0,0, - 114,3,1,0,0,114,123,0,0,0,114,153,0,0,0,114, - 155,0,0,0,114,226,0,0,0,41,6,114,167,0,0,0, - 114,122,0,0,0,114,37,0,0,0,114,176,0,0,0,114, - 161,0,0,0,114,2,1,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,177,0,0,0,115,4,0, - 0,115,26,0,0,0,0,6,8,1,6,1,14,1,8,1, - 6,1,10,1,6,1,4,3,6,1,16,1,6,2,6,2, - 122,20,80,97,116,104,70,105,110,100,101,114,46,102,105,110, - 100,95,115,112,101,99,99,3,0,0,0,0,0,0,0,4, - 0,0,0,3,0,0,0,67,0,0,0,115,30,0,0,0, - 124,0,106,0,124,1,124,2,131,2,125,3,124,3,100,1, - 107,8,114,24,100,1,83,0,124,3,106,1,83,0,41,2, - 122,170,102,105,110,100,32,116,104,101,32,109,111,100,117,108, - 101,32,111,110,32,115,121,115,46,112,97,116,104,32,111,114, - 32,39,112,97,116,104,39,32,98,97,115,101,100,32,111,110, - 32,115,121,115,46,112,97,116,104,95,104,111,111,107,115,32, - 97,110,100,10,32,32,32,32,32,32,32,32,115,121,115,46, - 112,97,116,104,95,105,109,112,111,114,116,101,114,95,99,97, - 99,104,101,46,10,10,32,32,32,32,32,32,32,32,84,104, - 105,115,32,109,101,116,104,111,100,32,105,115,32,100,101,112, - 114,101,99,97,116,101,100,46,32,32,85,115,101,32,102,105, - 110,100,95,115,112,101,99,40,41,32,105,110,115,116,101,97, - 100,46,10,10,32,32,32,32,32,32,32,32,78,41,2,114, - 177,0,0,0,114,123,0,0,0,41,4,114,167,0,0,0, - 114,122,0,0,0,114,37,0,0,0,114,161,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,178, - 0,0,0,139,4,0,0,115,8,0,0,0,0,8,12,1, - 8,1,4,1,122,22,80,97,116,104,70,105,110,100,101,114, - 46,102,105,110,100,95,109,111,100,117,108,101,41,1,78,41, - 2,78,78,41,1,78,41,12,114,108,0,0,0,114,107,0, - 0,0,114,109,0,0,0,114,110,0,0,0,114,179,0,0, - 0,114,248,0,0,0,114,253,0,0,0,114,255,0,0,0, - 114,0,1,0,0,114,3,1,0,0,114,177,0,0,0,114, - 178,0,0,0,114,4,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,247,0,0,0,21,4,0, - 0,115,22,0,0,0,8,2,4,2,12,8,12,13,12,22, - 12,15,2,1,12,31,2,1,12,23,2,1,114,247,0,0, - 0,99,0,0,0,0,0,0,0,0,0,0,0,0,3,0, - 0,0,64,0,0,0,115,90,0,0,0,101,0,90,1,100, - 0,90,2,100,1,90,3,100,2,100,3,132,0,90,4,100, - 4,100,5,132,0,90,5,101,6,90,7,100,6,100,7,132, - 0,90,8,100,8,100,9,132,0,90,9,100,19,100,11,100, - 12,132,1,90,10,100,13,100,14,132,0,90,11,101,12,100, - 15,100,16,132,0,131,1,90,13,100,17,100,18,132,0,90, - 14,100,10,83,0,41,20,218,10,70,105,108,101,70,105,110, - 100,101,114,122,172,70,105,108,101,45,98,97,115,101,100,32, - 102,105,110,100,101,114,46,10,10,32,32,32,32,73,110,116, - 101,114,97,99,116,105,111,110,115,32,119,105,116,104,32,116, - 104,101,32,102,105,108,101,32,115,121,115,116,101,109,32,97, - 114,101,32,99,97,99,104,101,100,32,102,111,114,32,112,101, - 114,102,111,114,109,97,110,99,101,44,32,98,101,105,110,103, - 10,32,32,32,32,114,101,102,114,101,115,104,101,100,32,119, - 104,101,110,32,116,104,101,32,100,105,114,101,99,116,111,114, - 121,32,116,104,101,32,102,105,110,100,101,114,32,105,115,32, - 104,97,110,100,108,105,110,103,32,104,97,115,32,98,101,101, - 110,32,109,111,100,105,102,105,101,100,46,10,10,32,32,32, - 32,99,2,0,0,0,0,0,0,0,5,0,0,0,5,0, - 0,0,7,0,0,0,115,88,0,0,0,103,0,125,3,120, - 40,124,2,68,0,93,32,92,2,137,0,125,4,124,3,106, - 0,135,0,102,1,100,1,100,2,132,8,124,4,68,0,131, - 1,131,1,1,0,113,10,87,0,124,3,124,0,95,1,124, - 1,112,58,100,3,124,0,95,2,100,6,124,0,95,3,116, - 4,131,0,124,0,95,5,116,4,131,0,124,0,95,6,100, - 5,83,0,41,7,122,154,73,110,105,116,105,97,108,105,122, - 101,32,119,105,116,104,32,116,104,101,32,112,97,116,104,32, - 116,111,32,115,101,97,114,99,104,32,111,110,32,97,110,100, - 32,97,32,118,97,114,105,97,98,108,101,32,110,117,109,98, - 101,114,32,111,102,10,32,32,32,32,32,32,32,32,50,45, - 116,117,112,108,101,115,32,99,111,110,116,97,105,110,105,110, - 103,32,116,104,101,32,108,111,97,100,101,114,32,97,110,100, - 32,116,104,101,32,102,105,108,101,32,115,117,102,102,105,120, - 101,115,32,116,104,101,32,108,111,97,100,101,114,10,32,32, - 32,32,32,32,32,32,114,101,99,111,103,110,105,122,101,115, - 46,99,1,0,0,0,0,0,0,0,2,0,0,0,3,0, - 0,0,51,0,0,0,115,22,0,0,0,124,0,93,14,125, - 1,124,1,136,0,102,2,86,0,1,0,113,2,100,0,83, - 0,41,1,78,114,4,0,0,0,41,2,114,24,0,0,0, - 114,221,0,0,0,41,1,114,123,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,223,0,0,0,168,4,0,0,115, - 2,0,0,0,4,0,122,38,70,105,108,101,70,105,110,100, - 101,114,46,95,95,105,110,105,116,95,95,46,60,108,111,99, - 97,108,115,62,46,60,103,101,110,101,120,112,114,62,114,61, - 0,0,0,114,31,0,0,0,78,114,90,0,0,0,41,7, - 114,146,0,0,0,218,8,95,108,111,97,100,101,114,115,114, - 37,0,0,0,218,11,95,112,97,116,104,95,109,116,105,109, - 101,218,3,115,101,116,218,11,95,112,97,116,104,95,99,97, - 99,104,101,218,19,95,114,101,108,97,120,101,100,95,112,97, - 116,104,95,99,97,99,104,101,41,5,114,103,0,0,0,114, - 37,0,0,0,218,14,108,111,97,100,101,114,95,100,101,116, - 97,105,108,115,90,7,108,111,97,100,101,114,115,114,163,0, - 0,0,114,4,0,0,0,41,1,114,123,0,0,0,114,6, - 0,0,0,114,181,0,0,0,162,4,0,0,115,16,0,0, - 0,0,4,4,1,14,1,28,1,6,2,10,1,6,1,8, - 1,122,19,70,105,108,101,70,105,110,100,101,114,46,95,95, - 105,110,105,116,95,95,99,1,0,0,0,0,0,0,0,1, - 0,0,0,2,0,0,0,67,0,0,0,115,10,0,0,0, - 100,3,124,0,95,0,100,2,83,0,41,4,122,31,73,110, - 118,97,108,105,100,97,116,101,32,116,104,101,32,100,105,114, - 101,99,116,111,114,121,32,109,116,105,109,101,46,114,31,0, - 0,0,78,114,90,0,0,0,41,1,114,6,1,0,0,41, - 1,114,103,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,248,0,0,0,176,4,0,0,115,2, - 0,0,0,0,2,122,28,70,105,108,101,70,105,110,100,101, - 114,46,105,110,118,97,108,105,100,97,116,101,95,99,97,99, - 104,101,115,99,2,0,0,0,0,0,0,0,3,0,0,0, - 2,0,0,0,67,0,0,0,115,42,0,0,0,124,0,106, - 0,124,1,131,1,125,2,124,2,100,1,107,8,114,26,100, - 1,103,0,102,2,83,0,124,2,106,1,124,2,106,2,112, - 38,103,0,102,2,83,0,41,2,122,197,84,114,121,32,116, - 111,32,102,105,110,100,32,97,32,108,111,97,100,101,114,32, - 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, - 100,32,109,111,100,117,108,101,44,32,111,114,32,116,104,101, - 32,110,97,109,101,115,112,97,99,101,10,32,32,32,32,32, - 32,32,32,112,97,99,107,97,103,101,32,112,111,114,116,105, - 111,110,115,46,32,82,101,116,117,114,110,115,32,40,108,111, - 97,100,101,114,44,32,108,105,115,116,45,111,102,45,112,111, - 114,116,105,111,110,115,41,46,10,10,32,32,32,32,32,32, - 32,32,84,104,105,115,32,109,101,116,104,111,100,32,105,115, - 32,100,101,112,114,101,99,97,116,101,100,46,32,32,85,115, - 101,32,102,105,110,100,95,115,112,101,99,40,41,32,105,110, - 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, - 78,41,3,114,177,0,0,0,114,123,0,0,0,114,153,0, - 0,0,41,3,114,103,0,0,0,114,122,0,0,0,114,161, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,120,0,0,0,182,4,0,0,115,8,0,0,0, - 0,7,10,1,8,1,8,1,122,22,70,105,108,101,70,105, - 110,100,101,114,46,102,105,110,100,95,108,111,97,100,101,114, - 99,6,0,0,0,0,0,0,0,7,0,0,0,7,0,0, - 0,67,0,0,0,115,30,0,0,0,124,1,124,2,124,3, - 131,2,125,6,116,0,124,2,124,3,100,1,124,6,100,2, - 124,4,144,2,131,2,83,0,41,3,78,114,123,0,0,0, - 114,153,0,0,0,41,1,114,164,0,0,0,41,7,114,103, - 0,0,0,114,162,0,0,0,114,122,0,0,0,114,37,0, - 0,0,90,4,115,109,115,108,114,176,0,0,0,114,123,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,3,1,0,0,194,4,0,0,115,6,0,0,0,0, - 1,10,1,12,1,122,20,70,105,108,101,70,105,110,100,101, - 114,46,95,103,101,116,95,115,112,101,99,78,99,3,0,0, - 0,0,0,0,0,14,0,0,0,15,0,0,0,67,0,0, - 0,115,100,1,0,0,100,1,125,3,124,1,106,0,100,2, - 131,1,100,3,25,0,125,4,121,24,116,1,124,0,106,2, - 112,34,116,3,106,4,131,0,131,1,106,5,125,5,87,0, - 110,24,4,0,116,6,107,10,114,66,1,0,1,0,1,0, - 100,10,125,5,89,0,110,2,88,0,124,5,124,0,106,7, - 107,3,114,92,124,0,106,8,131,0,1,0,124,5,124,0, - 95,7,116,9,131,0,114,114,124,0,106,10,125,6,124,4, - 106,11,131,0,125,7,110,10,124,0,106,12,125,6,124,4, - 125,7,124,7,124,6,107,6,114,218,116,13,124,0,106,2, - 124,4,131,2,125,8,120,72,124,0,106,14,68,0,93,54, - 92,2,125,9,125,10,100,5,124,9,23,0,125,11,116,13, - 124,8,124,11,131,2,125,12,116,15,124,12,131,1,114,152, - 124,0,106,16,124,10,124,1,124,12,124,8,103,1,124,2, - 131,5,83,0,113,152,87,0,116,17,124,8,131,1,125,3, - 120,90,124,0,106,14,68,0,93,80,92,2,125,9,125,10, - 116,13,124,0,106,2,124,4,124,9,23,0,131,2,125,12, - 116,18,106,19,100,6,124,12,100,7,100,3,144,1,131,2, - 1,0,124,7,124,9,23,0,124,6,107,6,114,226,116,15, - 124,12,131,1,114,226,124,0,106,16,124,10,124,1,124,12, - 100,8,124,2,131,5,83,0,113,226,87,0,124,3,144,1, - 114,96,116,18,106,19,100,9,124,8,131,2,1,0,116,18, - 106,20,124,1,100,8,131,2,125,13,124,8,103,1,124,13, - 95,21,124,13,83,0,100,8,83,0,41,11,122,111,84,114, - 121,32,116,111,32,102,105,110,100,32,97,32,115,112,101,99, - 32,102,111,114,32,116,104,101,32,115,112,101,99,105,102,105, - 101,100,32,109,111,100,117,108,101,46,10,10,32,32,32,32, - 32,32,32,32,82,101,116,117,114,110,115,32,116,104,101,32, - 109,97,116,99,104,105,110,103,32,115,112,101,99,44,32,111, - 114,32,78,111,110,101,32,105,102,32,110,111,116,32,102,111, - 117,110,100,46,10,32,32,32,32,32,32,32,32,70,114,61, - 0,0,0,114,59,0,0,0,114,31,0,0,0,114,181,0, - 0,0,122,9,116,114,121,105,110,103,32,123,125,90,9,118, - 101,114,98,111,115,105,116,121,78,122,25,112,111,115,115,105, - 98,108,101,32,110,97,109,101,115,112,97,99,101,32,102,111, - 114,32,123,125,114,90,0,0,0,41,22,114,34,0,0,0, - 114,41,0,0,0,114,37,0,0,0,114,3,0,0,0,114, - 47,0,0,0,114,215,0,0,0,114,42,0,0,0,114,6, - 1,0,0,218,11,95,102,105,108,108,95,99,97,99,104,101, - 114,7,0,0,0,114,9,1,0,0,114,91,0,0,0,114, - 8,1,0,0,114,30,0,0,0,114,5,1,0,0,114,46, - 0,0,0,114,3,1,0,0,114,48,0,0,0,114,117,0, - 0,0,114,132,0,0,0,114,157,0,0,0,114,153,0,0, - 0,41,14,114,103,0,0,0,114,122,0,0,0,114,176,0, - 0,0,90,12,105,115,95,110,97,109,101,115,112,97,99,101, - 90,11,116,97,105,108,95,109,111,100,117,108,101,114,129,0, - 0,0,90,5,99,97,99,104,101,90,12,99,97,99,104,101, - 95,109,111,100,117,108,101,90,9,98,97,115,101,95,112,97, - 116,104,114,221,0,0,0,114,162,0,0,0,90,13,105,110, - 105,116,95,102,105,108,101,110,97,109,101,90,9,102,117,108, - 108,95,112,97,116,104,114,161,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,177,0,0,0,199, - 4,0,0,115,70,0,0,0,0,5,4,1,14,1,2,1, - 24,1,14,1,10,1,10,1,8,1,6,2,6,1,6,1, - 10,2,6,1,4,2,8,1,12,1,16,1,8,1,10,1, - 8,1,24,4,8,2,16,1,16,1,18,1,12,1,8,1, - 10,1,12,1,6,1,12,1,12,1,8,1,4,1,122,20, - 70,105,108,101,70,105,110,100,101,114,46,102,105,110,100,95, - 115,112,101,99,99,1,0,0,0,0,0,0,0,9,0,0, - 0,13,0,0,0,67,0,0,0,115,194,0,0,0,124,0, - 106,0,125,1,121,22,116,1,106,2,124,1,112,22,116,1, - 106,3,131,0,131,1,125,2,87,0,110,30,4,0,116,4, - 116,5,116,6,102,3,107,10,114,58,1,0,1,0,1,0, - 103,0,125,2,89,0,110,2,88,0,116,7,106,8,106,9, - 100,1,131,1,115,84,116,10,124,2,131,1,124,0,95,11, - 110,78,116,10,131,0,125,3,120,64,124,2,68,0,93,56, - 125,4,124,4,106,12,100,2,131,1,92,3,125,5,125,6, - 125,7,124,6,114,138,100,3,106,13,124,5,124,7,106,14, - 131,0,131,2,125,8,110,4,124,5,125,8,124,3,106,15, - 124,8,131,1,1,0,113,96,87,0,124,3,124,0,95,11, - 116,7,106,8,106,9,116,16,131,1,114,190,100,4,100,5, - 132,0,124,2,68,0,131,1,124,0,95,17,100,6,83,0, - 41,7,122,68,70,105,108,108,32,116,104,101,32,99,97,99, - 104,101,32,111,102,32,112,111,116,101,110,116,105,97,108,32, - 109,111,100,117,108,101,115,32,97,110,100,32,112,97,99,107, - 97,103,101,115,32,102,111,114,32,116,104,105,115,32,100,105, - 114,101,99,116,111,114,121,46,114,0,0,0,0,114,61,0, - 0,0,122,5,123,125,46,123,125,99,1,0,0,0,0,0, - 0,0,2,0,0,0,3,0,0,0,83,0,0,0,115,20, - 0,0,0,104,0,124,0,93,12,125,1,124,1,106,0,131, - 0,146,2,113,4,83,0,114,4,0,0,0,41,1,114,91, - 0,0,0,41,2,114,24,0,0,0,90,2,102,110,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,250,9,60, - 115,101,116,99,111,109,112,62,20,5,0,0,115,2,0,0, - 0,6,0,122,41,70,105,108,101,70,105,110,100,101,114,46, - 95,102,105,108,108,95,99,97,99,104,101,46,60,108,111,99, - 97,108,115,62,46,60,115,101,116,99,111,109,112,62,78,41, - 18,114,37,0,0,0,114,3,0,0,0,90,7,108,105,115, - 116,100,105,114,114,47,0,0,0,114,254,0,0,0,218,15, - 80,101,114,109,105,115,115,105,111,110,69,114,114,111,114,218, - 18,78,111,116,65,68,105,114,101,99,116,111,114,121,69,114, - 114,111,114,114,8,0,0,0,114,9,0,0,0,114,10,0, - 0,0,114,7,1,0,0,114,8,1,0,0,114,86,0,0, - 0,114,50,0,0,0,114,91,0,0,0,218,3,97,100,100, - 114,11,0,0,0,114,9,1,0,0,41,9,114,103,0,0, - 0,114,37,0,0,0,90,8,99,111,110,116,101,110,116,115, - 90,21,108,111,119,101,114,95,115,117,102,102,105,120,95,99, - 111,110,116,101,110,116,115,114,243,0,0,0,114,101,0,0, - 0,114,233,0,0,0,114,221,0,0,0,90,8,110,101,119, - 95,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,11,1,0,0,247,4,0,0,115,34,0, - 0,0,0,2,6,1,2,1,22,1,20,3,10,3,12,1, - 12,7,6,1,10,1,16,1,4,1,18,2,4,1,14,1, - 6,1,12,1,122,22,70,105,108,101,70,105,110,100,101,114, - 46,95,102,105,108,108,95,99,97,99,104,101,99,1,0,0, - 0,0,0,0,0,3,0,0,0,3,0,0,0,7,0,0, - 0,115,18,0,0,0,135,0,135,1,102,2,100,1,100,2, - 132,8,125,2,124,2,83,0,41,3,97,20,1,0,0,65, - 32,99,108,97,115,115,32,109,101,116,104,111,100,32,119,104, - 105,99,104,32,114,101,116,117,114,110,115,32,97,32,99,108, - 111,115,117,114,101,32,116,111,32,117,115,101,32,111,110,32, - 115,121,115,46,112,97,116,104,95,104,111,111,107,10,32,32, - 32,32,32,32,32,32,119,104,105,99,104,32,119,105,108,108, - 32,114,101,116,117,114,110,32,97,110,32,105,110,115,116,97, - 110,99,101,32,117,115,105,110,103,32,116,104,101,32,115,112, - 101,99,105,102,105,101,100,32,108,111,97,100,101,114,115,32, - 97,110,100,32,116,104,101,32,112,97,116,104,10,32,32,32, - 32,32,32,32,32,99,97,108,108,101,100,32,111,110,32,116, - 104,101,32,99,108,111,115,117,114,101,46,10,10,32,32,32, - 32,32,32,32,32,73,102,32,116,104,101,32,112,97,116,104, - 32,99,97,108,108,101,100,32,111,110,32,116,104,101,32,99, - 108,111,115,117,114,101,32,105,115,32,110,111,116,32,97,32, - 100,105,114,101,99,116,111,114,121,44,32,73,109,112,111,114, - 116,69,114,114,111,114,32,105,115,10,32,32,32,32,32,32, - 32,32,114,97,105,115,101,100,46,10,10,32,32,32,32,32, - 32,32,32,99,1,0,0,0,0,0,0,0,1,0,0,0, - 4,0,0,0,19,0,0,0,115,32,0,0,0,116,0,124, - 0,131,1,115,22,116,1,100,1,100,2,124,0,144,1,131, - 1,130,1,136,0,124,0,136,1,140,1,83,0,41,3,122, - 45,80,97,116,104,32,104,111,111,107,32,102,111,114,32,105, - 109,112,111,114,116,108,105,98,46,109,97,99,104,105,110,101, - 114,121,46,70,105,108,101,70,105,110,100,101,114,46,122,30, - 111,110,108,121,32,100,105,114,101,99,116,111,114,105,101,115, - 32,97,114,101,32,115,117,112,112,111,114,116,101,100,114,37, - 0,0,0,41,2,114,48,0,0,0,114,102,0,0,0,41, - 1,114,37,0,0,0,41,2,114,167,0,0,0,114,10,1, - 0,0,114,4,0,0,0,114,6,0,0,0,218,24,112,97, - 116,104,95,104,111,111,107,95,102,111,114,95,70,105,108,101, - 70,105,110,100,101,114,32,5,0,0,115,6,0,0,0,0, - 2,8,1,14,1,122,54,70,105,108,101,70,105,110,100,101, - 114,46,112,97,116,104,95,104,111,111,107,46,60,108,111,99, - 97,108,115,62,46,112,97,116,104,95,104,111,111,107,95,102, - 111,114,95,70,105,108,101,70,105,110,100,101,114,114,4,0, - 0,0,41,3,114,167,0,0,0,114,10,1,0,0,114,16, - 1,0,0,114,4,0,0,0,41,2,114,167,0,0,0,114, - 10,1,0,0,114,6,0,0,0,218,9,112,97,116,104,95, - 104,111,111,107,22,5,0,0,115,4,0,0,0,0,10,14, - 6,122,20,70,105,108,101,70,105,110,100,101,114,46,112,97, - 116,104,95,104,111,111,107,99,1,0,0,0,0,0,0,0, - 1,0,0,0,2,0,0,0,67,0,0,0,115,12,0,0, - 0,100,1,106,0,124,0,106,1,131,1,83,0,41,2,78, - 122,16,70,105,108,101,70,105,110,100,101,114,40,123,33,114, - 125,41,41,2,114,50,0,0,0,114,37,0,0,0,41,1, - 114,103,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,242,0,0,0,40,5,0,0,115,2,0, - 0,0,0,1,122,19,70,105,108,101,70,105,110,100,101,114, - 46,95,95,114,101,112,114,95,95,41,1,78,41,15,114,108, - 0,0,0,114,107,0,0,0,114,109,0,0,0,114,110,0, - 0,0,114,181,0,0,0,114,248,0,0,0,114,126,0,0, - 0,114,178,0,0,0,114,120,0,0,0,114,3,1,0,0, - 114,177,0,0,0,114,11,1,0,0,114,179,0,0,0,114, - 17,1,0,0,114,242,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,4,1, - 0,0,153,4,0,0,115,20,0,0,0,8,7,4,2,8, - 14,8,4,4,2,8,12,8,5,10,48,8,31,12,18,114, - 4,1,0,0,99,4,0,0,0,0,0,0,0,6,0,0, - 0,11,0,0,0,67,0,0,0,115,148,0,0,0,124,0, - 106,0,100,1,131,1,125,4,124,0,106,0,100,2,131,1, - 125,5,124,4,115,66,124,5,114,36,124,5,106,1,125,4, - 110,30,124,2,124,3,107,2,114,56,116,2,124,1,124,2, - 131,2,125,4,110,10,116,3,124,1,124,2,131,2,125,4, - 124,5,115,86,116,4,124,1,124,2,100,3,124,4,144,1, - 131,2,125,5,121,36,124,5,124,0,100,2,60,0,124,4, - 124,0,100,1,60,0,124,2,124,0,100,4,60,0,124,3, - 124,0,100,5,60,0,87,0,110,20,4,0,116,5,107,10, - 114,142,1,0,1,0,1,0,89,0,110,2,88,0,100,0, - 83,0,41,6,78,218,10,95,95,108,111,97,100,101,114,95, - 95,218,8,95,95,115,112,101,99,95,95,114,123,0,0,0, - 90,8,95,95,102,105,108,101,95,95,90,10,95,95,99,97, - 99,104,101,100,95,95,41,6,218,3,103,101,116,114,123,0, - 0,0,114,219,0,0,0,114,214,0,0,0,114,164,0,0, - 0,218,9,69,120,99,101,112,116,105,111,110,41,6,90,2, - 110,115,114,101,0,0,0,90,8,112,97,116,104,110,97,109, - 101,90,9,99,112,97,116,104,110,97,109,101,114,123,0,0, - 0,114,161,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,218,14,95,102,105,120,95,117,112,95,109, - 111,100,117,108,101,46,5,0,0,115,34,0,0,0,0,2, - 10,1,10,1,4,1,4,1,8,1,8,1,12,2,10,1, - 4,1,16,1,2,1,8,1,8,1,8,1,12,1,14,2, - 114,22,1,0,0,99,0,0,0,0,0,0,0,0,3,0, - 0,0,3,0,0,0,67,0,0,0,115,38,0,0,0,116, - 0,116,1,106,2,131,0,102,2,125,0,116,3,116,4,102, - 2,125,1,116,5,116,6,102,2,125,2,124,0,124,1,124, - 2,103,3,83,0,41,1,122,95,82,101,116,117,114,110,115, - 32,97,32,108,105,115,116,32,111,102,32,102,105,108,101,45, - 98,97,115,101,100,32,109,111,100,117,108,101,32,108,111,97, - 100,101,114,115,46,10,10,32,32,32,32,69,97,99,104,32, - 105,116,101,109,32,105,115,32,97,32,116,117,112,108,101,32, - 40,108,111,97,100,101,114,44,32,115,117,102,102,105,120,101, - 115,41,46,10,32,32,32,32,41,7,114,220,0,0,0,114, - 142,0,0,0,218,18,101,120,116,101,110,115,105,111,110,95, - 115,117,102,102,105,120,101,115,114,214,0,0,0,114,87,0, - 0,0,114,219,0,0,0,114,77,0,0,0,41,3,90,10, - 101,120,116,101,110,115,105,111,110,115,90,6,115,111,117,114, - 99,101,90,8,98,121,116,101,99,111,100,101,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,158,0,0,0, - 69,5,0,0,115,8,0,0,0,0,5,12,1,8,1,8, - 1,114,158,0,0,0,99,1,0,0,0,0,0,0,0,12, - 0,0,0,12,0,0,0,67,0,0,0,115,188,1,0,0, - 124,0,97,0,116,0,106,1,97,1,116,0,106,2,97,2, - 116,1,106,3,116,4,25,0,125,1,120,56,100,26,68,0, - 93,48,125,2,124,2,116,1,106,3,107,7,114,58,116,0, - 106,5,124,2,131,1,125,3,110,10,116,1,106,3,124,2, - 25,0,125,3,116,6,124,1,124,2,124,3,131,3,1,0, - 113,32,87,0,100,5,100,6,103,1,102,2,100,7,100,8, - 100,6,103,2,102,2,102,2,125,4,120,118,124,4,68,0, - 93,102,92,2,125,5,125,6,116,7,100,9,100,10,132,0, - 124,6,68,0,131,1,131,1,115,142,116,8,130,1,124,6, - 100,11,25,0,125,7,124,5,116,1,106,3,107,6,114,174, - 116,1,106,3,124,5,25,0,125,8,80,0,113,112,121,16, - 116,0,106,5,124,5,131,1,125,8,80,0,87,0,113,112, - 4,0,116,9,107,10,114,212,1,0,1,0,1,0,119,112, - 89,0,113,112,88,0,113,112,87,0,116,9,100,12,131,1, - 130,1,116,6,124,1,100,13,124,8,131,3,1,0,116,6, - 124,1,100,14,124,7,131,3,1,0,116,6,124,1,100,15, - 100,16,106,10,124,6,131,1,131,3,1,0,121,14,116,0, - 106,5,100,17,131,1,125,9,87,0,110,26,4,0,116,9, - 107,10,144,1,114,52,1,0,1,0,1,0,100,18,125,9, - 89,0,110,2,88,0,116,6,124,1,100,17,124,9,131,3, - 1,0,116,0,106,5,100,19,131,1,125,10,116,6,124,1, - 100,19,124,10,131,3,1,0,124,5,100,7,107,2,144,1, - 114,120,116,0,106,5,100,20,131,1,125,11,116,6,124,1, - 100,21,124,11,131,3,1,0,116,6,124,1,100,22,116,11, - 131,0,131,3,1,0,116,12,106,13,116,2,106,14,131,0, - 131,1,1,0,124,5,100,7,107,2,144,1,114,184,116,15, - 106,16,100,23,131,1,1,0,100,24,116,12,107,6,144,1, - 114,184,100,25,116,17,95,18,100,18,83,0,41,27,122,205, - 83,101,116,117,112,32,116,104,101,32,112,97,116,104,45,98, - 97,115,101,100,32,105,109,112,111,114,116,101,114,115,32,102, - 111,114,32,105,109,112,111,114,116,108,105,98,32,98,121,32, - 105,109,112,111,114,116,105,110,103,32,110,101,101,100,101,100, - 10,32,32,32,32,98,117,105,108,116,45,105,110,32,109,111, - 100,117,108,101,115,32,97,110,100,32,105,110,106,101,99,116, - 105,110,103,32,116,104,101,109,32,105,110,116,111,32,116,104, - 101,32,103,108,111,98,97,108,32,110,97,109,101,115,112,97, - 99,101,46,10,10,32,32,32,32,79,116,104,101,114,32,99, - 111,109,112,111,110,101,110,116,115,32,97,114,101,32,101,120, - 116,114,97,99,116,101,100,32,102,114,111,109,32,116,104,101, - 32,99,111,114,101,32,98,111,111,116,115,116,114,97,112,32, - 109,111,100,117,108,101,46,10,10,32,32,32,32,114,52,0, - 0,0,114,63,0,0,0,218,8,98,117,105,108,116,105,110, - 115,114,139,0,0,0,90,5,112,111,115,105,120,250,1,47, - 218,2,110,116,250,1,92,99,1,0,0,0,0,0,0,0, - 2,0,0,0,3,0,0,0,115,0,0,0,115,26,0,0, - 0,124,0,93,18,125,1,116,0,124,1,131,1,100,0,107, - 2,86,0,1,0,113,2,100,1,83,0,41,2,114,31,0, - 0,0,78,41,1,114,33,0,0,0,41,2,114,24,0,0, - 0,114,80,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,223,0,0,0,105,5,0,0,115,2, - 0,0,0,4,0,122,25,95,115,101,116,117,112,46,60,108, - 111,99,97,108,115,62,46,60,103,101,110,101,120,112,114,62, - 114,62,0,0,0,122,30,105,109,112,111,114,116,108,105,98, - 32,114,101,113,117,105,114,101,115,32,112,111,115,105,120,32, - 111,114,32,110,116,114,3,0,0,0,114,27,0,0,0,114, - 23,0,0,0,114,32,0,0,0,90,7,95,116,104,114,101, - 97,100,78,90,8,95,119,101,97,107,114,101,102,90,6,119, - 105,110,114,101,103,114,166,0,0,0,114,7,0,0,0,122, - 4,46,112,121,119,122,6,95,100,46,112,121,100,84,41,4, - 122,3,95,105,111,122,9,95,119,97,114,110,105,110,103,115, - 122,8,98,117,105,108,116,105,110,115,122,7,109,97,114,115, - 104,97,108,41,19,114,117,0,0,0,114,8,0,0,0,114, - 142,0,0,0,114,235,0,0,0,114,108,0,0,0,90,18, - 95,98,117,105,108,116,105,110,95,102,114,111,109,95,110,97, - 109,101,114,112,0,0,0,218,3,97,108,108,218,14,65,115, - 115,101,114,116,105,111,110,69,114,114,111,114,114,102,0,0, - 0,114,28,0,0,0,114,13,0,0,0,114,225,0,0,0, - 114,146,0,0,0,114,23,1,0,0,114,87,0,0,0,114, - 160,0,0,0,114,165,0,0,0,114,169,0,0,0,41,12, - 218,17,95,98,111,111,116,115,116,114,97,112,95,109,111,100, - 117,108,101,90,11,115,101,108,102,95,109,111,100,117,108,101, - 90,12,98,117,105,108,116,105,110,95,110,97,109,101,90,14, - 98,117,105,108,116,105,110,95,109,111,100,117,108,101,90,10, - 111,115,95,100,101,116,97,105,108,115,90,10,98,117,105,108, - 116,105,110,95,111,115,114,23,0,0,0,114,27,0,0,0, - 90,9,111,115,95,109,111,100,117,108,101,90,13,116,104,114, - 101,97,100,95,109,111,100,117,108,101,90,14,119,101,97,107, - 114,101,102,95,109,111,100,117,108,101,90,13,119,105,110,114, - 101,103,95,109,111,100,117,108,101,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,6,95,115,101,116,117,112, - 80,5,0,0,115,82,0,0,0,0,8,4,1,6,1,6, - 3,10,1,10,1,10,1,12,2,10,1,16,3,22,1,14, - 2,22,1,8,1,10,1,10,1,4,2,2,1,10,1,6, - 1,14,1,12,2,8,1,12,1,12,1,18,3,2,1,14, - 1,16,2,10,1,12,3,10,1,12,3,10,1,10,1,12, - 3,14,1,14,1,10,1,10,1,10,1,114,31,1,0,0, + 0,114,178,0,0,0,203,4,0,0,115,70,0,0,0,0, + 5,4,1,14,1,2,1,24,1,14,1,10,1,10,1,8, + 1,6,2,6,1,6,1,10,2,6,1,4,2,8,1,12, + 1,16,1,8,1,10,1,8,1,24,4,8,2,16,1,16, + 1,18,1,12,1,8,1,10,1,12,1,6,1,12,1,12, + 1,8,1,4,1,122,20,70,105,108,101,70,105,110,100,101, + 114,46,102,105,110,100,95,115,112,101,99,99,1,0,0,0, + 0,0,0,0,9,0,0,0,13,0,0,0,67,0,0,0, + 115,194,0,0,0,124,0,106,0,125,1,121,22,116,1,106, + 2,124,1,112,22,116,1,106,3,131,0,131,1,125,2,87, + 0,110,30,4,0,116,4,116,5,116,6,102,3,107,10,114, + 58,1,0,1,0,1,0,103,0,125,2,89,0,110,2,88, + 0,116,7,106,8,106,9,100,1,131,1,115,84,116,10,124, + 2,131,1,124,0,95,11,110,78,116,10,131,0,125,3,120, + 64,124,2,68,0,93,56,125,4,124,4,106,12,100,2,131, + 1,92,3,125,5,125,6,125,7,124,6,114,138,100,3,106, + 13,124,5,124,7,106,14,131,0,131,2,125,8,110,4,124, + 5,125,8,124,3,106,15,124,8,131,1,1,0,113,96,87, + 0,124,3,124,0,95,11,116,7,106,8,106,9,116,16,131, + 1,114,190,100,4,100,5,132,0,124,2,68,0,131,1,124, + 0,95,17,100,6,83,0,41,7,122,68,70,105,108,108,32, + 116,104,101,32,99,97,99,104,101,32,111,102,32,112,111,116, + 101,110,116,105,97,108,32,109,111,100,117,108,101,115,32,97, + 110,100,32,112,97,99,107,97,103,101,115,32,102,111,114,32, + 116,104,105,115,32,100,105,114,101,99,116,111,114,121,46,114, + 0,0,0,0,114,61,0,0,0,122,5,123,125,46,123,125, 99,1,0,0,0,0,0,0,0,2,0,0,0,3,0,0, - 0,67,0,0,0,115,84,0,0,0,116,0,124,0,131,1, - 1,0,116,1,131,0,125,1,116,2,106,3,106,4,116,5, - 106,6,124,1,140,0,103,1,131,1,1,0,116,7,106,8, - 100,1,107,2,114,56,116,2,106,9,106,10,116,11,131,1, - 1,0,116,2,106,9,106,10,116,12,131,1,1,0,116,5, - 124,0,95,5,116,13,124,0,95,13,100,2,83,0,41,3, - 122,41,73,110,115,116,97,108,108,32,116,104,101,32,112,97, - 116,104,45,98,97,115,101,100,32,105,109,112,111,114,116,32, - 99,111,109,112,111,110,101,110,116,115,46,114,26,1,0,0, - 78,41,14,114,31,1,0,0,114,158,0,0,0,114,8,0, - 0,0,114,252,0,0,0,114,146,0,0,0,114,4,1,0, - 0,114,17,1,0,0,114,3,0,0,0,114,108,0,0,0, - 218,9,109,101,116,97,95,112,97,116,104,114,160,0,0,0, - 114,165,0,0,0,114,247,0,0,0,114,214,0,0,0,41, - 2,114,30,1,0,0,90,17,115,117,112,112,111,114,116,101, - 100,95,108,111,97,100,101,114,115,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,8,95,105,110,115,116,97, - 108,108,148,5,0,0,115,16,0,0,0,0,2,8,1,6, - 1,20,1,10,1,12,1,12,4,6,1,114,33,1,0,0, - 41,1,122,3,119,105,110,41,2,114,1,0,0,0,114,2, - 0,0,0,41,1,114,49,0,0,0,41,1,78,41,3,78, - 78,78,41,3,78,78,78,41,2,114,62,0,0,0,114,62, - 0,0,0,41,1,78,41,1,78,41,58,114,110,0,0,0, - 114,12,0,0,0,90,37,95,67,65,83,69,95,73,78,83, - 69,78,83,73,84,73,86,69,95,80,76,65,84,70,79,82, - 77,83,95,66,89,84,69,83,95,75,69,89,114,11,0,0, - 0,114,13,0,0,0,114,19,0,0,0,114,21,0,0,0, - 114,30,0,0,0,114,40,0,0,0,114,41,0,0,0,114, - 45,0,0,0,114,46,0,0,0,114,48,0,0,0,114,58, - 0,0,0,218,4,116,121,112,101,218,8,95,95,99,111,100, - 101,95,95,114,141,0,0,0,114,17,0,0,0,114,131,0, - 0,0,114,16,0,0,0,114,20,0,0,0,90,17,95,82, - 65,87,95,77,65,71,73,67,95,78,85,77,66,69,82,114, - 76,0,0,0,114,75,0,0,0,114,87,0,0,0,114,77, - 0,0,0,90,23,68,69,66,85,71,95,66,89,84,69,67, - 79,68,69,95,83,85,70,70,73,88,69,83,90,27,79,80, - 84,73,77,73,90,69,68,95,66,89,84,69,67,79,68,69, - 95,83,85,70,70,73,88,69,83,114,82,0,0,0,114,88, - 0,0,0,114,94,0,0,0,114,98,0,0,0,114,100,0, - 0,0,114,119,0,0,0,114,126,0,0,0,114,138,0,0, - 0,114,144,0,0,0,114,147,0,0,0,114,152,0,0,0, - 218,6,111,98,106,101,99,116,114,159,0,0,0,114,164,0, - 0,0,114,165,0,0,0,114,180,0,0,0,114,190,0,0, - 0,114,206,0,0,0,114,214,0,0,0,114,219,0,0,0, - 114,225,0,0,0,114,220,0,0,0,114,226,0,0,0,114, - 245,0,0,0,114,247,0,0,0,114,4,1,0,0,114,22, - 1,0,0,114,158,0,0,0,114,31,1,0,0,114,33,1, - 0,0,114,4,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,218,8,60,109,111,100,117,108,101,62, - 8,0,0,0,115,108,0,0,0,4,16,4,1,4,1,2, - 1,6,3,8,17,8,5,8,5,8,6,8,12,8,10,8, - 9,8,5,8,7,10,22,10,117,16,1,12,2,4,1,4, - 2,6,2,6,2,8,2,16,44,8,33,8,19,8,12,8, - 12,8,28,8,17,10,55,10,12,10,10,8,14,6,3,4, - 1,14,65,14,64,14,29,16,110,14,41,18,45,18,16,4, - 3,18,53,14,60,14,42,14,127,0,5,14,127,0,22,10, - 23,8,11,8,68, + 0,83,0,0,0,115,20,0,0,0,104,0,124,0,93,12, + 125,1,124,1,106,0,131,0,146,2,113,4,83,0,114,4, + 0,0,0,41,1,114,92,0,0,0,41,2,114,24,0,0, + 0,90,2,102,110,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,250,9,60,115,101,116,99,111,109,112,62,24, + 5,0,0,115,2,0,0,0,6,0,122,41,70,105,108,101, + 70,105,110,100,101,114,46,95,102,105,108,108,95,99,97,99, + 104,101,46,60,108,111,99,97,108,115,62,46,60,115,101,116, + 99,111,109,112,62,78,41,18,114,37,0,0,0,114,3,0, + 0,0,90,7,108,105,115,116,100,105,114,114,47,0,0,0, + 114,255,0,0,0,218,15,80,101,114,109,105,115,115,105,111, + 110,69,114,114,111,114,218,18,78,111,116,65,68,105,114,101, + 99,116,111,114,121,69,114,114,111,114,114,8,0,0,0,114, + 9,0,0,0,114,10,0,0,0,114,8,1,0,0,114,9, + 1,0,0,114,87,0,0,0,114,50,0,0,0,114,92,0, + 0,0,218,3,97,100,100,114,11,0,0,0,114,10,1,0, + 0,41,9,114,104,0,0,0,114,37,0,0,0,90,8,99, + 111,110,116,101,110,116,115,90,21,108,111,119,101,114,95,115, + 117,102,102,105,120,95,99,111,110,116,101,110,116,115,114,244, + 0,0,0,114,102,0,0,0,114,234,0,0,0,114,222,0, + 0,0,90,8,110,101,119,95,110,97,109,101,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,12,1,0,0, + 251,4,0,0,115,34,0,0,0,0,2,6,1,2,1,22, + 1,20,3,10,3,12,1,12,7,6,1,10,1,16,1,4, + 1,18,2,4,1,14,1,6,1,12,1,122,22,70,105,108, + 101,70,105,110,100,101,114,46,95,102,105,108,108,95,99,97, + 99,104,101,99,1,0,0,0,0,0,0,0,3,0,0,0, + 3,0,0,0,7,0,0,0,115,18,0,0,0,135,0,135, + 1,102,2,100,1,100,2,132,8,125,2,124,2,83,0,41, + 3,97,20,1,0,0,65,32,99,108,97,115,115,32,109,101, + 116,104,111,100,32,119,104,105,99,104,32,114,101,116,117,114, + 110,115,32,97,32,99,108,111,115,117,114,101,32,116,111,32, + 117,115,101,32,111,110,32,115,121,115,46,112,97,116,104,95, + 104,111,111,107,10,32,32,32,32,32,32,32,32,119,104,105, + 99,104,32,119,105,108,108,32,114,101,116,117,114,110,32,97, + 110,32,105,110,115,116,97,110,99,101,32,117,115,105,110,103, + 32,116,104,101,32,115,112,101,99,105,102,105,101,100,32,108, + 111,97,100,101,114,115,32,97,110,100,32,116,104,101,32,112, + 97,116,104,10,32,32,32,32,32,32,32,32,99,97,108,108, + 101,100,32,111,110,32,116,104,101,32,99,108,111,115,117,114, + 101,46,10,10,32,32,32,32,32,32,32,32,73,102,32,116, + 104,101,32,112,97,116,104,32,99,97,108,108,101,100,32,111, + 110,32,116,104,101,32,99,108,111,115,117,114,101,32,105,115, + 32,110,111,116,32,97,32,100,105,114,101,99,116,111,114,121, + 44,32,73,109,112,111,114,116,69,114,114,111,114,32,105,115, + 10,32,32,32,32,32,32,32,32,114,97,105,115,101,100,46, + 10,10,32,32,32,32,32,32,32,32,99,1,0,0,0,0, + 0,0,0,1,0,0,0,4,0,0,0,19,0,0,0,115, + 32,0,0,0,116,0,124,0,131,1,115,22,116,1,100,1, + 100,2,124,0,144,1,131,1,130,1,136,0,124,0,136,1, + 140,1,83,0,41,3,122,45,80,97,116,104,32,104,111,111, + 107,32,102,111,114,32,105,109,112,111,114,116,108,105,98,46, + 109,97,99,104,105,110,101,114,121,46,70,105,108,101,70,105, + 110,100,101,114,46,122,30,111,110,108,121,32,100,105,114,101, + 99,116,111,114,105,101,115,32,97,114,101,32,115,117,112,112, + 111,114,116,101,100,114,37,0,0,0,41,2,114,48,0,0, + 0,114,103,0,0,0,41,1,114,37,0,0,0,41,2,114, + 168,0,0,0,114,11,1,0,0,114,4,0,0,0,114,6, + 0,0,0,218,24,112,97,116,104,95,104,111,111,107,95,102, + 111,114,95,70,105,108,101,70,105,110,100,101,114,36,5,0, + 0,115,6,0,0,0,0,2,8,1,14,1,122,54,70,105, + 108,101,70,105,110,100,101,114,46,112,97,116,104,95,104,111, + 111,107,46,60,108,111,99,97,108,115,62,46,112,97,116,104, + 95,104,111,111,107,95,102,111,114,95,70,105,108,101,70,105, + 110,100,101,114,114,4,0,0,0,41,3,114,168,0,0,0, + 114,11,1,0,0,114,17,1,0,0,114,4,0,0,0,41, + 2,114,168,0,0,0,114,11,1,0,0,114,6,0,0,0, + 218,9,112,97,116,104,95,104,111,111,107,26,5,0,0,115, + 4,0,0,0,0,10,14,6,122,20,70,105,108,101,70,105, + 110,100,101,114,46,112,97,116,104,95,104,111,111,107,99,1, + 0,0,0,0,0,0,0,1,0,0,0,2,0,0,0,67, + 0,0,0,115,12,0,0,0,100,1,106,0,124,0,106,1, + 131,1,83,0,41,2,78,122,16,70,105,108,101,70,105,110, + 100,101,114,40,123,33,114,125,41,41,2,114,50,0,0,0, + 114,37,0,0,0,41,1,114,104,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,243,0,0,0, + 44,5,0,0,115,2,0,0,0,0,1,122,19,70,105,108, + 101,70,105,110,100,101,114,46,95,95,114,101,112,114,95,95, + 41,1,78,41,15,114,109,0,0,0,114,108,0,0,0,114, + 110,0,0,0,114,111,0,0,0,114,182,0,0,0,114,249, + 0,0,0,114,127,0,0,0,114,179,0,0,0,114,121,0, + 0,0,114,4,1,0,0,114,178,0,0,0,114,12,1,0, + 0,114,180,0,0,0,114,18,1,0,0,114,243,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,5,1,0,0,157,4,0,0,115,20,0, + 0,0,8,7,4,2,8,14,8,4,4,2,8,12,8,5, + 10,48,8,31,12,18,114,5,1,0,0,99,4,0,0,0, + 0,0,0,0,6,0,0,0,11,0,0,0,67,0,0,0, + 115,148,0,0,0,124,0,106,0,100,1,131,1,125,4,124, + 0,106,0,100,2,131,1,125,5,124,4,115,66,124,5,114, + 36,124,5,106,1,125,4,110,30,124,2,124,3,107,2,114, + 56,116,2,124,1,124,2,131,2,125,4,110,10,116,3,124, + 1,124,2,131,2,125,4,124,5,115,86,116,4,124,1,124, + 2,100,3,124,4,144,1,131,2,125,5,121,36,124,5,124, + 0,100,2,60,0,124,4,124,0,100,1,60,0,124,2,124, + 0,100,4,60,0,124,3,124,0,100,5,60,0,87,0,110, + 20,4,0,116,5,107,10,114,142,1,0,1,0,1,0,89, + 0,110,2,88,0,100,0,83,0,41,6,78,218,10,95,95, + 108,111,97,100,101,114,95,95,218,8,95,95,115,112,101,99, + 95,95,114,124,0,0,0,90,8,95,95,102,105,108,101,95, + 95,90,10,95,95,99,97,99,104,101,100,95,95,41,6,218, + 3,103,101,116,114,124,0,0,0,114,220,0,0,0,114,215, + 0,0,0,114,165,0,0,0,218,9,69,120,99,101,112,116, + 105,111,110,41,6,90,2,110,115,114,102,0,0,0,90,8, + 112,97,116,104,110,97,109,101,90,9,99,112,97,116,104,110, + 97,109,101,114,124,0,0,0,114,162,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,218,14,95,102, + 105,120,95,117,112,95,109,111,100,117,108,101,50,5,0,0, + 115,34,0,0,0,0,2,10,1,10,1,4,1,4,1,8, + 1,8,1,12,2,10,1,4,1,16,1,2,1,8,1,8, + 1,8,1,12,1,14,2,114,23,1,0,0,99,0,0,0, + 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, + 0,115,38,0,0,0,116,0,116,1,106,2,131,0,102,2, + 125,0,116,3,116,4,102,2,125,1,116,5,116,6,102,2, + 125,2,124,0,124,1,124,2,103,3,83,0,41,1,122,95, + 82,101,116,117,114,110,115,32,97,32,108,105,115,116,32,111, + 102,32,102,105,108,101,45,98,97,115,101,100,32,109,111,100, + 117,108,101,32,108,111,97,100,101,114,115,46,10,10,32,32, + 32,32,69,97,99,104,32,105,116,101,109,32,105,115,32,97, + 32,116,117,112,108,101,32,40,108,111,97,100,101,114,44,32, + 115,117,102,102,105,120,101,115,41,46,10,32,32,32,32,41, + 7,114,221,0,0,0,114,143,0,0,0,218,18,101,120,116, + 101,110,115,105,111,110,95,115,117,102,102,105,120,101,115,114, + 215,0,0,0,114,88,0,0,0,114,220,0,0,0,114,78, + 0,0,0,41,3,90,10,101,120,116,101,110,115,105,111,110, + 115,90,6,115,111,117,114,99,101,90,8,98,121,116,101,99, + 111,100,101,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,114,159,0,0,0,73,5,0,0,115,8,0,0,0, + 0,5,12,1,8,1,8,1,114,159,0,0,0,99,1,0, + 0,0,0,0,0,0,12,0,0,0,12,0,0,0,67,0, + 0,0,115,188,1,0,0,124,0,97,0,116,0,106,1,97, + 1,116,0,106,2,97,2,116,1,106,3,116,4,25,0,125, + 1,120,56,100,26,68,0,93,48,125,2,124,2,116,1,106, + 3,107,7,114,58,116,0,106,5,124,2,131,1,125,3,110, + 10,116,1,106,3,124,2,25,0,125,3,116,6,124,1,124, + 2,124,3,131,3,1,0,113,32,87,0,100,5,100,6,103, + 1,102,2,100,7,100,8,100,6,103,2,102,2,102,2,125, + 4,120,118,124,4,68,0,93,102,92,2,125,5,125,6,116, + 7,100,9,100,10,132,0,124,6,68,0,131,1,131,1,115, + 142,116,8,130,1,124,6,100,11,25,0,125,7,124,5,116, + 1,106,3,107,6,114,174,116,1,106,3,124,5,25,0,125, + 8,80,0,113,112,121,16,116,0,106,5,124,5,131,1,125, + 8,80,0,87,0,113,112,4,0,116,9,107,10,114,212,1, + 0,1,0,1,0,119,112,89,0,113,112,88,0,113,112,87, + 0,116,9,100,12,131,1,130,1,116,6,124,1,100,13,124, + 8,131,3,1,0,116,6,124,1,100,14,124,7,131,3,1, + 0,116,6,124,1,100,15,100,16,106,10,124,6,131,1,131, + 3,1,0,121,14,116,0,106,5,100,17,131,1,125,9,87, + 0,110,26,4,0,116,9,107,10,144,1,114,52,1,0,1, + 0,1,0,100,18,125,9,89,0,110,2,88,0,116,6,124, + 1,100,17,124,9,131,3,1,0,116,0,106,5,100,19,131, + 1,125,10,116,6,124,1,100,19,124,10,131,3,1,0,124, + 5,100,7,107,2,144,1,114,120,116,0,106,5,100,20,131, + 1,125,11,116,6,124,1,100,21,124,11,131,3,1,0,116, + 6,124,1,100,22,116,11,131,0,131,3,1,0,116,12,106, + 13,116,2,106,14,131,0,131,1,1,0,124,5,100,7,107, + 2,144,1,114,184,116,15,106,16,100,23,131,1,1,0,100, + 24,116,12,107,6,144,1,114,184,100,25,116,17,95,18,100, + 18,83,0,41,27,122,205,83,101,116,117,112,32,116,104,101, + 32,112,97,116,104,45,98,97,115,101,100,32,105,109,112,111, + 114,116,101,114,115,32,102,111,114,32,105,109,112,111,114,116, + 108,105,98,32,98,121,32,105,109,112,111,114,116,105,110,103, + 32,110,101,101,100,101,100,10,32,32,32,32,98,117,105,108, + 116,45,105,110,32,109,111,100,117,108,101,115,32,97,110,100, + 32,105,110,106,101,99,116,105,110,103,32,116,104,101,109,32, + 105,110,116,111,32,116,104,101,32,103,108,111,98,97,108,32, + 110,97,109,101,115,112,97,99,101,46,10,10,32,32,32,32, + 79,116,104,101,114,32,99,111,109,112,111,110,101,110,116,115, + 32,97,114,101,32,101,120,116,114,97,99,116,101,100,32,102, + 114,111,109,32,116,104,101,32,99,111,114,101,32,98,111,111, + 116,115,116,114,97,112,32,109,111,100,117,108,101,46,10,10, + 32,32,32,32,114,52,0,0,0,114,63,0,0,0,218,8, + 98,117,105,108,116,105,110,115,114,140,0,0,0,90,5,112, + 111,115,105,120,250,1,47,218,2,110,116,250,1,92,99,1, + 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,115, + 0,0,0,115,26,0,0,0,124,0,93,18,125,1,116,0, + 124,1,131,1,100,0,107,2,86,0,1,0,113,2,100,1, + 83,0,41,2,114,31,0,0,0,78,41,1,114,33,0,0, + 0,41,2,114,24,0,0,0,114,81,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,224,0,0, + 0,109,5,0,0,115,2,0,0,0,4,0,122,25,95,115, + 101,116,117,112,46,60,108,111,99,97,108,115,62,46,60,103, + 101,110,101,120,112,114,62,114,62,0,0,0,122,30,105,109, + 112,111,114,116,108,105,98,32,114,101,113,117,105,114,101,115, + 32,112,111,115,105,120,32,111,114,32,110,116,114,3,0,0, + 0,114,27,0,0,0,114,23,0,0,0,114,32,0,0,0, + 90,7,95,116,104,114,101,97,100,78,90,8,95,119,101,97, + 107,114,101,102,90,6,119,105,110,114,101,103,114,167,0,0, + 0,114,7,0,0,0,122,4,46,112,121,119,122,6,95,100, + 46,112,121,100,84,41,4,122,3,95,105,111,122,9,95,119, + 97,114,110,105,110,103,115,122,8,98,117,105,108,116,105,110, + 115,122,7,109,97,114,115,104,97,108,41,19,114,118,0,0, + 0,114,8,0,0,0,114,143,0,0,0,114,236,0,0,0, + 114,109,0,0,0,90,18,95,98,117,105,108,116,105,110,95, + 102,114,111,109,95,110,97,109,101,114,113,0,0,0,218,3, + 97,108,108,218,14,65,115,115,101,114,116,105,111,110,69,114, + 114,111,114,114,103,0,0,0,114,28,0,0,0,114,13,0, + 0,0,114,226,0,0,0,114,147,0,0,0,114,24,1,0, + 0,114,88,0,0,0,114,161,0,0,0,114,166,0,0,0, + 114,170,0,0,0,41,12,218,17,95,98,111,111,116,115,116, + 114,97,112,95,109,111,100,117,108,101,90,11,115,101,108,102, + 95,109,111,100,117,108,101,90,12,98,117,105,108,116,105,110, + 95,110,97,109,101,90,14,98,117,105,108,116,105,110,95,109, + 111,100,117,108,101,90,10,111,115,95,100,101,116,97,105,108, + 115,90,10,98,117,105,108,116,105,110,95,111,115,114,23,0, + 0,0,114,27,0,0,0,90,9,111,115,95,109,111,100,117, + 108,101,90,13,116,104,114,101,97,100,95,109,111,100,117,108, + 101,90,14,119,101,97,107,114,101,102,95,109,111,100,117,108, + 101,90,13,119,105,110,114,101,103,95,109,111,100,117,108,101, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, + 6,95,115,101,116,117,112,84,5,0,0,115,82,0,0,0, + 0,8,4,1,6,1,6,3,10,1,10,1,10,1,12,2, + 10,1,16,3,22,1,14,2,22,1,8,1,10,1,10,1, + 4,2,2,1,10,1,6,1,14,1,12,2,8,1,12,1, + 12,1,18,3,2,1,14,1,16,2,10,1,12,3,10,1, + 12,3,10,1,10,1,12,3,14,1,14,1,10,1,10,1, + 10,1,114,32,1,0,0,99,1,0,0,0,0,0,0,0, + 2,0,0,0,3,0,0,0,67,0,0,0,115,84,0,0, + 0,116,0,124,0,131,1,1,0,116,1,131,0,125,1,116, + 2,106,3,106,4,116,5,106,6,124,1,140,0,103,1,131, + 1,1,0,116,7,106,8,100,1,107,2,114,56,116,2,106, + 9,106,10,116,11,131,1,1,0,116,2,106,9,106,10,116, + 12,131,1,1,0,116,5,124,0,95,5,116,13,124,0,95, + 13,100,2,83,0,41,3,122,41,73,110,115,116,97,108,108, + 32,116,104,101,32,112,97,116,104,45,98,97,115,101,100,32, + 105,109,112,111,114,116,32,99,111,109,112,111,110,101,110,116, + 115,46,114,27,1,0,0,78,41,14,114,32,1,0,0,114, + 159,0,0,0,114,8,0,0,0,114,253,0,0,0,114,147, + 0,0,0,114,5,1,0,0,114,18,1,0,0,114,3,0, + 0,0,114,109,0,0,0,218,9,109,101,116,97,95,112,97, + 116,104,114,161,0,0,0,114,166,0,0,0,114,248,0,0, + 0,114,215,0,0,0,41,2,114,31,1,0,0,90,17,115, + 117,112,112,111,114,116,101,100,95,108,111,97,100,101,114,115, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, + 8,95,105,110,115,116,97,108,108,152,5,0,0,115,16,0, + 0,0,0,2,8,1,6,1,20,1,10,1,12,1,12,4, + 6,1,114,34,1,0,0,41,1,122,3,119,105,110,41,2, + 114,1,0,0,0,114,2,0,0,0,41,1,114,49,0,0, + 0,41,1,78,41,3,78,78,78,41,3,78,78,78,41,2, + 114,62,0,0,0,114,62,0,0,0,41,1,78,41,1,78, + 41,58,114,111,0,0,0,114,12,0,0,0,90,37,95,67, + 65,83,69,95,73,78,83,69,78,83,73,84,73,86,69,95, + 80,76,65,84,70,79,82,77,83,95,66,89,84,69,83,95, + 75,69,89,114,11,0,0,0,114,13,0,0,0,114,19,0, + 0,0,114,21,0,0,0,114,30,0,0,0,114,40,0,0, + 0,114,41,0,0,0,114,45,0,0,0,114,46,0,0,0, + 114,48,0,0,0,114,58,0,0,0,218,4,116,121,112,101, + 218,8,95,95,99,111,100,101,95,95,114,142,0,0,0,114, + 17,0,0,0,114,132,0,0,0,114,16,0,0,0,114,20, + 0,0,0,90,17,95,82,65,87,95,77,65,71,73,67,95, + 78,85,77,66,69,82,114,77,0,0,0,114,76,0,0,0, + 114,88,0,0,0,114,78,0,0,0,90,23,68,69,66,85, + 71,95,66,89,84,69,67,79,68,69,95,83,85,70,70,73, + 88,69,83,90,27,79,80,84,73,77,73,90,69,68,95,66, + 89,84,69,67,79,68,69,95,83,85,70,70,73,88,69,83, + 114,83,0,0,0,114,89,0,0,0,114,95,0,0,0,114, + 99,0,0,0,114,101,0,0,0,114,120,0,0,0,114,127, + 0,0,0,114,139,0,0,0,114,145,0,0,0,114,148,0, + 0,0,114,153,0,0,0,218,6,111,98,106,101,99,116,114, + 160,0,0,0,114,165,0,0,0,114,166,0,0,0,114,181, + 0,0,0,114,191,0,0,0,114,207,0,0,0,114,215,0, + 0,0,114,220,0,0,0,114,226,0,0,0,114,221,0,0, + 0,114,227,0,0,0,114,246,0,0,0,114,248,0,0,0, + 114,5,1,0,0,114,23,1,0,0,114,159,0,0,0,114, + 32,1,0,0,114,34,1,0,0,114,4,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,8,60, + 109,111,100,117,108,101,62,8,0,0,0,115,108,0,0,0, + 4,16,4,1,4,1,2,1,6,3,8,17,8,5,8,5, + 8,6,8,12,8,10,8,9,8,5,8,7,10,22,10,117, + 16,1,12,2,4,1,4,2,6,2,6,2,8,2,16,45, + 8,34,8,19,8,12,8,12,8,28,8,17,10,55,10,12, + 10,10,8,14,6,3,4,1,14,67,14,64,14,29,16,110, + 14,41,18,45,18,16,4,3,18,53,14,60,14,42,14,127, + 0,5,14,127,0,22,10,23,8,11,8,68, }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:41:58 2016 From: python-checkins at python.org (r.david.murray) Date: Thu, 08 Sep 2016 01:41:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2324277=3A_Fix_3=2E4_what?= =?utf-8?q?s_new_link_broken_by_email_doc_changes=2E?= Message-ID: <20160908014157.69505.63652.7A744E66@psf.io> https://hg.python.org/cpython/rev/ad297312a478 changeset: 103285:ad297312a478 user: R David Murray date: Wed Sep 07 21:39:40 2016 -0400 summary: #24277: Fix 3.4 whats new link broken by email doc changes. files: Doc/whatsnew/3.4.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -824,7 +824,7 @@ :term:`provisional API`. These classes provide a number of new methods that make extracting content from and inserting content into email messages much easier. For details, see the :mod:`~email.contentmanager` documentation and -the :ref:`email-contentmanager-api-examples`. These API additions complete the +the :ref:`email-examples`. These API additions complete the bulk of the work that was planned as part of the email6 project. The currently provisional API is scheduled to become final in Python 3.5 (possibly with a few minor additions in the area of error handling). (Contributed by R. David -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:48:33 2016 From: python-checkins at python.org (r.david.murray) Date: Thu, 08 Sep 2016 01:48:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2324277=3A_Fix_some_incor?= =?utf-8?q?rect_backslashes_in_email_example=2E?= Message-ID: <20160908014832.48373.33598.1116CD10@psf.io> https://hg.python.org/cpython/rev/d82927c18931 changeset: 103286:d82927c18931 user: R David Murray date: Wed Sep 07 21:48:21 2016 -0400 summary: #24277: Fix some incorrect backslashes in email example. files: Doc/includes/email-alternative.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/includes/email-alternative.py b/Doc/includes/email-alternative.py --- a/Doc/includes/email-alternative.py +++ b/Doc/includes/email-alternative.py @@ -30,13 +30,13 @@ -

Salut!<\p> +

Salut!

Cela ressemble ? un excellent + """.format(asparagus_cid=asparagus_cid[1:-1]), subtype='html') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 21:50:57 2016 From: python-checkins at python.org (eric.snow) Date: Thu, 08 Sep 2016 01:50:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2315352=3A_Rebuild_?= =?utf-8?q?frozen_modules_when_marshal=2Ec_is_changed=2E?= Message-ID: <20160908015057.8701.36354.D19DCC0A@psf.io> https://hg.python.org/cpython/rev/344f44bd793f changeset: 103287:344f44bd793f user: Eric Snow date: Wed Sep 07 18:48:06 2016 -0700 summary: Issue #15352: Rebuild frozen modules when marshal.c is changed. files: Makefile.pre.in | 4 ++-- Misc/NEWS | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -702,11 +702,11 @@ Programs/_freeze_importlib: Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LINKCC) $(PY_LDFLAGS) -o $@ Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) -Python/importlib_external.h: @GENERATED_COMMENT@ $(srcdir)/Lib/importlib/_bootstrap_external.py Programs/_freeze_importlib +Python/importlib_external.h: @GENERATED_COMMENT@ $(srcdir)/Lib/importlib/_bootstrap_external.py Programs/_freeze_importlib Python/marshal.c ./Programs/_freeze_importlib \ $(srcdir)/Lib/importlib/_bootstrap_external.py Python/importlib_external.h -Python/importlib.h: @GENERATED_COMMENT@ $(srcdir)/Lib/importlib/_bootstrap.py Programs/_freeze_importlib +Python/importlib.h: @GENERATED_COMMENT@ $(srcdir)/Lib/importlib/_bootstrap.py Programs/_freeze_importlib Python/marshal.c ./Programs/_freeze_importlib \ $(srcdir)/Lib/importlib/_bootstrap.py Python/importlib.h diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -7859,6 +7859,8 @@ - Issue #17119: Fixed integer overflows when processing large strings and tuples in the tkinter module. +- Issue #15352: Rebuild frozen modules when marshal.c is changed. + - Issue #18747: Re-seed OpenSSL's pseudo-random number generator after fork. A pthread_atfork() parent handler is used to seed the PRNG with pid, time and some stack data. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Wed Sep 7 22:09:01 2016 From: python-checkins at python.org (eric.snow) Date: Thu, 08 Sep 2016 02:09:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE1NTc4?= =?utf-8?q?=3A_Correctly_incref_the_parent_module_while_importing=2E?= Message-ID: <20160908020901.21124.23987.104FD5DD@psf.io> https://hg.python.org/cpython/rev/2d6dd8402d77 changeset: 103288:2d6dd8402d77 branch: 2.7 parent: 103278:26397c1ea557 user: Eric Snow date: Wed Sep 07 19:08:02 2016 -0700 summary: Issue #15578: Correctly incref the parent module while importing. files: Lib/test/test_import.py | 18 ++++++++++++++++++ Misc/NEWS | 2 ++ Python/import.c | 2 ++ 3 files changed, 22 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py --- a/Lib/test/test_import.py +++ b/Lib/test/test_import.py @@ -397,6 +397,24 @@ finally: sys.path.pop(0) + def test_replace_parent_in_sys_modules(self): + dir_name = os.path.abspath(TESTFN) + os.mkdir(dir_name) + try: + pkg_dir = os.path.join(dir_name, 'sa') + os.mkdir(pkg_dir) + with open(os.path.join(pkg_dir, '__init__.py'), 'w') as init_file: + init_file.write("import v1") + with open(os.path.join(pkg_dir, 'v1.py'), 'w') as v1_file: + v1_file.write("import sys;" + "sys.modules['sa'] = sys.modules[__name__];" + "import sa") + sys.path.insert(0, dir_name) + # a segfault means the test failed! + import sa + finally: + rmtree(dir_name) + class PycRewritingTests(unittest.TestCase): # Test that the `co_filename` attribute on code objects always points diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -7082,6 +7082,8 @@ - Add Py3k warnings for parameter names in parentheses. +- Issue #15578: Correctly incref the parent module while importing. + - Issue #7362: Give a proper error message for ``def f((x)=3): pass``. - Issue #7085: Fix crash when importing some extensions in a thread on MacOSX diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -2243,8 +2243,10 @@ if (parent == NULL) goto error_exit; + Py_INCREF(parent); head = load_next(parent, level < 0 ? Py_None : parent, &name, buf, &buflen); + Py_DECREF(parent); if (head == NULL) goto error_exit; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 01:26:23 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 05:26:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_placement_?= =?utf-8?q?of_Misc/NEWS_item_for_issue_=2315578=2E?= Message-ID: <20160908052623.1702.63272.DA9170CD@psf.io> https://hg.python.org/cpython/rev/731e5617cc8d changeset: 103289:731e5617cc8d branch: 2.7 user: Gregory P. Smith date: Wed Sep 07 22:26:08 2016 -0700 summary: Fix placement of Misc/NEWS item for issue #15578. files: Misc/NEWS | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #15578: Correctly incref the parent module while importing. + - Issue #26307: The profile-opt build now applys PGO to the built-in modules. - Issue #27870: A left shift of zero by a large integer no longer attempts @@ -7082,8 +7084,6 @@ - Add Py3k warnings for parameter names in parentheses. -- Issue #15578: Correctly incref the parent module while importing. - - Issue #7362: Give a proper error message for ``def f((x)=3): pass``. - Issue #7085: Fix crash when importing some extensions in a thread on MacOSX -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 01:42:49 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 08 Sep 2016 05:42:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3OTkz?= =?utf-8?q?=3A_Fix_problems_with_the_plural_=E2=80=9Cobjects=E2=80=9D_in_d?= =?utf-8?q?ocs_and_comments?= Message-ID: <20160908054249.1897.89505.EAD76C1A@psf.io> https://hg.python.org/cpython/rev/c07aadf9f5cb changeset: 103293:c07aadf9f5cb branch: 2.7 user: Martin Panter date: Thu Sep 08 05:39:59 2016 +0000 summary: Issue #27993: Fix problems with the plural ?objects? in docs and comments files: Doc/library/argparse.rst | 4 ++-- Doc/library/weakref.rst | 2 +- Doc/library/xml.dom.rst | 2 +- Lib/inspect.py | 5 ++--- Lib/json/tests/test_decode.py | 2 +- Misc/HISTORY | 2 +- Misc/NEWS | 2 +- 7 files changed, 9 insertions(+), 10 deletions(-) diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -181,7 +181,7 @@ prog ^^^^ -By default, :class:`ArgumentParser` objects uses ``sys.argv[0]`` to determine +By default, :class:`ArgumentParser` objects use ``sys.argv[0]`` to determine how to display the name of the program in help messages. This default is almost always desirable because it will make the help messages match how the program was invoked on the command line. For example, consider a file named @@ -512,7 +512,7 @@ ^^^^^^^^^^^^^^^^ :class:`ArgumentParser` objects do not allow two actions with the same option -string. By default, :class:`ArgumentParser` objects raises an exception if an +string. By default, :class:`ArgumentParser` objects raise an exception if an attempt is made to create an argument with an option string that is already in use:: diff --git a/Doc/library/weakref.rst b/Doc/library/weakref.rst --- a/Doc/library/weakref.rst +++ b/Doc/library/weakref.rst @@ -328,7 +328,7 @@ Example ------- -This simple example shows how an application can use objects IDs to retrieve +This simple example shows how an application can use object IDs to retrieve objects that it has seen before. The IDs of the objects can then be used in other data structures without forcing the objects to remain alive, but the objects can still be retrieved by ID if they do. diff --git a/Doc/library/xml.dom.rst b/Doc/library/xml.dom.rst --- a/Doc/library/xml.dom.rst +++ b/Doc/library/xml.dom.rst @@ -419,7 +419,7 @@ ^^^^^^^^^^^^^^^^ A :class:`NodeList` represents a sequence of nodes. These objects are used in -two ways in the DOM Core recommendation: the :class:`Element` objects provides +two ways in the DOM Core recommendation: an :class:`Element` object provides one as its list of child nodes, and the :meth:`getElementsByTagName` and :meth:`getElementsByTagNameNS` methods of :class:`Node` return objects with this interface to represent query results. diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -155,9 +155,8 @@ def isgeneratorfunction(object): """Return true if the object is a user-defined generator function. - Generator function objects provides same attributes as functions. - - See help(isfunction) for attributes listing.""" + Generator function objects provide the same attributes as functions. + See help(isfunction) for a list of attributes.""" return bool((isfunction(object) or ismethod(object)) and object.func_code.co_flags & CO_GENERATOR) diff --git a/Lib/json/tests/test_decode.py b/Lib/json/tests/test_decode.py --- a/Lib/json/tests/test_decode.py +++ b/Lib/json/tests/test_decode.py @@ -43,7 +43,7 @@ self.assertEqual(self.loads(s, object_pairs_hook=OrderedDict, object_hook=lambda x: None), OrderedDict(p)) - # check that empty objects literals work (see #17368) + # check that empty object literals work (see #17368) self.assertEqual(self.loads('{}', object_pairs_hook=OrderedDict), OrderedDict()) self.assertEqual(self.loads('{"empty": {}}', diff --git a/Misc/HISTORY b/Misc/HISTORY --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -15145,7 +15145,7 @@ module implementing deepcopy and normal (shallow) copy operations. See the library reference manual. -- Documentation strings for many objects types are accessible through +- Documentation strings for many object types are accessible through the __doc__ attribute. Modules, classes and functions support special syntax to initialize the __doc__ attribute: if the first statement consists of just a string literal, that string literal becomes the diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -1865,7 +1865,7 @@ - Issue #21350: Fix file.writelines() to accept arbitrary buffer objects, as advertised. Patch by Brian Kearns. -- Issue #20437: Fixed 43 potential bugs when deleting objects references. +- Issue #20437: Fixed 43 potential bugs when deleting object references. - Issue #21134: Fix segfault when str is called on an uninitialized UnicodeEncodeError, UnicodeDecodeError, or UnicodeTranslateError object. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 01:42:49 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 08 Sep 2016 05:42:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Correct_occura?= =?utf-8?q?nce_=E2=86=92_occurrence=3B_extracted_from_patch_by_Georg_Brand?= =?utf-8?q?l?= Message-ID: <20160908054249.87545.20844.D514450B@psf.io> https://hg.python.org/cpython/rev/4f1f3d7cf8cb changeset: 103292:4f1f3d7cf8cb branch: 2.7 user: Martin Panter date: Thu Sep 08 05:22:16 2016 +0000 summary: Correct occurance ? occurrence; extracted from patch by Georg Brandl files: Objects/bytearrayobject.c | 4 ++-- Objects/stringobject.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -1990,7 +1990,7 @@ } if (to_len == 0) { - /* delete all occurances of 'from' bytes */ + /* delete all occurrences of 'from' bytes */ if (from_len == 1) { return replace_delete_single_character( self, from_s[0], maxcount); @@ -2399,7 +2399,7 @@ PyDoc_STRVAR(remove__doc__, "B.remove(int) -> None\n\ \n\ -Remove the first occurance of a value in B."); +Remove the first occurrence of a value in B."); static PyObject * bytearray_remove(PyByteArrayObject *self, PyObject *arg) { diff --git a/Objects/stringobject.c b/Objects/stringobject.c --- a/Objects/stringobject.c +++ b/Objects/stringobject.c @@ -2775,7 +2775,7 @@ } if (to_len == 0) { - /* delete all occurances of 'from' string */ + /* delete all occurrences of 'from' string */ if (from_len == 1) { return replace_delete_single_character( self, from_s[0], maxcount); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 01:42:49 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 08 Sep 2016 05:42:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NTcw?= =?utf-8?q?=3A_Avoid_zero-length_memcpy=28=29_calls_with_null_source_point?= =?utf-8?q?ers?= Message-ID: <20160908054248.48696.19261.C248893F@psf.io> https://hg.python.org/cpython/rev/d465da1e5902 changeset: 103290:d465da1e5902 branch: 2.7 user: Martin Panter date: Wed Sep 07 11:04:41 2016 +0000 summary: Issue #27570: Avoid zero-length memcpy() calls with null source pointers files: Lib/test/test_array.py | 16 +++++++++++++--- Misc/NEWS | 3 +++ Modules/_ctypes/_ctypes.c | 6 ++++-- Modules/_ctypes/stgdict.c | 8 +++++--- Modules/arraymodule.c | 24 +++++++++++++++++------- 5 files changed, 42 insertions(+), 15 deletions(-) diff --git a/Lib/test/test_array.py b/Lib/test/test_array.py --- a/Lib/test/test_array.py +++ b/Lib/test/test_array.py @@ -22,9 +22,9 @@ if test_support.have_unicode: typecodes += "u" -class BadConstructorTest(unittest.TestCase): +class MiscTest(unittest.TestCase): - def test_constructor(self): + def test_bad_constructor(self): self.assertRaises(TypeError, array.array) self.assertRaises(TypeError, array.array, spam=42) self.assertRaises(TypeError, array.array, 'xx') @@ -40,7 +40,17 @@ self.assertRaises(ValueError, array.array, u'x') self.assertRaises(ValueError, array.array, u'\x80') -tests.append(BadConstructorTest) + def test_empty(self): + # Exercise code for handling zero-length arrays + a = array.array('B') + a[:] = a + self.assertEqual(len(a), 0) + self.assertEqual(len(a + a), 0) + self.assertEqual(len(a * 3), 0) + a += a + self.assertEqual(len(a), 0) + +tests.append(MiscTest) class BaseTest(unittest.TestCase): # Required class attributes (provided by subclasses diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -40,6 +40,9 @@ Library ------- +- Issue #27570: Avoid zero-length memcpy() etc calls with null source + pointers in the "ctypes" and "array" modules. + - lib2to3.pgen3.driver.load_grammar() now creates a stable cache file between runs given the same Grammar.txt input regardless of the hash randomization setting. diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -1426,8 +1426,10 @@ return NULL; } stgdict->shape[0] = length; - memmove(&stgdict->shape[1], itemdict->shape, - sizeof(Py_ssize_t) * (stgdict->ndim - 1)); + if (stgdict->ndim > 1) { + memmove(&stgdict->shape[1], itemdict->shape, + sizeof(Py_ssize_t) * (stgdict->ndim - 1)); + } itemsize = itemdict->size; if (length * itemsize < 0) { diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c --- a/Modules/_ctypes/stgdict.c +++ b/Modules/_ctypes/stgdict.c @@ -395,9 +395,11 @@ } memset(stgdict->ffi_type_pointer.elements, 0, sizeof(ffi_type *) * (basedict->length + len + 1)); - memcpy(stgdict->ffi_type_pointer.elements, - basedict->ffi_type_pointer.elements, - sizeof(ffi_type *) * (basedict->length)); + if (basedict->length > 0) { + memcpy(stgdict->ffi_type_pointer.elements, + basedict->ffi_type_pointer.elements, + sizeof(ffi_type *) * (basedict->length)); + } ffi_ofs = basedict->length; } else { offset = 0; diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -620,8 +620,10 @@ np = (arrayobject *) newarrayobject(&Arraytype, ihigh - ilow, a->ob_descr); if (np == NULL) return NULL; - memcpy(np->ob_item, a->ob_item + ilow * a->ob_descr->itemsize, - (ihigh-ilow) * a->ob_descr->itemsize); + if (ihigh > ilow) { + memcpy(np->ob_item, a->ob_item + ilow * a->ob_descr->itemsize, + (ihigh-ilow) * a->ob_descr->itemsize); + } return (PyObject *)np; } @@ -660,9 +662,13 @@ if (np == NULL) { return NULL; } - memcpy(np->ob_item, a->ob_item, Py_SIZE(a)*a->ob_descr->itemsize); - memcpy(np->ob_item + Py_SIZE(a)*a->ob_descr->itemsize, - b->ob_item, Py_SIZE(b)*b->ob_descr->itemsize); + if (Py_SIZE(a) > 0) { + memcpy(np->ob_item, a->ob_item, Py_SIZE(a)*a->ob_descr->itemsize); + } + if (Py_SIZE(b) > 0) { + memcpy(np->ob_item + Py_SIZE(a)*a->ob_descr->itemsize, + b->ob_item, Py_SIZE(b)*b->ob_descr->itemsize); + } return (PyObject *)np; #undef b } @@ -684,6 +690,8 @@ np = (arrayobject *) newarrayobject(&Arraytype, size, a->ob_descr); if (np == NULL) return NULL; + if (size == 0) + return (PyObject *)np; p = np->ob_item; nbytes = Py_SIZE(a) * a->ob_descr->itemsize; for (i = 0; i < n; i++) { @@ -838,8 +846,10 @@ PyErr_NoMemory(); return -1; } - memcpy(self->ob_item + Py_SIZE(self)*self->ob_descr->itemsize, - b->ob_item, Py_SIZE(b)*b->ob_descr->itemsize); + if (Py_SIZE(b) > 0) { + memcpy(self->ob_item + Py_SIZE(self)*self->ob_descr->itemsize, + b->ob_item, Py_SIZE(b)*b->ob_descr->itemsize); + } Py_SIZE(self) = size; self->allocated = size; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 01:42:49 2016 From: python-checkins at python.org (martin.panter) Date: Thu, 08 Sep 2016 05:42:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3ODk1?= =?utf-8?q?=3A__Spelling_fixes_=28Contributed_by_Ville_Skytt=C3=A4=29=2E?= Message-ID: <20160908054249.19423.5222.E6F8A88A@psf.io> https://hg.python.org/cpython/rev/9a68ab141c33 changeset: 103291:9a68ab141c33 branch: 2.7 user: Martin Panter date: Wed Sep 07 12:03:06 2016 +0000 summary: Issue #27895: Spelling fixes (Contributed by Ville Skytt?). files: Include/pymath.h | 2 +- Lib/distutils/tests/test_msvc9compiler.py | 2 +- Lib/doctest.py | 2 +- Lib/email/test/test_email.py | 4 +- Lib/httplib.py | 2 +- Lib/idlelib/idle_test/test_formatparagraph.py | 2 +- Lib/test/test_datetime.py | 2 +- Lib/test/test_descr.py | 4 +- Lib/test/test_pep247.py | 2 +- Lib/test/test_py3kwarn.py | 2 +- Lib/test/test_urllib.py | 2 +- Lib/test/test_urllib2_localnet.py | 2 +- Lib/test/test_winreg.py | 2 +- Mac/PythonLauncher/MyAppDelegate.m | 2 +- Misc/HISTORY | 14 +++++----- Misc/NEWS | 6 ++-- Modules/_ctypes/ctypes.h | 2 +- Modules/_hashopenssl.c | 2 +- Modules/_io/iobase.c | 2 +- Modules/zipimport.c | 2 +- Objects/codeobject.c | 2 +- Objects/listsort.txt | 2 +- Objects/longobject.c | 2 +- Objects/stringlib/formatter.h | 2 +- README | 2 +- 25 files changed, 35 insertions(+), 35 deletions(-) diff --git a/Include/pymath.h b/Include/pymath.h --- a/Include/pymath.h +++ b/Include/pymath.h @@ -43,7 +43,7 @@ extern double copysign(double, double); #endif -/* High precision defintion of pi and e (Euler) +/* High precision definition of pi and e (Euler) * The values are taken from libc6's math.h. */ #ifndef Py_MATH_PIl diff --git a/Lib/distutils/tests/test_msvc9compiler.py b/Lib/distutils/tests/test_msvc9compiler.py --- a/Lib/distutils/tests/test_msvc9compiler.py +++ b/Lib/distutils/tests/test_msvc9compiler.py @@ -125,7 +125,7 @@ self.assertRaises(KeyError, Reg.get_value, 'xxx', 'xxx') # looking for values that should exist on all - # windows registeries versions. + # windows registry versions. path = r'Control Panel\Desktop' v = Reg.get_value(path, u'dragfullwindows') self.assertIn(v, (u'0', u'1', u'2')) diff --git a/Lib/doctest.py b/Lib/doctest.py --- a/Lib/doctest.py +++ b/Lib/doctest.py @@ -219,7 +219,7 @@ with open(filename, 'U') as f: return f.read(), filename -# Use sys.stdout encoding for ouput. +# Use sys.stdout encoding for output. _encoding = getattr(sys.__stdout__, 'encoding', None) or 'utf-8' def _indent(s, indent=4): diff --git a/Lib/email/test/test_email.py b/Lib/email/test/test_email.py --- a/Lib/email/test/test_email.py +++ b/Lib/email/test/test_email.py @@ -561,12 +561,12 @@ # Issue 5871: reject an attempt to embed a header inside a header value # (header injection attack). - def test_embeded_header_via_Header_rejected(self): + def test_embedded_header_via_Header_rejected(self): msg = Message() msg['Dummy'] = Header('dummy\nX-Injected-Header: test') self.assertRaises(Errors.HeaderParseError, msg.as_string) - def test_embeded_header_via_string_rejected(self): + def test_embedded_header_via_string_rejected(self): msg = Message() msg['Dummy'] = 'dummy\nX-Injected-Header: test' self.assertRaises(Errors.HeaderParseError, msg.as_string) diff --git a/Lib/httplib.py b/Lib/httplib.py --- a/Lib/httplib.py +++ b/Lib/httplib.py @@ -242,7 +242,7 @@ # # VCHAR defined in http://tools.ietf.org/html/rfc5234#appendix-B.1 -# the patterns for both name and value are more leniant than RFC +# the patterns for both name and value are more lenient than RFC # definitions to allow for backwards compatibility _is_legal_header_name = re.compile(r'\A[^:\s][^:\r\n]*\Z').match _is_illegal_header_value = re.compile(r'\n(?![ \t])|\r(?![ \t\n])').search diff --git a/Lib/idlelib/idle_test/test_formatparagraph.py b/Lib/idlelib/idle_test/test_formatparagraph.py --- a/Lib/idlelib/idle_test/test_formatparagraph.py +++ b/Lib/idlelib/idle_test/test_formatparagraph.py @@ -159,7 +159,7 @@ class ReformatFunctionTest(unittest.TestCase): """Test the reformat_paragraph function without the editor window.""" - def test_reformat_paragrah(self): + def test_reformat_paragraph(self): Equal = self.assertEqual reform = fp.reformat_paragraph hw = "O hello world" diff --git a/Lib/test/test_datetime.py b/Lib/test/test_datetime.py --- a/Lib/test/test_datetime.py +++ b/Lib/test/test_datetime.py @@ -3343,7 +3343,7 @@ self.assertRaises(TypeError, lambda: as_date >= as_datetime) self.assertRaises(TypeError, lambda: as_datetime >= as_date) - # Neverthelss, comparison should work with the base-class (date) + # Nevertheless, comparison should work with the base-class (date) # projection if use of a date method is forced. self.assertTrue(as_date.__eq__(as_datetime)) different_day = (as_date.day + 1) % 20 + 1 diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -684,7 +684,7 @@ self.fail("inheriting from ModuleType and str at the same time " "should fail") - def test_multiple_inheritence(self): + def test_multiple_inheritance(self): # Testing multiple inheritance... class C(object): def __init__(self): @@ -815,7 +815,7 @@ else: self.fail("new class with only classic bases - shouldn't be") - def test_diamond_inheritence(self): + def test_diamond_inheritance(self): # Testing multiple inheritance special cases... class A(object): def spam(self): return "A" diff --git a/Lib/test/test_pep247.py b/Lib/test/test_pep247.py --- a/Lib/test/test_pep247.py +++ b/Lib/test/test_pep247.py @@ -1,5 +1,5 @@ """ -Test suite to check compilance with PEP 247, the standard API +Test suite to check compliance with PEP 247, the standard API for hashing algorithms """ diff --git a/Lib/test/test_py3kwarn.py b/Lib/test/test_py3kwarn.py --- a/Lib/test/test_py3kwarn.py +++ b/Lib/test/test_py3kwarn.py @@ -270,7 +270,7 @@ class NoWarningOnlyHash(object): def __hash__(self): pass self.assertEqual(len(w.warnings), 0) - # With an intermediate class in the heirarchy + # With an intermediate class in the hierarchy class DefinesAllThree(object): def __cmp__(self, other): pass def __eq__(self, other): pass diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py --- a/Lib/test/test_urllib.py +++ b/Lib/test/test_urllib.py @@ -1,4 +1,4 @@ -"""Regresssion tests for urllib""" +"""Regression tests for urllib""" import collections import urllib diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py --- a/Lib/test/test_urllib2_localnet.py +++ b/Lib/test/test_urllib2_localnet.py @@ -93,7 +93,7 @@ BaseHTTPServer.BaseHTTPRequestHandler.__init__(self, *args, **kwargs) def log_message(self, format, *args): - # Supress the HTTP Console log output + # Suppress the HTTP Console log output pass def do_HEAD(self): diff --git a/Lib/test/test_winreg.py b/Lib/test/test_winreg.py --- a/Lib/test/test_winreg.py +++ b/Lib/test/test_winreg.py @@ -174,7 +174,7 @@ DeleteKey(key, "sub_key") try: - # Shouldnt be able to delete it twice! + # Shouldn't be able to delete it twice! DeleteKey(key, "sub_key") self.fail("Deleting the key twice succeeded") except EnvironmentError: diff --git a/Mac/PythonLauncher/MyAppDelegate.m b/Mac/PythonLauncher/MyAppDelegate.m --- a/Mac/PythonLauncher/MyAppDelegate.m +++ b/Mac/PythonLauncher/MyAppDelegate.m @@ -34,7 +34,7 @@ - (BOOL)shouldShowUI { // if this call comes before applicationDidFinishLaunching: we - // should terminate immedeately after starting the script. + // should terminate immediately after starting the script. if (!initial_action_done) should_terminate = YES; initial_action_done = YES; diff --git a/Misc/HISTORY b/Misc/HISTORY --- a/Misc/HISTORY +++ b/Misc/HISTORY @@ -1995,7 +1995,7 @@ - Bug #947906: An object oriented interface has been added to the calendar module. It's possible to generate HTML calendar now and the module can be called as a script (e.g. via ``python -mcalendar``). Localized month and - weekday names can be ouput (even if an exotic encoding is used) using + weekday names can be output (even if an exotic encoding is used) using special classes that use unicode. Build @@ -2378,7 +2378,7 @@ ``True`` for ``!=``, and raises ``TypeError`` for other comparison operators. Because datetime is a subclass of date, comparing only the base class (date) members can still be done, if that's desired, by - forcing using of the approprate date method; e.g., + forcing using of the appropriate date method; e.g., ``a_date.__eq__(a_datetime)`` is true if and only if the year, month and day members of ``a_date`` and ``a_datetime`` are equal. @@ -12858,7 +12858,7 @@ - copy.py: Make sure the objects returned by __getinitargs__() are kept alive (in the memo) to avoid a certain kind of nasty crash. (Not -easily reproducable because it requires a later call to +easily reproducible because it requires a later call to __getinitargs__() to return a tuple that happens to be allocated at the same address.) @@ -16490,7 +16490,7 @@ There is now a script to patch Makefile and config.c to add a new optional built-in module: Addmodule.sh. Read the script before using! -Useing Addmodule.sh, all optional modules can now be configured at +Using Addmodule.sh, all optional modules can now be configured at compile time using Configure.py, so there are no modules left that require dynamic loading. @@ -16921,9 +16921,9 @@ SUNAUDIODEV: symbolic constant definitions for sunaudiodef (sun only) -SV: symbolic constat definitions for sv (sgi only) - -CD: symbolic constat definitions for cd (sgi only) +SV: symbolic constant definitions for sv (sgi only) + +CD: symbolic constant definitions for cd (sgi only) New demos diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -2516,7 +2516,7 @@ - Issue #18709: Fix CVE-2013-4238. The SSL module now handles NULL bytes inside subjectAltName correctly. Formerly the module has used OpenSSL's - GENERAL_NAME_print() function to get the string represention of ASN.1 + GENERAL_NAME_print() function to get the string representation of ASN.1 strings for ``rfc822Name`` (email), ``dNSName`` (DNS) and ``uniformResourceIdentifier`` (URI). @@ -3147,7 +3147,7 @@ using the same sys.flags as the current process. Initial patch by Sergey Mezentsev. -- Issue #8862: Fixed curses cleanup when getkey is interrputed by a signal. +- Issue #8862: Fixed curses cleanup when getkey is interrupted by a signal. - Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or EAGAIN, retry the select() loop instead of bailing out. This is because select() @@ -3571,7 +3571,7 @@ - Issue #14829: Fix bisect issues under 64-bit Windows. - Issue #14777: tkinter may return undecoded UTF-8 bytes as a string when - accessing the Tk clipboard. Modify clipboad_get() to first request type + accessing the Tk clipboard. Modify clipboard_get() to first request type UTF8_STRING when no specific type is requested in an X11 windowing environment, falling back to the current default type STRING if that fails. Original patch by Thomas Kluyver. diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -276,7 +276,7 @@ StgDictObject function to a generic one. Currently, PyCFuncPtr types have 'converters' and 'checker' entries in their - type dict. They are only used to cache attributes from other entries, whihc + type dict. They are only used to cache attributes from other entries, which is wrong. One use case is the .value attribute that all simple types have. But some diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -820,7 +820,7 @@ /* * This macro generates constructor function definitions for specific * hash algorithms. These constructors are much faster than calling - * the generic one passing it a python string and are noticably + * the generic one passing it a python string and are noticeably * faster than calling a python new() wrapper. Thats important for * code that wants to make hashes of a bunch of small strings. */ diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -71,7 +71,7 @@ return NULL; } -/* Positionning */ +/* Positioning */ PyDoc_STRVAR(iobase_seek_doc, "Change stream position.\n" diff --git a/Modules/zipimport.c b/Modules/zipimport.c --- a/Modules/zipimport.c +++ b/Modules/zipimport.c @@ -1096,7 +1096,7 @@ return code; } -/* Replace any occurances of "\r\n?" in the input string with "\n". +/* Replace any occurrences of "\r\n?" in the input string with "\n". This converts DOS and Mac line endings to Unix line endings. Also append a trailing "\n" to be compatible with PyParser_SimpleParseFile(). Returns a new reference. */ diff --git a/Objects/codeobject.c b/Objects/codeobject.c --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -690,7 +690,7 @@ /* possible optimization: if f->f_lasti == instr_ub (likely to be a common case) then we already know instr_lb -- if we stored the matching value of p - somwhere we could skip the first while loop. */ + somewhere we could skip the first while loop. */ /* See lnotab_notes.txt for the description of co_lnotab. A point to remember: increments to p diff --git a/Objects/listsort.txt b/Objects/listsort.txt --- a/Objects/listsort.txt +++ b/Objects/listsort.txt @@ -694,7 +694,7 @@ But in CPython's case, comparisons are extraordinarily expensive compared to moving data, and the details matter. Moving objects is just copying -pointers. Comparisons can be arbitrarily expensive (can invoke arbitary +pointers. Comparisons can be arbitrarily expensive (can invoke arbitrary user-supplied Python code), but even in simple cases (like 3 < 4) _all_ decisions are made at runtime: what's the type of the left comparand? the type of the right? do they need to be coerced to a common type? where's the diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -211,7 +211,7 @@ /* Checking for overflow in PyLong_AsLong is a PITA since C doesn't define * anything about what happens when a signed integer operation overflows, * and some compilers think they're doing you a favor by being "clever" - * then. The bit pattern for the largest postive signed long is + * then. The bit pattern for the largest positive signed long is * (unsigned long)LONG_MAX, and for the smallest negative signed long * it is abs(LONG_MIN), which we could write -(unsigned long)LONG_MIN. * However, some other compilers warn about applying unary minus to an diff --git a/Objects/stringlib/formatter.h b/Objects/stringlib/formatter.h --- a/Objects/stringlib/formatter.h +++ b/Objects/stringlib/formatter.h @@ -141,7 +141,7 @@ #if 0 -/* Occassionally useful for debugging. Should normally be commented out. */ +/* Occasionally useful for debugging. Should normally be commented out. */ static void DEBUG_PRINT_FORMAT_SPEC(InternalFormatSpec *format) { diff --git a/README b/README --- a/README +++ b/README @@ -207,7 +207,7 @@ After this instrumented version of the interpreter is built, the Makefile will automatically run a training workload. This is necessary in order to profile the interpreter execution. Note also that any output, both stdout -and stderr, that may appear at this step is supressed. +and stderr, that may appear at this step is suppressed. Finally, the last step is to rebuild the interpreter, using the information collected in the previous one. The end result will be a Python binary -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 02:43:35 2016 From: python-checkins at python.org (ethan.furman) Date: Thu, 08 Sep 2016 06:43:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_improve_Enum_docs?= Message-ID: <20160908064050.36352.82480.80565853@psf.io> https://hg.python.org/cpython/rev/86d72a1303d0 changeset: 103294:86d72a1303d0 parent: 103287:344f44bd793f user: Ethan Furman date: Wed Sep 07 23:40:31 2016 -0700 summary: improve Enum docs files: Doc/library/enum.rst | 23 ++++++++++++----------- 1 files changed, 12 insertions(+), 11 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -542,9 +542,9 @@ on :class:`int`. The difference being :class:`IntFlag` members can be combined using the bitwise operators (&, \|, ^, ~) and the result is still an :class:`IntFlag` member. However, as the name implies, :class:`IntFlag` -members also subclass :class:`int` and can be used wherever an :class:`int` is. -Any operation on an :class:`IntFlag` member besides the bit-wise operations -will lose the :class:`IntFlag` membership. +members also subclass :class:`int` and can be used wherever an :class:`int` is +used. Any operation on an :class:`IntFlag` member besides the bit-wise +operations will lose the :class:`IntFlag` membership. .. versionadded:: 3.6 @@ -955,10 +955,11 @@ ``Enum`` member type """""""""""""""""""" -:class:`Enum` members are instances of an :class:`Enum` class, and even -though they are accessible as `EnumClass.member`, they should not be accessed -directly from the member as that lookup may fail or, worse, return something -besides the ``Enum`` member you looking for:: +:class:`Enum` members are instances of their :class:`Enum` class, and are +normally accessed as ``EnumClass.member``. Under certain circumstances they +can also be accessed as ``EnumClass.member.member``, but you should never do +this as that lookup may fail or, worse, return something besides the +:class:`Enum` member you are looking for:: >>> class FieldTypes(Enum): ... name = 0 @@ -976,16 +977,16 @@ Boolean value of ``Enum`` classes and members """"""""""""""""""""""""""""""""""""""""""""" -``Enum`` members that are mixed with non-Enum types (such as +:class:`Enum` members that are mixed with non-:class:`Enum` types (such as :class:`int`, :class:`str`, etc.) are evaluated according to the mixed-in -type's rules; otherwise, all members evaluate as :data:`True`. To make your own -Enum's boolean evaluation depend on the member's value add the following to +type's rules; otherwise, all members evaluate as :data:`True`. To make your +own Enum's boolean evaluation depend on the member's value add the following to your class:: def __bool__(self): return bool(self.value) -``Enum`` classes always evaluate as :data:`True`. +:class:`Enum` classes always evaluate as :data:`True`. ``Enum`` classes with methods -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 03:10:06 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 07:10:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2MzU5?= =?utf-8?q?=3A_Add_the_--with-optimizations_flag=2E?= Message-ID: <20160908070833.8758.75312.A52F3319@psf.io> https://hg.python.org/cpython/rev/bc28cbd49070 changeset: 103295:bc28cbd49070 branch: 2.7 parent: 103293:c07aadf9f5cb user: Gregory P. Smith date: Wed Sep 07 23:28:23 2016 -0700 summary: Issue #26359: Add the --with-optimizations flag. files: Makefile.pre.in | 8 ++-- Misc/NEWS | 2 + configure | 55 +++++++++++++++++++++++++++++++++++++ configure.ac | 41 +++++++++++++++++++++++++++ 4 files changed, 102 insertions(+), 4 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -428,7 +428,7 @@ # Rules # Default target -all: build_all +all: @DEF_MAKE_ALL_RULE@ build_all: $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks # Compile a binary with profile guided optimization. @@ -452,7 +452,7 @@ $(MAKE) profile-removal build_all_generate_profile: - $(MAKE) all CFLAGS="$(CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" + $(MAKE) @DEF_MAKE_RULE@ CFLAGS="$(CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)" run_profile_task: : # FIXME: can't run for a cross build @@ -462,14 +462,14 @@ $(LLVM_PROF_MERGER) build_all_use_profile: - $(MAKE) all CFLAGS="$(CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@" + $(MAKE) @DEF_MAKE_RULE@ CFLAGS="$(CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@" # Compile and run with gcov .PHONY=coverage coverage-lcov coverage-report coverage: @echo "Building with support for coverage checking:" $(MAKE) clean profile-removal - $(MAKE) all CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov" + $(MAKE) @DEF_MAKE_RULE@ CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov" coverage-lcov: @echo "Creating Coverage HTML report with LCOV:" diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -156,6 +156,8 @@ Build ----- +- Issue #26359: Add the --with-optimizations configure flag. + - Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. Also update FreedBSD version checks for the original ctype UTF-8 workaround. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -668,6 +668,8 @@ PGO_PROF_USE_FLAG PGO_PROF_GEN_FLAG LTOFLAGS +DEF_MAKE_RULE +DEF_MAKE_ALL_RULE UNIVERSAL_ARCH_FLAGS BASECFLAGS OPT @@ -798,6 +800,7 @@ enable_shared enable_profiling with_pydebug +with_optimizations with_lto enable_toolbox_glue with_libs @@ -1491,6 +1494,8 @@ compiler --with-suffix=.exe set executable suffix --with-pydebug build with Py_DEBUG defined + --with-optimizations Enable all optimizations when available (LTO, PGO, + etc). Disabled by default. --with-lto Enable Link Time Optimization in PGO builds. Disabled by default. --with-libs='lib1 ...' link against additional libs @@ -6386,6 +6391,47 @@ fi +# Enable optimization flags + + +Py_OPT='false' +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-optimizations" >&5 +$as_echo_n "checking for --with-optimizations... " >&6; } + +# Check whether --with-optimizations was given. +if test "${with_optimizations+set}" = set; then : + withval=$with_optimizations; +if test "$withval" != no +then + Py_OPT='true' + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; }; +else + Py_OPT='false' + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; }; +fi +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + +if test "$Py_OPT" = 'true' ; then + case $ac_sys_system in + Darwin*) + # At least on macOS El Capitan, LTO does not work with PGO. + Py_LTO='false' + ;; + esac + Py_LTO='true' + DEF_MAKE_ALL_RULE="profile-opt" + DEF_MAKE_RULE="build_all" +else + DEF_MAKE_ALL_RULE="build_all" + DEF_MAKE_RULE="all" +fi + + # Enable LTO flags { $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-lto" >&5 @@ -16320,3 +16366,12 @@ esac mv config.c Modules + +if test "$Py_OPT" = 'false' -a "$Py_DEBUG" != 'true'; then + echo "" >&6 + echo "" >&6 + echo "If you want a release build with all optimizations active (LTO, PGO, etc)," + echo "please run ./configure --with-optimizations" >&6 + echo "" >&6 + echo "" >&6 +fi diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1380,6 +1380,38 @@ fi +# Enable optimization flags +AC_SUBST(DEF_MAKE_ALL_RULE) +AC_SUBST(DEF_MAKE_RULE) +Py_OPT='false' +AC_MSG_CHECKING(for --with-optimizations) +AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable all optimizations when available (LTO, PGO, etc). Disabled by default.]), +[ +if test "$withval" != no +then + Py_OPT='true' + AC_MSG_RESULT(yes); +else + Py_OPT='false' + AC_MSG_RESULT(no); +fi], +[AC_MSG_RESULT(no)]) +if test "$Py_OPT" = 'true' ; then + case $ac_sys_system in + Darwin*) + # At least on macOS El Capitan, LTO does not work with PGO. + Py_LTO='false' + ;; + esac + Py_LTO='true' + DEF_MAKE_ALL_RULE="profile-opt" + DEF_MAKE_RULE="build_all" +else + DEF_MAKE_ALL_RULE="build_all" + DEF_MAKE_RULE="all" +fi + + # Enable LTO flags AC_SUBST(LTOFLAGS) AC_MSG_CHECKING(for --with-lto) @@ -4775,3 +4807,12 @@ esac mv config.c Modules + +if test "$Py_OPT" = 'false' -a "$Py_DEBUG" != 'true'; then + echo "" >&AS_MESSAGE_FD + echo "" >&AS_MESSAGE_FD + echo "If you want a release build with all optimizations active (LTO, PGO, etc)," + echo "please run ./configure --with-optimizations" >&AS_MESSAGE_FD + echo "" >&AS_MESSAGE_FD + echo "" >&AS_MESSAGE_FD +fi -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 03:11:07 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 07:11:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogRml4ZXMgaXNzdWUj?= =?utf-8?q?_27983=3A_Cause_lack_of_llvm-profdata_tool_when_using_clang_-?= Message-ID: <20160908070844.87325.67294.5B1D2440@psf.io> https://hg.python.org/cpython/rev/3f04287e7bea changeset: 103296:3f04287e7bea branch: 2.7 user: Gregory P. Smith date: Thu Sep 08 00:07:40 2016 -0700 summary: Fixes issue# 27983: Cause lack of llvm-profdata tool when using clang - required for PGO linking - to be a configure time error rather than make time when --with-optimizations is enabled. Also improve our ability to find the llvm-profdata tool on MacOS and some Linuxes. files: Misc/NEWS | 5 + configure | 202 +++++++++++++++++++++++++++++++++----- configure.ac | 55 +++++++++- 3 files changed, 230 insertions(+), 32 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -156,6 +156,11 @@ Build ----- +- Issue #27983: Cause lack of llvm-profdata tool when using clang as + required for PGO linking to be a configure time error rather than + make time when --with-optimizations is enabled. Also improve our + ability to find the llvm-profdata tool on MacOS and some Linuxes. + - Issue #26359: Add the --with-optimizations configure flag. - Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -662,6 +662,11 @@ LIBTOOL_CRUFT OTHER_LIBTOOL_OPT LLVM_PROF_FOUND +target_os +target_vendor +target_cpu +target +LLVM_PROFDATA LLVM_PROF_ERR LLVM_PROF_FILE LLVM_PROF_MERGER @@ -1451,6 +1456,7 @@ System types: --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] + --target=TARGET configure for building compilers for TARGET [HOST] _ACEOF fi @@ -6425,9 +6431,11 @@ esac Py_LTO='true' DEF_MAKE_ALL_RULE="profile-opt" + REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" else DEF_MAKE_ALL_RULE="build_all" + REQUIRE_PGO="no" DEF_MAKE_RULE="all" fi @@ -6481,25 +6489,84 @@ - -# Extract the first word of "llvm-profdata", so it can be a program name with args. -set dummy llvm-profdata; ac_word=$2 +# Make this work on systems where llvm tools are not installed with their +# normal names in the default $PATH (ie: Ubuntu). They exist under the +# non-suffixed name in their versioned llvm directory. +llvm_bin_dir='' +llvm_path="${PATH}" +if test "${CC}" = "clang" +then + clang_bin=`which clang` + # Some systems install clang elsewhere as a symlink to the real path + # which is where the related llvm tools are located. + if test -L "${clang_bin}" + then + clang_dir=`dirname "${clang_bin}"` + clang_bin=`readlink "${clang_bin}"` + llvm_bin_dir="${clang_dir}/"`dirname "${clang_bin}"` + llvm_path="${llvm_path}${PATH_SEPARATOR}${llvm_bin_dir}" + fi +fi + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking target system type" >&5 +$as_echo_n "checking target system type... " >&6; } +if ${ac_cv_target+:} false; then : + $as_echo_n "(cached) " >&6 +else + if test "x$target_alias" = x; then + ac_cv_target=$ac_cv_host +else + ac_cv_target=`$SHELL "$ac_aux_dir/config.sub" $target_alias` || + as_fn_error $? "$SHELL $ac_aux_dir/config.sub $target_alias failed" "$LINENO" 5 +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_target" >&5 +$as_echo "$ac_cv_target" >&6; } +case $ac_cv_target in +*-*-*) ;; +*) as_fn_error $? "invalid value of canonical target" "$LINENO" 5;; +esac +target=$ac_cv_target +ac_save_IFS=$IFS; IFS='-' +set x $ac_cv_target +shift +target_cpu=$1 +target_vendor=$2 +shift; shift +# Remember, the first character of IFS is used to create $*, +# except with old shells: +target_os=$* +IFS=$ac_save_IFS +case $target_os in *\ *) target_os=`echo "$target_os" | sed 's/ /-/g'`;; esac + + +# The aliases save the names the user supplied, while $host etc. +# will get canonicalized. +test -n "$target_alias" && + test "$program_prefix$program_suffix$program_transform_name" = \ + NONENONEs,x,x, && + program_prefix=${target_alias}- +# Extract the first word of "$target_alias-llvm-profdata", so it can be a program name with args. +set dummy $target_alias-llvm-profdata; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_LLVM_PROF_FOUND+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$LLVM_PROF_FOUND"; then - ac_cv_prog_LLVM_PROF_FOUND="$LLVM_PROF_FOUND" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH +if ${ac_cv_path_LLVM_PROFDATA+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $LLVM_PROFDATA in + [\\/]* | ?:[\\/]*) + ac_cv_path_LLVM_PROFDATA="$LLVM_PROFDATA" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in ${llvm_path} do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_LLVM_PROF_FOUND="found" + ac_cv_path_LLVM_PROFDATA="$as_dir/$ac_word$ac_exec_ext" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi @@ -6507,30 +6574,105 @@ done IFS=$as_save_IFS - test -z "$ac_cv_prog_LLVM_PROF_FOUND" && ac_cv_prog_LLVM_PROF_FOUND="not-found" -fi -fi -LLVM_PROF_FOUND=$ac_cv_prog_LLVM_PROF_FOUND -if test -n "$LLVM_PROF_FOUND"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LLVM_PROF_FOUND" >&5 -$as_echo "$LLVM_PROF_FOUND" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - + ;; +esac +fi +LLVM_PROFDATA=$ac_cv_path_LLVM_PROFDATA +if test -n "$LLVM_PROFDATA"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LLVM_PROFDATA" >&5 +$as_echo "$LLVM_PROFDATA" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +if test -z "$ac_cv_path_LLVM_PROFDATA"; then + if test "$build" = "$target"; then + ac_pt_LLVM_PROFDATA=$LLVM_PROFDATA + # Extract the first word of "llvm-profdata", so it can be a program name with args. +set dummy llvm-profdata; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_LLVM_PROFDATA+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_LLVM_PROFDATA in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_LLVM_PROFDATA="$ac_pt_LLVM_PROFDATA" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in ${llvm_path} +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_LLVM_PROFDATA="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + test -z "$ac_cv_path_ac_pt_LLVM_PROFDATA" && ac_cv_path_ac_pt_LLVM_PROFDATA="''" + ;; +esac +fi +ac_pt_LLVM_PROFDATA=$ac_cv_path_ac_pt_LLVM_PROFDATA +if test -n "$ac_pt_LLVM_PROFDATA"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_LLVM_PROFDATA" >&5 +$as_echo "$ac_pt_LLVM_PROFDATA" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + LLVM_PROFDATA=$ac_pt_LLVM_PROFDATA + else + LLVM_PROFDATA="''" + fi +else + LLVM_PROFDATA="$ac_cv_path_LLVM_PROFDATA" +fi + + +if test -n "${LLVM_PROFDATA}" -a -x "${LLVM_PROFDATA}" +then + LLVM_PROF_FOUND="found" +else + LLVM_PROF_FOUND="not-found" +fi +if test "$ac_sys_system" = "Darwin" -a "${LLVM_PROF_FOUND}" = "not-found" +then + found_llvm_profdata=`/usr/bin/xcrun -find llvm-profdata 2>/dev/null` + if test -n "${found_llvm_profdata}" + then + # llvm-profdata isn't directly in $PATH in some cases. + # https://apple.stackexchange.com/questions/197053/ + LLVM_PROFDATA='/usr/bin/xcrun llvm-profdata' + LLVM_PROF_FOUND=found + { $as_echo "$as_me:${as_lineno-$LINENO}: llvm-profdata found via xcrun: ${LLVM_PROFDATA}" >&5 +$as_echo "$as_me: llvm-profdata found via xcrun: ${LLVM_PROFDATA}" >&6;} + fi +fi LLVM_PROF_ERR=no case $CC in *clang*) # Any changes made here should be reflected in the GCC+Darwin case below PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" if test $LLVM_PROF_FOUND = not-found then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5 + fi fi ;; *gcc*) @@ -6538,11 +6680,15 @@ Darwin*) PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" - if test $LLVM_PROF_FOUND = not-found + if test "${LLVM_PROF_FOUND}" = "not-found" then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + as_fn_error $? "llvm-profdata is required for a --with-optimizations build but could not be found." "$LINENO" 5 + fi fi ;; *) diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1405,9 +1405,11 @@ esac Py_LTO='true' DEF_MAKE_ALL_RULE="profile-opt" + REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" else DEF_MAKE_ALL_RULE="build_all" + REQUIRE_PGO="no" DEF_MAKE_RULE="all" fi @@ -1452,19 +1454,60 @@ AC_SUBST(LLVM_PROF_MERGER) AC_SUBST(LLVM_PROF_FILE) AC_SUBST(LLVM_PROF_ERR) +# Make this work on systems where llvm tools are not installed with their +# normal names in the default $PATH (ie: Ubuntu). They exist under the +# non-suffixed name in their versioned llvm directory. +llvm_bin_dir='' +llvm_path="${PATH}" +if test "${CC}" = "clang" +then + clang_bin=`which clang` + # Some systems install clang elsewhere as a symlink to the real path + # which is where the related llvm tools are located. + if test -L "${clang_bin}" + then + clang_dir=`dirname "${clang_bin}"` + clang_bin=`readlink "${clang_bin}"` + llvm_bin_dir="${clang_dir}/"`dirname "${clang_bin}"` + llvm_path="${llvm_path}${PATH_SEPARATOR}${llvm_bin_dir}" + fi +fi +AC_SUBST(LLVM_PROFDATA) +AC_PATH_TARGET_TOOL(LLVM_PROFDATA, llvm-profdata, '', ${llvm_path}) AC_SUBST(LLVM_PROF_FOUND) -AC_CHECK_PROG(LLVM_PROF_FOUND, llvm-profdata, found, not-found) +if test -n "${LLVM_PROFDATA}" -a -x "${LLVM_PROFDATA}" +then + LLVM_PROF_FOUND="found" +else + LLVM_PROF_FOUND="not-found" +fi +if test "$ac_sys_system" = "Darwin" -a "${LLVM_PROF_FOUND}" = "not-found" +then + found_llvm_profdata=`/usr/bin/xcrun -find llvm-profdata 2>/dev/null` + if test -n "${found_llvm_profdata}" + then + # llvm-profdata isn't directly in $PATH in some cases. + # https://apple.stackexchange.com/questions/197053/ + LLVM_PROFDATA='/usr/bin/xcrun llvm-profdata' + LLVM_PROF_FOUND=found + AC_MSG_NOTICE([llvm-profdata found via xcrun: ${LLVM_PROFDATA}]) + fi +fi LLVM_PROF_ERR=no case $CC in *clang*) # Any changes made here should be reflected in the GCC+Darwin case below PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" if test $LLVM_PROF_FOUND = not-found then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.]) + fi fi ;; *gcc*) @@ -1472,11 +1515,15 @@ Darwin*) PGO_PROF_GEN_FLAG="-fprofile-instr-generate" PGO_PROF_USE_FLAG="-fprofile-instr-use=code.profclangd" - LLVM_PROF_MERGER="llvm-profdata merge -output=code.profclangd *.profclangr" + LLVM_PROF_MERGER="${LLVM_PROFDATA} merge -output=code.profclangd *.profclangr" LLVM_PROF_FILE="LLVM_PROFILE_FILE=\"code-%p.profclangr\"" - if test $LLVM_PROF_FOUND = not-found + if test "${LLVM_PROF_FOUND}" = "not-found" then LLVM_PROF_ERR=yes + if test "${REQUIRE_PGO}" = "yes" + then + AC_MSG_ERROR([llvm-profdata is required for a --with-optimizations build but could not be found.]) + fi fi ;; *) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 03:15:29 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 07:15:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Move_my_news_e?= =?utf-8?q?ntries_to_the_build_section=2E?= Message-ID: <20160908071525.19574.88769.B474FDF4@psf.io> https://hg.python.org/cpython/rev/64ee75b7acc6 changeset: 103297:64ee75b7acc6 branch: 3.5 parent: 103274:99ce1713ae15 user: Gregory P. Smith date: Thu Sep 08 00:14:01 2016 -0700 summary: Move my news entries to the build section. files: Misc/NEWS | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,13 +10,6 @@ Core and Builtins ----------------- -- Issue #27983: Cause lack of llvm-profdata tool when using clang as - required for PGO linking to be a configure time error rather than - make time when --with-optimizations is enabled. Also improve our - ability to find the llvm-profdata tool on MacOS and some Linuxes. - -- Issue #26307: The profile-opt build now applys PGO to the built-in modules. - - Issue #27812: Properly clear out a generator's frame's backreference to the generator to prevent crashes in frame.clear(). @@ -277,6 +270,13 @@ Build ----- +- Issue #27983: Cause lack of llvm-profdata tool when using clang as + required for PGO linking to be a configure time error rather than + make time when --with-optimizations is enabled. Also improve our + ability to find the llvm-profdata tool on MacOS and some Linuxes. + +- Issue #26307: The profile-opt build now applys PGO to the built-in modules. + - Issue #26359: Add the --with-optimizations configure flag. - Issue #27713: Suppress spurious build warnings when updating importlib's -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 03:15:29 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 07:15:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_move_my_news_entries_to_the_build_section_as_appropriate?= =?utf-8?q?=2E?= Message-ID: <20160908071526.20950.35978.A0FECA8C@psf.io> https://hg.python.org/cpython/rev/139b1b6c324e changeset: 103298:139b1b6c324e parent: 103294:86d72a1303d0 parent: 103297:64ee75b7acc6 user: Gregory P. Smith date: Thu Sep 08 00:15:20 2016 -0700 summary: move my news entries to the build section as appropriate. files: Misc/NEWS | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -13,13 +13,6 @@ - Issue #27911: Remove unnecessary error checks in ``exec_builtin_or_dynamic()``. -- Issue #27983: Cause lack of llvm-profdata tool when using clang as - required for PGO linking to be a configure time error rather than - make time when --with-optimizations is enabled. Also improve our - ability to find the llvm-profdata tool on MacOS and some Linuxes. - -- Issue #26307: The profile-opt build now applys PGO to the built-in modules. - - Issue #27078: Added BUILD_STRING opcode. Optimized f-strings evaluation. - Issue #17884: Python now requires systems with inttypes.h and stdint.h @@ -265,6 +258,13 @@ Build ----- +- Issue #27983: Cause lack of llvm-profdata tool when using clang as + required for PGO linking to be a configure time error rather than + make time when --with-optimizations is enabled. Also improve our + ability to find the llvm-profdata tool on MacOS and some Linuxes. + +- Issue #26307: The profile-opt build now applys PGO to the built-in modules. + - Issue #26539: Add the --with-optimizations flag to turn on LTO and PGO build support when available. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 04:33:50 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 08:33:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_the_subjective_secu?= =?utf-8?q?rity_and_performance_claims=2C_fix_hyperlinks?= Message-ID: <20160908083350.36557.2967.ECEAB9BF@psf.io> https://hg.python.org/cpython/rev/368e0cfa5691 changeset: 103299:368e0cfa5691 user: Gregory P. Smith date: Thu Sep 08 01:33:43 2016 -0700 summary: Remove the subjective security and performance claims, fix hyperlinks to use https and add a link to RFC-7693. files: Doc/library/hashlib-blake2.rst | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Doc/library/hashlib-blake2.rst b/Doc/library/hashlib-blake2.rst --- a/Doc/library/hashlib-blake2.rst +++ b/Doc/library/hashlib-blake2.rst @@ -10,8 +10,8 @@ .. index:: single: blake2b, blake2s -BLAKE2_ is a cryptographic hash function, which offers highest security while -being as fast as MD5 or SHA-1, and comes in two flavors: +BLAKE2_ is a cryptographic hash function defined in RFC-7693_ that comes in two +flavors: * **BLAKE2b**, optimized for 64-bit platforms and produces digests of any size between 1 and 64 bytes, @@ -434,10 +434,11 @@ .. seealso:: Official BLAKE2 website: https://blake2.net +.. _RFC-7693: https://tools.ietf.org/html/rfc7693 .. _BLAKE2: https://blake2.net -.. _HMAC: http://en.wikipedia.org/wiki/Hash-based_message_authentication_code +.. _HMAC: https://en.wikipedia.org/wiki/Hash-based_message_authentication_code .. _BLAKE: https://131002.net/blake/ -.. _SHA-3: http://en.wikipedia.org/wiki/NIST_hash_function_competition -.. _ChaCha: http://cr.yp.to/chacha.html +.. _SHA-3: https://en.wikipedia.org/wiki/NIST_hash_function_competition +.. _ChaCha: https://cr.yp.to/chacha.html .. _pyblake2: https://pythonhosted.org/pyblake2/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 05:07:21 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 09:07:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_26798=3A_fetch_OSErr?= =?utf-8?q?or_and_HTTPException_like_other_tests_that_use?= Message-ID: <20160908090657.21237.40453.27829058@psf.io> https://hg.python.org/cpython/rev/46b34706eb41 changeset: 103300:46b34706eb41 user: Christian Heimes date: Thu Sep 08 10:53:40 2016 +0200 summary: Issue 26798: fetch OSError and HTTPException like other tests that use open_urlresource. files: Lib/test/test_hashlib.py | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py --- a/Lib/test/test_hashlib.py +++ b/Lib/test/test_hashlib.py @@ -20,6 +20,7 @@ import warnings from test import support from test.support import _4G, bigmemtest, import_fresh_module +from http.client import HTTPException # Were we compiled --with-pydebug or with #define Py_DEBUG? COMPILED_WITH_PYDEBUG = hasattr(sys, 'gettotalrefcount') @@ -54,8 +55,13 @@ URL = "http://www.pythontest.net/hashlib/{}.txt" def read_vectors(hash_name): - with support.open_urlresource(URL.format(hash_name)) as f: - for line in f: + url = URL.format(hash_name) + try: + testdata = support.open_urlresource(url) + except (OSError, HTTPException): + raise unittest.SkipTest("Could not retrieve {}".format(url)) + with testdata: + for line in testdata: line = line.strip() if line.startswith('#') or not line: continue -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 05:39:49 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 09:39:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_28017=3A_Use_-std=3D?= =?utf-8?q?gnu99_to_get_C99_with_GNU_extensions_for_bluetooth=2Eh_on?= Message-ID: <20160908093949.48610.70815.728297CB@psf.io> https://hg.python.org/cpython/rev/b5b2bb56d303 changeset: 103301:b5b2bb56d303 user: Christian Heimes date: Thu Sep 08 11:39:42 2016 +0200 summary: Issue 28017: Use -std=gnu99 to get C99 with GNU extensions for bluetooth.h on big endian. files: aclocal.m4 | 4 ++-- configure | 4 +++- configure.ac | 4 +++- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/aclocal.m4 b/aclocal.m4 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -13,7 +13,7 @@ m4_ifndef([AC_CONFIG_MACRO_DIRS], [m4_defun([_AM_CONFIG_MACRO_DIRS], [])m4_defun([AC_CONFIG_MACRO_DIRS], [_AM_CONFIG_MACRO_DIRS($@)])]) dnl pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*- -dnl serial 11 (pkg-config-0.29.1) +dnl serial 11 (pkg-config-0.29) dnl dnl Copyright ? 2004 Scott James Remnant . dnl Copyright ? 2012-2015 Dan Nicholson @@ -55,7 +55,7 @@ dnl See the "Since" comment for each macro you use to see what version dnl of the macros you require. m4_defun([PKG_PREREQ], -[m4_define([PKG_MACROS_VERSION], [0.29.1]) +[m4_define([PKG_MACROS_VERSION], [0.29]) m4_if(m4_version_compare(PKG_MACROS_VERSION, [$1]), -1, [m4_fatal([pkg.m4 version $1 or higher is required but ]PKG_MACROS_VERSION[ found])]) ])dnl PKG_PREREQ diff --git a/configure b/configure --- a/configure +++ b/configure @@ -6858,7 +6858,9 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) - CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" + # GNU dialect of C99, enables GNU extensions like __attribute__. GNU99 + # is required by bluetooth.h on big endian machines. + CFLAGS_NODIST="$CFLAGS_NODIST -std=gnu99" # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1523,7 +1523,9 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) - CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" + # GNU dialect of C99, enables GNU extensions like __attribute__. GNU99 + # is required by bluetooth.h on big endian machines. + CFLAGS_NODIST="$CFLAGS_NODIST -std=gnu99" # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 05:48:06 2016 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 08 Sep 2016 09:48:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Issue11551_-_I?= =?utf-8?q?ncrease_the_test_coverage_of_=5Fdummy=5Fthread_module_to_100=25?= =?utf-8?q?=2E?= Message-ID: <20160908094806.3133.9320.AD1FFF43@psf.io> https://hg.python.org/cpython/rev/08f446bf45a7 changeset: 103302:08f446bf45a7 branch: 3.5 parent: 103297:64ee75b7acc6 user: Senthil Kumaran date: Thu Sep 08 02:46:22 2016 -0700 summary: Issue11551 - Increase the test coverage of _dummy_thread module to 100%. Initial patch contributed by Denver Coneybeare. files: Lib/test/test_dummy_thread.py | 152 ++++++++++++++++----- 1 files changed, 113 insertions(+), 39 deletions(-) diff --git a/Lib/test/test_dummy_thread.py b/Lib/test/test_dummy_thread.py --- a/Lib/test/test_dummy_thread.py +++ b/Lib/test/test_dummy_thread.py @@ -1,19 +1,13 @@ -"""Generic thread tests. - -Meant to be used by dummy_thread and thread. To allow for different modules -to be used, test_main() can be called with the module to use as the thread -implementation as its sole argument. - -""" import _dummy_thread as _thread import time import queue import random import unittest from test import support +from unittest import mock -DELAY = 0 # Set > 0 when testing a module other than _dummy_thread, such as - # the '_thread' module. +DELAY = 0 + class LockTests(unittest.TestCase): """Test lock objects.""" @@ -34,6 +28,12 @@ self.assertFalse(self.lock.locked(), "Lock object did not release properly.") + def test_LockType_context_manager(self): + with _thread.LockType(): + pass + self.assertFalse(self.lock.locked(), + "Acquired Lock was not released") + def test_improper_release(self): #Make sure release of an unlocked thread raises RuntimeError self.assertRaises(RuntimeError, self.lock.release) @@ -83,39 +83,72 @@ self.assertGreaterEqual(end_time - start_time, DELAY, "Blocking by unconditional acquiring failed.") + @mock.patch('time.sleep') + def test_acquire_timeout(self, mock_sleep): + """Test invoking acquire() with a positive timeout when the lock is + already acquired. Ensure that time.sleep() is invoked with the given + timeout and that False is returned.""" + + self.lock.acquire() + retval = self.lock.acquire(waitflag=0, timeout=1) + self.assertTrue(mock_sleep.called) + mock_sleep.assert_called_once_with(1) + self.assertEqual(retval, False) + + def test_lock_representation(self): + self.lock.acquire() + self.assertIn("locked", repr(self.lock)) + self.lock.release() + self.assertIn("unlocked", repr(self.lock)) + + class MiscTests(unittest.TestCase): """Miscellaneous tests.""" def test_exit(self): - #Make sure _thread.exit() raises SystemExit self.assertRaises(SystemExit, _thread.exit) def test_ident(self): - #Test sanity of _thread.get_ident() self.assertIsInstance(_thread.get_ident(), int, "_thread.get_ident() returned a non-integer") self.assertNotEqual(_thread.get_ident(), 0, "_thread.get_ident() returned 0") def test_LockType(self): - #Make sure _thread.LockType is the same type as _thread.allocate_locke() self.assertIsInstance(_thread.allocate_lock(), _thread.LockType, "_thread.LockType is not an instance of what " "is returned by _thread.allocate_lock()") + def test_set_sentinel(self): + self.assertIsInstance(_thread._set_sentinel(), _thread.LockType, + "_thread._set_sentinel() did not return a " + "LockType instance.") + def test_interrupt_main(self): #Calling start_new_thread with a function that executes interrupt_main # should raise KeyboardInterrupt upon completion. def call_interrupt(): _thread.interrupt_main() - self.assertRaises(KeyboardInterrupt, _thread.start_new_thread, - call_interrupt, tuple()) + + self.assertRaises(KeyboardInterrupt, + _thread.start_new_thread, + call_interrupt, + tuple()) def test_interrupt_in_main(self): - # Make sure that if interrupt_main is called in main threat that - # KeyboardInterrupt is raised instantly. self.assertRaises(KeyboardInterrupt, _thread.interrupt_main) + def test_stack_size_None(self): + retval = _thread.stack_size(None) + self.assertEqual(retval, 0) + + def test_stack_size_not_None(self): + with self.assertRaises(_thread.error) as cm: + _thread.stack_size("") + self.assertEqual(cm.exception.args[0], + "setting thread stack size not supported") + + class ThreadTests(unittest.TestCase): """Test thread creation.""" @@ -129,31 +162,43 @@ _thread.start_new_thread(arg_tester, (testing_queue, True, True)) result = testing_queue.get() self.assertTrue(result[0] and result[1], - "Argument passing for thread creation using tuple failed") - _thread.start_new_thread(arg_tester, tuple(), {'queue':testing_queue, - 'arg1':True, 'arg2':True}) + "Argument passing for thread creation " + "using tuple failed") + + _thread.start_new_thread( + arg_tester, + tuple(), + {'queue':testing_queue, 'arg1':True, 'arg2':True}) + result = testing_queue.get() self.assertTrue(result[0] and result[1], - "Argument passing for thread creation using kwargs failed") - _thread.start_new_thread(arg_tester, (testing_queue, True), {'arg2':True}) + "Argument passing for thread creation " + "using kwargs failed") + + _thread.start_new_thread( + arg_tester, + (testing_queue, True), + {'arg2':True}) + result = testing_queue.get() self.assertTrue(result[0] and result[1], "Argument passing for thread creation using both tuple" " and kwargs failed") - def test_multi_creation(self): - #Make sure multiple threads can be created. + def test_multi_thread_creation(self): def queue_mark(queue, delay): - """Wait for ``delay`` seconds and then put something into ``queue``""" time.sleep(delay) queue.put(_thread.get_ident()) thread_count = 5 testing_queue = queue.Queue(thread_count) + if support.verbose: print() - print("*** Testing multiple thread creation "\ - "(will take approx. %s to %s sec.) ***" % (DELAY, thread_count)) + print("*** Testing multiple thread creation " + "(will take approx. %s to %s sec.) ***" % ( + DELAY, thread_count)) + for count in range(thread_count): if DELAY: local_delay = round(random.random(), 1) @@ -165,18 +210,47 @@ if support.verbose: print('done') self.assertEqual(testing_queue.qsize(), thread_count, - "Not all %s threads executed properly after %s sec." % - (thread_count, DELAY)) + "Not all %s threads executed properly " + "after %s sec." % (thread_count, DELAY)) -def test_main(imported_module=None): - global _thread, DELAY - if imported_module: - _thread = imported_module - DELAY = 2 - if support.verbose: - print() - print("*** Using %s as _thread module ***" % _thread) - support.run_unittest(LockTests, MiscTests, ThreadTests) + def test_args_not_tuple(self): + """ + Test invoking start_new_thread() with a non-tuple value for "args". + Expect TypeError with a meaningful error message to be raised. + """ + with self.assertRaises(TypeError) as cm: + _thread.start_new_thread(mock.Mock(), []) + self.assertEqual(cm.exception.args[0], "2nd arg must be a tuple") -if __name__ == '__main__': - test_main() + def test_kwargs_not_dict(self): + """ + Test invoking start_new_thread() with a non-dict value for "kwargs". + Expect TypeError with a meaningful error message to be raised. + """ + with self.assertRaises(TypeError) as cm: + _thread.start_new_thread(mock.Mock(), tuple(), kwargs=[]) + self.assertEqual(cm.exception.args[0], "3rd arg must be a dict") + + def test_SystemExit(self): + """ + Test invoking start_new_thread() with a function that raises + SystemExit. + The exception should be discarded. + """ + func = mock.Mock(side_effect=SystemExit()) + try: + _thread.start_new_thread(func, tuple()) + except SystemExit: + self.fail("start_new_thread raised SystemExit.") + + @mock.patch('traceback.print_exc') + def test_RaiseException(self, mock_print_exc): + """ + Test invoking start_new_thread() with a function that raises exception. + + The exception should be discarded and the traceback should be printed + via traceback.print_exc() + """ + func = mock.Mock(side_effect=Exception) + _thread.start_new_thread(func, tuple()) + self.assertTrue(mock_print_exc.called) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 05:48:06 2016 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 08 Sep 2016 09:48:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_=5Bmerge_from_3=2E5=5D_Issue11551_-_Increase_the_test_co?= =?utf-8?q?verage_of_=5Fdummy=5Fthread?= Message-ID: <20160908094806.69625.97315.5F1DB2DE@psf.io> https://hg.python.org/cpython/rev/d69e0df64b11 changeset: 103303:d69e0df64b11 parent: 103301:b5b2bb56d303 parent: 103302:08f446bf45a7 user: Senthil Kumaran date: Thu Sep 08 02:47:52 2016 -0700 summary: [merge from 3.5] Issue11551 - Increase the test coverage of _dummy_thread module to 100%. Initial patch contributed by Denver Coneybeare. files: Lib/test/test_dummy_thread.py | 152 ++++++++++++++++----- 1 files changed, 113 insertions(+), 39 deletions(-) diff --git a/Lib/test/test_dummy_thread.py b/Lib/test/test_dummy_thread.py --- a/Lib/test/test_dummy_thread.py +++ b/Lib/test/test_dummy_thread.py @@ -1,19 +1,13 @@ -"""Generic thread tests. - -Meant to be used by dummy_thread and thread. To allow for different modules -to be used, test_main() can be called with the module to use as the thread -implementation as its sole argument. - -""" import _dummy_thread as _thread import time import queue import random import unittest from test import support +from unittest import mock -DELAY = 0 # Set > 0 when testing a module other than _dummy_thread, such as - # the '_thread' module. +DELAY = 0 + class LockTests(unittest.TestCase): """Test lock objects.""" @@ -34,6 +28,12 @@ self.assertFalse(self.lock.locked(), "Lock object did not release properly.") + def test_LockType_context_manager(self): + with _thread.LockType(): + pass + self.assertFalse(self.lock.locked(), + "Acquired Lock was not released") + def test_improper_release(self): #Make sure release of an unlocked thread raises RuntimeError self.assertRaises(RuntimeError, self.lock.release) @@ -83,39 +83,72 @@ self.assertGreaterEqual(end_time - start_time, DELAY, "Blocking by unconditional acquiring failed.") + @mock.patch('time.sleep') + def test_acquire_timeout(self, mock_sleep): + """Test invoking acquire() with a positive timeout when the lock is + already acquired. Ensure that time.sleep() is invoked with the given + timeout and that False is returned.""" + + self.lock.acquire() + retval = self.lock.acquire(waitflag=0, timeout=1) + self.assertTrue(mock_sleep.called) + mock_sleep.assert_called_once_with(1) + self.assertEqual(retval, False) + + def test_lock_representation(self): + self.lock.acquire() + self.assertIn("locked", repr(self.lock)) + self.lock.release() + self.assertIn("unlocked", repr(self.lock)) + + class MiscTests(unittest.TestCase): """Miscellaneous tests.""" def test_exit(self): - #Make sure _thread.exit() raises SystemExit self.assertRaises(SystemExit, _thread.exit) def test_ident(self): - #Test sanity of _thread.get_ident() self.assertIsInstance(_thread.get_ident(), int, "_thread.get_ident() returned a non-integer") self.assertNotEqual(_thread.get_ident(), 0, "_thread.get_ident() returned 0") def test_LockType(self): - #Make sure _thread.LockType is the same type as _thread.allocate_locke() self.assertIsInstance(_thread.allocate_lock(), _thread.LockType, "_thread.LockType is not an instance of what " "is returned by _thread.allocate_lock()") + def test_set_sentinel(self): + self.assertIsInstance(_thread._set_sentinel(), _thread.LockType, + "_thread._set_sentinel() did not return a " + "LockType instance.") + def test_interrupt_main(self): #Calling start_new_thread with a function that executes interrupt_main # should raise KeyboardInterrupt upon completion. def call_interrupt(): _thread.interrupt_main() - self.assertRaises(KeyboardInterrupt, _thread.start_new_thread, - call_interrupt, tuple()) + + self.assertRaises(KeyboardInterrupt, + _thread.start_new_thread, + call_interrupt, + tuple()) def test_interrupt_in_main(self): - # Make sure that if interrupt_main is called in main threat that - # KeyboardInterrupt is raised instantly. self.assertRaises(KeyboardInterrupt, _thread.interrupt_main) + def test_stack_size_None(self): + retval = _thread.stack_size(None) + self.assertEqual(retval, 0) + + def test_stack_size_not_None(self): + with self.assertRaises(_thread.error) as cm: + _thread.stack_size("") + self.assertEqual(cm.exception.args[0], + "setting thread stack size not supported") + + class ThreadTests(unittest.TestCase): """Test thread creation.""" @@ -129,31 +162,43 @@ _thread.start_new_thread(arg_tester, (testing_queue, True, True)) result = testing_queue.get() self.assertTrue(result[0] and result[1], - "Argument passing for thread creation using tuple failed") - _thread.start_new_thread(arg_tester, tuple(), {'queue':testing_queue, - 'arg1':True, 'arg2':True}) + "Argument passing for thread creation " + "using tuple failed") + + _thread.start_new_thread( + arg_tester, + tuple(), + {'queue':testing_queue, 'arg1':True, 'arg2':True}) + result = testing_queue.get() self.assertTrue(result[0] and result[1], - "Argument passing for thread creation using kwargs failed") - _thread.start_new_thread(arg_tester, (testing_queue, True), {'arg2':True}) + "Argument passing for thread creation " + "using kwargs failed") + + _thread.start_new_thread( + arg_tester, + (testing_queue, True), + {'arg2':True}) + result = testing_queue.get() self.assertTrue(result[0] and result[1], "Argument passing for thread creation using both tuple" " and kwargs failed") - def test_multi_creation(self): - #Make sure multiple threads can be created. + def test_multi_thread_creation(self): def queue_mark(queue, delay): - """Wait for ``delay`` seconds and then put something into ``queue``""" time.sleep(delay) queue.put(_thread.get_ident()) thread_count = 5 testing_queue = queue.Queue(thread_count) + if support.verbose: print() - print("*** Testing multiple thread creation "\ - "(will take approx. %s to %s sec.) ***" % (DELAY, thread_count)) + print("*** Testing multiple thread creation " + "(will take approx. %s to %s sec.) ***" % ( + DELAY, thread_count)) + for count in range(thread_count): if DELAY: local_delay = round(random.random(), 1) @@ -165,18 +210,47 @@ if support.verbose: print('done') self.assertEqual(testing_queue.qsize(), thread_count, - "Not all %s threads executed properly after %s sec." % - (thread_count, DELAY)) + "Not all %s threads executed properly " + "after %s sec." % (thread_count, DELAY)) -def test_main(imported_module=None): - global _thread, DELAY - if imported_module: - _thread = imported_module - DELAY = 2 - if support.verbose: - print() - print("*** Using %s as _thread module ***" % _thread) - support.run_unittest(LockTests, MiscTests, ThreadTests) + def test_args_not_tuple(self): + """ + Test invoking start_new_thread() with a non-tuple value for "args". + Expect TypeError with a meaningful error message to be raised. + """ + with self.assertRaises(TypeError) as cm: + _thread.start_new_thread(mock.Mock(), []) + self.assertEqual(cm.exception.args[0], "2nd arg must be a tuple") -if __name__ == '__main__': - test_main() + def test_kwargs_not_dict(self): + """ + Test invoking start_new_thread() with a non-dict value for "kwargs". + Expect TypeError with a meaningful error message to be raised. + """ + with self.assertRaises(TypeError) as cm: + _thread.start_new_thread(mock.Mock(), tuple(), kwargs=[]) + self.assertEqual(cm.exception.args[0], "3rd arg must be a dict") + + def test_SystemExit(self): + """ + Test invoking start_new_thread() with a function that raises + SystemExit. + The exception should be discarded. + """ + func = mock.Mock(side_effect=SystemExit()) + try: + _thread.start_new_thread(func, tuple()) + except SystemExit: + self.fail("start_new_thread raised SystemExit.") + + @mock.patch('traceback.print_exc') + def test_RaiseException(self, mock_print_exc): + """ + Test invoking start_new_thread() with a function that raises exception. + + The exception should be discarded and the traceback should be printed + via traceback.print_exc() + """ + func = mock.Mock(side_effect=Exception) + _thread.start_new_thread(func, tuple()) + self.assertTrue(mock_print_exc.called) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 07:35:14 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 11:35:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2316113=3A_SHA3=3A_?= =?utf-8?q?allocate_extra_memory_for_lane_extraction_and_check_return?= Message-ID: <20160908113513.59502.39292.EC8D23B3@psf.io> https://hg.python.org/cpython/rev/e5871ffe9ac0 changeset: 103304:e5871ffe9ac0 user: Christian Heimes date: Thu Sep 08 13:35:00 2016 +0200 summary: Issue #16113: SHA3: allocate extra memory for lane extraction and check return value of PyModule_Create() files: Modules/_sha3/sha3module.c | 16 ++++++++++++---- 1 files changed, 12 insertions(+), 4 deletions(-) diff --git a/Modules/_sha3/sha3module.c b/Modules/_sha3/sha3module.c --- a/Modules/_sha3/sha3module.c +++ b/Modules/_sha3/sha3module.c @@ -114,6 +114,7 @@ #endif #define SHA3_MAX_DIGESTSIZE 64 /* 64 Bytes (512 Bits) for 224 to 512 */ +#define SHA3_LANESIZE 96 /* ExtractLane needs an extra 96 bytes */ #define SHA3_state Keccak_HashInstance #define SHA3_init Keccak_HashInitialize #define SHA3_process Keccak_HashUpdate @@ -310,7 +311,7 @@ _sha3_sha3_224_digest_impl(SHA3object *self) /*[clinic end generated code: output=fd531842e20b2d5b input=a5807917d219b30e]*/ { - unsigned char digest[SHA3_MAX_DIGESTSIZE]; + unsigned char digest[SHA3_MAX_DIGESTSIZE + SHA3_LANESIZE]; SHA3_state temp; HashReturn res; @@ -337,7 +338,7 @@ _sha3_sha3_224_hexdigest_impl(SHA3object *self) /*[clinic end generated code: output=75ad03257906918d input=2d91bb6e0d114ee3]*/ { - unsigned char digest[SHA3_MAX_DIGESTSIZE]; + unsigned char digest[SHA3_MAX_DIGESTSIZE + SHA3_LANESIZE]; SHA3_state temp; HashReturn res; @@ -601,7 +602,12 @@ int res; PyObject *result = NULL; - if ((digest = (unsigned char*)PyMem_Malloc(digestlen)) == NULL) { + /* ExtractLane needs at least SHA3_MAX_DIGESTSIZE + SHA3_LANESIZE and + * SHA3_LANESIZE extra space. + */ + digest = (unsigned char*)PyMem_Malloc(SHA3_LANESIZE + + ((digestlen > SHA3_MAX_DIGESTSIZE) ? digestlen : SHA3_MAX_DIGESTSIZE)); + if (digest == NULL) { return PyErr_NoMemory(); } @@ -708,7 +714,9 @@ { PyObject *m = NULL; - m = PyModule_Create(&_SHA3module); + if ((m = PyModule_Create(&_SHA3module)) == NULL) { + return NULL; + } #define init_sha3type(name, type) \ do { \ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 07:40:56 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 11:40:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326798=3A_Coverity?= =?utf-8?q?_complains_about_potential_memcpy=28=29_of_overlapped?= Message-ID: <20160908114048.19090.99503.699335FD@psf.io> https://hg.python.org/cpython/rev/fa89fff0b52c changeset: 103305:fa89fff0b52c user: Christian Heimes date: Thu Sep 08 13:40:25 2016 +0200 summary: Issue #26798: Coverity complains about potential memcpy() of overlapped regions. It doesn't hurt to use memmove() here. CID 1372514 / CID 1372515. Upstream https://github.com/BLAKE2/BLAKE2/issues/32 files: Modules/_blake2/impl/blake2b-ref.c | 2 +- Modules/_blake2/impl/blake2b.c | 2 +- Modules/_blake2/impl/blake2s-ref.c | 2 +- Modules/_blake2/impl/blake2s.c | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Modules/_blake2/impl/blake2b-ref.c b/Modules/_blake2/impl/blake2b-ref.c --- a/Modules/_blake2/impl/blake2b-ref.c +++ b/Modules/_blake2/impl/blake2b-ref.c @@ -334,7 +334,7 @@ blake2b_increment_counter( S, BLAKE2B_BLOCKBYTES ); blake2b_compress( S, S->buf ); S->buflen -= BLAKE2B_BLOCKBYTES; - memcpy( S->buf, S->buf + BLAKE2B_BLOCKBYTES, S->buflen ); + memmove( S->buf, S->buf + BLAKE2B_BLOCKBYTES, S->buflen ); } blake2b_increment_counter( S, S->buflen ); diff --git a/Modules/_blake2/impl/blake2b.c b/Modules/_blake2/impl/blake2b.c --- a/Modules/_blake2/impl/blake2b.c +++ b/Modules/_blake2/impl/blake2b.c @@ -371,7 +371,7 @@ blake2b_increment_counter( S, BLAKE2B_BLOCKBYTES ); blake2b_compress( S, S->buf ); S->buflen -= BLAKE2B_BLOCKBYTES; - memcpy( S->buf, S->buf + BLAKE2B_BLOCKBYTES, S->buflen ); + memmove( S->buf, S->buf + BLAKE2B_BLOCKBYTES, S->buflen ); } blake2b_increment_counter( S, S->buflen ); diff --git a/Modules/_blake2/impl/blake2s-ref.c b/Modules/_blake2/impl/blake2s-ref.c --- a/Modules/_blake2/impl/blake2s-ref.c +++ b/Modules/_blake2/impl/blake2s-ref.c @@ -325,7 +325,7 @@ blake2s_increment_counter( S, BLAKE2S_BLOCKBYTES ); blake2s_compress( S, S->buf ); S->buflen -= BLAKE2S_BLOCKBYTES; - memcpy( S->buf, S->buf + BLAKE2S_BLOCKBYTES, S->buflen ); + memmove( S->buf, S->buf + BLAKE2S_BLOCKBYTES, S->buflen ); } blake2s_increment_counter( S, ( uint32_t )S->buflen ); diff --git a/Modules/_blake2/impl/blake2s.c b/Modules/_blake2/impl/blake2s.c --- a/Modules/_blake2/impl/blake2s.c +++ b/Modules/_blake2/impl/blake2s.c @@ -348,7 +348,7 @@ blake2s_increment_counter( S, BLAKE2S_BLOCKBYTES ); blake2s_compress( S, S->buf ); S->buflen -= BLAKE2S_BLOCKBYTES; - memcpy( S->buf, S->buf + BLAKE2S_BLOCKBYTES, S->buflen ); + memmove( S->buf, S->buf + BLAKE2S_BLOCKBYTES, S->buflen ); } blake2s_increment_counter( S, ( uint32_t )S->buflen ); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 08:47:47 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Thu, 08 Sep 2016 12:47:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_old_typo=2E?= Message-ID: <20160908124746.48720.83571.20873465@psf.io> https://hg.python.org/cpython/rev/2150eadb54c7 changeset: 103306:2150eadb54c7 user: Serhiy Storchaka date: Thu Sep 08 15:47:27 2016 +0300 summary: Remove old typo. Initially (e0b7e34b5971) it should be \udef0, but after 52a77ef069cd (issue #3672) lone surrogates are not accepted and should be removed. files: Lib/test/test_bytes.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_bytes.py b/Lib/test/test_bytes.py --- a/Lib/test/test_bytes.py +++ b/Lib/test/test_bytes.py @@ -216,7 +216,7 @@ self.assertEqual(b, self.type2test(sample[:-3], "utf-8")) def test_decode(self): - sample = "Hello world\n\u1234\u5678\u9abc\def0\def0" + sample = "Hello world\n\u1234\u5678\u9abc" for enc in ("utf-8", "utf-16"): b = self.type2test(sample, enc) self.assertEqual(b.decode(enc), sample) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 09:05:00 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 13:05:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_sha3=3A_let=27s_keep_it_si?= =?utf-8?q?mple_and_always_allocate_enough_extra_space_for?= Message-ID: <20160908130453.48739.27379.1403557D@psf.io> https://hg.python.org/cpython/rev/01943b2114a6 changeset: 103307:01943b2114a6 user: Christian Heimes date: Thu Sep 08 15:04:38 2016 +0200 summary: sha3: let's keep it simple and always allocate enough extra space for uint64_t[20]. files: Modules/_sha3/sha3module.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Modules/_sha3/sha3module.c b/Modules/_sha3/sha3module.c --- a/Modules/_sha3/sha3module.c +++ b/Modules/_sha3/sha3module.c @@ -114,7 +114,7 @@ #endif #define SHA3_MAX_DIGESTSIZE 64 /* 64 Bytes (512 Bits) for 224 to 512 */ -#define SHA3_LANESIZE 96 /* ExtractLane needs an extra 96 bytes */ +#define SHA3_LANESIZE (20 * 8) /* ExtractLane needs max uint64_t[20] extra. */ #define SHA3_state Keccak_HashInstance #define SHA3_init Keccak_HashInitialize #define SHA3_process Keccak_HashUpdate @@ -605,8 +605,7 @@ /* ExtractLane needs at least SHA3_MAX_DIGESTSIZE + SHA3_LANESIZE and * SHA3_LANESIZE extra space. */ - digest = (unsigned char*)PyMem_Malloc(SHA3_LANESIZE + - ((digestlen > SHA3_MAX_DIGESTSIZE) ? digestlen : SHA3_MAX_DIGESTSIZE)); + digest = (unsigned char*)PyMem_Malloc(digestlen + SHA3_LANESIZE); if (digest == NULL) { return PyErr_NoMemory(); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:16:05 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 16:16:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_more_PY=5FLONG=5FLONG_to_l?= =?utf-8?q?ong_long?= Message-ID: <20160908161559.13065.38178.F3D553F1@psf.io> https://hg.python.org/cpython/rev/b0ef503c0e3a changeset: 103308:b0ef503c0e3a user: Benjamin Peterson date: Thu Sep 08 09:15:54 2016 -0700 summary: more PY_LONG_LONG to long long files: Doc/c-api/arg.rst | 23 +--- Doc/c-api/long.rst | 18 ++-- Doc/c-api/unicode.rst | 1 - Include/pyport.h | 30 ------- Lib/test/test_getargs2.py | 5 - Modules/_blake2/blake2b_impl.c | 4 +- Modules/_blake2/blake2s_impl.c | 4 +- Modules/_ctypes/_ctypes_test.c | 40 ++++---- Modules/_ctypes/callproc.c | 2 +- Modules/_ctypes/cfield.c | 48 +++++----- Modules/_ctypes/ctypes.h | 2 +- Modules/_io/_iomodule.h | 14 +- Modules/_io/textio.c | 4 +- Modules/_sqlite/util.c | 2 +- Objects/unicodeobject.c | 4 +- aclocal.m4 | 4 +- configure | 93 ---------------------- configure.ac | 61 -------------- pyconfig.h.in | 3 - 19 files changed, 81 insertions(+), 281 deletions(-) diff --git a/Doc/c-api/arg.rst b/Doc/c-api/arg.rst --- a/Doc/c-api/arg.rst +++ b/Doc/c-api/arg.rst @@ -265,15 +265,12 @@ Convert a Python integer to a C :c:type:`unsigned long` without overflow checking. -``L`` (:class:`int`) [PY_LONG_LONG] - Convert a Python integer to a C :c:type:`long long`. This format is only - available on platforms that support :c:type:`long long` (or :c:type:`_int64` on - Windows). +``L`` (:class:`int`) [long long] + Convert a Python integer to a C :c:type:`long long`. -``K`` (:class:`int`) [unsigned PY_LONG_LONG] +``K`` (:class:`int`) [unsigned long long] Convert a Python integer to a C :c:type:`unsigned long long` - without overflow checking. This format is only available on platforms that - support :c:type:`unsigned long long` (or :c:type:`unsigned _int64` on Windows). + without overflow checking. ``n`` (:class:`int`) [Py_ssize_t] Convert a Python integer to a C :c:type:`Py_ssize_t`. @@ -594,15 +591,11 @@ ``k`` (:class:`int`) [unsigned long] Convert a C :c:type:`unsigned long` to a Python integer object. - ``L`` (:class:`int`) [PY_LONG_LONG] - Convert a C :c:type:`long long` to a Python integer object. Only available - on platforms that support :c:type:`long long` (or :c:type:`_int64` on - Windows). + ``L`` (:class:`int`) [long long] + Convert a C :c:type:`long long` to a Python integer object. - ``K`` (:class:`int`) [unsigned PY_LONG_LONG] - Convert a C :c:type:`unsigned long long` to a Python integer object. Only - available on platforms that support :c:type:`unsigned long long` (or - :c:type:`unsigned _int64` on Windows). + ``K`` (:class:`int`) [unsigned long long] + Convert a C :c:type:`unsigned long long` to a Python integer object. ``n`` (:class:`int`) [Py_ssize_t] Convert a C :c:type:`Py_ssize_t` to a Python integer. diff --git a/Doc/c-api/long.rst b/Doc/c-api/long.rst --- a/Doc/c-api/long.rst +++ b/Doc/c-api/long.rst @@ -62,13 +62,13 @@ *NULL* on failure. -.. c:function:: PyObject* PyLong_FromLongLong(PY_LONG_LONG v) +.. c:function:: PyObject* PyLong_FromLongLong(long long v) Return a new :c:type:`PyLongObject` object from a C :c:type:`long long`, or *NULL* on failure. -.. c:function:: PyObject* PyLong_FromUnsignedLongLong(unsigned PY_LONG_LONG v) +.. c:function:: PyObject* PyLong_FromUnsignedLongLong(unsigned long long v) Return a new :c:type:`PyLongObject` object from a C :c:type:`unsigned long long`, or *NULL* on failure. @@ -148,7 +148,7 @@ occurs set *\*overflow* to ``0`` and return ``-1`` as usual. -.. c:function:: PY_LONG_LONG PyLong_AsLongLong(PyObject *obj) +.. c:function:: long long PyLong_AsLongLong(PyObject *obj) .. index:: single: OverflowError (built-in exception) @@ -161,7 +161,7 @@ :c:type:`long`. -.. c:function:: PY_LONG_LONG PyLong_AsLongLongAndOverflow(PyObject *obj, int *overflow) +.. c:function:: long long PyLong_AsLongLongAndOverflow(PyObject *obj, int *overflow) Return a C :c:type:`long long` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`__int__` method @@ -210,16 +210,16 @@ :c:type:`size_t`. -.. c:function:: unsigned PY_LONG_LONG PyLong_AsUnsignedLongLong(PyObject *pylong) +.. c:function:: unsigned long long PyLong_AsUnsignedLongLong(PyObject *pylong) .. index:: single: OverflowError (built-in exception) - Return a C :c:type:`unsigned PY_LONG_LONG` representation of *pylong*. - *pylong* must be an instance of :c:type:`PyLongObject`. + Return a C :c:type:`unsigned long long` representation of *pylong*. *pylong* + must be an instance of :c:type:`PyLongObject`. Raise :exc:`OverflowError` if the value of *pylong* is out of range for an - :c:type:`unsigned PY_LONG_LONG`. + :c:type:`unsigned long long`. .. versionchanged:: 3.1 A negative *pylong* now raises :exc:`OverflowError`, not :exc:`TypeError`. @@ -235,7 +235,7 @@ return the reduction of that value modulo :const:`ULONG_MAX + 1`. -.. c:function:: unsigned PY_LONG_LONG PyLong_AsUnsignedLongLongMask(PyObject *obj) +.. c:function:: unsigned long long PyLong_AsUnsignedLongLongMask(PyObject *obj) Return a C :c:type:`unsigned long long` representation of *obj*. If *obj* is not an instance of :c:type:`PyLongObject`, first call its :meth:`__int__` diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -440,7 +440,6 @@ .. % because not all compilers support the %z width modifier -- we fake it .. % when necessary via interpolating PY_FORMAT_SIZE_T. .. % Similar comments apply to the %ll width modifier and - .. % PY_FORMAT_LONG_LONG. .. tabularcolumns:: |l|l|L| diff --git a/Include/pyport.h b/Include/pyport.h --- a/Include/pyport.h +++ b/Include/pyport.h @@ -39,27 +39,10 @@ #ifndef PY_LONG_LONG #define PY_LONG_LONG long long -#if defined(LLONG_MAX) /* If LLONG_MAX is defined in limits.h, use that. */ #define PY_LLONG_MIN LLONG_MIN #define PY_LLONG_MAX LLONG_MAX #define PY_ULLONG_MAX ULLONG_MAX -#elif defined(__LONG_LONG_MAX__) -/* Otherwise, if GCC has a builtin define, use that. (Definition of - * PY_LLONG_MIN assumes two's complement with no trap representation.) */ -#define PY_LLONG_MAX __LONG_LONG_MAX__ -#define PY_LLONG_MIN (-PY_LLONG_MAX - 1) -#define PY_ULLONG_MAX (PY_LLONG_MAX * Py_ULL(2) + 1) -#elif defined(SIZEOF_LONG_LONG) -/* Otherwise compute from SIZEOF_LONG_LONG, assuming two's complement, no - padding bits, and no trap representation. Note: PY_ULLONG_MAX was - previously #defined as (~0ULL) here; but that'll give the wrong value in a - preprocessor expression on systems where long long != intmax_t. */ -#define PY_LLONG_MAX \ - (1 + 2 * ((Py_LL(1) << (CHAR_BIT * SIZEOF_LONG_LONG - 2)) - 1)) -#define PY_LLONG_MIN (-PY_LLONG_MAX - 1) -#define PY_ULLONG_MAX (PY_LLONG_MAX * Py_ULL(2) + 1) -#endif /* LLONG_MAX */ #endif #define PY_UINT32_T uint32_t @@ -159,19 +142,6 @@ # endif #endif -/* PY_FORMAT_LONG_LONG is analogous to PY_FORMAT_SIZE_T above, but for - * the long long type instead of the size_t type. The "high level" Python format - * functions listed above will interpret "lld" or "llu" correctly on - * all platforms. - */ -#ifndef PY_FORMAT_LONG_LONG -# ifdef MS_WINDOWS -# define PY_FORMAT_LONG_LONG "I64" -# else -# error "This platform's pyconfig.h needs to define PY_FORMAT_LONG_LONG" -# endif -#endif - /* Py_LOCAL can be used instead of static to get the fastest possible calling * convention for functions that are local to a given module. * diff --git a/Lib/test/test_getargs2.py b/Lib/test/test_getargs2.py --- a/Lib/test/test_getargs2.py +++ b/Lib/test/test_getargs2.py @@ -5,10 +5,6 @@ # Skip this test if the _testcapi module isn't available. support.import_module('_testcapi') from _testcapi import getargs_keywords, getargs_keyword_only -try: - from _testcapi import getargs_L, getargs_K -except ImportError: - getargs_L = None # PY_LONG_LONG not available # > How about the following counterproposal. This also changes some of # > the other format codes to be a little more regular. @@ -309,7 +305,6 @@ self.assertRaises(OverflowError, getargs_n, VERY_LARGE) - at unittest.skipIf(getargs_L is None, 'PY_LONG_LONG is not available') class LongLong_TestCase(unittest.TestCase): def test_L(self): from _testcapi import getargs_L diff --git a/Modules/_blake2/blake2b_impl.c b/Modules/_blake2/blake2b_impl.c --- a/Modules/_blake2/blake2b_impl.c +++ b/Modules/_blake2/blake2b_impl.c @@ -100,7 +100,7 @@ Py_buffer buf; unsigned long leaf_size = 0; - unsigned PY_LONG_LONG node_offset = 0; + unsigned long long node_offset = 0; self = new_BLAKE2bObject(type); if (self == NULL) { @@ -170,7 +170,7 @@ if (node_offset_obj != NULL) { node_offset = PyLong_AsUnsignedLongLong(node_offset_obj); - if (node_offset == (unsigned PY_LONG_LONG) -1 && PyErr_Occurred()) { + if (node_offset == (unsigned long long) -1 && PyErr_Occurred()) { goto error; } } diff --git a/Modules/_blake2/blake2s_impl.c b/Modules/_blake2/blake2s_impl.c --- a/Modules/_blake2/blake2s_impl.c +++ b/Modules/_blake2/blake2s_impl.c @@ -100,7 +100,7 @@ Py_buffer buf; unsigned long leaf_size = 0; - unsigned PY_LONG_LONG node_offset = 0; + unsigned long long node_offset = 0; self = new_BLAKE2sObject(type); if (self == NULL) { @@ -170,7 +170,7 @@ if (node_offset_obj != NULL) { node_offset = PyLong_AsUnsignedLongLong(node_offset_obj); - if (node_offset == (unsigned PY_LONG_LONG) -1 && PyErr_Occurred()) { + if (node_offset == (unsigned long long) -1 && PyErr_Occurred()) { goto error; } } diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -233,15 +233,15 @@ return (*func)(table); } -EXPORT(PY_LONG_LONG) _testfunc_q_bhilfdq(signed char b, short h, int i, long l, float f, - double d, PY_LONG_LONG q) +EXPORT(long long) _testfunc_q_bhilfdq(signed char b, short h, int i, long l, float f, + double d, long long q) { - return (PY_LONG_LONG)(b + h + i + l + f + d + q); + return (long long)(b + h + i + l + f + d + q); } -EXPORT(PY_LONG_LONG) _testfunc_q_bhilfd(signed char b, short h, int i, long l, float f, double d) +EXPORT(long long) _testfunc_q_bhilfd(signed char b, short h, int i, long l, float f, double d) { - return (PY_LONG_LONG)(b + h + i + l + f + d); + return (long long)(b + h + i + l + f + d); } EXPORT(int) _testfunc_callback_i_if(int value, int (*func)(int)) @@ -254,10 +254,10 @@ return sum; } -EXPORT(PY_LONG_LONG) _testfunc_callback_q_qf(PY_LONG_LONG value, - PY_LONG_LONG (*func)(PY_LONG_LONG)) +EXPORT(long long) _testfunc_callback_q_qf(long long value, + long long (*func)(long long)) { - PY_LONG_LONG sum = 0; + long long sum = 0; while (value != 0) { sum += func(value); @@ -381,8 +381,8 @@ { } -EXPORT(PY_LONG_LONG) last_tf_arg_s; -EXPORT(unsigned PY_LONG_LONG) last_tf_arg_u; +EXPORT(long long) last_tf_arg_s; +EXPORT(unsigned long long) last_tf_arg_u; struct BITS { int A: 1, B:2, C:3, D:4, E: 5, F: 6, G: 7, H: 8, I: 9; @@ -445,8 +445,8 @@ { NULL, NULL, 0, NULL}, }; -#define S last_tf_arg_s = (PY_LONG_LONG)c -#define U last_tf_arg_u = (unsigned PY_LONG_LONG)c +#define S last_tf_arg_s = (long long)c +#define U last_tf_arg_u = (unsigned long long)c EXPORT(signed char) tf_b(signed char c) { S; return c/3; } EXPORT(unsigned char) tf_B(unsigned char c) { U; return c/3; } @@ -456,8 +456,8 @@ EXPORT(unsigned int) tf_I(unsigned int c) { U; return c/3; } EXPORT(long) tf_l(long c) { S; return c/3; } EXPORT(unsigned long) tf_L(unsigned long c) { U; return c/3; } -EXPORT(PY_LONG_LONG) tf_q(PY_LONG_LONG c) { S; return c/3; } -EXPORT(unsigned PY_LONG_LONG) tf_Q(unsigned PY_LONG_LONG c) { U; return c/3; } +EXPORT(long long) tf_q(long long c) { S; return c/3; } +EXPORT(unsigned long long) tf_Q(unsigned long long c) { U; return c/3; } EXPORT(float) tf_f(float c) { S; return c/3; } EXPORT(double) tf_d(double c) { S; return c/3; } EXPORT(long double) tf_D(long double c) { S; return c/3; } @@ -471,8 +471,8 @@ EXPORT(unsigned int) __stdcall s_tf_I(unsigned int c) { U; return c/3; } EXPORT(long) __stdcall s_tf_l(long c) { S; return c/3; } EXPORT(unsigned long) __stdcall s_tf_L(unsigned long c) { U; return c/3; } -EXPORT(PY_LONG_LONG) __stdcall s_tf_q(PY_LONG_LONG c) { S; return c/3; } -EXPORT(unsigned PY_LONG_LONG) __stdcall s_tf_Q(unsigned PY_LONG_LONG c) { U; return c/3; } +EXPORT(long long) __stdcall s_tf_q(long long c) { S; return c/3; } +EXPORT(unsigned long long) __stdcall s_tf_Q(unsigned long long c) { U; return c/3; } EXPORT(float) __stdcall s_tf_f(float c) { S; return c/3; } EXPORT(double) __stdcall s_tf_d(double c) { S; return c/3; } EXPORT(long double) __stdcall s_tf_D(long double c) { S; return c/3; } @@ -487,8 +487,8 @@ EXPORT(unsigned int) tf_bI(signed char x, unsigned int c) { U; return c/3; } EXPORT(long) tf_bl(signed char x, long c) { S; return c/3; } EXPORT(unsigned long) tf_bL(signed char x, unsigned long c) { U; return c/3; } -EXPORT(PY_LONG_LONG) tf_bq(signed char x, PY_LONG_LONG c) { S; return c/3; } -EXPORT(unsigned PY_LONG_LONG) tf_bQ(signed char x, unsigned PY_LONG_LONG c) { U; return c/3; } +EXPORT(long long) tf_bq(signed char x, long long c) { S; return c/3; } +EXPORT(unsigned long long) tf_bQ(signed char x, unsigned long long c) { U; return c/3; } EXPORT(float) tf_bf(signed char x, float c) { S; return c/3; } EXPORT(double) tf_bd(signed char x, double c) { S; return c/3; } EXPORT(long double) tf_bD(signed char x, long double c) { S; return c/3; } @@ -503,8 +503,8 @@ EXPORT(unsigned int) __stdcall s_tf_bI(signed char x, unsigned int c) { U; return c/3; } EXPORT(long) __stdcall s_tf_bl(signed char x, long c) { S; return c/3; } EXPORT(unsigned long) __stdcall s_tf_bL(signed char x, unsigned long c) { U; return c/3; } -EXPORT(PY_LONG_LONG) __stdcall s_tf_bq(signed char x, PY_LONG_LONG c) { S; return c/3; } -EXPORT(unsigned PY_LONG_LONG) __stdcall s_tf_bQ(signed char x, unsigned PY_LONG_LONG c) { U; return c/3; } +EXPORT(long long) __stdcall s_tf_bq(signed char x, long long c) { S; return c/3; } +EXPORT(unsigned long long) __stdcall s_tf_bQ(signed char x, unsigned long long c) { U; return c/3; } EXPORT(float) __stdcall s_tf_bf(signed char x, float c) { S; return c/3; } EXPORT(double) __stdcall s_tf_bd(signed char x, double c) { S; return c/3; } EXPORT(long double) __stdcall s_tf_bD(signed char x, long double c) { S; return c/3; } diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -591,7 +591,7 @@ short h; int i; long l; - PY_LONG_LONG q; + long long q; long double D; double d; float f; diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c --- a/Modules/_ctypes/cfield.c +++ b/Modules/_ctypes/cfield.c @@ -382,9 +382,9 @@ /* Same, but handling native long long. */ static int -get_longlong(PyObject *v, PY_LONG_LONG *p) +get_longlong(PyObject *v, long long *p) { - PY_LONG_LONG x; + long long x; if (PyFloat_Check(v)) { PyErr_SetString(PyExc_TypeError, "int expected instead of float"); @@ -400,16 +400,16 @@ /* Same, but handling native unsigned long long. */ static int -get_ulonglong(PyObject *v, unsigned PY_LONG_LONG *p) +get_ulonglong(PyObject *v, unsigned long long *p) { - unsigned PY_LONG_LONG x; + unsigned long long x; if (PyFloat_Check(v)) { PyErr_SetString(PyExc_TypeError, "int expected instead of float"); return -1; } x = PyLong_AsUnsignedLongLongMask(v); - if (x == (unsigned PY_LONG_LONG)-1 && PyErr_Occurred()) + if (x == (unsigned long long)-1 && PyErr_Occurred()) return -1; *p = x; return 0; @@ -879,12 +879,12 @@ static PyObject * q_set(void *ptr, PyObject *value, Py_ssize_t size) { - PY_LONG_LONG val; - PY_LONG_LONG x; + long long val; + long long x; if (get_longlong(value, &val) < 0) return NULL; memcpy(&x, ptr, sizeof(x)); - x = SET(PY_LONG_LONG, x, val, size); + x = SET(long long, x, val, size); memcpy(ptr, &x, sizeof(x)); _RET(value); } @@ -892,13 +892,13 @@ static PyObject * q_set_sw(void *ptr, PyObject *value, Py_ssize_t size) { - PY_LONG_LONG val; - PY_LONG_LONG field; + long long val; + long long field; if (get_longlong(value, &val) < 0) return NULL; memcpy(&field, ptr, sizeof(field)); field = SWAP_8(field); - field = SET(PY_LONG_LONG, field, val, size); + field = SET(long long, field, val, size); field = SWAP_8(field); memcpy(ptr, &field, sizeof(field)); _RET(value); @@ -907,7 +907,7 @@ static PyObject * q_get(void *ptr, Py_ssize_t size) { - PY_LONG_LONG val; + long long val; memcpy(&val, ptr, sizeof(val)); GET_BITFIELD(val, size); return PyLong_FromLongLong(val); @@ -916,7 +916,7 @@ static PyObject * q_get_sw(void *ptr, Py_ssize_t size) { - PY_LONG_LONG val; + long long val; memcpy(&val, ptr, sizeof(val)); val = SWAP_8(val); GET_BITFIELD(val, size); @@ -926,12 +926,12 @@ static PyObject * Q_set(void *ptr, PyObject *value, Py_ssize_t size) { - unsigned PY_LONG_LONG val; - unsigned PY_LONG_LONG x; + unsigned long long val; + unsigned long long x; if (get_ulonglong(value, &val) < 0) return NULL; memcpy(&x, ptr, sizeof(x)); - x = SET(PY_LONG_LONG, x, val, size); + x = SET(long long, x, val, size); memcpy(ptr, &x, sizeof(x)); _RET(value); } @@ -939,13 +939,13 @@ static PyObject * Q_set_sw(void *ptr, PyObject *value, Py_ssize_t size) { - unsigned PY_LONG_LONG val; - unsigned PY_LONG_LONG field; + unsigned long long val; + unsigned long long field; if (get_ulonglong(value, &val) < 0) return NULL; memcpy(&field, ptr, sizeof(field)); field = SWAP_8(field); - field = SET(unsigned PY_LONG_LONG, field, val, size); + field = SET(unsigned long long, field, val, size); field = SWAP_8(field); memcpy(ptr, &field, sizeof(field)); _RET(value); @@ -954,7 +954,7 @@ static PyObject * Q_get(void *ptr, Py_ssize_t size) { - unsigned PY_LONG_LONG val; + unsigned long long val; memcpy(&val, ptr, sizeof(val)); GET_BITFIELD(val, size); return PyLong_FromUnsignedLongLong(val); @@ -963,7 +963,7 @@ static PyObject * Q_get_sw(void *ptr, Py_ssize_t size) { - unsigned PY_LONG_LONG val; + unsigned long long val; memcpy(&val, ptr, sizeof(val)); val = SWAP_8(val); GET_BITFIELD(val, size); @@ -1477,7 +1477,7 @@ v = (void *)PyLong_AsUnsignedLongMask(value); #else #if SIZEOF_LONG_LONG < SIZEOF_VOID_P -# error "PyLong_AsVoidPtr: sizeof(PY_LONG_LONG) < sizeof(void*)" +# error "PyLong_AsVoidPtr: sizeof(long long) < sizeof(void*)" #endif v = (void *)PyLong_AsUnsignedLongLongMask(value); #endif @@ -1617,8 +1617,8 @@ #endif */ -typedef struct { char c; PY_LONG_LONG x; } s_long_long; -#define LONG_LONG_ALIGN (sizeof(s_long_long) - sizeof(PY_LONG_LONG)) +typedef struct { char c; long long x; } s_long_long; +#define LONG_LONG_ALIGN (sizeof(s_long_long) - sizeof(long long)) /* from ffi.h: typedef struct _ffi_type diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -301,7 +301,7 @@ short h; int i; long l; - PY_LONG_LONG q; + long long q; long double D; double d; float f; diff --git a/Modules/_io/_iomodule.h b/Modules/_io/_iomodule.h --- a/Modules/_io/_iomodule.h +++ b/Modules/_io/_iomodule.h @@ -85,12 +85,12 @@ #ifdef MS_WINDOWS /* Windows uses long long for offsets */ -typedef PY_LONG_LONG Py_off_t; +typedef long long Py_off_t; # define PyLong_AsOff_t PyLong_AsLongLong # define PyLong_FromOff_t PyLong_FromLongLong -# define PY_OFF_T_MAX PY_LLONG_MAX -# define PY_OFF_T_MIN PY_LLONG_MIN -# define PY_OFF_T_COMPAT PY_LONG_LONG /* type compatible with off_t */ +# define PY_OFF_T_MAX LLONG_MAX +# define PY_OFF_T_MIN LLONG_MIN +# define PY_OFF_T_COMPAT long long /* type compatible with off_t */ # define PY_PRIdOFF "lld" /* format to use for that type */ #else @@ -107,9 +107,9 @@ #elif (SIZEOF_OFF_T == SIZEOF_LONG_LONG) # define PyLong_AsOff_t PyLong_AsLongLong # define PyLong_FromOff_t PyLong_FromLongLong -# define PY_OFF_T_MAX PY_LLONG_MAX -# define PY_OFF_T_MIN PY_LLONG_MIN -# define PY_OFF_T_COMPAT PY_LONG_LONG +# define PY_OFF_T_MAX LLONG_MAX +# define PY_OFF_T_MIN LLONG_MIN +# define PY_OFF_T_COMPAT long long # define PY_PRIdOFF "lld" #elif (SIZEOF_OFF_T == SIZEOF_LONG) # define PyLong_AsOff_t PyLong_AsLong diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -531,7 +531,7 @@ /*[clinic end generated code: output=f0d2c9c136f4e0d0 input=f8ff101825e32e7f]*/ { PyObject *buffer; - unsigned PY_LONG_LONG flag; + unsigned long long flag; if (self->decoder != Py_None) { PyObject *state = PyObject_CallMethodObjArgs(self->decoder, @@ -567,7 +567,7 @@ /*[clinic end generated code: output=c10c622508b576cb input=c53fb505a76dbbe2]*/ { PyObject *buffer; - unsigned PY_LONG_LONG flag; + unsigned long long flag; if (!PyArg_ParseTuple(state, "OK", &buffer, &flag)) return NULL; diff --git a/Modules/_sqlite/util.c b/Modules/_sqlite/util.c --- a/Modules/_sqlite/util.c +++ b/Modules/_sqlite/util.c @@ -130,7 +130,7 @@ _pysqlite_long_as_int64(PyObject * py_val) { int overflow; - PY_LONG_LONG value = PyLong_AsLongLongAndOverflow(py_val, &overflow); + long long value = PyLong_AsLongLongAndOverflow(py_val, &overflow); if (value == -1 && PyErr_Occurred()) return -1; if (!overflow) { diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -2680,7 +2680,7 @@ len = sprintf(buffer, "%lu", va_arg(*vargs, unsigned long)); else if (longlongflag) - len = sprintf(buffer, "%" PY_FORMAT_LONG_LONG "u", + len = sprintf(buffer, "%llu", va_arg(*vargs, unsigned long long)); else if (size_tflag) len = sprintf(buffer, "%" PY_FORMAT_SIZE_T "u", @@ -2697,7 +2697,7 @@ len = sprintf(buffer, "%li", va_arg(*vargs, long)); else if (longlongflag) - len = sprintf(buffer, "%" PY_FORMAT_LONG_LONG "i", + len = sprintf(buffer, "%lli", va_arg(*vargs, long long)); else if (size_tflag) len = sprintf(buffer, "%" PY_FORMAT_SIZE_T "i", diff --git a/aclocal.m4 b/aclocal.m4 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -13,7 +13,7 @@ m4_ifndef([AC_CONFIG_MACRO_DIRS], [m4_defun([_AM_CONFIG_MACRO_DIRS], [])m4_defun([AC_CONFIG_MACRO_DIRS], [_AM_CONFIG_MACRO_DIRS($@)])]) dnl pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*- -dnl serial 11 (pkg-config-0.29) +dnl serial 11 (pkg-config-0.29.1) dnl dnl Copyright ? 2004 Scott James Remnant . dnl Copyright ? 2012-2015 Dan Nicholson @@ -55,7 +55,7 @@ dnl See the "Since" comment for each macro you use to see what version dnl of the macros you require. m4_defun([PKG_PREREQ], -[m4_define([PKG_MACROS_VERSION], [0.29]) +[m4_define([PKG_MACROS_VERSION], [0.29.1]) m4_if(m4_version_compare(PKG_MACROS_VERSION, [$1]), -1, [m4_fatal([pkg.m4 version $1 or higher is required but ]PKG_MACROS_VERSION[ found])]) ])dnl PKG_PREREQ diff --git a/configure b/configure --- a/configure +++ b/configure @@ -15729,99 +15729,6 @@ fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for %lld and %llu printf() format support" >&5 -$as_echo_n "checking for %lld and %llu printf() format support... " >&6; } -if ${ac_cv_have_long_long_format+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test "$cross_compiling" = yes; then : - ac_cv_have_long_long_format="cross -- assuming no" -if test x$GCC = xyes; then -save_CFLAGS=$CFLAGS -CFLAGS="$CFLAGS -Werror -Wformat" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -#include -#include - -int -main () -{ - -char *buffer; -sprintf(buffer, "%lld", (long long)123); -sprintf(buffer, "%lld", (long long)-123); -sprintf(buffer, "%llu", (unsigned long long)123); - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_have_long_long_format=yes - -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -CFLAGS=$save_CFLAGS -fi -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -#include -#include -#include - -#ifdef HAVE_SYS_TYPES_H -#include -#endif - -int main() -{ -char buffer[256]; - -if (sprintf(buffer, "%lld", (long long)123) < 0) -return 1; -if (strcmp(buffer, "123")) -return 1; - -if (sprintf(buffer, "%lld", (long long)-123) < 0) -return 1; -if (strcmp(buffer, "-123")) -return 1; - -if (sprintf(buffer, "%llu", (unsigned long long)123) < 0) -return 1; -if (strcmp(buffer, "123")) -return 1; - -return 0; -} - -_ACEOF -if ac_fn_c_try_run "$LINENO"; then : - ac_cv_have_long_long_format=yes -else - ac_cv_have_long_long_format=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ - conftest.$ac_objext conftest.beam conftest.$ac_ext -fi - - -fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_have_long_long_format" >&5 -$as_echo "$ac_cv_have_long_long_format" >&6; } - -if test "$ac_cv_have_long_long_format" = yes -then - -$as_echo "#define PY_FORMAT_LONG_LONG \"ll\"" >>confdefs.h - -fi - if test $ac_sys_system = Darwin then LIBS="$LIBS -framework CoreFoundation" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -4938,67 +4938,6 @@ [Define to 1 if you have the /dev/ptc device file.]) fi -AC_MSG_CHECKING(for %lld and %llu printf() format support) -AC_CACHE_VAL(ac_cv_have_long_long_format, -AC_RUN_IFELSE([AC_LANG_SOURCE([[[ -#include -#include -#include - -#ifdef HAVE_SYS_TYPES_H -#include -#endif - -int main() -{ -char buffer[256]; - -if (sprintf(buffer, "%lld", (long long)123) < 0) -return 1; -if (strcmp(buffer, "123")) -return 1; - -if (sprintf(buffer, "%lld", (long long)-123) < 0) -return 1; -if (strcmp(buffer, "-123")) -return 1; - -if (sprintf(buffer, "%llu", (unsigned long long)123) < 0) -return 1; -if (strcmp(buffer, "123")) -return 1; - -return 0; -} -]]])], -[ac_cv_have_long_long_format=yes], -[ac_cv_have_long_long_format=no], -[ac_cv_have_long_long_format="cross -- assuming no" -if test x$GCC = xyes; then -save_CFLAGS=$CFLAGS -CFLAGS="$CFLAGS -Werror -Wformat" -AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ -#include -#include -]], [[ -char *buffer; -sprintf(buffer, "%lld", (long long)123); -sprintf(buffer, "%lld", (long long)-123); -sprintf(buffer, "%llu", (unsigned long long)123); -]])], -ac_cv_have_long_long_format=yes -) -CFLAGS=$save_CFLAGS -fi]) -) -AC_MSG_RESULT($ac_cv_have_long_long_format) - -if test "$ac_cv_have_long_long_format" = yes -then - AC_DEFINE(PY_FORMAT_LONG_LONG, "ll", - [Define to printf format modifier for long long type]) -fi - if test $ac_sys_system = Darwin then LIBS="$LIBS -framework CoreFoundation" diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -1241,9 +1241,6 @@ /* Define as the preferred size in bits of long digits */ #undef PYLONG_BITS_IN_DIGIT -/* Define to printf format modifier for long long type */ -#undef PY_FORMAT_LONG_LONG - /* Define to printf format modifier for Py_ssize_t */ #undef PY_FORMAT_SIZE_T -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:27:15 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 16:27:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Adds_MAX=5FPATH_button_to_?= =?utf-8?q?the_installer=2E?= Message-ID: <20160908162714.59525.18652.3C7A4793@psf.io> https://hg.python.org/cpython/rev/cad9440775cf changeset: 103309:cad9440775cf user: Steve Dower date: Thu Sep 08 09:26:42 2016 -0700 summary: Adds MAX_PATH button to the installer. files: Tools/msi/bundle/Default.thm | 7 +- Tools/msi/bundle/Default.wxl | 3 + Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 155 ++++----- 3 files changed, 76 insertions(+), 89 deletions(-) diff --git a/Tools/msi/bundle/Default.thm b/Tools/msi/bundle/Default.thm --- a/Tools/msi/bundle/Default.thm +++ b/Tools/msi/bundle/Default.thm @@ -116,10 +116,11 @@ #(loc.SuccessHeader) - + - #(loc.SuccessRestartText) - + + + #(loc.SuccessRestartText) diff --git a/Tools/msi/bundle/Default.wxl b/Tools/msi/bundle/Default.wxl --- a/Tools/msi/bundle/Default.wxl +++ b/Tools/msi/bundle/Default.wxl @@ -132,4 +132,7 @@ Windows Vista or later is required to install and use [WixBundleName]. Visit <a href="https://www.python.org/">python.org</a> to download Python 3.4. + + Disable path length limit + Changes your machine configuration to allow programs, including Python, to bypass the 260 character "MAX_PATH" limitation. diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -11,10 +11,6 @@ #include "pch.h" static const LPCWSTR PYBA_WINDOW_CLASS = L"PythonBA"; -static const LPCWSTR PYBA_VARIABLE_LAUNCH_TARGET_PATH = L"LaunchTarget"; -static const LPCWSTR PYBA_VARIABLE_LAUNCH_TARGET_ELEVATED_ID = L"LaunchTargetElevatedId"; -static const LPCWSTR PYBA_VARIABLE_LAUNCH_ARGUMENTS = L"LaunchArguments"; -static const LPCWSTR PYBA_VARIABLE_LAUNCH_HIDDEN = L"LaunchHidden"; static const DWORD PYBA_ACQUIRE_PERCENTAGE = 30; static const LPCWSTR PYBA_VARIABLE_BUNDLE_FILE_VERSION = L"WixBundleFileVersion"; @@ -129,11 +125,11 @@ ID_PROGRESS_CANCEL_BUTTON, // Success page - ID_LAUNCH_BUTTON, ID_SUCCESS_TEXT, ID_SUCCESS_RESTART_TEXT, ID_SUCCESS_RESTART_BUTTON, ID_SUCCESS_CANCEL_BUTTON, + ID_SUCCESS_MAX_PATH_BUTTON, // Failure page ID_FAILURE_LOGFILE_LINK, @@ -188,11 +184,11 @@ { ID_OVERALL_PROGRESS_TEXT, L"OverallProgressText" }, { ID_PROGRESS_CANCEL_BUTTON, L"ProgressCancelButton" }, - { ID_LAUNCH_BUTTON, L"LaunchButton" }, { ID_SUCCESS_TEXT, L"SuccessText" }, { ID_SUCCESS_RESTART_TEXT, L"SuccessRestartText" }, { ID_SUCCESS_RESTART_BUTTON, L"SuccessRestartButton" }, { ID_SUCCESS_CANCEL_BUTTON, L"SuccessCancelButton" }, + { ID_SUCCESS_MAX_PATH_BUTTON, L"SuccessMaxPathButton" }, { ID_FAILURE_LOGFILE_LINK, L"FailureLogFileLink" }, { ID_FAILURE_MESSAGE_TEXT, L"FailureMessageText" }, @@ -433,6 +429,11 @@ case ID_UNINSTALL_BUTTON: OnPlan(BOOTSTRAPPER_ACTION_UNINSTALL); break; + + case ID_SUCCESS_MAX_PATH_BUTTON: + EnableMaxPathSupport(); + ThemeControlEnable(_theme, ID_SUCCESS_MAX_PATH_BUTTON, FALSE); + break; } LExit: @@ -530,9 +531,8 @@ } void SuccessPage_Show() { - // on the "Success" page, check if the restart or launch button should be enabled. + // on the "Success" page, check if the restart button should be enabled. BOOL showRestartButton = FALSE; - BOOL launchTargetExists = FALSE; LOC_STRING *successText = nullptr; HRESULT hr = S_OK; @@ -540,8 +540,6 @@ if (BOOTSTRAPPER_RESTART_PROMPT == _command.restart) { showRestartButton = TRUE; } - } else if (ThemeControlExists(_theme, ID_LAUNCH_BUTTON)) { - launchTargetExists = BalStringVariableExists(PYBA_VARIABLE_LAUNCH_TARGET_PATH); } switch (_plannedAction) { @@ -568,9 +566,41 @@ } } - ThemeControlEnable(_theme, ID_LAUNCH_BUTTON, launchTargetExists && BOOTSTRAPPER_ACTION_UNINSTALL < _plannedAction); ThemeControlEnable(_theme, ID_SUCCESS_RESTART_TEXT, showRestartButton); ThemeControlEnable(_theme, ID_SUCCESS_RESTART_BUTTON, showRestartButton); + + if (_command.action != BOOTSTRAPPER_ACTION_INSTALL || + !IsWindowsVersionOrGreater(10, 0, 0)) { + ThemeControlEnable(_theme, ID_SUCCESS_MAX_PATH_BUTTON, FALSE); + } else { + DWORD dataType = 0, buffer = 0, bufferLen = sizeof(buffer); + HKEY hKey; + LRESULT res = RegOpenKeyExW( + HKEY_LOCAL_MACHINE, + L"SYSTEM\\CurrentControlSet\\Control\\FileSystem", + 0, + KEY_READ, + &hKey + ); + if (res == ERROR_SUCCESS) { + res = RegQueryValueExW(hKey, L"LongPathsEnabled", nullptr, &dataType, + (LPBYTE)&buffer, &bufferLen); + RegCloseKey(hKey); + } + else { + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Failed to open SYSTEM\\CurrentControlSet\\Control\\FileSystem: error code %d", res); + } + if (res == ERROR_SUCCESS && dataType == REG_DWORD && buffer == 0) { + ThemeControlElevates(_theme, ID_SUCCESS_MAX_PATH_BUTTON, TRUE); + } + else { + if (res == ERROR_SUCCESS) + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Failed to read LongPathsEnabled value: error code %d", res); + else + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Hiding MAX_PATH button because it is already enabled"); + ThemeControlEnable(_theme, ID_SUCCESS_MAX_PATH_BUTTON, FALSE); + } + } } void FailurePage_Show() { @@ -623,6 +653,34 @@ ThemeControlEnable(_theme, ID_FAILURE_RESTART_BUTTON, showRestartButton); } + static void EnableMaxPathSupport() { + LPWSTR targetDir = nullptr, defaultDir = nullptr; + HRESULT hr = BalGetStringVariable(L"TargetDir", &targetDir); + if (FAILED(hr) || !targetDir || !targetDir[0]) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Failed to get TargetDir"); + return; + } + + LPWSTR pythonw = nullptr; + StrAllocFormatted(&pythonw, L"%ls\\pythonw.exe", targetDir); + if (!pythonw || !pythonw[0]) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Failed to construct pythonw.exe path"); + return; + } + + LPCWSTR arguments = L"-c \"import winreg; " + "winreg.SetValueEx(" + "winreg.CreateKey(winreg.HKEY_LOCAL_MACHINE, " + "r'SYSTEM\\CurrentControlSet\\Control\\FileSystem'), " + "'LongPathsEnabled', " + "None, " + "winreg.REG_DWORD, " + "1" + ")\""; + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Executing %ls %ls", pythonw, arguments); + HINSTANCE res = ShellExecuteW(0, L"runas", pythonw, arguments, NULL, SW_HIDE); + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "return code 0x%08x", res); + } public: // IBootstrapperApplication virtual STDMETHODIMP OnStartup() { @@ -1213,12 +1271,6 @@ } virtual STDMETHODIMP_(void) OnLaunchApprovedExeComplete(__in HRESULT hrStatus, __in DWORD /*processId*/) { - if (HRESULT_FROM_WIN32(ERROR_ACCESS_DENIED) == hrStatus) { - //try with ShelExec next time - OnClickLaunchButton(); - } else { - ::PostMessageW(_hWnd, WM_CLOSE, 0, 0); - } } @@ -1864,10 +1916,6 @@ switch (LOWORD(wParam)) { // Customize commands // Success/failure commands - case ID_LAUNCH_BUTTON: - pBA->OnClickLaunchButton(); - return 0; - case ID_SUCCESS_RESTART_BUTTON: __fallthrough; case ID_FAILURE_RESTART_BUTTON: pBA->OnClickRestartButton(); @@ -2369,69 +2417,6 @@ } - // - // OnClickLaunchButton - launch the app from the success page. - // - void OnClickLaunchButton() { - HRESULT hr = S_OK; - LPWSTR sczUnformattedLaunchTarget = nullptr; - LPWSTR sczLaunchTarget = nullptr; - LPWSTR sczLaunchTargetElevatedId = nullptr; - LPWSTR sczUnformattedArguments = nullptr; - LPWSTR sczArguments = nullptr; - int nCmdShow = SW_SHOWNORMAL; - - hr = BalGetStringVariable(PYBA_VARIABLE_LAUNCH_TARGET_PATH, &sczUnformattedLaunchTarget); - BalExitOnFailure1(hr, "Failed to get launch target variable '%ls'.", PYBA_VARIABLE_LAUNCH_TARGET_PATH); - - hr = BalFormatString(sczUnformattedLaunchTarget, &sczLaunchTarget); - BalExitOnFailure1(hr, "Failed to format launch target variable: %ls", sczUnformattedLaunchTarget); - - if (BalStringVariableExists(PYBA_VARIABLE_LAUNCH_TARGET_ELEVATED_ID)) { - hr = BalGetStringVariable(PYBA_VARIABLE_LAUNCH_TARGET_ELEVATED_ID, &sczLaunchTargetElevatedId); - BalExitOnFailure1(hr, "Failed to get launch target elevated id '%ls'.", PYBA_VARIABLE_LAUNCH_TARGET_ELEVATED_ID); - } - - if (BalStringVariableExists(PYBA_VARIABLE_LAUNCH_ARGUMENTS)) { - hr = BalGetStringVariable(PYBA_VARIABLE_LAUNCH_ARGUMENTS, &sczUnformattedArguments); - BalExitOnFailure1(hr, "Failed to get launch arguments '%ls'.", PYBA_VARIABLE_LAUNCH_ARGUMENTS); - } - - if (BalStringVariableExists(PYBA_VARIABLE_LAUNCH_HIDDEN)) { - nCmdShow = SW_HIDE; - } - - if (sczLaunchTargetElevatedId && !_triedToLaunchElevated) { - _triedToLaunchElevated = TRUE; - hr = _engine->LaunchApprovedExe(_hWnd, sczLaunchTargetElevatedId, sczUnformattedArguments, 0); - if (FAILED(hr)) { - BalLogError(hr, "Failed to launch elevated target: %ls", sczLaunchTargetElevatedId); - - //try with ShelExec next time - OnClickLaunchButton(); - } - } else { - if (sczUnformattedArguments) { - hr = BalFormatString(sczUnformattedArguments, &sczArguments); - BalExitOnFailure1(hr, "Failed to format launch arguments variable: %ls", sczUnformattedArguments); - } - - hr = ShelExec(sczLaunchTarget, sczArguments, L"open", nullptr, nCmdShow, _hWnd, nullptr); - BalExitOnFailure1(hr, "Failed to launch target: %ls", sczLaunchTarget); - - ::PostMessageW(_hWnd, WM_CLOSE, 0, 0); - } - - LExit: - StrSecureZeroFreeString(sczArguments); - ReleaseStr(sczUnformattedArguments); - ReleaseStr(sczLaunchTargetElevatedId); - StrSecureZeroFreeString(sczLaunchTarget); - ReleaseStr(sczUnformattedLaunchTarget); - - return; - } - // // OnClickRestartButton - allows the restart and closes the app. @@ -3132,7 +3117,6 @@ _taskbarButtonCreatedMessage = UINT_MAX; _taskbarButtonOK = FALSE; _showingInternalUIThisPackage = FALSE; - _triedToLaunchElevated = FALSE; _suppressPaint = FALSE; @@ -3217,7 +3201,6 @@ UINT _taskbarButtonCreatedMessage; BOOL _taskbarButtonOK; BOOL _showingInternalUIThisPackage; - BOOL _triedToLaunchElevated; BOOL _suppressPaint; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:30:31 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 16:30:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_a_PY=5FLONG=5FLONG_str?= =?utf-8?q?aggler?= Message-ID: <20160908163030.87341.34119.0833FA1D@psf.io> https://hg.python.org/cpython/rev/7e3eaa25552b changeset: 103310:7e3eaa25552b user: Benjamin Peterson date: Thu Sep 08 09:25:03 2016 -0700 summary: fix a PY_LONG_LONG straggler files: Modules/_ctypes/ctypes.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -31,7 +31,7 @@ long l; float f; double d; - PY_LONG_LONG ll; + long long ll; long double D; }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:30:31 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 16:30:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_clinic=3A_PY=5FLONG=5FLONG?= =?utf-8?q?_-=3E_long_long?= Message-ID: <20160908163030.48632.32264.0EEF0291@psf.io> https://hg.python.org/cpython/rev/eab3422911e4 changeset: 103311:eab3422911e4 user: Benjamin Peterson date: Thu Sep 08 09:29:11 2016 -0700 summary: clinic: PY_LONG_LONG -> long long files: Doc/howto/clinic.rst | 4 +- Modules/_sha3/clinic/sha3module.c.h | 26 +++++--- Modules/clinic/sha512module.c.h | 50 +---------------- PC/clinic/msvcrtmodule.c.h | 10 +- PC/msvcrtmodule.c | 6 +- Tools/clinic/clinic.py | 10 +- 6 files changed, 32 insertions(+), 74 deletions(-) diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -827,9 +827,9 @@ ``'i'`` ``int`` ``'I'`` ``unsigned_int(bitwise=True)`` ``'k'`` ``unsigned_long(bitwise=True)`` -``'K'`` ``unsigned_PY_LONG_LONG(bitwise=True)`` +``'K'`` ``unsigned_long_long(bitwise=True)`` ``'l'`` ``long`` -``'L'`` ``PY_LONG_LONG`` +``'L'`` ``long long`` ``'n'`` ``Py_ssize_t`` ``'O'`` ``object`` ``'O!'`` ``object(subclass_of='&PySomething_Type')`` diff --git a/Modules/_sha3/clinic/sha3module.c.h b/Modules/_sha3/clinic/sha3module.c.h --- a/Modules/_sha3/clinic/sha3module.c.h +++ b/Modules/_sha3/clinic/sha3module.c.h @@ -15,12 +15,14 @@ py_sha3_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"string", NULL}; + static const char * const _keywords[] = {"string", NULL}; + static _PyArg_Parser _parser = {"|O:sha3_224", _keywords, 0}; PyObject *data = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O:sha3_224", _keywords, - &data)) + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &data)) { goto exit; + } return_value = py_sha3_new_impl(type, data); exit: @@ -106,12 +108,14 @@ _sha3_shake_128_digest(SHA3object *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"length", NULL}; + static const char * const _keywords[] = {"length", NULL}; + static _PyArg_Parser _parser = {"k:digest", _keywords, 0}; unsigned long length; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "k:digest", _keywords, - &length)) + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &length)) { goto exit; + } return_value = _sha3_shake_128_digest_impl(self, length); exit: @@ -134,15 +138,17 @@ _sha3_shake_128_hexdigest(SHA3object *self, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; - static char *_keywords[] = {"length", NULL}; + static const char * const _keywords[] = {"length", NULL}; + static _PyArg_Parser _parser = {"k:hexdigest", _keywords, 0}; unsigned long length; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "k:hexdigest", _keywords, - &length)) + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &length)) { goto exit; + } return_value = _sha3_shake_128_hexdigest_impl(self, length); exit: return return_value; } -/*[clinic end generated code: output=2eb6db41778eeb50 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=50cff05f2c74d41e input=a9049054013a1b77]*/ diff --git a/Modules/clinic/sha512module.c.h b/Modules/clinic/sha512module.c.h --- a/Modules/clinic/sha512module.c.h +++ b/Modules/clinic/sha512module.c.h @@ -2,8 +2,6 @@ preserve [clinic start generated code]*/ -#if defined(PY_LONG_LONG) - PyDoc_STRVAR(SHA512Type_copy__doc__, "copy($self, /)\n" "--\n" @@ -22,10 +20,6 @@ return SHA512Type_copy_impl(self); } -#endif /* defined(PY_LONG_LONG) */ - -#if defined(PY_LONG_LONG) - PyDoc_STRVAR(SHA512Type_digest__doc__, "digest($self, /)\n" "--\n" @@ -44,10 +38,6 @@ return SHA512Type_digest_impl(self); } -#endif /* defined(PY_LONG_LONG) */ - -#if defined(PY_LONG_LONG) - PyDoc_STRVAR(SHA512Type_hexdigest__doc__, "hexdigest($self, /)\n" "--\n" @@ -66,10 +56,6 @@ return SHA512Type_hexdigest_impl(self); } -#endif /* defined(PY_LONG_LONG) */ - -#if defined(PY_LONG_LONG) - PyDoc_STRVAR(SHA512Type_update__doc__, "update($self, obj, /)\n" "--\n" @@ -79,10 +65,6 @@ #define SHA512TYPE_UPDATE_METHODDEF \ {"update", (PyCFunction)SHA512Type_update, METH_O, SHA512Type_update__doc__}, -#endif /* defined(PY_LONG_LONG) */ - -#if defined(PY_LONG_LONG) - PyDoc_STRVAR(_sha512_sha512__doc__, "sha512($module, /, string=b\'\')\n" "--\n" @@ -113,10 +95,6 @@ return return_value; } -#endif /* defined(PY_LONG_LONG) */ - -#if defined(PY_LONG_LONG) - PyDoc_STRVAR(_sha512_sha384__doc__, "sha384($module, /, string=b\'\')\n" "--\n" @@ -146,30 +124,4 @@ exit: return return_value; } - -#endif /* defined(PY_LONG_LONG) */ - -#ifndef SHA512TYPE_COPY_METHODDEF - #define SHA512TYPE_COPY_METHODDEF -#endif /* !defined(SHA512TYPE_COPY_METHODDEF) */ - -#ifndef SHA512TYPE_DIGEST_METHODDEF - #define SHA512TYPE_DIGEST_METHODDEF -#endif /* !defined(SHA512TYPE_DIGEST_METHODDEF) */ - -#ifndef SHA512TYPE_HEXDIGEST_METHODDEF - #define SHA512TYPE_HEXDIGEST_METHODDEF -#endif /* !defined(SHA512TYPE_HEXDIGEST_METHODDEF) */ - -#ifndef SHA512TYPE_UPDATE_METHODDEF - #define SHA512TYPE_UPDATE_METHODDEF -#endif /* !defined(SHA512TYPE_UPDATE_METHODDEF) */ - -#ifndef _SHA512_SHA512_METHODDEF - #define _SHA512_SHA512_METHODDEF -#endif /* !defined(_SHA512_SHA512_METHODDEF) */ - -#ifndef _SHA512_SHA384_METHODDEF - #define _SHA512_SHA384_METHODDEF -#endif /* !defined(_SHA512_SHA384_METHODDEF) */ -/*[clinic end generated code: output=cf0da76cb603d1bf input=a9049054013a1b77]*/ +/*[clinic end generated code: output=8f7f6603a9c4e910 input=a9049054013a1b77]*/ diff --git a/PC/clinic/msvcrtmodule.c.h b/PC/clinic/msvcrtmodule.c.h --- a/PC/clinic/msvcrtmodule.c.h +++ b/PC/clinic/msvcrtmodule.c.h @@ -113,13 +113,13 @@ {"open_osfhandle", (PyCFunction)msvcrt_open_osfhandle, METH_VARARGS, msvcrt_open_osfhandle__doc__}, static long -msvcrt_open_osfhandle_impl(PyObject *module, Py_intptr_t handle, int flags); +msvcrt_open_osfhandle_impl(PyObject *module, intptr_t handle, int flags); static PyObject * msvcrt_open_osfhandle(PyObject *module, PyObject *args) { PyObject *return_value = NULL; - Py_intptr_t handle; + intptr_t handle; int flags; long _return_value; @@ -148,7 +148,7 @@ #define MSVCRT_GET_OSFHANDLE_METHODDEF \ {"get_osfhandle", (PyCFunction)msvcrt_get_osfhandle, METH_O, msvcrt_get_osfhandle__doc__}, -static Py_intptr_t +static intptr_t msvcrt_get_osfhandle_impl(PyObject *module, int fd); static PyObject * @@ -156,7 +156,7 @@ { PyObject *return_value = NULL; int fd; - Py_intptr_t _return_value; + intptr_t _return_value; if (!PyArg_Parse(arg, "i:get_osfhandle", &fd)) { goto exit; @@ -569,4 +569,4 @@ #ifndef MSVCRT_SET_ERROR_MODE_METHODDEF #define MSVCRT_SET_ERROR_MODE_METHODDEF #endif /* !defined(MSVCRT_SET_ERROR_MODE_METHODDEF) */ -/*[clinic end generated code: output=ece8106c0592ff1f input=a9049054013a1b77]*/ +/*[clinic end generated code: output=ae04e2b50eef8b63 input=a9049054013a1b77]*/ diff --git a/PC/msvcrtmodule.c b/PC/msvcrtmodule.c --- a/PC/msvcrtmodule.c +++ b/PC/msvcrtmodule.c @@ -59,7 +59,7 @@ data.return_conversion.append( 'return_value = PyUnicode_FromOrdinal(_return_value);\n') [python start generated code]*/ -/*[python end generated code: output=da39a3ee5e6b4b0d input=6a54fc4e73d0b367]*/ +/*[python end generated code: output=da39a3ee5e6b4b0d input=b59f1663dba11997]*/ /*[clinic input] module msvcrt @@ -161,7 +161,7 @@ static long msvcrt_open_osfhandle_impl(PyObject *module, intptr_t handle, int flags) -/*[clinic end generated code: output=bf65e422243a39f9 input=4d8516ed32db8f65]*/ +/*[clinic end generated code: output=cede871bf939d6e3 input=cb2108bbea84514e]*/ { int fd; @@ -185,7 +185,7 @@ static intptr_t msvcrt_get_osfhandle_impl(PyObject *module, int fd) -/*[clinic end generated code: output=eac47643338c0baa input=c7d18d02c8017ec1]*/ +/*[clinic end generated code: output=7ce761dd0de2b503 input=c7d18d02c8017ec1]*/ { intptr_t handle = -1; diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -2581,21 +2581,21 @@ if not bitwise: fail("Unsigned longs must be bitwise (for now).") -class PY_LONG_LONG_converter(CConverter): - type = 'PY_LONG_LONG' +class long_long_converter(CConverter): + type = 'long long' default_type = int format_unit = 'L' c_ignored_default = "0" -class unsigned_PY_LONG_LONG_converter(CConverter): - type = 'unsigned PY_LONG_LONG' +class unsigned_long_long_converter(CConverter): + type = 'unsigned long long' default_type = int format_unit = 'K' c_ignored_default = "0" def converter_init(self, *, bitwise=False): if not bitwise: - fail("Unsigned PY_LONG_LONGs must be bitwise (for now).") + fail("Unsigned long long must be bitwise (for now).") class Py_ssize_t_converter(CConverter): type = 'Py_ssize_t' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:41:57 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 08 Sep 2016 16:41:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327445=3A_Merge_from_3=2E5?= Message-ID: <20160908164156.87475.12595.0FBE3949@psf.io> https://hg.python.org/cpython/rev/c5cb8bd335ae changeset: 103313:c5cb8bd335ae parent: 103311:eab3422911e4 parent: 103312:4f6fef83cd0c user: Berker Peksag date: Thu Sep 08 19:42:11 2016 +0300 summary: Issue #27445: Merge from 3.5 files: Lib/email/mime/text.py | 4 +--- Lib/test/test_email/test_email.py | 5 ++++- Misc/NEWS | 3 +++ 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Lib/email/mime/text.py b/Lib/email/mime/text.py --- a/Lib/email/mime/text.py +++ b/Lib/email/mime/text.py @@ -35,10 +35,8 @@ _charset = 'us-ascii' except UnicodeEncodeError: _charset = 'utf-8' - if isinstance(_charset, Charset): - _charset = str(_charset) MIMENonMultipart.__init__(self, 'text', _subtype, policy=policy, - **{'charset': _charset}) + **{'charset': str(_charset)}) self.set_payload(_text, _charset) diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -1653,9 +1653,12 @@ eq(msg.get_charset().input_charset, 'us-ascii') eq(msg['content-type'], 'text/plain; charset="us-ascii"') # Also accept a Charset instance - msg = MIMEText('hello there', _charset=Charset('utf-8')) + charset = Charset('utf-8') + charset.body_encoding = None + msg = MIMEText('hello there', _charset=charset) eq(msg.get_charset().input_charset, 'utf-8') eq(msg['content-type'], 'text/plain; charset="utf-8"') + eq(msg.get_payload(), 'hello there') def test_7bit_input(self): eq = self.assertEqual diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -92,6 +92,9 @@ Library ------- +- Issue #27445: Don't pass str(_charset) to MIMEText.set_payload(). + Patch by Claude Paroz. + - Issue #24277: The new email API is no longer provisional, and the docs have been reorganized and rewritten to emphasize the new API. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:42:05 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 08 Sep 2016 16:42:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NDQ1?= =?utf-8?q?=3A_Don=27t_pass_str=28=5Fcharset=29_to_MIMEText=2Eset=5Fpayloa?= =?utf-8?b?ZCgp?= Message-ID: <20160908164155.3168.73834.BE7D6204@psf.io> https://hg.python.org/cpython/rev/4f6fef83cd0c changeset: 103312:4f6fef83cd0c branch: 3.5 parent: 103302:08f446bf45a7 user: Berker Peksag date: Thu Sep 08 19:40:30 2016 +0300 summary: Issue #27445: Don't pass str(_charset) to MIMEText.set_payload() Patch by Claude Paroz. files: Lib/email/mime/text.py | 4 +--- Lib/test/test_email/test_email.py | 5 ++++- Misc/NEWS | 3 +++ 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Lib/email/mime/text.py b/Lib/email/mime/text.py --- a/Lib/email/mime/text.py +++ b/Lib/email/mime/text.py @@ -35,10 +35,8 @@ _charset = 'us-ascii' except UnicodeEncodeError: _charset = 'utf-8' - if isinstance(_charset, Charset): - _charset = str(_charset) MIMENonMultipart.__init__(self, 'text', _subtype, - **{'charset': _charset}) + **{'charset': str(_charset)}) self.set_payload(_text, _charset) diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -1652,9 +1652,12 @@ eq(msg.get_charset().input_charset, 'us-ascii') eq(msg['content-type'], 'text/plain; charset="us-ascii"') # Also accept a Charset instance - msg = MIMEText('hello there', _charset=Charset('utf-8')) + charset = Charset('utf-8') + charset.body_encoding = None + msg = MIMEText('hello there', _charset=charset) eq(msg.get_charset().input_charset, 'utf-8') eq(msg['content-type'], 'text/plain; charset="utf-8"') + eq(msg.get_payload(), 'hello there') def test_7bit_input(self): eq = self.assertEqual diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -60,6 +60,9 @@ Library ------- +- Issue #27445: Don't pass str(_charset) to MIMEText.set_payload(). + Patch by Claude Paroz. + - lib2to3.pgen3.driver.load_grammar() now creates a stable cache file between runs given the same Grammar.txt input regardless of the hash randomization setting. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:44:47 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 08 Sep 2016 16:44:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_Py=5FMEMBER=5FSIZE_mac?= =?utf-8?q?ro?= Message-ID: <20160908164445.1800.99625.0B9B674B@psf.io> https://hg.python.org/cpython/rev/66a539984c9c changeset: 103315:66a539984c9c user: Victor Stinner date: Thu Sep 08 09:33:56 2016 -0700 summary: Add Py_MEMBER_SIZE macro Issue #27350: use Py_MEMBER_SIZE() macro to get the size of PyDictKeyEntry.dk_indices, rather than hardcoding 8. files: Include/pymacro.h | 3 +++ Objects/dictobject.c | 20 ++++++++++++-------- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/Include/pymacro.h b/Include/pymacro.h --- a/Include/pymacro.h +++ b/Include/pymacro.h @@ -18,6 +18,9 @@ by "__LINE__". */ #define Py_STRINGIFY(x) _Py_XSTRINGIFY(x) +/* Get the size of a structure member in bytes */ +#define Py_MEMBER_SIZE(type, member) sizeof(((type *)0)->member) + /* Argument must be a char or an int in [-128, 127] or [0, 255]. */ #define Py_CHARMASK(c) ((unsigned char)((c) & 0xff)) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -431,9 +431,10 @@ dk = keys_free_list[--numfreekeys]; } else { - dk = PyObject_MALLOC(sizeof(PyDictKeysObject) - 8 + - es * size + - sizeof(PyDictKeyEntry) * usable); + dk = PyObject_MALLOC(sizeof(PyDictKeysObject) + - Py_MEMBER_SIZE(PyDictKeysObject, dk_indices) + + es * size + + sizeof(PyDictKeyEntry) * usable); if (dk == NULL) { PyErr_NoMemory(); return NULL; @@ -2786,17 +2787,20 @@ /* If the dictionary is split, the keys portion is accounted-for in the type object. */ if (mp->ma_keys->dk_refcnt == 1) - res += sizeof(PyDictKeysObject) - 8 + DK_IXSIZE(mp->ma_keys) * size + - sizeof(PyDictKeyEntry) * usable; + res += (sizeof(PyDictKeysObject) + - Py_MEMBER_SIZE(PyDictKeysObject, dk_indices) + + DK_IXSIZE(mp->ma_keys) * size + + sizeof(PyDictKeyEntry) * usable); return res; } Py_ssize_t _PyDict_KeysSize(PyDictKeysObject *keys) { - return sizeof(PyDictKeysObject) - 8 - + DK_IXSIZE(keys) * DK_SIZE(keys) - + USABLE_FRACTION(DK_SIZE(keys)) * sizeof(PyDictKeyEntry); + return (sizeof(PyDictKeysObject) + - Py_MEMBER_SIZE(PyDictKeysObject, dk_indices) + + DK_IXSIZE(keys) * DK_SIZE(keys) + + USABLE_FRACTION(DK_SIZE(keys)) * sizeof(PyDictKeyEntry)); } static PyObject * -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:44:47 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 08 Sep 2016 16:44:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Implement_compact_dict?= Message-ID: <20160908164445.19517.6971.8980888D@psf.io> https://hg.python.org/cpython/rev/0bd618fe0639 changeset: 103314:0bd618fe0639 user: Victor Stinner date: Wed Sep 07 17:40:12 2016 -0700 summary: Implement compact dict Issue #27350: `dict` implementation is changed like PyPy. It is more compact and preserves insertion order. _PyDict_Dummy() function has been removed. Disable test_gdb: python-gdb.py is not updated yet to the new structure of compact dictionaries (issue #28023). Patch written by INADA Naoki. files: Doc/whatsnew/3.6.rst | 5 + Include/object.h | 1 - Lib/test/test_descr.py | 6 +- Lib/test/test_gdb.py | 3 + Lib/test/test_ordered_dict.py | 33 +- Lib/test/test_sys.py | 10 +- Lib/test/test_weakref.py | 7 +- Misc/NEWS | 3 + Objects/dict-common.h | 16 +- Objects/dictobject.c | 1287 +++++++++++--------- Objects/object.c | 6 - Objects/odictobject.c | 13 +- 12 files changed, 807 insertions(+), 583 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -343,6 +343,11 @@ Some smaller changes made to the core Python language are: +* `dict` implementation is changed like PyPy. It is more compact and preserves + insertion order. :pep:`PEP 468` (Preserving the order of `**kwargs` in a + function.) is implemented by this. + (Contributed by INADA Naoki in :issue:`27350`.) + * Long sequences of repeated traceback lines are now abbreviated as ``"[Previous line repeated {count} more times]"`` (see :ref:`py36-traceback` for an example). diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -710,7 +710,6 @@ PyAPI_DATA(Py_ssize_t) _Py_RefTotal; PyAPI_FUNC(void) _Py_NegativeRefcount(const char *fname, int lineno, PyObject *op); -PyAPI_FUNC(PyObject *) _PyDict_Dummy(void); PyAPI_FUNC(Py_ssize_t) _Py_GetRefTotal(void); #define _Py_INC_REFTOTAL _Py_RefTotal++ #define _Py_DEC_REFTOTAL _Py_RefTotal-- diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -5116,12 +5116,14 @@ a, b = A(), B() self.assertEqual(sys.getsizeof(vars(a)), sys.getsizeof(vars(b))) self.assertLess(sys.getsizeof(vars(a)), sys.getsizeof({})) - a.x, a.y, a.z, a.w = range(4) + # Initial hash table can contain at most 5 elements. + # Set 6 attributes to cause internal resizing. + a.x, a.y, a.z, a.w, a.v, a.u = range(6) self.assertNotEqual(sys.getsizeof(vars(a)), sys.getsizeof(vars(b))) a2 = A() self.assertEqual(sys.getsizeof(vars(a)), sys.getsizeof(vars(a2))) self.assertLess(sys.getsizeof(vars(a)), sys.getsizeof({})) - b.u, b.v, b.w, b.t = range(4) + b.u, b.v, b.w, b.t, b.s, b.r = range(6) self.assertLess(sys.getsizeof(vars(b)), sys.getsizeof({})) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -11,6 +11,9 @@ import unittest import locale +# FIXME: issue #28023 +raise unittest.SkipTest("FIXME: issue #28023, compact dict (issue #27350) broke python-gdb.py") + # Is this Python configured to support threads? try: import _thread diff --git a/Lib/test/test_ordered_dict.py b/Lib/test/test_ordered_dict.py --- a/Lib/test/test_ordered_dict.py +++ b/Lib/test/test_ordered_dict.py @@ -1,3 +1,4 @@ +import builtins import contextlib import copy import gc @@ -621,6 +622,25 @@ OrderedDict = py_coll.OrderedDict +class CPythonBuiltinDictTests(unittest.TestCase): + """Builtin dict preserves insertion order. + + Reuse some of tests in OrderedDict selectively. + """ + + module = builtins + OrderedDict = dict + +for method in ( + "test_init test_update test_abc test_clear test_delitem " + + "test_setitem test_detect_deletion_during_iteration " + + "test_popitem test_reinsert test_override_update " + + "test_highly_nested test_highly_nested_subclass " + + "test_delitem_hash_collision ").split(): + setattr(CPythonBuiltinDictTests, method, getattr(OrderedDictTests, method)) +del method + + @unittest.skipUnless(c_coll, 'requires the C version of the collections module') class CPythonOrderedDictTests(OrderedDictTests, unittest.TestCase): @@ -635,18 +655,19 @@ size = support.calcobjsize check = self.check_sizeof - basicsize = size('n2P' + '3PnPn2P') + calcsize('2nPn') - entrysize = calcsize('n2P') + calcsize('P') + basicsize = size('n2P' + '3PnPn2P') + calcsize('2nP2n') + entrysize = calcsize('n2P') + p = calcsize('P') nodesize = calcsize('Pn2P') od = OrderedDict() - check(od, basicsize + 8*entrysize) + check(od, basicsize + 8*p + 8 + 5*entrysize) # 8byte indicies + 8*2//3 * entry table od.x = 1 - check(od, basicsize + 8*entrysize) + check(od, basicsize + 8*p + 8 + 5*entrysize) od.update([(i, i) for i in range(3)]) - check(od, basicsize + 8*entrysize + 3*nodesize) + check(od, basicsize + 8*p + 8 + 5*entrysize + 3*nodesize) od.update([(i, i) for i in range(3, 10)]) - check(od, basicsize + 16*entrysize + 10*nodesize) + check(od, basicsize + 16*p + 16 + 10*entrysize + 10*nodesize) check(od.keys(), size('P')) check(od.items(), size('P')) diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -936,9 +936,9 @@ # method-wrapper (descriptor object) check({}.__iter__, size('2P')) # dict - check({}, size('n2P') + calcsize('2nPn') + 8*calcsize('n2P')) + check({}, size('n2P') + calcsize('2nP2n') + 8 + (8*2//3)*calcsize('n2P')) longdict = {1:1, 2:2, 3:3, 4:4, 5:5, 6:6, 7:7, 8:8} - check(longdict, size('n2P') + calcsize('2nPn') + 16*calcsize('n2P')) + check(longdict, size('n2P') + calcsize('2nP2n') + 16 + (16*2//3)*calcsize('n2P')) # dictionary-keyview check({}.keys(), size('P')) # dictionary-valueview @@ -1096,13 +1096,13 @@ '10P' # PySequenceMethods '2P' # PyBufferProcs '4P') - # Separate block for PyDictKeysObject with 4 entries - s += calcsize("2nPn") + 4*calcsize("n2P") + # Separate block for PyDictKeysObject with 8 keys and 5 entries + s += calcsize("2nP2n") + 8 + 5*calcsize("n2P") # class class newstyleclass(object): pass check(newstyleclass, s) # dict with shared keys - check(newstyleclass().__dict__, size('n2P' + '2nPn')) + check(newstyleclass().__dict__, size('n2P' + '2nP2n')) # unicode # each tuple contains a string and its expected character size # don't put any static strings here, as they may contain diff --git a/Lib/test/test_weakref.py b/Lib/test/test_weakref.py --- a/Lib/test/test_weakref.py +++ b/Lib/test/test_weakref.py @@ -1325,13 +1325,16 @@ o = Object(123456) with testcontext(): n = len(dict) - dict.popitem() + # Since underlaying dict is ordered, first item is popped + dict.pop(next(dict.keys())) self.assertEqual(len(dict), n - 1) dict[o] = o self.assertEqual(len(dict), n) + # last item in objects is removed from dict in context shutdown with testcontext(): self.assertEqual(len(dict), n - 1) - dict.pop(next(dict.keys())) + # Then, (o, o) is popped + dict.popitem() self.assertEqual(len(dict), n - 2) with testcontext(): self.assertEqual(len(dict), n - 3) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27350: `dict` implementation is changed like PyPy. It is more compact + and preserves insertion order. + - Issue #27911: Remove unnecessary error checks in ``exec_builtin_or_dynamic()``. diff --git a/Objects/dict-common.h b/Objects/dict-common.h --- a/Objects/dict-common.h +++ b/Objects/dict-common.h @@ -8,15 +8,25 @@ PyObject *me_value; /* This field is only meaningful for combined tables */ } PyDictKeyEntry; -typedef PyDictKeyEntry *(*dict_lookup_func) -(PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject ***value_addr); +/* dict_lookup_func() returns index of entry which can be used like DK_ENTRIES(dk)[index]. + * -1 when no entry found, -3 when compare raises error. + */ +typedef Py_ssize_t (*dict_lookup_func) +(PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject ***value_addr, + Py_ssize_t *hashpos); +#define DKIX_EMPTY (-1) +#define DKIX_DUMMY (-2) /* Used internally */ +#define DKIX_ERROR (-3) + +/* See dictobject.c for actual layout of DictKeysObject */ struct _dictkeysobject { Py_ssize_t dk_refcnt; Py_ssize_t dk_size; dict_lookup_func dk_lookup; Py_ssize_t dk_usable; - PyDictKeyEntry dk_entries[1]; + Py_ssize_t dk_nentries; /* How many entries are used. */ + char dk_indices[8]; /* dynamically sized. 8 is minimum. */ }; #endif diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -1,4 +1,3 @@ - /* Dictionary object implementation using a hash table */ /* The distribution includes a separate file, Objects/dictnotes.txt, @@ -7,64 +6,108 @@ tuning dictionaries, and several ideas for possible optimizations. */ +/* PyDictKeysObject + +This implements the dictionary's hashtable. + +As of Python 3.6, this is compact and orderd. Basic idea is described here. +https://morepypy.blogspot.jp/2015/01/faster-more-memory-efficient-and-more.html + +layout: + ++---------------+ +| dk_refcnt | +| dk_size | +| dk_lookup | +| dk_usable | +| dk_nentries | ++---------------+ +| dk_indices | +| | ++---------------+ +| dk_entries | +| | ++---------------+ + +dk_indices is actual hashtable. It holds index in entries, or DKIX_EMPTY(-1) +or DKIX_DUMMY(-2). +Size of indices is dk_size. Type of each index in indices is vary on dk_size: + +* int8 for dk_size <= 128 +* int16 for 256 <= dk_size <= 2**15 +* int32 for 2**16 <= dk_size <= 2**31 +* int64 for 2**32 <= dk_size + +dk_entries is array of PyDictKeyEntry. It's size is USABLE_FRACTION(dk_size). +DK_ENTRIES(dk) can be used to get pointer to entries. + +NOTE: Since negative value is used for DKIX_EMPTY and DKIX_DUMMY, type of +dk_indices entry is signed integer and int16 is used for table which +dk_size == 256. +*/ + /* -There are four kinds of slots in the table: - -1. Unused. me_key == me_value == NULL - Does not hold an active (key, value) pair now and never did. Unused can - transition to Active upon key insertion. This is the only case in which - me_key is NULL, and is each slot's initial state. - -2. Active. me_key != NULL and me_key != dummy and me_value != NULL - Holds an active (key, value) pair. Active can transition to Dummy or - Pending upon key deletion (for combined and split tables respectively). - This is the only case in which me_value != NULL. - -3. Dummy. me_key == dummy and me_value == NULL - Previously held an active (key, value) pair, but that was deleted and an - active pair has not yet overwritten the slot. Dummy can transition to - Active upon key insertion. Dummy slots cannot be made Unused again - (cannot have me_key set to NULL), else the probe sequence in case of - collision would have no way to know they were once active. - -4. Pending. Not yet inserted or deleted from a split-table. - key != NULL, key != dummy and value == NULL - The DictObject can be in one of two forms. + Either: A combined table: ma_values == NULL, dk_refcnt == 1. Values are stored in the me_value field of the PyDictKeysObject. - Slot kind 4 is not allowed i.e. - key != NULL, key != dummy and value == NULL is illegal. Or: A split table: ma_values != NULL, dk_refcnt >= 1 Values are stored in the ma_values array. - Only string (unicode) keys are allowed, no keys are present. - -Note: .popitem() abuses the me_hash field of an Unused or Dummy slot to -hold a search finger. The me_hash field of Unused or Dummy slots has no -meaning otherwise. As a consequence of this popitem always converts the dict -to the combined-table form. + Only string (unicode) keys are allowed. + All dicts sharing same key must have same insertion order. + +There are four kinds of slots in the table (slot is index, and +DK_ENTRIES(keys)[index] if index >= 0): + +1. Unused. index == DKIX_EMPTY + Does not hold an active (key, value) pair now and never did. Unused can + transition to Active upon key insertion. This is each slot's initial state. + +2. Active. index >= 0, me_key != NULL and me_value != NULL + Holds an active (key, value) pair. Active can transition to Dummy or + Pending upon key deletion (for combined and split tables respectively). + This is the only case in which me_value != NULL. + +3. Dummy. index == DKIX_DUMMY (combined only) + Previously held an active (key, value) pair, but that was deleted and an + active pair has not yet overwritten the slot. Dummy can transition to + Active upon key insertion. Dummy slots cannot be made Unused again + else the probe sequence in case of collision would have no way to know + they were once active. + +4. Pending. index >= 0, key != NULL, and value == NULL (split only) + Not yet inserted in split-table. */ -/* PyDict_MINSIZE_SPLIT is the minimum size of a split dictionary. - * It must be a power of 2, and at least 4. - * Resizing of split dictionaries is very rare, so the saving memory is more - * important than the cost of resizing. - */ -#define PyDict_MINSIZE_SPLIT 4 - -/* PyDict_MINSIZE_COMBINED is the starting size for any new, non-split dict. +/* +Preserving insertion order + +It's simple for combined table. Since dk_entries is mostly append only, we can +get insertion order by just iterating dk_entries. + +One exception is .popitem(). It removes last item in dk_entries and decrement +dk_nentries to achieve amortized O(1). Since there are DKIX_DUMMY remains in +dk_indices, we can't increment dk_usable even though dk_nentries is +decremented. + +In split table, inserting into pending entry is allowed only for dk_entries[ix] +where ix == mp->ma_used. Inserting into other index and deleting item cause +converting the dict to the combined table. +*/ + +/* PyDict_MINSIZE is the starting size for any new dict. * 8 allows dicts with no more than 5 active entries; experiments suggested * this suffices for the majority of dicts (consisting mostly of usually-small * dicts created to pass keyword arguments). * Making this 8, rather than 4 reduces the number of resizes for most * dictionaries, without any significant extra memory use. */ -#define PyDict_MINSIZE_COMBINED 8 +#define PyDict_MINSIZE 8 #include "Python.h" #include "dict-common.h" @@ -177,41 +220,31 @@ */ -/* Object used as dummy key to fill deleted entries - * This could be any unique object, - * use a custom type in order to minimise coupling. -*/ -static PyObject _dummy_struct; - -#define dummy (&_dummy_struct) - -#ifdef Py_REF_DEBUG -PyObject * -_PyDict_Dummy(void) -{ - return dummy; -} -#endif - /* forward declarations */ -static PyDictKeyEntry *lookdict(PyDictObject *mp, PyObject *key, - Py_hash_t hash, PyObject ***value_addr); -static PyDictKeyEntry *lookdict_unicode(PyDictObject *mp, PyObject *key, - Py_hash_t hash, PyObject ***value_addr); -static PyDictKeyEntry * +static Py_ssize_t lookdict(PyDictObject *mp, PyObject *key, + Py_hash_t hash, PyObject ***value_addr, + Py_ssize_t *hashpos); +static Py_ssize_t lookdict_unicode(PyDictObject *mp, PyObject *key, + Py_hash_t hash, PyObject ***value_addr, + Py_ssize_t *hashpos); +static Py_ssize_t lookdict_unicode_nodummy(PyDictObject *mp, PyObject *key, - Py_hash_t hash, PyObject ***value_addr); -static PyDictKeyEntry *lookdict_split(PyDictObject *mp, PyObject *key, - Py_hash_t hash, PyObject ***value_addr); + Py_hash_t hash, PyObject ***value_addr, + Py_ssize_t *hashpos); +static Py_ssize_t lookdict_split(PyDictObject *mp, PyObject *key, + Py_hash_t hash, PyObject ***value_addr, + Py_ssize_t *hashpos); static int dictresize(PyDictObject *mp, Py_ssize_t minused); -/* Dictionary reuse scheme to save calls to malloc, free, and memset */ +/* Dictionary reuse scheme to save calls to malloc and free */ #ifndef PyDict_MAXFREELIST #define PyDict_MAXFREELIST 80 #endif static PyDictObject *free_list[PyDict_MAXFREELIST]; static int numfree = 0; +static PyDictKeysObject *keys_free_list[PyDict_MAXFREELIST]; +static int numfreekeys = 0; #include "clinic/dictobject.c.h" @@ -219,12 +252,15 @@ PyDict_ClearFreeList(void) { PyDictObject *op; - int ret = numfree; + int ret = numfree + numfreekeys; while (numfree) { op = free_list[--numfree]; assert(PyDict_CheckExact(op)); PyObject_GC_Del(op); } + while (numfreekeys) { + PyObject_FREE(keys_free_list[--numfreekeys]); + } return ret; } @@ -243,40 +279,94 @@ PyDict_ClearFreeList(); } +#define DK_SIZE(dk) ((dk)->dk_size) +#if SIZEOF_VOID_P > 4 +#define DK_IXSIZE(dk) (DK_SIZE(dk) <= 0xff ? 1 : DK_SIZE(dk) <= 0xffff ? 2 : \ + DK_SIZE(dk) <= 0xffffffff ? 4 : sizeof(Py_ssize_t)) +#else +#define DK_IXSIZE(dk) (DK_SIZE(dk) <= 0xff ? 1 : DK_SIZE(dk) <= 0xffff ? 2 : \ + sizeof(Py_ssize_t)) +#endif +#define DK_ENTRIES(dk) ((PyDictKeyEntry*)(&(dk)->dk_indices[DK_SIZE(dk) * \ + DK_IXSIZE(dk)])) + #define DK_DEBUG_INCREF _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA #define DK_DEBUG_DECREF _Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA #define DK_INCREF(dk) (DK_DEBUG_INCREF ++(dk)->dk_refcnt) #define DK_DECREF(dk) if (DK_DEBUG_DECREF (--(dk)->dk_refcnt) == 0) free_keys_object(dk) -#define DK_SIZE(dk) ((dk)->dk_size) #define DK_MASK(dk) (((dk)->dk_size)-1) #define IS_POWER_OF_2(x) (((x) & (x-1)) == 0) + +/* lookup indices. returns DKIX_EMPTY, DKIX_DUMMY, or ix >=0 */ +Py_LOCAL_INLINE(Py_ssize_t) +dk_get_index(PyDictKeysObject *keys, Py_ssize_t i) +{ + Py_ssize_t s = DK_SIZE(keys); + if (s <= 0xff) { + return ((char*) &keys->dk_indices[0])[i]; + } + else if (s <= 0xffff) { + return ((int16_t*)&keys->dk_indices[0])[i]; + } +#if SIZEOF_VOID_P > 4 + else if (s <= 0xffffffff) { + return ((int32_t*)&keys->dk_indices[0])[i]; + } +#endif + else { + return ((Py_ssize_t*)&keys->dk_indices[0])[i]; + } +} + +/* write to indices. */ +Py_LOCAL_INLINE(void) +dk_set_index(PyDictKeysObject *keys, Py_ssize_t i, Py_ssize_t ix) +{ + Py_ssize_t s = DK_SIZE(keys); + if (s <= 0xff) { + ((char*) &keys->dk_indices[0])[i] = (char)ix; + } + else if (s <= 0xffff) { + ((int16_t*) &keys->dk_indices[0])[i] = (int16_t)ix; + } +#if SIZEOF_VOID_P > 4 + else if (s <= 0xffffffff) { + ((int32_t*) &keys->dk_indices[0])[i] = (int32_t)ix; + } +#endif + else { + ((Py_ssize_t*) &keys->dk_indices[0])[i] = ix; + } +} + + /* USABLE_FRACTION is the maximum dictionary load. - * Currently set to (2n+1)/3. Increasing this ratio makes dictionaries more - * dense resulting in more collisions. Decreasing it improves sparseness - * at the expense of spreading entries over more cache lines and at the - * cost of total memory consumed. + * Increasing this ratio makes dictionaries more dense resulting in more + * collisions. Decreasing it improves sparseness at the expense of spreading + * indices over more cache lines and at the cost of total memory consumed. * * USABLE_FRACTION must obey the following: * (0 < USABLE_FRACTION(n) < n) for all n >= 2 * - * USABLE_FRACTION should be very quick to calculate. - * Fractions around 5/8 to 2/3 seem to work well in practice. + * USABLE_FRACTION should be quick to calculate. + * Fractions around 1/2 to 2/3 seem to work well in practice. */ - -/* Use (2n+1)/3 rather than 2n+3 because: it makes no difference for - * combined tables (the two fractions round to the same number n < ), - * but 2*4/3 is 2 whereas (2*4+1)/3 is 3 which potentially saves quite - * a lot of space for small, split tables */ -#define USABLE_FRACTION(n) ((((n) << 1)+1)/3) - -/* Alternative fraction that is otherwise close enough to (2n+1)/3 to make +#define USABLE_FRACTION(n) (((n) << 1)/3) + +/* ESTIMATE_SIZE is reverse function of USABLE_FRACTION. + * This can be used to reserve enough size to insert n entries without + * resizing. + */ +#define ESTIMATE_SIZE(n) (((n)*3) >> 1) + +/* Alternative fraction that is otherwise close enough to 2n/3 to make * little difference. 8 * 2/3 == 8 * 5/8 == 5. 16 * 2/3 == 16 * 5/8 == 10. * 32 * 2/3 = 21, 32 * 5/8 = 20. * Its advantage is that it is faster to compute on machines with slow division. * #define USABLE_FRACTION(n) (((n) >> 1) + ((n) >> 2) - ((n) >> 3)) -*/ + */ /* GROWTH_RATE. Growth rate upon hitting maximum load. * Currently set to used*2 + capacity/2. @@ -304,9 +394,9 @@ 1, /* dk_size */ lookdict_split, /* dk_lookup */ 0, /* dk_usable (immutable) */ - { - { 0, 0, 0 } /* dk_entries (empty) */ - } + 0, /* dk_nentries */ + {DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, + DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY}, /* dk_indices */ }; static PyObject *empty_values[1] = { NULL }; @@ -316,45 +406,66 @@ static PyDictKeysObject *new_keys_object(Py_ssize_t size) { PyDictKeysObject *dk; - Py_ssize_t i; - PyDictKeyEntry *ep0; - - assert(size >= PyDict_MINSIZE_SPLIT); + Py_ssize_t es, usable; + + assert(size >= PyDict_MINSIZE); assert(IS_POWER_OF_2(size)); - dk = PyObject_MALLOC(sizeof(PyDictKeysObject) + - sizeof(PyDictKeyEntry) * (size-1)); - if (dk == NULL) { - PyErr_NoMemory(); - return NULL; + + usable = USABLE_FRACTION(size); + if (size <= 0xff) { + es = 1; + } + else if (size <= 0xffff) { + es = 2; + } +#if SIZEOF_VOID_P > 4 + else if (size <= 0xffffffff) { + es = 4; + } +#endif + else { + es = sizeof(Py_ssize_t); + } + + if (size == PyDict_MINSIZE && numfreekeys > 0) { + dk = keys_free_list[--numfreekeys]; + } + else { + dk = PyObject_MALLOC(sizeof(PyDictKeysObject) - 8 + + es * size + + sizeof(PyDictKeyEntry) * usable); + if (dk == NULL) { + PyErr_NoMemory(); + return NULL; + } } DK_DEBUG_INCREF dk->dk_refcnt = 1; dk->dk_size = size; - dk->dk_usable = USABLE_FRACTION(size); - ep0 = &dk->dk_entries[0]; - /* Hash value of slot 0 is used by popitem, so it must be initialized */ - ep0->me_hash = 0; - for (i = 0; i < size; i++) { - ep0[i].me_key = NULL; - ep0[i].me_value = NULL; - } + dk->dk_usable = usable; dk->dk_lookup = lookdict_unicode_nodummy; + dk->dk_nentries = 0; + memset(&dk->dk_indices[0], 0xff, es * size); + memset(DK_ENTRIES(dk), 0, sizeof(PyDictKeyEntry) * usable); return dk; } static void free_keys_object(PyDictKeysObject *keys) { - PyDictKeyEntry *entries = &keys->dk_entries[0]; + PyDictKeyEntry *entries = DK_ENTRIES(keys); Py_ssize_t i, n; - for (i = 0, n = DK_SIZE(keys); i < n; i++) { + for (i = 0, n = keys->dk_nentries; i < n; i++) { Py_XDECREF(entries[i].me_key); Py_XDECREF(entries[i].me_value); } + if (keys->dk_size == PyDict_MINSIZE && numfreekeys < PyDict_MAXFREELIST) { + keys_free_list[numfreekeys++] = keys; + return; + } PyObject_FREE(keys); } #define new_values(size) PyMem_NEW(PyObject *, size) - #define free_values(values) PyMem_FREE(values) /* Consumes a reference to the keys object */ @@ -390,7 +501,7 @@ PyObject **values; Py_ssize_t i, size; - size = DK_SIZE(keys); + size = USABLE_FRACTION(DK_SIZE(keys)); values = new_values(size); if (values == NULL) { DK_DECREF(keys); @@ -405,12 +516,43 @@ PyObject * PyDict_New(void) { - PyDictKeysObject *keys = new_keys_object(PyDict_MINSIZE_COMBINED); + PyDictKeysObject *keys = new_keys_object(PyDict_MINSIZE); if (keys == NULL) return NULL; return new_dict(keys, NULL); } +/* Search index of hash table from offset of entry table */ +static Py_ssize_t +lookdict_index(PyDictKeysObject *k, Py_hash_t hash, Py_ssize_t index) +{ + size_t i, perturb; + size_t mask = DK_MASK(k); + Py_ssize_t ix; + + i = (size_t)hash & mask; + ix = dk_get_index(k, i); + if (ix == index) { + return i; + } + if (ix == DKIX_EMPTY) { + return DKIX_EMPTY; + } + + for (perturb = hash; ; perturb >>= PERTURB_SHIFT) { + i = mask & ((i << 2) + i + perturb + 1); + ix = dk_get_index(k, i); + if (ix == index) { + return i; + } + if (ix == DKIX_EMPTY) { + return DKIX_EMPTY; + } + } + assert(0); /* NOT REACHED */ + return DKIX_ERROR; +} + /* The basic lookup function used by all operations. This is based on Algorithm D from Knuth Vol. 3, Sec. 6.4. @@ -426,52 +568,63 @@ contributions by Reimer Behrends, Jyrki Alakuijala, Vladimir Marangozov and Christian Tismer. -lookdict() is general-purpose, and may return NULL if (and only if) a +lookdict() is general-purpose, and may return DKIX_ERROR if (and only if) a comparison raises an exception (this was new in Python 2.5). lookdict_unicode() below is specialized to string keys, comparison of which can -never raise an exception; that function can never return NULL. +never raise an exception; that function can never return DKIX_ERROR. lookdict_unicode_nodummy is further specialized for string keys that cannot be the value. -For both, when the key isn't found a PyDictEntry* is returned -where the key would have been found, *value_addr points to the matching value -slot. +For both, when the key isn't found a DKIX_EMPTY is returned. hashpos returns +where the key index should be inserted. */ -static PyDictKeyEntry * +static Py_ssize_t lookdict(PyDictObject *mp, PyObject *key, - Py_hash_t hash, PyObject ***value_addr) + Py_hash_t hash, PyObject ***value_addr, Py_ssize_t *hashpos) { - size_t i; - size_t perturb; - PyDictKeyEntry *freeslot; - size_t mask; - PyDictKeyEntry *ep0; - PyDictKeyEntry *ep; + size_t i, perturb, mask; + Py_ssize_t ix, freeslot; int cmp; + PyDictKeysObject *dk; + PyDictKeyEntry *ep0, *ep; PyObject *startkey; top: - mask = DK_MASK(mp->ma_keys); - ep0 = &mp->ma_keys->dk_entries[0]; + dk = mp->ma_keys; + mask = DK_MASK(dk); + ep0 = DK_ENTRIES(dk); i = (size_t)hash & mask; - ep = &ep0[i]; - if (ep->me_key == NULL || ep->me_key == key) { - *value_addr = &ep->me_value; - return ep; + + ix = dk_get_index(dk, i); + if (ix == DKIX_EMPTY) { + if (hashpos != NULL) + *hashpos = i; + *value_addr = NULL; + return DKIX_EMPTY; } - if (ep->me_key == dummy) - freeslot = ep; + if (ix == DKIX_DUMMY) { + freeslot = i; + } else { - if (ep->me_hash == hash) { + ep = &ep0[ix]; + if (ep->me_key == key) { + *value_addr = &ep->me_value; + if (hashpos != NULL) + *hashpos = i; + return ix; + } + if (ep->me_key != NULL && ep->me_hash == hash) { startkey = ep->me_key; Py_INCREF(startkey); cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); Py_DECREF(startkey); if (cmp < 0) - return NULL; - if (ep0 == mp->ma_keys->dk_entries && ep->me_key == startkey) { + return DKIX_ERROR; + if (dk == mp->ma_keys && ep->me_key == startkey) { if (cmp > 0) { *value_addr = &ep->me_value; - return ep; + if (hashpos != NULL) + *hashpos = i; + return ix; } } else { @@ -479,40 +632,48 @@ goto top; } } - freeslot = NULL; + freeslot = -1; } - /* In the loop, me_key == dummy is by far (factor of 100s) the - least likely outcome, so test for that last. */ for (perturb = hash; ; perturb >>= PERTURB_SHIFT) { - i = (i << 2) + i + perturb + 1; - ep = &ep0[i & mask]; - if (ep->me_key == NULL) { - if (freeslot == NULL) { - *value_addr = &ep->me_value; - return ep; - } else { - *value_addr = &freeslot->me_value; - return freeslot; + i = ((i << 2) + i + perturb + 1) & mask; + ix = dk_get_index(dk, i); + if (ix == DKIX_EMPTY) { + if (hashpos != NULL) { + *hashpos = (freeslot == -1) ? (Py_ssize_t)i : freeslot; } + *value_addr = NULL; + return ix; } + if (ix == DKIX_DUMMY) { + if (freeslot == -1) + freeslot = i; + continue; + } + ep = &ep0[ix]; if (ep->me_key == key) { + if (hashpos != NULL) { + *hashpos = i; + } *value_addr = &ep->me_value; - return ep; + return ix; } - if (ep->me_hash == hash && ep->me_key != dummy) { + if (ep->me_hash == hash && ep->me_key != NULL) { startkey = ep->me_key; Py_INCREF(startkey); cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); Py_DECREF(startkey); if (cmp < 0) { *value_addr = NULL; - return NULL; + return DKIX_ERROR; } - if (ep0 == mp->ma_keys->dk_entries && ep->me_key == startkey) { + if (dk == mp->ma_keys && ep->me_key == startkey) { if (cmp > 0) { + if (hashpos != NULL) { + *hashpos = i; + } *value_addr = &ep->me_value; - return ep; + return ix; } } else { @@ -520,72 +681,80 @@ goto top; } } - else if (ep->me_key == dummy && freeslot == NULL) - freeslot = ep; } assert(0); /* NOT REACHED */ return 0; } /* Specialized version for string-only keys */ -static PyDictKeyEntry * +static Py_ssize_t lookdict_unicode(PyDictObject *mp, PyObject *key, - Py_hash_t hash, PyObject ***value_addr) + Py_hash_t hash, PyObject ***value_addr, Py_ssize_t *hashpos) { - size_t i; - size_t perturb; - PyDictKeyEntry *freeslot; + size_t i, perturb; size_t mask = DK_MASK(mp->ma_keys); - PyDictKeyEntry *ep0 = &mp->ma_keys->dk_entries[0]; - PyDictKeyEntry *ep; - + Py_ssize_t ix, freeslot; + PyDictKeyEntry *ep, *ep0 = DK_ENTRIES(mp->ma_keys); + + assert(mp->ma_values == NULL); /* Make sure this function doesn't have to handle non-unicode keys, including subclasses of str; e.g., one reason to subclass unicodes is to override __eq__, and for speed we don't cater to that here. */ if (!PyUnicode_CheckExact(key)) { mp->ma_keys->dk_lookup = lookdict; - return lookdict(mp, key, hash, value_addr); + return lookdict(mp, key, hash, value_addr, hashpos); } i = (size_t)hash & mask; - ep = &ep0[i]; - if (ep->me_key == NULL || ep->me_key == key) { - *value_addr = &ep->me_value; - return ep; + ix = dk_get_index(mp->ma_keys, i); + if (ix == DKIX_EMPTY) { + if (hashpos != NULL) + *hashpos = i; + *value_addr = NULL; + return DKIX_EMPTY; } - if (ep->me_key == dummy) - freeslot = ep; + if (ix == DKIX_DUMMY) { + freeslot = i; + } else { - if (ep->me_hash == hash && unicode_eq(ep->me_key, key)) { + ep = &ep0[ix]; + /* only split table can be ix != DKIX_DUMMY && me_key == NULL */ + assert(ep->me_key != NULL); + if (ep->me_key == key || (ep->me_hash == hash && unicode_eq(ep->me_key, key))) { + if (hashpos != NULL) + *hashpos = i; *value_addr = &ep->me_value; - return ep; + return ix; } - freeslot = NULL; + freeslot = -1; } - /* In the loop, me_key == dummy is by far (factor of 100s) the - least likely outcome, so test for that last. */ for (perturb = hash; ; perturb >>= PERTURB_SHIFT) { - i = (i << 2) + i + perturb + 1; - ep = &ep0[i & mask]; - if (ep->me_key == NULL) { - if (freeslot == NULL) { - *value_addr = &ep->me_value; - return ep; - } else { - *value_addr = &freeslot->me_value; - return freeslot; + i = mask & ((i << 2) + i + perturb + 1); + ix = dk_get_index(mp->ma_keys, i); + if (ix == DKIX_EMPTY) { + if (hashpos != NULL) { + *hashpos = (freeslot == -1) ? (Py_ssize_t)i : freeslot; } + *value_addr = NULL; + return DKIX_EMPTY; } + if (ix == DKIX_DUMMY) { + if (freeslot == -1) + freeslot = i; + continue; + } + ep = &ep0[ix]; if (ep->me_key == key || (ep->me_hash == hash - && ep->me_key != dummy - && unicode_eq(ep->me_key, key))) { + && ep->me_key != NULL + && unicode_eq(ep->me_key, key))) { *value_addr = &ep->me_value; - return ep; + if (hashpos != NULL) { + *hashpos = i; + } + return ix; } - if (ep->me_key == dummy && freeslot == NULL) - freeslot = ep; } assert(0); /* NOT REACHED */ return 0; @@ -593,40 +762,61 @@ /* Faster version of lookdict_unicode when it is known that no keys * will be present. */ -static PyDictKeyEntry * +static Py_ssize_t lookdict_unicode_nodummy(PyDictObject *mp, PyObject *key, - Py_hash_t hash, PyObject ***value_addr) + Py_hash_t hash, PyObject ***value_addr, + Py_ssize_t *hashpos) { - size_t i; - size_t perturb; + size_t i, perturb; size_t mask = DK_MASK(mp->ma_keys); - PyDictKeyEntry *ep0 = &mp->ma_keys->dk_entries[0]; - PyDictKeyEntry *ep; - + Py_ssize_t ix; + PyDictKeyEntry *ep, *ep0 = DK_ENTRIES(mp->ma_keys); + + assert(mp->ma_values == NULL); /* Make sure this function doesn't have to handle non-unicode keys, including subclasses of str; e.g., one reason to subclass unicodes is to override __eq__, and for speed we don't cater to that here. */ if (!PyUnicode_CheckExact(key)) { mp->ma_keys->dk_lookup = lookdict; - return lookdict(mp, key, hash, value_addr); + return lookdict(mp, key, hash, value_addr, hashpos); } i = (size_t)hash & mask; - ep = &ep0[i]; - assert(ep->me_key == NULL || PyUnicode_CheckExact(ep->me_key)); - if (ep->me_key == NULL || ep->me_key == key || + ix = dk_get_index(mp->ma_keys, i); + assert (ix != DKIX_DUMMY); + if (ix == DKIX_EMPTY) { + if (hashpos != NULL) + *hashpos = i; + *value_addr = NULL; + return DKIX_EMPTY; + } + ep = &ep0[ix]; + assert(ep->me_key != NULL && PyUnicode_CheckExact(ep->me_key)); + if (ep->me_key == key || (ep->me_hash == hash && unicode_eq(ep->me_key, key))) { + if (hashpos != NULL) + *hashpos = i; *value_addr = &ep->me_value; - return ep; + return ix; } for (perturb = hash; ; perturb >>= PERTURB_SHIFT) { - i = (i << 2) + i + perturb + 1; - ep = &ep0[i & mask]; - assert(ep->me_key == NULL || PyUnicode_CheckExact(ep->me_key)); - if (ep->me_key == NULL || ep->me_key == key || + i = mask & ((i << 2) + i + perturb + 1); + ix = dk_get_index(mp->ma_keys, i); + assert (ix != DKIX_DUMMY); + if (ix == DKIX_EMPTY) { + if (hashpos != NULL) + *hashpos = i; + *value_addr = NULL; + return DKIX_EMPTY; + } + ep = &ep0[ix]; + assert(ep->me_key != NULL && PyUnicode_CheckExact(ep->me_key)); + if (ep->me_key == key || (ep->me_hash == hash && unicode_eq(ep->me_key, key))) { + if (hashpos != NULL) + *hashpos = i; *value_addr = &ep->me_value; - return ep; + return ix; } } assert(0); /* NOT REACHED */ @@ -638,39 +828,61 @@ * Split tables only contain unicode keys and no dummy keys, * so algorithm is the same as lookdict_unicode_nodummy. */ -static PyDictKeyEntry * +static Py_ssize_t lookdict_split(PyDictObject *mp, PyObject *key, - Py_hash_t hash, PyObject ***value_addr) + Py_hash_t hash, PyObject ***value_addr, Py_ssize_t *hashpos) { - size_t i; - size_t perturb; + size_t i, perturb; size_t mask = DK_MASK(mp->ma_keys); - PyDictKeyEntry *ep0 = &mp->ma_keys->dk_entries[0]; - PyDictKeyEntry *ep; - + Py_ssize_t ix; + PyDictKeyEntry *ep, *ep0 = DK_ENTRIES(mp->ma_keys); + + /* mp must split table */ + assert(mp->ma_values != NULL); if (!PyUnicode_CheckExact(key)) { - ep = lookdict(mp, key, hash, value_addr); - /* lookdict expects a combined-table, so fix value_addr */ - i = ep - ep0; - *value_addr = &mp->ma_values[i]; - return ep; + ix = lookdict(mp, key, hash, value_addr, hashpos); + if (ix >= 0) { + *value_addr = &mp->ma_values[ix]; + } + return ix; } + i = (size_t)hash & mask; - ep = &ep0[i]; + ix = dk_get_index(mp->ma_keys, i); + if (ix == DKIX_EMPTY) { + if (hashpos != NULL) + *hashpos = i; + *value_addr = NULL; + return DKIX_EMPTY; + } + assert(ix >= 0); + ep = &ep0[ix]; assert(ep->me_key == NULL || PyUnicode_CheckExact(ep->me_key)); - if (ep->me_key == NULL || ep->me_key == key || + if (ep->me_key == key || (ep->me_hash == hash && unicode_eq(ep->me_key, key))) { - *value_addr = &mp->ma_values[i]; - return ep; + if (hashpos != NULL) + *hashpos = i; + *value_addr = &mp->ma_values[ix]; + return ix; } for (perturb = hash; ; perturb >>= PERTURB_SHIFT) { - i = (i << 2) + i + perturb + 1; - ep = &ep0[i & mask]; + i = mask & ((i << 2) + i + perturb + 1); + ix = dk_get_index(mp->ma_keys, i); + if (ix == DKIX_EMPTY) { + if (hashpos != NULL) + *hashpos = i; + *value_addr = NULL; + return DKIX_EMPTY; + } + assert(ix >= 0); + ep = &ep0[ix]; assert(ep->me_key == NULL || PyUnicode_CheckExact(ep->me_key)); - if (ep->me_key == NULL || ep->me_key == key || + if (ep->me_key == key || (ep->me_hash == hash && unicode_eq(ep->me_key, key))) { - *value_addr = &mp->ma_values[i & mask]; - return ep; + if (hashpos != NULL) + *hashpos = i; + *value_addr = &mp->ma_values[ix]; + return ix; } } assert(0); /* NOT REACHED */ @@ -707,27 +919,27 @@ { PyDictObject *mp; PyObject *value; - Py_ssize_t i, size; + Py_ssize_t i, numentries; + PyDictKeyEntry *ep0; if (!PyDict_CheckExact(op) || !_PyObject_GC_IS_TRACKED(op)) return; mp = (PyDictObject *) op; - size = DK_SIZE(mp->ma_keys); + ep0 = DK_ENTRIES(mp->ma_keys); + numentries = mp->ma_keys->dk_nentries; if (_PyDict_HasSplitTable(mp)) { - for (i = 0; i < size; i++) { + for (i = 0; i < numentries; i++) { if ((value = mp->ma_values[i]) == NULL) continue; if (_PyObject_GC_MAY_BE_TRACKED(value)) { - assert(!_PyObject_GC_MAY_BE_TRACKED( - mp->ma_keys->dk_entries[i].me_key)); + assert(!_PyObject_GC_MAY_BE_TRACKED(ep0[i].me_key)); return; } } } else { - PyDictKeyEntry *ep0 = &mp->ma_keys->dk_entries[0]; - for (i = 0; i < size; i++) { + for (i = 0; i < numentries; i++) { if ((value = ep0[i].me_value) == NULL) continue; if (_PyObject_GC_MAY_BE_TRACKED(value) || @@ -741,31 +953,33 @@ /* Internal function to find slot for an item from its hash * when it is known that the key is not present in the dict. */ -static PyDictKeyEntry * +static Py_ssize_t find_empty_slot(PyDictObject *mp, PyObject *key, Py_hash_t hash, - PyObject ***value_addr) + PyObject ***value_addr, Py_ssize_t *hashpos) { - size_t i; - size_t perturb; + size_t i, perturb; size_t mask = DK_MASK(mp->ma_keys); - PyDictKeyEntry *ep0 = &mp->ma_keys->dk_entries[0]; - PyDictKeyEntry *ep; - + Py_ssize_t ix; + PyDictKeyEntry *ep, *ep0 = DK_ENTRIES(mp->ma_keys); + + assert(hashpos != NULL); assert(key != NULL); if (!PyUnicode_CheckExact(key)) mp->ma_keys->dk_lookup = lookdict; i = hash & mask; - ep = &ep0[i]; - for (perturb = hash; ep->me_key != NULL; perturb >>= PERTURB_SHIFT) { + ix = dk_get_index(mp->ma_keys, i); + for (perturb = hash; ix != DKIX_EMPTY; perturb >>= PERTURB_SHIFT) { i = (i << 2) + i + perturb + 1; - ep = &ep0[i & mask]; + ix = dk_get_index(mp->ma_keys, i & mask); } + ep = &ep0[mp->ma_keys->dk_nentries]; + *hashpos = i & mask; assert(ep->me_value == NULL); if (mp->ma_values) - *value_addr = &mp->ma_values[i & mask]; + *value_addr = &mp->ma_values[ix]; else *value_addr = &ep->me_value; - return ep; + return ix; } static int @@ -784,58 +998,81 @@ { PyObject *old_value; PyObject **value_addr; - PyDictKeyEntry *ep; - assert(key != dummy); + PyDictKeyEntry *ep, *ep0; + Py_ssize_t hashpos, ix; if (mp->ma_values != NULL && !PyUnicode_CheckExact(key)) { if (insertion_resize(mp) < 0) return -1; } - ep = mp->ma_keys->dk_lookup(mp, key, hash, &value_addr); - if (ep == NULL) { + ix = mp->ma_keys->dk_lookup(mp, key, hash, &value_addr, &hashpos); + if (ix == DKIX_ERROR) { return -1; } + assert(PyUnicode_CheckExact(key) || mp->ma_keys->dk_lookup == lookdict); Py_INCREF(value); MAINTAIN_TRACKING(mp, key, value); + + /* When insertion order is different from shared key, we can't share + * the key anymore. Convert this instance to combine table. + */ + if (_PyDict_HasSplitTable(mp) && + ((ix >= 0 && *value_addr == NULL && mp->ma_used != ix) || + (ix == DKIX_EMPTY && mp->ma_used != mp->ma_keys->dk_nentries))) { + if (insertion_resize(mp) < 0) { + Py_DECREF(value); + return -1; + } + find_empty_slot(mp, key, hash, &value_addr, &hashpos); + ix = DKIX_EMPTY; + } + + if (ix == DKIX_EMPTY) { + /* Insert into new slot. */ + if (mp->ma_keys->dk_usable <= 0) { + /* Need to resize. */ + if (insertion_resize(mp) < 0) { + Py_DECREF(value); + return -1; + } + find_empty_slot(mp, key, hash, &value_addr, &hashpos); + } + ep0 = DK_ENTRIES(mp->ma_keys); + ep = &ep0[mp->ma_keys->dk_nentries]; + dk_set_index(mp->ma_keys, hashpos, mp->ma_keys->dk_nentries); + Py_INCREF(key); + ep->me_key = key; + ep->me_hash = hash; + if (mp->ma_values) { + assert (mp->ma_values[mp->ma_keys->dk_nentries] == NULL); + mp->ma_values[mp->ma_keys->dk_nentries] = value; + } + else { + ep->me_value = value; + } + mp->ma_used++; + mp->ma_keys->dk_usable--; + mp->ma_keys->dk_nentries++; + assert(mp->ma_keys->dk_usable >= 0); + return 0; + } + + assert(value_addr != NULL); + old_value = *value_addr; if (old_value != NULL) { - assert(ep->me_key != NULL && ep->me_key != dummy); *value_addr = value; Py_DECREF(old_value); /* which **CAN** re-enter (see issue #22653) */ + return 0; } - else { - if (ep->me_key == NULL) { - Py_INCREF(key); - if (mp->ma_keys->dk_usable <= 0) { - /* Need to resize. */ - if (insertion_resize(mp) < 0) { - Py_DECREF(key); - Py_DECREF(value); - return -1; - } - ep = find_empty_slot(mp, key, hash, &value_addr); - } - mp->ma_keys->dk_usable--; - assert(mp->ma_keys->dk_usable >= 0); - ep->me_key = key; - ep->me_hash = hash; - } - else { - if (ep->me_key == dummy) { - Py_INCREF(key); - ep->me_key = key; - ep->me_hash = hash; - Py_DECREF(dummy); - } else { - assert(_PyDict_HasSplitTable(mp)); - } - } - mp->ma_used++; - *value_addr = value; - assert(ep->me_key != NULL && ep->me_key != dummy); - } + + /* pending state */ + assert(_PyDict_HasSplitTable(mp)); + assert(ix == mp->ma_used); + *value_addr = value; + mp->ma_used++; return 0; } @@ -853,25 +1090,25 @@ insertdict_clean(PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject *value) { - size_t i; - size_t perturb; + size_t i, perturb; PyDictKeysObject *k = mp->ma_keys; size_t mask = (size_t)DK_SIZE(k)-1; - PyDictKeyEntry *ep0 = &k->dk_entries[0]; + PyDictKeyEntry *ep0 = DK_ENTRIES(mp->ma_keys); PyDictKeyEntry *ep; assert(k->dk_lookup != NULL); assert(value != NULL); assert(key != NULL); - assert(key != dummy); assert(PyUnicode_CheckExact(key) || k->dk_lookup == lookdict); i = hash & mask; - ep = &ep0[i]; - for (perturb = hash; ep->me_key != NULL; perturb >>= PERTURB_SHIFT) { - i = (i << 2) + i + perturb + 1; - ep = &ep0[i & mask]; + for (perturb = hash; dk_get_index(k, i) != DKIX_EMPTY; + perturb >>= PERTURB_SHIFT) { + i = mask & ((i << 2) + i + perturb + 1); } + ep = &ep0[k->dk_nentries]; assert(ep->me_value == NULL); + dk_set_index(k, i, k->dk_nentries); + k->dk_nentries++; ep->me_key = key; ep->me_hash = hash; ep->me_value = value; @@ -890,13 +1127,13 @@ static int dictresize(PyDictObject *mp, Py_ssize_t minused) { - Py_ssize_t newsize; + Py_ssize_t i, newsize; PyDictKeysObject *oldkeys; PyObject **oldvalues; - Py_ssize_t i, oldsize; - -/* Find the smallest table size > minused. */ - for (newsize = PyDict_MINSIZE_COMBINED; + PyDictKeyEntry *ep0; + + /* Find the smallest table size > minused. */ + for (newsize = PyDict_MINSIZE; newsize <= minused && newsize > 0; newsize <<= 1) ; @@ -914,52 +1151,39 @@ } if (oldkeys->dk_lookup == lookdict) mp->ma_keys->dk_lookup = lookdict; - oldsize = DK_SIZE(oldkeys); mp->ma_values = NULL; - /* If empty then nothing to copy so just return */ - if (oldsize == 1) { - assert(oldkeys == Py_EMPTY_KEYS); - DK_DECREF(oldkeys); - return 0; - } + ep0 = DK_ENTRIES(oldkeys); /* Main loop below assumes we can transfer refcount to new keys * and that value is stored in me_value. * Increment ref-counts and copy values here to compensate * This (resizing a split table) should be relatively rare */ if (oldvalues != NULL) { - for (i = 0; i < oldsize; i++) { + for (i = 0; i < oldkeys->dk_nentries; i++) { if (oldvalues[i] != NULL) { - Py_INCREF(oldkeys->dk_entries[i].me_key); - oldkeys->dk_entries[i].me_value = oldvalues[i]; + Py_INCREF(ep0[i].me_key); + ep0[i].me_value = oldvalues[i]; } } } /* Main loop */ - for (i = 0; i < oldsize; i++) { - PyDictKeyEntry *ep = &oldkeys->dk_entries[i]; + for (i = 0; i < oldkeys->dk_nentries; i++) { + PyDictKeyEntry *ep = &ep0[i]; if (ep->me_value != NULL) { - assert(ep->me_key != dummy); insertdict_clean(mp, ep->me_key, ep->me_hash, ep->me_value); } } mp->ma_keys->dk_usable -= mp->ma_used; if (oldvalues != NULL) { /* NULL out me_value slot in oldkeys, in case it was shared */ - for (i = 0; i < oldsize; i++) - oldkeys->dk_entries[i].me_value = NULL; - assert(oldvalues != empty_values); - free_values(oldvalues); + for (i = 0; i < oldkeys->dk_nentries; i++) + ep0[i].me_value = NULL; DK_DECREF(oldkeys); + if (oldvalues != empty_values) { + free_values(oldvalues); + } } else { assert(oldkeys->dk_lookup != lookdict_split); - if (oldkeys->dk_lookup != lookdict_unicode_nodummy) { - PyDictKeyEntry *ep0 = &oldkeys->dk_entries[0]; - for (i = 0; i < oldsize; i++) { - if (ep0[i].me_key == dummy) - Py_DECREF(dummy); - } - } assert(oldkeys->dk_refcnt == 1); DK_DEBUG_DECREF PyObject_FREE(oldkeys); } @@ -991,8 +1215,8 @@ } assert(mp->ma_keys->dk_lookup == lookdict_unicode_nodummy); /* Copy values into a new array */ - ep0 = &mp->ma_keys->dk_entries[0]; - size = DK_SIZE(mp->ma_keys); + ep0 = DK_ENTRIES(mp->ma_keys); + size = USABLE_FRACTION(DK_SIZE(mp->ma_keys)); values = new_values(size); if (values == NULL) { PyErr_SetString(PyExc_MemoryError, @@ -1015,7 +1239,7 @@ { Py_ssize_t newsize; PyDictKeysObject *new_keys; - for (newsize = PyDict_MINSIZE_COMBINED; + for (newsize = PyDict_MINSIZE; newsize <= minused && newsize > 0; newsize <<= 1) ; @@ -1039,8 +1263,8 @@ PyDict_GetItem(PyObject *op, PyObject *key) { Py_hash_t hash; + Py_ssize_t ix; PyDictObject *mp = (PyDictObject *)op; - PyDictKeyEntry *ep; PyThreadState *tstate; PyObject **value_addr; @@ -1066,15 +1290,15 @@ /* preserve the existing exception */ PyObject *err_type, *err_value, *err_tb; PyErr_Fetch(&err_type, &err_value, &err_tb); - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); /* ignore errors */ PyErr_Restore(err_type, err_value, err_tb); - if (ep == NULL) + if (ix < 0) return NULL; } else { - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) { + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); + if (ix < 0) { PyErr_Clear(); return NULL; } @@ -1085,8 +1309,8 @@ PyObject * _PyDict_GetItem_KnownHash(PyObject *op, PyObject *key, Py_hash_t hash) { + Py_ssize_t ix; PyDictObject *mp = (PyDictObject *)op; - PyDictKeyEntry *ep; PyThreadState *tstate; PyObject **value_addr; @@ -1103,15 +1327,15 @@ /* preserve the existing exception */ PyObject *err_type, *err_value, *err_tb; PyErr_Fetch(&err_type, &err_value, &err_tb); - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); /* ignore errors */ PyErr_Restore(err_type, err_value, err_tb); - if (ep == NULL) + if (ix == DKIX_EMPTY) return NULL; } else { - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) { + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); + if (ix == DKIX_EMPTY) { PyErr_Clear(); return NULL; } @@ -1126,9 +1350,9 @@ PyObject * PyDict_GetItemWithError(PyObject *op, PyObject *key) { + Py_ssize_t ix; Py_hash_t hash; PyDictObject*mp = (PyDictObject *)op; - PyDictKeyEntry *ep; PyObject **value_addr; if (!PyDict_Check(op)) { @@ -1144,8 +1368,8 @@ } } - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); + if (ix < 0) return NULL; return *value_addr; } @@ -1170,10 +1394,9 @@ PyObject * _PyDict_LoadGlobal(PyDictObject *globals, PyDictObject *builtins, PyObject *key) { + Py_ssize_t ix; Py_hash_t hash; - PyDictKeyEntry *entry; PyObject **value_addr; - PyObject *value; if (!PyUnicode_CheckExact(key) || (hash = ((PyASCIIObject *) key)->hash) == -1) @@ -1184,16 +1407,15 @@ } /* namespace 1: globals */ - entry = globals->ma_keys->dk_lookup(globals, key, hash, &value_addr); - if (entry == NULL) + ix = globals->ma_keys->dk_lookup(globals, key, hash, &value_addr, NULL); + if (ix == DKIX_ERROR) return NULL; - value = *value_addr; - if (value != NULL) - return value; + if (ix != DKIX_EMPTY && *value_addr != NULL) + return *value_addr; /* namespace 2: builtins */ - entry = builtins->ma_keys->dk_lookup(builtins, key, hash, &value_addr); - if (entry == NULL) + ix = builtins->ma_keys->dk_lookup(builtins, key, hash, &value_addr, NULL); + if (ix < 0) return NULL; return *value_addr; } @@ -1250,16 +1472,8 @@ int PyDict_DelItem(PyObject *op, PyObject *key) { - PyDictObject *mp; Py_hash_t hash; - PyDictKeyEntry *ep; - PyObject *old_key, *old_value; - PyObject **value_addr; - - if (!PyDict_Check(op)) { - PyErr_BadInternalCall(); - return -1; - } + assert(key); if (!PyUnicode_CheckExact(key) || (hash = ((PyASCIIObject *) key)->hash) == -1) { @@ -1267,31 +1481,14 @@ if (hash == -1) return -1; } - mp = (PyDictObject *)op; - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) - return -1; - if (*value_addr == NULL) { - _PyErr_SetKeyError(key); - return -1; - } - old_value = *value_addr; - *value_addr = NULL; - mp->ma_used--; - if (!_PyDict_HasSplitTable(mp)) { - ENSURE_ALLOWS_DELETIONS(mp); - old_key = ep->me_key; - Py_INCREF(dummy); - ep->me_key = dummy; - Py_DECREF(old_key); - } - Py_DECREF(old_value); - return 0; + + return _PyDict_DelItem_KnownHash(op, key, hash); } int _PyDict_DelItem_KnownHash(PyObject *op, PyObject *key, Py_hash_t hash) { + Py_ssize_t hashpos, ix; PyDictObject *mp; PyDictKeyEntry *ep; PyObject *old_key, *old_value; @@ -1304,21 +1501,26 @@ assert(key); assert(hash != -1); mp = (PyDictObject *)op; - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, &hashpos); + if (ix == DKIX_ERROR) return -1; - if (*value_addr == NULL) { + if (ix == DKIX_EMPTY || *value_addr == NULL) { _PyErr_SetKeyError(key); return -1; } + assert(dk_get_index(mp->ma_keys, hashpos) == ix); old_value = *value_addr; *value_addr = NULL; mp->ma_used--; - if (!_PyDict_HasSplitTable(mp)) { + if (_PyDict_HasSplitTable(mp)) { + mp->ma_keys->dk_usable = 0; + } + else { + ep = &DK_ENTRIES(mp->ma_keys)[ix]; + dk_set_index(mp->ma_keys, hashpos, DKIX_DUMMY); ENSURE_ALLOWS_DELETIONS(mp); old_key = ep->me_key; - Py_INCREF(dummy); - ep->me_key = dummy; + ep->me_key = NULL; Py_DECREF(old_key); } Py_DECREF(old_value); @@ -1347,7 +1549,7 @@ mp->ma_used = 0; /* ...then clear the keys and values */ if (oldvalues != NULL) { - n = DK_SIZE(oldkeys); + n = oldkeys->dk_nentries; for (i = 0; i < n; i++) Py_CLEAR(oldvalues[i]); free_values(oldvalues); @@ -1365,30 +1567,33 @@ Py_LOCAL_INLINE(Py_ssize_t) dict_next(PyObject *op, Py_ssize_t i, PyObject **pvalue) { - Py_ssize_t mask, offset; + Py_ssize_t n; PyDictObject *mp; - PyObject **value_ptr; - + PyObject **value_ptr = NULL; if (!PyDict_Check(op)) return -1; mp = (PyDictObject *)op; if (i < 0) return -1; + + n = mp->ma_keys->dk_nentries; if (mp->ma_values) { - value_ptr = &mp->ma_values[i]; - offset = sizeof(PyObject *); + for (; i < n; i++) { + value_ptr = &mp->ma_values[i]; + if (*value_ptr != NULL) + break; + } } else { - value_ptr = &mp->ma_keys->dk_entries[i].me_value; - offset = sizeof(PyDictKeyEntry); + PyDictKeyEntry *ep0 = DK_ENTRIES(mp->ma_keys); + for (; i < n; i++) { + value_ptr = &ep0[i].me_value; + if (*value_ptr != NULL) + break; + } } - mask = DK_MASK(mp->ma_keys); - while (i <= mask && *value_ptr == NULL) { - value_ptr = (PyObject **)(((char *)value_ptr) + offset); - i++; - } - if (i > mask) + if (i >= n) return -1; if (pvalue) *pvalue = *value_ptr; @@ -1413,14 +1618,14 @@ int PyDict_Next(PyObject *op, Py_ssize_t *ppos, PyObject **pkey, PyObject **pvalue) { - PyDictObject *mp; + PyDictObject *mp = (PyDictObject*)op; Py_ssize_t i = dict_next(op, *ppos, pvalue); if (i < 0) return 0; mp = (PyDictObject *)op; *ppos = i+1; if (pkey) - *pkey = mp->ma_keys->dk_entries[i].me_key; + *pkey = DK_ENTRIES(mp->ma_keys)[i].me_key; return 1; } @@ -1432,14 +1637,16 @@ PyObject **pvalue, Py_hash_t *phash) { PyDictObject *mp; + PyDictKeyEntry *ep0; Py_ssize_t i = dict_next(op, *ppos, pvalue); if (i < 0) return 0; mp = (PyDictObject *)op; + ep0 = DK_ENTRIES(mp->ma_keys); *ppos = i+1; - *phash = mp->ma_keys->dk_entries[i].me_hash; + *phash = ep0[i].me_hash; if (pkey) - *pkey = mp->ma_keys->dk_entries[i].me_key; + *pkey = ep0[i].me_key; return 1; } @@ -1448,6 +1655,7 @@ _PyDict_Pop(PyDictObject *mp, PyObject *key, PyObject *deflt) { Py_hash_t hash; + Py_ssize_t ix, hashpos; PyObject *old_value, *old_key; PyDictKeyEntry *ep; PyObject **value_addr; @@ -1466,11 +1674,10 @@ if (hash == -1) return NULL; } - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, &hashpos); + if (ix == DKIX_ERROR) return NULL; - old_value = *value_addr; - if (old_value == NULL) { + if (ix == DKIX_EMPTY) { if (deflt) { Py_INCREF(deflt); return deflt; @@ -1478,13 +1685,15 @@ _PyErr_SetKeyError(key); return NULL; } + old_value = *value_addr; *value_addr = NULL; mp->ma_used--; if (!_PyDict_HasSplitTable(mp)) { + dk_set_index(mp->ma_keys, hashpos, DKIX_DUMMY); + ep = &DK_ENTRIES(mp->ma_keys)[ix]; ENSURE_ALLOWS_DELETIONS(mp); old_key = ep->me_key; - Py_INCREF(dummy); - ep->me_key = dummy; + ep->me_key = NULL; Py_DECREF(old_key); } return old_value; @@ -1511,7 +1720,7 @@ PyObject *key; Py_hash_t hash; - if (dictresize(mp, Py_SIZE(iterable))) { + if (dictresize(mp, ESTIMATE_SIZE(Py_SIZE(iterable)))) { Py_DECREF(d); return NULL; } @@ -1530,7 +1739,7 @@ PyObject *key; Py_hash_t hash; - if (dictresize(mp, PySet_GET_SIZE(iterable))) { + if (dictresize(mp, ESTIMATE_SIZE(PySet_GET_SIZE(iterable)))) { Py_DECREF(d); return NULL; } @@ -1590,7 +1799,7 @@ Py_TRASHCAN_SAFE_BEGIN(mp) if (values != NULL) { if (values != empty_values) { - for (i = 0, n = DK_SIZE(mp->ma_keys); i < n; i++) { + for (i = 0, n = mp->ma_keys->dk_nentries; i < n; i++) { Py_XDECREF(values[i]); } free_values(values); @@ -1702,8 +1911,8 @@ dict_subscript(PyDictObject *mp, PyObject *key) { PyObject *v; + Py_ssize_t ix; Py_hash_t hash; - PyDictKeyEntry *ep; PyObject **value_addr; if (!PyUnicode_CheckExact(key) || @@ -1712,11 +1921,10 @@ if (hash == -1) return NULL; } - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); + if (ix == DKIX_ERROR) return NULL; - v = *value_addr; - if (v == NULL) { + if (ix == DKIX_EMPTY || *value_addr == NULL) { if (!PyDict_CheckExact(mp)) { /* Look up __missing__ method if we're a subclass. */ PyObject *missing, *res; @@ -1734,8 +1942,8 @@ _PyErr_SetKeyError(key); return NULL; } - else - Py_INCREF(v); + v = *value_addr; + Py_INCREF(v); return v; } @@ -1775,8 +1983,8 @@ Py_DECREF(v); goto again; } - ep = &mp->ma_keys->dk_entries[0]; - size = DK_SIZE(mp->ma_keys); + ep = DK_ENTRIES(mp->ma_keys); + size = mp->ma_keys->dk_nentries; if (mp->ma_values) { value_ptr = mp->ma_values; offset = sizeof(PyObject *); @@ -1818,13 +2026,13 @@ Py_DECREF(v); goto again; } - size = DK_SIZE(mp->ma_keys); + size = mp->ma_keys->dk_nentries; if (mp->ma_values) { value_ptr = mp->ma_values; offset = sizeof(PyObject *); } else { - value_ptr = &mp->ma_keys->dk_entries[0].me_value; + value_ptr = &(DK_ENTRIES(mp->ma_keys)[0].me_value); offset = sizeof(PyDictKeyEntry); } for (i = 0, j = 0; i < size; i++) { @@ -1875,8 +2083,8 @@ goto again; } /* Nothing we do below makes any function calls. */ - ep = mp->ma_keys->dk_entries; - size = DK_SIZE(mp->ma_keys); + ep = DK_ENTRIES(mp->ma_keys); + size = mp->ma_keys->dk_nentries; if (mp->ma_values) { value_ptr = mp->ma_values; offset = sizeof(PyObject *); @@ -1920,7 +2128,8 @@ } static int -dict_update_common(PyObject *self, PyObject *args, PyObject *kwds, const char *methname) +dict_update_common(PyObject *self, PyObject *args, PyObject *kwds, + const char *methname) { PyObject *arg = NULL; int result = 0; @@ -2043,7 +2252,7 @@ { PyDictObject *mp, *other; Py_ssize_t i, n; - PyDictKeyEntry *entry; + PyDictKeyEntry *entry, *ep0; /* We accept for the argument either a concrete dictionary object, * or an abstract "mapping" object. For the former, we can do @@ -2073,10 +2282,11 @@ if (mp->ma_keys->dk_usable * 3 < other->ma_used * 2) if (dictresize(mp, (mp->ma_used + other->ma_used)*2) != 0) return -1; - for (i = 0, n = DK_SIZE(other->ma_keys); i < n; i++) { + ep0 = DK_ENTRIES(other->ma_keys); + for (i = 0, n = other->ma_keys->dk_nentries; i < n; i++) { PyObject *key, *value; Py_hash_t hash; - entry = &other->ma_keys->dk_entries[i]; + entry = &ep0[i]; key = entry->me_key; hash = entry->me_hash; if (other->ma_values) @@ -2095,7 +2305,7 @@ if (err != 0) return -1; - if (n != DK_SIZE(other->ma_keys)) { + if (n != other->ma_keys->dk_nentries) { PyErr_SetString(PyExc_RuntimeError, "dict mutated during update"); return -1; @@ -2170,7 +2380,9 @@ mp = (PyDictObject *)o; if (_PyDict_HasSplitTable(mp)) { PyDictObject *split_copy; - PyObject **newvalues = new_values(DK_SIZE(mp->ma_keys)); + Py_ssize_t size = USABLE_FRACTION(DK_SIZE(mp->ma_keys)); + PyObject **newvalues; + newvalues = new_values(size); if (newvalues == NULL) return PyErr_NoMemory(); split_copy = PyObject_GC_New(PyDictObject, &PyDict_Type); @@ -2182,7 +2394,7 @@ split_copy->ma_keys = mp->ma_keys; split_copy->ma_used = mp->ma_used; DK_INCREF(mp->ma_keys); - for (i = 0, n = DK_SIZE(mp->ma_keys); i < n; i++) { + for (i = 0, n = size; i < n; i++) { PyObject *value = mp->ma_values[i]; Py_XINCREF(value); split_copy->ma_values[i] = value; @@ -2253,8 +2465,8 @@ /* can't be equal if # of entries differ */ return 0; /* Same # of entries -- check all of 'em. Exit early on any diff. */ - for (i = 0; i < DK_SIZE(a->ma_keys); i++) { - PyDictKeyEntry *ep = &a->ma_keys->dk_entries[i]; + for (i = 0; i < a->ma_keys->dk_nentries; i++) { + PyDictKeyEntry *ep = &DK_ENTRIES(a->ma_keys)[i]; PyObject *aval; if (a->ma_values) aval = a->ma_values[i]; @@ -2271,7 +2483,7 @@ /* ditto for key */ Py_INCREF(key); /* reuse the known hash value */ - if ((b->ma_keys->dk_lookup)(b, key, ep->me_hash, &vaddr) == NULL) + if ((b->ma_keys->dk_lookup)(b, key, ep->me_hash, &vaddr, NULL) < 0) bval = NULL; else bval = *vaddr; @@ -2329,7 +2541,7 @@ { register PyDictObject *mp = self; Py_hash_t hash; - PyDictKeyEntry *ep; + Py_ssize_t ix; PyObject **value_addr; if (!PyUnicode_CheckExact(key) || @@ -2338,10 +2550,12 @@ if (hash == -1) return NULL; } - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); + if (ix == DKIX_ERROR) return NULL; - return PyBool_FromLong(*value_addr != NULL); + if (ix == DKIX_EMPTY || *value_addr == NULL) + Py_RETURN_FALSE; + Py_RETURN_TRUE; } static PyObject * @@ -2351,7 +2565,7 @@ PyObject *failobj = Py_None; PyObject *val = NULL; Py_hash_t hash; - PyDictKeyEntry *ep; + Py_ssize_t ix; PyObject **value_addr; if (!PyArg_UnpackTuple(args, "get", 1, 2, &key, &failobj)) @@ -2363,12 +2577,13 @@ if (hash == -1) return NULL; } - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); + if (ix == DKIX_ERROR) return NULL; - val = *value_addr; - if (val == NULL) + if (ix == DKIX_EMPTY || *value_addr == NULL) val = failobj; + else + val = *value_addr; Py_INCREF(val); return val; } @@ -2379,6 +2594,7 @@ PyDictObject *mp = (PyDictObject *)d; PyObject *val = NULL; Py_hash_t hash; + Py_ssize_t hashpos, ix; PyDictKeyEntry *ep; PyObject **value_addr; @@ -2392,27 +2608,37 @@ if (hash == -1) return NULL; } - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - if (ep == NULL) + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, &hashpos); + if (ix == DKIX_ERROR) return NULL; - val = *value_addr; - if (val == NULL) { + if (ix == DKIX_EMPTY || *value_addr == NULL) { + val = defaultobj; if (mp->ma_keys->dk_usable <= 0) { /* Need to resize. */ if (insertion_resize(mp) < 0) return NULL; - ep = find_empty_slot(mp, key, hash, &value_addr); + find_empty_slot(mp, key, hash, &value_addr, &hashpos); } + ix = mp->ma_keys->dk_nentries; Py_INCREF(defaultobj); Py_INCREF(key); MAINTAIN_TRACKING(mp, key, defaultobj); + dk_set_index(mp->ma_keys, hashpos, ix); + ep = &DK_ENTRIES(mp->ma_keys)[ix]; ep->me_key = key; ep->me_hash = hash; - *value_addr = defaultobj; - val = defaultobj; + if (mp->ma_values) { + mp->ma_values[ix] = val; + } + else { + ep->me_value = val; + } mp->ma_keys->dk_usable--; + mp->ma_keys->dk_nentries++; mp->ma_used++; } + else + val = *value_addr; return val; } @@ -2451,11 +2677,10 @@ static PyObject * dict_popitem(PyDictObject *mp) { - Py_hash_t i = 0; - PyDictKeyEntry *ep; + Py_ssize_t i, j; + PyDictKeyEntry *ep0, *ep; PyObject *res; - /* Allocate the result tuple before checking the size. Believe it * or not, this allocation could trigger a garbage collection which * could empty the dict, so if we checked the size first and that @@ -2482,37 +2707,28 @@ } } ENSURE_ALLOWS_DELETIONS(mp); - /* Set ep to "the first" dict entry with a value. We abuse the hash - * field of slot 0 to hold a search finger: - * If slot 0 has a value, use slot 0. - * Else slot 0 is being used to hold a search finger, - * and we use its hash value as the first index to look. - */ - ep = &mp->ma_keys->dk_entries[0]; - if (ep->me_value == NULL) { - Py_ssize_t mask = DK_MASK(mp->ma_keys); - i = ep->me_hash; - /* The hash field may be a real hash value, or it may be a - * legit search finger, or it may be a once-legit search - * finger that's out of bounds now because it wrapped around - * or the table shrunk -- simply make sure it's in bounds now. - */ - if (i > mask || i < 1) - i = 1; /* skip slot 0 */ - while ((ep = &mp->ma_keys->dk_entries[i])->me_value == NULL) { - i++; - if (i > mask) - i = 1; - } + + /* Pop last item */ + ep0 = DK_ENTRIES(mp->ma_keys); + i = mp->ma_keys->dk_nentries - 1; + while (i >= 0 && ep0[i].me_value == NULL) { + i--; } + assert(i >= 0); + + ep = &ep0[i]; + j = lookdict_index(mp->ma_keys, ep->me_hash, i); + assert(j >= 0); + assert(dk_get_index(mp->ma_keys, j) == i); + dk_set_index(mp->ma_keys, j, DKIX_DUMMY); + PyTuple_SET_ITEM(res, 0, ep->me_key); PyTuple_SET_ITEM(res, 1, ep->me_value); - Py_INCREF(dummy); - ep->me_key = dummy; + ep->me_key = NULL; ep->me_value = NULL; + /* We can't dk_usable++ since there is DKIX_DUMMY in indices */ + mp->ma_keys->dk_nentries = i; mp->ma_used--; - assert(mp->ma_keys->dk_entries[0].me_value == NULL); - mp->ma_keys->dk_entries[0].me_hash = i + 1; /* next place to start */ return res; } @@ -2521,8 +2737,9 @@ { PyDictObject *mp = (PyDictObject *)op; PyDictKeysObject *keys = mp->ma_keys; - PyDictKeyEntry *entries = &keys->dk_entries[0]; - Py_ssize_t i, n = DK_SIZE(mp->ma_keys); + PyDictKeyEntry *entries = DK_ENTRIES(mp->ma_keys); + Py_ssize_t i, n = keys->dk_nentries; + if (keys->dk_lookup == lookdict) { for (i = 0; i < n; i++) { if (entries[i].me_value != NULL) { @@ -2530,7 +2747,8 @@ Py_VISIT(entries[i].me_key); } } - } else { + } + else { if (mp->ma_values != NULL) { for (i = 0; i < n; i++) { Py_VISIT(mp->ma_values[i]); @@ -2557,23 +2775,28 @@ Py_ssize_t _PyDict_SizeOf(PyDictObject *mp) { - Py_ssize_t size, res; + Py_ssize_t size, usable, res; size = DK_SIZE(mp->ma_keys); + usable = USABLE_FRACTION(size); + res = _PyObject_SIZE(Py_TYPE(mp)); if (mp->ma_values) - res += size * sizeof(PyObject*); + res += usable * sizeof(PyObject*); /* If the dictionary is split, the keys portion is accounted-for in the type object. */ if (mp->ma_keys->dk_refcnt == 1) - res += sizeof(PyDictKeysObject) + (size-1) * sizeof(PyDictKeyEntry); + res += sizeof(PyDictKeysObject) - 8 + DK_IXSIZE(mp->ma_keys) * size + + sizeof(PyDictKeyEntry) * usable; return res; } Py_ssize_t _PyDict_KeysSize(PyDictKeysObject *keys) { - return sizeof(PyDictKeysObject) + (DK_SIZE(keys)-1) * sizeof(PyDictKeyEntry); + return sizeof(PyDictKeysObject) - 8 + + DK_IXSIZE(keys) * DK_SIZE(keys) + + USABLE_FRACTION(DK_SIZE(keys)) * sizeof(PyDictKeyEntry); } static PyObject * @@ -2660,8 +2883,8 @@ PyDict_Contains(PyObject *op, PyObject *key) { Py_hash_t hash; + Py_ssize_t ix; PyDictObject *mp = (PyDictObject *)op; - PyDictKeyEntry *ep; PyObject **value_addr; if (!PyUnicode_CheckExact(key) || @@ -2670,8 +2893,10 @@ if (hash == -1) return -1; } - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - return (ep == NULL) ? -1 : (*value_addr != NULL); + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); + if (ix == DKIX_ERROR) + return -1; + return (ix != DKIX_EMPTY && *value_addr != NULL); } /* Internal version of PyDict_Contains used when the hash value is already known */ @@ -2679,11 +2904,13 @@ _PyDict_Contains(PyObject *op, PyObject *key, Py_hash_t hash) { PyDictObject *mp = (PyDictObject *)op; - PyDictKeyEntry *ep; PyObject **value_addr; - - ep = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr); - return (ep == NULL) ? -1 : (*value_addr != NULL); + Py_ssize_t ix; + + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, NULL); + if (ix == DKIX_ERROR) + return -1; + return (ix != DKIX_EMPTY && *value_addr != NULL); } /* Hack to implement "key in dict" */ @@ -2717,7 +2944,7 @@ _PyObject_GC_UNTRACK(d); d->ma_used = 0; - d->ma_keys = new_keys_object(PyDict_MINSIZE_COMBINED); + d->ma_keys = new_keys_object(PyDict_MINSIZE); if (d->ma_keys == NULL) { Py_DECREF(self); return NULL; @@ -2945,7 +3172,7 @@ static PyObject *dictiter_iternextkey(dictiterobject *di) { PyObject *key; - Py_ssize_t i, mask, offset; + Py_ssize_t i, n, offset; PyDictKeysObject *k; PyDictObject *d = di->di_dict; PyObject **value_ptr; @@ -2970,19 +3197,19 @@ offset = sizeof(PyObject *); } else { - value_ptr = &k->dk_entries[i].me_value; + value_ptr = &DK_ENTRIES(k)[i].me_value; offset = sizeof(PyDictKeyEntry); } - mask = DK_SIZE(k)-1; - while (i <= mask && *value_ptr == NULL) { + n = k->dk_nentries - 1; + while (i <= n && *value_ptr == NULL) { value_ptr = (PyObject **)(((char *)value_ptr) + offset); i++; } di->di_pos = i+1; - if (i > mask) + if (i > n) goto fail; di->len--; - key = k->dk_entries[i].me_key; + key = DK_ENTRIES(k)[i].me_key; Py_INCREF(key); return key; @@ -3028,7 +3255,7 @@ static PyObject *dictiter_iternextvalue(dictiterobject *di) { PyObject *value; - Py_ssize_t i, mask, offset; + Py_ssize_t i, n, offset; PyDictObject *d = di->di_dict; PyObject **value_ptr; @@ -3044,21 +3271,21 @@ } i = di->di_pos; - mask = DK_SIZE(d->ma_keys)-1; - if (i < 0 || i > mask) + n = d->ma_keys->dk_nentries - 1; + if (i < 0 || i > n) goto fail; if (d->ma_values) { value_ptr = &d->ma_values[i]; offset = sizeof(PyObject *); } else { - value_ptr = &d->ma_keys->dk_entries[i].me_value; + value_ptr = &DK_ENTRIES(d->ma_keys)[i].me_value; offset = sizeof(PyDictKeyEntry); } - while (i <= mask && *value_ptr == NULL) { + while (i <= n && *value_ptr == NULL) { value_ptr = (PyObject **)(((char *)value_ptr) + offset); i++; - if (i > mask) + if (i > n) goto fail; } di->di_pos = i+1; @@ -3109,7 +3336,7 @@ static PyObject *dictiter_iternextitem(dictiterobject *di) { PyObject *key, *value, *result = di->di_result; - Py_ssize_t i, mask, offset; + Py_ssize_t i, n, offset; PyDictObject *d = di->di_dict; PyObject **value_ptr; @@ -3127,21 +3354,21 @@ i = di->di_pos; if (i < 0) goto fail; - mask = DK_SIZE(d->ma_keys)-1; + n = d->ma_keys->dk_nentries - 1; if (d->ma_values) { value_ptr = &d->ma_values[i]; offset = sizeof(PyObject *); } else { - value_ptr = &d->ma_keys->dk_entries[i].me_value; + value_ptr = &DK_ENTRIES(d->ma_keys)[i].me_value; offset = sizeof(PyDictKeyEntry); } - while (i <= mask && *value_ptr == NULL) { + while (i <= n && *value_ptr == NULL) { value_ptr = (PyObject **)(((char *)value_ptr) + offset); i++; } di->di_pos = i+1; - if (i > mask) + if (i > n) goto fail; if (result->ob_refcnt == 1) { @@ -3154,7 +3381,7 @@ return NULL; } di->len--; - key = d->ma_keys->dk_entries[i].me_key; + key = DK_ENTRIES(d->ma_keys)[i].me_key; value = *value_ptr; Py_INCREF(key); Py_INCREF(value); @@ -3794,7 +4021,7 @@ PyDictKeysObject * _PyDict_NewKeysForClass(void) { - PyDictKeysObject *keys = new_keys_object(PyDict_MINSIZE_SPLIT); + PyDictKeysObject *keys = new_keys_object(PyDict_MINSIZE); if (keys == NULL) PyErr_Clear(); else @@ -3830,7 +4057,7 @@ int _PyObjectDict_SetItem(PyTypeObject *tp, PyObject **dictptr, - PyObject *key, PyObject *value) + PyObject *key, PyObject *value) { PyObject *dict; int res; @@ -3859,7 +4086,8 @@ /* Either update tp->ht_cached_keys or delete it */ if (cached->dk_refcnt == 1) { CACHED_KEYS(tp) = make_keys_shared(dict); - } else { + } + else { CACHED_KEYS(tp) = NULL; } DK_DECREF(cached); @@ -3889,50 +4117,3 @@ { DK_DECREF(keys); } - - -/* ARGSUSED */ -static PyObject * -dummy_repr(PyObject *op) -{ - return PyUnicode_FromString(""); -} - -/* ARGUSED */ -static void -dummy_dealloc(PyObject* ignore) -{ - /* This should never get called, but we also don't want to SEGV if - * we accidentally decref dummy-key out of existence. - */ - Py_FatalError("deallocating "); -} - -static PyTypeObject PyDictDummy_Type = { - PyVarObject_HEAD_INIT(&PyType_Type, 0) - " type", - 0, - 0, - dummy_dealloc, /*tp_dealloc*/ /*never called*/ - 0, /*tp_print*/ - 0, /*tp_getattr*/ - 0, /*tp_setattr*/ - 0, /*tp_reserved*/ - dummy_repr, /*tp_repr*/ - 0, /*tp_as_number*/ - 0, /*tp_as_sequence*/ - 0, /*tp_as_mapping*/ - 0, /*tp_hash */ - 0, /*tp_call */ - 0, /*tp_str */ - 0, /*tp_getattro */ - 0, /*tp_setattro */ - 0, /*tp_as_buffer */ - Py_TPFLAGS_DEFAULT, /*tp_flags */ -}; - -static PyObject _dummy_struct = { - _PyObject_EXTRA_INIT - 2, &PyDictDummy_Type -}; - diff --git a/Objects/object.c b/Objects/object.c --- a/Objects/object.c +++ b/Objects/object.c @@ -22,12 +22,6 @@ { PyObject *o; Py_ssize_t total = _Py_RefTotal; - /* ignore the references to the dummy object of the dicts and sets - because they are not reliable and not useful (now that the - hash table code is well-tested) */ - o = _PyDict_Dummy(); - if (o != NULL) - total -= o->ob_refcnt; o = _PySet_Dummy; if (o != NULL) total -= o->ob_refcnt; diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -536,14 +536,17 @@ _odict_get_index_raw(PyODictObject *od, PyObject *key, Py_hash_t hash) { PyObject **value_addr = NULL; - PyDictKeyEntry *ep; PyDictKeysObject *keys = ((PyDictObject *)od)->ma_keys; + Py_ssize_t ix; - ep = (keys->dk_lookup)((PyDictObject *)od, key, hash, &value_addr); - if (ep == NULL) + ix = (keys->dk_lookup)((PyDictObject *)od, key, hash, &value_addr, NULL); + if (ix == DKIX_EMPTY) { + return keys->dk_nentries; /* index of new entry */ + } + if (ix < 0) return -1; /* We use pointer arithmetic to get the entry's index into the table. */ - return ep - keys->dk_entries; + return ix; } /* Replace od->od_fast_nodes with a new table matching the size of dict's. */ @@ -565,7 +568,7 @@ /* Copy the current nodes into the table. */ _odict_FOREACH(od, node) { i = _odict_get_index_raw(od, _odictnode_KEY(node), - _odictnode_HASH(node)); + _odictnode_HASH(node)); if (i < 0) { PyMem_FREE(fast_nodes); return -1; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:50:49 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 16:50:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_rearrange_to_make_gcc_happ?= =?utf-8?q?y?= Message-ID: <20160908165046.1874.43427.BE2BDC34@psf.io> https://hg.python.org/cpython/rev/22c0c4d36782 changeset: 103316:22c0c4d36782 user: Benjamin Peterson date: Thu Sep 08 09:50:08 2016 -0700 summary: rearrange to make gcc happy files: Objects/dictobject.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -2012,6 +2012,7 @@ { PyObject *v; Py_ssize_t i, j; + PyDictKeyEntry *ep; Py_ssize_t size, n, offset; PyObject **value_ptr; @@ -2027,13 +2028,14 @@ Py_DECREF(v); goto again; } + ep = DK_ENTRIES(mp->ma_keys); size = mp->ma_keys->dk_nentries; if (mp->ma_values) { value_ptr = mp->ma_values; offset = sizeof(PyObject *); } else { - value_ptr = &(DK_ENTRIES(mp->ma_keys)[0].me_value); + value_ptr = &ep[0].me_value; offset = sizeof(PyDictKeyEntry); } for (i = 0, j = 0; i < size; i++) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 12:59:24 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 16:59:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_use_native_inline_instead_?= =?utf-8?q?of_Py=5FLOCAL=5FINLINE?= Message-ID: <20160908165909.21136.94573.FDFF721D@psf.io> https://hg.python.org/cpython/rev/9af1e36dda49 changeset: 103317:9af1e36dda49 user: Benjamin Peterson date: Thu Sep 08 09:58:47 2016 -0700 summary: use native inline instead of Py_LOCAL_INLINE files: Objects/dictobject.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -300,7 +300,7 @@ /* lookup indices. returns DKIX_EMPTY, DKIX_DUMMY, or ix >=0 */ -Py_LOCAL_INLINE(Py_ssize_t) +static inline Py_ssize_t dk_get_index(PyDictKeysObject *keys, Py_ssize_t i) { Py_ssize_t s = DK_SIZE(keys); @@ -321,7 +321,7 @@ } /* write to indices. */ -Py_LOCAL_INLINE(void) +static inline void dk_set_index(PyDictKeysObject *keys, Py_ssize_t i, Py_ssize_t ix) { Py_ssize_t s = DK_SIZE(keys); @@ -1565,7 +1565,7 @@ /* Returns -1 if no more items (or op is not a dict), * index of item otherwise. Stores value in pvalue */ -Py_LOCAL_INLINE(Py_ssize_t) +static inline Py_ssize_t dict_next(PyObject *op, Py_ssize_t i, PyObject **pvalue) { Py_ssize_t n; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 13:12:53 2016 From: python-checkins at python.org (brett.cannon) Date: Thu, 08 Sep 2016 17:12:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327853=3A_Add_sect?= =?utf-8?q?ion_headers_to_the_importlib_example_docs?= Message-ID: <20160908171252.59815.86029.9321D5F1@psf.io> https://hg.python.org/cpython/rev/a547d43c7258 changeset: 103318:a547d43c7258 user: Brett Cannon date: Thu Sep 08 10:12:47 2016 -0700 summary: Issue #27853: Add section headers to the importlib example docs files: Doc/library/importlib.rst | 19 +++++++++++++++++++ 1 files changed, 19 insertions(+), 0 deletions(-) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -1353,6 +1353,9 @@ Examples -------- +Importing programmatically +'''''''''''''''''''''''''' + To programmatically import a module, use :func:`importlib.import_module`. :: @@ -1360,6 +1363,10 @@ itertools = importlib.import_module('itertools') + +Checking if a module can be imported +'''''''''''''''''''''''''''''''''''' + If you need to find out if a module can be imported without actually doing the import, then you should use :func:`importlib.util.find_spec`. :: @@ -1380,6 +1387,10 @@ # Adding the module to sys.modules is optional. sys.modules[name] = module + +Importing a source file directly +'''''''''''''''''''''''''''''''' + To import a Python source file directly, use the following recipe (Python 3.4 and newer only):: @@ -1398,6 +1409,10 @@ # by name later. sys.modules[module_name] = module + +Setting up an importer +'''''''''''''''''''''' + For deep customizations of import, you typically want to implement an :term:`importer`. This means managing both the :term:`finder` and :term:`loader` side of things. For finders there are two flavours to choose from depending on @@ -1428,6 +1443,10 @@ # of priority. sys.path_hooks.append(SpamPathEntryFinder.path_hook(loader_details)) + +Approximating :func:`importlib.import_module` +''''''''''''''''''''''''''''''''''''''''''''' + Import itself is implemented in Python code, making it possible to expose most of the import machinery through importlib. The following helps illustrate the various APIs that importlib exposes by providing an -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 13:14:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 17:14:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_improve_compact_dict_chang?= =?utf-8?q?elog?= Message-ID: <20160908171431.21311.93498.8C5BBDA7@psf.io> https://hg.python.org/cpython/rev/643b147199c1 changeset: 103319:643b147199c1 user: Benjamin Peterson date: Thu Sep 08 10:13:42 2016 -0700 summary: improve compact dict changelog files: Doc/whatsnew/3.6.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -343,10 +343,10 @@ Some smaller changes made to the core Python language are: -* `dict` implementation is changed like PyPy. It is more compact and preserves - insertion order. :pep:`PEP 468` (Preserving the order of `**kwargs` in a - function.) is implemented by this. - (Contributed by INADA Naoki in :issue:`27350`.) +* :func:`dict` now uses a "compact" representation `pioneered by PyPy + `_. + :pep:`PEP 468` (Preserving the order of ``**kwargs`` in a function.) is + implemented by this. (Contributed by INADA Naoki in :issue:`27350`.) * Long sequences of repeated traceback lines are now abbreviated as ``"[Previous line repeated {count} more times]"`` (see -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 13:14:40 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 17:14:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_link_to_canonical_blogspot?= Message-ID: <20160908171439.21157.39781.9D07B0C2@psf.io> https://hg.python.org/cpython/rev/e70dcf7a6980 changeset: 103320:e70dcf7a6980 user: Benjamin Peterson date: Thu Sep 08 10:14:31 2016 -0700 summary: link to canonical blogspot files: Objects/dictobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -11,7 +11,7 @@ This implements the dictionary's hashtable. As of Python 3.6, this is compact and orderd. Basic idea is described here. -https://morepypy.blogspot.jp/2015/01/faster-more-memory-efficient-and-more.html +https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html layout: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 13:29:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 17:29:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_add_a_note_about_c99?= Message-ID: <20160908172932.12395.67854.04429BD4@psf.io> https://hg.python.org/cpython/rev/b84d32837663 changeset: 103321:b84d32837663 user: Benjamin Peterson date: Thu Sep 08 10:27:20 2016 -0700 summary: add a note about c99 files: Doc/whatsnew/3.6.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -845,6 +845,9 @@ Build and C API Changes ======================= +* Python now requires some C99 support in the toolchain to build. For more + information, see :pep:`7`. + * The ``--with-optimizations`` configure flag has been added. Turning it on will activate LTO and PGO build support (when available). (Original patch by Alecsandru Patrascu of Intel in :issue:`26539`.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 13:35:31 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 17:35:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327781=3A_Change_f?= =?utf-8?q?ile_system_encoding_on_Windows_to_UTF-8_=28PEP_529=29?= Message-ID: <20160908173531.2723.43888.E309E3C0@psf.io> https://hg.python.org/cpython/rev/e20c7d8a8187 changeset: 103322:e20c7d8a8187 user: Steve Dower date: Thu Sep 08 10:35:16 2016 -0700 summary: Issue #27781: Change file system encoding on Windows to UTF-8 (PEP 529) files: Doc/c-api/unicode.rst | 30 +- Doc/library/sys.rst | 51 +- Doc/using/cmdline.rst | 14 + Doc/whatsnew/3.6.rst | 29 + Include/fileobject.h | 1 + Include/unicodeobject.h | 8 +- Lib/os.py | 5 +- Lib/test/test_os.py | 113 +- Misc/NEWS | 6 +- Modules/_codecsmodule.c | 8 +- Modules/clinic/_codecsmodule.c.h | 26 +- Modules/clinic/posixmodule.c.h | 98 +- Modules/overlapped.c | 10 +- Modules/posixmodule.c | 961 +++++------------- Objects/unicodeobject.c | 46 +- Python/bltinmodule.c | 8 +- Python/pylifecycle.c | 20 + Python/sysmodule.c | 50 +- 18 files changed, 633 insertions(+), 851 deletions(-) diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -802,10 +802,11 @@ """""""""""""""""""" To encode and decode file names and other environment strings, -:c:data:`Py_FileSystemEncoding` should be used as the encoding, and -``"surrogateescape"`` should be used as the error handler (:pep:`383`). To -encode file names during argument parsing, the ``"O&"`` converter should be -used, passing :c:func:`PyUnicode_FSConverter` as the conversion function: +:c:data:`Py_FileSystemDefaultEncoding` should be used as the encoding, and +:c:data:`Py_FileSystemDefaultEncodeErrors` should be used as the error handler +(:pep:`383` and :pep:`529`). To encode file names to :class:`bytes` during +argument parsing, the ``"O&"`` converter should be used, passing +:c:func:`PyUnicode_FSConverter` as the conversion function: .. c:function:: int PyUnicode_FSConverter(PyObject* obj, void* result) @@ -820,8 +821,9 @@ .. versionchanged:: 3.6 Accepts a :term:`path-like object`. -To decode file names during argument parsing, the ``"O&"`` converter should be -used, passing :c:func:`PyUnicode_FSDecoder` as the conversion function: +To decode file names to :class:`str` during argument parsing, the ``"O&"`` +converter should be used, passing :c:func:`PyUnicode_FSDecoder` as the +conversion function: .. c:function:: int PyUnicode_FSDecoder(PyObject* obj, void* result) @@ -840,7 +842,7 @@ .. c:function:: PyObject* PyUnicode_DecodeFSDefaultAndSize(const char *s, Py_ssize_t size) Decode a string using :c:data:`Py_FileSystemDefaultEncoding` and the - ``"surrogateescape"`` error handler, or ``"strict"`` on Windows. + :c:data:`Py_FileSystemDefaultEncodeErrors` error handler. If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the locale encoding. @@ -854,28 +856,28 @@ The :c:func:`Py_DecodeLocale` function. - .. versionchanged:: 3.2 - Use ``"strict"`` error handler on Windows. + .. versionchanged:: 3.6 + Use :c:data:`Py_FileSystemDefaultEncodeErrors` error handler. .. c:function:: PyObject* PyUnicode_DecodeFSDefault(const char *s) Decode a null-terminated string using :c:data:`Py_FileSystemDefaultEncoding` - and the ``"surrogateescape"`` error handler, or ``"strict"`` on Windows. + and the :c:data:`Py_FileSystemDefaultEncodeErrors` error handler. If :c:data:`Py_FileSystemDefaultEncoding` is not set, fall back to the locale encoding. Use :c:func:`PyUnicode_DecodeFSDefaultAndSize` if you know the string length. - .. versionchanged:: 3.2 - Use ``"strict"`` error handler on Windows. + .. versionchanged:: 3.6 + Use :c:data:`Py_FileSystemDefaultEncodeErrors` error handler. .. c:function:: PyObject* PyUnicode_EncodeFSDefault(PyObject *unicode) Encode a Unicode object to :c:data:`Py_FileSystemDefaultEncoding` with the - ``"surrogateescape"`` error handler, or ``"strict"`` on Windows, and return + :c:data:`Py_FileSystemDefaultEncodeErrors` error handler, and return :class:`bytes`. Note that the resulting :class:`bytes` object may contain null bytes. @@ -892,6 +894,8 @@ .. versionadded:: 3.2 + .. versionchanged:: 3.6 + Use :c:data:`Py_FileSystemDefaultEncodeErrors` error handler. wchar_t Support """"""""""""""" diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -428,25 +428,42 @@ .. function:: getfilesystemencoding() - Return the name of the encoding used to convert Unicode filenames into - system file names. The result value depends on the operating system: + Return the name of the encoding used to convert between Unicode + filenames and bytes filenames. For best compatibility, str should be + used for filenames in all cases, although representing filenames as bytes + is also supported. Functions accepting or returning filenames should support + either str or bytes and internally convert to the system's preferred + representation. + + This encoding is always ASCII-compatible. + + :func:`os.fsencode` and :func:`os.fsdecode` should be used to ensure that + the correct encoding and errors mode are used. * On Mac OS X, the encoding is ``'utf-8'``. - * On Unix, the encoding is the user's preference according to the result of - nl_langinfo(CODESET). + * On Unix, the encoding is the locale encoding. - * On Windows NT+, file names are Unicode natively, so no conversion is - performed. :func:`getfilesystemencoding` still returns ``'mbcs'``, as - this is the encoding that applications should use when they explicitly - want to convert Unicode strings to byte strings that are equivalent when - used as file names. - - * On Windows 9x, the encoding is ``'mbcs'``. + * On Windows, the encoding may be ``'utf-8'`` or ``'mbcs'``, depending + on user configuration. .. versionchanged:: 3.2 :func:`getfilesystemencoding` result cannot be ``None`` anymore. + .. versionchanged:: 3.6 + Windows is no longer guaranteed to return ``'mbcs'``. See :pep:`529` + and :func:`_enablelegacywindowsfsencoding` for more information. + +.. function:: getfilesystemencodeerrors() + + Return the name of the error mode used to convert between Unicode filenames + and bytes filenames. The encoding name is returned from + :func:`getfilesystemencoding`. + + :func:`os.fsencode` and :func:`os.fsdecode` should be used to ensure that + the correct encoding and errors mode are used. + + .. versionadded:: 3.6 .. function:: getrefcount(object) @@ -1138,6 +1155,18 @@ This function has been added on a provisional basis (see :pep:`411` for details.) Use it only for debugging purposes. +.. function:: _enablelegacywindowsfsencoding() + + Changes the default filesystem encoding and errors mode to 'mbcs' and + 'replace' respectively, for consistency with versions of Python prior to 3.6. + + This is equivalent to defining the :envvar:`PYTHONLEGACYWINDOWSFSENCODING` + environment variable before launching Python. + + Availability: Windows + + .. versionadded:: 3.6 + See :pep:`529` for more details. .. data:: stdin stdout diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -672,6 +672,20 @@ It now has no effect if set to an empty string. +.. envvar:: PYTHONLEGACYWINDOWSFSENCODING + + If set to a non-empty string, the default filesystem encoding and errors mode + will revert to their pre-3.6 values of 'mbcs' and 'replace', respectively. + Otherwise, the new defaults 'utf-8' and 'surrogatepass' are used. + + This may also be enabled at runtime with + :func:`sys._enablelegacywindowsfsencoding()`. + + Availability: Windows + + .. versionadded:: 3.6 + See :pep:`529` for more details. + Debug-mode variables ~~~~~~~~~~~~~~~~~~~~ diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -76,6 +76,8 @@ Windows improvements: +* PEP 529: :ref:`Change Windows filesystem encoding to UTF-8 ` + * The ``py.exe`` launcher, when used interactively, no longer prefers Python 2 over Python 3 when the user doesn't specify a version (via command line arguments or a config file). Handling of shebang lines @@ -218,6 +220,33 @@ See :pep:`498` and the main documentation at :ref:`f-strings`. +.. _pep-529: + +PEP 529: Change Windows filesystem encoding to UTF-8 +---------------------------------------------------- + +Representing filesystem paths is best performed with str (Unicode) rather than +bytes. However, there are some situations where using bytes is sufficient and +correct. + +Prior to Python 3.6, data loss could result when using bytes paths on Windows. +With this change, using bytes to represent paths is now supported on Windows, +provided those bytes are encoded with the encoding returned by +:func:`sys.getfilesystemencoding()`, which now defaults to ``'utf-8'``. + +Applications that do not use str to represent paths should use +:func:`os.fsencode()` and :func:`os.fsdecode()` to ensure their bytes are +correctly encoded. To revert to the previous behaviour, set +:envvar:`PYTHONLEGACYWINDOWSFSENCODING` or call +:func:`sys._enablelegacywindowsfsencoding`. + +See :pep:`529` for more information and discussion of code modifications that +may be required. + +.. note:: + + This change is considered experimental for 3.6.0 beta releases. The default + encoding may change before the final release. PEP 487: Simpler customization of class creation ------------------------------------------------ diff --git a/Include/fileobject.h b/Include/fileobject.h --- a/Include/fileobject.h +++ b/Include/fileobject.h @@ -23,6 +23,7 @@ If non-NULL, this is different than the default encoding for strings */ PyAPI_DATA(const char *) Py_FileSystemDefaultEncoding; +PyAPI_DATA(const char *) Py_FileSystemDefaultEncodeErrors; PyAPI_DATA(int) Py_HasFileSystemDefaultEncoding; /* Internal API diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -103,10 +103,6 @@ # endif #endif -#if defined(MS_WINDOWS) -# define HAVE_MBCS -#endif - #ifdef HAVE_WCHAR_H /* Work around a cosmetic bug in BSDI 4.x wchar.h; thanks to Thomas Wouters */ # ifdef _HAVE_BSDI @@ -1657,7 +1653,7 @@ ); #endif -#ifdef HAVE_MBCS +#ifdef MS_WINDOWS /* --- MBCS codecs for Windows -------------------------------------------- */ @@ -1700,7 +1696,7 @@ const char *errors /* error handling */ ); -#endif /* HAVE_MBCS */ +#endif /* MS_WINDOWS */ /* --- Decimal Encoder ---------------------------------------------------- */ diff --git a/Lib/os.py b/Lib/os.py --- a/Lib/os.py +++ b/Lib/os.py @@ -851,10 +851,7 @@ def _fscodec(): encoding = sys.getfilesystemencoding() - if encoding == 'mbcs': - errors = 'strict' - else: - errors = 'surrogateescape' + errors = sys.getfilesystemencodeerrors() def fsencode(filename): """Encode filename (an os.PathLike, bytes, or str) to the filesystem diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -90,16 +90,6 @@ yield - at contextlib.contextmanager -def bytes_filename_warn(expected): - msg = 'The Windows bytes API has been deprecated' - if os.name == 'nt': - with ignore_deprecation_warnings(msg, quiet=not expected): - yield - else: - yield - - class _PathLike(os.PathLike): def __init__(self, path=""): @@ -342,8 +332,7 @@ fname = self.fname.encode(sys.getfilesystemencoding()) except UnicodeEncodeError: self.skipTest("cannot encode %a for the filesystem" % self.fname) - with bytes_filename_warn(True): - self.check_stat_attributes(fname) + self.check_stat_attributes(fname) def test_stat_result_pickle(self): result = os.stat(self.fname) @@ -1032,8 +1021,6 @@ def setUp(self): super().setUp() self.stack = contextlib.ExitStack() - if os.name == 'nt': - self.stack.enter_context(bytes_filename_warn(False)) def tearDown(self): self.stack.close() @@ -1640,8 +1627,7 @@ def _test_link(self, file1, file2): create_file(file1) - with bytes_filename_warn(False): - os.link(file1, file2) + os.link(file1, file2) with open(file1, "r") as f1, open(file2, "r") as f2: self.assertTrue(os.path.sameopenfile(f1.fileno(), f2.fileno())) @@ -1934,10 +1920,9 @@ self.created_paths) # bytes - with bytes_filename_warn(False): - self.assertEqual( - sorted(os.listdir(os.fsencode(support.TESTFN))), - [os.fsencode(path) for path in self.created_paths]) + self.assertEqual( + sorted(os.listdir(os.fsencode(support.TESTFN))), + [os.fsencode(path) for path in self.created_paths]) def test_listdir_extended_path(self): """Test when the path starts with '\\\\?\\'.""" @@ -1949,11 +1934,10 @@ self.created_paths) # bytes - with bytes_filename_warn(False): - path = b'\\\\?\\' + os.fsencode(os.path.abspath(support.TESTFN)) - self.assertEqual( - sorted(os.listdir(path)), - [os.fsencode(path) for path in self.created_paths]) + path = b'\\\\?\\' + os.fsencode(os.path.abspath(support.TESTFN)) + self.assertEqual( + sorted(os.listdir(path)), + [os.fsencode(path) for path in self.created_paths]) @unittest.skipUnless(sys.platform == "win32", "Win32 specific tests") @@ -2028,10 +2012,8 @@ self.assertNotEqual(os.lstat(link), os.stat(link)) bytes_link = os.fsencode(link) - with bytes_filename_warn(True): - self.assertEqual(os.stat(bytes_link), os.stat(target)) - with bytes_filename_warn(True): - self.assertNotEqual(os.lstat(bytes_link), os.stat(bytes_link)) + self.assertEqual(os.stat(bytes_link), os.stat(target)) + self.assertNotEqual(os.lstat(bytes_link), os.stat(bytes_link)) def test_12084(self): level1 = os.path.abspath(support.TESTFN) @@ -2589,46 +2571,6 @@ self._check_xattrs(getxattr, setxattr, removexattr, listxattr) - at unittest.skipUnless(sys.platform == "win32", "Win32 specific tests") -class Win32DeprecatedBytesAPI(unittest.TestCase): - def test_deprecated(self): - import nt - filename = os.fsencode(support.TESTFN) - for func, *args in ( - (nt._getfullpathname, filename), - (nt._isdir, filename), - (os.access, filename, os.R_OK), - (os.chdir, filename), - (os.chmod, filename, 0o777), - (os.getcwdb,), - (os.link, filename, filename), - (os.listdir, filename), - (os.lstat, filename), - (os.mkdir, filename), - (os.open, filename, os.O_RDONLY), - (os.rename, filename, filename), - (os.rmdir, filename), - (os.startfile, filename), - (os.stat, filename), - (os.unlink, filename), - (os.utime, filename), - ): - with bytes_filename_warn(True): - try: - func(*args) - except OSError: - # ignore OSError, we only care about DeprecationWarning - pass - - @support.skip_unless_symlink - def test_symlink(self): - self.addCleanup(support.unlink, support.TESTFN) - - filename = os.fsencode(support.TESTFN) - with bytes_filename_warn(True): - os.symlink(filename, filename) - - @unittest.skipUnless(hasattr(os, 'get_terminal_size'), "requires os.get_terminal_size") class TermsizeTests(unittest.TestCase): def test_does_not_crash(self): @@ -2712,16 +2654,7 @@ (self.bytes_filenames, os.replace, b"dst"), (self.unicode_filenames, os.rename, "dst"), (self.unicode_filenames, os.replace, "dst"), - # Issue #16414: Don't test undecodable names with listdir() - # because of a Windows bug. - # - # With the ANSI code page 932, os.listdir(b'\xe7') return an - # empty list (instead of failing), whereas os.listdir(b'\xff') - # raises a FileNotFoundError. It looks like a Windows bug: - # b'\xe7' directory does not exist, FindFirstFileA(b'\xe7') - # fails with ERROR_FILE_NOT_FOUND (2), instead of - # ERROR_PATH_NOT_FOUND (3). - (self.unicode_filenames, os.listdir,), + (self.unicode_filenames, os.listdir, ), )) else: funcs.extend(( @@ -2762,19 +2695,24 @@ else: funcs.append((self.filenames, os.readlink,)) + for filenames, func, *func_args in funcs: for name in filenames: try: - if isinstance(name, str): + if isinstance(name, (str, bytes)): func(name, *func_args) - elif isinstance(name, bytes): - with bytes_filename_warn(False): - func(name, *func_args) else: with self.assertWarnsRegex(DeprecationWarning, 'should be'): func(name, *func_args) except OSError as err: - self.assertIs(err.filename, name) + self.assertIs(err.filename, name, str(func)) + except RuntimeError as err: + if sys.platform != 'win32': + raise + + # issue27781: undecodable bytes currently raise RuntimeError + # by 3.6.0b4 this will become UnicodeDecodeError or nothing + self.assertIsInstance(err.__context__, UnicodeDecodeError) else: self.fail("No exception thrown by {}".format(func)) @@ -3086,7 +3024,6 @@ entry = self.create_file_entry() self.assertEqual(os.fspath(entry), os.path.join(self.path, 'file.txt')) - @unittest.skipIf(os.name == "nt", "test requires bytes path support") def test_fspath_protocol_bytes(self): bytes_filename = os.fsencode('bytesfile.txt') bytes_entry = self.create_file_entry(name=bytes_filename) @@ -3158,12 +3095,6 @@ entry.stat(follow_symlinks=False) def test_bytes(self): - if os.name == "nt": - # On Windows, os.scandir(bytes) must raise an exception - with bytes_filename_warn(True): - self.assertRaises(TypeError, os.scandir, b'.') - return - self.create_file("file.txt") path_bytes = os.fsencode(self.path) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -286,6 +286,8 @@ Windows ------- +- Issue #27781: Change file system encoding on Windows to UTF-8 (PEP 529) + - Issue #27731: Opt-out of MAX_PATH on Windows 10 - Issue #6135: Adds encoding and errors parameters to subprocess. @@ -2632,7 +2634,7 @@ - Issue #24774: Fix docstring in http.server.test. Patch from Chiu-Hsiang Hsu. - Issue #21159: Improve message in configparser.InterpolationMissingOptionError. - Patch from ??ukasz Langa. + Patch from ??ukasz Langa. - Issue #20362: Honour TestCase.longMessage correctly in assertRegex. Patch from Ilia Kurenkov. @@ -4560,7 +4562,7 @@ Based on patch by Martin Panter. - Issue #17293: uuid.getnode() now determines MAC address on AIX using netstat. - Based on patch by Aivars Kalv??ns. + Based on patch by Aivars Kalv??ns. - Issue #22769: Fixed ttk.Treeview.tag_has() when called without arguments. diff --git a/Modules/_codecsmodule.c b/Modules/_codecsmodule.c --- a/Modules/_codecsmodule.c +++ b/Modules/_codecsmodule.c @@ -604,7 +604,7 @@ return codec_tuple(decoded, data->len); } -#ifdef HAVE_MBCS +#ifdef MS_WINDOWS /*[clinic input] _codecs.mbcs_decode @@ -666,7 +666,7 @@ return codec_tuple(decoded, consumed); } -#endif /* HAVE_MBCS */ +#endif /* MS_WINDOWS */ /* --- Encoder ------------------------------------------------------------ */ @@ -972,7 +972,7 @@ return PyUnicode_BuildEncodingMap(map); } -#ifdef HAVE_MBCS +#ifdef MS_WINDOWS /*[clinic input] _codecs.mbcs_encode @@ -1021,7 +1021,7 @@ PyUnicode_GET_LENGTH(str)); } -#endif /* HAVE_MBCS */ +#endif /* MS_WINDOWS */ /* --- Error handler registry --------------------------------------------- */ diff --git a/Modules/clinic/_codecsmodule.c.h b/Modules/clinic/_codecsmodule.c.h --- a/Modules/clinic/_codecsmodule.c.h +++ b/Modules/clinic/_codecsmodule.c.h @@ -764,7 +764,7 @@ return return_value; } -#if defined(HAVE_MBCS) +#if defined(MS_WINDOWS) PyDoc_STRVAR(_codecs_mbcs_decode__doc__, "mbcs_decode($module, data, errors=None, final=False, /)\n" @@ -801,9 +801,9 @@ return return_value; } -#endif /* defined(HAVE_MBCS) */ +#endif /* defined(MS_WINDOWS) */ -#if defined(HAVE_MBCS) +#if defined(MS_WINDOWS) PyDoc_STRVAR(_codecs_oem_decode__doc__, "oem_decode($module, data, errors=None, final=False, /)\n" @@ -840,9 +840,9 @@ return return_value; } -#endif /* defined(HAVE_MBCS) */ +#endif /* defined(MS_WINDOWS) */ -#if defined(HAVE_MBCS) +#if defined(MS_WINDOWS) PyDoc_STRVAR(_codecs_code_page_decode__doc__, "code_page_decode($module, codepage, data, errors=None, final=False, /)\n" @@ -880,7 +880,7 @@ return return_value; } -#endif /* defined(HAVE_MBCS) */ +#endif /* defined(MS_WINDOWS) */ PyDoc_STRVAR(_codecs_readbuffer_encode__doc__, "readbuffer_encode($module, data, errors=None, /)\n" @@ -1351,7 +1351,7 @@ return return_value; } -#if defined(HAVE_MBCS) +#if defined(MS_WINDOWS) PyDoc_STRVAR(_codecs_mbcs_encode__doc__, "mbcs_encode($module, str, errors=None, /)\n" @@ -1381,9 +1381,9 @@ return return_value; } -#endif /* defined(HAVE_MBCS) */ +#endif /* defined(MS_WINDOWS) */ -#if defined(HAVE_MBCS) +#if defined(MS_WINDOWS) PyDoc_STRVAR(_codecs_oem_encode__doc__, "oem_encode($module, str, errors=None, /)\n" @@ -1413,9 +1413,9 @@ return return_value; } -#endif /* defined(HAVE_MBCS) */ +#endif /* defined(MS_WINDOWS) */ -#if defined(HAVE_MBCS) +#if defined(MS_WINDOWS) PyDoc_STRVAR(_codecs_code_page_encode__doc__, "code_page_encode($module, code_page, str, errors=None, /)\n" @@ -1447,7 +1447,7 @@ return return_value; } -#endif /* defined(HAVE_MBCS) */ +#endif /* defined(MS_WINDOWS) */ PyDoc_STRVAR(_codecs_register_error__doc__, "register_error($module, errors, handler, /)\n" @@ -1536,4 +1536,4 @@ #ifndef _CODECS_CODE_PAGE_ENCODE_METHODDEF #define _CODECS_CODE_PAGE_ENCODE_METHODDEF #endif /* !defined(_CODECS_CODE_PAGE_ENCODE_METHODDEF) */ -/*[clinic end generated code: output=7874e2d559d49368 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=ebe313ab417b17bb input=a9049054013a1b77]*/ diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -1649,24 +1649,24 @@ {"execv", (PyCFunction)os_execv, METH_VARARGS, os_execv__doc__}, static PyObject * -os_execv_impl(PyObject *module, PyObject *path, PyObject *argv); +os_execv_impl(PyObject *module, path_t *path, PyObject *argv); static PyObject * os_execv(PyObject *module, PyObject *args) { PyObject *return_value = NULL; - PyObject *path = NULL; + path_t path = PATH_T_INITIALIZE("execv", "path", 0, 0); PyObject *argv; if (!PyArg_ParseTuple(args, "O&O:execv", - PyUnicode_FSConverter, &path, &argv)) { + path_converter, &path, &argv)) { goto exit; } - return_value = os_execv_impl(module, path, argv); + return_value = os_execv_impl(module, &path, argv); exit: /* Cleanup for path */ - Py_XDECREF(path); + path_cleanup(&path); return return_value; } @@ -1719,7 +1719,7 @@ #endif /* defined(HAVE_EXECV) */ -#if defined(HAVE_SPAWNV) +#if (defined(HAVE_SPAWNV) || defined(HAVE_WSPAWNV)) PyDoc_STRVAR(os_spawnv__doc__, "spawnv($module, mode, path, argv, /)\n" @@ -1738,32 +1738,32 @@ {"spawnv", (PyCFunction)os_spawnv, METH_VARARGS, os_spawnv__doc__}, static PyObject * -os_spawnv_impl(PyObject *module, int mode, PyObject *path, PyObject *argv); +os_spawnv_impl(PyObject *module, int mode, path_t *path, PyObject *argv); static PyObject * os_spawnv(PyObject *module, PyObject *args) { PyObject *return_value = NULL; int mode; - PyObject *path = NULL; + path_t path = PATH_T_INITIALIZE("spawnv", "path", 0, 0); PyObject *argv; if (!PyArg_ParseTuple(args, "iO&O:spawnv", - &mode, PyUnicode_FSConverter, &path, &argv)) { + &mode, path_converter, &path, &argv)) { goto exit; } - return_value = os_spawnv_impl(module, mode, path, argv); + return_value = os_spawnv_impl(module, mode, &path, argv); exit: /* Cleanup for path */ - Py_XDECREF(path); + path_cleanup(&path); return return_value; } -#endif /* defined(HAVE_SPAWNV) */ - -#if defined(HAVE_SPAWNV) +#endif /* (defined(HAVE_SPAWNV) || defined(HAVE_WSPAWNV)) */ + +#if (defined(HAVE_SPAWNV) || defined(HAVE_WSPAWNV)) PyDoc_STRVAR(os_spawnve__doc__, "spawnve($module, mode, path, argv, env, /)\n" @@ -1784,7 +1784,7 @@ {"spawnve", (PyCFunction)os_spawnve, METH_VARARGS, os_spawnve__doc__}, static PyObject * -os_spawnve_impl(PyObject *module, int mode, PyObject *path, PyObject *argv, +os_spawnve_impl(PyObject *module, int mode, path_t *path, PyObject *argv, PyObject *env); static PyObject * @@ -1792,24 +1792,24 @@ { PyObject *return_value = NULL; int mode; - PyObject *path = NULL; + path_t path = PATH_T_INITIALIZE("spawnve", "path", 0, 0); PyObject *argv; PyObject *env; if (!PyArg_ParseTuple(args, "iO&OO:spawnve", - &mode, PyUnicode_FSConverter, &path, &argv, &env)) { + &mode, path_converter, &path, &argv, &env)) { goto exit; } - return_value = os_spawnve_impl(module, mode, path, argv, env); + return_value = os_spawnve_impl(module, mode, &path, argv, env); exit: /* Cleanup for path */ - Py_XDECREF(path); + path_cleanup(&path); return return_value; } -#endif /* defined(HAVE_SPAWNV) */ +#endif /* (defined(HAVE_SPAWNV) || defined(HAVE_WSPAWNV)) */ #if defined(HAVE_FORK1) @@ -4994,6 +4994,60 @@ return os_abort_impl(module); } +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(os_startfile__doc__, +"startfile($module, /, filepath, operation=None)\n" +"--\n" +"\n" +"startfile(filepath [, operation])\n" +"\n" +"Start a file with its associated application.\n" +"\n" +"When \"operation\" is not specified or \"open\", this acts like\n" +"double-clicking the file in Explorer, or giving the file name as an\n" +"argument to the DOS \"start\" command: the file is opened with whatever\n" +"application (if any) its extension is associated.\n" +"When another \"operation\" is given, it specifies what should be done with\n" +"the file. A typical operation is \"print\".\n" +"\n" +"startfile returns as soon as the associated application is launched.\n" +"There is no option to wait for the application to close, and no way\n" +"to retrieve the application\'s exit status.\n" +"\n" +"The filepath is relative to the current directory. If you want to use\n" +"an absolute path, make sure the first character is not a slash (\"/\");\n" +"the underlying Win32 ShellExecute function doesn\'t work if it is."); + +#define OS_STARTFILE_METHODDEF \ + {"startfile", (PyCFunction)os_startfile, METH_VARARGS|METH_KEYWORDS, os_startfile__doc__}, + +static PyObject * +os_startfile_impl(PyObject *module, path_t *filepath, Py_UNICODE *operation); + +static PyObject * +os_startfile(PyObject *module, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static char *_keywords[] = {"filepath", "operation", NULL}; + path_t filepath = PATH_T_INITIALIZE("startfile", "filepath", 0, 0); + Py_UNICODE *operation = NULL; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|u:startfile", _keywords, + path_converter, &filepath, &operation)) { + goto exit; + } + return_value = os_startfile_impl(module, &filepath, operation); + +exit: + /* Cleanup for filepath */ + path_cleanup(&filepath); + + return return_value; +} + +#endif /* defined(MS_WINDOWS) */ + #if defined(HAVE_GETLOADAVG) PyDoc_STRVAR(os_getloadavg__doc__, @@ -6034,6 +6088,10 @@ #define OS_SYSCONF_METHODDEF #endif /* !defined(OS_SYSCONF_METHODDEF) */ +#ifndef OS_STARTFILE_METHODDEF + #define OS_STARTFILE_METHODDEF +#endif /* !defined(OS_STARTFILE_METHODDEF) */ + #ifndef OS_GETLOADAVG_METHODDEF #define OS_GETLOADAVG_METHODDEF #endif /* !defined(OS_GETLOADAVG_METHODDEF) */ diff --git a/Modules/overlapped.c b/Modules/overlapped.c --- a/Modules/overlapped.c +++ b/Modules/overlapped.c @@ -973,28 +973,28 @@ static int parse_address(PyObject *obj, SOCKADDR *Address, int Length) { - char *Host; + Py_UNICODE *Host; unsigned short Port; unsigned long FlowInfo; unsigned long ScopeId; memset(Address, 0, Length); - if (PyArg_ParseTuple(obj, "sH", &Host, &Port)) + if (PyArg_ParseTuple(obj, "uH", &Host, &Port)) { Address->sa_family = AF_INET; - if (WSAStringToAddressA(Host, AF_INET, NULL, Address, &Length) < 0) { + if (WSAStringToAddressW(Host, AF_INET, NULL, Address, &Length) < 0) { SetFromWindowsErr(WSAGetLastError()); return -1; } ((SOCKADDR_IN*)Address)->sin_port = htons(Port); return Length; } - else if (PyArg_ParseTuple(obj, "sHkk", &Host, &Port, &FlowInfo, &ScopeId)) + else if (PyArg_ParseTuple(obj, "uHkk", &Host, &Port, &FlowInfo, &ScopeId)) { PyErr_Clear(); Address->sa_family = AF_INET6; - if (WSAStringToAddressA(Host, AF_INET6, NULL, Address, &Length) < 0) { + if (WSAStringToAddressW(Host, AF_INET6, NULL, Address, &Length) < 0) { SetFromWindowsErr(WSAGetLastError()); return -1; } diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -164,6 +164,8 @@ #define HAVE_GETLOGIN 1 #define HAVE_SPAWNV 1 #define HAVE_EXECV 1 +#define HAVE_WSPAWNV 1 +#define HAVE_WEXECV 1 #define HAVE_PIPE 1 #define HAVE_SYSTEM 1 #define HAVE_CWAIT 1 @@ -735,7 +737,7 @@ * * * On Windows, if we get a (Unicode) string we * extract the wchar_t * and return it; if we get - * bytes we extract the char * and return that. + * bytes we decode to wchar_t * and return that. * * * On all other platforms, strings are encoded * to bytes using PyUnicode_FSConverter, then we @@ -767,7 +769,9 @@ * and was not encoded. (Only used on Windows.) * path.narrow * Points to the path if it was expressed as bytes, - * or it was Unicode and was encoded to bytes. + * or it was Unicode and was encoded to bytes. (On Windows, + * is an non-zero integer if the path was expressed as bytes. + * The type is deliberately incompatible to prevent misuse.) * path.fd * Contains a file descriptor if path.accept_fd was true * and the caller provided a signed integer instead of any @@ -812,15 +816,24 @@ int nullable; int allow_fd; const wchar_t *wide; +#ifdef MS_WINDOWS + BOOL narrow; +#else const char *narrow; +#endif int fd; Py_ssize_t length; PyObject *object; PyObject *cleanup; } path_t; +#ifdef MS_WINDOWS +#define PATH_T_INITIALIZE(function_name, argument_name, nullable, allow_fd) \ + {function_name, argument_name, nullable, allow_fd, NULL, FALSE, -1, 0, NULL, NULL} +#else #define PATH_T_INITIALIZE(function_name, argument_name, nullable, allow_fd) \ {function_name, argument_name, nullable, allow_fd, NULL, NULL, -1, 0, NULL, NULL} +#endif static void path_cleanup(path_t *path) { @@ -839,6 +852,10 @@ /* Default to failure, forcing explicit signaling of succcess. */ int ret = 0; const char *narrow; +#ifdef MS_WINDOWS + PyObject *wo; + const wchar_t *wide; +#endif #define FORMAT_EXCEPTION(exc, fmt) \ PyErr_Format(exc, "%s%s" fmt, \ @@ -857,7 +874,11 @@ if ((o == Py_None) && path->nullable) { path->wide = NULL; +#ifdef MS_WINDOWS + path->narrow = FALSE; +#else path->narrow = NULL; +#endif path->length = 0; path->object = o; path->fd = -1; @@ -899,9 +920,7 @@ if (is_unicode) { #ifdef MS_WINDOWS - const wchar_t *wide; - - wide = PyUnicode_AsUnicodeAndSize(o, &length); + wide = PyUnicode_AsWideCharString(o, &length); if (!wide) { goto exit; } @@ -915,7 +934,6 @@ } path->wide = wide; - path->narrow = NULL; path->length = length; path->object = o; path->fd = -1; @@ -928,11 +946,6 @@ #endif } else if (is_bytes) { -#ifdef MS_WINDOWS - if (win32_warn_bytes_api()) { - goto exit; - } -#endif bytes = o; Py_INCREF(bytes); } @@ -950,22 +963,21 @@ Py_TYPE(o)->tp_name)) { goto exit; } -#ifdef MS_WINDOWS - if (win32_warn_bytes_api()) { - goto exit; - } -#endif bytes = PyBytes_FromObject(o); if (!bytes) { goto exit; } } - else if (path->allow_fd && PyIndex_Check(o)) { + else if (is_index) { if (!_fd_converter(o, &path->fd)) { goto exit; } path->wide = NULL; +#ifdef MS_WINDOWS + path->narrow = FALSE; +#else path->narrow = NULL; +#endif path->length = 0; path->object = o; ret = 1; @@ -987,14 +999,6 @@ } length = PyBytes_GET_SIZE(bytes); -#ifdef MS_WINDOWS - if (length > MAX_PATH-1) { - FORMAT_EXCEPTION(PyExc_ValueError, "%s too long for Windows"); - Py_DECREF(bytes); - goto exit; - } -#endif - narrow = PyBytes_AS_STRING(bytes); if ((size_t)length != strlen(narrow)) { FORMAT_EXCEPTION(PyExc_ValueError, "embedded null character in %s"); @@ -1002,8 +1006,35 @@ goto exit; } +#ifdef MS_WINDOWS + wo = PyUnicode_DecodeFSDefaultAndSize( + narrow, + length + ); + if (!wo) { + goto exit; + } + + wide = PyUnicode_AsWideCharString(wo, &length); + Py_DECREF(wo); + + if (!wide) { + goto exit; + } + if (length > 32767) { + FORMAT_EXCEPTION(PyExc_ValueError, "%s too long for Windows"); + goto exit; + } + if (wcslen(wide) != length) { + FORMAT_EXCEPTION(PyExc_ValueError, "embedded null character in %s"); + goto exit; + } + path->wide = wide; + path->narrow = TRUE; +#else path->wide = NULL; path->narrow = narrow; +#endif path->length = length; path->object = o; path->fd = -1; @@ -1067,7 +1098,11 @@ static int path_and_dir_fd_invalid(const char *function_name, path_t *path, int dir_fd) { - if (!path->narrow && !path->wide && (dir_fd != DEFAULT_DIR_FD)) { + if (!path->wide && (dir_fd != DEFAULT_DIR_FD) +#ifndef MS_WINDOWS + && !path->narrow +#endif + ) { PyErr_Format(PyExc_ValueError, "%s: can't specify dir_fd without matching path", function_name); @@ -1397,31 +1432,6 @@ it also needs to set "magic" environment variables indicating the per-drive current directory, which are of the form =: */ static BOOL __stdcall -win32_chdir(LPCSTR path) -{ - char new_path[MAX_PATH]; - int result; - char env[4] = "=x:"; - - if(!SetCurrentDirectoryA(path)) - return FALSE; - result = GetCurrentDirectoryA(Py_ARRAY_LENGTH(new_path), new_path); - if (!result) - return FALSE; - /* In the ANSI API, there should not be any paths longer - than MAX_PATH-1 (not including the final null character). */ - assert(result < Py_ARRAY_LENGTH(new_path)); - if (strncmp(new_path, "\\\\", 2) == 0 || - strncmp(new_path, "//", 2) == 0) - /* UNC path, nothing to do. */ - return TRUE; - env[1] = new_path[0]; - return SetEnvironmentVariableA(env, new_path); -} - -/* The Unicode version differs from the ANSI version - since the current directory might exceed MAX_PATH characters */ -static BOOL __stdcall win32_wchdir(LPCWSTR path) { wchar_t path_buf[MAX_PATH], *new_path = path_buf; @@ -1467,33 +1477,10 @@ #define HAVE_STAT_NSEC 1 #define HAVE_STRUCT_STAT_ST_FILE_ATTRIBUTES 1 -static BOOL -attributes_from_dir(LPCSTR pszFile, BY_HANDLE_FILE_INFORMATION *info, ULONG *reparse_tag) -{ - HANDLE hFindFile; - WIN32_FIND_DATAA FileData; - hFindFile = FindFirstFileA(pszFile, &FileData); - if (hFindFile == INVALID_HANDLE_VALUE) - return FALSE; - FindClose(hFindFile); - memset(info, 0, sizeof(*info)); - *reparse_tag = 0; - info->dwFileAttributes = FileData.dwFileAttributes; - info->ftCreationTime = FileData.ftCreationTime; - info->ftLastAccessTime = FileData.ftLastAccessTime; - info->ftLastWriteTime = FileData.ftLastWriteTime; - info->nFileSizeHigh = FileData.nFileSizeHigh; - info->nFileSizeLow = FileData.nFileSizeLow; -/* info->nNumberOfLinks = 1; */ - if (FileData.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) - *reparse_tag = FileData.dwReserved0; - return TRUE; -} - static void -find_data_to_file_info_w(WIN32_FIND_DATAW *pFileData, - BY_HANDLE_FILE_INFORMATION *info, - ULONG *reparse_tag) +find_data_to_file_info(WIN32_FIND_DATAW *pFileData, + BY_HANDLE_FILE_INFORMATION *info, + ULONG *reparse_tag) { memset(info, 0, sizeof(*info)); info->dwFileAttributes = pFileData->dwFileAttributes; @@ -1510,7 +1497,7 @@ } static BOOL -attributes_from_dir_w(LPCWSTR pszFile, BY_HANDLE_FILE_INFORMATION *info, ULONG *reparse_tag) +attributes_from_dir(LPCWSTR pszFile, BY_HANDLE_FILE_INFORMATION *info, ULONG *reparse_tag) { HANDLE hFindFile; WIN32_FIND_DATAW FileData; @@ -1518,7 +1505,7 @@ if (hFindFile == INVALID_HANDLE_VALUE) return FALSE; FindClose(hFindFile); - find_data_to_file_info_w(&FileData, info, reparse_tag); + find_data_to_file_info(&FileData, info, reparse_tag); return TRUE; } @@ -1561,10 +1548,7 @@ } static int -win32_xstat_impl_w(const wchar_t *path, struct _Py_stat_struct *result, - BOOL traverse); -static int -win32_xstat_impl(const char *path, struct _Py_stat_struct *result, +win32_xstat_impl(const wchar_t *path, struct _Py_stat_struct *result, BOOL traverse) { int code; @@ -1572,9 +1556,9 @@ BY_HANDLE_FILE_INFORMATION info; ULONG reparse_tag = 0; wchar_t *target_path; - const char *dot; - - hFile = CreateFileA( + const wchar_t *dot; + + hFile = CreateFileW( path, FILE_READ_ATTRIBUTES, /* desired access */ 0, /* share mode */ @@ -1623,96 +1607,6 @@ if (traverse) { /* In order to call GetFinalPathNameByHandle we need to open the file without the reparse handling flag set. */ - hFile2 = CreateFileA( - path, FILE_READ_ATTRIBUTES, FILE_SHARE_READ, - NULL, OPEN_EXISTING, - FILE_ATTRIBUTE_NORMAL|FILE_FLAG_BACKUP_SEMANTICS, - NULL); - if (hFile2 == INVALID_HANDLE_VALUE) - return -1; - - if (!get_target_path(hFile2, &target_path)) - return -1; - - code = win32_xstat_impl_w(target_path, result, FALSE); - PyMem_RawFree(target_path); - return code; - } - } else - CloseHandle(hFile); - } - _Py_attribute_data_to_stat(&info, reparse_tag, result); - - /* Set S_IEXEC if it is an .exe, .bat, ... */ - dot = strrchr(path, '.'); - if (dot) { - if (stricmp(dot, ".bat") == 0 || stricmp(dot, ".cmd") == 0 || - stricmp(dot, ".exe") == 0 || stricmp(dot, ".com") == 0) - result->st_mode |= 0111; - } - return 0; -} - -static int -win32_xstat_impl_w(const wchar_t *path, struct _Py_stat_struct *result, - BOOL traverse) -{ - int code; - HANDLE hFile, hFile2; - BY_HANDLE_FILE_INFORMATION info; - ULONG reparse_tag = 0; - wchar_t *target_path; - const wchar_t *dot; - - hFile = CreateFileW( - path, - FILE_READ_ATTRIBUTES, /* desired access */ - 0, /* share mode */ - NULL, /* security attributes */ - OPEN_EXISTING, - /* FILE_FLAG_BACKUP_SEMANTICS is required to open a directory */ - /* FILE_FLAG_OPEN_REPARSE_POINT does not follow the symlink. - Because of this, calls like GetFinalPathNameByHandle will return - the symlink path again and not the actual final path. */ - FILE_ATTRIBUTE_NORMAL|FILE_FLAG_BACKUP_SEMANTICS| - FILE_FLAG_OPEN_REPARSE_POINT, - NULL); - - if (hFile == INVALID_HANDLE_VALUE) { - /* Either the target doesn't exist, or we don't have access to - get a handle to it. If the former, we need to return an error. - If the latter, we can use attributes_from_dir. */ - if (GetLastError() != ERROR_SHARING_VIOLATION) - return -1; - /* Could not get attributes on open file. Fall back to - reading the directory. */ - if (!attributes_from_dir_w(path, &info, &reparse_tag)) - /* Very strange. This should not fail now */ - return -1; - if (info.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) { - if (traverse) { - /* Should traverse, but could not open reparse point handle */ - SetLastError(ERROR_SHARING_VIOLATION); - return -1; - } - } - } else { - if (!GetFileInformationByHandle(hFile, &info)) { - CloseHandle(hFile); - return -1; - } - if (info.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) { - if (!win32_get_reparse_tag(hFile, &reparse_tag)) - return -1; - - /* Close the outer open file handle now that we're about to - reopen it with different flags. */ - if (!CloseHandle(hFile)) - return -1; - - if (traverse) { - /* In order to call GetFinalPathNameByHandle we need to open - the file without the reparse handling flag set. */ hFile2 = CreateFileW( path, FILE_READ_ATTRIBUTES, FILE_SHARE_READ, NULL, OPEN_EXISTING, @@ -1724,7 +1618,7 @@ if (!get_target_path(hFile2, &target_path)) return -1; - code = win32_xstat_impl_w(target_path, result, FALSE); + code = win32_xstat_impl(target_path, result, FALSE); PyMem_RawFree(target_path); return code; } @@ -1744,7 +1638,7 @@ } static int -win32_xstat(const char *path, struct _Py_stat_struct *result, BOOL traverse) +win32_xstat(const wchar_t *path, struct _Py_stat_struct *result, BOOL traverse) { /* Protocol violation: we explicitly clear errno, instead of setting it to a POSIX error. Callers should use GetLastError. */ @@ -1752,16 +1646,6 @@ errno = 0; return code; } - -static int -win32_xstat_w(const wchar_t *path, struct _Py_stat_struct *result, BOOL traverse) -{ - /* Protocol violation: we explicitly clear errno, instead of - setting it to a POSIX error. Callers should use GetLastError. */ - int code = win32_xstat_impl_w(path, result, traverse); - errno = 0; - return code; -} /* About the following functions: win32_lstat_w, win32_stat, win32_stat_w In Posix, stat automatically traverses symlinks and returns the stat @@ -1771,34 +1655,20 @@ Therefore, win32_lstat will get the attributes traditionally, and win32_stat will first explicitly resolve the symlink target and then will - call win32_lstat on that result. - - The _w represent Unicode equivalents of the aforementioned ANSI functions. */ + call win32_lstat on that result. */ static int -win32_lstat(const char* path, struct _Py_stat_struct *result) +win32_lstat(const wchar_t* path, struct _Py_stat_struct *result) { return win32_xstat(path, result, FALSE); } static int -win32_lstat_w(const wchar_t* path, struct _Py_stat_struct *result) -{ - return win32_xstat_w(path, result, FALSE); -} - -static int -win32_stat(const char* path, struct _Py_stat_struct *result) +win32_stat(const wchar_t* path, struct _Py_stat_struct *result) { return win32_xstat(path, result, TRUE); } -static int -win32_stat_w(const wchar_t* path, struct _Py_stat_struct *result) -{ - return win32_xstat_w(path, result, TRUE); -} - #endif /* MS_WINDOWS */ PyDoc_STRVAR(stat_result__doc__, @@ -2200,26 +2070,25 @@ result = FSTAT(path->fd, &st); else #ifdef MS_WINDOWS - if (path->wide) { - if (follow_symlinks) - result = win32_stat_w(path->wide, &st); - else - result = win32_lstat_w(path->wide, &st); - } + if (follow_symlinks) + result = win32_stat(path->wide, &st); else -#endif -#if defined(HAVE_LSTAT) || defined(MS_WINDOWS) + result = win32_lstat(path->wide, &st); +#else + else +#if defined(HAVE_LSTAT) if ((!follow_symlinks) && (dir_fd == DEFAULT_DIR_FD)) result = LSTAT(path->narrow, &st); else -#endif +#endif /* HAVE_LSTAT */ #ifdef HAVE_FSTATAT if ((dir_fd != DEFAULT_DIR_FD) || !follow_symlinks) result = fstatat(dir_fd, path->narrow, &st, follow_symlinks ? 0 : AT_SYMLINK_NOFOLLOW); else -#endif +#endif /* HAVE_FSTATAT */ result = STAT(path->narrow, &st); +#endif /* MS_WINDOWS */ Py_END_ALLOW_THREADS if (result != 0) { @@ -2655,10 +2524,7 @@ #ifdef MS_WINDOWS Py_BEGIN_ALLOW_THREADS - if (path->wide != NULL) - attr = GetFileAttributesW(path->wide); - else - attr = GetFileAttributesA(path->narrow); + attr = GetFileAttributesW(path->wide); Py_END_ALLOW_THREADS /* @@ -2782,11 +2648,8 @@ Py_BEGIN_ALLOW_THREADS #ifdef MS_WINDOWS - if (path->wide) - result = win32_wchdir(path->wide); - else - result = win32_chdir(path->narrow); - result = !result; /* on unix, success = 0, on windows, success = !0 */ + /* on unix, success = 0, on windows, success = !0 */ + result = !win32_wchdir(path->wide); #else #ifdef HAVE_FCHDIR if (path->fd != -1) @@ -2881,10 +2744,7 @@ #ifdef MS_WINDOWS Py_BEGIN_ALLOW_THREADS - if (path->wide) - attr = GetFileAttributesW(path->wide); - else - attr = GetFileAttributesA(path->narrow); + attr = GetFileAttributesW(path->wide); if (attr == INVALID_FILE_ATTRIBUTES) result = 0; else { @@ -2892,10 +2752,7 @@ attr &= ~FILE_ATTRIBUTE_READONLY; else attr |= FILE_ATTRIBUTE_READONLY; - if (path->wide) - result = SetFileAttributesW(path->wide, attr); - else - result = SetFileAttributesA(path->narrow, attr); + result = SetFileAttributesW(path->wide, attr); } Py_END_ALLOW_THREADS @@ -3488,7 +3345,7 @@ /*[clinic end generated code: output=7f00f6007fd5269a input=b0095ebbcbaa7e04]*/ { #ifdef MS_WINDOWS - BOOL result; + BOOL result = FALSE; #else int result; #endif @@ -3500,18 +3357,18 @@ } #endif +#ifndef MS_WINDOWS if ((src->narrow && dst->wide) || (src->wide && dst->narrow)) { PyErr_SetString(PyExc_NotImplementedError, "link: src and dst must be the same type"); return NULL; } +#endif #ifdef MS_WINDOWS Py_BEGIN_ALLOW_THREADS if (src->wide) result = CreateHardLinkW(dst->wide, src->wide, NULL); - else - result = CreateHardLinkA(dst->narrow, src->narrow, NULL); Py_END_ALLOW_THREADS if (!result) @@ -3526,13 +3383,13 @@ dst_dir_fd, dst->narrow, follow_symlinks ? AT_SYMLINK_FOLLOW : 0); else -#endif +#endif /* HAVE_LINKAT */ result = link(src->narrow, dst->narrow); Py_END_ALLOW_THREADS if (result) return path_error2(src, dst); -#endif +#endif /* MS_WINDOWS */ Py_RETURN_NONE; } @@ -3546,97 +3403,39 @@ PyObject *v; HANDLE hFindFile = INVALID_HANDLE_VALUE; BOOL result; - WIN32_FIND_DATA FileData; - char namebuf[MAX_PATH+4]; /* Overallocate for "\*.*" */ + wchar_t namebuf[MAX_PATH+4]; /* Overallocate for "\*.*" */ /* only claim to have space for MAX_PATH */ Py_ssize_t len = Py_ARRAY_LENGTH(namebuf)-4; wchar_t *wnamebuf = NULL; - if (!path->narrow) { - WIN32_FIND_DATAW wFileData; - const wchar_t *po_wchars; - - if (!path->wide) { /* Default arg: "." */ - po_wchars = L"."; - len = 1; - } else { - po_wchars = path->wide; - len = wcslen(path->wide); - } - /* The +5 is so we can append "\\*.*\0" */ - wnamebuf = PyMem_New(wchar_t, len + 5); - if (!wnamebuf) { - PyErr_NoMemory(); - goto exit; - } - wcscpy(wnamebuf, po_wchars); - if (len > 0) { - wchar_t wch = wnamebuf[len-1]; - if (wch != SEP && wch != ALTSEP && wch != L':') - wnamebuf[len++] = SEP; - wcscpy(wnamebuf + len, L"*.*"); - } - if ((list = PyList_New(0)) == NULL) { - goto exit; - } - Py_BEGIN_ALLOW_THREADS - hFindFile = FindFirstFileW(wnamebuf, &wFileData); - Py_END_ALLOW_THREADS - if (hFindFile == INVALID_HANDLE_VALUE) { - int error = GetLastError(); - if (error == ERROR_FILE_NOT_FOUND) - goto exit; - Py_DECREF(list); - list = path_error(path); - goto exit; - } - do { - /* Skip over . and .. */ - if (wcscmp(wFileData.cFileName, L".") != 0 && - wcscmp(wFileData.cFileName, L"..") != 0) { - v = PyUnicode_FromWideChar(wFileData.cFileName, - wcslen(wFileData.cFileName)); - if (v == NULL) { - Py_DECREF(list); - list = NULL; - break; - } - if (PyList_Append(list, v) != 0) { - Py_DECREF(v); - Py_DECREF(list); - list = NULL; - break; - } - Py_DECREF(v); - } - Py_BEGIN_ALLOW_THREADS - result = FindNextFileW(hFindFile, &wFileData); - Py_END_ALLOW_THREADS - /* FindNextFile sets error to ERROR_NO_MORE_FILES if - it got to the end of the directory. */ - if (!result && GetLastError() != ERROR_NO_MORE_FILES) { - Py_DECREF(list); - list = path_error(path); - goto exit; - } - } while (result == TRUE); - + WIN32_FIND_DATAW wFileData; + const wchar_t *po_wchars; + + if (!path->wide) { /* Default arg: "." */ + po_wchars = L"."; + len = 1; + } else { + po_wchars = path->wide; + len = wcslen(path->wide); + } + /* The +5 is so we can append "\\*.*\0" */ + wnamebuf = PyMem_New(wchar_t, len + 5); + if (!wnamebuf) { + PyErr_NoMemory(); goto exit; } - strcpy(namebuf, path->narrow); - len = path->length; + wcscpy(wnamebuf, po_wchars); if (len > 0) { - char ch = namebuf[len-1]; - if (ch != '\\' && ch != '/' && ch != ':') - namebuf[len++] = '\\'; - strcpy(namebuf + len, "*.*"); - } - - if ((list = PyList_New(0)) == NULL) - return NULL; - + wchar_t wch = wnamebuf[len-1]; + if (wch != SEP && wch != ALTSEP && wch != L':') + wnamebuf[len++] = SEP; + wcscpy(wnamebuf + len, L"*.*"); + } + if ((list = PyList_New(0)) == NULL) { + goto exit; + } Py_BEGIN_ALLOW_THREADS - hFindFile = FindFirstFile(namebuf, &FileData); + hFindFile = FindFirstFileW(wnamebuf, &wFileData); Py_END_ALLOW_THREADS if (hFindFile == INVALID_HANDLE_VALUE) { int error = GetLastError(); @@ -3648,9 +3447,13 @@ } do { /* Skip over . and .. */ - if (strcmp(FileData.cFileName, ".") != 0 && - strcmp(FileData.cFileName, "..") != 0) { - v = PyBytes_FromString(FileData.cFileName); + if (wcscmp(wFileData.cFileName, L".") != 0 && + wcscmp(wFileData.cFileName, L"..") != 0) { + v = PyUnicode_FromWideChar(wFileData.cFileName, + wcslen(wFileData.cFileName)); + if (path->narrow && v) { + Py_SETREF(v, PyUnicode_EncodeFSDefault(v)); + } if (v == NULL) { Py_DECREF(list); list = NULL; @@ -3665,7 +3468,7 @@ Py_DECREF(v); } Py_BEGIN_ALLOW_THREADS - result = FindNextFile(hFindFile, &FileData); + result = FindNextFileW(hFindFile, &wFileData); Py_END_ALLOW_THREADS /* FindNextFile sets error to ERROR_NO_MORE_FILES if it got to the end of the directory. */ @@ -3846,41 +3649,29 @@ os__getfullpathname_impl(PyObject *module, path_t *path) /*[clinic end generated code: output=bb8679d56845bc9b input=332ed537c29d0a3e]*/ { - if (!path->narrow) - { - wchar_t woutbuf[MAX_PATH], *woutbufp = woutbuf; - wchar_t *wtemp; - DWORD result; - PyObject *v; - - result = GetFullPathNameW(path->wide, - Py_ARRAY_LENGTH(woutbuf), - woutbuf, &wtemp); - if (result > Py_ARRAY_LENGTH(woutbuf)) { - woutbufp = PyMem_New(wchar_t, result); - if (!woutbufp) - return PyErr_NoMemory(); - result = GetFullPathNameW(path->wide, result, woutbufp, &wtemp); - } - if (result) - v = PyUnicode_FromWideChar(woutbufp, wcslen(woutbufp)); - else - v = win32_error_object("GetFullPathNameW", path->object); - if (woutbufp != woutbuf) - PyMem_Free(woutbufp); - return v; - } - else { - char outbuf[MAX_PATH]; - char *temp; - - if (!GetFullPathName(path->narrow, Py_ARRAY_LENGTH(outbuf), - outbuf, &temp)) { - win32_error_object("GetFullPathName", path->object); - return NULL; - } - return PyBytes_FromString(outbuf); - } + wchar_t woutbuf[MAX_PATH], *woutbufp = woutbuf; + wchar_t *wtemp; + DWORD result; + PyObject *v; + + result = GetFullPathNameW(path->wide, + Py_ARRAY_LENGTH(woutbuf), + woutbuf, &wtemp); + if (result > Py_ARRAY_LENGTH(woutbuf)) { + woutbufp = PyMem_New(wchar_t, result); + if (!woutbufp) + return PyErr_NoMemory(); + result = GetFullPathNameW(path->wide, result, woutbufp, &wtemp); + } + if (result) { + v = PyUnicode_FromWideChar(woutbufp, wcslen(woutbufp)); + if (path->narrow) + Py_SETREF(v, PyUnicode_EncodeFSDefault(v)); + } else + v = win32_error_object("GetFullPathNameW", path->object); + if (woutbufp != woutbuf) + PyMem_Free(woutbufp); + return v; } @@ -3964,10 +3755,7 @@ DWORD attributes; Py_BEGIN_ALLOW_THREADS - if (!path->narrow) - attributes = GetFileAttributesW(path->wide); - else - attributes = GetFileAttributesA(path->narrow); + attributes = GetFileAttributesW(path->wide); Py_END_ALLOW_THREADS if (attributes == INVALID_FILE_ATTRIBUTES) @@ -4065,10 +3853,7 @@ #ifdef MS_WINDOWS Py_BEGIN_ALLOW_THREADS - if (path->wide) - result = CreateDirectoryW(path->wide, NULL); - else - result = CreateDirectoryA(path->narrow, NULL); + result = CreateDirectoryW(path->wide, NULL); Py_END_ALLOW_THREADS if (!result) @@ -4088,7 +3873,7 @@ Py_END_ALLOW_THREADS if (result < 0) return path_error(path); -#endif +#endif /* MS_WINDOWS */ Py_RETURN_NONE; } @@ -4211,31 +3996,28 @@ } #endif +#ifdef MS_WINDOWS + Py_BEGIN_ALLOW_THREADS + result = MoveFileExW(src->wide, dst->wide, flags); + Py_END_ALLOW_THREADS + + if (!result) + return path_error2(src, dst); + +#else if ((src->narrow && dst->wide) || (src->wide && dst->narrow)) { PyErr_Format(PyExc_ValueError, "%s: src and dst must be the same type", function_name); return NULL; } -#ifdef MS_WINDOWS - Py_BEGIN_ALLOW_THREADS - if (src->wide) - result = MoveFileExW(src->wide, dst->wide, flags); - else - result = MoveFileExA(src->narrow, dst->narrow, flags); - Py_END_ALLOW_THREADS - - if (!result) - return path_error2(src, dst); - -#else Py_BEGIN_ALLOW_THREADS #ifdef HAVE_RENAMEAT if (dir_fd_specified) result = renameat(src_dir_fd, src->narrow, dst_dir_fd, dst->narrow); else #endif - result = rename(src->narrow, dst->narrow); + result = rename(src->narrow, dst->narrow); Py_END_ALLOW_THREADS if (result) @@ -4316,11 +4098,8 @@ Py_BEGIN_ALLOW_THREADS #ifdef MS_WINDOWS - if (path->wide) - result = RemoveDirectoryW(path->wide); - else - result = RemoveDirectoryA(path->narrow); - result = !result; /* Windows, success=1, UNIX, success=0 */ + /* Windows, success=1, UNIX, success=0 */ + result = !RemoveDirectoryW(path->wide); #else #ifdef HAVE_UNLINKAT if (dir_fd != DEFAULT_DIR_FD) @@ -4466,11 +4245,8 @@ Py_BEGIN_ALLOW_THREADS _Py_BEGIN_SUPPRESS_IPH #ifdef MS_WINDOWS - if (path->wide) - result = Py_DeleteFileW(path->wide); - else - result = DeleteFileA(path->narrow); - result = !result; /* Windows, success=1, UNIX, success=0 */ + /* Windows, success=1, UNIX, success=0 */ + result = !Py_DeleteFileW(path->wide); #else #ifdef HAVE_UNLINKAT if (dir_fd != DEFAULT_DIR_FD) @@ -4881,14 +4657,9 @@ #ifdef MS_WINDOWS Py_BEGIN_ALLOW_THREADS - if (path->wide) - hFile = CreateFileW(path->wide, FILE_WRITE_ATTRIBUTES, 0, - NULL, OPEN_EXISTING, - FILE_FLAG_BACKUP_SEMANTICS, NULL); - else - hFile = CreateFileA(path->narrow, FILE_WRITE_ATTRIBUTES, 0, - NULL, OPEN_EXISTING, - FILE_FLAG_BACKUP_SEMANTICS, NULL); + hFile = CreateFileW(path->wide, FILE_WRITE_ATTRIBUTES, 0, + NULL, OPEN_EXISTING, + FILE_FLAG_BACKUP_SEMANTICS, NULL); Py_END_ALLOW_THREADS if (hFile == INVALID_HANDLE_VALUE) { path_error(path); @@ -4974,9 +4745,15 @@ return NULL; /* Make gcc -Wall happy */ } +#if defined(HAVE_WEXECV) || defined(HAVE_WSPAWNV) +#define EXECV_CHAR wchar_t +#else +#define EXECV_CHAR char +#endif + #if defined(HAVE_EXECV) || defined(HAVE_SPAWNV) static void -free_string_array(char **array, Py_ssize_t count) +free_string_array(EXECV_CHAR **array, Py_ssize_t count) { Py_ssize_t i; for (i = 0; i < count; i++) @@ -4985,10 +4762,15 @@ } static -int fsconvert_strdup(PyObject *o, char**out) -{ +int fsconvert_strdup(PyObject *o, EXECV_CHAR**out) +{ + Py_ssize_t size; +#if defined(HAVE_WEXECV) || defined(HAVE_WSPAWNV) + *out = PyUnicode_AsWideCharString(o, &size); + if (!*out) + return 0; +#else PyObject *bytes; - Py_ssize_t size; if (!PyUnicode_FSConverter(o, &bytes)) return 0; size = PyBytes_GET_SIZE(bytes); @@ -4999,26 +4781,24 @@ } memcpy(*out, PyBytes_AsString(bytes), size+1); Py_DECREF(bytes); +#endif return 1; } #endif #if defined(HAVE_EXECV) || defined (HAVE_FEXECVE) -static char** +static EXECV_CHAR** parse_envlist(PyObject* env, Py_ssize_t *envc_ptr) { - char **envlist; Py_ssize_t i, pos, envc; PyObject *keys=NULL, *vals=NULL; - PyObject *key, *val, *key2, *val2; - char *p; - const char *k, *v; - size_t len; + PyObject *key, *val, *keyval; + EXECV_CHAR **envlist; i = PyMapping_Size(env); if (i < 0) return NULL; - envlist = PyMem_NEW(char *, i + 1); + envlist = PyMem_NEW(EXECV_CHAR *, i + 1); if (envlist == NULL) { PyErr_NoMemory(); return NULL; @@ -5042,28 +4822,16 @@ if (!key || !val) goto error; - if (PyUnicode_FSConverter(key, &key2) == 0) + keyval = PyUnicode_FromFormat("%U=%U", key, val); + if (!keyval) goto error; - if (PyUnicode_FSConverter(val, &val2) == 0) { - Py_DECREF(key2); + + if (!fsconvert_strdup(keyval, &envlist[envc++])) { + Py_DECREF(keyval); goto error; } - - k = PyBytes_AsString(key2); - v = PyBytes_AsString(val2); - len = PyBytes_GET_SIZE(key2) + PyBytes_GET_SIZE(val2) + 2; - - p = PyMem_NEW(char, len); - if (p == NULL) { - PyErr_NoMemory(); - Py_DECREF(key2); - Py_DECREF(val2); - goto error; - } - PyOS_snprintf(p, len, "%s=%s", k, v); - envlist[envc++] = p; - Py_DECREF(key2); - Py_DECREF(val2); + + Py_DECREF(keyval); } Py_DECREF(vals); Py_DECREF(keys); @@ -5075,17 +4843,15 @@ error: Py_XDECREF(keys); Py_XDECREF(vals); - while (--envc >= 0) - PyMem_DEL(envlist[envc]); - PyMem_DEL(envlist); + free_string_array(envlist, envc); return NULL; } -static char** +static EXECV_CHAR** parse_arglist(PyObject* argv, Py_ssize_t *argc) { int i; - char **argvlist = PyMem_NEW(char *, *argc+1); + EXECV_CHAR **argvlist = PyMem_NEW(EXECV_CHAR *, *argc+1); if (argvlist == NULL) { PyErr_NoMemory(); return NULL; @@ -5107,6 +4873,7 @@ free_string_array(argvlist, *argc); return NULL; } + #endif @@ -5114,7 +4881,7 @@ /*[clinic input] os.execv - path: FSConverter + path: path_t Path of executable file. argv: object Tuple or list of strings. @@ -5124,17 +4891,15 @@ [clinic start generated code]*/ static PyObject * -os_execv_impl(PyObject *module, PyObject *path, PyObject *argv) -/*[clinic end generated code: output=b21dc34deeb5b004 input=96041559925e5229]*/ -{ - const char *path_char; - char **argvlist; +os_execv_impl(PyObject *module, path_t *path, PyObject *argv) +/*[clinic end generated code: output=3b52fec34cd0dafd input=9bac31efae07dac7]*/ +{ + EXECV_CHAR **argvlist; Py_ssize_t argc; /* execv has two arguments: (path, argv), where argv is a list or tuple of strings. */ - path_char = PyBytes_AsString(path); if (!PyList_Check(argv) && !PyTuple_Check(argv)) { PyErr_SetString(PyExc_TypeError, "execv() arg 2 must be a tuple or list"); @@ -5151,7 +4916,11 @@ return NULL; } - execv(path_char, argvlist); +#ifdef HAVE_WEXECV + _wexecv(path->wide, argvlist); +#else + execv(path->narrow, argvlist); +#endif /* If we get here it's definitely an error */ @@ -5177,8 +4946,8 @@ os_execve_impl(PyObject *module, path_t *path, PyObject *argv, PyObject *env) /*[clinic end generated code: output=ff9fa8e4da8bde58 input=626804fa092606d9]*/ { - char **argvlist = NULL; - char **envlist; + EXECV_CHAR **argvlist = NULL; + EXECV_CHAR **envlist; Py_ssize_t argc, envc; /* execve has three arguments: (path, argv, env), where @@ -5211,30 +4980,33 @@ fexecve(path->fd, argvlist, envlist); else #endif +#ifdef HAVE_WEXECV + _wexecve(path->wide, argvlist, envlist); +#else execve(path->narrow, argvlist, envlist); +#endif /* If we get here it's definitely an error */ path_error(path); - while (--envc >= 0) - PyMem_DEL(envlist[envc]); - PyMem_DEL(envlist); + free_string_array(envlist, envc); fail: if (argvlist) free_string_array(argvlist, argc); return NULL; } + #endif /* HAVE_EXECV */ -#ifdef HAVE_SPAWNV +#if defined(HAVE_SPAWNV) || defined(HAVE_WSPAWNV) /*[clinic input] os.spawnv mode: int Mode of process creation. - path: FSConverter + path: path_t Path of executable file. argv: object Tuple or list of strings. @@ -5244,11 +5016,10 @@ [clinic start generated code]*/ static PyObject * -os_spawnv_impl(PyObject *module, int mode, PyObject *path, PyObject *argv) -/*[clinic end generated code: output=c427c0ce40f10638 input=042c91dfc1e6debc]*/ -{ - const char *path_char; - char **argvlist; +os_spawnv_impl(PyObject *module, int mode, path_t *path, PyObject *argv) +/*[clinic end generated code: output=71cd037a9d96b816 input=43224242303291be]*/ +{ + EXECV_CHAR **argvlist; int i; Py_ssize_t argc; intptr_t spawnval; @@ -5257,7 +5028,6 @@ /* spawnv has three arguments: (mode, path, argv), where argv is a list or tuple of strings. */ - path_char = PyBytes_AsString(path); if (PyList_Check(argv)) { argc = PyList_Size(argv); getitem = PyList_GetItem; @@ -5272,7 +5042,7 @@ return NULL; } - argvlist = PyMem_NEW(char *, argc+1); + argvlist = PyMem_NEW(EXECV_CHAR *, argc+1); if (argvlist == NULL) { return PyErr_NoMemory(); } @@ -5292,7 +5062,11 @@ mode = _P_OVERLAY; Py_BEGIN_ALLOW_THREADS - spawnval = _spawnv(mode, path_char, argvlist); +#ifdef HAVE_WSPAWNV + spawnval = _wspawnv(mode, path->wide, argvlist); +#else + spawnval = _spawnv(mode, path->narrow, argvlist); +#endif Py_END_ALLOW_THREADS free_string_array(argvlist, argc); @@ -5309,7 +5083,7 @@ mode: int Mode of process creation. - path: FSConverter + path: path_t Path of executable file. argv: object Tuple or list of strings. @@ -5321,13 +5095,12 @@ [clinic start generated code]*/ static PyObject * -os_spawnve_impl(PyObject *module, int mode, PyObject *path, PyObject *argv, +os_spawnve_impl(PyObject *module, int mode, path_t *path, PyObject *argv, PyObject *env) -/*[clinic end generated code: output=ebcfa5f7ba2f4219 input=02362fd937963f8f]*/ -{ - const char *path_char; - char **argvlist; - char **envlist; +/*[clinic end generated code: output=30fe85be56fe37ad input=3e40803ee7c4c586]*/ +{ + EXECV_CHAR **argvlist; + EXECV_CHAR **envlist; PyObject *res = NULL; Py_ssize_t argc, i, envc; intptr_t spawnval; @@ -5338,7 +5111,6 @@ argv is a list or tuple of strings and env is a dictionary like posix.environ. */ - path_char = PyBytes_AsString(path); if (PyList_Check(argv)) { argc = PyList_Size(argv); getitem = PyList_GetItem; @@ -5358,7 +5130,7 @@ goto fail_0; } - argvlist = PyMem_NEW(char *, argc+1); + argvlist = PyMem_NEW(EXECV_CHAR *, argc+1); if (argvlist == NULL) { PyErr_NoMemory(); goto fail_0; @@ -5382,7 +5154,11 @@ mode = _P_OVERLAY; Py_BEGIN_ALLOW_THREADS - spawnval = _spawnve(mode, path_char, argvlist, envlist); +#ifdef HAVE_WSPAWNV + spawnval = _wspawnve(mode, path->wide, argvlist, envlist); +#else + spawnval = _spawnve(mode, path->narrow, argvlist, envlist); +#endif Py_END_ALLOW_THREADS if (spawnval == -1) @@ -7290,21 +7066,18 @@ /* Grab CreateSymbolicLinkW dynamically from kernel32 */ static DWORD (CALLBACK *Py_CreateSymbolicLinkW)(LPCWSTR, LPCWSTR, DWORD) = NULL; -static DWORD (CALLBACK *Py_CreateSymbolicLinkA)(LPCSTR, LPCSTR, DWORD) = NULL; static int check_CreateSymbolicLink(void) { HINSTANCE hKernel32; /* only recheck */ - if (Py_CreateSymbolicLinkW && Py_CreateSymbolicLinkA) + if (Py_CreateSymbolicLinkW) return 1; hKernel32 = GetModuleHandleW(L"KERNEL32"); *(FARPROC*)&Py_CreateSymbolicLinkW = GetProcAddress(hKernel32, "CreateSymbolicLinkW"); - *(FARPROC*)&Py_CreateSymbolicLinkA = GetProcAddress(hKernel32, - "CreateSymbolicLinkA"); - return (Py_CreateSymbolicLinkW && Py_CreateSymbolicLinkA); + return Py_CreateSymbolicLinkW != NULL; } /* Remove the last portion of the path */ @@ -7321,20 +7094,6 @@ *ptr = 0; } -/* Remove the last portion of the path */ -static void -_dirnameA(char *path) -{ - char *ptr; - - /* walk the path from the end until a backslash is encountered */ - for(ptr = path + strlen(path); ptr != path; ptr--) { - if (*ptr == '\\' || *ptr == '/') - break; - } - *ptr = 0; -} - /* Is this path absolute? */ static int _is_absW(const WCHAR *path) @@ -7343,14 +7102,6 @@ } -/* Is this path absolute? */ -static int -_is_absA(const char *path) -{ - return path[0] == '\\' || path[0] == '/' || path[1] == ':'; - -} - /* join root and rest with a backslash */ static void _joinW(WCHAR *dest_path, const WCHAR *root, const WCHAR *rest) @@ -7372,27 +7123,6 @@ wcscpy(dest_path+root_len, rest); } -/* join root and rest with a backslash */ -static void -_joinA(char *dest_path, const char *root, const char *rest) -{ - size_t root_len; - - if (_is_absA(rest)) { - strcpy(dest_path, rest); - return; - } - - root_len = strlen(root); - - strcpy(dest_path, root); - if(root_len) { - dest_path[root_len] = '\\'; - root_len++; - } - strcpy(dest_path+root_len, rest); -} - /* Return True if the path at src relative to dest is a directory */ static int _check_dirW(LPCWSTR src, LPCWSTR dest) @@ -7411,25 +7141,6 @@ && src_info.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY ); } - -/* Return True if the path at src relative to dest is a directory */ -static int -_check_dirA(LPCSTR src, LPCSTR dest) -{ - WIN32_FILE_ATTRIBUTE_DATA src_info; - char dest_parent[MAX_PATH]; - char src_resolved[MAX_PATH] = ""; - - /* dest_parent = os.path.dirname(dest) */ - strcpy(dest_parent, dest); - _dirnameA(dest_parent); - /* src_resolved = os.path.join(dest_parent, src) */ - _joinA(src_resolved, dest_parent, src); - return ( - GetFileAttributesExA(src_resolved, GetFileExInfoStandard, &src_info) - && src_info.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY - ); -} #endif @@ -7489,18 +7200,10 @@ #ifdef MS_WINDOWS Py_BEGIN_ALLOW_THREADS - if (dst->wide) { - /* if src is a directory, ensure target_is_directory==1 */ - target_is_directory |= _check_dirW(src->wide, dst->wide); - result = Py_CreateSymbolicLinkW(dst->wide, src->wide, - target_is_directory); - } - else { - /* if src is a directory, ensure target_is_directory==1 */ - target_is_directory |= _check_dirA(src->narrow, dst->narrow); - result = Py_CreateSymbolicLinkA(dst->narrow, src->narrow, - target_is_directory); - } + /* if src is a directory, ensure target_is_directory==1 */ + target_is_directory |= _check_dirW(src->wide, dst->wide); + result = Py_CreateSymbolicLinkW(dst->wide, src->wide, + target_is_directory); Py_END_ALLOW_THREADS if (!result) @@ -7805,16 +7508,14 @@ do { Py_BEGIN_ALLOW_THREADS #ifdef MS_WINDOWS - if (path->wide) - fd = _wopen(path->wide, flags, mode); - else + fd = _wopen(path->wide, flags, mode); #endif #ifdef HAVE_OPENAT if (dir_fd != DEFAULT_DIR_FD) fd = openat(dir_fd, path->narrow, flags, mode); else -#endif fd = open(path->narrow, flags, mode); +#endif Py_END_ALLOW_THREADS } while (fd < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); _Py_END_SUPPRESS_IPH @@ -8955,10 +8656,7 @@ Py_BEGIN_ALLOW_THREADS _Py_BEGIN_SUPPRESS_IPH #ifdef MS_WINDOWS - if (path->wide) - fd = _wopen(path->wide, _O_WRONLY | _O_BINARY | _O_NOINHERIT); - else - fd = _open(path->narrow, _O_WRONLY | _O_BINARY | _O_NOINHERIT); + fd = _wopen(path->wide, _O_WRONLY | _O_BINARY | _O_NOINHERIT); if (fd < 0) result = -1; else { @@ -10612,31 +10310,8 @@ } #ifdef MS_WINDOWS -/* AC 3.5: change to path_t? but that might change exceptions */ -PyDoc_STRVAR(win32_startfile__doc__, -"startfile(filepath [, operation])\n\ -\n\ -Start a file with its associated application.\n\ -\n\ -When \"operation\" is not specified or \"open\", this acts like\n\ -double-clicking the file in Explorer, or giving the file name as an\n\ -argument to the DOS \"start\" command: the file is opened with whatever\n\ -application (if any) its extension is associated.\n\ -When another \"operation\" is given, it specifies what should be done with\n\ -the file. A typical operation is \"print\".\n\ -\n\ -startfile returns as soon as the associated application is launched.\n\ -There is no option to wait for the application to close, and no way\n\ -to retrieve the application's exit status.\n\ -\n\ -The filepath is relative to the current directory. If you want to use\n\ -an absolute path, make sure the first character is not a slash (\"/\");\n\ -the underlying Win32 ShellExecute function doesn't work if it is."); - /* Grab ShellExecute dynamically from shell32 */ static int has_ShellExecute = -1; -static HINSTANCE (CALLBACK *Py_ShellExecuteA)(HWND, LPCSTR, LPCSTR, LPCSTR, - LPCSTR, INT); static HINSTANCE (CALLBACK *Py_ShellExecuteW)(HWND, LPCWSTR, LPCWSTR, LPCWSTR, LPCWSTR, INT); static int @@ -10650,12 +10325,9 @@ hShell32 = LoadLibraryW(L"SHELL32"); Py_END_ALLOW_THREADS if (hShell32) { - *(FARPROC*)&Py_ShellExecuteA = GetProcAddress(hShell32, - "ShellExecuteA"); *(FARPROC*)&Py_ShellExecuteW = GetProcAddress(hShell32, "ShellExecuteW"); - has_ShellExecute = Py_ShellExecuteA && - Py_ShellExecuteW; + has_ShellExecute = Py_ShellExecuteW != NULL; } else { has_ShellExecute = 0; } @@ -10664,17 +10336,37 @@ } -static PyObject * -win32_startfile(PyObject *self, PyObject *args) -{ - PyObject *ofilepath; - const char *filepath; - const char *operation = NULL; - const wchar_t *wpath, *woperation; +/*[clinic input] +os.startfile + filepath: path_t + operation: Py_UNICODE = NULL + +startfile(filepath [, operation]) + +Start a file with its associated application. + +When "operation" is not specified or "open", this acts like +double-clicking the file in Explorer, or giving the file name as an +argument to the DOS "start" command: the file is opened with whatever +application (if any) its extension is associated. +When another "operation" is given, it specifies what should be done with +the file. A typical operation is "print". + +startfile returns as soon as the associated application is launched. +There is no option to wait for the application to close, and no way +to retrieve the application's exit status. + +The filepath is relative to the current directory. If you want to use +an absolute path, make sure the first character is not a slash ("/"); +the underlying Win32 ShellExecute function doesn't work if it is. +[clinic start generated code]*/ + +static PyObject * +os_startfile_impl(PyObject *module, path_t *filepath, Py_UNICODE *operation) +/*[clinic end generated code: output=912ceba79acfa1c9 input=63950bf2986380d0]*/ +{ HINSTANCE rc; - PyObject *unipath, *uoperation = NULL; - if(!check_ShellExecute()) { /* If the OS doesn't have ShellExecute, return a NotImplementedError. */ @@ -10682,68 +10374,16 @@ "startfile not available on this platform"); } - if (!PyArg_ParseTuple(args, "U|s:startfile", - &unipath, &operation)) { - PyErr_Clear(); - goto normal; - } - - if (operation) { - uoperation = PyUnicode_DecodeASCII(operation, - strlen(operation), NULL); - if (!uoperation) { - PyErr_Clear(); - operation = NULL; - goto normal; - } - } - - wpath = PyUnicode_AsUnicode(unipath); - if (wpath == NULL) - goto normal; - if (uoperation) { - woperation = PyUnicode_AsUnicode(uoperation); - if (woperation == NULL) - goto normal; - } - else - woperation = NULL; - Py_BEGIN_ALLOW_THREADS - rc = Py_ShellExecuteW((HWND)0, woperation, wpath, + rc = Py_ShellExecuteW((HWND)0, operation, filepath->wide, NULL, NULL, SW_SHOWNORMAL); Py_END_ALLOW_THREADS - Py_XDECREF(uoperation); if (rc <= (HINSTANCE)32) { - win32_error_object("startfile", unipath); - return NULL; - } - Py_INCREF(Py_None); - return Py_None; - -normal: - if (!PyArg_ParseTuple(args, "O&|s:startfile", - PyUnicode_FSConverter, &ofilepath, - &operation)) - return NULL; - if (win32_warn_bytes_api()) { - Py_DECREF(ofilepath); - return NULL; - } - filepath = PyBytes_AsString(ofilepath); - Py_BEGIN_ALLOW_THREADS - rc = Py_ShellExecuteA((HWND)0, operation, filepath, - NULL, NULL, SW_SHOWNORMAL); - Py_END_ALLOW_THREADS - if (rc <= (HINSTANCE)32) { - PyObject *errval = win32_error("startfile", filepath); - Py_DECREF(ofilepath); - return errval; - } - Py_DECREF(ofilepath); - Py_INCREF(Py_None); - return Py_None; + win32_error_object("startfile", filepath->object); + return NULL; + } + Py_RETURN_NONE; } #endif /* MS_WINDOWS */ @@ -11560,9 +11200,9 @@ return NULL; if (follow_symlinks) - result = win32_stat_w(path, &st); + result = win32_stat(path, &st); else - result = win32_lstat_w(path, &st); + result = win32_lstat(path, &st); if (result != 0) { return PyErr_SetExcFromWindowsErrWithFilenameObject(PyExc_OSError, @@ -11761,7 +11401,7 @@ if (!path) return NULL; - if (win32_lstat_w(path, &stat) != 0) { + if (win32_lstat(path, &stat) != 0) { return PyErr_SetExcFromWindowsErrWithFilenameObject(PyExc_OSError, 0, self->path); } @@ -11910,6 +11550,11 @@ entry->name = PyUnicode_FromWideChar(dataW->cFileName, -1); if (!entry->name) goto error; + if (path->narrow) { + Py_SETREF(entry->name, PyUnicode_EncodeFSDefault(entry->name)); + if (!entry->name) + goto error; + } joined_path = join_path_filenameW(path->wide, dataW->cFileName); if (!joined_path) @@ -11919,8 +11564,13 @@ PyMem_Free(joined_path); if (!entry->path) goto error; - - find_data_to_file_info_w(dataW, &file_info, &reparse_tag); + if (path->narrow) { + Py_SETREF(entry->path, PyUnicode_EncodeFSDefault(entry->path)); + if (!entry->path) + goto error; + } + + find_data_to_file_info(dataW, &file_info, &reparse_tag); _Py_attribute_data_to_stat(&file_info, reparse_tag, &entry->win32_lstat); return (PyObject *)entry; @@ -12316,11 +11966,6 @@ Py_XINCREF(iterator->path.object); #ifdef MS_WINDOWS - if (iterator->path.narrow) { - PyErr_SetString(PyExc_TypeError, - "os.scandir() doesn't support bytes path on Windows, use Unicode instead"); - goto error; - } iterator->first_time = 1; path_strW = join_path_filenameW(iterator->path.wide, L"*.*"); @@ -12570,7 +12215,7 @@ OS_KILLPG_METHODDEF OS_PLOCK_METHODDEF #ifdef MS_WINDOWS - {"startfile", win32_startfile, METH_VARARGS, win32_startfile__doc__}, + OS_STARTFILE_METHODDEF #endif OS_SETUID_METHODDEF OS_SETEUID_METHODDEF diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3185,7 +3185,7 @@ || strcmp(lower, "us_ascii") == 0) { return PyUnicode_DecodeASCII(s, size, errors); } - #ifdef HAVE_MBCS + #ifdef MS_WINDOWS else if (strcmp(lower, "mbcs") == 0) { return PyUnicode_DecodeMBCS(s, size, errors); } @@ -3507,10 +3507,8 @@ PyObject * PyUnicode_EncodeFSDefault(PyObject *unicode) { -#ifdef HAVE_MBCS - return PyUnicode_EncodeCodePage(CP_ACP, unicode, NULL); -#elif defined(__APPLE__) - return _PyUnicode_AsUTF8String(unicode, "surrogateescape"); +#if defined(__APPLE__) + return _PyUnicode_AsUTF8String(unicode, Py_FileSystemDefaultEncodeErrors); #else PyInterpreterState *interp = PyThreadState_GET()->interp; /* Bootstrap check: if the filesystem codec is implemented in Python, we @@ -3525,10 +3523,10 @@ if (Py_FileSystemDefaultEncoding && interp->fscodec_initialized) { return PyUnicode_AsEncodedString(unicode, Py_FileSystemDefaultEncoding, - "surrogateescape"); + Py_FileSystemDefaultEncodeErrors); } else { - return PyUnicode_EncodeLocale(unicode, "surrogateescape"); + return PyUnicode_EncodeLocale(unicode, Py_FileSystemDefaultEncodeErrors); } #endif } @@ -3577,7 +3575,7 @@ || strcmp(lower, "us_ascii") == 0) { return _PyUnicode_AsASCIIString(unicode, errors); } -#ifdef HAVE_MBCS +#ifdef MS_WINDOWS else if (strcmp(lower, "mbcs") == 0) { return PyUnicode_EncodeCodePage(CP_ACP, unicode, errors); } @@ -3813,10 +3811,8 @@ PyObject* PyUnicode_DecodeFSDefaultAndSize(const char *s, Py_ssize_t size) { -#ifdef HAVE_MBCS - return PyUnicode_DecodeMBCS(s, size, NULL); -#elif defined(__APPLE__) - return PyUnicode_DecodeUTF8Stateful(s, size, "surrogateescape", NULL); +#if defined(__APPLE__) + return PyUnicode_DecodeUTF8Stateful(s, size, Py_FileSystemDefaultEncodeErrors, NULL); #else PyInterpreterState *interp = PyThreadState_GET()->interp; /* Bootstrap check: if the filesystem codec is implemented in Python, we @@ -3829,12 +3825,24 @@ cannot only rely on it: check also interp->fscodec_initialized for subinterpreters. */ if (Py_FileSystemDefaultEncoding && interp->fscodec_initialized) { - return PyUnicode_Decode(s, size, + PyObject *res = PyUnicode_Decode(s, size, Py_FileSystemDefaultEncoding, - "surrogateescape"); + Py_FileSystemDefaultEncodeErrors); +#ifdef MS_WINDOWS + if (!res && PyErr_ExceptionMatches(PyExc_UnicodeDecodeError)) { + PyObject *exc, *val, *tb; + PyErr_Fetch(&exc, &val, &tb); + PyErr_Format(PyExc_RuntimeError, + "filesystem path bytes were not correctly encoded with '%s'. " \ + "Please report this at http://bugs.python.org/issue27781", + Py_FileSystemDefaultEncoding); + _PyErr_ChainExceptions(exc, val, tb); + } +#endif + return res; } else { - return PyUnicode_DecodeLocaleAndSize(s, size, "surrogateescape"); + return PyUnicode_DecodeLocaleAndSize(s, size, Py_FileSystemDefaultEncodeErrors); } #endif } @@ -4218,7 +4226,7 @@ Py_CLEAR(*exceptionObject); } -#ifdef HAVE_MBCS +#ifdef MS_WINDOWS /* error handling callback helper: build arguments, call the callback and check the arguments, if no exception occurred, copy the replacement to the output @@ -4332,7 +4340,7 @@ Py_XDECREF(restuple); return -1; } -#endif /* HAVE_MBCS */ +#endif /* MS_WINDOWS */ static int unicode_decode_call_errorhandler_writer( @@ -7022,7 +7030,7 @@ return _PyUnicode_AsASCIIString(unicode, NULL); } -#ifdef HAVE_MBCS +#ifdef MS_WINDOWS /* --- MBCS codecs for Windows -------------------------------------------- */ @@ -7741,7 +7749,7 @@ #undef NEED_RETRY -#endif /* HAVE_MBCS */ +#endif /* MS_WINDOWS */ /* --- Character Mapping Codec -------------------------------------------- */ diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -21,16 +21,18 @@ Don't forget to modify PyUnicode_DecodeFSDefault() if you touch any of the values for Py_FileSystemDefaultEncoding! */ -#ifdef HAVE_MBCS -const char *Py_FileSystemDefaultEncoding = "mbcs"; +#if defined(__APPLE__) +const char *Py_FileSystemDefaultEncoding = "utf-8"; int Py_HasFileSystemDefaultEncoding = 1; -#elif defined(__APPLE__) +#elif defined(MS_WINDOWS) +/* may be changed by initfsencoding(), but should never be free()d */ const char *Py_FileSystemDefaultEncoding = "utf-8"; int Py_HasFileSystemDefaultEncoding = 1; #else const char *Py_FileSystemDefaultEncoding = NULL; /* set by initfsencoding() */ int Py_HasFileSystemDefaultEncoding = 0; #endif +const char *Py_FileSystemDefaultEncodeErrors = "surrogateescape"; _Py_IDENTIFIER(__builtins__); _Py_IDENTIFIER(__dict__); diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -90,6 +90,9 @@ int Py_UnbufferedStdioFlag = 0; /* Unbuffered binary std{in,out,err} */ int Py_HashRandomizationFlag = 0; /* for -R and PYTHONHASHSEED */ int Py_IsolatedFlag = 0; /* for -I, isolate from user's env */ +#ifdef MS_WINDOWS +int Py_LegacyWindowsFSEncodingFlag = 0; /* Uses mbcs instead of utf-8 */ +#endif PyThreadState *_Py_Finalizing = NULL; @@ -321,6 +324,10 @@ check its value further. */ if ((p = Py_GETENV("PYTHONHASHSEED")) && *p != '\0') Py_HashRandomizationFlag = add_flag(Py_HashRandomizationFlag, p); +#ifdef MS_WINDOWS + if ((p = Py_GETENV("PYTHONLEGACYWINDOWSFSENCODING")) && *p != '\0') + Py_LegacyWindowsFSEncodingFlag = add_flag(Py_LegacyWindowsFSEncodingFlag, p); +#endif _PyRandom_Init(); @@ -958,6 +965,18 @@ { PyObject *codec; +#ifdef MS_WINDOWS + if (Py_LegacyWindowsFSEncodingFlag) + { + Py_FileSystemDefaultEncoding = "mbcs"; + Py_FileSystemDefaultEncodeErrors = "replace"; + } + else + { + Py_FileSystemDefaultEncoding = "utf-8"; + Py_FileSystemDefaultEncodeErrors = "surrogatepass"; + } +#else if (Py_FileSystemDefaultEncoding == NULL) { Py_FileSystemDefaultEncoding = get_locale_encoding(); @@ -968,6 +987,7 @@ interp->fscodec_initialized = 1; return 0; } +#endif /* the encoding is mbcs, utf-8 or ascii */ codec = _PyCodec_Lookup(Py_FileSystemDefaultEncoding); diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -311,6 +311,23 @@ ); static PyObject * +sys_getfilesystemencodeerrors(PyObject *self) +{ + if (Py_FileSystemDefaultEncodeErrors) + return PyUnicode_FromString(Py_FileSystemDefaultEncodeErrors); + PyErr_SetString(PyExc_RuntimeError, + "filesystem encoding is not initialized"); + return NULL; +} + +PyDoc_STRVAR(getfilesystemencodeerrors_doc, + "getfilesystemencodeerrors() -> string\n\ +\n\ +Return the error mode used to convert Unicode filenames in\n\ +operating system filenames." +); + +static PyObject * sys_intern(PyObject *self, PyObject *args) { PyObject *s; @@ -866,6 +883,24 @@ #pragma warning(pop) +PyDoc_STRVAR(enablelegacywindowsfsencoding_doc, +"_enablelegacywindowsfsencoding()\n\ +\n\ +Changes the default filesystem encoding to mbcs:replace for consistency\n\ +with earlier versions of Python. See PEP 529 for more information.\n\ +\n\ +This is equivalent to defining the PYTHONLEGACYWINDOWSFSENCODING \n\ +environment variable before launching Python." +); + +static PyObject * +sys_enablelegacywindowsfsencoding(PyObject *self) +{ + Py_FileSystemDefaultEncoding = "mbcs"; + Py_FileSystemDefaultEncodeErrors = "replace"; + Py_RETURN_NONE; +} + #endif /* MS_WINDOWS */ #ifdef HAVE_DLOPEN @@ -1225,6 +1260,8 @@ #endif {"getfilesystemencoding", (PyCFunction)sys_getfilesystemencoding, METH_NOARGS, getfilesystemencoding_doc}, + { "getfilesystemencodeerrors", (PyCFunction)sys_getfilesystemencodeerrors, + METH_NOARGS, getfilesystemencodeerrors_doc }, #ifdef Py_TRACE_REFS {"getobjects", _Py_GetObjects, METH_VARARGS}, #endif @@ -1240,6 +1277,8 @@ #ifdef MS_WINDOWS {"getwindowsversion", (PyCFunction)sys_getwindowsversion, METH_NOARGS, getwindowsversion_doc}, + {"_enablelegacywindowsfsencoding", (PyCFunction)sys_enablelegacywindowsfsencoding, + METH_NOARGS, enablelegacywindowsfsencoding_doc }, #endif /* MS_WINDOWS */ {"intern", sys_intern, METH_VARARGS, intern_doc}, {"is_finalizing", sys_is_finalizing, METH_NOARGS, is_finalizing_doc}, @@ -1456,14 +1495,21 @@ version_info -- version information as a named tuple\n\ " ) -#ifdef MS_WINDOWS +#ifdef MS_COREDLL /* concatenating string here */ PyDoc_STR( "dllhandle -- [Windows only] integer handle of the Python DLL\n\ winver -- [Windows only] version number of the Python DLL\n\ " ) -#endif /* MS_WINDOWS */ +#endif /* MS_COREDLL */ +#ifdef MS_WINDOWS +/* concatenating string here */ +PyDoc_STR( +"_enablelegacywindowsfsencoding -- [Windows only] \n\ +" +) +#endif PyDoc_STR( "__stdin__ -- the original stdin; don't touch!\n\ __stdout__ -- the original stdout; don't touch!\n\ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 13:42:01 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 17:42:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_mismatched_if_blocks_i?= =?utf-8?q?n_posixmodule=2Ec=2E?= Message-ID: <20160908174200.36446.47068.868C4648@psf.io> https://hg.python.org/cpython/rev/aca5afbdbf39 changeset: 103323:aca5afbdbf39 user: Steve Dower date: Thu Sep 08 10:41:50 2016 -0700 summary: Fix mismatched if blocks in posixmodule.c. files: Modules/posixmodule.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -2068,9 +2068,8 @@ Py_BEGIN_ALLOW_THREADS if (path->fd != -1) result = FSTAT(path->fd, &st); - else -#ifdef MS_WINDOWS - if (follow_symlinks) +#ifdef MS_WINDOWS + else if (follow_symlinks) result = win32_stat(path->wide, &st); else result = win32_lstat(path->wide, &st); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:00:26 2016 From: python-checkins at python.org (r.david.murray) Date: Thu, 08 Sep 2016 18:00:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2327364=3A_fix_=22incorre?= =?utf-8?q?ct=22_uses_of_escape_character_in_the_stdlib=2E?= Message-ID: <20160908180025.59815.79633.21108A2F@psf.io> https://hg.python.org/cpython/rev/b4cc62473c13 changeset: 103324:b4cc62473c13 user: R David Murray date: Thu Sep 08 13:59:53 2016 -0400 summary: #27364: fix "incorrect" uses of escape character in the stdlib. And most of the tools. Patch by Emanual Barry, reviewed by me, Serhiy Storchaka, and Martin Panter. files: Lib/_osx_support.py | 10 +- Lib/csv.py | 8 +- Lib/difflib.py | 2 +- Lib/distutils/cmd.py | 2 +- Lib/distutils/command/bdist_msi.py | 4 +- Lib/distutils/command/build_scripts.py | 2 +- Lib/distutils/cygwinccompiler.py | 2 +- Lib/distutils/msvc9compiler.py | 2 +- Lib/distutils/sysconfig.py | 2 +- Lib/distutils/versionpredicate.py | 2 +- Lib/doctest.py | 8 +- Lib/email/_header_value_parser.py | 16 +- Lib/email/feedparser.py | 8 +- Lib/fnmatch.py | 2 +- Lib/ftplib.py | 2 +- Lib/html/parser.py | 4 +- Lib/http/client.py | 2 +- Lib/http/cookiejar.py | 6 +- Lib/http/cookies.py | 2 +- Lib/idlelib/calltips.py | 2 +- Lib/idlelib/idle_test/test_replace.py | 6 +- Lib/idlelib/idle_test/test_searchengine.py | 12 +- Lib/idlelib/paragraph.py | 2 +- Lib/imaplib.py | 4 +- Lib/msilib/__init__.py | 2 +- Lib/platform.py | 34 +- Lib/string.py | 2 +- Lib/sysconfig.py | 2 +- Lib/test/_test_multiprocessing.py | 2 +- Lib/test/datetimetester.py | 4 +- Lib/test/re_tests.py | 4 +- Lib/test/sortperf.py | 2 +- Lib/test/support/__init__.py | 2 +- Lib/test/test_asyncio/test_streams.py | 2 +- Lib/test/test_builtin.py | 4 +- Lib/test/test_capi.py | 10 +- Lib/test/test_cgi.py | 2 +- Lib/test/test_codeccallbacks.py | 4 +- Lib/test/test_codecs.py | 14 +- Lib/test/test_coroutines.py | 8 +- Lib/test/test_doctest.py | 2 +- Lib/test/test_email/test__header_value_parser.py | 4 +- Lib/test/test_email/test_email.py | 2 +- Lib/test/test_faulthandler.py | 8 +- Lib/test/test_fnmatch.py | 16 +- Lib/test/test_future.py | 2 +- Lib/test/test_gdb.py | 12 +- Lib/test/test_getargs2.py | 12 +- Lib/test/test_htmlparser.py | 2 +- Lib/test/test_http_cookiejar.py | 6 +- Lib/test/test_mailcap.py | 2 +- Lib/test/test_os.py | 2 +- Lib/test/test_platform.py | 4 +- Lib/test/test_re.py | 192 +++++----- Lib/test/test_regrtest.py | 4 +- Lib/test/test_ssl.py | 2 +- Lib/test/test_strftime.py | 6 +- Lib/test/test_strlit.py | 4 +- Lib/test/test_strptime.py | 10 +- Lib/test/test_unicode.py | 4 +- Lib/test/test_urllib.py | 4 +- Lib/test/test_xmlrpc.py | 2 +- Lib/unittest/runner.py | 2 +- Lib/unittest/test/test_assertions.py | 18 +- Lib/unittest/test/test_loader.py | 8 +- Lib/xml/etree/ElementPath.py | 22 +- Lib/xml/etree/ElementTree.py | 2 +- Tools/clinic/clinic.py | 2 +- Tools/demo/ss1.py | 2 +- Tools/freeze/winmakemakefile.py | 6 +- Tools/pybench/CommandLine.py | 4 +- Tools/pynche/ColorDB.py | 6 +- Tools/scripts/fixdiv.py | 4 +- Tools/scripts/h2py.py | 8 +- Tools/scripts/highlight.py | 16 +- Tools/scripts/mailerdaemon.py | 2 +- Tools/scripts/parseentities.py | 4 +- Tools/scripts/pathfix.py | 2 +- Tools/scripts/ptags.py | 2 +- Tools/scripts/svneol.py | 2 +- Tools/scripts/texi2html.py | 10 +- Tools/unicode/gencodec.py | 10 +- setup.py | 4 +- 83 files changed, 324 insertions(+), 324 deletions(-) diff --git a/Lib/_osx_support.py b/Lib/_osx_support.py --- a/Lib/_osx_support.py +++ b/Lib/_osx_support.py @@ -210,7 +210,7 @@ # Do not alter a config var explicitly overridden by env var if cv in _config_vars and cv not in os.environ: flags = _config_vars[cv] - flags = re.sub('-arch\s+\w+\s', ' ', flags, re.ASCII) + flags = re.sub(r'-arch\s+\w+\s', ' ', flags, re.ASCII) flags = re.sub('-isysroot [^ \t]*', ' ', flags) _save_modified_value(_config_vars, cv, flags) @@ -232,7 +232,7 @@ if 'CC' in os.environ: return _config_vars - if re.search('-arch\s+ppc', _config_vars['CFLAGS']) is not None: + if re.search(r'-arch\s+ppc', _config_vars['CFLAGS']) is not None: # NOTE: Cannot use subprocess here because of bootstrap # issues when building Python itself status = os.system( @@ -251,7 +251,7 @@ for cv in _UNIVERSAL_CONFIG_VARS: if cv in _config_vars and cv not in os.environ: flags = _config_vars[cv] - flags = re.sub('-arch\s+ppc\w*\s', ' ', flags) + flags = re.sub(r'-arch\s+ppc\w*\s', ' ', flags) _save_modified_value(_config_vars, cv, flags) return _config_vars @@ -267,7 +267,7 @@ for cv in _UNIVERSAL_CONFIG_VARS: if cv in _config_vars and '-arch' in _config_vars[cv]: flags = _config_vars[cv] - flags = re.sub('-arch\s+\w+\s', ' ', flags) + flags = re.sub(r'-arch\s+\w+\s', ' ', flags) flags = flags + ' ' + arch _save_modified_value(_config_vars, cv, flags) @@ -465,7 +465,7 @@ machine = 'fat' - archs = re.findall('-arch\s+(\S+)', cflags) + archs = re.findall(r'-arch\s+(\S+)', cflags) archs = tuple(sorted(set(archs))) if len(archs) == 1: diff --git a/Lib/csv.py b/Lib/csv.py --- a/Lib/csv.py +++ b/Lib/csv.py @@ -215,10 +215,10 @@ """ matches = [] - for restr in ('(?P[^\w\n"\'])(?P ?)(?P["\']).*?(?P=quote)(?P=delim)', # ,".*?", - '(?:^|\n)(?P["\']).*?(?P=quote)(?P[^\w\n"\'])(?P ?)', # ".*?", - '(?P>[^\w\n"\'])(?P ?)(?P["\']).*?(?P=quote)(?:$|\n)', # ,".*?" - '(?:^|\n)(?P["\']).*?(?P=quote)(?:$|\n)'): # ".*?" (no delim, no space) + for restr in (r'(?P[^\w\n"\'])(?P ?)(?P["\']).*?(?P=quote)(?P=delim)', # ,".*?", + r'(?:^|\n)(?P["\']).*?(?P=quote)(?P[^\w\n"\'])(?P ?)', # ".*?", + r'(?P>[^\w\n"\'])(?P ?)(?P["\']).*?(?P=quote)(?:$|\n)', # ,".*?" + r'(?:^|\n)(?P["\']).*?(?P=quote)(?:$|\n)'): # ".*?" (no delim, no space) regexp = re.compile(restr, re.DOTALL | re.MULTILINE) matches = regexp.findall(data) if matches: diff --git a/Lib/difflib.py b/Lib/difflib.py --- a/Lib/difflib.py +++ b/Lib/difflib.py @@ -1415,7 +1415,7 @@ import re # regular expression for finding intraline change indices - change_re = re.compile('(\++|\-+|\^+)') + change_re = re.compile(r'(\++|\-+|\^+)') # create the difference iterator to generate the differences diff_lines_iterator = ndiff(fromlines,tolines,linejunk,charjunk) diff --git a/Lib/distutils/cmd.py b/Lib/distutils/cmd.py --- a/Lib/distutils/cmd.py +++ b/Lib/distutils/cmd.py @@ -221,7 +221,7 @@ self._ensure_stringlike(option, "string", default) def ensure_string_list(self, option): - """Ensure that 'option' is a list of strings. If 'option' is + r"""Ensure that 'option' is a list of strings. If 'option' is currently a string, we split it either on /,\s*/ or /\s+/, so "foo bar baz", "foo,bar,baz", and "foo, bar baz" all become ["foo", "bar", "baz"]. diff --git a/Lib/distutils/command/bdist_msi.py b/Lib/distutils/command/bdist_msi.py --- a/Lib/distutils/command/bdist_msi.py +++ b/Lib/distutils/command/bdist_msi.py @@ -623,7 +623,7 @@ cost = PyDialog(db, "DiskCostDlg", x, y, w, h, modal, title, "OK", "OK", "OK", bitmap=False) cost.text("Title", 15, 6, 200, 15, 0x30003, - "{\DlgFontBold8}Disk Space Requirements") + r"{\DlgFontBold8}Disk Space Requirements") cost.text("Description", 20, 20, 280, 20, 0x30003, "The disk space required for the installation of the selected features.") cost.text("Text", 20, 53, 330, 60, 3, @@ -670,7 +670,7 @@ progress = PyDialog(db, "ProgressDlg", x, y, w, h, modeless, title, "Cancel", "Cancel", "Cancel", bitmap=False) progress.text("Title", 20, 15, 200, 15, 0x30003, - "{\DlgFontBold8}[Progress1] [ProductName]") + r"{\DlgFontBold8}[Progress1] [ProductName]") progress.text("Text", 35, 65, 300, 30, 3, "Please wait while the Installer [Progress2] [ProductName]. " "This may take several minutes.") diff --git a/Lib/distutils/command/build_scripts.py b/Lib/distutils/command/build_scripts.py --- a/Lib/distutils/command/build_scripts.py +++ b/Lib/distutils/command/build_scripts.py @@ -51,7 +51,7 @@ def copy_scripts(self): - """Copy each script listed in 'self.scripts'; if it's marked as a + r"""Copy each script listed in 'self.scripts'; if it's marked as a Python script in the Unix way (first line matches 'first_line_re', ie. starts with "\#!" and contains "python"), then adjust the first line to refer to the current Python interpreter as we copy. diff --git a/Lib/distutils/cygwinccompiler.py b/Lib/distutils/cygwinccompiler.py --- a/Lib/distutils/cygwinccompiler.py +++ b/Lib/distutils/cygwinccompiler.py @@ -368,7 +368,7 @@ return (CONFIG_H_UNCERTAIN, "couldn't read '%s': %s" % (fn, exc.strerror)) -RE_VERSION = re.compile(b'(\d+\.\d+(\.\d+)*)') +RE_VERSION = re.compile(br'(\d+\.\d+(\.\d+)*)') def _find_exe_version(cmd): """Find the version of an executable by running `cmd` in the shell. diff --git a/Lib/distutils/msvc9compiler.py b/Lib/distutils/msvc9compiler.py --- a/Lib/distutils/msvc9compiler.py +++ b/Lib/distutils/msvc9compiler.py @@ -716,7 +716,7 @@ r"""VC\d{2}\.CRT("|').*?(/>|)""", re.DOTALL) manifest_buf = re.sub(pattern, "", manifest_buf) - pattern = "\s*" + pattern = r"\s*" manifest_buf = re.sub(pattern, "", manifest_buf) # Now see if any other assemblies are referenced - if not, we # don't want a manifest embedded. diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py --- a/Lib/distutils/sysconfig.py +++ b/Lib/distutils/sysconfig.py @@ -278,7 +278,7 @@ # Regexes needed for parsing Makefile (and similar syntaxes, # like old-style Setup files). -_variable_rx = re.compile("([a-zA-Z][a-zA-Z0-9_]+)\s*=\s*(.*)") +_variable_rx = re.compile(r"([a-zA-Z][a-zA-Z0-9_]+)\s*=\s*(.*)") _findvar1_rx = re.compile(r"\$\(([A-Za-z][A-Za-z0-9_]*)\)") _findvar2_rx = re.compile(r"\${([A-Za-z][A-Za-z0-9_]*)}") diff --git a/Lib/distutils/versionpredicate.py b/Lib/distutils/versionpredicate.py --- a/Lib/distutils/versionpredicate.py +++ b/Lib/distutils/versionpredicate.py @@ -154,7 +154,7 @@ global _provision_rx if _provision_rx is None: _provision_rx = re.compile( - "([a-zA-Z_]\w*(?:\.[a-zA-Z_]\w*)*)(?:\s*\(\s*([^)\s]+)\s*\))?$", + r"([a-zA-Z_]\w*(?:\.[a-zA-Z_]\w*)*)(?:\s*\(\s*([^)\s]+)\s*\))?$", re.ASCII) value = value.strip() m = _provision_rx.match(value) diff --git a/Lib/doctest.py b/Lib/doctest.py --- a/Lib/doctest.py +++ b/Lib/doctest.py @@ -765,7 +765,7 @@ # This regular expression finds the indentation of every non-blank # line in a string. - _INDENT_RE = re.compile('^([ ]*)(?=\S)', re.MULTILINE) + _INDENT_RE = re.compile(r'^([ ]*)(?=\S)', re.MULTILINE) def _min_indent(self, s): "Return the minimum indentation of any non-blank line in `s`" @@ -1106,7 +1106,7 @@ if lineno is not None: if source_lines is None: return lineno+1 - pat = re.compile('(^|.*:)\s*\w*("|\')') + pat = re.compile(r'(^|.*:)\s*\w*("|\')') for lineno in range(lineno, len(source_lines)): if pat.match(source_lines[lineno]): return lineno @@ -1608,11 +1608,11 @@ # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used. if not (optionflags & DONT_ACCEPT_BLANKLINE): # Replace in want with a blank line. - want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER), + want = re.sub(r'(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER), '', want) # If a line in got contains only spaces, then remove the # spaces. - got = re.sub('(?m)^\s*?$', '', got) + got = re.sub(r'(?m)^\s*?$', '', got) if got == want: return True diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -652,8 +652,8 @@ if value.token_type == 'comment': return str(value) return str(value).replace('\\', '\\\\').replace( - '(', '\(').replace( - ')', '\)') + '(', r'\(').replace( + ')', r'\)') @property def content(self): @@ -1356,15 +1356,15 @@ _wsp_splitter = re.compile(r'([{}]+)'.format(''.join(WSP))).split _non_atom_end_matcher = re.compile(r"[^{}]+".format( - ''.join(ATOM_ENDS).replace('\\','\\\\').replace(']','\]'))).match + ''.join(ATOM_ENDS).replace('\\','\\\\').replace(']',r'\]'))).match _non_printable_finder = re.compile(r"[\x00-\x20\x7F]").findall _non_token_end_matcher = re.compile(r"[^{}]+".format( - ''.join(TOKEN_ENDS).replace('\\','\\\\').replace(']','\]'))).match + ''.join(TOKEN_ENDS).replace('\\','\\\\').replace(']',r'\]'))).match _non_attribute_end_matcher = re.compile(r"[^{}]+".format( - ''.join(ATTRIBUTE_ENDS).replace('\\','\\\\').replace(']','\]'))).match + ''.join(ATTRIBUTE_ENDS).replace('\\','\\\\').replace(']',r'\]'))).match _non_extended_attribute_end_matcher = re.compile(r"[^{}]+".format( ''.join(EXTENDED_ATTRIBUTE_ENDS).replace( - '\\','\\\\').replace(']','\]'))).match + '\\','\\\\').replace(']',r'\]'))).match def _validate_xtext(xtext): """If input token contains ASCII non-printables, register a defect.""" @@ -1517,7 +1517,7 @@ return unstructured def get_qp_ctext(value): - """ctext = + r"""ctext = This is not the RFC ctext, since we are handling nested comments in comment and unquoting quoted-pairs here. We allow anything except the '()' @@ -1878,7 +1878,7 @@ return obs_local_part, value def get_dtext(value): - """ dtext = / obs-dtext + r""" dtext = / obs-dtext obs-dtext = obs-NO-WS-CTL / quoted-pair We allow anything except the excluded characters, but if we find any diff --git a/Lib/email/feedparser.py b/Lib/email/feedparser.py --- a/Lib/email/feedparser.py +++ b/Lib/email/feedparser.py @@ -29,10 +29,10 @@ from collections import deque from io import StringIO -NLCRE = re.compile('\r\n|\r|\n') -NLCRE_bol = re.compile('(\r\n|\r|\n)') -NLCRE_eol = re.compile('(\r\n|\r|\n)\Z') -NLCRE_crack = re.compile('(\r\n|\r|\n)') +NLCRE = re.compile(r'\r\n|\r|\n') +NLCRE_bol = re.compile(r'(\r\n|\r|\n)') +NLCRE_eol = re.compile(r'(\r\n|\r|\n)\Z') +NLCRE_crack = re.compile(r'(\r\n|\r|\n)') # RFC 2822 $3.6.8 Optional fields. ftext is %d33-57 / %d59-126, Any character # except controls, SP, and ":". headerRE = re.compile(r'^(From |[\041-\071\073-\176]*:|[\t ])') diff --git a/Lib/fnmatch.py b/Lib/fnmatch.py --- a/Lib/fnmatch.py +++ b/Lib/fnmatch.py @@ -106,4 +106,4 @@ res = '%s[%s]' % (res, stuff) else: res = res + re.escape(c) - return res + '\Z(?ms)' + return res + r'\Z(?ms)' diff --git a/Lib/ftplib.py b/Lib/ftplib.py --- a/Lib/ftplib.py +++ b/Lib/ftplib.py @@ -821,7 +821,7 @@ if _150_re is None: import re _150_re = re.compile( - "150 .* \((\d+) bytes\)", re.IGNORECASE | re.ASCII) + r"150 .* \((\d+) bytes\)", re.IGNORECASE | re.ASCII) m = _150_re.match(resp) if not m: return None diff --git a/Lib/html/parser.py b/Lib/html/parser.py --- a/Lib/html/parser.py +++ b/Lib/html/parser.py @@ -34,7 +34,7 @@ # explode, so don't do it. # see http://www.w3.org/TR/html5/tokenization.html#tag-open-state # and http://www.w3.org/TR/html5/tokenization.html#tag-name-state -tagfind_tolerant = re.compile('([a-zA-Z][^\t\n\r\f />\x00]*)(?:\s|/(?!>))*') +tagfind_tolerant = re.compile(r'([a-zA-Z][^\t\n\r\f />\x00]*)(?:\s|/(?!>))*') attrfind_tolerant = re.compile( r'((?<=[\'"\s/])[^\s/>][^\s/=>]*)(\s*=+\s*' r'(\'[^\']*\'|"[^"]*"|(?![\'"])[^>\s]*))?(?:\s|/(?!>))*') @@ -56,7 +56,7 @@ endendtag = re.compile('>') # the HTML 5 spec, section 8.1.2.2, doesn't allow spaces between # ') +endtagfind = re.compile(r'') diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -1,4 +1,4 @@ -"""HTTP/1.1 client library +r"""HTTP/1.1 client library diff --git a/Lib/http/cookiejar.py b/Lib/http/cookiejar.py --- a/Lib/http/cookiejar.py +++ b/Lib/http/cookiejar.py @@ -200,7 +200,7 @@ STRICT_DATE_RE = re.compile( r"^[SMTWF][a-z][a-z], (\d\d) ([JFMASOND][a-z][a-z]) " - "(\d\d\d\d) (\d\d):(\d\d):(\d\d) GMT$", re.ASCII) + r"(\d\d\d\d) (\d\d):(\d\d):(\d\d) GMT$", re.ASCII) WEEKDAY_RE = re.compile( r"^(?:Sun|Mon|Tue|Wed|Thu|Fri|Sat)[a-z]*,?\s*", re.I | re.ASCII) LOOSE_HTTP_DATE_RE = re.compile( @@ -277,7 +277,7 @@ return _str2time(day, mon, yr, hr, min, sec, tz) ISO_DATE_RE = re.compile( - """^ + r"""^ (\d{4}) # year [-\/]? (\d\d?) # numerical month @@ -411,7 +411,7 @@ pairs = [] else: # skip junk - non_junk, nr_junk_chars = re.subn("^[=\s;]*", "", text) + non_junk, nr_junk_chars = re.subn(r"^[=\s;]*", "", text) assert nr_junk_chars > 0, ( "split_header_words bug: '%s', '%s', %s" % (orig_text, text, pairs)) diff --git a/Lib/http/cookies.py b/Lib/http/cookies.py --- a/Lib/http/cookies.py +++ b/Lib/http/cookies.py @@ -456,7 +456,7 @@ # _LegalKeyChars = r"\w\d!#%&'~_`><@,:/\$\*\+\-\.\^\|\)\(\?\}\{\=" -_LegalValueChars = _LegalKeyChars + '\[\]' +_LegalValueChars = _LegalKeyChars + r'\[\]' _CookiePattern = re.compile(r""" (?x) # This is a verbose pattern \s* # Optional whitespace at start of cookie diff --git a/Lib/idlelib/calltips.py b/Lib/idlelib/calltips.py --- a/Lib/idlelib/calltips.py +++ b/Lib/idlelib/calltips.py @@ -120,7 +120,7 @@ _MAX_COLS = 85 _MAX_LINES = 5 # enough for bytes _INDENT = ' '*4 # for wrapped signatures -_first_param = re.compile('(?<=\()\w*\,?\s*') +_first_param = re.compile(r'(?<=\()\w*\,?\s*') _default_callable_argspec = "See source or doc" diff --git a/Lib/idlelib/idle_test/test_replace.py b/Lib/idlelib/idle_test/test_replace.py --- a/Lib/idlelib/idle_test/test_replace.py +++ b/Lib/idlelib/idle_test/test_replace.py @@ -91,7 +91,7 @@ text.mark_set('insert', 'end') text.insert('insert', '\nline42:') before_text = text.get('1.0', 'end') - pv.set('[a-z][\d]+') + pv.set(r'[a-z][\d]+') replace() after_text = text.get('1.0', 'end') equal(before_text, after_text) @@ -192,7 +192,7 @@ self.engine.revar.set(True) before_text = text.get('1.0', 'end') - pv.set('[a-z][\d]+') + pv.set(r'[a-z][\d]+') rv.set('hello') replace() after_text = text.get('1.0', 'end') @@ -207,7 +207,7 @@ self.assertIn('error', showerror.title) self.assertIn('Empty', showerror.message) - pv.set('[\d') + pv.set(r'[\d') replace() self.assertIn('error', showerror.title) self.assertIn('Pattern', showerror.message) diff --git a/Lib/idlelib/idle_test/test_searchengine.py b/Lib/idlelib/idle_test/test_searchengine.py --- a/Lib/idlelib/idle_test/test_searchengine.py +++ b/Lib/idlelib/idle_test/test_searchengine.py @@ -139,10 +139,10 @@ def test_setcookedpat(self): engine = self.engine - engine.setcookedpat('\s') - self.assertEqual(engine.getpat(), '\s') + engine.setcookedpat(r'\s') + self.assertEqual(engine.getpat(), r'\s') engine.revar.set(1) - engine.setcookedpat('\s') + engine.setcookedpat(r'\s') self.assertEqual(engine.getpat(), r'\\s') def test_getcookedpat(self): @@ -156,10 +156,10 @@ Equal(engine.getcookedpat(), r'\bhello\b') engine.wordvar.set(False) - engine.setpat('\s') + engine.setpat(r'\s') Equal(engine.getcookedpat(), r'\\s') engine.revar.set(True) - Equal(engine.getcookedpat(), '\s') + Equal(engine.getcookedpat(), r'\s') def test_getprog(self): engine = self.engine @@ -282,7 +282,7 @@ cls.pat = re.compile('target') cls.res = (2, (10, 16)) # line, slice indexes of 'target' cls.failpat = re.compile('xyz') # not in text - cls.emptypat = re.compile('\w*') # empty match possible + cls.emptypat = re.compile(r'\w*') # empty match possible def make_search(self, func): def search(pat, line, col, wrap, ok=0): diff --git a/Lib/idlelib/paragraph.py b/Lib/idlelib/paragraph.py --- a/Lib/idlelib/paragraph.py +++ b/Lib/idlelib/paragraph.py @@ -130,7 +130,7 @@ partial = indent1 while i < n and not is_all_white(lines[i]): # XXX Should take double space after period (etc.) into account - words = re.split("(\s+)", lines[i]) + words = re.split(r"(\s+)", lines[i]) for j in range(0, len(words), 2): word = words[j] if not word: diff --git a/Lib/imaplib.py b/Lib/imaplib.py --- a/Lib/imaplib.py +++ b/Lib/imaplib.py @@ -132,7 +132,7 @@ class IMAP4: - """IMAP4 client class. + r"""IMAP4 client class. Instantiate with: IMAP4([host[, port]]) @@ -1535,7 +1535,7 @@ ('select', ('/tmp/yyz 2',)), ('search', (None, 'SUBJECT', 'test')), ('fetch', ('1', '(FLAGS INTERNALDATE RFC822)')), - ('store', ('1', 'FLAGS', '(\Deleted)')), + ('store', ('1', 'FLAGS', r'(\Deleted)')), ('namespace', ()), ('expunge', ()), ('recent', ()), diff --git a/Lib/msilib/__init__.py b/Lib/msilib/__init__.py --- a/Lib/msilib/__init__.py +++ b/Lib/msilib/__init__.py @@ -289,7 +289,7 @@ def make_short(self, file): oldfile = file file = file.replace('+', '_') - file = ''.join(c for c in file if not c in ' "/\[]:;=,') + file = ''.join(c for c in file if not c in r' "/\[]:;=,') parts = file.split(".") if len(parts) > 1: prefix = "".join(parts[:-1]).upper() diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -251,13 +251,13 @@ _release_filename = re.compile(r'(\w+)[-_](release|version)', re.ASCII) _lsb_release_version = re.compile(r'(.+)' - ' release ' - '([\d.]+)' - '[^(]*(?:\((.+)\))?', re.ASCII) + r' release ' + r'([\d.]+)' + r'[^(]*(?:\((.+)\))?', re.ASCII) _release_version = re.compile(r'([^0-9]+)' - '(?: release )?' - '([\d.]+)' - '[^(]*(?:\((.+)\))?', re.ASCII) + r'(?: release )?' + r'([\d.]+)' + r'[^(]*(?:\((.+)\))?', re.ASCII) # See also http://www.novell.com/coolsolutions/feature/11251.html # and http://linuxmafia.com/faq/Admin/release-files.html @@ -407,8 +407,8 @@ return version _ver_output = re.compile(r'(?:([\w ]+) ([\w.]+) ' - '.*' - '\[.* ([\d.]+)\])') + r'.*' + r'\[.* ([\d.]+)\])') # Examples of VER command output: # @@ -1153,22 +1153,22 @@ _ironpython_sys_version_parser = re.compile( r'IronPython\s*' - '([\d\.]+)' - '(?: \(([\d\.]+)\))?' - ' on (.NET [\d\.]+)', re.ASCII) + r'([\d\.]+)' + r'(?: \(([\d\.]+)\))?' + r' on (.NET [\d\.]+)', re.ASCII) # IronPython covering 2.6 and 2.7 _ironpython26_sys_version_parser = re.compile( r'([\d.]+)\s*' - '\(IronPython\s*' - '[\d.]+\s*' - '\(([\d.]+)\) on ([\w.]+ [\d.]+(?: \(\d+-bit\))?)\)' + r'\(IronPython\s*' + r'[\d.]+\s*' + r'\(([\d.]+)\) on ([\w.]+ [\d.]+(?: \(\d+-bit\))?)\)' ) _pypy_sys_version_parser = re.compile( r'([\w.+]+)\s*' - '\(#?([^,]+),\s*([\w ]+),\s*([\w :]+)\)\s*' - '\[PyPy [^\]]+\]?') + r'\(#?([^,]+),\s*([\w ]+),\s*([\w :]+)\)\s*' + r'\[PyPy [^\]]+\]?') _sys_version_cache = {} @@ -1403,7 +1403,7 @@ # see issue #1322 for more information warnings.filterwarnings( 'ignore', - 'dist\(\) and linux_distribution\(\) ' + r'dist\(\) and linux_distribution\(\) ' 'functions are deprecated .*', PendingDeprecationWarning, ) diff --git a/Lib/string.py b/Lib/string.py --- a/Lib/string.py +++ b/Lib/string.py @@ -28,7 +28,7 @@ digits = '0123456789' hexdigits = digits + 'abcdef' + 'ABCDEF' octdigits = '01234567' -punctuation = """!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~""" +punctuation = r"""!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~""" printable = digits + ascii_letters + punctuation + whitespace # Functions which aren't available as string methods. diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -215,7 +215,7 @@ # Regexes needed for parsing Makefile (and similar syntaxes, # like old-style Setup files). import re - _variable_rx = re.compile("([a-zA-Z][a-zA-Z0-9_]+)\s*=\s*(.*)") + _variable_rx = re.compile(r"([a-zA-Z][a-zA-Z0-9_]+)\s*=\s*(.*)") _findvar1_rx = re.compile(r"\$\(([A-Za-z][A-Za-z0-9_]*)\)") _findvar2_rx = re.compile(r"\${([A-Za-z][A-Za-z0-9_]*)}") diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -3857,7 +3857,7 @@ p.stderr.close() expected = 'semaphore_tracker: There appear to be 2 leaked semaphores' self.assertRegex(err, expected) - self.assertRegex(err, 'semaphore_tracker: %r: \[Errno' % name1) + self.assertRegex(err, r'semaphore_tracker: %r: \[Errno' % name1) # # Mixins diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -4001,12 +4001,12 @@ datetime(xx, xx, xx, xx, xx, xx, xx)) with self.assertRaisesRegex(TypeError, '^an integer is required ' - '\(got type str\)$'): + r'\(got type str\)$'): datetime(10, 10, '10') f10 = Number(10.9) with self.assertRaisesRegex(TypeError, '^__int__ returned non-int ' - '\(type float\)$'): + r'\(type float\)$'): datetime(10, 10, f10) class Float(float): diff --git a/Lib/test/re_tests.py b/Lib/test/re_tests.py --- a/Lib/test/re_tests.py +++ b/Lib/test/re_tests.py @@ -158,7 +158,7 @@ ('(abc', '-', SYNTAX_ERROR), ('a]', 'a]', SUCCEED, 'found', 'a]'), ('a[]]b', 'a]b', SUCCEED, 'found', 'a]b'), - ('a[\]]b', 'a]b', SUCCEED, 'found', 'a]b'), + ('a[\\]]b', 'a]b', SUCCEED, 'found', 'a]b'), ('a[^bc]d', 'aed', SUCCEED, 'found', 'aed'), ('a[^bc]d', 'abd', FAIL), ('a[^-b]c', 'adc', SUCCEED, 'found', 'adc'), @@ -551,7 +551,7 @@ # lookbehind: split by : but not if it is escaped by -. ('(?= 39 except OSError: can = False diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -831,7 +831,7 @@ stream._waiter = asyncio.Future(loop=self.loop) self.assertRegex( repr(stream), - ">") + r">") stream._waiter.set_result(None) self.loop.run_until_complete(stream._waiter) stream._waiter = None diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -83,7 +83,7 @@ ('', ValueError), (' ', ValueError), (' \t\t ', ValueError), - (str(b'\u0663\u0661\u0664 ','raw-unicode-escape'), 314), + (str(br'\u0663\u0661\u0664 ','raw-unicode-escape'), 314), (chr(0x200), ValueError), ] @@ -105,7 +105,7 @@ ('', ValueError), (' ', ValueError), (' \t\t ', ValueError), - (str(b'\u0663\u0661\u0664 ','raw-unicode-escape'), 314), + (str(br'\u0663\u0661\u0664 ','raw-unicode-escape'), 314), (chr(0x200), ValueError), ] diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py --- a/Lib/test/test_capi.py +++ b/Lib/test/test_capi.py @@ -533,21 +533,21 @@ parse((1, 2, 3), {}, b'OOO', ['', '', 'a']) parse((1, 2), {'a': 3}, b'OOO', ['', '', 'a']) with self.assertRaisesRegex(TypeError, - 'Function takes at least 2 positional arguments \(1 given\)'): + r'Function takes at least 2 positional arguments \(1 given\)'): parse((1,), {'a': 3}, b'OOO', ['', '', 'a']) parse((1,), {}, b'O|OO', ['', '', 'a']) with self.assertRaisesRegex(TypeError, - 'Function takes at least 1 positional arguments \(0 given\)'): + r'Function takes at least 1 positional arguments \(0 given\)'): parse((), {}, b'O|OO', ['', '', 'a']) parse((1, 2), {'a': 3}, b'OO$O', ['', '', 'a']) with self.assertRaisesRegex(TypeError, - 'Function takes exactly 2 positional arguments \(1 given\)'): + r'Function takes exactly 2 positional arguments \(1 given\)'): parse((1,), {'a': 3}, b'OO$O', ['', '', 'a']) parse((1,), {}, b'O|O$O', ['', '', 'a']) with self.assertRaisesRegex(TypeError, - 'Function takes at least 1 positional arguments \(0 given\)'): + r'Function takes at least 1 positional arguments \(0 given\)'): parse((), {}, b'O|O$O', ['', '', 'a']) - with self.assertRaisesRegex(SystemError, 'Empty parameter name after \$'): + with self.assertRaisesRegex(SystemError, r'Empty parameter name after \$'): parse((1,), {}, b'O|$OO', ['', '', 'a']) with self.assertRaisesRegex(SystemError, 'Empty keyword'): parse((1,), {}, b'O|OO', ['', 'a', '']) diff --git a/Lib/test/test_cgi.py b/Lib/test/test_cgi.py --- a/Lib/test/test_cgi.py +++ b/Lib/test/test_cgi.py @@ -148,7 +148,7 @@ def test_escape(self): # cgi.escape() is deprecated. with warnings.catch_warnings(): - warnings.filterwarnings('ignore', 'cgi\.escape', + warnings.filterwarnings('ignore', r'cgi\.escape', DeprecationWarning) self.assertEqual("test & string", cgi.escape("test & string")) self.assertEqual("<test string>", cgi.escape("")) diff --git a/Lib/test/test_codeccallbacks.py b/Lib/test/test_codeccallbacks.py --- a/Lib/test/test_codeccallbacks.py +++ b/Lib/test/test_codeccallbacks.py @@ -280,12 +280,12 @@ ) self.assertEqual( - b"\\u3042\u3xxx".decode("unicode-escape", "test.handler1"), + b"\\u3042\\u3xxx".decode("unicode-escape", "test.handler1"), "\u3042[<92><117><51>]xxx" ) self.assertEqual( - b"\\u3042\u3xx".decode("unicode-escape", "test.handler1"), + b"\\u3042\\u3xx".decode("unicode-escape", "test.handler1"), "\u3042[<92><117><51>]xx" ) diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2703,8 +2703,8 @@ bad_input = "bad input type" for encoding in bytes_transform_encodings: with self.subTest(encoding=encoding): - fmt = ( "{!r} is not a text encoding; " - "use codecs.encode\(\) to handle arbitrary codecs") + fmt = (r"{!r} is not a text encoding; " + r"use codecs.encode\(\) to handle arbitrary codecs") msg = fmt.format(encoding) with self.assertRaisesRegex(LookupError, msg) as failure: bad_input.encode(encoding) @@ -2713,7 +2713,7 @@ def test_text_to_binary_blacklists_text_transforms(self): # Check str.encode gives a good error message for str -> str codecs msg = (r"^'rot_13' is not a text encoding; " - "use codecs.encode\(\) to handle arbitrary codecs") + r"use codecs.encode\(\) to handle arbitrary codecs") with self.assertRaisesRegex(LookupError, msg): "just an example message".encode("rot_13") @@ -2725,7 +2725,7 @@ with self.subTest(encoding=encoding): encoded_data = codecs.encode(data, encoding) fmt = (r"{!r} is not a text encoding; " - "use codecs.decode\(\) to handle arbitrary codecs") + r"use codecs.decode\(\) to handle arbitrary codecs") msg = fmt.format(encoding) with self.assertRaisesRegex(LookupError, msg): encoded_data.decode(encoding) @@ -2737,7 +2737,7 @@ for bad_input in (b"immutable", bytearray(b"mutable")): with self.subTest(bad_input=bad_input): msg = (r"^'rot_13' is not a text encoding; " - "use codecs.decode\(\) to handle arbitrary codecs") + r"use codecs.decode\(\) to handle arbitrary codecs") with self.assertRaisesRegex(LookupError, msg) as failure: bad_input.decode("rot_13") self.assertIsNone(failure.exception.__cause__) @@ -2956,12 +2956,12 @@ self.assertEqual(decoded, b"not str!") # Text model methods should complain fmt = (r"^{!r} encoder returned 'str' instead of 'bytes'; " - "use codecs.encode\(\) to encode to arbitrary types$") + r"use codecs.encode\(\) to encode to arbitrary types$") msg = fmt.format(self.codec_name) with self.assertRaisesRegex(TypeError, msg): "str_input".encode(self.codec_name) fmt = (r"^{!r} decoder returned 'bytes' instead of 'str'; " - "use codecs.decode\(\) to decode to arbitrary types$") + r"use codecs.decode\(\) to decode to arbitrary types$") msg = fmt.format(self.codec_name) with self.assertRaisesRegex(TypeError, msg): b"bytes input".decode(self.codec_name) diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py --- a/Lib/test/test_coroutines.py +++ b/Lib/test/test_coroutines.py @@ -891,7 +891,7 @@ return await Awaitable() with self.assertRaisesRegex( - TypeError, "__await__\(\) returned a coroutine"): + TypeError, r"__await__\(\) returned a coroutine"): run_async(foo()) @@ -1333,7 +1333,7 @@ with self.assertRaisesRegex( TypeError, - "async for' received an invalid object.*__aiter.*\: I"): + r"async for' received an invalid object.*__aiter.*\: I"): run_async(foo()) @@ -1667,8 +1667,8 @@ try: with silence_coro_gc(), self.assertRaisesRegex( RuntimeError, - "coroutine wrapper.*\.wrapper at 0x.*attempted to " - "recursively wrap .* wrap .*"): + r"coroutine wrapper.*\.wrapper at 0x.*attempted to " + r"recursively wrap .* wrap .*"): foo() finally: diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest.py @@ -291,7 +291,7 @@ ... ... Non-example text. ... - ... >>> print('another\example') + ... >>> print('another\\example') ... another ... example ... ''' diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -581,7 +581,7 @@ def test_get_comment_quoted_parens(self): self._test_get_x(parser.get_comment, - '(foo\) \(\)bar)', '(foo\) \(\)bar)', ' ', [], '', ['foo) ()bar']) + r'(foo\) \(\)bar)', r'(foo\) \(\)bar)', ' ', [], '', ['foo) ()bar']) def test_get_comment_non_printable(self): self._test_get_x(parser.get_comment, @@ -625,7 +625,7 @@ def test_get_comment_qs_in_nested_comment(self): comment = self._test_get_x(parser.get_comment, - '(foo (b\)))', '(foo (b\)))', ' ', [], '', ['foo (b\))']) + r'(foo (b\)))', r'(foo (b\)))', ' ', [], '', [r'foo (b\))']) self.assertEqual(comment[2].content, 'b)') # get_cfws diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3040,7 +3040,7 @@ def test_escape_backslashes(self): self.assertEqual( - utils.formataddr(('Arthur \Backslash\ Foobar', 'person at dom.ain')), + utils.formataddr((r'Arthur \Backslash\ Foobar', 'person at dom.ain')), r'"Arthur \\Backslash\\ Foobar" ') a = r'Arthur \Backslash\ Foobar' b = 'person at dom.ain' diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py --- a/Lib/test/test_faulthandler.py +++ b/Lib/test/test_faulthandler.py @@ -93,7 +93,7 @@ header = 'Thread 0x[0-9a-f]+' else: header = 'Stack' - regex = """ + regex = r""" ^{fatal_error} {header} \(most recent call first\): @@ -490,7 +490,7 @@ lineno = 8 else: lineno = 10 - regex = """ + regex = r""" ^Thread 0x[0-9a-f]+ \(most recent call first\): (?: File ".*threading.py", line [0-9]+ in [_a-z]+ ){{1,3}} File "", line 23 in run @@ -669,9 +669,9 @@ trace = '\n'.join(trace) if not unregister: if all_threads: - regex = 'Current thread 0x[0-9a-f]+ \(most recent call first\):\n' + regex = r'Current thread 0x[0-9a-f]+ \(most recent call first\):\n' else: - regex = 'Stack \(most recent call first\):\n' + regex = r'Stack \(most recent call first\):\n' regex = expected_traceback(14, 32, regex) self.assertRegex(trace, regex) else: diff --git a/Lib/test/test_fnmatch.py b/Lib/test/test_fnmatch.py --- a/Lib/test/test_fnmatch.py +++ b/Lib/test/test_fnmatch.py @@ -62,14 +62,14 @@ class TranslateTestCase(unittest.TestCase): def test_translate(self): - self.assertEqual(translate('*'), '.*\Z(?ms)') - self.assertEqual(translate('?'), '.\Z(?ms)') - self.assertEqual(translate('a?b*'), 'a.b.*\Z(?ms)') - self.assertEqual(translate('[abc]'), '[abc]\Z(?ms)') - self.assertEqual(translate('[]]'), '[]]\Z(?ms)') - self.assertEqual(translate('[!x]'), '[^x]\Z(?ms)') - self.assertEqual(translate('[^x]'), '[\\^x]\Z(?ms)') - self.assertEqual(translate('[x'), '\\[x\Z(?ms)') + self.assertEqual(translate('*'), r'.*\Z(?ms)') + self.assertEqual(translate('?'), r'.\Z(?ms)') + self.assertEqual(translate('a?b*'), r'a.b.*\Z(?ms)') + self.assertEqual(translate('[abc]'), r'[abc]\Z(?ms)') + self.assertEqual(translate('[]]'), r'[]]\Z(?ms)') + self.assertEqual(translate('[!x]'), r'[^x]\Z(?ms)') + self.assertEqual(translate('[^x]'), r'[\^x]\Z(?ms)') + self.assertEqual(translate('[x'), r'\[x\Z(?ms)') class FilterTestCase(unittest.TestCase): diff --git a/Lib/test/test_future.py b/Lib/test/test_future.py --- a/Lib/test/test_future.py +++ b/Lib/test/test_future.py @@ -4,7 +4,7 @@ from test import support import re -rx = re.compile('\((\S+).py, line (\d+)') +rx = re.compile(r'\((\S+).py, line (\d+)') def get_error_location(msg): mo = rx.search(str(msg)) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -244,7 +244,7 @@ # gdb can insert additional '\n' and space characters in various places # in its output, depending on the width of the terminal it's connected # to (using its "wrap_here" function) - m = re.match('.*#0\s+builtin_id\s+\(self\=.*,\s+v=\s*(.*?)\)\s+at\s+\S*Python/bltinmodule.c.*', + m = re.match(r'.*#0\s+builtin_id\s+\(self\=.*,\s+v=\s*(.*?)\)\s+at\s+\S*Python/bltinmodule.c.*', gdb_output, re.DOTALL) if not m: self.fail('Unexpected gdb output: %r\n%s' % (gdb_output, gdb_output)) @@ -552,7 +552,7 @@ foo = Foo() foo.an_attr = foo id(foo)''') - self.assertTrue(re.match('\) at remote 0x-?[0-9a-f]+>', + self.assertTrue(re.match(r'\) at remote 0x-?[0-9a-f]+>', gdb_repr), 'Unexpected gdb representation: %r\n%s' % \ (gdb_repr, gdb_output)) @@ -565,7 +565,7 @@ foo = Foo() foo.an_attr = foo id(foo)''') - self.assertTrue(re.match('\) at remote 0x-?[0-9a-f]+>', + self.assertTrue(re.match(r'\) at remote 0x-?[0-9a-f]+>', gdb_repr), 'Unexpected gdb representation: %r\n%s' % \ (gdb_repr, gdb_output)) @@ -579,7 +579,7 @@ a.an_attr = b b.an_attr = a id(a)''') - self.assertTrue(re.match('\) at remote 0x-?[0-9a-f]+>\) at remote 0x-?[0-9a-f]+>', + self.assertTrue(re.match(r'\) at remote 0x-?[0-9a-f]+>\) at remote 0x-?[0-9a-f]+>', gdb_repr), 'Unexpected gdb representation: %r\n%s' % \ (gdb_repr, gdb_output)) @@ -614,7 +614,7 @@ def test_builtin_method(self): gdb_repr, gdb_output = self.get_gdb_repr('import sys; id(sys.stdout.readlines)') - self.assertTrue(re.match('', + self.assertTrue(re.match(r'', gdb_repr), 'Unexpected gdb representation: %r\n%s' % \ (gdb_repr, gdb_output)) @@ -629,7 +629,7 @@ breakpoint='builtin_id', cmds_after_breakpoint=['print (PyFrameObject*)(((PyCodeObject*)v)->co_zombieframe)'] ) - self.assertTrue(re.match('.*\s+\$1 =\s+Frame 0x-?[0-9a-f]+, for file , line 3, in foo \(\)\s+.*', + self.assertTrue(re.match(r'.*\s+\$1 =\s+Frame 0x-?[0-9a-f]+, for file , line 3, in foo \(\)\s+.*', gdb_output, re.DOTALL), 'Unexpected gdb representation: %r\n%s' % (gdb_output, gdb_output)) diff --git a/Lib/test/test_getargs2.py b/Lib/test/test_getargs2.py --- a/Lib/test/test_getargs2.py +++ b/Lib/test/test_getargs2.py @@ -625,20 +625,20 @@ ) # required arg missing with self.assertRaisesRegex(TypeError, - "Required argument 'required' \(pos 1\) not found"): + r"Required argument 'required' \(pos 1\) not found"): getargs_keyword_only(optional=2) with self.assertRaisesRegex(TypeError, - "Required argument 'required' \(pos 1\) not found"): + r"Required argument 'required' \(pos 1\) not found"): getargs_keyword_only(keyword_only=3) def test_too_many_args(self): with self.assertRaisesRegex(TypeError, - "Function takes at most 2 positional arguments \(3 given\)"): + r"Function takes at most 2 positional arguments \(3 given\)"): getargs_keyword_only(1, 2, 3) with self.assertRaisesRegex(TypeError, - "function takes at most 3 arguments \(4 given\)"): + r"function takes at most 3 arguments \(4 given\)"): getargs_keyword_only(1, 2, 3, keyword_only=5) def test_invalid_keyword(self): @@ -673,11 +673,11 @@ self.assertEqual(self.getargs(1), (1, -1, -1)) # required positional arg missing with self.assertRaisesRegex(TypeError, - "Function takes at least 1 positional arguments \(0 given\)"): + r"Function takes at least 1 positional arguments \(0 given\)"): self.getargs() with self.assertRaisesRegex(TypeError, - "Function takes at least 1 positional arguments \(0 given\)"): + r"Function takes at least 1 positional arguments \(0 given\)"): self.getargs(keyword=3) def test_empty_keyword(self): diff --git a/Lib/test/test_htmlparser.py b/Lib/test/test_htmlparser.py --- a/Lib/test/test_htmlparser.py +++ b/Lib/test/test_htmlparser.py @@ -701,7 +701,7 @@ def test_attr_funky_names2(self): self._run_check( - "", + r"", [("starttag", "a", [("$", None)]), ("starttag", "b", [("$", "%")]), ("starttag", "c", [("\\", "/")])]) diff --git a/Lib/test/test_http_cookiejar.py b/Lib/test/test_http_cookiejar.py --- a/Lib/test/test_http_cookiejar.py +++ b/Lib/test/test_http_cookiejar.py @@ -1051,7 +1051,7 @@ url = "http://foo.bar.com/" interact_2965(c, url, "spam=eggs; Version=1; Port") h = interact_2965(c, url) - self.assertRegex(h, "\$Port([^=]|$)", + self.assertRegex(h, r"\$Port([^=]|$)", "port with no value not returned with no value") c = CookieJar(pol) @@ -1396,9 +1396,9 @@ self.assertRegex(cookie, r'^\$Version="?1"?;') self.assertRegex(cookie, r'Part_Number="?Rocket_Launcher_0001"?;' - '\s*\$Path="\/acme"') + r'\s*\$Path="\/acme"') self.assertRegex(cookie, r'Customer="?WILE_E_COYOTE"?;' - '\s*\$Path="\/acme"') + r'\s*\$Path="\/acme"') # # 7. User Agent -> Server diff --git a/Lib/test/test_mailcap.py b/Lib/test/test_mailcap.py --- a/Lib/test/test_mailcap.py +++ b/Lib/test/test_mailcap.py @@ -101,7 +101,7 @@ (["echo foo", "audio/*", "foo.txt"], "echo foo"), (["echo %s", "audio/*", "foo.txt"], "echo foo.txt"), (["echo %t", "audio/*", "foo.txt"], "echo audio/*"), - (["echo \%t", "audio/*", "foo.txt"], "echo %t"), + (["echo \\%t", "audio/*", "foo.txt"], "echo %t"), (["echo foo", "audio/*", "foo.txt", plist], "echo foo"), (["echo %{total}", "audio/*", "foo.txt", plist], "echo 3") ] diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -2086,7 +2086,7 @@ class NonLocalSymlinkTests(unittest.TestCase): def setUp(self): - """ + r""" Create this structure: base diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -255,7 +255,7 @@ with warnings.catch_warnings(): warnings.filterwarnings( 'ignore', - 'dist\(\) and linux_distribution\(\) ' + r'dist\(\) and linux_distribution\(\) ' 'functions are deprecated .*', PendingDeprecationWarning, ) @@ -331,7 +331,7 @@ with warnings.catch_warnings(): warnings.filterwarnings( 'ignore', - 'dist\(\) and linux_distribution\(\) ' + r'dist\(\) and linux_distribution\(\) ' 'functions are deprecated .*', PendingDeprecationWarning, ) diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -113,10 +113,10 @@ self.assertEqual(re.sub('(.)', re.escape(s), 'x'), s) self.assertEqual(re.sub('(.)', lambda m: s, 'x'), s) - self.assertEqual(re.sub('(?Px)', '\g\g', 'xx'), 'xxxx') - self.assertEqual(re.sub('(?Px)', '\g\g<1>', 'xx'), 'xxxx') - self.assertEqual(re.sub('(?Px)', '\g\g', 'xx'), 'xxxx') - self.assertEqual(re.sub('(?Px)', '\g<1>\g<1>', 'xx'), 'xxxx') + self.assertEqual(re.sub('(?Px)', r'\g\g', 'xx'), 'xxxx') + self.assertEqual(re.sub('(?Px)', r'\g\g<1>', 'xx'), 'xxxx') + self.assertEqual(re.sub('(?Px)', r'\g\g', 'xx'), 'xxxx') + self.assertEqual(re.sub('(?Px)', r'\g<1>\g<1>', 'xx'), 'xxxx') self.assertEqual(re.sub('a', r'\t\n\v\r\f\a\b', 'a'), '\t\n\v\r\f\a\b') self.assertEqual(re.sub('a', '\t\n\v\r\f\a\b', 'a'), '\t\n\v\r\f\a\b') @@ -127,11 +127,11 @@ with self.assertRaises(re.error): self.assertEqual(re.sub('a', '\\' + c, 'a'), '\\' + c) - self.assertEqual(re.sub('^\s*', 'X', 'test'), 'Xtest') + self.assertEqual(re.sub(r'^\s*', 'X', 'test'), 'Xtest') def test_bug_449964(self): # fails for group followed by other escape - self.assertEqual(re.sub(r'(?Px)', '\g<1>\g<1>\\b', 'xx'), + self.assertEqual(re.sub(r'(?Px)', r'\g<1>\g<1>\b', 'xx'), 'xx\bxx\b') def test_bug_449000(self): @@ -218,26 +218,26 @@ self.assertEqual(re.sub('x+', '-', 'abxd'), 'ab-d') def test_symbolic_groups(self): - re.compile('(?Px)(?P=a)(?(a)y)') - re.compile('(?Px)(?P=a1)(?(a1)y)') - re.compile('(?Px)\1(?(1)y)') - self.checkPatternError('(?P)(?P)', + re.compile(r'(?Px)(?P=a)(?(a)y)') + re.compile(r'(?Px)(?P=a1)(?(a1)y)') + re.compile(r'(?Px)\1(?(1)y)') + self.checkPatternError(r'(?P)(?P)', "redefinition of group name 'a' as group 2; " "was group 1") - self.checkPatternError('(?P(?P=a))', + self.checkPatternError(r'(?P(?P=a))', "cannot refer to an open group", 10) - self.checkPatternError('(?Pxy)', 'unknown extension ?Px') - self.checkPatternError('(?P)(?P=a', 'missing ), unterminated name', 11) - self.checkPatternError('(?P=', 'missing group name', 4) - self.checkPatternError('(?P=)', 'missing group name', 4) - self.checkPatternError('(?P=1)', "bad character in group name '1'", 4) - self.checkPatternError('(?P=a)', "unknown group name 'a'") - self.checkPatternError('(?P=a1)', "unknown group name 'a1'") - self.checkPatternError('(?P=a.)', "bad character in group name 'a.'", 4) - self.checkPatternError('(?P<)', 'missing >, unterminated name', 4) - self.checkPatternError('(?P, unterminated name', 4) - self.checkPatternError('(?P<', 'missing group name', 4) - self.checkPatternError('(?P<>)', 'missing group name', 4) + self.checkPatternError(r'(?Pxy)', 'unknown extension ?Px') + self.checkPatternError(r'(?P)(?P=a', 'missing ), unterminated name', 11) + self.checkPatternError(r'(?P=', 'missing group name', 4) + self.checkPatternError(r'(?P=)', 'missing group name', 4) + self.checkPatternError(r'(?P=1)', "bad character in group name '1'", 4) + self.checkPatternError(r'(?P=a)', "unknown group name 'a'") + self.checkPatternError(r'(?P=a1)', "unknown group name 'a1'") + self.checkPatternError(r'(?P=a.)', "bad character in group name 'a.'", 4) + self.checkPatternError(r'(?P<)', 'missing >, unterminated name', 4) + self.checkPatternError(r'(?P, unterminated name', 4) + self.checkPatternError(r'(?P<', 'missing group name', 4) + self.checkPatternError(r'(?P<>)', 'missing group name', 4) self.checkPatternError(r'(?P<1>)', "bad character in group name '1'", 4) self.checkPatternError(r'(?P)', "bad character in group name 'a.'", 4) self.checkPatternError(r'(?(', 'missing group name', 3) @@ -256,35 +256,35 @@ self.assertEqual(re.match(pat, 'xc8yz').span(), (0, 5)) def test_symbolic_refs(self): - self.checkTemplateError('(?Px)', '\gx)', r'\g, unterminated name', 3) - self.checkTemplateError('(?Px)', '\g<', 'xx', + self.checkTemplateError('(?Px)', r'\g<', 'xx', 'missing group name', 3) - self.checkTemplateError('(?Px)', '\g', 'xx', 'missing <', 2) - self.checkTemplateError('(?Px)', '\g', 'xx', + self.checkTemplateError('(?Px)', r'\g', 'xx', 'missing <', 2) + self.checkTemplateError('(?Px)', r'\g', 'xx', "bad character in group name 'a a'", 3) - self.checkTemplateError('(?Px)', '\g<>', 'xx', + self.checkTemplateError('(?Px)', r'\g<>', 'xx', 'missing group name', 3) - self.checkTemplateError('(?Px)', '\g<1a1>', 'xx', + self.checkTemplateError('(?Px)', r'\g<1a1>', 'xx', "bad character in group name '1a1'", 3) self.checkTemplateError('(?Px)', r'\g<2>', 'xx', 'invalid group reference') self.checkTemplateError('(?Px)', r'\2', 'xx', 'invalid group reference') with self.assertRaisesRegex(IndexError, "unknown group name 'ab'"): - re.sub('(?Px)', '\g', 'xx') + re.sub('(?Px)', r'\g', 'xx') self.assertEqual(re.sub('(?Px)|(?Py)', r'\g', 'xx'), '') self.assertEqual(re.sub('(?Px)|(?Py)', r'\2', 'xx'), '') - self.checkTemplateError('(?Px)', '\g<-1>', 'xx', + self.checkTemplateError('(?Px)', r'\g<-1>', 'xx', "bad character in group name '-1'", 3) # New valid/invalid identifiers in Python 3 self.assertEqual(re.sub('(?Px)', r'\g', 'xx'), 'xx') self.assertEqual(re.sub('(?Px)', r'\g', 'xx'), 'xx') - self.checkTemplateError('(?Px)', '\g', 'xx', + self.checkTemplateError('(?Px)', r'\g', 'xx', "bad character in group name '?'", 3) # Support > 100 groups. pat = '|'.join('x(?P%x)y' % (i, i) for i in range(1, 200 + 1)) - self.assertEqual(re.sub(pat, '\g<200>', 'xc8yzxc8y'), 'c8zc8') + self.assertEqual(re.sub(pat, r'\g<200>', 'xc8yzxc8y'), 'c8zc8') def test_re_subn(self): self.assertEqual(re.subn("(?i)b+", "x", "bbbb BBBB"), ('x x', 2)) @@ -472,19 +472,19 @@ re.compile(r".*?").fullmatch("abcd", pos=1, endpos=3).span(), (1, 3)) def test_re_groupref_exists(self): - self.assertEqual(re.match('^(\()?([^()]+)(?(1)\))$', '(a)').groups(), + self.assertEqual(re.match(r'^(\()?([^()]+)(?(1)\))$', '(a)').groups(), ('(', 'a')) - self.assertEqual(re.match('^(\()?([^()]+)(?(1)\))$', 'a').groups(), + self.assertEqual(re.match(r'^(\()?([^()]+)(?(1)\))$', 'a').groups(), (None, 'a')) - self.assertIsNone(re.match('^(\()?([^()]+)(?(1)\))$', 'a)')) - self.assertIsNone(re.match('^(\()?([^()]+)(?(1)\))$', '(a')) + self.assertIsNone(re.match(r'^(\()?([^()]+)(?(1)\))$', 'a)')) + self.assertIsNone(re.match(r'^(\()?([^()]+)(?(1)\))$', '(a')) self.assertEqual(re.match('^(?:(a)|c)((?(1)b|d))$', 'ab').groups(), ('a', 'b')) - self.assertEqual(re.match('^(?:(a)|c)((?(1)b|d))$', 'cd').groups(), + self.assertEqual(re.match(r'^(?:(a)|c)((?(1)b|d))$', 'cd').groups(), (None, 'd')) - self.assertEqual(re.match('^(?:(a)|c)((?(1)|d))$', 'cd').groups(), + self.assertEqual(re.match(r'^(?:(a)|c)((?(1)|d))$', 'cd').groups(), (None, 'd')) - self.assertEqual(re.match('^(?:(a)|c)((?(1)|d))$', 'a').groups(), + self.assertEqual(re.match(r'^(?:(a)|c)((?(1)|d))$', 'a').groups(), ('a', '')) # Tests for bug #1177831: exercise groups other than the first group @@ -509,7 +509,7 @@ 'two branches', 10) def test_re_groupref_overflow(self): - self.checkTemplateError('()', '\g<%s>' % sre_constants.MAXGROUPS, 'xx', + self.checkTemplateError('()', r'\g<%s>' % sre_constants.MAXGROUPS, 'xx', 'invalid group reference', 3) self.checkPatternError(r'(?P)(?(%d))' % sre_constants.MAXGROUPS, 'invalid group reference', 10) @@ -544,37 +544,37 @@ " ") def test_repeat_minmax(self): - self.assertIsNone(re.match("^(\w){1}$", "abc")) - self.assertIsNone(re.match("^(\w){1}?$", "abc")) - self.assertIsNone(re.match("^(\w){1,2}$", "abc")) - self.assertIsNone(re.match("^(\w){1,2}?$", "abc")) + self.assertIsNone(re.match(r"^(\w){1}$", "abc")) + self.assertIsNone(re.match(r"^(\w){1}?$", "abc")) + self.assertIsNone(re.match(r"^(\w){1,2}$", "abc")) + self.assertIsNone(re.match(r"^(\w){1,2}?$", "abc")) - self.assertEqual(re.match("^(\w){3}$", "abc").group(1), "c") - self.assertEqual(re.match("^(\w){1,3}$", "abc").group(1), "c") - self.assertEqual(re.match("^(\w){1,4}$", "abc").group(1), "c") - self.assertEqual(re.match("^(\w){3,4}?$", "abc").group(1), "c") - self.assertEqual(re.match("^(\w){3}?$", "abc").group(1), "c") - self.assertEqual(re.match("^(\w){1,3}?$", "abc").group(1), "c") - self.assertEqual(re.match("^(\w){1,4}?$", "abc").group(1), "c") - self.assertEqual(re.match("^(\w){3,4}?$", "abc").group(1), "c") + self.assertEqual(re.match(r"^(\w){3}$", "abc").group(1), "c") + self.assertEqual(re.match(r"^(\w){1,3}$", "abc").group(1), "c") + self.assertEqual(re.match(r"^(\w){1,4}$", "abc").group(1), "c") + self.assertEqual(re.match(r"^(\w){3,4}?$", "abc").group(1), "c") + self.assertEqual(re.match(r"^(\w){3}?$", "abc").group(1), "c") + self.assertEqual(re.match(r"^(\w){1,3}?$", "abc").group(1), "c") + self.assertEqual(re.match(r"^(\w){1,4}?$", "abc").group(1), "c") + self.assertEqual(re.match(r"^(\w){3,4}?$", "abc").group(1), "c") - self.assertIsNone(re.match("^x{1}$", "xxx")) - self.assertIsNone(re.match("^x{1}?$", "xxx")) - self.assertIsNone(re.match("^x{1,2}$", "xxx")) - self.assertIsNone(re.match("^x{1,2}?$", "xxx")) + self.assertIsNone(re.match(r"^x{1}$", "xxx")) + self.assertIsNone(re.match(r"^x{1}?$", "xxx")) + self.assertIsNone(re.match(r"^x{1,2}$", "xxx")) + self.assertIsNone(re.match(r"^x{1,2}?$", "xxx")) - self.assertTrue(re.match("^x{3}$", "xxx")) - self.assertTrue(re.match("^x{1,3}$", "xxx")) - self.assertTrue(re.match("^x{3,3}$", "xxx")) - self.assertTrue(re.match("^x{1,4}$", "xxx")) - self.assertTrue(re.match("^x{3,4}?$", "xxx")) - self.assertTrue(re.match("^x{3}?$", "xxx")) - self.assertTrue(re.match("^x{1,3}?$", "xxx")) - self.assertTrue(re.match("^x{1,4}?$", "xxx")) - self.assertTrue(re.match("^x{3,4}?$", "xxx")) + self.assertTrue(re.match(r"^x{3}$", "xxx")) + self.assertTrue(re.match(r"^x{1,3}$", "xxx")) + self.assertTrue(re.match(r"^x{3,3}$", "xxx")) + self.assertTrue(re.match(r"^x{1,4}$", "xxx")) + self.assertTrue(re.match(r"^x{3,4}?$", "xxx")) + self.assertTrue(re.match(r"^x{3}?$", "xxx")) + self.assertTrue(re.match(r"^x{1,3}?$", "xxx")) + self.assertTrue(re.match(r"^x{1,4}?$", "xxx")) + self.assertTrue(re.match(r"^x{3,4}?$", "xxx")) - self.assertIsNone(re.match("^x{}$", "xxx")) - self.assertTrue(re.match("^x{}$", "x{}")) + self.assertIsNone(re.match(r"^x{}$", "xxx")) + self.assertTrue(re.match(r"^x{}$", "x{}")) self.checkPatternError(r'x{2,1}', 'min repeat greater than max repeat', 2) @@ -697,10 +697,10 @@ "a\n\nb") def test_lookahead(self): - self.assertEqual(re.match("(a(?=\s[^a]))", "a b").group(1), "a") - self.assertEqual(re.match("(a(?=\s[^a]*))", "a b").group(1), "a") - self.assertEqual(re.match("(a(?=\s[abc]))", "a b").group(1), "a") - self.assertEqual(re.match("(a(?=\s[abc]*))", "a bc").group(1), "a") + self.assertEqual(re.match(r"(a(?=\s[^a]))", "a b").group(1), "a") + self.assertEqual(re.match(r"(a(?=\s[^a]*))", "a b").group(1), "a") + self.assertEqual(re.match(r"(a(?=\s[abc]))", "a b").group(1), "a") + self.assertEqual(re.match(r"(a(?=\s[abc]*))", "a bc").group(1), "a") self.assertEqual(re.match(r"(a)(?=\s\1)", "a a").group(1), "a") self.assertEqual(re.match(r"(a)(?=\s\1*)", "a aa").group(1), "a") self.assertEqual(re.match(r"(a)(?=\s(abc|a))", "a a").group(1), "a") @@ -848,12 +848,12 @@ self.assertEqual(re.match(b"abc", b"ABC", re.I|re.L).group(0), b"ABC") def test_not_literal(self): - self.assertEqual(re.search("\s([^a])", " b").group(1), "b") - self.assertEqual(re.search("\s([^a]*)", " bb").group(1), "bb") + self.assertEqual(re.search(r"\s([^a])", " b").group(1), "b") + self.assertEqual(re.search(r"\s([^a]*)", " bb").group(1), "bb") def test_search_coverage(self): - self.assertEqual(re.search("\s(b)", " b").group(1), "b") - self.assertEqual(re.search("a\s", "a ").group(0), "a ") + self.assertEqual(re.search(r"\s(b)", " b").group(1), "b") + self.assertEqual(re.search(r"a\s", "a ").group(0), "a ") def assertMatch(self, pattern, text, match=None, span=None, matcher=re.match): @@ -1055,8 +1055,8 @@ self.assertIsNone(re.match(r'(a)?a','a').lastindex) self.assertEqual(re.match(r'(a)(b)?b','ab').lastindex, 1) self.assertEqual(re.match(r'(?Pa)(?Pb)?b','ab').lastgroup, 'a') - self.assertEqual(re.match("(?Pa(b))", "ab").lastgroup, 'a') - self.assertEqual(re.match("((a))", "a").lastindex, 1) + self.assertEqual(re.match(r"(?Pa(b))", "ab").lastgroup, 'a') + self.assertEqual(re.match(r"((a))", "a").lastindex, 1) def test_bug_418626(self): # bugs 418626 at al. -- Testing Greg Chapman's addition of op code @@ -1228,7 +1228,7 @@ '\uff10', # '\N{FULLWIDTH DIGIT ZERO}', category 'Nd' ] for x in decimal_digits: - self.assertEqual(re.match('^\d$', x).group(0), x) + self.assertEqual(re.match(r'^\d$', x).group(0), x) not_decimal_digits = [ '\u2165', # '\N{ROMAN NUMERAL SIX}', category 'Nl' @@ -1237,7 +1237,7 @@ '\u32b4', # '\N{CIRCLED NUMBER THIRTY NINE}', category 'No' ] for x in not_decimal_digits: - self.assertIsNone(re.match('^\d$', x)) + self.assertIsNone(re.match(r'^\d$', x)) def test_empty_array(self): # SF buf 1647541 @@ -1306,29 +1306,29 @@ for flags in (0, re.UNICODE): pat = re.compile('\xc0', flags | re.IGNORECASE) self.assertTrue(pat.match('\xe0')) - pat = re.compile('\w', flags) + pat = re.compile(r'\w', flags) self.assertTrue(pat.match('\xe0')) pat = re.compile('\xc0', re.ASCII | re.IGNORECASE) self.assertIsNone(pat.match('\xe0')) pat = re.compile('(?a)\xc0', re.IGNORECASE) self.assertIsNone(pat.match('\xe0')) - pat = re.compile('\w', re.ASCII) + pat = re.compile(r'\w', re.ASCII) self.assertIsNone(pat.match('\xe0')) - pat = re.compile('(?a)\w') + pat = re.compile(r'(?a)\w') self.assertIsNone(pat.match('\xe0')) # Bytes patterns for flags in (0, re.ASCII): pat = re.compile(b'\xc0', flags | re.IGNORECASE) self.assertIsNone(pat.match(b'\xe0')) - pat = re.compile(b'\w', flags) + pat = re.compile(br'\w', flags) self.assertIsNone(pat.match(b'\xe0')) # Incompatibilities - self.assertRaises(ValueError, re.compile, b'\w', re.UNICODE) - self.assertRaises(ValueError, re.compile, b'(?u)\w') - self.assertRaises(ValueError, re.compile, '\w', re.UNICODE | re.ASCII) - self.assertRaises(ValueError, re.compile, '(?u)\w', re.ASCII) - self.assertRaises(ValueError, re.compile, '(?a)\w', re.UNICODE) - self.assertRaises(ValueError, re.compile, '(?au)\w') + self.assertRaises(ValueError, re.compile, br'\w', re.UNICODE) + self.assertRaises(ValueError, re.compile, br'(?u)\w') + self.assertRaises(ValueError, re.compile, r'\w', re.UNICODE | re.ASCII) + self.assertRaises(ValueError, re.compile, r'(?u)\w', re.ASCII) + self.assertRaises(ValueError, re.compile, r'(?a)\w', re.UNICODE) + self.assertRaises(ValueError, re.compile, r'(?au)\w') def test_locale_flag(self): import locale @@ -1359,13 +1359,13 @@ pat = re.compile(bpat, re.IGNORECASE) if bletter: self.assertIsNone(pat.match(bletter)) - pat = re.compile(b'\w', re.LOCALE) + pat = re.compile(br'\w', re.LOCALE) if bletter: self.assertTrue(pat.match(bletter)) - pat = re.compile(b'(?L)\w') + pat = re.compile(br'(?L)\w') if bletter: self.assertTrue(pat.match(bletter)) - pat = re.compile(b'\w') + pat = re.compile(br'\w') if bletter: self.assertIsNone(pat.match(bletter)) # Incompatibilities @@ -1379,7 +1379,7 @@ def test_bug_6509(self): # Replacement strings of both types must parse properly. # all strings - pat = re.compile('a(\w)') + pat = re.compile(r'a(\w)') self.assertEqual(pat.sub('b\\1', 'ac'), 'bc') pat = re.compile('a(.)') self.assertEqual(pat.sub('b\\1', 'a\u1234'), 'b\u1234') @@ -1387,7 +1387,7 @@ self.assertEqual(pat.sub(lambda m: 'str', 'a5'), 'str') # all bytes - pat = re.compile(b'a(\w)') + pat = re.compile(br'a(\w)') self.assertEqual(pat.sub(b'b\\1', b'ac'), b'bc') pat = re.compile(b'a(.)') self.assertEqual(pat.sub(b'b\\1', b'a\xCD'), b'b\xCD') @@ -1509,7 +1509,7 @@ for string in (b'[abracadabra]', B(b'[abracadabra]'), bytearray(b'[abracadabra]'), memoryview(b'[abracadabra]')): - m = re.search(rb'(.+)(.*?)\1', string) + m = re.search(br'(.+)(.*?)\1', string) self.assertEqual(repr(m), "<%s.%s object; " "span=(1, 12), match=b'abracadabra'>" % (type(m).__module__, type(m).__qualname__)) diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -704,8 +704,8 @@ test = self.create_test('coverage') output = self.run_tests("--coverage", test) self.check_executed_tests(output, [test]) - regex = ('lines +cov% +module +\(path\)\n' - '(?: *[0-9]+ *[0-9]{1,2}% *[^ ]+ +\([^)]+\)+)+') + regex = (r'lines +cov% +module +\(path\)\n' + r'(?: *[0-9]+ *[0-9]{1,2}% *[^ ]+ +\([^)]+\)+)+') self.check_line(output, regex) def test_wait(self): diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -3405,7 +3405,7 @@ with warnings.catch_warnings(): warnings.filterwarnings( 'ignore', - 'dist\(\) and linux_distribution\(\) ' + r'dist\(\) and linux_distribution\(\) ' 'functions are deprecated .*', PendingDeprecationWarning, ) diff --git a/Lib/test/test_strftime.py b/Lib/test/test_strftime.py --- a/Lib/test/test_strftime.py +++ b/Lib/test/test_strftime.py @@ -23,9 +23,9 @@ """ new_text = re.escape(text) new_text = new_text.replace(re.escape(ampm), ampm) - new_text = new_text.replace('\%', '%') - new_text = new_text.replace('\:', ':') - new_text = new_text.replace('\?', '?') + new_text = new_text.replace(r'\%', '%') + new_text = new_text.replace(r'\:', ':') + new_text = new_text.replace(r'\?', '?') return new_text diff --git a/Lib/test/test_strlit.py b/Lib/test/test_strlit.py --- a/Lib/test/test_strlit.py +++ b/Lib/test/test_strlit.py @@ -121,9 +121,9 @@ self.assertEqual(eval(""" b'\x01' """), byte(1)) self.assertEqual(eval(r""" b'\x81' """), byte(0x81)) self.assertRaises(SyntaxError, eval, """ b'\x81' """) - self.assertEqual(eval(r""" b'\u1881' """), b'\\' + b'u1881') + self.assertEqual(eval(r""" br'\u1881' """), b'\\' + b'u1881') self.assertRaises(SyntaxError, eval, """ b'\u1881' """) - self.assertEqual(eval(r""" b'\U0001d120' """), b'\\' + b'U0001d120') + self.assertEqual(eval(r""" br'\U0001d120' """), b'\\' + b'U0001d120') self.assertRaises(SyntaxError, eval, """ b'\U0001d120' """) def test_eval_bytes_incomplete(self): diff --git a/Lib/test/test_strptime.py b/Lib/test/test_strptime.py --- a/Lib/test/test_strptime.py +++ b/Lib/test/test_strptime.py @@ -129,7 +129,7 @@ def test_pattern_escaping(self): # Make sure any characters in the format string that might be taken as # regex syntax is escaped. - pattern_string = self.time_re.pattern("\d+") + pattern_string = self.time_re.pattern(r"\d+") self.assertIn(r"\\d\+", pattern_string, "%s does not have re characters escaped properly" % pattern_string) @@ -170,9 +170,9 @@ def test_matching_with_escapes(self): # Make sure a format that requires escaping of characters works - compiled_re = self.time_re.compile("\w+ %m") - found = compiled_re.match("\w+ 10") - self.assertTrue(found, "Escaping failed of format '\w+ 10'") + compiled_re = self.time_re.compile(r"\w+ %m") + found = compiled_re.match(r"\w+ 10") + self.assertTrue(found, r"Escaping failed of format '\w+ 10'") def test_locale_data_w_regex_metacharacters(self): # Check that if locale data contains regex metacharacters they are @@ -403,7 +403,7 @@ # unbalanced parentheses when the regex is compiled if they are not # escaped. # Test instigated by bug #796149 . - need_escaping = ".^$*+?{}\[]|)(" + need_escaping = r".^$*+?{}\[]|)(" self.assertTrue(_strptime._strptime_time(need_escaping, need_escaping)) def test_feb29_on_leap_year_without_year(self): diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -1564,7 +1564,7 @@ ('+', b'+-'), ('+-', b'+--'), ('+?', b'+-?'), - ('\?', b'+AFw?'), + (r'\?', b'+AFw?'), ('+?', b'+-?'), (r'\\?', b'+AFwAXA?'), (r'\\\?', b'+AFwAXABc?'), @@ -2326,7 +2326,7 @@ # non-ascii format, ascii argument: ensure that PyUnicode_FromFormatV() # raises an error self.assertRaisesRegex(ValueError, - '^PyUnicode_FromFormatV\(\) expects an ASCII-encoded format ' + r'^PyUnicode_FromFormatV\(\) expects an ASCII-encoded format ' 'string, got a non-ASCII byte: 0xe9$', PyUnicode_FromFormat, b'unicode\xe9=%s', 'ascii') diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py --- a/Lib/test/test_urllib.py +++ b/Lib/test/test_urllib.py @@ -729,7 +729,7 @@ class QuotingTests(unittest.TestCase): - """Tests for urllib.quote() and urllib.quote_plus() + r"""Tests for urllib.quote() and urllib.quote_plus() According to RFC 2396 (Uniform Resource Identifiers), to escape a character you write it as '%' + <2 character US-ASCII hex value>. @@ -804,7 +804,7 @@ # Make sure all characters that should be quoted are by default sans # space (separate test for that). should_quote = [chr(num) for num in range(32)] # For 0x00 - 0x1F - should_quote.append('<>#%"{}|\^[]`') + should_quote.append(r'<>#%"{}|\^[]`') should_quote.append(chr(127)) # For 0x7F should_quote = ''.join(should_quote) for char in should_quote: diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -1218,7 +1218,7 @@ content = handle[handle.find("\d+)\s+(?P\d+)\s+(?P\d+)\s+(?P.*)') + r'\s*(?P\d+)\s+(?P\d+)\s+(?P\d+)\s+(?P.*)') class HTML40DB(ColorDB): - _re = re.compile('(?P\S+)\s+(?P#[0-9a-fA-F]{6})') + _re = re.compile(r'(?P\S+)\s+(?P#[0-9a-fA-F]{6})') def _extractrgb(self, mo): return rrggbb_to_triplet(mo.group('hexrgb')) class LightlinkDB(HTML40DB): - _re = re.compile('(?P(.+))\s+(?P#[0-9a-fA-F]{6})') + _re = re.compile(r'(?P(.+))\s+(?P#[0-9a-fA-F]{6})') def _extractname(self, mo): return mo.group('name').strip() diff --git a/Tools/scripts/fixdiv.py b/Tools/scripts/fixdiv.py --- a/Tools/scripts/fixdiv.py +++ b/Tools/scripts/fixdiv.py @@ -174,8 +174,8 @@ sys.stderr.write("Usage: %s [-m] warnings\n" % sys.argv[0]) sys.stderr.write("Try `%s -h' for more information.\n" % sys.argv[0]) -PATTERN = ("^(.+?):(\d+): DeprecationWarning: " - "classic (int|long|float|complex) division$") +PATTERN = (r"^(.+?):(\d+): DeprecationWarning: " + r"classic (int|long|float|complex) division$") def readwarnings(warningsfile): prog = re.compile(PATTERN) diff --git a/Tools/scripts/h2py.py b/Tools/scripts/h2py.py --- a/Tools/scripts/h2py.py +++ b/Tools/scripts/h2py.py @@ -23,13 +23,13 @@ import sys, re, getopt, os -p_define = re.compile('^[\t ]*#[\t ]*define[\t ]+([a-zA-Z0-9_]+)[\t ]+') +p_define = re.compile(r'^[\t ]*#[\t ]*define[\t ]+([a-zA-Z0-9_]+)[\t ]+') p_macro = re.compile( - '^[\t ]*#[\t ]*define[\t ]+' - '([a-zA-Z0-9_]+)\(([_a-zA-Z][_a-zA-Z0-9]*)\)[\t ]+') + r'^[\t ]*#[\t ]*define[\t ]+' + r'([a-zA-Z0-9_]+)\(([_a-zA-Z][_a-zA-Z0-9]*)\)[\t ]+') -p_include = re.compile('^[\t ]*#[\t ]*include[\t ]+<([^>\n]+)>') +p_include = re.compile(r'^[\t ]*#[\t ]*include[\t ]+<([^>\n]+)>') p_comment = re.compile(r'/\*([^*]+|\*+[^/])*(\*+/)?') p_cpp_comment = re.compile('//.*') diff --git a/Tools/scripts/highlight.py b/Tools/scripts/highlight.py --- a/Tools/scripts/highlight.py +++ b/Tools/scripts/highlight.py @@ -147,14 +147,14 @@ #### LaTeX Output ########################################## default_latex_commands = { - 'comment': '{\color{red}#1}', - 'string': '{\color{ForestGreen}#1}', - 'docstring': '{\emph{\color{ForestGreen}#1}}', - 'keyword': '{\color{orange}#1}', - 'builtin': '{\color{purple}#1}', - 'definition': '{\color{orange}#1}', - 'defname': '{\color{blue}#1}', - 'operator': '{\color{brown}#1}', + 'comment': r'{\color{red}#1}', + 'string': r'{\color{ForestGreen}#1}', + 'docstring': r'{\emph{\color{ForestGreen}#1}}', + 'keyword': r'{\color{orange}#1}', + 'builtin': r'{\color{purple}#1}', + 'definition': r'{\color{orange}#1}', + 'defname': r'{\color{blue}#1}', + 'operator': r'{\color{brown}#1}', } default_latex_document = r''' diff --git a/Tools/scripts/mailerdaemon.py b/Tools/scripts/mailerdaemon.py --- a/Tools/scripts/mailerdaemon.py +++ b/Tools/scripts/mailerdaemon.py @@ -88,7 +88,7 @@ # no more expressions are searched for. So, order is important. emparse_list_reason = [ r'^5\d{2} <>\.\.\. (?P.*)', - '<>\.\.\. (?P.*)', + r'<>\.\.\. (?P.*)', re.compile(r'^<<< 5\d{2} (?P.*)', re.MULTILINE), re.compile('===== stderr was =====\nrmail: (?P.*)'), re.compile('^Diagnostic-Code: (?P.*)', re.MULTILINE), diff --git a/Tools/scripts/parseentities.py b/Tools/scripts/parseentities.py --- a/Tools/scripts/parseentities.py +++ b/Tools/scripts/parseentities.py @@ -14,7 +14,7 @@ """ import re,sys -entityRE = re.compile('') +entityRE = re.compile(r'') def parse(text,pos=0,endpos=None): @@ -39,7 +39,7 @@ if charcode[:2] == '&#': code = int(charcode[2:-1]) if code < 256: - charcode = "'\%o'" % code + charcode = r"'\%o'" % code else: charcode = repr(charcode) else: diff --git a/Tools/scripts/pathfix.py b/Tools/scripts/pathfix.py --- a/Tools/scripts/pathfix.py +++ b/Tools/scripts/pathfix.py @@ -64,7 +64,7 @@ if fix(arg): bad = 1 sys.exit(bad) -ispythonprog = re.compile('^[a-zA-Z0-9_]+\.py$') +ispythonprog = re.compile(r'^[a-zA-Z0-9_]+\.py$') def ispython(name): return bool(ispythonprog.match(name)) diff --git a/Tools/scripts/ptags.py b/Tools/scripts/ptags.py --- a/Tools/scripts/ptags.py +++ b/Tools/scripts/ptags.py @@ -24,7 +24,7 @@ for s in tags: fp.write(s) -expr = '^[ \t]*(def|class)[ \t]+([a-zA-Z0-9_]+)[ \t]*[:\(]' +expr = r'^[ \t]*(def|class)[ \t]+([a-zA-Z0-9_]+)[ \t]*[:\(]' matcher = re.compile(expr) def treat_file(filename): diff --git a/Tools/scripts/svneol.py b/Tools/scripts/svneol.py --- a/Tools/scripts/svneol.py +++ b/Tools/scripts/svneol.py @@ -1,6 +1,6 @@ #! /usr/bin/env python3 -""" +r""" SVN helper script. Try to set the svn:eol-style property to "native" on every .py, .txt, .c and diff --git a/Tools/scripts/texi2html.py b/Tools/scripts/texi2html.py --- a/Tools/scripts/texi2html.py +++ b/Tools/scripts/texi2html.py @@ -78,11 +78,11 @@ # running text # # menu item (Yuck!) -miprog = re.compile('^\* ([^:]*):(:|[ \t]*([^\t,\n.]+)([^ \t\n]*))[ \t\n]*') -# 0 1 1 2 3 34 42 0 -# ----- ---------- --------- -# -|----------------------------- -# ----------------------------------------------------- +miprog = re.compile(r'^\* ([^:]*):(:|[ \t]*([^\t,\n.]+)([^ \t\n]*))[ \t\n]*') +# 0 1 1 2 3 34 42 0 +# ----- ---------- --------- +# -|----------------------------- +# ----------------------------------------------------- diff --git a/Tools/unicode/gencodec.py b/Tools/unicode/gencodec.py --- a/Tools/unicode/gencodec.py +++ b/Tools/unicode/gencodec.py @@ -37,11 +37,11 @@ # Placeholder for a missing code point MISSING_CODE = -1 -mapRE = re.compile('((?:0x[0-9a-fA-F]+\+?)+)' - '\s+' - '((?:(?:0x[0-9a-fA-Z]+|<[A-Za-z]+>)\+?)*)' - '\s*' - '(#.+)?') +mapRE = re.compile(r'((?:0x[0-9a-fA-F]+\+?)+)' + r'\s+' + r'((?:(?:0x[0-9a-fA-Z]+|<[A-Za-z]+>)\+?)*)' + r'\s*' + r'(#.+)?') def parsecodes(codes, len=len, range=range): diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -838,7 +838,7 @@ # find out which version of OpenSSL we have openssl_ver = 0 openssl_ver_re = re.compile( - '^\s*#\s*define\s+OPENSSL_VERSION_NUMBER\s+(0x[0-9a-fA-F]+)' ) + r'^\s*#\s*define\s+OPENSSL_VERSION_NUMBER\s+(0x[0-9a-fA-F]+)' ) # look for the openssl version header on the compiler search path. opensslv_h = find_file('openssl/opensslv.h', [], @@ -1724,7 +1724,7 @@ # All existing framework builds of Tcl/Tk don't support 64-bit # architectures. cflags = sysconfig.get_config_vars('CFLAGS')[0] - archs = re.findall('-arch\s+(\w+)', cflags) + archs = re.findall(r'-arch\s+(\w+)', cflags) tmpfile = os.path.join(self.build_temp, 'tk.arch') if not os.path.exists(self.build_temp): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:03:34 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 08 Sep 2016 18:03:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_assertions_to_dk=5Fset?= =?utf-8?b?X2luZGV4KCk=?= Message-ID: <20160908180334.2012.70277.33F05328@psf.io> https://hg.python.org/cpython/rev/36545fa93d2b changeset: 103325:36545fa93d2b user: Victor Stinner date: Thu Sep 08 10:52:46 2016 -0700 summary: Add assertions to dk_set_index() Issue #27350. files: Objects/dictobject.c | 18 ++++++++++++++---- 1 files changed, 14 insertions(+), 4 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -304,20 +304,24 @@ dk_get_index(PyDictKeysObject *keys, Py_ssize_t i) { Py_ssize_t s = DK_SIZE(keys); + Py_ssize_t ix; + if (s <= 0xff) { - return ((char*) &keys->dk_indices[0])[i]; + ix = ((char*) &keys->dk_indices[0])[i]; } else if (s <= 0xffff) { - return ((int16_t*)&keys->dk_indices[0])[i]; + ix = ((int16_t*)&keys->dk_indices[0])[i]; } #if SIZEOF_VOID_P > 4 else if (s <= 0xffffffff) { - return ((int32_t*)&keys->dk_indices[0])[i]; + ix = ((int32_t*)&keys->dk_indices[0])[i]; } #endif else { - return ((Py_ssize_t*)&keys->dk_indices[0])[i]; + ix = ((Py_ssize_t*)&keys->dk_indices[0])[i]; } + assert(ix >= DKIX_DUMMY); + return ix; } /* write to indices. */ @@ -325,14 +329,20 @@ dk_set_index(PyDictKeysObject *keys, Py_ssize_t i, Py_ssize_t ix) { Py_ssize_t s = DK_SIZE(keys); + + assert(ix >= DKIX_DUMMY); + if (s <= 0xff) { + assert(ix <= 0x7f); ((char*) &keys->dk_indices[0])[i] = (char)ix; } else if (s <= 0xffff) { + assert(ix <= 0x7fff); ((int16_t*) &keys->dk_indices[0])[i] = (int16_t)ix; } #if SIZEOF_VOID_P > 4 else if (s <= 0xffffffff) { + assert(ix <= 0x7fffffff); ((int32_t*) &keys->dk_indices[0])[i] = (int32_t)ix; } #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:04:55 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 18:04:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_pep_role?= Message-ID: <20160908180455.18872.96481.AEF59FE5@psf.io> https://hg.python.org/cpython/rev/e0922837e2d7 changeset: 103326:e0922837e2d7 user: Benjamin Peterson date: Thu Sep 08 11:03:55 2016 -0700 summary: fix pep role files: Doc/whatsnew/3.6.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -374,7 +374,7 @@ * :func:`dict` now uses a "compact" representation `pioneered by PyPy `_. - :pep:`PEP 468` (Preserving the order of ``**kwargs`` in a function.) is + :pep:`468` (Preserving the order of ``**kwargs`` in a function.) is implemented by this. (Contributed by INADA Naoki in :issue:`27350`.) * Long sequences of repeated traceback lines are now abbreviated as -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:07:39 2016 From: python-checkins at python.org (davin.potts) Date: Thu, 08 Sep 2016 18:07:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIxMjAx?= =?utf-8?q?=3A_Improves_readability_of_multiprocessing_error_message_from_?= =?utf-8?q?server?= Message-ID: <20160908180738.69446.24378.8DF849D4@psf.io> https://hg.python.org/cpython/rev/cbf81969aba4 changeset: 103327:cbf81969aba4 branch: 2.7 parent: 103296:3f04287e7bea user: Davin Potts date: Thu Sep 08 13:07:13 2016 -0500 summary: Issue #21201: Improves readability of multiprocessing error message from server to client for certain exceptions files: Lib/multiprocessing/managers.py | 2 +- Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -287,7 +287,7 @@ try: send(msg) except Exception, e: - send(('#UNSERIALIZABLE', repr(msg))) + send(('#UNSERIALIZABLE', format_exc())) except Exception, e: util.info('exception in thread serving %r', threading.current_thread().name) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -125,6 +125,9 @@ - Issue #25455: Fixed a crash in repr of cElementTree.Element with recursive tag. +- Issue #21201: Improves readability of multiprocessing error message. Thanks + to Wojciech Walczak for patch. + IDLE ---- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:08:41 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 18:08:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_spelling?= Message-ID: <20160908180841.12375.4210.1FA6AB7D@psf.io> https://hg.python.org/cpython/rev/7eda9b57531d changeset: 103328:7eda9b57531d parent: 103326:e0922837e2d7 user: Benjamin Peterson date: Thu Sep 08 11:08:30 2016 -0700 summary: fix spelling files: Objects/odictobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -39,7 +39,7 @@ __getitem__(), get(), etc. accordingly. The approach with the least performance impact (time and space) is #2, -mirroring the key order of dict's dk_enties with an array of node pointers. +mirroring the key order of dict's dk_entries with an array of node pointers. While lookdict() and friends (dk_lookup) don't give us the index into the array, we make use of pointer arithmetic to get that index. An alternative would be to refactor lookdict() to provide the index, explicitly exposing -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:11:34 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 18:11:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixes_tests_broken_by_issu?= =?utf-8?b?ZSAjMjc3ODEu?= Message-ID: <20160908181134.1702.47082.60EB1765@psf.io> https://hg.python.org/cpython/rev/faca0730270b changeset: 103329:faca0730270b user: Steve Dower date: Thu Sep 08 11:11:13 2016 -0700 summary: Fixes tests broken by issue #27781. files: Lib/test/test_genericpath.py | 5 ++++- Lib/test/test_httpservers.py | 2 ++ Lib/test/test_shutil.py | 4 +--- Lib/test/test_sys.py | 5 +++-- 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_genericpath.py b/Lib/test/test_genericpath.py --- a/Lib/test/test_genericpath.py +++ b/Lib/test/test_genericpath.py @@ -388,10 +388,13 @@ warnings.simplefilter("ignore", DeprecationWarning) self.assertIn(b"foo", self.pathmodule.abspath(b"foo")) + # avoid UnicodeDecodeError on Windows + undecodable_path = b'' if sys.platform == 'win32' else b'f\xf2\xf2' + # Abspath returns bytes when the arg is bytes with warnings.catch_warnings(): warnings.simplefilter("ignore", DeprecationWarning) - for path in (b'', b'foo', b'f\xf2\xf2', b'/foo', b'C:\\'): + for path in (b'', b'foo', undecodable_path, b'/foo', b'C:\\'): self.assertIsInstance(self.pathmodule.abspath(path), bytes) def test_realpath(self): diff --git a/Lib/test/test_httpservers.py b/Lib/test/test_httpservers.py --- a/Lib/test/test_httpservers.py +++ b/Lib/test/test_httpservers.py @@ -370,6 +370,8 @@ return body @support.requires_mac_ver(10, 5) + @unittest.skipIf(sys.platform == 'win32', + 'undecodable name cannot be decoded on win32') @unittest.skipUnless(support.TESTFN_UNDECODABLE, 'need support.TESTFN_UNDECODABLE') def test_undecodable_filename(self): diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -132,9 +132,7 @@ write_file(os.path.join(victim, 'somefile'), 'foo') victim = os.fsencode(victim) self.assertIsInstance(victim, bytes) - win = (os.name == 'nt') - with self.assertWarns(DeprecationWarning) if win else ExitStack(): - shutil.rmtree(victim) + shutil.rmtree(victim) @support.skip_unless_symlink def test_rmtree_fails_on_symlink(self): diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -10,6 +10,7 @@ import gc import sysconfig import platform +import locale # count the number of test runs, used to create unique # strings to intern in test_intern() @@ -627,6 +628,8 @@ @unittest.skipUnless(test.support.FS_NONASCII, 'requires OS support of non-ASCII encodings') + @unittest.skipUnless(sys.getfilesystemencoding() == locale.getpreferredencoding(False), + 'requires FS encoding to match locale') def test_ioencoding_nonascii(self): env = dict(os.environ) @@ -669,8 +672,6 @@ fs_encoding = sys.getfilesystemencoding() if sys.platform == 'darwin': expected = 'utf-8' - elif sys.platform == 'win32': - expected = 'mbcs' else: expected = None self.check_fsencoding(fs_encoding, expected) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:14:38 2016 From: python-checkins at python.org (eric.snow) Date: Thu, 08 Sep 2016 18:14:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328026=3A_Raise_Im?= =?utf-8?q?portError_when_exec=5Fmodule=28=29_exists_but_create=5Fmodule?= =?utf-8?b?KCk=?= Message-ID: <20160908181438.21237.42850.27F5BC46@psf.io> https://hg.python.org/cpython/rev/ada5620efd82 changeset: 103330:ada5620efd82 user: Eric Snow date: Thu Sep 08 11:12:31 2016 -0700 summary: Issue #28026: Raise ImportError when exec_module() exists but create_module() is missing. files: Lib/importlib/_bootstrap.py | 5 +- Lib/test/test_importlib/test_util.py | 8 +- Misc/NEWS | 3 + Python/importlib.h | 1798 ++++++------- 4 files changed, 903 insertions(+), 911 deletions(-) diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py --- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -559,9 +559,8 @@ # module creation should be used. module = spec.loader.create_module(spec) elif hasattr(spec.loader, 'exec_module'): - _warnings.warn('starting in Python 3.6, loaders defining exec_module() ' - 'must also define create_module()', - DeprecationWarning, stacklevel=2) + raise ImportError('loaders that define exec_module() ' + 'must also define create_module()') if module is None: module = _new_module(spec.name) _init_module_attrs(spec, module) diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py --- a/Lib/test/test_importlib/test_util.py +++ b/Lib/test/test_importlib/test_util.py @@ -47,14 +47,8 @@ def exec_module(self, module): pass spec = self.machinery.ModuleSpec('test', Loader()) - with warnings.catch_warnings(record=True) as w: - warnings.simplefilter('always') + with self.assertRaises(ImportError): module = self.util.module_from_spec(spec) - self.assertEqual(1, len(w)) - self.assertTrue(issubclass(w[0].category, DeprecationWarning)) - self.assertIn('create_module', str(w[0].message)) - self.assertIsInstance(module, types.ModuleType) - self.assertEqual(module.__name__, spec.name) def test_create_module_returns_None(self): class Loader(self.abc.Loader): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -6968,6 +6968,9 @@ - Issue #19369: Optimized the usage of __length_hint__(). +- Issue #28026: Raise ImportError when exec_module() exists but + create_module() is missing. + - Issue #18603: Ensure that PyOS_mystricmp and PyOS_mystrnicmp are in the Python executable and not removed by the linker's optimizer. diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:16:59 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 08 Sep 2016 18:16:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Split_lookdict=5Funicode?= =?utf-8?q?=5Fnodummy=28=29_assertion_to_debug?= Message-ID: <20160908181659.1897.49280.D7C8C4BC@psf.io> https://hg.python.org/cpython/rev/19902d840448 changeset: 103331:19902d840448 user: Victor Stinner date: Thu Sep 08 11:16:07 2016 -0700 summary: Split lookdict_unicode_nodummy() assertion to debug Issue #27350. files: Objects/dictobject.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -802,7 +802,8 @@ return DKIX_EMPTY; } ep = &ep0[ix]; - assert(ep->me_key != NULL && PyUnicode_CheckExact(ep->me_key)); + assert(ep->me_key != NULL); + assert(PyUnicode_CheckExact(ep->me_key)); if (ep->me_key == key || (ep->me_hash == hash && unicode_eq(ep->me_key, key))) { if (hashpos != NULL) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:22:03 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 18:22:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323524=3A_Finish_r?= =?utf-8?q?emoving_=5FPyVerify=5Ffd_from_sources?= Message-ID: <20160908182202.36273.10448.47F9748E@psf.io> https://hg.python.org/cpython/rev/e88e2049b793 changeset: 103332:e88e2049b793 user: Steve Dower date: Thu Sep 08 11:21:54 2016 -0700 summary: Issue #23524: Finish removing _PyVerify_fd from sources files: Include/fileutils.h | 12 -- Modules/_io/fileio.c | 39 ++----- Modules/faulthandler.c | 2 +- Modules/mmapmodule.c | 4 - Modules/posixmodule.c | 75 +---------------- Modules/signalmodule.c | 7 +- PC/msvcrtmodule.c | 13 +-- Parser/myreadline.c | 5 +- Python/fileutils.c | 128 +---------------------------- Python/pylifecycle.c | 2 +- 10 files changed, 28 insertions(+), 259 deletions(-) diff --git a/Include/fileutils.h b/Include/fileutils.h --- a/Include/fileutils.h +++ b/Include/fileutils.h @@ -121,18 +121,6 @@ PyAPI_FUNC(int) _Py_set_blocking(int fd, int blocking); #endif /* !MS_WINDOWS */ -#if defined _MSC_VER && _MSC_VER >= 1400 && _MSC_VER < 1900 -/* A routine to check if a file descriptor is valid on Windows. Returns 0 - * and sets errno to EBADF if it isn't. This is to avoid Assertions - * from various functions in the Windows CRT beginning with - * Visual Studio 2005 - */ -int _PyVerify_fd(int fd); - -#else -#define _PyVerify_fd(A) (1) /* dummy */ -#endif - #endif /* Py_LIMITED_API */ #ifdef __cplusplus diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c --- a/Modules/_io/fileio.c +++ b/Modules/_io/fileio.c @@ -117,18 +117,13 @@ int fd = self->fd; self->fd = -1; /* fd is accessible and someone else may have closed it */ - if (_PyVerify_fd(fd)) { - Py_BEGIN_ALLOW_THREADS - _Py_BEGIN_SUPPRESS_IPH - err = close(fd); - if (err < 0) - save_errno = errno; - _Py_END_SUPPRESS_IPH - Py_END_ALLOW_THREADS - } else { + Py_BEGIN_ALLOW_THREADS + _Py_BEGIN_SUPPRESS_IPH + err = close(fd); + if (err < 0) save_errno = errno; - err = -1; - } + _Py_END_SUPPRESS_IPH + Py_END_ALLOW_THREADS } if (err < 0) { errno = save_errno; @@ -700,8 +695,6 @@ if (self->fd < 0) return err_closed(); - if (!_PyVerify_fd(self->fd)) - return PyErr_SetFromErrno(PyExc_IOError); _Py_BEGIN_SUPPRESS_IPH #ifdef MS_WINDOWS @@ -914,18 +907,15 @@ return NULL; } - if (_PyVerify_fd(fd)) { - Py_BEGIN_ALLOW_THREADS - _Py_BEGIN_SUPPRESS_IPH + Py_BEGIN_ALLOW_THREADS + _Py_BEGIN_SUPPRESS_IPH #ifdef MS_WINDOWS - res = _lseeki64(fd, pos, whence); + res = _lseeki64(fd, pos, whence); #else - res = lseek(fd, pos, whence); + res = lseek(fd, pos, whence); #endif - _Py_END_SUPPRESS_IPH - Py_END_ALLOW_THREADS - } else - res = -1; + _Py_END_SUPPRESS_IPH + Py_END_ALLOW_THREADS if (res < 0) return PyErr_SetFromErrno(PyExc_IOError); @@ -1116,10 +1106,7 @@ return err_closed(); Py_BEGIN_ALLOW_THREADS _Py_BEGIN_SUPPRESS_IPH - if (_PyVerify_fd(self->fd)) - res = isatty(self->fd); - else - res = 0; + res = isatty(self->fd); _Py_END_SUPPRESS_IPH Py_END_ALLOW_THREADS return PyBool_FromLong(res); diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -159,7 +159,7 @@ fd = _PyLong_AsInt(file); if (fd == -1 && PyErr_Occurred()) return -1; - if (fd < 0 || !_PyVerify_fd(fd)) { + if (fd < 0) { PyErr_SetString(PyExc_ValueError, "file is not a valid file descripter"); return -1; diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -1326,10 +1326,6 @@ */ if (fileno != -1 && fileno != 0) { /* Ensure that fileno is within the CRT's valid range */ - if (!_PyVerify_fd(fileno)) { - PyErr_SetFromErrno(PyExc_OSError); - return NULL; - } _Py_BEGIN_SUPPRESS_IPH fh = (HANDLE)_get_osfhandle(fileno); _Py_END_SUPPRESS_IPH diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -1178,34 +1178,6 @@ #endif } - -#if defined _MSC_VER && _MSC_VER >= 1400 && _MSC_VER < 1900 -/* Legacy implementation of _PyVerify_fd_dup2 while transitioning to - * MSVC 14.0. This should eventually be removed. (issue23524) - */ -#define IOINFO_L2E 5 -#define IOINFO_ARRAYS 64 -#define IOINFO_ARRAY_ELTS (1 << IOINFO_L2E) -#define _NHANDLE_ (IOINFO_ARRAYS * IOINFO_ARRAY_ELTS) -#define _NO_CONSOLE_FILENO (intptr_t)-2 - -/* the special case of checking dup2. The target fd must be in a sensible range */ -static int -_PyVerify_fd_dup2(int fd1, int fd2) -{ - if (!_PyVerify_fd(fd1)) - return 0; - if (fd2 == _NO_CONSOLE_FILENO) - return 0; - if ((unsigned)fd2 < _NHANDLE_) - return 1; - else - return 0; -} -#else -#define _PyVerify_fd_dup2(fd1, fd2) (_PyVerify_fd(fd1) && (fd2) >= 0) -#endif - #ifdef MS_WINDOWS static int @@ -1409,9 +1381,6 @@ int res; int async_err = 0; - if (!_PyVerify_fd(fd)) - return posix_error(); - do { Py_BEGIN_ALLOW_THREADS _Py_BEGIN_SUPPRESS_IPH @@ -7549,8 +7518,6 @@ /*[clinic end generated code: output=2fe4e93602822c14 input=2bc42451ca5c3223]*/ { int res; - if (!_PyVerify_fd(fd)) - return posix_error(); /* We do not want to retry upon EINTR: see http://lwn.net/Articles/576478/ * and http://linux.derkeiler.com/Mailing-Lists/Kernel/2005-09/3000.html * for more details. @@ -7583,9 +7550,8 @@ int i; Py_BEGIN_ALLOW_THREADS _Py_BEGIN_SUPPRESS_IPH - for (i = fd_low; i < fd_high; i++) - if (_PyVerify_fd(i)) - close(i); + for (i = max(fd_low, 0); i < fd_high; i++) + close(i); _Py_END_SUPPRESS_IPH Py_END_ALLOW_THREADS Py_RETURN_NONE; @@ -7629,7 +7595,7 @@ int dup3_works = -1; #endif - if (!_PyVerify_fd_dup2(fd, fd2)) + if (fd < 0 || fd2 < 0) return posix_error(); /* dup2() can fail with EINTR if the target FD is already open, because it @@ -7753,10 +7719,6 @@ { Py_off_t result; - if (!_PyVerify_fd(fd)) { - posix_error(); - return -1; - } #ifdef SEEK_SET /* Turn 0, 1, 2 into SEEK_{SET,CUR,END} */ switch (how) { @@ -7769,10 +7731,6 @@ if (PyErr_Occurred()) return -1; - if (!_PyVerify_fd(fd)) { - posix_error(); - return -1; - } Py_BEGIN_ALLOW_THREADS _Py_BEGIN_SUPPRESS_IPH #ifdef MS_WINDOWS @@ -7980,10 +7938,6 @@ buffer = PyBytes_FromStringAndSize((char *)NULL, length); if (buffer == NULL) return NULL; - if (!_PyVerify_fd(fd)) { - Py_DECREF(buffer); - return posix_error(); - } do { Py_BEGIN_ALLOW_THREADS @@ -8226,8 +8180,6 @@ /*[clinic end generated code: output=6a48c8b4e644ca00 input=08ce94aa1eaf7b5e]*/ { int return_value; - if (!_PyVerify_fd(fd)) - return 0; _Py_BEGIN_SUPPRESS_IPH return_value = isatty(fd); _Py_END_SUPPRESS_IPH @@ -8419,11 +8371,6 @@ Py_ssize_t size; int async_err = 0; - if (!_PyVerify_fd(fd)) { - posix_error(); - return -1; - } - do { Py_BEGIN_ALLOW_THREADS _Py_BEGIN_SUPPRESS_IPH @@ -8606,9 +8553,6 @@ int result; int async_err = 0; - if (!_PyVerify_fd(fd)) - return posix_error(); - do { Py_BEGIN_ALLOW_THREADS _Py_BEGIN_SUPPRESS_IPH @@ -10979,11 +10923,6 @@ /*[clinic end generated code: output=0445e20e149aa5b8 input=89ac008dc9ab6b95]*/ { int return_value; - if (!_PyVerify_fd(fd)) { - posix_error(); - return -1; - } - _Py_BEGIN_SUPPRESS_IPH return_value = _Py_get_inheritable(fd); _Py_END_SUPPRESS_IPH @@ -11005,8 +10944,6 @@ /*[clinic end generated code: output=f1b1918a2f3c38c2 input=9ceaead87a1e2402]*/ { int result; - if (!_PyVerify_fd(fd)) - return posix_error(); _Py_BEGIN_SUPPRESS_IPH result = _Py_set_inheritable(fd, inheritable, NULL); @@ -11080,9 +11017,6 @@ if (!PyArg_ParseTuple(args, "i:get_blocking", &fd)) return NULL; - if (!_PyVerify_fd(fd)) - return posix_error(); - _Py_BEGIN_SUPPRESS_IPH blocking = _Py_get_blocking(fd); _Py_END_SUPPRESS_IPH @@ -11106,9 +11040,6 @@ if (!PyArg_ParseTuple(args, "ii:set_blocking", &fd, &blocking)) return NULL; - if (!_PyVerify_fd(fd)) - return posix_error(); - _Py_BEGIN_SUPPRESS_IPH result = _Py_set_blocking(fd, blocking); _Py_END_SUPPRESS_IPH diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -579,7 +579,7 @@ } fd = (int)sockfd; - if ((SOCKET_T)fd != sockfd || !_PyVerify_fd(fd)) { + if ((SOCKET_T)fd != sockfd) { PyErr_SetString(PyExc_ValueError, "invalid fd"); return NULL; } @@ -609,11 +609,6 @@ if (fd != -1) { int blocking; - if (!_PyVerify_fd(fd)) { - PyErr_SetString(PyExc_ValueError, "invalid fd"); - return NULL; - } - if (_Py_fstat(fd, &status) != 0) return NULL; diff --git a/PC/msvcrtmodule.c b/PC/msvcrtmodule.c --- a/PC/msvcrtmodule.c +++ b/PC/msvcrtmodule.c @@ -189,16 +189,11 @@ { intptr_t handle = -1; - if (!_PyVerify_fd(fd)) { + _Py_BEGIN_SUPPRESS_IPH + handle = _get_osfhandle(fd); + _Py_END_SUPPRESS_IPH + if (handle == -1) PyErr_SetFromErrno(PyExc_IOError); - } - else { - _Py_BEGIN_SUPPRESS_IPH - handle = _get_osfhandle(fd); - _Py_END_SUPPRESS_IPH - if (handle == -1) - PyErr_SetFromErrno(PyExc_IOError); - } return handle; } diff --git a/Parser/myreadline.c b/Parser/myreadline.c --- a/Parser/myreadline.c +++ b/Parser/myreadline.c @@ -41,10 +41,7 @@ (void)(PyOS_InputHook)(); errno = 0; clearerr(fp); - if (_PyVerify_fd(fileno(fp))) - p = fgets(buf, len, fp); - else - p = NULL; + p = fgets(buf, len, fp); if (p != NULL) return 0; /* No error */ err = errno; diff --git a/Python/fileutils.c b/Python/fileutils.c --- a/Python/fileutils.c +++ b/Python/fileutils.c @@ -44,7 +44,7 @@ #endif int valid; _Py_BEGIN_SUPPRESS_IPH - valid = _PyVerify_fd(fd) && isatty(fd); + valid = isatty(fd); _Py_END_SUPPRESS_IPH if (!valid) Py_RETURN_NONE; @@ -613,13 +613,9 @@ HANDLE h; int type; - if (!_PyVerify_fd(fd)) - h = INVALID_HANDLE_VALUE; - else { - _Py_BEGIN_SUPPRESS_IPH - h = (HANDLE)_get_osfhandle(fd); - _Py_END_SUPPRESS_IPH - } + _Py_BEGIN_SUPPRESS_IPH + h = (HANDLE)_get_osfhandle(fd); + _Py_END_SUPPRESS_IPH if (h == INVALID_HANDLE_VALUE) { /* errno is already set by _get_osfhandle, but we also set @@ -742,12 +738,6 @@ HANDLE handle; DWORD flags; - if (!_PyVerify_fd(fd)) { - if (raise) - PyErr_SetFromErrno(PyExc_OSError); - return -1; - } - _Py_BEGIN_SUPPRESS_IPH handle = (HANDLE)_get_osfhandle(fd); _Py_END_SUPPRESS_IPH @@ -819,12 +809,6 @@ } #ifdef MS_WINDOWS - if (!_PyVerify_fd(fd)) { - if (raise) - PyErr_SetFromErrno(PyExc_OSError); - return -1; - } - _Py_BEGIN_SUPPRESS_IPH handle = (HANDLE)_get_osfhandle(fd); _Py_END_SUPPRESS_IPH @@ -1190,14 +1174,6 @@ * handler raised an exception. */ assert(!PyErr_Occurred()); - if (!_PyVerify_fd(fd)) { - /* save/restore errno because PyErr_SetFromErrno() can modify it */ - err = errno; - PyErr_SetFromErrno(PyExc_OSError); - errno = err; - return -1; - } - #ifdef MS_WINDOWS if (count > INT_MAX) { /* On Windows, the count parameter of read() is an int */ @@ -1251,16 +1227,6 @@ int err; int async_err = 0; - if (!_PyVerify_fd(fd)) { - if (gil_held) { - /* save/restore errno because PyErr_SetFromErrno() can modify it */ - err = errno; - PyErr_SetFromErrno(PyExc_OSError); - errno = err; - } - return -1; - } - _Py_BEGIN_SUPPRESS_IPH #ifdef MS_WINDOWS if (count > 32767 && isatty(fd)) { @@ -1497,11 +1463,6 @@ assert(PyGILState_Check()); #endif - if (!_PyVerify_fd(fd)) { - PyErr_SetFromErrno(PyExc_OSError); - return -1; - } - #ifdef MS_WINDOWS _Py_BEGIN_SUPPRESS_IPH handle = (HANDLE)_get_osfhandle(fd); @@ -1624,84 +1585,3 @@ return -1; } #endif - -#if defined _MSC_VER && _MSC_VER >= 1400 && _MSC_VER < 1900 -/* Legacy implementation of _PyVerify_fd while transitioning to - * MSVC 14.0. This should eventually be removed. (issue23524) - */ - -/* Microsoft CRT in VS2005 and higher will verify that a filehandle is - * valid and raise an assertion if it isn't. - * Normally, an invalid fd is likely to be a C program error and therefore - * an assertion can be useful, but it does contradict the POSIX standard - * which for write(2) states: - * "Otherwise, -1 shall be returned and errno set to indicate the error." - * "[EBADF] The fildes argument is not a valid file descriptor open for - * writing." - * Furthermore, python allows the user to enter any old integer - * as a fd and should merely raise a python exception on error. - * The Microsoft CRT doesn't provide an official way to check for the - * validity of a file descriptor, but we can emulate its internal behaviour - * by using the exported __pinfo data member and knowledge of the - * internal structures involved. - * The structures below must be updated for each version of visual studio - * according to the file internal.h in the CRT source, until MS comes - * up with a less hacky way to do this. - * (all of this is to avoid globally modifying the CRT behaviour using - * _set_invalid_parameter_handler() and _CrtSetReportMode()) - */ -/* The actual size of the structure is determined at runtime. - * Only the first items must be present. - */ -typedef struct { - intptr_t osfhnd; - char osfile; -} my_ioinfo; - -extern __declspec(dllimport) char * __pioinfo[]; -#define IOINFO_L2E 5 -#define IOINFO_ARRAYS 64 -#define IOINFO_ARRAY_ELTS (1 << IOINFO_L2E) -#define _NHANDLE_ (IOINFO_ARRAYS * IOINFO_ARRAY_ELTS) -#define FOPEN 0x01 -#define _NO_CONSOLE_FILENO (intptr_t)-2 - -/* This function emulates what the windows CRT does to validate file handles */ -int -_PyVerify_fd(int fd) -{ - const int i1 = fd >> IOINFO_L2E; - const int i2 = fd & ((1 << IOINFO_L2E) - 1); - - static size_t sizeof_ioinfo = 0; - - /* Determine the actual size of the ioinfo structure, - * as used by the CRT loaded in memory - */ - if (sizeof_ioinfo == 0 && __pioinfo[0] != NULL) { - sizeof_ioinfo = _msize(__pioinfo[0]) / IOINFO_ARRAY_ELTS; - } - if (sizeof_ioinfo == 0) { - /* This should not happen... */ - goto fail; - } - - /* See that it isn't a special CLEAR fileno */ - if (fd != _NO_CONSOLE_FILENO) { - /* Microsoft CRT would check that 0<=fd<_nhandle but we can't do that. Instead - * we check pointer validity and other info - */ - if (0 <= i1 && i1 < IOINFO_ARRAYS && __pioinfo[i1] != NULL) { - /* finally, check that the file is open */ - my_ioinfo* info = (my_ioinfo*)(__pioinfo[i1] + i2 * sizeof_ioinfo); - if (info->osfile & FOPEN) { - return 1; - } - } - } - fail: - errno = EBADF; - return 0; -} - -#endif /* defined _MSC_VER && _MSC_VER >= 1400 && _MSC_VER < 1900 */ diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1026,7 +1026,7 @@ is_valid_fd(int fd) { int fd2; - if (fd < 0 || !_PyVerify_fd(fd)) + if (fd < 0) return 0; _Py_BEGIN_SUPPRESS_IPH /* Prefer dup() over fstat(). fstat() can require input/output whereas -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Thu Sep 8 14:22:33 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 8 Sep 2016 19:22:33 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-09-08 Message-ID: <1fe872e3-0bd5-41e7-8028-ba39a7f3a0b4@irsmsx101.ger.corp.intel.com> Results for project Python 2.7, build date 2016-09-08 02:47:18 +0000 commit: 2d6dd8402d77 previous commit: 75dae0b2ccb3 revision date: 2016-09-08 02:08:02 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.17% 1.77% 4.47% 3.01% :-) pybench 0.29% 0.66% 6.30% 2.91% :-( regex_v8 0.61% 0.12% -2.04% 9.95% :-) nbody 0.06% -0.18% 6.81% 5.39% :-| json_dump_v2 0.36% -0.94% 0.70% 10.11% :-| normal_startup 1.05% 0.45% -0.51% 2.33% :-) ssbench 0.20% -0.10% 2.48% 1.01% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-09-08/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From lp_benchmark_robot at intel.com Thu Sep 8 14:23:48 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Thu, 8 Sep 2016 19:23:48 +0100 Subject: [Python-checkins] BAD Benchmark Results for Python Default 2016-09-08 Message-ID: Results for project Python default, build date 2016-09-08 02:01:01 +0000 commit: 344f44bd793f previous commit: 4aa6233961e5 revision date: 2016-09-08 01:48:06 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.22% -0.11% 10.25% 17.77% :-) pybench 0.16% 0.76% 7.41% 6.22% :-( regex_v8 2.61% 0.88% -2.66% 1.42% :-( nbody 0.10% -3.52% -1.16% 11.46% :-( json_dump_v2 0.34% -0.17% -2.87% 12.78% :-| normal_startup 1.03% -0.24% -1.26% 6.82% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/bad-benchmark-results-for-python-default-2016-09-08/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Thu Sep 8 14:28:13 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 18:28:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_use_Py=5FMAX?= Message-ID: <20160908182812.19423.22533.5B1CEC7B@psf.io> https://hg.python.org/cpython/rev/5ffc6b528fb4 changeset: 103333:5ffc6b528fb4 user: Benjamin Peterson date: Thu Sep 08 11:28:06 2016 -0700 summary: use Py_MAX files: Modules/posixmodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -7550,7 +7550,7 @@ int i; Py_BEGIN_ALLOW_THREADS _Py_BEGIN_SUPPRESS_IPH - for (i = max(fd_low, 0); i < fd_high; i++) + for (i = Py_MAX(fd_low, 0); i < fd_high; i++) close(i); _Py_END_SUPPRESS_IPH Py_END_ALLOW_THREADS -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:34:47 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 18:34:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_document_--wit?= =?utf-8?q?h-optimizations_in_the_README_for_issue26359=2E?= Message-ID: <20160908183445.69669.69431.3E9FB994@psf.io> https://hg.python.org/cpython/rev/e1987bf14148 changeset: 103334:e1987bf14148 branch: 3.5 parent: 103312:4f6fef83cd0c user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 18:33:00 2016 +0000 summary: document --with-optimizations in the README for issue26359. files: README | 17 +++++++++++++---- 1 files changed, 13 insertions(+), 4 deletions(-) diff --git a/README b/README --- a/README +++ b/README @@ -46,16 +46,17 @@ (This will fail if you *also* built at the top-level directory. You should do a "make clean" at the toplevel first.) -If you need an optimized version of Python, you type "make profile-opt" in the -top level directory. This will rebuild the interpreter executable using Profile -Guided Optimization (PGO). For more details, see the section bellow. +To get an optimized build of Python, "configure --with-optimizations" before +you run make. This sets the default make targets up to enable Profile Guided +Optimization (PGO) and Link Time Optimization (LTO) on most platforms. +For more details, see the sections bellow. Profile Guided Optimization --------------------------- PGO takes advantage of recent versions of the GCC or Clang compilers. -If ran, the "profile-opt" rule will do several steps. +If ran, "make profile-opt" will do several steps. First, the entire Python directory is cleaned of temporary files that may have resulted in a previous compilation. @@ -75,6 +76,14 @@ that is optimized and suitable for distribution or production installation. +Link Time Optimization +---------------------- + +LTO takes advantages of recent compiler toolchains ability to optimize across +the otherwise arbitrary .o file boundary when building final executables or +shared libraries for additional performance gains. + + What's New ---------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:34:48 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 18:34:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_document_--with-optimizations_in_the_README_for_issue263?= =?utf-8?b?NTku?= Message-ID: <20160908183446.69505.76114.0C6C2D95@psf.io> https://hg.python.org/cpython/rev/4afb12077e08 changeset: 103335:4afb12077e08 parent: 103333:5ffc6b528fb4 parent: 103334:e1987bf14148 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 18:33:45 2016 +0000 summary: document --with-optimizations in the README for issue26359. files: README | 17 +++++++++++++---- 1 files changed, 13 insertions(+), 4 deletions(-) diff --git a/README b/README --- a/README +++ b/README @@ -46,16 +46,17 @@ (This will fail if you *also* built at the top-level directory. You should do a "make clean" at the toplevel first.) -If you need an optimized version of Python, you type "make profile-opt" in the -top level directory. This will rebuild the interpreter executable using Profile -Guided Optimization (PGO). For more details, see the section bellow. +To get an optimized build of Python, "configure --with-optimizations" before +you run make. This sets the default make targets up to enable Profile Guided +Optimization (PGO) and Link Time Optimization (LTO) on most platforms. +For more details, see the sections bellow. Profile Guided Optimization --------------------------- PGO takes advantage of recent versions of the GCC or Clang compilers. -If ran, the "profile-opt" rule will do several steps. +If ran, "make profile-opt" will do several steps. First, the entire Python directory is cleaned of temporary files that may have resulted in a previous compilation. @@ -75,6 +76,14 @@ that is optimized and suitable for distribution or production installation. +Link Time Optimization +---------------------- + +LTO takes advantages of recent compiler toolchains ability to optimize across +the otherwise arbitrary .o file boundary when building final executables or +shared libraries for additional performance gains. + + What's New ---------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:38:06 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 18:38:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_document_--wit?= =?utf-8?q?h-optimizations_in_the_README_for_issue26359=2E?= Message-ID: <20160908183806.13248.48665.F8D60DAA@psf.io> https://hg.python.org/cpython/rev/8567bc2876af changeset: 103336:8567bc2876af branch: 2.7 parent: 103327:cbf81969aba4 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 18:37:59 2016 +0000 summary: document --with-optimizations in the README for issue26359. files: README | 18 +++++++++++++----- 1 files changed, 13 insertions(+), 5 deletions(-) diff --git a/README b/README --- a/README +++ b/README @@ -173,10 +173,10 @@ build your desired target. The interpreter executable is built in the top level directory. -If you need an optimized version of Python, you type "make profile-opt" -in the top level directory. This will rebuild the interpreter executable -using Profile Guided Optimization (PGO). For more details, see the -section below. +To get an optimized build of Python, "configure --with-optimizations" before +you run make. This sets the default make targets up to enable Profile Guided +Optimization (PGO) and Link Time Optimization (LTO) on most platforms. +For more details, see the sections bellow. Once you have built a Python interpreter, see the subsections below on testing and installation. If you run into trouble, see the next @@ -194,7 +194,7 @@ --------------------------- PGO takes advantage of recent versions of the GCC or Clang compilers. -If ran, the "profile-opt" rule will do several steps. +If ran, "make profile-opt" will do several steps. First, the entire Python directory is cleaned of temporary files that may have resulted in a previous compilation. @@ -214,6 +214,14 @@ that is optimized and suitable for distribution or production installation. +Link Time Optimization +---------------------- + +LTO takes advantages of recent compiler toolchains ability to optimize across +the otherwise arbitrary .o file boundary when building final executables or +shared libraries for additional performance gains. + + Troubleshooting --------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:39:12 2016 From: python-checkins at python.org (zach.ware) Date: Thu, 08 Sep 2016 18:39:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328027=3A_Remove_L?= =?utf-8?q?ib/plat-*_files?= Message-ID: <20160908183911.69891.61382.0044F60A@psf.io> https://hg.python.org/cpython/rev/14d89359a389 changeset: 103337:14d89359a389 parent: 103335:4afb12077e08 user: Zachary Ware date: Thu Sep 08 11:38:46 2016 -0700 summary: Issue #28027: Remove Lib/plat-* files files: Doc/whatsnew/3.6.rst | 7 + Lib/plat-aix4/IN.py | 165 - Lib/plat-aix4/regen | 8 - Lib/plat-darwin/IN.py | 662 ------- Lib/plat-darwin/regen | 3 - Lib/plat-freebsd4/IN.py | 355 ---- Lib/plat-freebsd4/regen | 3 - Lib/plat-freebsd5/IN.py | 355 ---- Lib/plat-freebsd5/regen | 3 - Lib/plat-freebsd6/IN.py | 551 ------ Lib/plat-freebsd6/regen | 3 - Lib/plat-freebsd7/IN.py | 571 ------ Lib/plat-freebsd7/regen | 3 - Lib/plat-freebsd8/IN.py | 571 ------ Lib/plat-freebsd8/regen | 3 - Lib/plat-generic/regen | 3 - Lib/plat-linux/CDROM.py | 207 -- Lib/plat-linux/DLFCN.py | 83 - Lib/plat-linux/IN.py | 615 ------- Lib/plat-linux/TYPES.py | 170 - Lib/plat-linux/regen | 33 - Lib/plat-netbsd1/IN.py | 56 - Lib/plat-netbsd1/regen | 3 - Lib/plat-next3/regen | 6 - Lib/plat-sunos5/CDIO.py | 73 - Lib/plat-sunos5/DLFCN.py | 27 - Lib/plat-sunos5/IN.py | 1421 ---------------- Lib/plat-sunos5/STROPTS.py | 1813 --------------------- Lib/plat-sunos5/TYPES.py | 313 --- Lib/plat-sunos5/regen | 9 - Lib/plat-unixware7/IN.py | 836 --------- Lib/plat-unixware7/STROPTS.py | 328 --- Lib/plat-unixware7/regen | 9 - Makefile.pre.in | 22 - Misc/NEWS | 2 + 35 files changed, 9 insertions(+), 9283 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -987,6 +987,13 @@ Use :class:`io.TextIOWrapper` for reading compressed text files in :term:`universal newlines` mode. +* The undocumented ``IN``, ``CDROM``, ``DLFCN``, ``TYPES``, ``CDIO``, and + ``STROPTS`` modules have been removed. They had been available in the + platform specific ``Lib/plat-*/`` directories, but were chronically out of + date, inconsistently available across platforms, and unmaintained. The + script that created these modules is still available in the source + distribution at :source:`Tools/scripts/h2py.py`. + Porting to Python 3.6 ===================== diff --git a/Lib/plat-aix4/IN.py b/Lib/plat-aix4/IN.py deleted file mode 100644 --- a/Lib/plat-aix4/IN.py +++ /dev/null @@ -1,165 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from net/nh.h - -# Included from sys/machine.h -LITTLE_ENDIAN = 1234 -BIG_ENDIAN = 4321 -PDP_ENDIAN = 3412 -BYTE_ORDER = BIG_ENDIAN -DEFAULT_GPR = 0xDEADBEEF -MSR_EE = 0x8000 -MSR_PR = 0x4000 -MSR_FP = 0x2000 -MSR_ME = 0x1000 -MSR_FE = 0x0800 -MSR_FE0 = 0x0800 -MSR_SE = 0x0400 -MSR_BE = 0x0200 -MSR_IE = 0x0100 -MSR_FE1 = 0x0100 -MSR_AL = 0x0080 -MSR_IP = 0x0040 -MSR_IR = 0x0020 -MSR_DR = 0x0010 -MSR_PM = 0x0004 -DEFAULT_MSR = (MSR_EE | MSR_ME | MSR_AL | MSR_IR | MSR_DR) -DEFAULT_USER_MSR = (DEFAULT_MSR | MSR_PR) -CR_LT = 0x80000000 -CR_GT = 0x40000000 -CR_EQ = 0x20000000 -CR_SO = 0x10000000 -CR_FX = 0x08000000 -CR_FEX = 0x04000000 -CR_VX = 0x02000000 -CR_OX = 0x01000000 -XER_SO = 0x80000000 -XER_OV = 0x40000000 -XER_CA = 0x20000000 -def XER_COMP_BYTE(xer): return ((xer >> 8) & 0x000000FF) - -def XER_LENGTH(xer): return (xer & 0x0000007F) - -DSISR_IO = 0x80000000 -DSISR_PFT = 0x40000000 -DSISR_LOCK = 0x20000000 -DSISR_FPIO = 0x10000000 -DSISR_PROT = 0x08000000 -DSISR_LOOP = 0x04000000 -DSISR_DRST = 0x04000000 -DSISR_ST = 0x02000000 -DSISR_SEGB = 0x01000000 -DSISR_DABR = 0x00400000 -DSISR_EAR = 0x00100000 -SRR_IS_PFT = 0x40000000 -SRR_IS_ISPEC = 0x20000000 -SRR_IS_IIO = 0x10000000 -SRR_IS_GUARD = 0x10000000 -SRR_IS_PROT = 0x08000000 -SRR_IS_LOOP = 0x04000000 -SRR_PR_FPEN = 0x00100000 -SRR_PR_INVAL = 0x00080000 -SRR_PR_PRIV = 0x00040000 -SRR_PR_TRAP = 0x00020000 -SRR_PR_IMPRE = 0x00010000 -def BUID_7F_SRVAL(raddr): return (0x87F00000 | (((uint)(raddr)) >> 28)) - -BT_256M = 0x1FFC -BT_128M = 0x0FFC -BT_64M = 0x07FC -BT_32M = 0x03FC -BT_16M = 0x01FC -BT_8M = 0x00FC -BT_4M = 0x007C -BT_2M = 0x003C -BT_1M = 0x001C -BT_512K = 0x000C -BT_256K = 0x0004 -BT_128K = 0x0000 -BT_NOACCESS = 0x0 -BT_RDONLY = 0x1 -BT_WRITE = 0x2 -BT_VS = 0x2 -BT_VP = 0x1 -def BAT_ESEG(dbatu): return (((uint)(dbatu) >> 28)) - -MIN_BAT_SIZE = 0x00020000 -MAX_BAT_SIZE = 0x10000000 -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_TCP = 6 -IPPROTO_EGP = 8 -IPPROTO_PUP = 12 -IPPROTO_UDP = 17 -IPPROTO_IDP = 22 -IPPROTO_TP = 29 -IPPROTO_LOCAL = 63 -IPPROTO_EON = 80 -IPPROTO_BIP = 0x53 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -IPPORT_TIMESERVER = 37 -def IN_CLASSA(i): return (((int)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((int)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((int)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((int)(i) & 0xf0000000) == 0xe0000000) - -def IN_MULTICAST(i): return IN_CLASSD(i) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -INADDR_UNSPEC_GROUP = 0xe0000000 -INADDR_ALLHOSTS_GROUP = 0xe0000001 -INADDR_MAX_LOCAL_GROUP = 0xe00000ff -def IN_EXPERIMENTAL(i): return (((int)(i) & 0xe0000000) == 0xe0000000) - -def IN_BADCLASS(i): return (((int)(i) & 0xf0000000) == 0xf0000000) - -INADDR_ANY = 0x00000000 -INADDR_BROADCAST = 0xffffffff -INADDR_LOOPBACK = 0x7f000001 -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 diff --git a/Lib/plat-aix4/regen b/Lib/plat-aix4/regen deleted file mode 100755 --- a/Lib/plat-aix4/regen +++ /dev/null @@ -1,8 +0,0 @@ -#! /bin/sh -case `uname -sv` in -'AIX 4'*) ;; -*) echo Probably not on an AIX 4 system 1>&2 - exit 1;; -esac -set -v -h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-darwin/IN.py b/Lib/plat-darwin/IN.py deleted file mode 100644 --- a/Lib/plat-darwin/IN.py +++ /dev/null @@ -1,662 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/appleapiopts.h - -# Included from sys/_types.h - -# Included from sys/cdefs.h -def __P(protos): return protos - -def __STRING(x): return #x - -def __P(protos): return () - -def __STRING(x): return "x" - -def __attribute__(x): return - -def __COPYRIGHT(s): return __IDSTRING(copyright,s) - -def __RCSID(s): return __IDSTRING(rcsid,s) - -def __SCCSID(s): return __IDSTRING(sccsid,s) - -def __PROJECT_VERSION(s): return __IDSTRING(project_version,s) - -__DARWIN_UNIX03 = 1 -__DARWIN_UNIX03 = 0 -__DARWIN_UNIX03 = 0 -__DARWIN_UNIX03 = 1 -__DARWIN_64_BIT_INO_T = 1 -__DARWIN_64_BIT_INO_T = 0 -__DARWIN_64_BIT_INO_T = 0 -__DARWIN_NON_CANCELABLE = 0 -__DARWIN_VERS_1050 = 1 -__DARWIN_VERS_1050 = 0 -__DARWIN_SUF_UNIX03 = "$UNIX2003" -__DARWIN_SUF_UNIX03_SET = 1 -__DARWIN_SUF_UNIX03_SET = 0 -__DARWIN_SUF_64_BIT_INO_T = "$INODE64" -__DARWIN_SUF_NON_CANCELABLE = "$NOCANCEL" -__DARWIN_SUF_1050 = "$1050" -__DARWIN_SUF_UNIX03_SET = 0 -__DARWIN_SUF_EXTSN = "$DARWIN_EXTSN" -__DARWIN_LONG_DOUBLE_IS_DOUBLE = 0 -def __DARWIN_LDBL_COMPAT(x): return - -def __DARWIN_LDBL_COMPAT2(x): return - -__DARWIN_LONG_DOUBLE_IS_DOUBLE = 1 -def __DARWIN_LDBL_COMPAT(x): return - -def __DARWIN_LDBL_COMPAT2(x): return - -__DARWIN_LONG_DOUBLE_IS_DOUBLE = 0 -_DARWIN_FEATURE_LONG_DOUBLE_IS_DOUBLE = 1 -_DARWIN_FEATURE_UNIX_CONFORMANCE = 3 -_DARWIN_FEATURE_64_BIT_INODE = 1 - -# Included from machine/_types.h -__PTHREAD_SIZE__ = 1168 -__PTHREAD_ATTR_SIZE__ = 56 -__PTHREAD_MUTEXATTR_SIZE__ = 8 -__PTHREAD_MUTEX_SIZE__ = 56 -__PTHREAD_CONDATTR_SIZE__ = 8 -__PTHREAD_COND_SIZE__ = 40 -__PTHREAD_ONCE_SIZE__ = 8 -__PTHREAD_RWLOCK_SIZE__ = 192 -__PTHREAD_RWLOCKATTR_SIZE__ = 16 -__PTHREAD_SIZE__ = 596 -__PTHREAD_ATTR_SIZE__ = 36 -__PTHREAD_MUTEXATTR_SIZE__ = 8 -__PTHREAD_MUTEX_SIZE__ = 40 -__PTHREAD_CONDATTR_SIZE__ = 4 -__PTHREAD_COND_SIZE__ = 24 -__PTHREAD_ONCE_SIZE__ = 4 -__PTHREAD_RWLOCK_SIZE__ = 124 -__PTHREAD_RWLOCKATTR_SIZE__ = 12 -__DARWIN_NULL = 0 - -# Included from stdint.h -__WORDSIZE = 64 -__WORDSIZE = 32 -INT8_MAX = 127 -INT16_MAX = 32767 -INT32_MAX = 2147483647 -INT8_MIN = -128 -INT16_MIN = -32768 -INT32_MIN = (-INT32_MAX-1) -UINT8_MAX = 255 -UINT16_MAX = 65535 -INT_LEAST8_MIN = INT8_MIN -INT_LEAST16_MIN = INT16_MIN -INT_LEAST32_MIN = INT32_MIN -INT_LEAST8_MAX = INT8_MAX -INT_LEAST16_MAX = INT16_MAX -INT_LEAST32_MAX = INT32_MAX -UINT_LEAST8_MAX = UINT8_MAX -UINT_LEAST16_MAX = UINT16_MAX -INT_FAST8_MIN = INT8_MIN -INT_FAST16_MIN = INT16_MIN -INT_FAST32_MIN = INT32_MIN -INT_FAST8_MAX = INT8_MAX -INT_FAST16_MAX = INT16_MAX -INT_FAST32_MAX = INT32_MAX -UINT_FAST8_MAX = UINT8_MAX -UINT_FAST16_MAX = UINT16_MAX -INTPTR_MIN = INT32_MIN -INTPTR_MAX = INT32_MAX -PTRDIFF_MIN = INT32_MIN -PTRDIFF_MAX = INT32_MAX -WCHAR_MAX = 0x7fffffff -WCHAR_MIN = 0 -WCHAR_MIN = (-WCHAR_MAX-1) -WINT_MIN = INT32_MIN -WINT_MAX = INT32_MAX -SIG_ATOMIC_MIN = INT32_MIN -SIG_ATOMIC_MAX = INT32_MAX -def INT8_C(v): return (v) - -def INT16_C(v): return (v) - -def INT32_C(v): return (v) - - -# Included from sys/socket.h - -# Included from machine/_param.h -SOCK_STREAM = 1 -SOCK_DGRAM = 2 -SOCK_RAW = 3 -SOCK_RDM = 4 -SOCK_SEQPACKET = 5 -SO_DEBUG = 0x0001 -SO_ACCEPTCONN = 0x0002 -SO_REUSEADDR = 0x0004 -SO_KEEPALIVE = 0x0008 -SO_DONTROUTE = 0x0010 -SO_BROADCAST = 0x0020 -SO_USELOOPBACK = 0x0040 -SO_LINGER = 0x0080 -SO_LINGER = 0x1080 -SO_OOBINLINE = 0x0100 -SO_REUSEPORT = 0x0200 -SO_TIMESTAMP = 0x0400 -SO_ACCEPTFILTER = 0x1000 -SO_DONTTRUNC = 0x2000 -SO_WANTMORE = 0x4000 -SO_WANTOOBFLAG = 0x8000 -SO_SNDBUF = 0x1001 -SO_RCVBUF = 0x1002 -SO_SNDLOWAT = 0x1003 -SO_RCVLOWAT = 0x1004 -SO_SNDTIMEO = 0x1005 -SO_RCVTIMEO = 0x1006 -SO_ERROR = 0x1007 -SO_TYPE = 0x1008 -SO_NREAD = 0x1020 -SO_NKE = 0x1021 -SO_NOSIGPIPE = 0x1022 -SO_NOADDRERR = 0x1023 -SO_NWRITE = 0x1024 -SO_REUSESHAREUID = 0x1025 -SO_NOTIFYCONFLICT = 0x1026 -SO_LINGER_SEC = 0x1080 -SO_RESTRICTIONS = 0x1081 -SO_RESTRICT_DENYIN = 0x00000001 -SO_RESTRICT_DENYOUT = 0x00000002 -SO_RESTRICT_DENYSET = (-2147483648) -SO_LABEL = 0x1010 -SO_PEERLABEL = 0x1011 -SOL_SOCKET = 0xffff -AF_UNSPEC = 0 -AF_UNIX = 1 -AF_LOCAL = AF_UNIX -AF_INET = 2 -AF_IMPLINK = 3 -AF_PUP = 4 -AF_CHAOS = 5 -AF_NS = 6 -AF_ISO = 7 -AF_OSI = AF_ISO -AF_ECMA = 8 -AF_DATAKIT = 9 -AF_CCITT = 10 -AF_SNA = 11 -AF_DECnet = 12 -AF_DLI = 13 -AF_LAT = 14 -AF_HYLINK = 15 -AF_APPLETALK = 16 -AF_ROUTE = 17 -AF_LINK = 18 -pseudo_AF_XTP = 19 -AF_COIP = 20 -AF_CNT = 21 -pseudo_AF_RTIP = 22 -AF_IPX = 23 -AF_SIP = 24 -pseudo_AF_PIP = 25 -AF_NDRV = 27 -AF_ISDN = 28 -AF_E164 = AF_ISDN -pseudo_AF_KEY = 29 -AF_INET6 = 30 -AF_NATM = 31 -AF_SYSTEM = 32 -AF_NETBIOS = 33 -AF_PPP = 34 -AF_ATM = 30 -pseudo_AF_HDRCMPLT = 35 -AF_RESERVED_36 = 36 -AF_NETGRAPH = 32 -AF_MAX = 37 -SOCK_MAXADDRLEN = 255 -_SS_MAXSIZE = 128 -PF_UNSPEC = AF_UNSPEC -PF_LOCAL = AF_LOCAL -PF_UNIX = PF_LOCAL -PF_INET = AF_INET -PF_IMPLINK = AF_IMPLINK -PF_PUP = AF_PUP -PF_CHAOS = AF_CHAOS -PF_NS = AF_NS -PF_ISO = AF_ISO -PF_OSI = AF_ISO -PF_ECMA = AF_ECMA -PF_DATAKIT = AF_DATAKIT -PF_CCITT = AF_CCITT -PF_SNA = AF_SNA -PF_DECnet = AF_DECnet -PF_DLI = AF_DLI -PF_LAT = AF_LAT -PF_HYLINK = AF_HYLINK -PF_APPLETALK = AF_APPLETALK -PF_ROUTE = AF_ROUTE -PF_LINK = AF_LINK -PF_XTP = pseudo_AF_XTP -PF_COIP = AF_COIP -PF_CNT = AF_CNT -PF_SIP = AF_SIP -PF_IPX = AF_IPX -PF_RTIP = pseudo_AF_RTIP -PF_PIP = pseudo_AF_PIP -PF_NDRV = AF_NDRV -PF_ISDN = AF_ISDN -PF_KEY = pseudo_AF_KEY -PF_INET6 = AF_INET6 -PF_NATM = AF_NATM -PF_SYSTEM = AF_SYSTEM -PF_NETBIOS = AF_NETBIOS -PF_PPP = AF_PPP -PF_RESERVED_36 = AF_RESERVED_36 -PF_ATM = AF_ATM -PF_NETGRAPH = AF_NETGRAPH -PF_MAX = AF_MAX -NET_MAXID = AF_MAX -NET_RT_DUMP = 1 -NET_RT_FLAGS = 2 -NET_RT_IFLIST = 3 -NET_RT_STAT = 4 -NET_RT_TRASH = 5 -NET_RT_IFLIST2 = 6 -NET_RT_DUMP2 = 7 -NET_RT_MAXID = 8 -SOMAXCONN = 128 -MSG_OOB = 0x1 -MSG_PEEK = 0x2 -MSG_DONTROUTE = 0x4 -MSG_EOR = 0x8 -MSG_TRUNC = 0x10 -MSG_CTRUNC = 0x20 -MSG_WAITALL = 0x40 -MSG_DONTWAIT = 0x80 -MSG_EOF = 0x100 -MSG_WAITSTREAM = 0x200 -MSG_FLUSH = 0x400 -MSG_HOLD = 0x800 -MSG_SEND = 0x1000 -MSG_HAVEMORE = 0x2000 -MSG_RCVMORE = 0x4000 -MSG_NEEDSA = 0x10000 -CMGROUP_MAX = 16 -SCM_RIGHTS = 0x01 -SCM_TIMESTAMP = 0x02 -SCM_CREDS = 0x03 -SHUT_RD = 0 -SHUT_WR = 1 -SHUT_RDWR = 2 - -# Included from machine/endian.h - -# Included from sys/_endian.h -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -def NTOHL(x): return (x) - -def NTOHS(x): return (x) - -def HTONL(x): return (x) - -def HTONS(x): return (x) - - -# Included from libkern/_OSByteOrder.h -def __DARWIN_OSSwapConstInt16(x): return \ - -def __DARWIN_OSSwapConstInt32(x): return \ - -def __DARWIN_OSSwapConstInt64(x): return \ - - -# Included from libkern/i386/_OSByteOrder.h -def __DARWIN_OSSwapInt16(x): return \ - -def __DARWIN_OSSwapInt32(x): return \ - -def __DARWIN_OSSwapInt64(x): return \ - -def __DARWIN_OSSwapInt16(x): return _OSSwapInt16(x) - -def __DARWIN_OSSwapInt32(x): return _OSSwapInt32(x) - -def __DARWIN_OSSwapInt64(x): return _OSSwapInt64(x) - -def ntohs(x): return __DARWIN_OSSwapInt16(x) - -def htons(x): return __DARWIN_OSSwapInt16(x) - -def ntohl(x): return __DARWIN_OSSwapInt32(x) - -def htonl(x): return __DARWIN_OSSwapInt32(x) - -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_TCP = 6 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_UDP = 17 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_PIM = 103 -IPPROTO_PGM = 113 -IPPROTO_DIVERT = 254 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -__DARWIN_IPPORT_RESERVED = 1024 -IPPORT_RESERVED = __DARWIN_IPPORT_RESERVED -IPPORT_USERRESERVED = 5000 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -def IN_CLASSA(i): return (((u_int32_t)(i) & (-2147483648)) == 0) - -IN_CLASSA_NET = (-16777216) -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & (-1073741824)) == (-2147483648)) - -IN_CLASSB_NET = (-65536) -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & (-536870912)) == (-1073741824)) - -IN_CLASSC_NET = (-256) -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & (-268435456)) == (-536870912)) - -IN_CLASSD_NET = (-268435456) -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -INADDR_NONE = (-1) -def IN_LINKLOCAL(i): return (((u_int32_t)(i) & IN_CLASSB_NET) == IN_LINKLOCALNETNUM) - -IN_LOOPBACKNET = 127 -INET_ADDRSTRLEN = 16 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_STRIPHDR = 23 -IP_RECVTTL = 24 -IP_FW_ADD = 40 -IP_FW_DEL = 41 -IP_FW_FLUSH = 42 -IP_FW_ZERO = 43 -IP_FW_GET = 44 -IP_FW_RESETLOG = 45 -IP_OLD_FW_ADD = 50 -IP_OLD_FW_DEL = 51 -IP_OLD_FW_FLUSH = 52 -IP_OLD_FW_ZERO = 53 -IP_OLD_FW_GET = 54 -IP_NAT__XXX = 55 -IP_OLD_FW_RESETLOG = 56 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_TRAFFIC_MGT_BACKGROUND = 65 -IP_FORCE_OUT_IFP = 69 -TRAFFIC_MGT_SO_BACKGROUND = 0x0001 -TRAFFIC_MGT_SO_BG_SUPPRESSED = 0x0002 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 - -# Included from netinet6/in6.h -__KAME_VERSION = "20010528/apple-darwin" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_PKTINFO = 19 -IPV6_HOPLIMIT = 20 -IPV6_NEXTHOP = 21 -IPV6_HOPOPTS = 22 -IPV6_DSTOPTS = 23 -IPV6_RTHDR = 24 -IPV6_PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_V6ONLY = 27 -IPV6_BINDV6ONLY = IPV6_V6ONLY -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_V6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_USETEMPADDR = 32 -IPV6CTL_TEMPPLTIME = 33 -IPV6CTL_TEMPVLTIME = 34 -IPV6CTL_AUTO_LINKLOCAL = 35 -IPV6CTL_RIP6STATS = 36 -IPV6CTL_MAXFRAGS = 41 -IPV6CTL_MAXID = 42 diff --git a/Lib/plat-darwin/regen b/Lib/plat-darwin/regen deleted file mode 100755 --- a/Lib/plat-darwin/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python$EXE ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd4/IN.py b/Lib/plat-freebsd4/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd4/IN.py +++ /dev/null @@ -1,355 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_TCP = 6 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_UDP = 17 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_PIM = 103 -IPPROTO_PGM = 113 -IPPROTO_DIVERT = 254 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -def IN_CLASSA(i): return (((u_int32_t)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & 0xf0000000) == 0xe0000000) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -INET_ADDRSTRLEN = 16 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 - -# Included from netinet6/in6.h - -# Included from sys/queue.h -def SLIST_HEAD_INITIALIZER(head): return \ - -def SLIST_ENTRY(type): return \ - -def STAILQ_HEAD_INITIALIZER(head): return \ - -def STAILQ_ENTRY(type): return \ - -def LIST_HEAD_INITIALIZER(head): return \ - -def LIST_ENTRY(type): return \ - -def TAILQ_HEAD_INITIALIZER(head): return \ - -def TAILQ_ENTRY(type): return \ - -def CIRCLEQ_ENTRY(type): return \ - -__KAME_VERSION = "20000701/FreeBSD-current" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = 0xff010000 -IPV6_ADDR_INT32_MLL = 0xff020000 -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = 0xffff0000 -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_PKTINFO = 19 -IPV6_HOPLIMIT = 20 -IPV6_NEXTHOP = 21 -IPV6_HOPOPTS = 22 -IPV6_DSTOPTS = 23 -IPV6_RTHDR = 24 -IPV6_PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_BINDV6ONLY = 27 -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_BINDV6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_MAXID = 28 diff --git a/Lib/plat-freebsd4/regen b/Lib/plat-freebsd4/regen deleted file mode 100755 --- a/Lib/plat-freebsd4/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd5/IN.py b/Lib/plat-freebsd5/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd5/IN.py +++ /dev/null @@ -1,355 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_TCP = 6 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_UDP = 17 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_PIM = 103 -IPPROTO_PGM = 113 -IPPROTO_DIVERT = 254 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -def IN_CLASSA(i): return (((u_int32_t)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & 0xf0000000) == 0xe0000000) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -INET_ADDRSTRLEN = 16 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 - -# Included from netinet6/in6.h - -# Included from sys/queue.h -def SLIST_HEAD_INITIALIZER(head): return \ - -def SLIST_ENTRY(type): return \ - -def STAILQ_HEAD_INITIALIZER(head): return \ - -def STAILQ_ENTRY(type): return \ - -def LIST_HEAD_INITIALIZER(head): return \ - -def LIST_ENTRY(type): return \ - -def TAILQ_HEAD_INITIALIZER(head): return \ - -def TAILQ_ENTRY(type): return \ - -def CIRCLEQ_ENTRY(type): return \ - -__KAME_VERSION = "20000701/FreeBSD-current" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = 0xff010000 -IPV6_ADDR_INT32_MLL = 0xff020000 -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = 0xffff0000 -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_PKTINFO = 19 -IPV6_HOPLIMIT = 20 -IPV6_NEXTHOP = 21 -IPV6_HOPOPTS = 22 -IPV6_DSTOPTS = 23 -IPV6_RTHDR = 24 -IPV6_PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_BINDV6ONLY = 27 -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_BINDV6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_MAXID = 28 diff --git a/Lib/plat-freebsd5/regen b/Lib/plat-freebsd5/regen deleted file mode 100755 --- a/Lib/plat-freebsd5/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd6/IN.py b/Lib/plat-freebsd6/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd6/IN.py +++ /dev/null @@ -1,551 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/cdefs.h -__GNUCLIKE_ASM = 3 -__GNUCLIKE_ASM = 2 -__GNUCLIKE___TYPEOF = 1 -__GNUCLIKE___OFFSETOF = 1 -__GNUCLIKE___SECTION = 1 -__GNUCLIKE_ATTRIBUTE_MODE_DI = 1 -__GNUCLIKE_CTOR_SECTION_HANDLING = 1 -__GNUCLIKE_BUILTIN_CONSTANT_P = 1 -__GNUCLIKE_BUILTIN_VARARGS = 1 -__GNUCLIKE_BUILTIN_STDARG = 1 -__GNUCLIKE_BUILTIN_VAALIST = 1 -__GNUC_VA_LIST_COMPATIBILITY = 1 -__GNUCLIKE_BUILTIN_NEXT_ARG = 1 -__GNUCLIKE_BUILTIN_MEMCPY = 1 -__CC_SUPPORTS_INLINE = 1 -__CC_SUPPORTS___INLINE = 1 -__CC_SUPPORTS___INLINE__ = 1 -__CC_SUPPORTS___FUNC__ = 1 -__CC_SUPPORTS_WARNING = 1 -__CC_SUPPORTS_VARADIC_XXX = 1 -__CC_SUPPORTS_DYNAMIC_ARRAY_INIT = 1 -__CC_INT_IS_32BIT = 1 -def __P(protos): return protos - -def __STRING(x): return #x - -def __XSTRING(x): return __STRING(x) - -def __P(protos): return () - -def __STRING(x): return "x" - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __nonnull(x): return __attribute__((__nonnull__(x))) - -def __predict_true(exp): return __builtin_expect((exp), 1) - -def __predict_false(exp): return __builtin_expect((exp), 0) - -def __predict_true(exp): return (exp) - -def __predict_false(exp): return (exp) - -def __format_arg(fmtarg): return __attribute__((__format_arg__ (fmtarg))) - -def __FBSDID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID_SOURCE(s): return __IDSTRING(__CONCAT(__rcsid_source_,__LINE__),s) - -def __SCCSID(s): return __IDSTRING(__CONCAT(__sccsid_,__LINE__),s) - -def __COPYRIGHT(s): return __IDSTRING(__CONCAT(__copyright_,__LINE__),s) - -_POSIX_C_SOURCE = 199009 -_POSIX_C_SOURCE = 199209 -__XSI_VISIBLE = 600 -_POSIX_C_SOURCE = 200112 -__XSI_VISIBLE = 500 -_POSIX_C_SOURCE = 199506 -_POSIX_C_SOURCE = 198808 -__POSIX_VISIBLE = 200112 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 199506 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199309 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199209 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199009 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 198808 -__ISO_C_VISIBLE = 0 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 200112 -__XSI_VISIBLE = 600 -__BSD_VISIBLE = 1 -__ISO_C_VISIBLE = 1999 - -# Included from sys/_types.h - -# Included from machine/_types.h - -# Included from machine/endian.h -_QUAD_HIGHWORD = 1 -_QUAD_LOWWORD = 0 -_LITTLE_ENDIAN = 1234 -_BIG_ENDIAN = 4321 -_PDP_ENDIAN = 3412 -_BYTE_ORDER = _LITTLE_ENDIAN -LITTLE_ENDIAN = _LITTLE_ENDIAN -BIG_ENDIAN = _BIG_ENDIAN -PDP_ENDIAN = _PDP_ENDIAN -BYTE_ORDER = _BYTE_ORDER -def __word_swap_int_var(x): return \ - -def __word_swap_int_const(x): return \ - -def __word_swap_int(x): return __word_swap_int_var(x) - -def __byte_swap_int_var(x): return \ - -def __byte_swap_int_const(x): return \ - -def __byte_swap_int(x): return __byte_swap_int_var(x) - -def __byte_swap_long_var(x): return \ - -def __byte_swap_long_const(x): return \ - -def __byte_swap_long(x): return __byte_swap_long_var(x) - -def __byte_swap_word_var(x): return \ - -def __byte_swap_word_const(x): return \ - -def __byte_swap_word(x): return __byte_swap_word_var(x) - -def __htonl(x): return __bswap32(x) - -def __htons(x): return __bswap16(x) - -def __ntohl(x): return __bswap32(x) - -def __ntohs(x): return __bswap16(x) - -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_TCP = 6 -IPPROTO_UDP = 17 -def htonl(x): return __htonl(x) - -def htons(x): return __htons(x) - -def ntohl(x): return __ntohl(x) - -def ntohs(x): return __ntohs(x) - -IPPROTO_RAW = 255 -INET_ADDRSTRLEN = 16 -IPPROTO_HOPOPTS = 0 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_MOBILE = 55 -IPPROTO_TLSP = 56 -IPPROTO_SKIP = 57 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_SCTP = 132 -IPPROTO_PIM = 103 -IPPROTO_CARP = 112 -IPPROTO_PGM = 113 -IPPROTO_PFSYNC = 240 -IPPROTO_OLD_DIVERT = 254 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPROTO_DIVERT = 258 -IPPROTO_SPACER = 32767 -IPPORT_RESERVED = 1024 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -IPPORT_MAX = 65535 -def IN_CLASSA(i): return (((u_int32_t)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & 0xf0000000) == 0xe0000000) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_SENDSRCADDR = IP_RECVDSTADDR -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_ONESBCAST = 23 -IP_FW_TABLE_ADD = 40 -IP_FW_TABLE_DEL = 41 -IP_FW_TABLE_FLUSH = 42 -IP_FW_TABLE_GETSIZE = 43 -IP_FW_TABLE_LIST = 44 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_RECVTTL = 65 -IP_MINTTL = 66 -IP_DONTFRAG = 67 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 -def in_nullhost(x): return ((x).s_addr == INADDR_ANY) - - -# Included from netinet6/in6.h -__KAME_VERSION = "FreeBSD" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = 0xff010000 -IPV6_ADDR_INT32_MLL = 0xff020000 -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = 0xffff0000 -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_INTFACELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -def IFA6_IS_DEPRECATED(a): return \ - -def IFA6_IS_INVALID(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_2292PKTINFO = 19 -IPV6_2292HOPLIMIT = 20 -IPV6_2292NEXTHOP = 21 -IPV6_2292HOPOPTS = 22 -IPV6_2292DSTOPTS = 23 -IPV6_2292RTHDR = 24 -IPV6_2292PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_V6ONLY = 27 -IPV6_BINDV6ONLY = IPV6_V6ONLY -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDRDSTOPTS = 35 -IPV6_RECVPKTINFO = 36 -IPV6_RECVHOPLIMIT = 37 -IPV6_RECVRTHDR = 38 -IPV6_RECVHOPOPTS = 39 -IPV6_RECVDSTOPTS = 40 -IPV6_RECVRTHDRDSTOPTS = 41 -IPV6_USE_MIN_MTU = 42 -IPV6_RECVPATHMTU = 43 -IPV6_PATHMTU = 44 -IPV6_REACHCONF = 45 -IPV6_PKTINFO = 46 -IPV6_HOPLIMIT = 47 -IPV6_NEXTHOP = 48 -IPV6_HOPOPTS = 49 -IPV6_DSTOPTS = 50 -IPV6_RTHDR = 51 -IPV6_PKTOPTIONS = 52 -IPV6_RECVTCLASS = 57 -IPV6_AUTOFLOWLABEL = 59 -IPV6_TCLASS = 61 -IPV6_DONTFRAG = 62 -IPV6_PREFER_TEMPADDR = 63 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_V6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_USETEMPADDR = 32 -IPV6CTL_TEMPPLTIME = 33 -IPV6CTL_TEMPVLTIME = 34 -IPV6CTL_AUTO_LINKLOCAL = 35 -IPV6CTL_RIP6STATS = 36 -IPV6CTL_PREFER_TEMPADDR = 37 -IPV6CTL_ADDRCTLPOLICY = 38 -IPV6CTL_USE_DEFAULTZONE = 39 -IPV6CTL_MAXFRAGS = 41 -IPV6CTL_IFQ = 42 -IPV6CTL_ISATAPRTR = 43 -IPV6CTL_MCAST_PMTU = 44 -IPV6CTL_STEALTH = 45 -IPV6CTL_MAXID = 46 diff --git a/Lib/plat-freebsd6/regen b/Lib/plat-freebsd6/regen deleted file mode 100755 --- a/Lib/plat-freebsd6/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd7/IN.py b/Lib/plat-freebsd7/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd7/IN.py +++ /dev/null @@ -1,571 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/cdefs.h -__GNUCLIKE_ASM = 3 -__GNUCLIKE_ASM = 2 -__GNUCLIKE___TYPEOF = 1 -__GNUCLIKE___OFFSETOF = 1 -__GNUCLIKE___SECTION = 1 -__GNUCLIKE_ATTRIBUTE_MODE_DI = 1 -__GNUCLIKE_CTOR_SECTION_HANDLING = 1 -__GNUCLIKE_BUILTIN_CONSTANT_P = 1 -__GNUCLIKE_BUILTIN_VARARGS = 1 -__GNUCLIKE_BUILTIN_STDARG = 1 -__GNUCLIKE_BUILTIN_VAALIST = 1 -__GNUC_VA_LIST_COMPATIBILITY = 1 -__GNUCLIKE_BUILTIN_NEXT_ARG = 1 -__GNUCLIKE_BUILTIN_MEMCPY = 1 -__CC_SUPPORTS_INLINE = 1 -__CC_SUPPORTS___INLINE = 1 -__CC_SUPPORTS___INLINE__ = 1 -__CC_SUPPORTS___FUNC__ = 1 -__CC_SUPPORTS_WARNING = 1 -__CC_SUPPORTS_VARADIC_XXX = 1 -__CC_SUPPORTS_DYNAMIC_ARRAY_INIT = 1 -__CC_INT_IS_32BIT = 1 -def __P(protos): return protos - -def __STRING(x): return #x - -def __XSTRING(x): return __STRING(x) - -def __P(protos): return () - -def __STRING(x): return "x" - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __nonnull(x): return __attribute__((__nonnull__(x))) - -def __predict_true(exp): return __builtin_expect((exp), 1) - -def __predict_false(exp): return __builtin_expect((exp), 0) - -def __predict_true(exp): return (exp) - -def __predict_false(exp): return (exp) - -def __format_arg(fmtarg): return __attribute__((__format_arg__ (fmtarg))) - -def __FBSDID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID_SOURCE(s): return __IDSTRING(__CONCAT(__rcsid_source_,__LINE__),s) - -def __SCCSID(s): return __IDSTRING(__CONCAT(__sccsid_,__LINE__),s) - -def __COPYRIGHT(s): return __IDSTRING(__CONCAT(__copyright_,__LINE__),s) - -_POSIX_C_SOURCE = 199009 -_POSIX_C_SOURCE = 199209 -__XSI_VISIBLE = 600 -_POSIX_C_SOURCE = 200112 -__XSI_VISIBLE = 500 -_POSIX_C_SOURCE = 199506 -_POSIX_C_SOURCE = 198808 -__POSIX_VISIBLE = 200112 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 199506 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199309 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199209 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199009 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 198808 -__ISO_C_VISIBLE = 0 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 200112 -__XSI_VISIBLE = 600 -__BSD_VISIBLE = 1 -__ISO_C_VISIBLE = 1999 - -# Included from sys/_types.h - -# Included from machine/_types.h - -# Included from machine/endian.h -_QUAD_HIGHWORD = 1 -_QUAD_LOWWORD = 0 -_LITTLE_ENDIAN = 1234 -_BIG_ENDIAN = 4321 -_PDP_ENDIAN = 3412 -_BYTE_ORDER = _LITTLE_ENDIAN -LITTLE_ENDIAN = _LITTLE_ENDIAN -BIG_ENDIAN = _BIG_ENDIAN -PDP_ENDIAN = _PDP_ENDIAN -BYTE_ORDER = _BYTE_ORDER -def __word_swap_int_var(x): return \ - -def __word_swap_int_const(x): return \ - -def __word_swap_int(x): return __word_swap_int_var(x) - -def __byte_swap_int_var(x): return \ - -def __byte_swap_int_const(x): return \ - -def __byte_swap_int(x): return __byte_swap_int_var(x) - -def __byte_swap_word_var(x): return \ - -def __byte_swap_word_const(x): return \ - -def __byte_swap_word(x): return __byte_swap_word_var(x) - -def __htonl(x): return __bswap32(x) - -def __htons(x): return __bswap16(x) - -def __ntohl(x): return __bswap32(x) - -def __ntohs(x): return __bswap16(x) - -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_TCP = 6 -IPPROTO_UDP = 17 -def htonl(x): return __htonl(x) - -def htons(x): return __htons(x) - -def ntohl(x): return __ntohl(x) - -def ntohs(x): return __ntohs(x) - -IPPROTO_RAW = 255 -INET_ADDRSTRLEN = 16 -IPPROTO_HOPOPTS = 0 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_MOBILE = 55 -IPPROTO_TLSP = 56 -IPPROTO_SKIP = 57 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_SCTP = 132 -IPPROTO_PIM = 103 -IPPROTO_CARP = 112 -IPPROTO_PGM = 113 -IPPROTO_PFSYNC = 240 -IPPROTO_OLD_DIVERT = 254 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPROTO_DIVERT = 258 -IPPROTO_SPACER = 32767 -IPPORT_RESERVED = 1024 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -IPPORT_MAX = 65535 -def IN_CLASSA(i): return (((u_int32_t)(i) & (-2147483648)) == 0) - -IN_CLASSA_NET = (-16777216) -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & (-1073741824)) == (-2147483648)) - -IN_CLASSB_NET = (-65536) -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & (-536870912)) == (-1073741824)) - -IN_CLASSC_NET = (-256) -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & (-268435456)) == (-536870912)) - -IN_CLASSD_NET = (-268435456) -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_LINKLOCAL(i): return (((u_int32_t)(i) & (-65536)) == (-1442971648)) - -def IN_LOCAL_GROUP(i): return (((u_int32_t)(i) & (-256)) == (-536870912)) - -INADDR_NONE = (-1) -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_SENDSRCADDR = IP_RECVDSTADDR -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_ONESBCAST = 23 -IP_FW_TABLE_ADD = 40 -IP_FW_TABLE_DEL = 41 -IP_FW_TABLE_FLUSH = 42 -IP_FW_TABLE_GETSIZE = 43 -IP_FW_TABLE_LIST = 44 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_FW_NAT_CFG = 56 -IP_FW_NAT_DEL = 57 -IP_FW_NAT_GET_CONFIG = 58 -IP_FW_NAT_GET_LOG = 59 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_RECVTTL = 65 -IP_MINTTL = 66 -IP_DONTFRAG = 67 -IP_ADD_SOURCE_MEMBERSHIP = 70 -IP_DROP_SOURCE_MEMBERSHIP = 71 -IP_BLOCK_SOURCE = 72 -IP_UNBLOCK_SOURCE = 73 -IP_MSFILTER = 74 -MCAST_JOIN_GROUP = 80 -MCAST_LEAVE_GROUP = 81 -MCAST_JOIN_SOURCE_GROUP = 82 -MCAST_LEAVE_SOURCE_GROUP = 83 -MCAST_BLOCK_SOURCE = 84 -MCAST_UNBLOCK_SOURCE = 85 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MIN_MEMBERSHIPS = 31 -IP_MAX_MEMBERSHIPS = 4095 -IP_MAX_SOURCE_FILTER = 1024 -MCAST_INCLUDE = 1 -MCAST_EXCLUDE = 2 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 -def in_nullhost(x): return ((x).s_addr == INADDR_ANY) - - -# Included from netinet6/in6.h -__KAME_VERSION = "FreeBSD" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = (-16711680) -IPV6_ADDR_INT32_MLL = (-16646144) -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = (-65536) -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_INTFACELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -def IN6_IS_SCOPE_EMBED(a): return \ - -def IFA6_IS_DEPRECATED(a): return \ - -def IFA6_IS_INVALID(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_2292PKTINFO = 19 -IPV6_2292HOPLIMIT = 20 -IPV6_2292NEXTHOP = 21 -IPV6_2292HOPOPTS = 22 -IPV6_2292DSTOPTS = 23 -IPV6_2292RTHDR = 24 -IPV6_2292PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_V6ONLY = 27 -IPV6_BINDV6ONLY = IPV6_V6ONLY -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDRDSTOPTS = 35 -IPV6_RECVPKTINFO = 36 -IPV6_RECVHOPLIMIT = 37 -IPV6_RECVRTHDR = 38 -IPV6_RECVHOPOPTS = 39 -IPV6_RECVDSTOPTS = 40 -IPV6_RECVRTHDRDSTOPTS = 41 -IPV6_USE_MIN_MTU = 42 -IPV6_RECVPATHMTU = 43 -IPV6_PATHMTU = 44 -IPV6_REACHCONF = 45 -IPV6_PKTINFO = 46 -IPV6_HOPLIMIT = 47 -IPV6_NEXTHOP = 48 -IPV6_HOPOPTS = 49 -IPV6_DSTOPTS = 50 -IPV6_RTHDR = 51 -IPV6_PKTOPTIONS = 52 -IPV6_RECVTCLASS = 57 -IPV6_AUTOFLOWLABEL = 59 -IPV6_TCLASS = 61 -IPV6_DONTFRAG = 62 -IPV6_PREFER_TEMPADDR = 63 -IPV6_MSFILTER = 74 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_V6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_USETEMPADDR = 32 -IPV6CTL_TEMPPLTIME = 33 -IPV6CTL_TEMPVLTIME = 34 -IPV6CTL_AUTO_LINKLOCAL = 35 -IPV6CTL_RIP6STATS = 36 -IPV6CTL_PREFER_TEMPADDR = 37 -IPV6CTL_ADDRCTLPOLICY = 38 -IPV6CTL_USE_DEFAULTZONE = 39 -IPV6CTL_MAXFRAGS = 41 -IPV6CTL_IFQ = 42 -IPV6CTL_ISATAPRTR = 43 -IPV6CTL_MCAST_PMTU = 44 -IPV6CTL_STEALTH = 45 -IPV6CTL_MAXID = 46 diff --git a/Lib/plat-freebsd7/regen b/Lib/plat-freebsd7/regen deleted file mode 100755 --- a/Lib/plat-freebsd7/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd8/IN.py b/Lib/plat-freebsd8/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd8/IN.py +++ /dev/null @@ -1,571 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/cdefs.h -__GNUCLIKE_ASM = 3 -__GNUCLIKE_ASM = 2 -__GNUCLIKE___TYPEOF = 1 -__GNUCLIKE___OFFSETOF = 1 -__GNUCLIKE___SECTION = 1 -__GNUCLIKE_ATTRIBUTE_MODE_DI = 1 -__GNUCLIKE_CTOR_SECTION_HANDLING = 1 -__GNUCLIKE_BUILTIN_CONSTANT_P = 1 -__GNUCLIKE_BUILTIN_VARARGS = 1 -__GNUCLIKE_BUILTIN_STDARG = 1 -__GNUCLIKE_BUILTIN_VAALIST = 1 -__GNUC_VA_LIST_COMPATIBILITY = 1 -__GNUCLIKE_BUILTIN_NEXT_ARG = 1 -__GNUCLIKE_BUILTIN_MEMCPY = 1 -__CC_SUPPORTS_INLINE = 1 -__CC_SUPPORTS___INLINE = 1 -__CC_SUPPORTS___INLINE__ = 1 -__CC_SUPPORTS___FUNC__ = 1 -__CC_SUPPORTS_WARNING = 1 -__CC_SUPPORTS_VARADIC_XXX = 1 -__CC_SUPPORTS_DYNAMIC_ARRAY_INIT = 1 -__CC_INT_IS_32BIT = 1 -def __P(protos): return protos - -def __STRING(x): return #x - -def __XSTRING(x): return __STRING(x) - -def __P(protos): return () - -def __STRING(x): return "x" - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __nonnull(x): return __attribute__((__nonnull__(x))) - -def __predict_true(exp): return __builtin_expect((exp), 1) - -def __predict_false(exp): return __builtin_expect((exp), 0) - -def __predict_true(exp): return (exp) - -def __predict_false(exp): return (exp) - -def __format_arg(fmtarg): return __attribute__((__format_arg__ (fmtarg))) - -def __FBSDID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID_SOURCE(s): return __IDSTRING(__CONCAT(__rcsid_source_,__LINE__),s) - -def __SCCSID(s): return __IDSTRING(__CONCAT(__sccsid_,__LINE__),s) - -def __COPYRIGHT(s): return __IDSTRING(__CONCAT(__copyright_,__LINE__),s) - -_POSIX_C_SOURCE = 199009 -_POSIX_C_SOURCE = 199209 -__XSI_VISIBLE = 600 -_POSIX_C_SOURCE = 200112 -__XSI_VISIBLE = 500 -_POSIX_C_SOURCE = 199506 -_POSIX_C_SOURCE = 198808 -__POSIX_VISIBLE = 200112 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 199506 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199309 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199209 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199009 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 198808 -__ISO_C_VISIBLE = 0 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 200112 -__XSI_VISIBLE = 600 -__BSD_VISIBLE = 1 -__ISO_C_VISIBLE = 1999 - -# Included from sys/_types.h - -# Included from machine/_types.h - -# Included from machine/endian.h -_QUAD_HIGHWORD = 1 -_QUAD_LOWWORD = 0 -_LITTLE_ENDIAN = 1234 -_BIG_ENDIAN = 4321 -_PDP_ENDIAN = 3412 -_BYTE_ORDER = _LITTLE_ENDIAN -LITTLE_ENDIAN = _LITTLE_ENDIAN -BIG_ENDIAN = _BIG_ENDIAN -PDP_ENDIAN = _PDP_ENDIAN -BYTE_ORDER = _BYTE_ORDER -def __word_swap_int_var(x): return \ - -def __word_swap_int_const(x): return \ - -def __word_swap_int(x): return __word_swap_int_var(x) - -def __byte_swap_int_var(x): return \ - -def __byte_swap_int_const(x): return \ - -def __byte_swap_int(x): return __byte_swap_int_var(x) - -def __byte_swap_word_var(x): return \ - -def __byte_swap_word_const(x): return \ - -def __byte_swap_word(x): return __byte_swap_word_var(x) - -def __htonl(x): return __bswap32(x) - -def __htons(x): return __bswap16(x) - -def __ntohl(x): return __bswap32(x) - -def __ntohs(x): return __bswap16(x) - -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_TCP = 6 -IPPROTO_UDP = 17 -def htonl(x): return __htonl(x) - -def htons(x): return __htons(x) - -def ntohl(x): return __ntohl(x) - -def ntohs(x): return __ntohs(x) - -IPPROTO_RAW = 255 -INET_ADDRSTRLEN = 16 -IPPROTO_HOPOPTS = 0 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_MOBILE = 55 -IPPROTO_TLSP = 56 -IPPROTO_SKIP = 57 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_SCTP = 132 -IPPROTO_PIM = 103 -IPPROTO_CARP = 112 -IPPROTO_PGM = 113 -IPPROTO_PFSYNC = 240 -IPPROTO_OLD_DIVERT = 254 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPROTO_DIVERT = 258 -IPPROTO_SPACER = 32767 -IPPORT_RESERVED = 1024 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -IPPORT_MAX = 65535 -def IN_CLASSA(i): return (((u_int32_t)(i) & (-2147483648)) == 0) - -IN_CLASSA_NET = (-16777216) -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & (-1073741824)) == (-2147483648)) - -IN_CLASSB_NET = (-65536) -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & (-536870912)) == (-1073741824)) - -IN_CLASSC_NET = (-256) -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & (-268435456)) == (-536870912)) - -IN_CLASSD_NET = (-268435456) -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_LINKLOCAL(i): return (((u_int32_t)(i) & (-65536)) == (-1442971648)) - -def IN_LOCAL_GROUP(i): return (((u_int32_t)(i) & (-256)) == (-536870912)) - -INADDR_NONE = (-1) -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_SENDSRCADDR = IP_RECVDSTADDR -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_ONESBCAST = 23 -IP_FW_TABLE_ADD = 40 -IP_FW_TABLE_DEL = 41 -IP_FW_TABLE_FLUSH = 42 -IP_FW_TABLE_GETSIZE = 43 -IP_FW_TABLE_LIST = 44 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_FW_NAT_CFG = 56 -IP_FW_NAT_DEL = 57 -IP_FW_NAT_GET_CONFIG = 58 -IP_FW_NAT_GET_LOG = 59 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_RECVTTL = 65 -IP_MINTTL = 66 -IP_DONTFRAG = 67 -IP_ADD_SOURCE_MEMBERSHIP = 70 -IP_DROP_SOURCE_MEMBERSHIP = 71 -IP_BLOCK_SOURCE = 72 -IP_UNBLOCK_SOURCE = 73 -IP_MSFILTER = 74 -MCAST_JOIN_GROUP = 80 -MCAST_LEAVE_GROUP = 81 -MCAST_JOIN_SOURCE_GROUP = 82 -MCAST_LEAVE_SOURCE_GROUP = 83 -MCAST_BLOCK_SOURCE = 84 -MCAST_UNBLOCK_SOURCE = 85 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MIN_MEMBERSHIPS = 31 -IP_MAX_MEMBERSHIPS = 4095 -IP_MAX_SOURCE_FILTER = 1024 -MCAST_INCLUDE = 1 -MCAST_EXCLUDE = 2 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 -def in_nullhost(x): return ((x).s_addr == INADDR_ANY) - - -# Included from netinet6/in6.h -__KAME_VERSION = "FreeBSD" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = (-16711680) -IPV6_ADDR_INT32_MLL = (-16646144) -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = (-65536) -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_INTFACELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -def IN6_IS_SCOPE_EMBED(a): return \ - -def IFA6_IS_DEPRECATED(a): return \ - -def IFA6_IS_INVALID(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_2292PKTINFO = 19 -IPV6_2292HOPLIMIT = 20 -IPV6_2292NEXTHOP = 21 -IPV6_2292HOPOPTS = 22 -IPV6_2292DSTOPTS = 23 -IPV6_2292RTHDR = 24 -IPV6_2292PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_V6ONLY = 27 -IPV6_BINDV6ONLY = IPV6_V6ONLY -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDRDSTOPTS = 35 -IPV6_RECVPKTINFO = 36 -IPV6_RECVHOPLIMIT = 37 -IPV6_RECVRTHDR = 38 -IPV6_RECVHOPOPTS = 39 -IPV6_RECVDSTOPTS = 40 -IPV6_RECVRTHDRDSTOPTS = 41 -IPV6_USE_MIN_MTU = 42 -IPV6_RECVPATHMTU = 43 -IPV6_PATHMTU = 44 -IPV6_REACHCONF = 45 -IPV6_PKTINFO = 46 -IPV6_HOPLIMIT = 47 -IPV6_NEXTHOP = 48 -IPV6_HOPOPTS = 49 -IPV6_DSTOPTS = 50 -IPV6_RTHDR = 51 -IPV6_PKTOPTIONS = 52 -IPV6_RECVTCLASS = 57 -IPV6_AUTOFLOWLABEL = 59 -IPV6_TCLASS = 61 -IPV6_DONTFRAG = 62 -IPV6_PREFER_TEMPADDR = 63 -IPV6_MSFILTER = 74 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_V6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_USETEMPADDR = 32 -IPV6CTL_TEMPPLTIME = 33 -IPV6CTL_TEMPVLTIME = 34 -IPV6CTL_AUTO_LINKLOCAL = 35 -IPV6CTL_RIP6STATS = 36 -IPV6CTL_PREFER_TEMPADDR = 37 -IPV6CTL_ADDRCTLPOLICY = 38 -IPV6CTL_USE_DEFAULTZONE = 39 -IPV6CTL_MAXFRAGS = 41 -IPV6CTL_IFQ = 42 -IPV6CTL_ISATAPRTR = 43 -IPV6CTL_MCAST_PMTU = 44 -IPV6CTL_STEALTH = 45 -IPV6CTL_MAXID = 46 diff --git a/Lib/plat-freebsd8/regen b/Lib/plat-freebsd8/regen deleted file mode 100755 --- a/Lib/plat-freebsd8/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-generic/regen b/Lib/plat-generic/regen deleted file mode 100755 --- a/Lib/plat-generic/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -eval $PYTHON_FOR_BUILD ../../Tools/scripts/h2py.py -i "'(u_long)'" /usr/include/netinet/in.h diff --git a/Lib/plat-linux/CDROM.py b/Lib/plat-linux/CDROM.py deleted file mode 100644 --- a/Lib/plat-linux/CDROM.py +++ /dev/null @@ -1,207 +0,0 @@ -# Generated by h2py from /usr/include/linux/cdrom.h - -CDROMPAUSE = 0x5301 -CDROMRESUME = 0x5302 -CDROMPLAYMSF = 0x5303 -CDROMPLAYTRKIND = 0x5304 -CDROMREADTOCHDR = 0x5305 -CDROMREADTOCENTRY = 0x5306 -CDROMSTOP = 0x5307 -CDROMSTART = 0x5308 -CDROMEJECT = 0x5309 -CDROMVOLCTRL = 0x530a -CDROMSUBCHNL = 0x530b -CDROMREADMODE2 = 0x530c -CDROMREADMODE1 = 0x530d -CDROMREADAUDIO = 0x530e -CDROMEJECT_SW = 0x530f -CDROMMULTISESSION = 0x5310 -CDROM_GET_MCN = 0x5311 -CDROM_GET_UPC = CDROM_GET_MCN -CDROMRESET = 0x5312 -CDROMVOLREAD = 0x5313 -CDROMREADRAW = 0x5314 -CDROMREADCOOKED = 0x5315 -CDROMSEEK = 0x5316 -CDROMPLAYBLK = 0x5317 -CDROMREADALL = 0x5318 -CDROMGETSPINDOWN = 0x531d -CDROMSETSPINDOWN = 0x531e -CDROMCLOSETRAY = 0x5319 -CDROM_SET_OPTIONS = 0x5320 -CDROM_CLEAR_OPTIONS = 0x5321 -CDROM_SELECT_SPEED = 0x5322 -CDROM_SELECT_DISC = 0x5323 -CDROM_MEDIA_CHANGED = 0x5325 -CDROM_DRIVE_STATUS = 0x5326 -CDROM_DISC_STATUS = 0x5327 -CDROM_CHANGER_NSLOTS = 0x5328 -CDROM_LOCKDOOR = 0x5329 -CDROM_DEBUG = 0x5330 -CDROM_GET_CAPABILITY = 0x5331 -CDROMAUDIOBUFSIZ = 0x5382 -DVD_READ_STRUCT = 0x5390 -DVD_WRITE_STRUCT = 0x5391 -DVD_AUTH = 0x5392 -CDROM_SEND_PACKET = 0x5393 -CDROM_NEXT_WRITABLE = 0x5394 -CDROM_LAST_WRITTEN = 0x5395 -CDROM_PACKET_SIZE = 12 -CGC_DATA_UNKNOWN = 0 -CGC_DATA_WRITE = 1 -CGC_DATA_READ = 2 -CGC_DATA_NONE = 3 -CD_MINS = 74 -CD_SECS = 60 -CD_FRAMES = 75 -CD_SYNC_SIZE = 12 -CD_MSF_OFFSET = 150 -CD_CHUNK_SIZE = 24 -CD_NUM_OF_CHUNKS = 98 -CD_FRAMESIZE_SUB = 96 -CD_HEAD_SIZE = 4 -CD_SUBHEAD_SIZE = 8 -CD_EDC_SIZE = 4 -CD_ZERO_SIZE = 8 -CD_ECC_SIZE = 276 -CD_FRAMESIZE = 2048 -CD_FRAMESIZE_RAW = 2352 -CD_FRAMESIZE_RAWER = 2646 -CD_FRAMESIZE_RAW1 = (CD_FRAMESIZE_RAW-CD_SYNC_SIZE) -CD_FRAMESIZE_RAW0 = (CD_FRAMESIZE_RAW-CD_SYNC_SIZE-CD_HEAD_SIZE) -CD_XA_HEAD = (CD_HEAD_SIZE+CD_SUBHEAD_SIZE) -CD_XA_TAIL = (CD_EDC_SIZE+CD_ECC_SIZE) -CD_XA_SYNC_HEAD = (CD_SYNC_SIZE+CD_XA_HEAD) -CDROM_LBA = 0x01 -CDROM_MSF = 0x02 -CDROM_DATA_TRACK = 0x04 -CDROM_LEADOUT = 0xAA -CDROM_AUDIO_INVALID = 0x00 -CDROM_AUDIO_PLAY = 0x11 -CDROM_AUDIO_PAUSED = 0x12 -CDROM_AUDIO_COMPLETED = 0x13 -CDROM_AUDIO_ERROR = 0x14 -CDROM_AUDIO_NO_STATUS = 0x15 -CDC_CLOSE_TRAY = 0x1 -CDC_OPEN_TRAY = 0x2 -CDC_LOCK = 0x4 -CDC_SELECT_SPEED = 0x8 -CDC_SELECT_DISC = 0x10 -CDC_MULTI_SESSION = 0x20 -CDC_MCN = 0x40 -CDC_MEDIA_CHANGED = 0x80 -CDC_PLAY_AUDIO = 0x100 -CDC_RESET = 0x200 -CDC_IOCTLS = 0x400 -CDC_DRIVE_STATUS = 0x800 -CDC_GENERIC_PACKET = 0x1000 -CDC_CD_R = 0x2000 -CDC_CD_RW = 0x4000 -CDC_DVD = 0x8000 -CDC_DVD_R = 0x10000 -CDC_DVD_RAM = 0x20000 -CDS_NO_INFO = 0 -CDS_NO_DISC = 1 -CDS_TRAY_OPEN = 2 -CDS_DRIVE_NOT_READY = 3 -CDS_DISC_OK = 4 -CDS_AUDIO = 100 -CDS_DATA_1 = 101 -CDS_DATA_2 = 102 -CDS_XA_2_1 = 103 -CDS_XA_2_2 = 104 -CDS_MIXED = 105 -CDO_AUTO_CLOSE = 0x1 -CDO_AUTO_EJECT = 0x2 -CDO_USE_FFLAGS = 0x4 -CDO_LOCK = 0x8 -CDO_CHECK_TYPE = 0x10 -CD_PART_MAX = 64 -CD_PART_MASK = (CD_PART_MAX - 1) -GPCMD_BLANK = 0xa1 -GPCMD_CLOSE_TRACK = 0x5b -GPCMD_FLUSH_CACHE = 0x35 -GPCMD_FORMAT_UNIT = 0x04 -GPCMD_GET_CONFIGURATION = 0x46 -GPCMD_GET_EVENT_STATUS_NOTIFICATION = 0x4a -GPCMD_GET_PERFORMANCE = 0xac -GPCMD_INQUIRY = 0x12 -GPCMD_LOAD_UNLOAD = 0xa6 -GPCMD_MECHANISM_STATUS = 0xbd -GPCMD_MODE_SELECT_10 = 0x55 -GPCMD_MODE_SENSE_10 = 0x5a -GPCMD_PAUSE_RESUME = 0x4b -GPCMD_PLAY_AUDIO_10 = 0x45 -GPCMD_PLAY_AUDIO_MSF = 0x47 -GPCMD_PLAY_AUDIO_TI = 0x48 -GPCMD_PLAY_CD = 0xbc -GPCMD_PREVENT_ALLOW_MEDIUM_REMOVAL = 0x1e -GPCMD_READ_10 = 0x28 -GPCMD_READ_12 = 0xa8 -GPCMD_READ_CDVD_CAPACITY = 0x25 -GPCMD_READ_CD = 0xbe -GPCMD_READ_CD_MSF = 0xb9 -GPCMD_READ_DISC_INFO = 0x51 -GPCMD_READ_DVD_STRUCTURE = 0xad -GPCMD_READ_FORMAT_CAPACITIES = 0x23 -GPCMD_READ_HEADER = 0x44 -GPCMD_READ_TRACK_RZONE_INFO = 0x52 -GPCMD_READ_SUBCHANNEL = 0x42 -GPCMD_READ_TOC_PMA_ATIP = 0x43 -GPCMD_REPAIR_RZONE_TRACK = 0x58 -GPCMD_REPORT_KEY = 0xa4 -GPCMD_REQUEST_SENSE = 0x03 -GPCMD_RESERVE_RZONE_TRACK = 0x53 -GPCMD_SCAN = 0xba -GPCMD_SEEK = 0x2b -GPCMD_SEND_DVD_STRUCTURE = 0xad -GPCMD_SEND_EVENT = 0xa2 -GPCMD_SEND_KEY = 0xa3 -GPCMD_SEND_OPC = 0x54 -GPCMD_SET_READ_AHEAD = 0xa7 -GPCMD_SET_STREAMING = 0xb6 -GPCMD_START_STOP_UNIT = 0x1b -GPCMD_STOP_PLAY_SCAN = 0x4e -GPCMD_TEST_UNIT_READY = 0x00 -GPCMD_VERIFY_10 = 0x2f -GPCMD_WRITE_10 = 0x2a -GPCMD_WRITE_AND_VERIFY_10 = 0x2e -GPCMD_SET_SPEED = 0xbb -GPCMD_PLAYAUDIO_TI = 0x48 -GPCMD_GET_MEDIA_STATUS = 0xda -GPMODE_R_W_ERROR_PAGE = 0x01 -GPMODE_WRITE_PARMS_PAGE = 0x05 -GPMODE_AUDIO_CTL_PAGE = 0x0e -GPMODE_POWER_PAGE = 0x1a -GPMODE_FAULT_FAIL_PAGE = 0x1c -GPMODE_TO_PROTECT_PAGE = 0x1d -GPMODE_CAPABILITIES_PAGE = 0x2a -GPMODE_ALL_PAGES = 0x3f -GPMODE_CDROM_PAGE = 0x0d -DVD_STRUCT_PHYSICAL = 0x00 -DVD_STRUCT_COPYRIGHT = 0x01 -DVD_STRUCT_DISCKEY = 0x02 -DVD_STRUCT_BCA = 0x03 -DVD_STRUCT_MANUFACT = 0x04 -DVD_LAYERS = 4 -DVD_LU_SEND_AGID = 0 -DVD_HOST_SEND_CHALLENGE = 1 -DVD_LU_SEND_KEY1 = 2 -DVD_LU_SEND_CHALLENGE = 3 -DVD_HOST_SEND_KEY2 = 4 -DVD_AUTH_ESTABLISHED = 5 -DVD_AUTH_FAILURE = 6 -DVD_LU_SEND_TITLE_KEY = 7 -DVD_LU_SEND_ASF = 8 -DVD_INVALIDATE_AGID = 9 -DVD_LU_SEND_RPC_STATE = 10 -DVD_HOST_SEND_RPC_STATE = 11 -DVD_CPM_NO_COPYRIGHT = 0 -DVD_CPM_COPYRIGHTED = 1 -DVD_CP_SEC_NONE = 0 -DVD_CP_SEC_EXIST = 1 -DVD_CGMS_UNRESTRICTED = 0 -DVD_CGMS_SINGLE = 2 -DVD_CGMS_RESTRICTED = 3 - -CDROM_MAX_SLOTS = 256 diff --git a/Lib/plat-linux/DLFCN.py b/Lib/plat-linux/DLFCN.py deleted file mode 100644 --- a/Lib/plat-linux/DLFCN.py +++ /dev/null @@ -1,83 +0,0 @@ -# Generated by h2py from /usr/include/dlfcn.h -_DLFCN_H = 1 - -# Included from features.h -_FEATURES_H = 1 -__USE_ANSI = 1 -__FAVOR_BSD = 1 -_ISOC99_SOURCE = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 199506 -_XOPEN_SOURCE = 600 -_XOPEN_SOURCE_EXTENDED = 1 -_LARGEFILE64_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -__USE_ISOC99 = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 2 -_POSIX_C_SOURCE = 199506 -__USE_POSIX = 1 -__USE_POSIX2 = 1 -__USE_POSIX199309 = 1 -__USE_POSIX199506 = 1 -__USE_XOPEN = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_UNIX98 = 1 -_LARGEFILE_SOURCE = 1 -__USE_XOPEN2K = 1 -__USE_ISOC99 = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_FILE_OFFSET64 = 1 -__USE_MISC = 1 -__USE_BSD = 1 -__USE_SVID = 1 -__USE_GNU = 1 -__USE_REENTRANT = 1 -__STDC_IEC_559__ = 1 -__STDC_IEC_559_COMPLEX__ = 1 -__STDC_ISO_10646__ = 200009 -__GNU_LIBRARY__ = 6 -__GLIBC__ = 2 -__GLIBC_MINOR__ = 2 - -# Included from sys/cdefs.h -_SYS_CDEFS_H = 1 -def __PMT(args): return args - -def __P(args): return args - -def __PMT(args): return args - -def __STRING(x): return #x - -__flexarr = [] -__flexarr = [0] -__flexarr = [] -__flexarr = [1] -def __ASMNAME(cname): return __ASMNAME2 (__USER_LABEL_PREFIX__, cname) - -def __attribute__(xyz): return - -def __attribute_format_arg__(x): return __attribute__ ((__format_arg__ (x))) - -def __attribute_format_arg__(x): return - -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_EXTERN_INLINES = 1 - -# Included from gnu/stubs.h - -# Included from bits/dlfcn.h -RTLD_LAZY = 0x00001 -RTLD_NOW = 0x00002 -RTLD_BINDING_MASK = 0x3 -RTLD_NOLOAD = 0x00004 -RTLD_GLOBAL = 0x00100 -RTLD_LOCAL = 0 -RTLD_NODELETE = 0x01000 diff --git a/Lib/plat-linux/IN.py b/Lib/plat-linux/IN.py deleted file mode 100644 --- a/Lib/plat-linux/IN.py +++ /dev/null @@ -1,615 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h -_NETINET_IN_H = 1 - -# Included from features.h -_FEATURES_H = 1 -__USE_ANSI = 1 -__FAVOR_BSD = 1 -_ISOC99_SOURCE = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 199506 -_XOPEN_SOURCE = 600 -_XOPEN_SOURCE_EXTENDED = 1 -_LARGEFILE64_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -__USE_ISOC99 = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 2 -_POSIX_C_SOURCE = 199506 -__USE_POSIX = 1 -__USE_POSIX2 = 1 -__USE_POSIX199309 = 1 -__USE_POSIX199506 = 1 -__USE_XOPEN = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_UNIX98 = 1 -_LARGEFILE_SOURCE = 1 -__USE_XOPEN2K = 1 -__USE_ISOC99 = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_FILE_OFFSET64 = 1 -__USE_MISC = 1 -__USE_BSD = 1 -__USE_SVID = 1 -__USE_GNU = 1 -__USE_REENTRANT = 1 -__STDC_IEC_559__ = 1 -__STDC_IEC_559_COMPLEX__ = 1 -__STDC_ISO_10646__ = 200009 -__GNU_LIBRARY__ = 6 -__GLIBC__ = 2 -__GLIBC_MINOR__ = 2 - -# Included from sys/cdefs.h -_SYS_CDEFS_H = 1 -def __PMT(args): return args - -def __P(args): return args - -def __PMT(args): return args - -def __STRING(x): return #x - -__flexarr = [] -__flexarr = [0] -__flexarr = [] -__flexarr = [1] -def __ASMNAME(cname): return __ASMNAME2 (__USER_LABEL_PREFIX__, cname) - -def __attribute__(xyz): return - -def __attribute_format_arg__(x): return __attribute__ ((__format_arg__ (x))) - -def __attribute_format_arg__(x): return - -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_EXTERN_INLINES = 1 - -# Included from gnu/stubs.h - -# Included from stdint.h -_STDINT_H = 1 - -# Included from bits/wchar.h -_BITS_WCHAR_H = 1 -__WCHAR_MIN = (-2147483647 - 1) -__WCHAR_MAX = (2147483647) - -# Included from bits/wordsize.h -__WORDSIZE = 32 -def __INT64_C(c): return c ## L - -def __UINT64_C(c): return c ## UL - -def __INT64_C(c): return c ## LL - -def __UINT64_C(c): return c ## ULL - -INT8_MIN = (-128) -INT16_MIN = (-32767-1) -INT32_MIN = (-2147483647-1) -INT64_MIN = (-__INT64_C(9223372036854775807)-1) -INT8_MAX = (127) -INT16_MAX = (32767) -INT32_MAX = (2147483647) -INT64_MAX = (__INT64_C(9223372036854775807)) -UINT8_MAX = (255) -UINT16_MAX = (65535) -UINT64_MAX = (__UINT64_C(18446744073709551615)) -INT_LEAST8_MIN = (-128) -INT_LEAST16_MIN = (-32767-1) -INT_LEAST32_MIN = (-2147483647-1) -INT_LEAST64_MIN = (-__INT64_C(9223372036854775807)-1) -INT_LEAST8_MAX = (127) -INT_LEAST16_MAX = (32767) -INT_LEAST32_MAX = (2147483647) -INT_LEAST64_MAX = (__INT64_C(9223372036854775807)) -UINT_LEAST8_MAX = (255) -UINT_LEAST16_MAX = (65535) -UINT_LEAST64_MAX = (__UINT64_C(18446744073709551615)) -INT_FAST8_MIN = (-128) -INT_FAST16_MIN = (-9223372036854775807-1) -INT_FAST32_MIN = (-9223372036854775807-1) -INT_FAST16_MIN = (-2147483647-1) -INT_FAST32_MIN = (-2147483647-1) -INT_FAST64_MIN = (-__INT64_C(9223372036854775807)-1) -INT_FAST8_MAX = (127) -INT_FAST16_MAX = (9223372036854775807) -INT_FAST32_MAX = (9223372036854775807) -INT_FAST16_MAX = (2147483647) -INT_FAST32_MAX = (2147483647) -INT_FAST64_MAX = (__INT64_C(9223372036854775807)) -UINT_FAST8_MAX = (255) -UINT_FAST64_MAX = (__UINT64_C(18446744073709551615)) -INTPTR_MIN = (-9223372036854775807-1) -INTPTR_MAX = (9223372036854775807) -INTPTR_MIN = (-2147483647-1) -INTPTR_MAX = (2147483647) -INTMAX_MIN = (-__INT64_C(9223372036854775807)-1) -INTMAX_MAX = (__INT64_C(9223372036854775807)) -UINTMAX_MAX = (__UINT64_C(18446744073709551615)) -PTRDIFF_MIN = (-9223372036854775807-1) -PTRDIFF_MAX = (9223372036854775807) -PTRDIFF_MIN = (-2147483647-1) -PTRDIFF_MAX = (2147483647) -SIG_ATOMIC_MIN = (-2147483647-1) -SIG_ATOMIC_MAX = (2147483647) -WCHAR_MIN = __WCHAR_MIN -WCHAR_MAX = __WCHAR_MAX -def INT8_C(c): return c - -def INT16_C(c): return c - -def INT32_C(c): return c - -def INT64_C(c): return c ## L - -def INT64_C(c): return c ## LL - -def UINT8_C(c): return c ## U - -def UINT16_C(c): return c ## U - -def UINT32_C(c): return c ## U - -def UINT64_C(c): return c ## UL - -def UINT64_C(c): return c ## ULL - -def INTMAX_C(c): return c ## L - -def UINTMAX_C(c): return c ## UL - -def INTMAX_C(c): return c ## LL - -def UINTMAX_C(c): return c ## ULL - - -# Included from bits/types.h -_BITS_TYPES_H = 1 -__FD_SETSIZE = 1024 - -# Included from bits/pthreadtypes.h -_BITS_PTHREADTYPES_H = 1 - -# Included from bits/sched.h -SCHED_OTHER = 0 -SCHED_FIFO = 1 -SCHED_RR = 2 -CSIGNAL = 0x000000ff -CLONE_VM = 0x00000100 -CLONE_FS = 0x00000200 -CLONE_FILES = 0x00000400 -CLONE_SIGHAND = 0x00000800 -CLONE_PID = 0x00001000 -CLONE_PTRACE = 0x00002000 -CLONE_VFORK = 0x00004000 -__defined_schedparam = 1 -def IN_CLASSA(a): return ((((in_addr_t)(a)) & (-2147483648)) == 0) - -IN_CLASSA_NET = (-16777216) -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = ((-1) & ~IN_CLASSA_NET) -IN_CLASSA_MAX = 128 -def IN_CLASSB(a): return ((((in_addr_t)(a)) & (-1073741824)) == (-2147483648)) - -IN_CLASSB_NET = (-65536) -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = ((-1) & ~IN_CLASSB_NET) -IN_CLASSB_MAX = 65536 -def IN_CLASSC(a): return ((((in_addr_t)(a)) & (-536870912)) == (-1073741824)) - -IN_CLASSC_NET = (-256) -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = ((-1) & ~IN_CLASSC_NET) -def IN_CLASSD(a): return ((((in_addr_t)(a)) & (-268435456)) == (-536870912)) - -def IN_MULTICAST(a): return IN_CLASSD(a) - -def IN_EXPERIMENTAL(a): return ((((in_addr_t)(a)) & (-536870912)) == (-536870912)) - -def IN_BADCLASS(a): return ((((in_addr_t)(a)) & (-268435456)) == (-268435456)) - -IN_LOOPBACKNET = 127 -INET_ADDRSTRLEN = 16 -INET6_ADDRSTRLEN = 46 - -# Included from bits/socket.h - -# Included from limits.h -_LIBC_LIMITS_H_ = 1 -MB_LEN_MAX = 16 -_LIMITS_H = 1 -CHAR_BIT = 8 -SCHAR_MIN = (-128) -SCHAR_MAX = 127 -UCHAR_MAX = 255 -CHAR_MIN = 0 -CHAR_MAX = UCHAR_MAX -CHAR_MIN = SCHAR_MIN -CHAR_MAX = SCHAR_MAX -SHRT_MIN = (-32768) -SHRT_MAX = 32767 -USHRT_MAX = 65535 -INT_MAX = 2147483647 -LONG_MAX = 9223372036854775807 -LONG_MAX = 2147483647 -LONG_MIN = (-LONG_MAX - 1) - -# Included from bits/posix1_lim.h -_BITS_POSIX1_LIM_H = 1 -_POSIX_AIO_LISTIO_MAX = 2 -_POSIX_AIO_MAX = 1 -_POSIX_ARG_MAX = 4096 -_POSIX_CHILD_MAX = 6 -_POSIX_DELAYTIMER_MAX = 32 -_POSIX_LINK_MAX = 8 -_POSIX_MAX_CANON = 255 -_POSIX_MAX_INPUT = 255 -_POSIX_MQ_OPEN_MAX = 8 -_POSIX_MQ_PRIO_MAX = 32 -_POSIX_NGROUPS_MAX = 0 -_POSIX_OPEN_MAX = 16 -_POSIX_FD_SETSIZE = _POSIX_OPEN_MAX -_POSIX_NAME_MAX = 14 -_POSIX_PATH_MAX = 256 -_POSIX_PIPE_BUF = 512 -_POSIX_RTSIG_MAX = 8 -_POSIX_SEM_NSEMS_MAX = 256 -_POSIX_SEM_VALUE_MAX = 32767 -_POSIX_SIGQUEUE_MAX = 32 -_POSIX_SSIZE_MAX = 32767 -_POSIX_STREAM_MAX = 8 -_POSIX_TZNAME_MAX = 6 -_POSIX_QLIMIT = 1 -_POSIX_HIWAT = _POSIX_PIPE_BUF -_POSIX_UIO_MAXIOV = 16 -_POSIX_TTY_NAME_MAX = 9 -_POSIX_TIMER_MAX = 32 -_POSIX_LOGIN_NAME_MAX = 9 -_POSIX_CLOCKRES_MIN = 20000000 - -# Included from bits/local_lim.h - -# Included from linux/limits.h -NR_OPEN = 1024 -NGROUPS_MAX = 32 -ARG_MAX = 131072 -CHILD_MAX = 999 -OPEN_MAX = 256 -LINK_MAX = 127 -MAX_CANON = 255 -MAX_INPUT = 255 -NAME_MAX = 255 -PATH_MAX = 4096 -PIPE_BUF = 4096 -RTSIG_MAX = 32 -_POSIX_THREAD_KEYS_MAX = 128 -PTHREAD_KEYS_MAX = 1024 -_POSIX_THREAD_DESTRUCTOR_ITERATIONS = 4 -PTHREAD_DESTRUCTOR_ITERATIONS = _POSIX_THREAD_DESTRUCTOR_ITERATIONS -_POSIX_THREAD_THREADS_MAX = 64 -PTHREAD_THREADS_MAX = 1024 -AIO_PRIO_DELTA_MAX = 20 -PTHREAD_STACK_MIN = 16384 -TIMER_MAX = 256 -SSIZE_MAX = LONG_MAX -NGROUPS_MAX = _POSIX_NGROUPS_MAX - -# Included from bits/posix2_lim.h -_BITS_POSIX2_LIM_H = 1 -_POSIX2_BC_BASE_MAX = 99 -_POSIX2_BC_DIM_MAX = 2048 -_POSIX2_BC_SCALE_MAX = 99 -_POSIX2_BC_STRING_MAX = 1000 -_POSIX2_COLL_WEIGHTS_MAX = 2 -_POSIX2_EXPR_NEST_MAX = 32 -_POSIX2_LINE_MAX = 2048 -_POSIX2_RE_DUP_MAX = 255 -_POSIX2_CHARCLASS_NAME_MAX = 14 -BC_BASE_MAX = _POSIX2_BC_BASE_MAX -BC_DIM_MAX = _POSIX2_BC_DIM_MAX -BC_SCALE_MAX = _POSIX2_BC_SCALE_MAX -BC_STRING_MAX = _POSIX2_BC_STRING_MAX -COLL_WEIGHTS_MAX = 255 -EXPR_NEST_MAX = _POSIX2_EXPR_NEST_MAX -LINE_MAX = _POSIX2_LINE_MAX -CHARCLASS_NAME_MAX = 2048 -RE_DUP_MAX = (0x7fff) - -# Included from bits/xopen_lim.h -_XOPEN_LIM_H = 1 - -# Included from bits/stdio_lim.h -L_tmpnam = 20 -TMP_MAX = 238328 -FILENAME_MAX = 4096 -L_ctermid = 9 -L_cuserid = 9 -FOPEN_MAX = 16 -IOV_MAX = 1024 -_XOPEN_IOV_MAX = _POSIX_UIO_MAXIOV -NL_ARGMAX = _POSIX_ARG_MAX -NL_LANGMAX = _POSIX2_LINE_MAX -NL_MSGMAX = INT_MAX -NL_NMAX = INT_MAX -NL_SETMAX = INT_MAX -NL_TEXTMAX = INT_MAX -NZERO = 20 -WORD_BIT = 16 -WORD_BIT = 32 -WORD_BIT = 64 -WORD_BIT = 16 -WORD_BIT = 32 -WORD_BIT = 64 -WORD_BIT = 32 -LONG_BIT = 32 -LONG_BIT = 64 -LONG_BIT = 32 -LONG_BIT = 64 -LONG_BIT = 64 -LONG_BIT = 32 -from TYPES import * -PF_UNSPEC = 0 -PF_LOCAL = 1 -PF_UNIX = PF_LOCAL -PF_FILE = PF_LOCAL -PF_INET = 2 -PF_AX25 = 3 -PF_IPX = 4 -PF_APPLETALK = 5 -PF_NETROM = 6 -PF_BRIDGE = 7 -PF_ATMPVC = 8 -PF_X25 = 9 -PF_INET6 = 10 -PF_ROSE = 11 -PF_DECnet = 12 -PF_NETBEUI = 13 -PF_SECURITY = 14 -PF_KEY = 15 -PF_NETLINK = 16 -PF_ROUTE = PF_NETLINK -PF_PACKET = 17 -PF_ASH = 18 -PF_ECONET = 19 -PF_ATMSVC = 20 -PF_SNA = 22 -PF_IRDA = 23 -PF_PPPOX = 24 -PF_WANPIPE = 25 -PF_BLUETOOTH = 31 -PF_MAX = 32 -AF_UNSPEC = PF_UNSPEC -AF_LOCAL = PF_LOCAL -AF_UNIX = PF_UNIX -AF_FILE = PF_FILE -AF_INET = PF_INET -AF_AX25 = PF_AX25 -AF_IPX = PF_IPX -AF_APPLETALK = PF_APPLETALK -AF_NETROM = PF_NETROM -AF_BRIDGE = PF_BRIDGE -AF_ATMPVC = PF_ATMPVC -AF_X25 = PF_X25 -AF_INET6 = PF_INET6 -AF_ROSE = PF_ROSE -AF_DECnet = PF_DECnet -AF_NETBEUI = PF_NETBEUI -AF_SECURITY = PF_SECURITY -AF_KEY = PF_KEY -AF_NETLINK = PF_NETLINK -AF_ROUTE = PF_ROUTE -AF_PACKET = PF_PACKET -AF_ASH = PF_ASH -AF_ECONET = PF_ECONET -AF_ATMSVC = PF_ATMSVC -AF_SNA = PF_SNA -AF_IRDA = PF_IRDA -AF_PPPOX = PF_PPPOX -AF_WANPIPE = PF_WANPIPE -AF_BLUETOOTH = PF_BLUETOOTH -AF_MAX = PF_MAX -SOL_RAW = 255 -SOL_DECNET = 261 -SOL_X25 = 262 -SOL_PACKET = 263 -SOL_ATM = 264 -SOL_AAL = 265 -SOL_IRDA = 266 -SOMAXCONN = 128 - -# Included from bits/sockaddr.h -_BITS_SOCKADDR_H = 1 -def __SOCKADDR_COMMON(sa_prefix): return \ - -_SS_SIZE = 128 -def CMSG_FIRSTHDR(mhdr): return \ - - -# Included from asm/socket.h - -# Included from asm/sockios.h -FIOSETOWN = 0x8901 -SIOCSPGRP = 0x8902 -FIOGETOWN = 0x8903 -SIOCGPGRP = 0x8904 -SIOCATMARK = 0x8905 -SIOCGSTAMP = 0x8906 -SOL_SOCKET = 1 -SO_DEBUG = 1 -SO_REUSEADDR = 2 -SO_TYPE = 3 -SO_ERROR = 4 -SO_DONTROUTE = 5 -SO_BROADCAST = 6 -SO_SNDBUF = 7 -SO_RCVBUF = 8 -SO_KEEPALIVE = 9 -SO_OOBINLINE = 10 -SO_NO_CHECK = 11 -SO_PRIORITY = 12 -SO_LINGER = 13 -SO_BSDCOMPAT = 14 -SO_PASSCRED = 16 -SO_PEERCRED = 17 -SO_RCVLOWAT = 18 -SO_SNDLOWAT = 19 -SO_RCVTIMEO = 20 -SO_SNDTIMEO = 21 -SO_SECURITY_AUTHENTICATION = 22 -SO_SECURITY_ENCRYPTION_TRANSPORT = 23 -SO_SECURITY_ENCRYPTION_NETWORK = 24 -SO_BINDTODEVICE = 25 -SO_ATTACH_FILTER = 26 -SO_DETACH_FILTER = 27 -SO_PEERNAME = 28 -SO_TIMESTAMP = 29 -SCM_TIMESTAMP = SO_TIMESTAMP -SO_ACCEPTCONN = 30 -SOCK_STREAM = 1 -SOCK_DGRAM = 2 -SOCK_RAW = 3 -SOCK_RDM = 4 -SOCK_SEQPACKET = 5 -SOCK_PACKET = 10 -SOCK_MAX = (SOCK_PACKET+1) - -# Included from bits/in.h -IP_TOS = 1 -IP_TTL = 2 -IP_HDRINCL = 3 -IP_OPTIONS = 4 -IP_ROUTER_ALERT = 5 -IP_RECVOPTS = 6 -IP_RETOPTS = 7 -IP_PKTINFO = 8 -IP_PKTOPTIONS = 9 -IP_PMTUDISC = 10 -IP_MTU_DISCOVER = 10 -IP_RECVERR = 11 -IP_RECVTTL = 12 -IP_RECVTOS = 13 -IP_MULTICAST_IF = 32 -IP_MULTICAST_TTL = 33 -IP_MULTICAST_LOOP = 34 -IP_ADD_MEMBERSHIP = 35 -IP_DROP_MEMBERSHIP = 36 -IP_RECVRETOPTS = IP_RETOPTS -IP_PMTUDISC_DONT = 0 -IP_PMTUDISC_WANT = 1 -IP_PMTUDISC_DO = 2 -SOL_IP = 0 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IPV6_ADDRFORM = 1 -IPV6_PKTINFO = 2 -IPV6_HOPOPTS = 3 -IPV6_DSTOPTS = 4 -IPV6_RTHDR = 5 -IPV6_PKTOPTIONS = 6 -IPV6_CHECKSUM = 7 -IPV6_HOPLIMIT = 8 -IPV6_NEXTHOP = 9 -IPV6_AUTHHDR = 10 -IPV6_UNICAST_HOPS = 16 -IPV6_MULTICAST_IF = 17 -IPV6_MULTICAST_HOPS = 18 -IPV6_MULTICAST_LOOP = 19 -IPV6_JOIN_GROUP = 20 -IPV6_LEAVE_GROUP = 21 -IPV6_ROUTER_ALERT = 22 -IPV6_MTU_DISCOVER = 23 -IPV6_MTU = 24 -IPV6_RECVERR = 25 -IPV6_RXHOPOPTS = IPV6_HOPOPTS -IPV6_RXDSTOPTS = IPV6_DSTOPTS -IPV6_ADD_MEMBERSHIP = IPV6_JOIN_GROUP -IPV6_DROP_MEMBERSHIP = IPV6_LEAVE_GROUP -IPV6_PMTUDISC_DONT = 0 -IPV6_PMTUDISC_WANT = 1 -IPV6_PMTUDISC_DO = 2 -SOL_IPV6 = 41 -SOL_ICMPV6 = 58 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 - -# Included from endian.h -_ENDIAN_H = 1 -__LITTLE_ENDIAN = 1234 -__BIG_ENDIAN = 4321 -__PDP_ENDIAN = 3412 - -# Included from bits/endian.h -__BYTE_ORDER = __LITTLE_ENDIAN -__FLOAT_WORD_ORDER = __BYTE_ORDER -LITTLE_ENDIAN = __LITTLE_ENDIAN -BIG_ENDIAN = __BIG_ENDIAN -PDP_ENDIAN = __PDP_ENDIAN -BYTE_ORDER = __BYTE_ORDER - -# Included from bits/byteswap.h -_BITS_BYTESWAP_H = 1 -def __bswap_constant_16(x): return \ - -def __bswap_16(x): return \ - -def __bswap_16(x): return __bswap_constant_16 (x) - -def __bswap_constant_32(x): return \ - -def __bswap_32(x): return \ - -def __bswap_32(x): return \ - -def __bswap_32(x): return __bswap_constant_32 (x) - -def __bswap_constant_64(x): return \ - -def __bswap_64(x): return \ - -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -def ntohl(x): return __bswap_32 (x) - -def ntohs(x): return __bswap_16 (x) - -def htonl(x): return __bswap_32 (x) - -def htons(x): return __bswap_16 (x) - -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return diff --git a/Lib/plat-linux/TYPES.py b/Lib/plat-linux/TYPES.py deleted file mode 100644 --- a/Lib/plat-linux/TYPES.py +++ /dev/null @@ -1,170 +0,0 @@ -# Generated by h2py from /usr/include/sys/types.h -_SYS_TYPES_H = 1 - -# Included from features.h -_FEATURES_H = 1 -__USE_ANSI = 1 -__FAVOR_BSD = 1 -_ISOC99_SOURCE = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 199506 -_XOPEN_SOURCE = 600 -_XOPEN_SOURCE_EXTENDED = 1 -_LARGEFILE64_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -__USE_ISOC99 = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 2 -_POSIX_C_SOURCE = 199506 -__USE_POSIX = 1 -__USE_POSIX2 = 1 -__USE_POSIX199309 = 1 -__USE_POSIX199506 = 1 -__USE_XOPEN = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_UNIX98 = 1 -_LARGEFILE_SOURCE = 1 -__USE_XOPEN2K = 1 -__USE_ISOC99 = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_FILE_OFFSET64 = 1 -__USE_MISC = 1 -__USE_BSD = 1 -__USE_SVID = 1 -__USE_GNU = 1 -__USE_REENTRANT = 1 -__STDC_IEC_559__ = 1 -__STDC_IEC_559_COMPLEX__ = 1 -__STDC_ISO_10646__ = 200009 -__GNU_LIBRARY__ = 6 -__GLIBC__ = 2 -__GLIBC_MINOR__ = 2 - -# Included from sys/cdefs.h -_SYS_CDEFS_H = 1 -def __PMT(args): return args - -def __P(args): return args - -def __PMT(args): return args - -def __STRING(x): return #x - -__flexarr = [] -__flexarr = [0] -__flexarr = [] -__flexarr = [1] -def __ASMNAME(cname): return __ASMNAME2 (__USER_LABEL_PREFIX__, cname) - -def __attribute__(xyz): return - -def __attribute_format_arg__(x): return __attribute__ ((__format_arg__ (x))) - -def __attribute_format_arg__(x): return - -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_EXTERN_INLINES = 1 - -# Included from gnu/stubs.h - -# Included from bits/types.h -_BITS_TYPES_H = 1 -__FD_SETSIZE = 1024 - -# Included from bits/pthreadtypes.h -_BITS_PTHREADTYPES_H = 1 - -# Included from bits/sched.h -SCHED_OTHER = 0 -SCHED_FIFO = 1 -SCHED_RR = 2 -CSIGNAL = 0x000000ff -CLONE_VM = 0x00000100 -CLONE_FS = 0x00000200 -CLONE_FILES = 0x00000400 -CLONE_SIGHAND = 0x00000800 -CLONE_PID = 0x00001000 -CLONE_PTRACE = 0x00002000 -CLONE_VFORK = 0x00004000 -__defined_schedparam = 1 - -# Included from time.h -_TIME_H = 1 - -# Included from bits/time.h -_BITS_TIME_H = 1 -CLOCKS_PER_SEC = 1000000 -CLOCK_REALTIME = 0 -CLOCK_PROCESS_CPUTIME_ID = 2 -CLOCK_THREAD_CPUTIME_ID = 3 -TIMER_ABSTIME = 1 -_STRUCT_TIMEVAL = 1 -CLK_TCK = CLOCKS_PER_SEC -__clock_t_defined = 1 -__time_t_defined = 1 -__clockid_t_defined = 1 -__timer_t_defined = 1 -__timespec_defined = 1 -def __isleap(year): return \ - -__BIT_TYPES_DEFINED__ = 1 - -# Included from endian.h -_ENDIAN_H = 1 -__LITTLE_ENDIAN = 1234 -__BIG_ENDIAN = 4321 -__PDP_ENDIAN = 3412 - -# Included from bits/endian.h -__BYTE_ORDER = __LITTLE_ENDIAN -__FLOAT_WORD_ORDER = __BYTE_ORDER -LITTLE_ENDIAN = __LITTLE_ENDIAN -BIG_ENDIAN = __BIG_ENDIAN -PDP_ENDIAN = __PDP_ENDIAN -BYTE_ORDER = __BYTE_ORDER - -# Included from sys/select.h -_SYS_SELECT_H = 1 - -# Included from bits/select.h -def __FD_ZERO(fdsp): return \ - -def __FD_ZERO(set): return \ - - -# Included from bits/sigset.h -_SIGSET_H_types = 1 -_SIGSET_H_fns = 1 -def __sigmask(sig): return \ - -def __sigemptyset(set): return \ - -def __sigfillset(set): return \ - -def __sigisemptyset(set): return \ - -def __FDELT(d): return ((d) / __NFDBITS) - -FD_SETSIZE = __FD_SETSIZE -def FD_ZERO(fdsetp): return __FD_ZERO (fdsetp) - - -# Included from sys/sysmacros.h -_SYS_SYSMACROS_H = 1 -def major(dev): return ((int)(((dev) >> 8) & 0xff)) - -def minor(dev): return ((int)((dev) & 0xff)) - -def major(dev): return (((dev).__val[1] >> 8) & 0xff) - -def minor(dev): return ((dev).__val[1] & 0xff) - -def major(dev): return (((dev).__val[0] >> 8) & 0xff) - -def minor(dev): return ((dev).__val[0] & 0xff) diff --git a/Lib/plat-linux/regen b/Lib/plat-linux/regen deleted file mode 100755 --- a/Lib/plat-linux/regen +++ /dev/null @@ -1,33 +0,0 @@ -#! /bin/sh -case `uname` in -Linux*|GNU*) ;; -*) echo Probably not on a Linux system 1>&2 - exit 1;; -esac -if [ -z "$CC" ]; then - echo >&2 "$(basename $0): CC is not set" - exit 1 -fi -headers="sys/types.h netinet/in.h dlfcn.h" -incdirs="$(echo $($CC -v -E - < /dev/null 2>&1|awk '/^#include/, /^End of search/' | grep '^ '))" -if [ -z "$incdirs" ]; then - incdirs="/usr/include" -fi -for h in $headers; do - absh= - for d in $incdirs; do - if [ -f "$d/$h" ]; then - absh="$d/$h" - break - fi - done - if [ -n "$absh" ]; then - absheaders="$absheaders $absh" - else - echo >&2 "$(basename $0): header $h not found" - exit 1 - fi -done - -set -x -${H2PY:-h2py} -i '(u_long)' $absheaders diff --git a/Lib/plat-netbsd1/IN.py b/Lib/plat-netbsd1/IN.py deleted file mode 100644 --- a/Lib/plat-netbsd1/IN.py +++ /dev/null @@ -1,56 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPIP = 4 -IPPROTO_TCP = 6 -IPPROTO_EGP = 8 -IPPROTO_PUP = 12 -IPPROTO_UDP = 17 -IPPROTO_IDP = 22 -IPPROTO_TP = 29 -IPPROTO_EON = 80 -IPPROTO_ENCAP = 98 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -def __IPADDR(x): return ((u_int32_t)(x)) - -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_MAX = 128 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_MAX = 65536 -IN_CLASSC_NSHIFT = 8 -IN_CLASSD_NSHIFT = 28 -def IN_MULTICAST(i): return IN_CLASSD(i) - -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_RECVIF = 20 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IPPROTO_MAXID = (IPPROTO_IDP + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_FORWSRCRT = 5 -IPCTL_DIRECTEDBCAST = 6 -IPCTL_ALLOWSRCRT = 7 -IPCTL_MAXID = 8 -def in_nullhost(x): return ((x).s_addr == INADDR_ANY) diff --git a/Lib/plat-netbsd1/regen b/Lib/plat-netbsd1/regen deleted file mode 100755 --- a/Lib/plat-netbsd1/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-next3/regen b/Lib/plat-next3/regen deleted file mode 100755 --- a/Lib/plat-next3/regen +++ /dev/null @@ -1,6 +0,0 @@ -#! /bin/sh -set -v -INCLUDE="/NextDeveloper/Headers;/NextDeveloper/Headers/ansi;/NextDeveloper/Headers/bsd" -export INCLUDE - -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/bsd/netinet/in.h diff --git a/Lib/plat-sunos5/CDIO.py b/Lib/plat-sunos5/CDIO.py deleted file mode 100644 --- a/Lib/plat-sunos5/CDIO.py +++ /dev/null @@ -1,73 +0,0 @@ -# Generated by h2py from /usr/include/sys/cdio.h -CDROM_LBA = 0x01 -CDROM_MSF = 0x02 -CDROM_DATA_TRACK = 0x04 -CDROM_LEADOUT = 0xAA -CDROM_AUDIO_INVALID = 0x00 -CDROM_AUDIO_PLAY = 0x11 -CDROM_AUDIO_PAUSED = 0x12 -CDROM_AUDIO_COMPLETED = 0x13 -CDROM_AUDIO_ERROR = 0x14 -CDROM_AUDIO_NO_STATUS = 0x15 -CDROM_DA_NO_SUBCODE = 0x00 -CDROM_DA_SUBQ = 0x01 -CDROM_DA_ALL_SUBCODE = 0x02 -CDROM_DA_SUBCODE_ONLY = 0x03 -CDROM_XA_DATA = 0x00 -CDROM_XA_SECTOR_DATA = 0x01 -CDROM_XA_DATA_W_ERROR = 0x02 -CDROM_BLK_512 = 512 -CDROM_BLK_1024 = 1024 -CDROM_BLK_2048 = 2048 -CDROM_BLK_2056 = 2056 -CDROM_BLK_2336 = 2336 -CDROM_BLK_2340 = 2340 -CDROM_BLK_2352 = 2352 -CDROM_BLK_2368 = 2368 -CDROM_BLK_2448 = 2448 -CDROM_BLK_2646 = 2646 -CDROM_BLK_2647 = 2647 -CDROM_BLK_SUBCODE = 96 -CDROM_NORMAL_SPEED = 0x00 -CDROM_DOUBLE_SPEED = 0x01 -CDROM_QUAD_SPEED = 0x03 -CDROM_TWELVE_SPEED = 0x0C -CDROM_MAXIMUM_SPEED = 0xff -CDIOC = (0x04 << 8) -CDROMPAUSE = (CDIOC|151) -CDROMRESUME = (CDIOC|152) -CDROMPLAYMSF = (CDIOC|153) -CDROMPLAYTRKIND = (CDIOC|154) -CDROMREADTOCHDR = (CDIOC|155) -CDROMREADTOCENTRY = (CDIOC|156) -CDROMSTOP = (CDIOC|157) -CDROMSTART = (CDIOC|158) -CDROMEJECT = (CDIOC|159) -CDROMVOLCTRL = (CDIOC|160) -CDROMSUBCHNL = (CDIOC|161) -CDROMREADMODE2 = (CDIOC|162) -CDROMREADMODE1 = (CDIOC|163) -CDROMREADOFFSET = (CDIOC|164) -CDROMGBLKMODE = (CDIOC|165) -CDROMSBLKMODE = (CDIOC|166) -CDROMCDDA = (CDIOC|167) -CDROMCDXA = (CDIOC|168) -CDROMSUBCODE = (CDIOC|169) -CDROMGDRVSPEED = (CDIOC|170) -CDROMSDRVSPEED = (CDIOC|171) -SCMD_READ_TOC = 0x43 -SCMD_PLAYAUDIO_MSF = 0x47 -SCMD_PLAYAUDIO_TI = 0x48 -SCMD_PAUSE_RESUME = 0x4B -SCMD_READ_SUBCHANNEL = 0x42 -SCMD_PLAYAUDIO10 = 0x45 -SCMD_PLAYTRACK_REL10 = 0x49 -SCMD_READ_HEADER = 0x44 -SCMD_PLAYAUDIO12 = 0xA5 -SCMD_PLAYTRACK_REL12 = 0xA9 -SCMD_CD_PLAYBACK_CONTROL = 0xC9 -SCMD_CD_PLAYBACK_STATUS = 0xC4 -SCMD_READ_CDDA = 0xD8 -SCMD_READ_CDXA = 0xDB -SCMD_READ_ALL_SUBCODES = 0xDF -CDROM_MODE2_SIZE = 2336 diff --git a/Lib/plat-sunos5/DLFCN.py b/Lib/plat-sunos5/DLFCN.py deleted file mode 100644 --- a/Lib/plat-sunos5/DLFCN.py +++ /dev/null @@ -1,27 +0,0 @@ -# Generated by h2py from /usr/include/dlfcn.h -from TYPES import * -RTLD_LAZY = 0x00001 -RTLD_NOW = 0x00002 -RTLD_NOLOAD = 0x00004 -RTLD_GLOBAL = 0x00100 -RTLD_LOCAL = 0x00000 -RTLD_PARENT = 0x00200 -RTLD_GROUP = 0x00400 -RTLD_WORLD = 0x00800 -RTLD_NODELETE = 0x01000 -RTLD_CONFGEN = 0x10000 -RTLD_REL_RELATIVE = 0x00001 -RTLD_REL_EXEC = 0x00002 -RTLD_REL_DEPENDS = 0x00004 -RTLD_REL_PRELOAD = 0x00008 -RTLD_REL_SELF = 0x00010 -RTLD_REL_WEAK = 0x00020 -RTLD_REL_ALL = 0x00fff -RTLD_MEMORY = 0x01000 -RTLD_STRIP = 0x02000 -RTLD_NOHEAP = 0x04000 -RTLD_CONFSET = 0x10000 -RTLD_DI_LMID = 1 -RTLD_DI_LINKMAP = 2 -RTLD_DI_CONFIGADDR = 3 -RTLD_DI_MAX = 3 diff --git a/Lib/plat-sunos5/IN.py b/Lib/plat-sunos5/IN.py deleted file mode 100644 --- a/Lib/plat-sunos5/IN.py +++ /dev/null @@ -1,1421 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/feature_tests.h - -# Included from sys/isa_defs.h -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 8 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_ALIGNMENT_REQUIRED = 1 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 4 -_DOUBLE_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 4 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 4 -_ALIGNMENT_REQUIRED = 0 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_ALIGNMENT_REQUIRED = 1 -_LONG_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 8 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 8 -_LONG_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_POSIX_C_SOURCE = 1 -_LARGEFILE64_SOURCE = 1 -_LARGEFILE_SOURCE = 1 -_FILE_OFFSET_BITS = 64 -_FILE_OFFSET_BITS = 32 -_POSIX_C_SOURCE = 199506 -_POSIX_PTHREAD_SEMANTICS = 1 -_XOPEN_VERSION = 500 -_XOPEN_VERSION = 4 -_XOPEN_VERSION = 3 -from TYPES import * - -# Included from sys/stream.h - -# Included from sys/vnode.h -from TYPES import * - -# Included from sys/t_lock.h - -# Included from sys/machlock.h -from TYPES import * -LOCK_HELD_VALUE = 0xff -def SPIN_LOCK(pl): return ((pl) > ipltospl(LOCK_LEVEL)) - -def LOCK_SAMPLE_INTERVAL(i): return (((i) & 0xff) == 0) - -CLOCK_LEVEL = 10 -LOCK_LEVEL = 10 -DISP_LEVEL = (LOCK_LEVEL + 1) -PTR24_LSB = 5 -PTR24_MSB = (PTR24_LSB + 24) -PTR24_ALIGN = 32 -PTR24_BASE = 0xe0000000 - -# Included from sys/param.h -from TYPES import * -_POSIX_VDISABLE = 0 -MAX_INPUT = 512 -MAX_CANON = 256 -UID_NOBODY = 60001 -GID_NOBODY = UID_NOBODY -UID_NOACCESS = 60002 -MAX_TASKID = 999999 -MAX_MAXPID = 999999 -DEFAULT_MAXPID = 999999 -DEFAULT_JUMPPID = 100000 -DEFAULT_MAXPID = 30000 -DEFAULT_JUMPPID = 0 -MAXUID = 2147483647 -MAXPROJID = MAXUID -MAXLINK = 32767 -NMOUNT = 40 -CANBSIZ = 256 -NOFILE = 20 -NGROUPS_UMIN = 0 -NGROUPS_UMAX = 32 -NGROUPS_MAX_DEFAULT = 16 -NZERO = 20 -NULL = 0 -NULL = 0 -CMASK = 0o22 -CDLIMIT = (1<<11) -NBPS = 0x20000 -NBPSCTR = 512 -UBSIZE = 512 -SCTRSHFT = 9 -SYSNAME = 9 -PREMOTE = 39 -MAXPATHLEN = 1024 -MAXSYMLINKS = 20 -MAXNAMELEN = 256 -NADDR = 13 -PIPE_BUF = 5120 -PIPE_MAX = 5120 -NBBY = 8 -MAXBSIZE = 8192 -DEV_BSIZE = 512 -DEV_BSHIFT = 9 -MAXFRAG = 8 -MAXOFF32_T = 0x7fffffff -MAXOFF_T = 0x7fffffffffffffff -MAXOFFSET_T = 0x7fffffffffffffff -MAXOFF_T = 0x7fffffff -MAXOFFSET_T = 0x7fffffff -def btodb(bytes): return \ - -def dbtob(db): return \ - -def lbtodb(bytes): return \ - -def ldbtob(db): return \ - -NCARGS32 = 0x100000 -NCARGS64 = 0x200000 -NCARGS = NCARGS64 -NCARGS = NCARGS32 -FSHIFT = 8 -FSCALE = (1<> MMU_PAGESHIFT) - -def mmu_btopr(x): return ((((x) + MMU_PAGEOFFSET) >> MMU_PAGESHIFT)) - -def mmu_ptod(x): return ((x) << (MMU_PAGESHIFT - DEV_BSHIFT)) - -def ptod(x): return ((x) << (PAGESHIFT - DEV_BSHIFT)) - -def ptob(x): return ((x) << PAGESHIFT) - -def btop(x): return (((x) >> PAGESHIFT)) - -def btopr(x): return ((((x) + PAGEOFFSET) >> PAGESHIFT)) - -def dtop(DD): return (((DD) + NDPP - 1) >> (PAGESHIFT - DEV_BSHIFT)) - -def dtopt(DD): return ((DD) >> (PAGESHIFT - DEV_BSHIFT)) - -_AIO_LISTIO_MAX = (4096) -_AIO_MAX = (-1) -_MQ_OPEN_MAX = (32) -_MQ_PRIO_MAX = (32) -_SEM_NSEMS_MAX = INT_MAX -_SEM_VALUE_MAX = INT_MAX - -# Included from sys/unistd.h -_CS_PATH = 65 -_CS_LFS_CFLAGS = 68 -_CS_LFS_LDFLAGS = 69 -_CS_LFS_LIBS = 70 -_CS_LFS_LINTFLAGS = 71 -_CS_LFS64_CFLAGS = 72 -_CS_LFS64_LDFLAGS = 73 -_CS_LFS64_LIBS = 74 -_CS_LFS64_LINTFLAGS = 75 -_CS_XBS5_ILP32_OFF32_CFLAGS = 700 -_CS_XBS5_ILP32_OFF32_LDFLAGS = 701 -_CS_XBS5_ILP32_OFF32_LIBS = 702 -_CS_XBS5_ILP32_OFF32_LINTFLAGS = 703 -_CS_XBS5_ILP32_OFFBIG_CFLAGS = 705 -_CS_XBS5_ILP32_OFFBIG_LDFLAGS = 706 -_CS_XBS5_ILP32_OFFBIG_LIBS = 707 -_CS_XBS5_ILP32_OFFBIG_LINTFLAGS = 708 -_CS_XBS5_LP64_OFF64_CFLAGS = 709 -_CS_XBS5_LP64_OFF64_LDFLAGS = 710 -_CS_XBS5_LP64_OFF64_LIBS = 711 -_CS_XBS5_LP64_OFF64_LINTFLAGS = 712 -_CS_XBS5_LPBIG_OFFBIG_CFLAGS = 713 -_CS_XBS5_LPBIG_OFFBIG_LDFLAGS = 714 -_CS_XBS5_LPBIG_OFFBIG_LIBS = 715 -_CS_XBS5_LPBIG_OFFBIG_LINTFLAGS = 716 -_SC_ARG_MAX = 1 -_SC_CHILD_MAX = 2 -_SC_CLK_TCK = 3 -_SC_NGROUPS_MAX = 4 -_SC_OPEN_MAX = 5 -_SC_JOB_CONTROL = 6 -_SC_SAVED_IDS = 7 -_SC_VERSION = 8 -_SC_PASS_MAX = 9 -_SC_LOGNAME_MAX = 10 -_SC_PAGESIZE = 11 -_SC_XOPEN_VERSION = 12 -_SC_NPROCESSORS_CONF = 14 -_SC_NPROCESSORS_ONLN = 15 -_SC_STREAM_MAX = 16 -_SC_TZNAME_MAX = 17 -_SC_AIO_LISTIO_MAX = 18 -_SC_AIO_MAX = 19 -_SC_AIO_PRIO_DELTA_MAX = 20 -_SC_ASYNCHRONOUS_IO = 21 -_SC_DELAYTIMER_MAX = 22 -_SC_FSYNC = 23 -_SC_MAPPED_FILES = 24 -_SC_MEMLOCK = 25 -_SC_MEMLOCK_RANGE = 26 -_SC_MEMORY_PROTECTION = 27 -_SC_MESSAGE_PASSING = 28 -_SC_MQ_OPEN_MAX = 29 -_SC_MQ_PRIO_MAX = 30 -_SC_PRIORITIZED_IO = 31 -_SC_PRIORITY_SCHEDULING = 32 -_SC_REALTIME_SIGNALS = 33 -_SC_RTSIG_MAX = 34 -_SC_SEMAPHORES = 35 -_SC_SEM_NSEMS_MAX = 36 -_SC_SEM_VALUE_MAX = 37 -_SC_SHARED_MEMORY_OBJECTS = 38 -_SC_SIGQUEUE_MAX = 39 -_SC_SIGRT_MIN = 40 -_SC_SIGRT_MAX = 41 -_SC_SYNCHRONIZED_IO = 42 -_SC_TIMERS = 43 -_SC_TIMER_MAX = 44 -_SC_2_C_BIND = 45 -_SC_2_C_DEV = 46 -_SC_2_C_VERSION = 47 -_SC_2_FORT_DEV = 48 -_SC_2_FORT_RUN = 49 -_SC_2_LOCALEDEF = 50 -_SC_2_SW_DEV = 51 -_SC_2_UPE = 52 -_SC_2_VERSION = 53 -_SC_BC_BASE_MAX = 54 -_SC_BC_DIM_MAX = 55 -_SC_BC_SCALE_MAX = 56 -_SC_BC_STRING_MAX = 57 -_SC_COLL_WEIGHTS_MAX = 58 -_SC_EXPR_NEST_MAX = 59 -_SC_LINE_MAX = 60 -_SC_RE_DUP_MAX = 61 -_SC_XOPEN_CRYPT = 62 -_SC_XOPEN_ENH_I18N = 63 -_SC_XOPEN_SHM = 64 -_SC_2_CHAR_TERM = 66 -_SC_XOPEN_XCU_VERSION = 67 -_SC_ATEXIT_MAX = 76 -_SC_IOV_MAX = 77 -_SC_XOPEN_UNIX = 78 -_SC_PAGE_SIZE = _SC_PAGESIZE -_SC_T_IOV_MAX = 79 -_SC_PHYS_PAGES = 500 -_SC_AVPHYS_PAGES = 501 -_SC_COHER_BLKSZ = 503 -_SC_SPLIT_CACHE = 504 -_SC_ICACHE_SZ = 505 -_SC_DCACHE_SZ = 506 -_SC_ICACHE_LINESZ = 507 -_SC_DCACHE_LINESZ = 508 -_SC_ICACHE_BLKSZ = 509 -_SC_DCACHE_BLKSZ = 510 -_SC_DCACHE_TBLKSZ = 511 -_SC_ICACHE_ASSOC = 512 -_SC_DCACHE_ASSOC = 513 -_SC_MAXPID = 514 -_SC_STACK_PROT = 515 -_SC_THREAD_DESTRUCTOR_ITERATIONS = 568 -_SC_GETGR_R_SIZE_MAX = 569 -_SC_GETPW_R_SIZE_MAX = 570 -_SC_LOGIN_NAME_MAX = 571 -_SC_THREAD_KEYS_MAX = 572 -_SC_THREAD_STACK_MIN = 573 -_SC_THREAD_THREADS_MAX = 574 -_SC_TTY_NAME_MAX = 575 -_SC_THREADS = 576 -_SC_THREAD_ATTR_STACKADDR = 577 -_SC_THREAD_ATTR_STACKSIZE = 578 -_SC_THREAD_PRIORITY_SCHEDULING = 579 -_SC_THREAD_PRIO_INHERIT = 580 -_SC_THREAD_PRIO_PROTECT = 581 -_SC_THREAD_PROCESS_SHARED = 582 -_SC_THREAD_SAFE_FUNCTIONS = 583 -_SC_XOPEN_LEGACY = 717 -_SC_XOPEN_REALTIME = 718 -_SC_XOPEN_REALTIME_THREADS = 719 -_SC_XBS5_ILP32_OFF32 = 720 -_SC_XBS5_ILP32_OFFBIG = 721 -_SC_XBS5_LP64_OFF64 = 722 -_SC_XBS5_LPBIG_OFFBIG = 723 -_PC_LINK_MAX = 1 -_PC_MAX_CANON = 2 -_PC_MAX_INPUT = 3 -_PC_NAME_MAX = 4 -_PC_PATH_MAX = 5 -_PC_PIPE_BUF = 6 -_PC_NO_TRUNC = 7 -_PC_VDISABLE = 8 -_PC_CHOWN_RESTRICTED = 9 -_PC_ASYNC_IO = 10 -_PC_PRIO_IO = 11 -_PC_SYNC_IO = 12 -_PC_FILESIZEBITS = 67 -_PC_LAST = 67 -_POSIX_VERSION = 199506 -_POSIX2_VERSION = 199209 -_POSIX2_C_VERSION = 199209 -_XOPEN_XCU_VERSION = 4 -_XOPEN_REALTIME = 1 -_XOPEN_ENH_I18N = 1 -_XOPEN_SHM = 1 -_POSIX2_C_BIND = 1 -_POSIX2_CHAR_TERM = 1 -_POSIX2_LOCALEDEF = 1 -_POSIX2_C_DEV = 1 -_POSIX2_SW_DEV = 1 -_POSIX2_UPE = 1 - -# Included from sys/mutex.h -from TYPES import * -def MUTEX_HELD(x): return (mutex_owned(x)) - - -# Included from sys/rwlock.h -from TYPES import * -def RW_READ_HELD(x): return (rw_read_held((x))) - -def RW_WRITE_HELD(x): return (rw_write_held((x))) - -def RW_LOCK_HELD(x): return (rw_lock_held((x))) - -def RW_ISWRITER(x): return (rw_iswriter(x)) - - -# Included from sys/semaphore.h - -# Included from sys/thread.h -from TYPES import * - -# Included from sys/klwp.h -from TYPES import * - -# Included from sys/condvar.h -from TYPES import * - -# Included from sys/time.h - -# Included from sys/types32.h - -# Included from sys/int_types.h -TIME32_MAX = INT32_MAX -TIME32_MIN = INT32_MIN -def TIMEVAL_OVERFLOW(tv): return \ - -from TYPES import * -DST_NONE = 0 -DST_USA = 1 -DST_AUST = 2 -DST_WET = 3 -DST_MET = 4 -DST_EET = 5 -DST_CAN = 6 -DST_GB = 7 -DST_RUM = 8 -DST_TUR = 9 -DST_AUSTALT = 10 -ITIMER_REAL = 0 -ITIMER_VIRTUAL = 1 -ITIMER_PROF = 2 -ITIMER_REALPROF = 3 -def ITIMERVAL_OVERFLOW(itv): return \ - -SEC = 1 -MILLISEC = 1000 -MICROSEC = 1000000 -NANOSEC = 1000000000 - -# Included from sys/time_impl.h -def TIMESPEC_OVERFLOW(ts): return \ - -def ITIMERSPEC_OVERFLOW(it): return \ - -__CLOCK_REALTIME0 = 0 -CLOCK_VIRTUAL = 1 -CLOCK_PROF = 2 -__CLOCK_REALTIME3 = 3 -CLOCK_HIGHRES = 4 -CLOCK_MAX = 5 -CLOCK_REALTIME = __CLOCK_REALTIME3 -CLOCK_REALTIME = __CLOCK_REALTIME0 -TIMER_RELTIME = 0x0 -TIMER_ABSTIME = 0x1 -def TICK_TO_SEC(tick): return ((tick) / hz) - -def SEC_TO_TICK(sec): return ((sec) * hz) - -def TICK_TO_MSEC(tick): return \ - -def MSEC_TO_TICK(msec): return \ - -def MSEC_TO_TICK_ROUNDUP(msec): return \ - -def TICK_TO_USEC(tick): return ((tick) * usec_per_tick) - -def USEC_TO_TICK(usec): return ((usec) / usec_per_tick) - -def USEC_TO_TICK_ROUNDUP(usec): return \ - -def TICK_TO_NSEC(tick): return ((tick) * nsec_per_tick) - -def NSEC_TO_TICK(nsec): return ((nsec) / nsec_per_tick) - -def NSEC_TO_TICK_ROUNDUP(nsec): return \ - -def TIMEVAL_TO_TICK(tvp): return \ - -def TIMESTRUC_TO_TICK(tsp): return \ - - -# Included from time.h -from TYPES import * - -# Included from iso/time_iso.h -NULL = 0 -NULL = 0 -CLOCKS_PER_SEC = 1000000 - -# Included from sys/select.h -FD_SETSIZE = 65536 -FD_SETSIZE = 1024 -_NBBY = 8 -NBBY = _NBBY -def FD_ZERO(p): return bzero((p), sizeof (*(p))) - - -# Included from sys/signal.h - -# Included from sys/iso/signal_iso.h -SIGHUP = 1 -SIGINT = 2 -SIGQUIT = 3 -SIGILL = 4 -SIGTRAP = 5 -SIGIOT = 6 -SIGABRT = 6 -SIGEMT = 7 -SIGFPE = 8 -SIGKILL = 9 -SIGBUS = 10 -SIGSEGV = 11 -SIGSYS = 12 -SIGPIPE = 13 -SIGALRM = 14 -SIGTERM = 15 -SIGUSR1 = 16 -SIGUSR2 = 17 -SIGCLD = 18 -SIGCHLD = 18 -SIGPWR = 19 -SIGWINCH = 20 -SIGURG = 21 -SIGPOLL = 22 -SIGIO = SIGPOLL -SIGSTOP = 23 -SIGTSTP = 24 -SIGCONT = 25 -SIGTTIN = 26 -SIGTTOU = 27 -SIGVTALRM = 28 -SIGPROF = 29 -SIGXCPU = 30 -SIGXFSZ = 31 -SIGWAITING = 32 -SIGLWP = 33 -SIGFREEZE = 34 -SIGTHAW = 35 -SIGCANCEL = 36 -SIGLOST = 37 -_SIGRTMIN = 38 -_SIGRTMAX = 45 -SIG_BLOCK = 1 -SIG_UNBLOCK = 2 -SIG_SETMASK = 3 -SIGNO_MASK = 0xFF -SIGDEFER = 0x100 -SIGHOLD = 0x200 -SIGRELSE = 0x400 -SIGIGNORE = 0x800 -SIGPAUSE = 0x1000 - -# Included from sys/siginfo.h -from TYPES import * -SIGEV_NONE = 1 -SIGEV_SIGNAL = 2 -SIGEV_THREAD = 3 -SI_NOINFO = 32767 -SI_USER = 0 -SI_LWP = (-1) -SI_QUEUE = (-2) -SI_TIMER = (-3) -SI_ASYNCIO = (-4) -SI_MESGQ = (-5) - -# Included from sys/machsig.h -ILL_ILLOPC = 1 -ILL_ILLOPN = 2 -ILL_ILLADR = 3 -ILL_ILLTRP = 4 -ILL_PRVOPC = 5 -ILL_PRVREG = 6 -ILL_COPROC = 7 -ILL_BADSTK = 8 -NSIGILL = 8 -EMT_TAGOVF = 1 -EMT_CPCOVF = 2 -NSIGEMT = 2 -FPE_INTDIV = 1 -FPE_INTOVF = 2 -FPE_FLTDIV = 3 -FPE_FLTOVF = 4 -FPE_FLTUND = 5 -FPE_FLTRES = 6 -FPE_FLTINV = 7 -FPE_FLTSUB = 8 -NSIGFPE = 8 -SEGV_MAPERR = 1 -SEGV_ACCERR = 2 -NSIGSEGV = 2 -BUS_ADRALN = 1 -BUS_ADRERR = 2 -BUS_OBJERR = 3 -NSIGBUS = 3 -TRAP_BRKPT = 1 -TRAP_TRACE = 2 -TRAP_RWATCH = 3 -TRAP_WWATCH = 4 -TRAP_XWATCH = 5 -NSIGTRAP = 5 -CLD_EXITED = 1 -CLD_KILLED = 2 -CLD_DUMPED = 3 -CLD_TRAPPED = 4 -CLD_STOPPED = 5 -CLD_CONTINUED = 6 -NSIGCLD = 6 -POLL_IN = 1 -POLL_OUT = 2 -POLL_MSG = 3 -POLL_ERR = 4 -POLL_PRI = 5 -POLL_HUP = 6 -NSIGPOLL = 6 -PROF_SIG = 1 -NSIGPROF = 1 -SI_MAXSZ = 256 -SI_MAXSZ = 128 - -# Included from sys/time_std_impl.h -from TYPES import * -SI32_MAXSZ = 128 -def SI_CANQUEUE(c): return ((c) <= SI_QUEUE) - -SA_NOCLDSTOP = 0x00020000 -SA_ONSTACK = 0x00000001 -SA_RESETHAND = 0x00000002 -SA_RESTART = 0x00000004 -SA_SIGINFO = 0x00000008 -SA_NODEFER = 0x00000010 -SA_NOCLDWAIT = 0x00010000 -SA_WAITSIG = 0x00010000 -NSIG = 46 -MAXSIG = 45 -S_SIGNAL = 1 -S_SIGSET = 2 -S_SIGACTION = 3 -S_NONE = 4 -MINSIGSTKSZ = 2048 -SIGSTKSZ = 8192 -SS_ONSTACK = 0x00000001 -SS_DISABLE = 0x00000002 -SN_PROC = 1 -SN_CANCEL = 2 -SN_SEND = 3 - -# Included from sys/ucontext.h -from TYPES import * - -# Included from sys/regset.h -REG_CCR = (0) -REG_PSR = (0) -REG_PSR = (0) -REG_PC = (1) -REG_nPC = (2) -REG_Y = (3) -REG_G1 = (4) -REG_G2 = (5) -REG_G3 = (6) -REG_G4 = (7) -REG_G5 = (8) -REG_G6 = (9) -REG_G7 = (10) -REG_O0 = (11) -REG_O1 = (12) -REG_O2 = (13) -REG_O3 = (14) -REG_O4 = (15) -REG_O5 = (16) -REG_O6 = (17) -REG_O7 = (18) -REG_ASI = (19) -REG_FPRS = (20) -REG_PS = REG_PSR -REG_SP = REG_O6 -REG_R0 = REG_O0 -REG_R1 = REG_O1 -_NGREG = 21 -_NGREG = 19 -NGREG = _NGREG -_NGREG32 = 19 -_NGREG64 = 21 -SPARC_MAXREGWINDOW = 31 -MAXFPQ = 16 -XRS_ID = 0x78727300 - -# Included from v7/sys/privregs.h - -# Included from v7/sys/psr.h -PSR_CWP = 0x0000001F -PSR_ET = 0x00000020 -PSR_PS = 0x00000040 -PSR_S = 0x00000080 -PSR_PIL = 0x00000F00 -PSR_EF = 0x00001000 -PSR_EC = 0x00002000 -PSR_RSV = 0x000FC000 -PSR_ICC = 0x00F00000 -PSR_C = 0x00100000 -PSR_V = 0x00200000 -PSR_Z = 0x00400000 -PSR_N = 0x00800000 -PSR_VER = 0x0F000000 -PSR_IMPL = 0xF0000000 -PSL_ALLCC = PSR_ICC -PSL_USER = (PSR_S) -PSL_USERMASK = (PSR_ICC) -PSL_UBITS = (PSR_ICC|PSR_EF) -def USERMODE(ps): return (((ps) & PSR_PS) == 0) - - -# Included from sys/fsr.h -FSR_CEXC = 0x0000001f -FSR_AEXC = 0x000003e0 -FSR_FCC = 0x00000c00 -FSR_PR = 0x00001000 -FSR_QNE = 0x00002000 -FSR_FTT = 0x0001c000 -FSR_VER = 0x000e0000 -FSR_TEM = 0x0f800000 -FSR_RP = 0x30000000 -FSR_RD = 0xc0000000 -FSR_VER_SHIFT = 17 -FSR_FCC1 = 0x00000003 -FSR_FCC2 = 0x0000000C -FSR_FCC3 = 0x00000030 -FSR_CEXC_NX = 0x00000001 -FSR_CEXC_DZ = 0x00000002 -FSR_CEXC_UF = 0x00000004 -FSR_CEXC_OF = 0x00000008 -FSR_CEXC_NV = 0x00000010 -FSR_AEXC_NX = (0x1 << 5) -FSR_AEXC_DZ = (0x2 << 5) -FSR_AEXC_UF = (0x4 << 5) -FSR_AEXC_OF = (0x8 << 5) -FSR_AEXC_NV = (0x10 << 5) -FTT_NONE = 0 -FTT_IEEE = 1 -FTT_UNFIN = 2 -FTT_UNIMP = 3 -FTT_SEQ = 4 -FTT_ALIGN = 5 -FTT_DFAULT = 6 -FSR_FTT_SHIFT = 14 -FSR_FTT_IEEE = (FTT_IEEE << FSR_FTT_SHIFT) -FSR_FTT_UNFIN = (FTT_UNFIN << FSR_FTT_SHIFT) -FSR_FTT_UNIMP = (FTT_UNIMP << FSR_FTT_SHIFT) -FSR_FTT_SEQ = (FTT_SEQ << FSR_FTT_SHIFT) -FSR_FTT_ALIGN = (FTT_ALIGN << FSR_FTT_SHIFT) -FSR_FTT_DFAULT = (FTT_DFAULT << FSR_FTT_SHIFT) -FSR_TEM_NX = (0x1 << 23) -FSR_TEM_DZ = (0x2 << 23) -FSR_TEM_UF = (0x4 << 23) -FSR_TEM_OF = (0x8 << 23) -FSR_TEM_NV = (0x10 << 23) -RP_DBLEXT = 0 -RP_SINGLE = 1 -RP_DOUBLE = 2 -RP_RESERVED = 3 -RD_NEAR = 0 -RD_ZER0 = 1 -RD_POSINF = 2 -RD_NEGINF = 3 -FPRS_DL = 0x1 -FPRS_DU = 0x2 -FPRS_FEF = 0x4 -PIL_MAX = 0xf -def SAVE_GLOBALS(RP): return \ - -def RESTORE_GLOBALS(RP): return \ - -def SAVE_OUTS(RP): return \ - -def RESTORE_OUTS(RP): return \ - -def SAVE_WINDOW(SBP): return \ - -def RESTORE_WINDOW(SBP): return \ - -def STORE_FPREGS(FP): return \ - -def LOAD_FPREGS(FP): return \ - -_SPARC_MAXREGWINDOW = 31 -_XRS_ID = 0x78727300 -GETCONTEXT = 0 -SETCONTEXT = 1 -UC_SIGMASK = 0o01 -UC_STACK = 0o02 -UC_CPU = 0o04 -UC_MAU = 0o10 -UC_FPU = UC_MAU -UC_INTR = 0o20 -UC_ASR = 0o40 -UC_MCONTEXT = (UC_CPU|UC_FPU|UC_ASR) -UC_ALL = (UC_SIGMASK|UC_STACK|UC_MCONTEXT) -_SIGQUEUE_MAX = 32 -_SIGNOTIFY_MAX = 32 - -# Included from sys/pcb.h -INSTR_VALID = 0x02 -NORMAL_STEP = 0x04 -WATCH_STEP = 0x08 -CPC_OVERFLOW = 0x10 -ASYNC_HWERR = 0x20 -STEP_NONE = 0 -STEP_REQUESTED = 1 -STEP_ACTIVE = 2 -STEP_WASACTIVE = 3 - -# Included from sys/msacct.h -LMS_USER = 0 -LMS_SYSTEM = 1 -LMS_TRAP = 2 -LMS_TFAULT = 3 -LMS_DFAULT = 4 -LMS_KFAULT = 5 -LMS_USER_LOCK = 6 -LMS_SLEEP = 7 -LMS_WAIT_CPU = 8 -LMS_STOPPED = 9 -NMSTATES = 10 - -# Included from sys/lwp.h - -# Included from sys/synch.h -from TYPES import * -USYNC_THREAD = 0x00 -USYNC_PROCESS = 0x01 -LOCK_NORMAL = 0x00 -LOCK_ERRORCHECK = 0x02 -LOCK_RECURSIVE = 0x04 -USYNC_PROCESS_ROBUST = 0x08 -LOCK_PRIO_NONE = 0x00 -LOCK_PRIO_INHERIT = 0x10 -LOCK_PRIO_PROTECT = 0x20 -LOCK_STALL_NP = 0x00 -LOCK_ROBUST_NP = 0x40 -LOCK_OWNERDEAD = 0x1 -LOCK_NOTRECOVERABLE = 0x2 -LOCK_INITED = 0x4 -LOCK_UNMAPPED = 0x8 -LWP_DETACHED = 0x00000040 -LWP_SUSPENDED = 0x00000080 -__LWP_ASLWP = 0x00000100 -MAXSYSARGS = 8 -NORMALRETURN = 0 -JUSTRETURN = 1 -LWP_USER = 0x01 -LWP_SYS = 0x02 -TS_FREE = 0x00 -TS_SLEEP = 0x01 -TS_RUN = 0x02 -TS_ONPROC = 0x04 -TS_ZOMB = 0x08 -TS_STOPPED = 0x10 -T_INTR_THREAD = 0x0001 -T_WAKEABLE = 0x0002 -T_TOMASK = 0x0004 -T_TALLOCSTK = 0x0008 -T_WOULDBLOCK = 0x0020 -T_DONTBLOCK = 0x0040 -T_DONTPEND = 0x0080 -T_SYS_PROF = 0x0100 -T_WAITCVSEM = 0x0200 -T_WATCHPT = 0x0400 -T_PANIC = 0x0800 -TP_HOLDLWP = 0x0002 -TP_TWAIT = 0x0004 -TP_LWPEXIT = 0x0008 -TP_PRSTOP = 0x0010 -TP_CHKPT = 0x0020 -TP_EXITLWP = 0x0040 -TP_PRVSTOP = 0x0080 -TP_MSACCT = 0x0100 -TP_STOPPING = 0x0200 -TP_WATCHPT = 0x0400 -TP_PAUSE = 0x0800 -TP_CHANGEBIND = 0x1000 -TS_LOAD = 0x0001 -TS_DONT_SWAP = 0x0002 -TS_SWAPENQ = 0x0004 -TS_ON_SWAPQ = 0x0008 -TS_CSTART = 0x0100 -TS_UNPAUSE = 0x0200 -TS_XSTART = 0x0400 -TS_PSTART = 0x0800 -TS_RESUME = 0x1000 -TS_CREATE = 0x2000 -TS_ALLSTART = \ - (TS_CSTART|TS_UNPAUSE|TS_XSTART|TS_PSTART|TS_RESUME|TS_CREATE) -def CPR_VSTOPPED(t): return \ - -def THREAD_TRANSITION(tp): return thread_transition(tp); - -def THREAD_STOP(tp): return \ - -def THREAD_ZOMB(tp): return THREAD_SET_STATE(tp, TS_ZOMB, NULL) - -def SEMA_HELD(x): return (sema_held((x))) - -NO_LOCKS_HELD = 1 -NO_COMPETING_THREADS = 1 - -# Included from sys/cred.h - -# Included from sys/uio.h -from TYPES import * - -# Included from sys/resource.h -from TYPES import * -PRIO_PROCESS = 0 -PRIO_PGRP = 1 -PRIO_USER = 2 -RLIMIT_CPU = 0 -RLIMIT_FSIZE = 1 -RLIMIT_DATA = 2 -RLIMIT_STACK = 3 -RLIMIT_CORE = 4 -RLIMIT_NOFILE = 5 -RLIMIT_VMEM = 6 -RLIMIT_AS = RLIMIT_VMEM -RLIM_NLIMITS = 7 -RLIM_INFINITY = (-3) -RLIM_SAVED_MAX = (-2) -RLIM_SAVED_CUR = (-1) -RLIM_INFINITY = 0x7fffffff -RLIM_SAVED_MAX = 0x7ffffffe -RLIM_SAVED_CUR = 0x7ffffffd -RLIM32_INFINITY = 0x7fffffff -RLIM32_SAVED_MAX = 0x7ffffffe -RLIM32_SAVED_CUR = 0x7ffffffd - -# Included from sys/model.h - -# Included from sys/debug.h -def ASSERT64(x): return ASSERT(x) - -def ASSERT32(x): return ASSERT(x) - -DATAMODEL_MASK = 0x0FF00000 -DATAMODEL_ILP32 = 0x00100000 -DATAMODEL_LP64 = 0x00200000 -DATAMODEL_NONE = 0 -DATAMODEL_NATIVE = DATAMODEL_LP64 -DATAMODEL_NATIVE = DATAMODEL_ILP32 -def STRUCT_SIZE(handle): return \ - -def STRUCT_BUF(handle): return ((handle).ptr.m64) - -def SIZEOF_PTR(umodel): return \ - -def STRUCT_SIZE(handle): return (sizeof (*(handle).ptr)) - -def STRUCT_BUF(handle): return ((handle).ptr) - -def SIZEOF_PTR(umodel): return sizeof (caddr_t) - -def lwp_getdatamodel(t): return DATAMODEL_ILP32 - -RUSAGE_SELF = 0 -RUSAGE_CHILDREN = -1 - -# Included from vm/seg_enum.h - -# Included from sys/buf.h - -# Included from sys/kstat.h -from TYPES import * -KSTAT_STRLEN = 31 -def KSTAT_ENTER(k): return \ - -def KSTAT_EXIT(k): return \ - -KSTAT_TYPE_RAW = 0 -KSTAT_TYPE_NAMED = 1 -KSTAT_TYPE_INTR = 2 -KSTAT_TYPE_IO = 3 -KSTAT_TYPE_TIMER = 4 -KSTAT_NUM_TYPES = 5 -KSTAT_FLAG_VIRTUAL = 0x01 -KSTAT_FLAG_VAR_SIZE = 0x02 -KSTAT_FLAG_WRITABLE = 0x04 -KSTAT_FLAG_PERSISTENT = 0x08 -KSTAT_FLAG_DORMANT = 0x10 -KSTAT_FLAG_INVALID = 0x20 -KSTAT_READ = 0 -KSTAT_WRITE = 1 -KSTAT_DATA_CHAR = 0 -KSTAT_DATA_INT32 = 1 -KSTAT_DATA_UINT32 = 2 -KSTAT_DATA_INT64 = 3 -KSTAT_DATA_UINT64 = 4 -KSTAT_DATA_LONG = KSTAT_DATA_INT32 -KSTAT_DATA_ULONG = KSTAT_DATA_UINT32 -KSTAT_DATA_LONG = KSTAT_DATA_INT64 -KSTAT_DATA_ULONG = KSTAT_DATA_UINT64 -KSTAT_DATA_LONG = 7 -KSTAT_DATA_ULONG = 8 -KSTAT_DATA_LONGLONG = KSTAT_DATA_INT64 -KSTAT_DATA_ULONGLONG = KSTAT_DATA_UINT64 -KSTAT_DATA_FLOAT = 5 -KSTAT_DATA_DOUBLE = 6 -KSTAT_INTR_HARD = 0 -KSTAT_INTR_SOFT = 1 -KSTAT_INTR_WATCHDOG = 2 -KSTAT_INTR_SPURIOUS = 3 -KSTAT_INTR_MULTSVC = 4 -KSTAT_NUM_INTRS = 5 -B_BUSY = 0x0001 -B_DONE = 0x0002 -B_ERROR = 0x0004 -B_PAGEIO = 0x0010 -B_PHYS = 0x0020 -B_READ = 0x0040 -B_WRITE = 0x0100 -B_KERNBUF = 0x0008 -B_WANTED = 0x0080 -B_AGE = 0x000200 -B_ASYNC = 0x000400 -B_DELWRI = 0x000800 -B_STALE = 0x001000 -B_DONTNEED = 0x002000 -B_REMAPPED = 0x004000 -B_FREE = 0x008000 -B_INVAL = 0x010000 -B_FORCE = 0x020000 -B_HEAD = 0x040000 -B_NOCACHE = 0x080000 -B_TRUNC = 0x100000 -B_SHADOW = 0x200000 -B_RETRYWRI = 0x400000 -def notavail(bp): return \ - -def BWRITE(bp): return \ - -def BWRITE2(bp): return \ - -VROOT = 0x01 -VNOCACHE = 0x02 -VNOMAP = 0x04 -VDUP = 0x08 -VNOSWAP = 0x10 -VNOMOUNT = 0x20 -VISSWAP = 0x40 -VSWAPLIKE = 0x80 -VVFSLOCK = 0x100 -VVFSWAIT = 0x200 -VVMLOCK = 0x400 -VDIROPEN = 0x800 -VVMEXEC = 0x1000 -VPXFS = 0x2000 -AT_TYPE = 0x0001 -AT_MODE = 0x0002 -AT_UID = 0x0004 -AT_GID = 0x0008 -AT_FSID = 0x0010 -AT_NODEID = 0x0020 -AT_NLINK = 0x0040 -AT_SIZE = 0x0080 -AT_ATIME = 0x0100 -AT_MTIME = 0x0200 -AT_CTIME = 0x0400 -AT_RDEV = 0x0800 -AT_BLKSIZE = 0x1000 -AT_NBLOCKS = 0x2000 -AT_VCODE = 0x4000 -AT_ALL = (AT_TYPE|AT_MODE|AT_UID|AT_GID|AT_FSID|AT_NODEID|\ - AT_NLINK|AT_SIZE|AT_ATIME|AT_MTIME|AT_CTIME|\ - AT_RDEV|AT_BLKSIZE|AT_NBLOCKS|AT_VCODE) -AT_STAT = (AT_MODE|AT_UID|AT_GID|AT_FSID|AT_NODEID|AT_NLINK|\ - AT_SIZE|AT_ATIME|AT_MTIME|AT_CTIME|AT_RDEV) -AT_TIMES = (AT_ATIME|AT_MTIME|AT_CTIME) -AT_NOSET = (AT_NLINK|AT_RDEV|AT_FSID|AT_NODEID|AT_TYPE|\ - AT_BLKSIZE|AT_NBLOCKS|AT_VCODE) -VSUID = 0o4000 -VSGID = 0o2000 -VSVTX = 0o1000 -VREAD = 0o0400 -VWRITE = 0o0200 -VEXEC = 0o0100 -MODEMASK = 0o7777 -PERMMASK = 0o0777 -def MANDMODE(mode): return (((mode) & (VSGID|(VEXEC>>3))) == VSGID) - -VSA_ACL = 0x0001 -VSA_ACLCNT = 0x0002 -VSA_DFACL = 0x0004 -VSA_DFACLCNT = 0x0008 -LOOKUP_DIR = 0x01 -DUMP_ALLOC = 0 -DUMP_FREE = 1 -DUMP_SCAN = 2 -ATTR_UTIME = 0x01 -ATTR_EXEC = 0x02 -ATTR_COMM = 0x04 -ATTR_HINT = 0x08 -ATTR_REAL = 0x10 - -# Included from sys/poll.h -POLLIN = 0x0001 -POLLPRI = 0x0002 -POLLOUT = 0x0004 -POLLRDNORM = 0x0040 -POLLWRNORM = POLLOUT -POLLRDBAND = 0x0080 -POLLWRBAND = 0x0100 -POLLNORM = POLLRDNORM -POLLERR = 0x0008 -POLLHUP = 0x0010 -POLLNVAL = 0x0020 -POLLREMOVE = 0x0800 -POLLRDDATA = 0x0200 -POLLNOERR = 0x0400 -POLLCLOSED = 0x8000 - -# Included from sys/strmdep.h -def str_aligned(X): return (((ulong_t)(X) & (sizeof (int) - 1)) == 0) - - -# Included from sys/strft.h -tdelta_t_sz = 12 -FTEV_MASK = 0x1FFF -FTEV_ISWR = 0x8000 -FTEV_CS = 0x4000 -FTEV_PS = 0x2000 -FTEV_QMASK = 0x1F00 -FTEV_ALLOCMASK = 0x1FF8 -FTEV_ALLOCB = 0x0000 -FTEV_ESBALLOC = 0x0001 -FTEV_DESBALLOC = 0x0002 -FTEV_ESBALLOCA = 0x0003 -FTEV_DESBALLOCA = 0x0004 -FTEV_ALLOCBIG = 0x0005 -FTEV_ALLOCBW = 0x0006 -FTEV_FREEB = 0x0008 -FTEV_DUPB = 0x0009 -FTEV_COPYB = 0x000A -FTEV_CALLER = 0x000F -FTEV_PUT = 0x0100 -FTEV_FSYNCQ = 0x0103 -FTEV_DSYNCQ = 0x0104 -FTEV_PUTQ = 0x0105 -FTEV_GETQ = 0x0106 -FTEV_RMVQ = 0x0107 -FTEV_INSQ = 0x0108 -FTEV_PUTBQ = 0x0109 -FTEV_FLUSHQ = 0x010A -FTEV_REPLYQ = 0x010B -FTEV_PUTNEXT = 0x010D -FTEV_RWNEXT = 0x010E -FTEV_QWINNER = 0x010F -FTEV_GEWRITE = 0x0101 -def FTFLW_HASH(h): return (((unsigned)(h))%ftflw_hash_sz) - -FTBLK_EVNTS = 0x9 -QENAB = 0x00000001 -QWANTR = 0x00000002 -QWANTW = 0x00000004 -QFULL = 0x00000008 -QREADR = 0x00000010 -QUSE = 0x00000020 -QNOENB = 0x00000040 -QBACK = 0x00000100 -QHLIST = 0x00000200 -QPAIR = 0x00000800 -QPERQ = 0x00001000 -QPERMOD = 0x00002000 -QMTSAFE = 0x00004000 -QMTOUTPERIM = 0x00008000 -QMT_TYPEMASK = (QPAIR|QPERQ|QPERMOD|QMTSAFE|QMTOUTPERIM) -QINSERVICE = 0x00010000 -QWCLOSE = 0x00020000 -QEND = 0x00040000 -QWANTWSYNC = 0x00080000 -QSYNCSTR = 0x00100000 -QISDRV = 0x00200000 -QHOT = 0x00400000 -QNEXTHOT = 0x00800000 -_QINSERTING = 0x04000000 -_QREMOVING = 0x08000000 -Q_SQQUEUED = 0x01 -Q_SQDRAINING = 0x02 -QB_FULL = 0x01 -QB_WANTW = 0x02 -QB_BACK = 0x04 -NBAND = 256 -STRUIOT_NONE = -1 -STRUIOT_DONTCARE = 0 -STRUIOT_STANDARD = 1 -STRUIOT_IP = 2 -DBLK_REFMIN = 0x01 -STRUIO_SPEC = 0x01 -STRUIO_DONE = 0x02 -STRUIO_IP = 0x04 -STRUIO_ZC = 0x08 -STRUIO_ICK = 0x10 -MSGMARK = 0x01 -MSGNOLOOP = 0x02 -MSGDELIM = 0x04 -MSGNOGET = 0x08 -MSGMARKNEXT = 0x10 -MSGNOTMARKNEXT = 0x20 -M_DATA = 0x00 -M_PROTO = 0x01 -M_BREAK = 0x08 -M_PASSFP = 0x09 -M_EVENT = 0x0a -M_SIG = 0x0b -M_DELAY = 0x0c -M_CTL = 0x0d -M_IOCTL = 0x0e -M_SETOPTS = 0x10 -M_RSE = 0x11 -M_IOCACK = 0x81 -M_IOCNAK = 0x82 -M_PCPROTO = 0x83 -M_PCSIG = 0x84 -M_READ = 0x85 -M_FLUSH = 0x86 -M_STOP = 0x87 -M_START = 0x88 -M_HANGUP = 0x89 -M_ERROR = 0x8a -M_COPYIN = 0x8b -M_COPYOUT = 0x8c -M_IOCDATA = 0x8d -M_PCRSE = 0x8e -M_STOPI = 0x8f -M_STARTI = 0x90 -M_PCEVENT = 0x91 -M_UNHANGUP = 0x92 -QNORM = 0x00 -QPCTL = 0x80 -IOC_MODELS = DATAMODEL_MASK -IOC_ILP32 = DATAMODEL_ILP32 -IOC_LP64 = DATAMODEL_LP64 -IOC_NATIVE = DATAMODEL_NATIVE -IOC_NONE = DATAMODEL_NONE -STRCANON = 0x01 -RECOPY = 0x02 -SO_ALL = 0x003f -SO_READOPT = 0x0001 -SO_WROFF = 0x0002 -SO_MINPSZ = 0x0004 -SO_MAXPSZ = 0x0008 -SO_HIWAT = 0x0010 -SO_LOWAT = 0x0020 -SO_MREADON = 0x0040 -SO_MREADOFF = 0x0080 -SO_NDELON = 0x0100 -SO_NDELOFF = 0x0200 -SO_ISTTY = 0x0400 -SO_ISNTTY = 0x0800 -SO_TOSTOP = 0x1000 -SO_TONSTOP = 0x2000 -SO_BAND = 0x4000 -SO_DELIM = 0x8000 -SO_NODELIM = 0x010000 -SO_STRHOLD = 0x020000 -SO_ERROPT = 0x040000 -SO_COPYOPT = 0x080000 -SO_MAXBLK = 0x100000 -DEF_IOV_MAX = 16 -INFOD_FIRSTBYTES = 0x02 -INFOD_BYTES = 0x04 -INFOD_COUNT = 0x08 -INFOD_COPYOUT = 0x10 -MODOPEN = 0x1 -CLONEOPEN = 0x2 -CONSOPEN = 0x4 -OPENFAIL = -1 -BPRI_LO = 1 -BPRI_MED = 2 -BPRI_HI = 3 -BPRI_FT = 4 -INFPSZ = -1 -FLUSHALL = 1 -FLUSHDATA = 0 -STRHIGH = 5120 -STRLOW = 1024 -MAXIOCBSZ = 1024 -PERIM_INNER = 1 -PERIM_OUTER = 2 -def datamsg(type): return \ - -def straln(a): return (caddr_t)((intptr_t)(a) & ~(sizeof (int)-1)) - - -# Included from sys/byteorder.h -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_ENCAP = 4 -IPPROTO_TCP = 6 -IPPROTO_EGP = 8 -IPPROTO_PUP = 12 -IPPROTO_UDP = 17 -IPPROTO_IDP = 22 -IPPROTO_IPV6 = 41 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_RSVP = 46 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_HELLO = 63 -IPPROTO_ND = 77 -IPPROTO_EON = 80 -IPPROTO_PIM = 103 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPORT_ECHO = 7 -IPPORT_DISCARD = 9 -IPPORT_SYSTAT = 11 -IPPORT_DAYTIME = 13 -IPPORT_NETSTAT = 15 -IPPORT_FTP = 21 -IPPORT_TELNET = 23 -IPPORT_SMTP = 25 -IPPORT_TIMESERVER = 37 -IPPORT_NAMESERVER = 42 -IPPORT_WHOIS = 43 -IPPORT_MTP = 57 -IPPORT_BOOTPS = 67 -IPPORT_BOOTPC = 68 -IPPORT_TFTP = 69 -IPPORT_RJE = 77 -IPPORT_FINGER = 79 -IPPORT_TTYLINK = 87 -IPPORT_SUPDUP = 95 -IPPORT_EXECSERVER = 512 -IPPORT_LOGINSERVER = 513 -IPPORT_CMDSERVER = 514 -IPPORT_EFSSERVER = 520 -IPPORT_BIFFUDP = 512 -IPPORT_WHOSERVER = 513 -IPPORT_ROUTESERVER = 520 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -IMPLINK_IP = 155 -IMPLINK_LOWEXPER = 156 -IMPLINK_HIGHEXPER = 158 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_MAX = 128 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_MAX = 65536 -IN_CLASSC_NSHIFT = 8 -IN_CLASSD_NSHIFT = 28 -def IN_MULTICAST(i): return IN_CLASSD(i) - -IN_LOOPBACKNET = 127 -def IN_SET_LOOPBACK_ADDR(a): return \ - -def IN6_IS_ADDR_UNSPECIFIED(addr): return \ - -def IN6_IS_ADDR_LOOPBACK(addr): return \ - -def IN6_IS_ADDR_LOOPBACK(addr): return \ - -def IN6_IS_ADDR_MULTICAST(addr): return \ - -def IN6_IS_ADDR_MULTICAST(addr): return \ - -def IN6_IS_ADDR_LINKLOCAL(addr): return \ - -def IN6_IS_ADDR_LINKLOCAL(addr): return \ - -def IN6_IS_ADDR_SITELOCAL(addr): return \ - -def IN6_IS_ADDR_SITELOCAL(addr): return \ - -def IN6_IS_ADDR_V4MAPPED(addr): return \ - -def IN6_IS_ADDR_V4MAPPED(addr): return \ - -def IN6_IS_ADDR_V4MAPPED_ANY(addr): return \ - -def IN6_IS_ADDR_V4MAPPED_ANY(addr): return \ - -def IN6_IS_ADDR_V4COMPAT(addr): return \ - -def IN6_IS_ADDR_V4COMPAT(addr): return \ - -def IN6_IS_ADDR_MC_RESERVED(addr): return \ - -def IN6_IS_ADDR_MC_RESERVED(addr): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(addr): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(addr): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(addr): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(addr): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(addr): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(addr): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(addr): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(addr): return \ - -def IN6_IS_ADDR_MC_GLOBAL(addr): return \ - -def IN6_IS_ADDR_MC_GLOBAL(addr): return \ - -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 0x10 -IP_MULTICAST_TTL = 0x11 -IP_MULTICAST_LOOP = 0x12 -IP_ADD_MEMBERSHIP = 0x13 -IP_DROP_MEMBERSHIP = 0x14 -IP_SEC_OPT = 0x22 -IPSEC_PREF_NEVER = 0x01 -IPSEC_PREF_REQUIRED = 0x02 -IPSEC_PREF_UNIQUE = 0x04 -IP_ADD_PROXY_ADDR = 0x40 -IP_BOUND_IF = 0x41 -IP_UNSPEC_SRC = 0x42 -IP_REUSEADDR = 0x104 -IP_DONTROUTE = 0x105 -IP_BROADCAST = 0x106 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_UNICAST_HOPS = 0x5 -IPV6_MULTICAST_IF = 0x6 -IPV6_MULTICAST_HOPS = 0x7 -IPV6_MULTICAST_LOOP = 0x8 -IPV6_JOIN_GROUP = 0x9 -IPV6_LEAVE_GROUP = 0xa -IPV6_ADD_MEMBERSHIP = 0x9 -IPV6_DROP_MEMBERSHIP = 0xa -IPV6_PKTINFO = 0xb -IPV6_HOPLIMIT = 0xc -IPV6_NEXTHOP = 0xd -IPV6_HOPOPTS = 0xe -IPV6_DSTOPTS = 0xf -IPV6_RTHDR = 0x10 -IPV6_RTHDRDSTOPTS = 0x11 -IPV6_RECVPKTINFO = 0x12 -IPV6_RECVHOPLIMIT = 0x13 -IPV6_RECVHOPOPTS = 0x14 -IPV6_RECVDSTOPTS = 0x15 -IPV6_RECVRTHDR = 0x16 -IPV6_RECVRTHDRDSTOPTS = 0x17 -IPV6_CHECKSUM = 0x18 -IPV6_BOUND_IF = 0x41 -IPV6_UNSPEC_SRC = 0x42 -INET_ADDRSTRLEN = 16 -INET6_ADDRSTRLEN = 46 -IPV6_PAD1_OPT = 0 diff --git a/Lib/plat-sunos5/STROPTS.py b/Lib/plat-sunos5/STROPTS.py deleted file mode 100644 --- a/Lib/plat-sunos5/STROPTS.py +++ /dev/null @@ -1,1813 +0,0 @@ -# Generated by h2py from /usr/include/sys/stropts.h - -# Included from sys/feature_tests.h - -# Included from sys/isa_defs.h -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 8 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_ALIGNMENT_REQUIRED = 1 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 4 -_DOUBLE_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 4 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 4 -_ALIGNMENT_REQUIRED = 0 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_ALIGNMENT_REQUIRED = 1 -_LONG_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 8 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 8 -_LONG_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_POSIX_C_SOURCE = 1 -_LARGEFILE64_SOURCE = 1 -_LARGEFILE_SOURCE = 1 -_FILE_OFFSET_BITS = 64 -_FILE_OFFSET_BITS = 32 -_POSIX_C_SOURCE = 199506 -_POSIX_PTHREAD_SEMANTICS = 1 -_XOPEN_VERSION = 500 -_XOPEN_VERSION = 4 -_XOPEN_VERSION = 3 -from TYPES import * - -# Included from sys/conf.h - -# Included from sys/t_lock.h - -# Included from sys/machlock.h -from TYPES import * -LOCK_HELD_VALUE = 0xff -def SPIN_LOCK(pl): return ((pl) > ipltospl(LOCK_LEVEL)) - -def LOCK_SAMPLE_INTERVAL(i): return (((i) & 0xff) == 0) - -CLOCK_LEVEL = 10 -LOCK_LEVEL = 10 -DISP_LEVEL = (LOCK_LEVEL + 1) -PTR24_LSB = 5 -PTR24_MSB = (PTR24_LSB + 24) -PTR24_ALIGN = 32 -PTR24_BASE = 0xe0000000 - -# Included from sys/param.h -from TYPES import * -_POSIX_VDISABLE = 0 -MAX_INPUT = 512 -MAX_CANON = 256 -UID_NOBODY = 60001 -GID_NOBODY = UID_NOBODY -UID_NOACCESS = 60002 -MAX_TASKID = 999999 -MAX_MAXPID = 999999 -DEFAULT_MAXPID = 999999 -DEFAULT_JUMPPID = 100000 -DEFAULT_MAXPID = 30000 -DEFAULT_JUMPPID = 0 -MAXUID = 2147483647 -MAXPROJID = MAXUID -MAXLINK = 32767 -NMOUNT = 40 -CANBSIZ = 256 -NOFILE = 20 -NGROUPS_UMIN = 0 -NGROUPS_UMAX = 32 -NGROUPS_MAX_DEFAULT = 16 -NZERO = 20 -NULL = 0 -NULL = 0 -CMASK = 0o22 -CDLIMIT = (1<<11) -NBPS = 0x20000 -NBPSCTR = 512 -UBSIZE = 512 -SCTRSHFT = 9 -SYSNAME = 9 -PREMOTE = 39 -MAXPATHLEN = 1024 -MAXSYMLINKS = 20 -MAXNAMELEN = 256 -NADDR = 13 -PIPE_BUF = 5120 -PIPE_MAX = 5120 -NBBY = 8 -MAXBSIZE = 8192 -DEV_BSIZE = 512 -DEV_BSHIFT = 9 -MAXFRAG = 8 -MAXOFF32_T = 0x7fffffff -MAXOFF_T = 0x7fffffffffffffff -MAXOFFSET_T = 0x7fffffffffffffff -MAXOFF_T = 0x7fffffff -MAXOFFSET_T = 0x7fffffff -def btodb(bytes): return \ - -def dbtob(db): return \ - -def lbtodb(bytes): return \ - -def ldbtob(db): return \ - -NCARGS32 = 0x100000 -NCARGS64 = 0x200000 -NCARGS = NCARGS64 -NCARGS = NCARGS32 -FSHIFT = 8 -FSCALE = (1<> MMU_PAGESHIFT) - -def mmu_btopr(x): return ((((x) + MMU_PAGEOFFSET) >> MMU_PAGESHIFT)) - -def mmu_ptod(x): return ((x) << (MMU_PAGESHIFT - DEV_BSHIFT)) - -def ptod(x): return ((x) << (PAGESHIFT - DEV_BSHIFT)) - -def ptob(x): return ((x) << PAGESHIFT) - -def btop(x): return (((x) >> PAGESHIFT)) - -def btopr(x): return ((((x) + PAGEOFFSET) >> PAGESHIFT)) - -def dtop(DD): return (((DD) + NDPP - 1) >> (PAGESHIFT - DEV_BSHIFT)) - -def dtopt(DD): return ((DD) >> (PAGESHIFT - DEV_BSHIFT)) - -_AIO_LISTIO_MAX = (4096) -_AIO_MAX = (-1) -_MQ_OPEN_MAX = (32) -_MQ_PRIO_MAX = (32) -_SEM_NSEMS_MAX = INT_MAX -_SEM_VALUE_MAX = INT_MAX - -# Included from sys/unistd.h -_CS_PATH = 65 -_CS_LFS_CFLAGS = 68 -_CS_LFS_LDFLAGS = 69 -_CS_LFS_LIBS = 70 -_CS_LFS_LINTFLAGS = 71 -_CS_LFS64_CFLAGS = 72 -_CS_LFS64_LDFLAGS = 73 -_CS_LFS64_LIBS = 74 -_CS_LFS64_LINTFLAGS = 75 -_CS_XBS5_ILP32_OFF32_CFLAGS = 700 -_CS_XBS5_ILP32_OFF32_LDFLAGS = 701 -_CS_XBS5_ILP32_OFF32_LIBS = 702 -_CS_XBS5_ILP32_OFF32_LINTFLAGS = 703 -_CS_XBS5_ILP32_OFFBIG_CFLAGS = 705 -_CS_XBS5_ILP32_OFFBIG_LDFLAGS = 706 -_CS_XBS5_ILP32_OFFBIG_LIBS = 707 -_CS_XBS5_ILP32_OFFBIG_LINTFLAGS = 708 -_CS_XBS5_LP64_OFF64_CFLAGS = 709 -_CS_XBS5_LP64_OFF64_LDFLAGS = 710 -_CS_XBS5_LP64_OFF64_LIBS = 711 -_CS_XBS5_LP64_OFF64_LINTFLAGS = 712 -_CS_XBS5_LPBIG_OFFBIG_CFLAGS = 713 -_CS_XBS5_LPBIG_OFFBIG_LDFLAGS = 714 -_CS_XBS5_LPBIG_OFFBIG_LIBS = 715 -_CS_XBS5_LPBIG_OFFBIG_LINTFLAGS = 716 -_SC_ARG_MAX = 1 -_SC_CHILD_MAX = 2 -_SC_CLK_TCK = 3 -_SC_NGROUPS_MAX = 4 -_SC_OPEN_MAX = 5 -_SC_JOB_CONTROL = 6 -_SC_SAVED_IDS = 7 -_SC_VERSION = 8 -_SC_PASS_MAX = 9 -_SC_LOGNAME_MAX = 10 -_SC_PAGESIZE = 11 -_SC_XOPEN_VERSION = 12 -_SC_NPROCESSORS_CONF = 14 -_SC_NPROCESSORS_ONLN = 15 -_SC_STREAM_MAX = 16 -_SC_TZNAME_MAX = 17 -_SC_AIO_LISTIO_MAX = 18 -_SC_AIO_MAX = 19 -_SC_AIO_PRIO_DELTA_MAX = 20 -_SC_ASYNCHRONOUS_IO = 21 -_SC_DELAYTIMER_MAX = 22 -_SC_FSYNC = 23 -_SC_MAPPED_FILES = 24 -_SC_MEMLOCK = 25 -_SC_MEMLOCK_RANGE = 26 -_SC_MEMORY_PROTECTION = 27 -_SC_MESSAGE_PASSING = 28 -_SC_MQ_OPEN_MAX = 29 -_SC_MQ_PRIO_MAX = 30 -_SC_PRIORITIZED_IO = 31 -_SC_PRIORITY_SCHEDULING = 32 -_SC_REALTIME_SIGNALS = 33 -_SC_RTSIG_MAX = 34 -_SC_SEMAPHORES = 35 -_SC_SEM_NSEMS_MAX = 36 -_SC_SEM_VALUE_MAX = 37 -_SC_SHARED_MEMORY_OBJECTS = 38 -_SC_SIGQUEUE_MAX = 39 -_SC_SIGRT_MIN = 40 -_SC_SIGRT_MAX = 41 -_SC_SYNCHRONIZED_IO = 42 -_SC_TIMERS = 43 -_SC_TIMER_MAX = 44 -_SC_2_C_BIND = 45 -_SC_2_C_DEV = 46 -_SC_2_C_VERSION = 47 -_SC_2_FORT_DEV = 48 -_SC_2_FORT_RUN = 49 -_SC_2_LOCALEDEF = 50 -_SC_2_SW_DEV = 51 -_SC_2_UPE = 52 -_SC_2_VERSION = 53 -_SC_BC_BASE_MAX = 54 -_SC_BC_DIM_MAX = 55 -_SC_BC_SCALE_MAX = 56 -_SC_BC_STRING_MAX = 57 -_SC_COLL_WEIGHTS_MAX = 58 -_SC_EXPR_NEST_MAX = 59 -_SC_LINE_MAX = 60 -_SC_RE_DUP_MAX = 61 -_SC_XOPEN_CRYPT = 62 -_SC_XOPEN_ENH_I18N = 63 -_SC_XOPEN_SHM = 64 -_SC_2_CHAR_TERM = 66 -_SC_XOPEN_XCU_VERSION = 67 -_SC_ATEXIT_MAX = 76 -_SC_IOV_MAX = 77 -_SC_XOPEN_UNIX = 78 -_SC_PAGE_SIZE = _SC_PAGESIZE -_SC_T_IOV_MAX = 79 -_SC_PHYS_PAGES = 500 -_SC_AVPHYS_PAGES = 501 -_SC_COHER_BLKSZ = 503 -_SC_SPLIT_CACHE = 504 -_SC_ICACHE_SZ = 505 -_SC_DCACHE_SZ = 506 -_SC_ICACHE_LINESZ = 507 -_SC_DCACHE_LINESZ = 508 -_SC_ICACHE_BLKSZ = 509 -_SC_DCACHE_BLKSZ = 510 -_SC_DCACHE_TBLKSZ = 511 -_SC_ICACHE_ASSOC = 512 -_SC_DCACHE_ASSOC = 513 -_SC_MAXPID = 514 -_SC_STACK_PROT = 515 -_SC_THREAD_DESTRUCTOR_ITERATIONS = 568 -_SC_GETGR_R_SIZE_MAX = 569 -_SC_GETPW_R_SIZE_MAX = 570 -_SC_LOGIN_NAME_MAX = 571 -_SC_THREAD_KEYS_MAX = 572 -_SC_THREAD_STACK_MIN = 573 -_SC_THREAD_THREADS_MAX = 574 -_SC_TTY_NAME_MAX = 575 -_SC_THREADS = 576 -_SC_THREAD_ATTR_STACKADDR = 577 -_SC_THREAD_ATTR_STACKSIZE = 578 -_SC_THREAD_PRIORITY_SCHEDULING = 579 -_SC_THREAD_PRIO_INHERIT = 580 -_SC_THREAD_PRIO_PROTECT = 581 -_SC_THREAD_PROCESS_SHARED = 582 -_SC_THREAD_SAFE_FUNCTIONS = 583 -_SC_XOPEN_LEGACY = 717 -_SC_XOPEN_REALTIME = 718 -_SC_XOPEN_REALTIME_THREADS = 719 -_SC_XBS5_ILP32_OFF32 = 720 -_SC_XBS5_ILP32_OFFBIG = 721 -_SC_XBS5_LP64_OFF64 = 722 -_SC_XBS5_LPBIG_OFFBIG = 723 -_PC_LINK_MAX = 1 -_PC_MAX_CANON = 2 -_PC_MAX_INPUT = 3 -_PC_NAME_MAX = 4 -_PC_PATH_MAX = 5 -_PC_PIPE_BUF = 6 -_PC_NO_TRUNC = 7 -_PC_VDISABLE = 8 -_PC_CHOWN_RESTRICTED = 9 -_PC_ASYNC_IO = 10 -_PC_PRIO_IO = 11 -_PC_SYNC_IO = 12 -_PC_FILESIZEBITS = 67 -_PC_LAST = 67 -_POSIX_VERSION = 199506 -_POSIX2_VERSION = 199209 -_POSIX2_C_VERSION = 199209 -_XOPEN_XCU_VERSION = 4 -_XOPEN_REALTIME = 1 -_XOPEN_ENH_I18N = 1 -_XOPEN_SHM = 1 -_POSIX2_C_BIND = 1 -_POSIX2_CHAR_TERM = 1 -_POSIX2_LOCALEDEF = 1 -_POSIX2_C_DEV = 1 -_POSIX2_SW_DEV = 1 -_POSIX2_UPE = 1 - -# Included from sys/mutex.h -from TYPES import * -def MUTEX_HELD(x): return (mutex_owned(x)) - - -# Included from sys/rwlock.h -from TYPES import * -def RW_READ_HELD(x): return (rw_read_held((x))) - -def RW_WRITE_HELD(x): return (rw_write_held((x))) - -def RW_LOCK_HELD(x): return (rw_lock_held((x))) - -def RW_ISWRITER(x): return (rw_iswriter(x)) - - -# Included from sys/semaphore.h - -# Included from sys/thread.h -from TYPES import * - -# Included from sys/klwp.h -from TYPES import * - -# Included from sys/condvar.h -from TYPES import * - -# Included from sys/time.h - -# Included from sys/types32.h - -# Included from sys/int_types.h -TIME32_MAX = INT32_MAX -TIME32_MIN = INT32_MIN -def TIMEVAL_OVERFLOW(tv): return \ - -from TYPES import * -DST_NONE = 0 -DST_USA = 1 -DST_AUST = 2 -DST_WET = 3 -DST_MET = 4 -DST_EET = 5 -DST_CAN = 6 -DST_GB = 7 -DST_RUM = 8 -DST_TUR = 9 -DST_AUSTALT = 10 -ITIMER_REAL = 0 -ITIMER_VIRTUAL = 1 -ITIMER_PROF = 2 -ITIMER_REALPROF = 3 -def ITIMERVAL_OVERFLOW(itv): return \ - -SEC = 1 -MILLISEC = 1000 -MICROSEC = 1000000 -NANOSEC = 1000000000 - -# Included from sys/time_impl.h -def TIMESPEC_OVERFLOW(ts): return \ - -def ITIMERSPEC_OVERFLOW(it): return \ - -__CLOCK_REALTIME0 = 0 -CLOCK_VIRTUAL = 1 -CLOCK_PROF = 2 -__CLOCK_REALTIME3 = 3 -CLOCK_HIGHRES = 4 -CLOCK_MAX = 5 -CLOCK_REALTIME = __CLOCK_REALTIME3 -CLOCK_REALTIME = __CLOCK_REALTIME0 -TIMER_RELTIME = 0x0 -TIMER_ABSTIME = 0x1 -def TICK_TO_SEC(tick): return ((tick) / hz) - -def SEC_TO_TICK(sec): return ((sec) * hz) - -def TICK_TO_MSEC(tick): return \ - -def MSEC_TO_TICK(msec): return \ - -def MSEC_TO_TICK_ROUNDUP(msec): return \ - -def TICK_TO_USEC(tick): return ((tick) * usec_per_tick) - -def USEC_TO_TICK(usec): return ((usec) / usec_per_tick) - -def USEC_TO_TICK_ROUNDUP(usec): return \ - -def TICK_TO_NSEC(tick): return ((tick) * nsec_per_tick) - -def NSEC_TO_TICK(nsec): return ((nsec) / nsec_per_tick) - -def NSEC_TO_TICK_ROUNDUP(nsec): return \ - -def TIMEVAL_TO_TICK(tvp): return \ - -def TIMESTRUC_TO_TICK(tsp): return \ - - -# Included from time.h -from TYPES import * - -# Included from iso/time_iso.h -NULL = 0 -NULL = 0 -CLOCKS_PER_SEC = 1000000 - -# Included from sys/select.h -FD_SETSIZE = 65536 -FD_SETSIZE = 1024 -_NBBY = 8 -NBBY = _NBBY -def FD_ZERO(p): return bzero((p), sizeof (*(p))) - - -# Included from sys/signal.h - -# Included from sys/iso/signal_iso.h -SIGHUP = 1 -SIGINT = 2 -SIGQUIT = 3 -SIGILL = 4 -SIGTRAP = 5 -SIGIOT = 6 -SIGABRT = 6 -SIGEMT = 7 -SIGFPE = 8 -SIGKILL = 9 -SIGBUS = 10 -SIGSEGV = 11 -SIGSYS = 12 -SIGPIPE = 13 -SIGALRM = 14 -SIGTERM = 15 -SIGUSR1 = 16 -SIGUSR2 = 17 -SIGCLD = 18 -SIGCHLD = 18 -SIGPWR = 19 -SIGWINCH = 20 -SIGURG = 21 -SIGPOLL = 22 -SIGIO = SIGPOLL -SIGSTOP = 23 -SIGTSTP = 24 -SIGCONT = 25 -SIGTTIN = 26 -SIGTTOU = 27 -SIGVTALRM = 28 -SIGPROF = 29 -SIGXCPU = 30 -SIGXFSZ = 31 -SIGWAITING = 32 -SIGLWP = 33 -SIGFREEZE = 34 -SIGTHAW = 35 -SIGCANCEL = 36 -SIGLOST = 37 -_SIGRTMIN = 38 -_SIGRTMAX = 45 -SIG_BLOCK = 1 -SIG_UNBLOCK = 2 -SIG_SETMASK = 3 -SIGNO_MASK = 0xFF -SIGDEFER = 0x100 -SIGHOLD = 0x200 -SIGRELSE = 0x400 -SIGIGNORE = 0x800 -SIGPAUSE = 0x1000 - -# Included from sys/siginfo.h -from TYPES import * -SIGEV_NONE = 1 -SIGEV_SIGNAL = 2 -SIGEV_THREAD = 3 -SI_NOINFO = 32767 -SI_USER = 0 -SI_LWP = (-1) -SI_QUEUE = (-2) -SI_TIMER = (-3) -SI_ASYNCIO = (-4) -SI_MESGQ = (-5) - -# Included from sys/machsig.h -ILL_ILLOPC = 1 -ILL_ILLOPN = 2 -ILL_ILLADR = 3 -ILL_ILLTRP = 4 -ILL_PRVOPC = 5 -ILL_PRVREG = 6 -ILL_COPROC = 7 -ILL_BADSTK = 8 -NSIGILL = 8 -EMT_TAGOVF = 1 -EMT_CPCOVF = 2 -NSIGEMT = 2 -FPE_INTDIV = 1 -FPE_INTOVF = 2 -FPE_FLTDIV = 3 -FPE_FLTOVF = 4 -FPE_FLTUND = 5 -FPE_FLTRES = 6 -FPE_FLTINV = 7 -FPE_FLTSUB = 8 -NSIGFPE = 8 -SEGV_MAPERR = 1 -SEGV_ACCERR = 2 -NSIGSEGV = 2 -BUS_ADRALN = 1 -BUS_ADRERR = 2 -BUS_OBJERR = 3 -NSIGBUS = 3 -TRAP_BRKPT = 1 -TRAP_TRACE = 2 -TRAP_RWATCH = 3 -TRAP_WWATCH = 4 -TRAP_XWATCH = 5 -NSIGTRAP = 5 -CLD_EXITED = 1 -CLD_KILLED = 2 -CLD_DUMPED = 3 -CLD_TRAPPED = 4 -CLD_STOPPED = 5 -CLD_CONTINUED = 6 -NSIGCLD = 6 -POLL_IN = 1 -POLL_OUT = 2 -POLL_MSG = 3 -POLL_ERR = 4 -POLL_PRI = 5 -POLL_HUP = 6 -NSIGPOLL = 6 -PROF_SIG = 1 -NSIGPROF = 1 -SI_MAXSZ = 256 -SI_MAXSZ = 128 - -# Included from sys/time_std_impl.h -from TYPES import * -SI32_MAXSZ = 128 -def SI_CANQUEUE(c): return ((c) <= SI_QUEUE) - -SA_NOCLDSTOP = 0x00020000 -SA_ONSTACK = 0x00000001 -SA_RESETHAND = 0x00000002 -SA_RESTART = 0x00000004 -SA_SIGINFO = 0x00000008 -SA_NODEFER = 0x00000010 -SA_NOCLDWAIT = 0x00010000 -SA_WAITSIG = 0x00010000 -NSIG = 46 -MAXSIG = 45 -S_SIGNAL = 1 -S_SIGSET = 2 -S_SIGACTION = 3 -S_NONE = 4 -MINSIGSTKSZ = 2048 -SIGSTKSZ = 8192 -SS_ONSTACK = 0x00000001 -SS_DISABLE = 0x00000002 -SN_PROC = 1 -SN_CANCEL = 2 -SN_SEND = 3 - -# Included from sys/ucontext.h -from TYPES import * - -# Included from sys/regset.h -REG_CCR = (0) -REG_PSR = (0) -REG_PSR = (0) -REG_PC = (1) -REG_nPC = (2) -REG_Y = (3) -REG_G1 = (4) -REG_G2 = (5) -REG_G3 = (6) -REG_G4 = (7) -REG_G5 = (8) -REG_G6 = (9) -REG_G7 = (10) -REG_O0 = (11) -REG_O1 = (12) -REG_O2 = (13) -REG_O3 = (14) -REG_O4 = (15) -REG_O5 = (16) -REG_O6 = (17) -REG_O7 = (18) -REG_ASI = (19) -REG_FPRS = (20) -REG_PS = REG_PSR -REG_SP = REG_O6 -REG_R0 = REG_O0 -REG_R1 = REG_O1 -_NGREG = 21 -_NGREG = 19 -NGREG = _NGREG -_NGREG32 = 19 -_NGREG64 = 21 -SPARC_MAXREGWINDOW = 31 -MAXFPQ = 16 -XRS_ID = 0x78727300 - -# Included from v7/sys/privregs.h - -# Included from v7/sys/psr.h -PSR_CWP = 0x0000001F -PSR_ET = 0x00000020 -PSR_PS = 0x00000040 -PSR_S = 0x00000080 -PSR_PIL = 0x00000F00 -PSR_EF = 0x00001000 -PSR_EC = 0x00002000 -PSR_RSV = 0x000FC000 -PSR_ICC = 0x00F00000 -PSR_C = 0x00100000 -PSR_V = 0x00200000 -PSR_Z = 0x00400000 -PSR_N = 0x00800000 -PSR_VER = 0x0F000000 -PSR_IMPL = 0xF0000000 -PSL_ALLCC = PSR_ICC -PSL_USER = (PSR_S) -PSL_USERMASK = (PSR_ICC) -PSL_UBITS = (PSR_ICC|PSR_EF) -def USERMODE(ps): return (((ps) & PSR_PS) == 0) - - -# Included from sys/fsr.h -FSR_CEXC = 0x0000001f -FSR_AEXC = 0x000003e0 -FSR_FCC = 0x00000c00 -FSR_PR = 0x00001000 -FSR_QNE = 0x00002000 -FSR_FTT = 0x0001c000 -FSR_VER = 0x000e0000 -FSR_TEM = 0x0f800000 -FSR_RP = 0x30000000 -FSR_RD = 0xc0000000 -FSR_VER_SHIFT = 17 -FSR_FCC1 = 0x00000003 -FSR_FCC2 = 0x0000000C -FSR_FCC3 = 0x00000030 -FSR_CEXC_NX = 0x00000001 -FSR_CEXC_DZ = 0x00000002 -FSR_CEXC_UF = 0x00000004 -FSR_CEXC_OF = 0x00000008 -FSR_CEXC_NV = 0x00000010 -FSR_AEXC_NX = (0x1 << 5) -FSR_AEXC_DZ = (0x2 << 5) -FSR_AEXC_UF = (0x4 << 5) -FSR_AEXC_OF = (0x8 << 5) -FSR_AEXC_NV = (0x10 << 5) -FTT_NONE = 0 -FTT_IEEE = 1 -FTT_UNFIN = 2 -FTT_UNIMP = 3 -FTT_SEQ = 4 -FTT_ALIGN = 5 -FTT_DFAULT = 6 -FSR_FTT_SHIFT = 14 -FSR_FTT_IEEE = (FTT_IEEE << FSR_FTT_SHIFT) -FSR_FTT_UNFIN = (FTT_UNFIN << FSR_FTT_SHIFT) -FSR_FTT_UNIMP = (FTT_UNIMP << FSR_FTT_SHIFT) -FSR_FTT_SEQ = (FTT_SEQ << FSR_FTT_SHIFT) -FSR_FTT_ALIGN = (FTT_ALIGN << FSR_FTT_SHIFT) -FSR_FTT_DFAULT = (FTT_DFAULT << FSR_FTT_SHIFT) -FSR_TEM_NX = (0x1 << 23) -FSR_TEM_DZ = (0x2 << 23) -FSR_TEM_UF = (0x4 << 23) -FSR_TEM_OF = (0x8 << 23) -FSR_TEM_NV = (0x10 << 23) -RP_DBLEXT = 0 -RP_SINGLE = 1 -RP_DOUBLE = 2 -RP_RESERVED = 3 -RD_NEAR = 0 -RD_ZER0 = 1 -RD_POSINF = 2 -RD_NEGINF = 3 -FPRS_DL = 0x1 -FPRS_DU = 0x2 -FPRS_FEF = 0x4 -PIL_MAX = 0xf -def SAVE_GLOBALS(RP): return \ - -def RESTORE_GLOBALS(RP): return \ - -def SAVE_OUTS(RP): return \ - -def RESTORE_OUTS(RP): return \ - -def SAVE_WINDOW(SBP): return \ - -def RESTORE_WINDOW(SBP): return \ - -def STORE_FPREGS(FP): return \ - -def LOAD_FPREGS(FP): return \ - -_SPARC_MAXREGWINDOW = 31 -_XRS_ID = 0x78727300 -GETCONTEXT = 0 -SETCONTEXT = 1 -UC_SIGMASK = 0o01 -UC_STACK = 0o02 -UC_CPU = 0o04 -UC_MAU = 0o10 -UC_FPU = UC_MAU -UC_INTR = 0o20 -UC_ASR = 0o40 -UC_MCONTEXT = (UC_CPU|UC_FPU|UC_ASR) -UC_ALL = (UC_SIGMASK|UC_STACK|UC_MCONTEXT) -_SIGQUEUE_MAX = 32 -_SIGNOTIFY_MAX = 32 - -# Included from sys/pcb.h -INSTR_VALID = 0x02 -NORMAL_STEP = 0x04 -WATCH_STEP = 0x08 -CPC_OVERFLOW = 0x10 -ASYNC_HWERR = 0x20 -STEP_NONE = 0 -STEP_REQUESTED = 1 -STEP_ACTIVE = 2 -STEP_WASACTIVE = 3 - -# Included from sys/msacct.h -LMS_USER = 0 -LMS_SYSTEM = 1 -LMS_TRAP = 2 -LMS_TFAULT = 3 -LMS_DFAULT = 4 -LMS_KFAULT = 5 -LMS_USER_LOCK = 6 -LMS_SLEEP = 7 -LMS_WAIT_CPU = 8 -LMS_STOPPED = 9 -NMSTATES = 10 - -# Included from sys/lwp.h - -# Included from sys/synch.h -from TYPES import * -USYNC_THREAD = 0x00 -USYNC_PROCESS = 0x01 -LOCK_NORMAL = 0x00 -LOCK_ERRORCHECK = 0x02 -LOCK_RECURSIVE = 0x04 -USYNC_PROCESS_ROBUST = 0x08 -LOCK_PRIO_NONE = 0x00 -LOCK_PRIO_INHERIT = 0x10 -LOCK_PRIO_PROTECT = 0x20 -LOCK_STALL_NP = 0x00 -LOCK_ROBUST_NP = 0x40 -LOCK_OWNERDEAD = 0x1 -LOCK_NOTRECOVERABLE = 0x2 -LOCK_INITED = 0x4 -LOCK_UNMAPPED = 0x8 -LWP_DETACHED = 0x00000040 -LWP_SUSPENDED = 0x00000080 -__LWP_ASLWP = 0x00000100 -MAXSYSARGS = 8 -NORMALRETURN = 0 -JUSTRETURN = 1 -LWP_USER = 0x01 -LWP_SYS = 0x02 -TS_FREE = 0x00 -TS_SLEEP = 0x01 -TS_RUN = 0x02 -TS_ONPROC = 0x04 -TS_ZOMB = 0x08 -TS_STOPPED = 0x10 -T_INTR_THREAD = 0x0001 -T_WAKEABLE = 0x0002 -T_TOMASK = 0x0004 -T_TALLOCSTK = 0x0008 -T_WOULDBLOCK = 0x0020 -T_DONTBLOCK = 0x0040 -T_DONTPEND = 0x0080 -T_SYS_PROF = 0x0100 -T_WAITCVSEM = 0x0200 -T_WATCHPT = 0x0400 -T_PANIC = 0x0800 -TP_HOLDLWP = 0x0002 -TP_TWAIT = 0x0004 -TP_LWPEXIT = 0x0008 -TP_PRSTOP = 0x0010 -TP_CHKPT = 0x0020 -TP_EXITLWP = 0x0040 -TP_PRVSTOP = 0x0080 -TP_MSACCT = 0x0100 -TP_STOPPING = 0x0200 -TP_WATCHPT = 0x0400 -TP_PAUSE = 0x0800 -TP_CHANGEBIND = 0x1000 -TS_LOAD = 0x0001 -TS_DONT_SWAP = 0x0002 -TS_SWAPENQ = 0x0004 -TS_ON_SWAPQ = 0x0008 -TS_CSTART = 0x0100 -TS_UNPAUSE = 0x0200 -TS_XSTART = 0x0400 -TS_PSTART = 0x0800 -TS_RESUME = 0x1000 -TS_CREATE = 0x2000 -TS_ALLSTART = \ - (TS_CSTART|TS_UNPAUSE|TS_XSTART|TS_PSTART|TS_RESUME|TS_CREATE) -def CPR_VSTOPPED(t): return \ - -def THREAD_TRANSITION(tp): return thread_transition(tp); - -def THREAD_STOP(tp): return \ - -def THREAD_ZOMB(tp): return THREAD_SET_STATE(tp, TS_ZOMB, NULL) - -def SEMA_HELD(x): return (sema_held((x))) - -NO_LOCKS_HELD = 1 -NO_COMPETING_THREADS = 1 -FMNAMESZ = 8 - -# Included from sys/systm.h -from TYPES import * - -# Included from sys/proc.h - -# Included from sys/cred.h - -# Included from sys/user.h -from TYPES import * - -# Included from sys/resource.h -from TYPES import * -PRIO_PROCESS = 0 -PRIO_PGRP = 1 -PRIO_USER = 2 -RLIMIT_CPU = 0 -RLIMIT_FSIZE = 1 -RLIMIT_DATA = 2 -RLIMIT_STACK = 3 -RLIMIT_CORE = 4 -RLIMIT_NOFILE = 5 -RLIMIT_VMEM = 6 -RLIMIT_AS = RLIMIT_VMEM -RLIM_NLIMITS = 7 -RLIM_INFINITY = (-3) -RLIM_SAVED_MAX = (-2) -RLIM_SAVED_CUR = (-1) -RLIM_INFINITY = 0x7fffffff -RLIM_SAVED_MAX = 0x7ffffffe -RLIM_SAVED_CUR = 0x7ffffffd -RLIM32_INFINITY = 0x7fffffff -RLIM32_SAVED_MAX = 0x7ffffffe -RLIM32_SAVED_CUR = 0x7ffffffd - -# Included from sys/model.h - -# Included from sys/debug.h -def ASSERT64(x): return ASSERT(x) - -def ASSERT32(x): return ASSERT(x) - -DATAMODEL_MASK = 0x0FF00000 -DATAMODEL_ILP32 = 0x00100000 -DATAMODEL_LP64 = 0x00200000 -DATAMODEL_NONE = 0 -DATAMODEL_NATIVE = DATAMODEL_LP64 -DATAMODEL_NATIVE = DATAMODEL_ILP32 -def STRUCT_SIZE(handle): return \ - -def STRUCT_BUF(handle): return ((handle).ptr.m64) - -def SIZEOF_PTR(umodel): return \ - -def STRUCT_SIZE(handle): return (sizeof (*(handle).ptr)) - -def STRUCT_BUF(handle): return ((handle).ptr) - -def SIZEOF_PTR(umodel): return sizeof (caddr_t) - -def lwp_getdatamodel(t): return DATAMODEL_ILP32 - -RUSAGE_SELF = 0 -RUSAGE_CHILDREN = -1 - -# Included from sys/auxv.h -AT_NULL = 0 -AT_IGNORE = 1 -AT_EXECFD = 2 -AT_PHDR = 3 -AT_PHENT = 4 -AT_PHNUM = 5 -AT_PAGESZ = 6 -AT_BASE = 7 -AT_FLAGS = 8 -AT_ENTRY = 9 -AT_DCACHEBSIZE = 10 -AT_ICACHEBSIZE = 11 -AT_UCACHEBSIZE = 12 -AT_SUN_UID = 2000 -AT_SUN_RUID = 2001 -AT_SUN_GID = 2002 -AT_SUN_RGID = 2003 -AT_SUN_LDELF = 2004 -AT_SUN_LDSHDR = 2005 -AT_SUN_LDNAME = 2006 -AT_SUN_LPAGESZ = 2007 -AT_SUN_PLATFORM = 2008 -AT_SUN_HWCAP = 2009 -AT_SUN_IFLUSH = 2010 -AT_SUN_CPU = 2011 -AT_SUN_EMUL_ENTRY = 2012 -AT_SUN_EMUL_EXECFD = 2013 -AT_SUN_EXECNAME = 2014 -AT_SUN_MMU = 2015 - -# Included from sys/errno.h -EPERM = 1 -ENOENT = 2 -ESRCH = 3 -EINTR = 4 -EIO = 5 -ENXIO = 6 -E2BIG = 7 -ENOEXEC = 8 -EBADF = 9 -ECHILD = 10 -EAGAIN = 11 -ENOMEM = 12 -EACCES = 13 -EFAULT = 14 -ENOTBLK = 15 -EBUSY = 16 -EEXIST = 17 -EXDEV = 18 -ENODEV = 19 -ENOTDIR = 20 -EISDIR = 21 -EINVAL = 22 -ENFILE = 23 -EMFILE = 24 -ENOTTY = 25 -ETXTBSY = 26 -EFBIG = 27 -ENOSPC = 28 -ESPIPE = 29 -EROFS = 30 -EMLINK = 31 -EPIPE = 32 -EDOM = 33 -ERANGE = 34 -ENOMSG = 35 -EIDRM = 36 -ECHRNG = 37 -EL2NSYNC = 38 -EL3HLT = 39 -EL3RST = 40 -ELNRNG = 41 -EUNATCH = 42 -ENOCSI = 43 -EL2HLT = 44 -EDEADLK = 45 -ENOLCK = 46 -ECANCELED = 47 -ENOTSUP = 48 -EDQUOT = 49 -EBADE = 50 -EBADR = 51 -EXFULL = 52 -ENOANO = 53 -EBADRQC = 54 -EBADSLT = 55 -EDEADLOCK = 56 -EBFONT = 57 -EOWNERDEAD = 58 -ENOTRECOVERABLE = 59 -ENOSTR = 60 -ENODATA = 61 -ETIME = 62 -ENOSR = 63 -ENONET = 64 -ENOPKG = 65 -EREMOTE = 66 -ENOLINK = 67 -EADV = 68 -ESRMNT = 69 -ECOMM = 70 -EPROTO = 71 -ELOCKUNMAPPED = 72 -ENOTACTIVE = 73 -EMULTIHOP = 74 -EBADMSG = 77 -ENAMETOOLONG = 78 -EOVERFLOW = 79 -ENOTUNIQ = 80 -EBADFD = 81 -EREMCHG = 82 -ELIBACC = 83 -ELIBBAD = 84 -ELIBSCN = 85 -ELIBMAX = 86 -ELIBEXEC = 87 -EILSEQ = 88 -ENOSYS = 89 -ELOOP = 90 -ERESTART = 91 -ESTRPIPE = 92 -ENOTEMPTY = 93 -EUSERS = 94 -ENOTSOCK = 95 -EDESTADDRREQ = 96 -EMSGSIZE = 97 -EPROTOTYPE = 98 -ENOPROTOOPT = 99 -EPROTONOSUPPORT = 120 -ESOCKTNOSUPPORT = 121 -EOPNOTSUPP = 122 -EPFNOSUPPORT = 123 -EAFNOSUPPORT = 124 -EADDRINUSE = 125 -EADDRNOTAVAIL = 126 -ENETDOWN = 127 -ENETUNREACH = 128 -ENETRESET = 129 -ECONNABORTED = 130 -ECONNRESET = 131 -ENOBUFS = 132 -EISCONN = 133 -ENOTCONN = 134 -ESHUTDOWN = 143 -ETOOMANYREFS = 144 -ETIMEDOUT = 145 -ECONNREFUSED = 146 -EHOSTDOWN = 147 -EHOSTUNREACH = 148 -EWOULDBLOCK = EAGAIN -EALREADY = 149 -EINPROGRESS = 150 -ESTALE = 151 -PSARGSZ = 80 -PSCOMSIZ = 14 -MAXCOMLEN = 16 -__KERN_NAUXV_IMPL = 19 -__KERN_NAUXV_IMPL = 21 -__KERN_NAUXV_IMPL = 21 -PSARGSZ = 80 - -# Included from sys/watchpoint.h -from TYPES import * - -# Included from vm/seg_enum.h - -# Included from sys/copyops.h -from TYPES import * - -# Included from sys/buf.h - -# Included from sys/kstat.h -from TYPES import * -KSTAT_STRLEN = 31 -def KSTAT_ENTER(k): return \ - -def KSTAT_EXIT(k): return \ - -KSTAT_TYPE_RAW = 0 -KSTAT_TYPE_NAMED = 1 -KSTAT_TYPE_INTR = 2 -KSTAT_TYPE_IO = 3 -KSTAT_TYPE_TIMER = 4 -KSTAT_NUM_TYPES = 5 -KSTAT_FLAG_VIRTUAL = 0x01 -KSTAT_FLAG_VAR_SIZE = 0x02 -KSTAT_FLAG_WRITABLE = 0x04 -KSTAT_FLAG_PERSISTENT = 0x08 -KSTAT_FLAG_DORMANT = 0x10 -KSTAT_FLAG_INVALID = 0x20 -KSTAT_READ = 0 -KSTAT_WRITE = 1 -KSTAT_DATA_CHAR = 0 -KSTAT_DATA_INT32 = 1 -KSTAT_DATA_UINT32 = 2 -KSTAT_DATA_INT64 = 3 -KSTAT_DATA_UINT64 = 4 -KSTAT_DATA_LONG = KSTAT_DATA_INT32 -KSTAT_DATA_ULONG = KSTAT_DATA_UINT32 -KSTAT_DATA_LONG = KSTAT_DATA_INT64 -KSTAT_DATA_ULONG = KSTAT_DATA_UINT64 -KSTAT_DATA_LONG = 7 -KSTAT_DATA_ULONG = 8 -KSTAT_DATA_LONGLONG = KSTAT_DATA_INT64 -KSTAT_DATA_ULONGLONG = KSTAT_DATA_UINT64 -KSTAT_DATA_FLOAT = 5 -KSTAT_DATA_DOUBLE = 6 -KSTAT_INTR_HARD = 0 -KSTAT_INTR_SOFT = 1 -KSTAT_INTR_WATCHDOG = 2 -KSTAT_INTR_SPURIOUS = 3 -KSTAT_INTR_MULTSVC = 4 -KSTAT_NUM_INTRS = 5 -B_BUSY = 0x0001 -B_DONE = 0x0002 -B_ERROR = 0x0004 -B_PAGEIO = 0x0010 -B_PHYS = 0x0020 -B_READ = 0x0040 -B_WRITE = 0x0100 -B_KERNBUF = 0x0008 -B_WANTED = 0x0080 -B_AGE = 0x000200 -B_ASYNC = 0x000400 -B_DELWRI = 0x000800 -B_STALE = 0x001000 -B_DONTNEED = 0x002000 -B_REMAPPED = 0x004000 -B_FREE = 0x008000 -B_INVAL = 0x010000 -B_FORCE = 0x020000 -B_HEAD = 0x040000 -B_NOCACHE = 0x080000 -B_TRUNC = 0x100000 -B_SHADOW = 0x200000 -B_RETRYWRI = 0x400000 -def notavail(bp): return \ - -def BWRITE(bp): return \ - -def BWRITE2(bp): return \ - - -# Included from sys/aio_req.h - -# Included from sys/uio.h -from TYPES import * -WP_NOWATCH = 0x01 -WP_SETPROT = 0x02 - -# Included from sys/timer.h -from TYPES import * -_TIMER_MAX = 32 -ITLK_LOCKED = 0x01 -ITLK_WANTED = 0x02 -ITLK_REMOVE = 0x04 -IT_PERLWP = 0x01 -IT_SIGNAL = 0x02 - -# Included from sys/utrap.h -UT_INSTRUCTION_DISABLED = 1 -UT_INSTRUCTION_ERROR = 2 -UT_INSTRUCTION_PROTECTION = 3 -UT_ILLTRAP_INSTRUCTION = 4 -UT_ILLEGAL_INSTRUCTION = 5 -UT_PRIVILEGED_OPCODE = 6 -UT_FP_DISABLED = 7 -UT_FP_EXCEPTION_IEEE_754 = 8 -UT_FP_EXCEPTION_OTHER = 9 -UT_TAG_OVERFLOW = 10 -UT_DIVISION_BY_ZERO = 11 -UT_DATA_EXCEPTION = 12 -UT_DATA_ERROR = 13 -UT_DATA_PROTECTION = 14 -UT_MEM_ADDRESS_NOT_ALIGNED = 15 -UT_PRIVILEGED_ACTION = 16 -UT_ASYNC_DATA_ERROR = 17 -UT_TRAP_INSTRUCTION_16 = 18 -UT_TRAP_INSTRUCTION_17 = 19 -UT_TRAP_INSTRUCTION_18 = 20 -UT_TRAP_INSTRUCTION_19 = 21 -UT_TRAP_INSTRUCTION_20 = 22 -UT_TRAP_INSTRUCTION_21 = 23 -UT_TRAP_INSTRUCTION_22 = 24 -UT_TRAP_INSTRUCTION_23 = 25 -UT_TRAP_INSTRUCTION_24 = 26 -UT_TRAP_INSTRUCTION_25 = 27 -UT_TRAP_INSTRUCTION_26 = 28 -UT_TRAP_INSTRUCTION_27 = 29 -UT_TRAP_INSTRUCTION_28 = 30 -UT_TRAP_INSTRUCTION_29 = 31 -UT_TRAP_INSTRUCTION_30 = 32 -UT_TRAP_INSTRUCTION_31 = 33 -UTRAP_V8P_FP_DISABLED = UT_FP_DISABLED -UTRAP_V8P_MEM_ADDRESS_NOT_ALIGNED = UT_MEM_ADDRESS_NOT_ALIGNED -UT_PRECISE_MAXTRAPS = 33 - -# Included from sys/refstr.h - -# Included from sys/task.h -from TYPES import * -TASK_NORMAL = 0x0 -TASK_FINAL = 0x1 -TASK_FINALITY = 0x1 - -# Included from sys/id_space.h -from TYPES import * - -# Included from sys/vmem.h -from TYPES import * -VM_SLEEP = 0x00000000 -VM_NOSLEEP = 0x00000001 -VM_PANIC = 0x00000002 -VM_KMFLAGS = 0x000000ff -VM_BESTFIT = 0x00000100 -VMEM_ALLOC = 0x01 -VMEM_FREE = 0x02 -VMEM_SPAN = 0x10 -ISP_NORMAL = 0x0 -ISP_RESERVE = 0x1 - -# Included from sys/exacct_impl.h -from TYPES import * - -# Included from sys/kmem.h -from TYPES import * -KM_SLEEP = 0x0000 -KM_NOSLEEP = 0x0001 -KM_PANIC = 0x0002 -KM_VMFLAGS = 0x00ff -KM_FLAGS = 0xffff -KMC_NOTOUCH = 0x00010000 -KMC_NODEBUG = 0x00020000 -KMC_NOMAGAZINE = 0x00040000 -KMC_NOHASH = 0x00080000 -KMC_QCACHE = 0x00100000 -_ISA_IA32 = 0 -_ISA_IA64 = 1 -SSLEEP = 1 -SRUN = 2 -SZOMB = 3 -SSTOP = 4 -SIDL = 5 -SONPROC = 6 -CLDPEND = 0x0001 -CLDCONT = 0x0002 -SSYS = 0x00000001 -STRC = 0x00000002 -SLOAD = 0x00000008 -SLOCK = 0x00000010 -SPREXEC = 0x00000020 -SPROCTR = 0x00000040 -SPRFORK = 0x00000080 -SKILLED = 0x00000100 -SULOAD = 0x00000200 -SRUNLCL = 0x00000400 -SBPTADJ = 0x00000800 -SKILLCL = 0x00001000 -SOWEUPC = 0x00002000 -SEXECED = 0x00004000 -SPASYNC = 0x00008000 -SJCTL = 0x00010000 -SNOWAIT = 0x00020000 -SVFORK = 0x00040000 -SVFWAIT = 0x00080000 -EXITLWPS = 0x00100000 -HOLDFORK = 0x00200000 -SWAITSIG = 0x00400000 -HOLDFORK1 = 0x00800000 -COREDUMP = 0x01000000 -SMSACCT = 0x02000000 -ASLWP = 0x04000000 -SPRLOCK = 0x08000000 -NOCD = 0x10000000 -HOLDWATCH = 0x20000000 -SMSFORK = 0x40000000 -SDOCORE = 0x80000000 -FORREAL = 0 -JUSTLOOKING = 1 -SUSPEND_NORMAL = 0 -SUSPEND_PAUSE = 1 -NOCLASS = (-1) - -# Included from sys/dditypes.h -DDI_DEVICE_ATTR_V0 = 0x0001 -DDI_NEVERSWAP_ACC = 0x00 -DDI_STRUCTURE_LE_ACC = 0x01 -DDI_STRUCTURE_BE_ACC = 0x02 -DDI_STRICTORDER_ACC = 0x00 -DDI_UNORDERED_OK_ACC = 0x01 -DDI_MERGING_OK_ACC = 0x02 -DDI_LOADCACHING_OK_ACC = 0x03 -DDI_STORECACHING_OK_ACC = 0x04 -DDI_DATA_SZ01_ACC = 1 -DDI_DATA_SZ02_ACC = 2 -DDI_DATA_SZ04_ACC = 4 -DDI_DATA_SZ08_ACC = 8 -VERS_ACCHDL = 0x0001 -DEVID_NONE = 0 -DEVID_SCSI3_WWN = 1 -DEVID_SCSI_SERIAL = 2 -DEVID_FAB = 3 -DEVID_ENCAP = 4 -DEVID_MAXTYPE = 4 - -# Included from sys/varargs.h - -# Included from sys/va_list.h -VA_ALIGN = 8 -def _ARGSIZEOF(t): return ((sizeof (t) + VA_ALIGN - 1) & ~(VA_ALIGN - 1)) - -VA_ALIGN = 8 -def _ARGSIZEOF(t): return ((sizeof (t) + VA_ALIGN - 1) & ~(VA_ALIGN - 1)) - -NSYSCALL = 256 -SE_32RVAL1 = 0x0 -SE_32RVAL2 = 0x1 -SE_64RVAL = 0x2 -SE_RVAL_MASK = 0x3 -SE_LOADABLE = 0x08 -SE_LOADED = 0x10 -SE_NOUNLOAD = 0x20 -SE_ARGC = 0x40 - -# Included from sys/devops.h -from TYPES import * - -# Included from sys/poll.h -POLLIN = 0x0001 -POLLPRI = 0x0002 -POLLOUT = 0x0004 -POLLRDNORM = 0x0040 -POLLWRNORM = POLLOUT -POLLRDBAND = 0x0080 -POLLWRBAND = 0x0100 -POLLNORM = POLLRDNORM -POLLERR = 0x0008 -POLLHUP = 0x0010 -POLLNVAL = 0x0020 -POLLREMOVE = 0x0800 -POLLRDDATA = 0x0200 -POLLNOERR = 0x0400 -POLLCLOSED = 0x8000 - -# Included from vm/as.h - -# Included from vm/seg.h - -# Included from sys/vnode.h -from TYPES import * -VROOT = 0x01 -VNOCACHE = 0x02 -VNOMAP = 0x04 -VDUP = 0x08 -VNOSWAP = 0x10 -VNOMOUNT = 0x20 -VISSWAP = 0x40 -VSWAPLIKE = 0x80 -VVFSLOCK = 0x100 -VVFSWAIT = 0x200 -VVMLOCK = 0x400 -VDIROPEN = 0x800 -VVMEXEC = 0x1000 -VPXFS = 0x2000 -AT_TYPE = 0x0001 -AT_MODE = 0x0002 -AT_UID = 0x0004 -AT_GID = 0x0008 -AT_FSID = 0x0010 -AT_NODEID = 0x0020 -AT_NLINK = 0x0040 -AT_SIZE = 0x0080 -AT_ATIME = 0x0100 -AT_MTIME = 0x0200 -AT_CTIME = 0x0400 -AT_RDEV = 0x0800 -AT_BLKSIZE = 0x1000 -AT_NBLOCKS = 0x2000 -AT_VCODE = 0x4000 -AT_ALL = (AT_TYPE|AT_MODE|AT_UID|AT_GID|AT_FSID|AT_NODEID|\ - AT_NLINK|AT_SIZE|AT_ATIME|AT_MTIME|AT_CTIME|\ - AT_RDEV|AT_BLKSIZE|AT_NBLOCKS|AT_VCODE) -AT_STAT = (AT_MODE|AT_UID|AT_GID|AT_FSID|AT_NODEID|AT_NLINK|\ - AT_SIZE|AT_ATIME|AT_MTIME|AT_CTIME|AT_RDEV) -AT_TIMES = (AT_ATIME|AT_MTIME|AT_CTIME) -AT_NOSET = (AT_NLINK|AT_RDEV|AT_FSID|AT_NODEID|AT_TYPE|\ - AT_BLKSIZE|AT_NBLOCKS|AT_VCODE) -VSUID = 0o4000 -VSGID = 0o2000 -VSVTX = 0o1000 -VREAD = 0o0400 -VWRITE = 0o0200 -VEXEC = 0o0100 -MODEMASK = 0o7777 -PERMMASK = 0o0777 -def MANDMODE(mode): return (((mode) & (VSGID|(VEXEC>>3))) == VSGID) - -VSA_ACL = 0x0001 -VSA_ACLCNT = 0x0002 -VSA_DFACL = 0x0004 -VSA_DFACLCNT = 0x0008 -LOOKUP_DIR = 0x01 -DUMP_ALLOC = 0 -DUMP_FREE = 1 -DUMP_SCAN = 2 -ATTR_UTIME = 0x01 -ATTR_EXEC = 0x02 -ATTR_COMM = 0x04 -ATTR_HINT = 0x08 -ATTR_REAL = 0x10 - -# Included from vm/faultcode.h -FC_HWERR = 0x1 -FC_ALIGN = 0x2 -FC_OBJERR = 0x3 -FC_PROT = 0x4 -FC_NOMAP = 0x5 -FC_NOSUPPORT = 0x6 -def FC_MAKE_ERR(e): return (((e) << 8) | FC_OBJERR) - -def FC_CODE(fc): return ((fc) & 0xff) - -def FC_ERRNO(fc): return ((unsigned)(fc) >> 8) - - -# Included from vm/hat.h -from TYPES import * - -# Included from vm/page.h -PAGE_HASHAVELEN = 4 -PAGE_HASHVPSHIFT = 6 -PG_EXCL = 0x0001 -PG_WAIT = 0x0002 -PG_PHYSCONTIG = 0x0004 -PG_MATCH_COLOR = 0x0008 -PG_NORELOC = 0x0010 -PG_FREE_LIST = 1 -PG_CACHE_LIST = 2 -PG_LIST_TAIL = 0 -PG_LIST_HEAD = 1 -def page_next_raw(PP): return page_nextn_raw((PP), 1) - -PAGE_IO_INUSE = 0x1 -PAGE_IO_WANTED = 0x2 -PGREL_NOTREL = 0x1 -PGREL_CLEAN = 0x2 -PGREL_MOD = 0x3 -P_FREE = 0x80 -P_NORELOC = 0x40 -def PP_SETAGED(pp): return ASSERT(PP_ISAGED(pp)) - -HAT_FLAGS_RESV = 0xFF000000 -HAT_LOAD = 0x00 -HAT_LOAD_LOCK = 0x01 -HAT_LOAD_ADV = 0x04 -HAT_LOAD_CONTIG = 0x10 -HAT_LOAD_NOCONSIST = 0x20 -HAT_LOAD_SHARE = 0x40 -HAT_LOAD_REMAP = 0x80 -HAT_RELOAD_SHARE = 0x100 -HAT_PLAT_ATTR_MASK = 0xF00000 -HAT_PROT_MASK = 0x0F -HAT_NOFAULT = 0x10 -HAT_NOSYNC = 0x20 -HAT_STRICTORDER = 0x0000 -HAT_UNORDERED_OK = 0x0100 -HAT_MERGING_OK = 0x0200 -HAT_LOADCACHING_OK = 0x0300 -HAT_STORECACHING_OK = 0x0400 -HAT_ORDER_MASK = 0x0700 -HAT_NEVERSWAP = 0x0000 -HAT_STRUCTURE_BE = 0x1000 -HAT_STRUCTURE_LE = 0x2000 -HAT_ENDIAN_MASK = 0x3000 -HAT_COW = 0x0001 -HAT_UNLOAD = 0x00 -HAT_UNLOAD_NOSYNC = 0x02 -HAT_UNLOAD_UNLOCK = 0x04 -HAT_UNLOAD_OTHER = 0x08 -HAT_UNLOAD_UNMAP = 0x10 -HAT_SYNC_DONTZERO = 0x00 -HAT_SYNC_ZERORM = 0x01 -HAT_SYNC_STOPON_REF = 0x02 -HAT_SYNC_STOPON_MOD = 0x04 -HAT_SYNC_STOPON_RM = (HAT_SYNC_STOPON_REF | HAT_SYNC_STOPON_MOD) -HAT_DUP_ALL = 1 -HAT_DUP_COW = 2 -HAT_MAP = 0x00 -HAT_ADV_PGUNLOAD = 0x00 -HAT_FORCE_PGUNLOAD = 0x01 -P_MOD = 0x1 -P_REF = 0x2 -P_RO = 0x4 -def hat_ismod(pp): return (hat_page_getattr(pp, P_MOD)) - -def hat_isref(pp): return (hat_page_getattr(pp, P_REF)) - -def hat_isro(pp): return (hat_page_getattr(pp, P_RO)) - -def hat_setmod(pp): return (hat_page_setattr(pp, P_MOD)) - -def hat_setref(pp): return (hat_page_setattr(pp, P_REF)) - -def hat_setrefmod(pp): return (hat_page_setattr(pp, P_REF|P_MOD)) - -def hat_clrmod(pp): return (hat_page_clrattr(pp, P_MOD)) - -def hat_clrref(pp): return (hat_page_clrattr(pp, P_REF)) - -def hat_clrrefmod(pp): return (hat_page_clrattr(pp, P_REF|P_MOD)) - -def hat_page_is_mapped(pp): return (hat_page_getshare(pp)) - -HAT_DONTALLOC = 0 -HAT_ALLOC = 1 -HRM_SHIFT = 4 -HRM_BYTES = (1 << HRM_SHIFT) -HRM_PAGES = ((HRM_BYTES * NBBY) / 2) -HRM_PGPERBYTE = (NBBY/2) -HRM_PGBYTEMASK = (HRM_PGPERBYTE-1) -HRM_HASHSIZE = 0x200 -HRM_HASHMASK = (HRM_HASHSIZE - 1) -HRM_BLIST_INCR = 0x200 -HRM_SWSMONID = 1 -SSL_NLEVELS = 4 -SSL_BFACTOR = 4 -SSL_LOG2BF = 2 -SEGP_ASYNC_FLUSH = 0x1 -SEGP_FORCE_WIRED = 0x2 -SEGP_SUCCESS = 0 -SEGP_FAIL = 1 -def seg_pages(seg): return \ - -IE_NOMEM = -1 -AS_PAGLCK = 0x80 -AS_CLAIMGAP = 0x40 -AS_UNMAPWAIT = 0x20 -def AS_TYPE_64BIT(as_): return \ - -AS_LREP_LINKEDLIST = 0 -AS_LREP_SKIPLIST = 1 -AS_MUTATION_THRESH = 225 -AH_DIR = 0x1 -AH_LO = 0x0 -AH_HI = 0x1 -AH_CONTAIN = 0x2 - -# Included from sys/ddidmareq.h -DMA_UNIT_8 = 1 -DMA_UNIT_16 = 2 -DMA_UNIT_32 = 4 -DMALIM_VER0 = ((0x86000000) + 0) -DDI_DMA_FORCE_PHYSICAL = 0x0100 -DMA_ATTR_V0 = 0 -DMA_ATTR_VERSION = DMA_ATTR_V0 -DDI_DMA_CALLBACK_RUNOUT = 0 -DDI_DMA_CALLBACK_DONE = 1 -DDI_DMA_WRITE = 0x0001 -DDI_DMA_READ = 0x0002 -DDI_DMA_RDWR = (DDI_DMA_READ | DDI_DMA_WRITE) -DDI_DMA_REDZONE = 0x0004 -DDI_DMA_PARTIAL = 0x0008 -DDI_DMA_CONSISTENT = 0x0010 -DDI_DMA_EXCLUSIVE = 0x0020 -DDI_DMA_STREAMING = 0x0040 -DDI_DMA_SBUS_64BIT = 0x2000 -DDI_DMA_MAPPED = 0 -DDI_DMA_MAPOK = 0 -DDI_DMA_PARTIAL_MAP = 1 -DDI_DMA_DONE = 2 -DDI_DMA_NORESOURCES = -1 -DDI_DMA_NOMAPPING = -2 -DDI_DMA_TOOBIG = -3 -DDI_DMA_TOOSMALL = -4 -DDI_DMA_LOCKED = -5 -DDI_DMA_BADLIMITS = -6 -DDI_DMA_STALE = -7 -DDI_DMA_BADATTR = -8 -DDI_DMA_INUSE = -9 -DDI_DMA_SYNC_FORDEV = 0x0 -DDI_DMA_SYNC_FORCPU = 0x1 -DDI_DMA_SYNC_FORKERNEL = 0x2 - -# Included from sys/ddimapreq.h - -# Included from sys/mman.h -PROT_READ = 0x1 -PROT_WRITE = 0x2 -PROT_EXEC = 0x4 -PROT_USER = 0x8 -PROT_ZFOD = (PROT_READ | PROT_WRITE | PROT_EXEC | PROT_USER) -PROT_ALL = (PROT_READ | PROT_WRITE | PROT_EXEC | PROT_USER) -PROT_NONE = 0x0 -MAP_SHARED = 1 -MAP_PRIVATE = 2 -MAP_TYPE = 0xf -MAP_FIXED = 0x10 -MAP_NORESERVE = 0x40 -MAP_ANON = 0x100 -MAP_ANONYMOUS = MAP_ANON -MAP_RENAME = 0x20 -PROC_TEXT = (PROT_EXEC | PROT_READ) -PROC_DATA = (PROT_READ | PROT_WRITE | PROT_EXEC) -SHARED = 0x10 -PRIVATE = 0x20 -VALID_ATTR = (PROT_READ|PROT_WRITE|PROT_EXEC|SHARED|PRIVATE) -PROT_EXCL = 0x20 -_MAP_LOW32 = 0x80 -_MAP_NEW = 0x80000000 -from TYPES import * -MADV_NORMAL = 0 -MADV_RANDOM = 1 -MADV_SEQUENTIAL = 2 -MADV_WILLNEED = 3 -MADV_DONTNEED = 4 -MADV_FREE = 5 -MS_OLDSYNC = 0x0 -MS_SYNC = 0x4 -MS_ASYNC = 0x1 -MS_INVALIDATE = 0x2 -MC_SYNC = 1 -MC_LOCK = 2 -MC_UNLOCK = 3 -MC_ADVISE = 4 -MC_LOCKAS = 5 -MC_UNLOCKAS = 6 -MCL_CURRENT = 0x1 -MCL_FUTURE = 0x2 -DDI_MAP_VERSION = 0x0001 -DDI_MF_USER_MAPPING = 0x1 -DDI_MF_KERNEL_MAPPING = 0x2 -DDI_MF_DEVICE_MAPPING = 0x4 -DDI_ME_GENERIC = (-1) -DDI_ME_UNIMPLEMENTED = (-2) -DDI_ME_NORESOURCES = (-3) -DDI_ME_UNSUPPORTED = (-4) -DDI_ME_REGSPEC_RANGE = (-5) -DDI_ME_RNUMBER_RANGE = (-6) -DDI_ME_INVAL = (-7) - -# Included from sys/ddipropdefs.h -def CELLS_1275_TO_BYTES(n): return ((n) * PROP_1275_CELL_SIZE) - -def BYTES_TO_1275_CELLS(n): return ((n) / PROP_1275_CELL_SIZE) - -PH_FROM_PROM = 0x01 -DDI_PROP_SUCCESS = 0 -DDI_PROP_NOT_FOUND = 1 -DDI_PROP_UNDEFINED = 2 -DDI_PROP_NO_MEMORY = 3 -DDI_PROP_INVAL_ARG = 4 -DDI_PROP_BUF_TOO_SMALL = 5 -DDI_PROP_CANNOT_DECODE = 6 -DDI_PROP_CANNOT_ENCODE = 7 -DDI_PROP_END_OF_DATA = 8 -DDI_PROP_FOUND_1275 = 255 -PROP_1275_INT_SIZE = 4 -DDI_PROP_DONTPASS = 0x0001 -DDI_PROP_CANSLEEP = 0x0002 -DDI_PROP_SYSTEM_DEF = 0x0004 -DDI_PROP_NOTPROM = 0x0008 -DDI_PROP_DONTSLEEP = 0x0010 -DDI_PROP_STACK_CREATE = 0x0020 -DDI_PROP_UNDEF_IT = 0x0040 -DDI_PROP_HW_DEF = 0x0080 -DDI_PROP_TYPE_INT = 0x0100 -DDI_PROP_TYPE_STRING = 0x0200 -DDI_PROP_TYPE_BYTE = 0x0400 -DDI_PROP_TYPE_COMPOSITE = 0x0800 -DDI_PROP_TYPE_ANY = (DDI_PROP_TYPE_INT | \ - DDI_PROP_TYPE_STRING | \ - DDI_PROP_TYPE_BYTE | \ - DDI_PROP_TYPE_COMPOSITE) -DDI_PROP_TYPE_MASK = (DDI_PROP_TYPE_INT | \ - DDI_PROP_TYPE_STRING | \ - DDI_PROP_TYPE_BYTE | \ - DDI_PROP_TYPE_COMPOSITE) -DDI_RELATIVE_ADDRESSING = "relative-addressing" -DDI_GENERIC_ADDRESSING = "generic-addressing" - -# Included from sys/ddidevmap.h -KMEM_PAGEABLE = 0x100 -KMEM_NON_PAGEABLE = 0x200 -UMEM_LOCKED = 0x400 -UMEM_TRASH = 0x800 -DEVMAP_OPS_REV = 1 -DEVMAP_DEFAULTS = 0x00 -DEVMAP_MAPPING_INVALID = 0x01 -DEVMAP_ALLOW_REMAP = 0x02 -DEVMAP_USE_PAGESIZE = 0x04 -DEVMAP_SETUP_FLAGS = \ - (DEVMAP_MAPPING_INVALID | DEVMAP_ALLOW_REMAP | DEVMAP_USE_PAGESIZE) -DEVMAP_SETUP_DONE = 0x100 -DEVMAP_LOCK_INITED = 0x200 -DEVMAP_FAULTING = 0x400 -DEVMAP_LOCKED = 0x800 -DEVMAP_FLAG_LARGE = 0x1000 -DDI_UMEM_SLEEP = 0x0 -DDI_UMEM_NOSLEEP = 0x01 -DDI_UMEM_PAGEABLE = 0x02 -DDI_UMEM_TRASH = 0x04 -DDI_UMEMLOCK_READ = 0x01 -DDI_UMEMLOCK_WRITE = 0x02 - -# Included from sys/nexusdefs.h - -# Included from sys/nexusintr.h -BUSO_REV = 4 -BUSO_REV_3 = 3 -BUSO_REV_4 = 4 -DEVO_REV = 3 -CB_REV = 1 -DDI_IDENTIFIED = (0) -DDI_NOT_IDENTIFIED = (-1) -DDI_PROBE_FAILURE = ENXIO -DDI_PROBE_DONTCARE = 0 -DDI_PROBE_PARTIAL = 1 -DDI_PROBE_SUCCESS = 2 -MAPDEV_REV = 1 -from TYPES import * -D_NEW = 0x00 -_D_OLD = 0x01 -D_TAPE = 0x08 -D_MTSAFE = 0x0020 -_D_QNEXTLESS = 0x0040 -_D_MTOCSHARED = 0x0080 -D_MTOCEXCL = 0x0800 -D_MTPUTSHARED = 0x1000 -D_MTPERQ = 0x2000 -D_MTQPAIR = 0x4000 -D_MTPERMOD = 0x6000 -D_MTOUTPERIM = 0x8000 -_D_MTCBSHARED = 0x10000 -D_MTINNER_MOD = (D_MTPUTSHARED|_D_MTOCSHARED|_D_MTCBSHARED) -D_MTOUTER_MOD = (D_MTOCEXCL) -D_MP = D_MTSAFE -D_64BIT = 0x200 -D_SYNCSTR = 0x400 -D_DEVMAP = 0x100 -D_HOTPLUG = 0x4 -SNDZERO = 0x001 -SNDPIPE = 0x002 -RNORM = 0x000 -RMSGD = 0x001 -RMSGN = 0x002 -RMODEMASK = 0x003 -RPROTDAT = 0x004 -RPROTDIS = 0x008 -RPROTNORM = 0x010 -RPROTMASK = 0x01c -RFLUSHMASK = 0x020 -RFLUSHPCPROT = 0x020 -RERRNORM = 0x001 -RERRNONPERSIST = 0x002 -RERRMASK = (RERRNORM|RERRNONPERSIST) -WERRNORM = 0x004 -WERRNONPERSIST = 0x008 -WERRMASK = (WERRNORM|WERRNONPERSIST) -FLUSHR = 0x01 -FLUSHW = 0x02 -FLUSHRW = 0x03 -FLUSHBAND = 0x04 -MAPINOK = 0x01 -NOMAPIN = 0x02 -REMAPOK = 0x04 -NOREMAP = 0x08 -S_INPUT = 0x0001 -S_HIPRI = 0x0002 -S_OUTPUT = 0x0004 -S_MSG = 0x0008 -S_ERROR = 0x0010 -S_HANGUP = 0x0020 -S_RDNORM = 0x0040 -S_WRNORM = S_OUTPUT -S_RDBAND = 0x0080 -S_WRBAND = 0x0100 -S_BANDURG = 0x0200 -RS_HIPRI = 0x01 -STRUIO_POSTPONE = 0x08 -STRUIO_MAPIN = 0x10 -MSG_HIPRI = 0x01 -MSG_ANY = 0x02 -MSG_BAND = 0x04 -MSG_XPG4 = 0x08 -MSG_IPEEK = 0x10 -MSG_DISCARDTAIL = 0x20 -MSG_HOLDSIG = 0x40 -MSG_IGNERROR = 0x80 -MSG_DELAYERROR = 0x100 -MSG_IGNFLOW = 0x200 -MSG_NOMARK = 0x400 -MORECTL = 1 -MOREDATA = 2 -MUXID_ALL = (-1) -ANYMARK = 0x01 -LASTMARK = 0x02 -_INFTIM = -1 -INFTIM = _INFTIM diff --git a/Lib/plat-sunos5/TYPES.py b/Lib/plat-sunos5/TYPES.py deleted file mode 100644 --- a/Lib/plat-sunos5/TYPES.py +++ /dev/null @@ -1,313 +0,0 @@ -# Generated by h2py from /usr/include/sys/types.h - -# Included from sys/isa_defs.h -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 8 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_ALIGNMENT_REQUIRED = 1 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 4 -_DOUBLE_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 4 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 4 -_ALIGNMENT_REQUIRED = 0 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_ALIGNMENT_REQUIRED = 1 -_LONG_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 8 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 8 -_LONG_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 - -# Included from sys/feature_tests.h -_POSIX_C_SOURCE = 1 -_LARGEFILE64_SOURCE = 1 -_LARGEFILE_SOURCE = 1 -_FILE_OFFSET_BITS = 64 -_FILE_OFFSET_BITS = 32 -_POSIX_C_SOURCE = 199506 -_POSIX_PTHREAD_SEMANTICS = 1 -_XOPEN_VERSION = 500 -_XOPEN_VERSION = 4 -_XOPEN_VERSION = 3 - -# Included from sys/machtypes.h - -# Included from sys/inttypes.h - -# Included from sys/int_types.h - -# Included from sys/int_limits.h -INT8_MAX = (127) -INT16_MAX = (32767) -INT32_MAX = (2147483647) -INTMAX_MAX = INT32_MAX -INT_LEAST8_MAX = INT8_MAX -INT_LEAST16_MAX = INT16_MAX -INT_LEAST32_MAX = INT32_MAX -INT8_MIN = (-128) -INT16_MIN = (-32767-1) -INT32_MIN = (-2147483647-1) -INTMAX_MIN = INT32_MIN -INT_LEAST8_MIN = INT8_MIN -INT_LEAST16_MIN = INT16_MIN -INT_LEAST32_MIN = INT32_MIN - -# Included from sys/int_const.h -def INT8_C(c): return (c) - -def INT16_C(c): return (c) - -def INT32_C(c): return (c) - -def INT64_C(c): return __CONCAT__(c,l) - -def INT64_C(c): return __CONCAT__(c,ll) - -def UINT8_C(c): return __CONCAT__(c,u) - -def UINT16_C(c): return __CONCAT__(c,u) - -def UINT32_C(c): return __CONCAT__(c,u) - -def UINT64_C(c): return __CONCAT__(c,ul) - -def UINT64_C(c): return __CONCAT__(c,ull) - -def INTMAX_C(c): return __CONCAT__(c,l) - -def UINTMAX_C(c): return __CONCAT__(c,ul) - -def INTMAX_C(c): return __CONCAT__(c,ll) - -def UINTMAX_C(c): return __CONCAT__(c,ull) - -def INTMAX_C(c): return (c) - -def UINTMAX_C(c): return (c) - - -# Included from sys/int_fmtio.h -PRId8 = "d" -PRId16 = "d" -PRId32 = "d" -PRId64 = "ld" -PRId64 = "lld" -PRIdLEAST8 = "d" -PRIdLEAST16 = "d" -PRIdLEAST32 = "d" -PRIdLEAST64 = "ld" -PRIdLEAST64 = "lld" -PRIi8 = "i" -PRIi16 = "i" -PRIi32 = "i" -PRIi64 = "li" -PRIi64 = "lli" -PRIiLEAST8 = "i" -PRIiLEAST16 = "i" -PRIiLEAST32 = "i" -PRIiLEAST64 = "li" -PRIiLEAST64 = "lli" -PRIo8 = "o" -PRIo16 = "o" -PRIo32 = "o" -PRIo64 = "lo" -PRIo64 = "llo" -PRIoLEAST8 = "o" -PRIoLEAST16 = "o" -PRIoLEAST32 = "o" -PRIoLEAST64 = "lo" -PRIoLEAST64 = "llo" -PRIx8 = "x" -PRIx16 = "x" -PRIx32 = "x" -PRIx64 = "lx" -PRIx64 = "llx" -PRIxLEAST8 = "x" -PRIxLEAST16 = "x" -PRIxLEAST32 = "x" -PRIxLEAST64 = "lx" -PRIxLEAST64 = "llx" -PRIX8 = "X" -PRIX16 = "X" -PRIX32 = "X" -PRIX64 = "lX" -PRIX64 = "llX" -PRIXLEAST8 = "X" -PRIXLEAST16 = "X" -PRIXLEAST32 = "X" -PRIXLEAST64 = "lX" -PRIXLEAST64 = "llX" -PRIu8 = "u" -PRIu16 = "u" -PRIu32 = "u" -PRIu64 = "lu" -PRIu64 = "llu" -PRIuLEAST8 = "u" -PRIuLEAST16 = "u" -PRIuLEAST32 = "u" -PRIuLEAST64 = "lu" -PRIuLEAST64 = "llu" -SCNd16 = "hd" -SCNd32 = "d" -SCNd64 = "ld" -SCNd64 = "lld" -SCNi16 = "hi" -SCNi32 = "i" -SCNi64 = "li" -SCNi64 = "lli" -SCNo16 = "ho" -SCNo32 = "o" -SCNo64 = "lo" -SCNo64 = "llo" -SCNu16 = "hu" -SCNu32 = "u" -SCNu64 = "lu" -SCNu64 = "llu" -SCNx16 = "hx" -SCNx32 = "x" -SCNx64 = "lx" -SCNx64 = "llx" -PRIdMAX = "ld" -PRIoMAX = "lo" -PRIxMAX = "lx" -PRIuMAX = "lu" -PRIdMAX = "lld" -PRIoMAX = "llo" -PRIxMAX = "llx" -PRIuMAX = "llu" -PRIdMAX = "d" -PRIoMAX = "o" -PRIxMAX = "x" -PRIuMAX = "u" -SCNiMAX = "li" -SCNdMAX = "ld" -SCNoMAX = "lo" -SCNxMAX = "lx" -SCNiMAX = "lli" -SCNdMAX = "lld" -SCNoMAX = "llo" -SCNxMAX = "llx" -SCNiMAX = "i" -SCNdMAX = "d" -SCNoMAX = "o" -SCNxMAX = "x" - -# Included from sys/types32.h -SHRT_MIN = (-32768) -SHRT_MAX = 32767 -USHRT_MAX = 65535 -INT_MIN = (-2147483647-1) -INT_MAX = 2147483647 -LONG_MIN = (-9223372036854775807-1) -LONG_MAX = 9223372036854775807 -LONG_MIN = (-2147483647-1) -LONG_MAX = 2147483647 -P_MYID = (-1) - -# Included from sys/select.h - -# Included from sys/time.h -TIME32_MAX = INT32_MAX -TIME32_MIN = INT32_MIN -def TIMEVAL_OVERFLOW(tv): return \ - -from TYPES import * -DST_NONE = 0 -DST_USA = 1 -DST_AUST = 2 -DST_WET = 3 -DST_MET = 4 -DST_EET = 5 -DST_CAN = 6 -DST_GB = 7 -DST_RUM = 8 -DST_TUR = 9 -DST_AUSTALT = 10 -ITIMER_REAL = 0 -ITIMER_VIRTUAL = 1 -ITIMER_PROF = 2 -ITIMER_REALPROF = 3 -def ITIMERVAL_OVERFLOW(itv): return \ - -SEC = 1 -MILLISEC = 1000 -MICROSEC = 1000000 -NANOSEC = 1000000000 - -# Included from sys/time_impl.h -def TIMESPEC_OVERFLOW(ts): return \ - -def ITIMERSPEC_OVERFLOW(it): return \ - -__CLOCK_REALTIME0 = 0 -CLOCK_VIRTUAL = 1 -CLOCK_PROF = 2 -__CLOCK_REALTIME3 = 3 -CLOCK_HIGHRES = 4 -CLOCK_MAX = 5 -CLOCK_REALTIME = __CLOCK_REALTIME3 -CLOCK_REALTIME = __CLOCK_REALTIME0 -TIMER_RELTIME = 0x0 -TIMER_ABSTIME = 0x1 - -# Included from sys/mutex.h -from TYPES import * -def MUTEX_HELD(x): return (mutex_owned(x)) - -def TICK_TO_SEC(tick): return ((tick) / hz) - -def SEC_TO_TICK(sec): return ((sec) * hz) - -def TICK_TO_MSEC(tick): return \ - -def MSEC_TO_TICK(msec): return \ - -def MSEC_TO_TICK_ROUNDUP(msec): return \ - -def TICK_TO_USEC(tick): return ((tick) * usec_per_tick) - -def USEC_TO_TICK(usec): return ((usec) / usec_per_tick) - -def USEC_TO_TICK_ROUNDUP(usec): return \ - -def TICK_TO_NSEC(tick): return ((tick) * nsec_per_tick) - -def NSEC_TO_TICK(nsec): return ((nsec) / nsec_per_tick) - -def NSEC_TO_TICK_ROUNDUP(nsec): return \ - -def TIMEVAL_TO_TICK(tvp): return \ - -def TIMESTRUC_TO_TICK(tsp): return \ - - -# Included from time.h -from TYPES import * - -# Included from iso/time_iso.h -NULL = 0 -NULL = 0 -CLOCKS_PER_SEC = 1000000 -FD_SETSIZE = 65536 -FD_SETSIZE = 1024 -_NBBY = 8 -NBBY = _NBBY -def FD_ZERO(p): return bzero((p), sizeof (*(p))) diff --git a/Lib/plat-sunos5/regen b/Lib/plat-sunos5/regen deleted file mode 100755 --- a/Lib/plat-sunos5/regen +++ /dev/null @@ -1,9 +0,0 @@ -#! /bin/sh -case `uname -sr` in -'SunOS 5.'*) ;; -*) echo Probably not on a Solaris 2 system 1>&2 - exit 1;; -esac -set -v -h2py -i '(u_long)' /usr/include/sys/types.h /usr/include/netinet/in.h /usr/include/sys/stropts.h /usr/include/dlfcn.h - diff --git a/Lib/plat-unixware7/IN.py b/Lib/plat-unixware7/IN.py deleted file mode 100644 --- a/Lib/plat-unixware7/IN.py +++ /dev/null @@ -1,836 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from netinet/in_f.h -def IN_CLASSA(i): return (((int)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((int)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((int)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((int)(i) & 0xf0000000) == 0xe0000000) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((int)(i) & 0xe0000000) == 0xe0000000) - -def IN_BADCLASS(i): return (((int)(i) & 0xf0000000) == 0xf0000000) - -INADDR_ANY = 0x00000000 -INADDR_LOOPBACK = 0x7f000001 -INADDR_BROADCAST = 0xffffffff -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -INADDR_UNSPEC_GROUP = 0xe0000000 -INADDR_ALLHOSTS_GROUP = 0xe0000001 -INADDR_ALLRTRS_GROUP = 0xe0000002 -INADDR_MAX_LOCAL_GROUP = 0xe00000ff - -# Included from netinet/in6.h - -# Included from sys/types.h -def quad_low(x): return x.val[0] - -ADT_EMASKSIZE = 8 -SHRT_MIN = -32768 -SHRT_MAX = 32767 -INT_MIN = (-2147483647-1) -INT_MAX = 2147483647 -LONG_MIN = (-2147483647-1) -LONG_MAX = 2147483647 -OFF32_MAX = LONG_MAX -ISTAT_ASSERTED = 0 -ISTAT_ASSUMED = 1 -ISTAT_NONE = 2 -OFF_MAX = OFF32_MAX -CLOCK_MAX = LONG_MAX -P_MYID = (-1) -P_MYHOSTID = (-1) - -# Included from sys/select.h -FD_SETSIZE = 4096 -NBBY = 8 -NULL = 0 - -# Included from sys/bitypes.h - -# Included from netinet/in6_f.h -def IN6_IS_ADDR_UNSPECIFIED(a): return IN6_ADDR_EQUAL_L(a, 0, 0, 0, 0) - -def IN6_SET_ADDR_UNSPECIFIED(a): return IN6_ADDR_COPY_L(a, 0, 0, 0, 0) - -def IN6_IS_ADDR_ANY(a): return IN6_ADDR_EQUAL_L(a, 0, 0, 0, 0) - -def IN6_SET_ADDR_ANY(a): return IN6_ADDR_COPY_L(a, 0, 0, 0, 0) - -def IN6_IS_ADDR_LOOPBACK(a): return IN6_ADDR_EQUAL_L(a, 0, 0, 0, 0x01000000) - -def IN6_SET_ADDR_LOOPBACK(a): return IN6_ADDR_COPY_L(a, 0, 0, 0, 0x01000000) - -IN6_MC_FLAG_PERMANENT = 0x0 -IN6_MC_FLAG_TRANSIENT = 0x1 -IN6_MC_SCOPE_NODELOCAL = 0x1 -IN6_MC_SCOPE_LINKLOCAL = 0x2 -IN6_MC_SCOPE_SITELOCAL = 0x5 -IN6_MC_SCOPE_ORGLOCAL = 0x8 -IN6_MC_SCOPE_GLOBAL = 0xE -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - - -# Included from sys/convsa.h -__NETLIB_UW211_SVR4 = 1 -__NETLIB_UW211_XPG4 = 2 -__NETLIB_GEMINI_SVR4 = 3 -__NETLIB_GEMINI_XPG4 = 4 -__NETLIB_FP1_SVR4 = 5 -__NETLIB_FP1_XPG4 = 6 -__NETLIB_BASE_VERSION__ = __NETLIB_UW211_SVR4 -__NETLIB_VERSION__ = __NETLIB_FP1_SVR4 -__NETLIB_VERSION__ = __NETLIB_FP1_XPG4 -__NETLIB_VERSION__ = __NETLIB_GEMINI_SVR4 -__NETLIB_VERSION__ = __NETLIB_GEMINI_XPG4 -__NETLIB_VERSION__ = __NETLIB_UW211_SVR4 -__NETLIB_VERSION__ = __NETLIB_UW211_XPG4 -__NETLIB_VERSION__ = __NETLIB_FP1_XPG4 - -# Included from sys/byteorder.h -LITTLE_ENDIAN = 1234 -BIG_ENDIAN = 4321 -PDP_ENDIAN = 3412 - -# Included from sys/byteorder_f.h -BYTE_ORDER = LITTLE_ENDIAN -def htonl(hl): return __htonl(hl) - -def ntohl(nl): return __ntohl(nl) - -def htons(hs): return __htons(hs) - -def ntohs(ns): return __ntohs(ns) - -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -def __NETLIB_VERSION_IS_XPG4(version): return (((version) % 2) == 0) - -def __NETLIB_VERSION_HAS_SALEN(version): return ((version) >= __NETLIB_GEMINI_SVR4) - -def __NETLIB_VERSION_IS_IKS(version): return ((version) >= __NETLIB_FP1_SVR4) - -def SA_FAMILY_GET(sa): return \ - -INET6_ADDRSTRLEN = 46 -IPV6_UNICAST_HOPS = 3 -IPV6_ADDRFORM = 24 -IPV6_MULTICAST_HOPS = 25 -IPV6_MULTICAST_IF = 26 -IPV6_MULTICAST_LOOP = 27 -IPV6_ADD_MEMBERSHIP = 28 -IPV6_DROP_MEMBERSHIP = 29 - -# Included from sys/insrem.h -def LIST_INIT(head): return \ - -def LIST_INIT(head): return \ - -def remque(a): return REMQUE(a) - - -# Included from sys/socket.h - -# Included from sys/uio.h -SHUT_RD = 0 -SHUT_WR = 1 -SHUT_RDWR = 2 - -# Included from sys/netconfig.h - -# Included from sys/cdefs.h -def __P(protos): return protos - -def __STRING(x): return #x - -def __P(protos): return () - -def __STRING(x): return "x" - -NETCONFIG = "/etc/netconfig" -NETPATH = "NETPATH" -NC_TPI_CLTS = 1 -NC_TPI_COTS = 2 -NC_TPI_COTS_ORD = 3 -NC_TPI_RAW = 4 -NC_NOFLAG = 00 -NC_VISIBLE = 0o1 -NC_BROADCAST = 0o2 -NC_NOPROTOFMLY = "-" -NC_LOOPBACK = "loopback" -NC_INET = "inet" -NC_INET6 = "inet6" -NC_IMPLINK = "implink" -NC_PUP = "pup" -NC_CHAOS = "chaos" -NC_NS = "ns" -NC_NBS = "nbs" -NC_ECMA = "ecma" -NC_DATAKIT = "datakit" -NC_CCITT = "ccitt" -NC_SNA = "sna" -NC_DECNET = "decnet" -NC_DLI = "dli" -NC_LAT = "lat" -NC_HYLINK = "hylink" -NC_APPLETALK = "appletalk" -NC_NIT = "nit" -NC_IEEE802 = "ieee802" -NC_OSI = "osi" -NC_X25 = "x25" -NC_OSINET = "osinet" -NC_GOSIP = "gosip" -NC_NETWARE = "netware" -NC_NOPROTO = "-" -NC_TCP = "tcp" -NC_UDP = "udp" -NC_ICMP = "icmp" -NC_IPX = "ipx" -NC_SPX = "spx" -NC_TPI_CLTS = 1 -NC_TPI_COTS = 2 -NC_TPI_COTS_ORD = 3 -NC_TPI_RAW = 4 -SOCK_STREAM = 2 -SOCK_DGRAM = 1 -SOCK_RAW = 4 -SOCK_RDM = 5 -SOCK_SEQPACKET = 6 -SO_DEBUG = 0x0001 -SO_ACCEPTCONN = 0x0002 -SO_REUSEADDR = 0x0004 -SO_KEEPALIVE = 0x0008 -SO_DONTROUTE = 0x0010 -SO_BROADCAST = 0x0020 -SO_USELOOPBACK = 0x0040 -SO_LINGER = 0x0080 -SO_OOBINLINE = 0x0100 -SO_ORDREL = 0x0200 -SO_IMASOCKET = 0x0400 -SO_MGMT = 0x0800 -SO_REUSEPORT = 0x1000 -SO_LISTENING = 0x2000 -SO_RDWR = 0x4000 -SO_SEMA = 0x8000 -SO_DONTLINGER = (~SO_LINGER) -SO_SNDBUF = 0x1001 -SO_RCVBUF = 0x1002 -SO_SNDLOWAT = 0x1003 -SO_RCVLOWAT = 0x1004 -SO_SNDTIMEO = 0x1005 -SO_RCVTIMEO = 0x1006 -SO_ERROR = 0x1007 -SO_TYPE = 0x1008 -SO_PROTOTYPE = 0x1009 -SO_ALLRAW = 0x100a -SOL_SOCKET = 0xffff -AF_UNSPEC = 0 -AF_UNIX = 1 -AF_LOCAL = AF_UNIX -AF_INET = 2 -AF_IMPLINK = 3 -AF_PUP = 4 -AF_CHAOS = 5 -AF_NS = 6 -AF_NBS = 7 -AF_ECMA = 8 -AF_DATAKIT = 9 -AF_CCITT = 10 -AF_SNA = 11 -AF_DECnet = 12 -AF_DLI = 13 -AF_LAT = 14 -AF_HYLINK = 15 -AF_APPLETALK = 16 -AF_NIT = 17 -AF_802 = 18 -AF_OSI = 19 -AF_ISO = AF_OSI -AF_X25 = 20 -AF_OSINET = 21 -AF_GOSIP = 22 -AF_YNET = 23 -AF_ROUTE = 24 -AF_LINK = 25 -pseudo_AF_XTP = 26 -AF_INET6 = 27 -AF_MAX = 27 -AF_INET_BSWAP = 0x0200 -PF_UNSPEC = AF_UNSPEC -PF_UNIX = AF_UNIX -PF_LOCAL = AF_LOCAL -PF_INET = AF_INET -PF_IMPLINK = AF_IMPLINK -PF_PUP = AF_PUP -PF_CHAOS = AF_CHAOS -PF_NS = AF_NS -PF_NBS = AF_NBS -PF_ECMA = AF_ECMA -PF_DATAKIT = AF_DATAKIT -PF_CCITT = AF_CCITT -PF_SNA = AF_SNA -PF_DECnet = AF_DECnet -PF_DLI = AF_DLI -PF_LAT = AF_LAT -PF_HYLINK = AF_HYLINK -PF_APPLETALK = AF_APPLETALK -PF_NIT = AF_NIT -PF_802 = AF_802 -PF_OSI = AF_OSI -PF_ISO = PF_OSI -PF_X25 = AF_X25 -PF_OSINET = AF_OSINET -PF_GOSIP = AF_GOSIP -PF_YNET = AF_YNET -PF_ROUTE = AF_ROUTE -PF_LINK = AF_LINK -pseudo_PF_XTP = pseudo_AF_XTP -PF_INET6 = AF_INET6 -PF_MAX = AF_MAX -SOMAXCONN = 5 -SCM_RIGHTS = 1 -MSG_OOB = 0x1 -MSG_PEEK = 0x2 -MSG_DONTROUTE = 0x4 -MSG_CTRUNC = 0x8 -MSG_TRUNC = 0x10 -MSG_EOR = 0x30 -MSG_WAITALL = 0x20 -MSG_MAXIOVLEN = 16 -def OPTLEN(x): return ((((x) + sizeof(int) - 1) / sizeof(int)) * sizeof(int)) - -GIARG = 0x1 -CONTI = 0x2 -GITAB = 0x4 -SOCKETSYS = 88 -SOCKETSYS = 83 -SO_ACCEPT = 1 -SO_BIND = 2 -SO_CONNECT = 3 -SO_GETPEERNAME = 4 -SO_GETSOCKNAME = 5 -SO_GETSOCKOPT = 6 -SO_LISTEN = 7 -SO_RECV = 8 -SO_RECVFROM = 9 -SO_SEND = 10 -SO_SENDTO = 11 -SO_SETSOCKOPT = 12 -SO_SHUTDOWN = 13 -SO_SOCKET = 14 -SO_SOCKPOLL = 15 -SO_GETIPDOMAIN = 16 -SO_SETIPDOMAIN = 17 -SO_ADJTIME = 18 - -# Included from sys/stream.h - -# Included from sys/cred.h - -# Included from sys/ksynch.h - -# Included from sys/dl.h -SIGNBIT = 0x80000000 - -# Included from sys/ipl.h - -# Included from sys/disp_p.h - -# Included from sys/trap.h -DIVERR = 0 -SGLSTP = 1 -NMIFLT = 2 -BPTFLT = 3 -INTOFLT = 4 -BOUNDFLT = 5 -INVOPFLT = 6 -NOEXTFLT = 7 -DBLFLT = 8 -EXTOVRFLT = 9 -INVTSSFLT = 10 -SEGNPFLT = 11 -STKFLT = 12 -GPFLT = 13 -PGFLT = 14 -EXTERRFLT = 16 -ALIGNFLT = 17 -MCEFLT = 18 -USERFLT = 0x100 -TRP_PREEMPT = 0x200 -TRP_UNUSED = 0x201 -PF_ERR_MASK = 0x01 -PF_ERR_PAGE = 0 -PF_ERR_PROT = 1 -PF_ERR_WRITE = 2 -PF_ERR_USER = 4 -EVT_STRSCHED = 0x04 -EVT_GLOBCALLOUT = 0x08 -EVT_LCLCALLOUT = 0x10 -EVT_SOFTINTMASK = (EVT_STRSCHED|EVT_GLOBCALLOUT|EVT_LCLCALLOUT) -PL0 = 0 -PL1 = 1 -PL2 = 2 -PL3 = 3 -PL4 = 4 -PL5 = 5 -PL6 = 6 -PLHI = 8 -PL7 = PLHI -PLBASE = PL0 -PLTIMEOUT = PL1 -PLDISK = PL5 -PLSTR = PL6 -PLTTY = PLSTR -PLMIN = PL0 -PLMIN = PL1 -MAX_INTR_LEVELS = 10 -MAX_INTR_NESTING = 50 -STRSCHED = EVT_STRSCHED -GLOBALSOFTINT = EVT_GLOBCALLOUT -LOCALSOFTINT = EVT_LCLCALLOUT - -# Included from sys/ksynch_p.h -def GET_TIME(timep): return \ - -LK_THRESHOLD = 500000 - -# Included from sys/list.h - -# Included from sys/listasm.h -def remque_null(e): return \ - -def LS_ISEMPTY(listp): return \ - -LK_BASIC = 0x1 -LK_SLEEP = 0x2 -LK_NOSTATS = 0x4 -def CYCLES_SINCE(c): return CYCLES_BETWEEN((c), CYCLES()) - -LSB_NLKDS = 92 -EVT_RUNRUN = 0x01 -EVT_KPRUNRUN = 0x02 -SP_UNLOCKED = 0 -SP_LOCKED = 1 -KS_LOCKTEST = 0x01 -KS_MPSTATS = 0x02 -KS_DEINITED = 0x04 -KS_NVLTTRACE = 0x08 -RWS_READ = (ord('r')) -RWS_WRITE = (ord('w')) -RWS_UNLOCKED = (ord('u')) -RWS_BUSY = (ord('b')) -def SLEEP_LOCKOWNED(lkp): return \ - -def SLEEP_DISOWN(lkp): return \ - -KS_NOPRMPT = 0x00000001 -__KS_LOCKTEST = KS_LOCKTEST -__KS_LOCKTEST = 0 -__KS_MPSTATS = KS_MPSTATS -__KS_MPSTATS = 0 -__KS_NVLTTRACE = KS_NVLTTRACE -__KS_NVLTTRACE = 0 -KSFLAGS = (__KS_LOCKTEST|__KS_MPSTATS|__KS_NVLTTRACE) -KSVUNIPROC = 1 -KSVMPDEBUG = 2 -KSVMPNODEBUG = 3 -KSVFLAG = KSVUNIPROC -KSVFLAG = KSVMPDEBUG -KSVFLAG = KSVMPNODEBUG - -# Included from sys/ksinline.h -_A_SP_LOCKED = 1 -_A_SP_UNLOCKED = 0 -_A_INVPL = -1 -def _ATOMIC_INT_INCR(atomic_intp): return \ - -def _ATOMIC_INT_DECR(atomic_intp): return \ - -def ATOMIC_INT_READ(atomic_intp): return _ATOMIC_INT_READ(atomic_intp) - -def ATOMIC_INT_INCR(atomic_intp): return _ATOMIC_INT_INCR(atomic_intp) - -def ATOMIC_INT_DECR(atomic_intp): return _ATOMIC_INT_DECR(atomic_intp) - -def FSPIN_INIT(lp): return - -def FSPIN_LOCK(l): return DISABLE() - -def FSPIN_TRYLOCK(l): return (DISABLE(), B_TRUE) - -def FSPIN_UNLOCK(l): return ENABLE() - -def LOCK_DEINIT(lp): return - -def LOCK_DEALLOC(lp): return - -def LOCK_OWNED(lp): return (B_TRUE) - -def RW_DEINIT(lp): return - -def RW_DEALLOC(lp): return - -def RW_OWNED(lp): return (B_TRUE) - -def IS_LOCKED(lockp): return B_FALSE - -def LOCK_PLMIN(lockp): return \ - -def TRYLOCK_PLMIN(lockp): return LOCK_PLMIN(lockp) - -def LOCK_SH_PLMIN(lockp): return LOCK_PLMIN(lockp) - -def RW_RDLOCK_PLMIN(lockp): return LOCK_PLMIN(lockp) - -def RW_WRLOCK_PLMIN(lockp): return LOCK_PLMIN(lockp) - -def LOCK_DEINIT(l): return - -def LOCK_PLMIN(lockp): return LOCK((lockp), PLMIN) - -def TRYLOCK_PLMIN(lockp): return TRYLOCK((lockp), PLMIN) - -def LOCK_SH_PLMIN(lockp): return LOCK_SH((lockp), PLMIN) - -def RW_RDLOCK_PLMIN(lockp): return RW_RDLOCK((lockp), PLMIN) - -def RW_WRLOCK_PLMIN(lockp): return RW_WRLOCK((lockp), PLMIN) - -def FSPIN_IS_LOCKED(fsp): return B_FALSE - -def SPIN_IS_LOCKED(lockp): return B_FALSE - -def FSPIN_OWNED(l): return (B_TRUE) - -CR_MLDREAL = 0x00000001 -CR_RDUMP = 0x00000002 -def crhold(credp): return crholdn((credp), 1) - -def crfree(credp): return crfreen((credp), 1) - - -# Included from sys/strmdep.h -def str_aligned(X): return (((uint)(X) & (sizeof(int) - 1)) == 0) - - -# Included from sys/engine.h - -# Included from sys/clock.h - -# Included from sys/time.h -DST_NONE = 0 -DST_USA = 1 -DST_AUST = 2 -DST_WET = 3 -DST_MET = 4 -DST_EET = 5 -DST_CAN = 6 -DST_GB = 7 -DST_RUM = 8 -DST_TUR = 9 -DST_AUSTALT = 10 -ITIMER_REAL = 0 -ITIMER_VIRTUAL = 1 -ITIMER_PROF = 2 -FD_SETSIZE = 4096 -FD_NBBY = 8 - -# Included from time.h -NULL = 0 -CLOCKS_PER_SEC = 1000000 - -# Included from sys/clock_p.h -CGBITS = 4 -IDBITS = 28 -def toid_unpackcg(idval): return (((idval) >> IDBITS) & 0xf) - -def toid_unpackid(idval): return ((idval) & 0xfffffff) - -def toid_unpackcg(idval): return 0 - -def toid_unpackid(idval): return (idval) - -NCALLOUT_HASH = 1024 -CALLOUT_MAXVAL = 0x7fffffff -TO_PERIODIC = 0x80000000 -TO_IMMEDIATE = 0x80000000 -SEC = 1 -MILLISEC = 1000 -MICROSEC = 1000000 -NANOSEC = 1000000000 -SECHR = (60*60) -SECDAY = (24*SECHR) -SECYR = (365*SECDAY) -def TIME_OWNED_R(cgnum): return (B_TRUE) - -LOOPSECONDS = 1800 -LOOPMICROSECONDS = (LOOPSECONDS * MICROSEC) -def TICKS_SINCE(t): return TICKS_BETWEEN(t, TICKS()) - -MAXRQS = 2 -E_OFFLINE = 0x01 -E_BAD = 0x02 -E_SHUTDOWN = 0x04 -E_DRIVER = 0x08 -E_DEFAULTKEEP = 0x100 -E_DRIVERBOUND = 0x200 -E_EXCLUSIVE = 0x400 -E_CGLEADER = 0x800 -E_NOWAY = (E_OFFLINE|E_BAD|E_SHUTDOWN) -E_BOUND = 0x01 -E_GLOBAL = 0x00 -E_UNAVAIL = -1 -ENGINE_ONLINE = 1 -def PROCESSOR_UNMAP(e): return ((e) - engine) - -BOOTENG = 0 -QMOVED = 0x0001 -QWANTR = 0x0002 -QWANTW = 0x0004 -QFULL = 0x0008 -QREADR = 0x0010 -QUSE = 0x0020 -QNOENB = 0x0040 -QUP = 0x0080 -QBACK = 0x0100 -QINTER = 0x0200 -QPROCSON = 0x0400 -QTOENAB = 0x0800 -QFREEZE = 0x1000 -QBOUND = 0x2000 -QDEFCNT = 0x4000 -QENAB = 0x0001 -QSVCBUSY = 0x0002 -STRM_PUTCNT_TABLES = 31 -def STRM_MYENG_PUTCNT(sdp): return STRM_PUTCNT(l.eng_num, sdp) - -QB_FULL = 0x01 -QB_WANTW = 0x02 -QB_BACK = 0x04 -NBAND = 256 -DB_WASDUPED = 0x1 -DB_2PIECE = 0x2 -STRLEAKHASHSZ = 1021 -MSGMARK = 0x01 -MSGNOLOOP = 0x02 -MSGDELIM = 0x04 -MSGNOGET = 0x08 -MSGLOG = 0x10 -M_DATA = 0x00 -M_PROTO = 0x01 -M_BREAK = 0x08 -M_PASSFP = 0x09 -M_SIG = 0x0b -M_DELAY = 0x0c -M_CTL = 0x0d -M_IOCTL = 0x0e -M_SETOPTS = 0x10 -M_RSE = 0x11 -M_TRAIL = 0x12 -M_IOCACK = 0x81 -M_IOCNAK = 0x82 -M_PCPROTO = 0x83 -M_PCSIG = 0x84 -M_READ = 0x85 -M_FLUSH = 0x86 -M_STOP = 0x87 -M_START = 0x88 -M_HANGUP = 0x89 -M_ERROR = 0x8a -M_COPYIN = 0x8b -M_COPYOUT = 0x8c -M_IOCDATA = 0x8d -M_PCRSE = 0x8e -M_STOPI = 0x8f -M_STARTI = 0x90 -M_PCCTL = 0x91 -M_PCSETOPTS = 0x92 -QNORM = 0x00 -QPCTL = 0x80 -STRCANON = 0x01 -RECOPY = 0x02 -SO_ALL = 0x003f -SO_READOPT = 0x0001 -SO_WROFF = 0x0002 -SO_MINPSZ = 0x0004 -SO_MAXPSZ = 0x0008 -SO_HIWAT = 0x0010 -SO_LOWAT = 0x0020 -SO_MREADON = 0x0040 -SO_MREADOFF = 0x0080 -SO_NDELON = 0x0100 -SO_NDELOFF = 0x0200 -SO_ISTTY = 0x0400 -SO_ISNTTY = 0x0800 -SO_TOSTOP = 0x1000 -SO_TONSTOP = 0x2000 -SO_BAND = 0x4000 -SO_DELIM = 0x8000 -SO_NODELIM = 0x010000 -SO_STRHOLD = 0x020000 -SO_LOOP = 0x040000 -DRVOPEN = 0x0 -MODOPEN = 0x1 -CLONEOPEN = 0x2 -OPENFAIL = -1 -BPRI_LO = 1 -BPRI_MED = 2 -BPRI_HI = 3 -INFPSZ = -1 -FLUSHALL = 1 -FLUSHDATA = 0 -STRHIGH = 5120 -STRLOW = 1024 -MAXIOCBSZ = 1024 -def straln(a): return (caddr_t)((int)(a) & ~(sizeof(int)-1)) - -IPM_ID = 200 -ICMPM_ID = 201 -TCPM_ID = 202 -UDPM_ID = 203 -ARPM_ID = 204 -APPM_ID = 205 -RIPM_ID = 206 -PPPM_ID = 207 -AHDLCM_ID = 208 -MHDLCRIPM_ID = 209 -HDLCM_ID = 210 -PPCID_ID = 211 -IGMPM_ID = 212 -IPIPM_ID = 213 -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPIP = 4 -IPPROTO_TCP = 6 -IPPROTO_EGP = 8 -IPPROTO_PUP = 12 -IPPROTO_UDP = 17 -IPPROTO_IDP = 22 -IPPROTO_TP = 29 -IPPROTO_IPV6 = 41 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_HELLO = 63 -IPPROTO_ND = 77 -IPPROTO_EON = 80 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPORT_ECHO = 7 -IPPORT_DISCARD = 9 -IPPORT_SYSTAT = 11 -IPPORT_DAYTIME = 13 -IPPORT_NETSTAT = 15 -IPPORT_FTP = 21 -IPPORT_TELNET = 23 -IPPORT_SMTP = 25 -IPPORT_TIMESERVER = 37 -IPPORT_NAMESERVER = 42 -IPPORT_WHOIS = 43 -IPPORT_MTP = 57 -IPPORT_TFTP = 69 -IPPORT_RJE = 77 -IPPORT_FINGER = 79 -IPPORT_TTYLINK = 87 -IPPORT_SUPDUP = 95 -IPPORT_EXECSERVER = 512 -IPPORT_LOGINSERVER = 513 -IPPORT_CMDSERVER = 514 -IPPORT_EFSSERVER = 520 -IPPORT_BIFFUDP = 512 -IPPORT_WHOSERVER = 513 -IPPORT_ROUTESERVER = 520 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 65535 -IPPORT_RESERVED_LOW = 512 -IPPORT_RESERVED_HIGH = 1023 -IPPORT_USERRESERVED_LOW = 32768 -IPPORT_USERRESERVED_HIGH = 65535 -INET_ADDRSTRLEN = 16 -IP_OPTIONS = 1 -IP_TOS = 2 -IP_TTL = 3 -IP_HDRINCL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_LOOP = 10 -IP_ADD_MEMBERSHIP = 11 -IP_DROP_MEMBERSHIP = 12 -IP_BROADCAST_IF = 14 -IP_RECVIFINDEX = 15 -IP_MULTICAST_TTL = 16 -MRT_INIT = 17 -MRT_DONE = 18 -MRT_ADD_VIF = 19 -MRT_DEL_VIF = 20 -MRT_ADD_MFC = 21 -MRT_DEL_MFC = 22 -MRT_VERSION = 23 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -INADDR_UNSPEC_GROUP = 0xe0000000 -INADDR_ALLHOSTS_GROUP = 0xe0000001 -INADDR_ALLRTRS_GROUP = 0xe0000002 -INADDR_MAX_LOCAL_GROUP = 0xe00000ff - -# Included from netinet/in_mp.h - -# Included from netinet/in_mp_ddi.h - -# Included from sys/inline.h -IP_HIER_BASE = (20) -def ASSERT_LOCK(x): return - -def ASSERT_WRLOCK(x): return - -def ASSERT_UNLOCK(x): return - -def CANPUT(q): return canput((q)) - -def CANPUTNEXT(q): return canputnext((q)) - -INET_DEBUG = 1 diff --git a/Lib/plat-unixware7/STROPTS.py b/Lib/plat-unixware7/STROPTS.py deleted file mode 100644 --- a/Lib/plat-unixware7/STROPTS.py +++ /dev/null @@ -1,328 +0,0 @@ -# Generated by h2py from /usr/include/sys/stropts.h - -# Included from sys/types.h -def quad_low(x): return x.val[0] - -ADT_EMASKSIZE = 8 -SHRT_MIN = -32768 -SHRT_MAX = 32767 -INT_MIN = (-2147483647-1) -INT_MAX = 2147483647 -LONG_MIN = (-2147483647-1) -LONG_MAX = 2147483647 -OFF32_MAX = LONG_MAX -ISTAT_ASSERTED = 0 -ISTAT_ASSUMED = 1 -ISTAT_NONE = 2 -OFF_MAX = OFF32_MAX -CLOCK_MAX = LONG_MAX -P_MYID = (-1) -P_MYHOSTID = (-1) - -# Included from sys/select.h -FD_SETSIZE = 4096 -NBBY = 8 -NULL = 0 - -# Included from sys/conf.h -D_NEW = 0x00 -D_OLD = 0x01 -D_DMA = 0x02 -D_BLKOFF = 0x400 -D_LFS = 0x8000 -D_STR = 0x0800 -D_MOD = 0x1000 -D_PSEUDO = 0x2000 -D_RANDOM = 0x4000 -D_HOT = 0x10000 -D_SEEKNEG = 0x04 -D_TAPE = 0x08 -D_NOBRKUP = 0x10 -D_INITPUB = 0x20 -D_NOSPECMACDATA = 0x40 -D_RDWEQ = 0x80 -SECMASK = (D_INITPUB|D_NOSPECMACDATA|D_RDWEQ) -DAF_REQDMA = 0x1 -DAF_PHYSREQ = 0x2 -DAF_PRE8 = 0x4 -DAF_STATIC = 0x8 -DAF_STR = 0x10 -D_MP = 0x100 -D_UPF = 0x200 -ROOTFS_NAMESZ = 7 -FMNAMESZ = 8 -MCD_VERSION = 1 -DI_BCBP = 0 -DI_MEDIA = 1 - -# Included from sys/secsys.h -ES_MACOPENLID = 1 -ES_MACSYSLID = 2 -ES_MACROOTLID = 3 -ES_PRVINFO = 4 -ES_PRVSETCNT = 5 -ES_PRVSETS = 6 -ES_MACADTLID = 7 -ES_PRVID = 8 -ES_TPGETMAJOR = 9 -SA_EXEC = 0o01 -SA_WRITE = 0o02 -SA_READ = 0o04 -SA_SUBSIZE = 0o10 - -# Included from sys/stropts_f.h -X_STR = (ord('S')<<8) -X_I_BASE = (X_STR|0o200) -X_I_NREAD = (X_STR|0o201) -X_I_PUSH = (X_STR|0o202) -X_I_POP = (X_STR|0o203) -X_I_LOOK = (X_STR|0o204) -X_I_FLUSH = (X_STR|0o205) -X_I_SRDOPT = (X_STR|0o206) -X_I_GRDOPT = (X_STR|0o207) -X_I_STR = (X_STR|0o210) -X_I_SETSIG = (X_STR|0o211) -X_I_GETSIG = (X_STR|0o212) -X_I_FIND = (X_STR|0o213) -X_I_LINK = (X_STR|0o214) -X_I_UNLINK = (X_STR|0o215) -X_I_PEEK = (X_STR|0o217) -X_I_FDINSERT = (X_STR|0o220) -X_I_SENDFD = (X_STR|0o221) -X_I_RECVFD = (X_STR|0o222) - -# Included from unistd.h - -# Included from sys/unistd.h -R_OK = 0o04 -W_OK = 0o02 -X_OK = 0o01 -F_OK = 000 -EFF_ONLY_OK = 0o10 -EX_OK = 0o20 -SEEK_SET = 0 -SEEK_CUR = 1 -SEEK_END = 2 -_SC_ARG_MAX = 1 -_SC_CHILD_MAX = 2 -_SC_CLK_TCK = 3 -_SC_NGROUPS_MAX = 4 -_SC_OPEN_MAX = 5 -_SC_JOB_CONTROL = 6 -_SC_SAVED_IDS = 7 -_SC_VERSION = 8 -_SC_PASS_MAX = 9 -_SC_LOGNAME_MAX = 10 -_SC_PAGESIZE = 11 -_SC_PAGE_SIZE = _SC_PAGESIZE -_SC_XOPEN_VERSION = 12 -_SC_NACLS_MAX = 13 -_SC_NPROCESSORS_CONF = 14 -_SC_NPROCESSORS_ONLN = 15 -_SC_NPROCESSES = 39 -_SC_TOTAL_MEMORY = 40 -_SC_USEABLE_MEMORY = 41 -_SC_GENERAL_MEMORY = 42 -_SC_DEDICATED_MEMORY = 43 -_SC_NCGS_CONF = 44 -_SC_NCGS_ONLN = 45 -_SC_MAX_CPUS_PER_CG = 46 -_SC_CG_SIMPLE_IMPL = 47 -_SC_CACHE_LINE = 48 -_SC_SYSTEM_ID = 49 -_SC_THREADS = 51 -_SC_THREAD_ATTR_STACKADDR = 52 -_SC_THREAD_ATTR_STACKSIZE = 53 -_SC_THREAD_DESTRUCTOR_ITERATIONS = 54 -_SC_THREAD_KEYS_MAX = 55 -_SC_THREAD_PRIORITY_SCHEDULING = 56 -_SC_THREAD_PRIO_INHERIT = 57 -_SC_THREAD_PRIO_PROTECT = 58 -_SC_THREAD_STACK_MIN = 59 -_SC_THREAD_PROCESS_SHARED = 60 -_SC_THREAD_SAFE_FUNCTIONS = 61 -_SC_THREAD_THREADS_MAX = 62 -_SC_KERNEL_VM = 63 -_SC_TZNAME_MAX = 320 -_SC_STREAM_MAX = 321 -_SC_XOPEN_CRYPT = 323 -_SC_XOPEN_ENH_I18N = 324 -_SC_XOPEN_SHM = 325 -_SC_XOPEN_XCU_VERSION = 327 -_SC_AES_OS_VERSION = 330 -_SC_ATEXIT_MAX = 331 -_SC_2_C_BIND = 350 -_SC_2_C_DEV = 351 -_SC_2_C_VERSION = 352 -_SC_2_CHAR_TERM = 353 -_SC_2_FORT_DEV = 354 -_SC_2_FORT_RUN = 355 -_SC_2_LOCALEDEF = 356 -_SC_2_SW_DEV = 357 -_SC_2_UPE = 358 -_SC_2_VERSION = 359 -_SC_BC_BASE_MAX = 370 -_SC_BC_DIM_MAX = 371 -_SC_BC_SCALE_MAX = 372 -_SC_BC_STRING_MAX = 373 -_SC_COLL_WEIGHTS_MAX = 380 -_SC_EXPR_NEST_MAX = 381 -_SC_LINE_MAX = 382 -_SC_RE_DUP_MAX = 383 -_SC_IOV_MAX = 390 -_SC_NPROC_CONF = 391 -_SC_NPROC_ONLN = 392 -_SC_XOPEN_UNIX = 400 -_SC_SEMAPHORES = 440 -_CS_PATH = 1 -__O_CS_HOSTNAME = 2 -_CS_RELEASE = 3 -_CS_VERSION = 4 -__O_CS_MACHINE = 5 -__O_CS_ARCHITECTURE = 6 -_CS_HW_SERIAL = 7 -__O_CS_HW_PROVIDER = 8 -_CS_SRPC_DOMAIN = 9 -_CS_INITTAB_NAME = 10 -__O_CS_SYSNAME = 11 -_CS_LFS_CFLAGS = 20 -_CS_LFS_LDFLAGS = 21 -_CS_LFS_LIBS = 22 -_CS_LFS_LINTFLAGS = 23 -_CS_LFS64_CFLAGS = 24 -_CS_LFS64_LDFLAGS = 25 -_CS_LFS64_LIBS = 26 -_CS_LFS64_LINTFLAGS = 27 -_CS_ARCHITECTURE = 100 -_CS_BUSTYPES = 101 -_CS_HOSTNAME = 102 -_CS_HW_PROVIDER = 103 -_CS_KERNEL_STAMP = 104 -_CS_MACHINE = 105 -_CS_OS_BASE = 106 -_CS_OS_PROVIDER = 107 -_CS_SYSNAME = 108 -_CS_USER_LIMIT = 109 -_PC_LINK_MAX = 1 -_PC_MAX_CANON = 2 -_PC_MAX_INPUT = 3 -_PC_NAME_MAX = 4 -_PC_PATH_MAX = 5 -_PC_PIPE_BUF = 6 -_PC_NO_TRUNC = 7 -_PC_VDISABLE = 8 -_PC_CHOWN_RESTRICTED = 9 -_PC_FILESIZEBITS = 10 -_POSIX_VERSION = 199009 -_XOPEN_VERSION = 4 -GF_PATH = "/etc/group" -PF_PATH = "/etc/passwd" -F_ULOCK = 0 -F_LOCK = 1 -F_TLOCK = 2 -F_TEST = 3 -_POSIX_JOB_CONTROL = 1 -_POSIX_SAVED_IDS = 1 -_POSIX_VDISABLE = 0 -NULL = 0 -STDIN_FILENO = 0 -STDOUT_FILENO = 1 -STDERR_FILENO = 2 -_XOPEN_UNIX = 1 -_XOPEN_ENH_I18N = 1 -_XOPEN_XPG4 = 1 -_POSIX2_C_VERSION = 199209 -_POSIX2_VERSION = 199209 -_XOPEN_XCU_VERSION = 4 -_POSIX_SEMAPHORES = 1 -_POSIX_THREADS = 1 -_POSIX_THREAD_ATTR_STACKADDR = 1 -_POSIX_THREAD_ATTR_STACKSIZE = 1 -_POSIX_THREAD_PRIORITY_SCHEDULING = 1 -_POSIX_THREAD_PROCESS_SHARED = 1 -_POSIX_THREAD_SAFE_FUNCTIONS = 1 -_POSIX2_C_BIND = 1 -_POSIX2_CHAR_TERM = 1 -_POSIX2_FORT_RUN = 1 -_POSIX2_LOCALEDEF = 1 -_POSIX2_UPE = 1 -_LFS_ASYNCHRONOUS_IO = 1 -_LFS_LARGEFILE = 1 -_LFS64_ASYNCHRONOUS_IO = 1 -_LFS64_LARGEFILE = 1 -_LFS64_STDIO = 1 -FMNAMESZ = 8 -SNDZERO = 0x001 -SNDPIPE = 0x002 -RNORM = 0x000 -RMSGD = 0x001 -RMSGN = 0x002 -RMODEMASK = 0x003 -RPROTDAT = 0x004 -RPROTDIS = 0x008 -RPROTNORM = 0x010 -RPROTMASK = 0x01c -FLUSHR = 0x01 -FLUSHW = 0x02 -FLUSHRW = 0x03 -FLUSHBAND = 0x04 -S_INPUT = 0x0001 -S_HIPRI = 0x0002 -S_OUTPUT = 0x0004 -S_MSG = 0x0008 -S_ERROR = 0x0010 -S_HANGUP = 0x0020 -S_RDNORM = 0x0040 -S_WRNORM = S_OUTPUT -S_RDBAND = 0x0080 -S_WRBAND = 0x0100 -S_BANDURG = 0x0200 -RS_HIPRI = 0x01 -MSG_HIPRI = 0x01 -MSG_ANY = 0x02 -MSG_BAND = 0x04 -MSG_DISCARD = 0x08 -MSG_PEEKIOCTL = 0x10 -MORECTL = 1 -MOREDATA = 2 -MUXID_ALL = (-1) -ANYMARK = 0x01 -LASTMARK = 0x02 -STR = (ord('S')<<8) -I_NREAD = (STR|0o1) -I_PUSH = (STR|0o2) -I_POP = (STR|0o3) -I_LOOK = (STR|0o4) -I_FLUSH = (STR|0o5) -I_SRDOPT = (STR|0o6) -I_GRDOPT = (STR|0o7) -I_STR = (STR|0o10) -I_SETSIG = (STR|0o11) -I_GETSIG = (STR|0o12) -I_FIND = (STR|0o13) -I_LINK = (STR|0o14) -I_UNLINK = (STR|0o15) -I_PEEK = (STR|0o17) -I_FDINSERT = (STR|0o20) -I_SENDFD = (STR|0o21) -I_RECVFD = (STR|0o22) -I_E_RECVFD = (STR|0o16) -I_RECVFD = (STR|0o16) -I_RECVFD = (STR|0o22) -I_SWROPT = (STR|0o23) -I_GWROPT = (STR|0o24) -I_LIST = (STR|0o25) -I_PLINK = (STR|0o26) -I_PUNLINK = (STR|0o27) -I_FLUSHBAND = (STR|0o34) -I_CKBAND = (STR|0o35) -I_GETBAND = (STR|0o36) -I_ATMARK = (STR|0o37) -I_SETCLTIME = (STR|0o40) -I_GETCLTIME = (STR|0o41) -I_CANPUT = (STR|0o42) -I_S_RECVFD = (STR|0o43) -I_STATS = (STR|0o44) -I_BIGPIPE = (STR|0o45) -I_GETTP = (STR|0o46) -INFTIM = -1 diff --git a/Lib/plat-unixware7/regen b/Lib/plat-unixware7/regen deleted file mode 100755 --- a/Lib/plat-unixware7/regen +++ /dev/null @@ -1,9 +0,0 @@ -#! /bin/sh -case `uname -sr` in -UnixWare*) ;; -*) echo Probably not on a UnixWare system 1>&2 - exit 1;; -esac -set -v -h2py -i '(u_long)' /usr/include/netinet/in.h -h2py /usr/include/sys/stropts.h diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1310,29 +1310,8 @@ -PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \ $(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/PatternGrammar.txt -# Create the PLATDIR source directory, if one wasn't distributed.. -# For multiarch targets, use the plat-linux/regen script. $(srcdir)/Lib/$(PLATDIR): mkdir $(srcdir)/Lib/$(PLATDIR) - if [ -n "$(MULTIARCH)" ]; then \ - cp $(srcdir)/Lib/plat-linux/regen $(srcdir)/Lib/$(PLATDIR)/regen; \ - else \ - cp $(srcdir)/Lib/plat-generic/regen $(srcdir)/Lib/$(PLATDIR)/regen; \ - fi; \ - export PATH; PATH="`pwd`:$$PATH"; \ - export PYTHONPATH; PYTHONPATH="`pwd`/Lib"; \ - export DYLD_FRAMEWORK_PATH; DYLD_FRAMEWORK_PATH="`pwd`"; \ - export EXE; EXE="$(BUILDEXE)"; \ - export CC; CC="$(CC)"; \ - if [ -n "$(MULTIARCH)" ]; then export MULTIARCH; MULTIARCH=$(MULTIARCH); fi; \ - export PYTHON_FOR_BUILD; \ - if [ "$(BUILD_GNU_TYPE)" = "$(HOST_GNU_TYPE)" ]; then \ - PYTHON_FOR_BUILD="$(BUILDPYTHON)"; \ - else \ - PYTHON_FOR_BUILD="$(PYTHON_FOR_BUILD)"; \ - fi; \ - export H2PY; H2PY="$$PYTHON_FOR_BUILD $(abs_srcdir)/Tools/scripts/h2py.py"; \ - cd $(srcdir)/Lib/$(PLATDIR); $(RUNSHARED) ./regen python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh # Substitution happens here, as the completely-expanded BINDIR @@ -1657,7 +1636,6 @@ -o -name '*.tmSnippet' \ -o -name 'Setup' \ -o -name 'Setup.*' \ - -o -name regen \ -o -name README \ -o -name NEWS \ -o -name HISTORY \ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -95,6 +95,8 @@ Library ------- +- Issue #28027: Remove undocumented modules from ``Lib/plat-*``. + - Issue #27445: Don't pass str(_charset) to MIMEText.set_payload(). Patch by Claude Paroz. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:43:22 2016 From: python-checkins at python.org (zach.ware) Date: Thu, 08 Sep 2016 18:43:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328027=3A_Mention_?= =?utf-8?q?the_names_of_the_removed_modules_in_Misc/NEWS?= Message-ID: <20160908184321.87525.98366.DFE40AA9@psf.io> https://hg.python.org/cpython/rev/e256583e6062 changeset: 103338:e256583e6062 user: Zachary Ware date: Thu Sep 08 11:43:09 2016 -0700 summary: Issue #28027: Mention the names of the removed modules in Misc/NEWS files: Misc/NEWS | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -95,7 +95,8 @@ Library ------- -- Issue #28027: Remove undocumented modules from ``Lib/plat-*``. +- Issue #28027: Remove undocumented modules from ``Lib/plat-*``: IN, CDROM, + DLFCN, TYPES, CDIO, and STROPTS. - Issue #27445: Don't pass str(_charset) to MIMEText.set_payload(). Patch by Claude Paroz. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:44:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 18:44:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_simplify_Py=5FUCSN_definit?= =?utf-8?q?ions_with_stdint_types?= Message-ID: <20160908184432.19406.39372.5FD1488A@psf.io> https://hg.python.org/cpython/rev/d1dace3c9871 changeset: 103339:d1dace3c9871 parent: 103333:5ffc6b528fb4 user: Benjamin Peterson date: Thu Sep 08 11:38:28 2016 -0700 summary: simplify Py_UCSN definitions with stdint types files: Include/unicodeobject.h | 18 +++--------------- 1 files changed, 3 insertions(+), 15 deletions(-) diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -113,21 +113,9 @@ /* Py_UCS4 and Py_UCS2 are typedefs for the respective unicode representations. */ -#if SIZEOF_INT == 4 -typedef unsigned int Py_UCS4; -#elif SIZEOF_LONG == 4 -typedef unsigned long Py_UCS4; -#else -#error "Could not find a proper typedef for Py_UCS4" -#endif - -#if SIZEOF_SHORT == 2 -typedef unsigned short Py_UCS2; -#else -#error "Could not find a proper typedef for Py_UCS2" -#endif - -typedef unsigned char Py_UCS1; +typedef uint32_t Py_UCS4; +typedef uint16_t Py_UCS2; +typedef uint8_t Py_UCS1; /* --- Internal Unicode Operations ---------------------------------------- */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:44:34 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 18:44:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_merge_heads?= Message-ID: <20160908184433.69808.82180.999E28A5@psf.io> https://hg.python.org/cpython/rev/8c91348e1dd4 changeset: 103340:8c91348e1dd4 parent: 103339:d1dace3c9871 parent: 103338:e256583e6062 user: Benjamin Peterson date: Thu Sep 08 11:44:26 2016 -0700 summary: merge heads files: Doc/whatsnew/3.6.rst | 7 + Lib/plat-aix4/IN.py | 165 - Lib/plat-aix4/regen | 8 - Lib/plat-darwin/IN.py | 662 ------- Lib/plat-darwin/regen | 3 - Lib/plat-freebsd4/IN.py | 355 ---- Lib/plat-freebsd4/regen | 3 - Lib/plat-freebsd5/IN.py | 355 ---- Lib/plat-freebsd5/regen | 3 - Lib/plat-freebsd6/IN.py | 551 ------ Lib/plat-freebsd6/regen | 3 - Lib/plat-freebsd7/IN.py | 571 ------ Lib/plat-freebsd7/regen | 3 - Lib/plat-freebsd8/IN.py | 571 ------ Lib/plat-freebsd8/regen | 3 - Lib/plat-generic/regen | 3 - Lib/plat-linux/CDROM.py | 207 -- Lib/plat-linux/DLFCN.py | 83 - Lib/plat-linux/IN.py | 615 ------- Lib/plat-linux/TYPES.py | 170 - Lib/plat-linux/regen | 33 - Lib/plat-netbsd1/IN.py | 56 - Lib/plat-netbsd1/regen | 3 - Lib/plat-next3/regen | 6 - Lib/plat-sunos5/CDIO.py | 73 - Lib/plat-sunos5/DLFCN.py | 27 - Lib/plat-sunos5/IN.py | 1421 ---------------- Lib/plat-sunos5/STROPTS.py | 1813 --------------------- Lib/plat-sunos5/TYPES.py | 313 --- Lib/plat-sunos5/regen | 9 - Lib/plat-unixware7/IN.py | 836 --------- Lib/plat-unixware7/STROPTS.py | 328 --- Lib/plat-unixware7/regen | 9 - Makefile.pre.in | 22 - Misc/NEWS | 3 + README | 17 +- 36 files changed, 23 insertions(+), 9287 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -987,6 +987,13 @@ Use :class:`io.TextIOWrapper` for reading compressed text files in :term:`universal newlines` mode. +* The undocumented ``IN``, ``CDROM``, ``DLFCN``, ``TYPES``, ``CDIO``, and + ``STROPTS`` modules have been removed. They had been available in the + platform specific ``Lib/plat-*/`` directories, but were chronically out of + date, inconsistently available across platforms, and unmaintained. The + script that created these modules is still available in the source + distribution at :source:`Tools/scripts/h2py.py`. + Porting to Python 3.6 ===================== diff --git a/Lib/plat-aix4/IN.py b/Lib/plat-aix4/IN.py deleted file mode 100644 --- a/Lib/plat-aix4/IN.py +++ /dev/null @@ -1,165 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from net/nh.h - -# Included from sys/machine.h -LITTLE_ENDIAN = 1234 -BIG_ENDIAN = 4321 -PDP_ENDIAN = 3412 -BYTE_ORDER = BIG_ENDIAN -DEFAULT_GPR = 0xDEADBEEF -MSR_EE = 0x8000 -MSR_PR = 0x4000 -MSR_FP = 0x2000 -MSR_ME = 0x1000 -MSR_FE = 0x0800 -MSR_FE0 = 0x0800 -MSR_SE = 0x0400 -MSR_BE = 0x0200 -MSR_IE = 0x0100 -MSR_FE1 = 0x0100 -MSR_AL = 0x0080 -MSR_IP = 0x0040 -MSR_IR = 0x0020 -MSR_DR = 0x0010 -MSR_PM = 0x0004 -DEFAULT_MSR = (MSR_EE | MSR_ME | MSR_AL | MSR_IR | MSR_DR) -DEFAULT_USER_MSR = (DEFAULT_MSR | MSR_PR) -CR_LT = 0x80000000 -CR_GT = 0x40000000 -CR_EQ = 0x20000000 -CR_SO = 0x10000000 -CR_FX = 0x08000000 -CR_FEX = 0x04000000 -CR_VX = 0x02000000 -CR_OX = 0x01000000 -XER_SO = 0x80000000 -XER_OV = 0x40000000 -XER_CA = 0x20000000 -def XER_COMP_BYTE(xer): return ((xer >> 8) & 0x000000FF) - -def XER_LENGTH(xer): return (xer & 0x0000007F) - -DSISR_IO = 0x80000000 -DSISR_PFT = 0x40000000 -DSISR_LOCK = 0x20000000 -DSISR_FPIO = 0x10000000 -DSISR_PROT = 0x08000000 -DSISR_LOOP = 0x04000000 -DSISR_DRST = 0x04000000 -DSISR_ST = 0x02000000 -DSISR_SEGB = 0x01000000 -DSISR_DABR = 0x00400000 -DSISR_EAR = 0x00100000 -SRR_IS_PFT = 0x40000000 -SRR_IS_ISPEC = 0x20000000 -SRR_IS_IIO = 0x10000000 -SRR_IS_GUARD = 0x10000000 -SRR_IS_PROT = 0x08000000 -SRR_IS_LOOP = 0x04000000 -SRR_PR_FPEN = 0x00100000 -SRR_PR_INVAL = 0x00080000 -SRR_PR_PRIV = 0x00040000 -SRR_PR_TRAP = 0x00020000 -SRR_PR_IMPRE = 0x00010000 -def BUID_7F_SRVAL(raddr): return (0x87F00000 | (((uint)(raddr)) >> 28)) - -BT_256M = 0x1FFC -BT_128M = 0x0FFC -BT_64M = 0x07FC -BT_32M = 0x03FC -BT_16M = 0x01FC -BT_8M = 0x00FC -BT_4M = 0x007C -BT_2M = 0x003C -BT_1M = 0x001C -BT_512K = 0x000C -BT_256K = 0x0004 -BT_128K = 0x0000 -BT_NOACCESS = 0x0 -BT_RDONLY = 0x1 -BT_WRITE = 0x2 -BT_VS = 0x2 -BT_VP = 0x1 -def BAT_ESEG(dbatu): return (((uint)(dbatu) >> 28)) - -MIN_BAT_SIZE = 0x00020000 -MAX_BAT_SIZE = 0x10000000 -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_TCP = 6 -IPPROTO_EGP = 8 -IPPROTO_PUP = 12 -IPPROTO_UDP = 17 -IPPROTO_IDP = 22 -IPPROTO_TP = 29 -IPPROTO_LOCAL = 63 -IPPROTO_EON = 80 -IPPROTO_BIP = 0x53 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -IPPORT_TIMESERVER = 37 -def IN_CLASSA(i): return (((int)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((int)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((int)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((int)(i) & 0xf0000000) == 0xe0000000) - -def IN_MULTICAST(i): return IN_CLASSD(i) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -INADDR_UNSPEC_GROUP = 0xe0000000 -INADDR_ALLHOSTS_GROUP = 0xe0000001 -INADDR_MAX_LOCAL_GROUP = 0xe00000ff -def IN_EXPERIMENTAL(i): return (((int)(i) & 0xe0000000) == 0xe0000000) - -def IN_BADCLASS(i): return (((int)(i) & 0xf0000000) == 0xf0000000) - -INADDR_ANY = 0x00000000 -INADDR_BROADCAST = 0xffffffff -INADDR_LOOPBACK = 0x7f000001 -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 diff --git a/Lib/plat-aix4/regen b/Lib/plat-aix4/regen deleted file mode 100755 --- a/Lib/plat-aix4/regen +++ /dev/null @@ -1,8 +0,0 @@ -#! /bin/sh -case `uname -sv` in -'AIX 4'*) ;; -*) echo Probably not on an AIX 4 system 1>&2 - exit 1;; -esac -set -v -h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-darwin/IN.py b/Lib/plat-darwin/IN.py deleted file mode 100644 --- a/Lib/plat-darwin/IN.py +++ /dev/null @@ -1,662 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/appleapiopts.h - -# Included from sys/_types.h - -# Included from sys/cdefs.h -def __P(protos): return protos - -def __STRING(x): return #x - -def __P(protos): return () - -def __STRING(x): return "x" - -def __attribute__(x): return - -def __COPYRIGHT(s): return __IDSTRING(copyright,s) - -def __RCSID(s): return __IDSTRING(rcsid,s) - -def __SCCSID(s): return __IDSTRING(sccsid,s) - -def __PROJECT_VERSION(s): return __IDSTRING(project_version,s) - -__DARWIN_UNIX03 = 1 -__DARWIN_UNIX03 = 0 -__DARWIN_UNIX03 = 0 -__DARWIN_UNIX03 = 1 -__DARWIN_64_BIT_INO_T = 1 -__DARWIN_64_BIT_INO_T = 0 -__DARWIN_64_BIT_INO_T = 0 -__DARWIN_NON_CANCELABLE = 0 -__DARWIN_VERS_1050 = 1 -__DARWIN_VERS_1050 = 0 -__DARWIN_SUF_UNIX03 = "$UNIX2003" -__DARWIN_SUF_UNIX03_SET = 1 -__DARWIN_SUF_UNIX03_SET = 0 -__DARWIN_SUF_64_BIT_INO_T = "$INODE64" -__DARWIN_SUF_NON_CANCELABLE = "$NOCANCEL" -__DARWIN_SUF_1050 = "$1050" -__DARWIN_SUF_UNIX03_SET = 0 -__DARWIN_SUF_EXTSN = "$DARWIN_EXTSN" -__DARWIN_LONG_DOUBLE_IS_DOUBLE = 0 -def __DARWIN_LDBL_COMPAT(x): return - -def __DARWIN_LDBL_COMPAT2(x): return - -__DARWIN_LONG_DOUBLE_IS_DOUBLE = 1 -def __DARWIN_LDBL_COMPAT(x): return - -def __DARWIN_LDBL_COMPAT2(x): return - -__DARWIN_LONG_DOUBLE_IS_DOUBLE = 0 -_DARWIN_FEATURE_LONG_DOUBLE_IS_DOUBLE = 1 -_DARWIN_FEATURE_UNIX_CONFORMANCE = 3 -_DARWIN_FEATURE_64_BIT_INODE = 1 - -# Included from machine/_types.h -__PTHREAD_SIZE__ = 1168 -__PTHREAD_ATTR_SIZE__ = 56 -__PTHREAD_MUTEXATTR_SIZE__ = 8 -__PTHREAD_MUTEX_SIZE__ = 56 -__PTHREAD_CONDATTR_SIZE__ = 8 -__PTHREAD_COND_SIZE__ = 40 -__PTHREAD_ONCE_SIZE__ = 8 -__PTHREAD_RWLOCK_SIZE__ = 192 -__PTHREAD_RWLOCKATTR_SIZE__ = 16 -__PTHREAD_SIZE__ = 596 -__PTHREAD_ATTR_SIZE__ = 36 -__PTHREAD_MUTEXATTR_SIZE__ = 8 -__PTHREAD_MUTEX_SIZE__ = 40 -__PTHREAD_CONDATTR_SIZE__ = 4 -__PTHREAD_COND_SIZE__ = 24 -__PTHREAD_ONCE_SIZE__ = 4 -__PTHREAD_RWLOCK_SIZE__ = 124 -__PTHREAD_RWLOCKATTR_SIZE__ = 12 -__DARWIN_NULL = 0 - -# Included from stdint.h -__WORDSIZE = 64 -__WORDSIZE = 32 -INT8_MAX = 127 -INT16_MAX = 32767 -INT32_MAX = 2147483647 -INT8_MIN = -128 -INT16_MIN = -32768 -INT32_MIN = (-INT32_MAX-1) -UINT8_MAX = 255 -UINT16_MAX = 65535 -INT_LEAST8_MIN = INT8_MIN -INT_LEAST16_MIN = INT16_MIN -INT_LEAST32_MIN = INT32_MIN -INT_LEAST8_MAX = INT8_MAX -INT_LEAST16_MAX = INT16_MAX -INT_LEAST32_MAX = INT32_MAX -UINT_LEAST8_MAX = UINT8_MAX -UINT_LEAST16_MAX = UINT16_MAX -INT_FAST8_MIN = INT8_MIN -INT_FAST16_MIN = INT16_MIN -INT_FAST32_MIN = INT32_MIN -INT_FAST8_MAX = INT8_MAX -INT_FAST16_MAX = INT16_MAX -INT_FAST32_MAX = INT32_MAX -UINT_FAST8_MAX = UINT8_MAX -UINT_FAST16_MAX = UINT16_MAX -INTPTR_MIN = INT32_MIN -INTPTR_MAX = INT32_MAX -PTRDIFF_MIN = INT32_MIN -PTRDIFF_MAX = INT32_MAX -WCHAR_MAX = 0x7fffffff -WCHAR_MIN = 0 -WCHAR_MIN = (-WCHAR_MAX-1) -WINT_MIN = INT32_MIN -WINT_MAX = INT32_MAX -SIG_ATOMIC_MIN = INT32_MIN -SIG_ATOMIC_MAX = INT32_MAX -def INT8_C(v): return (v) - -def INT16_C(v): return (v) - -def INT32_C(v): return (v) - - -# Included from sys/socket.h - -# Included from machine/_param.h -SOCK_STREAM = 1 -SOCK_DGRAM = 2 -SOCK_RAW = 3 -SOCK_RDM = 4 -SOCK_SEQPACKET = 5 -SO_DEBUG = 0x0001 -SO_ACCEPTCONN = 0x0002 -SO_REUSEADDR = 0x0004 -SO_KEEPALIVE = 0x0008 -SO_DONTROUTE = 0x0010 -SO_BROADCAST = 0x0020 -SO_USELOOPBACK = 0x0040 -SO_LINGER = 0x0080 -SO_LINGER = 0x1080 -SO_OOBINLINE = 0x0100 -SO_REUSEPORT = 0x0200 -SO_TIMESTAMP = 0x0400 -SO_ACCEPTFILTER = 0x1000 -SO_DONTTRUNC = 0x2000 -SO_WANTMORE = 0x4000 -SO_WANTOOBFLAG = 0x8000 -SO_SNDBUF = 0x1001 -SO_RCVBUF = 0x1002 -SO_SNDLOWAT = 0x1003 -SO_RCVLOWAT = 0x1004 -SO_SNDTIMEO = 0x1005 -SO_RCVTIMEO = 0x1006 -SO_ERROR = 0x1007 -SO_TYPE = 0x1008 -SO_NREAD = 0x1020 -SO_NKE = 0x1021 -SO_NOSIGPIPE = 0x1022 -SO_NOADDRERR = 0x1023 -SO_NWRITE = 0x1024 -SO_REUSESHAREUID = 0x1025 -SO_NOTIFYCONFLICT = 0x1026 -SO_LINGER_SEC = 0x1080 -SO_RESTRICTIONS = 0x1081 -SO_RESTRICT_DENYIN = 0x00000001 -SO_RESTRICT_DENYOUT = 0x00000002 -SO_RESTRICT_DENYSET = (-2147483648) -SO_LABEL = 0x1010 -SO_PEERLABEL = 0x1011 -SOL_SOCKET = 0xffff -AF_UNSPEC = 0 -AF_UNIX = 1 -AF_LOCAL = AF_UNIX -AF_INET = 2 -AF_IMPLINK = 3 -AF_PUP = 4 -AF_CHAOS = 5 -AF_NS = 6 -AF_ISO = 7 -AF_OSI = AF_ISO -AF_ECMA = 8 -AF_DATAKIT = 9 -AF_CCITT = 10 -AF_SNA = 11 -AF_DECnet = 12 -AF_DLI = 13 -AF_LAT = 14 -AF_HYLINK = 15 -AF_APPLETALK = 16 -AF_ROUTE = 17 -AF_LINK = 18 -pseudo_AF_XTP = 19 -AF_COIP = 20 -AF_CNT = 21 -pseudo_AF_RTIP = 22 -AF_IPX = 23 -AF_SIP = 24 -pseudo_AF_PIP = 25 -AF_NDRV = 27 -AF_ISDN = 28 -AF_E164 = AF_ISDN -pseudo_AF_KEY = 29 -AF_INET6 = 30 -AF_NATM = 31 -AF_SYSTEM = 32 -AF_NETBIOS = 33 -AF_PPP = 34 -AF_ATM = 30 -pseudo_AF_HDRCMPLT = 35 -AF_RESERVED_36 = 36 -AF_NETGRAPH = 32 -AF_MAX = 37 -SOCK_MAXADDRLEN = 255 -_SS_MAXSIZE = 128 -PF_UNSPEC = AF_UNSPEC -PF_LOCAL = AF_LOCAL -PF_UNIX = PF_LOCAL -PF_INET = AF_INET -PF_IMPLINK = AF_IMPLINK -PF_PUP = AF_PUP -PF_CHAOS = AF_CHAOS -PF_NS = AF_NS -PF_ISO = AF_ISO -PF_OSI = AF_ISO -PF_ECMA = AF_ECMA -PF_DATAKIT = AF_DATAKIT -PF_CCITT = AF_CCITT -PF_SNA = AF_SNA -PF_DECnet = AF_DECnet -PF_DLI = AF_DLI -PF_LAT = AF_LAT -PF_HYLINK = AF_HYLINK -PF_APPLETALK = AF_APPLETALK -PF_ROUTE = AF_ROUTE -PF_LINK = AF_LINK -PF_XTP = pseudo_AF_XTP -PF_COIP = AF_COIP -PF_CNT = AF_CNT -PF_SIP = AF_SIP -PF_IPX = AF_IPX -PF_RTIP = pseudo_AF_RTIP -PF_PIP = pseudo_AF_PIP -PF_NDRV = AF_NDRV -PF_ISDN = AF_ISDN -PF_KEY = pseudo_AF_KEY -PF_INET6 = AF_INET6 -PF_NATM = AF_NATM -PF_SYSTEM = AF_SYSTEM -PF_NETBIOS = AF_NETBIOS -PF_PPP = AF_PPP -PF_RESERVED_36 = AF_RESERVED_36 -PF_ATM = AF_ATM -PF_NETGRAPH = AF_NETGRAPH -PF_MAX = AF_MAX -NET_MAXID = AF_MAX -NET_RT_DUMP = 1 -NET_RT_FLAGS = 2 -NET_RT_IFLIST = 3 -NET_RT_STAT = 4 -NET_RT_TRASH = 5 -NET_RT_IFLIST2 = 6 -NET_RT_DUMP2 = 7 -NET_RT_MAXID = 8 -SOMAXCONN = 128 -MSG_OOB = 0x1 -MSG_PEEK = 0x2 -MSG_DONTROUTE = 0x4 -MSG_EOR = 0x8 -MSG_TRUNC = 0x10 -MSG_CTRUNC = 0x20 -MSG_WAITALL = 0x40 -MSG_DONTWAIT = 0x80 -MSG_EOF = 0x100 -MSG_WAITSTREAM = 0x200 -MSG_FLUSH = 0x400 -MSG_HOLD = 0x800 -MSG_SEND = 0x1000 -MSG_HAVEMORE = 0x2000 -MSG_RCVMORE = 0x4000 -MSG_NEEDSA = 0x10000 -CMGROUP_MAX = 16 -SCM_RIGHTS = 0x01 -SCM_TIMESTAMP = 0x02 -SCM_CREDS = 0x03 -SHUT_RD = 0 -SHUT_WR = 1 -SHUT_RDWR = 2 - -# Included from machine/endian.h - -# Included from sys/_endian.h -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -def NTOHL(x): return (x) - -def NTOHS(x): return (x) - -def HTONL(x): return (x) - -def HTONS(x): return (x) - - -# Included from libkern/_OSByteOrder.h -def __DARWIN_OSSwapConstInt16(x): return \ - -def __DARWIN_OSSwapConstInt32(x): return \ - -def __DARWIN_OSSwapConstInt64(x): return \ - - -# Included from libkern/i386/_OSByteOrder.h -def __DARWIN_OSSwapInt16(x): return \ - -def __DARWIN_OSSwapInt32(x): return \ - -def __DARWIN_OSSwapInt64(x): return \ - -def __DARWIN_OSSwapInt16(x): return _OSSwapInt16(x) - -def __DARWIN_OSSwapInt32(x): return _OSSwapInt32(x) - -def __DARWIN_OSSwapInt64(x): return _OSSwapInt64(x) - -def ntohs(x): return __DARWIN_OSSwapInt16(x) - -def htons(x): return __DARWIN_OSSwapInt16(x) - -def ntohl(x): return __DARWIN_OSSwapInt32(x) - -def htonl(x): return __DARWIN_OSSwapInt32(x) - -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_TCP = 6 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_UDP = 17 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_PIM = 103 -IPPROTO_PGM = 113 -IPPROTO_DIVERT = 254 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -__DARWIN_IPPORT_RESERVED = 1024 -IPPORT_RESERVED = __DARWIN_IPPORT_RESERVED -IPPORT_USERRESERVED = 5000 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -def IN_CLASSA(i): return (((u_int32_t)(i) & (-2147483648)) == 0) - -IN_CLASSA_NET = (-16777216) -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & (-1073741824)) == (-2147483648)) - -IN_CLASSB_NET = (-65536) -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & (-536870912)) == (-1073741824)) - -IN_CLASSC_NET = (-256) -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & (-268435456)) == (-536870912)) - -IN_CLASSD_NET = (-268435456) -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -INADDR_NONE = (-1) -def IN_LINKLOCAL(i): return (((u_int32_t)(i) & IN_CLASSB_NET) == IN_LINKLOCALNETNUM) - -IN_LOOPBACKNET = 127 -INET_ADDRSTRLEN = 16 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_STRIPHDR = 23 -IP_RECVTTL = 24 -IP_FW_ADD = 40 -IP_FW_DEL = 41 -IP_FW_FLUSH = 42 -IP_FW_ZERO = 43 -IP_FW_GET = 44 -IP_FW_RESETLOG = 45 -IP_OLD_FW_ADD = 50 -IP_OLD_FW_DEL = 51 -IP_OLD_FW_FLUSH = 52 -IP_OLD_FW_ZERO = 53 -IP_OLD_FW_GET = 54 -IP_NAT__XXX = 55 -IP_OLD_FW_RESETLOG = 56 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_TRAFFIC_MGT_BACKGROUND = 65 -IP_FORCE_OUT_IFP = 69 -TRAFFIC_MGT_SO_BACKGROUND = 0x0001 -TRAFFIC_MGT_SO_BG_SUPPRESSED = 0x0002 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 - -# Included from netinet6/in6.h -__KAME_VERSION = "20010528/apple-darwin" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_PKTINFO = 19 -IPV6_HOPLIMIT = 20 -IPV6_NEXTHOP = 21 -IPV6_HOPOPTS = 22 -IPV6_DSTOPTS = 23 -IPV6_RTHDR = 24 -IPV6_PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_V6ONLY = 27 -IPV6_BINDV6ONLY = IPV6_V6ONLY -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_V6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_USETEMPADDR = 32 -IPV6CTL_TEMPPLTIME = 33 -IPV6CTL_TEMPVLTIME = 34 -IPV6CTL_AUTO_LINKLOCAL = 35 -IPV6CTL_RIP6STATS = 36 -IPV6CTL_MAXFRAGS = 41 -IPV6CTL_MAXID = 42 diff --git a/Lib/plat-darwin/regen b/Lib/plat-darwin/regen deleted file mode 100755 --- a/Lib/plat-darwin/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python$EXE ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd4/IN.py b/Lib/plat-freebsd4/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd4/IN.py +++ /dev/null @@ -1,355 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_TCP = 6 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_UDP = 17 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_PIM = 103 -IPPROTO_PGM = 113 -IPPROTO_DIVERT = 254 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -def IN_CLASSA(i): return (((u_int32_t)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & 0xf0000000) == 0xe0000000) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -INET_ADDRSTRLEN = 16 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 - -# Included from netinet6/in6.h - -# Included from sys/queue.h -def SLIST_HEAD_INITIALIZER(head): return \ - -def SLIST_ENTRY(type): return \ - -def STAILQ_HEAD_INITIALIZER(head): return \ - -def STAILQ_ENTRY(type): return \ - -def LIST_HEAD_INITIALIZER(head): return \ - -def LIST_ENTRY(type): return \ - -def TAILQ_HEAD_INITIALIZER(head): return \ - -def TAILQ_ENTRY(type): return \ - -def CIRCLEQ_ENTRY(type): return \ - -__KAME_VERSION = "20000701/FreeBSD-current" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = 0xff010000 -IPV6_ADDR_INT32_MLL = 0xff020000 -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = 0xffff0000 -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_PKTINFO = 19 -IPV6_HOPLIMIT = 20 -IPV6_NEXTHOP = 21 -IPV6_HOPOPTS = 22 -IPV6_DSTOPTS = 23 -IPV6_RTHDR = 24 -IPV6_PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_BINDV6ONLY = 27 -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_BINDV6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_MAXID = 28 diff --git a/Lib/plat-freebsd4/regen b/Lib/plat-freebsd4/regen deleted file mode 100755 --- a/Lib/plat-freebsd4/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd5/IN.py b/Lib/plat-freebsd5/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd5/IN.py +++ /dev/null @@ -1,355 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_TCP = 6 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_UDP = 17 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_PIM = 103 -IPPROTO_PGM = 113 -IPPROTO_DIVERT = 254 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -def IN_CLASSA(i): return (((u_int32_t)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & 0xf0000000) == 0xe0000000) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -INET_ADDRSTRLEN = 16 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 - -# Included from netinet6/in6.h - -# Included from sys/queue.h -def SLIST_HEAD_INITIALIZER(head): return \ - -def SLIST_ENTRY(type): return \ - -def STAILQ_HEAD_INITIALIZER(head): return \ - -def STAILQ_ENTRY(type): return \ - -def LIST_HEAD_INITIALIZER(head): return \ - -def LIST_ENTRY(type): return \ - -def TAILQ_HEAD_INITIALIZER(head): return \ - -def TAILQ_ENTRY(type): return \ - -def CIRCLEQ_ENTRY(type): return \ - -__KAME_VERSION = "20000701/FreeBSD-current" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = 0xff010000 -IPV6_ADDR_INT32_MLL = 0xff020000 -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = 0xffff0000 -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_PKTINFO = 19 -IPV6_HOPLIMIT = 20 -IPV6_NEXTHOP = 21 -IPV6_HOPOPTS = 22 -IPV6_DSTOPTS = 23 -IPV6_RTHDR = 24 -IPV6_PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_BINDV6ONLY = 27 -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_BINDV6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_MAXID = 28 diff --git a/Lib/plat-freebsd5/regen b/Lib/plat-freebsd5/regen deleted file mode 100755 --- a/Lib/plat-freebsd5/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd6/IN.py b/Lib/plat-freebsd6/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd6/IN.py +++ /dev/null @@ -1,551 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/cdefs.h -__GNUCLIKE_ASM = 3 -__GNUCLIKE_ASM = 2 -__GNUCLIKE___TYPEOF = 1 -__GNUCLIKE___OFFSETOF = 1 -__GNUCLIKE___SECTION = 1 -__GNUCLIKE_ATTRIBUTE_MODE_DI = 1 -__GNUCLIKE_CTOR_SECTION_HANDLING = 1 -__GNUCLIKE_BUILTIN_CONSTANT_P = 1 -__GNUCLIKE_BUILTIN_VARARGS = 1 -__GNUCLIKE_BUILTIN_STDARG = 1 -__GNUCLIKE_BUILTIN_VAALIST = 1 -__GNUC_VA_LIST_COMPATIBILITY = 1 -__GNUCLIKE_BUILTIN_NEXT_ARG = 1 -__GNUCLIKE_BUILTIN_MEMCPY = 1 -__CC_SUPPORTS_INLINE = 1 -__CC_SUPPORTS___INLINE = 1 -__CC_SUPPORTS___INLINE__ = 1 -__CC_SUPPORTS___FUNC__ = 1 -__CC_SUPPORTS_WARNING = 1 -__CC_SUPPORTS_VARADIC_XXX = 1 -__CC_SUPPORTS_DYNAMIC_ARRAY_INIT = 1 -__CC_INT_IS_32BIT = 1 -def __P(protos): return protos - -def __STRING(x): return #x - -def __XSTRING(x): return __STRING(x) - -def __P(protos): return () - -def __STRING(x): return "x" - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __nonnull(x): return __attribute__((__nonnull__(x))) - -def __predict_true(exp): return __builtin_expect((exp), 1) - -def __predict_false(exp): return __builtin_expect((exp), 0) - -def __predict_true(exp): return (exp) - -def __predict_false(exp): return (exp) - -def __format_arg(fmtarg): return __attribute__((__format_arg__ (fmtarg))) - -def __FBSDID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID_SOURCE(s): return __IDSTRING(__CONCAT(__rcsid_source_,__LINE__),s) - -def __SCCSID(s): return __IDSTRING(__CONCAT(__sccsid_,__LINE__),s) - -def __COPYRIGHT(s): return __IDSTRING(__CONCAT(__copyright_,__LINE__),s) - -_POSIX_C_SOURCE = 199009 -_POSIX_C_SOURCE = 199209 -__XSI_VISIBLE = 600 -_POSIX_C_SOURCE = 200112 -__XSI_VISIBLE = 500 -_POSIX_C_SOURCE = 199506 -_POSIX_C_SOURCE = 198808 -__POSIX_VISIBLE = 200112 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 199506 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199309 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199209 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199009 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 198808 -__ISO_C_VISIBLE = 0 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 200112 -__XSI_VISIBLE = 600 -__BSD_VISIBLE = 1 -__ISO_C_VISIBLE = 1999 - -# Included from sys/_types.h - -# Included from machine/_types.h - -# Included from machine/endian.h -_QUAD_HIGHWORD = 1 -_QUAD_LOWWORD = 0 -_LITTLE_ENDIAN = 1234 -_BIG_ENDIAN = 4321 -_PDP_ENDIAN = 3412 -_BYTE_ORDER = _LITTLE_ENDIAN -LITTLE_ENDIAN = _LITTLE_ENDIAN -BIG_ENDIAN = _BIG_ENDIAN -PDP_ENDIAN = _PDP_ENDIAN -BYTE_ORDER = _BYTE_ORDER -def __word_swap_int_var(x): return \ - -def __word_swap_int_const(x): return \ - -def __word_swap_int(x): return __word_swap_int_var(x) - -def __byte_swap_int_var(x): return \ - -def __byte_swap_int_const(x): return \ - -def __byte_swap_int(x): return __byte_swap_int_var(x) - -def __byte_swap_long_var(x): return \ - -def __byte_swap_long_const(x): return \ - -def __byte_swap_long(x): return __byte_swap_long_var(x) - -def __byte_swap_word_var(x): return \ - -def __byte_swap_word_const(x): return \ - -def __byte_swap_word(x): return __byte_swap_word_var(x) - -def __htonl(x): return __bswap32(x) - -def __htons(x): return __bswap16(x) - -def __ntohl(x): return __bswap32(x) - -def __ntohs(x): return __bswap16(x) - -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_TCP = 6 -IPPROTO_UDP = 17 -def htonl(x): return __htonl(x) - -def htons(x): return __htons(x) - -def ntohl(x): return __ntohl(x) - -def ntohs(x): return __ntohs(x) - -IPPROTO_RAW = 255 -INET_ADDRSTRLEN = 16 -IPPROTO_HOPOPTS = 0 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_MOBILE = 55 -IPPROTO_TLSP = 56 -IPPROTO_SKIP = 57 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_SCTP = 132 -IPPROTO_PIM = 103 -IPPROTO_CARP = 112 -IPPROTO_PGM = 113 -IPPROTO_PFSYNC = 240 -IPPROTO_OLD_DIVERT = 254 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPROTO_DIVERT = 258 -IPPROTO_SPACER = 32767 -IPPORT_RESERVED = 1024 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -IPPORT_MAX = 65535 -def IN_CLASSA(i): return (((u_int32_t)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & 0xf0000000) == 0xe0000000) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & 0xf0000000) == 0xf0000000) - -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_SENDSRCADDR = IP_RECVDSTADDR -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_ONESBCAST = 23 -IP_FW_TABLE_ADD = 40 -IP_FW_TABLE_DEL = 41 -IP_FW_TABLE_FLUSH = 42 -IP_FW_TABLE_GETSIZE = 43 -IP_FW_TABLE_LIST = 44 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_RECVTTL = 65 -IP_MINTTL = 66 -IP_DONTFRAG = 67 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 -def in_nullhost(x): return ((x).s_addr == INADDR_ANY) - - -# Included from netinet6/in6.h -__KAME_VERSION = "FreeBSD" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = 0xff010000 -IPV6_ADDR_INT32_MLL = 0xff020000 -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = 0xffff0000 -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_INTFACELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -def IFA6_IS_DEPRECATED(a): return \ - -def IFA6_IS_INVALID(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_2292PKTINFO = 19 -IPV6_2292HOPLIMIT = 20 -IPV6_2292NEXTHOP = 21 -IPV6_2292HOPOPTS = 22 -IPV6_2292DSTOPTS = 23 -IPV6_2292RTHDR = 24 -IPV6_2292PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_V6ONLY = 27 -IPV6_BINDV6ONLY = IPV6_V6ONLY -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDRDSTOPTS = 35 -IPV6_RECVPKTINFO = 36 -IPV6_RECVHOPLIMIT = 37 -IPV6_RECVRTHDR = 38 -IPV6_RECVHOPOPTS = 39 -IPV6_RECVDSTOPTS = 40 -IPV6_RECVRTHDRDSTOPTS = 41 -IPV6_USE_MIN_MTU = 42 -IPV6_RECVPATHMTU = 43 -IPV6_PATHMTU = 44 -IPV6_REACHCONF = 45 -IPV6_PKTINFO = 46 -IPV6_HOPLIMIT = 47 -IPV6_NEXTHOP = 48 -IPV6_HOPOPTS = 49 -IPV6_DSTOPTS = 50 -IPV6_RTHDR = 51 -IPV6_PKTOPTIONS = 52 -IPV6_RECVTCLASS = 57 -IPV6_AUTOFLOWLABEL = 59 -IPV6_TCLASS = 61 -IPV6_DONTFRAG = 62 -IPV6_PREFER_TEMPADDR = 63 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_V6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_USETEMPADDR = 32 -IPV6CTL_TEMPPLTIME = 33 -IPV6CTL_TEMPVLTIME = 34 -IPV6CTL_AUTO_LINKLOCAL = 35 -IPV6CTL_RIP6STATS = 36 -IPV6CTL_PREFER_TEMPADDR = 37 -IPV6CTL_ADDRCTLPOLICY = 38 -IPV6CTL_USE_DEFAULTZONE = 39 -IPV6CTL_MAXFRAGS = 41 -IPV6CTL_IFQ = 42 -IPV6CTL_ISATAPRTR = 43 -IPV6CTL_MCAST_PMTU = 44 -IPV6CTL_STEALTH = 45 -IPV6CTL_MAXID = 46 diff --git a/Lib/plat-freebsd6/regen b/Lib/plat-freebsd6/regen deleted file mode 100755 --- a/Lib/plat-freebsd6/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd7/IN.py b/Lib/plat-freebsd7/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd7/IN.py +++ /dev/null @@ -1,571 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/cdefs.h -__GNUCLIKE_ASM = 3 -__GNUCLIKE_ASM = 2 -__GNUCLIKE___TYPEOF = 1 -__GNUCLIKE___OFFSETOF = 1 -__GNUCLIKE___SECTION = 1 -__GNUCLIKE_ATTRIBUTE_MODE_DI = 1 -__GNUCLIKE_CTOR_SECTION_HANDLING = 1 -__GNUCLIKE_BUILTIN_CONSTANT_P = 1 -__GNUCLIKE_BUILTIN_VARARGS = 1 -__GNUCLIKE_BUILTIN_STDARG = 1 -__GNUCLIKE_BUILTIN_VAALIST = 1 -__GNUC_VA_LIST_COMPATIBILITY = 1 -__GNUCLIKE_BUILTIN_NEXT_ARG = 1 -__GNUCLIKE_BUILTIN_MEMCPY = 1 -__CC_SUPPORTS_INLINE = 1 -__CC_SUPPORTS___INLINE = 1 -__CC_SUPPORTS___INLINE__ = 1 -__CC_SUPPORTS___FUNC__ = 1 -__CC_SUPPORTS_WARNING = 1 -__CC_SUPPORTS_VARADIC_XXX = 1 -__CC_SUPPORTS_DYNAMIC_ARRAY_INIT = 1 -__CC_INT_IS_32BIT = 1 -def __P(protos): return protos - -def __STRING(x): return #x - -def __XSTRING(x): return __STRING(x) - -def __P(protos): return () - -def __STRING(x): return "x" - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __nonnull(x): return __attribute__((__nonnull__(x))) - -def __predict_true(exp): return __builtin_expect((exp), 1) - -def __predict_false(exp): return __builtin_expect((exp), 0) - -def __predict_true(exp): return (exp) - -def __predict_false(exp): return (exp) - -def __format_arg(fmtarg): return __attribute__((__format_arg__ (fmtarg))) - -def __FBSDID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID_SOURCE(s): return __IDSTRING(__CONCAT(__rcsid_source_,__LINE__),s) - -def __SCCSID(s): return __IDSTRING(__CONCAT(__sccsid_,__LINE__),s) - -def __COPYRIGHT(s): return __IDSTRING(__CONCAT(__copyright_,__LINE__),s) - -_POSIX_C_SOURCE = 199009 -_POSIX_C_SOURCE = 199209 -__XSI_VISIBLE = 600 -_POSIX_C_SOURCE = 200112 -__XSI_VISIBLE = 500 -_POSIX_C_SOURCE = 199506 -_POSIX_C_SOURCE = 198808 -__POSIX_VISIBLE = 200112 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 199506 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199309 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199209 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199009 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 198808 -__ISO_C_VISIBLE = 0 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 200112 -__XSI_VISIBLE = 600 -__BSD_VISIBLE = 1 -__ISO_C_VISIBLE = 1999 - -# Included from sys/_types.h - -# Included from machine/_types.h - -# Included from machine/endian.h -_QUAD_HIGHWORD = 1 -_QUAD_LOWWORD = 0 -_LITTLE_ENDIAN = 1234 -_BIG_ENDIAN = 4321 -_PDP_ENDIAN = 3412 -_BYTE_ORDER = _LITTLE_ENDIAN -LITTLE_ENDIAN = _LITTLE_ENDIAN -BIG_ENDIAN = _BIG_ENDIAN -PDP_ENDIAN = _PDP_ENDIAN -BYTE_ORDER = _BYTE_ORDER -def __word_swap_int_var(x): return \ - -def __word_swap_int_const(x): return \ - -def __word_swap_int(x): return __word_swap_int_var(x) - -def __byte_swap_int_var(x): return \ - -def __byte_swap_int_const(x): return \ - -def __byte_swap_int(x): return __byte_swap_int_var(x) - -def __byte_swap_word_var(x): return \ - -def __byte_swap_word_const(x): return \ - -def __byte_swap_word(x): return __byte_swap_word_var(x) - -def __htonl(x): return __bswap32(x) - -def __htons(x): return __bswap16(x) - -def __ntohl(x): return __bswap32(x) - -def __ntohs(x): return __bswap16(x) - -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_TCP = 6 -IPPROTO_UDP = 17 -def htonl(x): return __htonl(x) - -def htons(x): return __htons(x) - -def ntohl(x): return __ntohl(x) - -def ntohs(x): return __ntohs(x) - -IPPROTO_RAW = 255 -INET_ADDRSTRLEN = 16 -IPPROTO_HOPOPTS = 0 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_MOBILE = 55 -IPPROTO_TLSP = 56 -IPPROTO_SKIP = 57 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_SCTP = 132 -IPPROTO_PIM = 103 -IPPROTO_CARP = 112 -IPPROTO_PGM = 113 -IPPROTO_PFSYNC = 240 -IPPROTO_OLD_DIVERT = 254 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPROTO_DIVERT = 258 -IPPROTO_SPACER = 32767 -IPPORT_RESERVED = 1024 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -IPPORT_MAX = 65535 -def IN_CLASSA(i): return (((u_int32_t)(i) & (-2147483648)) == 0) - -IN_CLASSA_NET = (-16777216) -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & (-1073741824)) == (-2147483648)) - -IN_CLASSB_NET = (-65536) -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & (-536870912)) == (-1073741824)) - -IN_CLASSC_NET = (-256) -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & (-268435456)) == (-536870912)) - -IN_CLASSD_NET = (-268435456) -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_LINKLOCAL(i): return (((u_int32_t)(i) & (-65536)) == (-1442971648)) - -def IN_LOCAL_GROUP(i): return (((u_int32_t)(i) & (-256)) == (-536870912)) - -INADDR_NONE = (-1) -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_SENDSRCADDR = IP_RECVDSTADDR -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_ONESBCAST = 23 -IP_FW_TABLE_ADD = 40 -IP_FW_TABLE_DEL = 41 -IP_FW_TABLE_FLUSH = 42 -IP_FW_TABLE_GETSIZE = 43 -IP_FW_TABLE_LIST = 44 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_FW_NAT_CFG = 56 -IP_FW_NAT_DEL = 57 -IP_FW_NAT_GET_CONFIG = 58 -IP_FW_NAT_GET_LOG = 59 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_RECVTTL = 65 -IP_MINTTL = 66 -IP_DONTFRAG = 67 -IP_ADD_SOURCE_MEMBERSHIP = 70 -IP_DROP_SOURCE_MEMBERSHIP = 71 -IP_BLOCK_SOURCE = 72 -IP_UNBLOCK_SOURCE = 73 -IP_MSFILTER = 74 -MCAST_JOIN_GROUP = 80 -MCAST_LEAVE_GROUP = 81 -MCAST_JOIN_SOURCE_GROUP = 82 -MCAST_LEAVE_SOURCE_GROUP = 83 -MCAST_BLOCK_SOURCE = 84 -MCAST_UNBLOCK_SOURCE = 85 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MIN_MEMBERSHIPS = 31 -IP_MAX_MEMBERSHIPS = 4095 -IP_MAX_SOURCE_FILTER = 1024 -MCAST_INCLUDE = 1 -MCAST_EXCLUDE = 2 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 -def in_nullhost(x): return ((x).s_addr == INADDR_ANY) - - -# Included from netinet6/in6.h -__KAME_VERSION = "FreeBSD" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = (-16711680) -IPV6_ADDR_INT32_MLL = (-16646144) -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = (-65536) -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_INTFACELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -def IN6_IS_SCOPE_EMBED(a): return \ - -def IFA6_IS_DEPRECATED(a): return \ - -def IFA6_IS_INVALID(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_2292PKTINFO = 19 -IPV6_2292HOPLIMIT = 20 -IPV6_2292NEXTHOP = 21 -IPV6_2292HOPOPTS = 22 -IPV6_2292DSTOPTS = 23 -IPV6_2292RTHDR = 24 -IPV6_2292PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_V6ONLY = 27 -IPV6_BINDV6ONLY = IPV6_V6ONLY -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDRDSTOPTS = 35 -IPV6_RECVPKTINFO = 36 -IPV6_RECVHOPLIMIT = 37 -IPV6_RECVRTHDR = 38 -IPV6_RECVHOPOPTS = 39 -IPV6_RECVDSTOPTS = 40 -IPV6_RECVRTHDRDSTOPTS = 41 -IPV6_USE_MIN_MTU = 42 -IPV6_RECVPATHMTU = 43 -IPV6_PATHMTU = 44 -IPV6_REACHCONF = 45 -IPV6_PKTINFO = 46 -IPV6_HOPLIMIT = 47 -IPV6_NEXTHOP = 48 -IPV6_HOPOPTS = 49 -IPV6_DSTOPTS = 50 -IPV6_RTHDR = 51 -IPV6_PKTOPTIONS = 52 -IPV6_RECVTCLASS = 57 -IPV6_AUTOFLOWLABEL = 59 -IPV6_TCLASS = 61 -IPV6_DONTFRAG = 62 -IPV6_PREFER_TEMPADDR = 63 -IPV6_MSFILTER = 74 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_V6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_USETEMPADDR = 32 -IPV6CTL_TEMPPLTIME = 33 -IPV6CTL_TEMPVLTIME = 34 -IPV6CTL_AUTO_LINKLOCAL = 35 -IPV6CTL_RIP6STATS = 36 -IPV6CTL_PREFER_TEMPADDR = 37 -IPV6CTL_ADDRCTLPOLICY = 38 -IPV6CTL_USE_DEFAULTZONE = 39 -IPV6CTL_MAXFRAGS = 41 -IPV6CTL_IFQ = 42 -IPV6CTL_ISATAPRTR = 43 -IPV6CTL_MCAST_PMTU = 44 -IPV6CTL_STEALTH = 45 -IPV6CTL_MAXID = 46 diff --git a/Lib/plat-freebsd7/regen b/Lib/plat-freebsd7/regen deleted file mode 100755 --- a/Lib/plat-freebsd7/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-freebsd8/IN.py b/Lib/plat-freebsd8/IN.py deleted file mode 100644 --- a/Lib/plat-freebsd8/IN.py +++ /dev/null @@ -1,571 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/cdefs.h -__GNUCLIKE_ASM = 3 -__GNUCLIKE_ASM = 2 -__GNUCLIKE___TYPEOF = 1 -__GNUCLIKE___OFFSETOF = 1 -__GNUCLIKE___SECTION = 1 -__GNUCLIKE_ATTRIBUTE_MODE_DI = 1 -__GNUCLIKE_CTOR_SECTION_HANDLING = 1 -__GNUCLIKE_BUILTIN_CONSTANT_P = 1 -__GNUCLIKE_BUILTIN_VARARGS = 1 -__GNUCLIKE_BUILTIN_STDARG = 1 -__GNUCLIKE_BUILTIN_VAALIST = 1 -__GNUC_VA_LIST_COMPATIBILITY = 1 -__GNUCLIKE_BUILTIN_NEXT_ARG = 1 -__GNUCLIKE_BUILTIN_MEMCPY = 1 -__CC_SUPPORTS_INLINE = 1 -__CC_SUPPORTS___INLINE = 1 -__CC_SUPPORTS___INLINE__ = 1 -__CC_SUPPORTS___FUNC__ = 1 -__CC_SUPPORTS_WARNING = 1 -__CC_SUPPORTS_VARADIC_XXX = 1 -__CC_SUPPORTS_DYNAMIC_ARRAY_INIT = 1 -__CC_INT_IS_32BIT = 1 -def __P(protos): return protos - -def __STRING(x): return #x - -def __XSTRING(x): return __STRING(x) - -def __P(protos): return () - -def __STRING(x): return "x" - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __aligned(x): return __attribute__((__aligned__(x))) - -def __section(x): return __attribute__((__section__(x))) - -def __nonnull(x): return __attribute__((__nonnull__(x))) - -def __predict_true(exp): return __builtin_expect((exp), 1) - -def __predict_false(exp): return __builtin_expect((exp), 0) - -def __predict_true(exp): return (exp) - -def __predict_false(exp): return (exp) - -def __format_arg(fmtarg): return __attribute__((__format_arg__ (fmtarg))) - -def __FBSDID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID(s): return __IDSTRING(__CONCAT(__rcsid_,__LINE__),s) - -def __RCSID_SOURCE(s): return __IDSTRING(__CONCAT(__rcsid_source_,__LINE__),s) - -def __SCCSID(s): return __IDSTRING(__CONCAT(__sccsid_,__LINE__),s) - -def __COPYRIGHT(s): return __IDSTRING(__CONCAT(__copyright_,__LINE__),s) - -_POSIX_C_SOURCE = 199009 -_POSIX_C_SOURCE = 199209 -__XSI_VISIBLE = 600 -_POSIX_C_SOURCE = 200112 -__XSI_VISIBLE = 500 -_POSIX_C_SOURCE = 199506 -_POSIX_C_SOURCE = 198808 -__POSIX_VISIBLE = 200112 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 199506 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199309 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199209 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 199009 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 198808 -__ISO_C_VISIBLE = 0 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1990 -__POSIX_VISIBLE = 0 -__XSI_VISIBLE = 0 -__BSD_VISIBLE = 0 -__ISO_C_VISIBLE = 1999 -__POSIX_VISIBLE = 200112 -__XSI_VISIBLE = 600 -__BSD_VISIBLE = 1 -__ISO_C_VISIBLE = 1999 - -# Included from sys/_types.h - -# Included from machine/_types.h - -# Included from machine/endian.h -_QUAD_HIGHWORD = 1 -_QUAD_LOWWORD = 0 -_LITTLE_ENDIAN = 1234 -_BIG_ENDIAN = 4321 -_PDP_ENDIAN = 3412 -_BYTE_ORDER = _LITTLE_ENDIAN -LITTLE_ENDIAN = _LITTLE_ENDIAN -BIG_ENDIAN = _BIG_ENDIAN -PDP_ENDIAN = _PDP_ENDIAN -BYTE_ORDER = _BYTE_ORDER -def __word_swap_int_var(x): return \ - -def __word_swap_int_const(x): return \ - -def __word_swap_int(x): return __word_swap_int_var(x) - -def __byte_swap_int_var(x): return \ - -def __byte_swap_int_const(x): return \ - -def __byte_swap_int(x): return __byte_swap_int_var(x) - -def __byte_swap_word_var(x): return \ - -def __byte_swap_word_const(x): return \ - -def __byte_swap_word(x): return __byte_swap_word_var(x) - -def __htonl(x): return __bswap32(x) - -def __htons(x): return __bswap16(x) - -def __ntohl(x): return __bswap32(x) - -def __ntohs(x): return __bswap16(x) - -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_TCP = 6 -IPPROTO_UDP = 17 -def htonl(x): return __htonl(x) - -def htons(x): return __htons(x) - -def ntohl(x): return __ntohl(x) - -def ntohs(x): return __ntohs(x) - -IPPROTO_RAW = 255 -INET_ADDRSTRLEN = 16 -IPPROTO_HOPOPTS = 0 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPV4 = 4 -IPPROTO_IPIP = IPPROTO_IPV4 -IPPROTO_ST = 7 -IPPROTO_EGP = 8 -IPPROTO_PIGP = 9 -IPPROTO_RCCMON = 10 -IPPROTO_NVPII = 11 -IPPROTO_PUP = 12 -IPPROTO_ARGUS = 13 -IPPROTO_EMCON = 14 -IPPROTO_XNET = 15 -IPPROTO_CHAOS = 16 -IPPROTO_MUX = 18 -IPPROTO_MEAS = 19 -IPPROTO_HMP = 20 -IPPROTO_PRM = 21 -IPPROTO_IDP = 22 -IPPROTO_TRUNK1 = 23 -IPPROTO_TRUNK2 = 24 -IPPROTO_LEAF1 = 25 -IPPROTO_LEAF2 = 26 -IPPROTO_RDP = 27 -IPPROTO_IRTP = 28 -IPPROTO_TP = 29 -IPPROTO_BLT = 30 -IPPROTO_NSP = 31 -IPPROTO_INP = 32 -IPPROTO_SEP = 33 -IPPROTO_3PC = 34 -IPPROTO_IDPR = 35 -IPPROTO_XTP = 36 -IPPROTO_DDP = 37 -IPPROTO_CMTP = 38 -IPPROTO_TPXX = 39 -IPPROTO_IL = 40 -IPPROTO_IPV6 = 41 -IPPROTO_SDRP = 42 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_IDRP = 45 -IPPROTO_RSVP = 46 -IPPROTO_GRE = 47 -IPPROTO_MHRP = 48 -IPPROTO_BHA = 49 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_INLSP = 52 -IPPROTO_SWIPE = 53 -IPPROTO_NHRP = 54 -IPPROTO_MOBILE = 55 -IPPROTO_TLSP = 56 -IPPROTO_SKIP = 57 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_AHIP = 61 -IPPROTO_CFTP = 62 -IPPROTO_HELLO = 63 -IPPROTO_SATEXPAK = 64 -IPPROTO_KRYPTOLAN = 65 -IPPROTO_RVD = 66 -IPPROTO_IPPC = 67 -IPPROTO_ADFS = 68 -IPPROTO_SATMON = 69 -IPPROTO_VISA = 70 -IPPROTO_IPCV = 71 -IPPROTO_CPNX = 72 -IPPROTO_CPHB = 73 -IPPROTO_WSN = 74 -IPPROTO_PVP = 75 -IPPROTO_BRSATMON = 76 -IPPROTO_ND = 77 -IPPROTO_WBMON = 78 -IPPROTO_WBEXPAK = 79 -IPPROTO_EON = 80 -IPPROTO_VMTP = 81 -IPPROTO_SVMTP = 82 -IPPROTO_VINES = 83 -IPPROTO_TTP = 84 -IPPROTO_IGP = 85 -IPPROTO_DGP = 86 -IPPROTO_TCF = 87 -IPPROTO_IGRP = 88 -IPPROTO_OSPFIGP = 89 -IPPROTO_SRPC = 90 -IPPROTO_LARP = 91 -IPPROTO_MTP = 92 -IPPROTO_AX25 = 93 -IPPROTO_IPEIP = 94 -IPPROTO_MICP = 95 -IPPROTO_SCCSP = 96 -IPPROTO_ETHERIP = 97 -IPPROTO_ENCAP = 98 -IPPROTO_APES = 99 -IPPROTO_GMTP = 100 -IPPROTO_IPCOMP = 108 -IPPROTO_SCTP = 132 -IPPROTO_PIM = 103 -IPPROTO_CARP = 112 -IPPROTO_PGM = 113 -IPPROTO_PFSYNC = 240 -IPPROTO_OLD_DIVERT = 254 -IPPROTO_MAX = 256 -IPPROTO_DONE = 257 -IPPROTO_DIVERT = 258 -IPPROTO_SPACER = 32767 -IPPORT_RESERVED = 1024 -IPPORT_HIFIRSTAUTO = 49152 -IPPORT_HILASTAUTO = 65535 -IPPORT_RESERVEDSTART = 600 -IPPORT_MAX = 65535 -def IN_CLASSA(i): return (((u_int32_t)(i) & (-2147483648)) == 0) - -IN_CLASSA_NET = (-16777216) -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((u_int32_t)(i) & (-1073741824)) == (-2147483648)) - -IN_CLASSB_NET = (-65536) -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((u_int32_t)(i) & (-536870912)) == (-1073741824)) - -IN_CLASSC_NET = (-256) -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((u_int32_t)(i) & (-268435456)) == (-536870912)) - -IN_CLASSD_NET = (-268435456) -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_BADCLASS(i): return (((u_int32_t)(i) & (-268435456)) == (-268435456)) - -def IN_LINKLOCAL(i): return (((u_int32_t)(i) & (-65536)) == (-1442971648)) - -def IN_LOCAL_GROUP(i): return (((u_int32_t)(i) & (-256)) == (-536870912)) - -INADDR_NONE = (-1) -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_SENDSRCADDR = IP_RECVDSTADDR -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_MULTICAST_VIF = 14 -IP_RSVP_ON = 15 -IP_RSVP_OFF = 16 -IP_RSVP_VIF_ON = 17 -IP_RSVP_VIF_OFF = 18 -IP_PORTRANGE = 19 -IP_RECVIF = 20 -IP_IPSEC_POLICY = 21 -IP_FAITH = 22 -IP_ONESBCAST = 23 -IP_FW_TABLE_ADD = 40 -IP_FW_TABLE_DEL = 41 -IP_FW_TABLE_FLUSH = 42 -IP_FW_TABLE_GETSIZE = 43 -IP_FW_TABLE_LIST = 44 -IP_FW_ADD = 50 -IP_FW_DEL = 51 -IP_FW_FLUSH = 52 -IP_FW_ZERO = 53 -IP_FW_GET = 54 -IP_FW_RESETLOG = 55 -IP_FW_NAT_CFG = 56 -IP_FW_NAT_DEL = 57 -IP_FW_NAT_GET_CONFIG = 58 -IP_FW_NAT_GET_LOG = 59 -IP_DUMMYNET_CONFIGURE = 60 -IP_DUMMYNET_DEL = 61 -IP_DUMMYNET_FLUSH = 62 -IP_DUMMYNET_GET = 64 -IP_RECVTTL = 65 -IP_MINTTL = 66 -IP_DONTFRAG = 67 -IP_ADD_SOURCE_MEMBERSHIP = 70 -IP_DROP_SOURCE_MEMBERSHIP = 71 -IP_BLOCK_SOURCE = 72 -IP_UNBLOCK_SOURCE = 73 -IP_MSFILTER = 74 -MCAST_JOIN_GROUP = 80 -MCAST_LEAVE_GROUP = 81 -MCAST_JOIN_SOURCE_GROUP = 82 -MCAST_LEAVE_SOURCE_GROUP = 83 -MCAST_BLOCK_SOURCE = 84 -MCAST_UNBLOCK_SOURCE = 85 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MIN_MEMBERSHIPS = 31 -IP_MAX_MEMBERSHIPS = 4095 -IP_MAX_SOURCE_FILTER = 1024 -MCAST_INCLUDE = 1 -MCAST_EXCLUDE = 2 -IP_PORTRANGE_DEFAULT = 0 -IP_PORTRANGE_HIGH = 1 -IP_PORTRANGE_LOW = 2 -IPPROTO_MAXID = (IPPROTO_AH + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_RTEXPIRE = 5 -IPCTL_RTMINEXPIRE = 6 -IPCTL_RTMAXCACHE = 7 -IPCTL_SOURCEROUTE = 8 -IPCTL_DIRECTEDBROADCAST = 9 -IPCTL_INTRQMAXLEN = 10 -IPCTL_INTRQDROPS = 11 -IPCTL_STATS = 12 -IPCTL_ACCEPTSOURCEROUTE = 13 -IPCTL_FASTFORWARDING = 14 -IPCTL_KEEPFAITH = 15 -IPCTL_GIF_TTL = 16 -IPCTL_MAXID = 17 -def in_nullhost(x): return ((x).s_addr == INADDR_ANY) - - -# Included from netinet6/in6.h -__KAME_VERSION = "FreeBSD" -IPV6PORT_RESERVED = 1024 -IPV6PORT_ANONMIN = 49152 -IPV6PORT_ANONMAX = 65535 -IPV6PORT_RESERVEDMIN = 600 -IPV6PORT_RESERVEDMAX = (IPV6PORT_RESERVED-1) -INET6_ADDRSTRLEN = 46 -IPV6_ADDR_INT32_ONE = 1 -IPV6_ADDR_INT32_TWO = 2 -IPV6_ADDR_INT32_MNL = (-16711680) -IPV6_ADDR_INT32_MLL = (-16646144) -IPV6_ADDR_INT32_SMP = 0x0000ffff -IPV6_ADDR_INT16_ULL = 0xfe80 -IPV6_ADDR_INT16_USL = 0xfec0 -IPV6_ADDR_INT16_MLL = 0xff02 -IPV6_ADDR_INT32_ONE = 0x01000000 -IPV6_ADDR_INT32_TWO = 0x02000000 -IPV6_ADDR_INT32_MNL = 0x000001ff -IPV6_ADDR_INT32_MLL = 0x000002ff -IPV6_ADDR_INT32_SMP = (-65536) -IPV6_ADDR_INT16_ULL = 0x80fe -IPV6_ADDR_INT16_USL = 0xc0fe -IPV6_ADDR_INT16_MLL = 0x02ff -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -IPV6_ADDR_SCOPE_GLOBAL = 0x0e -__IPV6_ADDR_SCOPE_NODELOCAL = 0x01 -__IPV6_ADDR_SCOPE_INTFACELOCAL = 0x01 -__IPV6_ADDR_SCOPE_LINKLOCAL = 0x02 -__IPV6_ADDR_SCOPE_SITELOCAL = 0x05 -__IPV6_ADDR_SCOPE_ORGLOCAL = 0x08 -__IPV6_ADDR_SCOPE_GLOBAL = 0x0e -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_INTFACELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - -def IN6_IS_SCOPE_LINKLOCAL(a): return \ - -def IN6_IS_SCOPE_EMBED(a): return \ - -def IFA6_IS_DEPRECATED(a): return \ - -def IFA6_IS_INVALID(a): return \ - -IPV6_OPTIONS = 1 -IPV6_RECVOPTS = 5 -IPV6_RECVRETOPTS = 6 -IPV6_RECVDSTADDR = 7 -IPV6_RETOPTS = 8 -IPV6_SOCKOPT_RESERVED1 = 3 -IPV6_UNICAST_HOPS = 4 -IPV6_MULTICAST_IF = 9 -IPV6_MULTICAST_HOPS = 10 -IPV6_MULTICAST_LOOP = 11 -IPV6_JOIN_GROUP = 12 -IPV6_LEAVE_GROUP = 13 -IPV6_PORTRANGE = 14 -ICMP6_FILTER = 18 -IPV6_2292PKTINFO = 19 -IPV6_2292HOPLIMIT = 20 -IPV6_2292NEXTHOP = 21 -IPV6_2292HOPOPTS = 22 -IPV6_2292DSTOPTS = 23 -IPV6_2292RTHDR = 24 -IPV6_2292PKTOPTIONS = 25 -IPV6_CHECKSUM = 26 -IPV6_V6ONLY = 27 -IPV6_BINDV6ONLY = IPV6_V6ONLY -IPV6_IPSEC_POLICY = 28 -IPV6_FAITH = 29 -IPV6_FW_ADD = 30 -IPV6_FW_DEL = 31 -IPV6_FW_FLUSH = 32 -IPV6_FW_ZERO = 33 -IPV6_FW_GET = 34 -IPV6_RTHDRDSTOPTS = 35 -IPV6_RECVPKTINFO = 36 -IPV6_RECVHOPLIMIT = 37 -IPV6_RECVRTHDR = 38 -IPV6_RECVHOPOPTS = 39 -IPV6_RECVDSTOPTS = 40 -IPV6_RECVRTHDRDSTOPTS = 41 -IPV6_USE_MIN_MTU = 42 -IPV6_RECVPATHMTU = 43 -IPV6_PATHMTU = 44 -IPV6_REACHCONF = 45 -IPV6_PKTINFO = 46 -IPV6_HOPLIMIT = 47 -IPV6_NEXTHOP = 48 -IPV6_HOPOPTS = 49 -IPV6_DSTOPTS = 50 -IPV6_RTHDR = 51 -IPV6_PKTOPTIONS = 52 -IPV6_RECVTCLASS = 57 -IPV6_AUTOFLOWLABEL = 59 -IPV6_TCLASS = 61 -IPV6_DONTFRAG = 62 -IPV6_PREFER_TEMPADDR = 63 -IPV6_MSFILTER = 74 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_DEFAULT_MULTICAST_HOPS = 1 -IPV6_DEFAULT_MULTICAST_LOOP = 1 -IPV6_PORTRANGE_DEFAULT = 0 -IPV6_PORTRANGE_HIGH = 1 -IPV6_PORTRANGE_LOW = 2 -IPV6PROTO_MAXID = (IPPROTO_PIM + 1) -IPV6CTL_FORWARDING = 1 -IPV6CTL_SENDREDIRECTS = 2 -IPV6CTL_DEFHLIM = 3 -IPV6CTL_DEFMTU = 4 -IPV6CTL_FORWSRCRT = 5 -IPV6CTL_STATS = 6 -IPV6CTL_MRTSTATS = 7 -IPV6CTL_MRTPROTO = 8 -IPV6CTL_MAXFRAGPACKETS = 9 -IPV6CTL_SOURCECHECK = 10 -IPV6CTL_SOURCECHECK_LOGINT = 11 -IPV6CTL_ACCEPT_RTADV = 12 -IPV6CTL_KEEPFAITH = 13 -IPV6CTL_LOG_INTERVAL = 14 -IPV6CTL_HDRNESTLIMIT = 15 -IPV6CTL_DAD_COUNT = 16 -IPV6CTL_AUTO_FLOWLABEL = 17 -IPV6CTL_DEFMCASTHLIM = 18 -IPV6CTL_GIF_HLIM = 19 -IPV6CTL_KAME_VERSION = 20 -IPV6CTL_USE_DEPRECATED = 21 -IPV6CTL_RR_PRUNE = 22 -IPV6CTL_MAPPED_ADDR = 23 -IPV6CTL_V6ONLY = 24 -IPV6CTL_RTEXPIRE = 25 -IPV6CTL_RTMINEXPIRE = 26 -IPV6CTL_RTMAXCACHE = 27 -IPV6CTL_USETEMPADDR = 32 -IPV6CTL_TEMPPLTIME = 33 -IPV6CTL_TEMPVLTIME = 34 -IPV6CTL_AUTO_LINKLOCAL = 35 -IPV6CTL_RIP6STATS = 36 -IPV6CTL_PREFER_TEMPADDR = 37 -IPV6CTL_ADDRCTLPOLICY = 38 -IPV6CTL_USE_DEFAULTZONE = 39 -IPV6CTL_MAXFRAGS = 41 -IPV6CTL_IFQ = 42 -IPV6CTL_ISATAPRTR = 43 -IPV6CTL_MCAST_PMTU = 44 -IPV6CTL_STEALTH = 45 -IPV6CTL_MAXID = 46 diff --git a/Lib/plat-freebsd8/regen b/Lib/plat-freebsd8/regen deleted file mode 100755 --- a/Lib/plat-freebsd8/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-generic/regen b/Lib/plat-generic/regen deleted file mode 100755 --- a/Lib/plat-generic/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -eval $PYTHON_FOR_BUILD ../../Tools/scripts/h2py.py -i "'(u_long)'" /usr/include/netinet/in.h diff --git a/Lib/plat-linux/CDROM.py b/Lib/plat-linux/CDROM.py deleted file mode 100644 --- a/Lib/plat-linux/CDROM.py +++ /dev/null @@ -1,207 +0,0 @@ -# Generated by h2py from /usr/include/linux/cdrom.h - -CDROMPAUSE = 0x5301 -CDROMRESUME = 0x5302 -CDROMPLAYMSF = 0x5303 -CDROMPLAYTRKIND = 0x5304 -CDROMREADTOCHDR = 0x5305 -CDROMREADTOCENTRY = 0x5306 -CDROMSTOP = 0x5307 -CDROMSTART = 0x5308 -CDROMEJECT = 0x5309 -CDROMVOLCTRL = 0x530a -CDROMSUBCHNL = 0x530b -CDROMREADMODE2 = 0x530c -CDROMREADMODE1 = 0x530d -CDROMREADAUDIO = 0x530e -CDROMEJECT_SW = 0x530f -CDROMMULTISESSION = 0x5310 -CDROM_GET_MCN = 0x5311 -CDROM_GET_UPC = CDROM_GET_MCN -CDROMRESET = 0x5312 -CDROMVOLREAD = 0x5313 -CDROMREADRAW = 0x5314 -CDROMREADCOOKED = 0x5315 -CDROMSEEK = 0x5316 -CDROMPLAYBLK = 0x5317 -CDROMREADALL = 0x5318 -CDROMGETSPINDOWN = 0x531d -CDROMSETSPINDOWN = 0x531e -CDROMCLOSETRAY = 0x5319 -CDROM_SET_OPTIONS = 0x5320 -CDROM_CLEAR_OPTIONS = 0x5321 -CDROM_SELECT_SPEED = 0x5322 -CDROM_SELECT_DISC = 0x5323 -CDROM_MEDIA_CHANGED = 0x5325 -CDROM_DRIVE_STATUS = 0x5326 -CDROM_DISC_STATUS = 0x5327 -CDROM_CHANGER_NSLOTS = 0x5328 -CDROM_LOCKDOOR = 0x5329 -CDROM_DEBUG = 0x5330 -CDROM_GET_CAPABILITY = 0x5331 -CDROMAUDIOBUFSIZ = 0x5382 -DVD_READ_STRUCT = 0x5390 -DVD_WRITE_STRUCT = 0x5391 -DVD_AUTH = 0x5392 -CDROM_SEND_PACKET = 0x5393 -CDROM_NEXT_WRITABLE = 0x5394 -CDROM_LAST_WRITTEN = 0x5395 -CDROM_PACKET_SIZE = 12 -CGC_DATA_UNKNOWN = 0 -CGC_DATA_WRITE = 1 -CGC_DATA_READ = 2 -CGC_DATA_NONE = 3 -CD_MINS = 74 -CD_SECS = 60 -CD_FRAMES = 75 -CD_SYNC_SIZE = 12 -CD_MSF_OFFSET = 150 -CD_CHUNK_SIZE = 24 -CD_NUM_OF_CHUNKS = 98 -CD_FRAMESIZE_SUB = 96 -CD_HEAD_SIZE = 4 -CD_SUBHEAD_SIZE = 8 -CD_EDC_SIZE = 4 -CD_ZERO_SIZE = 8 -CD_ECC_SIZE = 276 -CD_FRAMESIZE = 2048 -CD_FRAMESIZE_RAW = 2352 -CD_FRAMESIZE_RAWER = 2646 -CD_FRAMESIZE_RAW1 = (CD_FRAMESIZE_RAW-CD_SYNC_SIZE) -CD_FRAMESIZE_RAW0 = (CD_FRAMESIZE_RAW-CD_SYNC_SIZE-CD_HEAD_SIZE) -CD_XA_HEAD = (CD_HEAD_SIZE+CD_SUBHEAD_SIZE) -CD_XA_TAIL = (CD_EDC_SIZE+CD_ECC_SIZE) -CD_XA_SYNC_HEAD = (CD_SYNC_SIZE+CD_XA_HEAD) -CDROM_LBA = 0x01 -CDROM_MSF = 0x02 -CDROM_DATA_TRACK = 0x04 -CDROM_LEADOUT = 0xAA -CDROM_AUDIO_INVALID = 0x00 -CDROM_AUDIO_PLAY = 0x11 -CDROM_AUDIO_PAUSED = 0x12 -CDROM_AUDIO_COMPLETED = 0x13 -CDROM_AUDIO_ERROR = 0x14 -CDROM_AUDIO_NO_STATUS = 0x15 -CDC_CLOSE_TRAY = 0x1 -CDC_OPEN_TRAY = 0x2 -CDC_LOCK = 0x4 -CDC_SELECT_SPEED = 0x8 -CDC_SELECT_DISC = 0x10 -CDC_MULTI_SESSION = 0x20 -CDC_MCN = 0x40 -CDC_MEDIA_CHANGED = 0x80 -CDC_PLAY_AUDIO = 0x100 -CDC_RESET = 0x200 -CDC_IOCTLS = 0x400 -CDC_DRIVE_STATUS = 0x800 -CDC_GENERIC_PACKET = 0x1000 -CDC_CD_R = 0x2000 -CDC_CD_RW = 0x4000 -CDC_DVD = 0x8000 -CDC_DVD_R = 0x10000 -CDC_DVD_RAM = 0x20000 -CDS_NO_INFO = 0 -CDS_NO_DISC = 1 -CDS_TRAY_OPEN = 2 -CDS_DRIVE_NOT_READY = 3 -CDS_DISC_OK = 4 -CDS_AUDIO = 100 -CDS_DATA_1 = 101 -CDS_DATA_2 = 102 -CDS_XA_2_1 = 103 -CDS_XA_2_2 = 104 -CDS_MIXED = 105 -CDO_AUTO_CLOSE = 0x1 -CDO_AUTO_EJECT = 0x2 -CDO_USE_FFLAGS = 0x4 -CDO_LOCK = 0x8 -CDO_CHECK_TYPE = 0x10 -CD_PART_MAX = 64 -CD_PART_MASK = (CD_PART_MAX - 1) -GPCMD_BLANK = 0xa1 -GPCMD_CLOSE_TRACK = 0x5b -GPCMD_FLUSH_CACHE = 0x35 -GPCMD_FORMAT_UNIT = 0x04 -GPCMD_GET_CONFIGURATION = 0x46 -GPCMD_GET_EVENT_STATUS_NOTIFICATION = 0x4a -GPCMD_GET_PERFORMANCE = 0xac -GPCMD_INQUIRY = 0x12 -GPCMD_LOAD_UNLOAD = 0xa6 -GPCMD_MECHANISM_STATUS = 0xbd -GPCMD_MODE_SELECT_10 = 0x55 -GPCMD_MODE_SENSE_10 = 0x5a -GPCMD_PAUSE_RESUME = 0x4b -GPCMD_PLAY_AUDIO_10 = 0x45 -GPCMD_PLAY_AUDIO_MSF = 0x47 -GPCMD_PLAY_AUDIO_TI = 0x48 -GPCMD_PLAY_CD = 0xbc -GPCMD_PREVENT_ALLOW_MEDIUM_REMOVAL = 0x1e -GPCMD_READ_10 = 0x28 -GPCMD_READ_12 = 0xa8 -GPCMD_READ_CDVD_CAPACITY = 0x25 -GPCMD_READ_CD = 0xbe -GPCMD_READ_CD_MSF = 0xb9 -GPCMD_READ_DISC_INFO = 0x51 -GPCMD_READ_DVD_STRUCTURE = 0xad -GPCMD_READ_FORMAT_CAPACITIES = 0x23 -GPCMD_READ_HEADER = 0x44 -GPCMD_READ_TRACK_RZONE_INFO = 0x52 -GPCMD_READ_SUBCHANNEL = 0x42 -GPCMD_READ_TOC_PMA_ATIP = 0x43 -GPCMD_REPAIR_RZONE_TRACK = 0x58 -GPCMD_REPORT_KEY = 0xa4 -GPCMD_REQUEST_SENSE = 0x03 -GPCMD_RESERVE_RZONE_TRACK = 0x53 -GPCMD_SCAN = 0xba -GPCMD_SEEK = 0x2b -GPCMD_SEND_DVD_STRUCTURE = 0xad -GPCMD_SEND_EVENT = 0xa2 -GPCMD_SEND_KEY = 0xa3 -GPCMD_SEND_OPC = 0x54 -GPCMD_SET_READ_AHEAD = 0xa7 -GPCMD_SET_STREAMING = 0xb6 -GPCMD_START_STOP_UNIT = 0x1b -GPCMD_STOP_PLAY_SCAN = 0x4e -GPCMD_TEST_UNIT_READY = 0x00 -GPCMD_VERIFY_10 = 0x2f -GPCMD_WRITE_10 = 0x2a -GPCMD_WRITE_AND_VERIFY_10 = 0x2e -GPCMD_SET_SPEED = 0xbb -GPCMD_PLAYAUDIO_TI = 0x48 -GPCMD_GET_MEDIA_STATUS = 0xda -GPMODE_R_W_ERROR_PAGE = 0x01 -GPMODE_WRITE_PARMS_PAGE = 0x05 -GPMODE_AUDIO_CTL_PAGE = 0x0e -GPMODE_POWER_PAGE = 0x1a -GPMODE_FAULT_FAIL_PAGE = 0x1c -GPMODE_TO_PROTECT_PAGE = 0x1d -GPMODE_CAPABILITIES_PAGE = 0x2a -GPMODE_ALL_PAGES = 0x3f -GPMODE_CDROM_PAGE = 0x0d -DVD_STRUCT_PHYSICAL = 0x00 -DVD_STRUCT_COPYRIGHT = 0x01 -DVD_STRUCT_DISCKEY = 0x02 -DVD_STRUCT_BCA = 0x03 -DVD_STRUCT_MANUFACT = 0x04 -DVD_LAYERS = 4 -DVD_LU_SEND_AGID = 0 -DVD_HOST_SEND_CHALLENGE = 1 -DVD_LU_SEND_KEY1 = 2 -DVD_LU_SEND_CHALLENGE = 3 -DVD_HOST_SEND_KEY2 = 4 -DVD_AUTH_ESTABLISHED = 5 -DVD_AUTH_FAILURE = 6 -DVD_LU_SEND_TITLE_KEY = 7 -DVD_LU_SEND_ASF = 8 -DVD_INVALIDATE_AGID = 9 -DVD_LU_SEND_RPC_STATE = 10 -DVD_HOST_SEND_RPC_STATE = 11 -DVD_CPM_NO_COPYRIGHT = 0 -DVD_CPM_COPYRIGHTED = 1 -DVD_CP_SEC_NONE = 0 -DVD_CP_SEC_EXIST = 1 -DVD_CGMS_UNRESTRICTED = 0 -DVD_CGMS_SINGLE = 2 -DVD_CGMS_RESTRICTED = 3 - -CDROM_MAX_SLOTS = 256 diff --git a/Lib/plat-linux/DLFCN.py b/Lib/plat-linux/DLFCN.py deleted file mode 100644 --- a/Lib/plat-linux/DLFCN.py +++ /dev/null @@ -1,83 +0,0 @@ -# Generated by h2py from /usr/include/dlfcn.h -_DLFCN_H = 1 - -# Included from features.h -_FEATURES_H = 1 -__USE_ANSI = 1 -__FAVOR_BSD = 1 -_ISOC99_SOURCE = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 199506 -_XOPEN_SOURCE = 600 -_XOPEN_SOURCE_EXTENDED = 1 -_LARGEFILE64_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -__USE_ISOC99 = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 2 -_POSIX_C_SOURCE = 199506 -__USE_POSIX = 1 -__USE_POSIX2 = 1 -__USE_POSIX199309 = 1 -__USE_POSIX199506 = 1 -__USE_XOPEN = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_UNIX98 = 1 -_LARGEFILE_SOURCE = 1 -__USE_XOPEN2K = 1 -__USE_ISOC99 = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_FILE_OFFSET64 = 1 -__USE_MISC = 1 -__USE_BSD = 1 -__USE_SVID = 1 -__USE_GNU = 1 -__USE_REENTRANT = 1 -__STDC_IEC_559__ = 1 -__STDC_IEC_559_COMPLEX__ = 1 -__STDC_ISO_10646__ = 200009 -__GNU_LIBRARY__ = 6 -__GLIBC__ = 2 -__GLIBC_MINOR__ = 2 - -# Included from sys/cdefs.h -_SYS_CDEFS_H = 1 -def __PMT(args): return args - -def __P(args): return args - -def __PMT(args): return args - -def __STRING(x): return #x - -__flexarr = [] -__flexarr = [0] -__flexarr = [] -__flexarr = [1] -def __ASMNAME(cname): return __ASMNAME2 (__USER_LABEL_PREFIX__, cname) - -def __attribute__(xyz): return - -def __attribute_format_arg__(x): return __attribute__ ((__format_arg__ (x))) - -def __attribute_format_arg__(x): return - -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_EXTERN_INLINES = 1 - -# Included from gnu/stubs.h - -# Included from bits/dlfcn.h -RTLD_LAZY = 0x00001 -RTLD_NOW = 0x00002 -RTLD_BINDING_MASK = 0x3 -RTLD_NOLOAD = 0x00004 -RTLD_GLOBAL = 0x00100 -RTLD_LOCAL = 0 -RTLD_NODELETE = 0x01000 diff --git a/Lib/plat-linux/IN.py b/Lib/plat-linux/IN.py deleted file mode 100644 --- a/Lib/plat-linux/IN.py +++ /dev/null @@ -1,615 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h -_NETINET_IN_H = 1 - -# Included from features.h -_FEATURES_H = 1 -__USE_ANSI = 1 -__FAVOR_BSD = 1 -_ISOC99_SOURCE = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 199506 -_XOPEN_SOURCE = 600 -_XOPEN_SOURCE_EXTENDED = 1 -_LARGEFILE64_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -__USE_ISOC99 = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 2 -_POSIX_C_SOURCE = 199506 -__USE_POSIX = 1 -__USE_POSIX2 = 1 -__USE_POSIX199309 = 1 -__USE_POSIX199506 = 1 -__USE_XOPEN = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_UNIX98 = 1 -_LARGEFILE_SOURCE = 1 -__USE_XOPEN2K = 1 -__USE_ISOC99 = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_FILE_OFFSET64 = 1 -__USE_MISC = 1 -__USE_BSD = 1 -__USE_SVID = 1 -__USE_GNU = 1 -__USE_REENTRANT = 1 -__STDC_IEC_559__ = 1 -__STDC_IEC_559_COMPLEX__ = 1 -__STDC_ISO_10646__ = 200009 -__GNU_LIBRARY__ = 6 -__GLIBC__ = 2 -__GLIBC_MINOR__ = 2 - -# Included from sys/cdefs.h -_SYS_CDEFS_H = 1 -def __PMT(args): return args - -def __P(args): return args - -def __PMT(args): return args - -def __STRING(x): return #x - -__flexarr = [] -__flexarr = [0] -__flexarr = [] -__flexarr = [1] -def __ASMNAME(cname): return __ASMNAME2 (__USER_LABEL_PREFIX__, cname) - -def __attribute__(xyz): return - -def __attribute_format_arg__(x): return __attribute__ ((__format_arg__ (x))) - -def __attribute_format_arg__(x): return - -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_EXTERN_INLINES = 1 - -# Included from gnu/stubs.h - -# Included from stdint.h -_STDINT_H = 1 - -# Included from bits/wchar.h -_BITS_WCHAR_H = 1 -__WCHAR_MIN = (-2147483647 - 1) -__WCHAR_MAX = (2147483647) - -# Included from bits/wordsize.h -__WORDSIZE = 32 -def __INT64_C(c): return c ## L - -def __UINT64_C(c): return c ## UL - -def __INT64_C(c): return c ## LL - -def __UINT64_C(c): return c ## ULL - -INT8_MIN = (-128) -INT16_MIN = (-32767-1) -INT32_MIN = (-2147483647-1) -INT64_MIN = (-__INT64_C(9223372036854775807)-1) -INT8_MAX = (127) -INT16_MAX = (32767) -INT32_MAX = (2147483647) -INT64_MAX = (__INT64_C(9223372036854775807)) -UINT8_MAX = (255) -UINT16_MAX = (65535) -UINT64_MAX = (__UINT64_C(18446744073709551615)) -INT_LEAST8_MIN = (-128) -INT_LEAST16_MIN = (-32767-1) -INT_LEAST32_MIN = (-2147483647-1) -INT_LEAST64_MIN = (-__INT64_C(9223372036854775807)-1) -INT_LEAST8_MAX = (127) -INT_LEAST16_MAX = (32767) -INT_LEAST32_MAX = (2147483647) -INT_LEAST64_MAX = (__INT64_C(9223372036854775807)) -UINT_LEAST8_MAX = (255) -UINT_LEAST16_MAX = (65535) -UINT_LEAST64_MAX = (__UINT64_C(18446744073709551615)) -INT_FAST8_MIN = (-128) -INT_FAST16_MIN = (-9223372036854775807-1) -INT_FAST32_MIN = (-9223372036854775807-1) -INT_FAST16_MIN = (-2147483647-1) -INT_FAST32_MIN = (-2147483647-1) -INT_FAST64_MIN = (-__INT64_C(9223372036854775807)-1) -INT_FAST8_MAX = (127) -INT_FAST16_MAX = (9223372036854775807) -INT_FAST32_MAX = (9223372036854775807) -INT_FAST16_MAX = (2147483647) -INT_FAST32_MAX = (2147483647) -INT_FAST64_MAX = (__INT64_C(9223372036854775807)) -UINT_FAST8_MAX = (255) -UINT_FAST64_MAX = (__UINT64_C(18446744073709551615)) -INTPTR_MIN = (-9223372036854775807-1) -INTPTR_MAX = (9223372036854775807) -INTPTR_MIN = (-2147483647-1) -INTPTR_MAX = (2147483647) -INTMAX_MIN = (-__INT64_C(9223372036854775807)-1) -INTMAX_MAX = (__INT64_C(9223372036854775807)) -UINTMAX_MAX = (__UINT64_C(18446744073709551615)) -PTRDIFF_MIN = (-9223372036854775807-1) -PTRDIFF_MAX = (9223372036854775807) -PTRDIFF_MIN = (-2147483647-1) -PTRDIFF_MAX = (2147483647) -SIG_ATOMIC_MIN = (-2147483647-1) -SIG_ATOMIC_MAX = (2147483647) -WCHAR_MIN = __WCHAR_MIN -WCHAR_MAX = __WCHAR_MAX -def INT8_C(c): return c - -def INT16_C(c): return c - -def INT32_C(c): return c - -def INT64_C(c): return c ## L - -def INT64_C(c): return c ## LL - -def UINT8_C(c): return c ## U - -def UINT16_C(c): return c ## U - -def UINT32_C(c): return c ## U - -def UINT64_C(c): return c ## UL - -def UINT64_C(c): return c ## ULL - -def INTMAX_C(c): return c ## L - -def UINTMAX_C(c): return c ## UL - -def INTMAX_C(c): return c ## LL - -def UINTMAX_C(c): return c ## ULL - - -# Included from bits/types.h -_BITS_TYPES_H = 1 -__FD_SETSIZE = 1024 - -# Included from bits/pthreadtypes.h -_BITS_PTHREADTYPES_H = 1 - -# Included from bits/sched.h -SCHED_OTHER = 0 -SCHED_FIFO = 1 -SCHED_RR = 2 -CSIGNAL = 0x000000ff -CLONE_VM = 0x00000100 -CLONE_FS = 0x00000200 -CLONE_FILES = 0x00000400 -CLONE_SIGHAND = 0x00000800 -CLONE_PID = 0x00001000 -CLONE_PTRACE = 0x00002000 -CLONE_VFORK = 0x00004000 -__defined_schedparam = 1 -def IN_CLASSA(a): return ((((in_addr_t)(a)) & (-2147483648)) == 0) - -IN_CLASSA_NET = (-16777216) -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = ((-1) & ~IN_CLASSA_NET) -IN_CLASSA_MAX = 128 -def IN_CLASSB(a): return ((((in_addr_t)(a)) & (-1073741824)) == (-2147483648)) - -IN_CLASSB_NET = (-65536) -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = ((-1) & ~IN_CLASSB_NET) -IN_CLASSB_MAX = 65536 -def IN_CLASSC(a): return ((((in_addr_t)(a)) & (-536870912)) == (-1073741824)) - -IN_CLASSC_NET = (-256) -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = ((-1) & ~IN_CLASSC_NET) -def IN_CLASSD(a): return ((((in_addr_t)(a)) & (-268435456)) == (-536870912)) - -def IN_MULTICAST(a): return IN_CLASSD(a) - -def IN_EXPERIMENTAL(a): return ((((in_addr_t)(a)) & (-536870912)) == (-536870912)) - -def IN_BADCLASS(a): return ((((in_addr_t)(a)) & (-268435456)) == (-268435456)) - -IN_LOOPBACKNET = 127 -INET_ADDRSTRLEN = 16 -INET6_ADDRSTRLEN = 46 - -# Included from bits/socket.h - -# Included from limits.h -_LIBC_LIMITS_H_ = 1 -MB_LEN_MAX = 16 -_LIMITS_H = 1 -CHAR_BIT = 8 -SCHAR_MIN = (-128) -SCHAR_MAX = 127 -UCHAR_MAX = 255 -CHAR_MIN = 0 -CHAR_MAX = UCHAR_MAX -CHAR_MIN = SCHAR_MIN -CHAR_MAX = SCHAR_MAX -SHRT_MIN = (-32768) -SHRT_MAX = 32767 -USHRT_MAX = 65535 -INT_MAX = 2147483647 -LONG_MAX = 9223372036854775807 -LONG_MAX = 2147483647 -LONG_MIN = (-LONG_MAX - 1) - -# Included from bits/posix1_lim.h -_BITS_POSIX1_LIM_H = 1 -_POSIX_AIO_LISTIO_MAX = 2 -_POSIX_AIO_MAX = 1 -_POSIX_ARG_MAX = 4096 -_POSIX_CHILD_MAX = 6 -_POSIX_DELAYTIMER_MAX = 32 -_POSIX_LINK_MAX = 8 -_POSIX_MAX_CANON = 255 -_POSIX_MAX_INPUT = 255 -_POSIX_MQ_OPEN_MAX = 8 -_POSIX_MQ_PRIO_MAX = 32 -_POSIX_NGROUPS_MAX = 0 -_POSIX_OPEN_MAX = 16 -_POSIX_FD_SETSIZE = _POSIX_OPEN_MAX -_POSIX_NAME_MAX = 14 -_POSIX_PATH_MAX = 256 -_POSIX_PIPE_BUF = 512 -_POSIX_RTSIG_MAX = 8 -_POSIX_SEM_NSEMS_MAX = 256 -_POSIX_SEM_VALUE_MAX = 32767 -_POSIX_SIGQUEUE_MAX = 32 -_POSIX_SSIZE_MAX = 32767 -_POSIX_STREAM_MAX = 8 -_POSIX_TZNAME_MAX = 6 -_POSIX_QLIMIT = 1 -_POSIX_HIWAT = _POSIX_PIPE_BUF -_POSIX_UIO_MAXIOV = 16 -_POSIX_TTY_NAME_MAX = 9 -_POSIX_TIMER_MAX = 32 -_POSIX_LOGIN_NAME_MAX = 9 -_POSIX_CLOCKRES_MIN = 20000000 - -# Included from bits/local_lim.h - -# Included from linux/limits.h -NR_OPEN = 1024 -NGROUPS_MAX = 32 -ARG_MAX = 131072 -CHILD_MAX = 999 -OPEN_MAX = 256 -LINK_MAX = 127 -MAX_CANON = 255 -MAX_INPUT = 255 -NAME_MAX = 255 -PATH_MAX = 4096 -PIPE_BUF = 4096 -RTSIG_MAX = 32 -_POSIX_THREAD_KEYS_MAX = 128 -PTHREAD_KEYS_MAX = 1024 -_POSIX_THREAD_DESTRUCTOR_ITERATIONS = 4 -PTHREAD_DESTRUCTOR_ITERATIONS = _POSIX_THREAD_DESTRUCTOR_ITERATIONS -_POSIX_THREAD_THREADS_MAX = 64 -PTHREAD_THREADS_MAX = 1024 -AIO_PRIO_DELTA_MAX = 20 -PTHREAD_STACK_MIN = 16384 -TIMER_MAX = 256 -SSIZE_MAX = LONG_MAX -NGROUPS_MAX = _POSIX_NGROUPS_MAX - -# Included from bits/posix2_lim.h -_BITS_POSIX2_LIM_H = 1 -_POSIX2_BC_BASE_MAX = 99 -_POSIX2_BC_DIM_MAX = 2048 -_POSIX2_BC_SCALE_MAX = 99 -_POSIX2_BC_STRING_MAX = 1000 -_POSIX2_COLL_WEIGHTS_MAX = 2 -_POSIX2_EXPR_NEST_MAX = 32 -_POSIX2_LINE_MAX = 2048 -_POSIX2_RE_DUP_MAX = 255 -_POSIX2_CHARCLASS_NAME_MAX = 14 -BC_BASE_MAX = _POSIX2_BC_BASE_MAX -BC_DIM_MAX = _POSIX2_BC_DIM_MAX -BC_SCALE_MAX = _POSIX2_BC_SCALE_MAX -BC_STRING_MAX = _POSIX2_BC_STRING_MAX -COLL_WEIGHTS_MAX = 255 -EXPR_NEST_MAX = _POSIX2_EXPR_NEST_MAX -LINE_MAX = _POSIX2_LINE_MAX -CHARCLASS_NAME_MAX = 2048 -RE_DUP_MAX = (0x7fff) - -# Included from bits/xopen_lim.h -_XOPEN_LIM_H = 1 - -# Included from bits/stdio_lim.h -L_tmpnam = 20 -TMP_MAX = 238328 -FILENAME_MAX = 4096 -L_ctermid = 9 -L_cuserid = 9 -FOPEN_MAX = 16 -IOV_MAX = 1024 -_XOPEN_IOV_MAX = _POSIX_UIO_MAXIOV -NL_ARGMAX = _POSIX_ARG_MAX -NL_LANGMAX = _POSIX2_LINE_MAX -NL_MSGMAX = INT_MAX -NL_NMAX = INT_MAX -NL_SETMAX = INT_MAX -NL_TEXTMAX = INT_MAX -NZERO = 20 -WORD_BIT = 16 -WORD_BIT = 32 -WORD_BIT = 64 -WORD_BIT = 16 -WORD_BIT = 32 -WORD_BIT = 64 -WORD_BIT = 32 -LONG_BIT = 32 -LONG_BIT = 64 -LONG_BIT = 32 -LONG_BIT = 64 -LONG_BIT = 64 -LONG_BIT = 32 -from TYPES import * -PF_UNSPEC = 0 -PF_LOCAL = 1 -PF_UNIX = PF_LOCAL -PF_FILE = PF_LOCAL -PF_INET = 2 -PF_AX25 = 3 -PF_IPX = 4 -PF_APPLETALK = 5 -PF_NETROM = 6 -PF_BRIDGE = 7 -PF_ATMPVC = 8 -PF_X25 = 9 -PF_INET6 = 10 -PF_ROSE = 11 -PF_DECnet = 12 -PF_NETBEUI = 13 -PF_SECURITY = 14 -PF_KEY = 15 -PF_NETLINK = 16 -PF_ROUTE = PF_NETLINK -PF_PACKET = 17 -PF_ASH = 18 -PF_ECONET = 19 -PF_ATMSVC = 20 -PF_SNA = 22 -PF_IRDA = 23 -PF_PPPOX = 24 -PF_WANPIPE = 25 -PF_BLUETOOTH = 31 -PF_MAX = 32 -AF_UNSPEC = PF_UNSPEC -AF_LOCAL = PF_LOCAL -AF_UNIX = PF_UNIX -AF_FILE = PF_FILE -AF_INET = PF_INET -AF_AX25 = PF_AX25 -AF_IPX = PF_IPX -AF_APPLETALK = PF_APPLETALK -AF_NETROM = PF_NETROM -AF_BRIDGE = PF_BRIDGE -AF_ATMPVC = PF_ATMPVC -AF_X25 = PF_X25 -AF_INET6 = PF_INET6 -AF_ROSE = PF_ROSE -AF_DECnet = PF_DECnet -AF_NETBEUI = PF_NETBEUI -AF_SECURITY = PF_SECURITY -AF_KEY = PF_KEY -AF_NETLINK = PF_NETLINK -AF_ROUTE = PF_ROUTE -AF_PACKET = PF_PACKET -AF_ASH = PF_ASH -AF_ECONET = PF_ECONET -AF_ATMSVC = PF_ATMSVC -AF_SNA = PF_SNA -AF_IRDA = PF_IRDA -AF_PPPOX = PF_PPPOX -AF_WANPIPE = PF_WANPIPE -AF_BLUETOOTH = PF_BLUETOOTH -AF_MAX = PF_MAX -SOL_RAW = 255 -SOL_DECNET = 261 -SOL_X25 = 262 -SOL_PACKET = 263 -SOL_ATM = 264 -SOL_AAL = 265 -SOL_IRDA = 266 -SOMAXCONN = 128 - -# Included from bits/sockaddr.h -_BITS_SOCKADDR_H = 1 -def __SOCKADDR_COMMON(sa_prefix): return \ - -_SS_SIZE = 128 -def CMSG_FIRSTHDR(mhdr): return \ - - -# Included from asm/socket.h - -# Included from asm/sockios.h -FIOSETOWN = 0x8901 -SIOCSPGRP = 0x8902 -FIOGETOWN = 0x8903 -SIOCGPGRP = 0x8904 -SIOCATMARK = 0x8905 -SIOCGSTAMP = 0x8906 -SOL_SOCKET = 1 -SO_DEBUG = 1 -SO_REUSEADDR = 2 -SO_TYPE = 3 -SO_ERROR = 4 -SO_DONTROUTE = 5 -SO_BROADCAST = 6 -SO_SNDBUF = 7 -SO_RCVBUF = 8 -SO_KEEPALIVE = 9 -SO_OOBINLINE = 10 -SO_NO_CHECK = 11 -SO_PRIORITY = 12 -SO_LINGER = 13 -SO_BSDCOMPAT = 14 -SO_PASSCRED = 16 -SO_PEERCRED = 17 -SO_RCVLOWAT = 18 -SO_SNDLOWAT = 19 -SO_RCVTIMEO = 20 -SO_SNDTIMEO = 21 -SO_SECURITY_AUTHENTICATION = 22 -SO_SECURITY_ENCRYPTION_TRANSPORT = 23 -SO_SECURITY_ENCRYPTION_NETWORK = 24 -SO_BINDTODEVICE = 25 -SO_ATTACH_FILTER = 26 -SO_DETACH_FILTER = 27 -SO_PEERNAME = 28 -SO_TIMESTAMP = 29 -SCM_TIMESTAMP = SO_TIMESTAMP -SO_ACCEPTCONN = 30 -SOCK_STREAM = 1 -SOCK_DGRAM = 2 -SOCK_RAW = 3 -SOCK_RDM = 4 -SOCK_SEQPACKET = 5 -SOCK_PACKET = 10 -SOCK_MAX = (SOCK_PACKET+1) - -# Included from bits/in.h -IP_TOS = 1 -IP_TTL = 2 -IP_HDRINCL = 3 -IP_OPTIONS = 4 -IP_ROUTER_ALERT = 5 -IP_RECVOPTS = 6 -IP_RETOPTS = 7 -IP_PKTINFO = 8 -IP_PKTOPTIONS = 9 -IP_PMTUDISC = 10 -IP_MTU_DISCOVER = 10 -IP_RECVERR = 11 -IP_RECVTTL = 12 -IP_RECVTOS = 13 -IP_MULTICAST_IF = 32 -IP_MULTICAST_TTL = 33 -IP_MULTICAST_LOOP = 34 -IP_ADD_MEMBERSHIP = 35 -IP_DROP_MEMBERSHIP = 36 -IP_RECVRETOPTS = IP_RETOPTS -IP_PMTUDISC_DONT = 0 -IP_PMTUDISC_WANT = 1 -IP_PMTUDISC_DO = 2 -SOL_IP = 0 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IPV6_ADDRFORM = 1 -IPV6_PKTINFO = 2 -IPV6_HOPOPTS = 3 -IPV6_DSTOPTS = 4 -IPV6_RTHDR = 5 -IPV6_PKTOPTIONS = 6 -IPV6_CHECKSUM = 7 -IPV6_HOPLIMIT = 8 -IPV6_NEXTHOP = 9 -IPV6_AUTHHDR = 10 -IPV6_UNICAST_HOPS = 16 -IPV6_MULTICAST_IF = 17 -IPV6_MULTICAST_HOPS = 18 -IPV6_MULTICAST_LOOP = 19 -IPV6_JOIN_GROUP = 20 -IPV6_LEAVE_GROUP = 21 -IPV6_ROUTER_ALERT = 22 -IPV6_MTU_DISCOVER = 23 -IPV6_MTU = 24 -IPV6_RECVERR = 25 -IPV6_RXHOPOPTS = IPV6_HOPOPTS -IPV6_RXDSTOPTS = IPV6_DSTOPTS -IPV6_ADD_MEMBERSHIP = IPV6_JOIN_GROUP -IPV6_DROP_MEMBERSHIP = IPV6_LEAVE_GROUP -IPV6_PMTUDISC_DONT = 0 -IPV6_PMTUDISC_WANT = 1 -IPV6_PMTUDISC_DO = 2 -SOL_IPV6 = 41 -SOL_ICMPV6 = 58 -IPV6_RTHDR_LOOSE = 0 -IPV6_RTHDR_STRICT = 1 -IPV6_RTHDR_TYPE_0 = 0 - -# Included from endian.h -_ENDIAN_H = 1 -__LITTLE_ENDIAN = 1234 -__BIG_ENDIAN = 4321 -__PDP_ENDIAN = 3412 - -# Included from bits/endian.h -__BYTE_ORDER = __LITTLE_ENDIAN -__FLOAT_WORD_ORDER = __BYTE_ORDER -LITTLE_ENDIAN = __LITTLE_ENDIAN -BIG_ENDIAN = __BIG_ENDIAN -PDP_ENDIAN = __PDP_ENDIAN -BYTE_ORDER = __BYTE_ORDER - -# Included from bits/byteswap.h -_BITS_BYTESWAP_H = 1 -def __bswap_constant_16(x): return \ - -def __bswap_16(x): return \ - -def __bswap_16(x): return __bswap_constant_16 (x) - -def __bswap_constant_32(x): return \ - -def __bswap_32(x): return \ - -def __bswap_32(x): return \ - -def __bswap_32(x): return __bswap_constant_32 (x) - -def __bswap_constant_64(x): return \ - -def __bswap_64(x): return \ - -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -def ntohl(x): return __bswap_32 (x) - -def ntohs(x): return __bswap_16 (x) - -def htonl(x): return __bswap_32 (x) - -def htons(x): return __bswap_16 (x) - -def IN6_IS_ADDR_UNSPECIFIED(a): return \ - -def IN6_IS_ADDR_LOOPBACK(a): return \ - -def IN6_IS_ADDR_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_SITELOCAL(a): return \ - -def IN6_IS_ADDR_V4MAPPED(a): return \ - -def IN6_IS_ADDR_V4COMPAT(a): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return diff --git a/Lib/plat-linux/TYPES.py b/Lib/plat-linux/TYPES.py deleted file mode 100644 --- a/Lib/plat-linux/TYPES.py +++ /dev/null @@ -1,170 +0,0 @@ -# Generated by h2py from /usr/include/sys/types.h -_SYS_TYPES_H = 1 - -# Included from features.h -_FEATURES_H = 1 -__USE_ANSI = 1 -__FAVOR_BSD = 1 -_ISOC99_SOURCE = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 199506 -_XOPEN_SOURCE = 600 -_XOPEN_SOURCE_EXTENDED = 1 -_LARGEFILE64_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -_BSD_SOURCE = 1 -_SVID_SOURCE = 1 -__USE_ISOC99 = 1 -_POSIX_SOURCE = 1 -_POSIX_C_SOURCE = 2 -_POSIX_C_SOURCE = 199506 -__USE_POSIX = 1 -__USE_POSIX2 = 1 -__USE_POSIX199309 = 1 -__USE_POSIX199506 = 1 -__USE_XOPEN = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_UNIX98 = 1 -_LARGEFILE_SOURCE = 1 -__USE_XOPEN2K = 1 -__USE_ISOC99 = 1 -__USE_XOPEN_EXTENDED = 1 -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_FILE_OFFSET64 = 1 -__USE_MISC = 1 -__USE_BSD = 1 -__USE_SVID = 1 -__USE_GNU = 1 -__USE_REENTRANT = 1 -__STDC_IEC_559__ = 1 -__STDC_IEC_559_COMPLEX__ = 1 -__STDC_ISO_10646__ = 200009 -__GNU_LIBRARY__ = 6 -__GLIBC__ = 2 -__GLIBC_MINOR__ = 2 - -# Included from sys/cdefs.h -_SYS_CDEFS_H = 1 -def __PMT(args): return args - -def __P(args): return args - -def __PMT(args): return args - -def __STRING(x): return #x - -__flexarr = [] -__flexarr = [0] -__flexarr = [] -__flexarr = [1] -def __ASMNAME(cname): return __ASMNAME2 (__USER_LABEL_PREFIX__, cname) - -def __attribute__(xyz): return - -def __attribute_format_arg__(x): return __attribute__ ((__format_arg__ (x))) - -def __attribute_format_arg__(x): return - -__USE_LARGEFILE = 1 -__USE_LARGEFILE64 = 1 -__USE_EXTERN_INLINES = 1 - -# Included from gnu/stubs.h - -# Included from bits/types.h -_BITS_TYPES_H = 1 -__FD_SETSIZE = 1024 - -# Included from bits/pthreadtypes.h -_BITS_PTHREADTYPES_H = 1 - -# Included from bits/sched.h -SCHED_OTHER = 0 -SCHED_FIFO = 1 -SCHED_RR = 2 -CSIGNAL = 0x000000ff -CLONE_VM = 0x00000100 -CLONE_FS = 0x00000200 -CLONE_FILES = 0x00000400 -CLONE_SIGHAND = 0x00000800 -CLONE_PID = 0x00001000 -CLONE_PTRACE = 0x00002000 -CLONE_VFORK = 0x00004000 -__defined_schedparam = 1 - -# Included from time.h -_TIME_H = 1 - -# Included from bits/time.h -_BITS_TIME_H = 1 -CLOCKS_PER_SEC = 1000000 -CLOCK_REALTIME = 0 -CLOCK_PROCESS_CPUTIME_ID = 2 -CLOCK_THREAD_CPUTIME_ID = 3 -TIMER_ABSTIME = 1 -_STRUCT_TIMEVAL = 1 -CLK_TCK = CLOCKS_PER_SEC -__clock_t_defined = 1 -__time_t_defined = 1 -__clockid_t_defined = 1 -__timer_t_defined = 1 -__timespec_defined = 1 -def __isleap(year): return \ - -__BIT_TYPES_DEFINED__ = 1 - -# Included from endian.h -_ENDIAN_H = 1 -__LITTLE_ENDIAN = 1234 -__BIG_ENDIAN = 4321 -__PDP_ENDIAN = 3412 - -# Included from bits/endian.h -__BYTE_ORDER = __LITTLE_ENDIAN -__FLOAT_WORD_ORDER = __BYTE_ORDER -LITTLE_ENDIAN = __LITTLE_ENDIAN -BIG_ENDIAN = __BIG_ENDIAN -PDP_ENDIAN = __PDP_ENDIAN -BYTE_ORDER = __BYTE_ORDER - -# Included from sys/select.h -_SYS_SELECT_H = 1 - -# Included from bits/select.h -def __FD_ZERO(fdsp): return \ - -def __FD_ZERO(set): return \ - - -# Included from bits/sigset.h -_SIGSET_H_types = 1 -_SIGSET_H_fns = 1 -def __sigmask(sig): return \ - -def __sigemptyset(set): return \ - -def __sigfillset(set): return \ - -def __sigisemptyset(set): return \ - -def __FDELT(d): return ((d) / __NFDBITS) - -FD_SETSIZE = __FD_SETSIZE -def FD_ZERO(fdsetp): return __FD_ZERO (fdsetp) - - -# Included from sys/sysmacros.h -_SYS_SYSMACROS_H = 1 -def major(dev): return ((int)(((dev) >> 8) & 0xff)) - -def minor(dev): return ((int)((dev) & 0xff)) - -def major(dev): return (((dev).__val[1] >> 8) & 0xff) - -def minor(dev): return ((dev).__val[1] & 0xff) - -def major(dev): return (((dev).__val[0] >> 8) & 0xff) - -def minor(dev): return ((dev).__val[0] & 0xff) diff --git a/Lib/plat-linux/regen b/Lib/plat-linux/regen deleted file mode 100755 --- a/Lib/plat-linux/regen +++ /dev/null @@ -1,33 +0,0 @@ -#! /bin/sh -case `uname` in -Linux*|GNU*) ;; -*) echo Probably not on a Linux system 1>&2 - exit 1;; -esac -if [ -z "$CC" ]; then - echo >&2 "$(basename $0): CC is not set" - exit 1 -fi -headers="sys/types.h netinet/in.h dlfcn.h" -incdirs="$(echo $($CC -v -E - < /dev/null 2>&1|awk '/^#include/, /^End of search/' | grep '^ '))" -if [ -z "$incdirs" ]; then - incdirs="/usr/include" -fi -for h in $headers; do - absh= - for d in $incdirs; do - if [ -f "$d/$h" ]; then - absh="$d/$h" - break - fi - done - if [ -n "$absh" ]; then - absheaders="$absheaders $absh" - else - echo >&2 "$(basename $0): header $h not found" - exit 1 - fi -done - -set -x -${H2PY:-h2py} -i '(u_long)' $absheaders diff --git a/Lib/plat-netbsd1/IN.py b/Lib/plat-netbsd1/IN.py deleted file mode 100644 --- a/Lib/plat-netbsd1/IN.py +++ /dev/null @@ -1,56 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h -IPPROTO_IP = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPIP = 4 -IPPROTO_TCP = 6 -IPPROTO_EGP = 8 -IPPROTO_PUP = 12 -IPPROTO_UDP = 17 -IPPROTO_IDP = 22 -IPPROTO_TP = 29 -IPPROTO_EON = 80 -IPPROTO_ENCAP = 98 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -def __IPADDR(x): return ((u_int32_t)(x)) - -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_MAX = 128 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_MAX = 65536 -IN_CLASSC_NSHIFT = 8 -IN_CLASSD_NSHIFT = 28 -def IN_MULTICAST(i): return IN_CLASSD(i) - -IN_LOOPBACKNET = 127 -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_TTL = 10 -IP_MULTICAST_LOOP = 11 -IP_ADD_MEMBERSHIP = 12 -IP_DROP_MEMBERSHIP = 13 -IP_RECVIF = 20 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -IPPROTO_MAXID = (IPPROTO_IDP + 1) -IPCTL_FORWARDING = 1 -IPCTL_SENDREDIRECTS = 2 -IPCTL_DEFTTL = 3 -IPCTL_DEFMTU = 4 -IPCTL_FORWSRCRT = 5 -IPCTL_DIRECTEDBCAST = 6 -IPCTL_ALLOWSRCRT = 7 -IPCTL_MAXID = 8 -def in_nullhost(x): return ((x).s_addr == INADDR_ANY) diff --git a/Lib/plat-netbsd1/regen b/Lib/plat-netbsd1/regen deleted file mode 100755 --- a/Lib/plat-netbsd1/regen +++ /dev/null @@ -1,3 +0,0 @@ -#! /bin/sh -set -v -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h diff --git a/Lib/plat-next3/regen b/Lib/plat-next3/regen deleted file mode 100755 --- a/Lib/plat-next3/regen +++ /dev/null @@ -1,6 +0,0 @@ -#! /bin/sh -set -v -INCLUDE="/NextDeveloper/Headers;/NextDeveloper/Headers/ansi;/NextDeveloper/Headers/bsd" -export INCLUDE - -python ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/bsd/netinet/in.h diff --git a/Lib/plat-sunos5/CDIO.py b/Lib/plat-sunos5/CDIO.py deleted file mode 100644 --- a/Lib/plat-sunos5/CDIO.py +++ /dev/null @@ -1,73 +0,0 @@ -# Generated by h2py from /usr/include/sys/cdio.h -CDROM_LBA = 0x01 -CDROM_MSF = 0x02 -CDROM_DATA_TRACK = 0x04 -CDROM_LEADOUT = 0xAA -CDROM_AUDIO_INVALID = 0x00 -CDROM_AUDIO_PLAY = 0x11 -CDROM_AUDIO_PAUSED = 0x12 -CDROM_AUDIO_COMPLETED = 0x13 -CDROM_AUDIO_ERROR = 0x14 -CDROM_AUDIO_NO_STATUS = 0x15 -CDROM_DA_NO_SUBCODE = 0x00 -CDROM_DA_SUBQ = 0x01 -CDROM_DA_ALL_SUBCODE = 0x02 -CDROM_DA_SUBCODE_ONLY = 0x03 -CDROM_XA_DATA = 0x00 -CDROM_XA_SECTOR_DATA = 0x01 -CDROM_XA_DATA_W_ERROR = 0x02 -CDROM_BLK_512 = 512 -CDROM_BLK_1024 = 1024 -CDROM_BLK_2048 = 2048 -CDROM_BLK_2056 = 2056 -CDROM_BLK_2336 = 2336 -CDROM_BLK_2340 = 2340 -CDROM_BLK_2352 = 2352 -CDROM_BLK_2368 = 2368 -CDROM_BLK_2448 = 2448 -CDROM_BLK_2646 = 2646 -CDROM_BLK_2647 = 2647 -CDROM_BLK_SUBCODE = 96 -CDROM_NORMAL_SPEED = 0x00 -CDROM_DOUBLE_SPEED = 0x01 -CDROM_QUAD_SPEED = 0x03 -CDROM_TWELVE_SPEED = 0x0C -CDROM_MAXIMUM_SPEED = 0xff -CDIOC = (0x04 << 8) -CDROMPAUSE = (CDIOC|151) -CDROMRESUME = (CDIOC|152) -CDROMPLAYMSF = (CDIOC|153) -CDROMPLAYTRKIND = (CDIOC|154) -CDROMREADTOCHDR = (CDIOC|155) -CDROMREADTOCENTRY = (CDIOC|156) -CDROMSTOP = (CDIOC|157) -CDROMSTART = (CDIOC|158) -CDROMEJECT = (CDIOC|159) -CDROMVOLCTRL = (CDIOC|160) -CDROMSUBCHNL = (CDIOC|161) -CDROMREADMODE2 = (CDIOC|162) -CDROMREADMODE1 = (CDIOC|163) -CDROMREADOFFSET = (CDIOC|164) -CDROMGBLKMODE = (CDIOC|165) -CDROMSBLKMODE = (CDIOC|166) -CDROMCDDA = (CDIOC|167) -CDROMCDXA = (CDIOC|168) -CDROMSUBCODE = (CDIOC|169) -CDROMGDRVSPEED = (CDIOC|170) -CDROMSDRVSPEED = (CDIOC|171) -SCMD_READ_TOC = 0x43 -SCMD_PLAYAUDIO_MSF = 0x47 -SCMD_PLAYAUDIO_TI = 0x48 -SCMD_PAUSE_RESUME = 0x4B -SCMD_READ_SUBCHANNEL = 0x42 -SCMD_PLAYAUDIO10 = 0x45 -SCMD_PLAYTRACK_REL10 = 0x49 -SCMD_READ_HEADER = 0x44 -SCMD_PLAYAUDIO12 = 0xA5 -SCMD_PLAYTRACK_REL12 = 0xA9 -SCMD_CD_PLAYBACK_CONTROL = 0xC9 -SCMD_CD_PLAYBACK_STATUS = 0xC4 -SCMD_READ_CDDA = 0xD8 -SCMD_READ_CDXA = 0xDB -SCMD_READ_ALL_SUBCODES = 0xDF -CDROM_MODE2_SIZE = 2336 diff --git a/Lib/plat-sunos5/DLFCN.py b/Lib/plat-sunos5/DLFCN.py deleted file mode 100644 --- a/Lib/plat-sunos5/DLFCN.py +++ /dev/null @@ -1,27 +0,0 @@ -# Generated by h2py from /usr/include/dlfcn.h -from TYPES import * -RTLD_LAZY = 0x00001 -RTLD_NOW = 0x00002 -RTLD_NOLOAD = 0x00004 -RTLD_GLOBAL = 0x00100 -RTLD_LOCAL = 0x00000 -RTLD_PARENT = 0x00200 -RTLD_GROUP = 0x00400 -RTLD_WORLD = 0x00800 -RTLD_NODELETE = 0x01000 -RTLD_CONFGEN = 0x10000 -RTLD_REL_RELATIVE = 0x00001 -RTLD_REL_EXEC = 0x00002 -RTLD_REL_DEPENDS = 0x00004 -RTLD_REL_PRELOAD = 0x00008 -RTLD_REL_SELF = 0x00010 -RTLD_REL_WEAK = 0x00020 -RTLD_REL_ALL = 0x00fff -RTLD_MEMORY = 0x01000 -RTLD_STRIP = 0x02000 -RTLD_NOHEAP = 0x04000 -RTLD_CONFSET = 0x10000 -RTLD_DI_LMID = 1 -RTLD_DI_LINKMAP = 2 -RTLD_DI_CONFIGADDR = 3 -RTLD_DI_MAX = 3 diff --git a/Lib/plat-sunos5/IN.py b/Lib/plat-sunos5/IN.py deleted file mode 100644 --- a/Lib/plat-sunos5/IN.py +++ /dev/null @@ -1,1421 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from sys/feature_tests.h - -# Included from sys/isa_defs.h -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 8 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_ALIGNMENT_REQUIRED = 1 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 4 -_DOUBLE_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 4 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 4 -_ALIGNMENT_REQUIRED = 0 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_ALIGNMENT_REQUIRED = 1 -_LONG_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 8 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 8 -_LONG_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_POSIX_C_SOURCE = 1 -_LARGEFILE64_SOURCE = 1 -_LARGEFILE_SOURCE = 1 -_FILE_OFFSET_BITS = 64 -_FILE_OFFSET_BITS = 32 -_POSIX_C_SOURCE = 199506 -_POSIX_PTHREAD_SEMANTICS = 1 -_XOPEN_VERSION = 500 -_XOPEN_VERSION = 4 -_XOPEN_VERSION = 3 -from TYPES import * - -# Included from sys/stream.h - -# Included from sys/vnode.h -from TYPES import * - -# Included from sys/t_lock.h - -# Included from sys/machlock.h -from TYPES import * -LOCK_HELD_VALUE = 0xff -def SPIN_LOCK(pl): return ((pl) > ipltospl(LOCK_LEVEL)) - -def LOCK_SAMPLE_INTERVAL(i): return (((i) & 0xff) == 0) - -CLOCK_LEVEL = 10 -LOCK_LEVEL = 10 -DISP_LEVEL = (LOCK_LEVEL + 1) -PTR24_LSB = 5 -PTR24_MSB = (PTR24_LSB + 24) -PTR24_ALIGN = 32 -PTR24_BASE = 0xe0000000 - -# Included from sys/param.h -from TYPES import * -_POSIX_VDISABLE = 0 -MAX_INPUT = 512 -MAX_CANON = 256 -UID_NOBODY = 60001 -GID_NOBODY = UID_NOBODY -UID_NOACCESS = 60002 -MAX_TASKID = 999999 -MAX_MAXPID = 999999 -DEFAULT_MAXPID = 999999 -DEFAULT_JUMPPID = 100000 -DEFAULT_MAXPID = 30000 -DEFAULT_JUMPPID = 0 -MAXUID = 2147483647 -MAXPROJID = MAXUID -MAXLINK = 32767 -NMOUNT = 40 -CANBSIZ = 256 -NOFILE = 20 -NGROUPS_UMIN = 0 -NGROUPS_UMAX = 32 -NGROUPS_MAX_DEFAULT = 16 -NZERO = 20 -NULL = 0 -NULL = 0 -CMASK = 0o22 -CDLIMIT = (1<<11) -NBPS = 0x20000 -NBPSCTR = 512 -UBSIZE = 512 -SCTRSHFT = 9 -SYSNAME = 9 -PREMOTE = 39 -MAXPATHLEN = 1024 -MAXSYMLINKS = 20 -MAXNAMELEN = 256 -NADDR = 13 -PIPE_BUF = 5120 -PIPE_MAX = 5120 -NBBY = 8 -MAXBSIZE = 8192 -DEV_BSIZE = 512 -DEV_BSHIFT = 9 -MAXFRAG = 8 -MAXOFF32_T = 0x7fffffff -MAXOFF_T = 0x7fffffffffffffff -MAXOFFSET_T = 0x7fffffffffffffff -MAXOFF_T = 0x7fffffff -MAXOFFSET_T = 0x7fffffff -def btodb(bytes): return \ - -def dbtob(db): return \ - -def lbtodb(bytes): return \ - -def ldbtob(db): return \ - -NCARGS32 = 0x100000 -NCARGS64 = 0x200000 -NCARGS = NCARGS64 -NCARGS = NCARGS32 -FSHIFT = 8 -FSCALE = (1<> MMU_PAGESHIFT) - -def mmu_btopr(x): return ((((x) + MMU_PAGEOFFSET) >> MMU_PAGESHIFT)) - -def mmu_ptod(x): return ((x) << (MMU_PAGESHIFT - DEV_BSHIFT)) - -def ptod(x): return ((x) << (PAGESHIFT - DEV_BSHIFT)) - -def ptob(x): return ((x) << PAGESHIFT) - -def btop(x): return (((x) >> PAGESHIFT)) - -def btopr(x): return ((((x) + PAGEOFFSET) >> PAGESHIFT)) - -def dtop(DD): return (((DD) + NDPP - 1) >> (PAGESHIFT - DEV_BSHIFT)) - -def dtopt(DD): return ((DD) >> (PAGESHIFT - DEV_BSHIFT)) - -_AIO_LISTIO_MAX = (4096) -_AIO_MAX = (-1) -_MQ_OPEN_MAX = (32) -_MQ_PRIO_MAX = (32) -_SEM_NSEMS_MAX = INT_MAX -_SEM_VALUE_MAX = INT_MAX - -# Included from sys/unistd.h -_CS_PATH = 65 -_CS_LFS_CFLAGS = 68 -_CS_LFS_LDFLAGS = 69 -_CS_LFS_LIBS = 70 -_CS_LFS_LINTFLAGS = 71 -_CS_LFS64_CFLAGS = 72 -_CS_LFS64_LDFLAGS = 73 -_CS_LFS64_LIBS = 74 -_CS_LFS64_LINTFLAGS = 75 -_CS_XBS5_ILP32_OFF32_CFLAGS = 700 -_CS_XBS5_ILP32_OFF32_LDFLAGS = 701 -_CS_XBS5_ILP32_OFF32_LIBS = 702 -_CS_XBS5_ILP32_OFF32_LINTFLAGS = 703 -_CS_XBS5_ILP32_OFFBIG_CFLAGS = 705 -_CS_XBS5_ILP32_OFFBIG_LDFLAGS = 706 -_CS_XBS5_ILP32_OFFBIG_LIBS = 707 -_CS_XBS5_ILP32_OFFBIG_LINTFLAGS = 708 -_CS_XBS5_LP64_OFF64_CFLAGS = 709 -_CS_XBS5_LP64_OFF64_LDFLAGS = 710 -_CS_XBS5_LP64_OFF64_LIBS = 711 -_CS_XBS5_LP64_OFF64_LINTFLAGS = 712 -_CS_XBS5_LPBIG_OFFBIG_CFLAGS = 713 -_CS_XBS5_LPBIG_OFFBIG_LDFLAGS = 714 -_CS_XBS5_LPBIG_OFFBIG_LIBS = 715 -_CS_XBS5_LPBIG_OFFBIG_LINTFLAGS = 716 -_SC_ARG_MAX = 1 -_SC_CHILD_MAX = 2 -_SC_CLK_TCK = 3 -_SC_NGROUPS_MAX = 4 -_SC_OPEN_MAX = 5 -_SC_JOB_CONTROL = 6 -_SC_SAVED_IDS = 7 -_SC_VERSION = 8 -_SC_PASS_MAX = 9 -_SC_LOGNAME_MAX = 10 -_SC_PAGESIZE = 11 -_SC_XOPEN_VERSION = 12 -_SC_NPROCESSORS_CONF = 14 -_SC_NPROCESSORS_ONLN = 15 -_SC_STREAM_MAX = 16 -_SC_TZNAME_MAX = 17 -_SC_AIO_LISTIO_MAX = 18 -_SC_AIO_MAX = 19 -_SC_AIO_PRIO_DELTA_MAX = 20 -_SC_ASYNCHRONOUS_IO = 21 -_SC_DELAYTIMER_MAX = 22 -_SC_FSYNC = 23 -_SC_MAPPED_FILES = 24 -_SC_MEMLOCK = 25 -_SC_MEMLOCK_RANGE = 26 -_SC_MEMORY_PROTECTION = 27 -_SC_MESSAGE_PASSING = 28 -_SC_MQ_OPEN_MAX = 29 -_SC_MQ_PRIO_MAX = 30 -_SC_PRIORITIZED_IO = 31 -_SC_PRIORITY_SCHEDULING = 32 -_SC_REALTIME_SIGNALS = 33 -_SC_RTSIG_MAX = 34 -_SC_SEMAPHORES = 35 -_SC_SEM_NSEMS_MAX = 36 -_SC_SEM_VALUE_MAX = 37 -_SC_SHARED_MEMORY_OBJECTS = 38 -_SC_SIGQUEUE_MAX = 39 -_SC_SIGRT_MIN = 40 -_SC_SIGRT_MAX = 41 -_SC_SYNCHRONIZED_IO = 42 -_SC_TIMERS = 43 -_SC_TIMER_MAX = 44 -_SC_2_C_BIND = 45 -_SC_2_C_DEV = 46 -_SC_2_C_VERSION = 47 -_SC_2_FORT_DEV = 48 -_SC_2_FORT_RUN = 49 -_SC_2_LOCALEDEF = 50 -_SC_2_SW_DEV = 51 -_SC_2_UPE = 52 -_SC_2_VERSION = 53 -_SC_BC_BASE_MAX = 54 -_SC_BC_DIM_MAX = 55 -_SC_BC_SCALE_MAX = 56 -_SC_BC_STRING_MAX = 57 -_SC_COLL_WEIGHTS_MAX = 58 -_SC_EXPR_NEST_MAX = 59 -_SC_LINE_MAX = 60 -_SC_RE_DUP_MAX = 61 -_SC_XOPEN_CRYPT = 62 -_SC_XOPEN_ENH_I18N = 63 -_SC_XOPEN_SHM = 64 -_SC_2_CHAR_TERM = 66 -_SC_XOPEN_XCU_VERSION = 67 -_SC_ATEXIT_MAX = 76 -_SC_IOV_MAX = 77 -_SC_XOPEN_UNIX = 78 -_SC_PAGE_SIZE = _SC_PAGESIZE -_SC_T_IOV_MAX = 79 -_SC_PHYS_PAGES = 500 -_SC_AVPHYS_PAGES = 501 -_SC_COHER_BLKSZ = 503 -_SC_SPLIT_CACHE = 504 -_SC_ICACHE_SZ = 505 -_SC_DCACHE_SZ = 506 -_SC_ICACHE_LINESZ = 507 -_SC_DCACHE_LINESZ = 508 -_SC_ICACHE_BLKSZ = 509 -_SC_DCACHE_BLKSZ = 510 -_SC_DCACHE_TBLKSZ = 511 -_SC_ICACHE_ASSOC = 512 -_SC_DCACHE_ASSOC = 513 -_SC_MAXPID = 514 -_SC_STACK_PROT = 515 -_SC_THREAD_DESTRUCTOR_ITERATIONS = 568 -_SC_GETGR_R_SIZE_MAX = 569 -_SC_GETPW_R_SIZE_MAX = 570 -_SC_LOGIN_NAME_MAX = 571 -_SC_THREAD_KEYS_MAX = 572 -_SC_THREAD_STACK_MIN = 573 -_SC_THREAD_THREADS_MAX = 574 -_SC_TTY_NAME_MAX = 575 -_SC_THREADS = 576 -_SC_THREAD_ATTR_STACKADDR = 577 -_SC_THREAD_ATTR_STACKSIZE = 578 -_SC_THREAD_PRIORITY_SCHEDULING = 579 -_SC_THREAD_PRIO_INHERIT = 580 -_SC_THREAD_PRIO_PROTECT = 581 -_SC_THREAD_PROCESS_SHARED = 582 -_SC_THREAD_SAFE_FUNCTIONS = 583 -_SC_XOPEN_LEGACY = 717 -_SC_XOPEN_REALTIME = 718 -_SC_XOPEN_REALTIME_THREADS = 719 -_SC_XBS5_ILP32_OFF32 = 720 -_SC_XBS5_ILP32_OFFBIG = 721 -_SC_XBS5_LP64_OFF64 = 722 -_SC_XBS5_LPBIG_OFFBIG = 723 -_PC_LINK_MAX = 1 -_PC_MAX_CANON = 2 -_PC_MAX_INPUT = 3 -_PC_NAME_MAX = 4 -_PC_PATH_MAX = 5 -_PC_PIPE_BUF = 6 -_PC_NO_TRUNC = 7 -_PC_VDISABLE = 8 -_PC_CHOWN_RESTRICTED = 9 -_PC_ASYNC_IO = 10 -_PC_PRIO_IO = 11 -_PC_SYNC_IO = 12 -_PC_FILESIZEBITS = 67 -_PC_LAST = 67 -_POSIX_VERSION = 199506 -_POSIX2_VERSION = 199209 -_POSIX2_C_VERSION = 199209 -_XOPEN_XCU_VERSION = 4 -_XOPEN_REALTIME = 1 -_XOPEN_ENH_I18N = 1 -_XOPEN_SHM = 1 -_POSIX2_C_BIND = 1 -_POSIX2_CHAR_TERM = 1 -_POSIX2_LOCALEDEF = 1 -_POSIX2_C_DEV = 1 -_POSIX2_SW_DEV = 1 -_POSIX2_UPE = 1 - -# Included from sys/mutex.h -from TYPES import * -def MUTEX_HELD(x): return (mutex_owned(x)) - - -# Included from sys/rwlock.h -from TYPES import * -def RW_READ_HELD(x): return (rw_read_held((x))) - -def RW_WRITE_HELD(x): return (rw_write_held((x))) - -def RW_LOCK_HELD(x): return (rw_lock_held((x))) - -def RW_ISWRITER(x): return (rw_iswriter(x)) - - -# Included from sys/semaphore.h - -# Included from sys/thread.h -from TYPES import * - -# Included from sys/klwp.h -from TYPES import * - -# Included from sys/condvar.h -from TYPES import * - -# Included from sys/time.h - -# Included from sys/types32.h - -# Included from sys/int_types.h -TIME32_MAX = INT32_MAX -TIME32_MIN = INT32_MIN -def TIMEVAL_OVERFLOW(tv): return \ - -from TYPES import * -DST_NONE = 0 -DST_USA = 1 -DST_AUST = 2 -DST_WET = 3 -DST_MET = 4 -DST_EET = 5 -DST_CAN = 6 -DST_GB = 7 -DST_RUM = 8 -DST_TUR = 9 -DST_AUSTALT = 10 -ITIMER_REAL = 0 -ITIMER_VIRTUAL = 1 -ITIMER_PROF = 2 -ITIMER_REALPROF = 3 -def ITIMERVAL_OVERFLOW(itv): return \ - -SEC = 1 -MILLISEC = 1000 -MICROSEC = 1000000 -NANOSEC = 1000000000 - -# Included from sys/time_impl.h -def TIMESPEC_OVERFLOW(ts): return \ - -def ITIMERSPEC_OVERFLOW(it): return \ - -__CLOCK_REALTIME0 = 0 -CLOCK_VIRTUAL = 1 -CLOCK_PROF = 2 -__CLOCK_REALTIME3 = 3 -CLOCK_HIGHRES = 4 -CLOCK_MAX = 5 -CLOCK_REALTIME = __CLOCK_REALTIME3 -CLOCK_REALTIME = __CLOCK_REALTIME0 -TIMER_RELTIME = 0x0 -TIMER_ABSTIME = 0x1 -def TICK_TO_SEC(tick): return ((tick) / hz) - -def SEC_TO_TICK(sec): return ((sec) * hz) - -def TICK_TO_MSEC(tick): return \ - -def MSEC_TO_TICK(msec): return \ - -def MSEC_TO_TICK_ROUNDUP(msec): return \ - -def TICK_TO_USEC(tick): return ((tick) * usec_per_tick) - -def USEC_TO_TICK(usec): return ((usec) / usec_per_tick) - -def USEC_TO_TICK_ROUNDUP(usec): return \ - -def TICK_TO_NSEC(tick): return ((tick) * nsec_per_tick) - -def NSEC_TO_TICK(nsec): return ((nsec) / nsec_per_tick) - -def NSEC_TO_TICK_ROUNDUP(nsec): return \ - -def TIMEVAL_TO_TICK(tvp): return \ - -def TIMESTRUC_TO_TICK(tsp): return \ - - -# Included from time.h -from TYPES import * - -# Included from iso/time_iso.h -NULL = 0 -NULL = 0 -CLOCKS_PER_SEC = 1000000 - -# Included from sys/select.h -FD_SETSIZE = 65536 -FD_SETSIZE = 1024 -_NBBY = 8 -NBBY = _NBBY -def FD_ZERO(p): return bzero((p), sizeof (*(p))) - - -# Included from sys/signal.h - -# Included from sys/iso/signal_iso.h -SIGHUP = 1 -SIGINT = 2 -SIGQUIT = 3 -SIGILL = 4 -SIGTRAP = 5 -SIGIOT = 6 -SIGABRT = 6 -SIGEMT = 7 -SIGFPE = 8 -SIGKILL = 9 -SIGBUS = 10 -SIGSEGV = 11 -SIGSYS = 12 -SIGPIPE = 13 -SIGALRM = 14 -SIGTERM = 15 -SIGUSR1 = 16 -SIGUSR2 = 17 -SIGCLD = 18 -SIGCHLD = 18 -SIGPWR = 19 -SIGWINCH = 20 -SIGURG = 21 -SIGPOLL = 22 -SIGIO = SIGPOLL -SIGSTOP = 23 -SIGTSTP = 24 -SIGCONT = 25 -SIGTTIN = 26 -SIGTTOU = 27 -SIGVTALRM = 28 -SIGPROF = 29 -SIGXCPU = 30 -SIGXFSZ = 31 -SIGWAITING = 32 -SIGLWP = 33 -SIGFREEZE = 34 -SIGTHAW = 35 -SIGCANCEL = 36 -SIGLOST = 37 -_SIGRTMIN = 38 -_SIGRTMAX = 45 -SIG_BLOCK = 1 -SIG_UNBLOCK = 2 -SIG_SETMASK = 3 -SIGNO_MASK = 0xFF -SIGDEFER = 0x100 -SIGHOLD = 0x200 -SIGRELSE = 0x400 -SIGIGNORE = 0x800 -SIGPAUSE = 0x1000 - -# Included from sys/siginfo.h -from TYPES import * -SIGEV_NONE = 1 -SIGEV_SIGNAL = 2 -SIGEV_THREAD = 3 -SI_NOINFO = 32767 -SI_USER = 0 -SI_LWP = (-1) -SI_QUEUE = (-2) -SI_TIMER = (-3) -SI_ASYNCIO = (-4) -SI_MESGQ = (-5) - -# Included from sys/machsig.h -ILL_ILLOPC = 1 -ILL_ILLOPN = 2 -ILL_ILLADR = 3 -ILL_ILLTRP = 4 -ILL_PRVOPC = 5 -ILL_PRVREG = 6 -ILL_COPROC = 7 -ILL_BADSTK = 8 -NSIGILL = 8 -EMT_TAGOVF = 1 -EMT_CPCOVF = 2 -NSIGEMT = 2 -FPE_INTDIV = 1 -FPE_INTOVF = 2 -FPE_FLTDIV = 3 -FPE_FLTOVF = 4 -FPE_FLTUND = 5 -FPE_FLTRES = 6 -FPE_FLTINV = 7 -FPE_FLTSUB = 8 -NSIGFPE = 8 -SEGV_MAPERR = 1 -SEGV_ACCERR = 2 -NSIGSEGV = 2 -BUS_ADRALN = 1 -BUS_ADRERR = 2 -BUS_OBJERR = 3 -NSIGBUS = 3 -TRAP_BRKPT = 1 -TRAP_TRACE = 2 -TRAP_RWATCH = 3 -TRAP_WWATCH = 4 -TRAP_XWATCH = 5 -NSIGTRAP = 5 -CLD_EXITED = 1 -CLD_KILLED = 2 -CLD_DUMPED = 3 -CLD_TRAPPED = 4 -CLD_STOPPED = 5 -CLD_CONTINUED = 6 -NSIGCLD = 6 -POLL_IN = 1 -POLL_OUT = 2 -POLL_MSG = 3 -POLL_ERR = 4 -POLL_PRI = 5 -POLL_HUP = 6 -NSIGPOLL = 6 -PROF_SIG = 1 -NSIGPROF = 1 -SI_MAXSZ = 256 -SI_MAXSZ = 128 - -# Included from sys/time_std_impl.h -from TYPES import * -SI32_MAXSZ = 128 -def SI_CANQUEUE(c): return ((c) <= SI_QUEUE) - -SA_NOCLDSTOP = 0x00020000 -SA_ONSTACK = 0x00000001 -SA_RESETHAND = 0x00000002 -SA_RESTART = 0x00000004 -SA_SIGINFO = 0x00000008 -SA_NODEFER = 0x00000010 -SA_NOCLDWAIT = 0x00010000 -SA_WAITSIG = 0x00010000 -NSIG = 46 -MAXSIG = 45 -S_SIGNAL = 1 -S_SIGSET = 2 -S_SIGACTION = 3 -S_NONE = 4 -MINSIGSTKSZ = 2048 -SIGSTKSZ = 8192 -SS_ONSTACK = 0x00000001 -SS_DISABLE = 0x00000002 -SN_PROC = 1 -SN_CANCEL = 2 -SN_SEND = 3 - -# Included from sys/ucontext.h -from TYPES import * - -# Included from sys/regset.h -REG_CCR = (0) -REG_PSR = (0) -REG_PSR = (0) -REG_PC = (1) -REG_nPC = (2) -REG_Y = (3) -REG_G1 = (4) -REG_G2 = (5) -REG_G3 = (6) -REG_G4 = (7) -REG_G5 = (8) -REG_G6 = (9) -REG_G7 = (10) -REG_O0 = (11) -REG_O1 = (12) -REG_O2 = (13) -REG_O3 = (14) -REG_O4 = (15) -REG_O5 = (16) -REG_O6 = (17) -REG_O7 = (18) -REG_ASI = (19) -REG_FPRS = (20) -REG_PS = REG_PSR -REG_SP = REG_O6 -REG_R0 = REG_O0 -REG_R1 = REG_O1 -_NGREG = 21 -_NGREG = 19 -NGREG = _NGREG -_NGREG32 = 19 -_NGREG64 = 21 -SPARC_MAXREGWINDOW = 31 -MAXFPQ = 16 -XRS_ID = 0x78727300 - -# Included from v7/sys/privregs.h - -# Included from v7/sys/psr.h -PSR_CWP = 0x0000001F -PSR_ET = 0x00000020 -PSR_PS = 0x00000040 -PSR_S = 0x00000080 -PSR_PIL = 0x00000F00 -PSR_EF = 0x00001000 -PSR_EC = 0x00002000 -PSR_RSV = 0x000FC000 -PSR_ICC = 0x00F00000 -PSR_C = 0x00100000 -PSR_V = 0x00200000 -PSR_Z = 0x00400000 -PSR_N = 0x00800000 -PSR_VER = 0x0F000000 -PSR_IMPL = 0xF0000000 -PSL_ALLCC = PSR_ICC -PSL_USER = (PSR_S) -PSL_USERMASK = (PSR_ICC) -PSL_UBITS = (PSR_ICC|PSR_EF) -def USERMODE(ps): return (((ps) & PSR_PS) == 0) - - -# Included from sys/fsr.h -FSR_CEXC = 0x0000001f -FSR_AEXC = 0x000003e0 -FSR_FCC = 0x00000c00 -FSR_PR = 0x00001000 -FSR_QNE = 0x00002000 -FSR_FTT = 0x0001c000 -FSR_VER = 0x000e0000 -FSR_TEM = 0x0f800000 -FSR_RP = 0x30000000 -FSR_RD = 0xc0000000 -FSR_VER_SHIFT = 17 -FSR_FCC1 = 0x00000003 -FSR_FCC2 = 0x0000000C -FSR_FCC3 = 0x00000030 -FSR_CEXC_NX = 0x00000001 -FSR_CEXC_DZ = 0x00000002 -FSR_CEXC_UF = 0x00000004 -FSR_CEXC_OF = 0x00000008 -FSR_CEXC_NV = 0x00000010 -FSR_AEXC_NX = (0x1 << 5) -FSR_AEXC_DZ = (0x2 << 5) -FSR_AEXC_UF = (0x4 << 5) -FSR_AEXC_OF = (0x8 << 5) -FSR_AEXC_NV = (0x10 << 5) -FTT_NONE = 0 -FTT_IEEE = 1 -FTT_UNFIN = 2 -FTT_UNIMP = 3 -FTT_SEQ = 4 -FTT_ALIGN = 5 -FTT_DFAULT = 6 -FSR_FTT_SHIFT = 14 -FSR_FTT_IEEE = (FTT_IEEE << FSR_FTT_SHIFT) -FSR_FTT_UNFIN = (FTT_UNFIN << FSR_FTT_SHIFT) -FSR_FTT_UNIMP = (FTT_UNIMP << FSR_FTT_SHIFT) -FSR_FTT_SEQ = (FTT_SEQ << FSR_FTT_SHIFT) -FSR_FTT_ALIGN = (FTT_ALIGN << FSR_FTT_SHIFT) -FSR_FTT_DFAULT = (FTT_DFAULT << FSR_FTT_SHIFT) -FSR_TEM_NX = (0x1 << 23) -FSR_TEM_DZ = (0x2 << 23) -FSR_TEM_UF = (0x4 << 23) -FSR_TEM_OF = (0x8 << 23) -FSR_TEM_NV = (0x10 << 23) -RP_DBLEXT = 0 -RP_SINGLE = 1 -RP_DOUBLE = 2 -RP_RESERVED = 3 -RD_NEAR = 0 -RD_ZER0 = 1 -RD_POSINF = 2 -RD_NEGINF = 3 -FPRS_DL = 0x1 -FPRS_DU = 0x2 -FPRS_FEF = 0x4 -PIL_MAX = 0xf -def SAVE_GLOBALS(RP): return \ - -def RESTORE_GLOBALS(RP): return \ - -def SAVE_OUTS(RP): return \ - -def RESTORE_OUTS(RP): return \ - -def SAVE_WINDOW(SBP): return \ - -def RESTORE_WINDOW(SBP): return \ - -def STORE_FPREGS(FP): return \ - -def LOAD_FPREGS(FP): return \ - -_SPARC_MAXREGWINDOW = 31 -_XRS_ID = 0x78727300 -GETCONTEXT = 0 -SETCONTEXT = 1 -UC_SIGMASK = 0o01 -UC_STACK = 0o02 -UC_CPU = 0o04 -UC_MAU = 0o10 -UC_FPU = UC_MAU -UC_INTR = 0o20 -UC_ASR = 0o40 -UC_MCONTEXT = (UC_CPU|UC_FPU|UC_ASR) -UC_ALL = (UC_SIGMASK|UC_STACK|UC_MCONTEXT) -_SIGQUEUE_MAX = 32 -_SIGNOTIFY_MAX = 32 - -# Included from sys/pcb.h -INSTR_VALID = 0x02 -NORMAL_STEP = 0x04 -WATCH_STEP = 0x08 -CPC_OVERFLOW = 0x10 -ASYNC_HWERR = 0x20 -STEP_NONE = 0 -STEP_REQUESTED = 1 -STEP_ACTIVE = 2 -STEP_WASACTIVE = 3 - -# Included from sys/msacct.h -LMS_USER = 0 -LMS_SYSTEM = 1 -LMS_TRAP = 2 -LMS_TFAULT = 3 -LMS_DFAULT = 4 -LMS_KFAULT = 5 -LMS_USER_LOCK = 6 -LMS_SLEEP = 7 -LMS_WAIT_CPU = 8 -LMS_STOPPED = 9 -NMSTATES = 10 - -# Included from sys/lwp.h - -# Included from sys/synch.h -from TYPES import * -USYNC_THREAD = 0x00 -USYNC_PROCESS = 0x01 -LOCK_NORMAL = 0x00 -LOCK_ERRORCHECK = 0x02 -LOCK_RECURSIVE = 0x04 -USYNC_PROCESS_ROBUST = 0x08 -LOCK_PRIO_NONE = 0x00 -LOCK_PRIO_INHERIT = 0x10 -LOCK_PRIO_PROTECT = 0x20 -LOCK_STALL_NP = 0x00 -LOCK_ROBUST_NP = 0x40 -LOCK_OWNERDEAD = 0x1 -LOCK_NOTRECOVERABLE = 0x2 -LOCK_INITED = 0x4 -LOCK_UNMAPPED = 0x8 -LWP_DETACHED = 0x00000040 -LWP_SUSPENDED = 0x00000080 -__LWP_ASLWP = 0x00000100 -MAXSYSARGS = 8 -NORMALRETURN = 0 -JUSTRETURN = 1 -LWP_USER = 0x01 -LWP_SYS = 0x02 -TS_FREE = 0x00 -TS_SLEEP = 0x01 -TS_RUN = 0x02 -TS_ONPROC = 0x04 -TS_ZOMB = 0x08 -TS_STOPPED = 0x10 -T_INTR_THREAD = 0x0001 -T_WAKEABLE = 0x0002 -T_TOMASK = 0x0004 -T_TALLOCSTK = 0x0008 -T_WOULDBLOCK = 0x0020 -T_DONTBLOCK = 0x0040 -T_DONTPEND = 0x0080 -T_SYS_PROF = 0x0100 -T_WAITCVSEM = 0x0200 -T_WATCHPT = 0x0400 -T_PANIC = 0x0800 -TP_HOLDLWP = 0x0002 -TP_TWAIT = 0x0004 -TP_LWPEXIT = 0x0008 -TP_PRSTOP = 0x0010 -TP_CHKPT = 0x0020 -TP_EXITLWP = 0x0040 -TP_PRVSTOP = 0x0080 -TP_MSACCT = 0x0100 -TP_STOPPING = 0x0200 -TP_WATCHPT = 0x0400 -TP_PAUSE = 0x0800 -TP_CHANGEBIND = 0x1000 -TS_LOAD = 0x0001 -TS_DONT_SWAP = 0x0002 -TS_SWAPENQ = 0x0004 -TS_ON_SWAPQ = 0x0008 -TS_CSTART = 0x0100 -TS_UNPAUSE = 0x0200 -TS_XSTART = 0x0400 -TS_PSTART = 0x0800 -TS_RESUME = 0x1000 -TS_CREATE = 0x2000 -TS_ALLSTART = \ - (TS_CSTART|TS_UNPAUSE|TS_XSTART|TS_PSTART|TS_RESUME|TS_CREATE) -def CPR_VSTOPPED(t): return \ - -def THREAD_TRANSITION(tp): return thread_transition(tp); - -def THREAD_STOP(tp): return \ - -def THREAD_ZOMB(tp): return THREAD_SET_STATE(tp, TS_ZOMB, NULL) - -def SEMA_HELD(x): return (sema_held((x))) - -NO_LOCKS_HELD = 1 -NO_COMPETING_THREADS = 1 - -# Included from sys/cred.h - -# Included from sys/uio.h -from TYPES import * - -# Included from sys/resource.h -from TYPES import * -PRIO_PROCESS = 0 -PRIO_PGRP = 1 -PRIO_USER = 2 -RLIMIT_CPU = 0 -RLIMIT_FSIZE = 1 -RLIMIT_DATA = 2 -RLIMIT_STACK = 3 -RLIMIT_CORE = 4 -RLIMIT_NOFILE = 5 -RLIMIT_VMEM = 6 -RLIMIT_AS = RLIMIT_VMEM -RLIM_NLIMITS = 7 -RLIM_INFINITY = (-3) -RLIM_SAVED_MAX = (-2) -RLIM_SAVED_CUR = (-1) -RLIM_INFINITY = 0x7fffffff -RLIM_SAVED_MAX = 0x7ffffffe -RLIM_SAVED_CUR = 0x7ffffffd -RLIM32_INFINITY = 0x7fffffff -RLIM32_SAVED_MAX = 0x7ffffffe -RLIM32_SAVED_CUR = 0x7ffffffd - -# Included from sys/model.h - -# Included from sys/debug.h -def ASSERT64(x): return ASSERT(x) - -def ASSERT32(x): return ASSERT(x) - -DATAMODEL_MASK = 0x0FF00000 -DATAMODEL_ILP32 = 0x00100000 -DATAMODEL_LP64 = 0x00200000 -DATAMODEL_NONE = 0 -DATAMODEL_NATIVE = DATAMODEL_LP64 -DATAMODEL_NATIVE = DATAMODEL_ILP32 -def STRUCT_SIZE(handle): return \ - -def STRUCT_BUF(handle): return ((handle).ptr.m64) - -def SIZEOF_PTR(umodel): return \ - -def STRUCT_SIZE(handle): return (sizeof (*(handle).ptr)) - -def STRUCT_BUF(handle): return ((handle).ptr) - -def SIZEOF_PTR(umodel): return sizeof (caddr_t) - -def lwp_getdatamodel(t): return DATAMODEL_ILP32 - -RUSAGE_SELF = 0 -RUSAGE_CHILDREN = -1 - -# Included from vm/seg_enum.h - -# Included from sys/buf.h - -# Included from sys/kstat.h -from TYPES import * -KSTAT_STRLEN = 31 -def KSTAT_ENTER(k): return \ - -def KSTAT_EXIT(k): return \ - -KSTAT_TYPE_RAW = 0 -KSTAT_TYPE_NAMED = 1 -KSTAT_TYPE_INTR = 2 -KSTAT_TYPE_IO = 3 -KSTAT_TYPE_TIMER = 4 -KSTAT_NUM_TYPES = 5 -KSTAT_FLAG_VIRTUAL = 0x01 -KSTAT_FLAG_VAR_SIZE = 0x02 -KSTAT_FLAG_WRITABLE = 0x04 -KSTAT_FLAG_PERSISTENT = 0x08 -KSTAT_FLAG_DORMANT = 0x10 -KSTAT_FLAG_INVALID = 0x20 -KSTAT_READ = 0 -KSTAT_WRITE = 1 -KSTAT_DATA_CHAR = 0 -KSTAT_DATA_INT32 = 1 -KSTAT_DATA_UINT32 = 2 -KSTAT_DATA_INT64 = 3 -KSTAT_DATA_UINT64 = 4 -KSTAT_DATA_LONG = KSTAT_DATA_INT32 -KSTAT_DATA_ULONG = KSTAT_DATA_UINT32 -KSTAT_DATA_LONG = KSTAT_DATA_INT64 -KSTAT_DATA_ULONG = KSTAT_DATA_UINT64 -KSTAT_DATA_LONG = 7 -KSTAT_DATA_ULONG = 8 -KSTAT_DATA_LONGLONG = KSTAT_DATA_INT64 -KSTAT_DATA_ULONGLONG = KSTAT_DATA_UINT64 -KSTAT_DATA_FLOAT = 5 -KSTAT_DATA_DOUBLE = 6 -KSTAT_INTR_HARD = 0 -KSTAT_INTR_SOFT = 1 -KSTAT_INTR_WATCHDOG = 2 -KSTAT_INTR_SPURIOUS = 3 -KSTAT_INTR_MULTSVC = 4 -KSTAT_NUM_INTRS = 5 -B_BUSY = 0x0001 -B_DONE = 0x0002 -B_ERROR = 0x0004 -B_PAGEIO = 0x0010 -B_PHYS = 0x0020 -B_READ = 0x0040 -B_WRITE = 0x0100 -B_KERNBUF = 0x0008 -B_WANTED = 0x0080 -B_AGE = 0x000200 -B_ASYNC = 0x000400 -B_DELWRI = 0x000800 -B_STALE = 0x001000 -B_DONTNEED = 0x002000 -B_REMAPPED = 0x004000 -B_FREE = 0x008000 -B_INVAL = 0x010000 -B_FORCE = 0x020000 -B_HEAD = 0x040000 -B_NOCACHE = 0x080000 -B_TRUNC = 0x100000 -B_SHADOW = 0x200000 -B_RETRYWRI = 0x400000 -def notavail(bp): return \ - -def BWRITE(bp): return \ - -def BWRITE2(bp): return \ - -VROOT = 0x01 -VNOCACHE = 0x02 -VNOMAP = 0x04 -VDUP = 0x08 -VNOSWAP = 0x10 -VNOMOUNT = 0x20 -VISSWAP = 0x40 -VSWAPLIKE = 0x80 -VVFSLOCK = 0x100 -VVFSWAIT = 0x200 -VVMLOCK = 0x400 -VDIROPEN = 0x800 -VVMEXEC = 0x1000 -VPXFS = 0x2000 -AT_TYPE = 0x0001 -AT_MODE = 0x0002 -AT_UID = 0x0004 -AT_GID = 0x0008 -AT_FSID = 0x0010 -AT_NODEID = 0x0020 -AT_NLINK = 0x0040 -AT_SIZE = 0x0080 -AT_ATIME = 0x0100 -AT_MTIME = 0x0200 -AT_CTIME = 0x0400 -AT_RDEV = 0x0800 -AT_BLKSIZE = 0x1000 -AT_NBLOCKS = 0x2000 -AT_VCODE = 0x4000 -AT_ALL = (AT_TYPE|AT_MODE|AT_UID|AT_GID|AT_FSID|AT_NODEID|\ - AT_NLINK|AT_SIZE|AT_ATIME|AT_MTIME|AT_CTIME|\ - AT_RDEV|AT_BLKSIZE|AT_NBLOCKS|AT_VCODE) -AT_STAT = (AT_MODE|AT_UID|AT_GID|AT_FSID|AT_NODEID|AT_NLINK|\ - AT_SIZE|AT_ATIME|AT_MTIME|AT_CTIME|AT_RDEV) -AT_TIMES = (AT_ATIME|AT_MTIME|AT_CTIME) -AT_NOSET = (AT_NLINK|AT_RDEV|AT_FSID|AT_NODEID|AT_TYPE|\ - AT_BLKSIZE|AT_NBLOCKS|AT_VCODE) -VSUID = 0o4000 -VSGID = 0o2000 -VSVTX = 0o1000 -VREAD = 0o0400 -VWRITE = 0o0200 -VEXEC = 0o0100 -MODEMASK = 0o7777 -PERMMASK = 0o0777 -def MANDMODE(mode): return (((mode) & (VSGID|(VEXEC>>3))) == VSGID) - -VSA_ACL = 0x0001 -VSA_ACLCNT = 0x0002 -VSA_DFACL = 0x0004 -VSA_DFACLCNT = 0x0008 -LOOKUP_DIR = 0x01 -DUMP_ALLOC = 0 -DUMP_FREE = 1 -DUMP_SCAN = 2 -ATTR_UTIME = 0x01 -ATTR_EXEC = 0x02 -ATTR_COMM = 0x04 -ATTR_HINT = 0x08 -ATTR_REAL = 0x10 - -# Included from sys/poll.h -POLLIN = 0x0001 -POLLPRI = 0x0002 -POLLOUT = 0x0004 -POLLRDNORM = 0x0040 -POLLWRNORM = POLLOUT -POLLRDBAND = 0x0080 -POLLWRBAND = 0x0100 -POLLNORM = POLLRDNORM -POLLERR = 0x0008 -POLLHUP = 0x0010 -POLLNVAL = 0x0020 -POLLREMOVE = 0x0800 -POLLRDDATA = 0x0200 -POLLNOERR = 0x0400 -POLLCLOSED = 0x8000 - -# Included from sys/strmdep.h -def str_aligned(X): return (((ulong_t)(X) & (sizeof (int) - 1)) == 0) - - -# Included from sys/strft.h -tdelta_t_sz = 12 -FTEV_MASK = 0x1FFF -FTEV_ISWR = 0x8000 -FTEV_CS = 0x4000 -FTEV_PS = 0x2000 -FTEV_QMASK = 0x1F00 -FTEV_ALLOCMASK = 0x1FF8 -FTEV_ALLOCB = 0x0000 -FTEV_ESBALLOC = 0x0001 -FTEV_DESBALLOC = 0x0002 -FTEV_ESBALLOCA = 0x0003 -FTEV_DESBALLOCA = 0x0004 -FTEV_ALLOCBIG = 0x0005 -FTEV_ALLOCBW = 0x0006 -FTEV_FREEB = 0x0008 -FTEV_DUPB = 0x0009 -FTEV_COPYB = 0x000A -FTEV_CALLER = 0x000F -FTEV_PUT = 0x0100 -FTEV_FSYNCQ = 0x0103 -FTEV_DSYNCQ = 0x0104 -FTEV_PUTQ = 0x0105 -FTEV_GETQ = 0x0106 -FTEV_RMVQ = 0x0107 -FTEV_INSQ = 0x0108 -FTEV_PUTBQ = 0x0109 -FTEV_FLUSHQ = 0x010A -FTEV_REPLYQ = 0x010B -FTEV_PUTNEXT = 0x010D -FTEV_RWNEXT = 0x010E -FTEV_QWINNER = 0x010F -FTEV_GEWRITE = 0x0101 -def FTFLW_HASH(h): return (((unsigned)(h))%ftflw_hash_sz) - -FTBLK_EVNTS = 0x9 -QENAB = 0x00000001 -QWANTR = 0x00000002 -QWANTW = 0x00000004 -QFULL = 0x00000008 -QREADR = 0x00000010 -QUSE = 0x00000020 -QNOENB = 0x00000040 -QBACK = 0x00000100 -QHLIST = 0x00000200 -QPAIR = 0x00000800 -QPERQ = 0x00001000 -QPERMOD = 0x00002000 -QMTSAFE = 0x00004000 -QMTOUTPERIM = 0x00008000 -QMT_TYPEMASK = (QPAIR|QPERQ|QPERMOD|QMTSAFE|QMTOUTPERIM) -QINSERVICE = 0x00010000 -QWCLOSE = 0x00020000 -QEND = 0x00040000 -QWANTWSYNC = 0x00080000 -QSYNCSTR = 0x00100000 -QISDRV = 0x00200000 -QHOT = 0x00400000 -QNEXTHOT = 0x00800000 -_QINSERTING = 0x04000000 -_QREMOVING = 0x08000000 -Q_SQQUEUED = 0x01 -Q_SQDRAINING = 0x02 -QB_FULL = 0x01 -QB_WANTW = 0x02 -QB_BACK = 0x04 -NBAND = 256 -STRUIOT_NONE = -1 -STRUIOT_DONTCARE = 0 -STRUIOT_STANDARD = 1 -STRUIOT_IP = 2 -DBLK_REFMIN = 0x01 -STRUIO_SPEC = 0x01 -STRUIO_DONE = 0x02 -STRUIO_IP = 0x04 -STRUIO_ZC = 0x08 -STRUIO_ICK = 0x10 -MSGMARK = 0x01 -MSGNOLOOP = 0x02 -MSGDELIM = 0x04 -MSGNOGET = 0x08 -MSGMARKNEXT = 0x10 -MSGNOTMARKNEXT = 0x20 -M_DATA = 0x00 -M_PROTO = 0x01 -M_BREAK = 0x08 -M_PASSFP = 0x09 -M_EVENT = 0x0a -M_SIG = 0x0b -M_DELAY = 0x0c -M_CTL = 0x0d -M_IOCTL = 0x0e -M_SETOPTS = 0x10 -M_RSE = 0x11 -M_IOCACK = 0x81 -M_IOCNAK = 0x82 -M_PCPROTO = 0x83 -M_PCSIG = 0x84 -M_READ = 0x85 -M_FLUSH = 0x86 -M_STOP = 0x87 -M_START = 0x88 -M_HANGUP = 0x89 -M_ERROR = 0x8a -M_COPYIN = 0x8b -M_COPYOUT = 0x8c -M_IOCDATA = 0x8d -M_PCRSE = 0x8e -M_STOPI = 0x8f -M_STARTI = 0x90 -M_PCEVENT = 0x91 -M_UNHANGUP = 0x92 -QNORM = 0x00 -QPCTL = 0x80 -IOC_MODELS = DATAMODEL_MASK -IOC_ILP32 = DATAMODEL_ILP32 -IOC_LP64 = DATAMODEL_LP64 -IOC_NATIVE = DATAMODEL_NATIVE -IOC_NONE = DATAMODEL_NONE -STRCANON = 0x01 -RECOPY = 0x02 -SO_ALL = 0x003f -SO_READOPT = 0x0001 -SO_WROFF = 0x0002 -SO_MINPSZ = 0x0004 -SO_MAXPSZ = 0x0008 -SO_HIWAT = 0x0010 -SO_LOWAT = 0x0020 -SO_MREADON = 0x0040 -SO_MREADOFF = 0x0080 -SO_NDELON = 0x0100 -SO_NDELOFF = 0x0200 -SO_ISTTY = 0x0400 -SO_ISNTTY = 0x0800 -SO_TOSTOP = 0x1000 -SO_TONSTOP = 0x2000 -SO_BAND = 0x4000 -SO_DELIM = 0x8000 -SO_NODELIM = 0x010000 -SO_STRHOLD = 0x020000 -SO_ERROPT = 0x040000 -SO_COPYOPT = 0x080000 -SO_MAXBLK = 0x100000 -DEF_IOV_MAX = 16 -INFOD_FIRSTBYTES = 0x02 -INFOD_BYTES = 0x04 -INFOD_COUNT = 0x08 -INFOD_COPYOUT = 0x10 -MODOPEN = 0x1 -CLONEOPEN = 0x2 -CONSOPEN = 0x4 -OPENFAIL = -1 -BPRI_LO = 1 -BPRI_MED = 2 -BPRI_HI = 3 -BPRI_FT = 4 -INFPSZ = -1 -FLUSHALL = 1 -FLUSHDATA = 0 -STRHIGH = 5120 -STRLOW = 1024 -MAXIOCBSZ = 1024 -PERIM_INNER = 1 -PERIM_OUTER = 2 -def datamsg(type): return \ - -def straln(a): return (caddr_t)((intptr_t)(a) & ~(sizeof (int)-1)) - - -# Included from sys/byteorder.h -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_ENCAP = 4 -IPPROTO_TCP = 6 -IPPROTO_EGP = 8 -IPPROTO_PUP = 12 -IPPROTO_UDP = 17 -IPPROTO_IDP = 22 -IPPROTO_IPV6 = 41 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_RSVP = 46 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_HELLO = 63 -IPPROTO_ND = 77 -IPPROTO_EON = 80 -IPPROTO_PIM = 103 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPORT_ECHO = 7 -IPPORT_DISCARD = 9 -IPPORT_SYSTAT = 11 -IPPORT_DAYTIME = 13 -IPPORT_NETSTAT = 15 -IPPORT_FTP = 21 -IPPORT_TELNET = 23 -IPPORT_SMTP = 25 -IPPORT_TIMESERVER = 37 -IPPORT_NAMESERVER = 42 -IPPORT_WHOIS = 43 -IPPORT_MTP = 57 -IPPORT_BOOTPS = 67 -IPPORT_BOOTPC = 68 -IPPORT_TFTP = 69 -IPPORT_RJE = 77 -IPPORT_FINGER = 79 -IPPORT_TTYLINK = 87 -IPPORT_SUPDUP = 95 -IPPORT_EXECSERVER = 512 -IPPORT_LOGINSERVER = 513 -IPPORT_CMDSERVER = 514 -IPPORT_EFSSERVER = 520 -IPPORT_BIFFUDP = 512 -IPPORT_WHOSERVER = 513 -IPPORT_ROUTESERVER = 520 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 5000 -IMPLINK_IP = 155 -IMPLINK_LOWEXPER = 156 -IMPLINK_HIGHEXPER = 158 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_MAX = 128 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_MAX = 65536 -IN_CLASSC_NSHIFT = 8 -IN_CLASSD_NSHIFT = 28 -def IN_MULTICAST(i): return IN_CLASSD(i) - -IN_LOOPBACKNET = 127 -def IN_SET_LOOPBACK_ADDR(a): return \ - -def IN6_IS_ADDR_UNSPECIFIED(addr): return \ - -def IN6_IS_ADDR_LOOPBACK(addr): return \ - -def IN6_IS_ADDR_LOOPBACK(addr): return \ - -def IN6_IS_ADDR_MULTICAST(addr): return \ - -def IN6_IS_ADDR_MULTICAST(addr): return \ - -def IN6_IS_ADDR_LINKLOCAL(addr): return \ - -def IN6_IS_ADDR_LINKLOCAL(addr): return \ - -def IN6_IS_ADDR_SITELOCAL(addr): return \ - -def IN6_IS_ADDR_SITELOCAL(addr): return \ - -def IN6_IS_ADDR_V4MAPPED(addr): return \ - -def IN6_IS_ADDR_V4MAPPED(addr): return \ - -def IN6_IS_ADDR_V4MAPPED_ANY(addr): return \ - -def IN6_IS_ADDR_V4MAPPED_ANY(addr): return \ - -def IN6_IS_ADDR_V4COMPAT(addr): return \ - -def IN6_IS_ADDR_V4COMPAT(addr): return \ - -def IN6_IS_ADDR_MC_RESERVED(addr): return \ - -def IN6_IS_ADDR_MC_RESERVED(addr): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(addr): return \ - -def IN6_IS_ADDR_MC_NODELOCAL(addr): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(addr): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(addr): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(addr): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(addr): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(addr): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(addr): return \ - -def IN6_IS_ADDR_MC_GLOBAL(addr): return \ - -def IN6_IS_ADDR_MC_GLOBAL(addr): return \ - -IP_OPTIONS = 1 -IP_HDRINCL = 2 -IP_TOS = 3 -IP_TTL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 0x10 -IP_MULTICAST_TTL = 0x11 -IP_MULTICAST_LOOP = 0x12 -IP_ADD_MEMBERSHIP = 0x13 -IP_DROP_MEMBERSHIP = 0x14 -IP_SEC_OPT = 0x22 -IPSEC_PREF_NEVER = 0x01 -IPSEC_PREF_REQUIRED = 0x02 -IPSEC_PREF_UNIQUE = 0x04 -IP_ADD_PROXY_ADDR = 0x40 -IP_BOUND_IF = 0x41 -IP_UNSPEC_SRC = 0x42 -IP_REUSEADDR = 0x104 -IP_DONTROUTE = 0x105 -IP_BROADCAST = 0x106 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IPV6_RTHDR_TYPE_0 = 0 -IPV6_UNICAST_HOPS = 0x5 -IPV6_MULTICAST_IF = 0x6 -IPV6_MULTICAST_HOPS = 0x7 -IPV6_MULTICAST_LOOP = 0x8 -IPV6_JOIN_GROUP = 0x9 -IPV6_LEAVE_GROUP = 0xa -IPV6_ADD_MEMBERSHIP = 0x9 -IPV6_DROP_MEMBERSHIP = 0xa -IPV6_PKTINFO = 0xb -IPV6_HOPLIMIT = 0xc -IPV6_NEXTHOP = 0xd -IPV6_HOPOPTS = 0xe -IPV6_DSTOPTS = 0xf -IPV6_RTHDR = 0x10 -IPV6_RTHDRDSTOPTS = 0x11 -IPV6_RECVPKTINFO = 0x12 -IPV6_RECVHOPLIMIT = 0x13 -IPV6_RECVHOPOPTS = 0x14 -IPV6_RECVDSTOPTS = 0x15 -IPV6_RECVRTHDR = 0x16 -IPV6_RECVRTHDRDSTOPTS = 0x17 -IPV6_CHECKSUM = 0x18 -IPV6_BOUND_IF = 0x41 -IPV6_UNSPEC_SRC = 0x42 -INET_ADDRSTRLEN = 16 -INET6_ADDRSTRLEN = 46 -IPV6_PAD1_OPT = 0 diff --git a/Lib/plat-sunos5/STROPTS.py b/Lib/plat-sunos5/STROPTS.py deleted file mode 100644 --- a/Lib/plat-sunos5/STROPTS.py +++ /dev/null @@ -1,1813 +0,0 @@ -# Generated by h2py from /usr/include/sys/stropts.h - -# Included from sys/feature_tests.h - -# Included from sys/isa_defs.h -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 8 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_ALIGNMENT_REQUIRED = 1 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 4 -_DOUBLE_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 4 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 4 -_ALIGNMENT_REQUIRED = 0 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_ALIGNMENT_REQUIRED = 1 -_LONG_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 8 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 8 -_LONG_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_POSIX_C_SOURCE = 1 -_LARGEFILE64_SOURCE = 1 -_LARGEFILE_SOURCE = 1 -_FILE_OFFSET_BITS = 64 -_FILE_OFFSET_BITS = 32 -_POSIX_C_SOURCE = 199506 -_POSIX_PTHREAD_SEMANTICS = 1 -_XOPEN_VERSION = 500 -_XOPEN_VERSION = 4 -_XOPEN_VERSION = 3 -from TYPES import * - -# Included from sys/conf.h - -# Included from sys/t_lock.h - -# Included from sys/machlock.h -from TYPES import * -LOCK_HELD_VALUE = 0xff -def SPIN_LOCK(pl): return ((pl) > ipltospl(LOCK_LEVEL)) - -def LOCK_SAMPLE_INTERVAL(i): return (((i) & 0xff) == 0) - -CLOCK_LEVEL = 10 -LOCK_LEVEL = 10 -DISP_LEVEL = (LOCK_LEVEL + 1) -PTR24_LSB = 5 -PTR24_MSB = (PTR24_LSB + 24) -PTR24_ALIGN = 32 -PTR24_BASE = 0xe0000000 - -# Included from sys/param.h -from TYPES import * -_POSIX_VDISABLE = 0 -MAX_INPUT = 512 -MAX_CANON = 256 -UID_NOBODY = 60001 -GID_NOBODY = UID_NOBODY -UID_NOACCESS = 60002 -MAX_TASKID = 999999 -MAX_MAXPID = 999999 -DEFAULT_MAXPID = 999999 -DEFAULT_JUMPPID = 100000 -DEFAULT_MAXPID = 30000 -DEFAULT_JUMPPID = 0 -MAXUID = 2147483647 -MAXPROJID = MAXUID -MAXLINK = 32767 -NMOUNT = 40 -CANBSIZ = 256 -NOFILE = 20 -NGROUPS_UMIN = 0 -NGROUPS_UMAX = 32 -NGROUPS_MAX_DEFAULT = 16 -NZERO = 20 -NULL = 0 -NULL = 0 -CMASK = 0o22 -CDLIMIT = (1<<11) -NBPS = 0x20000 -NBPSCTR = 512 -UBSIZE = 512 -SCTRSHFT = 9 -SYSNAME = 9 -PREMOTE = 39 -MAXPATHLEN = 1024 -MAXSYMLINKS = 20 -MAXNAMELEN = 256 -NADDR = 13 -PIPE_BUF = 5120 -PIPE_MAX = 5120 -NBBY = 8 -MAXBSIZE = 8192 -DEV_BSIZE = 512 -DEV_BSHIFT = 9 -MAXFRAG = 8 -MAXOFF32_T = 0x7fffffff -MAXOFF_T = 0x7fffffffffffffff -MAXOFFSET_T = 0x7fffffffffffffff -MAXOFF_T = 0x7fffffff -MAXOFFSET_T = 0x7fffffff -def btodb(bytes): return \ - -def dbtob(db): return \ - -def lbtodb(bytes): return \ - -def ldbtob(db): return \ - -NCARGS32 = 0x100000 -NCARGS64 = 0x200000 -NCARGS = NCARGS64 -NCARGS = NCARGS32 -FSHIFT = 8 -FSCALE = (1<> MMU_PAGESHIFT) - -def mmu_btopr(x): return ((((x) + MMU_PAGEOFFSET) >> MMU_PAGESHIFT)) - -def mmu_ptod(x): return ((x) << (MMU_PAGESHIFT - DEV_BSHIFT)) - -def ptod(x): return ((x) << (PAGESHIFT - DEV_BSHIFT)) - -def ptob(x): return ((x) << PAGESHIFT) - -def btop(x): return (((x) >> PAGESHIFT)) - -def btopr(x): return ((((x) + PAGEOFFSET) >> PAGESHIFT)) - -def dtop(DD): return (((DD) + NDPP - 1) >> (PAGESHIFT - DEV_BSHIFT)) - -def dtopt(DD): return ((DD) >> (PAGESHIFT - DEV_BSHIFT)) - -_AIO_LISTIO_MAX = (4096) -_AIO_MAX = (-1) -_MQ_OPEN_MAX = (32) -_MQ_PRIO_MAX = (32) -_SEM_NSEMS_MAX = INT_MAX -_SEM_VALUE_MAX = INT_MAX - -# Included from sys/unistd.h -_CS_PATH = 65 -_CS_LFS_CFLAGS = 68 -_CS_LFS_LDFLAGS = 69 -_CS_LFS_LIBS = 70 -_CS_LFS_LINTFLAGS = 71 -_CS_LFS64_CFLAGS = 72 -_CS_LFS64_LDFLAGS = 73 -_CS_LFS64_LIBS = 74 -_CS_LFS64_LINTFLAGS = 75 -_CS_XBS5_ILP32_OFF32_CFLAGS = 700 -_CS_XBS5_ILP32_OFF32_LDFLAGS = 701 -_CS_XBS5_ILP32_OFF32_LIBS = 702 -_CS_XBS5_ILP32_OFF32_LINTFLAGS = 703 -_CS_XBS5_ILP32_OFFBIG_CFLAGS = 705 -_CS_XBS5_ILP32_OFFBIG_LDFLAGS = 706 -_CS_XBS5_ILP32_OFFBIG_LIBS = 707 -_CS_XBS5_ILP32_OFFBIG_LINTFLAGS = 708 -_CS_XBS5_LP64_OFF64_CFLAGS = 709 -_CS_XBS5_LP64_OFF64_LDFLAGS = 710 -_CS_XBS5_LP64_OFF64_LIBS = 711 -_CS_XBS5_LP64_OFF64_LINTFLAGS = 712 -_CS_XBS5_LPBIG_OFFBIG_CFLAGS = 713 -_CS_XBS5_LPBIG_OFFBIG_LDFLAGS = 714 -_CS_XBS5_LPBIG_OFFBIG_LIBS = 715 -_CS_XBS5_LPBIG_OFFBIG_LINTFLAGS = 716 -_SC_ARG_MAX = 1 -_SC_CHILD_MAX = 2 -_SC_CLK_TCK = 3 -_SC_NGROUPS_MAX = 4 -_SC_OPEN_MAX = 5 -_SC_JOB_CONTROL = 6 -_SC_SAVED_IDS = 7 -_SC_VERSION = 8 -_SC_PASS_MAX = 9 -_SC_LOGNAME_MAX = 10 -_SC_PAGESIZE = 11 -_SC_XOPEN_VERSION = 12 -_SC_NPROCESSORS_CONF = 14 -_SC_NPROCESSORS_ONLN = 15 -_SC_STREAM_MAX = 16 -_SC_TZNAME_MAX = 17 -_SC_AIO_LISTIO_MAX = 18 -_SC_AIO_MAX = 19 -_SC_AIO_PRIO_DELTA_MAX = 20 -_SC_ASYNCHRONOUS_IO = 21 -_SC_DELAYTIMER_MAX = 22 -_SC_FSYNC = 23 -_SC_MAPPED_FILES = 24 -_SC_MEMLOCK = 25 -_SC_MEMLOCK_RANGE = 26 -_SC_MEMORY_PROTECTION = 27 -_SC_MESSAGE_PASSING = 28 -_SC_MQ_OPEN_MAX = 29 -_SC_MQ_PRIO_MAX = 30 -_SC_PRIORITIZED_IO = 31 -_SC_PRIORITY_SCHEDULING = 32 -_SC_REALTIME_SIGNALS = 33 -_SC_RTSIG_MAX = 34 -_SC_SEMAPHORES = 35 -_SC_SEM_NSEMS_MAX = 36 -_SC_SEM_VALUE_MAX = 37 -_SC_SHARED_MEMORY_OBJECTS = 38 -_SC_SIGQUEUE_MAX = 39 -_SC_SIGRT_MIN = 40 -_SC_SIGRT_MAX = 41 -_SC_SYNCHRONIZED_IO = 42 -_SC_TIMERS = 43 -_SC_TIMER_MAX = 44 -_SC_2_C_BIND = 45 -_SC_2_C_DEV = 46 -_SC_2_C_VERSION = 47 -_SC_2_FORT_DEV = 48 -_SC_2_FORT_RUN = 49 -_SC_2_LOCALEDEF = 50 -_SC_2_SW_DEV = 51 -_SC_2_UPE = 52 -_SC_2_VERSION = 53 -_SC_BC_BASE_MAX = 54 -_SC_BC_DIM_MAX = 55 -_SC_BC_SCALE_MAX = 56 -_SC_BC_STRING_MAX = 57 -_SC_COLL_WEIGHTS_MAX = 58 -_SC_EXPR_NEST_MAX = 59 -_SC_LINE_MAX = 60 -_SC_RE_DUP_MAX = 61 -_SC_XOPEN_CRYPT = 62 -_SC_XOPEN_ENH_I18N = 63 -_SC_XOPEN_SHM = 64 -_SC_2_CHAR_TERM = 66 -_SC_XOPEN_XCU_VERSION = 67 -_SC_ATEXIT_MAX = 76 -_SC_IOV_MAX = 77 -_SC_XOPEN_UNIX = 78 -_SC_PAGE_SIZE = _SC_PAGESIZE -_SC_T_IOV_MAX = 79 -_SC_PHYS_PAGES = 500 -_SC_AVPHYS_PAGES = 501 -_SC_COHER_BLKSZ = 503 -_SC_SPLIT_CACHE = 504 -_SC_ICACHE_SZ = 505 -_SC_DCACHE_SZ = 506 -_SC_ICACHE_LINESZ = 507 -_SC_DCACHE_LINESZ = 508 -_SC_ICACHE_BLKSZ = 509 -_SC_DCACHE_BLKSZ = 510 -_SC_DCACHE_TBLKSZ = 511 -_SC_ICACHE_ASSOC = 512 -_SC_DCACHE_ASSOC = 513 -_SC_MAXPID = 514 -_SC_STACK_PROT = 515 -_SC_THREAD_DESTRUCTOR_ITERATIONS = 568 -_SC_GETGR_R_SIZE_MAX = 569 -_SC_GETPW_R_SIZE_MAX = 570 -_SC_LOGIN_NAME_MAX = 571 -_SC_THREAD_KEYS_MAX = 572 -_SC_THREAD_STACK_MIN = 573 -_SC_THREAD_THREADS_MAX = 574 -_SC_TTY_NAME_MAX = 575 -_SC_THREADS = 576 -_SC_THREAD_ATTR_STACKADDR = 577 -_SC_THREAD_ATTR_STACKSIZE = 578 -_SC_THREAD_PRIORITY_SCHEDULING = 579 -_SC_THREAD_PRIO_INHERIT = 580 -_SC_THREAD_PRIO_PROTECT = 581 -_SC_THREAD_PROCESS_SHARED = 582 -_SC_THREAD_SAFE_FUNCTIONS = 583 -_SC_XOPEN_LEGACY = 717 -_SC_XOPEN_REALTIME = 718 -_SC_XOPEN_REALTIME_THREADS = 719 -_SC_XBS5_ILP32_OFF32 = 720 -_SC_XBS5_ILP32_OFFBIG = 721 -_SC_XBS5_LP64_OFF64 = 722 -_SC_XBS5_LPBIG_OFFBIG = 723 -_PC_LINK_MAX = 1 -_PC_MAX_CANON = 2 -_PC_MAX_INPUT = 3 -_PC_NAME_MAX = 4 -_PC_PATH_MAX = 5 -_PC_PIPE_BUF = 6 -_PC_NO_TRUNC = 7 -_PC_VDISABLE = 8 -_PC_CHOWN_RESTRICTED = 9 -_PC_ASYNC_IO = 10 -_PC_PRIO_IO = 11 -_PC_SYNC_IO = 12 -_PC_FILESIZEBITS = 67 -_PC_LAST = 67 -_POSIX_VERSION = 199506 -_POSIX2_VERSION = 199209 -_POSIX2_C_VERSION = 199209 -_XOPEN_XCU_VERSION = 4 -_XOPEN_REALTIME = 1 -_XOPEN_ENH_I18N = 1 -_XOPEN_SHM = 1 -_POSIX2_C_BIND = 1 -_POSIX2_CHAR_TERM = 1 -_POSIX2_LOCALEDEF = 1 -_POSIX2_C_DEV = 1 -_POSIX2_SW_DEV = 1 -_POSIX2_UPE = 1 - -# Included from sys/mutex.h -from TYPES import * -def MUTEX_HELD(x): return (mutex_owned(x)) - - -# Included from sys/rwlock.h -from TYPES import * -def RW_READ_HELD(x): return (rw_read_held((x))) - -def RW_WRITE_HELD(x): return (rw_write_held((x))) - -def RW_LOCK_HELD(x): return (rw_lock_held((x))) - -def RW_ISWRITER(x): return (rw_iswriter(x)) - - -# Included from sys/semaphore.h - -# Included from sys/thread.h -from TYPES import * - -# Included from sys/klwp.h -from TYPES import * - -# Included from sys/condvar.h -from TYPES import * - -# Included from sys/time.h - -# Included from sys/types32.h - -# Included from sys/int_types.h -TIME32_MAX = INT32_MAX -TIME32_MIN = INT32_MIN -def TIMEVAL_OVERFLOW(tv): return \ - -from TYPES import * -DST_NONE = 0 -DST_USA = 1 -DST_AUST = 2 -DST_WET = 3 -DST_MET = 4 -DST_EET = 5 -DST_CAN = 6 -DST_GB = 7 -DST_RUM = 8 -DST_TUR = 9 -DST_AUSTALT = 10 -ITIMER_REAL = 0 -ITIMER_VIRTUAL = 1 -ITIMER_PROF = 2 -ITIMER_REALPROF = 3 -def ITIMERVAL_OVERFLOW(itv): return \ - -SEC = 1 -MILLISEC = 1000 -MICROSEC = 1000000 -NANOSEC = 1000000000 - -# Included from sys/time_impl.h -def TIMESPEC_OVERFLOW(ts): return \ - -def ITIMERSPEC_OVERFLOW(it): return \ - -__CLOCK_REALTIME0 = 0 -CLOCK_VIRTUAL = 1 -CLOCK_PROF = 2 -__CLOCK_REALTIME3 = 3 -CLOCK_HIGHRES = 4 -CLOCK_MAX = 5 -CLOCK_REALTIME = __CLOCK_REALTIME3 -CLOCK_REALTIME = __CLOCK_REALTIME0 -TIMER_RELTIME = 0x0 -TIMER_ABSTIME = 0x1 -def TICK_TO_SEC(tick): return ((tick) / hz) - -def SEC_TO_TICK(sec): return ((sec) * hz) - -def TICK_TO_MSEC(tick): return \ - -def MSEC_TO_TICK(msec): return \ - -def MSEC_TO_TICK_ROUNDUP(msec): return \ - -def TICK_TO_USEC(tick): return ((tick) * usec_per_tick) - -def USEC_TO_TICK(usec): return ((usec) / usec_per_tick) - -def USEC_TO_TICK_ROUNDUP(usec): return \ - -def TICK_TO_NSEC(tick): return ((tick) * nsec_per_tick) - -def NSEC_TO_TICK(nsec): return ((nsec) / nsec_per_tick) - -def NSEC_TO_TICK_ROUNDUP(nsec): return \ - -def TIMEVAL_TO_TICK(tvp): return \ - -def TIMESTRUC_TO_TICK(tsp): return \ - - -# Included from time.h -from TYPES import * - -# Included from iso/time_iso.h -NULL = 0 -NULL = 0 -CLOCKS_PER_SEC = 1000000 - -# Included from sys/select.h -FD_SETSIZE = 65536 -FD_SETSIZE = 1024 -_NBBY = 8 -NBBY = _NBBY -def FD_ZERO(p): return bzero((p), sizeof (*(p))) - - -# Included from sys/signal.h - -# Included from sys/iso/signal_iso.h -SIGHUP = 1 -SIGINT = 2 -SIGQUIT = 3 -SIGILL = 4 -SIGTRAP = 5 -SIGIOT = 6 -SIGABRT = 6 -SIGEMT = 7 -SIGFPE = 8 -SIGKILL = 9 -SIGBUS = 10 -SIGSEGV = 11 -SIGSYS = 12 -SIGPIPE = 13 -SIGALRM = 14 -SIGTERM = 15 -SIGUSR1 = 16 -SIGUSR2 = 17 -SIGCLD = 18 -SIGCHLD = 18 -SIGPWR = 19 -SIGWINCH = 20 -SIGURG = 21 -SIGPOLL = 22 -SIGIO = SIGPOLL -SIGSTOP = 23 -SIGTSTP = 24 -SIGCONT = 25 -SIGTTIN = 26 -SIGTTOU = 27 -SIGVTALRM = 28 -SIGPROF = 29 -SIGXCPU = 30 -SIGXFSZ = 31 -SIGWAITING = 32 -SIGLWP = 33 -SIGFREEZE = 34 -SIGTHAW = 35 -SIGCANCEL = 36 -SIGLOST = 37 -_SIGRTMIN = 38 -_SIGRTMAX = 45 -SIG_BLOCK = 1 -SIG_UNBLOCK = 2 -SIG_SETMASK = 3 -SIGNO_MASK = 0xFF -SIGDEFER = 0x100 -SIGHOLD = 0x200 -SIGRELSE = 0x400 -SIGIGNORE = 0x800 -SIGPAUSE = 0x1000 - -# Included from sys/siginfo.h -from TYPES import * -SIGEV_NONE = 1 -SIGEV_SIGNAL = 2 -SIGEV_THREAD = 3 -SI_NOINFO = 32767 -SI_USER = 0 -SI_LWP = (-1) -SI_QUEUE = (-2) -SI_TIMER = (-3) -SI_ASYNCIO = (-4) -SI_MESGQ = (-5) - -# Included from sys/machsig.h -ILL_ILLOPC = 1 -ILL_ILLOPN = 2 -ILL_ILLADR = 3 -ILL_ILLTRP = 4 -ILL_PRVOPC = 5 -ILL_PRVREG = 6 -ILL_COPROC = 7 -ILL_BADSTK = 8 -NSIGILL = 8 -EMT_TAGOVF = 1 -EMT_CPCOVF = 2 -NSIGEMT = 2 -FPE_INTDIV = 1 -FPE_INTOVF = 2 -FPE_FLTDIV = 3 -FPE_FLTOVF = 4 -FPE_FLTUND = 5 -FPE_FLTRES = 6 -FPE_FLTINV = 7 -FPE_FLTSUB = 8 -NSIGFPE = 8 -SEGV_MAPERR = 1 -SEGV_ACCERR = 2 -NSIGSEGV = 2 -BUS_ADRALN = 1 -BUS_ADRERR = 2 -BUS_OBJERR = 3 -NSIGBUS = 3 -TRAP_BRKPT = 1 -TRAP_TRACE = 2 -TRAP_RWATCH = 3 -TRAP_WWATCH = 4 -TRAP_XWATCH = 5 -NSIGTRAP = 5 -CLD_EXITED = 1 -CLD_KILLED = 2 -CLD_DUMPED = 3 -CLD_TRAPPED = 4 -CLD_STOPPED = 5 -CLD_CONTINUED = 6 -NSIGCLD = 6 -POLL_IN = 1 -POLL_OUT = 2 -POLL_MSG = 3 -POLL_ERR = 4 -POLL_PRI = 5 -POLL_HUP = 6 -NSIGPOLL = 6 -PROF_SIG = 1 -NSIGPROF = 1 -SI_MAXSZ = 256 -SI_MAXSZ = 128 - -# Included from sys/time_std_impl.h -from TYPES import * -SI32_MAXSZ = 128 -def SI_CANQUEUE(c): return ((c) <= SI_QUEUE) - -SA_NOCLDSTOP = 0x00020000 -SA_ONSTACK = 0x00000001 -SA_RESETHAND = 0x00000002 -SA_RESTART = 0x00000004 -SA_SIGINFO = 0x00000008 -SA_NODEFER = 0x00000010 -SA_NOCLDWAIT = 0x00010000 -SA_WAITSIG = 0x00010000 -NSIG = 46 -MAXSIG = 45 -S_SIGNAL = 1 -S_SIGSET = 2 -S_SIGACTION = 3 -S_NONE = 4 -MINSIGSTKSZ = 2048 -SIGSTKSZ = 8192 -SS_ONSTACK = 0x00000001 -SS_DISABLE = 0x00000002 -SN_PROC = 1 -SN_CANCEL = 2 -SN_SEND = 3 - -# Included from sys/ucontext.h -from TYPES import * - -# Included from sys/regset.h -REG_CCR = (0) -REG_PSR = (0) -REG_PSR = (0) -REG_PC = (1) -REG_nPC = (2) -REG_Y = (3) -REG_G1 = (4) -REG_G2 = (5) -REG_G3 = (6) -REG_G4 = (7) -REG_G5 = (8) -REG_G6 = (9) -REG_G7 = (10) -REG_O0 = (11) -REG_O1 = (12) -REG_O2 = (13) -REG_O3 = (14) -REG_O4 = (15) -REG_O5 = (16) -REG_O6 = (17) -REG_O7 = (18) -REG_ASI = (19) -REG_FPRS = (20) -REG_PS = REG_PSR -REG_SP = REG_O6 -REG_R0 = REG_O0 -REG_R1 = REG_O1 -_NGREG = 21 -_NGREG = 19 -NGREG = _NGREG -_NGREG32 = 19 -_NGREG64 = 21 -SPARC_MAXREGWINDOW = 31 -MAXFPQ = 16 -XRS_ID = 0x78727300 - -# Included from v7/sys/privregs.h - -# Included from v7/sys/psr.h -PSR_CWP = 0x0000001F -PSR_ET = 0x00000020 -PSR_PS = 0x00000040 -PSR_S = 0x00000080 -PSR_PIL = 0x00000F00 -PSR_EF = 0x00001000 -PSR_EC = 0x00002000 -PSR_RSV = 0x000FC000 -PSR_ICC = 0x00F00000 -PSR_C = 0x00100000 -PSR_V = 0x00200000 -PSR_Z = 0x00400000 -PSR_N = 0x00800000 -PSR_VER = 0x0F000000 -PSR_IMPL = 0xF0000000 -PSL_ALLCC = PSR_ICC -PSL_USER = (PSR_S) -PSL_USERMASK = (PSR_ICC) -PSL_UBITS = (PSR_ICC|PSR_EF) -def USERMODE(ps): return (((ps) & PSR_PS) == 0) - - -# Included from sys/fsr.h -FSR_CEXC = 0x0000001f -FSR_AEXC = 0x000003e0 -FSR_FCC = 0x00000c00 -FSR_PR = 0x00001000 -FSR_QNE = 0x00002000 -FSR_FTT = 0x0001c000 -FSR_VER = 0x000e0000 -FSR_TEM = 0x0f800000 -FSR_RP = 0x30000000 -FSR_RD = 0xc0000000 -FSR_VER_SHIFT = 17 -FSR_FCC1 = 0x00000003 -FSR_FCC2 = 0x0000000C -FSR_FCC3 = 0x00000030 -FSR_CEXC_NX = 0x00000001 -FSR_CEXC_DZ = 0x00000002 -FSR_CEXC_UF = 0x00000004 -FSR_CEXC_OF = 0x00000008 -FSR_CEXC_NV = 0x00000010 -FSR_AEXC_NX = (0x1 << 5) -FSR_AEXC_DZ = (0x2 << 5) -FSR_AEXC_UF = (0x4 << 5) -FSR_AEXC_OF = (0x8 << 5) -FSR_AEXC_NV = (0x10 << 5) -FTT_NONE = 0 -FTT_IEEE = 1 -FTT_UNFIN = 2 -FTT_UNIMP = 3 -FTT_SEQ = 4 -FTT_ALIGN = 5 -FTT_DFAULT = 6 -FSR_FTT_SHIFT = 14 -FSR_FTT_IEEE = (FTT_IEEE << FSR_FTT_SHIFT) -FSR_FTT_UNFIN = (FTT_UNFIN << FSR_FTT_SHIFT) -FSR_FTT_UNIMP = (FTT_UNIMP << FSR_FTT_SHIFT) -FSR_FTT_SEQ = (FTT_SEQ << FSR_FTT_SHIFT) -FSR_FTT_ALIGN = (FTT_ALIGN << FSR_FTT_SHIFT) -FSR_FTT_DFAULT = (FTT_DFAULT << FSR_FTT_SHIFT) -FSR_TEM_NX = (0x1 << 23) -FSR_TEM_DZ = (0x2 << 23) -FSR_TEM_UF = (0x4 << 23) -FSR_TEM_OF = (0x8 << 23) -FSR_TEM_NV = (0x10 << 23) -RP_DBLEXT = 0 -RP_SINGLE = 1 -RP_DOUBLE = 2 -RP_RESERVED = 3 -RD_NEAR = 0 -RD_ZER0 = 1 -RD_POSINF = 2 -RD_NEGINF = 3 -FPRS_DL = 0x1 -FPRS_DU = 0x2 -FPRS_FEF = 0x4 -PIL_MAX = 0xf -def SAVE_GLOBALS(RP): return \ - -def RESTORE_GLOBALS(RP): return \ - -def SAVE_OUTS(RP): return \ - -def RESTORE_OUTS(RP): return \ - -def SAVE_WINDOW(SBP): return \ - -def RESTORE_WINDOW(SBP): return \ - -def STORE_FPREGS(FP): return \ - -def LOAD_FPREGS(FP): return \ - -_SPARC_MAXREGWINDOW = 31 -_XRS_ID = 0x78727300 -GETCONTEXT = 0 -SETCONTEXT = 1 -UC_SIGMASK = 0o01 -UC_STACK = 0o02 -UC_CPU = 0o04 -UC_MAU = 0o10 -UC_FPU = UC_MAU -UC_INTR = 0o20 -UC_ASR = 0o40 -UC_MCONTEXT = (UC_CPU|UC_FPU|UC_ASR) -UC_ALL = (UC_SIGMASK|UC_STACK|UC_MCONTEXT) -_SIGQUEUE_MAX = 32 -_SIGNOTIFY_MAX = 32 - -# Included from sys/pcb.h -INSTR_VALID = 0x02 -NORMAL_STEP = 0x04 -WATCH_STEP = 0x08 -CPC_OVERFLOW = 0x10 -ASYNC_HWERR = 0x20 -STEP_NONE = 0 -STEP_REQUESTED = 1 -STEP_ACTIVE = 2 -STEP_WASACTIVE = 3 - -# Included from sys/msacct.h -LMS_USER = 0 -LMS_SYSTEM = 1 -LMS_TRAP = 2 -LMS_TFAULT = 3 -LMS_DFAULT = 4 -LMS_KFAULT = 5 -LMS_USER_LOCK = 6 -LMS_SLEEP = 7 -LMS_WAIT_CPU = 8 -LMS_STOPPED = 9 -NMSTATES = 10 - -# Included from sys/lwp.h - -# Included from sys/synch.h -from TYPES import * -USYNC_THREAD = 0x00 -USYNC_PROCESS = 0x01 -LOCK_NORMAL = 0x00 -LOCK_ERRORCHECK = 0x02 -LOCK_RECURSIVE = 0x04 -USYNC_PROCESS_ROBUST = 0x08 -LOCK_PRIO_NONE = 0x00 -LOCK_PRIO_INHERIT = 0x10 -LOCK_PRIO_PROTECT = 0x20 -LOCK_STALL_NP = 0x00 -LOCK_ROBUST_NP = 0x40 -LOCK_OWNERDEAD = 0x1 -LOCK_NOTRECOVERABLE = 0x2 -LOCK_INITED = 0x4 -LOCK_UNMAPPED = 0x8 -LWP_DETACHED = 0x00000040 -LWP_SUSPENDED = 0x00000080 -__LWP_ASLWP = 0x00000100 -MAXSYSARGS = 8 -NORMALRETURN = 0 -JUSTRETURN = 1 -LWP_USER = 0x01 -LWP_SYS = 0x02 -TS_FREE = 0x00 -TS_SLEEP = 0x01 -TS_RUN = 0x02 -TS_ONPROC = 0x04 -TS_ZOMB = 0x08 -TS_STOPPED = 0x10 -T_INTR_THREAD = 0x0001 -T_WAKEABLE = 0x0002 -T_TOMASK = 0x0004 -T_TALLOCSTK = 0x0008 -T_WOULDBLOCK = 0x0020 -T_DONTBLOCK = 0x0040 -T_DONTPEND = 0x0080 -T_SYS_PROF = 0x0100 -T_WAITCVSEM = 0x0200 -T_WATCHPT = 0x0400 -T_PANIC = 0x0800 -TP_HOLDLWP = 0x0002 -TP_TWAIT = 0x0004 -TP_LWPEXIT = 0x0008 -TP_PRSTOP = 0x0010 -TP_CHKPT = 0x0020 -TP_EXITLWP = 0x0040 -TP_PRVSTOP = 0x0080 -TP_MSACCT = 0x0100 -TP_STOPPING = 0x0200 -TP_WATCHPT = 0x0400 -TP_PAUSE = 0x0800 -TP_CHANGEBIND = 0x1000 -TS_LOAD = 0x0001 -TS_DONT_SWAP = 0x0002 -TS_SWAPENQ = 0x0004 -TS_ON_SWAPQ = 0x0008 -TS_CSTART = 0x0100 -TS_UNPAUSE = 0x0200 -TS_XSTART = 0x0400 -TS_PSTART = 0x0800 -TS_RESUME = 0x1000 -TS_CREATE = 0x2000 -TS_ALLSTART = \ - (TS_CSTART|TS_UNPAUSE|TS_XSTART|TS_PSTART|TS_RESUME|TS_CREATE) -def CPR_VSTOPPED(t): return \ - -def THREAD_TRANSITION(tp): return thread_transition(tp); - -def THREAD_STOP(tp): return \ - -def THREAD_ZOMB(tp): return THREAD_SET_STATE(tp, TS_ZOMB, NULL) - -def SEMA_HELD(x): return (sema_held((x))) - -NO_LOCKS_HELD = 1 -NO_COMPETING_THREADS = 1 -FMNAMESZ = 8 - -# Included from sys/systm.h -from TYPES import * - -# Included from sys/proc.h - -# Included from sys/cred.h - -# Included from sys/user.h -from TYPES import * - -# Included from sys/resource.h -from TYPES import * -PRIO_PROCESS = 0 -PRIO_PGRP = 1 -PRIO_USER = 2 -RLIMIT_CPU = 0 -RLIMIT_FSIZE = 1 -RLIMIT_DATA = 2 -RLIMIT_STACK = 3 -RLIMIT_CORE = 4 -RLIMIT_NOFILE = 5 -RLIMIT_VMEM = 6 -RLIMIT_AS = RLIMIT_VMEM -RLIM_NLIMITS = 7 -RLIM_INFINITY = (-3) -RLIM_SAVED_MAX = (-2) -RLIM_SAVED_CUR = (-1) -RLIM_INFINITY = 0x7fffffff -RLIM_SAVED_MAX = 0x7ffffffe -RLIM_SAVED_CUR = 0x7ffffffd -RLIM32_INFINITY = 0x7fffffff -RLIM32_SAVED_MAX = 0x7ffffffe -RLIM32_SAVED_CUR = 0x7ffffffd - -# Included from sys/model.h - -# Included from sys/debug.h -def ASSERT64(x): return ASSERT(x) - -def ASSERT32(x): return ASSERT(x) - -DATAMODEL_MASK = 0x0FF00000 -DATAMODEL_ILP32 = 0x00100000 -DATAMODEL_LP64 = 0x00200000 -DATAMODEL_NONE = 0 -DATAMODEL_NATIVE = DATAMODEL_LP64 -DATAMODEL_NATIVE = DATAMODEL_ILP32 -def STRUCT_SIZE(handle): return \ - -def STRUCT_BUF(handle): return ((handle).ptr.m64) - -def SIZEOF_PTR(umodel): return \ - -def STRUCT_SIZE(handle): return (sizeof (*(handle).ptr)) - -def STRUCT_BUF(handle): return ((handle).ptr) - -def SIZEOF_PTR(umodel): return sizeof (caddr_t) - -def lwp_getdatamodel(t): return DATAMODEL_ILP32 - -RUSAGE_SELF = 0 -RUSAGE_CHILDREN = -1 - -# Included from sys/auxv.h -AT_NULL = 0 -AT_IGNORE = 1 -AT_EXECFD = 2 -AT_PHDR = 3 -AT_PHENT = 4 -AT_PHNUM = 5 -AT_PAGESZ = 6 -AT_BASE = 7 -AT_FLAGS = 8 -AT_ENTRY = 9 -AT_DCACHEBSIZE = 10 -AT_ICACHEBSIZE = 11 -AT_UCACHEBSIZE = 12 -AT_SUN_UID = 2000 -AT_SUN_RUID = 2001 -AT_SUN_GID = 2002 -AT_SUN_RGID = 2003 -AT_SUN_LDELF = 2004 -AT_SUN_LDSHDR = 2005 -AT_SUN_LDNAME = 2006 -AT_SUN_LPAGESZ = 2007 -AT_SUN_PLATFORM = 2008 -AT_SUN_HWCAP = 2009 -AT_SUN_IFLUSH = 2010 -AT_SUN_CPU = 2011 -AT_SUN_EMUL_ENTRY = 2012 -AT_SUN_EMUL_EXECFD = 2013 -AT_SUN_EXECNAME = 2014 -AT_SUN_MMU = 2015 - -# Included from sys/errno.h -EPERM = 1 -ENOENT = 2 -ESRCH = 3 -EINTR = 4 -EIO = 5 -ENXIO = 6 -E2BIG = 7 -ENOEXEC = 8 -EBADF = 9 -ECHILD = 10 -EAGAIN = 11 -ENOMEM = 12 -EACCES = 13 -EFAULT = 14 -ENOTBLK = 15 -EBUSY = 16 -EEXIST = 17 -EXDEV = 18 -ENODEV = 19 -ENOTDIR = 20 -EISDIR = 21 -EINVAL = 22 -ENFILE = 23 -EMFILE = 24 -ENOTTY = 25 -ETXTBSY = 26 -EFBIG = 27 -ENOSPC = 28 -ESPIPE = 29 -EROFS = 30 -EMLINK = 31 -EPIPE = 32 -EDOM = 33 -ERANGE = 34 -ENOMSG = 35 -EIDRM = 36 -ECHRNG = 37 -EL2NSYNC = 38 -EL3HLT = 39 -EL3RST = 40 -ELNRNG = 41 -EUNATCH = 42 -ENOCSI = 43 -EL2HLT = 44 -EDEADLK = 45 -ENOLCK = 46 -ECANCELED = 47 -ENOTSUP = 48 -EDQUOT = 49 -EBADE = 50 -EBADR = 51 -EXFULL = 52 -ENOANO = 53 -EBADRQC = 54 -EBADSLT = 55 -EDEADLOCK = 56 -EBFONT = 57 -EOWNERDEAD = 58 -ENOTRECOVERABLE = 59 -ENOSTR = 60 -ENODATA = 61 -ETIME = 62 -ENOSR = 63 -ENONET = 64 -ENOPKG = 65 -EREMOTE = 66 -ENOLINK = 67 -EADV = 68 -ESRMNT = 69 -ECOMM = 70 -EPROTO = 71 -ELOCKUNMAPPED = 72 -ENOTACTIVE = 73 -EMULTIHOP = 74 -EBADMSG = 77 -ENAMETOOLONG = 78 -EOVERFLOW = 79 -ENOTUNIQ = 80 -EBADFD = 81 -EREMCHG = 82 -ELIBACC = 83 -ELIBBAD = 84 -ELIBSCN = 85 -ELIBMAX = 86 -ELIBEXEC = 87 -EILSEQ = 88 -ENOSYS = 89 -ELOOP = 90 -ERESTART = 91 -ESTRPIPE = 92 -ENOTEMPTY = 93 -EUSERS = 94 -ENOTSOCK = 95 -EDESTADDRREQ = 96 -EMSGSIZE = 97 -EPROTOTYPE = 98 -ENOPROTOOPT = 99 -EPROTONOSUPPORT = 120 -ESOCKTNOSUPPORT = 121 -EOPNOTSUPP = 122 -EPFNOSUPPORT = 123 -EAFNOSUPPORT = 124 -EADDRINUSE = 125 -EADDRNOTAVAIL = 126 -ENETDOWN = 127 -ENETUNREACH = 128 -ENETRESET = 129 -ECONNABORTED = 130 -ECONNRESET = 131 -ENOBUFS = 132 -EISCONN = 133 -ENOTCONN = 134 -ESHUTDOWN = 143 -ETOOMANYREFS = 144 -ETIMEDOUT = 145 -ECONNREFUSED = 146 -EHOSTDOWN = 147 -EHOSTUNREACH = 148 -EWOULDBLOCK = EAGAIN -EALREADY = 149 -EINPROGRESS = 150 -ESTALE = 151 -PSARGSZ = 80 -PSCOMSIZ = 14 -MAXCOMLEN = 16 -__KERN_NAUXV_IMPL = 19 -__KERN_NAUXV_IMPL = 21 -__KERN_NAUXV_IMPL = 21 -PSARGSZ = 80 - -# Included from sys/watchpoint.h -from TYPES import * - -# Included from vm/seg_enum.h - -# Included from sys/copyops.h -from TYPES import * - -# Included from sys/buf.h - -# Included from sys/kstat.h -from TYPES import * -KSTAT_STRLEN = 31 -def KSTAT_ENTER(k): return \ - -def KSTAT_EXIT(k): return \ - -KSTAT_TYPE_RAW = 0 -KSTAT_TYPE_NAMED = 1 -KSTAT_TYPE_INTR = 2 -KSTAT_TYPE_IO = 3 -KSTAT_TYPE_TIMER = 4 -KSTAT_NUM_TYPES = 5 -KSTAT_FLAG_VIRTUAL = 0x01 -KSTAT_FLAG_VAR_SIZE = 0x02 -KSTAT_FLAG_WRITABLE = 0x04 -KSTAT_FLAG_PERSISTENT = 0x08 -KSTAT_FLAG_DORMANT = 0x10 -KSTAT_FLAG_INVALID = 0x20 -KSTAT_READ = 0 -KSTAT_WRITE = 1 -KSTAT_DATA_CHAR = 0 -KSTAT_DATA_INT32 = 1 -KSTAT_DATA_UINT32 = 2 -KSTAT_DATA_INT64 = 3 -KSTAT_DATA_UINT64 = 4 -KSTAT_DATA_LONG = KSTAT_DATA_INT32 -KSTAT_DATA_ULONG = KSTAT_DATA_UINT32 -KSTAT_DATA_LONG = KSTAT_DATA_INT64 -KSTAT_DATA_ULONG = KSTAT_DATA_UINT64 -KSTAT_DATA_LONG = 7 -KSTAT_DATA_ULONG = 8 -KSTAT_DATA_LONGLONG = KSTAT_DATA_INT64 -KSTAT_DATA_ULONGLONG = KSTAT_DATA_UINT64 -KSTAT_DATA_FLOAT = 5 -KSTAT_DATA_DOUBLE = 6 -KSTAT_INTR_HARD = 0 -KSTAT_INTR_SOFT = 1 -KSTAT_INTR_WATCHDOG = 2 -KSTAT_INTR_SPURIOUS = 3 -KSTAT_INTR_MULTSVC = 4 -KSTAT_NUM_INTRS = 5 -B_BUSY = 0x0001 -B_DONE = 0x0002 -B_ERROR = 0x0004 -B_PAGEIO = 0x0010 -B_PHYS = 0x0020 -B_READ = 0x0040 -B_WRITE = 0x0100 -B_KERNBUF = 0x0008 -B_WANTED = 0x0080 -B_AGE = 0x000200 -B_ASYNC = 0x000400 -B_DELWRI = 0x000800 -B_STALE = 0x001000 -B_DONTNEED = 0x002000 -B_REMAPPED = 0x004000 -B_FREE = 0x008000 -B_INVAL = 0x010000 -B_FORCE = 0x020000 -B_HEAD = 0x040000 -B_NOCACHE = 0x080000 -B_TRUNC = 0x100000 -B_SHADOW = 0x200000 -B_RETRYWRI = 0x400000 -def notavail(bp): return \ - -def BWRITE(bp): return \ - -def BWRITE2(bp): return \ - - -# Included from sys/aio_req.h - -# Included from sys/uio.h -from TYPES import * -WP_NOWATCH = 0x01 -WP_SETPROT = 0x02 - -# Included from sys/timer.h -from TYPES import * -_TIMER_MAX = 32 -ITLK_LOCKED = 0x01 -ITLK_WANTED = 0x02 -ITLK_REMOVE = 0x04 -IT_PERLWP = 0x01 -IT_SIGNAL = 0x02 - -# Included from sys/utrap.h -UT_INSTRUCTION_DISABLED = 1 -UT_INSTRUCTION_ERROR = 2 -UT_INSTRUCTION_PROTECTION = 3 -UT_ILLTRAP_INSTRUCTION = 4 -UT_ILLEGAL_INSTRUCTION = 5 -UT_PRIVILEGED_OPCODE = 6 -UT_FP_DISABLED = 7 -UT_FP_EXCEPTION_IEEE_754 = 8 -UT_FP_EXCEPTION_OTHER = 9 -UT_TAG_OVERFLOW = 10 -UT_DIVISION_BY_ZERO = 11 -UT_DATA_EXCEPTION = 12 -UT_DATA_ERROR = 13 -UT_DATA_PROTECTION = 14 -UT_MEM_ADDRESS_NOT_ALIGNED = 15 -UT_PRIVILEGED_ACTION = 16 -UT_ASYNC_DATA_ERROR = 17 -UT_TRAP_INSTRUCTION_16 = 18 -UT_TRAP_INSTRUCTION_17 = 19 -UT_TRAP_INSTRUCTION_18 = 20 -UT_TRAP_INSTRUCTION_19 = 21 -UT_TRAP_INSTRUCTION_20 = 22 -UT_TRAP_INSTRUCTION_21 = 23 -UT_TRAP_INSTRUCTION_22 = 24 -UT_TRAP_INSTRUCTION_23 = 25 -UT_TRAP_INSTRUCTION_24 = 26 -UT_TRAP_INSTRUCTION_25 = 27 -UT_TRAP_INSTRUCTION_26 = 28 -UT_TRAP_INSTRUCTION_27 = 29 -UT_TRAP_INSTRUCTION_28 = 30 -UT_TRAP_INSTRUCTION_29 = 31 -UT_TRAP_INSTRUCTION_30 = 32 -UT_TRAP_INSTRUCTION_31 = 33 -UTRAP_V8P_FP_DISABLED = UT_FP_DISABLED -UTRAP_V8P_MEM_ADDRESS_NOT_ALIGNED = UT_MEM_ADDRESS_NOT_ALIGNED -UT_PRECISE_MAXTRAPS = 33 - -# Included from sys/refstr.h - -# Included from sys/task.h -from TYPES import * -TASK_NORMAL = 0x0 -TASK_FINAL = 0x1 -TASK_FINALITY = 0x1 - -# Included from sys/id_space.h -from TYPES import * - -# Included from sys/vmem.h -from TYPES import * -VM_SLEEP = 0x00000000 -VM_NOSLEEP = 0x00000001 -VM_PANIC = 0x00000002 -VM_KMFLAGS = 0x000000ff -VM_BESTFIT = 0x00000100 -VMEM_ALLOC = 0x01 -VMEM_FREE = 0x02 -VMEM_SPAN = 0x10 -ISP_NORMAL = 0x0 -ISP_RESERVE = 0x1 - -# Included from sys/exacct_impl.h -from TYPES import * - -# Included from sys/kmem.h -from TYPES import * -KM_SLEEP = 0x0000 -KM_NOSLEEP = 0x0001 -KM_PANIC = 0x0002 -KM_VMFLAGS = 0x00ff -KM_FLAGS = 0xffff -KMC_NOTOUCH = 0x00010000 -KMC_NODEBUG = 0x00020000 -KMC_NOMAGAZINE = 0x00040000 -KMC_NOHASH = 0x00080000 -KMC_QCACHE = 0x00100000 -_ISA_IA32 = 0 -_ISA_IA64 = 1 -SSLEEP = 1 -SRUN = 2 -SZOMB = 3 -SSTOP = 4 -SIDL = 5 -SONPROC = 6 -CLDPEND = 0x0001 -CLDCONT = 0x0002 -SSYS = 0x00000001 -STRC = 0x00000002 -SLOAD = 0x00000008 -SLOCK = 0x00000010 -SPREXEC = 0x00000020 -SPROCTR = 0x00000040 -SPRFORK = 0x00000080 -SKILLED = 0x00000100 -SULOAD = 0x00000200 -SRUNLCL = 0x00000400 -SBPTADJ = 0x00000800 -SKILLCL = 0x00001000 -SOWEUPC = 0x00002000 -SEXECED = 0x00004000 -SPASYNC = 0x00008000 -SJCTL = 0x00010000 -SNOWAIT = 0x00020000 -SVFORK = 0x00040000 -SVFWAIT = 0x00080000 -EXITLWPS = 0x00100000 -HOLDFORK = 0x00200000 -SWAITSIG = 0x00400000 -HOLDFORK1 = 0x00800000 -COREDUMP = 0x01000000 -SMSACCT = 0x02000000 -ASLWP = 0x04000000 -SPRLOCK = 0x08000000 -NOCD = 0x10000000 -HOLDWATCH = 0x20000000 -SMSFORK = 0x40000000 -SDOCORE = 0x80000000 -FORREAL = 0 -JUSTLOOKING = 1 -SUSPEND_NORMAL = 0 -SUSPEND_PAUSE = 1 -NOCLASS = (-1) - -# Included from sys/dditypes.h -DDI_DEVICE_ATTR_V0 = 0x0001 -DDI_NEVERSWAP_ACC = 0x00 -DDI_STRUCTURE_LE_ACC = 0x01 -DDI_STRUCTURE_BE_ACC = 0x02 -DDI_STRICTORDER_ACC = 0x00 -DDI_UNORDERED_OK_ACC = 0x01 -DDI_MERGING_OK_ACC = 0x02 -DDI_LOADCACHING_OK_ACC = 0x03 -DDI_STORECACHING_OK_ACC = 0x04 -DDI_DATA_SZ01_ACC = 1 -DDI_DATA_SZ02_ACC = 2 -DDI_DATA_SZ04_ACC = 4 -DDI_DATA_SZ08_ACC = 8 -VERS_ACCHDL = 0x0001 -DEVID_NONE = 0 -DEVID_SCSI3_WWN = 1 -DEVID_SCSI_SERIAL = 2 -DEVID_FAB = 3 -DEVID_ENCAP = 4 -DEVID_MAXTYPE = 4 - -# Included from sys/varargs.h - -# Included from sys/va_list.h -VA_ALIGN = 8 -def _ARGSIZEOF(t): return ((sizeof (t) + VA_ALIGN - 1) & ~(VA_ALIGN - 1)) - -VA_ALIGN = 8 -def _ARGSIZEOF(t): return ((sizeof (t) + VA_ALIGN - 1) & ~(VA_ALIGN - 1)) - -NSYSCALL = 256 -SE_32RVAL1 = 0x0 -SE_32RVAL2 = 0x1 -SE_64RVAL = 0x2 -SE_RVAL_MASK = 0x3 -SE_LOADABLE = 0x08 -SE_LOADED = 0x10 -SE_NOUNLOAD = 0x20 -SE_ARGC = 0x40 - -# Included from sys/devops.h -from TYPES import * - -# Included from sys/poll.h -POLLIN = 0x0001 -POLLPRI = 0x0002 -POLLOUT = 0x0004 -POLLRDNORM = 0x0040 -POLLWRNORM = POLLOUT -POLLRDBAND = 0x0080 -POLLWRBAND = 0x0100 -POLLNORM = POLLRDNORM -POLLERR = 0x0008 -POLLHUP = 0x0010 -POLLNVAL = 0x0020 -POLLREMOVE = 0x0800 -POLLRDDATA = 0x0200 -POLLNOERR = 0x0400 -POLLCLOSED = 0x8000 - -# Included from vm/as.h - -# Included from vm/seg.h - -# Included from sys/vnode.h -from TYPES import * -VROOT = 0x01 -VNOCACHE = 0x02 -VNOMAP = 0x04 -VDUP = 0x08 -VNOSWAP = 0x10 -VNOMOUNT = 0x20 -VISSWAP = 0x40 -VSWAPLIKE = 0x80 -VVFSLOCK = 0x100 -VVFSWAIT = 0x200 -VVMLOCK = 0x400 -VDIROPEN = 0x800 -VVMEXEC = 0x1000 -VPXFS = 0x2000 -AT_TYPE = 0x0001 -AT_MODE = 0x0002 -AT_UID = 0x0004 -AT_GID = 0x0008 -AT_FSID = 0x0010 -AT_NODEID = 0x0020 -AT_NLINK = 0x0040 -AT_SIZE = 0x0080 -AT_ATIME = 0x0100 -AT_MTIME = 0x0200 -AT_CTIME = 0x0400 -AT_RDEV = 0x0800 -AT_BLKSIZE = 0x1000 -AT_NBLOCKS = 0x2000 -AT_VCODE = 0x4000 -AT_ALL = (AT_TYPE|AT_MODE|AT_UID|AT_GID|AT_FSID|AT_NODEID|\ - AT_NLINK|AT_SIZE|AT_ATIME|AT_MTIME|AT_CTIME|\ - AT_RDEV|AT_BLKSIZE|AT_NBLOCKS|AT_VCODE) -AT_STAT = (AT_MODE|AT_UID|AT_GID|AT_FSID|AT_NODEID|AT_NLINK|\ - AT_SIZE|AT_ATIME|AT_MTIME|AT_CTIME|AT_RDEV) -AT_TIMES = (AT_ATIME|AT_MTIME|AT_CTIME) -AT_NOSET = (AT_NLINK|AT_RDEV|AT_FSID|AT_NODEID|AT_TYPE|\ - AT_BLKSIZE|AT_NBLOCKS|AT_VCODE) -VSUID = 0o4000 -VSGID = 0o2000 -VSVTX = 0o1000 -VREAD = 0o0400 -VWRITE = 0o0200 -VEXEC = 0o0100 -MODEMASK = 0o7777 -PERMMASK = 0o0777 -def MANDMODE(mode): return (((mode) & (VSGID|(VEXEC>>3))) == VSGID) - -VSA_ACL = 0x0001 -VSA_ACLCNT = 0x0002 -VSA_DFACL = 0x0004 -VSA_DFACLCNT = 0x0008 -LOOKUP_DIR = 0x01 -DUMP_ALLOC = 0 -DUMP_FREE = 1 -DUMP_SCAN = 2 -ATTR_UTIME = 0x01 -ATTR_EXEC = 0x02 -ATTR_COMM = 0x04 -ATTR_HINT = 0x08 -ATTR_REAL = 0x10 - -# Included from vm/faultcode.h -FC_HWERR = 0x1 -FC_ALIGN = 0x2 -FC_OBJERR = 0x3 -FC_PROT = 0x4 -FC_NOMAP = 0x5 -FC_NOSUPPORT = 0x6 -def FC_MAKE_ERR(e): return (((e) << 8) | FC_OBJERR) - -def FC_CODE(fc): return ((fc) & 0xff) - -def FC_ERRNO(fc): return ((unsigned)(fc) >> 8) - - -# Included from vm/hat.h -from TYPES import * - -# Included from vm/page.h -PAGE_HASHAVELEN = 4 -PAGE_HASHVPSHIFT = 6 -PG_EXCL = 0x0001 -PG_WAIT = 0x0002 -PG_PHYSCONTIG = 0x0004 -PG_MATCH_COLOR = 0x0008 -PG_NORELOC = 0x0010 -PG_FREE_LIST = 1 -PG_CACHE_LIST = 2 -PG_LIST_TAIL = 0 -PG_LIST_HEAD = 1 -def page_next_raw(PP): return page_nextn_raw((PP), 1) - -PAGE_IO_INUSE = 0x1 -PAGE_IO_WANTED = 0x2 -PGREL_NOTREL = 0x1 -PGREL_CLEAN = 0x2 -PGREL_MOD = 0x3 -P_FREE = 0x80 -P_NORELOC = 0x40 -def PP_SETAGED(pp): return ASSERT(PP_ISAGED(pp)) - -HAT_FLAGS_RESV = 0xFF000000 -HAT_LOAD = 0x00 -HAT_LOAD_LOCK = 0x01 -HAT_LOAD_ADV = 0x04 -HAT_LOAD_CONTIG = 0x10 -HAT_LOAD_NOCONSIST = 0x20 -HAT_LOAD_SHARE = 0x40 -HAT_LOAD_REMAP = 0x80 -HAT_RELOAD_SHARE = 0x100 -HAT_PLAT_ATTR_MASK = 0xF00000 -HAT_PROT_MASK = 0x0F -HAT_NOFAULT = 0x10 -HAT_NOSYNC = 0x20 -HAT_STRICTORDER = 0x0000 -HAT_UNORDERED_OK = 0x0100 -HAT_MERGING_OK = 0x0200 -HAT_LOADCACHING_OK = 0x0300 -HAT_STORECACHING_OK = 0x0400 -HAT_ORDER_MASK = 0x0700 -HAT_NEVERSWAP = 0x0000 -HAT_STRUCTURE_BE = 0x1000 -HAT_STRUCTURE_LE = 0x2000 -HAT_ENDIAN_MASK = 0x3000 -HAT_COW = 0x0001 -HAT_UNLOAD = 0x00 -HAT_UNLOAD_NOSYNC = 0x02 -HAT_UNLOAD_UNLOCK = 0x04 -HAT_UNLOAD_OTHER = 0x08 -HAT_UNLOAD_UNMAP = 0x10 -HAT_SYNC_DONTZERO = 0x00 -HAT_SYNC_ZERORM = 0x01 -HAT_SYNC_STOPON_REF = 0x02 -HAT_SYNC_STOPON_MOD = 0x04 -HAT_SYNC_STOPON_RM = (HAT_SYNC_STOPON_REF | HAT_SYNC_STOPON_MOD) -HAT_DUP_ALL = 1 -HAT_DUP_COW = 2 -HAT_MAP = 0x00 -HAT_ADV_PGUNLOAD = 0x00 -HAT_FORCE_PGUNLOAD = 0x01 -P_MOD = 0x1 -P_REF = 0x2 -P_RO = 0x4 -def hat_ismod(pp): return (hat_page_getattr(pp, P_MOD)) - -def hat_isref(pp): return (hat_page_getattr(pp, P_REF)) - -def hat_isro(pp): return (hat_page_getattr(pp, P_RO)) - -def hat_setmod(pp): return (hat_page_setattr(pp, P_MOD)) - -def hat_setref(pp): return (hat_page_setattr(pp, P_REF)) - -def hat_setrefmod(pp): return (hat_page_setattr(pp, P_REF|P_MOD)) - -def hat_clrmod(pp): return (hat_page_clrattr(pp, P_MOD)) - -def hat_clrref(pp): return (hat_page_clrattr(pp, P_REF)) - -def hat_clrrefmod(pp): return (hat_page_clrattr(pp, P_REF|P_MOD)) - -def hat_page_is_mapped(pp): return (hat_page_getshare(pp)) - -HAT_DONTALLOC = 0 -HAT_ALLOC = 1 -HRM_SHIFT = 4 -HRM_BYTES = (1 << HRM_SHIFT) -HRM_PAGES = ((HRM_BYTES * NBBY) / 2) -HRM_PGPERBYTE = (NBBY/2) -HRM_PGBYTEMASK = (HRM_PGPERBYTE-1) -HRM_HASHSIZE = 0x200 -HRM_HASHMASK = (HRM_HASHSIZE - 1) -HRM_BLIST_INCR = 0x200 -HRM_SWSMONID = 1 -SSL_NLEVELS = 4 -SSL_BFACTOR = 4 -SSL_LOG2BF = 2 -SEGP_ASYNC_FLUSH = 0x1 -SEGP_FORCE_WIRED = 0x2 -SEGP_SUCCESS = 0 -SEGP_FAIL = 1 -def seg_pages(seg): return \ - -IE_NOMEM = -1 -AS_PAGLCK = 0x80 -AS_CLAIMGAP = 0x40 -AS_UNMAPWAIT = 0x20 -def AS_TYPE_64BIT(as_): return \ - -AS_LREP_LINKEDLIST = 0 -AS_LREP_SKIPLIST = 1 -AS_MUTATION_THRESH = 225 -AH_DIR = 0x1 -AH_LO = 0x0 -AH_HI = 0x1 -AH_CONTAIN = 0x2 - -# Included from sys/ddidmareq.h -DMA_UNIT_8 = 1 -DMA_UNIT_16 = 2 -DMA_UNIT_32 = 4 -DMALIM_VER0 = ((0x86000000) + 0) -DDI_DMA_FORCE_PHYSICAL = 0x0100 -DMA_ATTR_V0 = 0 -DMA_ATTR_VERSION = DMA_ATTR_V0 -DDI_DMA_CALLBACK_RUNOUT = 0 -DDI_DMA_CALLBACK_DONE = 1 -DDI_DMA_WRITE = 0x0001 -DDI_DMA_READ = 0x0002 -DDI_DMA_RDWR = (DDI_DMA_READ | DDI_DMA_WRITE) -DDI_DMA_REDZONE = 0x0004 -DDI_DMA_PARTIAL = 0x0008 -DDI_DMA_CONSISTENT = 0x0010 -DDI_DMA_EXCLUSIVE = 0x0020 -DDI_DMA_STREAMING = 0x0040 -DDI_DMA_SBUS_64BIT = 0x2000 -DDI_DMA_MAPPED = 0 -DDI_DMA_MAPOK = 0 -DDI_DMA_PARTIAL_MAP = 1 -DDI_DMA_DONE = 2 -DDI_DMA_NORESOURCES = -1 -DDI_DMA_NOMAPPING = -2 -DDI_DMA_TOOBIG = -3 -DDI_DMA_TOOSMALL = -4 -DDI_DMA_LOCKED = -5 -DDI_DMA_BADLIMITS = -6 -DDI_DMA_STALE = -7 -DDI_DMA_BADATTR = -8 -DDI_DMA_INUSE = -9 -DDI_DMA_SYNC_FORDEV = 0x0 -DDI_DMA_SYNC_FORCPU = 0x1 -DDI_DMA_SYNC_FORKERNEL = 0x2 - -# Included from sys/ddimapreq.h - -# Included from sys/mman.h -PROT_READ = 0x1 -PROT_WRITE = 0x2 -PROT_EXEC = 0x4 -PROT_USER = 0x8 -PROT_ZFOD = (PROT_READ | PROT_WRITE | PROT_EXEC | PROT_USER) -PROT_ALL = (PROT_READ | PROT_WRITE | PROT_EXEC | PROT_USER) -PROT_NONE = 0x0 -MAP_SHARED = 1 -MAP_PRIVATE = 2 -MAP_TYPE = 0xf -MAP_FIXED = 0x10 -MAP_NORESERVE = 0x40 -MAP_ANON = 0x100 -MAP_ANONYMOUS = MAP_ANON -MAP_RENAME = 0x20 -PROC_TEXT = (PROT_EXEC | PROT_READ) -PROC_DATA = (PROT_READ | PROT_WRITE | PROT_EXEC) -SHARED = 0x10 -PRIVATE = 0x20 -VALID_ATTR = (PROT_READ|PROT_WRITE|PROT_EXEC|SHARED|PRIVATE) -PROT_EXCL = 0x20 -_MAP_LOW32 = 0x80 -_MAP_NEW = 0x80000000 -from TYPES import * -MADV_NORMAL = 0 -MADV_RANDOM = 1 -MADV_SEQUENTIAL = 2 -MADV_WILLNEED = 3 -MADV_DONTNEED = 4 -MADV_FREE = 5 -MS_OLDSYNC = 0x0 -MS_SYNC = 0x4 -MS_ASYNC = 0x1 -MS_INVALIDATE = 0x2 -MC_SYNC = 1 -MC_LOCK = 2 -MC_UNLOCK = 3 -MC_ADVISE = 4 -MC_LOCKAS = 5 -MC_UNLOCKAS = 6 -MCL_CURRENT = 0x1 -MCL_FUTURE = 0x2 -DDI_MAP_VERSION = 0x0001 -DDI_MF_USER_MAPPING = 0x1 -DDI_MF_KERNEL_MAPPING = 0x2 -DDI_MF_DEVICE_MAPPING = 0x4 -DDI_ME_GENERIC = (-1) -DDI_ME_UNIMPLEMENTED = (-2) -DDI_ME_NORESOURCES = (-3) -DDI_ME_UNSUPPORTED = (-4) -DDI_ME_REGSPEC_RANGE = (-5) -DDI_ME_RNUMBER_RANGE = (-6) -DDI_ME_INVAL = (-7) - -# Included from sys/ddipropdefs.h -def CELLS_1275_TO_BYTES(n): return ((n) * PROP_1275_CELL_SIZE) - -def BYTES_TO_1275_CELLS(n): return ((n) / PROP_1275_CELL_SIZE) - -PH_FROM_PROM = 0x01 -DDI_PROP_SUCCESS = 0 -DDI_PROP_NOT_FOUND = 1 -DDI_PROP_UNDEFINED = 2 -DDI_PROP_NO_MEMORY = 3 -DDI_PROP_INVAL_ARG = 4 -DDI_PROP_BUF_TOO_SMALL = 5 -DDI_PROP_CANNOT_DECODE = 6 -DDI_PROP_CANNOT_ENCODE = 7 -DDI_PROP_END_OF_DATA = 8 -DDI_PROP_FOUND_1275 = 255 -PROP_1275_INT_SIZE = 4 -DDI_PROP_DONTPASS = 0x0001 -DDI_PROP_CANSLEEP = 0x0002 -DDI_PROP_SYSTEM_DEF = 0x0004 -DDI_PROP_NOTPROM = 0x0008 -DDI_PROP_DONTSLEEP = 0x0010 -DDI_PROP_STACK_CREATE = 0x0020 -DDI_PROP_UNDEF_IT = 0x0040 -DDI_PROP_HW_DEF = 0x0080 -DDI_PROP_TYPE_INT = 0x0100 -DDI_PROP_TYPE_STRING = 0x0200 -DDI_PROP_TYPE_BYTE = 0x0400 -DDI_PROP_TYPE_COMPOSITE = 0x0800 -DDI_PROP_TYPE_ANY = (DDI_PROP_TYPE_INT | \ - DDI_PROP_TYPE_STRING | \ - DDI_PROP_TYPE_BYTE | \ - DDI_PROP_TYPE_COMPOSITE) -DDI_PROP_TYPE_MASK = (DDI_PROP_TYPE_INT | \ - DDI_PROP_TYPE_STRING | \ - DDI_PROP_TYPE_BYTE | \ - DDI_PROP_TYPE_COMPOSITE) -DDI_RELATIVE_ADDRESSING = "relative-addressing" -DDI_GENERIC_ADDRESSING = "generic-addressing" - -# Included from sys/ddidevmap.h -KMEM_PAGEABLE = 0x100 -KMEM_NON_PAGEABLE = 0x200 -UMEM_LOCKED = 0x400 -UMEM_TRASH = 0x800 -DEVMAP_OPS_REV = 1 -DEVMAP_DEFAULTS = 0x00 -DEVMAP_MAPPING_INVALID = 0x01 -DEVMAP_ALLOW_REMAP = 0x02 -DEVMAP_USE_PAGESIZE = 0x04 -DEVMAP_SETUP_FLAGS = \ - (DEVMAP_MAPPING_INVALID | DEVMAP_ALLOW_REMAP | DEVMAP_USE_PAGESIZE) -DEVMAP_SETUP_DONE = 0x100 -DEVMAP_LOCK_INITED = 0x200 -DEVMAP_FAULTING = 0x400 -DEVMAP_LOCKED = 0x800 -DEVMAP_FLAG_LARGE = 0x1000 -DDI_UMEM_SLEEP = 0x0 -DDI_UMEM_NOSLEEP = 0x01 -DDI_UMEM_PAGEABLE = 0x02 -DDI_UMEM_TRASH = 0x04 -DDI_UMEMLOCK_READ = 0x01 -DDI_UMEMLOCK_WRITE = 0x02 - -# Included from sys/nexusdefs.h - -# Included from sys/nexusintr.h -BUSO_REV = 4 -BUSO_REV_3 = 3 -BUSO_REV_4 = 4 -DEVO_REV = 3 -CB_REV = 1 -DDI_IDENTIFIED = (0) -DDI_NOT_IDENTIFIED = (-1) -DDI_PROBE_FAILURE = ENXIO -DDI_PROBE_DONTCARE = 0 -DDI_PROBE_PARTIAL = 1 -DDI_PROBE_SUCCESS = 2 -MAPDEV_REV = 1 -from TYPES import * -D_NEW = 0x00 -_D_OLD = 0x01 -D_TAPE = 0x08 -D_MTSAFE = 0x0020 -_D_QNEXTLESS = 0x0040 -_D_MTOCSHARED = 0x0080 -D_MTOCEXCL = 0x0800 -D_MTPUTSHARED = 0x1000 -D_MTPERQ = 0x2000 -D_MTQPAIR = 0x4000 -D_MTPERMOD = 0x6000 -D_MTOUTPERIM = 0x8000 -_D_MTCBSHARED = 0x10000 -D_MTINNER_MOD = (D_MTPUTSHARED|_D_MTOCSHARED|_D_MTCBSHARED) -D_MTOUTER_MOD = (D_MTOCEXCL) -D_MP = D_MTSAFE -D_64BIT = 0x200 -D_SYNCSTR = 0x400 -D_DEVMAP = 0x100 -D_HOTPLUG = 0x4 -SNDZERO = 0x001 -SNDPIPE = 0x002 -RNORM = 0x000 -RMSGD = 0x001 -RMSGN = 0x002 -RMODEMASK = 0x003 -RPROTDAT = 0x004 -RPROTDIS = 0x008 -RPROTNORM = 0x010 -RPROTMASK = 0x01c -RFLUSHMASK = 0x020 -RFLUSHPCPROT = 0x020 -RERRNORM = 0x001 -RERRNONPERSIST = 0x002 -RERRMASK = (RERRNORM|RERRNONPERSIST) -WERRNORM = 0x004 -WERRNONPERSIST = 0x008 -WERRMASK = (WERRNORM|WERRNONPERSIST) -FLUSHR = 0x01 -FLUSHW = 0x02 -FLUSHRW = 0x03 -FLUSHBAND = 0x04 -MAPINOK = 0x01 -NOMAPIN = 0x02 -REMAPOK = 0x04 -NOREMAP = 0x08 -S_INPUT = 0x0001 -S_HIPRI = 0x0002 -S_OUTPUT = 0x0004 -S_MSG = 0x0008 -S_ERROR = 0x0010 -S_HANGUP = 0x0020 -S_RDNORM = 0x0040 -S_WRNORM = S_OUTPUT -S_RDBAND = 0x0080 -S_WRBAND = 0x0100 -S_BANDURG = 0x0200 -RS_HIPRI = 0x01 -STRUIO_POSTPONE = 0x08 -STRUIO_MAPIN = 0x10 -MSG_HIPRI = 0x01 -MSG_ANY = 0x02 -MSG_BAND = 0x04 -MSG_XPG4 = 0x08 -MSG_IPEEK = 0x10 -MSG_DISCARDTAIL = 0x20 -MSG_HOLDSIG = 0x40 -MSG_IGNERROR = 0x80 -MSG_DELAYERROR = 0x100 -MSG_IGNFLOW = 0x200 -MSG_NOMARK = 0x400 -MORECTL = 1 -MOREDATA = 2 -MUXID_ALL = (-1) -ANYMARK = 0x01 -LASTMARK = 0x02 -_INFTIM = -1 -INFTIM = _INFTIM diff --git a/Lib/plat-sunos5/TYPES.py b/Lib/plat-sunos5/TYPES.py deleted file mode 100644 --- a/Lib/plat-sunos5/TYPES.py +++ /dev/null @@ -1,313 +0,0 @@ -# Generated by h2py from /usr/include/sys/types.h - -# Included from sys/isa_defs.h -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 8 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 -_ALIGNMENT_REQUIRED = 1 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 4 -_DOUBLE_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 4 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 4 -_ALIGNMENT_REQUIRED = 0 -_CHAR_ALIGNMENT = 1 -_SHORT_ALIGNMENT = 2 -_INT_ALIGNMENT = 4 -_LONG_LONG_ALIGNMENT = 8 -_DOUBLE_ALIGNMENT = 8 -_ALIGNMENT_REQUIRED = 1 -_LONG_ALIGNMENT = 4 -_LONG_DOUBLE_ALIGNMENT = 8 -_POINTER_ALIGNMENT = 4 -_MAX_ALIGNMENT = 8 -_LONG_ALIGNMENT = 8 -_LONG_DOUBLE_ALIGNMENT = 16 -_POINTER_ALIGNMENT = 8 -_MAX_ALIGNMENT = 16 - -# Included from sys/feature_tests.h -_POSIX_C_SOURCE = 1 -_LARGEFILE64_SOURCE = 1 -_LARGEFILE_SOURCE = 1 -_FILE_OFFSET_BITS = 64 -_FILE_OFFSET_BITS = 32 -_POSIX_C_SOURCE = 199506 -_POSIX_PTHREAD_SEMANTICS = 1 -_XOPEN_VERSION = 500 -_XOPEN_VERSION = 4 -_XOPEN_VERSION = 3 - -# Included from sys/machtypes.h - -# Included from sys/inttypes.h - -# Included from sys/int_types.h - -# Included from sys/int_limits.h -INT8_MAX = (127) -INT16_MAX = (32767) -INT32_MAX = (2147483647) -INTMAX_MAX = INT32_MAX -INT_LEAST8_MAX = INT8_MAX -INT_LEAST16_MAX = INT16_MAX -INT_LEAST32_MAX = INT32_MAX -INT8_MIN = (-128) -INT16_MIN = (-32767-1) -INT32_MIN = (-2147483647-1) -INTMAX_MIN = INT32_MIN -INT_LEAST8_MIN = INT8_MIN -INT_LEAST16_MIN = INT16_MIN -INT_LEAST32_MIN = INT32_MIN - -# Included from sys/int_const.h -def INT8_C(c): return (c) - -def INT16_C(c): return (c) - -def INT32_C(c): return (c) - -def INT64_C(c): return __CONCAT__(c,l) - -def INT64_C(c): return __CONCAT__(c,ll) - -def UINT8_C(c): return __CONCAT__(c,u) - -def UINT16_C(c): return __CONCAT__(c,u) - -def UINT32_C(c): return __CONCAT__(c,u) - -def UINT64_C(c): return __CONCAT__(c,ul) - -def UINT64_C(c): return __CONCAT__(c,ull) - -def INTMAX_C(c): return __CONCAT__(c,l) - -def UINTMAX_C(c): return __CONCAT__(c,ul) - -def INTMAX_C(c): return __CONCAT__(c,ll) - -def UINTMAX_C(c): return __CONCAT__(c,ull) - -def INTMAX_C(c): return (c) - -def UINTMAX_C(c): return (c) - - -# Included from sys/int_fmtio.h -PRId8 = "d" -PRId16 = "d" -PRId32 = "d" -PRId64 = "ld" -PRId64 = "lld" -PRIdLEAST8 = "d" -PRIdLEAST16 = "d" -PRIdLEAST32 = "d" -PRIdLEAST64 = "ld" -PRIdLEAST64 = "lld" -PRIi8 = "i" -PRIi16 = "i" -PRIi32 = "i" -PRIi64 = "li" -PRIi64 = "lli" -PRIiLEAST8 = "i" -PRIiLEAST16 = "i" -PRIiLEAST32 = "i" -PRIiLEAST64 = "li" -PRIiLEAST64 = "lli" -PRIo8 = "o" -PRIo16 = "o" -PRIo32 = "o" -PRIo64 = "lo" -PRIo64 = "llo" -PRIoLEAST8 = "o" -PRIoLEAST16 = "o" -PRIoLEAST32 = "o" -PRIoLEAST64 = "lo" -PRIoLEAST64 = "llo" -PRIx8 = "x" -PRIx16 = "x" -PRIx32 = "x" -PRIx64 = "lx" -PRIx64 = "llx" -PRIxLEAST8 = "x" -PRIxLEAST16 = "x" -PRIxLEAST32 = "x" -PRIxLEAST64 = "lx" -PRIxLEAST64 = "llx" -PRIX8 = "X" -PRIX16 = "X" -PRIX32 = "X" -PRIX64 = "lX" -PRIX64 = "llX" -PRIXLEAST8 = "X" -PRIXLEAST16 = "X" -PRIXLEAST32 = "X" -PRIXLEAST64 = "lX" -PRIXLEAST64 = "llX" -PRIu8 = "u" -PRIu16 = "u" -PRIu32 = "u" -PRIu64 = "lu" -PRIu64 = "llu" -PRIuLEAST8 = "u" -PRIuLEAST16 = "u" -PRIuLEAST32 = "u" -PRIuLEAST64 = "lu" -PRIuLEAST64 = "llu" -SCNd16 = "hd" -SCNd32 = "d" -SCNd64 = "ld" -SCNd64 = "lld" -SCNi16 = "hi" -SCNi32 = "i" -SCNi64 = "li" -SCNi64 = "lli" -SCNo16 = "ho" -SCNo32 = "o" -SCNo64 = "lo" -SCNo64 = "llo" -SCNu16 = "hu" -SCNu32 = "u" -SCNu64 = "lu" -SCNu64 = "llu" -SCNx16 = "hx" -SCNx32 = "x" -SCNx64 = "lx" -SCNx64 = "llx" -PRIdMAX = "ld" -PRIoMAX = "lo" -PRIxMAX = "lx" -PRIuMAX = "lu" -PRIdMAX = "lld" -PRIoMAX = "llo" -PRIxMAX = "llx" -PRIuMAX = "llu" -PRIdMAX = "d" -PRIoMAX = "o" -PRIxMAX = "x" -PRIuMAX = "u" -SCNiMAX = "li" -SCNdMAX = "ld" -SCNoMAX = "lo" -SCNxMAX = "lx" -SCNiMAX = "lli" -SCNdMAX = "lld" -SCNoMAX = "llo" -SCNxMAX = "llx" -SCNiMAX = "i" -SCNdMAX = "d" -SCNoMAX = "o" -SCNxMAX = "x" - -# Included from sys/types32.h -SHRT_MIN = (-32768) -SHRT_MAX = 32767 -USHRT_MAX = 65535 -INT_MIN = (-2147483647-1) -INT_MAX = 2147483647 -LONG_MIN = (-9223372036854775807-1) -LONG_MAX = 9223372036854775807 -LONG_MIN = (-2147483647-1) -LONG_MAX = 2147483647 -P_MYID = (-1) - -# Included from sys/select.h - -# Included from sys/time.h -TIME32_MAX = INT32_MAX -TIME32_MIN = INT32_MIN -def TIMEVAL_OVERFLOW(tv): return \ - -from TYPES import * -DST_NONE = 0 -DST_USA = 1 -DST_AUST = 2 -DST_WET = 3 -DST_MET = 4 -DST_EET = 5 -DST_CAN = 6 -DST_GB = 7 -DST_RUM = 8 -DST_TUR = 9 -DST_AUSTALT = 10 -ITIMER_REAL = 0 -ITIMER_VIRTUAL = 1 -ITIMER_PROF = 2 -ITIMER_REALPROF = 3 -def ITIMERVAL_OVERFLOW(itv): return \ - -SEC = 1 -MILLISEC = 1000 -MICROSEC = 1000000 -NANOSEC = 1000000000 - -# Included from sys/time_impl.h -def TIMESPEC_OVERFLOW(ts): return \ - -def ITIMERSPEC_OVERFLOW(it): return \ - -__CLOCK_REALTIME0 = 0 -CLOCK_VIRTUAL = 1 -CLOCK_PROF = 2 -__CLOCK_REALTIME3 = 3 -CLOCK_HIGHRES = 4 -CLOCK_MAX = 5 -CLOCK_REALTIME = __CLOCK_REALTIME3 -CLOCK_REALTIME = __CLOCK_REALTIME0 -TIMER_RELTIME = 0x0 -TIMER_ABSTIME = 0x1 - -# Included from sys/mutex.h -from TYPES import * -def MUTEX_HELD(x): return (mutex_owned(x)) - -def TICK_TO_SEC(tick): return ((tick) / hz) - -def SEC_TO_TICK(sec): return ((sec) * hz) - -def TICK_TO_MSEC(tick): return \ - -def MSEC_TO_TICK(msec): return \ - -def MSEC_TO_TICK_ROUNDUP(msec): return \ - -def TICK_TO_USEC(tick): return ((tick) * usec_per_tick) - -def USEC_TO_TICK(usec): return ((usec) / usec_per_tick) - -def USEC_TO_TICK_ROUNDUP(usec): return \ - -def TICK_TO_NSEC(tick): return ((tick) * nsec_per_tick) - -def NSEC_TO_TICK(nsec): return ((nsec) / nsec_per_tick) - -def NSEC_TO_TICK_ROUNDUP(nsec): return \ - -def TIMEVAL_TO_TICK(tvp): return \ - -def TIMESTRUC_TO_TICK(tsp): return \ - - -# Included from time.h -from TYPES import * - -# Included from iso/time_iso.h -NULL = 0 -NULL = 0 -CLOCKS_PER_SEC = 1000000 -FD_SETSIZE = 65536 -FD_SETSIZE = 1024 -_NBBY = 8 -NBBY = _NBBY -def FD_ZERO(p): return bzero((p), sizeof (*(p))) diff --git a/Lib/plat-sunos5/regen b/Lib/plat-sunos5/regen deleted file mode 100755 --- a/Lib/plat-sunos5/regen +++ /dev/null @@ -1,9 +0,0 @@ -#! /bin/sh -case `uname -sr` in -'SunOS 5.'*) ;; -*) echo Probably not on a Solaris 2 system 1>&2 - exit 1;; -esac -set -v -h2py -i '(u_long)' /usr/include/sys/types.h /usr/include/netinet/in.h /usr/include/sys/stropts.h /usr/include/dlfcn.h - diff --git a/Lib/plat-unixware7/IN.py b/Lib/plat-unixware7/IN.py deleted file mode 100644 --- a/Lib/plat-unixware7/IN.py +++ /dev/null @@ -1,836 +0,0 @@ -# Generated by h2py from /usr/include/netinet/in.h - -# Included from netinet/in_f.h -def IN_CLASSA(i): return (((int)(i) & 0x80000000) == 0) - -IN_CLASSA_NET = 0xff000000 -IN_CLASSA_NSHIFT = 24 -IN_CLASSA_HOST = 0x00ffffff -IN_CLASSA_MAX = 128 -def IN_CLASSB(i): return (((int)(i) & 0xc0000000) == 0x80000000) - -IN_CLASSB_NET = 0xffff0000 -IN_CLASSB_NSHIFT = 16 -IN_CLASSB_HOST = 0x0000ffff -IN_CLASSB_MAX = 65536 -def IN_CLASSC(i): return (((int)(i) & 0xe0000000) == 0xc0000000) - -IN_CLASSC_NET = 0xffffff00 -IN_CLASSC_NSHIFT = 8 -IN_CLASSC_HOST = 0x000000ff -def IN_CLASSD(i): return (((int)(i) & 0xf0000000) == 0xe0000000) - -IN_CLASSD_NET = 0xf0000000 -IN_CLASSD_NSHIFT = 28 -IN_CLASSD_HOST = 0x0fffffff -def IN_MULTICAST(i): return IN_CLASSD(i) - -def IN_EXPERIMENTAL(i): return (((int)(i) & 0xe0000000) == 0xe0000000) - -def IN_BADCLASS(i): return (((int)(i) & 0xf0000000) == 0xf0000000) - -INADDR_ANY = 0x00000000 -INADDR_LOOPBACK = 0x7f000001 -INADDR_BROADCAST = 0xffffffff -INADDR_NONE = 0xffffffff -IN_LOOPBACKNET = 127 -INADDR_UNSPEC_GROUP = 0xe0000000 -INADDR_ALLHOSTS_GROUP = 0xe0000001 -INADDR_ALLRTRS_GROUP = 0xe0000002 -INADDR_MAX_LOCAL_GROUP = 0xe00000ff - -# Included from netinet/in6.h - -# Included from sys/types.h -def quad_low(x): return x.val[0] - -ADT_EMASKSIZE = 8 -SHRT_MIN = -32768 -SHRT_MAX = 32767 -INT_MIN = (-2147483647-1) -INT_MAX = 2147483647 -LONG_MIN = (-2147483647-1) -LONG_MAX = 2147483647 -OFF32_MAX = LONG_MAX -ISTAT_ASSERTED = 0 -ISTAT_ASSUMED = 1 -ISTAT_NONE = 2 -OFF_MAX = OFF32_MAX -CLOCK_MAX = LONG_MAX -P_MYID = (-1) -P_MYHOSTID = (-1) - -# Included from sys/select.h -FD_SETSIZE = 4096 -NBBY = 8 -NULL = 0 - -# Included from sys/bitypes.h - -# Included from netinet/in6_f.h -def IN6_IS_ADDR_UNSPECIFIED(a): return IN6_ADDR_EQUAL_L(a, 0, 0, 0, 0) - -def IN6_SET_ADDR_UNSPECIFIED(a): return IN6_ADDR_COPY_L(a, 0, 0, 0, 0) - -def IN6_IS_ADDR_ANY(a): return IN6_ADDR_EQUAL_L(a, 0, 0, 0, 0) - -def IN6_SET_ADDR_ANY(a): return IN6_ADDR_COPY_L(a, 0, 0, 0, 0) - -def IN6_IS_ADDR_LOOPBACK(a): return IN6_ADDR_EQUAL_L(a, 0, 0, 0, 0x01000000) - -def IN6_SET_ADDR_LOOPBACK(a): return IN6_ADDR_COPY_L(a, 0, 0, 0, 0x01000000) - -IN6_MC_FLAG_PERMANENT = 0x0 -IN6_MC_FLAG_TRANSIENT = 0x1 -IN6_MC_SCOPE_NODELOCAL = 0x1 -IN6_MC_SCOPE_LINKLOCAL = 0x2 -IN6_MC_SCOPE_SITELOCAL = 0x5 -IN6_MC_SCOPE_ORGLOCAL = 0x8 -IN6_MC_SCOPE_GLOBAL = 0xE -def IN6_IS_ADDR_MC_NODELOCAL(a): return \ - -def IN6_IS_ADDR_MC_LINKLOCAL(a): return \ - -def IN6_IS_ADDR_MC_SITELOCAL(a): return \ - -def IN6_IS_ADDR_MC_ORGLOCAL(a): return \ - -def IN6_IS_ADDR_MC_GLOBAL(a): return \ - - -# Included from sys/convsa.h -__NETLIB_UW211_SVR4 = 1 -__NETLIB_UW211_XPG4 = 2 -__NETLIB_GEMINI_SVR4 = 3 -__NETLIB_GEMINI_XPG4 = 4 -__NETLIB_FP1_SVR4 = 5 -__NETLIB_FP1_XPG4 = 6 -__NETLIB_BASE_VERSION__ = __NETLIB_UW211_SVR4 -__NETLIB_VERSION__ = __NETLIB_FP1_SVR4 -__NETLIB_VERSION__ = __NETLIB_FP1_XPG4 -__NETLIB_VERSION__ = __NETLIB_GEMINI_SVR4 -__NETLIB_VERSION__ = __NETLIB_GEMINI_XPG4 -__NETLIB_VERSION__ = __NETLIB_UW211_SVR4 -__NETLIB_VERSION__ = __NETLIB_UW211_XPG4 -__NETLIB_VERSION__ = __NETLIB_FP1_XPG4 - -# Included from sys/byteorder.h -LITTLE_ENDIAN = 1234 -BIG_ENDIAN = 4321 -PDP_ENDIAN = 3412 - -# Included from sys/byteorder_f.h -BYTE_ORDER = LITTLE_ENDIAN -def htonl(hl): return __htonl(hl) - -def ntohl(nl): return __ntohl(nl) - -def htons(hs): return __htons(hs) - -def ntohs(ns): return __ntohs(ns) - -def ntohl(x): return (x) - -def ntohs(x): return (x) - -def htonl(x): return (x) - -def htons(x): return (x) - -def __NETLIB_VERSION_IS_XPG4(version): return (((version) % 2) == 0) - -def __NETLIB_VERSION_HAS_SALEN(version): return ((version) >= __NETLIB_GEMINI_SVR4) - -def __NETLIB_VERSION_IS_IKS(version): return ((version) >= __NETLIB_FP1_SVR4) - -def SA_FAMILY_GET(sa): return \ - -INET6_ADDRSTRLEN = 46 -IPV6_UNICAST_HOPS = 3 -IPV6_ADDRFORM = 24 -IPV6_MULTICAST_HOPS = 25 -IPV6_MULTICAST_IF = 26 -IPV6_MULTICAST_LOOP = 27 -IPV6_ADD_MEMBERSHIP = 28 -IPV6_DROP_MEMBERSHIP = 29 - -# Included from sys/insrem.h -def LIST_INIT(head): return \ - -def LIST_INIT(head): return \ - -def remque(a): return REMQUE(a) - - -# Included from sys/socket.h - -# Included from sys/uio.h -SHUT_RD = 0 -SHUT_WR = 1 -SHUT_RDWR = 2 - -# Included from sys/netconfig.h - -# Included from sys/cdefs.h -def __P(protos): return protos - -def __STRING(x): return #x - -def __P(protos): return () - -def __STRING(x): return "x" - -NETCONFIG = "/etc/netconfig" -NETPATH = "NETPATH" -NC_TPI_CLTS = 1 -NC_TPI_COTS = 2 -NC_TPI_COTS_ORD = 3 -NC_TPI_RAW = 4 -NC_NOFLAG = 00 -NC_VISIBLE = 0o1 -NC_BROADCAST = 0o2 -NC_NOPROTOFMLY = "-" -NC_LOOPBACK = "loopback" -NC_INET = "inet" -NC_INET6 = "inet6" -NC_IMPLINK = "implink" -NC_PUP = "pup" -NC_CHAOS = "chaos" -NC_NS = "ns" -NC_NBS = "nbs" -NC_ECMA = "ecma" -NC_DATAKIT = "datakit" -NC_CCITT = "ccitt" -NC_SNA = "sna" -NC_DECNET = "decnet" -NC_DLI = "dli" -NC_LAT = "lat" -NC_HYLINK = "hylink" -NC_APPLETALK = "appletalk" -NC_NIT = "nit" -NC_IEEE802 = "ieee802" -NC_OSI = "osi" -NC_X25 = "x25" -NC_OSINET = "osinet" -NC_GOSIP = "gosip" -NC_NETWARE = "netware" -NC_NOPROTO = "-" -NC_TCP = "tcp" -NC_UDP = "udp" -NC_ICMP = "icmp" -NC_IPX = "ipx" -NC_SPX = "spx" -NC_TPI_CLTS = 1 -NC_TPI_COTS = 2 -NC_TPI_COTS_ORD = 3 -NC_TPI_RAW = 4 -SOCK_STREAM = 2 -SOCK_DGRAM = 1 -SOCK_RAW = 4 -SOCK_RDM = 5 -SOCK_SEQPACKET = 6 -SO_DEBUG = 0x0001 -SO_ACCEPTCONN = 0x0002 -SO_REUSEADDR = 0x0004 -SO_KEEPALIVE = 0x0008 -SO_DONTROUTE = 0x0010 -SO_BROADCAST = 0x0020 -SO_USELOOPBACK = 0x0040 -SO_LINGER = 0x0080 -SO_OOBINLINE = 0x0100 -SO_ORDREL = 0x0200 -SO_IMASOCKET = 0x0400 -SO_MGMT = 0x0800 -SO_REUSEPORT = 0x1000 -SO_LISTENING = 0x2000 -SO_RDWR = 0x4000 -SO_SEMA = 0x8000 -SO_DONTLINGER = (~SO_LINGER) -SO_SNDBUF = 0x1001 -SO_RCVBUF = 0x1002 -SO_SNDLOWAT = 0x1003 -SO_RCVLOWAT = 0x1004 -SO_SNDTIMEO = 0x1005 -SO_RCVTIMEO = 0x1006 -SO_ERROR = 0x1007 -SO_TYPE = 0x1008 -SO_PROTOTYPE = 0x1009 -SO_ALLRAW = 0x100a -SOL_SOCKET = 0xffff -AF_UNSPEC = 0 -AF_UNIX = 1 -AF_LOCAL = AF_UNIX -AF_INET = 2 -AF_IMPLINK = 3 -AF_PUP = 4 -AF_CHAOS = 5 -AF_NS = 6 -AF_NBS = 7 -AF_ECMA = 8 -AF_DATAKIT = 9 -AF_CCITT = 10 -AF_SNA = 11 -AF_DECnet = 12 -AF_DLI = 13 -AF_LAT = 14 -AF_HYLINK = 15 -AF_APPLETALK = 16 -AF_NIT = 17 -AF_802 = 18 -AF_OSI = 19 -AF_ISO = AF_OSI -AF_X25 = 20 -AF_OSINET = 21 -AF_GOSIP = 22 -AF_YNET = 23 -AF_ROUTE = 24 -AF_LINK = 25 -pseudo_AF_XTP = 26 -AF_INET6 = 27 -AF_MAX = 27 -AF_INET_BSWAP = 0x0200 -PF_UNSPEC = AF_UNSPEC -PF_UNIX = AF_UNIX -PF_LOCAL = AF_LOCAL -PF_INET = AF_INET -PF_IMPLINK = AF_IMPLINK -PF_PUP = AF_PUP -PF_CHAOS = AF_CHAOS -PF_NS = AF_NS -PF_NBS = AF_NBS -PF_ECMA = AF_ECMA -PF_DATAKIT = AF_DATAKIT -PF_CCITT = AF_CCITT -PF_SNA = AF_SNA -PF_DECnet = AF_DECnet -PF_DLI = AF_DLI -PF_LAT = AF_LAT -PF_HYLINK = AF_HYLINK -PF_APPLETALK = AF_APPLETALK -PF_NIT = AF_NIT -PF_802 = AF_802 -PF_OSI = AF_OSI -PF_ISO = PF_OSI -PF_X25 = AF_X25 -PF_OSINET = AF_OSINET -PF_GOSIP = AF_GOSIP -PF_YNET = AF_YNET -PF_ROUTE = AF_ROUTE -PF_LINK = AF_LINK -pseudo_PF_XTP = pseudo_AF_XTP -PF_INET6 = AF_INET6 -PF_MAX = AF_MAX -SOMAXCONN = 5 -SCM_RIGHTS = 1 -MSG_OOB = 0x1 -MSG_PEEK = 0x2 -MSG_DONTROUTE = 0x4 -MSG_CTRUNC = 0x8 -MSG_TRUNC = 0x10 -MSG_EOR = 0x30 -MSG_WAITALL = 0x20 -MSG_MAXIOVLEN = 16 -def OPTLEN(x): return ((((x) + sizeof(int) - 1) / sizeof(int)) * sizeof(int)) - -GIARG = 0x1 -CONTI = 0x2 -GITAB = 0x4 -SOCKETSYS = 88 -SOCKETSYS = 83 -SO_ACCEPT = 1 -SO_BIND = 2 -SO_CONNECT = 3 -SO_GETPEERNAME = 4 -SO_GETSOCKNAME = 5 -SO_GETSOCKOPT = 6 -SO_LISTEN = 7 -SO_RECV = 8 -SO_RECVFROM = 9 -SO_SEND = 10 -SO_SENDTO = 11 -SO_SETSOCKOPT = 12 -SO_SHUTDOWN = 13 -SO_SOCKET = 14 -SO_SOCKPOLL = 15 -SO_GETIPDOMAIN = 16 -SO_SETIPDOMAIN = 17 -SO_ADJTIME = 18 - -# Included from sys/stream.h - -# Included from sys/cred.h - -# Included from sys/ksynch.h - -# Included from sys/dl.h -SIGNBIT = 0x80000000 - -# Included from sys/ipl.h - -# Included from sys/disp_p.h - -# Included from sys/trap.h -DIVERR = 0 -SGLSTP = 1 -NMIFLT = 2 -BPTFLT = 3 -INTOFLT = 4 -BOUNDFLT = 5 -INVOPFLT = 6 -NOEXTFLT = 7 -DBLFLT = 8 -EXTOVRFLT = 9 -INVTSSFLT = 10 -SEGNPFLT = 11 -STKFLT = 12 -GPFLT = 13 -PGFLT = 14 -EXTERRFLT = 16 -ALIGNFLT = 17 -MCEFLT = 18 -USERFLT = 0x100 -TRP_PREEMPT = 0x200 -TRP_UNUSED = 0x201 -PF_ERR_MASK = 0x01 -PF_ERR_PAGE = 0 -PF_ERR_PROT = 1 -PF_ERR_WRITE = 2 -PF_ERR_USER = 4 -EVT_STRSCHED = 0x04 -EVT_GLOBCALLOUT = 0x08 -EVT_LCLCALLOUT = 0x10 -EVT_SOFTINTMASK = (EVT_STRSCHED|EVT_GLOBCALLOUT|EVT_LCLCALLOUT) -PL0 = 0 -PL1 = 1 -PL2 = 2 -PL3 = 3 -PL4 = 4 -PL5 = 5 -PL6 = 6 -PLHI = 8 -PL7 = PLHI -PLBASE = PL0 -PLTIMEOUT = PL1 -PLDISK = PL5 -PLSTR = PL6 -PLTTY = PLSTR -PLMIN = PL0 -PLMIN = PL1 -MAX_INTR_LEVELS = 10 -MAX_INTR_NESTING = 50 -STRSCHED = EVT_STRSCHED -GLOBALSOFTINT = EVT_GLOBCALLOUT -LOCALSOFTINT = EVT_LCLCALLOUT - -# Included from sys/ksynch_p.h -def GET_TIME(timep): return \ - -LK_THRESHOLD = 500000 - -# Included from sys/list.h - -# Included from sys/listasm.h -def remque_null(e): return \ - -def LS_ISEMPTY(listp): return \ - -LK_BASIC = 0x1 -LK_SLEEP = 0x2 -LK_NOSTATS = 0x4 -def CYCLES_SINCE(c): return CYCLES_BETWEEN((c), CYCLES()) - -LSB_NLKDS = 92 -EVT_RUNRUN = 0x01 -EVT_KPRUNRUN = 0x02 -SP_UNLOCKED = 0 -SP_LOCKED = 1 -KS_LOCKTEST = 0x01 -KS_MPSTATS = 0x02 -KS_DEINITED = 0x04 -KS_NVLTTRACE = 0x08 -RWS_READ = (ord('r')) -RWS_WRITE = (ord('w')) -RWS_UNLOCKED = (ord('u')) -RWS_BUSY = (ord('b')) -def SLEEP_LOCKOWNED(lkp): return \ - -def SLEEP_DISOWN(lkp): return \ - -KS_NOPRMPT = 0x00000001 -__KS_LOCKTEST = KS_LOCKTEST -__KS_LOCKTEST = 0 -__KS_MPSTATS = KS_MPSTATS -__KS_MPSTATS = 0 -__KS_NVLTTRACE = KS_NVLTTRACE -__KS_NVLTTRACE = 0 -KSFLAGS = (__KS_LOCKTEST|__KS_MPSTATS|__KS_NVLTTRACE) -KSVUNIPROC = 1 -KSVMPDEBUG = 2 -KSVMPNODEBUG = 3 -KSVFLAG = KSVUNIPROC -KSVFLAG = KSVMPDEBUG -KSVFLAG = KSVMPNODEBUG - -# Included from sys/ksinline.h -_A_SP_LOCKED = 1 -_A_SP_UNLOCKED = 0 -_A_INVPL = -1 -def _ATOMIC_INT_INCR(atomic_intp): return \ - -def _ATOMIC_INT_DECR(atomic_intp): return \ - -def ATOMIC_INT_READ(atomic_intp): return _ATOMIC_INT_READ(atomic_intp) - -def ATOMIC_INT_INCR(atomic_intp): return _ATOMIC_INT_INCR(atomic_intp) - -def ATOMIC_INT_DECR(atomic_intp): return _ATOMIC_INT_DECR(atomic_intp) - -def FSPIN_INIT(lp): return - -def FSPIN_LOCK(l): return DISABLE() - -def FSPIN_TRYLOCK(l): return (DISABLE(), B_TRUE) - -def FSPIN_UNLOCK(l): return ENABLE() - -def LOCK_DEINIT(lp): return - -def LOCK_DEALLOC(lp): return - -def LOCK_OWNED(lp): return (B_TRUE) - -def RW_DEINIT(lp): return - -def RW_DEALLOC(lp): return - -def RW_OWNED(lp): return (B_TRUE) - -def IS_LOCKED(lockp): return B_FALSE - -def LOCK_PLMIN(lockp): return \ - -def TRYLOCK_PLMIN(lockp): return LOCK_PLMIN(lockp) - -def LOCK_SH_PLMIN(lockp): return LOCK_PLMIN(lockp) - -def RW_RDLOCK_PLMIN(lockp): return LOCK_PLMIN(lockp) - -def RW_WRLOCK_PLMIN(lockp): return LOCK_PLMIN(lockp) - -def LOCK_DEINIT(l): return - -def LOCK_PLMIN(lockp): return LOCK((lockp), PLMIN) - -def TRYLOCK_PLMIN(lockp): return TRYLOCK((lockp), PLMIN) - -def LOCK_SH_PLMIN(lockp): return LOCK_SH((lockp), PLMIN) - -def RW_RDLOCK_PLMIN(lockp): return RW_RDLOCK((lockp), PLMIN) - -def RW_WRLOCK_PLMIN(lockp): return RW_WRLOCK((lockp), PLMIN) - -def FSPIN_IS_LOCKED(fsp): return B_FALSE - -def SPIN_IS_LOCKED(lockp): return B_FALSE - -def FSPIN_OWNED(l): return (B_TRUE) - -CR_MLDREAL = 0x00000001 -CR_RDUMP = 0x00000002 -def crhold(credp): return crholdn((credp), 1) - -def crfree(credp): return crfreen((credp), 1) - - -# Included from sys/strmdep.h -def str_aligned(X): return (((uint)(X) & (sizeof(int) - 1)) == 0) - - -# Included from sys/engine.h - -# Included from sys/clock.h - -# Included from sys/time.h -DST_NONE = 0 -DST_USA = 1 -DST_AUST = 2 -DST_WET = 3 -DST_MET = 4 -DST_EET = 5 -DST_CAN = 6 -DST_GB = 7 -DST_RUM = 8 -DST_TUR = 9 -DST_AUSTALT = 10 -ITIMER_REAL = 0 -ITIMER_VIRTUAL = 1 -ITIMER_PROF = 2 -FD_SETSIZE = 4096 -FD_NBBY = 8 - -# Included from time.h -NULL = 0 -CLOCKS_PER_SEC = 1000000 - -# Included from sys/clock_p.h -CGBITS = 4 -IDBITS = 28 -def toid_unpackcg(idval): return (((idval) >> IDBITS) & 0xf) - -def toid_unpackid(idval): return ((idval) & 0xfffffff) - -def toid_unpackcg(idval): return 0 - -def toid_unpackid(idval): return (idval) - -NCALLOUT_HASH = 1024 -CALLOUT_MAXVAL = 0x7fffffff -TO_PERIODIC = 0x80000000 -TO_IMMEDIATE = 0x80000000 -SEC = 1 -MILLISEC = 1000 -MICROSEC = 1000000 -NANOSEC = 1000000000 -SECHR = (60*60) -SECDAY = (24*SECHR) -SECYR = (365*SECDAY) -def TIME_OWNED_R(cgnum): return (B_TRUE) - -LOOPSECONDS = 1800 -LOOPMICROSECONDS = (LOOPSECONDS * MICROSEC) -def TICKS_SINCE(t): return TICKS_BETWEEN(t, TICKS()) - -MAXRQS = 2 -E_OFFLINE = 0x01 -E_BAD = 0x02 -E_SHUTDOWN = 0x04 -E_DRIVER = 0x08 -E_DEFAULTKEEP = 0x100 -E_DRIVERBOUND = 0x200 -E_EXCLUSIVE = 0x400 -E_CGLEADER = 0x800 -E_NOWAY = (E_OFFLINE|E_BAD|E_SHUTDOWN) -E_BOUND = 0x01 -E_GLOBAL = 0x00 -E_UNAVAIL = -1 -ENGINE_ONLINE = 1 -def PROCESSOR_UNMAP(e): return ((e) - engine) - -BOOTENG = 0 -QMOVED = 0x0001 -QWANTR = 0x0002 -QWANTW = 0x0004 -QFULL = 0x0008 -QREADR = 0x0010 -QUSE = 0x0020 -QNOENB = 0x0040 -QUP = 0x0080 -QBACK = 0x0100 -QINTER = 0x0200 -QPROCSON = 0x0400 -QTOENAB = 0x0800 -QFREEZE = 0x1000 -QBOUND = 0x2000 -QDEFCNT = 0x4000 -QENAB = 0x0001 -QSVCBUSY = 0x0002 -STRM_PUTCNT_TABLES = 31 -def STRM_MYENG_PUTCNT(sdp): return STRM_PUTCNT(l.eng_num, sdp) - -QB_FULL = 0x01 -QB_WANTW = 0x02 -QB_BACK = 0x04 -NBAND = 256 -DB_WASDUPED = 0x1 -DB_2PIECE = 0x2 -STRLEAKHASHSZ = 1021 -MSGMARK = 0x01 -MSGNOLOOP = 0x02 -MSGDELIM = 0x04 -MSGNOGET = 0x08 -MSGLOG = 0x10 -M_DATA = 0x00 -M_PROTO = 0x01 -M_BREAK = 0x08 -M_PASSFP = 0x09 -M_SIG = 0x0b -M_DELAY = 0x0c -M_CTL = 0x0d -M_IOCTL = 0x0e -M_SETOPTS = 0x10 -M_RSE = 0x11 -M_TRAIL = 0x12 -M_IOCACK = 0x81 -M_IOCNAK = 0x82 -M_PCPROTO = 0x83 -M_PCSIG = 0x84 -M_READ = 0x85 -M_FLUSH = 0x86 -M_STOP = 0x87 -M_START = 0x88 -M_HANGUP = 0x89 -M_ERROR = 0x8a -M_COPYIN = 0x8b -M_COPYOUT = 0x8c -M_IOCDATA = 0x8d -M_PCRSE = 0x8e -M_STOPI = 0x8f -M_STARTI = 0x90 -M_PCCTL = 0x91 -M_PCSETOPTS = 0x92 -QNORM = 0x00 -QPCTL = 0x80 -STRCANON = 0x01 -RECOPY = 0x02 -SO_ALL = 0x003f -SO_READOPT = 0x0001 -SO_WROFF = 0x0002 -SO_MINPSZ = 0x0004 -SO_MAXPSZ = 0x0008 -SO_HIWAT = 0x0010 -SO_LOWAT = 0x0020 -SO_MREADON = 0x0040 -SO_MREADOFF = 0x0080 -SO_NDELON = 0x0100 -SO_NDELOFF = 0x0200 -SO_ISTTY = 0x0400 -SO_ISNTTY = 0x0800 -SO_TOSTOP = 0x1000 -SO_TONSTOP = 0x2000 -SO_BAND = 0x4000 -SO_DELIM = 0x8000 -SO_NODELIM = 0x010000 -SO_STRHOLD = 0x020000 -SO_LOOP = 0x040000 -DRVOPEN = 0x0 -MODOPEN = 0x1 -CLONEOPEN = 0x2 -OPENFAIL = -1 -BPRI_LO = 1 -BPRI_MED = 2 -BPRI_HI = 3 -INFPSZ = -1 -FLUSHALL = 1 -FLUSHDATA = 0 -STRHIGH = 5120 -STRLOW = 1024 -MAXIOCBSZ = 1024 -def straln(a): return (caddr_t)((int)(a) & ~(sizeof(int)-1)) - -IPM_ID = 200 -ICMPM_ID = 201 -TCPM_ID = 202 -UDPM_ID = 203 -ARPM_ID = 204 -APPM_ID = 205 -RIPM_ID = 206 -PPPM_ID = 207 -AHDLCM_ID = 208 -MHDLCRIPM_ID = 209 -HDLCM_ID = 210 -PPCID_ID = 211 -IGMPM_ID = 212 -IPIPM_ID = 213 -IPPROTO_IP = 0 -IPPROTO_HOPOPTS = 0 -IPPROTO_ICMP = 1 -IPPROTO_IGMP = 2 -IPPROTO_GGP = 3 -IPPROTO_IPIP = 4 -IPPROTO_TCP = 6 -IPPROTO_EGP = 8 -IPPROTO_PUP = 12 -IPPROTO_UDP = 17 -IPPROTO_IDP = 22 -IPPROTO_TP = 29 -IPPROTO_IPV6 = 41 -IPPROTO_ROUTING = 43 -IPPROTO_FRAGMENT = 44 -IPPROTO_ESP = 50 -IPPROTO_AH = 51 -IPPROTO_ICMPV6 = 58 -IPPROTO_NONE = 59 -IPPROTO_DSTOPTS = 60 -IPPROTO_HELLO = 63 -IPPROTO_ND = 77 -IPPROTO_EON = 80 -IPPROTO_RAW = 255 -IPPROTO_MAX = 256 -IPPORT_ECHO = 7 -IPPORT_DISCARD = 9 -IPPORT_SYSTAT = 11 -IPPORT_DAYTIME = 13 -IPPORT_NETSTAT = 15 -IPPORT_FTP = 21 -IPPORT_TELNET = 23 -IPPORT_SMTP = 25 -IPPORT_TIMESERVER = 37 -IPPORT_NAMESERVER = 42 -IPPORT_WHOIS = 43 -IPPORT_MTP = 57 -IPPORT_TFTP = 69 -IPPORT_RJE = 77 -IPPORT_FINGER = 79 -IPPORT_TTYLINK = 87 -IPPORT_SUPDUP = 95 -IPPORT_EXECSERVER = 512 -IPPORT_LOGINSERVER = 513 -IPPORT_CMDSERVER = 514 -IPPORT_EFSSERVER = 520 -IPPORT_BIFFUDP = 512 -IPPORT_WHOSERVER = 513 -IPPORT_ROUTESERVER = 520 -IPPORT_RESERVED = 1024 -IPPORT_USERRESERVED = 65535 -IPPORT_RESERVED_LOW = 512 -IPPORT_RESERVED_HIGH = 1023 -IPPORT_USERRESERVED_LOW = 32768 -IPPORT_USERRESERVED_HIGH = 65535 -INET_ADDRSTRLEN = 16 -IP_OPTIONS = 1 -IP_TOS = 2 -IP_TTL = 3 -IP_HDRINCL = 4 -IP_RECVOPTS = 5 -IP_RECVRETOPTS = 6 -IP_RECVDSTADDR = 7 -IP_RETOPTS = 8 -IP_MULTICAST_IF = 9 -IP_MULTICAST_LOOP = 10 -IP_ADD_MEMBERSHIP = 11 -IP_DROP_MEMBERSHIP = 12 -IP_BROADCAST_IF = 14 -IP_RECVIFINDEX = 15 -IP_MULTICAST_TTL = 16 -MRT_INIT = 17 -MRT_DONE = 18 -MRT_ADD_VIF = 19 -MRT_DEL_VIF = 20 -MRT_ADD_MFC = 21 -MRT_DEL_MFC = 22 -MRT_VERSION = 23 -IP_DEFAULT_MULTICAST_TTL = 1 -IP_DEFAULT_MULTICAST_LOOP = 1 -IP_MAX_MEMBERSHIPS = 20 -INADDR_UNSPEC_GROUP = 0xe0000000 -INADDR_ALLHOSTS_GROUP = 0xe0000001 -INADDR_ALLRTRS_GROUP = 0xe0000002 -INADDR_MAX_LOCAL_GROUP = 0xe00000ff - -# Included from netinet/in_mp.h - -# Included from netinet/in_mp_ddi.h - -# Included from sys/inline.h -IP_HIER_BASE = (20) -def ASSERT_LOCK(x): return - -def ASSERT_WRLOCK(x): return - -def ASSERT_UNLOCK(x): return - -def CANPUT(q): return canput((q)) - -def CANPUTNEXT(q): return canputnext((q)) - -INET_DEBUG = 1 diff --git a/Lib/plat-unixware7/STROPTS.py b/Lib/plat-unixware7/STROPTS.py deleted file mode 100644 --- a/Lib/plat-unixware7/STROPTS.py +++ /dev/null @@ -1,328 +0,0 @@ -# Generated by h2py from /usr/include/sys/stropts.h - -# Included from sys/types.h -def quad_low(x): return x.val[0] - -ADT_EMASKSIZE = 8 -SHRT_MIN = -32768 -SHRT_MAX = 32767 -INT_MIN = (-2147483647-1) -INT_MAX = 2147483647 -LONG_MIN = (-2147483647-1) -LONG_MAX = 2147483647 -OFF32_MAX = LONG_MAX -ISTAT_ASSERTED = 0 -ISTAT_ASSUMED = 1 -ISTAT_NONE = 2 -OFF_MAX = OFF32_MAX -CLOCK_MAX = LONG_MAX -P_MYID = (-1) -P_MYHOSTID = (-1) - -# Included from sys/select.h -FD_SETSIZE = 4096 -NBBY = 8 -NULL = 0 - -# Included from sys/conf.h -D_NEW = 0x00 -D_OLD = 0x01 -D_DMA = 0x02 -D_BLKOFF = 0x400 -D_LFS = 0x8000 -D_STR = 0x0800 -D_MOD = 0x1000 -D_PSEUDO = 0x2000 -D_RANDOM = 0x4000 -D_HOT = 0x10000 -D_SEEKNEG = 0x04 -D_TAPE = 0x08 -D_NOBRKUP = 0x10 -D_INITPUB = 0x20 -D_NOSPECMACDATA = 0x40 -D_RDWEQ = 0x80 -SECMASK = (D_INITPUB|D_NOSPECMACDATA|D_RDWEQ) -DAF_REQDMA = 0x1 -DAF_PHYSREQ = 0x2 -DAF_PRE8 = 0x4 -DAF_STATIC = 0x8 -DAF_STR = 0x10 -D_MP = 0x100 -D_UPF = 0x200 -ROOTFS_NAMESZ = 7 -FMNAMESZ = 8 -MCD_VERSION = 1 -DI_BCBP = 0 -DI_MEDIA = 1 - -# Included from sys/secsys.h -ES_MACOPENLID = 1 -ES_MACSYSLID = 2 -ES_MACROOTLID = 3 -ES_PRVINFO = 4 -ES_PRVSETCNT = 5 -ES_PRVSETS = 6 -ES_MACADTLID = 7 -ES_PRVID = 8 -ES_TPGETMAJOR = 9 -SA_EXEC = 0o01 -SA_WRITE = 0o02 -SA_READ = 0o04 -SA_SUBSIZE = 0o10 - -# Included from sys/stropts_f.h -X_STR = (ord('S')<<8) -X_I_BASE = (X_STR|0o200) -X_I_NREAD = (X_STR|0o201) -X_I_PUSH = (X_STR|0o202) -X_I_POP = (X_STR|0o203) -X_I_LOOK = (X_STR|0o204) -X_I_FLUSH = (X_STR|0o205) -X_I_SRDOPT = (X_STR|0o206) -X_I_GRDOPT = (X_STR|0o207) -X_I_STR = (X_STR|0o210) -X_I_SETSIG = (X_STR|0o211) -X_I_GETSIG = (X_STR|0o212) -X_I_FIND = (X_STR|0o213) -X_I_LINK = (X_STR|0o214) -X_I_UNLINK = (X_STR|0o215) -X_I_PEEK = (X_STR|0o217) -X_I_FDINSERT = (X_STR|0o220) -X_I_SENDFD = (X_STR|0o221) -X_I_RECVFD = (X_STR|0o222) - -# Included from unistd.h - -# Included from sys/unistd.h -R_OK = 0o04 -W_OK = 0o02 -X_OK = 0o01 -F_OK = 000 -EFF_ONLY_OK = 0o10 -EX_OK = 0o20 -SEEK_SET = 0 -SEEK_CUR = 1 -SEEK_END = 2 -_SC_ARG_MAX = 1 -_SC_CHILD_MAX = 2 -_SC_CLK_TCK = 3 -_SC_NGROUPS_MAX = 4 -_SC_OPEN_MAX = 5 -_SC_JOB_CONTROL = 6 -_SC_SAVED_IDS = 7 -_SC_VERSION = 8 -_SC_PASS_MAX = 9 -_SC_LOGNAME_MAX = 10 -_SC_PAGESIZE = 11 -_SC_PAGE_SIZE = _SC_PAGESIZE -_SC_XOPEN_VERSION = 12 -_SC_NACLS_MAX = 13 -_SC_NPROCESSORS_CONF = 14 -_SC_NPROCESSORS_ONLN = 15 -_SC_NPROCESSES = 39 -_SC_TOTAL_MEMORY = 40 -_SC_USEABLE_MEMORY = 41 -_SC_GENERAL_MEMORY = 42 -_SC_DEDICATED_MEMORY = 43 -_SC_NCGS_CONF = 44 -_SC_NCGS_ONLN = 45 -_SC_MAX_CPUS_PER_CG = 46 -_SC_CG_SIMPLE_IMPL = 47 -_SC_CACHE_LINE = 48 -_SC_SYSTEM_ID = 49 -_SC_THREADS = 51 -_SC_THREAD_ATTR_STACKADDR = 52 -_SC_THREAD_ATTR_STACKSIZE = 53 -_SC_THREAD_DESTRUCTOR_ITERATIONS = 54 -_SC_THREAD_KEYS_MAX = 55 -_SC_THREAD_PRIORITY_SCHEDULING = 56 -_SC_THREAD_PRIO_INHERIT = 57 -_SC_THREAD_PRIO_PROTECT = 58 -_SC_THREAD_STACK_MIN = 59 -_SC_THREAD_PROCESS_SHARED = 60 -_SC_THREAD_SAFE_FUNCTIONS = 61 -_SC_THREAD_THREADS_MAX = 62 -_SC_KERNEL_VM = 63 -_SC_TZNAME_MAX = 320 -_SC_STREAM_MAX = 321 -_SC_XOPEN_CRYPT = 323 -_SC_XOPEN_ENH_I18N = 324 -_SC_XOPEN_SHM = 325 -_SC_XOPEN_XCU_VERSION = 327 -_SC_AES_OS_VERSION = 330 -_SC_ATEXIT_MAX = 331 -_SC_2_C_BIND = 350 -_SC_2_C_DEV = 351 -_SC_2_C_VERSION = 352 -_SC_2_CHAR_TERM = 353 -_SC_2_FORT_DEV = 354 -_SC_2_FORT_RUN = 355 -_SC_2_LOCALEDEF = 356 -_SC_2_SW_DEV = 357 -_SC_2_UPE = 358 -_SC_2_VERSION = 359 -_SC_BC_BASE_MAX = 370 -_SC_BC_DIM_MAX = 371 -_SC_BC_SCALE_MAX = 372 -_SC_BC_STRING_MAX = 373 -_SC_COLL_WEIGHTS_MAX = 380 -_SC_EXPR_NEST_MAX = 381 -_SC_LINE_MAX = 382 -_SC_RE_DUP_MAX = 383 -_SC_IOV_MAX = 390 -_SC_NPROC_CONF = 391 -_SC_NPROC_ONLN = 392 -_SC_XOPEN_UNIX = 400 -_SC_SEMAPHORES = 440 -_CS_PATH = 1 -__O_CS_HOSTNAME = 2 -_CS_RELEASE = 3 -_CS_VERSION = 4 -__O_CS_MACHINE = 5 -__O_CS_ARCHITECTURE = 6 -_CS_HW_SERIAL = 7 -__O_CS_HW_PROVIDER = 8 -_CS_SRPC_DOMAIN = 9 -_CS_INITTAB_NAME = 10 -__O_CS_SYSNAME = 11 -_CS_LFS_CFLAGS = 20 -_CS_LFS_LDFLAGS = 21 -_CS_LFS_LIBS = 22 -_CS_LFS_LINTFLAGS = 23 -_CS_LFS64_CFLAGS = 24 -_CS_LFS64_LDFLAGS = 25 -_CS_LFS64_LIBS = 26 -_CS_LFS64_LINTFLAGS = 27 -_CS_ARCHITECTURE = 100 -_CS_BUSTYPES = 101 -_CS_HOSTNAME = 102 -_CS_HW_PROVIDER = 103 -_CS_KERNEL_STAMP = 104 -_CS_MACHINE = 105 -_CS_OS_BASE = 106 -_CS_OS_PROVIDER = 107 -_CS_SYSNAME = 108 -_CS_USER_LIMIT = 109 -_PC_LINK_MAX = 1 -_PC_MAX_CANON = 2 -_PC_MAX_INPUT = 3 -_PC_NAME_MAX = 4 -_PC_PATH_MAX = 5 -_PC_PIPE_BUF = 6 -_PC_NO_TRUNC = 7 -_PC_VDISABLE = 8 -_PC_CHOWN_RESTRICTED = 9 -_PC_FILESIZEBITS = 10 -_POSIX_VERSION = 199009 -_XOPEN_VERSION = 4 -GF_PATH = "/etc/group" -PF_PATH = "/etc/passwd" -F_ULOCK = 0 -F_LOCK = 1 -F_TLOCK = 2 -F_TEST = 3 -_POSIX_JOB_CONTROL = 1 -_POSIX_SAVED_IDS = 1 -_POSIX_VDISABLE = 0 -NULL = 0 -STDIN_FILENO = 0 -STDOUT_FILENO = 1 -STDERR_FILENO = 2 -_XOPEN_UNIX = 1 -_XOPEN_ENH_I18N = 1 -_XOPEN_XPG4 = 1 -_POSIX2_C_VERSION = 199209 -_POSIX2_VERSION = 199209 -_XOPEN_XCU_VERSION = 4 -_POSIX_SEMAPHORES = 1 -_POSIX_THREADS = 1 -_POSIX_THREAD_ATTR_STACKADDR = 1 -_POSIX_THREAD_ATTR_STACKSIZE = 1 -_POSIX_THREAD_PRIORITY_SCHEDULING = 1 -_POSIX_THREAD_PROCESS_SHARED = 1 -_POSIX_THREAD_SAFE_FUNCTIONS = 1 -_POSIX2_C_BIND = 1 -_POSIX2_CHAR_TERM = 1 -_POSIX2_FORT_RUN = 1 -_POSIX2_LOCALEDEF = 1 -_POSIX2_UPE = 1 -_LFS_ASYNCHRONOUS_IO = 1 -_LFS_LARGEFILE = 1 -_LFS64_ASYNCHRONOUS_IO = 1 -_LFS64_LARGEFILE = 1 -_LFS64_STDIO = 1 -FMNAMESZ = 8 -SNDZERO = 0x001 -SNDPIPE = 0x002 -RNORM = 0x000 -RMSGD = 0x001 -RMSGN = 0x002 -RMODEMASK = 0x003 -RPROTDAT = 0x004 -RPROTDIS = 0x008 -RPROTNORM = 0x010 -RPROTMASK = 0x01c -FLUSHR = 0x01 -FLUSHW = 0x02 -FLUSHRW = 0x03 -FLUSHBAND = 0x04 -S_INPUT = 0x0001 -S_HIPRI = 0x0002 -S_OUTPUT = 0x0004 -S_MSG = 0x0008 -S_ERROR = 0x0010 -S_HANGUP = 0x0020 -S_RDNORM = 0x0040 -S_WRNORM = S_OUTPUT -S_RDBAND = 0x0080 -S_WRBAND = 0x0100 -S_BANDURG = 0x0200 -RS_HIPRI = 0x01 -MSG_HIPRI = 0x01 -MSG_ANY = 0x02 -MSG_BAND = 0x04 -MSG_DISCARD = 0x08 -MSG_PEEKIOCTL = 0x10 -MORECTL = 1 -MOREDATA = 2 -MUXID_ALL = (-1) -ANYMARK = 0x01 -LASTMARK = 0x02 -STR = (ord('S')<<8) -I_NREAD = (STR|0o1) -I_PUSH = (STR|0o2) -I_POP = (STR|0o3) -I_LOOK = (STR|0o4) -I_FLUSH = (STR|0o5) -I_SRDOPT = (STR|0o6) -I_GRDOPT = (STR|0o7) -I_STR = (STR|0o10) -I_SETSIG = (STR|0o11) -I_GETSIG = (STR|0o12) -I_FIND = (STR|0o13) -I_LINK = (STR|0o14) -I_UNLINK = (STR|0o15) -I_PEEK = (STR|0o17) -I_FDINSERT = (STR|0o20) -I_SENDFD = (STR|0o21) -I_RECVFD = (STR|0o22) -I_E_RECVFD = (STR|0o16) -I_RECVFD = (STR|0o16) -I_RECVFD = (STR|0o22) -I_SWROPT = (STR|0o23) -I_GWROPT = (STR|0o24) -I_LIST = (STR|0o25) -I_PLINK = (STR|0o26) -I_PUNLINK = (STR|0o27) -I_FLUSHBAND = (STR|0o34) -I_CKBAND = (STR|0o35) -I_GETBAND = (STR|0o36) -I_ATMARK = (STR|0o37) -I_SETCLTIME = (STR|0o40) -I_GETCLTIME = (STR|0o41) -I_CANPUT = (STR|0o42) -I_S_RECVFD = (STR|0o43) -I_STATS = (STR|0o44) -I_BIGPIPE = (STR|0o45) -I_GETTP = (STR|0o46) -INFTIM = -1 diff --git a/Lib/plat-unixware7/regen b/Lib/plat-unixware7/regen deleted file mode 100755 --- a/Lib/plat-unixware7/regen +++ /dev/null @@ -1,9 +0,0 @@ -#! /bin/sh -case `uname -sr` in -UnixWare*) ;; -*) echo Probably not on a UnixWare system 1>&2 - exit 1;; -esac -set -v -h2py -i '(u_long)' /usr/include/netinet/in.h -h2py /usr/include/sys/stropts.h diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1310,29 +1310,8 @@ -PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \ $(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/PatternGrammar.txt -# Create the PLATDIR source directory, if one wasn't distributed.. -# For multiarch targets, use the plat-linux/regen script. $(srcdir)/Lib/$(PLATDIR): mkdir $(srcdir)/Lib/$(PLATDIR) - if [ -n "$(MULTIARCH)" ]; then \ - cp $(srcdir)/Lib/plat-linux/regen $(srcdir)/Lib/$(PLATDIR)/regen; \ - else \ - cp $(srcdir)/Lib/plat-generic/regen $(srcdir)/Lib/$(PLATDIR)/regen; \ - fi; \ - export PATH; PATH="`pwd`:$$PATH"; \ - export PYTHONPATH; PYTHONPATH="`pwd`/Lib"; \ - export DYLD_FRAMEWORK_PATH; DYLD_FRAMEWORK_PATH="`pwd`"; \ - export EXE; EXE="$(BUILDEXE)"; \ - export CC; CC="$(CC)"; \ - if [ -n "$(MULTIARCH)" ]; then export MULTIARCH; MULTIARCH=$(MULTIARCH); fi; \ - export PYTHON_FOR_BUILD; \ - if [ "$(BUILD_GNU_TYPE)" = "$(HOST_GNU_TYPE)" ]; then \ - PYTHON_FOR_BUILD="$(BUILDPYTHON)"; \ - else \ - PYTHON_FOR_BUILD="$(PYTHON_FOR_BUILD)"; \ - fi; \ - export H2PY; H2PY="$$PYTHON_FOR_BUILD $(abs_srcdir)/Tools/scripts/h2py.py"; \ - cd $(srcdir)/Lib/$(PLATDIR); $(RUNSHARED) ./regen python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh # Substitution happens here, as the completely-expanded BINDIR @@ -1657,7 +1636,6 @@ -o -name '*.tmSnippet' \ -o -name 'Setup' \ -o -name 'Setup.*' \ - -o -name regen \ -o -name README \ -o -name NEWS \ -o -name HISTORY \ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -95,6 +95,9 @@ Library ------- +- Issue #28027: Remove undocumented modules from ``Lib/plat-*``: IN, CDROM, + DLFCN, TYPES, CDIO, and STROPTS. + - Issue #27445: Don't pass str(_charset) to MIMEText.set_payload(). Patch by Claude Paroz. diff --git a/README b/README --- a/README +++ b/README @@ -46,16 +46,17 @@ (This will fail if you *also* built at the top-level directory. You should do a "make clean" at the toplevel first.) -If you need an optimized version of Python, you type "make profile-opt" in the -top level directory. This will rebuild the interpreter executable using Profile -Guided Optimization (PGO). For more details, see the section bellow. +To get an optimized build of Python, "configure --with-optimizations" before +you run make. This sets the default make targets up to enable Profile Guided +Optimization (PGO) and Link Time Optimization (LTO) on most platforms. +For more details, see the sections bellow. Profile Guided Optimization --------------------------- PGO takes advantage of recent versions of the GCC or Clang compilers. -If ran, the "profile-opt" rule will do several steps. +If ran, "make profile-opt" will do several steps. First, the entire Python directory is cleaned of temporary files that may have resulted in a previous compilation. @@ -75,6 +76,14 @@ that is optimized and suitable for distribution or production installation. +Link Time Optimization +---------------------- + +LTO takes advantages of recent compiler toolchains ability to optimize across +the otherwise arbitrary .o file boundary when building final executables or +shared libraries for additional performance gains. + + What's New ---------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:56:06 2016 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 08 Sep 2016 18:56:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327350=3A_Add_cred?= =?utf-8?q?its?= Message-ID: <20160908185605.48584.90564.7F59C4B2@psf.io> https://hg.python.org/cpython/rev/9434f511937d changeset: 103341:9434f511937d user: Raymond Hettinger date: Thu Sep 08 11:55:38 2016 -0700 summary: Issue #27350: Add credits files: Misc/NEWS | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -12,6 +12,7 @@ - Issue #27350: `dict` implementation is changed like PyPy. It is more compact and preserves insertion order. + (Concept developed by Raymond Hettinger and patch by Inada Naoki.) - Issue #27911: Remove unnecessary error checks in ``exec_builtin_or_dynamic()``. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:57:02 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 18:57:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_use_static_inline_instead_?= =?utf-8?q?of_Py=5FLOCAL=5FINLINE?= Message-ID: <20160908185702.87494.19239.22D7CB59@psf.io> https://hg.python.org/cpython/rev/50cb3c6f5f72 changeset: 103342:50cb3c6f5f72 user: Benjamin Peterson date: Thu Sep 08 11:56:06 2016 -0700 summary: use static inline instead of Py_LOCAL_INLINE files: Objects/memoryobject.c | 32 +++++++++++++++--------------- 1 files changed, 16 insertions(+), 16 deletions(-) diff --git a/Objects/memoryobject.c b/Objects/memoryobject.c --- a/Objects/memoryobject.c +++ b/Objects/memoryobject.c @@ -57,7 +57,7 @@ } -Py_LOCAL_INLINE(_PyManagedBufferObject *) +static inline _PyManagedBufferObject * mbuf_alloc(void) { _PyManagedBufferObject *mbuf; @@ -243,7 +243,7 @@ #define HAVE_SUBOFFSETS_IN_LAST_DIM(view) \ (view->suboffsets && view->suboffsets[dest->ndim-1] >= 0) -Py_LOCAL_INLINE(int) +static inline int last_dim_is_contiguous(const Py_buffer *dest, const Py_buffer *src) { assert(dest->ndim > 0 && src->ndim > 0); @@ -259,7 +259,7 @@ assignments, where the lvalue is already known to have a single character format. This is a performance hack that could be rewritten (if properly benchmarked). */ -Py_LOCAL_INLINE(int) +static inline int equiv_format(const Py_buffer *dest, const Py_buffer *src) { const char *dfmt, *sfmt; @@ -279,7 +279,7 @@ /* Two shapes are equivalent if they are either equal or identical up to a zero element at the same position. For example, in NumPy arrays the shapes [1, 0, 5] and [1, 0, 7] are equivalent. */ -Py_LOCAL_INLINE(int) +static inline int equiv_shape(const Py_buffer *dest, const Py_buffer *src) { int i; @@ -438,7 +438,7 @@ } /* Initialize strides for a C-contiguous array. */ -Py_LOCAL_INLINE(void) +static inline void init_strides_from_shape(Py_buffer *view) { Py_ssize_t i; @@ -451,7 +451,7 @@ } /* Initialize strides for a Fortran-contiguous array. */ -Py_LOCAL_INLINE(void) +static inline void init_fortran_strides_from_shape(Py_buffer *view) { Py_ssize_t i; @@ -513,7 +513,7 @@ /****************************************************************************/ /* Initialize values that are shared with the managed buffer. */ -Py_LOCAL_INLINE(void) +static inline void init_shared_values(Py_buffer *dest, const Py_buffer *src) { dest->obj = src->obj; @@ -553,7 +553,7 @@ } } -Py_LOCAL_INLINE(void) +static inline void init_suboffsets(Py_buffer *dest, const Py_buffer *src) { Py_ssize_t i; @@ -567,7 +567,7 @@ } /* len = product(shape) * itemsize */ -Py_LOCAL_INLINE(void) +static inline void init_len(Py_buffer *view) { Py_ssize_t i, len; @@ -614,7 +614,7 @@ /* Allocate a new memoryview and perform basic initialization. New memoryviews are exclusively created through the mbuf_add functions. */ -Py_LOCAL_INLINE(PyMemoryViewObject *) +static inline PyMemoryViewObject * memory_alloc(int ndim) { PyMemoryViewObject *mv; @@ -1099,7 +1099,7 @@ #define IS_BYTE_FORMAT(f) (f == 'b' || f == 'B' || f == 'c') -Py_LOCAL_INLINE(Py_ssize_t) +static inline Py_ssize_t get_native_fmtchar(char *result, const char *fmt) { Py_ssize_t size = -1; @@ -1127,7 +1127,7 @@ return -1; } -Py_LOCAL_INLINE(const char *) +static inline const char * get_native_fmtstr(const char *fmt) { int at = 0; @@ -1642,7 +1642,7 @@ /* Unpack a single item. 'fmt' can be any native format character in struct module syntax. This function is very sensitive to small changes. With this layout gcc automatically generates a fast jump table. */ -Py_LOCAL_INLINE(PyObject *) +static inline PyObject * unpack_single(const char *ptr, const char *fmt) { unsigned long long llu; @@ -1998,7 +1998,7 @@ /****************************************************************************/ /* allow explicit form of native format */ -Py_LOCAL_INLINE(const char *) +static inline const char * adjust_fmt(const Py_buffer *view) { const char *fmt; @@ -2280,7 +2280,7 @@ return unpack_single(ptr, fmt); } -Py_LOCAL_INLINE(int) +static inline int init_slice(Py_buffer *base, PyObject *key, int dim) { Py_ssize_t start, stop, step, slicelength; @@ -2602,7 +2602,7 @@ equal = (x == y); \ } while (0) -Py_LOCAL_INLINE(int) +static inline int unpack_cmp(const char *p, const char *q, char fmt, struct unpacker *unpack_p, struct unpacker *unpack_q) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 14:58:51 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 18:58:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_make_some_peps_high_level_?= =?utf-8?q?sections?= Message-ID: <20160908185851.18971.98748.7B53D078@psf.io> https://hg.python.org/cpython/rev/e701ca63897e changeset: 103343:e701ca63897e user: Benjamin Peterson date: Thu Sep 08 11:58:40 2016 -0700 summary: make some peps high level sections files: Doc/whatsnew/3.6.rst | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -223,7 +223,7 @@ .. _pep-529: PEP 529: Change Windows filesystem encoding to UTF-8 ----------------------------------------------------- +==================================================== Representing filesystem paths is best performed with str (Unicode) rather than bytes. However, there are some situations where using bytes is sufficient and @@ -249,7 +249,7 @@ encoding may change before the final release. PEP 487: Simpler customization of class creation ------------------------------------------------- +================================================ Upon subclassing a class, the ``__init_subclass__`` classmethod (if defined) is called on the base class. This makes it straightforward to write classes that @@ -269,7 +269,7 @@ PYTHONMALLOC environment variable ---------------------------------- +================================= The new :envvar:`PYTHONMALLOC` environment variable allows setting the Python memory allocators and/or install debug hooks. @@ -345,7 +345,7 @@ .. _whatsnew-deforder: PEP 520: Preserving Class Attribute Definition Order ----------------------------------------------------- +==================================================== Attributes in a class definition body have a natural ordering: the same order in which the names appear in the source. This order is now -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 15:03:10 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 08 Sep 2016 19:03:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Reindeint_DK=5Fxxx_macros?= Message-ID: <20160908190310.87325.7077.81C0D8DE@psf.io> https://hg.python.org/cpython/rev/fc8aaa073eb4 changeset: 103345:fc8aaa073eb4 user: Victor Stinner date: Thu Sep 08 11:37:36 2016 -0700 summary: Reindeint DK_xxx macros Issue #27350. files: Objects/dictobject.c | 17 +++++++++++------ 1 files changed, 11 insertions(+), 6 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -281,14 +281,19 @@ #define DK_SIZE(dk) ((dk)->dk_size) #if SIZEOF_VOID_P > 4 -#define DK_IXSIZE(dk) (DK_SIZE(dk) <= 0xff ? 1 : DK_SIZE(dk) <= 0xffff ? 2 : \ - DK_SIZE(dk) <= 0xffffffff ? 4 : sizeof(Py_ssize_t)) +#define DK_IXSIZE(dk) \ + (DK_SIZE(dk) <= 0xff ? \ + 1 : DK_SIZE(dk) <= 0xffff ? \ + 2 : DK_SIZE(dk) <= 0xffffffff ? \ + 4 : sizeof(Py_ssize_t)) #else -#define DK_IXSIZE(dk) (DK_SIZE(dk) <= 0xff ? 1 : DK_SIZE(dk) <= 0xffff ? 2 : \ - sizeof(Py_ssize_t)) +#define DK_IXSIZE(dk) \ + (DK_SIZE(dk) <= 0xff ? \ + 1 : DK_SIZE(dk) <= 0xffff ? \ + 2 : sizeof(Py_ssize_t)) #endif -#define DK_ENTRIES(dk) ((PyDictKeyEntry*)(&(dk)->dk_indices[DK_SIZE(dk) * \ - DK_IXSIZE(dk)])) +#define DK_ENTRIES(dk) \ + ((PyDictKeyEntry*)(&(dk)->dk_indices[DK_SIZE(dk) * DK_IXSIZE(dk)])) #define DK_DEBUG_INCREF _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA #define DK_DEBUG_DECREF _Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 15:03:10 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 08 Sep 2016 19:03:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_dk=5Fget=5Findex/dk=5Fset?= =?utf-8?q?=5Findex_uses_a_type_indices_variable?= Message-ID: <20160908190310.8701.78944.501FFC39@psf.io> https://hg.python.org/cpython/rev/48a1f97d03b4 changeset: 103344:48a1f97d03b4 user: Victor Stinner date: Thu Sep 08 11:35:46 2016 -0700 summary: dk_get_index/dk_set_index uses a type indices variable Issue #27350. files: Objects/dictobject.c | 24 ++++++++++++++++-------- 1 files changed, 16 insertions(+), 8 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -307,18 +307,22 @@ Py_ssize_t ix; if (s <= 0xff) { - ix = ((char*) &keys->dk_indices[0])[i]; + char *indices = (char*)keys->dk_indices; + ix = indices[i]; } else if (s <= 0xffff) { - ix = ((int16_t*)&keys->dk_indices[0])[i]; + int16_t *indices = (int16_t*)keys->dk_indices; + ix = indices[i]; } #if SIZEOF_VOID_P > 4 else if (s <= 0xffffffff) { - ix = ((int32_t*)&keys->dk_indices[0])[i]; + int32_t *indices = (int32_t*)keys->dk_indices; + ix = indices[i]; } #endif else { - ix = ((Py_ssize_t*)&keys->dk_indices[0])[i]; + Py_ssize_t *indices = (Py_ssize_t*)keys->dk_indices; + ix = indices[i]; } assert(ix >= DKIX_DUMMY); return ix; @@ -333,21 +337,25 @@ assert(ix >= DKIX_DUMMY); if (s <= 0xff) { + char *indices = (char*)keys->dk_indices; assert(ix <= 0x7f); - ((char*) &keys->dk_indices[0])[i] = (char)ix; + indices[i] = (char)ix; } else if (s <= 0xffff) { + int16_t *indices = (int16_t*)keys->dk_indices; assert(ix <= 0x7fff); - ((int16_t*) &keys->dk_indices[0])[i] = (int16_t)ix; + indices[i] = (int16_t)ix; } #if SIZEOF_VOID_P > 4 else if (s <= 0xffffffff) { + int32_t *indices = (int32_t*)keys->dk_indices; assert(ix <= 0x7fffffff); - ((int32_t*) &keys->dk_indices[0])[i] = (int32_t)ix; + indices[i] = (int32_t)ix; } #endif else { - ((Py_ssize_t*) &keys->dk_indices[0])[i] = ix; + Py_ssize_t *indices = (Py_ssize_t*)keys->dk_indices; + indices[i] = ix; } } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 15:03:11 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 08 Sep 2016 19:03:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_documentation_to_the_d?= =?utf-8?q?ict_implementation?= Message-ID: <20160908190310.2439.84.5BE514E9@psf.io> https://hg.python.org/cpython/rev/378e000a6878 changeset: 103346:378e000a6878 user: Victor Stinner date: Thu Sep 08 12:01:25 2016 -0700 summary: Add documentation to the dict implementation Issue #27350. files: Include/dictobject.h | 9 ++++++ Objects/dict-common.h | 43 +++++++++++++++++++++++++++++- Objects/dictobject.c | 2 +- 3 files changed, 51 insertions(+), 3 deletions(-) diff --git a/Include/dictobject.h b/Include/dictobject.h --- a/Include/dictobject.h +++ b/Include/dictobject.h @@ -22,8 +22,17 @@ */ typedef struct { PyObject_HEAD + + /* Number of items in the dictionary */ Py_ssize_t ma_used; + PyDictKeysObject *ma_keys; + + /* If ma_values is NULL, the table is "combined": keys and values + are stored in ma_keys (and ma_keys->dk_refcnt == 1). + + If ma_values is not NULL, the table is splitted: + keys are stored in ma_keys and values are stored in ma_values */ PyObject **ma_values; } PyDictObject; diff --git a/Objects/dict-common.h b/Objects/dict-common.h --- a/Objects/dict-common.h +++ b/Objects/dict-common.h @@ -22,11 +22,50 @@ /* See dictobject.c for actual layout of DictKeysObject */ struct _dictkeysobject { Py_ssize_t dk_refcnt; + + /* Size of the hash table (dk_indices). It must be a power of 2. */ Py_ssize_t dk_size; + + /* Function to lookup in the hash table (dk_indices): + + - lookdict(): general-purpose, and may return DKIX_ERROR if (and + only if) a comparison raises an exception. + + - lookdict_unicode(): specialized to Unicode string keys, comparison of + which can never raise an exception; that function can never return + DKIX_ERROR. + + - lookdict_unicode_nodummy(): similar to lookdict_unicode() but further + specialized for Unicode string keys that cannot be the value. + + - lookdict_split(): Version of lookdict() for split tables. */ dict_lookup_func dk_lookup; + + /* Number of usable entries in dk_entries. + 0 <= dk_usable <= USABLE_FRACTION(dk_size) */ Py_ssize_t dk_usable; - Py_ssize_t dk_nentries; /* How many entries are used. */ - char dk_indices[8]; /* dynamically sized. 8 is minimum. */ + + /* Number of used entries in dk_entries. + 0 <= dk_nentries < dk_size */ + Py_ssize_t dk_nentries; + + /* Actual hash table of dk_size entries. It holds indices in dk_entries, + or DKIX_EMPTY(-1) or DKIX_DUMMY(-2). + + Indices must be: 0 <= indice < USABLE_FRACTION(dk_size). + + The size in bytes of an indice depends on dk_size: + + - 1 byte if dk_size <= 0xff (char*) + - 2 bytes if dk_size <= 0xffff (int16_t*) + - 4 bytes if dk_size <= 0xffffffff (int32_t*) + - 8 bytes otherwise (Py_ssize_t*) + + Dynamically sized, 8 is minimum. */ + char dk_indices[8]; + + /* "PyDictKeyEntry dk_entries[dk_usable];" array follows: + see the DK_ENTRIES() macro */ }; #endif diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -593,7 +593,7 @@ Christian Tismer. lookdict() is general-purpose, and may return DKIX_ERROR if (and only if) a -comparison raises an exception (this was new in Python 2.5). +comparison raises an exception. lookdict_unicode() below is specialized to string keys, comparison of which can never raise an exception; that function can never return DKIX_ERROR. lookdict_unicode_nodummy is further specialized for string keys that cannot be -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 15:20:39 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 19:20:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_access_dk=5Findices_throug?= =?utf-8?q?h_a_union?= Message-ID: <20160908192039.69465.79181.ADCE37DA@psf.io> https://hg.python.org/cpython/rev/2f0289c5ee73 changeset: 103347:2f0289c5ee73 user: Benjamin Peterson date: Thu Sep 08 12:20:12 2016 -0700 summary: access dk_indices through a union files: Objects/dict-common.h | 7 ++++++- Objects/dictobject.c | 28 ++++++++++++---------------- 2 files changed, 18 insertions(+), 17 deletions(-) diff --git a/Objects/dict-common.h b/Objects/dict-common.h --- a/Objects/dict-common.h +++ b/Objects/dict-common.h @@ -62,7 +62,12 @@ - 8 bytes otherwise (Py_ssize_t*) Dynamically sized, 8 is minimum. */ - char dk_indices[8]; + union { + int8_t as_1[8]; + int16_t as_2[4]; + int32_t as_4[2]; + int64_t as_8[1]; + } dk_indices; /* "PyDictKeyEntry dk_entries[dk_usable];" array follows: see the DK_ENTRIES() macro */ diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -293,7 +293,7 @@ 2 : sizeof(Py_ssize_t)) #endif #define DK_ENTRIES(dk) \ - ((PyDictKeyEntry*)(&(dk)->dk_indices[DK_SIZE(dk) * DK_IXSIZE(dk)])) + ((PyDictKeyEntry*)(&(dk)->dk_indices.as_1[DK_SIZE(dk) * DK_IXSIZE(dk)])) #define DK_DEBUG_INCREF _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA #define DK_DEBUG_DECREF _Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA @@ -312,21 +312,19 @@ Py_ssize_t ix; if (s <= 0xff) { - char *indices = (char*)keys->dk_indices; + int8_t *indices = keys->dk_indices.as_1; ix = indices[i]; } else if (s <= 0xffff) { - int16_t *indices = (int16_t*)keys->dk_indices; + int16_t *indices = keys->dk_indices.as_2; ix = indices[i]; } -#if SIZEOF_VOID_P > 4 else if (s <= 0xffffffff) { - int32_t *indices = (int32_t*)keys->dk_indices; + int32_t *indices = keys->dk_indices.as_4; ix = indices[i]; } -#endif else { - Py_ssize_t *indices = (Py_ssize_t*)keys->dk_indices; + int64_t *indices = keys->dk_indices.as_8; ix = indices[i]; } assert(ix >= DKIX_DUMMY); @@ -342,24 +340,22 @@ assert(ix >= DKIX_DUMMY); if (s <= 0xff) { - char *indices = (char*)keys->dk_indices; + int8_t *indices = keys->dk_indices.as_1; assert(ix <= 0x7f); indices[i] = (char)ix; } else if (s <= 0xffff) { - int16_t *indices = (int16_t*)keys->dk_indices; + int16_t *indices = keys->dk_indices.as_2; assert(ix <= 0x7fff); indices[i] = (int16_t)ix; } -#if SIZEOF_VOID_P > 4 else if (s <= 0xffffffff) { - int32_t *indices = (int32_t*)keys->dk_indices; + int32_t *indices = keys->dk_indices.as_4; assert(ix <= 0x7fffffff); indices[i] = (int32_t)ix; } -#endif else { - Py_ssize_t *indices = (Py_ssize_t*)keys->dk_indices; + int64_t *indices = keys->dk_indices.as_8; indices[i] = ix; } } @@ -418,8 +414,8 @@ lookdict_split, /* dk_lookup */ 0, /* dk_usable (immutable) */ 0, /* dk_nentries */ - {DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, - DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY}, /* dk_indices */ + .dk_indices = { .as_1 = {DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, + DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY}}, }; static PyObject *empty_values[1] = { NULL }; @@ -468,7 +464,7 @@ dk->dk_usable = usable; dk->dk_lookup = lookdict_unicode_nodummy; dk->dk_nentries = 0; - memset(&dk->dk_indices[0], 0xff, es * size); + memset(&dk->dk_indices.as_1[0], 0xff, es * size); memset(DK_ENTRIES(dk), 0, sizeof(PyDictKeyEntry) * usable); return dk; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 15:34:34 2016 From: python-checkins at python.org (r.david.murray) Date: Thu, 08 Sep 2016 19:34:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2327364=3A_Deprecate_inva?= =?utf-8?q?lid_escape_strings_in_str/byutes=2E?= Message-ID: <20160908193433.8543.65517.3D5180EC@psf.io> https://hg.python.org/cpython/rev/38802c38cfe1 changeset: 103348:38802c38cfe1 user: R David Murray date: Thu Sep 08 15:34:08 2016 -0400 summary: #27364: Deprecate invalid escape strings in str/byutes. Patch by Emanuel Barry, reviewed by Serhiy Storchaka and Martin Panter. files: Doc/reference/lexical_analysis.rst | 4 ++ Doc/whatsnew/3.6.rst | 5 ++ Lib/test/test_codecs.py | 35 ++++++++++++----- Lib/test/test_unicode.py | 7 +++ Misc/NEWS | 3 + Objects/bytesobject.c | 3 +- Objects/unicodeobject.c | 3 + 7 files changed, 48 insertions(+), 12 deletions(-) diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -560,6 +560,10 @@ escape sequences only recognized in string literals fall into the category of unrecognized escapes for bytes literals. + .. versionchanged:: 3.6 + Unrecognized escape sequences produce a DeprecationWarning. In + some future version of Python they will be a SyntaxError. + Even in a raw literal, quotes can be escaped with a backslash, but the backslash remains in the result; for example, ``r"\""`` is a valid string literal consisting of two characters: a backslash and a double quote; ``r"\"`` diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -952,6 +952,11 @@ parameter will be dropped in a future Python release and likely earlier through third party tools. See :issue:`27919` for details. +* A backslash-character pair that is not a valid escape sequence now generates + a DeprecationWarning. Although this will eventually become a SyntaxError, + that will not be for several Python releases. (Contributed by Emanuel Barry + in :issue:`27364`.) + Deprecated Python behavior -------------------------- diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -1175,7 +1175,7 @@ check(b"[\\\n]", b"[]") check(br'[\"]', b'["]') check(br"[\']", b"[']") - check(br"[\\]", br"[\]") + check(br"[\\]", b"[\\]") check(br"[\a]", b"[\x07]") check(br"[\b]", b"[\x08]") check(br"[\t]", b"[\x09]") @@ -1184,7 +1184,6 @@ check(br"[\f]", b"[\x0c]") check(br"[\r]", b"[\x0d]") check(br"[\7]", b"[\x07]") - check(br"[\8]", br"[\8]") check(br"[\78]", b"[\x078]") check(br"[\41]", b"[!]") check(br"[\418]", b"[!8]") @@ -1192,12 +1191,18 @@ check(br"[\1010]", b"[A0]") check(br"[\501]", b"[A]") check(br"[\x41]", b"[A]") - check(br"[\X41]", br"[\X41]") check(br"[\x410]", b"[A0]") - for b in range(256): - if b not in b'\n"\'\\abtnvfr01234567x': - b = bytes([b]) - check(b'\\' + b, b'\\' + b) + for i in range(97, 123): + b = bytes([i]) + if b not in b'abfnrtvx': + with self.assertWarns(DeprecationWarning): + check(b"\\" + b, b"\\" + b) + with self.assertWarns(DeprecationWarning): + check(b"\\" + b.upper(), b"\\" + b.upper()) + with self.assertWarns(DeprecationWarning): + check(br"\8", b"\\8") + with self.assertWarns(DeprecationWarning): + check(br"\9", b"\\9") def test_errors(self): decode = codecs.escape_decode @@ -2448,7 +2453,6 @@ check(br"[\f]", "[\x0c]") check(br"[\r]", "[\x0d]") check(br"[\7]", "[\x07]") - check(br"[\8]", r"[\8]") check(br"[\78]", "[\x078]") check(br"[\41]", "[!]") check(br"[\418]", "[!8]") @@ -2458,9 +2462,18 @@ check(br"[\x410]", "[A0]") check(br"\u20ac", "\u20ac") check(br"\U0001d120", "\U0001d120") - for b in range(256): - if b not in b'\n"\'\\abtnvfr01234567xuUN': - check(b'\\' + bytes([b]), '\\' + chr(b)) + for i in range(97, 123): + b = bytes([i]) + if b not in b'abfnrtuvx': + with self.assertWarns(DeprecationWarning): + check(b"\\" + b, "\\" + chr(i)) + if b.upper() not in b'UN': + with self.assertWarns(DeprecationWarning): + check(b"\\" + b.upper(), "\\" + chr(i-32)) + with self.assertWarns(DeprecationWarning): + check(br"\8", "\\8") + with self.assertWarns(DeprecationWarning): + check(br"\9", "\\9") def test_decode_errors(self): decode = codecs.unicode_escape_decode diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -10,6 +10,7 @@ import itertools import operator import struct +import string import sys import unittest import warnings @@ -2752,6 +2753,12 @@ support.check_free_after_iterating(self, iter, str) support.check_free_after_iterating(self, reversed, str) + def test_invalid_sequences(self): + for letter in string.ascii_letters + "89": # 0-7 are octal escapes + if letter in "abfnrtuvxNU": + continue + with self.assertWarns(DeprecationWarning): + eval(r"'\%s'" % letter) class StringModuleTest(unittest.TestCase): def test_formatter_parser(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27364: A backslash-character pair that is not a valid escape sequence + now generates a DeprecationWarning. + - Issue #27350: `dict` implementation is changed like PyPy. It is more compact and preserves insertion order. (Concept developed by Raymond Hettinger and patch by Inada Naoki.) diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1207,8 +1207,9 @@ break; default: + if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, "invalid escape sequence '\\%c'", *(--s)) < 0) + goto failed; *p++ = '\\'; - s--; goto non_esc; /* an arbitrary number of unescaped UTF-8 bytes may follow. */ } diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -6065,6 +6065,9 @@ goto error; default: + if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, + "invalid escape sequence '\\%c'", c) < 0) + goto onError; WRITE_ASCII_CHAR('\\'); WRITE_CHAR(c); continue; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 15:47:45 2016 From: python-checkins at python.org (davin.potts) Date: Thu, 08 Sep 2016 19:47:45 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIxMjAx?= =?utf-8?q?=3A_Improves_readability_of_multiprocessing_error_message_from_?= =?utf-8?q?server?= Message-ID: <20160908194745.12492.43305.5698DAFF@psf.io> https://hg.python.org/cpython/rev/9b1f8c68de4c changeset: 103349:9b1f8c68de4c branch: 3.5 parent: 103334:e1987bf14148 user: Davin Potts date: Thu Sep 08 14:40:36 2016 -0500 summary: Issue #21201: Improves readability of multiprocessing error message from server to client for certain exceptions files: Lib/multiprocessing/managers.py | 2 +- Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -275,7 +275,7 @@ try: send(msg) except Exception as e: - send(('#UNSERIALIZABLE', repr(msg))) + send(('#UNSERIALIZABLE', format_exc())) except Exception as e: util.info('exception in thread serving %r', threading.current_thread().name) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -221,6 +221,9 @@ - Issue #27930: Improved behaviour of logging.handlers.QueueListener. Thanks to Paulo Andrade and Petr Viktorin for the analysis and patch. +- Issue #21201: Improves readability of multiprocessing error message. Thanks + to Wojciech Walczak for patch. + IDLE ---- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 15:47:46 2016 From: python-checkins at python.org (davin.potts) Date: Thu, 08 Sep 2016 19:47:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2321201=3A_Improves_readability_of_multiprocessin?= =?utf-8?q?g_error_message_from_server?= Message-ID: <20160908194745.59502.88926.C7CF9758@psf.io> https://hg.python.org/cpython/rev/199aca18d9a1 changeset: 103350:199aca18d9a1 parent: 103348:38802c38cfe1 parent: 103349:9b1f8c68de4c user: Davin Potts date: Thu Sep 08 14:47:23 2016 -0500 summary: Issue #21201: Improves readability of multiprocessing error message from server to client for certain exceptions files: Lib/multiprocessing/managers.py | 2 +- Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -283,7 +283,7 @@ try: send(msg) except Exception as e: - send(('#UNSERIALIZABLE', repr(msg))) + send(('#UNSERIALIZABLE', format_exc())) except Exception as e: util.info('exception in thread serving %r', threading.current_thread().name) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -248,6 +248,9 @@ - Issue #6766: Distributed reference counting added to multiprocessing to support nesting of shared values / proxy objects. +- Issue #21201: Improves readability of multiprocessing error message. Thanks + to Wojciech Walczak for patch. + C API ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 16:06:38 2016 From: python-checkins at python.org (victor.stinner) Date: Thu, 08 Sep 2016 20:06:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_a_new_private_version_?= =?utf-8?q?to_the_builtin_dict_type?= Message-ID: <20160908200635.48474.2718.775982DC@psf.io> https://hg.python.org/cpython/rev/d43f819caea7 changeset: 103351:d43f819caea7 user: Victor Stinner date: Thu Sep 08 12:51:24 2016 -0700 summary: Add a new private version to the builtin dict type Issue #26058: Add a new private version to the builtin dict type, incremented at each dictionary creation and at each dictionary change. Implementation of the PEP 509. files: Doc/whatsnew/3.6.rst | 10 + Include/dictobject.h | 4 + Lib/test/test_ordered_dict.py | 2 +- Lib/test/test_pep509.py | 186 ++++++++++++++++++++++ Lib/test/test_sys.py | 6 +- Misc/NEWS | 4 + Modules/_testcapimodule.c | 16 + Objects/dictobject.c | 19 ++ 8 files changed, 243 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -367,6 +367,16 @@ PEP written and implemented by Eric Snow. +PEP 509: Add a private version to dict +-------------------------------------- + +Add a new private version to the builtin ``dict`` type, incremented at +each dictionary creation and at each dictionary change, to implement +fast guards on namespaces. + +(Contributed by Victor Stinner in :issue:`26058`.) + + Other Language Changes ====================== diff --git a/Include/dictobject.h b/Include/dictobject.h --- a/Include/dictobject.h +++ b/Include/dictobject.h @@ -26,6 +26,10 @@ /* Number of items in the dictionary */ Py_ssize_t ma_used; + /* Dictionary version: globally unique, value change each time + the dictionary is modified */ + uint64_t ma_version_tag; + PyDictKeysObject *ma_keys; /* If ma_values is NULL, the table is "combined": keys and values diff --git a/Lib/test/test_ordered_dict.py b/Lib/test/test_ordered_dict.py --- a/Lib/test/test_ordered_dict.py +++ b/Lib/test/test_ordered_dict.py @@ -655,7 +655,7 @@ size = support.calcobjsize check = self.check_sizeof - basicsize = size('n2P' + '3PnPn2P') + calcsize('2nP2n') + basicsize = size('n2P3PnPn2P') + 8 + calcsize('2nP2n') entrysize = calcsize('n2P') p = calcsize('P') nodesize = calcsize('Pn2P') diff --git a/Lib/test/test_pep509.py b/Lib/test/test_pep509.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_pep509.py @@ -0,0 +1,186 @@ +""" +Test implementation of the PEP 509: dictionary versionning. +""" +import unittest +from test import support + +# PEP 509 is implemented in CPython but other Python implementations +# don't require to implement it +_testcapi = support.import_module('_testcapi') + + +class DictVersionTests(unittest.TestCase): + type2test = dict + + def setUp(self): + self.seen_versions = set() + self.dict = None + + def check_version_unique(self, mydict): + version = _testcapi.dict_get_version(mydict) + self.assertNotIn(version, self.seen_versions) + self.seen_versions.add(version) + + def check_version_changed(self, mydict, method, *args, **kw): + result = method(*args, **kw) + self.check_version_unique(mydict) + return result + + def check_version_dont_change(self, mydict, method, *args, **kw): + version1 = _testcapi.dict_get_version(mydict) + self.seen_versions.add(version1) + + result = method(*args, **kw) + + version2 = _testcapi.dict_get_version(mydict) + self.assertEqual(version2, version1, "version changed") + + return result + + def new_dict(self, *args, **kw): + d = self.type2test(*args, **kw) + self.check_version_unique(d) + return d + + def test_constructor(self): + # new empty dictionaries must all have an unique version + empty1 = self.new_dict() + empty2 = self.new_dict() + empty3 = self.new_dict() + + # non-empty dictionaries must also have an unique version + nonempty1 = self.new_dict(x='x') + nonempty2 = self.new_dict(x='x', y='y') + + def test_copy(self): + d = self.new_dict(a=1, b=2) + + d2 = self.check_version_dont_change(d, d.copy) + + # dict.copy() must create a dictionary with a new unique version + self.check_version_unique(d2) + + def test_setitem(self): + d = self.new_dict() + + # creating new keys must change the version + self.check_version_changed(d, d.__setitem__, 'x', 'x') + self.check_version_changed(d, d.__setitem__, 'y', 'y') + + # changing values must change the version + self.check_version_changed(d, d.__setitem__, 'x', 1) + self.check_version_changed(d, d.__setitem__, 'y', 2) + + def test_setitem_same_value(self): + value = object() + d = self.new_dict() + + # setting a key must change the version + self.check_version_changed(d, d.__setitem__, 'key', value) + + # setting a key to the same value with dict.__setitem__ + # must change the version + self.check_version_changed(d, d.__setitem__, 'key', value) + + # setting a key to the same value with dict.update + # must change the version + self.check_version_changed(d, d.update, key=value) + + d2 = self.new_dict(key=value) + self.check_version_changed(d, d.update, d2) + + def test_setitem_equal(self): + class AlwaysEqual: + def __eq__(self, other): + return True + + value1 = AlwaysEqual() + value2 = AlwaysEqual() + self.assertTrue(value1 == value2) + self.assertFalse(value1 != value2) + + d = self.new_dict() + self.check_version_changed(d, d.__setitem__, 'key', value1) + + # setting a key to a value equal to the current value + # with dict.__setitem__() must change the version + self.check_version_changed(d, d.__setitem__, 'key', value2) + + # setting a key to a value equal to the current value + # with dict.update() must change the version + self.check_version_changed(d, d.update, key=value1) + + d2 = self.new_dict(key=value2) + self.check_version_changed(d, d.update, d2) + + def test_setdefault(self): + d = self.new_dict() + + # setting a key with dict.setdefault() must change the version + self.check_version_changed(d, d.setdefault, 'key', 'value1') + + # don't change the version if the key already exists + self.check_version_dont_change(d, d.setdefault, 'key', 'value2') + + def test_delitem(self): + d = self.new_dict(key='value') + + # deleting a key with dict.__delitem__() must change the version + self.check_version_changed(d, d.__delitem__, 'key') + + # don't change the version if the key doesn't exist + self.check_version_dont_change(d, self.assertRaises, KeyError, + d.__delitem__, 'key') + + def test_pop(self): + d = self.new_dict(key='value') + + # pop() must change the version if the key exists + self.check_version_changed(d, d.pop, 'key') + + # pop() must not change the version if the key does not exist + self.check_version_dont_change(d, self.assertRaises, KeyError, + d.pop, 'key') + + def test_popitem(self): + d = self.new_dict(key='value') + + # popitem() must change the version if the dict is not empty + self.check_version_changed(d, d.popitem) + + # popitem() must not change the version if the dict is empty + self.check_version_dont_change(d, self.assertRaises, KeyError, + d.popitem) + + def test_update(self): + d = self.new_dict(key='value') + + # update() calling with no argument must not change the version + self.check_version_dont_change(d, d.update) + + # update() must change the version + self.check_version_changed(d, d.update, key='new value') + + d2 = self.new_dict(key='value 3') + self.check_version_changed(d, d.update, d2) + + def test_clear(self): + d = self.new_dict(key='value') + + # clear() must change the version if the dict is not empty + self.check_version_changed(d, d.clear) + + # clear() must not change the version if the dict is empty + self.check_version_dont_change(d, d.clear) + + +class Dict(dict): + pass + + +class DictSubtypeVersionTests(DictVersionTests): + type2test = Dict + + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -937,9 +937,9 @@ # method-wrapper (descriptor object) check({}.__iter__, size('2P')) # dict - check({}, size('n2P') + calcsize('2nP2n') + 8 + (8*2//3)*calcsize('n2P')) + check({}, size('n2P') + 8 + calcsize('2nP2n') + 8 + (8*2//3)*calcsize('n2P')) longdict = {1:1, 2:2, 3:3, 4:4, 5:5, 6:6, 7:7, 8:8} - check(longdict, size('n2P') + calcsize('2nP2n') + 16 + (16*2//3)*calcsize('n2P')) + check(longdict, size('n2P') + 8 + calcsize('2nP2n') + 16 + (16*2//3)*calcsize('n2P')) # dictionary-keyview check({}.keys(), size('P')) # dictionary-valueview @@ -1103,7 +1103,7 @@ class newstyleclass(object): pass check(newstyleclass, s) # dict with shared keys - check(newstyleclass().__dict__, size('n2P' + '2nP2n')) + check(newstyleclass().__dict__, size('n2P' + '2nP2n') + 8) # unicode # each tuple contains a string and its expected character size # don't put any static strings here, as they may contain diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #26058: Add a new private version to the builtin dict type, incremented + at each dictionary creation and at each dictionary change. Implementation of + the PEP 509. + - Issue #27364: A backslash-character pair that is not a valid escape sequence now generates a DeprecationWarning. diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -3915,6 +3915,21 @@ return _PyTraceMalloc_GetTraceback(domain, (uintptr_t)ptr); } +static PyObject * +dict_get_version(PyObject *self, PyObject *args) +{ + PyDictObject *dict; + uint64_t version; + + if (!PyArg_ParseTuple(args, "O!", &PyDict_Type, &dict)) + return NULL; + + version = dict->ma_version_tag; + + Py_BUILD_ASSERT(sizeof(unsigned PY_LONG_LONG) >= sizeof(version)); + return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG)version); +} + static PyMethodDef TestMethods[] = { {"raise_exception", raise_exception, METH_VARARGS}, @@ -4114,6 +4129,7 @@ {"tracemalloc_track", tracemalloc_track, METH_VARARGS}, {"tracemalloc_untrack", tracemalloc_untrack, METH_VARARGS}, {"tracemalloc_get_traceback", tracemalloc_get_traceback, METH_VARARGS}, + {"dict_get_version", dict_get_version, METH_VARARGS}, {NULL, NULL} /* sentinel */ }; diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -237,6 +237,13 @@ static int dictresize(PyDictObject *mp, Py_ssize_t minused); +/* Global counter used to set ma_version_tag field of dictionary. + * It is incremented each time that a dictionary is created and each + * time that a dictionary is modified. */ +static uint64_t pydict_global_version = 0; + +#define DICT_NEXT_VERSION() (++pydict_global_version) + /* Dictionary reuse scheme to save calls to malloc and free */ #ifndef PyDict_MAXFREELIST #define PyDict_MAXFREELIST 80 @@ -511,6 +518,7 @@ mp->ma_keys = keys; mp->ma_values = values; mp->ma_used = 0; + mp->ma_version_tag = DICT_NEXT_VERSION(); return (PyObject *)mp; } @@ -1074,6 +1082,7 @@ ep->me_value = value; } mp->ma_used++; + mp->ma_version_tag = DICT_NEXT_VERSION(); mp->ma_keys->dk_usable--; mp->ma_keys->dk_nentries++; assert(mp->ma_keys->dk_usable >= 0); @@ -1085,6 +1094,8 @@ old_value = *value_addr; if (old_value != NULL) { *value_addr = value; + mp->ma_version_tag = DICT_NEXT_VERSION(); + Py_DECREF(old_value); /* which **CAN** re-enter (see issue #22653) */ return 0; } @@ -1094,6 +1105,7 @@ assert(ix == mp->ma_used); *value_addr = value; mp->ma_used++; + mp->ma_version_tag = DICT_NEXT_VERSION(); return 0; } @@ -1533,6 +1545,7 @@ old_value = *value_addr; *value_addr = NULL; mp->ma_used--; + mp->ma_version_tag = DICT_NEXT_VERSION(); if (_PyDict_HasSplitTable(mp)) { mp->ma_keys->dk_usable = 0; } @@ -1568,6 +1581,7 @@ mp->ma_keys = Py_EMPTY_KEYS; mp->ma_values = empty_values; mp->ma_used = 0; + mp->ma_version_tag = DICT_NEXT_VERSION(); /* ...then clear the keys and values */ if (oldvalues != NULL) { n = oldkeys->dk_nentries; @@ -1706,9 +1720,11 @@ _PyErr_SetKeyError(key); return NULL; } + old_value = *value_addr; *value_addr = NULL; mp->ma_used--; + mp->ma_version_tag = DICT_NEXT_VERSION(); if (!_PyDict_HasSplitTable(mp)) { dk_set_index(mp->ma_keys, hashpos, DKIX_DUMMY); ep = &DK_ENTRIES(mp->ma_keys)[ix]; @@ -2659,6 +2675,7 @@ mp->ma_keys->dk_usable--; mp->ma_keys->dk_nentries++; mp->ma_used++; + mp->ma_version_tag = DICT_NEXT_VERSION(); } else val = *value_addr; @@ -2752,6 +2769,7 @@ /* We can't dk_usable++ since there is DKIX_DUMMY in indices */ mp->ma_keys->dk_nentries = i; mp->ma_used--; + mp->ma_version_tag = DICT_NEXT_VERSION(); return res; } @@ -2970,6 +2988,7 @@ _PyObject_GC_UNTRACK(d); d->ma_used = 0; + d->ma_version_tag = DICT_NEXT_VERSION(); d->ma_keys = new_keys_object(PyDict_MINSIZE); if (d->ma_keys == NULL) { Py_DECREF(self); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 16:17:54 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 20:17:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_do_not_worry_about_64-bit_?= =?utf-8?q?dict_sizes_on_32-bit_platforms?= Message-ID: <20160908201753.87446.21581.C6F5E8C6@psf.io> https://hg.python.org/cpython/rev/af8cb71edcb7 changeset: 103352:af8cb71edcb7 user: Benjamin Peterson date: Thu Sep 08 13:16:41 2016 -0700 summary: do not worry about 64-bit dict sizes on 32-bit platforms files: Objects/dict-common.h | 4 +++- Objects/dictobject.c | 10 +++++++--- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/Objects/dict-common.h b/Objects/dict-common.h --- a/Objects/dict-common.h +++ b/Objects/dict-common.h @@ -59,14 +59,16 @@ - 1 byte if dk_size <= 0xff (char*) - 2 bytes if dk_size <= 0xffff (int16_t*) - 4 bytes if dk_size <= 0xffffffff (int32_t*) - - 8 bytes otherwise (Py_ssize_t*) + - 8 bytes otherwise (int64_t*) Dynamically sized, 8 is minimum. */ union { int8_t as_1[8]; int16_t as_2[4]; int32_t as_4[2]; +#if SIZEOF_VOID_P > 4 int64_t as_8[1]; +#endif } dk_indices; /* "PyDictKeyEntry dk_entries[dk_usable];" array follows: diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -237,7 +237,7 @@ static int dictresize(PyDictObject *mp, Py_ssize_t minused); -/* Global counter used to set ma_version_tag field of dictionary. +/*Global counter used to set ma_version_tag field of dictionary. * It is incremented each time that a dictionary is created and each * time that a dictionary is modified. */ static uint64_t pydict_global_version = 0; @@ -292,12 +292,12 @@ (DK_SIZE(dk) <= 0xff ? \ 1 : DK_SIZE(dk) <= 0xffff ? \ 2 : DK_SIZE(dk) <= 0xffffffff ? \ - 4 : sizeof(Py_ssize_t)) + 4 : sizeof(int64_t)) #else #define DK_IXSIZE(dk) \ (DK_SIZE(dk) <= 0xff ? \ 1 : DK_SIZE(dk) <= 0xffff ? \ - 2 : sizeof(Py_ssize_t)) + 2 : sizeof(int32_t)) #endif #define DK_ENTRIES(dk) \ ((PyDictKeyEntry*)(&(dk)->dk_indices.as_1[DK_SIZE(dk) * DK_IXSIZE(dk)])) @@ -330,10 +330,12 @@ int32_t *indices = keys->dk_indices.as_4; ix = indices[i]; } +#if SIZEOF_VOID_P > 4 else { int64_t *indices = keys->dk_indices.as_8; ix = indices[i]; } +#endif assert(ix >= DKIX_DUMMY); return ix; } @@ -361,10 +363,12 @@ assert(ix <= 0x7fffffff); indices[i] = (int32_t)ix; } +#if SIZEOF_VOID_P > 4 else { int64_t *indices = keys->dk_indices.as_8; indices[i] = ix; } +#endif } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 16:19:43 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 20:19:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_indicate_the_dependence_of?= =?utf-8?q?_odict_and_dictobject_on_dict-common=2Eh?= Message-ID: <20160908201938.19423.47117.6E97DE58@psf.io> https://hg.python.org/cpython/rev/a0a47e7642a0 changeset: 103353:a0a47e7642a0 user: Benjamin Peterson date: Thu Sep 08 13:19:14 2016 -0700 summary: indicate the dependence of odict and dictobject on dict-common.h files: Makefile.pre.in | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -846,7 +846,8 @@ Objects/unicodeobject.o: $(srcdir)/Objects/unicodeobject.c $(UNICODE_DEPS) -Objects/dictobject.o: $(srcdir)/Objects/stringlib/eq.h +Objects/odictobject.o: $(srcdir)/Objects/dict-common.h +Objects/dictobject.o: $(srcdir)/Objects/stringlib/eq.h $(srcdir)/Objects/dict-common.h Objects/setobject.o: $(srcdir)/Objects/stringlib/eq.h $(OPCODETARGETS_H): $(OPCODETARGETGEN_FILES) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 16:36:03 2016 From: python-checkins at python.org (berker.peksag) Date: Thu, 08 Sep 2016 20:36:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_missing_versionadded_d?= =?utf-8?q?irective?= Message-ID: <20160908203602.87386.17855.63863D15@psf.io> https://hg.python.org/cpython/rev/e7bd2da68d2b changeset: 103354:e7bd2da68d2b user: Berker Peksag date: Thu Sep 08 23:36:25 2016 +0300 summary: Add missing versionadded directive files: Doc/library/pkgutil.rst | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Doc/library/pkgutil.rst b/Doc/library/pkgutil.rst --- a/Doc/library/pkgutil.rst +++ b/Doc/library/pkgutil.rst @@ -15,6 +15,7 @@ A namedtuple that holds a brief summary of a module's info. + .. versionadded:: 3.6 .. function:: extend_path(path, name) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 16:49:09 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 20:49:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_legacy_=22from_=5F?= =?utf-8?q?=5Ffuture=5F=5F_import_with=5Fstatement=22_lines=2E?= Message-ID: <20160908204907.19423.8805.181F6ADD@psf.io> https://hg.python.org/cpython/rev/7eaaac5ad4e8 changeset: 103355:7eaaac5ad4e8 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 13:47:41 2016 -0700 summary: Remove legacy "from __future__ import with_statement" lines. files: Doc/tools/rstlint.py | 2 -- Lib/lib2to3/refactor.py | 2 -- Lib/lib2to3/tests/test_parser.py | 2 -- Lib/lib2to3/tests/test_pytree.py | 2 -- Lib/lib2to3/tests/test_refactor.py | 2 -- Tools/gdb/libpython.py | 2 +- Tools/hg/hgtouch.py | 1 - Tools/pybench/With.py | 1 - Tools/scripts/generate_opcode_h.py | 2 -- 9 files changed, 1 insertions(+), 15 deletions(-) diff --git a/Doc/tools/rstlint.py b/Doc/tools/rstlint.py --- a/Doc/tools/rstlint.py +++ b/Doc/tools/rstlint.py @@ -9,8 +9,6 @@ # TODO: - wrong versions in versionadded/changed # - wrong markup after versionchanged directive -from __future__ import with_statement - import os import re import sys diff --git a/Lib/lib2to3/refactor.py b/Lib/lib2to3/refactor.py --- a/Lib/lib2to3/refactor.py +++ b/Lib/lib2to3/refactor.py @@ -8,8 +8,6 @@ provides infrastructure to write your own refactoring tool. """ -from __future__ import with_statement - __author__ = "Guido van Rossum " diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -6,8 +6,6 @@ test_grammar.py files from both Python 2 and Python 3. """ -from __future__ import with_statement - # Testing imports from . import support from .support import driver diff --git a/Lib/lib2to3/tests/test_pytree.py b/Lib/lib2to3/tests/test_pytree.py --- a/Lib/lib2to3/tests/test_pytree.py +++ b/Lib/lib2to3/tests/test_pytree.py @@ -9,8 +9,6 @@ especially when debugging a test. """ -from __future__ import with_statement - # Testing imports from . import support diff --git a/Lib/lib2to3/tests/test_refactor.py b/Lib/lib2to3/tests/test_refactor.py --- a/Lib/lib2to3/tests/test_refactor.py +++ b/Lib/lib2to3/tests/test_refactor.py @@ -2,8 +2,6 @@ Unit tests for refactor.py. """ -from __future__ import with_statement - import sys import os import codecs diff --git a/Tools/gdb/libpython.py b/Tools/gdb/libpython.py --- a/Tools/gdb/libpython.py +++ b/Tools/gdb/libpython.py @@ -44,7 +44,7 @@ # NOTE: some gdbs are linked with Python 3, so this file should be dual-syntax # compatible (2.6+ and 3.0+). See #19308. -from __future__ import print_function, with_statement +from __future__ import print_function import gdb import os import locale diff --git a/Tools/hg/hgtouch.py b/Tools/hg/hgtouch.py --- a/Tools/hg/hgtouch.py +++ b/Tools/hg/hgtouch.py @@ -7,7 +7,6 @@ In addition to the dependency syntax, #-comments are supported. """ -from __future__ import with_statement import errno import os import time diff --git a/Tools/pybench/With.py b/Tools/pybench/With.py --- a/Tools/pybench/With.py +++ b/Tools/pybench/With.py @@ -1,4 +1,3 @@ -from __future__ import with_statement from pybench import Test class WithFinally(Test): diff --git a/Tools/scripts/generate_opcode_h.py b/Tools/scripts/generate_opcode_h.py --- a/Tools/scripts/generate_opcode_h.py +++ b/Tools/scripts/generate_opcode_h.py @@ -1,7 +1,5 @@ # This script generates the opcode.h header file. -from __future__ import with_statement - import sys header = """/* Auto-generated by Tools/scripts/generate_opcode_h.py */ #ifndef Py_OPCODE_H -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:02:03 2016 From: python-checkins at python.org (eric.snow) Date: Thu, 08 Sep 2016 21:02:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328030=3A_Update_t?= =?utf-8?q?he_language_reference_for_PEP_468=2E?= Message-ID: <20160908210155.87446.67669.E4E88F0E@psf.io> https://hg.python.org/cpython/rev/cdbc5ee744a1 changeset: 103356:cdbc5ee744a1 user: Eric Snow date: Thu Sep 08 13:59:58 2016 -0700 summary: Issue #28030: Update the language reference for PEP 468. files: Doc/reference/compound_stmts.rst | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -546,11 +546,12 @@ function call always assigns values to all parameters mentioned in the parameter list, either from position arguments, from keyword arguments, or from default values. If the form "``*identifier``" is present, it is initialized to a tuple -receiving any excess positional parameters, defaulting to the empty tuple. If -the form "``**identifier``" is present, it is initialized to a new dictionary -receiving any excess keyword arguments, defaulting to a new empty dictionary. -Parameters after "``*``" or "``*identifier``" are keyword-only parameters and -may only be passed used keyword arguments. +receiving any excess positional parameters, defaulting to the empty tuple. +If the form "``**identifier``" is present, it is initialized to a new +ordered mapping receiving any excess keyword arguments, defaulting to a +new empty mapping of the same type. Parameters after "``*``" or +"``*identifier``" are keyword-only parameters and may only be passed +used keyword arguments. .. index:: pair: function; annotations -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:15:20 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 21:15:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=231602=3A_Windows_c?= =?utf-8?q?onsole_doesn=27t_input_or_print_Unicode_=28PEP_528=29?= Message-ID: <20160908211512.59674.48406.0287A8A9@psf.io> https://hg.python.org/cpython/rev/6142d2d3c471 changeset: 103357:6142d2d3c471 user: Steve Dower date: Tue Aug 30 21:22:36 2016 -0700 summary: Issue #1602: Windows console doesn't input or print Unicode (PEP 528) Closes #17602: Adds a readline implementation for the Windows console files: Doc/library/functions.rst | 40 +- Doc/using/cmdline.rst | 17 + Doc/whatsnew/3.6.rst | 19 + Include/pydebug.h | 4 + Lib/io.py | 7 + Lib/test/test_os.py | 2 +- Lib/test/test_winconsoleio.py | 72 + Misc/NEWS | 3 +- Modules/_io/_iomodule.c | 24 +- Modules/_io/_iomodule.h | 10 + Modules/_io/clinic/winconsoleio.c.h | 331 ++++ Modules/_io/winconsoleio.c | 1096 +++++++++++++++ PCbuild/pythoncore.vcxproj | 1 + PCbuild/pythoncore.vcxproj.filters | 3 + Parser/myreadline.c | 113 + Python/pylifecycle.c | 18 + 16 files changed, 1739 insertions(+), 21 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1055,30 +1055,38 @@ (where :func:`open` is declared), :mod:`os`, :mod:`os.path`, :mod:`tempfile`, and :mod:`shutil`. - .. versionchanged:: 3.3 - The *opener* parameter was added. - The ``'x'`` mode was added. - :exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`. - :exc:`FileExistsError` is now raised if the file opened in exclusive - creation mode (``'x'``) already exists. + .. versionchanged:: + 3.3 - .. versionchanged:: 3.4 - The file is now non-inheritable. + * The *opener* parameter was added. + * The ``'x'`` mode was added. + * :exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`. + * :exc:`FileExistsError` is now raised if the file opened in exclusive + * creation mode (``'x'``) already exists. + + .. versionchanged:: + 3.4 + + * The file is now non-inheritable. .. deprecated-removed:: 3.4 4.0 The ``'U'`` mode. - .. versionchanged:: 3.5 - If the system call is interrupted and the signal handler does not raise an - exception, the function now retries the system call instead of raising an - :exc:`InterruptedError` exception (see :pep:`475` for the rationale). + .. versionchanged:: + 3.5 - .. versionchanged:: 3.5 - The ``'namereplace'`` error handler was added. + * If the system call is interrupted and the signal handler does not raise an + exception, the function now retries the system call instead of raising an + :exc:`InterruptedError` exception (see :pep:`475` for the rationale). + * The ``'namereplace'`` error handler was added. - .. versionchanged:: 3.6 - Support added to accept objects implementing :class:`os.PathLike`. + .. versionchanged:: + 3.6 + + * Support added to accept objects implementing :class:`os.PathLike`. + * On Windows, opening a console buffer may return a subclass of + :class:`io.RawIOBase` other than :class:`io.FileIO`. .. function:: ord(c) diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -559,6 +559,10 @@ .. versionchanged:: 3.4 The ``encodingname`` part is now optional. + .. versionchanged:: 3.6 + On Windows, the encoding specified by this variable is ignored for interactive + console buffers unless :envvar:`PYTHONLEGACYWINDOWSIOENCODING` is also specified. + Files and pipes redirected through the standard streams are not affected. .. envvar:: PYTHONNOUSERSITE @@ -686,6 +690,19 @@ .. versionadded:: 3.6 See :pep:`529` for more details. +.. envvar:: PYTHONLEGACYWINDOWSIOENCODING + + If set to a non-empty string, does not use the new console reader and + writer. This means that Unicode characters will be encoded according to + the active console code page, rather than using utf-8. + + This variable is ignored if the standard streams are redirected (to files + or pipes) rather than referring to console buffers. + + Availability: Windows + + .. versionadded:: 3.6 + Debug-mode variables ~~~~~~~~~~~~~~~~~~~~ diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -78,6 +78,8 @@ * PEP 529: :ref:`Change Windows filesystem encoding to UTF-8 ` +* PEP 528: :ref:`Change Windows console encoding to UTF-8 ` + * The ``py.exe`` launcher, when used interactively, no longer prefers Python 2 over Python 3 when the user doesn't specify a version (via command line arguments or a config file). Handling of shebang lines @@ -267,6 +269,23 @@ (Contributed by Martin Teichmann in :issue:`27366`) +.. _pep-528: + +PEP 528: Change Windows console encoding to UTF-8 +------------------------------------------------- + +The default console on Windows will now accept all Unicode characters and +provide correctly read str objects to Python code. ``sys.stdin``, +``sys.stdout`` and ``sys.stderr`` now default to utf-8 encoding. + +This change only applies when using an interactive console, and not when +redirecting files or pipes. To revert to the previous behaviour for interactive +console use, set :envvar:`PYTHONLEGACYWINDOWSIOENCODING`. + +.. seealso:: + + :pep:`528` -- Change Windows console encoding to UTF-8 + PEP written and implemented by Steve Dower. PYTHONMALLOC environment variable ================================= diff --git a/Include/pydebug.h b/Include/pydebug.h --- a/Include/pydebug.h +++ b/Include/pydebug.h @@ -24,6 +24,10 @@ PyAPI_DATA(int) Py_HashRandomizationFlag; PyAPI_DATA(int) Py_IsolatedFlag; +#ifdef MS_WINDOWS +PyAPI_DATA(int) Py_LegacyWindowsStdioFlag; +#endif + /* this is a wrapper around getenv() that pays attention to Py_IgnoreEnvironmentFlag. It should be used for getting variables like PYTHONPATH and PYTHONHOME from the environment */ diff --git a/Lib/io.py b/Lib/io.py --- a/Lib/io.py +++ b/Lib/io.py @@ -90,3 +90,10 @@ for klass in (StringIO, TextIOWrapper): TextIOBase.register(klass) del klass + +try: + from _io import _WindowsConsoleIO +except ImportError: + pass +else: + RawIOBase.register(_WindowsConsoleIO) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1518,7 +1518,7 @@ singles = ["fchdir", "dup", "fdopen", "fdatasync", "fstat", "fstatvfs", "fsync", "tcgetpgrp", "ttyname"] #singles.append("close") - #We omit close because it doesn'r raise an exception on some platforms + #We omit close because it doesn't raise an exception on some platforms def get_single(f): def helper(self): if hasattr(os, f): diff --git a/Lib/test/test_winconsoleio.py b/Lib/test/test_winconsoleio.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_winconsoleio.py @@ -0,0 +1,72 @@ +'''Tests for WindowsConsoleIO + +Unfortunately, most testing requires interactive use, since we have no +API to read back from a real console, and this class is only for use +with real consoles. + +Instead, we validate that basic functionality such as opening, closing +and in particular fileno() work, but are forced to leave real testing +to real people with real keyborads. +''' + +import io +import unittest +import sys + +if sys.platform != 'win32': + raise unittest.SkipTest("test only relevant on win32") + +ConIO = io._WindowsConsoleIO + +class WindowsConsoleIOTests(unittest.TestCase): + def test_abc(self): + self.assertTrue(issubclass(ConIO, io.RawIOBase)) + self.assertFalse(issubclass(ConIO, io.BufferedIOBase)) + self.assertFalse(issubclass(ConIO, io.TextIOBase)) + + def test_open_fd(self): + f = ConIO(0) + self.assertTrue(f.readable()) + self.assertFalse(f.writable()) + self.assertEqual(0, f.fileno()) + f.close() # multiple close should not crash + f.close() + + f = ConIO(1, 'w') + self.assertFalse(f.readable()) + self.assertTrue(f.writable()) + self.assertEqual(1, f.fileno()) + f.close() + f.close() + + f = ConIO(2, 'w') + self.assertFalse(f.readable()) + self.assertTrue(f.writable()) + self.assertEqual(2, f.fileno()) + f.close() + f.close() + + def test_open_name(self): + f = ConIO("CON") + self.assertTrue(f.readable()) + self.assertFalse(f.writable()) + self.assertIsNotNone(f.fileno()) + f.close() # multiple close should not crash + f.close() + + f = ConIO('CONIN$') + self.assertTrue(f.readable()) + self.assertFalse(f.writable()) + self.assertIsNotNone(f.fileno()) + f.close() + f.close() + + f = ConIO('CONOUT$', 'w') + self.assertFalse(f.readable()) + self.assertTrue(f.writable()) + self.assertIsNotNone(f.fileno()) + f.close() + f.close() + +if __name__ == "__main__": + unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -300,6 +300,8 @@ Windows ------- +- Issue #1602: Windows console doesn't input or print Unicode (PEP 528) + - Issue #27781: Change file system encoding on Windows to UTF-8 (PEP 529) - Issue #27731: Opt-out of MAX_PATH on Windows 10 @@ -556,7 +558,6 @@ - Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. Also update FreedBSD version checks for the original ctype UTF-8 workaround. - What's New in Python 3.6.0 alpha 3 ================================== diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c --- a/Modules/_io/_iomodule.c +++ b/Modules/_io/_iomodule.c @@ -20,6 +20,9 @@ #include #endif /* HAVE_SYS_STAT_H */ +#ifdef MS_WINDOWS +#include +#endif /* Various interned strings */ @@ -52,7 +55,6 @@ PyObject *_PyIO_empty_bytes; PyObject *_PyIO_zero; - PyDoc_STRVAR(module_doc, "The io module provides the Python interfaces to stream handling. The\n" "builtin open function is defined in this module.\n" @@ -362,8 +364,18 @@ } /* Create the Raw file stream */ - raw = PyObject_CallFunction((PyObject *)&PyFileIO_Type, - "OsiO", path_or_fd, rawmode, closefd, opener); + { + PyObject *RawIO_class = (PyObject *)&PyFileIO_Type; +#ifdef MS_WINDOWS + if (!Py_LegacyWindowsStdioFlag && _PyIO_get_console_type(path_or_fd) != '\0') { + RawIO_class = (PyObject *)&PyWindowsConsoleIO_Type; + encoding = "utf-8"; + } +#endif + raw = PyObject_CallFunction(RawIO_class, + "OsiO", path_or_fd, rawmode, closefd, opener); + } + if (raw == NULL) goto error; result = raw; @@ -708,6 +720,12 @@ PyStringIO_Type.tp_base = &PyTextIOBase_Type; ADD_TYPE(&PyStringIO_Type, "StringIO"); +#ifdef MS_WINDOWS + /* WindowsConsoleIO */ + PyWindowsConsoleIO_Type.tp_base = &PyRawIOBase_Type; + ADD_TYPE(&PyWindowsConsoleIO_Type, "_WindowsConsoleIO"); +#endif + /* BufferedReader */ PyBufferedReader_Type.tp_base = &PyBufferedIOBase_Type; ADD_TYPE(&PyBufferedReader_Type, "BufferedReader"); diff --git a/Modules/_io/_iomodule.h b/Modules/_io/_iomodule.h --- a/Modules/_io/_iomodule.h +++ b/Modules/_io/_iomodule.h @@ -19,6 +19,12 @@ extern PyTypeObject PyTextIOWrapper_Type; extern PyTypeObject PyIncrementalNewlineDecoder_Type; +#ifndef Py_LIMITED_API +#ifdef MS_WINDOWS +extern PyTypeObject PyWindowsConsoleIO_Type; +#define PyWindowsConsoleIO_Check(op) (PyObject_TypeCheck((op), &PyWindowsConsoleIO_Type)) +#endif /* MS_WINDOWS */ +#endif /* Py_LIMITED_API */ extern int _PyIO_ConvertSsize_t(PyObject *, void *); @@ -145,6 +151,10 @@ extern _PyIO_State *_PyIO_get_module_state(void); extern PyObject *_PyIO_get_locale_module(_PyIO_State *); +#ifdef MS_WINDOWS +extern char _PyIO_get_console_type(PyObject *); +#endif + extern PyObject *_PyIO_str_close; extern PyObject *_PyIO_str_closed; extern PyObject *_PyIO_str_decode; diff --git a/Modules/_io/clinic/winconsoleio.c.h b/Modules/_io/clinic/winconsoleio.c.h new file mode 100644 --- /dev/null +++ b/Modules/_io/clinic/winconsoleio.c.h @@ -0,0 +1,331 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO_close__doc__, +"close($self, /)\n" +"--\n" +"\n" +"Close the handle.\n" +"\n" +"A closed handle cannot be used for further I/O operations. close() may be\n" +"called more than once without error."); + +#define _IO__WINDOWSCONSOLEIO_CLOSE_METHODDEF \ + {"close", (PyCFunction)_io__WindowsConsoleIO_close, METH_NOARGS, _io__WindowsConsoleIO_close__doc__}, + +static PyObject * +_io__WindowsConsoleIO_close_impl(winconsoleio *self); + +static PyObject * +_io__WindowsConsoleIO_close(winconsoleio *self, PyObject *Py_UNUSED(ignored)) +{ + return _io__WindowsConsoleIO_close_impl(self); +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO___init____doc__, +"_WindowsConsoleIO(file, mode=\'r\', closefd=True, opener=None)\n" +"--\n" +"\n" +"Open a console buffer by file descriptor.\n" +"\n" +"The mode can be \'rb\' (default), or \'wb\' for reading or writing bytes. All\n" +"other mode characters will be ignored. Mode \'b\' will be assumed if it is\n" +"omitted. The *opener* parameter is always ignored."); + +static int +_io__WindowsConsoleIO___init___impl(winconsoleio *self, PyObject *nameobj, + const char *mode, int closefd, + PyObject *opener); + +static int +_io__WindowsConsoleIO___init__(PyObject *self, PyObject *args, PyObject *kwargs) +{ + int return_value = -1; + static const char * const _keywords[] = {"file", "mode", "closefd", "opener", NULL}; + static _PyArg_Parser _parser = {"O|siO:_WindowsConsoleIO", _keywords, 0}; + PyObject *nameobj; + const char *mode = "r"; + int closefd = 1; + PyObject *opener = Py_None; + + if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + &nameobj, &mode, &closefd, &opener)) { + goto exit; + } + return_value = _io__WindowsConsoleIO___init___impl((winconsoleio *)self, nameobj, mode, closefd, opener); + +exit: + return return_value; +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO_fileno__doc__, +"fileno($self, /)\n" +"--\n" +"\n" +"Return the underlying file descriptor (an integer).\n" +"\n" +"fileno is only set when a file descriptor is used to open\n" +"one of the standard streams."); + +#define _IO__WINDOWSCONSOLEIO_FILENO_METHODDEF \ + {"fileno", (PyCFunction)_io__WindowsConsoleIO_fileno, METH_NOARGS, _io__WindowsConsoleIO_fileno__doc__}, + +static PyObject * +_io__WindowsConsoleIO_fileno_impl(winconsoleio *self); + +static PyObject * +_io__WindowsConsoleIO_fileno(winconsoleio *self, PyObject *Py_UNUSED(ignored)) +{ + return _io__WindowsConsoleIO_fileno_impl(self); +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO_readable__doc__, +"readable($self, /)\n" +"--\n" +"\n" +"True if console is an input buffer."); + +#define _IO__WINDOWSCONSOLEIO_READABLE_METHODDEF \ + {"readable", (PyCFunction)_io__WindowsConsoleIO_readable, METH_NOARGS, _io__WindowsConsoleIO_readable__doc__}, + +static PyObject * +_io__WindowsConsoleIO_readable_impl(winconsoleio *self); + +static PyObject * +_io__WindowsConsoleIO_readable(winconsoleio *self, PyObject *Py_UNUSED(ignored)) +{ + return _io__WindowsConsoleIO_readable_impl(self); +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO_writable__doc__, +"writable($self, /)\n" +"--\n" +"\n" +"True if console is an output buffer."); + +#define _IO__WINDOWSCONSOLEIO_WRITABLE_METHODDEF \ + {"writable", (PyCFunction)_io__WindowsConsoleIO_writable, METH_NOARGS, _io__WindowsConsoleIO_writable__doc__}, + +static PyObject * +_io__WindowsConsoleIO_writable_impl(winconsoleio *self); + +static PyObject * +_io__WindowsConsoleIO_writable(winconsoleio *self, PyObject *Py_UNUSED(ignored)) +{ + return _io__WindowsConsoleIO_writable_impl(self); +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO_readinto__doc__, +"readinto($self, buffer, /)\n" +"--\n" +"\n" +"Same as RawIOBase.readinto()."); + +#define _IO__WINDOWSCONSOLEIO_READINTO_METHODDEF \ + {"readinto", (PyCFunction)_io__WindowsConsoleIO_readinto, METH_O, _io__WindowsConsoleIO_readinto__doc__}, + +static PyObject * +_io__WindowsConsoleIO_readinto_impl(winconsoleio *self, Py_buffer *buffer); + +static PyObject * +_io__WindowsConsoleIO_readinto(winconsoleio *self, PyObject *arg) +{ + PyObject *return_value = NULL; + Py_buffer buffer = {NULL, NULL}; + + if (!PyArg_Parse(arg, "w*:readinto", &buffer)) { + goto exit; + } + return_value = _io__WindowsConsoleIO_readinto_impl(self, &buffer); + +exit: + /* Cleanup for buffer */ + if (buffer.obj) { + PyBuffer_Release(&buffer); + } + + return return_value; +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO_readall__doc__, +"readall($self, /)\n" +"--\n" +"\n" +"Read all data from the console, returned as bytes.\n" +"\n" +"Return an empty bytes object at EOF."); + +#define _IO__WINDOWSCONSOLEIO_READALL_METHODDEF \ + {"readall", (PyCFunction)_io__WindowsConsoleIO_readall, METH_NOARGS, _io__WindowsConsoleIO_readall__doc__}, + +static PyObject * +_io__WindowsConsoleIO_readall_impl(winconsoleio *self); + +static PyObject * +_io__WindowsConsoleIO_readall(winconsoleio *self, PyObject *Py_UNUSED(ignored)) +{ + return _io__WindowsConsoleIO_readall_impl(self); +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO_read__doc__, +"read($self, size=-1, /)\n" +"--\n" +"\n" +"Read at most size bytes, returned as bytes.\n" +"\n" +"Only makes one system call when size is a positive integer,\n" +"so less data may be returned than requested.\n" +"Return an empty bytes object at EOF."); + +#define _IO__WINDOWSCONSOLEIO_READ_METHODDEF \ + {"read", (PyCFunction)_io__WindowsConsoleIO_read, METH_VARARGS, _io__WindowsConsoleIO_read__doc__}, + +static PyObject * +_io__WindowsConsoleIO_read_impl(winconsoleio *self, Py_ssize_t size); + +static PyObject * +_io__WindowsConsoleIO_read(winconsoleio *self, PyObject *args) +{ + PyObject *return_value = NULL; + Py_ssize_t size = -1; + + if (!PyArg_ParseTuple(args, "|O&:read", + _PyIO_ConvertSsize_t, &size)) { + goto exit; + } + return_value = _io__WindowsConsoleIO_read_impl(self, size); + +exit: + return return_value; +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO_write__doc__, +"write($self, b, /)\n" +"--\n" +"\n" +"Write buffer b to file, return number of bytes written.\n" +"\n" +"Only makes one system call, so not all of the data may be written.\n" +"The number of bytes actually written is returned."); + +#define _IO__WINDOWSCONSOLEIO_WRITE_METHODDEF \ + {"write", (PyCFunction)_io__WindowsConsoleIO_write, METH_O, _io__WindowsConsoleIO_write__doc__}, + +static PyObject * +_io__WindowsConsoleIO_write_impl(winconsoleio *self, Py_buffer *b); + +static PyObject * +_io__WindowsConsoleIO_write(winconsoleio *self, PyObject *arg) +{ + PyObject *return_value = NULL; + Py_buffer b = {NULL, NULL}; + + if (!PyArg_Parse(arg, "y*:write", &b)) { + goto exit; + } + return_value = _io__WindowsConsoleIO_write_impl(self, &b); + +exit: + /* Cleanup for b */ + if (b.obj) { + PyBuffer_Release(&b); + } + + return return_value; +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(_io__WindowsConsoleIO_isatty__doc__, +"isatty($self, /)\n" +"--\n" +"\n" +"Always True."); + +#define _IO__WINDOWSCONSOLEIO_ISATTY_METHODDEF \ + {"isatty", (PyCFunction)_io__WindowsConsoleIO_isatty, METH_NOARGS, _io__WindowsConsoleIO_isatty__doc__}, + +static PyObject * +_io__WindowsConsoleIO_isatty_impl(winconsoleio *self); + +static PyObject * +_io__WindowsConsoleIO_isatty(winconsoleio *self, PyObject *Py_UNUSED(ignored)) +{ + return _io__WindowsConsoleIO_isatty_impl(self); +} + +#endif /* defined(MS_WINDOWS) */ + +#ifndef _IO__WINDOWSCONSOLEIO_CLOSE_METHODDEF + #define _IO__WINDOWSCONSOLEIO_CLOSE_METHODDEF +#endif /* !defined(_IO__WINDOWSCONSOLEIO_CLOSE_METHODDEF) */ + +#ifndef _IO__WINDOWSCONSOLEIO_FILENO_METHODDEF + #define _IO__WINDOWSCONSOLEIO_FILENO_METHODDEF +#endif /* !defined(_IO__WINDOWSCONSOLEIO_FILENO_METHODDEF) */ + +#ifndef _IO__WINDOWSCONSOLEIO_READABLE_METHODDEF + #define _IO__WINDOWSCONSOLEIO_READABLE_METHODDEF +#endif /* !defined(_IO__WINDOWSCONSOLEIO_READABLE_METHODDEF) */ + +#ifndef _IO__WINDOWSCONSOLEIO_WRITABLE_METHODDEF + #define _IO__WINDOWSCONSOLEIO_WRITABLE_METHODDEF +#endif /* !defined(_IO__WINDOWSCONSOLEIO_WRITABLE_METHODDEF) */ + +#ifndef _IO__WINDOWSCONSOLEIO_READINTO_METHODDEF + #define _IO__WINDOWSCONSOLEIO_READINTO_METHODDEF +#endif /* !defined(_IO__WINDOWSCONSOLEIO_READINTO_METHODDEF) */ + +#ifndef _IO__WINDOWSCONSOLEIO_READALL_METHODDEF + #define _IO__WINDOWSCONSOLEIO_READALL_METHODDEF +#endif /* !defined(_IO__WINDOWSCONSOLEIO_READALL_METHODDEF) */ + +#ifndef _IO__WINDOWSCONSOLEIO_READ_METHODDEF + #define _IO__WINDOWSCONSOLEIO_READ_METHODDEF +#endif /* !defined(_IO__WINDOWSCONSOLEIO_READ_METHODDEF) */ + +#ifndef _IO__WINDOWSCONSOLEIO_WRITE_METHODDEF + #define _IO__WINDOWSCONSOLEIO_WRITE_METHODDEF +#endif /* !defined(_IO__WINDOWSCONSOLEIO_WRITE_METHODDEF) */ + +#ifndef _IO__WINDOWSCONSOLEIO_ISATTY_METHODDEF + #define _IO__WINDOWSCONSOLEIO_ISATTY_METHODDEF +#endif /* !defined(_IO__WINDOWSCONSOLEIO_ISATTY_METHODDEF) */ +/*[clinic end generated code: output=9eba916f8537fff7 input=a9049054013a1b77]*/ diff --git a/Modules/_io/winconsoleio.c b/Modules/_io/winconsoleio.c new file mode 100644 --- /dev/null +++ b/Modules/_io/winconsoleio.c @@ -0,0 +1,1096 @@ +/* + An implementation of Windows console I/O + + Classes defined here: _WindowsConsoleIO + + Written by Steve Dower +*/ + +#define PY_SSIZE_T_CLEAN +#include "Python.h" + +#ifdef MS_WINDOWS + +#include "structmember.h" +#ifdef HAVE_SYS_TYPES_H +#include +#endif +#ifdef HAVE_SYS_STAT_H +#include +#endif +#include /* For offsetof */ + +#define WIN32_LEAN_AND_MEAN +#include + +#include "_iomodule.h" + +/* BUFSIZ determines how many characters can be typed at the console + before it starts blocking. */ +#if BUFSIZ < (16*1024) +#define SMALLCHUNK (2*1024) +#elif (BUFSIZ >= (2 << 25)) +#error "unreasonable BUFSIZ > 64MB defined" +#else +#define SMALLCHUNK BUFSIZ +#endif + +/* BUFMAX determines how many bytes can be read in one go. */ +#define BUFMAX (32*1024*1024) + +char _get_console_type(HANDLE handle) { + DWORD mode, peek_count; + + if (handle == INVALID_HANDLE_VALUE) + return '\0'; + + if (!GetConsoleMode(handle, &mode)) + return '\0'; + + /* Peek at the handle to see whether it is an input or output handle */ + if (GetNumberOfConsoleInputEvents(handle, &peek_count)) + return 'r'; + return 'w'; +} + +char _PyIO_get_console_type(PyObject *path_or_fd) { + int fd; + + fd = PyLong_AsLong(path_or_fd); + PyErr_Clear(); + if (fd >= 0) { + HANDLE handle; + _Py_BEGIN_SUPPRESS_IPH + handle = (HANDLE)_get_osfhandle(fd); + _Py_END_SUPPRESS_IPH + if (!handle) + return '\0'; + return _get_console_type(handle); + } + + PyObject *decoded = Py_None; + Py_INCREF(decoded); + + int d = PyUnicode_FSDecoder(path_or_fd, &decoded); + if (!d) { + PyErr_Clear(); + Py_CLEAR(decoded); + return '\0'; + } + + char m = '\0'; + if (!PyUnicode_Check(decoded)) { + return '\0'; + } else if (PyUnicode_CompareWithASCIIString(decoded, "CONIN$") == 0) { + m = 'r'; + } else if (PyUnicode_CompareWithASCIIString(decoded, "CONOUT$") == 0 || + PyUnicode_CompareWithASCIIString(decoded, "CON") == 0) { + m = 'w'; + } + + Py_CLEAR(decoded); + return m; +} + +/*[clinic input] +module _io +class _io._WindowsConsoleIO "winconsoleio *" "&PyWindowsConsoleIO_Type" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=e897fdc1fba4e131]*/ + +/*[python input] +class io_ssize_t_converter(CConverter): + type = 'Py_ssize_t' + converter = '_PyIO_ConvertSsize_t' +[python start generated code]*/ +/*[python end generated code: output=da39a3ee5e6b4b0d input=d0a811d3cbfd1b33]*/ + +typedef struct { + PyObject_HEAD + HANDLE handle; + int fd; + unsigned int created : 1; + unsigned int readable : 1; + unsigned int writable : 1; + unsigned int closehandle : 1; + char finalizing; + unsigned int blksize; + PyObject *weakreflist; + PyObject *dict; + char buf[4]; +} winconsoleio; + +PyTypeObject PyWindowsConsoleIO_Type; + +_Py_IDENTIFIER(name); + +int +_PyWindowsConsoleIO_closed(PyObject *self) +{ + return ((winconsoleio *)self)->handle == INVALID_HANDLE_VALUE; +} + + +/* Returns 0 on success, -1 with exception set on failure. */ +static int +internal_close(winconsoleio *self) +{ + if (self->handle != INVALID_HANDLE_VALUE) { + if (self->closehandle) { + if (self->fd >= 0) { + _Py_BEGIN_SUPPRESS_IPH + close(self->fd); + _Py_END_SUPPRESS_IPH + } + CloseHandle(self->handle); + } + self->handle = INVALID_HANDLE_VALUE; + self->fd = -1; + } + return 0; +} + +/*[clinic input] +_io._WindowsConsoleIO.close + +Close the handle. + +A closed handle cannot be used for further I/O operations. close() may be +called more than once without error. +[clinic start generated code]*/ + +static PyObject * +_io__WindowsConsoleIO_close_impl(winconsoleio *self) +/*[clinic end generated code: output=27ef95b66c29057b input=185617e349ae4c7b]*/ +{ + PyObject *res; + PyObject *exc, *val, *tb; + int rc; + _Py_IDENTIFIER(close); + res = _PyObject_CallMethodId((PyObject*)&PyRawIOBase_Type, + &PyId_close, "O", self); + if (!self->closehandle) { + self->handle = INVALID_HANDLE_VALUE; + return res; + } + if (res == NULL) + PyErr_Fetch(&exc, &val, &tb); + rc = internal_close(self); + if (res == NULL) + _PyErr_ChainExceptions(exc, val, tb); + if (rc < 0) + Py_CLEAR(res); + return res; +} + +static PyObject * +winconsoleio_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +{ + winconsoleio *self; + + assert(type != NULL && type->tp_alloc != NULL); + + self = (winconsoleio *) type->tp_alloc(type, 0); + if (self != NULL) { + self->handle = INVALID_HANDLE_VALUE; + self->fd = -1; + self->created = 0; + self->readable = 0; + self->writable = 0; + self->closehandle = 0; + self->blksize = 0; + self->weakreflist = NULL; + } + + return (PyObject *) self; +} + +/*[clinic input] +_io._WindowsConsoleIO.__init__ + file as nameobj: object + mode: str = "r" + closefd: int(c_default="1") = True + opener: object = None + +Open a console buffer by file descriptor. + +The mode can be 'rb' (default), or 'wb' for reading or writing bytes. All +other mode characters will be ignored. Mode 'b' will be assumed if it is +omitted. The *opener* parameter is always ignored. +[clinic start generated code]*/ + +static int +_io__WindowsConsoleIO___init___impl(winconsoleio *self, PyObject *nameobj, + const char *mode, int closefd, + PyObject *opener) +/*[clinic end generated code: output=3fd9cbcdd8d95429 input=61be39633a86f5d7]*/ +{ + const char *s; + wchar_t *name = NULL; + int ret = 0; + int rwa = 0; + int fd = -1; + int fd_is_own = 0; + + assert(PyWindowsConsoleIO_Check(self)); + if (self->handle >= 0) { + if (self->closehandle) { + /* Have to close the existing file first. */ + if (internal_close(self) < 0) + return -1; + } + else + self->handle = INVALID_HANDLE_VALUE; + } + + if (PyFloat_Check(nameobj)) { + PyErr_SetString(PyExc_TypeError, + "integer argument expected, got float"); + return -1; + } + + fd = _PyLong_AsInt(nameobj); + if (fd < 0) { + if (!PyErr_Occurred()) { + PyErr_SetString(PyExc_ValueError, + "negative file descriptor"); + return -1; + } + PyErr_Clear(); + } + self->fd = fd; + + if (fd < 0) { + PyObject *decodedname = Py_None; + Py_INCREF(decodedname); + + int d = PyUnicode_FSDecoder(nameobj, (void*)&decodedname); + if (!d) + return -1; + + Py_ssize_t length; + name = PyUnicode_AsWideCharString(decodedname, &length); + Py_CLEAR(decodedname); + if (name == NULL) + return -1; + + if (wcslen(name) != length) { + PyMem_Free(name); + PyErr_SetString(PyExc_ValueError, "embedded null character"); + return -1; + } + } + + s = mode; + while (*s) { + switch (*s++) { + case '+': + case 'a': + case 'b': + case 'x': + break; + case 'r': + if (rwa) + goto bad_mode; + rwa = 1; + self->readable = 1; + break; + case 'w': + if (rwa) + goto bad_mode; + rwa = 1; + self->writable = 1; + break; + default: + PyErr_Format(PyExc_ValueError, + "invalid mode: %.200s", mode); + goto error; + } + } + + if (!rwa) + goto bad_mode; + + if (fd >= 0) { + _Py_BEGIN_SUPPRESS_IPH + self->handle = (HANDLE)_get_osfhandle(fd); + _Py_END_SUPPRESS_IPH + self->closehandle = 0; + } else { + DWORD access = GENERIC_READ; + + self->closehandle = 1; + if (!closefd) { + PyErr_SetString(PyExc_ValueError, + "Cannot use closefd=False with file name"); + goto error; + } + + if (self->writable) + access |= GENERIC_WRITE; + + Py_BEGIN_ALLOW_THREADS + /* Attempt to open for read/write initially, then fall back + on the specific access. This is required for modern names + CONIN$ and CONOUT$, which allow reading/writing state as + well as reading/writing content. */ + self->handle = CreateFileW(name, GENERIC_READ | GENERIC_WRITE, + FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL); + if (self->handle == INVALID_HANDLE_VALUE) + self->handle = CreateFileW(name, access, + FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL); + Py_END_ALLOW_THREADS + + if (self->handle == INVALID_HANDLE_VALUE) { + PyErr_SetExcFromWindowsErrWithFilenameObject(PyExc_OSError, GetLastError(), nameobj); + goto error; + } + } + + if (self->writable && _get_console_type(self->handle) != 'w') { + PyErr_SetString(PyExc_ValueError, + "Cannot open console input buffer for writing"); + goto error; + } + if (self->readable && _get_console_type(self->handle) != 'r') { + PyErr_SetString(PyExc_ValueError, + "Cannot open console output buffer for reading"); + goto error; + } + + self->blksize = DEFAULT_BUFFER_SIZE; + memset(self->buf, 0, 4); + + if (_PyObject_SetAttrId((PyObject *)self, &PyId_name, nameobj) < 0) + goto error; + + goto done; + +bad_mode: + PyErr_SetString(PyExc_ValueError, + "Must have exactly one of read or write mode"); +error: + ret = -1; + internal_close(self); + +done: + if (name) + PyMem_Free(name); + return ret; +} + +static int +winconsoleio_traverse(winconsoleio *self, visitproc visit, void *arg) +{ + Py_VISIT(self->dict); + return 0; +} + +static int +winconsoleio_clear(winconsoleio *self) +{ + Py_CLEAR(self->dict); + return 0; +} + +static void +winconsoleio_dealloc(winconsoleio *self) +{ + self->finalizing = 1; + if (_PyIOBase_finalize((PyObject *) self) < 0) + return; + _PyObject_GC_UNTRACK(self); + if (self->weakreflist != NULL) + PyObject_ClearWeakRefs((PyObject *) self); + Py_CLEAR(self->dict); + Py_TYPE(self)->tp_free((PyObject *)self); +} + +static PyObject * +err_closed(void) +{ + PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); + return NULL; +} + +static PyObject * +err_mode(const char *action) +{ + _PyIO_State *state = IO_STATE(); + if (state != NULL) + PyErr_Format(state->unsupported_operation, + "Console buffer does not support %s", action); + return NULL; +} + +/*[clinic input] +_io._WindowsConsoleIO.fileno + +Return the underlying file descriptor (an integer). + +fileno is only set when a file descriptor is used to open +one of the standard streams. + +[clinic start generated code]*/ + +static PyObject * +_io__WindowsConsoleIO_fileno_impl(winconsoleio *self) +/*[clinic end generated code: output=006fa74ce3b5cfbf input=079adc330ddaabe6]*/ +{ + if (self->fd < 0 && self->handle != INVALID_HANDLE_VALUE) { + _Py_BEGIN_SUPPRESS_IPH + if (self->writable) + self->fd = _open_osfhandle((intptr_t)self->handle, 'wb'); + else + self->fd = _open_osfhandle((intptr_t)self->handle, 'rb'); + _Py_END_SUPPRESS_IPH + } + if (self->fd < 0) + return err_mode("fileno"); + return PyLong_FromLong(self->fd); +} + +/*[clinic input] +_io._WindowsConsoleIO.readable + +True if console is an input buffer. +[clinic start generated code]*/ + +static PyObject * +_io__WindowsConsoleIO_readable_impl(winconsoleio *self) +/*[clinic end generated code: output=daf9cef2743becf0 input=6be9defb5302daae]*/ +{ + if (self->handle == INVALID_HANDLE_VALUE) + return err_closed(); + return PyBool_FromLong((long) self->readable); +} + +/*[clinic input] +_io._WindowsConsoleIO.writable + +True if console is an output buffer. +[clinic start generated code]*/ + +static PyObject * +_io__WindowsConsoleIO_writable_impl(winconsoleio *self) +/*[clinic end generated code: output=e0a2ad7eae5abf67 input=cefbd8abc24df6a0]*/ +{ + if (self->handle == INVALID_HANDLE_VALUE) + return err_closed(); + return PyBool_FromLong((long) self->writable); +} + +static DWORD +_buflen(winconsoleio *self) +{ + for (DWORD i = 0; i < 4; ++i) { + if (!self->buf[i]) + return i; + } + return 4; +} + +static DWORD +_copyfrombuf(winconsoleio *self, char *buf, DWORD len) +{ + DWORD n = 0; + + while (self->buf[0] && len--) { + n += 1; + buf[0] = self->buf[0]; + self->buf[0] = self->buf[1]; + self->buf[1] = self->buf[2]; + self->buf[2] = self->buf[3]; + self->buf[3] = 0; + } + + return n; +} + +static wchar_t * +read_console_w(HANDLE handle, DWORD maxlen, DWORD *readlen) { + int err = 0, sig = 0; + + wchar_t *buf = (wchar_t*)PyMem_Malloc(maxlen * sizeof(wchar_t)); + if (!buf) + goto error; + *readlen = 0; + + Py_BEGIN_ALLOW_THREADS + for (DWORD off = 0; off < maxlen; off += BUFSIZ) { + DWORD n, len = min(maxlen - off, BUFSIZ); + SetLastError(0); + BOOL res = ReadConsoleW(handle, &buf[off], len, &n, NULL); + + if (!res) { + err = GetLastError(); + break; + } + if (n == 0) { + err = GetLastError(); + if (err != ERROR_OPERATION_ABORTED) + break; + err = 0; + HANDLE hInterruptEvent = _PyOS_SigintEvent(); + if (WaitForSingleObjectEx(hInterruptEvent, 100, FALSE) + == WAIT_OBJECT_0) { + ResetEvent(hInterruptEvent); + Py_BLOCK_THREADS + sig = PyErr_CheckSignals(); + Py_UNBLOCK_THREADS + if (sig < 0) + break; + } + } + *readlen += n; + + /* If we didn't read a full buffer that time, don't try + again or we will block a second time. */ + if (n < len) + break; + /* If the buffer ended with a newline, break out */ + if (buf[*readlen - 1] == '\n') + break; + } + Py_END_ALLOW_THREADS + + if (sig) + goto error; + if (err) { + PyErr_SetFromWindowsErr(err); + goto error; + } + + if (*readlen > 0 && buf[0] == L'\x1a') { + PyMem_Free(buf); + buf = (wchar_t *)PyMem_Malloc(sizeof(wchar_t)); + if (!buf) + goto error; + buf[0] = L'\0'; + *readlen = 0; + } + + return buf; + +error: + if (buf) + PyMem_Free(buf); + return NULL; +} + + +static Py_ssize_t +readinto(winconsoleio *self, char *buf, Py_ssize_t len) +{ + if (self->handle == INVALID_HANDLE_VALUE) { + err_closed(); + return -1; + } + if (!self->readable) { + err_mode("reading"); + return -1; + } + if (len == 0) + return 0; + if (len > BUFMAX) { + PyErr_Format(PyExc_ValueError, "cannot read more than %d bytes", BUFMAX); + return -1; + } + + /* Each character may take up to 4 bytes in the final buffer. + This is highly conservative, but necessary to avoid + failure for any given Unicode input (e.g. \U0010ffff). + If the caller requests fewer than 4 bytes, we buffer one + character. + */ + DWORD wlen = (DWORD)(len / 4); + if (wlen == 0) { + wlen = 1; + } + + DWORD read_len = _copyfrombuf(self, buf, (DWORD)len); + if (read_len) { + buf = &buf[read_len]; + len -= read_len; + wlen -= 1; + } + if (len == read_len || wlen == 0) + return read_len; + + DWORD n; + wchar_t *wbuf = read_console_w(self->handle, wlen, &n); + if (wbuf == NULL) + return -1; + if (n == 0) { + PyMem_Free(wbuf); + return read_len; + } + + int err = 0; + DWORD u8n = 0; + + Py_BEGIN_ALLOW_THREADS + if (len < 4) { + if (WideCharToMultiByte(CP_UTF8, 0, wbuf, n, + self->buf, sizeof(self->buf) / sizeof(self->buf[0]), + NULL, NULL)) + u8n = _copyfrombuf(self, buf, (DWORD)len); + } else { + u8n = WideCharToMultiByte(CP_UTF8, 0, wbuf, n, + buf, (DWORD)len, NULL, NULL); + } + + if (u8n) { + read_len += u8n; + u8n = 0; + } else { + err = GetLastError(); + if (err == ERROR_INSUFFICIENT_BUFFER) { + /* Calculate the needed buffer for a more useful error, as this + means our "/ 4" logic above is insufficient for some input. + */ + u8n = WideCharToMultiByte(CP_UTF8, 0, wbuf, n, + NULL, 0, NULL, NULL); + } + } + Py_END_ALLOW_THREADS + + PyMem_Free(wbuf); + + if (u8n) { + PyErr_Format(PyExc_SystemError, + "Buffer had room for %d bytes but %d bytes required", + len, u8n); + return -1; + } + if (err) { + PyErr_SetFromWindowsErr(err); + return -1; + } + + return read_len; +} + +/*[clinic input] +_io._WindowsConsoleIO.readinto + buffer: Py_buffer(accept={rwbuffer}) + / + +Same as RawIOBase.readinto(). +[clinic start generated code]*/ + +static PyObject * +_io__WindowsConsoleIO_readinto_impl(winconsoleio *self, Py_buffer *buffer) +/*[clinic end generated code: output=66d1bdfa3f20af39 input=4ed68da48a6baffe]*/ +{ + Py_ssize_t len = readinto(self, buffer->buf, buffer->len); + if (len < 0) + return NULL; + + return PyLong_FromSsize_t(len); +} + +static DWORD +new_buffersize(winconsoleio *self, DWORD currentsize) +{ + DWORD addend; + + /* Expand the buffer by an amount proportional to the current size, + giving us amortized linear-time behavior. For bigger sizes, use a + less-than-double growth factor to avoid excessive allocation. */ + if (currentsize > 65536) + addend = currentsize >> 3; + else + addend = 256 + currentsize; + if (addend < SMALLCHUNK) + /* Avoid tiny read() calls. */ + addend = SMALLCHUNK; + return addend + currentsize; +} + +/*[clinic input] +_io._WindowsConsoleIO.readall + +Read all data from the console, returned as bytes. + +Return an empty bytes object at EOF. +[clinic start generated code]*/ + +static PyObject * +_io__WindowsConsoleIO_readall_impl(winconsoleio *self) +/*[clinic end generated code: output=e6d312c684f6e23b input=4024d649a1006e69]*/ +{ + wchar_t *buf; + DWORD bufsize, n, len = 0; + PyObject *bytes; + DWORD bytes_size, rn; + + if (self->handle == INVALID_HANDLE_VALUE) + return err_closed(); + + bufsize = BUFSIZ; + + buf = (wchar_t*)PyMem_Malloc((bufsize + 1) * sizeof(wchar_t)); + if (buf == NULL) + return NULL; + + while (1) { + wchar_t *subbuf; + + if (len >= (Py_ssize_t)bufsize) { + DWORD newsize = new_buffersize(self, len); + if (newsize > BUFMAX) + break; + if (newsize < bufsize) { + PyErr_SetString(PyExc_OverflowError, + "unbounded read returned more bytes " + "than a Python bytes object can hold"); + PyMem_Free(buf); + return NULL; + } + bufsize = newsize; + + buf = PyMem_Realloc(buf, (bufsize + 1) * sizeof(wchar_t)); + if (!buf) { + PyMem_Free(buf); + return NULL; + } + } + + subbuf = read_console_w(self->handle, bufsize - len, &n); + + if (subbuf == NULL) { + PyMem_Free(buf); + return NULL; + } + + if (n > 0) + wcsncpy_s(&buf[len], bufsize - len + 1, subbuf, n); + + PyMem_Free(subbuf); + + /* when the read starts with ^Z or is empty we break */ + if (n == 0 || buf[len] == '\x1a') + break; + + len += n; + } + + if (len > 0 && buf[0] == '\x1a' && _buflen(self) == 0) { + /* when the result starts with ^Z we return an empty buffer */ + PyMem_Free(buf); + return PyBytes_FromStringAndSize(NULL, 0); + } + + Py_BEGIN_ALLOW_THREADS + bytes_size = WideCharToMultiByte(CP_UTF8, 0, buf, len, + NULL, 0, NULL, NULL); + Py_END_ALLOW_THREADS + + if (!bytes_size) { + DWORD err = GetLastError(); + PyMem_Free(buf); + return PyErr_SetFromWindowsErr(err); + } + + bytes_size += _buflen(self); + bytes = PyBytes_FromStringAndSize(NULL, bytes_size); + rn = _copyfrombuf(self, PyBytes_AS_STRING(bytes), bytes_size); + + Py_BEGIN_ALLOW_THREADS + bytes_size = WideCharToMultiByte(CP_UTF8, 0, buf, len, + &PyBytes_AS_STRING(bytes)[rn], bytes_size - rn, NULL, NULL); + Py_END_ALLOW_THREADS + + if (!bytes_size) { + DWORD err = GetLastError(); + PyMem_Free(buf); + Py_CLEAR(bytes); + return PyErr_SetFromWindowsErr(err); + } + + PyMem_Free(buf); + if (bytes_size < (size_t)PyBytes_GET_SIZE(bytes)) { + if (_PyBytes_Resize(&bytes, n * sizeof(wchar_t)) < 0) { + Py_CLEAR(bytes); + return NULL; + } + } + return bytes; +} + +/*[clinic input] +_io._WindowsConsoleIO.read + size: io_ssize_t = -1 + / + +Read at most size bytes, returned as bytes. + +Only makes one system call when size is a positive integer, +so less data may be returned than requested. +Return an empty bytes object at EOF. +[clinic start generated code]*/ + +static PyObject * +_io__WindowsConsoleIO_read_impl(winconsoleio *self, Py_ssize_t size) +/*[clinic end generated code: output=57df68af9f4b22d0 input=6c56fceec460f1dd]*/ +{ + PyObject *bytes; + Py_ssize_t bytes_size; + + if (self->handle == INVALID_HANDLE_VALUE) + return err_closed(); + if (!self->readable) + return err_mode("reading"); + + if (size < 0) + return _io__WindowsConsoleIO_readall_impl(self); + if (size > BUFMAX) { + PyErr_Format(PyExc_ValueError, "cannot read more than %d bytes", BUFMAX); + return NULL; + } + + bytes = PyBytes_FromStringAndSize(NULL, size); + if (bytes == NULL) + return NULL; + + bytes_size = readinto(self, PyBytes_AS_STRING(bytes), PyBytes_GET_SIZE(bytes)); + if (bytes_size < 0) { + Py_CLEAR(bytes); + return NULL; + } + + if (bytes_size < PyBytes_GET_SIZE(bytes)) { + if (_PyBytes_Resize(&bytes, bytes_size) < 0) { + Py_CLEAR(bytes); + return NULL; + } + } + + return bytes; +} + +/*[clinic input] +_io._WindowsConsoleIO.write + b: Py_buffer + / + +Write buffer b to file, return number of bytes written. + +Only makes one system call, so not all of the data may be written. +The number of bytes actually written is returned. +[clinic start generated code]*/ + +static PyObject * +_io__WindowsConsoleIO_write_impl(winconsoleio *self, Py_buffer *b) +/*[clinic end generated code: output=775bdb16fbf9137b input=be35fb624f97c941]*/ +{ + BOOL res = TRUE; + wchar_t *wbuf; + DWORD len, wlen, n = 0; + + if (self->handle == INVALID_HANDLE_VALUE) + return err_closed(); + if (!self->writable) + return err_mode("writing"); + + if (b->len > BUFMAX) + len = BUFMAX; + else + len = (DWORD)b->len; + + Py_BEGIN_ALLOW_THREADS + wlen = MultiByteToWideChar(CP_UTF8, 0, b->buf, len, NULL, 0); + + /* issue11395 there is an unspecified upper bound on how many bytes + can be written at once. We cap at 32k - the caller will have to + handle partial writes. + Since we don't know how many input bytes are being ignored, we + have to reduce and recalculate. */ + while (wlen > 32766 / sizeof(wchar_t)) { + len /= 2; + wlen = MultiByteToWideChar(CP_UTF8, 0, b->buf, len, NULL, 0); + } + Py_END_ALLOW_THREADS + + if (!wlen) + return PyErr_SetFromWindowsErr(0); + + wbuf = (wchar_t*)PyMem_Malloc(wlen * sizeof(wchar_t)); + + Py_BEGIN_ALLOW_THREADS + wlen = MultiByteToWideChar(CP_UTF8, 0, b->buf, len, wbuf, wlen); + if (wlen) { + res = WriteConsoleW(self->handle, wbuf, wlen, &n, NULL); + if (n < wlen) { + /* Wrote fewer characters than expected, which means our + * len value may be wrong. So recalculate it from the + * characters that were written. As this could potentially + * result in a different value, we also validate that value. + */ + len = WideCharToMultiByte(CP_UTF8, 0, wbuf, n, + NULL, 0, NULL, NULL); + if (len) { + wlen = MultiByteToWideChar(CP_UTF8, 0, b->buf, len, + NULL, 0); + assert(wlen == len); + } + } + } else + res = 0; + Py_END_ALLOW_THREADS + + if (!res) { + DWORD err = GetLastError(); + PyMem_Free(wbuf); + return PyErr_SetFromWindowsErr(err); + } + + PyMem_Free(wbuf); + return PyLong_FromSsize_t(len); +} + +static PyObject * +winconsoleio_repr(winconsoleio *self) +{ + if (self->handle == INVALID_HANDLE_VALUE) + return PyUnicode_FromFormat("<_io._WindowsConsoleIO [closed]>"); + + if (self->readable) + return PyUnicode_FromFormat("<_io._WindowsConsoleIO mode='rb' closefd=%s>", + self->closehandle ? "True" : "False"); + if (self->writable) + return PyUnicode_FromFormat("<_io._WindowsConsoleIO mode='wb' closefd=%s>", + self->closehandle ? "True" : "False"); + + PyErr_SetString(PyExc_SystemError, "_WindowsConsoleIO has invalid mode"); + return NULL; +} + +/*[clinic input] +_io._WindowsConsoleIO.isatty + +Always True. +[clinic start generated code]*/ + +static PyObject * +_io__WindowsConsoleIO_isatty_impl(winconsoleio *self) +/*[clinic end generated code: output=9eac09d287c11bd7 input=9b91591dbe356f86]*/ +{ + if (self->handle == INVALID_HANDLE_VALUE) + return err_closed(); + + Py_RETURN_TRUE; +} + +static PyObject * +winconsoleio_getstate(winconsoleio *self) +{ + PyErr_Format(PyExc_TypeError, + "cannot serialize '%s' object", Py_TYPE(self)->tp_name); + return NULL; +} + +#include "clinic/winconsoleio.c.h" + +static PyMethodDef winconsoleio_methods[] = { + _IO__WINDOWSCONSOLEIO_READ_METHODDEF + _IO__WINDOWSCONSOLEIO_READALL_METHODDEF + _IO__WINDOWSCONSOLEIO_READINTO_METHODDEF + _IO__WINDOWSCONSOLEIO_WRITE_METHODDEF + _IO__WINDOWSCONSOLEIO_CLOSE_METHODDEF + _IO__WINDOWSCONSOLEIO_READABLE_METHODDEF + _IO__WINDOWSCONSOLEIO_WRITABLE_METHODDEF + _IO__WINDOWSCONSOLEIO_FILENO_METHODDEF + _IO__WINDOWSCONSOLEIO_ISATTY_METHODDEF + {"__getstate__", (PyCFunction)winconsoleio_getstate, METH_NOARGS, NULL}, + {NULL, NULL} /* sentinel */ +}; + +/* 'closed' and 'mode' are attributes for compatibility with FileIO. */ + +static PyObject * +get_closed(winconsoleio *self, void *closure) +{ + return PyBool_FromLong((long)(self->handle == INVALID_HANDLE_VALUE)); +} + +static PyObject * +get_closefd(winconsoleio *self, void *closure) +{ + return PyBool_FromLong((long)(self->closehandle)); +} + +static PyObject * +get_mode(winconsoleio *self, void *closure) +{ + return PyUnicode_FromString(self->readable ? "rb" : "wb"); +} + +static PyGetSetDef winconsoleio_getsetlist[] = { + {"closed", (getter)get_closed, NULL, "True if the file is closed"}, + {"closefd", (getter)get_closefd, NULL, + "True if the file descriptor will be closed by close()."}, + {"mode", (getter)get_mode, NULL, "String giving the file mode"}, + {NULL}, +}; + +static PyMemberDef winconsoleio_members[] = { + {"_blksize", T_UINT, offsetof(winconsoleio, blksize), 0}, + {"_finalizing", T_BOOL, offsetof(winconsoleio, finalizing), 0}, + {NULL} +}; + +PyTypeObject PyWindowsConsoleIO_Type = { + PyVarObject_HEAD_INIT(NULL, 0) + "_io._WindowsConsoleIO", + sizeof(winconsoleio), + 0, + (destructor)winconsoleio_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_reserved */ + (reprfunc)winconsoleio_repr, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE + | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_HAVE_FINALIZE, /* tp_flags */ + _io__WindowsConsoleIO___init____doc__, /* tp_doc */ + (traverseproc)winconsoleio_traverse, /* tp_traverse */ + (inquiry)winconsoleio_clear, /* tp_clear */ + 0, /* tp_richcompare */ + offsetof(winconsoleio, weakreflist), /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + winconsoleio_methods, /* tp_methods */ + winconsoleio_members, /* tp_members */ + winconsoleio_getsetlist, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + offsetof(winconsoleio, dict), /* tp_dictoffset */ + _io__WindowsConsoleIO___init__, /* tp_init */ + PyType_GenericAlloc, /* tp_alloc */ + winconsoleio_new, /* tp_new */ + PyObject_GC_Del, /* tp_free */ + 0, /* tp_is_gc */ + 0, /* tp_bases */ + 0, /* tp_mro */ + 0, /* tp_cache */ + 0, /* tp_subclasses */ + 0, /* tp_weaklist */ + 0, /* tp_del */ + 0, /* tp_version_tag */ + 0, /* tp_finalize */ +}; + +#endif /* MS_WINDOWS */ diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -270,6 +270,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -605,6 +605,9 @@ Modules\_io + + Modules\_io + Modules\_io diff --git a/Parser/myreadline.c b/Parser/myreadline.c --- a/Parser/myreadline.c +++ b/Parser/myreadline.c @@ -98,6 +98,100 @@ /* NOTREACHED */ } +#ifdef MS_WINDOWS +/* Readline implementation using ReadConsoleW */ + +extern char _get_console_type(HANDLE handle); + +char * +_PyOS_WindowsConsoleReadline(HANDLE hStdIn) +{ + static wchar_t wbuf_local[1024 * 16]; + const DWORD chunk_size = 1024; + + DWORD n_read, total_read, wbuflen, u8len; + wchar_t *wbuf; + char *buf = NULL; + int err = 0; + + n_read = 0; + total_read = 0; + wbuf = wbuf_local; + wbuflen = sizeof(wbuf_local) / sizeof(wbuf_local[0]) - 1; + while (1) { + if (!ReadConsoleW(hStdIn, &wbuf[total_read], wbuflen - total_read, &n_read, NULL)) { + err = GetLastError(); + goto exit; + } + if (n_read == 0) { + int s; + err = GetLastError(); + if (err != ERROR_OPERATION_ABORTED) + goto exit; + err = 0; + HANDLE hInterruptEvent = _PyOS_SigintEvent(); + if (WaitForSingleObjectEx(hInterruptEvent, 100, FALSE) + == WAIT_OBJECT_0) { + ResetEvent(hInterruptEvent); +#ifdef WITH_THREAD + PyEval_RestoreThread(_PyOS_ReadlineTState); +#endif + s = PyErr_CheckSignals(); +#ifdef WITH_THREAD + PyEval_SaveThread(); +#endif + if (s < 0) + goto exit; + } + break; + } + + total_read += n_read; + if (total_read == 0 || wbuf[total_read - 1] == L'\n') { + break; + } + wbuflen += chunk_size; + if (wbuf == wbuf_local) { + wbuf[total_read] = '\0'; + wbuf = (wchar_t*)PyMem_RawMalloc(wbuflen * sizeof(wchar_t)); + if (wbuf) + wcscpy_s(wbuf, wbuflen, wbuf_local); + } + else + wbuf = (wchar_t*)PyMem_RawRealloc(wbuf, wbuflen * sizeof(wchar_t)); + } + + if (wbuf[0] == '\x1a') { + buf = PyMem_RawMalloc(1); + if (buf) + buf[0] = '\0'; + goto exit; + } + + u8len = WideCharToMultiByte(CP_UTF8, 0, wbuf, total_read, NULL, 0, NULL, NULL); + buf = PyMem_RawMalloc(u8len + 1); + u8len = WideCharToMultiByte(CP_UTF8, 0, wbuf, total_read, buf, u8len, NULL, NULL); + buf[u8len] = '\0'; + +exit: + if (wbuf != wbuf_local) + PyMem_RawFree(wbuf); + + if (err) { +#ifdef WITH_THREAD + PyEval_RestoreThread(_PyOS_ReadlineTState); +#endif + PyErr_SetFromWindowsErr(err); +#ifdef WITH_THREAD + PyEval_SaveThread(); +#endif + } + + return buf; +} + +#endif + /* Readline implementation using fgets() */ @@ -107,6 +201,25 @@ size_t n; char *p, *pr; +#ifdef MS_WINDOWS + if (!Py_LegacyWindowsStdioFlag && sys_stdin == stdin) { + HANDLE hStdIn; + + _Py_BEGIN_SUPPRESS_IPH + hStdIn = (HANDLE)_get_osfhandle(fileno(sys_stdin)); + _Py_END_SUPPRESS_IPH + + if (_get_console_type(hStdIn) == 'r') { + fflush(sys_stdout); + if (prompt) + fprintf(stderr, "%s", prompt); + fflush(stderr); + clearerr(sys_stdin); + return _PyOS_WindowsConsoleReadline(hStdIn); + } + } +#endif + n = 100; p = (char *)PyMem_RawMalloc(n); if (p == NULL) diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -31,6 +31,9 @@ #ifdef MS_WINDOWS #undef BYTE #include "windows.h" + +extern PyTypeObject PyWindowsConsoleIO_Type; +#define PyWindowsConsoleIO_Check(op) (PyObject_TypeCheck((op), &PyWindowsConsoleIO_Type)) #endif _Py_IDENTIFIER(flush); @@ -92,6 +95,7 @@ int Py_IsolatedFlag = 0; /* for -I, isolate from user's env */ #ifdef MS_WINDOWS int Py_LegacyWindowsFSEncodingFlag = 0; /* Uses mbcs instead of utf-8 */ +int Py_LegacyWindowsStdioFlag = 0; /* Uses FileIO instead of WindowsConsoleIO */ #endif PyThreadState *_Py_Finalizing = NULL; @@ -154,6 +158,12 @@ return -3; } } +#ifdef MS_WINDOWS + if (_Py_StandardStreamEncoding) { + /* Overriding the stream encoding implies legacy streams */ + Py_LegacyWindowsStdioFlag = 1; + } +#endif return 0; } @@ -327,6 +337,8 @@ #ifdef MS_WINDOWS if ((p = Py_GETENV("PYTHONLEGACYWINDOWSFSENCODING")) && *p != '\0') Py_LegacyWindowsFSEncodingFlag = add_flag(Py_LegacyWindowsFSEncodingFlag, p); + if ((p = Py_GETENV("PYTHONLEGACYWINDOWSSTDIO")) && *p != '\0') + Py_LegacyWindowsStdioFlag = add_flag(Py_LegacyWindowsStdioFlag, p); #endif _PyRandom_Init(); @@ -1089,6 +1101,12 @@ Py_INCREF(raw); } +#ifdef MS_WINDOWS + /* Windows console IO is always UTF-8 encoded */ + if (PyWindowsConsoleIO_Check(raw)) + encoding = "utf-8"; +#endif + text = PyUnicode_FromString(name); if (text == NULL || _PyObject_SetAttrId(raw, &PyId_name, text) < 0) goto error; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:30:19 2016 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 08 Sep 2016 21:30:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_=5Bmerge_from_3=2E5=5D_-_Issue28010_-_Make_http=2Eclient?= =?utf-8?q?=2EHTTPConnection=2Eputrequest?= Message-ID: <20160908212951.19131.50491.91D3AD74@psf.io> https://hg.python.org/cpython/rev/93c7a893f9b4 changeset: 103359:93c7a893f9b4 parent: 103357:6142d2d3c471 parent: 103358:f6b62cf4a436 user: Senthil Kumaran date: Thu Sep 08 14:29:23 2016 -0700 summary: [merge from 3.5] - Issue28010 - Make http.client.HTTPConnection.putrequest documentation consistent with the code. files: Doc/library/http.client.rst | 24 +++++++++--------------- Lib/http/client.py | 3 ++- 2 files changed, 11 insertions(+), 16 deletions(-) diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -31,8 +31,7 @@ The module provides the following classes: -.. class:: HTTPConnection(host, port=None[, timeout], \ - source_address=None) +.. class:: HTTPConnection(host, port=None[, timeout], source_address=None) An :class:`HTTPConnection` instance represents one transaction with an HTTP server. It should be instantiated passing it a host and optional port @@ -196,7 +195,6 @@ The default port for the HTTP protocol (always ``80``). - .. data:: HTTPS_PORT The default port for the HTTPS protocol (always ``443``). @@ -340,14 +338,15 @@ also send your request step by step, by using the four functions below. -.. method:: HTTPConnection.putrequest(request, selector, skip_host=False, skip_accept_encoding=False) +.. method:: HTTPConnection.putrequest(method, url, skip_host=False, \ + skip_accept_encoding=False) - This should be the first call after the connection to the server has been made. - It sends a line to the server consisting of the *request* string, the *selector* - string, and the HTTP version (``HTTP/1.1``). To disable automatic sending of - ``Host:`` or ``Accept-Encoding:`` headers (for example to accept additional - content encodings), specify *skip_host* or *skip_accept_encoding* with non-False - values. + This should be the first call after the connection to the server has been + made. It sends a line to the server consisting of the *method* string, + the *url* string, and the HTTP version (``HTTP/1.1``). To disable automatic + sending of ``Host:`` or ``Accept-Encoding:`` headers (for example to accept + additional content encodings), specify *skip_host* or *skip_accept_encoding* + with non-False values. .. method:: HTTPConnection.putheader(header, argument[, ...]) @@ -425,7 +424,6 @@ return all of the values joined by ', '. If 'default' is any iterable other than a single string, its elements are similarly returned joined by commas. - .. method:: HTTPResponse.getheaders() Return a list of (header, value) tuples. @@ -440,22 +438,18 @@ headers. :class:`http.client.HTTPMessage` is a subclass of :class:`email.message.Message`. - .. attribute:: HTTPResponse.version HTTP protocol version used by server. 10 for HTTP/1.0, 11 for HTTP/1.1. - .. attribute:: HTTPResponse.status Status code returned by server. - .. attribute:: HTTPResponse.reason Reason phrase returned by server. - .. attribute:: HTTPResponse.debuglevel A debugging hook. If :attr:`debuglevel` is greater than zero, messages diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -1068,7 +1068,8 @@ # end chunked transfer self.send(b'0\r\n\r\n') - def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0): + def putrequest(self, method, url, skip_host=False, + skip_accept_encoding=False): """Send a request to the server. `method' specifies an HTTP request method, e.g. 'GET'. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:30:19 2016 From: python-checkins at python.org (senthil.kumaran) Date: Thu, 08 Sep 2016 21:30:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Issue28010_-_M?= =?utf-8?q?ake_http=2Eclient=2EHTTPConnection=2Eputrequest_documentation?= Message-ID: <20160908212951.13088.59468.2C499E2F@psf.io> https://hg.python.org/cpython/rev/f6b62cf4a436 changeset: 103358:f6b62cf4a436 branch: 3.5 parent: 103349:9b1f8c68de4c user: Senthil Kumaran date: Thu Sep 08 14:28:01 2016 -0700 summary: Issue28010 - Make http.client.HTTPConnection.putrequest documentation consistent with the code. files: Doc/library/http.client.rst | 24 +++++++++--------------- Lib/http/client.py | 3 ++- 2 files changed, 11 insertions(+), 16 deletions(-) diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -31,8 +31,7 @@ The module provides the following classes: -.. class:: HTTPConnection(host, port=None[, timeout], \ - source_address=None) +.. class:: HTTPConnection(host, port=None[, timeout], source_address=None) An :class:`HTTPConnection` instance represents one transaction with an HTTP server. It should be instantiated passing it a host and optional port @@ -196,7 +195,6 @@ The default port for the HTTP protocol (always ``80``). - .. data:: HTTPS_PORT The default port for the HTTPS protocol (always ``443``). @@ -318,14 +316,15 @@ also send your request step by step, by using the four functions below. -.. method:: HTTPConnection.putrequest(request, selector, skip_host=False, skip_accept_encoding=False) +.. method:: HTTPConnection.putrequest(method, url, skip_host=False, \ + skip_accept_encoding=False) - This should be the first call after the connection to the server has been made. - It sends a line to the server consisting of the *request* string, the *selector* - string, and the HTTP version (``HTTP/1.1``). To disable automatic sending of - ``Host:`` or ``Accept-Encoding:`` headers (for example to accept additional - content encodings), specify *skip_host* or *skip_accept_encoding* with non-False - values. + This should be the first call after the connection to the server has been + made. It sends a line to the server consisting of the *method* string, + the *url* string, and the HTTP version (``HTTP/1.1``). To disable automatic + sending of ``Host:`` or ``Accept-Encoding:`` headers (for example to accept + additional content encodings), specify *skip_host* or *skip_accept_encoding* + with non-False values. .. method:: HTTPConnection.putheader(header, argument[, ...]) @@ -384,7 +383,6 @@ return all of the values joined by ', '. If 'default' is any iterable other than a single string, its elements are similarly returned joined by commas. - .. method:: HTTPResponse.getheaders() Return a list of (header, value) tuples. @@ -399,22 +397,18 @@ headers. :class:`http.client.HTTPMessage` is a subclass of :class:`email.message.Message`. - .. attribute:: HTTPResponse.version HTTP protocol version used by server. 10 for HTTP/1.0, 11 for HTTP/1.1. - .. attribute:: HTTPResponse.status Status code returned by server. - .. attribute:: HTTPResponse.reason Reason phrase returned by server. - .. attribute:: HTTPResponse.debuglevel A debugging hook. If :attr:`debuglevel` is greater than zero, messages diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -935,7 +935,8 @@ if message_body is not None: self.send(message_body) - def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0): + def putrequest(self, method, url, skip_host=False, + skip_accept_encoding=False): """Send a request to the server. `method' specifies an HTTP request method, e.g. 'GET'. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:34:40 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 21:34:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Skips_console_open=5Ffd_te?= =?utf-8?q?sts_when_we_don=27t_have_real_consoles=2E?= Message-ID: <20160908213435.21136.73607.94EBB2B3@psf.io> https://hg.python.org/cpython/rev/7a14822547db changeset: 103360:7a14822547db user: Steve Dower date: Thu Sep 08 14:34:24 2016 -0700 summary: Skips console open_fd tests when we don't have real consoles. files: Lib/test/test_winconsoleio.py | 39 ++++++++++++---------- 1 files changed, 21 insertions(+), 18 deletions(-) diff --git a/Lib/test/test_winconsoleio.py b/Lib/test/test_winconsoleio.py --- a/Lib/test/test_winconsoleio.py +++ b/Lib/test/test_winconsoleio.py @@ -25,26 +25,29 @@ self.assertFalse(issubclass(ConIO, io.TextIOBase)) def test_open_fd(self): - f = ConIO(0) - self.assertTrue(f.readable()) - self.assertFalse(f.writable()) - self.assertEqual(0, f.fileno()) - f.close() # multiple close should not crash - f.close() + if sys.stdin.fileno() == 0: + f = ConIO(0) + self.assertTrue(f.readable()) + self.assertFalse(f.writable()) + self.assertEqual(0, f.fileno()) + f.close() # multiple close should not crash + f.close() - f = ConIO(1, 'w') - self.assertFalse(f.readable()) - self.assertTrue(f.writable()) - self.assertEqual(1, f.fileno()) - f.close() - f.close() + if sys.stdout.fileno() == 1: + f = ConIO(1, 'w') + self.assertFalse(f.readable()) + self.assertTrue(f.writable()) + self.assertEqual(1, f.fileno()) + f.close() + f.close() - f = ConIO(2, 'w') - self.assertFalse(f.readable()) - self.assertTrue(f.writable()) - self.assertEqual(2, f.fileno()) - f.close() - f.close() + if sys.stderr.fileno() == 2: + f = ConIO(2, 'w') + self.assertFalse(f.readable()) + self.assertTrue(f.writable()) + self.assertEqual(2, f.fileno()) + f.close() + f.close() def test_open_name(self): f = ConIO("CON") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:36:30 2016 From: python-checkins at python.org (steve.dower) Date: Thu, 08 Sep 2016 21:36:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_More_lenient_skipping_of_c?= =?utf-8?q?onsole_tests=2E?= Message-ID: <20160908213629.69700.51776.1C58E46C@psf.io> https://hg.python.org/cpython/rev/3c7fb7aa6d2c changeset: 103361:3c7fb7aa6d2c user: Steve Dower date: Thu Sep 08 14:36:18 2016 -0700 summary: More lenient skipping of console tests. files: Lib/test/test_winconsoleio.py | 18 +++++++++++++++--- 1 files changed, 15 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_winconsoleio.py b/Lib/test/test_winconsoleio.py --- a/Lib/test/test_winconsoleio.py +++ b/Lib/test/test_winconsoleio.py @@ -25,24 +25,36 @@ self.assertFalse(issubclass(ConIO, io.TextIOBase)) def test_open_fd(self): - if sys.stdin.fileno() == 0: + try: f = ConIO(0) + except ValueError: + # cannot open console because it's not a real console + pass + else: self.assertTrue(f.readable()) self.assertFalse(f.writable()) self.assertEqual(0, f.fileno()) f.close() # multiple close should not crash f.close() - if sys.stdout.fileno() == 1: + try: f = ConIO(1, 'w') + except ValueError: + # cannot open console because it's not a real console + pass + else: self.assertFalse(f.readable()) self.assertTrue(f.writable()) self.assertEqual(1, f.fileno()) f.close() f.close() - if sys.stderr.fileno() == 2: + try: f = ConIO(2, 'w') + except ValueError: + # cannot open console because it's not a real console + pass + else: self.assertFalse(f.readable()) self.assertTrue(f.writable()) self.assertEqual(2, f.fileno()) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:46:16 2016 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 08 Sep 2016 21:46:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Merge?= Message-ID: <20160908214615.69577.80184.A9D10C18@psf.io> https://hg.python.org/cpython/rev/dcd0fa6ab699 changeset: 103363:dcd0fa6ab699 parent: 103361:3c7fb7aa6d2c user: Raymond Hettinger date: Thu Sep 08 14:45:40 2016 -0700 summary: Merge files: Lib/test/test_set.py | 15 +++++++++++++++ Python/ceval.c | 6 ++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py --- a/Lib/test/test_set.py +++ b/Lib/test/test_set.py @@ -389,6 +389,21 @@ t = {1,2,3} self.assertEqual(s, t) + def test_set_literal_insertion_order(self): + # SF Issue #26020 -- Expect left to right insertion + s = {1, 1.0, True} + self.assertEqual(len(s), 1) + stored_value = s.pop() + self.assertEqual(type(stored_value), int) + + def test_set_literal_evaluation_order(self): + # Expect left to right expression evaluation + events = [] + def record(obj): + events.append(obj) + s = {record(1), record(2), record(3)} + self.assertEqual(events, [1, 2, 3]) + def test_hash(self): self.assertRaises(TypeError, hash, self.s) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2619,14 +2619,16 @@ TARGET(BUILD_SET) { PyObject *set = PySet_New(NULL); int err = 0; + int i; if (set == NULL) goto error; - while (--oparg >= 0) { - PyObject *item = POP(); + for (i = oparg; i > 0; i--) { + PyObject *item = PEEK(i); if (err == 0) err = PySet_Add(set, item); Py_DECREF(item); } + STACKADJ(-oparg); if (err != 0) { Py_DECREF(set); goto error; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:46:17 2016 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 08 Sep 2016 21:46:17 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MDIw?= =?utf-8?q?=3A__Fix_evaluation_order_for_set_literals?= Message-ID: <20160908214614.19052.76438.7C93F0F9@psf.io> https://hg.python.org/cpython/rev/0967531fc762 changeset: 103362:0967531fc762 branch: 3.5 parent: 103358:f6b62cf4a436 user: Raymond Hettinger date: Thu Sep 08 14:40:36 2016 -0700 summary: Issue #26020: Fix evaluation order for set literals files: Lib/test/test_set.py | 15 +++++++++++++++ Misc/NEWS | 2 ++ Python/ceval.c | 6 ++++-- 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py --- a/Lib/test/test_set.py +++ b/Lib/test/test_set.py @@ -388,6 +388,21 @@ t = {1,2,3} self.assertEqual(s, t) + def test_set_literal_insertion_order(self): + # SF Issue #26020 -- Expect left to right insertion + s = {1, 1.0, True} + self.assertEqual(len(s), 1) + stored_value = s.pop() + self.assertEqual(type(stored_value), int) + + def test_set_literal_evaluation_order(self): + # Expect left to right expression evaluation + events = [] + def record(obj): + events.append(obj) + s = {record(1), record(2), record(3)} + self.assertEqual(events, [1, 2, 3]) + def test_hash(self): self.assertRaises(TypeError, hash, self.s) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -20,6 +20,8 @@ after use of 'def' in _PyState_AddModule(). Initial patch by Christian Heimes. +- Issue #26020: set literal evaluation order did not match documented behaviour. + - Issue #27782: Multi-phase extension module import now correctly allows the ``m_methods`` field to be used to add module level functions to instances of non-module types returned from ``Py_create_mod``. Patch by Xiang Zhang. diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2580,14 +2580,16 @@ TARGET(BUILD_SET) { PyObject *set = PySet_New(NULL); int err = 0; + int i; if (set == NULL) goto error; - while (--oparg >= 0) { - PyObject *item = POP(); + for (i = oparg; i > 0; i--) { + PyObject *item = PEEK(i); if (err == 0) err = PySet_Add(set, item); Py_DECREF(item); } + STACKADJ(-oparg); if (err != 0) { Py_DECREF(set); goto error; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:51:33 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 21:51:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_skip_two_test?= =?utf-8?q?=5Fgdb_tests_that_fail_when_compiled_in_profile-opt_mode=3A?= Message-ID: <20160908215133.8893.21917.F3CDCA74@psf.io> https://hg.python.org/cpython/rev/c5b8de10aac2 changeset: 103364:c5b8de10aac2 branch: 2.7 parent: 103336:8567bc2876af user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 21:51:26 2016 +0000 summary: skip two test_gdb tests that fail when compiled in profile-opt mode: StackNavigationTests.test_up_at_top and PyBtTests.test_threads. Unfortunately we have no way of identifying if we were compiled profile-opt or not from what is in sysconfig.get_config_vars() so we just disable it for all optimized builds. This test suite crazy fragile. files: Lib/test/test_gdb.py | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -717,6 +717,8 @@ 'Unable to find a newer python frame\n') @unittest.skipUnless(HAS_PYUP_PYDOWN, "test requires py-up/py-down commands") + @unittest.skipIf(python_is_optimized(), + "Python was compiled with optimizations") def test_up_at_top(self): 'Verify handling of "py-up" at the top of the stack' bt = self.get_stack_trace(script=self.get_sample_script(), @@ -777,6 +779,8 @@ @unittest.skipUnless(thread, "Python was compiled without thread support") + @unittest.skipIf(python_is_optimized(), + "Python was compiled with optimizations") def test_threads(self): 'Verify that "py-bt" indicates threads that are waiting for the GIL' cmd = ''' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 17:57:19 2016 From: python-checkins at python.org (r.david.murray) Date: Thu, 08 Sep 2016 21:57:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_policy_keyword_to_emai?= =?utf-8?q?l=2Egenerator=2EDecodedGenerator=2E?= Message-ID: <20160908215719.8517.60660.4A29DC59@psf.io> https://hg.python.org/cpython/rev/9d50309875ff changeset: 103365:9d50309875ff parent: 103363:dcd0fa6ab699 user: R David Murray date: Thu Sep 08 17:57:06 2016 -0400 summary: Add policy keyword to email.generator.DecodedGenerator. files: Doc/library/email.generator.rst | 6 +++--- Doc/whatsnew/3.6.rst | 4 ++++ Lib/email/generator.py | 6 ++++-- Misc/NEWS | 2 ++ 4 files changed, 13 insertions(+), 5 deletions(-) diff --git a/Doc/library/email.generator.rst b/Doc/library/email.generator.rst --- a/Doc/library/email.generator.rst +++ b/Doc/library/email.generator.rst @@ -234,7 +234,8 @@ represented in the output stream by a string derived from a template filled in with information about the part. -.. class:: DecodedGenerator(outfp, mangle_from_=None, maxheaderlen=78, fmt=None) +.. class:: DecodedGenerator(outfp, mangle_from_=None, maxheaderlen=None, \ + fmt=None, *, policy=None) Act like :class:`Generator`, except that for any subpart of the message passed to :meth:`Generator.flatten`, if the subpart is of main type @@ -263,8 +264,7 @@ "[Non-text (%(type)s) part of message omitted, filename %(filename)s]" Optional *_mangle_from_* and *maxheaderlen* are as with the - :class:`Generator` base class, except that the default value for - *maxheaderlen* is ``78`` (the RFC standard default header length). + :class:`Generator` base class. .. rubric:: Footnotes diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -531,6 +531,9 @@ The :mod:`email.mime` classes now all accept an optional *policy* keyword. (Contributed by Berker Peksag in :issue:`27331`.) +The :class:`~email.generator.DecodedGenerator` now supports the *policy* +keyword. + encodings --------- @@ -538,6 +541,7 @@ On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP`` and the ``'ansi'`` alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page. + faulthandler ------------ diff --git a/Lib/email/generator.py b/Lib/email/generator.py --- a/Lib/email/generator.py +++ b/Lib/email/generator.py @@ -452,7 +452,8 @@ Like the Generator base class, except that non-text parts are substituted with a format string representing the part. """ - def __init__(self, outfp, mangle_from_=None, maxheaderlen=78, fmt=None): + def __init__(self, outfp, mangle_from_=None, maxheaderlen=None, fmt=None, *, + policy=None): """Like Generator.__init__() except that an additional optional argument is allowed. @@ -474,7 +475,8 @@ [Non-text (%(type)s) part of message omitted, filename %(filename)s] """ - Generator.__init__(self, outfp, mangle_from_, maxheaderlen) + Generator.__init__(self, outfp, mangle_from_, maxheaderlen, + policy=policy) if fmt is None: self._fmt = _FMT else: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -103,6 +103,8 @@ Library ------- +- email.generator.DecodedGenerator now supports the policy keyword. + - Issue #28027: Remove undocumented modules from ``Lib/plat-*``: IN, CDROM, DLFCN, TYPES, CDIO, and STROPTS. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:12:36 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:12:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Skip_unused_value_in_token?= =?utf-8?q?izer_code?= Message-ID: <20160908221236.87545.26394.A593FCCC@psf.io> https://hg.python.org/cpython/rev/08c08a3f5ac1 changeset: 103367:08c08a3f5ac1 user: Christian Heimes date: Fri Sep 09 00:09:45 2016 +0200 summary: Skip unused value in tokenizer code In the case of an escape character, c is never read. tok_next() is used to advance the pointer. CID 1225097 files: Parser/tokenizer.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1737,7 +1737,7 @@ else { end_quote_size = 0; if (c == '\\') - c = tok_nextc(tok); /* skip escaped char */ + tok_nextc(tok); /* skip escaped char */ } } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:12:37 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:12:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Check_return_value_of_PyLi?= =?utf-8?q?st=5FAppend=28=29_in_Py=5FMain=28=29=2E_CID_1353200?= Message-ID: <20160908221235.20970.26498.C6447845@psf.io> https://hg.python.org/cpython/rev/cce7a59396d6 changeset: 103366:cce7a59396d6 user: Christian Heimes date: Fri Sep 09 00:08:35 2016 +0200 summary: Check return value of PyList_Append() in Py_Main(). CID 1353200 files: Modules/main.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Modules/main.c b/Modules/main.c --- a/Modules/main.c +++ b/Modules/main.c @@ -482,7 +482,8 @@ warning_option = PyUnicode_FromWideChar(_PyOS_optarg, -1); if (warning_option == NULL) Py_FatalError("failure in handling of -W argument"); - PyList_Append(warning_options, warning_option); + if (PyList_Append(warning_options, warning_option) == -1) + Py_FatalError("failure in handling of -W argument"); Py_DECREF(warning_option); break; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:12:37 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:12:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Use_PyModule=5FAddIntMacro?= =?utf-8?q?=28=29_in_signal_module?= Message-ID: <20160908221236.2439.94733.8E8EFC32@psf.io> https://hg.python.org/cpython/rev/5d7892fa3e6d changeset: 103368:5d7892fa3e6d user: Christian Heimes date: Fri Sep 09 00:11:45 2016 +0200 summary: Use PyModule_AddIntMacro() in signal module The signal module was using old-style module initialization with potential NULL dereferencing. CID 1295026 files: Modules/signalmodule.c | 215 +++++++++++----------------- 1 files changed, 86 insertions(+), 129 deletions(-) diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -1266,210 +1266,169 @@ } #ifdef SIGHUP - x = PyLong_FromLong(SIGHUP); - PyDict_SetItemString(d, "SIGHUP", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGHUP)) + goto finally; #endif #ifdef SIGINT - x = PyLong_FromLong(SIGINT); - PyDict_SetItemString(d, "SIGINT", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGINT)) + goto finally; #endif #ifdef SIGBREAK - x = PyLong_FromLong(SIGBREAK); - PyDict_SetItemString(d, "SIGBREAK", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGBREAK)) + goto finally; #endif #ifdef SIGQUIT - x = PyLong_FromLong(SIGQUIT); - PyDict_SetItemString(d, "SIGQUIT", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGQUIT)) + goto finally; #endif #ifdef SIGILL - x = PyLong_FromLong(SIGILL); - PyDict_SetItemString(d, "SIGILL", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGILL)) + goto finally; #endif #ifdef SIGTRAP - x = PyLong_FromLong(SIGTRAP); - PyDict_SetItemString(d, "SIGTRAP", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGTRAP)) + goto finally; #endif #ifdef SIGIOT - x = PyLong_FromLong(SIGIOT); - PyDict_SetItemString(d, "SIGIOT", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGIOT)) + goto finally; #endif #ifdef SIGABRT - x = PyLong_FromLong(SIGABRT); - PyDict_SetItemString(d, "SIGABRT", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGABRT)) + goto finally; #endif #ifdef SIGEMT - x = PyLong_FromLong(SIGEMT); - PyDict_SetItemString(d, "SIGEMT", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGEMT)) + goto finally; #endif #ifdef SIGFPE - x = PyLong_FromLong(SIGFPE); - PyDict_SetItemString(d, "SIGFPE", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGFPE)) + goto finally; #endif #ifdef SIGKILL - x = PyLong_FromLong(SIGKILL); - PyDict_SetItemString(d, "SIGKILL", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGKILL)) + goto finally; #endif #ifdef SIGBUS - x = PyLong_FromLong(SIGBUS); - PyDict_SetItemString(d, "SIGBUS", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGBUS)) + goto finally; #endif #ifdef SIGSEGV - x = PyLong_FromLong(SIGSEGV); - PyDict_SetItemString(d, "SIGSEGV", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGSEGV)) + goto finally; #endif #ifdef SIGSYS - x = PyLong_FromLong(SIGSYS); - PyDict_SetItemString(d, "SIGSYS", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGSYS)) + goto finally; #endif #ifdef SIGPIPE - x = PyLong_FromLong(SIGPIPE); - PyDict_SetItemString(d, "SIGPIPE", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGPIPE)) + goto finally; #endif #ifdef SIGALRM - x = PyLong_FromLong(SIGALRM); - PyDict_SetItemString(d, "SIGALRM", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGALRM)) + goto finally; #endif #ifdef SIGTERM - x = PyLong_FromLong(SIGTERM); - PyDict_SetItemString(d, "SIGTERM", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGTERM)) + goto finally; #endif #ifdef SIGUSR1 - x = PyLong_FromLong(SIGUSR1); - PyDict_SetItemString(d, "SIGUSR1", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGUSR1)) + goto finally; #endif #ifdef SIGUSR2 - x = PyLong_FromLong(SIGUSR2); - PyDict_SetItemString(d, "SIGUSR2", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGUSR2)) + goto finally; #endif #ifdef SIGCLD - x = PyLong_FromLong(SIGCLD); - PyDict_SetItemString(d, "SIGCLD", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGCLD)) + goto finally; #endif #ifdef SIGCHLD - x = PyLong_FromLong(SIGCHLD); - PyDict_SetItemString(d, "SIGCHLD", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGCHLD)) + goto finally; #endif #ifdef SIGPWR - x = PyLong_FromLong(SIGPWR); - PyDict_SetItemString(d, "SIGPWR", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGPWR)) + goto finally; #endif #ifdef SIGIO - x = PyLong_FromLong(SIGIO); - PyDict_SetItemString(d, "SIGIO", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGIO)) + goto finally; #endif #ifdef SIGURG - x = PyLong_FromLong(SIGURG); - PyDict_SetItemString(d, "SIGURG", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGURG)) + goto finally; #endif #ifdef SIGWINCH - x = PyLong_FromLong(SIGWINCH); - PyDict_SetItemString(d, "SIGWINCH", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGWINCH)) + goto finally; #endif #ifdef SIGPOLL - x = PyLong_FromLong(SIGPOLL); - PyDict_SetItemString(d, "SIGPOLL", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGPOLL)) + goto finally; #endif #ifdef SIGSTOP - x = PyLong_FromLong(SIGSTOP); - PyDict_SetItemString(d, "SIGSTOP", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGSTOP)) + goto finally; #endif #ifdef SIGTSTP - x = PyLong_FromLong(SIGTSTP); - PyDict_SetItemString(d, "SIGTSTP", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGTSTP)) + goto finally; #endif #ifdef SIGCONT - x = PyLong_FromLong(SIGCONT); - PyDict_SetItemString(d, "SIGCONT", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGCONT)) + goto finally; #endif #ifdef SIGTTIN - x = PyLong_FromLong(SIGTTIN); - PyDict_SetItemString(d, "SIGTTIN", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGTTIN)) + goto finally; #endif #ifdef SIGTTOU - x = PyLong_FromLong(SIGTTOU); - PyDict_SetItemString(d, "SIGTTOU", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGTTOU)) + goto finally; #endif #ifdef SIGVTALRM - x = PyLong_FromLong(SIGVTALRM); - PyDict_SetItemString(d, "SIGVTALRM", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGVTALRM)) + goto finally; #endif #ifdef SIGPROF - x = PyLong_FromLong(SIGPROF); - PyDict_SetItemString(d, "SIGPROF", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGPROF)) + goto finally; #endif #ifdef SIGXCPU - x = PyLong_FromLong(SIGXCPU); - PyDict_SetItemString(d, "SIGXCPU", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGXCPU)) + goto finally; #endif #ifdef SIGXFSZ - x = PyLong_FromLong(SIGXFSZ); - PyDict_SetItemString(d, "SIGXFSZ", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGXFSZ)) + goto finally; #endif #ifdef SIGRTMIN - x = PyLong_FromLong(SIGRTMIN); - PyDict_SetItemString(d, "SIGRTMIN", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGRTMIN)) + goto finally; #endif #ifdef SIGRTMAX - x = PyLong_FromLong(SIGRTMAX); - PyDict_SetItemString(d, "SIGRTMAX", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGRTMAX)) + goto finally; #endif #ifdef SIGINFO - x = PyLong_FromLong(SIGINFO); - PyDict_SetItemString(d, "SIGINFO", x); - Py_XDECREF(x); + if (PyModule_AddIntMacro(m, SIGINFO)) + goto finally; #endif #ifdef ITIMER_REAL - x = PyLong_FromLong(ITIMER_REAL); - PyDict_SetItemString(d, "ITIMER_REAL", x); - Py_DECREF(x); + if (PyModule_AddIntMacro(m, ITIMER_REAL)) + goto finally; #endif #ifdef ITIMER_VIRTUAL - x = PyLong_FromLong(ITIMER_VIRTUAL); - PyDict_SetItemString(d, "ITIMER_VIRTUAL", x); - Py_DECREF(x); + if (PyModule_AddIntMacro(m, ITIMER_VIRTUAL)) + goto finally; #endif #ifdef ITIMER_PROF - x = PyLong_FromLong(ITIMER_PROF); - PyDict_SetItemString(d, "ITIMER_PROF", x); - Py_DECREF(x); + if (PyModule_AddIntMacro(m, ITIMER_PROF)) + goto finally; #endif #if defined (HAVE_SETITIMER) || defined (HAVE_GETITIMER) @@ -1480,15 +1439,13 @@ #endif #ifdef CTRL_C_EVENT - x = PyLong_FromLong(CTRL_C_EVENT); - PyDict_SetItemString(d, "CTRL_C_EVENT", x); - Py_DECREF(x); + if (PyModule_AddIntMacro(m, CTRL_C_EVENT)) + goto finally; #endif #ifdef CTRL_BREAK_EVENT - x = PyLong_FromLong(CTRL_BREAK_EVENT); - PyDict_SetItemString(d, "CTRL_BREAK_EVENT", x); - Py_DECREF(x); + if (PyModule_AddIntMacro(m, CTRL_BREAK_EVENT)) + goto finally; #endif #ifdef MS_WINDOWS -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:21:31 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:21:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_error_checking_to_PyIn?= =?utf-8?q?it=5Fpyexpact?= Message-ID: <20160908222130.69446.15795.322AD692@psf.io> https://hg.python.org/cpython/rev/1241ee79fc41 changeset: 103369:1241ee79fc41 user: Christian Heimes date: Fri Sep 09 00:13:35 2016 +0200 summary: Add error checking to PyInit_pyexpact The module initializer of the pyexpat module failed to check the return value of PySys_GetObject() for NULL. CID 982779 files: Modules/pyexpat.c | 12 +++++++++++- 1 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1701,7 +1701,15 @@ PyModule_AddStringConstant(m, "native_encoding", "UTF-8"); sys_modules = PySys_GetObject("modules"); + if (sys_modules == NULL) { + Py_DECREF(m); + return NULL; + } d = PyModule_GetDict(m); + if (d == NULL) { + Py_DECREF(m); + return NULL; + } errors_module = PyDict_GetItem(d, errmod_name); if (errors_module == NULL) { errors_module = PyModule_New(MODULE_NAME ".errors"); @@ -1722,9 +1730,11 @@ } } Py_DECREF(modelmod_name); - if (errors_module == NULL || model_module == NULL) + if (errors_module == NULL || model_module == NULL) { /* Don't core dump later! */ + Py_DECREF(m); return NULL; + } #if XML_COMBINED_VERSION > 19505 { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:21:31 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:21:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Additional_safe-guard_agai?= =?utf-8?q?nst_dereferencing_NULL_in_reduce=5Fnewobj?= Message-ID: <20160908222130.48560.87271.1D832F52@psf.io> https://hg.python.org/cpython/rev/5861d9e8463d changeset: 103371:5861d9e8463d user: Christian Heimes date: Fri Sep 09 00:21:22 2016 +0200 summary: Additional safe-guard against dereferencing NULL in reduce_newobj _PyObject_GetNewArguments() can leave args == NULL but the __newobj_ex__ branch expects args to be not-NULL. CID 1353201 files: Objects/typeobject.c | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -4263,7 +4263,7 @@ } Py_XDECREF(args); } - else { + else if (args != NULL) { _Py_IDENTIFIER(__newobj_ex__); newobj = _PyObject_GetAttrId(copyreg, &PyId___newobj_ex__); @@ -4281,6 +4281,12 @@ return NULL; } } + else { + /* args == NULL */ + Py_DECREF(kwargs); + PyErr_BadInternalCall(); + return NULL; + } state = _PyObject_GetState(obj, !hasargs && !PyList_Check(obj) && !PyDict_Check(obj)); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:21:31 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:21:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_NULL_check_for_gen-=3E?= =?utf-8?b?Z2lfY29kZSBpbiBnZW5fc2VuZF9leCgp?= Message-ID: <20160908222130.59651.98861.A35480B0@psf.io> https://hg.python.org/cpython/rev/9336b31aa6b3 changeset: 103370:9336b31aa6b3 user: Christian Heimes date: Fri Sep 09 00:20:13 2016 +0200 summary: Add NULL check for gen->gi_code in gen_send_ex() _PyGen_Finalize() checks that gen->gi_code is not NULL before it accesses the flags of the code object. This means that the flag could be NULL. It passes down the generatore to gen_close() and gen_send_ex(). gen_send_ex() did not check for gen->gi_code != NULL. CID 1297900 files: Objects/genobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -167,7 +167,7 @@ /* Check for __future__ generator_stop and conditionally turn * a leaking StopIteration into RuntimeError (with its cause * set appropriately). */ - if (((PyCodeObject *)gen->gi_code)->co_flags & + if (gen->gi_code != NULL && ((PyCodeObject *)gen->gi_code)->co_flags & (CO_FUTURE_GENERATOR_STOP | CO_COROUTINE | CO_ITERABLE_COROUTINE)) { PyObject *exc, *val, *val2, *tb; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:23:17 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 22:23:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_replace_PyInt16_with_int16?= =?utf-8?q?=5Ft?= Message-ID: <20160908222316.20950.25624.179C7F92@psf.io> https://hg.python.org/cpython/rev/a347cc6eb050 changeset: 103372:a347cc6eb050 user: Benjamin Peterson date: Thu Sep 08 15:08:02 2016 -0700 summary: replace PyInt16 with int16_t files: Modules/audioop.c | 25 +++++++++++-------------- 1 files changed, 11 insertions(+), 14 deletions(-) diff --git a/Modules/audioop.c b/Modules/audioop.c --- a/Modules/audioop.c +++ b/Modules/audioop.c @@ -4,9 +4,6 @@ #define PY_SSIZE_T_CLEAN #include "Python.h" -#include - -typedef short PyInt16; #if defined(__CHAR_UNSIGNED__) #if defined(signed) @@ -52,15 +49,15 @@ #define SEG_SHIFT (4) /* Left shift for segment number. */ #define SEG_MASK (0x70) /* Segment field mask. */ -static const PyInt16 seg_aend[8] = { +static const int16_t seg_aend[8] = { 0x1F, 0x3F, 0x7F, 0xFF, 0x1FF, 0x3FF, 0x7FF, 0xFFF }; -static const PyInt16 seg_uend[8] = { +static const int16_t seg_uend[8] = { 0x3F, 0x7F, 0xFF, 0x1FF, 0x3FF, 0x7FF, 0xFFF, 0x1FFF }; -static PyInt16 -search(PyInt16 val, const PyInt16 *table, int size) +static int16_t +search(int16_t val, const int16_t *table, int size) { int i; @@ -73,7 +70,7 @@ #define st_ulaw2linear16(uc) (_st_ulaw2linear16[uc]) #define st_alaw2linear16(uc) (_st_alaw2linear16[uc]) -static const PyInt16 _st_ulaw2linear16[256] = { +static const int16_t _st_ulaw2linear16[256] = { -32124, -31100, -30076, -29052, -28028, -27004, -25980, -24956, -23932, -22908, -21884, -20860, -19836, -18812, -17788, -16764, -15996, -15484, -14972, -14460, -13948, @@ -146,10 +143,10 @@ * John Wiley & Sons, pps 98-111 and 472-476. */ static unsigned char -st_14linear2ulaw(PyInt16 pcm_val) /* 2's complement (14-bit range) */ +st_14linear2ulaw(int16_t pcm_val) /* 2's complement (14-bit range) */ { - PyInt16 mask; - PyInt16 seg; + int16_t mask; + int16_t seg; unsigned char uval; /* u-law inverts all bits */ @@ -179,7 +176,7 @@ } -static const PyInt16 _st_alaw2linear16[256] = { +static const int16_t _st_alaw2linear16[256] = { -5504, -5248, -6016, -5760, -4480, -4224, -4992, -4736, -7552, -7296, -8064, -7808, -6528, -6272, -7040, -6784, -2752, -2624, -3008, -2880, -2240, @@ -240,9 +237,9 @@ * John Wiley & Sons, pps 98-111 and 472-476. */ static unsigned char -st_linear2alaw(PyInt16 pcm_val) /* 2's complement (13-bit range) */ +st_linear2alaw(int16_t pcm_val) /* 2's complement (13-bit range) */ { - PyInt16 mask; + int16_t mask; short seg; unsigned char aval; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:24:33 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:24:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_NULL_checks_to_the_ini?= =?utf-8?q?tializer_of_the_locale_module?= Message-ID: <20160908222432.21136.18420.2AB28BBE@psf.io> https://hg.python.org/cpython/rev/c5d4d1f480f5 changeset: 103374:c5d4d1f480f5 user: Christian Heimes date: Fri Sep 09 00:24:12 2016 +0200 summary: Add NULL checks to the initializer of the locale module The _locale module was using old-style APIs to set numeric module constants from macros. The new way requires less code and properly checks for NULL. CID 1295027 files: Modules/_localemodule.c | 54 ++++++++++------------------ 1 files changed, 20 insertions(+), 34 deletions(-) diff --git a/Modules/_localemodule.c b/Modules/_localemodule.c --- a/Modules/_localemodule.c +++ b/Modules/_localemodule.c @@ -621,53 +621,34 @@ PyMODINIT_FUNC PyInit__locale(void) { - PyObject *m, *d, *x; + PyObject *m; #ifdef HAVE_LANGINFO_H int i; #endif m = PyModule_Create(&_localemodule); if (m == NULL) - return NULL; + return NULL; - d = PyModule_GetDict(m); - - x = PyLong_FromLong(LC_CTYPE); - PyDict_SetItemString(d, "LC_CTYPE", x); - Py_XDECREF(x); - - x = PyLong_FromLong(LC_TIME); - PyDict_SetItemString(d, "LC_TIME", x); - Py_XDECREF(x); - - x = PyLong_FromLong(LC_COLLATE); - PyDict_SetItemString(d, "LC_COLLATE", x); - Py_XDECREF(x); - - x = PyLong_FromLong(LC_MONETARY); - PyDict_SetItemString(d, "LC_MONETARY", x); - Py_XDECREF(x); + PyModule_AddIntMacro(m, LC_CTYPE); + PyModule_AddIntMacro(m, LC_TIME); + PyModule_AddIntMacro(m, LC_COLLATE); + PyModule_AddIntMacro(m, LC_MONETARY); #ifdef LC_MESSAGES - x = PyLong_FromLong(LC_MESSAGES); - PyDict_SetItemString(d, "LC_MESSAGES", x); - Py_XDECREF(x); + PyModule_AddIntMacro(m, LC_MESSAGES); #endif /* LC_MESSAGES */ - x = PyLong_FromLong(LC_NUMERIC); - PyDict_SetItemString(d, "LC_NUMERIC", x); - Py_XDECREF(x); - - x = PyLong_FromLong(LC_ALL); - PyDict_SetItemString(d, "LC_ALL", x); - Py_XDECREF(x); - - x = PyLong_FromLong(CHAR_MAX); - PyDict_SetItemString(d, "CHAR_MAX", x); - Py_XDECREF(x); + PyModule_AddIntMacro(m, LC_NUMERIC); + PyModule_AddIntMacro(m, LC_ALL); + PyModule_AddIntMacro(m, CHAR_MAX); Error = PyErr_NewException("locale.Error", NULL, NULL); - PyDict_SetItemString(d, "Error", Error); + if (Error == NULL) { + Py_DECREF(m); + return NULL; + } + PyModule_AddObject(m, "Error", Error); #ifdef HAVE_LANGINFO_H for (i = 0; langinfo_constants[i].name; i++) { @@ -675,6 +656,11 @@ langinfo_constants[i].value); } #endif + + if (PyErr_Occurred()) { + Py_DECREF(m); + return NULL; + } return m; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:24:32 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:24:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_potential_NULL_pointer?= =?utf-8?q?_dereference_in_update=5Fsymbols=28=29?= Message-ID: <20160908222432.1680.95880.F1A2655B@psf.io> https://hg.python.org/cpython/rev/61052fd221fa changeset: 103373:61052fd221fa user: Christian Heimes date: Fri Sep 09 00:22:28 2016 +0200 summary: Fix potential NULL pointer dereference in update_symbols() symtable_analyze() calls analyze_block() with bound=NULL. Theoretically that NULL can be passed down to update_symbols(). update_symbols() may deference NULL and pass it to PySet_Contains() files: Python/symtable.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Python/symtable.c b/Python/symtable.c --- a/Python/symtable.c +++ b/Python/symtable.c @@ -652,7 +652,7 @@ continue; } /* Handle global symbol */ - if (!PySet_Contains(bound, name)) { + if (bound && !PySet_Contains(bound, name)) { Py_DECREF(name); continue; /* it's a global */ } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:25:34 2016 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 08 Sep 2016 22:25:34 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2MDIw?= =?utf-8?q?=3A_Fix_evaluation_order_for_set_literals?= Message-ID: <20160908222534.19592.72250.E6E135E5@psf.io> https://hg.python.org/cpython/rev/a8d504600c18 changeset: 103375:a8d504600c18 branch: 2.7 parent: 103364:c5b8de10aac2 user: Raymond Hettinger date: Thu Sep 08 15:25:19 2016 -0700 summary: Issue #26020: Fix evaluation order for set literals files: Lib/test/test_set.py | 15 +++++++++++++++ Python/ceval.c | 6 ++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py --- a/Lib/test/test_set.py +++ b/Lib/test/test_set.py @@ -360,6 +360,21 @@ t = self.thetype(s) self.assertNotEqual(id(s), id(t)) + def test_set_literal_insertion_order(self): + # SF Issue #26020 -- Expect left to right insertion + s = {1, 1.0, True} + self.assertEqual(len(s), 1) + stored_value = s.pop() + self.assertEqual(type(stored_value), int) + + def test_set_literal_evaluation_order(self): + # Expect left to right expression evaluation + events = [] + def record(obj): + events.append(obj) + s = {record(1), record(2), record(3)} + self.assertEqual(events, [1, 2, 3]) + def test_hash(self): self.assertRaises(TypeError, hash, self.s) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2477,14 +2477,16 @@ TARGET(BUILD_SET) { + int i; x = PySet_New(NULL); if (x != NULL) { - for (; --oparg >= 0;) { - w = POP(); + for (i = oparg; i > 0; i--) { + w = PEEK(i); if (err == 0) err = PySet_Add(x, w); Py_DECREF(w); } + STACKADJ(-oparg); if (err != 0) { Py_DECREF(x); break; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:27:39 2016 From: python-checkins at python.org (raymond.hettinger) Date: Thu, 08 Sep 2016 22:27:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2MDIw?= =?utf-8?q?=3A_Add_news_entry?= Message-ID: <20160908222739.36323.49489.251213D1@psf.io> https://hg.python.org/cpython/rev/aa2bf603a9bd changeset: 103376:aa2bf603a9bd branch: 2.7 user: Raymond Hettinger date: Thu Sep 08 15:27:27 2016 -0700 summary: Issue #26020: Add news entry files: Misc/NEWS | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -14,6 +14,8 @@ - Issue #26307: The profile-opt build now applys PGO to the built-in modules. +- Issue #26020: set literal evaluation order did not match documented behaviour. + - Issue #27870: A left shift of zero by a large integer no longer attempts to allocate large amounts of memory. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:28:52 2016 From: python-checkins at python.org (r.david.murray) Date: Thu, 08 Sep 2016 22:28:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_24277=3A_Make_it_clearer_t?= =?utf-8?q?hat_the_new_modules_are_not_provisional=2E?= Message-ID: <20160908222852.13248.78093.8CDF5E50@psf.io> https://hg.python.org/cpython/rev/a06299a4b00c changeset: 103377:a06299a4b00c parent: 103374:c5d4d1f480f5 user: R David Murray date: Thu Sep 08 18:28:43 2016 -0400 summary: 24277: Make it clearer that the new modules are not provisional. Also make it clear on the contents page what chapters are about the legacy API. files: Doc/library/email.contentmanager.rst | 10 +++++++--- Doc/library/email.headerregistry.rst | 10 +++++++--- Doc/library/email.message.rst | 15 ++++++++------- Doc/library/email.policy.rst | 10 +++++++--- Doc/library/email.rst | 9 +++++++++ 5 files changed, 38 insertions(+), 16 deletions(-) diff --git a/Doc/library/email.contentmanager.rst b/Doc/library/email.contentmanager.rst --- a/Doc/library/email.contentmanager.rst +++ b/Doc/library/email.contentmanager.rst @@ -11,9 +11,7 @@ ------------ -.. versionadded:: 3.4 as a :term:`provisional module `. - -.. versionchanged:: 3.6 provisional status removed. +.. versionadded:: 3.6 [1]_ .. class:: ContentManager() @@ -201,3 +199,9 @@ ``headername: headervalue`` or a list of ``header`` objects (distinguished from strings by having a ``name`` attribute), add the headers to *msg*. + + +.. rubric:: Footnotes + +.. [1] Oringally added in 3.4 as a :term:`provisional module ` diff --git a/Doc/library/email.headerregistry.rst b/Doc/library/email.headerregistry.rst --- a/Doc/library/email.headerregistry.rst +++ b/Doc/library/email.headerregistry.rst @@ -11,9 +11,7 @@ -------------- -.. versionadded:: 3.3 as a :term:`provisional module `. - -.. versionchanged:: 3.6 provisonal status removed. +.. versionadded:: 3.6 [1]_ Headers are represented by customized subclasses of :class:`str`. The particular class used to represent a given header is determined by the @@ -449,3 +447,9 @@ ``display_name`` is none and there is a single ``Address`` in the ``addresses`` list, the ``str`` value will be the same as the ``str`` of that single ``Address``. + + +.. rubric:: Footnotes + +.. [1] Oringally added in 3.3 as a :term:`provisional module ` diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -11,13 +11,7 @@ -------------- -.. versionadded:: 3.4 - the classes documented here were added :term:`provisionaly `. - -.. versionchanged:: 3.6 - provisional status removed, docs for legacy message class moved - to :ref:`compat32_message`. +.. versionadded:: 3.6 [1]_ The central class in the :mod:`email` package is the :class:`EmailMessage` class, imported from the :mod:`email.message` module. It is the base class for @@ -748,3 +742,10 @@ :class:`EmailMessage`, except that no :mailheader:`MIME-Version` headers are added when :meth:`~EmailMessage.set_content` is called, since sub-parts do not need their own :mailheader:`MIME-Version` headers. + + +.. rubric:: Footnotes + +.. [1] Oringally added in 3.4 as a :term:`provisional module `. Docs for legacy message class moved to + :ref:`compat32_message`. diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -371,9 +371,7 @@ In addition to the settable attributes listed above that apply to all policies, this policy adds the following additional attributes: - .. versionadded:: 3.3 as a :term:`provisional feature `. - - .. versionchanged:: 3.6 provisional status removed. + .. versionadded:: 3.6 [1]_ .. attribute:: utf8 @@ -634,3 +632,9 @@ An instance of :class:`Compat32`, providing backward compatibility with the behavior of the email package in Python 3.2. + + +.. rubric:: Footnotes + +.. [1] Oringally added in 3.3 as a :term:`provisional feature `. diff --git a/Doc/library/email.rst b/Doc/library/email.rst --- a/Doc/library/email.rst +++ b/Doc/library/email.rst @@ -97,6 +97,11 @@ are still using the :mod:`~email.policy.compat32` API for backward compatibility reasons. +.. versionchanged:: 3.6 + Docs reorganized and rewritten to promote the new + :class:`~email.message.EmailMessage`/:class:`~email.policy.EmailPolicy` + API. + Contents of the :mod:`email` package documentation: .. toctree:: @@ -112,6 +117,10 @@ email.examples.rst +Legacy API: + +.. toctree:: + email.compat32-message.rst email.mime.rst email.header.rst -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:31:44 2016 From: python-checkins at python.org (eric.snow) Date: Thu, 08 Sep 2016 22:31:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324254=3A_Drop_cls?= =?utf-8?b?Ll9fZGVmaW5pdGlvbl9vcmRlcl9fLg==?= Message-ID: <20160908223143.69669.70256.67C601A3@psf.io> https://hg.python.org/cpython/rev/9aa5424fd1df changeset: 103378:9aa5424fd1df user: Eric Snow date: Thu Sep 08 15:11:11 2016 -0700 summary: Issue #24254: Drop cls.__definition_order__. files: Doc/library/inspect.rst | 367 +++++++++--------- Doc/library/types.rst | 10 +- Doc/reference/compound_stmts.rst | 12 +- Doc/reference/datamodel.rst | 7 - Include/object.h | 2 - Include/odictobject.h | 4 - Lib/test/test_builtin.py | 192 +--------- Lib/test/test_metaclass.py | 11 +- Lib/test/test_pydoc.py | 8 +- Lib/test/test_sys.py | 2 +- Lib/test/test_types.py | 22 - Lib/types.py | 5 +- Lib/typing.py | 1 - Objects/odictobject.c | 15 - Objects/typeobject.c | 66 +--- Python/bltinmodule.c | 2 +- 16 files changed, 193 insertions(+), 533 deletions(-) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -34,190 +34,185 @@ They also help you determine when you can expect to find the following special attributes: -+-----------+----------------------+---------------------------+ -| Type | Attribute | Description | -+===========+======================+===========================+ -| module | __doc__ | documentation string | -+-----------+----------------------+---------------------------+ -| | __file__ | filename (missing for | -| | | built-in modules) | -+-----------+----------------------+---------------------------+ -| class | __doc__ | documentation string | -+-----------+----------------------+---------------------------+ -| | __name__ | name with which this | -| | | class was defined | -+-----------+----------------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+----------------------+---------------------------+ -| | __module__ | name of module in which | -| | | this class was defined | -+-----------+----------------------+---------------------------+ -| | __definition_order__ | the names of the class's | -| | | attributes, in the order | -| | | in which they were | -| | | defined (if known) | -+-----------+----------------------+---------------------------+ -| method | __doc__ | documentation string | -+-----------+----------------------+---------------------------+ -| | __name__ | name with which this | -| | | method was defined | -+-----------+----------------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+----------------------+---------------------------+ -| | __func__ | function object | -| | | containing implementation | -| | | of method | -+-----------+----------------------+---------------------------+ -| | __self__ | instance to which this | -| | | method is bound, or | -| | | ``None`` | -+-----------+----------------------+---------------------------+ -| function | __doc__ | documentation string | -+-----------+----------------------+---------------------------+ -| | __name__ | name with which this | -| | | function was defined | -+-----------+----------------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+----------------------+---------------------------+ -| | __code__ | code object containing | -| | | compiled function | -| | | :term:`bytecode` | -+-----------+----------------------+---------------------------+ -| | __defaults__ | tuple of any default | -| | | values for positional or | -| | | keyword parameters | -+-----------+----------------------+---------------------------+ -| | __kwdefaults__ | mapping of any default | -| | | values for keyword-only | -| | | parameters | -+-----------+----------------------+---------------------------+ -| | __globals__ | global namespace in which | -| | | this function was defined | -+-----------+----------------------+---------------------------+ -| | __annotations__ | mapping of parameters | -| | | names to annotations; | -| | | ``"return"`` key is | -| | | reserved for return | -| | | annotations. | -+-----------+----------------------+---------------------------+ -| traceback | tb_frame | frame object at this | -| | | level | -+-----------+----------------------+---------------------------+ -| | tb_lasti | index of last attempted | -| | | instruction in bytecode | -+-----------+----------------------+---------------------------+ -| | tb_lineno | current line number in | -| | | Python source code | -+-----------+----------------------+---------------------------+ -| | tb_next | next inner traceback | -| | | object (called by this | -| | | level) | -+-----------+----------------------+---------------------------+ -| frame | f_back | next outer frame object | -| | | (this frame's caller) | -+-----------+----------------------+---------------------------+ -| | f_builtins | builtins namespace seen | -| | | by this frame | -+-----------+----------------------+---------------------------+ -| | f_code | code object being | -| | | executed in this frame | -+-----------+----------------------+---------------------------+ -| | f_globals | global namespace seen by | -| | | this frame | -+-----------+----------------------+---------------------------+ -| | f_lasti | index of last attempted | -| | | instruction in bytecode | -+-----------+----------------------+---------------------------+ -| | f_lineno | current line number in | -| | | Python source code | -+-----------+----------------------+---------------------------+ -| | f_locals | local namespace seen by | -| | | this frame | -+-----------+----------------------+---------------------------+ -| | f_restricted | 0 or 1 if frame is in | -| | | restricted execution mode | -+-----------+----------------------+---------------------------+ -| | f_trace | tracing function for this | -| | | frame, or ``None`` | -+-----------+----------------------+---------------------------+ -| code | co_argcount | number of arguments (not | -| | | including \* or \*\* | -| | | args) | -+-----------+----------------------+---------------------------+ -| | co_code | string of raw compiled | -| | | bytecode | -+-----------+----------------------+---------------------------+ -| | co_consts | tuple of constants used | -| | | in the bytecode | -+-----------+----------------------+---------------------------+ -| | co_filename | name of file in which | -| | | this code object was | -| | | created | -+-----------+----------------------+---------------------------+ -| | co_firstlineno | number of first line in | -| | | Python source code | -+-----------+----------------------+---------------------------+ -| | co_flags | bitmap: 1=optimized ``|`` | -| | | 2=newlocals ``|`` 4=\*arg | -| | | ``|`` 8=\*\*arg | -+-----------+----------------------+---------------------------+ -| | co_lnotab | encoded mapping of line | -| | | numbers to bytecode | -| | | indices | -+-----------+----------------------+---------------------------+ -| | co_name | name with which this code | -| | | object was defined | -+-----------+----------------------+---------------------------+ -| | co_names | tuple of names of local | -| | | variables | -+-----------+----------------------+---------------------------+ -| | co_nlocals | number of local variables | -+-----------+----------------------+---------------------------+ -| | co_stacksize | virtual machine stack | -| | | space required | -+-----------+----------------------+---------------------------+ -| | co_varnames | tuple of names of | -| | | arguments and local | -| | | variables | -+-----------+----------------------+---------------------------+ -| generator | __name__ | name | -+-----------+----------------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+----------------------+---------------------------+ -| | gi_frame | frame | -+-----------+----------------------+---------------------------+ -| | gi_running | is the generator running? | -+-----------+----------------------+---------------------------+ -| | gi_code | code | -+-----------+----------------------+---------------------------+ -| | gi_yieldfrom | object being iterated by | -| | | ``yield from``, or | -| | | ``None`` | -+-----------+----------------------+---------------------------+ -| coroutine | __name__ | name | -+-----------+----------------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+----------------------+---------------------------+ -| | cr_await | object being awaited on, | -| | | or ``None`` | -+-----------+----------------------+---------------------------+ -| | cr_frame | frame | -+-----------+----------------------+---------------------------+ -| | cr_running | is the coroutine running? | -+-----------+----------------------+---------------------------+ -| | cr_code | code | -+-----------+----------------------+---------------------------+ -| builtin | __doc__ | documentation string | -+-----------+----------------------+---------------------------+ -| | __name__ | original name of this | -| | | function or method | -+-----------+----------------------+---------------------------+ -| | __qualname__ | qualified name | -+-----------+----------------------+---------------------------+ -| | __self__ | instance to which a | -| | | method is bound, or | -| | | ``None`` | -+-----------+----------------------+---------------------------+ ++-----------+-----------------+---------------------------+ +| Type | Attribute | Description | ++===========+=================+===========================+ +| module | __doc__ | documentation string | ++-----------+-----------------+---------------------------+ +| | __file__ | filename (missing for | +| | | built-in modules) | ++-----------+-----------------+---------------------------+ +| class | __doc__ | documentation string | ++-----------+-----------------+---------------------------+ +| | __name__ | name with which this | +| | | class was defined | ++-----------+-----------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+-----------------+---------------------------+ +| | __module__ | name of module in which | +| | | this class was defined | ++-----------+-----------------+---------------------------+ +| method | __doc__ | documentation string | ++-----------+-----------------+---------------------------+ +| | __name__ | name with which this | +| | | method was defined | ++-----------+-----------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+-----------------+---------------------------+ +| | __func__ | function object | +| | | containing implementation | +| | | of method | ++-----------+-----------------+---------------------------+ +| | __self__ | instance to which this | +| | | method is bound, or | +| | | ``None`` | ++-----------+-----------------+---------------------------+ +| function | __doc__ | documentation string | ++-----------+-----------------+---------------------------+ +| | __name__ | name with which this | +| | | function was defined | ++-----------+-----------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+-----------------+---------------------------+ +| | __code__ | code object containing | +| | | compiled function | +| | | :term:`bytecode` | ++-----------+-----------------+---------------------------+ +| | __defaults__ | tuple of any default | +| | | values for positional or | +| | | keyword parameters | ++-----------+-----------------+---------------------------+ +| | __kwdefaults__ | mapping of any default | +| | | values for keyword-only | +| | | parameters | ++-----------+-----------------+---------------------------+ +| | __globals__ | global namespace in which | +| | | this function was defined | ++-----------+-----------------+---------------------------+ +| | __annotations__ | mapping of parameters | +| | | names to annotations; | +| | | ``"return"`` key is | +| | | reserved for return | +| | | annotations. | ++-----------+-----------------+---------------------------+ +| traceback | tb_frame | frame object at this | +| | | level | ++-----------+-----------------+---------------------------+ +| | tb_lasti | index of last attempted | +| | | instruction in bytecode | ++-----------+-----------------+---------------------------+ +| | tb_lineno | current line number in | +| | | Python source code | ++-----------+-----------------+---------------------------+ +| | tb_next | next inner traceback | +| | | object (called by this | +| | | level) | ++-----------+-----------------+---------------------------+ +| frame | f_back | next outer frame object | +| | | (this frame's caller) | ++-----------+-----------------+---------------------------+ +| | f_builtins | builtins namespace seen | +| | | by this frame | ++-----------+-----------------+---------------------------+ +| | f_code | code object being | +| | | executed in this frame | ++-----------+-----------------+---------------------------+ +| | f_globals | global namespace seen by | +| | | this frame | ++-----------+-----------------+---------------------------+ +| | f_lasti | index of last attempted | +| | | instruction in bytecode | ++-----------+-----------------+---------------------------+ +| | f_lineno | current line number in | +| | | Python source code | ++-----------+-----------------+---------------------------+ +| | f_locals | local namespace seen by | +| | | this frame | ++-----------+-----------------+---------------------------+ +| | f_restricted | 0 or 1 if frame is in | +| | | restricted execution mode | ++-----------+-----------------+---------------------------+ +| | f_trace | tracing function for this | +| | | frame, or ``None`` | ++-----------+-----------------+---------------------------+ +| code | co_argcount | number of arguments (not | +| | | including \* or \*\* | +| | | args) | ++-----------+-----------------+---------------------------+ +| | co_code | string of raw compiled | +| | | bytecode | ++-----------+-----------------+---------------------------+ +| | co_consts | tuple of constants used | +| | | in the bytecode | ++-----------+-----------------+---------------------------+ +| | co_filename | name of file in which | +| | | this code object was | +| | | created | ++-----------+-----------------+---------------------------+ +| | co_firstlineno | number of first line in | +| | | Python source code | ++-----------+-----------------+---------------------------+ +| | co_flags | bitmap: 1=optimized ``|`` | +| | | 2=newlocals ``|`` 4=\*arg | +| | | ``|`` 8=\*\*arg | ++-----------+-----------------+---------------------------+ +| | co_lnotab | encoded mapping of line | +| | | numbers to bytecode | +| | | indices | ++-----------+-----------------+---------------------------+ +| | co_name | name with which this code | +| | | object was defined | ++-----------+-----------------+---------------------------+ +| | co_names | tuple of names of local | +| | | variables | ++-----------+-----------------+---------------------------+ +| | co_nlocals | number of local variables | ++-----------+-----------------+---------------------------+ +| | co_stacksize | virtual machine stack | +| | | space required | ++-----------+-----------------+---------------------------+ +| | co_varnames | tuple of names of | +| | | arguments and local | +| | | variables | ++-----------+-----------------+---------------------------+ +| generator | __name__ | name | ++-----------+-----------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+-----------------+---------------------------+ +| | gi_frame | frame | ++-----------+-----------------+---------------------------+ +| | gi_running | is the generator running? | ++-----------+-----------------+---------------------------+ +| | gi_code | code | ++-----------+-----------------+---------------------------+ +| | gi_yieldfrom | object being iterated by | +| | | ``yield from``, or | +| | | ``None`` | ++-----------+-----------------+---------------------------+ +| coroutine | __name__ | name | ++-----------+-----------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+-----------------+---------------------------+ +| | cr_await | object being awaited on, | +| | | or ``None`` | ++-----------+-----------------+---------------------------+ +| | cr_frame | frame | ++-----------+-----------------+---------------------------+ +| | cr_running | is the coroutine running? | ++-----------+-----------------+---------------------------+ +| | cr_code | code | ++-----------+-----------------+---------------------------+ +| builtin | __doc__ | documentation string | ++-----------+-----------------+---------------------------+ +| | __name__ | original name of this | +| | | function or method | ++-----------+-----------------+---------------------------+ +| | __qualname__ | qualified name | ++-----------+-----------------+---------------------------+ +| | __self__ | instance to which a | +| | | method is bound, or | +| | | ``None`` | ++-----------+-----------------+---------------------------+ .. versionchanged:: 3.5 @@ -226,10 +221,6 @@ The ``__name__`` attribute of generators is now set from the function name, instead of the code name, and it can now be modified. -.. versionchanged:: 3.6 - - Add ``__definition_order__`` to classes. - .. function:: getmembers(object[, predicate]) diff --git a/Doc/library/types.rst b/Doc/library/types.rst --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -53,19 +53,13 @@ in *kwds* argument with any ``'metaclass'`` entry removed. If no *kwds* argument is passed in, this will be an empty dict. - .. impl-detail:: - - CPython uses :class:`collections.OrderedDict` for the default - namespace. - .. versionadded:: 3.3 .. versionchanged:: 3.6 The default value for the ``namespace`` element of the returned - tuple has changed from :func:`dict`. Now an insertion-order- - preserving mapping is used when the metaclass does not have a - ``__prepare__`` method, + tuple has changed. Now an insertion-order-preserving mapping is + used when the metaclass does not have a ``__prepare__`` method, .. seealso:: diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -634,15 +634,9 @@ namespace. The order in which attributes are defined in the class body is preserved -in the ``__definition_order__`` attribute on the new class. If that order -is not known then the attribute is set to :const:`None`. The class body -may include a ``__definition_order__`` attribute. In that case it is used -directly. The value must be a tuple of identifiers or ``None``, otherwise -:exc:`TypeError` will be raised when the class statement is executed. - -.. versionchanged:: 3.6 - - Add ``__definition_order__`` to classes. +in the new class's ``__dict__``. Note that this is reliable only right +after the class is created and only for classes that were defined using +the definition syntax. Class creation can be customized heavily using :ref:`metaclasses `. diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1752,13 +1752,6 @@ If the metaclass has no ``__prepare__`` attribute, then the class namespace is initialised as an empty ordered mapping. -.. impl-detail:: - - In CPython the default is :class:`collections.OrderedDict`. - -.. versionchanged:: 3.6 - Defaults to :class:`collections.OrderedDict` instead of :func:`dict`. - .. seealso:: :pep:`3115` - Metaclasses in Python 3000 diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -421,8 +421,6 @@ destructor tp_finalize; - PyObject *tp_deforder; - #ifdef COUNT_ALLOCS /* these must be last and never explicitly initialized */ Py_ssize_t tp_allocs; diff --git a/Include/odictobject.h b/Include/odictobject.h --- a/Include/odictobject.h +++ b/Include/odictobject.h @@ -28,10 +28,6 @@ PyAPI_FUNC(int) PyODict_SetItem(PyObject *od, PyObject *key, PyObject *item); PyAPI_FUNC(int) PyODict_DelItem(PyObject *od, PyObject *key); -#ifndef Py_LIMITED_API -PyAPI_FUNC(PyObject *) _PyODict_KeysAsTuple(PyObject *od); -#endif - /* wrappers around PyDict* functions */ #define PyODict_GetItem(od, key) PyDict_GetItem((PyObject *)od, key) #define PyODict_GetItemWithError(od, key) \ diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -16,10 +16,8 @@ import types import unittest import warnings -from collections import OrderedDict from operator import neg -from test.support import (TESTFN, unlink, run_unittest, check_warnings, - cpython_only) +from test.support import TESTFN, unlink, run_unittest, check_warnings from test.support.script_helper import assert_python_ok try: import pty, signal @@ -1780,194 +1778,6 @@ A.__doc__ = doc self.assertEqual(A.__doc__, doc) - def test_type_definition_order_nonempty(self): - class Spam: - b = 1 - c = 3 - a = 2 - d = 4 - eggs = 2 - e = 5 - b = 42 - - self.assertEqual(Spam.__definition_order__, - ('__module__', '__qualname__', - 'b', 'c', 'a', 'd', 'eggs', 'e')) - - def test_type_definition_order_empty(self): - class Empty: - pass - - self.assertEqual(Empty.__definition_order__, - ('__module__', '__qualname__')) - - def test_type_definition_order_on_instance(self): - class Spam: - a = 2 - b = 1 - c = 3 - with self.assertRaises(AttributeError): - Spam().__definition_order__ - - def test_type_definition_order_set_to_None(self): - class Spam: - a = 2 - b = 1 - c = 3 - Spam.__definition_order__ = None - self.assertEqual(Spam.__definition_order__, None) - - def test_type_definition_order_set_to_tuple(self): - class Spam: - a = 2 - b = 1 - c = 3 - Spam.__definition_order__ = ('x', 'y', 'z') - self.assertEqual(Spam.__definition_order__, ('x', 'y', 'z')) - - def test_type_definition_order_deleted(self): - class Spam: - a = 2 - b = 1 - c = 3 - del Spam.__definition_order__ - self.assertEqual(Spam.__definition_order__, None) - - def test_type_definition_order_set_to_bad_type(self): - class Spam: - a = 2 - b = 1 - c = 3 - Spam.__definition_order__ = 42 - self.assertEqual(Spam.__definition_order__, 42) - - def test_type_definition_order_builtins(self): - self.assertEqual(object.__definition_order__, None) - self.assertEqual(type.__definition_order__, None) - self.assertEqual(dict.__definition_order__, None) - self.assertEqual(type(None).__definition_order__, None) - - def test_type_definition_order_dunder_names_included(self): - class Dunder: - VAR = 3 - def __init__(self): - pass - - self.assertEqual(Dunder.__definition_order__, - ('__module__', '__qualname__', - 'VAR', '__init__')) - - def test_type_definition_order_only_dunder_names(self): - class DunderOnly: - __xyz__ = None - def __init__(self): - pass - - self.assertEqual(DunderOnly.__definition_order__, - ('__module__', '__qualname__', - '__xyz__', '__init__')) - - def test_type_definition_order_underscore_names(self): - class HalfDunder: - __whether_to_be = True - or_not_to_be__ = False - - self.assertEqual(HalfDunder.__definition_order__, - ('__module__', '__qualname__', - '_HalfDunder__whether_to_be', 'or_not_to_be__')) - - def test_type_definition_order_with_slots(self): - class Slots: - __slots__ = ('x', 'y') - a = 1 - b = 2 - - self.assertEqual(Slots.__definition_order__, - ('__module__', '__qualname__', - '__slots__', 'a', 'b')) - - def test_type_definition_order_overwritten_None(self): - class OverwrittenNone: - __definition_order__ = None - a = 1 - b = 2 - c = 3 - - self.assertEqual(OverwrittenNone.__definition_order__, None) - - def test_type_definition_order_overwritten_tuple(self): - class OverwrittenTuple: - __definition_order__ = ('x', 'y', 'z') - a = 1 - b = 2 - c = 3 - - self.assertEqual(OverwrittenTuple.__definition_order__, - ('x', 'y', 'z')) - - def test_type_definition_order_overwritten_bad_item(self): - with self.assertRaises(TypeError): - class PoorlyOverwritten: - __definition_order__ = ('a', 2, 'c') - a = 1 - b = 2 - c = 3 - - def test_type_definition_order_overwritten_bad_type(self): - with self.assertRaises(TypeError): - class PoorlyOverwritten: - __definition_order__ = ['a', 2, 'c'] - a = 1 - b = 2 - c = 3 - - def test_type_definition_order_metaclass(self): - class Meta(type): - SPAM = 42 - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - - self.assertEqual(Meta.__definition_order__, - ('__module__', '__qualname__', - 'SPAM', '__init__')) - - def test_type_definition_order_OrderedDict(self): - class Meta(type): - def __prepare__(self, *args, **kwargs): - return OrderedDict() - - class WithODict(metaclass=Meta): - x='y' - - self.assertEqual(WithODict.__definition_order__, - ('__module__', '__qualname__', 'x')) - - class Meta(type): - def __prepare__(self, *args, **kwargs): - class ODictSub(OrderedDict): - pass - return ODictSub() - - class WithODictSub(metaclass=Meta): - x='y' - - self.assertEqual(WithODictSub.__definition_order__, - ('__module__', '__qualname__', 'x')) - - @cpython_only - def test_type_definition_order_cpython(self): - # some implementations will have an ordered-by-default dict. - - class Meta(type): - def __prepare__(self, *args, **kwargs): - return {} - - class NotOrdered(metaclass=Meta): - x='y' - - self.assertEqual(NotOrdered.__definition_order__, None) - def test_bad_args(self): with self.assertRaises(TypeError): type() diff --git a/Lib/test/test_metaclass.py b/Lib/test/test_metaclass.py --- a/Lib/test/test_metaclass.py +++ b/Lib/test/test_metaclass.py @@ -180,7 +180,7 @@ meta: C () ns: [('__module__', 'test.test_metaclass'), ('__qualname__', 'C'), ('a', 42), ('b', 24)] kw: [] - >>> type(C) is types._DefaultClassNamespaceType + >>> type(C) is dict True >>> print(sorted(C.items())) [('__module__', 'test.test_metaclass'), ('__qualname__', 'C'), ('a', 42), ('b', 24)] @@ -211,11 +211,8 @@ The default metaclass must define a __prepare__() method. - >>> ns = type.__prepare__() - >>> type(ns) is types._DefaultClassNamespaceType - True - >>> list(ns) == [] - True + >>> type.__prepare__() + {} >>> Make sure it works with subclassing. @@ -251,9 +248,7 @@ """ -from collections import OrderedDict import sys -import types # Trace function introduces __locals__ which causes various tests to fail. if hasattr(sys, 'gettrace') and sys.gettrace(): diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -427,7 +427,6 @@ expected_html = expected_html_pattern % ( (mod_url, mod_file, doc_loc) + expected_html_data_docstrings) - self.maxDiff = None self.assertEqual(result, expected_html) @unittest.skipIf(sys.flags.optimize >= 2, @@ -474,18 +473,13 @@ def test_non_str_name(self): # issue14638 # Treat illegal (non-str) name like no name - # Definition order is set to None so it looks the same in both - # cases. class A: - __definition_order__ = None __name__ = 42 class B: pass adoc = pydoc.render_doc(A()) bdoc = pydoc.render_doc(B()) - self.maxDiff = None - expected = adoc.replace("A", "B") - self.assertEqual(bdoc, expected) + self.assertEqual(adoc.replace("A", "B"), bdoc) def test_not_here(self): missing_module = "test.i_am_not_here" diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -1085,7 +1085,7 @@ check((1,2,3), vsize('') + 3*self.P) # type # static type: PyTypeObject - fmt = 'P2n15Pl4Pn9Pn11PIPP' + fmt = 'P2n15Pl4Pn9Pn11PIP' if hasattr(sys, 'getcounts'): fmt += '3n2P' s = vsize(fmt) diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -825,28 +825,6 @@ self.assertEqual(C.y, 1) self.assertEqual(C.z, 2) - def test_new_class_deforder(self): - C = types.new_class("C") - self.assertEqual(C.__definition_order__, tuple()) - - Meta = self.Meta - def func(ns): - ns["x"] = 0 - D = types.new_class("D", (), {"metaclass": Meta, "z": 2}, func) - self.assertEqual(D.__definition_order__, ('y', 'z', 'x')) - - def func(ns): - ns["__definition_order__"] = None - ns["x"] = 0 - D = types.new_class("D", (), {"metaclass": Meta, "z": 2}, func) - self.assertEqual(D.__definition_order__, None) - - def func(ns): - ns["__definition_order__"] = ('a', 'b', 'c') - ns["x"] = 0 - D = types.new_class("D", (), {"metaclass": Meta, "z": 2}, func) - self.assertEqual(D.__definition_order__, ('a', 'b', 'c')) - # Many of the following tests are derived from test_descr.py def test_prepare_class(self): # Basic test of metaclass derivation diff --git a/Lib/types.py b/Lib/types.py --- a/Lib/types.py +++ b/Lib/types.py @@ -25,11 +25,8 @@ _c.close() # Prevent ResourceWarning class _C: - _nsType = type(locals()) def _m(self): pass MethodType = type(_C()._m) -# In CPython, this should end up as OrderedDict. -_DefaultClassNamespaceType = _C._nsType BuiltinFunctionType = type(len) BuiltinMethodType = type([].append) # Same as BuiltinFunctionType @@ -88,7 +85,7 @@ if hasattr(meta, '__prepare__'): ns = meta.__prepare__(name, bases, **kwds) else: - ns = _DefaultClassNamespaceType() + ns = {} return meta, ns, kwds def _calculate_meta(meta, bases): diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -1301,7 +1301,6 @@ if (not attr.startswith('_abc_') and attr != '__abstractmethods__' and attr != '_is_protocol' and - attr != '__definition_order__' and attr != '__dict__' and attr != '__args__' and attr != '__slots__' and diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -1765,21 +1765,6 @@ return _PyDict_DelItem_KnownHash(od, key, hash); } -PyObject * -_PyODict_KeysAsTuple(PyObject *od) { - Py_ssize_t i = 0; - _ODictNode *node; - PyObject *keys = PyTuple_New(PyODict_Size(od)); - if (keys == NULL) - return NULL; - _odict_FOREACH((PyODictObject *)od, node) { - Py_INCREF(_odictnode_KEY(node)); - PyTuple_SET_ITEM(keys, i, _odictnode_KEY(node)); - i++; - } - return keys; -} - /* ------------------------------------------- * The OrderedDict views (keys/values/items) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -48,7 +48,6 @@ _Py_IDENTIFIER(__abstractmethods__); _Py_IDENTIFIER(__class__); _Py_IDENTIFIER(__delitem__); -_Py_IDENTIFIER(__definition_order__); _Py_IDENTIFIER(__dict__); _Py_IDENTIFIER(__doc__); _Py_IDENTIFIER(__getattribute__); @@ -490,23 +489,6 @@ } static PyObject * -type_deforder(PyTypeObject *type, void *context) -{ - if (type->tp_deforder == NULL) - Py_RETURN_NONE; - Py_INCREF(type->tp_deforder); - return type->tp_deforder; -} - -static int -type_set_deforder(PyTypeObject *type, PyObject *value, void *context) -{ - Py_XINCREF(value); - Py_XSETREF(type->tp_deforder, value); - return 0; -} - -static PyObject * type_abstractmethods(PyTypeObject *type, void *context) { PyObject *mod = NULL; @@ -852,8 +834,6 @@ {"__qualname__", (getter)type_qualname, (setter)type_set_qualname, NULL}, {"__bases__", (getter)type_get_bases, (setter)type_set_bases, NULL}, {"__module__", (getter)type_module, (setter)type_set_module, NULL}, - {"__definition_order__", (getter)type_deforder, - (setter)type_set_deforder, NULL}, {"__abstractmethods__", (getter)type_abstractmethods, (setter)type_set_abstractmethods, NULL}, {"__dict__", (getter)type_dict, NULL, NULL}, @@ -2371,7 +2351,6 @@ goto error; } - /* Copy the definition namespace into a new dict. */ dict = PyDict_Copy(orig_dict); if (dict == NULL) goto error; @@ -2580,48 +2559,6 @@ if (qualname != NULL && PyDict_DelItem(dict, PyId___qualname__.object) < 0) goto error; - /* Set tp_deforder to the extracted definition order, if any. */ - type->tp_deforder = _PyDict_GetItemId(dict, &PyId___definition_order__); - if (type->tp_deforder != NULL) { - Py_INCREF(type->tp_deforder); - - // Due to subclass lookup, __definition_order__ can't be in __dict__. - if (_PyDict_DelItemId(dict, &PyId___definition_order__) != 0) { - goto error; - } - - if (type->tp_deforder != Py_None) { - Py_ssize_t numnames; - - if (!PyTuple_Check(type->tp_deforder)) { - PyErr_SetString(PyExc_TypeError, - "__definition_order__ must be a tuple or None"); - goto error; - } - - // Make sure they are identifers. - numnames = PyTuple_Size(type->tp_deforder); - for (i = 0; i < numnames; i++) { - PyObject *name = PyTuple_GET_ITEM(type->tp_deforder, i); - if (name == NULL) { - goto error; - } - if (!PyUnicode_Check(name) || !PyUnicode_IsIdentifier(name)) { - PyErr_Format(PyExc_TypeError, - "__definition_order__ must " - "contain only identifiers, got '%s'", - name); - goto error; - } - } - } - } - else if (PyODict_Check(orig_dict)) { - type->tp_deforder = _PyODict_KeysAsTuple(orig_dict); - if (type->tp_deforder == NULL) - goto error; - } - /* Set tp_doc to a copy of dict['__doc__'], if the latter is there and is a string. The __doc__ accessor will first look for tp_doc; if that fails, it will still look into __dict__. @@ -3136,7 +3073,6 @@ Py_XDECREF(type->tp_mro); Py_XDECREF(type->tp_cache); Py_XDECREF(type->tp_subclasses); - Py_XDECREF(type->tp_deforder); /* A type's tp_doc is heap allocated, unlike the tp_doc slots * of most other objects. It's okay to cast it to char *. */ @@ -3179,7 +3115,7 @@ static PyObject * type_prepare(PyObject *self, PyObject *args, PyObject *kwds) { - return PyODict_New(); + return PyDict_New(); } /* diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -147,7 +147,7 @@ if (prep == NULL) { if (PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Clear(); - ns = PyODict_New(); + ns = PyDict_New(); } else { Py_DECREF(meta); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:34:02 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:34:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_potential_NULL_pointer?= =?utf-8?q?_dereference_in_=5Fimp=5Fcreate=5Fbuiltin?= Message-ID: <20160908223401.12395.29590.D2EEB427@psf.io> https://hg.python.org/cpython/rev/4af9d196cd21 changeset: 103379:4af9d196cd21 user: Christian Heimes date: Fri Sep 09 00:25:03 2016 +0200 summary: Fix potential NULL pointer dereference in _imp_create_builtin PyModule_GetDef() can return NULL. Let's check the return value properly like in the other five cases. CID 1299590 files: Python/import.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Python/import.c b/Python/import.c --- a/Python/import.c +++ b/Python/import.c @@ -1077,6 +1077,10 @@ } else { /* Remember pointer to module init function. */ def = PyModule_GetDef(mod); + if (def == NULL) { + Py_DECREF(name); + return NULL; + } def->m_base.m_init = p->initfunc; if (_PyImport_FixupExtensionObject(mod, name, name) < 0) { Py_DECREF(name); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:34:02 2016 From: python-checkins at python.org (christian.heimes) Date: Thu, 08 Sep 2016 22:34:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_18550=3A_Check_retur?= =?utf-8?q?n_value_of_ioctl=28=29_/_fnctl=28=29_in_internal=5Fsetblocking?= Message-ID: <20160908223401.12492.48930.26AA4472@psf.io> https://hg.python.org/cpython/rev/0a52c66f31f5 changeset: 103380:0a52c66f31f5 user: Christian Heimes date: Fri Sep 09 00:28:57 2016 +0200 summary: Issue 18550: Check return value of ioctl() / fnctl() in internal_setblocking The function internal_setblocking() of the socket module did not check the return values of ioctl() and fnctl(). CID 1294328 files: Modules/socketmodule.c | 63 ++++++++++++++++++++---------- 1 files changed, 42 insertions(+), 21 deletions(-) diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -640,24 +640,35 @@ #ifndef MS_WINDOWS #if (defined(HAVE_SYS_IOCTL_H) && defined(FIONBIO)) block = !block; - ioctl(s->sock_fd, FIONBIO, (unsigned int *)&block); + if (ioctl(s->sock_fd, FIONBIO, (unsigned int *)&block) == -1) + goto error; #else delay_flag = fcntl(s->sock_fd, F_GETFL, 0); + if (delay_flag == -1) + goto error; if (block) new_delay_flag = delay_flag & (~O_NONBLOCK); else new_delay_flag = delay_flag | O_NONBLOCK; if (new_delay_flag != delay_flag) - fcntl(s->sock_fd, F_SETFL, new_delay_flag); + if (fcntl(s->sock_fd, F_SETFL, new_delay_flag) == -1) + goto error; #endif #else /* MS_WINDOWS */ arg = !block; - ioctlsocket(s->sock_fd, FIONBIO, &arg); + if (ioctlsocket(s->sock_fd, FIONBIO, &arg) != 0) + goto error; #endif /* MS_WINDOWS */ Py_END_ALLOW_THREADS - /* Since these don't return anything */ - return 1; + return 0; + error: +#ifndef MS_WINDOWS + PyErr_SetFromErrno(PyExc_OSError); +#else + PyErr_SetExcFromWindowsErr(PyExc_OSError, WSAGetLastError()); +#endif + return -1; } static int @@ -905,7 +916,7 @@ /* Default timeout for new sockets */ static _PyTime_t defaulttimeout = _PYTIME_FROMSECONDS(-1); -static void +static int init_sockobject(PySocketSockObject *s, SOCKET_T fd, int family, int type, int proto) { @@ -922,10 +933,13 @@ #endif { s->sock_timeout = defaulttimeout; - if (defaulttimeout >= 0) - internal_setblocking(s, 0); - } - + if (defaulttimeout >= 0) { + if (internal_setblocking(s, 0) == -1) { + return -1; + } + } + } + return 0; } @@ -940,8 +954,12 @@ PySocketSockObject *s; s = (PySocketSockObject *) PyType_GenericNew(&sock_type, NULL, NULL); - if (s != NULL) - init_sockobject(s, fd, family, type, proto); + if (s == NULL) + return NULL; + if (init_sockobject(s, fd, family, type, proto) == -1) { + Py_DECREF(s); + return NULL; + } return s; } @@ -2423,10 +2441,10 @@ return NULL; s->sock_timeout = _PyTime_FromSeconds(block ? -1 : 0); - internal_setblocking(s, block); - - Py_INCREF(Py_None); - return Py_None; + if (internal_setblocking(s, block) == -1) { + return NULL; + } + Py_RETURN_NONE; } PyDoc_STRVAR(setblocking_doc, @@ -2492,10 +2510,10 @@ return NULL; s->sock_timeout = timeout; - internal_setblocking(s, timeout < 0); - - Py_INCREF(Py_None); - return Py_None; + if (internal_setblocking(s, timeout < 0) == -1) { + return NULL; + } + Py_RETURN_NONE; } PyDoc_STRVAR(settimeout_doc, @@ -4720,7 +4738,10 @@ } #endif } - init_sockobject(s, fd, family, type, proto); + if (init_sockobject(s, fd, family, type, proto) == -1) { + SOCKETCLOSE(fd); + return -1; + } return 0; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:40:56 2016 From: python-checkins at python.org (benjamin.peterson) Date: Thu, 08 Sep 2016 22:40:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41?= Message-ID: <20160908224056.1897.95150.C0CAD1A0@psf.io> https://hg.python.org/cpython/rev/2e65943a35bc changeset: 103381:2e65943a35bc parent: 103380:0a52c66f31f5 parent: 103362:0967531fc762 user: Benjamin Peterson date: Thu Sep 08 15:40:49 2016 -0700 summary: merge 3.5 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:43:01 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 22:43:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogaXNzdWUyODAzMjog?= =?utf-8?q?never_imply_--with-lto_as_part_of_--with-optimizations=2E?= Message-ID: <20160908224300.59448.31682.9EE5144E@psf.io> https://hg.python.org/cpython/rev/b77465544f28 changeset: 103382:b77465544f28 branch: 3.5 parent: 103362:0967531fc762 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 22:38:46 2016 +0000 summary: issue28032: never imply --with-lto as part of --with-optimizations. Too many build tool chains do not properly support it, including building and linking an executable fine that simply segfaults when you try to run it (such as debian jessie 8.5's gcc 4.9). On others where it does appear to build (ubuntu 16.04's gcc 5.4) there are still test_distutils and test_gdb failures to deal with. We're not going to spend time attempting to maintain a complicated white list of what does and doesn't work in our configure.ac file. files: README | 11 ++++++----- configure | 13 +++++-------- configure.ac | 13 +++++-------- 3 files changed, 16 insertions(+), 21 deletions(-) diff --git a/README b/README --- a/README +++ b/README @@ -48,8 +48,8 @@ To get an optimized build of Python, "configure --with-optimizations" before you run make. This sets the default make targets up to enable Profile Guided -Optimization (PGO) and Link Time Optimization (LTO) on most platforms. -For more details, see the sections bellow. +Optimization (PGO) and may be used to auto-enable Link Time Optimization (LTO) +on some platforms. For more details, see the sections bellow. Profile Guided Optimization @@ -79,9 +79,10 @@ Link Time Optimization ---------------------- -LTO takes advantages of recent compiler toolchains ability to optimize across -the otherwise arbitrary .o file boundary when building final executables or -shared libraries for additional performance gains. +Enabled via configure's --with-lto flag. LTO takes advantages of recent +compiler toolchains ability to optimize across the otherwise arbitrary .o file +boundary when building final executables or shared libraries for additional +performance gains. What's New diff --git a/configure b/configure --- a/configure +++ b/configure @@ -1509,7 +1509,7 @@ compiler --with-suffix=.exe set executable suffix --with-pydebug build with Py_DEBUG defined - --with-optimizations Enable all optimizations when available (LTO, PGO, + --with-optimizations Enable expensive optimizations (PGO, maybe LTO, etc). Disabled by default. --with-lto Enable Link Time Optimization in PGO builds. Disabled by default. @@ -6565,13 +6565,10 @@ fi if test "$Py_OPT" = 'true' ; then - Py_LTO='true' - case $ac_sys_system in - Darwin*) - # At least on macOS El Capitan, LTO does not work with PGO. - Py_LTO='false' - ;; - esac + # Intentionally not forcing Py_LTO='true' here. Too many toolchains do not + # compile working code using it and both test_distutils and test_gdb are + # broken when you do managed to get a toolchain that works with it. People + # who want LTO need to use --with-lto themselves. DEF_MAKE_ALL_RULE="profile-opt" REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1235,7 +1235,7 @@ AC_SUBST(DEF_MAKE_RULE) Py_OPT='false' AC_MSG_CHECKING(for --with-optimizations) -AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable all optimizations when available (LTO, PGO, etc). Disabled by default.]), +AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, maybe LTO, etc). Disabled by default.]), [ if test "$withval" != no then @@ -1247,13 +1247,10 @@ fi], [AC_MSG_RESULT(no)]) if test "$Py_OPT" = 'true' ; then - Py_LTO='true' - case $ac_sys_system in - Darwin*) - # At least on macOS El Capitan, LTO does not work with PGO. - Py_LTO='false' - ;; - esac + # Intentionally not forcing Py_LTO='true' here. Too many toolchains do not + # compile working code using it and both test_distutils and test_gdb are + # broken when you do managed to get a toolchain that works with it. People + # who want LTO need to use --with-lto themselves. DEF_MAKE_ALL_RULE="profile-opt" REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:43:01 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 22:43:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_issue28032=3A_never_imply_--with-lto_as_part_of_--with-o?= =?utf-8?q?ptimizations=2E?= Message-ID: <20160908224300.48546.62605.ED449684@psf.io> https://hg.python.org/cpython/rev/4a50d87808a6 changeset: 103383:4a50d87808a6 parent: 103381:2e65943a35bc parent: 103382:b77465544f28 user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 22:42:45 2016 +0000 summary: issue28032: never imply --with-lto as part of --with-optimizations. Too many build tool chains do not properly support it, including building and linking an executable fine that simply segfaults when you try to run it (such as debian jessie 8.5's gcc 4.9). On others where it does appear to build (ubuntu 16.04's gcc 5.4) there are still test_distutils and test_gdb failures to deal with. We're not going to spend time attempting to maintain a complicated white list of what does and doesn't work in our configure.ac file. files: README | 11 ++++++----- configure | 13 +++++-------- configure.ac | 13 +++++-------- 3 files changed, 16 insertions(+), 21 deletions(-) diff --git a/README b/README --- a/README +++ b/README @@ -48,8 +48,8 @@ To get an optimized build of Python, "configure --with-optimizations" before you run make. This sets the default make targets up to enable Profile Guided -Optimization (PGO) and Link Time Optimization (LTO) on most platforms. -For more details, see the sections bellow. +Optimization (PGO) and may be used to auto-enable Link Time Optimization (LTO) +on some platforms. For more details, see the sections bellow. Profile Guided Optimization @@ -79,9 +79,10 @@ Link Time Optimization ---------------------- -LTO takes advantages of recent compiler toolchains ability to optimize across -the otherwise arbitrary .o file boundary when building final executables or -shared libraries for additional performance gains. +Enabled via configure's --with-lto flag. LTO takes advantages of recent +compiler toolchains ability to optimize across the otherwise arbitrary .o file +boundary when building final executables or shared libraries for additional +performance gains. What's New diff --git a/configure b/configure --- a/configure +++ b/configure @@ -1510,7 +1510,7 @@ compiler --with-suffix=.exe set executable suffix --with-pydebug build with Py_DEBUG defined - --with-optimizations Enable all optimizations when available (LTO, PGO, + --with-optimizations Enable expensive optimizations (PGO, maybe LTO, etc). Disabled by default. --with-lto Enable Link Time Optimization in PGO builds. Disabled by default. @@ -6495,13 +6495,10 @@ fi if test "$Py_OPT" = 'true' ; then - Py_LTO='true' - case $ac_sys_system in - Darwin*) - # At least on macOS El Capitan, LTO does not work with PGO. - Py_LTO='false' - ;; - esac + # Intentionally not forcing Py_LTO='true' here. Too many toolchains do not + # compile working code using it and both test_distutils and test_gdb are + # broken when you do managed to get a toolchain that works with it. People + # who want LTO need to use --with-lto themselves. DEF_MAKE_ALL_RULE="profile-opt" REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1287,7 +1287,7 @@ AC_SUBST(DEF_MAKE_RULE) Py_OPT='false' AC_MSG_CHECKING(for --with-optimizations) -AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable all optimizations when available (LTO, PGO, etc). Disabled by default.]), +AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, maybe LTO, etc). Disabled by default.]), [ if test "$withval" != no then @@ -1299,13 +1299,10 @@ fi], [AC_MSG_RESULT(no)]) if test "$Py_OPT" = 'true' ; then - Py_LTO='true' - case $ac_sys_system in - Darwin*) - # At least on macOS El Capitan, LTO does not work with PGO. - Py_LTO='false' - ;; - esac + # Intentionally not forcing Py_LTO='true' here. Too many toolchains do not + # compile working code using it and both test_distutils and test_gdb are + # broken when you do managed to get a toolchain that works with it. People + # who want LTO need to use --with-lto themselves. DEF_MAKE_ALL_RULE="profile-opt" REQUIRE_PGO="yes" DEF_MAKE_RULE="build_all" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 18:45:26 2016 From: python-checkins at python.org (gregory.p.smith) Date: Thu, 08 Sep 2016 22:45:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogaXNzdWUyODAzMjog?= =?utf-8?q?never_imply_--with-lto_as_part_of_--with-optimizations=2E?= Message-ID: <20160908224525.87446.95966.AFC85E0F@psf.io> https://hg.python.org/cpython/rev/7e89469e4342 changeset: 103384:7e89469e4342 branch: 2.7 parent: 103376:aa2bf603a9bd user: Gregory P. Smith [Google Inc.] date: Thu Sep 08 22:44:44 2016 +0000 summary: issue28032: never imply --with-lto as part of --with-optimizations. Too many build tool chains do not properly support it, including building and linking an executable fine that simply segfaults when you try to run it (such as debian jessie 8.5's gcc 4.9). On others where it does appear to build (ubuntu 16.04's gcc 5.4) there are still test_distutils and test_gdb failures to deal with. We're not going to spend time attempting to maintain a complicated white list of what does and doesn't work in our configure.ac file. files: README | 11 ++++++----- configure | 12 +++++------- configure.ac | 12 +++++------- 3 files changed, 16 insertions(+), 19 deletions(-) diff --git a/README b/README --- a/README +++ b/README @@ -175,8 +175,8 @@ To get an optimized build of Python, "configure --with-optimizations" before you run make. This sets the default make targets up to enable Profile Guided -Optimization (PGO) and Link Time Optimization (LTO) on most platforms. -For more details, see the sections bellow. +Optimization (PGO) and may be used to auto-enable Link Time Optimization (LTO) +on some platforms. For more details, see the sections bellow. Once you have built a Python interpreter, see the subsections below on testing and installation. If you run into trouble, see the next @@ -217,9 +217,10 @@ Link Time Optimization ---------------------- -LTO takes advantages of recent compiler toolchains ability to optimize across -the otherwise arbitrary .o file boundary when building final executables or -shared libraries for additional performance gains. +Enabled via configure's --with-lto flag. LTO takes advantages of recent +compiler toolchains ability to optimize across the otherwise arbitrary .o file +boundary when building final executables or shared libraries for additional +performance gains. Troubleshooting diff --git a/configure b/configure --- a/configure +++ b/configure @@ -1500,7 +1500,7 @@ compiler --with-suffix=.exe set executable suffix --with-pydebug build with Py_DEBUG defined - --with-optimizations Enable all optimizations when available (LTO, PGO, + --with-optimizations Enable expensive optimizations (PGO, maybe LTO, etc). Disabled by default. --with-lto Enable Link Time Optimization in PGO builds. Disabled by default. @@ -6423,12 +6423,10 @@ fi if test "$Py_OPT" = 'true' ; then - case $ac_sys_system in - Darwin*) - # At least on macOS El Capitan, LTO does not work with PGO. - Py_LTO='false' - ;; - esac + # Intentionally not forcing Py_LTO='true' here. Too many toolchains do not + # compile working code using it and both test_distutils and test_gdb are + # broken when you do managed to get a toolchain that works with it. People + # who want LTO need to use --with-lto themselves. Py_LTO='true' DEF_MAKE_ALL_RULE="profile-opt" REQUIRE_PGO="yes" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1385,7 +1385,7 @@ AC_SUBST(DEF_MAKE_RULE) Py_OPT='false' AC_MSG_CHECKING(for --with-optimizations) -AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable all optimizations when available (LTO, PGO, etc). Disabled by default.]), +AC_ARG_WITH(optimizations, AS_HELP_STRING([--with-optimizations], [Enable expensive optimizations (PGO, maybe LTO, etc). Disabled by default.]), [ if test "$withval" != no then @@ -1397,12 +1397,10 @@ fi], [AC_MSG_RESULT(no)]) if test "$Py_OPT" = 'true' ; then - case $ac_sys_system in - Darwin*) - # At least on macOS El Capitan, LTO does not work with PGO. - Py_LTO='false' - ;; - esac + # Intentionally not forcing Py_LTO='true' here. Too many toolchains do not + # compile working code using it and both test_distutils and test_gdb are + # broken when you do managed to get a toolchain that works with it. People + # who want LTO need to use --with-lto themselves. Py_LTO='true' DEF_MAKE_ALL_RULE="profile-opt" REQUIRE_PGO="yes" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 20:57:00 2016 From: python-checkins at python.org (berker.peksag) Date: Fri, 09 Sep 2016 00:57:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328033=3A_Fix_typo?= =?utf-8?q?_in_dictobject=2Ec?= Message-ID: <20160909005700.69581.83407.BEA4B372@psf.io> https://hg.python.org/cpython/rev/30f20247bd14 changeset: 103385:30f20247bd14 parent: 103383:4a50d87808a6 user: Berker Peksag date: Fri Sep 09 03:57:23 2016 +0300 summary: Issue #28033: Fix typo in dictobject.c Patch by Wesley Emeneker. files: Objects/dictobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -10,7 +10,7 @@ This implements the dictionary's hashtable. -As of Python 3.6, this is compact and orderd. Basic idea is described here. +As of Python 3.6, this is compact and ordered. Basic idea is described here. https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html layout: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 22:28:33 2016 From: python-checkins at python.org (r.david.murray) Date: Fri, 09 Sep 2016 02:28:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI3NjMwOiBCZSBj?= =?utf-8?q?onsistent_in_how_=5FXXX/=5Fencoded=5FXXX_vars_are_initialized?= =?utf-8?q?=2E?= Message-ID: <20160909022833.48417.33245.D842EBC3@psf.io> https://hg.python.org/cpython/rev/468961cea562 changeset: 103386:468961cea562 branch: 3.5 parent: 103382:b77465544f28 user: R David Murray date: Thu Sep 08 22:21:27 2016 -0400 summary: #27630: Be consistent in how _XXX/_encoded_XXX vars are initialized. files: Lib/email/generator.py | 10 +--------- 1 files changed, 1 insertions(+), 9 deletions(-) diff --git a/Lib/email/generator.py b/Lib/email/generator.py --- a/Lib/email/generator.py +++ b/Lib/email/generator.py @@ -97,7 +97,7 @@ self._NL = policy.linesep self._encoded_NL = self._encode(self._NL) self._EMPTY = '' - self._encoded_EMTPY = self._encode('') + self._encoded_EMPTY = self._encode(self._EMPTY) # Because we use clone (below) when we recursively process message # subparts, and because clone uses the computed policy (not None), # submessages will automatically get set to the computed policy when @@ -137,10 +137,6 @@ # it has already transformed the input; but, since this whole thing is a # hack anyway this seems good enough. - # Similarly, we have _XXX and _encoded_XXX attributes that are used on - # source and buffer data, respectively. - _encoded_EMPTY = '' - def _new_buffer(self): # BytesGenerator overrides this to return BytesIO. return StringIO() @@ -402,10 +398,6 @@ The outfp object must accept bytes in its write method. """ - # Bytes versions of this constant for use in manipulating data from - # the BytesIO buffer. - _encoded_EMPTY = b'' - def write(self, s): self._fp.write(s.encode('ascii', 'surrogateescape')) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 22:28:33 2016 From: python-checkins at python.org (r.david.murray) Date: Fri, 09 Sep 2016 02:28:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2327630=3A_Be_consistent_in_how_=5FXXX/=5Fenco?= =?utf-8?q?ded=5FXXX_vars_are_initialized=2E?= Message-ID: <20160909022833.13248.42119.5F1F183E@psf.io> https://hg.python.org/cpython/rev/ddb1cf7b7eb1 changeset: 103387:ddb1cf7b7eb1 parent: 103385:30f20247bd14 parent: 103386:468961cea562 user: R David Murray date: Thu Sep 08 22:27:39 2016 -0400 summary: Merge: #27630: Be consistent in how _XXX/_encoded_XXX vars are initialized. files: Lib/email/generator.py | 10 +--------- 1 files changed, 1 insertions(+), 9 deletions(-) diff --git a/Lib/email/generator.py b/Lib/email/generator.py --- a/Lib/email/generator.py +++ b/Lib/email/generator.py @@ -97,7 +97,7 @@ self._NL = policy.linesep self._encoded_NL = self._encode(self._NL) self._EMPTY = '' - self._encoded_EMPTY = self._encode('') + self._encoded_EMPTY = self._encode(self._EMPTY) # Because we use clone (below) when we recursively process message # subparts, and because clone uses the computed policy (not None), # submessages will automatically get set to the computed policy when @@ -137,10 +137,6 @@ # it has already transformed the input; but, since this whole thing is a # hack anyway this seems good enough. - # Similarly, we have _XXX and _encoded_XXX attributes that are used on - # source and buffer data, respectively. - _encoded_EMPTY = '' - def _new_buffer(self): # BytesGenerator overrides this to return BytesIO. return StringIO() @@ -402,10 +398,6 @@ The outfp object must accept bytes in its write method. """ - # Bytes versions of this constant for use in manipulating data from - # the BytesIO buffer. - _encoded_EMPTY = b'' - def write(self, s): self._fp.write(s.encode('ascii', 'surrogateescape')) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 22:37:43 2016 From: python-checkins at python.org (r.david.murray) Date: Fri, 09 Sep 2016 02:37:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2327364=3A_Credit_Emanuel?= =?utf-8?q?_Barry_in_NEWS_item=2E?= Message-ID: <20160909023743.13175.27168.2F997F76@psf.io> https://hg.python.org/cpython/rev/60085c8f01fe changeset: 103388:60085c8f01fe user: R David Murray date: Thu Sep 08 22:37:34 2016 -0400 summary: #27364: Credit Emanuel Barry in NEWS item. files: Misc/NEWS | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -15,7 +15,7 @@ the PEP 509. - Issue #27364: A backslash-character pair that is not a valid escape sequence - now generates a DeprecationWarning. + now generates a DeprecationWarning. Patch by Emanuel Barry. - Issue #27350: `dict` implementation is changed like PyPy. It is more compact and preserves insertion order. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Thu Sep 8 23:51:44 2016 From: python-checkins at python.org (yury.selivanov) Date: Fri, 09 Sep 2016 03:51:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327985=3A_Implemen?= =?utf-8?q?t_PEP_526_--_Syntax_for_Variable_Annotations=2E?= Message-ID: <20160909035144.19406.16717.D95F5283@psf.io> https://hg.python.org/cpython/rev/49533a9fe322 changeset: 103389:49533a9fe322 user: Yury Selivanov date: Thu Sep 08 20:50:03 2016 -0700 summary: Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi. files: Doc/glossary.rst | 12 + Doc/library/dis.rst | 11 + Doc/reference/datamodel.rst | 35 +- Doc/reference/simple_stmts.rst | 44 + Grammar/Grammar | 5 +- Include/Python-ast.h | 21 +- Include/graminit.h | 137 +- Include/opcode.h | 2 + Include/symtable.h | 1 + Lib/importlib/_bootstrap_external.py | 4 +- Lib/lib2to3/Grammar.txt | 5 +- Lib/lib2to3/tests/test_parser.py | 30 + Lib/opcode.py | 3 +- Lib/symbol.py | 137 +- Lib/symtable.py | 5 +- Lib/test/ann_module.py | 53 + Lib/test/ann_module2.py | 36 + Lib/test/ann_module3.py | 18 + Lib/test/pydoc_mod.py | 2 +- Lib/test/test___all__.py | 2 + Lib/test/test_dis.py | 33 + Lib/test/test_grammar.py | 178 + Lib/test/test_opcodes.py | 27 + Lib/test/test_parser.py | 39 + Lib/test/test_pydoc.py | 4 + Lib/test/test_symtable.py | 11 + Lib/test/test_tools/test_com2ann.py | 260 + Lib/test/test_typing.py | 83 +- Lib/typing.py | 146 +- Misc/ACKS | 1 + Misc/NEWS | 3 + Modules/symtablemodule.c | 1 + PC/launcher.c | 2 +- Parser/Python.asdl | 2 + Python/Python-ast.c | 124 +- Python/ast.c | 86 +- Python/ceval.c | 112 + Python/compile.c | 215 +- Python/graminit.c | 2089 +++++++------ Python/importlib_external.h | 208 +- Python/opcode_targets.h | 4 +- Python/pylifecycle.c | 8 +- Python/symtable.c | 56 + Tools/parser/com2ann.py | 308 ++ Tools/parser/unparse.py | 13 + 45 files changed, 3255 insertions(+), 1321 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -964,6 +964,18 @@ ``'\r'``. See :pep:`278` and :pep:`3116`, as well as :func:`bytes.splitlines` for an additional use. + variable annotation + A type metadata value associated with a module global variable or + a class attribute. Its syntax is explained in section :ref:`annassign`. + Annotations are stored in the :attr:`__annotations__` special + attribute of a class or module object and can be accessed using + :func:`typing.get_type_hints`. + + Python itself does not assign any particular meaning to variable + annotations. They are intended to be interpreted by third-party libraries + or type checking tools. See :pep:`526`, :pep:`484` which describe + some of their potential uses. + virtual environment A cooperatively isolated runtime environment that allows Python users and applications to install and upgrade Python distribution packages diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -607,6 +607,12 @@ .. versionadded:: 3.3 +.. opcode:: SETUP_ANNOTATIONS + + Checks whether ``__annotations__`` is defined in ``locals()``, if not it is + set up to an empty ``dict``. This opcode is only emmitted if a class + or module body contains :term:`variable annotations ` + statically. .. opcode:: IMPORT_STAR @@ -890,6 +896,11 @@ Deletes local ``co_varnames[var_num]``. +.. opcode:: STORE_ANNOTATION (namei) + + Stores TOS as ``locals()['__annotations__'][co_names[namei]] = TOS``. + + .. opcode:: LOAD_CLOSURE (i) Pushes a reference to the cell contained in slot *i* of the cell and free diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -686,33 +686,36 @@ Attribute assignment updates the module's namespace dictionary, e.g., ``m.x = 1`` is equivalent to ``m.__dict__["x"] = 1``. - .. index:: single: __dict__ (module attribute) - - Special read-only attribute: :attr:`~object.__dict__` is the module's namespace as a - dictionary object. - - .. impl-detail:: - - Because of the way CPython clears module dictionaries, the module - dictionary will be cleared when the module falls out of scope even if the - dictionary still has live references. To avoid this, copy the dictionary - or keep the module around while using its dictionary directly. - .. index:: single: __name__ (module attribute) single: __doc__ (module attribute) single: __file__ (module attribute) + single: __annotations__ (module attribute) pair: module; namespace Predefined (writable) attributes: :attr:`__name__` is the module's name; :attr:`__doc__` is the module's documentation string, or ``None`` if - unavailable; :attr:`__file__` is the pathname of the file from which the + unavailable; :attr:`__annotations__` (optional) is a dictionary containing + :term:`variable annotations ` collected during module + body execution; :attr:`__file__` is the pathname of the file from which the module was loaded, if it was loaded from a file. The :attr:`__file__` attribute may be missing for certain types of modules, such as C modules that are statically linked into the interpreter; for extension modules loaded dynamically from a shared library, it is the pathname of the shared library file. + .. index:: single: __dict__ (module attribute) + + Special read-only attribute: :attr:`~object.__dict__` is the module's + namespace as a dictionary object. + + .. impl-detail:: + + Because of the way CPython clears module dictionaries, the module + dictionary will be cleared when the module falls out of scope even if the + dictionary still has live references. To avoid this, copy the dictionary + or keep the module around while using its dictionary directly. + Custom classes Custom class types are typically created by class definitions (see section :ref:`class`). A class has a namespace implemented by a dictionary object. @@ -761,13 +764,17 @@ single: __dict__ (class attribute) single: __bases__ (class attribute) single: __doc__ (class attribute) + single: __annotations__ (class attribute) Special attributes: :attr:`~definition.__name__` is the class name; :attr:`__module__` is the module name in which the class was defined; :attr:`~object.__dict__` is the dictionary containing the class's namespace; :attr:`~class.__bases__` is a tuple (possibly empty or a singleton) containing the base classes, in the order of their occurrence in the base class list; :attr:`__doc__` is the - class's documentation string, or None if undefined. + class's documentation string, or None if undefined; + :attr:`__annotations__` (optional) is a dictionary containing + :term:`variable annotations ` collected during + class body execution. Class instances .. index:: diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -16,6 +16,7 @@ : | `assert_stmt` : | `assignment_stmt` : | `augmented_assignment_stmt` + : | `annotated_assignment_stmt` : | `pass_stmt` : | `del_stmt` : | `return_stmt` @@ -312,6 +313,49 @@ and instance attributes ` applies as for regular assignments. +.. _annassign: + +Annotated assignment statements +------------------------------- + +.. index:: + pair: annotated; assignment + single: statement; assignment, annotated + +Annotation assignment is the combination, in a single statement, +of a variable or attribute annotation and an optional assignment statement: + +.. productionlist:: + annotated_assignment_stmt: `augtarget` ":" `expression` ["=" `expression`] + +The difference from normal :ref:`assignment` is that only single target and +only single right hand side value is allowed. + +For simple names as assignment targets, if in class or module scope, +the annotations are evaluated and stored in a special class or module +attribute :attr:`__annotations__` +that is a dictionary mapping from variable names to evaluated annotations. +This attribute is writable and is automatically created at the start +of class or module body execution, if annotations are found statically. + +For expressions as assignment targets, the annotations are evaluated if +in class or module scope, but not stored. + +If a name is annotated in a function scope, then this name is local for +that scope. Annotations are never evaluated and stored in function scopes. + +If the right hand side is present, an annotated +assignment performs the actual assignment before evaluating annotations +(where applicable). If the right hand side is not present for an expression +target, then the interpreter evaluates the target except for the last +:meth:`__setitem__` or :meth:`__setattr__` call. + +.. seealso:: + + :pep:`526` - Variable and attribute annotation syntax + :pep:`484` - Type hints + + .. _assert: The :keyword:`assert` statement diff --git a/Grammar/Grammar b/Grammar/Grammar --- a/Grammar/Grammar +++ b/Grammar/Grammar @@ -45,12 +45,13 @@ simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE small_stmt: (expr_stmt | del_stmt | pass_stmt | flow_stmt | import_stmt | global_stmt | nonlocal_stmt | assert_stmt) -expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) | +expr_stmt: testlist_star_expr (annassign | augassign (yield_expr|testlist) | ('=' (yield_expr|testlist_star_expr))*) +annassign: ':' test ['=' test] testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [','] augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=') -# For normal assignments, additional restrictions enforced by the interpreter +# For normal and annotated assignments, additional restrictions enforced by the interpreter del_stmt: 'del' exprlist pass_stmt: 'pass' flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt diff --git a/Include/Python-ast.h b/Include/Python-ast.h --- a/Include/Python-ast.h +++ b/Include/Python-ast.h @@ -65,11 +65,12 @@ enum _stmt_kind {FunctionDef_kind=1, AsyncFunctionDef_kind=2, ClassDef_kind=3, Return_kind=4, Delete_kind=5, Assign_kind=6, - AugAssign_kind=7, For_kind=8, AsyncFor_kind=9, While_kind=10, - If_kind=11, With_kind=12, AsyncWith_kind=13, Raise_kind=14, - Try_kind=15, Assert_kind=16, Import_kind=17, - ImportFrom_kind=18, Global_kind=19, Nonlocal_kind=20, - Expr_kind=21, Pass_kind=22, Break_kind=23, Continue_kind=24}; + AugAssign_kind=7, AnnAssign_kind=8, For_kind=9, + AsyncFor_kind=10, While_kind=11, If_kind=12, With_kind=13, + AsyncWith_kind=14, Raise_kind=15, Try_kind=16, + Assert_kind=17, Import_kind=18, ImportFrom_kind=19, + Global_kind=20, Nonlocal_kind=21, Expr_kind=22, Pass_kind=23, + Break_kind=24, Continue_kind=25}; struct _stmt { enum _stmt_kind kind; union { @@ -118,6 +119,13 @@ struct { expr_ty target; + expr_ty annotation; + expr_ty value; + int simple; + } AnnAssign; + + struct { + expr_ty target; expr_ty iter; asdl_seq *body; asdl_seq *orelse; @@ -461,6 +469,9 @@ #define AugAssign(a0, a1, a2, a3, a4, a5) _Py_AugAssign(a0, a1, a2, a3, a4, a5) stmt_ty _Py_AugAssign(expr_ty target, operator_ty op, expr_ty value, int lineno, int col_offset, PyArena *arena); +#define AnnAssign(a0, a1, a2, a3, a4, a5, a6) _Py_AnnAssign(a0, a1, a2, a3, a4, a5, a6) +stmt_ty _Py_AnnAssign(expr_ty target, expr_ty annotation, expr_ty value, int + simple, int lineno, int col_offset, PyArena *arena); #define For(a0, a1, a2, a3, a4, a5, a6) _Py_For(a0, a1, a2, a3, a4, a5, a6) stmt_ty _Py_For(expr_ty target, expr_ty iter, asdl_seq * body, asdl_seq * orelse, int lineno, int col_offset, PyArena *arena); diff --git a/Include/graminit.h b/Include/graminit.h --- a/Include/graminit.h +++ b/Include/graminit.h @@ -17,71 +17,72 @@ #define simple_stmt 270 #define small_stmt 271 #define expr_stmt 272 -#define testlist_star_expr 273 -#define augassign 274 -#define del_stmt 275 -#define pass_stmt 276 -#define flow_stmt 277 -#define break_stmt 278 -#define continue_stmt 279 -#define return_stmt 280 -#define yield_stmt 281 -#define raise_stmt 282 -#define import_stmt 283 -#define import_name 284 -#define import_from 285 -#define import_as_name 286 -#define dotted_as_name 287 -#define import_as_names 288 -#define dotted_as_names 289 -#define dotted_name 290 -#define global_stmt 291 -#define nonlocal_stmt 292 -#define assert_stmt 293 -#define compound_stmt 294 -#define async_stmt 295 -#define if_stmt 296 -#define while_stmt 297 -#define for_stmt 298 -#define try_stmt 299 -#define with_stmt 300 -#define with_item 301 -#define except_clause 302 -#define suite 303 -#define test 304 -#define test_nocond 305 -#define lambdef 306 -#define lambdef_nocond 307 -#define or_test 308 -#define and_test 309 -#define not_test 310 -#define comparison 311 -#define comp_op 312 -#define star_expr 313 -#define expr 314 -#define xor_expr 315 -#define and_expr 316 -#define shift_expr 317 -#define arith_expr 318 -#define term 319 -#define factor 320 -#define power 321 -#define atom_expr 322 -#define atom 323 -#define testlist_comp 324 -#define trailer 325 -#define subscriptlist 326 -#define subscript 327 -#define sliceop 328 -#define exprlist 329 -#define testlist 330 -#define dictorsetmaker 331 -#define classdef 332 -#define arglist 333 -#define argument 334 -#define comp_iter 335 -#define comp_for 336 -#define comp_if 337 -#define encoding_decl 338 -#define yield_expr 339 -#define yield_arg 340 +#define annassign 273 +#define testlist_star_expr 274 +#define augassign 275 +#define del_stmt 276 +#define pass_stmt 277 +#define flow_stmt 278 +#define break_stmt 279 +#define continue_stmt 280 +#define return_stmt 281 +#define yield_stmt 282 +#define raise_stmt 283 +#define import_stmt 284 +#define import_name 285 +#define import_from 286 +#define import_as_name 287 +#define dotted_as_name 288 +#define import_as_names 289 +#define dotted_as_names 290 +#define dotted_name 291 +#define global_stmt 292 +#define nonlocal_stmt 293 +#define assert_stmt 294 +#define compound_stmt 295 +#define async_stmt 296 +#define if_stmt 297 +#define while_stmt 298 +#define for_stmt 299 +#define try_stmt 300 +#define with_stmt 301 +#define with_item 302 +#define except_clause 303 +#define suite 304 +#define test 305 +#define test_nocond 306 +#define lambdef 307 +#define lambdef_nocond 308 +#define or_test 309 +#define and_test 310 +#define not_test 311 +#define comparison 312 +#define comp_op 313 +#define star_expr 314 +#define expr 315 +#define xor_expr 316 +#define and_expr 317 +#define shift_expr 318 +#define arith_expr 319 +#define term 320 +#define factor 321 +#define power 322 +#define atom_expr 323 +#define atom 324 +#define testlist_comp 325 +#define trailer 326 +#define subscriptlist 327 +#define subscript 328 +#define sliceop 329 +#define exprlist 330 +#define testlist 331 +#define dictorsetmaker 332 +#define classdef 333 +#define arglist 334 +#define argument 335 +#define comp_iter 336 +#define comp_for 337 +#define comp_if 338 +#define encoding_decl 339 +#define yield_expr 340 +#define yield_arg 341 diff --git a/Include/opcode.h b/Include/opcode.h --- a/Include/opcode.h +++ b/Include/opcode.h @@ -60,6 +60,7 @@ #define WITH_CLEANUP_FINISH 82 #define RETURN_VALUE 83 #define IMPORT_STAR 84 +#define SETUP_ANNOTATIONS 85 #define YIELD_VALUE 86 #define POP_BLOCK 87 #define END_FINALLY 88 @@ -98,6 +99,7 @@ #define LOAD_FAST 124 #define STORE_FAST 125 #define DELETE_FAST 126 +#define STORE_ANNOTATION 127 #define RAISE_VARARGS 130 #define CALL_FUNCTION 131 #define MAKE_FUNCTION 132 diff --git a/Include/symtable.h b/Include/symtable.h --- a/Include/symtable.h +++ b/Include/symtable.h @@ -91,6 +91,7 @@ #define DEF_FREE 2<<4 /* name used but not defined in nested block */ #define DEF_FREE_CLASS 2<<5 /* free variable from class's method */ #define DEF_IMPORT 2<<6 /* assignment occurred via import */ +#define DEF_ANNOT 2<<7 /* this name is annotated */ #define DEF_BOUND (DEF_LOCAL | DEF_PARAM | DEF_IMPORT) diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -234,6 +234,8 @@ # Python 3.6a1 3372 (MAKE_FUNCTION simplification, remove MAKE_CLOSURE # #27095) # Python 3.6b1 3373 (add BUILD_STRING opcode #27078) +# Python 3.6b1 3375 (add SETUP_ANNOTATIONS and STORE_ANNOTATION opcodes +# #27985) # # MAGIC must change whenever the bytecode emitted by the compiler may no # longer be understood by older implementations of the eval loop (usually @@ -242,7 +244,7 @@ # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # in PC/launcher.c must also be updated. -MAGIC_NUMBER = (3373).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3375).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -54,12 +54,13 @@ simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE small_stmt: (expr_stmt | print_stmt | del_stmt | pass_stmt | flow_stmt | import_stmt | global_stmt | exec_stmt | assert_stmt) -expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) | +expr_stmt: testlist_star_expr (annassign | augassign (yield_expr|testlist) | ('=' (yield_expr|testlist_star_expr))*) +annassign: ':' test ['=' test] testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [','] augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=') -# For normal assignments, additional restrictions enforced by the interpreter +# For normal and annotated assignments, additional restrictions enforced by the interpreter print_stmt: 'print' ( [ test (',' test)* [','] ] | '>>' test [ (',' test)+ [','] ] ) del_stmt: 'del' exprlist diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -237,6 +237,36 @@ self.validate(s) +# Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.test_var_annot +class TestFunctionAnnotations(GrammarTest): + def test_1(self): + self.validate("var1: int = 5") + + def test_2(self): + self.validate("var2: [int, str]") + + def test_3(self): + self.validate("def f():\n" + " st: str = 'Hello'\n" + " a.b: int = (1, 2)\n" + " return st\n") + + def test_4(self): + self.validate("def fbad():\n" + " x: int\n" + " print(x)\n") + + def test_5(self): + self.validate("class C:\n" + " x: int\n" + " s: str = 'attr'\n" + " z = 2\n" + " def __init__(self, x):\n" + " self.x: int = x\n") + + def test_6(self): + self.validate("lst: List[int] = []") + class TestExcept(GrammarTest): def test_new(self): s = """ diff --git a/Lib/opcode.py b/Lib/opcode.py --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -119,7 +119,7 @@ def_op('RETURN_VALUE', 83) def_op('IMPORT_STAR', 84) - +def_op('SETUP_ANNOTATIONS', 85) def_op('YIELD_VALUE', 86) def_op('POP_BLOCK', 87) def_op('END_FINALLY', 88) @@ -169,6 +169,7 @@ haslocal.append(125) def_op('DELETE_FAST', 126) # Local variable number haslocal.append(126) +name_op('STORE_ANNOTATION', 127) # Index in name list def_op('RAISE_VARARGS', 130) # Number of raise arguments (1, 2, or 3) def_op('CALL_FUNCTION', 131) # #args + (#kwargs << 8) diff --git a/Lib/symbol.py b/Lib/symbol.py --- a/Lib/symbol.py +++ b/Lib/symbol.py @@ -27,74 +27,75 @@ simple_stmt = 270 small_stmt = 271 expr_stmt = 272 -testlist_star_expr = 273 -augassign = 274 -del_stmt = 275 -pass_stmt = 276 -flow_stmt = 277 -break_stmt = 278 -continue_stmt = 279 -return_stmt = 280 -yield_stmt = 281 -raise_stmt = 282 -import_stmt = 283 -import_name = 284 -import_from = 285 -import_as_name = 286 -dotted_as_name = 287 -import_as_names = 288 -dotted_as_names = 289 -dotted_name = 290 -global_stmt = 291 -nonlocal_stmt = 292 -assert_stmt = 293 -compound_stmt = 294 -async_stmt = 295 -if_stmt = 296 -while_stmt = 297 -for_stmt = 298 -try_stmt = 299 -with_stmt = 300 -with_item = 301 -except_clause = 302 -suite = 303 -test = 304 -test_nocond = 305 -lambdef = 306 -lambdef_nocond = 307 -or_test = 308 -and_test = 309 -not_test = 310 -comparison = 311 -comp_op = 312 -star_expr = 313 -expr = 314 -xor_expr = 315 -and_expr = 316 -shift_expr = 317 -arith_expr = 318 -term = 319 -factor = 320 -power = 321 -atom_expr = 322 -atom = 323 -testlist_comp = 324 -trailer = 325 -subscriptlist = 326 -subscript = 327 -sliceop = 328 -exprlist = 329 -testlist = 330 -dictorsetmaker = 331 -classdef = 332 -arglist = 333 -argument = 334 -comp_iter = 335 -comp_for = 336 -comp_if = 337 -encoding_decl = 338 -yield_expr = 339 -yield_arg = 340 +annassign = 273 +testlist_star_expr = 274 +augassign = 275 +del_stmt = 276 +pass_stmt = 277 +flow_stmt = 278 +break_stmt = 279 +continue_stmt = 280 +return_stmt = 281 +yield_stmt = 282 +raise_stmt = 283 +import_stmt = 284 +import_name = 285 +import_from = 286 +import_as_name = 287 +dotted_as_name = 288 +import_as_names = 289 +dotted_as_names = 290 +dotted_name = 291 +global_stmt = 292 +nonlocal_stmt = 293 +assert_stmt = 294 +compound_stmt = 295 +async_stmt = 296 +if_stmt = 297 +while_stmt = 298 +for_stmt = 299 +try_stmt = 300 +with_stmt = 301 +with_item = 302 +except_clause = 303 +suite = 304 +test = 305 +test_nocond = 306 +lambdef = 307 +lambdef_nocond = 308 +or_test = 309 +and_test = 310 +not_test = 311 +comparison = 312 +comp_op = 313 +star_expr = 314 +expr = 315 +xor_expr = 316 +and_expr = 317 +shift_expr = 318 +arith_expr = 319 +term = 320 +factor = 321 +power = 322 +atom_expr = 323 +atom = 324 +testlist_comp = 325 +trailer = 326 +subscriptlist = 327 +subscript = 328 +sliceop = 329 +exprlist = 330 +testlist = 331 +dictorsetmaker = 332 +classdef = 333 +arglist = 334 +argument = 335 +comp_iter = 336 +comp_for = 337 +comp_if = 338 +encoding_decl = 339 +yield_expr = 340 +yield_arg = 341 #--end constants-- sym_name = {} diff --git a/Lib/symtable.py b/Lib/symtable.py --- a/Lib/symtable.py +++ b/Lib/symtable.py @@ -2,7 +2,7 @@ import _symtable from _symtable import (USE, DEF_GLOBAL, DEF_LOCAL, DEF_PARAM, - DEF_IMPORT, DEF_BOUND, SCOPE_OFF, SCOPE_MASK, FREE, + DEF_IMPORT, DEF_BOUND, DEF_ANNOT, SCOPE_OFF, SCOPE_MASK, FREE, LOCAL, GLOBAL_IMPLICIT, GLOBAL_EXPLICIT, CELL) import weakref @@ -190,6 +190,9 @@ def is_local(self): return bool(self.__flags & DEF_BOUND) + def is_annotated(self): + return bool(self.__flags & DEF_ANNOT) + def is_free(self): return bool(self.__scope == FREE) diff --git a/Lib/test/ann_module.py b/Lib/test/ann_module.py new file mode 100644 --- /dev/null +++ b/Lib/test/ann_module.py @@ -0,0 +1,53 @@ + + +""" +The module for testing variable annotations. +Empty lines above are for good reason (testing for correct line numbers) +""" + +from typing import Optional + +__annotations__[1] = 2 + +class C: + + x = 5; y: Optional['C'] = None + +from typing import Tuple +x: int = 5; y: str = x; f: Tuple[int, int] + +class M(type): + + __annotations__['123'] = 123 + o: type = object + +(pars): bool = True + +class D(C): + j: str = 'hi'; k: str= 'bye' + +from types import new_class +h_class = new_class('H', (C,)) +j_class = new_class('J') + +class F(): + z: int = 5 + def __init__(self, x): + pass + +class Y(F): + def __init__(self): + super(F, self).__init__(123) + +class Meta(type): + def __new__(meta, name, bases, namespace): + return super().__new__(meta, name, bases, namespace) + +class S(metaclass = Meta): + x: str = 'something' + y: str = 'something else' + +def foo(x: int = 10): + def bar(y: List[str]): + x: str = 'yes' + bar() diff --git a/Lib/test/ann_module2.py b/Lib/test/ann_module2.py new file mode 100644 --- /dev/null +++ b/Lib/test/ann_module2.py @@ -0,0 +1,36 @@ +""" +Some correct syntax for variable annotation here. +More examples are in test_grammar and test_parser. +""" + +from typing import no_type_check, ClassVar + +i: int = 1 +j: int +x: float = i/10 + +def f(): + class C: ... + return C() + +f().new_attr: object = object() + +class C: + def __init__(self, x: int) -> None: + self.x = x + +c = C(5) +c.new_attr: int = 10 + +__annotations__ = {} + + + at no_type_check +class NTC: + def meth(self, param: complex) -> None: + ... + +class CV: + var: ClassVar['CV'] + +CV.var = CV() diff --git a/Lib/test/ann_module3.py b/Lib/test/ann_module3.py new file mode 100644 --- /dev/null +++ b/Lib/test/ann_module3.py @@ -0,0 +1,18 @@ +""" +Correct syntax for variable annotation that should fail at runtime +in a certain manner. More examples are in test_grammar and test_parser. +""" + +def f_bad_ann(): + __annotations__[1] = 2 + +class C_OK: + def __init__(self, x: int) -> None: + self.x: no_such_name = x # This one is OK as proposed by Guido + +class D_bad_ann: + def __init__(self, x: int) -> None: + sfel.y: int = 0 + +def g_bad_ann(): + no_such_name.attr: int = 0 diff --git a/Lib/test/pydoc_mod.py b/Lib/test/pydoc_mod.py --- a/Lib/test/pydoc_mod.py +++ b/Lib/test/pydoc_mod.py @@ -12,7 +12,7 @@ pass class B(object): - NO_MEANING = "eggs" + NO_MEANING: str = "eggs" pass class C(object): diff --git a/Lib/test/test___all__.py b/Lib/test/test___all__.py --- a/Lib/test/test___all__.py +++ b/Lib/test/test___all__.py @@ -38,6 +38,8 @@ modname, e.__class__.__name__, e)) if "__builtins__" in names: del names["__builtins__"] + if '__annotations__' in names: + del names['__annotations__'] keys = set(names) all_list = sys.modules[modname].__all__ all_set = set(all_list) diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -207,6 +207,38 @@ 10 RETURN_VALUE """ +annot_stmt_str = """\ + +x: int = 1 +y: fun(1) +lst[fun(0)]: int = 1 +""" +# leading newline is for a reason (tests lineno) + +dis_annot_stmt_str = """\ + 2 0 SETUP_ANNOTATIONS + 2 LOAD_CONST 0 (1) + 4 STORE_NAME 0 (x) + 6 LOAD_NAME 1 (int) + 8 STORE_ANNOTATION 0 (x) + + 3 10 LOAD_NAME 2 (fun) + 12 LOAD_CONST 0 (1) + 14 CALL_FUNCTION 1 (1 positional, 0 keyword pair) + 16 STORE_ANNOTATION 3 (y) + + 4 18 LOAD_CONST 0 (1) + 20 LOAD_NAME 4 (lst) + 22 LOAD_NAME 2 (fun) + 24 LOAD_CONST 1 (0) + 26 CALL_FUNCTION 1 (1 positional, 0 keyword pair) + 28 STORE_SUBSCR + 30 LOAD_NAME 1 (int) + 32 POP_TOP + 34 LOAD_CONST 2 (None) + 36 RETURN_VALUE +""" + compound_stmt_str = """\ x = 0 while 1: @@ -345,6 +377,7 @@ def test_disassemble_str(self): self.do_disassembly_test(expr_str, dis_expr_str) self.do_disassembly_test(simple_stmt_str, dis_simple_stmt_str) + self.do_disassembly_test(annot_stmt_str, dis_annot_stmt_str) self.do_disassembly_test(compound_stmt_str, dis_compound_stmt_str) def test_disassemble_bytes(self): diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -8,6 +8,14 @@ # testing import * from sys import * +# different import patterns to check that __annotations__ does not interfere +# with import machinery +import test.ann_module as ann_module +import typing +from collections import ChainMap +from test import ann_module2 +import test + class TokenTests(unittest.TestCase): @@ -139,6 +147,19 @@ compile(s, "", "exec") self.assertIn("unexpected EOF", str(cm.exception)) +var_annot_global: int # a global annotated is necessary for test_var_annot + +# custom namespace for testing __annotations__ + +class CNS: + def __init__(self): + self._dct = {} + def __setitem__(self, item, value): + self._dct[item.lower()] = value + def __getitem__(self, item): + return self._dct[item] + + class GrammarTests(unittest.TestCase): # single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE @@ -154,6 +175,163 @@ # testlist ENDMARKER x = eval('1, 0 or 1') + def test_var_annot_basics(self): + # all these should be allowed + var1: int = 5 + var2: [int, str] + my_lst = [42] + def one(): + return 1 + int.new_attr: int + [list][0]: type + my_lst[one()-1]: int = 5 + self.assertEqual(my_lst, [5]) + + def test_var_annot_syntax_errors(self): + # parser pass + check_syntax_error(self, "def f: int") + check_syntax_error(self, "x: int: str") + check_syntax_error(self, "def f():\n" + " nonlocal x: int\n") + # AST pass + check_syntax_error(self, "[x, 0]: int\n") + check_syntax_error(self, "f(): int\n") + check_syntax_error(self, "(x,): int") + check_syntax_error(self, "def f():\n" + " (x, y): int = (1, 2)\n") + # symtable pass + check_syntax_error(self, "def f():\n" + " x: int\n" + " global x\n") + check_syntax_error(self, "def f():\n" + " global x\n" + " x: int\n") + + def test_var_annot_basic_semantics(self): + # execution order + with self.assertRaises(ZeroDivisionError): + no_name[does_not_exist]: no_name_again = 1/0 + with self.assertRaises(NameError): + no_name[does_not_exist]: 1/0 = 0 + global var_annot_global + + # function semantics + def f(): + st: str = "Hello" + a.b: int = (1, 2) + return st + self.assertEqual(f.__annotations__, {}) + def f_OK(): + x: 1/0 + f_OK() + def fbad(): + x: int + print(x) + with self.assertRaises(UnboundLocalError): + fbad() + def f2bad(): + (no_such_global): int + print(no_such_global) + try: + f2bad() + except Exception as e: + self.assertIs(type(e), NameError) + + # class semantics + class C: + x: int + s: str = "attr" + z = 2 + def __init__(self, x): + self.x: int = x + self.assertEqual(C.__annotations__, {'x': int, 's': str}) + with self.assertRaises(NameError): + class CBad: + no_such_name_defined.attr: int = 0 + with self.assertRaises(NameError): + class Cbad2(C): + x: int + x.y: list = [] + + def test_var_annot_metaclass_semantics(self): + class CMeta(type): + @classmethod + def __prepare__(metacls, name, bases, **kwds): + return {'__annotations__': CNS()} + class CC(metaclass=CMeta): + XX: 'ANNOT' + self.assertEqual(CC.__annotations__['xx'], 'ANNOT') + + def test_var_annot_module_semantics(self): + with self.assertRaises(AttributeError): + print(test.__annotations__) + self.assertEqual(ann_module.__annotations__, + {1: 2, 'x': int, 'y': str, 'f': typing.Tuple[int, int]}) + self.assertEqual(ann_module.M.__annotations__, + {'123': 123, 'o': type}) + self.assertEqual(ann_module2.__annotations__, {}) + self.assertEqual(typing.get_type_hints(ann_module2.CV, + ann_module2.__dict__), + ChainMap({'var': typing.ClassVar[ann_module2.CV]}, {})) + + def test_var_annot_in_module(self): + # check that functions fail the same way when executed + # outside of module where they were defined + from test.ann_module3 import f_bad_ann, g_bad_ann, D_bad_ann + with self.assertRaises(NameError): + f_bad_ann() + with self.assertRaises(NameError): + g_bad_ann() + with self.assertRaises(NameError): + D_bad_ann(5) + + def test_var_annot_simple_exec(self): + gns = {}; lns= {} + exec("'docstring'\n" + "__annotations__[1] = 2\n" + "x: int = 5\n", gns, lns) + self.assertEqual(lns["__annotations__"], {1: 2, 'x': int}) + with self.assertRaises(KeyError): + gns['__annotations__'] + + def test_var_annot_custom_maps(self): + # tests with custom locals() and __annotations__ + ns = {'__annotations__': CNS()} + exec('X: int; Z: str = "Z"; (w): complex = 1j', ns) + self.assertEqual(ns['__annotations__']['x'], int) + self.assertEqual(ns['__annotations__']['z'], str) + with self.assertRaises(KeyError): + ns['__annotations__']['w'] + nonloc_ns = {} + class CNS2: + def __init__(self): + self._dct = {} + def __setitem__(self, item, value): + nonlocal nonloc_ns + self._dct[item] = value + nonloc_ns[item] = value + def __getitem__(self, item): + return self._dct[item] + exec('x: int = 1', {}, CNS2()) + self.assertEqual(nonloc_ns['__annotations__']['x'], int) + + def test_var_annot_refleak(self): + # complex case: custom locals plus custom __annotations__ + # this was causing refleak + cns = CNS() + nonloc_ns = {'__annotations__': cns} + class CNS2: + def __init__(self): + self._dct = {'__annotations__': cns} + def __setitem__(self, item, value): + nonlocal nonloc_ns + self._dct[item] = value + nonloc_ns[item] = value + def __getitem__(self, item): + return self._dct[item] + exec('X: str', {}, CNS2()) + self.assertEqual(nonloc_ns['__annotations__']['x'], str) + def test_funcdef(self): ### [decorators] 'def' NAME parameters ['->' test] ':' suite ### decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE diff --git a/Lib/test/test_opcodes.py b/Lib/test/test_opcodes.py --- a/Lib/test/test_opcodes.py +++ b/Lib/test/test_opcodes.py @@ -1,6 +1,7 @@ # Python test set -- part 2, opcodes import unittest +from test import ann_module class OpcodeTest(unittest.TestCase): @@ -20,6 +21,32 @@ if n != 90: self.fail('try inside for') + def test_setup_annotations_line(self): + # check that SETUP_ANNOTATIONS does not create spurious line numbers + try: + with open(ann_module.__file__) as f: + txt = f.read() + co = compile(txt, ann_module.__file__, 'exec') + self.assertEqual(co.co_firstlineno, 6) + except OSError: + pass + + def test_no_annotations_if_not_needed(self): + class C: pass + with self.assertRaises(AttributeError): + C.__annotations__ + + def test_use_existing_annotations(self): + ns = {'__annotations__': {1: 2}} + exec('x: int', ns) + self.assertEqual(ns['__annotations__'], {'x': int, 1: 2}) + + def test_do_not_recreate_annotations(self): + class C: + del __annotations__ + with self.assertRaises(NameError): + x: int + def test_raise_class_exceptions(self): class AClass(Exception): pass diff --git a/Lib/test/test_parser.py b/Lib/test/test_parser.py --- a/Lib/test/test_parser.py +++ b/Lib/test/test_parser.py @@ -138,6 +138,45 @@ self.check_suite("a = b") self.check_suite("a = b = c = d = e") + def test_var_annot(self): + self.check_suite("x: int = 5") + self.check_suite("y: List[T] = []; z: [list] = fun()") + self.check_suite("x: tuple = (1, 2)") + self.check_suite("d[f()]: int = 42") + self.check_suite("f(d[x]): str = 'abc'") + self.check_suite("x.y.z.w: complex = 42j") + self.check_suite("x: int") + self.check_suite("def f():\n" + " x: str\n" + " y: int = 5\n") + self.check_suite("class C:\n" + " x: str\n" + " y: int = 5\n") + self.check_suite("class C:\n" + " def __init__(self, x: int) -> None:\n" + " self.x: int = x\n") + # double check for nonsense + with self.assertRaises(SyntaxError): + exec("2+2: int", {}, {}) + with self.assertRaises(SyntaxError): + exec("[]: int = 5", {}, {}) + with self.assertRaises(SyntaxError): + exec("x, *y, z: int = range(5)", {}, {}) + with self.assertRaises(SyntaxError): + exec("t: tuple = 1, 2", {}, {}) + with self.assertRaises(SyntaxError): + exec("u = v: int", {}, {}) + with self.assertRaises(SyntaxError): + exec("False: int", {}, {}) + with self.assertRaises(SyntaxError): + exec("x.False: int", {}, {}) + with self.assertRaises(SyntaxError): + exec("x.y,: int", {}, {}) + with self.assertRaises(SyntaxError): + exec("[0]: int", {}, {}) + with self.assertRaises(SyntaxError): + exec("f(): int", {}, {}) + def test_simple_augmented_assignments(self): self.check_suite("a += b") self.check_suite("a -= b") diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py --- a/Lib/test/test_pydoc.py +++ b/Lib/test/test_pydoc.py @@ -83,6 +83,8 @@ | Data and other attributes defined here: |\x20\x20 | NO_MEANING = 'eggs' + |\x20\x20 + | __annotations__ = {'NO_MEANING': } \x20\x20\x20\x20 class C(builtins.object) | Methods defined here: @@ -195,6 +197,8 @@ Data and other attributes defined here:

NO_MEANING = 'eggs'
+
__annotations__ = {'NO_MEANING': <class 'str'>}
+

diff --git a/Lib/test/test_symtable.py b/Lib/test/test_symtable.py --- a/Lib/test/test_symtable.py +++ b/Lib/test/test_symtable.py @@ -133,6 +133,17 @@ self.assertTrue(self.Mine.lookup("a_method").is_assigned()) self.assertFalse(self.internal.lookup("x").is_assigned()) + def test_annotated(self): + st1 = symtable.symtable('def f():\n x: int\n', 'test', 'exec') + st2 = st1.get_children()[0] + self.assertTrue(st2.lookup('x').is_local()) + self.assertTrue(st2.lookup('x').is_annotated()) + self.assertFalse(st2.lookup('x').is_global()) + st3 = symtable.symtable('def f():\n x = 1\n', 'test', 'exec') + st4 = st3.get_children()[0] + self.assertTrue(st4.lookup('x').is_local()) + self.assertFalse(st4.lookup('x').is_annotated()) + def test_imported(self): self.assertTrue(self.top.lookup("sys").is_imported()) diff --git a/Lib/test/test_tools/test_com2ann.py b/Lib/test/test_tools/test_com2ann.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_tools/test_com2ann.py @@ -0,0 +1,260 @@ +"""Tests for the com2ann.py script in the Tools/parser directory.""" + +import unittest +import test.support +import os +import re + +from test.test_tools import basepath, toolsdir, skip_if_missing + +skip_if_missing() + +parser_path = os.path.join(toolsdir, "parser") + +with test.support.DirsOnSysPath(parser_path): + from com2ann import * + +class BaseTestCase(unittest.TestCase): + + def check(self, code, expected, n=False, e=False): + self.assertEqual(com2ann(code, + drop_None=n, drop_Ellipsis=e, silent=True), + expected) + +class SimpleTestCase(BaseTestCase): + # Tests for basic conversions + + def test_basics(self): + self.check("z = 5", "z = 5") + self.check("z: int = 5", "z: int = 5") + self.check("z = 5 # type: int", "z: int = 5") + self.check("z = 5 # type: int # comment", + "z: int = 5 # comment") + + def test_type_ignore(self): + self.check("foobar = foobaz() #type: ignore", + "foobar = foobaz() #type: ignore") + self.check("a = 42 #type: ignore #comment", + "a = 42 #type: ignore #comment") + + def test_complete_tuple(self): + self.check("t = 1, 2, 3 # type: Tuple[int, ...]", + "t: Tuple[int, ...] = (1, 2, 3)") + self.check("t = 1, # type: Tuple[int]", + "t: Tuple[int] = (1,)") + self.check("t = (1, 2, 3) # type: Tuple[int, ...]", + "t: Tuple[int, ...] = (1, 2, 3)") + + def test_drop_None(self): + self.check("x = None # type: int", + "x: int", True) + self.check("x = None # type: int # another", + "x: int # another", True) + self.check("x = None # type: int # None", + "x: int # None", True) + + def test_drop_Ellipsis(self): + self.check("x = ... # type: int", + "x: int", False, True) + self.check("x = ... # type: int # another", + "x: int # another", False, True) + self.check("x = ... # type: int # ...", + "x: int # ...", False, True) + + def test_newline(self): + self.check("z = 5 # type: int\r\n", "z: int = 5\r\n") + self.check("z = 5 # type: int # comment\x85", + "z: int = 5 # comment\x85") + + def test_wrong(self): + self.check("#type : str", "#type : str") + self.check("x==y #type: bool", "x==y #type: bool") + + def test_pattern(self): + for line in ["#type: int", " # type: str[:] # com"]: + self.assertTrue(re.search(TYPE_COM, line)) + for line in ["", "#", "# comment", "#type", "type int:"]: + self.assertFalse(re.search(TYPE_COM, line)) + +class BigTestCase(BaseTestCase): + # Tests for really crazy formatting, to be sure + # that script works reasonably in extreme situations + + def test_crazy(self): + self.maxDiff = None + self.check(crazy_code, big_result, False, False) + self.check(crazy_code, big_result_ne, True, True) + +crazy_code = """\ +# -*- coding: utf-8 -*- # this should not be spoiled +''' +Docstring here +''' + +import testmod +x = 5 #type : int # this one is OK +ttt \\ + = \\ + 1.0, \\ + 2.0, \\ + 3.0, #type: Tuple[float, float, float] +with foo(x==1) as f: #type: str + print(f) + +for i, j in my_inter(x=1): # type: ignore + i + j # type: int # what about this + +x = y = z = 1 # type: int +x, y, z = [], [], [] # type: (List[int], List[int], List[str]) +class C: + + + l[f(x + =1)] = [ + + 1, + 2, + ] # type: List[int] + + + (C.x[1]) = \\ + 42 == 5# type: bool +lst[...] = \\ + ((\\ +...)) # type: int # comment .. + +y = ... # type: int # comment ... +z = ... +#type: int + + +#DONE placement of annotation after target rather than before = + +TD.x[1] \\ + = 0 == 5# type: bool + +TD.y[1] =5 == 5# type: bool # one more here +F[G(x == y, + +# hm... + + z)]\\ + = None # type: OMG[int] # comment: None +x = None#type:int #comment : None""" + +big_result = """\ +# -*- coding: utf-8 -*- # this should not be spoiled +''' +Docstring here +''' + +import testmod +x: int = 5 # this one is OK +ttt: Tuple[float, float, float] \\ + = \\ + (1.0, \\ + 2.0, \\ + 3.0,) +with foo(x==1) as f: #type: str + print(f) + +for i, j in my_inter(x=1): # type: ignore + i + j # type: int # what about this + +x = y = z = 1 # type: int +x, y, z = [], [], [] # type: (List[int], List[int], List[str]) +class C: + + + l[f(x + =1)]: List[int] = [ + + 1, + 2, + ] + + + (C.x[1]): bool = \\ + 42 == 5 +lst[...]: int = \\ + ((\\ +...)) # comment .. + +y: int = ... # comment ... +z = ... +#type: int + + +#DONE placement of annotation after target rather than before = + +TD.x[1]: bool \\ + = 0 == 5 + +TD.y[1]: bool =5 == 5 # one more here +F[G(x == y, + +# hm... + + z)]: OMG[int]\\ + = None # comment: None +x: int = None #comment : None""" + +big_result_ne = """\ +# -*- coding: utf-8 -*- # this should not be spoiled +''' +Docstring here +''' + +import testmod +x: int = 5 # this one is OK +ttt: Tuple[float, float, float] \\ + = \\ + (1.0, \\ + 2.0, \\ + 3.0,) +with foo(x==1) as f: #type: str + print(f) + +for i, j in my_inter(x=1): # type: ignore + i + j # type: int # what about this + +x = y = z = 1 # type: int +x, y, z = [], [], [] # type: (List[int], List[int], List[str]) +class C: + + + l[f(x + =1)]: List[int] = [ + + 1, + 2, + ] + + + (C.x[1]): bool = \\ + 42 == 5 +lst[...]: int \\ + \\ + # comment .. + +y: int # comment ... +z = ... +#type: int + + +#DONE placement of annotation after target rather than before = + +TD.x[1]: bool \\ + = 0 == 5 + +TD.y[1]: bool =5 == 5 # one more here +F[G(x == y, + +# hm... + + z)]: OMG[int]\\ + # comment: None +x: int #comment : None""" + +if __name__ == '__main__': + unittest.main() diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4,14 +4,16 @@ import re import sys from unittest import TestCase, main, skipUnless, SkipTest +from collections import ChainMap +from test import ann_module, ann_module2, ann_module3 from typing import Any from typing import TypeVar, AnyStr from typing import T, KT, VT # Not in __all__. from typing import Union, Optional -from typing import Tuple +from typing import Tuple, List from typing import Callable -from typing import Generic +from typing import Generic, ClassVar from typing import cast from typing import get_type_hints from typing import no_type_check, no_type_check_decorator @@ -827,6 +829,43 @@ with self.assertRaises(Exception): D[T] +class ClassVarTests(BaseTestCase): + + def test_basics(self): + with self.assertRaises(TypeError): + ClassVar[1] + with self.assertRaises(TypeError): + ClassVar[int, str] + with self.assertRaises(TypeError): + ClassVar[int][str] + + def test_repr(self): + self.assertEqual(repr(ClassVar), 'typing.ClassVar') + cv = ClassVar[int] + self.assertEqual(repr(cv), 'typing.ClassVar[int]') + cv = ClassVar[Employee] + self.assertEqual(repr(cv), 'typing.ClassVar[%s.Employee]' % __name__) + + def test_cannot_subclass(self): + with self.assertRaises(TypeError): + class C(type(ClassVar)): + pass + with self.assertRaises(TypeError): + class C(type(ClassVar[int])): + pass + + def test_cannot_init(self): + with self.assertRaises(TypeError): + type(ClassVar)() + with self.assertRaises(TypeError): + type(ClassVar[Optional[int]])() + + def test_no_isinstance(self): + with self.assertRaises(TypeError): + isinstance(1, ClassVar[int]) + with self.assertRaises(TypeError): + issubclass(int, ClassVar) + class VarianceTests(BaseTestCase): @@ -930,6 +969,46 @@ right_hints = get_type_hints(t.add_right, globals(), locals()) self.assertEqual(right_hints['node'], Optional[Node[T]]) + def test_get_type_hints(self): + gth = get_type_hints + self.assertEqual(gth(ann_module), {'x': int, 'y': str}) + self.assertEqual(gth(ann_module.C, ann_module.__dict__), + ChainMap({'y': Optional[ann_module.C]}, {})) + self.assertEqual(gth(ann_module2), {}) + self.assertEqual(gth(ann_module3), {}) + self.assertEqual(repr(gth(ann_module.j_class)), 'ChainMap({}, {})') + self.assertEqual(gth(ann_module.M), ChainMap({'123': 123, 'o': type}, + {}, {})) + self.assertEqual(gth(ann_module.D), + ChainMap({'j': str, 'k': str, + 'y': Optional[ann_module.C]}, {})) + self.assertEqual(gth(ann_module.Y), ChainMap({'z': int}, {})) + self.assertEqual(gth(ann_module.h_class), + ChainMap({}, {'y': Optional[ann_module.C]}, {})) + self.assertEqual(gth(ann_module.S), ChainMap({'x': str, 'y': str}, + {})) + self.assertEqual(gth(ann_module.foo), {'x': int}) + + def testf(x, y): ... + testf.__annotations__['x'] = 'int' + self.assertEqual(gth(testf), {'x': int}) + self.assertEqual(gth(ann_module2.NTC.meth), {}) + + # interactions with ClassVar + class B: + x: ClassVar[Optional['B']] = None + y: int + class C(B): + z: ClassVar['C'] = B() + class G(Generic[T]): + lst: ClassVar[List[T]] = [] + self.assertEqual(gth(B, locals()), + ChainMap({'y': int, 'x': ClassVar[Optional[B]]}, {})) + self.assertEqual(gth(C, locals()), + ChainMap({'z': ClassVar[C]}, + {'y': int, 'x': ClassVar[Optional[B]]}, {})) + self.assertEqual(gth(G), ChainMap({'lst': ClassVar[List[T]]},{},{})) + def test_forwardref_instance_type_error(self): fr = typing._ForwardRef('int') with self.assertRaises(TypeError): diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -6,6 +6,7 @@ import re as stdlib_re # Avoid confusion with the re we export. import sys import types + try: import collections.abc as collections_abc except ImportError: @@ -17,6 +18,7 @@ # Super-special typing primitives. 'Any', 'Callable', + 'ClassVar', 'Generic', 'Optional', 'Tuple', @@ -270,7 +272,7 @@ def _get_type_vars(types, tvars): for t in types: - if isinstance(t, TypingMeta): + if isinstance(t, TypingMeta) or isinstance(t, _ClassVar): t._get_type_vars(tvars) @@ -281,7 +283,7 @@ def _eval_type(t, globalns, localns): - if isinstance(t, TypingMeta): + if isinstance(t, TypingMeta) or isinstance(t, _ClassVar): return t._eval_type(globalns, localns) else: return t @@ -1114,6 +1116,67 @@ return obj +class _ClassVar(metaclass=TypingMeta, _root=True): + """Special type construct to mark class variables. + + An annotation wrapped in ClassVar indicates that a given + attribute is intended to be used as a class variable and + should not be set on instances of that class. Usage:: + + class Starship: + stats: ClassVar[Dict[str, int]] = {} # class variable + damage: int = 10 # instance variable + + ClassVar accepts only types and cannot be further subscribed. + + Note that ClassVar is not a class itself, and should not + be used with isinstance() or issubclass(). + """ + + def __init__(self, tp=None, _root=False): + cls = type(self) + if _root: + self.__type__ = tp + else: + raise TypeError('Cannot initialize {}'.format(cls.__name__[1:])) + + def __getitem__(self, item): + cls = type(self) + if self.__type__ is None: + return cls(_type_check(item, + '{} accepts only types.'.format(cls.__name__[1:])), + _root=True) + raise TypeError('{} cannot be further subscripted' + .format(cls.__name__[1:])) + + def _eval_type(self, globalns, localns): + return type(self)(_eval_type(self.__type__, globalns, localns), + _root=True) + + def _get_type_vars(self, tvars): + if self.__type__: + _get_type_vars(self.__type__, tvars) + + def __repr__(self): + cls = type(self) + if not self.__type__: + return '{}.{}'.format(cls.__module__, cls.__name__[1:]) + return '{}.{}[{}]'.format(cls.__module__, cls.__name__[1:], + _type_repr(self.__type__)) + + def __hash__(self): + return hash((type(self).__name__, self.__type__)) + + def __eq__(self, other): + if not isinstance(other, _ClassVar): + return NotImplemented + if self.__type__ is not None: + return self.__type__ == other.__type__ + return self is other + +ClassVar = _ClassVar(_root=True) + + def cast(typ, val): """Cast a value to a type. @@ -1142,12 +1205,20 @@ def get_type_hints(obj, globalns=None, localns=None): - """Return type hints for a function or method object. + """Return type hints for an object. This is often the same as obj.__annotations__, but it handles forward references encoded as string literals, and if necessary adds Optional[t] if a default value equal to None is set. + The argument may be a module, class, method, or function. The annotations + are returned as a dictionary, or in the case of a class, a ChainMap of + dictionaries. + + TypeError is raised if the argument is not of a type that can contain + annotations, and an empty dictionary is returned if no annotations are + present. + BEWARE -- the behavior of globalns and localns is counterintuitive (unless you are familiar with how eval() and exec() work). The search order is locals first, then globals. @@ -1162,6 +1233,7 @@ - If two dict arguments are passed, they specify globals and locals, respectively. """ + if getattr(obj, '__no_type_check__', None): return {} if globalns is None: @@ -1170,16 +1242,62 @@ localns = globalns elif localns is None: localns = globalns - defaults = _get_defaults(obj) - hints = dict(obj.__annotations__) - for name, value in hints.items(): - if isinstance(value, str): - value = _ForwardRef(value) - value = _eval_type(value, globalns, localns) - if name in defaults and defaults[name] is None: - value = Optional[value] - hints[name] = value - return hints + + if (isinstance(obj, types.FunctionType) or + isinstance(obj, types.BuiltinFunctionType) or + isinstance(obj, types.MethodType)): + defaults = _get_defaults(obj) + hints = obj.__annotations__ + for name, value in hints.items(): + if value is None: + value = type(None) + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + if name in defaults and defaults[name] is None: + value = Optional[value] + hints[name] = value + return hints + + if isinstance(obj, types.ModuleType): + try: + hints = obj.__annotations__ + except AttributeError: + return {} + # we keep only those annotations that can be accessed on module + members = obj.__dict__ + hints = {name: value for name, value in hints.items() + if name in members} + for name, value in hints.items(): + if value is None: + value = type(None) + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + hints[name] = value + return hints + + if isinstance(object, type): + cmap = None + for base in reversed(obj.__mro__): + new_map = collections.ChainMap if cmap is None else cmap.new_child + try: + hints = base.__dict__['__annotations__'] + except KeyError: + cmap = new_map() + else: + for name, value in hints.items(): + if value is None: + value = type(None) + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + hints[name] = value + cmap = new_map(hints) + return cmap + + raise TypeError('{!r} is not a module, class, method, ' + 'or function.'.format(obj)) def no_type_check(arg): @@ -1300,6 +1418,8 @@ else: if (not attr.startswith('_abc_') and attr != '__abstractmethods__' and + attr != '__annotations__' and + attr != '__weakref__' and attr != '_is_protocol' and attr != '__dict__' and attr != '__args__' and diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -468,6 +468,7 @@ Robin Friedrich Bradley Froehle Ivan Frohne +Ivan Levkivskyi Matthias Fuchs Jim Fulton Tadayoshi Funaba diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -100,6 +100,9 @@ In a brand new thread, raise a RuntimeError since there is no active exception to reraise. Patch written by Xiang Zhang. +- Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. + Patch by Ivan Levkivskyi. + Library ------- diff --git a/Modules/symtablemodule.c b/Modules/symtablemodule.c --- a/Modules/symtablemodule.c +++ b/Modules/symtablemodule.c @@ -79,6 +79,7 @@ PyModule_AddIntMacro(m, DEF_FREE_CLASS); PyModule_AddIntMacro(m, DEF_IMPORT); PyModule_AddIntMacro(m, DEF_BOUND); + PyModule_AddIntMacro(m, DEF_ANNOT); PyModule_AddIntConstant(m, "TYPE_FUNCTION", FunctionBlock); PyModule_AddIntConstant(m, "TYPE_CLASS", ClassBlock); diff --git a/PC/launcher.c b/PC/launcher.c --- a/PC/launcher.c +++ b/PC/launcher.c @@ -1089,7 +1089,7 @@ { 3190, 3230, L"3.3" }, { 3250, 3310, L"3.4" }, { 3320, 3351, L"3.5" }, - { 3360, 3373, L"3.6" }, + { 3360, 3375, L"3.6" }, { 0 } }; diff --git a/Parser/Python.asdl b/Parser/Python.asdl --- a/Parser/Python.asdl +++ b/Parser/Python.asdl @@ -28,6 +28,8 @@ | Delete(expr* targets) | Assign(expr* targets, expr value) | AugAssign(expr target, operator op, expr value) + -- 'simple' indicates that we annotate simple name without parens + | AnnAssign(expr target, expr annotation, expr? value, int simple) -- use 'orelse' because else is a keyword in target languages | For(expr target, expr iter, stmt* body, stmt* orelse) diff --git a/Python/Python-ast.c b/Python/Python-ast.c --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -86,6 +86,15 @@ "op", "value", }; +static PyTypeObject *AnnAssign_type; +_Py_IDENTIFIER(annotation); +_Py_IDENTIFIER(simple); +static char *AnnAssign_fields[]={ + "target", + "annotation", + "value", + "simple", +}; static PyTypeObject *For_type; _Py_IDENTIFIER(iter); _Py_IDENTIFIER(orelse); @@ -466,7 +475,6 @@ "col_offset", }; _Py_IDENTIFIER(arg); -_Py_IDENTIFIER(annotation); static char *arg_fields[]={ "arg", "annotation", @@ -873,6 +881,8 @@ if (!Assign_type) return 0; AugAssign_type = make_type("AugAssign", stmt_type, AugAssign_fields, 3); if (!AugAssign_type) return 0; + AnnAssign_type = make_type("AnnAssign", stmt_type, AnnAssign_fields, 4); + if (!AnnAssign_type) return 0; For_type = make_type("For", stmt_type, For_fields, 4); if (!For_type) return 0; AsyncFor_type = make_type("AsyncFor", stmt_type, AsyncFor_fields, 4); @@ -1407,6 +1417,34 @@ } stmt_ty +AnnAssign(expr_ty target, expr_ty annotation, expr_ty value, int simple, int + lineno, int col_offset, PyArena *arena) +{ + stmt_ty p; + if (!target) { + PyErr_SetString(PyExc_ValueError, + "field target is required for AnnAssign"); + return NULL; + } + if (!annotation) { + PyErr_SetString(PyExc_ValueError, + "field annotation is required for AnnAssign"); + return NULL; + } + p = (stmt_ty)PyArena_Malloc(arena, sizeof(*p)); + if (!p) + return NULL; + p->kind = AnnAssign_kind; + p->v.AnnAssign.target = target; + p->v.AnnAssign.annotation = annotation; + p->v.AnnAssign.value = value; + p->v.AnnAssign.simple = simple; + p->lineno = lineno; + p->col_offset = col_offset; + return p; +} + +stmt_ty For(expr_ty target, expr_ty iter, asdl_seq * body, asdl_seq * orelse, int lineno, int col_offset, PyArena *arena) { @@ -2740,6 +2778,30 @@ goto failed; Py_DECREF(value); break; + case AnnAssign_kind: + result = PyType_GenericNew(AnnAssign_type, NULL, NULL); + if (!result) goto failed; + value = ast2obj_expr(o->v.AnnAssign.target); + if (!value) goto failed; + if (_PyObject_SetAttrId(result, &PyId_target, value) == -1) + goto failed; + Py_DECREF(value); + value = ast2obj_expr(o->v.AnnAssign.annotation); + if (!value) goto failed; + if (_PyObject_SetAttrId(result, &PyId_annotation, value) == -1) + goto failed; + Py_DECREF(value); + value = ast2obj_expr(o->v.AnnAssign.value); + if (!value) goto failed; + if (_PyObject_SetAttrId(result, &PyId_value, value) == -1) + goto failed; + Py_DECREF(value); + value = ast2obj_int(o->v.AnnAssign.simple); + if (!value) goto failed; + if (_PyObject_SetAttrId(result, &PyId_simple, value) == -1) + goto failed; + Py_DECREF(value); + break; case For_kind: result = PyType_GenericNew(For_type, NULL, NULL); if (!result) goto failed; @@ -4535,6 +4597,64 @@ if (*out == NULL) goto failed; return 0; } + isinstance = PyObject_IsInstance(obj, (PyObject*)AnnAssign_type); + if (isinstance == -1) { + return 1; + } + if (isinstance) { + expr_ty target; + expr_ty annotation; + expr_ty value; + int simple; + + if (_PyObject_HasAttrId(obj, &PyId_target)) { + int res; + tmp = _PyObject_GetAttrId(obj, &PyId_target); + if (tmp == NULL) goto failed; + res = obj2ast_expr(tmp, &target, arena); + if (res != 0) goto failed; + Py_CLEAR(tmp); + } else { + PyErr_SetString(PyExc_TypeError, "required field \"target\" missing from AnnAssign"); + return 1; + } + if (_PyObject_HasAttrId(obj, &PyId_annotation)) { + int res; + tmp = _PyObject_GetAttrId(obj, &PyId_annotation); + if (tmp == NULL) goto failed; + res = obj2ast_expr(tmp, &annotation, arena); + if (res != 0) goto failed; + Py_CLEAR(tmp); + } else { + PyErr_SetString(PyExc_TypeError, "required field \"annotation\" missing from AnnAssign"); + return 1; + } + if (exists_not_none(obj, &PyId_value)) { + int res; + tmp = _PyObject_GetAttrId(obj, &PyId_value); + if (tmp == NULL) goto failed; + res = obj2ast_expr(tmp, &value, arena); + if (res != 0) goto failed; + Py_CLEAR(tmp); + } else { + value = NULL; + } + if (_PyObject_HasAttrId(obj, &PyId_simple)) { + int res; + tmp = _PyObject_GetAttrId(obj, &PyId_simple); + if (tmp == NULL) goto failed; + res = obj2ast_int(tmp, &simple, arena); + if (res != 0) goto failed; + Py_CLEAR(tmp); + } else { + PyErr_SetString(PyExc_TypeError, "required field \"simple\" missing from AnnAssign"); + return 1; + } + *out = AnnAssign(target, annotation, value, simple, lineno, col_offset, + arena); + if (*out == NULL) goto failed; + return 0; + } isinstance = PyObject_IsInstance(obj, (PyObject*)For_type); if (isinstance == -1) { return 1; @@ -7517,6 +7637,8 @@ NULL; if (PyDict_SetItemString(d, "AugAssign", (PyObject*)AugAssign_type) < 0) return NULL; + if (PyDict_SetItemString(d, "AnnAssign", (PyObject*)AnnAssign_type) < 0) + return NULL; if (PyDict_SetItemString(d, "For", (PyObject*)For_type) < 0) return NULL; if (PyDict_SetItemString(d, "AsyncFor", (PyObject*)AsyncFor_type) < 0) return NULL; diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -397,6 +397,17 @@ case AugAssign_kind: return validate_expr(stmt->v.AugAssign.target, Store) && validate_expr(stmt->v.AugAssign.value, Load); + case AnnAssign_kind: + if (stmt->v.AnnAssign.target->kind != Name_kind && + stmt->v.AnnAssign.simple) { + PyErr_SetString(PyExc_TypeError, + "AnnAssign with simple non-Name target"); + return 0; + } + return validate_expr(stmt->v.AnnAssign.target, Store) && + (!stmt->v.AnnAssign.value || + validate_expr(stmt->v.AnnAssign.value, Load)) && + validate_expr(stmt->v.AnnAssign.annotation, Load); case For_kind: return validate_expr(stmt->v.For.target, Store) && validate_expr(stmt->v.For.iter, Load) && @@ -2847,8 +2858,9 @@ ast_for_expr_stmt(struct compiling *c, const node *n) { REQ(n, expr_stmt); - /* expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) - | ('=' (yield_expr|testlist))*) + /* expr_stmt: testlist_star_expr (annassign | augassign (yield_expr|testlist) | + ('=' (yield_expr|testlist_star_expr))*) + annassign: ':' test ['=' test] testlist_star_expr: (test|star_expr) (',' test|star_expr)* [','] augassign: '+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=' | '**=' | '//=' @@ -2900,6 +2912,76 @@ return AugAssign(expr1, newoperator, expr2, LINENO(n), n->n_col_offset, c->c_arena); } + else if (TYPE(CHILD(n, 1)) == annassign) { + expr_ty expr1, expr2, expr3; + node *ch = CHILD(n, 0); + node *deep, *ann = CHILD(n, 1); + int simple = 1; + + /* we keep track of parens to qualify (x) as expression not name */ + deep = ch; + while (NCH(deep) == 1) { + deep = CHILD(deep, 0); + } + if (NCH(deep) > 0 && TYPE(CHILD(deep, 0)) == LPAR) { + simple = 0; + } + expr1 = ast_for_testlist(c, ch); + if (!expr1) { + return NULL; + } + switch (expr1->kind) { + case Name_kind: + if (forbidden_name(c, expr1->v.Name.id, n, 0)) { + return NULL; + } + expr1->v.Name.ctx = Store; + break; + case Attribute_kind: + if (forbidden_name(c, expr1->v.Attribute.attr, n, 1)) { + return NULL; + } + expr1->v.Attribute.ctx = Store; + break; + case Subscript_kind: + expr1->v.Subscript.ctx = Store; + break; + case List_kind: + ast_error(c, ch, + "only single target (not list) can be annotated"); + return NULL; + case Tuple_kind: + ast_error(c, ch, + "only single target (not tuple) can be annotated"); + return NULL; + default: + ast_error(c, ch, + "illegal target for annotation"); + return NULL; + } + + if (expr1->kind != Name_kind) { + simple = 0; + } + ch = CHILD(ann, 1); + expr2 = ast_for_expr(c, ch); + if (!expr2) { + return NULL; + } + if (NCH(ann) == 2) { + return AnnAssign(expr1, expr2, NULL, simple, + LINENO(n), n->n_col_offset, c->c_arena); + } + else { + ch = CHILD(ann, 3); + expr3 = ast_for_expr(c, ch); + if (!expr3) { + return NULL; + } + return AnnAssign(expr1, expr2, expr3, simple, + LINENO(n), n->n_col_offset, c->c_arena); + } + } else { int i; asdl_seq *targets; diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1873,6 +1873,62 @@ DISPATCH(); } + TARGET(STORE_ANNOTATION) { + _Py_IDENTIFIER(__annotations__); + PyObject *ann_dict; + PyObject *ann = POP(); + PyObject *name = GETITEM(names, oparg); + int err; + if (f->f_locals == NULL) { + PyErr_Format(PyExc_SystemError, + "no locals found when storing annotation"); + Py_DECREF(ann); + goto error; + } + /* first try to get __annotations__ from locals... */ + if (PyDict_CheckExact(f->f_locals)) { + ann_dict = _PyDict_GetItemId(f->f_locals, + &PyId___annotations__); + if (ann_dict == NULL) { + PyErr_SetString(PyExc_NameError, + "__annotations__ not found"); + Py_DECREF(ann); + goto error; + } + Py_INCREF(ann_dict); + } + else { + PyObject *ann_str = _PyUnicode_FromId(&PyId___annotations__); + if (ann_str == NULL) { + Py_DECREF(ann); + goto error; + } + ann_dict = PyObject_GetItem(f->f_locals, ann_str); + if (ann_dict == NULL) { + if (PyErr_ExceptionMatches(PyExc_KeyError)) { + PyErr_SetString(PyExc_NameError, + "__annotations__ not found"); + } + Py_DECREF(ann); + goto error; + } + } + /* ...if succeeded, __annotations__[name] = ann */ + if (PyDict_CheckExact(ann_dict)) { + err = PyDict_SetItem(ann_dict, name, ann); + } + else { + err = PyObject_SetItem(ann_dict, name, ann); + } + Py_DECREF(ann_dict); + if (err != 0) { + Py_DECREF(ann); + goto error; + } + Py_DECREF(ann); + DISPATCH(); + } + TARGET(DELETE_SUBSCR) { PyObject *sub = TOP(); PyObject *container = SECOND(); @@ -2680,6 +2736,62 @@ DISPATCH(); } + TARGET(SETUP_ANNOTATIONS) { + _Py_IDENTIFIER(__annotations__); + int err; + PyObject *ann_dict; + if (f->f_locals == NULL) { + PyErr_Format(PyExc_SystemError, + "no locals found when setting up annotations"); + goto error; + } + /* check if __annotations__ in locals()... */ + if (PyDict_CheckExact(f->f_locals)) { + ann_dict = _PyDict_GetItemId(f->f_locals, + &PyId___annotations__); + if (ann_dict == NULL) { + /* ...if not, create a new one */ + ann_dict = PyDict_New(); + if (ann_dict == NULL) { + goto error; + } + err = _PyDict_SetItemId(f->f_locals, + &PyId___annotations__, ann_dict); + Py_DECREF(ann_dict); + if (err != 0) { + goto error; + } + } + } + else { + /* do the same if locals() is not a dict */ + PyObject *ann_str = _PyUnicode_FromId(&PyId___annotations__); + if (ann_str == NULL) { + break; + } + ann_dict = PyObject_GetItem(f->f_locals, ann_str); + if (ann_dict == NULL) { + if (!PyErr_ExceptionMatches(PyExc_KeyError)) { + goto error; + } + PyErr_Clear(); + ann_dict = PyDict_New(); + if (ann_dict == NULL) { + goto error; + } + err = PyObject_SetItem(f->f_locals, ann_str, ann_dict); + Py_DECREF(ann_dict); + if (err != 0) { + goto error; + } + } + else { + Py_DECREF(ann_dict); + } + } + DISPATCH(); + } + TARGET(BUILD_CONST_KEY_MAP) { Py_ssize_t i; PyObject *map; diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -179,6 +179,7 @@ static int compiler_visit_keyword(struct compiler *, keyword_ty); static int compiler_visit_expr(struct compiler *, expr_ty); static int compiler_augassign(struct compiler *, stmt_ty); +static int compiler_annassign(struct compiler *, stmt_ty); static int compiler_visit_slice(struct compiler *, slice_ty, expr_context_ty); @@ -933,6 +934,8 @@ return -1; case IMPORT_STAR: return -1; + case SETUP_ANNOTATIONS: + return 0; case YIELD_VALUE: return 0; case YIELD_FROM: @@ -1020,6 +1023,8 @@ return -1; case DELETE_FAST: return 0; + case STORE_ANNOTATION: + return -1; case RAISE_VARARGS: return -oparg; @@ -1358,7 +1363,65 @@ } } -/* Compile a sequence of statements, checking for a docstring. */ +/* Search if variable annotations are present statically in a block. */ + +static int +find_ann(asdl_seq *stmts) +{ + int i, j, res = 0; + stmt_ty st; + + for (i = 0; i < asdl_seq_LEN(stmts); i++) { + st = (stmt_ty)asdl_seq_GET(stmts, i); + switch (st->kind) { + case AnnAssign_kind: + return 1; + case For_kind: + res = find_ann(st->v.For.body) || + find_ann(st->v.For.orelse); + break; + case AsyncFor_kind: + res = find_ann(st->v.AsyncFor.body) || + find_ann(st->v.AsyncFor.orelse); + break; + case While_kind: + res = find_ann(st->v.While.body) || + find_ann(st->v.While.orelse); + break; + case If_kind: + res = find_ann(st->v.If.body) || + find_ann(st->v.If.orelse); + break; + case With_kind: + res = find_ann(st->v.With.body); + break; + case AsyncWith_kind: + res = find_ann(st->v.AsyncWith.body); + break; + case Try_kind: + for (j = 0; j < asdl_seq_LEN(st->v.Try.handlers); j++) { + excepthandler_ty handler = (excepthandler_ty)asdl_seq_GET( + st->v.Try.handlers, j); + if (find_ann(handler->v.ExceptHandler.body)) { + return 1; + } + } + res = find_ann(st->v.Try.body) || + find_ann(st->v.Try.finalbody) || + find_ann(st->v.Try.orelse); + break; + default: + res = 0; + } + if (res) { + break; + } + } + return res; +} + +/* Compile a sequence of statements, checking for a docstring + and for annotations. */ static int compiler_body(struct compiler *c, asdl_seq *stmts) @@ -1366,6 +1429,19 @@ int i = 0; stmt_ty st; + /* Set current line number to the line number of first statement. + This way line number for SETUP_ANNOTATIONS will always + coincide with the line number of first "real" statement in module. + If body is empy, then lineno will be set later in assemble. */ + if (c->u->u_scope_type == COMPILER_SCOPE_MODULE && + !c->u->u_lineno && asdl_seq_LEN(stmts)) { + st = (stmt_ty)asdl_seq_GET(stmts, 0); + c->u->u_lineno = st->lineno; + } + /* Every annotated class and module should have __annotations__. */ + if (find_ann(stmts)) { + ADDOP(c, SETUP_ANNOTATIONS); + } if (!asdl_seq_LEN(stmts)) return 1; st = (stmt_ty)asdl_seq_GET(stmts, 0); @@ -1403,6 +1479,9 @@ } break; case Interactive_kind: + if (find_ann(mod->v.Interactive.body)) { + ADDOP(c, SETUP_ANNOTATIONS); + } c->c_interactive = 1; VISIT_SEQ_IN_SCOPE(c, stmt, mod->v.Interactive.body); @@ -2743,6 +2822,8 @@ break; case AugAssign_kind: return compiler_augassign(c, s); + case AnnAssign_kind: + return compiler_annassign(c, s); case For_kind: return compiler_for(c, s); case While_kind: @@ -4223,6 +4304,138 @@ } static int +check_ann_expr(struct compiler *c, expr_ty e) +{ + VISIT(c, expr, e); + ADDOP(c, POP_TOP); + return 1; +} + +static int +check_annotation(struct compiler *c, stmt_ty s) +{ + /* Annotations are only evaluated in a module or class. */ + if (c->u->u_scope_type == COMPILER_SCOPE_MODULE || + c->u->u_scope_type == COMPILER_SCOPE_CLASS) { + return check_ann_expr(c, s->v.AnnAssign.annotation); + } + return 1; +} + +static int +check_ann_slice(struct compiler *c, slice_ty sl) +{ + switch(sl->kind) { + case Index_kind: + return check_ann_expr(c, sl->v.Index.value); + case Slice_kind: + if (sl->v.Slice.lower && !check_ann_expr(c, sl->v.Slice.lower)) { + return 0; + } + if (sl->v.Slice.upper && !check_ann_expr(c, sl->v.Slice.upper)) { + return 0; + } + if (sl->v.Slice.step && !check_ann_expr(c, sl->v.Slice.step)) { + return 0; + } + break; + default: + PyErr_SetString(PyExc_SystemError, + "unexpected slice kind"); + return 0; + } + return 1; +} + +static int +check_ann_subscr(struct compiler *c, slice_ty sl) +{ + /* We check that everything in a subscript is defined at runtime. */ + Py_ssize_t i, n; + + switch (sl->kind) { + case Index_kind: + case Slice_kind: + if (!check_ann_slice(c, sl)) { + return 0; + } + break; + case ExtSlice_kind: + n = asdl_seq_LEN(sl->v.ExtSlice.dims); + for (i = 0; i < n; i++) { + slice_ty subsl = (slice_ty)asdl_seq_GET(sl->v.ExtSlice.dims, i); + switch (subsl->kind) { + case Index_kind: + case Slice_kind: + if (!check_ann_slice(c, subsl)) { + return 0; + } + break; + case ExtSlice_kind: + default: + PyErr_SetString(PyExc_SystemError, + "extended slice invalid in nested slice"); + return 0; + } + } + break; + default: + PyErr_Format(PyExc_SystemError, + "invalid subscript kind %d", sl->kind); + return 0; + } + return 1; +} + +static int +compiler_annassign(struct compiler *c, stmt_ty s) +{ + expr_ty targ = s->v.AnnAssign.target; + + assert(s->kind == AnnAssign_kind); + + /* We perform the actual assignment first. */ + if (s->v.AnnAssign.value) { + VISIT(c, expr, s->v.AnnAssign.value); + VISIT(c, expr, targ); + } + switch (targ->kind) { + case Name_kind: + /* If we have a simple name in a module or class, store annotation. */ + if (s->v.AnnAssign.simple && + (c->u->u_scope_type == COMPILER_SCOPE_MODULE || + c->u->u_scope_type == COMPILER_SCOPE_CLASS)) { + VISIT(c, expr, s->v.AnnAssign.annotation); + ADDOP_O(c, STORE_ANNOTATION, targ->v.Name.id, names) + } + break; + case Attribute_kind: + if (!s->v.AnnAssign.value && + !check_ann_expr(c, targ->v.Attribute.value)) { + return 0; + } + break; + case Subscript_kind: + if (!s->v.AnnAssign.value && + (!check_ann_expr(c, targ->v.Subscript.value) || + !check_ann_subscr(c, targ->v.Subscript.slice))) { + return 0; + } + break; + default: + PyErr_Format(PyExc_SystemError, + "invalid node type (%d) for annotated assignment", + targ->kind); + return 0; + } + /* Annotation is evaluated last. */ + if (!s->v.AnnAssign.simple && !check_annotation(c, s)) { + return 0; + } + return 1; +} + +static int compiler_push_fblock(struct compiler *c, enum fblocktype t, basicblock *b) { struct fblockinfo *f; diff --git a/Python/graminit.c b/Python/graminit.c --- a/Python/graminit.c +++ b/Python/graminit.c @@ -459,54 +459,77 @@ static arc arcs_16_0[1] = { {47, 1}, }; -static arc arcs_16_1[3] = { +static arc arcs_16_1[4] = { {48, 2}, - {31, 3}, + {49, 3}, + {31, 4}, {0, 1}, }; -static arc arcs_16_2[2] = { - {49, 4}, - {9, 4}, +static arc arcs_16_2[1] = { + {0, 2}, }; static arc arcs_16_3[2] = { - {49, 5}, + {50, 2}, + {9, 2}, +}; +static arc arcs_16_4[2] = { + {50, 5}, {47, 5}, }; -static arc arcs_16_4[1] = { - {0, 4}, -}; static arc arcs_16_5[2] = { - {31, 3}, + {31, 4}, {0, 5}, }; static state states_16[6] = { {1, arcs_16_0}, - {3, arcs_16_1}, - {2, arcs_16_2}, + {4, arcs_16_1}, + {1, arcs_16_2}, {2, arcs_16_3}, - {1, arcs_16_4}, + {2, arcs_16_4}, {2, arcs_16_5}, }; -static arc arcs_17_0[2] = { +static arc arcs_17_0[1] = { + {27, 1}, +}; +static arc arcs_17_1[1] = { + {26, 2}, +}; +static arc arcs_17_2[2] = { + {31, 3}, + {0, 2}, +}; +static arc arcs_17_3[1] = { + {26, 4}, +}; +static arc arcs_17_4[1] = { + {0, 4}, +}; +static state states_17[5] = { + {1, arcs_17_0}, + {1, arcs_17_1}, + {2, arcs_17_2}, + {1, arcs_17_3}, + {1, arcs_17_4}, +}; +static arc arcs_18_0[2] = { {26, 1}, - {50, 1}, -}; -static arc arcs_17_1[2] = { + {51, 1}, +}; +static arc arcs_18_1[2] = { {32, 2}, {0, 1}, }; -static arc arcs_17_2[3] = { +static arc arcs_18_2[3] = { {26, 1}, - {50, 1}, + {51, 1}, {0, 2}, }; -static state states_17[3] = { - {2, arcs_17_0}, - {2, arcs_17_1}, - {3, arcs_17_2}, -}; -static arc arcs_18_0[13] = { - {51, 1}, +static state states_18[3] = { + {2, arcs_18_0}, + {2, arcs_18_1}, + {3, arcs_18_2}, +}; +static arc arcs_19_0[13] = { {52, 1}, {53, 1}, {54, 1}, @@ -519,60 +542,51 @@ {61, 1}, {62, 1}, {63, 1}, -}; -static arc arcs_18_1[1] = { + {64, 1}, +}; +static arc arcs_19_1[1] = { {0, 1}, }; -static state states_18[2] = { - {13, arcs_18_0}, - {1, arcs_18_1}, -}; -static arc arcs_19_0[1] = { - {64, 1}, -}; -static arc arcs_19_1[1] = { - {65, 2}, -}; -static arc arcs_19_2[1] = { +static state states_19[2] = { + {13, arcs_19_0}, + {1, arcs_19_1}, +}; +static arc arcs_20_0[1] = { + {65, 1}, +}; +static arc arcs_20_1[1] = { + {66, 2}, +}; +static arc arcs_20_2[1] = { {0, 2}, }; -static state states_19[3] = { - {1, arcs_19_0}, - {1, arcs_19_1}, - {1, arcs_19_2}, -}; -static arc arcs_20_0[1] = { - {66, 1}, -}; -static arc arcs_20_1[1] = { - {0, 1}, -}; -static state states_20[2] = { +static state states_20[3] = { {1, arcs_20_0}, {1, arcs_20_1}, -}; -static arc arcs_21_0[5] = { + {1, arcs_20_2}, +}; +static arc arcs_21_0[1] = { {67, 1}, +}; +static arc arcs_21_1[1] = { + {0, 1}, +}; +static state states_21[2] = { + {1, arcs_21_0}, + {1, arcs_21_1}, +}; +static arc arcs_22_0[5] = { {68, 1}, {69, 1}, {70, 1}, {71, 1}, -}; -static arc arcs_21_1[1] = { - {0, 1}, -}; -static state states_21[2] = { - {5, arcs_21_0}, - {1, arcs_21_1}, -}; -static arc arcs_22_0[1] = { {72, 1}, }; static arc arcs_22_1[1] = { {0, 1}, }; static state states_22[2] = { - {1, arcs_22_0}, + {5, arcs_22_0}, {1, arcs_22_1}, }; static arc arcs_23_0[1] = { @@ -588,142 +602,133 @@ static arc arcs_24_0[1] = { {74, 1}, }; -static arc arcs_24_1[2] = { +static arc arcs_24_1[1] = { + {0, 1}, +}; +static state states_24[2] = { + {1, arcs_24_0}, + {1, arcs_24_1}, +}; +static arc arcs_25_0[1] = { + {75, 1}, +}; +static arc arcs_25_1[2] = { {9, 2}, {0, 1}, }; -static arc arcs_24_2[1] = { +static arc arcs_25_2[1] = { {0, 2}, }; -static state states_24[3] = { - {1, arcs_24_0}, - {2, arcs_24_1}, - {1, arcs_24_2}, -}; -static arc arcs_25_0[1] = { - {49, 1}, -}; -static arc arcs_25_1[1] = { +static state states_25[3] = { + {1, arcs_25_0}, + {2, arcs_25_1}, + {1, arcs_25_2}, +}; +static arc arcs_26_0[1] = { + {50, 1}, +}; +static arc arcs_26_1[1] = { {0, 1}, }; -static state states_25[2] = { - {1, arcs_25_0}, - {1, arcs_25_1}, -}; -static arc arcs_26_0[1] = { - {75, 1}, -}; -static arc arcs_26_1[2] = { +static state states_26[2] = { + {1, arcs_26_0}, + {1, arcs_26_1}, +}; +static arc arcs_27_0[1] = { + {76, 1}, +}; +static arc arcs_27_1[2] = { {26, 2}, {0, 1}, }; -static arc arcs_26_2[2] = { - {76, 3}, +static arc arcs_27_2[2] = { + {77, 3}, {0, 2}, }; -static arc arcs_26_3[1] = { +static arc arcs_27_3[1] = { {26, 4}, }; -static arc arcs_26_4[1] = { +static arc arcs_27_4[1] = { {0, 4}, }; -static state states_26[5] = { - {1, arcs_26_0}, - {2, arcs_26_1}, - {2, arcs_26_2}, - {1, arcs_26_3}, - {1, arcs_26_4}, -}; -static arc arcs_27_0[2] = { +static state states_27[5] = { + {1, arcs_27_0}, + {2, arcs_27_1}, + {2, arcs_27_2}, + {1, arcs_27_3}, + {1, arcs_27_4}, +}; +static arc arcs_28_0[2] = { + {78, 1}, + {79, 1}, +}; +static arc arcs_28_1[1] = { + {0, 1}, +}; +static state states_28[2] = { + {2, arcs_28_0}, + {1, arcs_28_1}, +}; +static arc arcs_29_0[1] = { + {80, 1}, +}; +static arc arcs_29_1[1] = { + {81, 2}, +}; +static arc arcs_29_2[1] = { + {0, 2}, +}; +static state states_29[3] = { + {1, arcs_29_0}, + {1, arcs_29_1}, + {1, arcs_29_2}, +}; +static arc arcs_30_0[1] = { {77, 1}, - {78, 1}, -}; -static arc arcs_27_1[1] = { - {0, 1}, -}; -static state states_27[2] = { - {2, arcs_27_0}, - {1, arcs_27_1}, -}; -static arc arcs_28_0[1] = { - {79, 1}, -}; -static arc arcs_28_1[1] = { - {80, 2}, -}; -static arc arcs_28_2[1] = { - {0, 2}, -}; -static state states_28[3] = { - {1, arcs_28_0}, - {1, arcs_28_1}, - {1, arcs_28_2}, -}; -static arc arcs_29_0[1] = { - {76, 1}, -}; -static arc arcs_29_1[3] = { - {81, 2}, +}; +static arc arcs_30_1[3] = { {82, 2}, + {83, 2}, {12, 3}, }; -static arc arcs_29_2[4] = { - {81, 2}, +static arc arcs_30_2[4] = { {82, 2}, + {83, 2}, {12, 3}, - {79, 4}, -}; -static arc arcs_29_3[1] = { - {79, 4}, -}; -static arc arcs_29_4[3] = { + {80, 4}, +}; +static arc arcs_30_3[1] = { + {80, 4}, +}; +static arc arcs_30_4[3] = { {33, 5}, {13, 6}, - {83, 5}, -}; -static arc arcs_29_5[1] = { + {84, 5}, +}; +static arc arcs_30_5[1] = { {0, 5}, }; -static arc arcs_29_6[1] = { - {83, 7}, -}; -static arc arcs_29_7[1] = { +static arc arcs_30_6[1] = { + {84, 7}, +}; +static arc arcs_30_7[1] = { {15, 5}, }; -static state states_29[8] = { - {1, arcs_29_0}, - {3, arcs_29_1}, - {4, arcs_29_2}, - {1, arcs_29_3}, - {3, arcs_29_4}, - {1, arcs_29_5}, - {1, arcs_29_6}, - {1, arcs_29_7}, -}; -static arc arcs_30_0[1] = { +static state states_30[8] = { + {1, arcs_30_0}, + {3, arcs_30_1}, + {4, arcs_30_2}, + {1, arcs_30_3}, + {3, arcs_30_4}, + {1, arcs_30_5}, + {1, arcs_30_6}, + {1, arcs_30_7}, +}; +static arc arcs_31_0[1] = { {23, 1}, }; -static arc arcs_30_1[2] = { - {85, 2}, - {0, 1}, -}; -static arc arcs_30_2[1] = { - {23, 3}, -}; -static arc arcs_30_3[1] = { - {0, 3}, -}; -static state states_30[4] = { - {1, arcs_30_0}, - {2, arcs_30_1}, - {1, arcs_30_2}, - {1, arcs_30_3}, -}; -static arc arcs_31_0[1] = { - {12, 1}, -}; static arc arcs_31_1[2] = { - {85, 2}, + {86, 2}, {0, 1}, }; static arc arcs_31_2[1] = { @@ -739,37 +744,45 @@ {1, arcs_31_3}, }; static arc arcs_32_0[1] = { - {84, 1}, + {12, 1}, }; static arc arcs_32_1[2] = { + {86, 2}, + {0, 1}, +}; +static arc arcs_32_2[1] = { + {23, 3}, +}; +static arc arcs_32_3[1] = { + {0, 3}, +}; +static state states_32[4] = { + {1, arcs_32_0}, + {2, arcs_32_1}, + {1, arcs_32_2}, + {1, arcs_32_3}, +}; +static arc arcs_33_0[1] = { + {85, 1}, +}; +static arc arcs_33_1[2] = { {32, 2}, {0, 1}, }; -static arc arcs_32_2[2] = { - {84, 1}, +static arc arcs_33_2[2] = { + {85, 1}, {0, 2}, }; -static state states_32[3] = { - {1, arcs_32_0}, - {2, arcs_32_1}, - {2, arcs_32_2}, -}; -static arc arcs_33_0[1] = { - {86, 1}, -}; -static arc arcs_33_1[2] = { - {32, 0}, - {0, 1}, -}; -static state states_33[2] = { +static state states_33[3] = { {1, arcs_33_0}, {2, arcs_33_1}, + {2, arcs_33_2}, }; static arc arcs_34_0[1] = { - {23, 1}, + {87, 1}, }; static arc arcs_34_1[2] = { - {81, 0}, + {32, 0}, {0, 1}, }; static state states_34[2] = { @@ -777,19 +790,15 @@ {2, arcs_34_1}, }; static arc arcs_35_0[1] = { - {87, 1}, -}; -static arc arcs_35_1[1] = { - {23, 2}, -}; -static arc arcs_35_2[2] = { - {32, 1}, - {0, 2}, -}; -static state states_35[3] = { + {23, 1}, +}; +static arc arcs_35_1[2] = { + {82, 0}, + {0, 1}, +}; +static state states_35[2] = { {1, arcs_35_0}, - {1, arcs_35_1}, - {2, arcs_35_2}, + {2, arcs_35_1}, }; static arc arcs_36_0[1] = { {88, 1}, @@ -810,97 +819,76 @@ {89, 1}, }; static arc arcs_37_1[1] = { - {26, 2}, + {23, 2}, }; static arc arcs_37_2[2] = { - {32, 3}, + {32, 1}, {0, 2}, }; -static arc arcs_37_3[1] = { - {26, 4}, -}; -static arc arcs_37_4[1] = { - {0, 4}, -}; -static state states_37[5] = { +static state states_37[3] = { {1, arcs_37_0}, {1, arcs_37_1}, {2, arcs_37_2}, - {1, arcs_37_3}, - {1, arcs_37_4}, -}; -static arc arcs_38_0[9] = { +}; +static arc arcs_38_0[1] = { {90, 1}, +}; +static arc arcs_38_1[1] = { + {26, 2}, +}; +static arc arcs_38_2[2] = { + {32, 3}, + {0, 2}, +}; +static arc arcs_38_3[1] = { + {26, 4}, +}; +static arc arcs_38_4[1] = { + {0, 4}, +}; +static state states_38[5] = { + {1, arcs_38_0}, + {1, arcs_38_1}, + {2, arcs_38_2}, + {1, arcs_38_3}, + {1, arcs_38_4}, +}; +static arc arcs_39_0[9] = { {91, 1}, {92, 1}, {93, 1}, {94, 1}, + {95, 1}, {19, 1}, {18, 1}, {17, 1}, - {95, 1}, -}; -static arc arcs_38_1[1] = { + {96, 1}, +}; +static arc arcs_39_1[1] = { {0, 1}, }; -static state states_38[2] = { - {9, arcs_38_0}, - {1, arcs_38_1}, -}; -static arc arcs_39_0[1] = { +static state states_39[2] = { + {9, arcs_39_0}, + {1, arcs_39_1}, +}; +static arc arcs_40_0[1] = { {21, 1}, }; -static arc arcs_39_1[3] = { +static arc arcs_40_1[3] = { {19, 2}, - {94, 2}, - {92, 2}, -}; -static arc arcs_39_2[1] = { + {95, 2}, + {93, 2}, +}; +static arc arcs_40_2[1] = { {0, 2}, }; -static state states_39[3] = { - {1, arcs_39_0}, - {3, arcs_39_1}, - {1, arcs_39_2}, -}; -static arc arcs_40_0[1] = { - {96, 1}, -}; -static arc arcs_40_1[1] = { - {26, 2}, -}; -static arc arcs_40_2[1] = { - {27, 3}, -}; -static arc arcs_40_3[1] = { - {28, 4}, -}; -static arc arcs_40_4[3] = { +static state states_40[3] = { + {1, arcs_40_0}, + {3, arcs_40_1}, + {1, arcs_40_2}, +}; +static arc arcs_41_0[1] = { {97, 1}, - {98, 5}, - {0, 4}, -}; -static arc arcs_40_5[1] = { - {27, 6}, -}; -static arc arcs_40_6[1] = { - {28, 7}, -}; -static arc arcs_40_7[1] = { - {0, 7}, -}; -static state states_40[8] = { - {1, arcs_40_0}, - {1, arcs_40_1}, - {1, arcs_40_2}, - {1, arcs_40_3}, - {3, arcs_40_4}, - {1, arcs_40_5}, - {1, arcs_40_6}, - {1, arcs_40_7}, -}; -static arc arcs_41_0[1] = { - {99, 1}, }; static arc arcs_41_1[1] = { {26, 2}, @@ -911,8 +899,9 @@ static arc arcs_41_3[1] = { {28, 4}, }; -static arc arcs_41_4[2] = { - {98, 5}, +static arc arcs_41_4[3] = { + {98, 1}, + {99, 5}, {0, 4}, }; static arc arcs_41_5[1] = { @@ -929,7 +918,7 @@ {1, arcs_41_1}, {1, arcs_41_2}, {1, arcs_41_3}, - {2, arcs_41_4}, + {3, arcs_41_4}, {1, arcs_41_5}, {1, arcs_41_6}, {1, arcs_41_7}, @@ -938,258 +927,270 @@ {100, 1}, }; static arc arcs_42_1[1] = { - {65, 2}, + {26, 2}, }; static arc arcs_42_2[1] = { - {101, 3}, + {27, 3}, }; static arc arcs_42_3[1] = { - {9, 4}, -}; -static arc arcs_42_4[1] = { - {27, 5}, + {28, 4}, +}; +static arc arcs_42_4[2] = { + {99, 5}, + {0, 4}, }; static arc arcs_42_5[1] = { - {28, 6}, -}; -static arc arcs_42_6[2] = { - {98, 7}, - {0, 6}, + {27, 6}, +}; +static arc arcs_42_6[1] = { + {28, 7}, }; static arc arcs_42_7[1] = { - {27, 8}, -}; -static arc arcs_42_8[1] = { - {28, 9}, -}; -static arc arcs_42_9[1] = { - {0, 9}, -}; -static state states_42[10] = { + {0, 7}, +}; +static state states_42[8] = { {1, arcs_42_0}, {1, arcs_42_1}, {1, arcs_42_2}, {1, arcs_42_3}, - {1, arcs_42_4}, + {2, arcs_42_4}, {1, arcs_42_5}, - {2, arcs_42_6}, + {1, arcs_42_6}, {1, arcs_42_7}, - {1, arcs_42_8}, - {1, arcs_42_9}, }; static arc arcs_43_0[1] = { - {102, 1}, + {101, 1}, }; static arc arcs_43_1[1] = { - {27, 2}, + {66, 2}, }; static arc arcs_43_2[1] = { - {28, 3}, -}; -static arc arcs_43_3[2] = { - {103, 4}, - {104, 5}, + {102, 3}, +}; +static arc arcs_43_3[1] = { + {9, 4}, }; static arc arcs_43_4[1] = { - {27, 6}, + {27, 5}, }; static arc arcs_43_5[1] = { - {27, 7}, -}; -static arc arcs_43_6[1] = { - {28, 8}, + {28, 6}, +}; +static arc arcs_43_6[2] = { + {99, 7}, + {0, 6}, }; static arc arcs_43_7[1] = { + {27, 8}, +}; +static arc arcs_43_8[1] = { {28, 9}, }; -static arc arcs_43_8[4] = { - {103, 4}, - {98, 10}, - {104, 5}, - {0, 8}, -}; static arc arcs_43_9[1] = { {0, 9}, }; -static arc arcs_43_10[1] = { - {27, 11}, -}; -static arc arcs_43_11[1] = { - {28, 12}, -}; -static arc arcs_43_12[2] = { - {104, 5}, - {0, 12}, -}; -static state states_43[13] = { +static state states_43[10] = { {1, arcs_43_0}, {1, arcs_43_1}, {1, arcs_43_2}, - {2, arcs_43_3}, + {1, arcs_43_3}, {1, arcs_43_4}, {1, arcs_43_5}, - {1, arcs_43_6}, + {2, arcs_43_6}, {1, arcs_43_7}, - {4, arcs_43_8}, + {1, arcs_43_8}, {1, arcs_43_9}, - {1, arcs_43_10}, - {1, arcs_43_11}, - {2, arcs_43_12}, }; static arc arcs_44_0[1] = { - {105, 1}, + {103, 1}, }; static arc arcs_44_1[1] = { - {106, 2}, -}; -static arc arcs_44_2[2] = { + {27, 2}, +}; +static arc arcs_44_2[1] = { + {28, 3}, +}; +static arc arcs_44_3[2] = { + {104, 4}, + {105, 5}, +}; +static arc arcs_44_4[1] = { + {27, 6}, +}; +static arc arcs_44_5[1] = { + {27, 7}, +}; +static arc arcs_44_6[1] = { + {28, 8}, +}; +static arc arcs_44_7[1] = { + {28, 9}, +}; +static arc arcs_44_8[4] = { + {104, 4}, + {99, 10}, + {105, 5}, + {0, 8}, +}; +static arc arcs_44_9[1] = { + {0, 9}, +}; +static arc arcs_44_10[1] = { + {27, 11}, +}; +static arc arcs_44_11[1] = { + {28, 12}, +}; +static arc arcs_44_12[2] = { + {105, 5}, + {0, 12}, +}; +static state states_44[13] = { + {1, arcs_44_0}, + {1, arcs_44_1}, + {1, arcs_44_2}, + {2, arcs_44_3}, + {1, arcs_44_4}, + {1, arcs_44_5}, + {1, arcs_44_6}, + {1, arcs_44_7}, + {4, arcs_44_8}, + {1, arcs_44_9}, + {1, arcs_44_10}, + {1, arcs_44_11}, + {2, arcs_44_12}, +}; +static arc arcs_45_0[1] = { + {106, 1}, +}; +static arc arcs_45_1[1] = { + {107, 2}, +}; +static arc arcs_45_2[2] = { {32, 1}, {27, 3}, }; -static arc arcs_44_3[1] = { +static arc arcs_45_3[1] = { {28, 4}, }; -static arc arcs_44_4[1] = { +static arc arcs_45_4[1] = { {0, 4}, }; -static state states_44[5] = { - {1, arcs_44_0}, - {1, arcs_44_1}, - {2, arcs_44_2}, - {1, arcs_44_3}, - {1, arcs_44_4}, -}; -static arc arcs_45_0[1] = { +static state states_45[5] = { + {1, arcs_45_0}, + {1, arcs_45_1}, + {2, arcs_45_2}, + {1, arcs_45_3}, + {1, arcs_45_4}, +}; +static arc arcs_46_0[1] = { {26, 1}, }; -static arc arcs_45_1[2] = { - {85, 2}, +static arc arcs_46_1[2] = { + {86, 2}, {0, 1}, }; -static arc arcs_45_2[1] = { - {107, 3}, -}; -static arc arcs_45_3[1] = { +static arc arcs_46_2[1] = { + {108, 3}, +}; +static arc arcs_46_3[1] = { {0, 3}, }; -static state states_45[4] = { - {1, arcs_45_0}, - {2, arcs_45_1}, - {1, arcs_45_2}, - {1, arcs_45_3}, -}; -static arc arcs_46_0[1] = { - {108, 1}, -}; -static arc arcs_46_1[2] = { +static state states_46[4] = { + {1, arcs_46_0}, + {2, arcs_46_1}, + {1, arcs_46_2}, + {1, arcs_46_3}, +}; +static arc arcs_47_0[1] = { + {109, 1}, +}; +static arc arcs_47_1[2] = { {26, 2}, {0, 1}, }; -static arc arcs_46_2[2] = { - {85, 3}, +static arc arcs_47_2[2] = { + {86, 3}, {0, 2}, }; -static arc arcs_46_3[1] = { +static arc arcs_47_3[1] = { {23, 4}, }; -static arc arcs_46_4[1] = { +static arc arcs_47_4[1] = { {0, 4}, }; -static state states_46[5] = { - {1, arcs_46_0}, - {2, arcs_46_1}, - {2, arcs_46_2}, - {1, arcs_46_3}, - {1, arcs_46_4}, -}; -static arc arcs_47_0[2] = { +static state states_47[5] = { + {1, arcs_47_0}, + {2, arcs_47_1}, + {2, arcs_47_2}, + {1, arcs_47_3}, + {1, arcs_47_4}, +}; +static arc arcs_48_0[2] = { {3, 1}, {2, 2}, }; -static arc arcs_47_1[1] = { +static arc arcs_48_1[1] = { {0, 1}, }; -static arc arcs_47_2[1] = { - {109, 3}, -}; -static arc arcs_47_3[1] = { +static arc arcs_48_2[1] = { + {110, 3}, +}; +static arc arcs_48_3[1] = { {6, 4}, }; -static arc arcs_47_4[2] = { +static arc arcs_48_4[2] = { {6, 4}, - {110, 1}, -}; -static state states_47[5] = { - {2, arcs_47_0}, - {1, arcs_47_1}, - {1, arcs_47_2}, - {1, arcs_47_3}, - {2, arcs_47_4}, -}; -static arc arcs_48_0[2] = { {111, 1}, - {112, 2}, -}; -static arc arcs_48_1[2] = { - {96, 3}, - {0, 1}, -}; -static arc arcs_48_2[1] = { - {0, 2}, -}; -static arc arcs_48_3[1] = { - {111, 4}, -}; -static arc arcs_48_4[1] = { - {98, 5}, -}; -static arc arcs_48_5[1] = { - {26, 2}, -}; -static state states_48[6] = { +}; +static state states_48[5] = { {2, arcs_48_0}, - {2, arcs_48_1}, + {1, arcs_48_1}, {1, arcs_48_2}, {1, arcs_48_3}, - {1, arcs_48_4}, - {1, arcs_48_5}, + {2, arcs_48_4}, }; static arc arcs_49_0[2] = { - {111, 1}, - {114, 1}, -}; -static arc arcs_49_1[1] = { + {112, 1}, + {113, 2}, +}; +static arc arcs_49_1[2] = { + {97, 3}, {0, 1}, }; -static state states_49[2] = { +static arc arcs_49_2[1] = { + {0, 2}, +}; +static arc arcs_49_3[1] = { + {112, 4}, +}; +static arc arcs_49_4[1] = { + {99, 5}, +}; +static arc arcs_49_5[1] = { + {26, 2}, +}; +static state states_49[6] = { {2, arcs_49_0}, - {1, arcs_49_1}, -}; -static arc arcs_50_0[1] = { + {2, arcs_49_1}, + {1, arcs_49_2}, + {1, arcs_49_3}, + {1, arcs_49_4}, + {1, arcs_49_5}, +}; +static arc arcs_50_0[2] = { + {112, 1}, {115, 1}, }; -static arc arcs_50_1[2] = { - {35, 2}, - {27, 3}, -}; -static arc arcs_50_2[1] = { - {27, 3}, -}; -static arc arcs_50_3[1] = { - {26, 4}, -}; -static arc arcs_50_4[1] = { - {0, 4}, -}; -static state states_50[5] = { - {1, arcs_50_0}, - {2, arcs_50_1}, - {1, arcs_50_2}, - {1, arcs_50_3}, - {1, arcs_50_4}, +static arc arcs_50_1[1] = { + {0, 1}, +}; +static state states_50[2] = { + {2, arcs_50_0}, + {1, arcs_50_1}, }; static arc arcs_51_0[1] = { - {115, 1}, + {116, 1}, }; static arc arcs_51_1[2] = { {35, 2}, @@ -1199,7 +1200,7 @@ {27, 3}, }; static arc arcs_51_3[1] = { - {113, 4}, + {26, 4}, }; static arc arcs_51_4[1] = { {0, 4}, @@ -1215,108 +1216,120 @@ {116, 1}, }; static arc arcs_52_1[2] = { - {117, 0}, - {0, 1}, -}; -static state states_52[2] = { + {35, 2}, + {27, 3}, +}; +static arc arcs_52_2[1] = { + {27, 3}, +}; +static arc arcs_52_3[1] = { + {114, 4}, +}; +static arc arcs_52_4[1] = { + {0, 4}, +}; +static state states_52[5] = { {1, arcs_52_0}, {2, arcs_52_1}, + {1, arcs_52_2}, + {1, arcs_52_3}, + {1, arcs_52_4}, }; static arc arcs_53_0[1] = { - {118, 1}, + {117, 1}, }; static arc arcs_53_1[2] = { - {119, 0}, + {118, 0}, {0, 1}, }; static state states_53[2] = { {1, arcs_53_0}, {2, arcs_53_1}, }; -static arc arcs_54_0[2] = { - {120, 1}, - {121, 2}, -}; -static arc arcs_54_1[1] = { - {118, 2}, -}; -static arc arcs_54_2[1] = { +static arc arcs_54_0[1] = { + {119, 1}, +}; +static arc arcs_54_1[2] = { + {120, 0}, + {0, 1}, +}; +static state states_54[2] = { + {1, arcs_54_0}, + {2, arcs_54_1}, +}; +static arc arcs_55_0[2] = { + {121, 1}, + {122, 2}, +}; +static arc arcs_55_1[1] = { + {119, 2}, +}; +static arc arcs_55_2[1] = { {0, 2}, }; -static state states_54[3] = { - {2, arcs_54_0}, - {1, arcs_54_1}, - {1, arcs_54_2}, -}; -static arc arcs_55_0[1] = { - {107, 1}, -}; -static arc arcs_55_1[2] = { - {122, 0}, +static state states_55[3] = { + {2, arcs_55_0}, + {1, arcs_55_1}, + {1, arcs_55_2}, +}; +static arc arcs_56_0[1] = { + {108, 1}, +}; +static arc arcs_56_1[2] = { + {123, 0}, {0, 1}, }; -static state states_55[2] = { - {1, arcs_55_0}, - {2, arcs_55_1}, -}; -static arc arcs_56_0[10] = { - {123, 1}, +static state states_56[2] = { + {1, arcs_56_0}, + {2, arcs_56_1}, +}; +static arc arcs_57_0[10] = { {124, 1}, {125, 1}, {126, 1}, {127, 1}, {128, 1}, {129, 1}, - {101, 1}, - {120, 2}, - {130, 3}, -}; -static arc arcs_56_1[1] = { + {130, 1}, + {102, 1}, + {121, 2}, + {131, 3}, +}; +static arc arcs_57_1[1] = { {0, 1}, }; -static arc arcs_56_2[1] = { - {101, 1}, -}; -static arc arcs_56_3[2] = { - {120, 1}, +static arc arcs_57_2[1] = { + {102, 1}, +}; +static arc arcs_57_3[2] = { + {121, 1}, {0, 3}, }; -static state states_56[4] = { - {10, arcs_56_0}, - {1, arcs_56_1}, - {1, arcs_56_2}, - {2, arcs_56_3}, -}; -static arc arcs_57_0[1] = { - {33, 1}, -}; -static arc arcs_57_1[1] = { - {107, 2}, -}; -static arc arcs_57_2[1] = { - {0, 2}, -}; -static state states_57[3] = { - {1, arcs_57_0}, +static state states_57[4] = { + {10, arcs_57_0}, {1, arcs_57_1}, {1, arcs_57_2}, + {2, arcs_57_3}, }; static arc arcs_58_0[1] = { - {131, 1}, -}; -static arc arcs_58_1[2] = { - {132, 0}, - {0, 1}, -}; -static state states_58[2] = { + {33, 1}, +}; +static arc arcs_58_1[1] = { + {108, 2}, +}; +static arc arcs_58_2[1] = { + {0, 2}, +}; +static state states_58[3] = { {1, arcs_58_0}, - {2, arcs_58_1}, + {1, arcs_58_1}, + {1, arcs_58_2}, }; static arc arcs_59_0[1] = { - {133, 1}, + {132, 1}, }; static arc arcs_59_1[2] = { - {134, 0}, + {133, 0}, {0, 1}, }; static state states_59[2] = { @@ -1324,10 +1337,10 @@ {2, arcs_59_1}, }; static arc arcs_60_0[1] = { - {135, 1}, + {134, 1}, }; static arc arcs_60_1[2] = { - {136, 0}, + {135, 0}, {0, 1}, }; static state states_60[2] = { @@ -1335,23 +1348,22 @@ {2, arcs_60_1}, }; static arc arcs_61_0[1] = { - {137, 1}, -}; -static arc arcs_61_1[3] = { - {138, 0}, - {139, 0}, + {136, 1}, +}; +static arc arcs_61_1[2] = { + {137, 0}, {0, 1}, }; static state states_61[2] = { {1, arcs_61_0}, - {3, arcs_61_1}, + {2, arcs_61_1}, }; static arc arcs_62_0[1] = { - {140, 1}, + {138, 1}, }; static arc arcs_62_1[3] = { - {141, 0}, - {142, 0}, + {139, 0}, + {140, 0}, {0, 1}, }; static state states_62[2] = { @@ -1359,528 +1371,540 @@ {3, arcs_62_1}, }; static arc arcs_63_0[1] = { - {143, 1}, -}; -static arc arcs_63_1[6] = { + {141, 1}, +}; +static arc arcs_63_1[3] = { + {142, 0}, + {143, 0}, + {0, 1}, +}; +static state states_63[2] = { + {1, arcs_63_0}, + {3, arcs_63_1}, +}; +static arc arcs_64_0[1] = { + {144, 1}, +}; +static arc arcs_64_1[6] = { {33, 0}, {11, 0}, - {144, 0}, {145, 0}, {146, 0}, + {147, 0}, {0, 1}, }; -static state states_63[2] = { - {1, arcs_63_0}, - {6, arcs_63_1}, -}; -static arc arcs_64_0[4] = { - {141, 1}, +static state states_64[2] = { + {1, arcs_64_0}, + {6, arcs_64_1}, +}; +static arc arcs_65_0[4] = { {142, 1}, - {147, 1}, - {148, 2}, -}; -static arc arcs_64_1[1] = { - {143, 2}, -}; -static arc arcs_64_2[1] = { + {143, 1}, + {148, 1}, + {149, 2}, +}; +static arc arcs_65_1[1] = { + {144, 2}, +}; +static arc arcs_65_2[1] = { {0, 2}, }; -static state states_64[3] = { - {4, arcs_64_0}, - {1, arcs_64_1}, - {1, arcs_64_2}, -}; -static arc arcs_65_0[1] = { - {149, 1}, -}; -static arc arcs_65_1[2] = { +static state states_65[3] = { + {4, arcs_65_0}, + {1, arcs_65_1}, + {1, arcs_65_2}, +}; +static arc arcs_66_0[1] = { + {150, 1}, +}; +static arc arcs_66_1[2] = { {34, 2}, {0, 1}, }; -static arc arcs_65_2[1] = { - {143, 3}, -}; -static arc arcs_65_3[1] = { +static arc arcs_66_2[1] = { + {144, 3}, +}; +static arc arcs_66_3[1] = { {0, 3}, }; -static state states_65[4] = { - {1, arcs_65_0}, - {2, arcs_65_1}, - {1, arcs_65_2}, - {1, arcs_65_3}, -}; -static arc arcs_66_0[2] = { - {150, 1}, - {151, 2}, -}; -static arc arcs_66_1[1] = { - {151, 2}, -}; -static arc arcs_66_2[2] = { +static state states_66[4] = { + {1, arcs_66_0}, + {2, arcs_66_1}, + {1, arcs_66_2}, + {1, arcs_66_3}, +}; +static arc arcs_67_0[2] = { + {151, 1}, {152, 2}, +}; +static arc arcs_67_1[1] = { + {152, 2}, +}; +static arc arcs_67_2[2] = { + {153, 2}, {0, 2}, }; -static state states_66[3] = { - {2, arcs_66_0}, - {1, arcs_66_1}, - {2, arcs_66_2}, -}; -static arc arcs_67_0[10] = { +static state states_67[3] = { + {2, arcs_67_0}, + {1, arcs_67_1}, + {2, arcs_67_2}, +}; +static arc arcs_68_0[10] = { {13, 1}, - {154, 2}, - {156, 3}, + {155, 2}, + {157, 3}, {23, 4}, - {159, 4}, - {160, 5}, - {82, 4}, - {161, 4}, + {160, 4}, + {161, 5}, + {83, 4}, {162, 4}, {163, 4}, -}; -static arc arcs_67_1[3] = { - {49, 6}, - {153, 6}, + {164, 4}, +}; +static arc arcs_68_1[3] = { + {50, 6}, + {154, 6}, {15, 4}, }; -static arc arcs_67_2[2] = { - {153, 7}, - {155, 4}, -}; -static arc arcs_67_3[2] = { - {157, 8}, - {158, 4}, -}; -static arc arcs_67_4[1] = { +static arc arcs_68_2[2] = { + {154, 7}, + {156, 4}, +}; +static arc arcs_68_3[2] = { + {158, 8}, + {159, 4}, +}; +static arc arcs_68_4[1] = { {0, 4}, }; -static arc arcs_67_5[2] = { - {160, 5}, +static arc arcs_68_5[2] = { + {161, 5}, {0, 5}, }; -static arc arcs_67_6[1] = { +static arc arcs_68_6[1] = { {15, 4}, }; -static arc arcs_67_7[1] = { - {155, 4}, -}; -static arc arcs_67_8[1] = { - {158, 4}, -}; -static state states_67[9] = { - {10, arcs_67_0}, - {3, arcs_67_1}, - {2, arcs_67_2}, - {2, arcs_67_3}, - {1, arcs_67_4}, - {2, arcs_67_5}, - {1, arcs_67_6}, - {1, arcs_67_7}, - {1, arcs_67_8}, -}; -static arc arcs_68_0[2] = { +static arc arcs_68_7[1] = { + {156, 4}, +}; +static arc arcs_68_8[1] = { + {159, 4}, +}; +static state states_68[9] = { + {10, arcs_68_0}, + {3, arcs_68_1}, + {2, arcs_68_2}, + {2, arcs_68_3}, + {1, arcs_68_4}, + {2, arcs_68_5}, + {1, arcs_68_6}, + {1, arcs_68_7}, + {1, arcs_68_8}, +}; +static arc arcs_69_0[2] = { {26, 1}, - {50, 1}, -}; -static arc arcs_68_1[3] = { - {164, 2}, + {51, 1}, +}; +static arc arcs_69_1[3] = { + {165, 2}, {32, 3}, {0, 1}, }; -static arc arcs_68_2[1] = { +static arc arcs_69_2[1] = { {0, 2}, }; -static arc arcs_68_3[3] = { +static arc arcs_69_3[3] = { {26, 4}, - {50, 4}, + {51, 4}, {0, 3}, }; -static arc arcs_68_4[2] = { +static arc arcs_69_4[2] = { {32, 3}, {0, 4}, }; -static state states_68[5] = { - {2, arcs_68_0}, - {3, arcs_68_1}, - {1, arcs_68_2}, - {3, arcs_68_3}, - {2, arcs_68_4}, -}; -static arc arcs_69_0[3] = { +static state states_69[5] = { + {2, arcs_69_0}, + {3, arcs_69_1}, + {1, arcs_69_2}, + {3, arcs_69_3}, + {2, arcs_69_4}, +}; +static arc arcs_70_0[3] = { {13, 1}, - {154, 2}, - {81, 3}, -}; -static arc arcs_69_1[2] = { + {155, 2}, + {82, 3}, +}; +static arc arcs_70_1[2] = { {14, 4}, {15, 5}, }; -static arc arcs_69_2[1] = { - {165, 6}, -}; -static arc arcs_69_3[1] = { +static arc arcs_70_2[1] = { + {166, 6}, +}; +static arc arcs_70_3[1] = { {23, 5}, }; -static arc arcs_69_4[1] = { +static arc arcs_70_4[1] = { {15, 5}, }; -static arc arcs_69_5[1] = { +static arc arcs_70_5[1] = { {0, 5}, }; -static arc arcs_69_6[1] = { - {155, 5}, -}; -static state states_69[7] = { - {3, arcs_69_0}, - {2, arcs_69_1}, - {1, arcs_69_2}, - {1, arcs_69_3}, - {1, arcs_69_4}, - {1, arcs_69_5}, - {1, arcs_69_6}, -}; -static arc arcs_70_0[1] = { - {166, 1}, -}; -static arc arcs_70_1[2] = { +static arc arcs_70_6[1] = { + {156, 5}, +}; +static state states_70[7] = { + {3, arcs_70_0}, + {2, arcs_70_1}, + {1, arcs_70_2}, + {1, arcs_70_3}, + {1, arcs_70_4}, + {1, arcs_70_5}, + {1, arcs_70_6}, +}; +static arc arcs_71_0[1] = { + {167, 1}, +}; +static arc arcs_71_1[2] = { {32, 2}, {0, 1}, }; -static arc arcs_70_2[2] = { - {166, 1}, +static arc arcs_71_2[2] = { + {167, 1}, {0, 2}, }; -static state states_70[3] = { - {1, arcs_70_0}, - {2, arcs_70_1}, - {2, arcs_70_2}, -}; -static arc arcs_71_0[2] = { +static state states_71[3] = { + {1, arcs_71_0}, + {2, arcs_71_1}, + {2, arcs_71_2}, +}; +static arc arcs_72_0[2] = { {26, 1}, {27, 2}, }; -static arc arcs_71_1[2] = { +static arc arcs_72_1[2] = { {27, 2}, {0, 1}, }; -static arc arcs_71_2[3] = { +static arc arcs_72_2[3] = { {26, 3}, - {167, 4}, + {168, 4}, {0, 2}, }; -static arc arcs_71_3[2] = { - {167, 4}, +static arc arcs_72_3[2] = { + {168, 4}, {0, 3}, }; -static arc arcs_71_4[1] = { +static arc arcs_72_4[1] = { {0, 4}, }; -static state states_71[5] = { - {2, arcs_71_0}, - {2, arcs_71_1}, - {3, arcs_71_2}, - {2, arcs_71_3}, - {1, arcs_71_4}, -}; -static arc arcs_72_0[1] = { +static state states_72[5] = { + {2, arcs_72_0}, + {2, arcs_72_1}, + {3, arcs_72_2}, + {2, arcs_72_3}, + {1, arcs_72_4}, +}; +static arc arcs_73_0[1] = { {27, 1}, }; -static arc arcs_72_1[2] = { +static arc arcs_73_1[2] = { {26, 2}, {0, 1}, }; -static arc arcs_72_2[1] = { +static arc arcs_73_2[1] = { {0, 2}, }; -static state states_72[3] = { - {1, arcs_72_0}, - {2, arcs_72_1}, - {1, arcs_72_2}, -}; -static arc arcs_73_0[2] = { - {107, 1}, - {50, 1}, -}; -static arc arcs_73_1[2] = { - {32, 2}, - {0, 1}, -}; -static arc arcs_73_2[3] = { - {107, 1}, - {50, 1}, - {0, 2}, -}; static state states_73[3] = { - {2, arcs_73_0}, + {1, arcs_73_0}, {2, arcs_73_1}, - {3, arcs_73_2}, -}; -static arc arcs_74_0[1] = { - {26, 1}, + {1, arcs_73_2}, +}; +static arc arcs_74_0[2] = { + {108, 1}, + {51, 1}, }; static arc arcs_74_1[2] = { {32, 2}, {0, 1}, }; -static arc arcs_74_2[2] = { +static arc arcs_74_2[3] = { + {108, 1}, + {51, 1}, + {0, 2}, +}; +static state states_74[3] = { + {2, arcs_74_0}, + {2, arcs_74_1}, + {3, arcs_74_2}, +}; +static arc arcs_75_0[1] = { + {26, 1}, +}; +static arc arcs_75_1[2] = { + {32, 2}, + {0, 1}, +}; +static arc arcs_75_2[2] = { {26, 1}, {0, 2}, }; -static state states_74[3] = { - {1, arcs_74_0}, - {2, arcs_74_1}, - {2, arcs_74_2}, -}; -static arc arcs_75_0[3] = { +static state states_75[3] = { + {1, arcs_75_0}, + {2, arcs_75_1}, + {2, arcs_75_2}, +}; +static arc arcs_76_0[3] = { {26, 1}, {34, 2}, - {50, 3}, -}; -static arc arcs_75_1[4] = { + {51, 3}, +}; +static arc arcs_76_1[4] = { {27, 4}, - {164, 5}, + {165, 5}, {32, 6}, {0, 1}, }; -static arc arcs_75_2[1] = { - {107, 7}, -}; -static arc arcs_75_3[3] = { - {164, 5}, +static arc arcs_76_2[1] = { + {108, 7}, +}; +static arc arcs_76_3[3] = { + {165, 5}, {32, 6}, {0, 3}, }; -static arc arcs_75_4[1] = { +static arc arcs_76_4[1] = { {26, 7}, }; -static arc arcs_75_5[1] = { +static arc arcs_76_5[1] = { {0, 5}, }; -static arc arcs_75_6[3] = { +static arc arcs_76_6[3] = { {26, 8}, - {50, 8}, + {51, 8}, {0, 6}, }; -static arc arcs_75_7[3] = { - {164, 5}, +static arc arcs_76_7[3] = { + {165, 5}, {32, 9}, {0, 7}, }; -static arc arcs_75_8[2] = { +static arc arcs_76_8[2] = { {32, 6}, {0, 8}, }; -static arc arcs_75_9[3] = { +static arc arcs_76_9[3] = { {26, 10}, {34, 11}, {0, 9}, }; -static arc arcs_75_10[1] = { +static arc arcs_76_10[1] = { {27, 12}, }; -static arc arcs_75_11[1] = { - {107, 13}, -}; -static arc arcs_75_12[1] = { +static arc arcs_76_11[1] = { + {108, 13}, +}; +static arc arcs_76_12[1] = { {26, 13}, }; -static arc arcs_75_13[2] = { +static arc arcs_76_13[2] = { {32, 9}, {0, 13}, }; -static state states_75[14] = { - {3, arcs_75_0}, - {4, arcs_75_1}, - {1, arcs_75_2}, - {3, arcs_75_3}, - {1, arcs_75_4}, - {1, arcs_75_5}, - {3, arcs_75_6}, - {3, arcs_75_7}, - {2, arcs_75_8}, - {3, arcs_75_9}, - {1, arcs_75_10}, - {1, arcs_75_11}, - {1, arcs_75_12}, - {2, arcs_75_13}, -}; -static arc arcs_76_0[1] = { - {168, 1}, -}; -static arc arcs_76_1[1] = { +static state states_76[14] = { + {3, arcs_76_0}, + {4, arcs_76_1}, + {1, arcs_76_2}, + {3, arcs_76_3}, + {1, arcs_76_4}, + {1, arcs_76_5}, + {3, arcs_76_6}, + {3, arcs_76_7}, + {2, arcs_76_8}, + {3, arcs_76_9}, + {1, arcs_76_10}, + {1, arcs_76_11}, + {1, arcs_76_12}, + {2, arcs_76_13}, +}; +static arc arcs_77_0[1] = { + {169, 1}, +}; +static arc arcs_77_1[1] = { {23, 2}, }; -static arc arcs_76_2[2] = { +static arc arcs_77_2[2] = { {13, 3}, {27, 4}, }; -static arc arcs_76_3[2] = { +static arc arcs_77_3[2] = { {14, 5}, {15, 6}, }; -static arc arcs_76_4[1] = { +static arc arcs_77_4[1] = { {28, 7}, }; -static arc arcs_76_5[1] = { +static arc arcs_77_5[1] = { {15, 6}, }; -static arc arcs_76_6[1] = { +static arc arcs_77_6[1] = { {27, 4}, }; -static arc arcs_76_7[1] = { +static arc arcs_77_7[1] = { {0, 7}, }; -static state states_76[8] = { - {1, arcs_76_0}, - {1, arcs_76_1}, - {2, arcs_76_2}, - {2, arcs_76_3}, - {1, arcs_76_4}, - {1, arcs_76_5}, - {1, arcs_76_6}, - {1, arcs_76_7}, -}; -static arc arcs_77_0[1] = { - {169, 1}, -}; -static arc arcs_77_1[2] = { +static state states_77[8] = { + {1, arcs_77_0}, + {1, arcs_77_1}, + {2, arcs_77_2}, + {2, arcs_77_3}, + {1, arcs_77_4}, + {1, arcs_77_5}, + {1, arcs_77_6}, + {1, arcs_77_7}, +}; +static arc arcs_78_0[1] = { + {170, 1}, +}; +static arc arcs_78_1[2] = { {32, 2}, {0, 1}, }; -static arc arcs_77_2[2] = { - {169, 1}, +static arc arcs_78_2[2] = { + {170, 1}, {0, 2}, }; -static state states_77[3] = { - {1, arcs_77_0}, - {2, arcs_77_1}, - {2, arcs_77_2}, -}; -static arc arcs_78_0[3] = { +static state states_78[3] = { + {1, arcs_78_0}, + {2, arcs_78_1}, + {2, arcs_78_2}, +}; +static arc arcs_79_0[3] = { {26, 1}, {34, 2}, {33, 2}, }; -static arc arcs_78_1[3] = { - {164, 3}, +static arc arcs_79_1[3] = { + {165, 3}, {31, 2}, {0, 1}, }; -static arc arcs_78_2[1] = { +static arc arcs_79_2[1] = { {26, 3}, }; -static arc arcs_78_3[1] = { +static arc arcs_79_3[1] = { {0, 3}, }; -static state states_78[4] = { - {3, arcs_78_0}, - {3, arcs_78_1}, - {1, arcs_78_2}, - {1, arcs_78_3}, -}; -static arc arcs_79_0[2] = { - {164, 1}, - {171, 1}, -}; -static arc arcs_79_1[1] = { +static state states_79[4] = { + {3, arcs_79_0}, + {3, arcs_79_1}, + {1, arcs_79_2}, + {1, arcs_79_3}, +}; +static arc arcs_80_0[2] = { + {165, 1}, + {172, 1}, +}; +static arc arcs_80_1[1] = { {0, 1}, }; -static state states_79[2] = { - {2, arcs_79_0}, - {1, arcs_79_1}, -}; -static arc arcs_80_0[1] = { - {100, 1}, -}; -static arc arcs_80_1[1] = { - {65, 2}, -}; -static arc arcs_80_2[1] = { - {101, 3}, -}; -static arc arcs_80_3[1] = { - {111, 4}, -}; -static arc arcs_80_4[2] = { - {170, 5}, +static state states_80[2] = { + {2, arcs_80_0}, + {1, arcs_80_1}, +}; +static arc arcs_81_0[1] = { + {101, 1}, +}; +static arc arcs_81_1[1] = { + {66, 2}, +}; +static arc arcs_81_2[1] = { + {102, 3}, +}; +static arc arcs_81_3[1] = { + {112, 4}, +}; +static arc arcs_81_4[2] = { + {171, 5}, {0, 4}, }; -static arc arcs_80_5[1] = { +static arc arcs_81_5[1] = { {0, 5}, }; -static state states_80[6] = { - {1, arcs_80_0}, - {1, arcs_80_1}, - {1, arcs_80_2}, - {1, arcs_80_3}, - {2, arcs_80_4}, - {1, arcs_80_5}, -}; -static arc arcs_81_0[1] = { - {96, 1}, -}; -static arc arcs_81_1[1] = { - {113, 2}, -}; -static arc arcs_81_2[2] = { - {170, 3}, - {0, 2}, -}; -static arc arcs_81_3[1] = { - {0, 3}, -}; -static state states_81[4] = { +static state states_81[6] = { {1, arcs_81_0}, {1, arcs_81_1}, - {2, arcs_81_2}, + {1, arcs_81_2}, {1, arcs_81_3}, + {2, arcs_81_4}, + {1, arcs_81_5}, }; static arc arcs_82_0[1] = { - {23, 1}, + {97, 1}, }; static arc arcs_82_1[1] = { - {0, 1}, -}; -static state states_82[2] = { + {114, 2}, +}; +static arc arcs_82_2[2] = { + {171, 3}, + {0, 2}, +}; +static arc arcs_82_3[1] = { + {0, 3}, +}; +static state states_82[4] = { {1, arcs_82_0}, {1, arcs_82_1}, + {2, arcs_82_2}, + {1, arcs_82_3}, }; static arc arcs_83_0[1] = { - {173, 1}, -}; -static arc arcs_83_1[2] = { - {174, 2}, + {23, 1}, +}; +static arc arcs_83_1[1] = { {0, 1}, }; -static arc arcs_83_2[1] = { - {0, 2}, -}; -static state states_83[3] = { +static state states_83[2] = { {1, arcs_83_0}, - {2, arcs_83_1}, - {1, arcs_83_2}, -}; -static arc arcs_84_0[2] = { - {76, 1}, - {9, 2}, -}; -static arc arcs_84_1[1] = { - {26, 2}, + {1, arcs_83_1}, +}; +static arc arcs_84_0[1] = { + {174, 1}, +}; +static arc arcs_84_1[2] = { + {175, 2}, + {0, 1}, }; static arc arcs_84_2[1] = { {0, 2}, }; static state states_84[3] = { - {2, arcs_84_0}, - {1, arcs_84_1}, + {1, arcs_84_0}, + {2, arcs_84_1}, {1, arcs_84_2}, }; -static dfa dfas[85] = { +static arc arcs_85_0[2] = { + {77, 1}, + {9, 2}, +}; +static arc arcs_85_1[1] = { + {26, 2}, +}; +static arc arcs_85_2[1] = { + {0, 2}, +}; +static state states_85[3] = { + {2, arcs_85_0}, + {1, arcs_85_1}, + {1, arcs_85_2}, +}; +static dfa dfas[86] = { {256, "single_input", 0, 3, states_0, - "\004\050\340\000\002\000\000\000\005\237\204\003\131\002\010\001\000\140\110\224\017\041"}, + "\004\050\340\000\002\000\000\000\012\076\011\007\262\004\020\002\000\300\220\050\037\102"}, {257, "file_input", 0, 2, states_1, - "\204\050\340\000\002\000\000\000\005\237\204\003\131\002\010\001\000\140\110\224\017\041"}, + "\204\050\340\000\002\000\000\000\012\076\011\007\262\004\020\002\000\300\220\050\037\102"}, {258, "eval_input", 0, 3, states_2, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, {259, "decorator", 0, 7, states_3, "\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {260, "decorators", 0, 2, states_4, @@ -1902,170 +1926,172 @@ {268, "vfpdef", 0, 2, states_12, "\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, {269, "stmt", 0, 2, states_13, - "\000\050\340\000\002\000\000\000\005\237\204\003\131\002\010\001\000\140\110\224\017\041"}, + "\000\050\340\000\002\000\000\000\012\076\011\007\262\004\020\002\000\300\220\050\037\102"}, {270, "simple_stmt", 0, 4, states_14, - "\000\040\200\000\002\000\000\000\005\237\204\003\000\000\010\001\000\140\110\224\017\040"}, + "\000\040\200\000\002\000\000\000\012\076\011\007\000\000\020\002\000\300\220\050\037\100"}, {271, "small_stmt", 0, 2, states_15, - "\000\040\200\000\002\000\000\000\005\237\204\003\000\000\010\001\000\140\110\224\017\040"}, + "\000\040\200\000\002\000\000\000\012\076\011\007\000\000\020\002\000\300\220\050\037\100"}, {272, "expr_stmt", 0, 6, states_16, - "\000\040\200\000\002\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {273, "testlist_star_expr", 0, 3, states_17, - "\000\040\200\000\002\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {274, "augassign", 0, 2, states_18, - "\000\000\000\000\000\000\370\377\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {275, "del_stmt", 0, 3, states_19, - "\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {276, "pass_stmt", 0, 2, states_20, - "\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {277, "flow_stmt", 0, 2, states_21, - "\000\000\000\000\000\000\000\000\000\017\000\000\000\000\000\000\000\000\000\000\000\040"}, - {278, "break_stmt", 0, 2, states_22, - "\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000"}, - {279, "continue_stmt", 0, 2, states_23, + "\000\040\200\000\002\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {273, "annassign", 0, 5, states_17, + "\000\000\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + {274, "testlist_star_expr", 0, 3, states_18, + "\000\040\200\000\002\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {275, "augassign", 0, 2, states_19, + "\000\000\000\000\000\000\360\377\001\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + {276, "del_stmt", 0, 3, states_20, + "\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + {277, "pass_stmt", 0, 2, states_21, + "\000\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000"}, + {278, "flow_stmt", 0, 2, states_22, + "\000\000\000\000\000\000\000\000\000\036\000\000\000\000\000\000\000\000\000\000\000\100"}, + {279, "break_stmt", 0, 2, states_23, "\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000"}, - {280, "return_stmt", 0, 3, states_24, + {280, "continue_stmt", 0, 2, states_24, "\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000\000"}, - {281, "yield_stmt", 0, 2, states_25, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\040"}, - {282, "raise_stmt", 0, 5, states_26, + {281, "return_stmt", 0, 3, states_25, "\000\000\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\000\000\000\000\000"}, - {283, "import_stmt", 0, 2, states_27, - "\000\000\000\000\000\000\000\000\000\220\000\000\000\000\000\000\000\000\000\000\000\000"}, - {284, "import_name", 0, 3, states_28, - "\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000"}, - {285, "import_from", 0, 8, states_29, + {282, "yield_stmt", 0, 2, states_26, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\100"}, + {283, "raise_stmt", 0, 5, states_27, "\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000\000\000"}, - {286, "import_as_name", 0, 4, states_30, + {284, "import_stmt", 0, 2, states_28, + "\000\000\000\000\000\000\000\000\000\040\001\000\000\000\000\000\000\000\000\000\000\000"}, + {285, "import_name", 0, 3, states_29, + "\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000"}, + {286, "import_from", 0, 8, states_30, + "\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000"}, + {287, "import_as_name", 0, 4, states_31, "\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {287, "dotted_as_name", 0, 4, states_31, + {288, "dotted_as_name", 0, 4, states_32, "\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {288, "import_as_names", 0, 3, states_32, + {289, "import_as_names", 0, 3, states_33, "\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {289, "dotted_as_names", 0, 2, states_33, + {290, "dotted_as_names", 0, 2, states_34, "\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {290, "dotted_name", 0, 2, states_34, + {291, "dotted_name", 0, 2, states_35, "\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {291, "global_stmt", 0, 3, states_35, - "\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000"}, - {292, "nonlocal_stmt", 0, 3, states_36, + {292, "global_stmt", 0, 3, states_36, "\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000"}, - {293, "assert_stmt", 0, 5, states_37, + {293, "nonlocal_stmt", 0, 3, states_37, "\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000"}, - {294, "compound_stmt", 0, 2, states_38, - "\000\010\140\000\000\000\000\000\000\000\000\000\131\002\000\000\000\000\000\000\000\001"}, - {295, "async_stmt", 0, 3, states_39, + {294, "assert_stmt", 0, 5, states_38, + "\000\000\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\000\000"}, + {295, "compound_stmt", 0, 2, states_39, + "\000\010\140\000\000\000\000\000\000\000\000\000\262\004\000\000\000\000\000\000\000\002"}, + {296, "async_stmt", 0, 3, states_40, "\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {296, "if_stmt", 0, 8, states_40, - "\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000"}, - {297, "while_stmt", 0, 8, states_41, - "\000\000\000\000\000\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\000\000"}, - {298, "for_stmt", 0, 10, states_42, + {297, "if_stmt", 0, 8, states_41, + "\000\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000"}, + {298, "while_stmt", 0, 8, states_42, "\000\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000"}, - {299, "try_stmt", 0, 13, states_43, - "\000\000\000\000\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000"}, - {300, "with_stmt", 0, 5, states_44, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000"}, - {301, "with_item", 0, 4, states_45, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {302, "except_clause", 0, 5, states_46, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000"}, - {303, "suite", 0, 5, states_47, - "\004\040\200\000\002\000\000\000\005\237\204\003\000\000\010\001\000\140\110\224\017\040"}, - {304, "test", 0, 6, states_48, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {305, "test_nocond", 0, 2, states_49, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {306, "lambdef", 0, 5, states_50, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000"}, - {307, "lambdef_nocond", 0, 5, states_51, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000"}, - {308, "or_test", 0, 2, states_52, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\001\000\140\110\224\017\000"}, - {309, "and_test", 0, 2, states_53, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\001\000\140\110\224\017\000"}, - {310, "not_test", 0, 3, states_54, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\001\000\140\110\224\017\000"}, - {311, "comparison", 0, 2, states_55, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\140\110\224\017\000"}, - {312, "comp_op", 0, 4, states_56, - "\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\371\007\000\000\000\000\000"}, - {313, "star_expr", 0, 3, states_57, + {299, "for_stmt", 0, 10, states_43, + "\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000"}, + {300, "try_stmt", 0, 13, states_44, + "\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000"}, + {301, "with_stmt", 0, 5, states_45, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000"}, + {302, "with_item", 0, 4, states_46, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {303, "except_clause", 0, 5, states_47, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000"}, + {304, "suite", 0, 5, states_48, + "\004\040\200\000\002\000\000\000\012\076\011\007\000\000\020\002\000\300\220\050\037\100"}, + {305, "test", 0, 6, states_49, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {306, "test_nocond", 0, 2, states_50, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {307, "lambdef", 0, 5, states_51, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000"}, + {308, "lambdef_nocond", 0, 5, states_52, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000"}, + {309, "or_test", 0, 2, states_53, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\002\000\300\220\050\037\000"}, + {310, "and_test", 0, 2, states_54, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\002\000\300\220\050\037\000"}, + {311, "not_test", 0, 3, states_55, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\002\000\300\220\050\037\000"}, + {312, "comparison", 0, 2, states_56, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\300\220\050\037\000"}, + {313, "comp_op", 0, 4, states_57, + "\000\000\000\000\000\000\000\000\000\000\000\000\100\000\000\362\017\000\000\000\000\000"}, + {314, "star_expr", 0, 3, states_58, "\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {314, "expr", 0, 2, states_58, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\140\110\224\017\000"}, - {315, "xor_expr", 0, 2, states_59, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\140\110\224\017\000"}, - {316, "and_expr", 0, 2, states_60, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\140\110\224\017\000"}, - {317, "shift_expr", 0, 2, states_61, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\140\110\224\017\000"}, - {318, "arith_expr", 0, 2, states_62, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\140\110\224\017\000"}, - {319, "term", 0, 2, states_63, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\140\110\224\017\000"}, - {320, "factor", 0, 3, states_64, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\140\110\224\017\000"}, - {321, "power", 0, 4, states_65, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\100\224\017\000"}, - {322, "atom_expr", 0, 3, states_66, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\100\224\017\000"}, - {323, "atom", 0, 9, states_67, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\224\017\000"}, - {324, "testlist_comp", 0, 5, states_68, - "\000\040\200\000\002\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {325, "trailer", 0, 7, states_69, - "\000\040\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\004\000\000"}, - {326, "subscriptlist", 0, 3, states_70, - "\000\040\200\010\000\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {327, "subscript", 0, 5, states_71, - "\000\040\200\010\000\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {328, "sliceop", 0, 3, states_72, + {315, "expr", 0, 2, states_59, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\300\220\050\037\000"}, + {316, "xor_expr", 0, 2, states_60, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\300\220\050\037\000"}, + {317, "and_expr", 0, 2, states_61, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\300\220\050\037\000"}, + {318, "shift_expr", 0, 2, states_62, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\300\220\050\037\000"}, + {319, "arith_expr", 0, 2, states_63, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\300\220\050\037\000"}, + {320, "term", 0, 2, states_64, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\300\220\050\037\000"}, + {321, "factor", 0, 3, states_65, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\300\220\050\037\000"}, + {322, "power", 0, 4, states_66, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\200\050\037\000"}, + {323, "atom_expr", 0, 3, states_67, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\200\050\037\000"}, + {324, "atom", 0, 9, states_68, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\000\000\000\000\000\050\037\000"}, + {325, "testlist_comp", 0, 5, states_69, + "\000\040\200\000\002\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {326, "trailer", 0, 7, states_70, + "\000\040\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\010\000\000"}, + {327, "subscriptlist", 0, 3, states_71, + "\000\040\200\010\000\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {328, "subscript", 0, 5, states_72, + "\000\040\200\010\000\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {329, "sliceop", 0, 3, states_73, "\000\000\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {329, "exprlist", 0, 3, states_73, - "\000\040\200\000\002\000\000\000\000\000\004\000\000\000\000\000\000\140\110\224\017\000"}, - {330, "testlist", 0, 3, states_74, - "\000\040\200\000\000\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {331, "dictorsetmaker", 0, 14, states_75, - "\000\040\200\000\006\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {332, "classdef", 0, 8, states_76, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\001"}, - {333, "arglist", 0, 3, states_77, - "\000\040\200\000\006\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {334, "argument", 0, 4, states_78, - "\000\040\200\000\006\000\000\000\000\000\004\000\000\000\010\001\000\140\110\224\017\000"}, - {335, "comp_iter", 0, 2, states_79, - "\000\000\000\000\000\000\000\000\000\000\000\000\021\000\000\000\000\000\000\000\000\000"}, - {336, "comp_for", 0, 6, states_80, - "\000\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000"}, - {337, "comp_if", 0, 4, states_81, - "\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000"}, - {338, "encoding_decl", 0, 2, states_82, + {330, "exprlist", 0, 3, states_74, + "\000\040\200\000\002\000\000\000\000\000\010\000\000\000\000\000\000\300\220\050\037\000"}, + {331, "testlist", 0, 3, states_75, + "\000\040\200\000\000\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {332, "dictorsetmaker", 0, 14, states_76, + "\000\040\200\000\006\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {333, "classdef", 0, 8, states_77, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\002"}, + {334, "arglist", 0, 3, states_78, + "\000\040\200\000\006\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {335, "argument", 0, 4, states_79, + "\000\040\200\000\006\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, + {336, "comp_iter", 0, 2, states_80, + "\000\000\000\000\000\000\000\000\000\000\000\000\042\000\000\000\000\000\000\000\000\000"}, + {337, "comp_for", 0, 6, states_81, + "\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000"}, + {338, "comp_if", 0, 4, states_82, + "\000\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000"}, + {339, "encoding_decl", 0, 2, states_83, "\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"}, - {339, "yield_expr", 0, 3, states_83, - "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\040"}, - {340, "yield_arg", 0, 3, states_84, - "\000\040\200\000\000\000\000\000\000\020\004\000\000\000\010\001\000\140\110\224\017\000"}, -}; -static label labels[175] = { + {340, "yield_expr", 0, 3, states_84, + "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\100"}, + {341, "yield_arg", 0, 3, states_85, + "\000\040\200\000\000\000\000\000\000\040\010\000\000\000\020\002\000\300\220\050\037\000"}, +}; +static label labels[176] = { {0, "EMPTY"}, {256, 0}, {4, 0}, {270, 0}, - {294, 0}, + {295, 0}, {257, 0}, {269, 0}, {0, 0}, {258, 0}, - {330, 0}, + {331, 0}, {259, 0}, {49, 0}, - {290, 0}, + {291, 0}, {7, 0}, - {333, 0}, + {334, 0}, {8, 0}, {260, 0}, {261, 0}, - {332, 0}, + {333, 0}, {263, 0}, {262, 0}, {55, 0}, @@ -2073,9 +2099,9 @@ {1, 0}, {264, 0}, {51, 0}, + {305, 0}, + {11, 0}, {304, 0}, - {11, 0}, - {303, 0}, {265, 0}, {266, 0}, {22, 0}, @@ -2087,17 +2113,18 @@ {271, 0}, {13, 0}, {272, 0}, - {275, 0}, {276, 0}, {277, 0}, - {283, 0}, - {291, 0}, + {278, 0}, + {284, 0}, {292, 0}, {293, 0}, + {294, 0}, + {274, 0}, {273, 0}, - {274, 0}, - {339, 0}, - {313, 0}, + {275, 0}, + {340, 0}, + {314, 0}, {36, 0}, {37, 0}, {38, 0}, @@ -2112,37 +2139,37 @@ {46, 0}, {48, 0}, {1, "del"}, - {329, 0}, + {330, 0}, {1, "pass"}, - {278, 0}, {279, 0}, {280, 0}, + {281, 0}, + {283, 0}, {282, 0}, - {281, 0}, {1, "break"}, {1, "continue"}, {1, "return"}, {1, "raise"}, {1, "from"}, - {284, 0}, {285, 0}, + {286, 0}, {1, "import"}, - {289, 0}, + {290, 0}, {23, 0}, {52, 0}, + {289, 0}, + {287, 0}, + {1, "as"}, {288, 0}, - {286, 0}, - {1, "as"}, - {287, 0}, {1, "global"}, {1, "nonlocal"}, {1, "assert"}, - {296, 0}, {297, 0}, {298, 0}, {299, 0}, {300, 0}, - {295, 0}, + {301, 0}, + {296, 0}, {1, "if"}, {1, "elif"}, {1, "else"}, @@ -2150,26 +2177,26 @@ {1, "for"}, {1, "in"}, {1, "try"}, - {302, 0}, + {303, 0}, {1, "finally"}, {1, "with"}, - {301, 0}, - {314, 0}, + {302, 0}, + {315, 0}, {1, "except"}, {5, 0}, {6, 0}, + {309, 0}, + {307, 0}, + {306, 0}, {308, 0}, - {306, 0}, - {305, 0}, - {307, 0}, {1, "lambda"}, - {309, 0}, + {310, 0}, {1, "or"}, - {310, 0}, + {311, 0}, {1, "and"}, {1, "not"}, - {311, 0}, {312, 0}, + {313, 0}, {20, 0}, {21, 0}, {27, 0}, @@ -2178,54 +2205,54 @@ {28, 0}, {28, 0}, {1, "is"}, - {315, 0}, + {316, 0}, {18, 0}, - {316, 0}, + {317, 0}, {32, 0}, - {317, 0}, + {318, 0}, {19, 0}, - {318, 0}, + {319, 0}, {33, 0}, {34, 0}, - {319, 0}, + {320, 0}, {14, 0}, {15, 0}, - {320, 0}, + {321, 0}, {17, 0}, {24, 0}, {47, 0}, {31, 0}, - {321, 0}, {322, 0}, + {323, 0}, {54, 0}, - {323, 0}, + {324, 0}, + {326, 0}, {325, 0}, - {324, 0}, {9, 0}, {10, 0}, {25, 0}, - {331, 0}, + {332, 0}, {26, 0}, {2, 0}, {3, 0}, {1, "None"}, {1, "True"}, {1, "False"}, - {336, 0}, - {326, 0}, + {337, 0}, {327, 0}, {328, 0}, + {329, 0}, {1, "class"}, - {334, 0}, {335, 0}, - {337, 0}, + {336, 0}, {338, 0}, + {339, 0}, {1, "yield"}, - {340, 0}, + {341, 0}, }; grammar _PyParser_Grammar = { - 85, + 86, dfas, - {175, labels}, + {176, labels}, 256 }; diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -242,7 +242,7 @@ 101,95,97,116,111,109,105,99,106,0,0,0,115,26,0,0, 0,0,5,16,1,6,1,26,1,2,3,14,1,20,1,16, 1,14,1,2,1,14,1,14,1,6,1,114,58,0,0,0, - 105,45,13,0,0,233,2,0,0,0,114,15,0,0,0,115, + 105,47,13,0,0,233,2,0,0,0,114,15,0,0,0,115, 2,0,0,0,13,10,90,11,95,95,112,121,99,97,99,104, 101,95,95,122,4,111,112,116,45,122,3,46,112,121,122,4, 46,112,121,99,78,41,1,218,12,111,112,116,105,109,105,122, @@ -346,7 +346,7 @@ 103,90,15,97,108,109,111,115,116,95,102,105,108,101,110,97, 109,101,114,4,0,0,0,114,4,0,0,0,114,6,0,0, 0,218,17,99,97,99,104,101,95,102,114,111,109,95,115,111, - 117,114,99,101,1,1,0,0,115,48,0,0,0,0,18,8, + 117,114,99,101,3,1,0,0,115,48,0,0,0,0,18,8, 1,6,1,6,1,8,1,4,1,8,1,12,1,10,1,12, 1,16,1,8,1,8,1,8,1,24,1,8,1,12,1,6, 2,8,1,8,1,8,1,8,1,14,1,14,1,114,83,0, @@ -420,7 +420,7 @@ 112,116,95,108,101,118,101,108,90,13,98,97,115,101,95,102, 105,108,101,110,97,109,101,114,4,0,0,0,114,4,0,0, 0,114,6,0,0,0,218,17,115,111,117,114,99,101,95,102, - 114,111,109,95,99,97,99,104,101,46,1,0,0,115,46,0, + 114,111,109,95,99,97,99,104,101,48,1,0,0,115,46,0, 0,0,0,9,12,1,8,1,10,1,12,1,12,1,8,1, 6,1,10,1,10,1,8,1,6,1,10,1,8,1,16,1, 10,1,6,1,8,1,16,1,8,1,6,1,8,1,14,1, @@ -456,7 +456,7 @@ 115,105,111,110,218,11,115,111,117,114,99,101,95,112,97,116, 104,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, 218,15,95,103,101,116,95,115,111,117,114,99,101,102,105,108, - 101,80,1,0,0,115,20,0,0,0,0,7,12,1,4,1, + 101,82,1,0,0,115,20,0,0,0,0,7,12,1,4,1, 16,1,26,1,4,1,2,1,12,1,18,1,18,1,114,95, 0,0,0,99,1,0,0,0,0,0,0,0,1,0,0,0, 11,0,0,0,67,0,0,0,115,74,0,0,0,124,0,106, @@ -469,7 +469,7 @@ 0,0,114,83,0,0,0,114,70,0,0,0,114,78,0,0, 0,41,1,218,8,102,105,108,101,110,97,109,101,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,11,95,103, - 101,116,95,99,97,99,104,101,100,99,1,0,0,115,16,0, + 101,116,95,99,97,99,104,101,100,101,1,0,0,115,16,0, 0,0,0,1,14,1,2,1,8,1,14,1,8,1,14,1, 6,2,114,99,0,0,0,99,1,0,0,0,0,0,0,0, 2,0,0,0,11,0,0,0,67,0,0,0,115,52,0,0, @@ -483,7 +483,7 @@ 0,233,128,0,0,0,41,3,114,41,0,0,0,114,43,0, 0,0,114,42,0,0,0,41,2,114,37,0,0,0,114,44, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,10,95,99,97,108,99,95,109,111,100,101,111,1, + 0,0,218,10,95,99,97,108,99,95,109,111,100,101,113,1, 0,0,115,12,0,0,0,0,2,2,1,14,1,14,1,10, 3,8,1,114,101,0,0,0,99,1,0,0,0,0,0,0, 0,3,0,0,0,11,0,0,0,3,0,0,0,115,68,0, @@ -521,7 +521,7 @@ 115,90,6,107,119,97,114,103,115,41,1,218,6,109,101,116, 104,111,100,114,4,0,0,0,114,6,0,0,0,218,19,95, 99,104,101,99,107,95,110,97,109,101,95,119,114,97,112,112, - 101,114,131,1,0,0,115,12,0,0,0,0,1,8,1,8, + 101,114,133,1,0,0,115,12,0,0,0,0,1,8,1,8, 1,10,1,4,1,20,1,122,40,95,99,104,101,99,107,95, 110,97,109,101,46,60,108,111,99,97,108,115,62,46,95,99, 104,101,99,107,95,110,97,109,101,95,119,114,97,112,112,101, @@ -541,7 +541,7 @@ 116,116,114,218,8,95,95,100,105,99,116,95,95,218,6,117, 112,100,97,116,101,41,3,90,3,110,101,119,90,3,111,108, 100,114,55,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,218,5,95,119,114,97,112,142,1,0,0, + 114,6,0,0,0,218,5,95,119,114,97,112,144,1,0,0, 115,8,0,0,0,0,1,10,1,10,1,22,1,122,26,95, 99,104,101,99,107,95,110,97,109,101,46,60,108,111,99,97, 108,115,62,46,95,119,114,97,112,41,1,78,41,3,218,10, @@ -549,7 +549,7 @@ 9,78,97,109,101,69,114,114,111,114,41,3,114,106,0,0, 0,114,107,0,0,0,114,117,0,0,0,114,4,0,0,0, 41,1,114,106,0,0,0,114,6,0,0,0,218,11,95,99, - 104,101,99,107,95,110,97,109,101,123,1,0,0,115,14,0, + 104,101,99,107,95,110,97,109,101,125,1,0,0,115,14,0, 0,0,0,8,14,7,2,1,10,1,14,2,14,5,10,1, 114,120,0,0,0,99,2,0,0,0,0,0,0,0,5,0, 0,0,4,0,0,0,67,0,0,0,115,60,0,0,0,124, @@ -577,7 +577,7 @@ 101,218,6,108,111,97,100,101,114,218,8,112,111,114,116,105, 111,110,115,218,3,109,115,103,114,4,0,0,0,114,4,0, 0,0,114,6,0,0,0,218,17,95,102,105,110,100,95,109, - 111,100,117,108,101,95,115,104,105,109,151,1,0,0,115,10, + 111,100,117,108,101,95,115,104,105,109,153,1,0,0,115,10, 0,0,0,0,10,14,1,16,1,4,1,22,1,114,127,0, 0,0,99,4,0,0,0,0,0,0,0,11,0,0,0,19, 0,0,0,67,0,0,0,115,128,1,0,0,105,0,125,4, @@ -657,7 +657,7 @@ 111,117,114,99,101,95,115,105,122,101,114,4,0,0,0,114, 4,0,0,0,114,6,0,0,0,218,25,95,118,97,108,105, 100,97,116,101,95,98,121,116,101,99,111,100,101,95,104,101, - 97,100,101,114,168,1,0,0,115,76,0,0,0,0,11,4, + 97,100,101,114,170,1,0,0,115,76,0,0,0,0,11,4, 1,8,1,10,3,4,1,8,1,8,1,12,1,12,1,12, 1,8,1,12,1,12,1,12,1,12,1,10,1,12,1,10, 1,12,1,10,1,12,1,8,1,10,1,2,1,16,1,14, @@ -686,7 +686,7 @@ 5,114,56,0,0,0,114,102,0,0,0,114,93,0,0,0, 114,94,0,0,0,218,4,99,111,100,101,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,17,95,99,111,109, - 112,105,108,101,95,98,121,116,101,99,111,100,101,223,1,0, + 112,105,108,101,95,98,121,116,101,99,111,100,101,225,1,0, 0,115,16,0,0,0,0,2,10,1,10,1,12,1,8,1, 12,1,6,2,12,1,114,145,0,0,0,114,62,0,0,0, 99,3,0,0,0,0,0,0,0,4,0,0,0,3,0,0, @@ -705,7 +705,7 @@ 115,41,4,114,144,0,0,0,114,130,0,0,0,114,138,0, 0,0,114,56,0,0,0,114,4,0,0,0,114,4,0,0, 0,114,6,0,0,0,218,17,95,99,111,100,101,95,116,111, - 95,98,121,116,101,99,111,100,101,235,1,0,0,115,10,0, + 95,98,121,116,101,99,111,100,101,237,1,0,0,115,10,0, 0,0,0,3,8,1,14,1,14,1,16,1,114,148,0,0, 0,99,1,0,0,0,0,0,0,0,5,0,0,0,4,0, 0,0,67,0,0,0,115,62,0,0,0,100,1,100,2,108, @@ -732,7 +732,7 @@ 101,218,8,101,110,99,111,100,105,110,103,90,15,110,101,119, 108,105,110,101,95,100,101,99,111,100,101,114,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,13,100,101,99, - 111,100,101,95,115,111,117,114,99,101,245,1,0,0,115,10, + 111,100,101,95,115,111,117,114,99,101,247,1,0,0,115,10, 0,0,0,0,5,8,1,12,1,10,1,12,1,114,153,0, 0,0,41,2,114,124,0,0,0,218,26,115,117,98,109,111, 100,117,108,101,95,115,101,97,114,99,104,95,108,111,99,97, @@ -794,7 +794,7 @@ 0,0,90,7,100,105,114,110,97,109,101,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,23,115,112,101,99, 95,102,114,111,109,95,102,105,108,101,95,108,111,99,97,116, - 105,111,110,6,2,0,0,115,62,0,0,0,0,12,8,4, + 105,111,110,8,2,0,0,115,62,0,0,0,0,12,8,4, 4,1,10,2,2,1,14,1,14,1,8,2,10,8,18,1, 6,3,8,1,16,1,14,1,10,1,6,1,6,2,4,3, 8,2,10,1,2,1,14,1,14,1,6,2,4,1,8,2, @@ -830,7 +830,7 @@ 72,75,69,89,95,76,79,67,65,76,95,77,65,67,72,73, 78,69,41,2,218,3,99,108,115,114,5,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,218,14,95, - 111,112,101,110,95,114,101,103,105,115,116,114,121,86,2,0, + 111,112,101,110,95,114,101,103,105,115,116,114,121,88,2,0, 0,115,8,0,0,0,0,2,2,1,14,1,14,1,122,36, 87,105,110,100,111,119,115,82,101,103,105,115,116,114,121,70, 105,110,100,101,114,46,95,111,112,101,110,95,114,101,103,105, @@ -856,7 +856,7 @@ 114,121,95,107,101,121,114,5,0,0,0,90,4,104,107,101, 121,218,8,102,105,108,101,112,97,116,104,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,16,95,115,101,97, - 114,99,104,95,114,101,103,105,115,116,114,121,93,2,0,0, + 114,99,104,95,114,101,103,105,115,116,114,121,95,2,0,0, 115,22,0,0,0,0,2,6,1,8,2,6,1,10,1,22, 1,2,1,12,1,26,1,14,1,6,1,122,38,87,105,110, 100,111,119,115,82,101,103,105,115,116,114,121,70,105,110,100, @@ -878,7 +878,7 @@ 0,0,0,114,37,0,0,0,218,6,116,97,114,103,101,116, 114,174,0,0,0,114,124,0,0,0,114,164,0,0,0,114, 162,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,9,102,105,110,100,95,115,112,101,99,108,2, + 0,0,0,218,9,102,105,110,100,95,115,112,101,99,110,2, 0,0,115,26,0,0,0,0,2,10,1,8,1,4,1,2, 1,12,1,14,1,6,1,16,1,14,1,6,1,10,1,8, 1,122,31,87,105,110,100,111,119,115,82,101,103,105,115,116, @@ -897,7 +897,7 @@ 78,41,2,114,178,0,0,0,114,124,0,0,0,41,4,114, 168,0,0,0,114,123,0,0,0,114,37,0,0,0,114,162, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,102,105,110,100,95,109,111,100,117,108,101,124, + 0,0,218,11,102,105,110,100,95,109,111,100,117,108,101,126, 2,0,0,115,8,0,0,0,0,7,12,1,8,1,8,2, 122,33,87,105,110,100,111,119,115,82,101,103,105,115,116,114, 121,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, @@ -907,7 +907,7 @@ 11,99,108,97,115,115,109,101,116,104,111,100,114,169,0,0, 0,114,175,0,0,0,114,178,0,0,0,114,179,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,166,0,0,0,74,2,0,0,115,20,0, + 6,0,0,0,114,166,0,0,0,76,2,0,0,115,20,0, 0,0,8,2,4,3,4,3,4,2,4,2,12,7,12,15, 2,1,12,15,2,1,114,166,0,0,0,99,0,0,0,0, 0,0,0,0,0,0,0,0,2,0,0,0,64,0,0,0, @@ -942,7 +942,7 @@ 0,114,123,0,0,0,114,98,0,0,0,90,13,102,105,108, 101,110,97,109,101,95,98,97,115,101,90,9,116,97,105,108, 95,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,157,0,0,0,143,2,0,0,115,8,0, + 6,0,0,0,114,157,0,0,0,145,2,0,0,115,8,0, 0,0,0,3,18,1,16,1,14,1,122,24,95,76,111,97, 100,101,114,66,97,115,105,99,115,46,105,115,95,112,97,99, 107,97,103,101,99,2,0,0,0,0,0,0,0,2,0,0, @@ -953,7 +953,7 @@ 78,114,4,0,0,0,41,2,114,104,0,0,0,114,162,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, 0,218,13,99,114,101,97,116,101,95,109,111,100,117,108,101, - 151,2,0,0,115,0,0,0,0,122,27,95,76,111,97,100, + 153,2,0,0,115,0,0,0,0,122,27,95,76,111,97,100, 101,114,66,97,115,105,99,115,46,99,114,101,97,116,101,95, 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,3, 0,0,0,4,0,0,0,67,0,0,0,115,56,0,0,0, @@ -972,7 +972,7 @@ 100,218,4,101,120,101,99,114,115,0,0,0,41,3,114,104, 0,0,0,218,6,109,111,100,117,108,101,114,144,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 11,101,120,101,99,95,109,111,100,117,108,101,154,2,0,0, + 11,101,120,101,99,95,109,111,100,117,108,101,156,2,0,0, 115,10,0,0,0,0,2,12,1,8,1,6,1,10,1,122, 25,95,76,111,97,100,101,114,66,97,115,105,99,115,46,101, 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, @@ -984,13 +984,13 @@ 117,108,101,95,115,104,105,109,41,2,114,104,0,0,0,114, 123,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, 0,0,0,218,11,108,111,97,100,95,109,111,100,117,108,101, - 162,2,0,0,115,2,0,0,0,0,2,122,25,95,76,111, + 164,2,0,0,115,2,0,0,0,0,2,122,25,95,76,111, 97,100,101,114,66,97,115,105,99,115,46,108,111,97,100,95, 109,111,100,117,108,101,78,41,8,114,109,0,0,0,114,108, 0,0,0,114,110,0,0,0,114,111,0,0,0,114,157,0, 0,0,114,183,0,0,0,114,188,0,0,0,114,190,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,181,0,0,0,138,2,0,0,115,10, + 114,6,0,0,0,114,181,0,0,0,140,2,0,0,115,10, 0,0,0,8,3,4,2,8,8,8,3,8,8,114,181,0, 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,3, 0,0,0,64,0,0,0,115,74,0,0,0,101,0,90,1, @@ -1016,7 +1016,7 @@ 1,218,7,73,79,69,114,114,111,114,41,2,114,104,0,0, 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, 114,6,0,0,0,218,10,112,97,116,104,95,109,116,105,109, - 101,169,2,0,0,115,2,0,0,0,0,6,122,23,83,111, + 101,171,2,0,0,115,2,0,0,0,0,6,122,23,83,111, 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, 109,116,105,109,101,99,2,0,0,0,0,0,0,0,2,0, 0,0,3,0,0,0,67,0,0,0,115,14,0,0,0,100, @@ -1051,7 +1051,7 @@ 0,0,0,41,1,114,193,0,0,0,41,2,114,104,0,0, 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, 114,6,0,0,0,218,10,112,97,116,104,95,115,116,97,116, - 115,177,2,0,0,115,2,0,0,0,0,11,122,23,83,111, + 115,179,2,0,0,115,2,0,0,0,0,11,122,23,83,111, 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, 115,116,97,116,115,99,4,0,0,0,0,0,0,0,4,0, 0,0,3,0,0,0,67,0,0,0,115,12,0,0,0,124, @@ -1074,7 +1074,7 @@ 4,114,104,0,0,0,114,94,0,0,0,90,10,99,97,99, 104,101,95,112,97,116,104,114,56,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,15,95,99,97, - 99,104,101,95,98,121,116,101,99,111,100,101,190,2,0,0, + 99,104,101,95,98,121,116,101,99,111,100,101,192,2,0,0, 115,2,0,0,0,0,8,122,28,83,111,117,114,99,101,76, 111,97,100,101,114,46,95,99,97,99,104,101,95,98,121,116, 101,99,111,100,101,99,3,0,0,0,0,0,0,0,3,0, @@ -1091,7 +1091,7 @@ 108,101,115,46,10,32,32,32,32,32,32,32,32,78,114,4, 0,0,0,41,3,114,104,0,0,0,114,37,0,0,0,114, 56,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,195,0,0,0,200,2,0,0,115,0,0,0, + 0,0,0,114,195,0,0,0,202,2,0,0,115,0,0,0, 0,122,21,83,111,117,114,99,101,76,111,97,100,101,114,46, 115,101,116,95,100,97,116,97,99,2,0,0,0,0,0,0, 0,5,0,0,0,16,0,0,0,67,0,0,0,115,84,0, @@ -1112,7 +1112,7 @@ 104,0,0,0,114,123,0,0,0,114,37,0,0,0,114,151, 0,0,0,218,3,101,120,99,114,4,0,0,0,114,4,0, 0,0,114,6,0,0,0,218,10,103,101,116,95,115,111,117, - 114,99,101,207,2,0,0,115,14,0,0,0,0,2,10,1, + 114,99,101,209,2,0,0,115,14,0,0,0,0,2,10,1, 2,1,14,1,16,1,6,1,28,1,122,23,83,111,117,114, 99,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, 114,99,101,114,31,0,0,0,41,1,218,9,95,111,112,116, @@ -1134,7 +1134,7 @@ 104,0,0,0,114,56,0,0,0,114,37,0,0,0,114,200, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, 0,0,218,14,115,111,117,114,99,101,95,116,111,95,99,111, - 100,101,217,2,0,0,115,4,0,0,0,0,5,14,1,122, + 100,101,219,2,0,0,115,4,0,0,0,0,5,14,1,122, 27,83,111,117,114,99,101,76,111,97,100,101,114,46,115,111, 117,114,99,101,95,116,111,95,99,111,100,101,99,2,0,0, 0,0,0,0,0,10,0,0,0,43,0,0,0,67,0,0, @@ -1190,7 +1190,7 @@ 0,0,218,2,115,116,114,56,0,0,0,218,10,98,121,116, 101,115,95,100,97,116,97,114,151,0,0,0,90,11,99,111, 100,101,95,111,98,106,101,99,116,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,184,0,0,0,225,2,0, + 0,0,0,114,6,0,0,0,114,184,0,0,0,227,2,0, 0,115,78,0,0,0,0,7,10,1,4,1,2,1,12,1, 14,1,10,2,2,1,14,1,14,1,6,2,12,1,2,1, 14,1,14,1,6,2,2,1,6,1,8,1,12,1,18,1, @@ -1202,7 +1202,7 @@ 0,0,114,193,0,0,0,114,194,0,0,0,114,196,0,0, 0,114,195,0,0,0,114,199,0,0,0,114,203,0,0,0, 114,184,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,191,0,0,0,167,2, + 4,0,0,0,114,6,0,0,0,114,191,0,0,0,169,2, 0,0,115,14,0,0,0,8,2,8,8,8,13,8,10,8, 7,8,10,14,8,114,191,0,0,0,99,0,0,0,0,0, 0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,115, @@ -1229,7 +1229,7 @@ 78,41,2,114,102,0,0,0,114,37,0,0,0,41,3,114, 104,0,0,0,114,123,0,0,0,114,37,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,182,0, - 0,0,26,3,0,0,115,4,0,0,0,0,3,6,1,122, + 0,0,28,3,0,0,115,4,0,0,0,0,3,6,1,122, 19,70,105,108,101,76,111,97,100,101,114,46,95,95,105,110, 105,116,95,95,99,2,0,0,0,0,0,0,0,2,0,0, 0,2,0,0,0,67,0,0,0,115,24,0,0,0,124,0, @@ -1238,7 +1238,7 @@ 108,97,115,115,95,95,114,115,0,0,0,41,2,114,104,0, 0,0,218,5,111,116,104,101,114,114,4,0,0,0,114,4, 0,0,0,114,6,0,0,0,218,6,95,95,101,113,95,95, - 32,3,0,0,115,4,0,0,0,0,1,12,1,122,17,70, + 34,3,0,0,115,4,0,0,0,0,1,12,1,122,17,70, 105,108,101,76,111,97,100,101,114,46,95,95,101,113,95,95, 99,1,0,0,0,0,0,0,0,1,0,0,0,3,0,0, 0,67,0,0,0,115,20,0,0,0,116,0,124,0,106,1, @@ -1246,7 +1246,7 @@ 78,41,3,218,4,104,97,115,104,114,102,0,0,0,114,37, 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, 4,0,0,0,114,6,0,0,0,218,8,95,95,104,97,115, - 104,95,95,36,3,0,0,115,2,0,0,0,0,1,122,19, + 104,95,95,38,3,0,0,115,2,0,0,0,0,1,122,19, 70,105,108,101,76,111,97,100,101,114,46,95,95,104,97,115, 104,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, 3,0,0,0,3,0,0,0,115,16,0,0,0,116,0,116, @@ -1260,7 +1260,7 @@ 32,32,32,32,32,41,3,218,5,115,117,112,101,114,114,207, 0,0,0,114,190,0,0,0,41,2,114,104,0,0,0,114, 123,0,0,0,41,1,114,208,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,190,0,0,0,39,3,0,0,115,2, + 114,6,0,0,0,114,190,0,0,0,41,3,0,0,115,2, 0,0,0,0,10,122,22,70,105,108,101,76,111,97,100,101, 114,46,108,111,97,100,95,109,111,100,117,108,101,99,2,0, 0,0,0,0,0,0,2,0,0,0,1,0,0,0,67,0, @@ -1271,7 +1271,7 @@ 116,104,101,32,102,105,110,100,101,114,46,41,1,114,37,0, 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,155,0, - 0,0,51,3,0,0,115,2,0,0,0,0,3,122,23,70, + 0,0,53,3,0,0,115,2,0,0,0,0,3,122,23,70, 105,108,101,76,111,97,100,101,114,46,103,101,116,95,102,105, 108,101,110,97,109,101,99,2,0,0,0,0,0,0,0,3, 0,0,0,9,0,0,0,67,0,0,0,115,32,0,0,0, @@ -1283,14 +1283,14 @@ 3,114,52,0,0,0,114,53,0,0,0,90,4,114,101,97, 100,41,3,114,104,0,0,0,114,37,0,0,0,114,57,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,197,0,0,0,56,3,0,0,115,4,0,0,0,0, + 0,114,197,0,0,0,58,3,0,0,115,4,0,0,0,0, 2,14,1,122,19,70,105,108,101,76,111,97,100,101,114,46, 103,101,116,95,100,97,116,97,41,11,114,109,0,0,0,114, 108,0,0,0,114,110,0,0,0,114,111,0,0,0,114,182, 0,0,0,114,210,0,0,0,114,212,0,0,0,114,120,0, 0,0,114,190,0,0,0,114,155,0,0,0,114,197,0,0, 0,114,4,0,0,0,114,4,0,0,0,41,1,114,208,0, - 0,0,114,6,0,0,0,114,207,0,0,0,21,3,0,0, + 0,0,114,6,0,0,0,114,207,0,0,0,23,3,0,0, 115,14,0,0,0,8,3,4,2,8,6,8,4,8,3,16, 12,12,5,114,207,0,0,0,99,0,0,0,0,0,0,0, 0,0,0,0,0,3,0,0,0,64,0,0,0,115,46,0, @@ -1312,7 +1312,7 @@ 8,115,116,95,109,116,105,109,101,90,7,115,116,95,115,105, 122,101,41,3,114,104,0,0,0,114,37,0,0,0,114,205, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,194,0,0,0,66,3,0,0,115,4,0,0,0, + 0,0,114,194,0,0,0,68,3,0,0,115,4,0,0,0, 0,2,8,1,122,27,83,111,117,114,99,101,70,105,108,101, 76,111,97,100,101,114,46,112,97,116,104,95,115,116,97,116, 115,99,4,0,0,0,0,0,0,0,5,0,0,0,5,0, @@ -1322,7 +1322,7 @@ 2,114,101,0,0,0,114,195,0,0,0,41,5,114,104,0, 0,0,114,94,0,0,0,114,93,0,0,0,114,56,0,0, 0,114,44,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,196,0,0,0,71,3,0,0,115,4, + 114,6,0,0,0,114,196,0,0,0,73,3,0,0,115,4, 0,0,0,0,2,8,1,122,32,83,111,117,114,99,101,70, 105,108,101,76,111,97,100,101,114,46,95,99,97,99,104,101, 95,98,121,116,101,99,111,100,101,105,182,1,0,0,41,1, @@ -1357,7 +1357,7 @@ 0,114,217,0,0,0,218,6,112,97,114,101,110,116,114,98, 0,0,0,114,29,0,0,0,114,25,0,0,0,114,198,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,195,0,0,0,76,3,0,0,115,42,0,0,0,0, + 0,114,195,0,0,0,78,3,0,0,115,42,0,0,0,0, 2,12,1,4,2,16,1,12,1,14,2,14,1,10,1,2, 1,14,1,14,2,6,1,16,3,6,1,8,1,20,1,2, 1,12,1,16,1,16,2,8,1,122,25,83,111,117,114,99, @@ -1366,7 +1366,7 @@ 0,114,110,0,0,0,114,111,0,0,0,114,194,0,0,0, 114,196,0,0,0,114,195,0,0,0,114,4,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,215, - 0,0,0,62,3,0,0,115,8,0,0,0,8,2,4,2, + 0,0,0,64,3,0,0,115,8,0,0,0,8,2,4,2, 8,5,8,5,114,215,0,0,0,99,0,0,0,0,0,0, 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,32, 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, @@ -1386,7 +1386,7 @@ 145,0,0,0,41,5,114,104,0,0,0,114,123,0,0,0, 114,37,0,0,0,114,56,0,0,0,114,206,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,184, - 0,0,0,111,3,0,0,115,8,0,0,0,0,1,10,1, + 0,0,0,113,3,0,0,115,8,0,0,0,0,1,10,1, 10,1,18,1,122,29,83,111,117,114,99,101,108,101,115,115, 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, @@ -1396,13 +1396,13 @@ 115,111,117,114,99,101,32,99,111,100,101,46,78,114,4,0, 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, - 0,0,117,3,0,0,115,2,0,0,0,0,2,122,31,83, + 0,0,119,3,0,0,115,2,0,0,0,0,2,122,31,83, 111,117,114,99,101,108,101,115,115,70,105,108,101,76,111,97, 100,101,114,46,103,101,116,95,115,111,117,114,99,101,78,41, 6,114,109,0,0,0,114,108,0,0,0,114,110,0,0,0, 114,111,0,0,0,114,184,0,0,0,114,199,0,0,0,114, 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,220,0,0,0,107,3,0,0,115,6,0,0, + 0,0,0,114,220,0,0,0,109,3,0,0,115,6,0,0, 0,8,2,4,2,8,6,114,220,0,0,0,99,0,0,0, 0,0,0,0,0,0,0,0,0,3,0,0,0,64,0,0, 0,115,92,0,0,0,101,0,90,1,100,0,90,2,100,1, @@ -1424,7 +1424,7 @@ 83,0,41,1,78,41,2,114,102,0,0,0,114,37,0,0, 0,41,3,114,104,0,0,0,114,102,0,0,0,114,37,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,182,0,0,0,134,3,0,0,115,4,0,0,0,0, + 0,114,182,0,0,0,136,3,0,0,115,4,0,0,0,0, 1,6,1,122,28,69,120,116,101,110,115,105,111,110,70,105, 108,101,76,111,97,100,101,114,46,95,95,105,110,105,116,95, 95,99,2,0,0,0,0,0,0,0,2,0,0,0,2,0, @@ -1433,7 +1433,7 @@ 2,83,0,41,1,78,41,2,114,208,0,0,0,114,115,0, 0,0,41,2,114,104,0,0,0,114,209,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,210,0, - 0,0,138,3,0,0,115,4,0,0,0,0,1,12,1,122, + 0,0,140,3,0,0,115,4,0,0,0,0,1,12,1,122, 26,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, 97,100,101,114,46,95,95,101,113,95,95,99,1,0,0,0, 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, @@ -1441,7 +1441,7 @@ 0,106,2,131,1,65,0,83,0,41,1,78,41,3,114,211, 0,0,0,114,102,0,0,0,114,37,0,0,0,41,1,114, 104,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,212,0,0,0,142,3,0,0,115,2,0,0, + 0,0,0,114,212,0,0,0,144,3,0,0,115,2,0,0, 0,0,1,122,28,69,120,116,101,110,115,105,111,110,70,105, 108,101,76,111,97,100,101,114,46,95,95,104,97,115,104,95, 95,99,2,0,0,0,0,0,0,0,3,0,0,0,4,0, @@ -1458,7 +1458,7 @@ 105,99,114,133,0,0,0,114,102,0,0,0,114,37,0,0, 0,41,3,114,104,0,0,0,114,162,0,0,0,114,187,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,183,0,0,0,145,3,0,0,115,10,0,0,0,0, + 0,114,183,0,0,0,147,3,0,0,115,10,0,0,0,0, 2,4,1,10,1,6,1,12,1,122,33,69,120,116,101,110, 115,105,111,110,70,105,108,101,76,111,97,100,101,114,46,99, 114,101,97,116,101,95,109,111,100,117,108,101,99,2,0,0, @@ -1475,7 +1475,7 @@ 121,110,97,109,105,99,114,133,0,0,0,114,102,0,0,0, 114,37,0,0,0,41,2,114,104,0,0,0,114,187,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 114,188,0,0,0,153,3,0,0,115,6,0,0,0,0,2, + 114,188,0,0,0,155,3,0,0,115,6,0,0,0,0,2, 14,1,6,1,122,31,69,120,116,101,110,115,105,111,110,70, 105,108,101,76,111,97,100,101,114,46,101,120,101,99,95,109, 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, @@ -1492,7 +1492,7 @@ 0,41,2,114,182,0,0,0,78,114,4,0,0,0,41,2, 114,24,0,0,0,218,6,115,117,102,102,105,120,41,1,218, 9,102,105,108,101,95,110,97,109,101,114,4,0,0,0,114, - 6,0,0,0,250,9,60,103,101,110,101,120,112,114,62,162, + 6,0,0,0,250,9,60,103,101,110,101,120,112,114,62,164, 3,0,0,115,2,0,0,0,4,1,122,49,69,120,116,101, 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, 105,115,95,112,97,99,107,97,103,101,46,60,108,111,99,97, @@ -1501,7 +1501,7 @@ 69,88,84,69,78,83,73,79,78,95,83,85,70,70,73,88, 69,83,41,2,114,104,0,0,0,114,123,0,0,0,114,4, 0,0,0,41,1,114,223,0,0,0,114,6,0,0,0,114, - 157,0,0,0,159,3,0,0,115,6,0,0,0,0,2,14, + 157,0,0,0,161,3,0,0,115,6,0,0,0,0,2,14, 1,12,1,122,30,69,120,116,101,110,115,105,111,110,70,105, 108,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, 97,103,101,99,2,0,0,0,0,0,0,0,2,0,0,0, @@ -1512,7 +1512,7 @@ 114,101,97,116,101,32,97,32,99,111,100,101,32,111,98,106, 101,99,116,46,78,114,4,0,0,0,41,2,114,104,0,0, 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,184,0,0,0,165,3,0,0,115,2, + 114,6,0,0,0,114,184,0,0,0,167,3,0,0,115,2, 0,0,0,0,2,122,28,69,120,116,101,110,115,105,111,110, 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, @@ -1523,7 +1523,7 @@ 117,114,99,101,32,99,111,100,101,46,78,114,4,0,0,0, 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,114,199,0,0,0, - 169,3,0,0,115,2,0,0,0,0,2,122,30,69,120,116, + 171,3,0,0,115,2,0,0,0,0,2,122,30,69,120,116, 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, 46,103,101,116,95,115,111,117,114,99,101,99,2,0,0,0, 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, @@ -1534,7 +1534,7 @@ 101,32,102,105,110,100,101,114,46,41,1,114,37,0,0,0, 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,114,155,0,0,0, - 173,3,0,0,115,2,0,0,0,0,3,122,32,69,120,116, + 175,3,0,0,115,2,0,0,0,0,3,122,32,69,120,116, 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, 46,103,101,116,95,102,105,108,101,110,97,109,101,78,41,14, 114,109,0,0,0,114,108,0,0,0,114,110,0,0,0,114, @@ -1542,7 +1542,7 @@ 0,0,0,114,183,0,0,0,114,188,0,0,0,114,157,0, 0,0,114,184,0,0,0,114,199,0,0,0,114,120,0,0, 0,114,155,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,221,0,0,0,126, + 114,4,0,0,0,114,6,0,0,0,114,221,0,0,0,128, 3,0,0,115,20,0,0,0,8,6,4,2,8,4,8,4, 8,3,8,8,8,6,8,6,8,4,8,4,114,221,0,0, 0,99,0,0,0,0,0,0,0,0,0,0,0,0,2,0, @@ -1584,7 +1584,7 @@ 114,41,4,114,104,0,0,0,114,102,0,0,0,114,37,0, 0,0,218,11,112,97,116,104,95,102,105,110,100,101,114,114, 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,182, - 0,0,0,186,3,0,0,115,8,0,0,0,0,1,6,1, + 0,0,0,188,3,0,0,115,8,0,0,0,0,1,6,1, 6,1,14,1,122,23,95,78,97,109,101,115,112,97,99,101, 80,97,116,104,46,95,95,105,110,105,116,95,95,99,1,0, 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, @@ -1602,7 +1602,7 @@ 0,0,218,3,100,111,116,90,2,109,101,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,23,95,102,105,110, 100,95,112,97,114,101,110,116,95,112,97,116,104,95,110,97, - 109,101,115,192,3,0,0,115,8,0,0,0,0,2,18,1, + 109,101,115,194,3,0,0,115,8,0,0,0,0,2,18,1, 8,2,4,3,122,38,95,78,97,109,101,115,112,97,99,101, 80,97,116,104,46,95,102,105,110,100,95,112,97,114,101,110, 116,95,112,97,116,104,95,110,97,109,101,115,99,1,0,0, @@ -1614,7 +1614,7 @@ 3,114,104,0,0,0,90,18,112,97,114,101,110,116,95,109, 111,100,117,108,101,95,110,97,109,101,90,14,112,97,116,104, 95,97,116,116,114,95,110,97,109,101,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,230,0,0,0,202,3, + 4,0,0,0,114,6,0,0,0,114,230,0,0,0,204,3, 0,0,115,4,0,0,0,0,1,12,1,122,31,95,78,97, 109,101,115,112,97,99,101,80,97,116,104,46,95,103,101,116, 95,112,97,114,101,110,116,95,112,97,116,104,99,1,0,0, @@ -1630,7 +1630,7 @@ 0,0,0,41,3,114,104,0,0,0,90,11,112,97,114,101, 110,116,95,112,97,116,104,114,162,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,12,95,114,101, - 99,97,108,99,117,108,97,116,101,206,3,0,0,115,16,0, + 99,97,108,99,117,108,97,116,101,208,3,0,0,115,16,0, 0,0,0,2,12,1,10,1,14,3,18,1,6,1,8,1, 6,1,122,27,95,78,97,109,101,115,112,97,99,101,80,97, 116,104,46,95,114,101,99,97,108,99,117,108,97,116,101,99, @@ -1639,7 +1639,7 @@ 0,131,1,83,0,41,1,78,41,2,218,4,105,116,101,114, 114,237,0,0,0,41,1,114,104,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,8,95,95,105, - 116,101,114,95,95,219,3,0,0,115,2,0,0,0,0,1, + 116,101,114,95,95,221,3,0,0,115,2,0,0,0,0,1, 122,23,95,78,97,109,101,115,112,97,99,101,80,97,116,104, 46,95,95,105,116,101,114,95,95,99,3,0,0,0,0,0, 0,0,3,0,0,0,3,0,0,0,67,0,0,0,115,14, @@ -1647,7 +1647,7 @@ 0,41,1,78,41,1,114,229,0,0,0,41,3,114,104,0, 0,0,218,5,105,110,100,101,120,114,37,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,218,11,95, - 95,115,101,116,105,116,101,109,95,95,222,3,0,0,115,2, + 95,115,101,116,105,116,101,109,95,95,224,3,0,0,115,2, 0,0,0,0,1,122,26,95,78,97,109,101,115,112,97,99, 101,80,97,116,104,46,95,95,115,101,116,105,116,101,109,95, 95,99,1,0,0,0,0,0,0,0,1,0,0,0,2,0, @@ -1655,7 +1655,7 @@ 1,131,0,131,1,83,0,41,1,78,41,2,114,33,0,0, 0,114,237,0,0,0,41,1,114,104,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,7,95,95, - 108,101,110,95,95,225,3,0,0,115,2,0,0,0,0,1, + 108,101,110,95,95,227,3,0,0,115,2,0,0,0,0,1, 122,22,95,78,97,109,101,115,112,97,99,101,80,97,116,104, 46,95,95,108,101,110,95,95,99,1,0,0,0,0,0,0, 0,1,0,0,0,2,0,0,0,67,0,0,0,115,12,0, @@ -1664,7 +1664,7 @@ 104,40,123,33,114,125,41,41,2,114,50,0,0,0,114,229, 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, 4,0,0,0,114,6,0,0,0,218,8,95,95,114,101,112, - 114,95,95,228,3,0,0,115,2,0,0,0,0,1,122,23, + 114,95,95,230,3,0,0,115,2,0,0,0,0,1,122,23, 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, 95,114,101,112,114,95,95,99,2,0,0,0,0,0,0,0, 2,0,0,0,2,0,0,0,67,0,0,0,115,12,0,0, @@ -1672,7 +1672,7 @@ 41,1,114,237,0,0,0,41,2,114,104,0,0,0,218,4, 105,116,101,109,114,4,0,0,0,114,4,0,0,0,114,6, 0,0,0,218,12,95,95,99,111,110,116,97,105,110,115,95, - 95,231,3,0,0,115,2,0,0,0,0,1,122,27,95,78, + 95,233,3,0,0,115,2,0,0,0,0,1,122,27,95,78, 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,99, 111,110,116,97,105,110,115,95,95,99,2,0,0,0,0,0, 0,0,2,0,0,0,2,0,0,0,67,0,0,0,115,16, @@ -1680,7 +1680,7 @@ 0,83,0,41,1,78,41,2,114,229,0,0,0,114,161,0, 0,0,41,2,114,104,0,0,0,114,244,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,161,0, - 0,0,234,3,0,0,115,2,0,0,0,0,1,122,21,95, + 0,0,236,3,0,0,115,2,0,0,0,0,1,122,21,95, 78,97,109,101,115,112,97,99,101,80,97,116,104,46,97,112, 112,101,110,100,78,41,14,114,109,0,0,0,114,108,0,0, 0,114,110,0,0,0,114,111,0,0,0,114,182,0,0,0, @@ -1688,7 +1688,7 @@ 239,0,0,0,114,241,0,0,0,114,242,0,0,0,114,243, 0,0,0,114,245,0,0,0,114,161,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,227,0,0,0,179,3,0,0,115,22,0,0,0,8, + 0,114,227,0,0,0,181,3,0,0,115,22,0,0,0,8, 5,4,2,8,6,8,10,8,4,8,13,8,3,8,3,8, 3,8,3,8,3,114,227,0,0,0,99,0,0,0,0,0, 0,0,0,0,0,0,0,3,0,0,0,64,0,0,0,115, @@ -1704,7 +1704,7 @@ 100,0,83,0,41,1,78,41,2,114,227,0,0,0,114,229, 0,0,0,41,4,114,104,0,0,0,114,102,0,0,0,114, 37,0,0,0,114,233,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,182,0,0,0,240,3,0, + 0,0,0,114,6,0,0,0,114,182,0,0,0,242,3,0, 0,115,2,0,0,0,0,1,122,25,95,78,97,109,101,115, 112,97,99,101,76,111,97,100,101,114,46,95,95,105,110,105, 116,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, @@ -1721,14 +1721,14 @@ 110,97,109,101,115,112,97,99,101,41,62,41,2,114,50,0, 0,0,114,109,0,0,0,41,2,114,168,0,0,0,114,187, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,109,111,100,117,108,101,95,114,101,112,114,243, + 0,0,218,11,109,111,100,117,108,101,95,114,101,112,114,245, 3,0,0,115,2,0,0,0,0,7,122,28,95,78,97,109, 101,115,112,97,99,101,76,111,97,100,101,114,46,109,111,100, 117,108,101,95,114,101,112,114,99,2,0,0,0,0,0,0, 0,2,0,0,0,1,0,0,0,67,0,0,0,115,4,0, 0,0,100,1,83,0,41,2,78,84,114,4,0,0,0,41, 2,114,104,0,0,0,114,123,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,157,0,0,0,252, + 114,4,0,0,0,114,6,0,0,0,114,157,0,0,0,254, 3,0,0,115,2,0,0,0,0,1,122,27,95,78,97,109, 101,115,112,97,99,101,76,111,97,100,101,114,46,105,115,95, 112,97,99,107,97,103,101,99,2,0,0,0,0,0,0,0, @@ -1736,7 +1736,7 @@ 0,100,1,83,0,41,2,78,114,32,0,0,0,114,4,0, 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, - 0,0,255,3,0,0,115,2,0,0,0,0,1,122,27,95, + 0,0,1,4,0,0,115,2,0,0,0,0,1,122,27,95, 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, 0,0,0,2,0,0,0,6,0,0,0,67,0,0,0,115, @@ -1745,7 +1745,7 @@ 60,115,116,114,105,110,103,62,114,186,0,0,0,114,201,0, 0,0,84,41,1,114,202,0,0,0,41,2,114,104,0,0, 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,184,0,0,0,2,4,0,0,115,2, + 114,6,0,0,0,114,184,0,0,0,4,4,0,0,115,2, 0,0,0,0,1,122,25,95,78,97,109,101,115,112,97,99, 101,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, @@ -1755,14 +1755,14 @@ 108,101,32,99,114,101,97,116,105,111,110,46,78,114,4,0, 0,0,41,2,114,104,0,0,0,114,162,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,183,0, - 0,0,5,4,0,0,115,0,0,0,0,122,30,95,78,97, + 0,0,7,4,0,0,115,0,0,0,0,122,30,95,78,97, 109,101,115,112,97,99,101,76,111,97,100,101,114,46,99,114, 101,97,116,101,95,109,111,100,117,108,101,99,2,0,0,0, 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, 115,4,0,0,0,100,0,83,0,41,1,78,114,4,0,0, 0,41,2,114,104,0,0,0,114,187,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,114,188,0,0, - 0,8,4,0,0,115,2,0,0,0,0,1,122,28,95,78, + 0,10,4,0,0,115,2,0,0,0,0,1,122,28,95,78, 97,109,101,115,112,97,99,101,76,111,97,100,101,114,46,101, 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, @@ -1780,7 +1780,7 @@ 0,0,0,114,133,0,0,0,114,229,0,0,0,114,189,0, 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,190,0, - 0,0,11,4,0,0,115,6,0,0,0,0,7,6,1,8, + 0,0,13,4,0,0,115,6,0,0,0,0,7,6,1,8, 1,122,28,95,78,97,109,101,115,112,97,99,101,76,111,97, 100,101,114,46,108,111,97,100,95,109,111,100,117,108,101,78, 41,12,114,109,0,0,0,114,108,0,0,0,114,110,0,0, @@ -1788,7 +1788,7 @@ 114,157,0,0,0,114,199,0,0,0,114,184,0,0,0,114, 183,0,0,0,114,188,0,0,0,114,190,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,246,0,0,0,239,3,0,0,115,16,0,0,0, + 0,0,114,246,0,0,0,241,3,0,0,115,16,0,0,0, 8,1,8,3,12,9,8,3,8,3,8,3,8,3,8,3, 114,246,0,0,0,99,0,0,0,0,0,0,0,0,0,0, 0,0,4,0,0,0,64,0,0,0,115,106,0,0,0,101, @@ -1822,7 +1822,7 @@ 108,117,101,115,114,112,0,0,0,114,249,0,0,0,41,2, 114,168,0,0,0,218,6,102,105,110,100,101,114,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, - 0,29,4,0,0,115,6,0,0,0,0,4,16,1,10,1, + 0,31,4,0,0,115,6,0,0,0,0,4,16,1,10,1, 122,28,80,97,116,104,70,105,110,100,101,114,46,105,110,118, 97,108,105,100,97,116,101,95,99,97,99,104,101,115,99,2, 0,0,0,0,0,0,0,3,0,0,0,12,0,0,0,67, @@ -1841,7 +1841,7 @@ 0,0,114,64,0,0,0,114,122,0,0,0,114,103,0,0, 0,41,3,114,168,0,0,0,114,37,0,0,0,90,4,104, 111,111,107,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,95,112,97,116,104,95,104,111,111,107,115,37, + 0,0,218,11,95,112,97,116,104,95,104,111,111,107,115,39, 4,0,0,115,16,0,0,0,0,3,18,1,12,1,12,1, 2,1,8,1,14,1,12,2,122,22,80,97,116,104,70,105, 110,100,101,114,46,95,112,97,116,104,95,104,111,111,107,115, @@ -1873,7 +1873,7 @@ 3,114,168,0,0,0,114,37,0,0,0,114,252,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, 20,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, - 99,97,99,104,101,50,4,0,0,115,22,0,0,0,0,8, + 99,97,99,104,101,52,4,0,0,115,22,0,0,0,0,8, 8,1,2,1,12,1,14,3,6,1,2,1,14,1,14,1, 10,1,16,1,122,31,80,97,116,104,70,105,110,100,101,114, 46,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, @@ -1890,7 +1890,7 @@ 114,168,0,0,0,114,123,0,0,0,114,252,0,0,0,114, 124,0,0,0,114,125,0,0,0,114,162,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,218,16,95, - 108,101,103,97,99,121,95,103,101,116,95,115,112,101,99,72, + 108,101,103,97,99,121,95,103,101,116,95,115,112,101,99,74, 4,0,0,115,18,0,0,0,0,4,10,1,16,2,10,1, 4,1,8,1,12,1,12,1,6,1,122,27,80,97,116,104, 70,105,110,100,101,114,46,95,108,101,103,97,99,121,95,103, @@ -1922,7 +1922,7 @@ 95,112,97,116,104,90,5,101,110,116,114,121,114,252,0,0, 0,114,162,0,0,0,114,125,0,0,0,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,9,95,103,101,116, - 95,115,112,101,99,87,4,0,0,115,40,0,0,0,0,5, + 95,115,112,101,99,89,4,0,0,115,40,0,0,0,0,5, 4,1,10,1,14,1,2,1,10,1,8,1,10,1,14,2, 12,1,8,1,2,1,10,1,4,1,6,1,8,1,8,5, 14,2,12,1,6,1,122,20,80,97,116,104,70,105,110,100, @@ -1950,7 +1950,7 @@ 41,6,114,168,0,0,0,114,123,0,0,0,114,37,0,0, 0,114,177,0,0,0,114,162,0,0,0,114,3,1,0,0, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, - 178,0,0,0,119,4,0,0,115,26,0,0,0,0,6,8, + 178,0,0,0,121,4,0,0,115,26,0,0,0,0,6,8, 1,6,1,14,1,8,1,6,1,10,1,6,1,4,3,6, 1,16,1,6,2,6,2,122,20,80,97,116,104,70,105,110, 100,101,114,46,102,105,110,100,95,115,112,101,99,99,3,0, @@ -1971,7 +1971,7 @@ 32,32,32,78,41,2,114,178,0,0,0,114,124,0,0,0, 41,4,114,168,0,0,0,114,123,0,0,0,114,37,0,0, 0,114,162,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,179,0,0,0,143,4,0,0,115,8, + 114,6,0,0,0,114,179,0,0,0,145,4,0,0,115,8, 0,0,0,0,8,12,1,8,1,4,1,122,22,80,97,116, 104,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, 117,108,101,41,1,78,41,2,78,78,41,1,78,41,12,114, @@ -1980,7 +1980,7 @@ 0,0,114,0,1,0,0,114,1,1,0,0,114,4,1,0, 0,114,178,0,0,0,114,179,0,0,0,114,4,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, - 248,0,0,0,25,4,0,0,115,22,0,0,0,8,2,4, + 248,0,0,0,27,4,0,0,115,22,0,0,0,8,2,4, 2,12,8,12,13,12,22,12,15,2,1,12,31,2,1,12, 23,2,1,114,248,0,0,0,99,0,0,0,0,0,0,0, 0,0,0,0,0,3,0,0,0,64,0,0,0,115,90,0, @@ -2024,7 +2024,7 @@ 1,0,113,2,100,0,83,0,41,1,78,114,4,0,0,0, 41,2,114,24,0,0,0,114,222,0,0,0,41,1,114,124, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,224,0, - 0,0,172,4,0,0,115,2,0,0,0,4,0,122,38,70, + 0,0,174,4,0,0,115,2,0,0,0,4,0,122,38,70, 105,108,101,70,105,110,100,101,114,46,95,95,105,110,105,116, 95,95,46,60,108,111,99,97,108,115,62,46,60,103,101,110, 101,120,112,114,62,114,61,0,0,0,114,31,0,0,0,78, @@ -2036,7 +2036,7 @@ 5,114,104,0,0,0,114,37,0,0,0,218,14,108,111,97, 100,101,114,95,100,101,116,97,105,108,115,90,7,108,111,97, 100,101,114,115,114,164,0,0,0,114,4,0,0,0,41,1, - 114,124,0,0,0,114,6,0,0,0,114,182,0,0,0,166, + 114,124,0,0,0,114,6,0,0,0,114,182,0,0,0,168, 4,0,0,115,16,0,0,0,0,4,4,1,14,1,28,1, 6,2,10,1,6,1,8,1,122,19,70,105,108,101,70,105, 110,100,101,114,46,95,95,105,110,105,116,95,95,99,1,0, @@ -2047,7 +2047,7 @@ 105,109,101,46,114,31,0,0,0,78,114,91,0,0,0,41, 1,114,7,1,0,0,41,1,114,104,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, - 0,180,4,0,0,115,2,0,0,0,0,2,122,28,70,105, + 0,182,4,0,0,115,2,0,0,0,0,2,122,28,70,105, 108,101,70,105,110,100,101,114,46,105,110,118,97,108,105,100, 97,116,101,95,99,97,99,104,101,115,99,2,0,0,0,0, 0,0,0,3,0,0,0,2,0,0,0,67,0,0,0,115, @@ -2069,7 +2069,7 @@ 32,32,32,32,32,32,32,78,41,3,114,178,0,0,0,114, 124,0,0,0,114,154,0,0,0,41,3,114,104,0,0,0, 114,123,0,0,0,114,162,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,121,0,0,0,186,4, + 4,0,0,0,114,6,0,0,0,114,121,0,0,0,188,4, 0,0,115,8,0,0,0,0,7,10,1,8,1,8,1,122, 22,70,105,108,101,70,105,110,100,101,114,46,102,105,110,100, 95,108,111,97,100,101,114,99,6,0,0,0,0,0,0,0, @@ -2080,7 +2080,7 @@ 0,0,0,41,7,114,104,0,0,0,114,163,0,0,0,114, 123,0,0,0,114,37,0,0,0,90,4,115,109,115,108,114, 177,0,0,0,114,124,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,4,1,0,0,198,4,0, + 0,0,0,114,6,0,0,0,114,4,1,0,0,200,4,0, 0,115,6,0,0,0,0,1,10,1,12,1,122,20,70,105, 108,101,70,105,110,100,101,114,46,95,103,101,116,95,115,112, 101,99,78,99,3,0,0,0,0,0,0,0,14,0,0,0, @@ -2135,7 +2135,7 @@ 0,0,0,90,13,105,110,105,116,95,102,105,108,101,110,97, 109,101,90,9,102,117,108,108,95,112,97,116,104,114,162,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,178,0,0,0,203,4,0,0,115,70,0,0,0,0, + 0,114,178,0,0,0,205,4,0,0,115,70,0,0,0,0, 5,4,1,14,1,2,1,24,1,14,1,10,1,10,1,8, 1,6,2,6,1,6,1,10,2,6,1,4,2,8,1,12, 1,16,1,8,1,10,1,8,1,24,4,8,2,16,1,16, @@ -2166,7 +2166,7 @@ 125,1,124,1,106,0,131,0,146,2,113,4,83,0,114,4, 0,0,0,41,1,114,92,0,0,0,41,2,114,24,0,0, 0,90,2,102,110,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,250,9,60,115,101,116,99,111,109,112,62,24, + 6,0,0,0,250,9,60,115,101,116,99,111,109,112,62,26, 5,0,0,115,2,0,0,0,6,0,122,41,70,105,108,101, 70,105,110,100,101,114,46,95,102,105,108,108,95,99,97,99, 104,101,46,60,108,111,99,97,108,115,62,46,60,115,101,116, @@ -2184,7 +2184,7 @@ 0,0,0,114,102,0,0,0,114,234,0,0,0,114,222,0, 0,0,90,8,110,101,119,95,110,97,109,101,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,114,12,1,0,0, - 251,4,0,0,115,34,0,0,0,0,2,6,1,2,1,22, + 253,4,0,0,115,34,0,0,0,0,2,6,1,2,1,22, 1,20,3,10,3,12,1,12,7,6,1,10,1,16,1,4, 1,18,2,4,1,14,1,6,1,12,1,122,22,70,105,108, 101,70,105,110,100,101,114,46,95,102,105,108,108,95,99,97, @@ -2221,7 +2221,7 @@ 0,114,103,0,0,0,41,1,114,37,0,0,0,41,2,114, 168,0,0,0,114,11,1,0,0,114,4,0,0,0,114,6, 0,0,0,218,24,112,97,116,104,95,104,111,111,107,95,102, - 111,114,95,70,105,108,101,70,105,110,100,101,114,36,5,0, + 111,114,95,70,105,108,101,70,105,110,100,101,114,38,5,0, 0,115,6,0,0,0,0,2,8,1,14,1,122,54,70,105, 108,101,70,105,110,100,101,114,46,112,97,116,104,95,104,111, 111,107,46,60,108,111,99,97,108,115,62,46,112,97,116,104, @@ -2229,7 +2229,7 @@ 110,100,101,114,114,4,0,0,0,41,3,114,168,0,0,0, 114,11,1,0,0,114,17,1,0,0,114,4,0,0,0,41, 2,114,168,0,0,0,114,11,1,0,0,114,6,0,0,0, - 218,9,112,97,116,104,95,104,111,111,107,26,5,0,0,115, + 218,9,112,97,116,104,95,104,111,111,107,28,5,0,0,115, 4,0,0,0,0,10,14,6,122,20,70,105,108,101,70,105, 110,100,101,114,46,112,97,116,104,95,104,111,111,107,99,1, 0,0,0,0,0,0,0,1,0,0,0,2,0,0,0,67, @@ -2238,7 +2238,7 @@ 100,101,114,40,123,33,114,125,41,41,2,114,50,0,0,0, 114,37,0,0,0,41,1,114,104,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,114,243,0,0,0, - 44,5,0,0,115,2,0,0,0,0,1,122,19,70,105,108, + 46,5,0,0,115,2,0,0,0,0,1,122,19,70,105,108, 101,70,105,110,100,101,114,46,95,95,114,101,112,114,95,95, 41,1,78,41,15,114,109,0,0,0,114,108,0,0,0,114, 110,0,0,0,114,111,0,0,0,114,182,0,0,0,114,249, @@ -2246,7 +2246,7 @@ 0,0,114,4,1,0,0,114,178,0,0,0,114,12,1,0, 0,114,180,0,0,0,114,18,1,0,0,114,243,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,5,1,0,0,157,4,0,0,115,20,0, + 6,0,0,0,114,5,1,0,0,159,4,0,0,115,20,0, 0,0,8,7,4,2,8,14,8,4,4,2,8,12,8,5, 10,48,8,31,12,18,114,5,1,0,0,99,4,0,0,0, 0,0,0,0,6,0,0,0,11,0,0,0,67,0,0,0, @@ -2269,7 +2269,7 @@ 112,97,116,104,110,97,109,101,90,9,99,112,97,116,104,110, 97,109,101,114,124,0,0,0,114,162,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,14,95,102, - 105,120,95,117,112,95,109,111,100,117,108,101,50,5,0,0, + 105,120,95,117,112,95,109,111,100,117,108,101,52,5,0,0, 115,34,0,0,0,0,2,10,1,10,1,4,1,4,1,8, 1,8,1,12,2,10,1,4,1,16,1,2,1,8,1,8, 1,8,1,12,1,14,2,114,23,1,0,0,99,0,0,0, @@ -2289,7 +2289,7 @@ 0,0,0,41,3,90,10,101,120,116,101,110,115,105,111,110, 115,90,6,115,111,117,114,99,101,90,8,98,121,116,101,99, 111,100,101,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,159,0,0,0,73,5,0,0,115,8,0,0,0, + 0,0,114,159,0,0,0,75,5,0,0,115,8,0,0,0, 0,5,12,1,8,1,8,1,114,159,0,0,0,99,1,0, 0,0,0,0,0,0,12,0,0,0,12,0,0,0,67,0, 0,0,115,188,1,0,0,124,0,97,0,116,0,106,1,97, @@ -2342,7 +2342,7 @@ 83,0,41,2,114,31,0,0,0,78,41,1,114,33,0,0, 0,41,2,114,24,0,0,0,114,81,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,114,224,0,0, - 0,109,5,0,0,115,2,0,0,0,4,0,122,25,95,115, + 0,111,5,0,0,115,2,0,0,0,4,0,122,25,95,115, 101,116,117,112,46,60,108,111,99,97,108,115,62,46,60,103, 101,110,101,120,112,114,62,114,62,0,0,0,122,30,105,109, 112,111,114,116,108,105,98,32,114,101,113,117,105,114,101,115, @@ -2372,7 +2372,7 @@ 101,90,14,119,101,97,107,114,101,102,95,109,111,100,117,108, 101,90,13,119,105,110,114,101,103,95,109,111,100,117,108,101, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 6,95,115,101,116,117,112,84,5,0,0,115,82,0,0,0, + 6,95,115,101,116,117,112,86,5,0,0,115,82,0,0,0, 0,8,4,1,6,1,6,3,10,1,10,1,10,1,12,2, 10,1,16,3,22,1,14,2,22,1,8,1,10,1,10,1, 4,2,2,1,10,1,6,1,14,1,12,2,8,1,12,1, @@ -2396,7 +2396,7 @@ 0,114,215,0,0,0,41,2,114,31,1,0,0,90,17,115, 117,112,112,111,114,116,101,100,95,108,111,97,100,101,114,115, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 8,95,105,110,115,116,97,108,108,152,5,0,0,115,16,0, + 8,95,105,110,115,116,97,108,108,154,5,0,0,115,16,0, 0,0,0,2,8,1,6,1,20,1,10,1,12,1,12,4, 6,1,114,34,1,0,0,41,1,122,3,119,105,110,41,2, 114,1,0,0,0,114,2,0,0,0,41,1,114,49,0,0, @@ -2430,7 +2430,7 @@ 0,0,0,114,4,0,0,0,114,6,0,0,0,218,8,60, 109,111,100,117,108,101,62,8,0,0,0,115,108,0,0,0, 4,16,4,1,4,1,2,1,6,3,8,17,8,5,8,5, - 8,6,8,12,8,10,8,9,8,5,8,7,10,22,10,117, + 8,6,8,12,8,10,8,9,8,5,8,7,10,22,10,119, 16,1,12,2,4,1,4,2,6,2,6,2,8,2,16,45, 8,34,8,19,8,12,8,12,8,28,8,17,10,55,10,12, 10,10,8,14,6,3,4,1,14,67,14,64,14,29,16,110, diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -84,7 +84,7 @@ &&TARGET_WITH_CLEANUP_FINISH, &&TARGET_RETURN_VALUE, &&TARGET_IMPORT_STAR, - &&_unknown_opcode, + &&TARGET_SETUP_ANNOTATIONS, &&TARGET_YIELD_VALUE, &&TARGET_POP_BLOCK, &&TARGET_END_FINALLY, @@ -126,7 +126,7 @@ &&TARGET_LOAD_FAST, &&TARGET_STORE_FAST, &&TARGET_DELETE_FAST, - &&_unknown_opcode, + &&TARGET_STORE_ANNOTATION, &&_unknown_opcode, &&_unknown_opcode, &&TARGET_RAISE_VARARGS, diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -937,11 +937,17 @@ static void initmain(PyInterpreterState *interp) { - PyObject *m, *d, *loader; + PyObject *m, *d, *loader, *ann_dict; m = PyImport_AddModule("__main__"); if (m == NULL) Py_FatalError("can't create __main__ module"); d = PyModule_GetDict(m); + ann_dict = PyDict_New(); + if ((ann_dict == NULL) || + (PyDict_SetItemString(d, "__annotations__", ann_dict) < 0)) { + Py_FatalError("Failed to initialize __main__.__annotations__"); + } + Py_DECREF(ann_dict); if (PyDict_GetItemString(d, "__builtins__") == NULL) { PyObject *bimod = PyImport_ImportModule("builtins"); if (bimod == NULL) { diff --git a/Python/symtable.c b/Python/symtable.c --- a/Python/symtable.c +++ b/Python/symtable.c @@ -1201,6 +1201,44 @@ VISIT_SEQ(st, expr, s->v.Assign.targets); VISIT(st, expr, s->v.Assign.value); break; + case AnnAssign_kind: + if (s->v.AnnAssign.target->kind == Name_kind) { + expr_ty e_name = s->v.AnnAssign.target; + long cur = symtable_lookup(st, e_name->v.Name.id); + if (cur < 0) { + VISIT_QUIT(st, 0); + } + if ((cur & (DEF_GLOBAL | DEF_NONLOCAL)) + && s->v.AnnAssign.simple) { + PyErr_Format(PyExc_SyntaxError, + "annotated name '%U' can't be %s", + e_name->v.Name.id, + cur & DEF_GLOBAL ? "global" : "nonlocal"); + PyErr_SyntaxLocationObject(st->st_filename, + s->lineno, + s->col_offset); + VISIT_QUIT(st, 0); + } + if (s->v.AnnAssign.simple && + !symtable_add_def(st, e_name->v.Name.id, + DEF_ANNOT | DEF_LOCAL)) { + VISIT_QUIT(st, 0); + } + else { + if (s->v.AnnAssign.value + && !symtable_add_def(st, e_name->v.Name.id, DEF_LOCAL)) { + VISIT_QUIT(st, 0); + } + } + } + else { + VISIT(st, expr, s->v.AnnAssign.target); + } + VISIT(st, expr, s->v.AnnAssign.annotation); + if (s->v.AnnAssign.value) { + VISIT(st, expr, s->v.AnnAssign.value); + } + break; case AugAssign_kind: VISIT(st, expr, s->v.AugAssign.target); VISIT(st, expr, s->v.AugAssign.value); @@ -1258,6 +1296,15 @@ long cur = symtable_lookup(st, name); if (cur < 0) VISIT_QUIT(st, 0); + if (cur & DEF_ANNOT) { + PyErr_Format(PyExc_SyntaxError, + "annotated name '%U' can't be global", + name); + PyErr_SyntaxLocationObject(st->st_filename, + s->lineno, + s->col_offset); + VISIT_QUIT(st, 0); + } if (cur & (DEF_LOCAL | USE)) { char buf[256]; char *c_name = _PyUnicode_AsString(name); @@ -1289,6 +1336,15 @@ long cur = symtable_lookup(st, name); if (cur < 0) VISIT_QUIT(st, 0); + if (cur & DEF_ANNOT) { + PyErr_Format(PyExc_SyntaxError, + "annotated name '%U' can't be nonlocal", + name); + PyErr_SyntaxLocationObject(st->st_filename, + s->lineno, + s->col_offset); + VISIT_QUIT(st, 0); + } if (cur & (DEF_LOCAL | USE)) { char buf[256]; char *c_name = _PyUnicode_AsString(name); diff --git a/Tools/parser/com2ann.py b/Tools/parser/com2ann.py new file mode 100644 --- /dev/null +++ b/Tools/parser/com2ann.py @@ -0,0 +1,308 @@ +"""Helper module to tranlate 3.5 type comments to 3.6 variable annotations.""" +import re +import os +import ast +import argparse +import tokenize +from collections import defaultdict +from textwrap import dedent +from io import BytesIO + +__all__ = ['com2ann', 'TYPE_COM'] + +TYPE_COM = re.compile('\s*#\s*type\s*:.*$', flags=re.DOTALL) +TRAIL_OR_COM = re.compile('\s*$|\s*#.*$', flags=re.DOTALL) + + +class _Data: + """Internal class describing global data on file.""" + def __init__(self, lines, tokens): + self.lines = lines + self.tokens = tokens + ttab = defaultdict(list) # maps line number to token numbers + for i, tok in enumerate(tokens): + ttab[tok.start[0]].append(i) + self.ttab = ttab + self.success = [] # list of lines where type comments where processed + self.fail = [] # list of lines where type comments where rejected + + +def skip_blank(d, lno): + while d.lines[lno].strip() == '': + lno += 1 + return lno + + +def find_start(d, lcom): + """Find first char of the assignment target.""" + i = d.ttab[lcom + 1][-2] # index of type comment token in tokens list + while ((d.tokens[i].exact_type != tokenize.NEWLINE) and + (d.tokens[i].exact_type != tokenize.ENCODING)): + i -= 1 + lno = d.tokens[i].start[0] + return skip_blank(d, lno) + + +def check_target(stmt): + if len(stmt.body): + assign = stmt.body[0] + else: + return False + if isinstance(assign, ast.Assign) and len(assign.targets) == 1: + targ = assign.targets[0] + else: + return False + if (isinstance(targ, ast.Name) or isinstance(targ, ast.Attribute) + or isinstance(targ, ast.Subscript)): + return True + return False + + +def find_eq(d, lstart): + """Find equal sign starting from lstart taking care about d[f(x=1)] = 5.""" + col = pars = 0 + lno = lstart + while d.lines[lno][col] != '=' or pars != 0: + ch = d.lines[lno][col] + if ch in '([{': + pars += 1 + elif ch in ')]}': + pars -= 1 + if ch == '#' or col == len(d.lines[lno])-1: + lno = skip_blank(d, lno+1) + col = 0 + else: + col += 1 + return lno, col + + +def find_val(d, poseq): + """Find position of first char of assignment value starting from poseq.""" + lno, col = poseq + while (d.lines[lno][col].isspace() or d.lines[lno][col] in '=\\'): + if col == len(d.lines[lno])-1: + lno += 1 + col = 0 + else: + col += 1 + return lno, col + + +def find_targ(d, poseq): + """Find position of last char of target (annotation goes here).""" + lno, col = poseq + while (d.lines[lno][col].isspace() or d.lines[lno][col] in '=\\'): + if col == 0: + lno -= 1 + col = len(d.lines[lno])-1 + else: + col -= 1 + return lno, col+1 + + +def trim(new_lines, string, ltarg, poseq, lcom, ccom): + """Remove None or Ellipsis from assignment value. + + Also remove parens if one has (None), (...) etc. + string -- 'None' or '...' + ltarg -- line where last char of target is located + poseq -- position of equal sign + lcom, ccom -- position of type comment + """ + nopars = lambda s: s.replace('(', '').replace(')', '') + leq, ceq = poseq + end = ccom if leq == lcom else len(new_lines[leq]) + subline = new_lines[leq][:ceq] + if leq == ltarg: + subline = subline.rstrip() + new_lines[leq] = subline + (new_lines[leq][end:] if leq == lcom + else new_lines[leq][ceq+1:end]) + + for lno in range(leq+1,lcom): + new_lines[lno] = nopars(new_lines[lno]) + + if lcom != leq: + subline = nopars(new_lines[lcom][:ccom]).replace(string, '') + if (not subline.isspace()): + subline = subline.rstrip() + new_lines[lcom] = subline + new_lines[lcom][ccom:] + + +def _com2ann(d, drop_None, drop_Ellipsis): + new_lines = d.lines[:] + for lcom, line in enumerate(d.lines): + match = re.search(TYPE_COM, line) + if match: + # strip " # type : annotation \n" -> "annotation \n" + tp = match.group().lstrip()[1:].lstrip()[4:].lstrip()[1:].lstrip() + submatch = re.search(TRAIL_OR_COM, tp) + subcom = '' + if submatch and submatch.group(): + subcom = submatch.group() + tp = tp[:submatch.start()] + if tp == 'ignore': + continue + ccom = match.start() + if not any(d.tokens[i].exact_type == tokenize.COMMENT + for i in d.ttab[lcom + 1]): + d.fail.append(lcom) + continue # type comment inside string + lstart = find_start(d, lcom) + stmt_str = dedent(''.join(d.lines[lstart:lcom+1])) + try: + stmt = ast.parse(stmt_str) + except SyntaxError: + d.fail.append(lcom) + continue # for or with statements + if not check_target(stmt): + d.fail.append(lcom) + continue + + d.success.append(lcom) + val = stmt.body[0].value + + # writing output now + poseq = find_eq(d, lstart) + lval, cval = find_val(d, poseq) + ltarg, ctarg = find_targ(d, poseq) + + op_par = '' + cl_par = '' + if isinstance(val, ast.Tuple): + if d.lines[lval][cval] != '(': + op_par = '(' + cl_par = ')' + # write the comment first + new_lines[lcom] = d.lines[lcom][:ccom].rstrip() + cl_par + subcom + ccom = len(d.lines[lcom][:ccom].rstrip()) + + string = False + if isinstance(val, ast.Tuple): + # t = 1, 2 -> t = (1, 2); only latter is allowed with annotation + free_place = int(new_lines[lval][cval-2:cval] == ' ') + new_lines[lval] = (new_lines[lval][:cval-free_place] + + op_par + new_lines[lval][cval:]) + elif isinstance(val, ast.Ellipsis) and drop_Ellipsis: + string = '...' + elif (isinstance(val, ast.NameConstant) and + val.value is None and drop_None): + string = 'None' + if string: + trim(new_lines, string, ltarg, poseq, lcom, ccom) + + # finally write an annotation + new_lines[ltarg] = (new_lines[ltarg][:ctarg] + + ': ' + tp + new_lines[ltarg][ctarg:]) + return ''.join(new_lines) + + +def com2ann(code, *, drop_None=False, drop_Ellipsis=False, silent=False): + """Translate type comments to type annotations in code. + + Take code as string and return this string where:: + + variable = value # type: annotation # real comment + + is translated to:: + + variable: annotation = value # real comment + + For unsupported syntax cases, the type comments are + left intact. If drop_None is True or if drop_Ellipsis + is True translate correcpondingly:: + + variable = None # type: annotation + variable = ... # type: annotation + + into:: + + variable: annotation + + The tool tries to preserve code formatting as much as + possible, but an exact translation is not guarateed. + A summary of translated comments id printed by default. + """ + try: + ast.parse(code) # we want to work only with file without syntax errors + except SyntaxError: + return None + lines = code.splitlines(keepends=True) + rl = BytesIO(code.encode('utf-8')).readline + tokens = list(tokenize.tokenize(rl)) + + data = _Data(lines, tokens) + new_code = _com2ann(data, drop_None, drop_Ellipsis) + + if not silent: + if data.success: + print('Comments translated on lines:', + ', '.join(str(lno+1) for lno in data.success)) + if data.fail: + print('Comments rejected on lines:', + ', '.join(str(lno+1) for lno in data.fail)) + if not data.success and not data.fail: + print('No type comments found') + + return new_code + + +def translate_file(infile, outfile, dnone, dell, silent): + try: + descr = tokenize.open(infile) + except SyntaxError: + print("Cannot open", infile) + return + with descr as f: + code = f.read() + enc = f.encoding + if not silent: + print('File:', infile) + new_code = com2ann(code, drop_None=dnone, + drop_Ellipsis=dell, + silent=silent) + if new_code is None: + print("SyntaxError in", infile) + return + with open(outfile, 'wb') as f: + f.write((new_code).encode(enc)) + + +if __name__ == '__main__': + + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("-o", "--outfile", + help="output file, will be overwritten if exists,\n" + "defaults to input file") + parser.add_argument("infile", + help="input file or directory for translation, must\n" + "contain no syntax errors, for directory\n" + "the outfile is ignored and translation is\n" + "made in place") + parser.add_argument("-s", "--silent", + help="Do not print summary for line numbers of\n" + "translated and rejected comments", + action="store_true") + parser.add_argument("-n", "--drop-none", + help="drop any None as assignment value during\n" + "translation if it is annotated by a type coment", + action="store_true") + parser.add_argument("-e", "--drop-ellipsis", + help="drop any Ellipsis (...) as assignment value during\n" + "translation if it is annotated by a type coment", + action="store_true") + args = parser.parse_args() + if args.outfile is None: + args.outfile = args.infile + + if os.path.isfile(args.infile): + translate_file(args.infile, args.outfile, + args.drop_none, args.drop_ellipsis, args.silent) + else: + for root, dirs, files in os.walk(args.infile): + for afile in files: + _, ext = os.path.splitext(afile) + if ext == '.py' or ext == '.pyi': + fname = os.path.join(root, afile) + translate_file(fname, fname, + args.drop_none, args.drop_ellipsis, + args.silent) diff --git a/Tools/parser/unparse.py b/Tools/parser/unparse.py --- a/Tools/parser/unparse.py +++ b/Tools/parser/unparse.py @@ -104,6 +104,19 @@ self.write(" "+self.binop[t.op.__class__.__name__]+"= ") self.dispatch(t.value) + def _AnnAssign(self, t): + self.fill() + if not t.simple and isinstance(t.target, ast.Name): + self.write('(') + self.dispatch(t.target) + if not t.simple and isinstance(t.target, ast.Name): + self.write(')') + self.write(": ") + self.dispatch(t.annotation) + if t.value: + self.write(" = ") + self.dispatch(t.value) + def _Return(self, t): self.fill("return") if t.value: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 00:47:40 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 09 Sep 2016 04:47:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_regrtest=3A_log_FS_and_loc?= =?utf-8?q?ale_encodings?= Message-ID: <20160909044740.69529.1973.E25F2E99@psf.io> https://hg.python.org/cpython/rev/4ac88398319d changeset: 103390:4ac88398319d user: Victor Stinner date: Thu Sep 08 21:46:56 2016 -0700 summary: regrtest: log FS and locale encodings files: Lib/test/libregrtest/main.py | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -1,5 +1,6 @@ import datetime import faulthandler +import locale import os import platform import random @@ -392,7 +393,10 @@ "%s-endian" % sys.byteorder) print("== ", "hash algorithm:", sys.hash_info.algorithm, "64bit" if sys.maxsize > 2**32 else "32bit") - print("== ", os.getcwd()) + print("== cwd:", os.getcwd()) + print("== encodings: locale=%s, FS=%s" + % (locale.getpreferredencoding(False), + sys.getfilesystemencoding())) print("Testing with flags:", sys.flags) if self.ns.randomize: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 00:50:57 2016 From: python-checkins at python.org (gregory.p.smith) Date: Fri, 09 Sep 2016 04:50:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Disable_test?= =?utf-8?q?=5Fgdb=2EPrettyPrintTests_when_compiled_with_optimizations=2C?= Message-ID: <20160909045057.8881.71376.00E7816A@psf.io> https://hg.python.org/cpython/rev/3cce329c20e6 changeset: 103391:3cce329c20e6 branch: 2.7 parent: 103384:7e89469e4342 user: Gregory P. Smith date: Thu Sep 08 21:50:44 2016 -0700 summary: Disable test_gdb.PrettyPrintTests when compiled with optimizations, these often fail on PGO builds. files: Lib/test/test_gdb.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -265,6 +265,9 @@ def get_sample_script(self): return findfile('gdb_sample.py') + + at unittest.skipIf(python_is_optimized(), + "Python was compiled with optimizations") class PrettyPrintTests(DebuggerTests): def test_getting_backtrace(self): gdb_output = self.get_stack_trace('print 42') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 01:02:00 2016 From: python-checkins at python.org (yury.selivanov) Date: Fri, 09 Sep 2016 05:02:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328003=3A_Implemen?= =?utf-8?q?t_PEP_525_--_Asynchronous_Generators=2E?= Message-ID: <20160909050159.69549.56001.75F0AE80@psf.io> https://hg.python.org/cpython/rev/5259588983ca changeset: 103392:5259588983ca parent: 103390:4ac88398319d user: Yury Selivanov date: Thu Sep 08 22:01:51 2016 -0700 summary: Issue #28003: Implement PEP 525 -- Asynchronous Generators. files: Include/ceval.h | 4 + Include/code.h | 1 + Include/genobject.h | 31 + Include/pylifecycle.h | 1 + Include/pystate.h | 3 + Include/symtable.h | 1 + Lib/asyncio/base_events.py | 57 +- Lib/asyncio/coroutines.py | 5 +- Lib/asyncio/events.py | 4 + Lib/dis.py | 1 + Lib/inspect.py | 7 + Lib/test/badsyntax_async6.py | 2 - Lib/test/test_asyncgen.py | 823 +++++++++++++++++ Lib/test/test_coroutines.py | 6 - Lib/test/test_dis.py | 2 +- Lib/test/test_inspect.py | 12 +- Lib/test/test_sys.py | 26 + Lib/types.py | 5 + Misc/NEWS | 3 + Modules/gcmodule.c | 1 + Objects/genobject.c | 1032 +++++++++++++++++++++- Python/ceval.c | 116 +- Python/compile.c | 13 +- Python/pylifecycle.c | 1 + Python/pystate.c | 5 + Python/symtable.c | 6 +- Python/sysmodule.c | 119 ++ 27 files changed, 2190 insertions(+), 97 deletions(-) diff --git a/Include/ceval.h b/Include/ceval.h --- a/Include/ceval.h +++ b/Include/ceval.h @@ -25,6 +25,10 @@ PyAPI_FUNC(void) PyEval_SetTrace(Py_tracefunc, PyObject *); PyAPI_FUNC(void) _PyEval_SetCoroutineWrapper(PyObject *); PyAPI_FUNC(PyObject *) _PyEval_GetCoroutineWrapper(void); +PyAPI_FUNC(void) _PyEval_SetAsyncGenFirstiter(PyObject *); +PyAPI_FUNC(PyObject *) _PyEval_GetAsyncGenFirstiter(void); +PyAPI_FUNC(void) _PyEval_SetAsyncGenFinalizer(PyObject *); +PyAPI_FUNC(PyObject *) _PyEval_GetAsyncGenFinalizer(void); #endif struct _frame; /* Avoid including frameobject.h */ diff --git a/Include/code.h b/Include/code.h --- a/Include/code.h +++ b/Include/code.h @@ -59,6 +59,7 @@ ``async def`` keywords) */ #define CO_COROUTINE 0x0080 #define CO_ITERABLE_COROUTINE 0x0100 +#define CO_ASYNC_GENERATOR 0x0200 /* These are no longer used. */ #if 0 diff --git a/Include/genobject.h b/Include/genobject.h --- a/Include/genobject.h +++ b/Include/genobject.h @@ -61,6 +61,37 @@ PyObject *_PyCoro_GetAwaitableIter(PyObject *o); PyAPI_FUNC(PyObject *) PyCoro_New(struct _frame *, PyObject *name, PyObject *qualname); + +/* Asynchronous Generators */ + +typedef struct { + _PyGenObject_HEAD(ag) + PyObject *ag_finalizer; + + /* Flag is set to 1 when hooks set up by sys.set_asyncgen_hooks + were called on the generator, to avoid calling them more + than once. */ + int ag_hooks_inited; + + /* Flag is set to 1 when aclose() is called for the first time, or + when a StopAsyncIteration exception is raised. */ + int ag_closed; +} PyAsyncGenObject; + +PyAPI_DATA(PyTypeObject) PyAsyncGen_Type; +PyAPI_DATA(PyTypeObject) _PyAsyncGenASend_Type; +PyAPI_DATA(PyTypeObject) _PyAsyncGenWrappedValue_Type; +PyAPI_DATA(PyTypeObject) _PyAsyncGenAThrow_Type; + +PyAPI_FUNC(PyObject *) PyAsyncGen_New(struct _frame *, + PyObject *name, PyObject *qualname); + +#define PyAsyncGen_CheckExact(op) (Py_TYPE(op) == &PyAsyncGen_Type) + +PyObject *_PyAsyncGenValueWrapperNew(PyObject *); + +int PyAsyncGen_ClearFreeLists(void); + #endif #undef _PyGenObject_HEAD diff --git a/Include/pylifecycle.h b/Include/pylifecycle.h --- a/Include/pylifecycle.h +++ b/Include/pylifecycle.h @@ -107,6 +107,7 @@ PyAPI_FUNC(void) PySlice_Fini(void); PyAPI_FUNC(void) _PyType_Fini(void); PyAPI_FUNC(void) _PyRandom_Fini(void); +PyAPI_FUNC(void) PyAsyncGen_Fini(void); PyAPI_DATA(PyThreadState *) _Py_Finalizing; #endif diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -148,6 +148,9 @@ Py_ssize_t co_extra_user_count; freefunc co_extra_freefuncs[MAX_CO_EXTRA_USERS]; + PyObject *async_gen_firstiter; + PyObject *async_gen_finalizer; + /* XXX signal handlers should also be here */ } PyThreadState; diff --git a/Include/symtable.h b/Include/symtable.h --- a/Include/symtable.h +++ b/Include/symtable.h @@ -48,6 +48,7 @@ unsigned ste_child_free : 1; /* true if a child block has free vars, including free refs to globals */ unsigned ste_generator : 1; /* true if namespace is a generator */ + unsigned ste_coroutine : 1; /* true if namespace is a coroutine */ unsigned ste_varargs : 1; /* true if block has varargs */ unsigned ste_varkeywords : 1; /* true if block has varkeywords */ unsigned ste_returns_value : 1; /* true if namespace uses return with diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -13,7 +13,6 @@ to modify the meaning of the API call itself. """ - import collections import concurrent.futures import heapq @@ -28,6 +27,7 @@ import traceback import sys import warnings +import weakref from . import compat from . import coroutines @@ -242,6 +242,13 @@ self._task_factory = None self._coroutine_wrapper_set = False + # A weak set of all asynchronous generators that are being iterated + # by the loop. + self._asyncgens = weakref.WeakSet() + + # Set to True when `loop.shutdown_asyncgens` is called. + self._asyncgens_shutdown_called = False + def __repr__(self): return ('<%s running=%s closed=%s debug=%s>' % (self.__class__.__name__, self.is_running(), @@ -333,6 +340,46 @@ if self._closed: raise RuntimeError('Event loop is closed') + def _asyncgen_finalizer_hook(self, agen): + self._asyncgens.discard(agen) + if not self.is_closed(): + self.create_task(agen.aclose()) + + def _asyncgen_firstiter_hook(self, agen): + if self._asyncgens_shutdown_called: + warnings.warn( + "asynchronous generator {!r} was scheduled after " + "loop.shutdown_asyncgens() call".format(agen), + ResourceWarning, source=self) + + self._asyncgens.add(agen) + + @coroutine + def shutdown_asyncgens(self): + """Shutdown all active asynchronous generators.""" + self._asyncgens_shutdown_called = True + + if not len(self._asyncgens): + return + + closing_agens = list(self._asyncgens) + self._asyncgens.clear() + + shutdown_coro = tasks.gather( + *[ag.aclose() for ag in closing_agens], + return_exceptions=True, + loop=self) + + results = yield from shutdown_coro + for result, agen in zip(results, closing_agens): + if isinstance(result, Exception): + self.call_exception_handler({ + 'message': 'an error occurred during closing of ' + 'asynchronous generator {!r}'.format(agen), + 'exception': result, + 'asyncgen': agen + }) + def run_forever(self): """Run until stop() is called.""" self._check_closed() @@ -340,6 +387,9 @@ raise RuntimeError('Event loop is running.') self._set_coroutine_wrapper(self._debug) self._thread_id = threading.get_ident() + old_agen_hooks = sys.get_asyncgen_hooks() + sys.set_asyncgen_hooks(firstiter=self._asyncgen_firstiter_hook, + finalizer=self._asyncgen_finalizer_hook) try: while True: self._run_once() @@ -349,6 +399,7 @@ self._stopping = False self._thread_id = None self._set_coroutine_wrapper(False) + sys.set_asyncgen_hooks(*old_agen_hooks) def run_until_complete(self, future): """Run until the Future is done. @@ -1179,7 +1230,9 @@ - 'handle' (optional): Handle instance; - 'protocol' (optional): Protocol instance; - 'transport' (optional): Transport instance; - - 'socket' (optional): Socket instance. + - 'socket' (optional): Socket instance; + - 'asyncgen' (optional): Asynchronous generator that caused + the exception. New keys maybe introduced in the future. diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py --- a/Lib/asyncio/coroutines.py +++ b/Lib/asyncio/coroutines.py @@ -276,7 +276,10 @@ try: coro_code = coro.gi_code except AttributeError: - coro_code = coro.cr_code + try: + coro_code = coro.cr_code + except AttributeError: + return repr(coro) try: coro_frame = coro.gi_frame diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -248,6 +248,10 @@ """ raise NotImplementedError + def shutdown_asyncgens(self): + """Shutdown all active asynchronous generators.""" + raise NotImplementedError + # Methods scheduling callbacks. All these return Handles. def _timer_handle_cancelled(self, handle): diff --git a/Lib/dis.py b/Lib/dis.py --- a/Lib/dis.py +++ b/Lib/dis.py @@ -87,6 +87,7 @@ 64: "NOFREE", 128: "COROUTINE", 256: "ITERABLE_COROUTINE", + 512: "ASYNC_GENERATOR", } def pretty_flags(flags): diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -185,6 +185,13 @@ return bool((isfunction(object) or ismethod(object)) and object.__code__.co_flags & CO_COROUTINE) +def isasyncgenfunction(object): + return bool((isfunction(object) or ismethod(object)) and + object.__code__.co_flags & CO_ASYNC_GENERATOR) + +def isasyncgen(object): + return isinstance(object, types.AsyncGeneratorType) + def isgenerator(object): """Return true if the object is a generator. diff --git a/Lib/test/badsyntax_async6.py b/Lib/test/badsyntax_async6.py deleted file mode 100644 --- a/Lib/test/badsyntax_async6.py +++ /dev/null @@ -1,2 +0,0 @@ -async def foo(): - yield diff --git a/Lib/test/test_asyncgen.py b/Lib/test/test_asyncgen.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_asyncgen.py @@ -0,0 +1,823 @@ +import asyncio +import inspect +import sys +import types +import unittest + +from unittest import mock + + +class AwaitException(Exception): + pass + + + at types.coroutine +def awaitable(*, throw=False): + if throw: + yield ('throw',) + else: + yield ('result',) + + +def run_until_complete(coro): + exc = False + while True: + try: + if exc: + exc = False + fut = coro.throw(AwaitException) + else: + fut = coro.send(None) + except StopIteration as ex: + return ex.args[0] + + if fut == ('throw',): + exc = True + + +def to_list(gen): + async def iterate(): + res = [] + async for i in gen: + res.append(i) + return res + + return run_until_complete(iterate()) + + +class AsyncGenSyntaxTest(unittest.TestCase): + + def test_async_gen_syntax_01(self): + code = '''async def foo(): + await abc + yield from 123 + ''' + + with self.assertRaisesRegex(SyntaxError, 'yield from.*inside async'): + exec(code, {}, {}) + + def test_async_gen_syntax_02(self): + code = '''async def foo(): + yield from 123 + ''' + + with self.assertRaisesRegex(SyntaxError, 'yield from.*inside async'): + exec(code, {}, {}) + + def test_async_gen_syntax_03(self): + code = '''async def foo(): + await abc + yield + return 123 + ''' + + with self.assertRaisesRegex(SyntaxError, 'return.*value.*async gen'): + exec(code, {}, {}) + + def test_async_gen_syntax_04(self): + code = '''async def foo(): + yield + return 123 + ''' + + with self.assertRaisesRegex(SyntaxError, 'return.*value.*async gen'): + exec(code, {}, {}) + + def test_async_gen_syntax_05(self): + code = '''async def foo(): + if 0: + yield + return 12 + ''' + + with self.assertRaisesRegex(SyntaxError, 'return.*value.*async gen'): + exec(code, {}, {}) + + +class AsyncGenTest(unittest.TestCase): + + def compare_generators(self, sync_gen, async_gen): + def sync_iterate(g): + res = [] + while True: + try: + res.append(g.__next__()) + except StopIteration: + res.append('STOP') + break + except Exception as ex: + res.append(str(type(ex))) + return res + + def async_iterate(g): + res = [] + while True: + try: + g.__anext__().__next__() + except StopAsyncIteration: + res.append('STOP') + break + except StopIteration as ex: + if ex.args: + res.append(ex.args[0]) + else: + res.append('EMPTY StopIteration') + break + except Exception as ex: + res.append(str(type(ex))) + return res + + sync_gen_result = sync_iterate(sync_gen) + async_gen_result = async_iterate(async_gen) + self.assertEqual(sync_gen_result, async_gen_result) + return async_gen_result + + def test_async_gen_iteration_01(self): + async def gen(): + await awaitable() + a = yield 123 + self.assertIs(a, None) + await awaitable() + yield 456 + await awaitable() + yield 789 + + self.assertEqual(to_list(gen()), [123, 456, 789]) + + def test_async_gen_iteration_02(self): + async def gen(): + await awaitable() + yield 123 + await awaitable() + + g = gen() + ai = g.__aiter__() + self.assertEqual(ai.__anext__().__next__(), ('result',)) + + try: + ai.__anext__().__next__() + except StopIteration as ex: + self.assertEqual(ex.args[0], 123) + else: + self.fail('StopIteration was not raised') + + self.assertEqual(ai.__anext__().__next__(), ('result',)) + + try: + ai.__anext__().__next__() + except StopAsyncIteration as ex: + self.assertFalse(ex.args) + else: + self.fail('StopAsyncIteration was not raised') + + def test_async_gen_exception_03(self): + async def gen(): + await awaitable() + yield 123 + await awaitable(throw=True) + yield 456 + + with self.assertRaises(AwaitException): + to_list(gen()) + + def test_async_gen_exception_04(self): + async def gen(): + await awaitable() + yield 123 + 1 / 0 + + g = gen() + ai = g.__aiter__() + self.assertEqual(ai.__anext__().__next__(), ('result',)) + + try: + ai.__anext__().__next__() + except StopIteration as ex: + self.assertEqual(ex.args[0], 123) + else: + self.fail('StopIteration was not raised') + + with self.assertRaises(ZeroDivisionError): + ai.__anext__().__next__() + + def test_async_gen_exception_05(self): + async def gen(): + yield 123 + raise StopAsyncIteration + + with self.assertRaisesRegex(RuntimeError, + 'async generator.*StopAsyncIteration'): + to_list(gen()) + + def test_async_gen_exception_06(self): + async def gen(): + yield 123 + raise StopIteration + + with self.assertRaisesRegex(RuntimeError, + 'async generator.*StopIteration'): + to_list(gen()) + + def test_async_gen_exception_07(self): + def sync_gen(): + try: + yield 1 + 1 / 0 + finally: + yield 2 + yield 3 + + yield 100 + + async def async_gen(): + try: + yield 1 + 1 / 0 + finally: + yield 2 + yield 3 + + yield 100 + + self.compare_generators(sync_gen(), async_gen()) + + def test_async_gen_exception_08(self): + def sync_gen(): + try: + yield 1 + finally: + yield 2 + 1 / 0 + yield 3 + + yield 100 + + async def async_gen(): + try: + yield 1 + await awaitable() + finally: + await awaitable() + yield 2 + 1 / 0 + yield 3 + + yield 100 + + self.compare_generators(sync_gen(), async_gen()) + + def test_async_gen_exception_09(self): + def sync_gen(): + try: + yield 1 + 1 / 0 + finally: + yield 2 + yield 3 + + yield 100 + + async def async_gen(): + try: + await awaitable() + yield 1 + 1 / 0 + finally: + yield 2 + await awaitable() + yield 3 + + yield 100 + + self.compare_generators(sync_gen(), async_gen()) + + def test_async_gen_exception_10(self): + async def gen(): + yield 123 + with self.assertRaisesRegex(TypeError, + "non-None value .* async generator"): + gen().__anext__().send(100) + + def test_async_gen_api_01(self): + async def gen(): + yield 123 + + g = gen() + + self.assertEqual(g.__name__, 'gen') + g.__name__ = '123' + self.assertEqual(g.__name__, '123') + + self.assertIn('.gen', g.__qualname__) + g.__qualname__ = '123' + self.assertEqual(g.__qualname__, '123') + + self.assertIsNone(g.ag_await) + self.assertIsInstance(g.ag_frame, types.FrameType) + self.assertFalse(g.ag_running) + self.assertIsInstance(g.ag_code, types.CodeType) + + self.assertTrue(inspect.isawaitable(g.aclose())) + + +class AsyncGenAsyncioTest(unittest.TestCase): + + def setUp(self): + self.loop = asyncio.new_event_loop() + asyncio.set_event_loop(None) + + def tearDown(self): + self.loop.close() + self.loop = None + + async def to_list(self, gen): + res = [] + async for i in gen: + res.append(i) + return res + + def test_async_gen_asyncio_01(self): + async def gen(): + yield 1 + await asyncio.sleep(0.01, loop=self.loop) + yield 2 + await asyncio.sleep(0.01, loop=self.loop) + return + yield 3 + + res = self.loop.run_until_complete(self.to_list(gen())) + self.assertEqual(res, [1, 2]) + + def test_async_gen_asyncio_02(self): + async def gen(): + yield 1 + await asyncio.sleep(0.01, loop=self.loop) + yield 2 + 1 / 0 + yield 3 + + with self.assertRaises(ZeroDivisionError): + self.loop.run_until_complete(self.to_list(gen())) + + def test_async_gen_asyncio_03(self): + loop = self.loop + + class Gen: + async def __aiter__(self): + yield 1 + await asyncio.sleep(0.01, loop=loop) + yield 2 + + res = loop.run_until_complete(self.to_list(Gen())) + self.assertEqual(res, [1, 2]) + + def test_async_gen_asyncio_anext_04(self): + async def foo(): + yield 1 + await asyncio.sleep(0.01, loop=self.loop) + try: + yield 2 + yield 3 + except ZeroDivisionError: + yield 1000 + await asyncio.sleep(0.01, loop=self.loop) + yield 4 + + async def run1(): + it = foo().__aiter__() + + self.assertEqual(await it.__anext__(), 1) + self.assertEqual(await it.__anext__(), 2) + self.assertEqual(await it.__anext__(), 3) + self.assertEqual(await it.__anext__(), 4) + with self.assertRaises(StopAsyncIteration): + await it.__anext__() + with self.assertRaises(StopAsyncIteration): + await it.__anext__() + + async def run2(): + it = foo().__aiter__() + + self.assertEqual(await it.__anext__(), 1) + self.assertEqual(await it.__anext__(), 2) + try: + it.__anext__().throw(ZeroDivisionError) + except StopIteration as ex: + self.assertEqual(ex.args[0], 1000) + else: + self.fail('StopIteration was not raised') + self.assertEqual(await it.__anext__(), 4) + with self.assertRaises(StopAsyncIteration): + await it.__anext__() + + self.loop.run_until_complete(run1()) + self.loop.run_until_complete(run2()) + + def test_async_gen_asyncio_anext_05(self): + async def foo(): + v = yield 1 + v = yield v + yield v * 100 + + async def run(): + it = foo().__aiter__() + + try: + it.__anext__().send(None) + except StopIteration as ex: + self.assertEqual(ex.args[0], 1) + else: + self.fail('StopIteration was not raised') + + try: + it.__anext__().send(10) + except StopIteration as ex: + self.assertEqual(ex.args[0], 10) + else: + self.fail('StopIteration was not raised') + + try: + it.__anext__().send(12) + except StopIteration as ex: + self.assertEqual(ex.args[0], 1200) + else: + self.fail('StopIteration was not raised') + + with self.assertRaises(StopAsyncIteration): + await it.__anext__() + + self.loop.run_until_complete(run()) + + def test_async_gen_asyncio_aclose_06(self): + async def foo(): + try: + yield 1 + 1 / 0 + finally: + await asyncio.sleep(0.01, loop=self.loop) + yield 12 + + async def run(): + gen = foo() + it = gen.__aiter__() + await it.__anext__() + await gen.aclose() + + with self.assertRaisesRegex( + RuntimeError, + "async generator ignored GeneratorExit"): + self.loop.run_until_complete(run()) + + def test_async_gen_asyncio_aclose_07(self): + DONE = 0 + + async def foo(): + nonlocal DONE + try: + yield 1 + 1 / 0 + finally: + await asyncio.sleep(0.01, loop=self.loop) + await asyncio.sleep(0.01, loop=self.loop) + DONE += 1 + DONE += 1000 + + async def run(): + gen = foo() + it = gen.__aiter__() + await it.__anext__() + await gen.aclose() + + self.loop.run_until_complete(run()) + self.assertEqual(DONE, 1) + + def test_async_gen_asyncio_aclose_08(self): + DONE = 0 + + fut = asyncio.Future(loop=self.loop) + + async def foo(): + nonlocal DONE + try: + yield 1 + await fut + DONE += 1000 + yield 2 + finally: + await asyncio.sleep(0.01, loop=self.loop) + await asyncio.sleep(0.01, loop=self.loop) + DONE += 1 + DONE += 1000 + + async def run(): + gen = foo() + it = gen.__aiter__() + self.assertEqual(await it.__anext__(), 1) + t = self.loop.create_task(it.__anext__()) + await asyncio.sleep(0.01, loop=self.loop) + await gen.aclose() + return t + + t = self.loop.run_until_complete(run()) + self.assertEqual(DONE, 1) + + # Silence ResourceWarnings + fut.cancel() + t.cancel() + self.loop.run_until_complete(asyncio.sleep(0.01, loop=self.loop)) + + def test_async_gen_asyncio_gc_aclose_09(self): + DONE = 0 + + async def gen(): + nonlocal DONE + try: + while True: + yield 1 + finally: + await asyncio.sleep(0.01, loop=self.loop) + await asyncio.sleep(0.01, loop=self.loop) + DONE = 1 + + async def run(): + g = gen() + await g.__anext__() + await g.__anext__() + del g + + await asyncio.sleep(0.1, loop=self.loop) + + self.loop.run_until_complete(run()) + self.assertEqual(DONE, 1) + + def test_async_gen_asyncio_asend_01(self): + DONE = 0 + + # Sanity check: + def sgen(): + v = yield 1 + yield v * 2 + sg = sgen() + v = sg.send(None) + self.assertEqual(v, 1) + v = sg.send(100) + self.assertEqual(v, 200) + + async def gen(): + nonlocal DONE + try: + await asyncio.sleep(0.01, loop=self.loop) + v = yield 1 + await asyncio.sleep(0.01, loop=self.loop) + yield v * 2 + await asyncio.sleep(0.01, loop=self.loop) + return + finally: + await asyncio.sleep(0.01, loop=self.loop) + await asyncio.sleep(0.01, loop=self.loop) + DONE = 1 + + async def run(): + g = gen() + + v = await g.asend(None) + self.assertEqual(v, 1) + + v = await g.asend(100) + self.assertEqual(v, 200) + + with self.assertRaises(StopAsyncIteration): + await g.asend(None) + + self.loop.run_until_complete(run()) + self.assertEqual(DONE, 1) + + def test_async_gen_asyncio_asend_02(self): + DONE = 0 + + async def sleep_n_crash(delay): + await asyncio.sleep(delay, loop=self.loop) + 1 / 0 + + async def gen(): + nonlocal DONE + try: + await asyncio.sleep(0.01, loop=self.loop) + v = yield 1 + await sleep_n_crash(0.01) + DONE += 1000 + yield v * 2 + finally: + await asyncio.sleep(0.01, loop=self.loop) + await asyncio.sleep(0.01, loop=self.loop) + DONE = 1 + + async def run(): + g = gen() + + v = await g.asend(None) + self.assertEqual(v, 1) + + await g.asend(100) + + with self.assertRaises(ZeroDivisionError): + self.loop.run_until_complete(run()) + self.assertEqual(DONE, 1) + + def test_async_gen_asyncio_asend_03(self): + DONE = 0 + + async def sleep_n_crash(delay): + fut = asyncio.ensure_future(asyncio.sleep(delay, loop=self.loop), + loop=self.loop) + self.loop.call_later(delay / 2, lambda: fut.cancel()) + return await fut + + async def gen(): + nonlocal DONE + try: + await asyncio.sleep(0.01, loop=self.loop) + v = yield 1 + await sleep_n_crash(0.01) + DONE += 1000 + yield v * 2 + finally: + await asyncio.sleep(0.01, loop=self.loop) + await asyncio.sleep(0.01, loop=self.loop) + DONE = 1 + + async def run(): + g = gen() + + v = await g.asend(None) + self.assertEqual(v, 1) + + await g.asend(100) + + with self.assertRaises(asyncio.CancelledError): + self.loop.run_until_complete(run()) + self.assertEqual(DONE, 1) + + def test_async_gen_asyncio_athrow_01(self): + DONE = 0 + + class FooEr(Exception): + pass + + # Sanity check: + def sgen(): + try: + v = yield 1 + except FooEr: + v = 1000 + yield v * 2 + sg = sgen() + v = sg.send(None) + self.assertEqual(v, 1) + v = sg.throw(FooEr) + self.assertEqual(v, 2000) + with self.assertRaises(StopIteration): + sg.send(None) + + async def gen(): + nonlocal DONE + try: + await asyncio.sleep(0.01, loop=self.loop) + try: + v = yield 1 + except FooEr: + v = 1000 + await asyncio.sleep(0.01, loop=self.loop) + yield v * 2 + await asyncio.sleep(0.01, loop=self.loop) + # return + finally: + await asyncio.sleep(0.01, loop=self.loop) + await asyncio.sleep(0.01, loop=self.loop) + DONE = 1 + + async def run(): + g = gen() + + v = await g.asend(None) + self.assertEqual(v, 1) + + v = await g.athrow(FooEr) + self.assertEqual(v, 2000) + + with self.assertRaises(StopAsyncIteration): + await g.asend(None) + + self.loop.run_until_complete(run()) + self.assertEqual(DONE, 1) + + def test_async_gen_asyncio_athrow_02(self): + DONE = 0 + + class FooEr(Exception): + pass + + async def sleep_n_crash(delay): + fut = asyncio.ensure_future(asyncio.sleep(delay, loop=self.loop), + loop=self.loop) + self.loop.call_later(delay / 2, lambda: fut.cancel()) + return await fut + + async def gen(): + nonlocal DONE + try: + await asyncio.sleep(0.01, loop=self.loop) + try: + v = yield 1 + except FooEr: + await sleep_n_crash(0.01) + yield v * 2 + await asyncio.sleep(0.01, loop=self.loop) + # return + finally: + await asyncio.sleep(0.01, loop=self.loop) + await asyncio.sleep(0.01, loop=self.loop) + DONE = 1 + + async def run(): + g = gen() + + v = await g.asend(None) + self.assertEqual(v, 1) + + try: + await g.athrow(FooEr) + except asyncio.CancelledError: + self.assertEqual(DONE, 1) + raise + else: + self.fail('CancelledError was not raised') + + with self.assertRaises(asyncio.CancelledError): + self.loop.run_until_complete(run()) + self.assertEqual(DONE, 1) + + def test_async_gen_asyncio_shutdown_01(self): + finalized = 0 + + async def waiter(timeout): + nonlocal finalized + try: + await asyncio.sleep(timeout, loop=self.loop) + yield 1 + finally: + await asyncio.sleep(0, loop=self.loop) + finalized += 1 + + async def wait(): + async for _ in waiter(1): + pass + + t1 = self.loop.create_task(wait()) + t2 = self.loop.create_task(wait()) + + self.loop.run_until_complete(asyncio.sleep(0.1, loop=self.loop)) + + self.loop.run_until_complete(self.loop.shutdown_asyncgens()) + self.assertEqual(finalized, 2) + + # Silence warnings + t1.cancel() + t2.cancel() + self.loop.run_until_complete(asyncio.sleep(0.1, loop=self.loop)) + + def test_async_gen_asyncio_shutdown_02(self): + logged = 0 + + def logger(loop, context): + nonlocal logged + self.assertIn('asyncgen', context) + expected = 'an error occurred during closing of asynchronous' + if expected in context['message']: + logged += 1 + + async def waiter(timeout): + try: + await asyncio.sleep(timeout, loop=self.loop) + yield 1 + finally: + 1 / 0 + + async def wait(): + async for _ in waiter(1): + pass + + t = self.loop.create_task(wait()) + self.loop.run_until_complete(asyncio.sleep(0.1, loop=self.loop)) + + self.loop.set_exception_handler(logger) + self.loop.run_until_complete(self.loop.shutdown_asyncgens()) + + self.assertEqual(logged, 1) + + # Silence warnings + t.cancel() + self.loop.run_until_complete(asyncio.sleep(0.1, loop=self.loop)) + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py --- a/Lib/test/test_coroutines.py +++ b/Lib/test/test_coroutines.py @@ -88,12 +88,6 @@ with self.assertRaisesRegex(SyntaxError, 'invalid syntax'): import test.badsyntax_async5 - def test_badsyntax_6(self): - with self.assertRaisesRegex( - SyntaxError, "'yield' inside async function"): - - import test.badsyntax_async6 - def test_badsyntax_7(self): with self.assertRaisesRegex( SyntaxError, "'yield from' inside async function"): diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -574,7 +574,7 @@ Kw-only arguments: 0 Number of locals: 2 Stack size: 17 -Flags: OPTIMIZED, NEWLOCALS, GENERATOR, NOFREE, COROUTINE +Flags: OPTIMIZED, NEWLOCALS, NOFREE, COROUTINE Constants: 0: None 1: 1""" diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -65,7 +65,8 @@ inspect.isframe, inspect.isfunction, inspect.ismethod, inspect.ismodule, inspect.istraceback, inspect.isgenerator, inspect.isgeneratorfunction, - inspect.iscoroutine, inspect.iscoroutinefunction]) + inspect.iscoroutine, inspect.iscoroutinefunction, + inspect.isasyncgen, inspect.isasyncgenfunction]) def istest(self, predicate, exp): obj = eval(exp) @@ -73,6 +74,7 @@ for other in self.predicates - set([predicate]): if (predicate == inspect.isgeneratorfunction or \ + predicate == inspect.isasyncgenfunction or \ predicate == inspect.iscoroutinefunction) and \ other == inspect.isfunction: continue @@ -82,6 +84,10 @@ for i in range(2): yield i +async def async_generator_function_example(self): + async for i in range(2): + yield i + async def coroutine_function_example(self): return 'spam' @@ -122,6 +128,10 @@ self.istest(inspect.isdatadescriptor, 'collections.defaultdict.default_factory') self.istest(inspect.isgenerator, '(x for x in range(2))') self.istest(inspect.isgeneratorfunction, 'generator_function_example') + self.istest(inspect.isasyncgen, + 'async_generator_function_example(1)') + self.istest(inspect.isasyncgenfunction, + 'async_generator_function_example') with warnings.catch_warnings(): warnings.simplefilter("ignore") diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -1192,6 +1192,32 @@ # sys.flags check(sys.flags, vsize('') + self.P * len(sys.flags)) + def test_asyncgen_hooks(self): + old = sys.get_asyncgen_hooks() + self.assertIsNone(old.firstiter) + self.assertIsNone(old.finalizer) + + firstiter = lambda *a: None + sys.set_asyncgen_hooks(firstiter=firstiter) + hooks = sys.get_asyncgen_hooks() + self.assertIs(hooks.firstiter, firstiter) + self.assertIs(hooks[0], firstiter) + self.assertIs(hooks.finalizer, None) + self.assertIs(hooks[1], None) + + finalizer = lambda *a: None + sys.set_asyncgen_hooks(finalizer=finalizer) + hooks = sys.get_asyncgen_hooks() + self.assertIs(hooks.firstiter, firstiter) + self.assertIs(hooks[0], firstiter) + self.assertIs(hooks.finalizer, finalizer) + self.assertIs(hooks[1], finalizer) + + sys.set_asyncgen_hooks(*old) + cur = sys.get_asyncgen_hooks() + self.assertIsNone(cur.firstiter) + self.assertIsNone(cur.finalizer) + def test_main(): test.support.run_unittest(SysModuleTest, SizeofTest) diff --git a/Lib/types.py b/Lib/types.py --- a/Lib/types.py +++ b/Lib/types.py @@ -24,6 +24,11 @@ CoroutineType = type(_c) _c.close() # Prevent ResourceWarning +async def _ag(): + yield +_ag = _ag() +AsyncGeneratorType = type(_ag) + class _C: def _m(self): pass MethodType = type(_C()._m) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -103,6 +103,9 @@ - Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi. +- Issue #28003: Implement PEP 525 -- Asynchronous Generators. + + Library ------- diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c --- a/Modules/gcmodule.c +++ b/Modules/gcmodule.c @@ -892,6 +892,7 @@ (void)PyList_ClearFreeList(); (void)PyDict_ClearFreeList(); (void)PySet_ClearFreeList(); + (void)PyAsyncGen_ClearFreeLists(); } /* This is the main function. Read this to understand how the diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -5,7 +5,15 @@ #include "structmember.h" #include "opcode.h" -static PyObject *gen_close(PyGenObject *gen, PyObject *args); +static PyObject *gen_close(PyGenObject *, PyObject *); +static PyObject *async_gen_asend_new(PyAsyncGenObject *, PyObject *); +static PyObject *async_gen_athrow_new(PyAsyncGenObject *, PyObject *); + +static char *NON_INIT_CORO_MSG = "can't send non-None value to a " + "just-started coroutine"; + +static char *ASYNC_GEN_IGNORED_EXIT_MSG = + "async generator ignored GeneratorExit"; static int gen_traverse(PyGenObject *gen, visitproc visit, void *arg) @@ -28,6 +36,26 @@ /* Generator isn't paused, so no need to close */ return; + if (PyAsyncGen_CheckExact(self)) { + PyAsyncGenObject *agen = (PyAsyncGenObject*)self; + PyObject *finalizer = agen->ag_finalizer; + if (finalizer && !agen->ag_closed) { + /* Save the current exception, if any. */ + PyErr_Fetch(&error_type, &error_value, &error_traceback); + + res = PyObject_CallFunctionObjArgs(finalizer, self, NULL); + + if (res == NULL) { + PyErr_WriteUnraisable(self); + } else { + Py_DECREF(res); + } + /* Restore the saved exception. */ + PyErr_Restore(error_type, error_value, error_traceback); + return; + } + } + /* Save the current exception, if any. */ PyErr_Fetch(&error_type, &error_value, &error_traceback); @@ -74,6 +102,12 @@ return; /* resurrected. :( */ _PyObject_GC_UNTRACK(self); + if (PyAsyncGen_CheckExact(gen)) { + /* We have to handle this case for asynchronous generators + right here, because this code has to be between UNTRACK + and GC_Del. */ + Py_CLEAR(((PyAsyncGenObject*)gen)->ag_finalizer); + } if (gen->gi_frame != NULL) { gen->gi_frame->f_gen = NULL; Py_CLEAR(gen->gi_frame); @@ -84,6 +118,33 @@ PyObject_GC_Del(gen); } +static void +gen_chain_runtime_error(const char *msg) +{ + PyObject *exc, *val, *val2, *tb; + + /* TODO: This about rewriting using _PyErr_ChainExceptions. */ + + PyErr_Fetch(&exc, &val, &tb); + PyErr_NormalizeException(&exc, &val, &tb); + if (tb != NULL) { + PyException_SetTraceback(val, tb); + } + + Py_DECREF(exc); + Py_XDECREF(tb); + + PyErr_SetString(PyExc_RuntimeError, msg); + PyErr_Fetch(&exc, &val2, &tb); + PyErr_NormalizeException(&exc, &val2, &tb); + + Py_INCREF(val); + PyException_SetCause(val2, val); + PyException_SetContext(val2, val); + + PyErr_Restore(exc, val2, tb); +} + static PyObject * gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing) { @@ -93,8 +154,12 @@ if (gen->gi_running) { char *msg = "generator already executing"; - if (PyCoro_CheckExact(gen)) + if (PyCoro_CheckExact(gen)) { msg = "coroutine already executing"; + } + else if (PyAsyncGen_CheckExact(gen)) { + msg = "async generator already executing"; + } PyErr_SetString(PyExc_ValueError, msg); return NULL; } @@ -106,10 +171,16 @@ PyErr_SetString( PyExc_RuntimeError, "cannot reuse already awaited coroutine"); - } else if (arg && !exc) { + } + else if (arg && !exc) { /* `gen` is an exhausted generator: only set exception if called from send(). */ - PyErr_SetNone(PyExc_StopIteration); + if (PyAsyncGen_CheckExact(gen)) { + PyErr_SetNone(PyExc_StopAsyncIteration); + } + else { + PyErr_SetNone(PyExc_StopIteration); + } } return NULL; } @@ -118,9 +189,13 @@ if (arg && arg != Py_None) { char *msg = "can't send non-None value to a " "just-started generator"; - if (PyCoro_CheckExact(gen)) + if (PyCoro_CheckExact(gen)) { + msg = NON_INIT_CORO_MSG; + } + else if (PyAsyncGen_CheckExact(gen)) { msg = "can't send non-None value to a " - "just-started coroutine"; + "just-started async generator"; + } PyErr_SetString(PyExc_TypeError, msg); return NULL; } @@ -152,10 +227,20 @@ if (result && f->f_stacktop == NULL) { if (result == Py_None) { /* Delay exception instantiation if we can */ - PyErr_SetNone(PyExc_StopIteration); - } else { + if (PyAsyncGen_CheckExact(gen)) { + PyErr_SetNone(PyExc_StopAsyncIteration); + } + else { + PyErr_SetNone(PyExc_StopIteration); + } + } + else { PyObject *e = PyObject_CallFunctionObjArgs( PyExc_StopIteration, result, NULL); + + /* Async generators cannot return anything but None */ + assert(!PyAsyncGen_CheckExact(gen)); + if (e != NULL) { PyErr_SetObject(PyExc_StopIteration, e); Py_DECREF(e); @@ -167,28 +252,38 @@ /* Check for __future__ generator_stop and conditionally turn * a leaking StopIteration into RuntimeError (with its cause * set appropriately). */ - if (gen->gi_code != NULL && ((PyCodeObject *)gen->gi_code)->co_flags & - (CO_FUTURE_GENERATOR_STOP | CO_COROUTINE | CO_ITERABLE_COROUTINE)) + + const int check_stop_iter_error_flags = CO_FUTURE_GENERATOR_STOP | + CO_COROUTINE | + CO_ITERABLE_COROUTINE | + CO_ASYNC_GENERATOR; + + if (gen->gi_code != NULL && + ((PyCodeObject *)gen->gi_code)->co_flags & + check_stop_iter_error_flags) { - PyObject *exc, *val, *val2, *tb; - char *msg = "generator raised StopIteration"; - if (PyCoro_CheckExact(gen)) + /* `gen` is either: + * a generator with CO_FUTURE_GENERATOR_STOP flag; + * a coroutine; + * a generator with CO_ITERABLE_COROUTINE flag + (decorated with types.coroutine decorator); + * an async generator. + */ + const char *msg = "generator raised StopIteration"; + if (PyCoro_CheckExact(gen)) { msg = "coroutine raised StopIteration"; - PyErr_Fetch(&exc, &val, &tb); - PyErr_NormalizeException(&exc, &val, &tb); - if (tb != NULL) - PyException_SetTraceback(val, tb); - Py_DECREF(exc); - Py_XDECREF(tb); - PyErr_SetString(PyExc_RuntimeError, msg); - PyErr_Fetch(&exc, &val2, &tb); - PyErr_NormalizeException(&exc, &val2, &tb); - Py_INCREF(val); - PyException_SetCause(val2, val); - PyException_SetContext(val2, val); - PyErr_Restore(exc, val2, tb); + } + else if PyAsyncGen_CheckExact(gen) { + msg = "async generator raised StopIteration"; + } + /* Raise a RuntimeError */ + gen_chain_runtime_error(msg); } else { + /* `gen` is an ordinary generator without + CO_FUTURE_GENERATOR_STOP flag. + */ + PyObject *exc, *val, *tb; /* Pop the exception before issuing a warning. */ @@ -207,6 +302,15 @@ } } } + else if (PyAsyncGen_CheckExact(gen) && !result && + PyErr_ExceptionMatches(PyExc_StopAsyncIteration)) + { + /* code in `gen` raised a StopAsyncIteration error: + raise a RuntimeError. + */ + const char *msg = "async generator raised StopAsyncIteration"; + gen_chain_runtime_error(msg); + } if (!result || f->f_stacktop == NULL) { /* generator can't be rerun, so release the frame */ @@ -253,17 +357,19 @@ PyObject *retval = NULL; _Py_IDENTIFIER(close); - if (PyGen_CheckExact(yf)) { + if (PyGen_CheckExact(yf) || PyCoro_CheckExact(yf)) { retval = gen_close((PyGenObject *)yf, NULL); if (retval == NULL) return -1; - } else { + } + else { PyObject *meth = _PyObject_GetAttrId(yf, &PyId_close); if (meth == NULL) { if (!PyErr_ExceptionMatches(PyExc_AttributeError)) PyErr_WriteUnraisable(yf); PyErr_Clear(); - } else { + } + else { retval = _PyObject_CallNoArg(meth); Py_DECREF(meth); if (retval == NULL) @@ -311,8 +417,11 @@ retval = gen_send_ex(gen, Py_None, 1, 1); if (retval) { char *msg = "generator ignored GeneratorExit"; - if (PyCoro_CheckExact(gen)) + if (PyCoro_CheckExact(gen)) { msg = "coroutine ignored GeneratorExit"; + } else if (PyAsyncGen_CheckExact(gen)) { + msg = ASYNC_GEN_IGNORED_EXIT_MSG; + } Py_DECREF(retval); PyErr_SetString(PyExc_RuntimeError, msg); return NULL; @@ -332,21 +441,22 @@ return next yielded value or raise StopIteration."); static PyObject * -gen_throw(PyGenObject *gen, PyObject *args) +_gen_throw(PyGenObject *gen, int close_on_genexit, + PyObject *typ, PyObject *val, PyObject *tb) { - PyObject *typ; - PyObject *tb = NULL; - PyObject *val = NULL; PyObject *yf = _PyGen_yf(gen); _Py_IDENTIFIER(throw); - if (!PyArg_UnpackTuple(args, "throw", 1, 3, &typ, &val, &tb)) - return NULL; - if (yf) { PyObject *ret; int err; - if (PyErr_GivenExceptionMatches(typ, PyExc_GeneratorExit)) { + if (PyErr_GivenExceptionMatches(typ, PyExc_GeneratorExit) && + close_on_genexit + ) { + /* Asynchronous generators *should not* be closed right away. + We have to allow some awaits to work it through, hence the + `close_on_genexit` parameter here. + */ gen->gi_running = 1; err = gen_close_iter(yf); gen->gi_running = 0; @@ -355,11 +465,16 @@ return gen_send_ex(gen, Py_None, 1, 0); goto throw_here; } - if (PyGen_CheckExact(yf)) { + if (PyGen_CheckExact(yf) || PyCoro_CheckExact(yf)) { + /* `yf` is a generator or a coroutine. */ gen->gi_running = 1; - ret = gen_throw((PyGenObject *)yf, args); + /* Close the generator that we are currently iterating with + 'yield from' or awaiting on with 'await'. */ + ret = _gen_throw((PyGenObject *)yf, close_on_genexit, + typ, val, tb); gen->gi_running = 0; } else { + /* `yf` is an iterator or a coroutine-like object. */ PyObject *meth = _PyObject_GetAttrId(yf, &PyId_throw); if (meth == NULL) { if (!PyErr_ExceptionMatches(PyExc_AttributeError)) { @@ -371,7 +486,7 @@ goto throw_here; } gen->gi_running = 1; - ret = PyObject_CallObject(meth, args); + ret = PyObject_CallFunctionObjArgs(meth, typ, val, tb, NULL); gen->gi_running = 0; Py_DECREF(meth); } @@ -454,6 +569,21 @@ static PyObject * +gen_throw(PyGenObject *gen, PyObject *args) +{ + PyObject *typ; + PyObject *tb = NULL; + PyObject *val = NULL; + + if (!PyArg_UnpackTuple(args, "throw", 1, 3, &typ, &val, &tb)) { + return NULL; + } + + return _gen_throw(gen, 1, typ, val, tb); +} + + +static PyObject * gen_iternext(PyGenObject *gen) { return gen_send_ex(gen, NULL, 0, 0); @@ -997,21 +1127,21 @@ typedef struct { PyObject_HEAD - PyObject *aw_aiter; + PyObject *ags_aiter; } PyAIterWrapper; static PyObject * aiter_wrapper_iternext(PyAIterWrapper *aw) { - PyErr_SetObject(PyExc_StopIteration, aw->aw_aiter); + PyErr_SetObject(PyExc_StopIteration, aw->ags_aiter); return NULL; } static int aiter_wrapper_traverse(PyAIterWrapper *aw, visitproc visit, void *arg) { - Py_VISIT((PyObject *)aw->aw_aiter); + Py_VISIT((PyObject *)aw->ags_aiter); return 0; } @@ -1019,7 +1149,7 @@ aiter_wrapper_dealloc(PyAIterWrapper *aw) { _PyObject_GC_UNTRACK((PyObject *)aw); - Py_CLEAR(aw->aw_aiter); + Py_CLEAR(aw->ags_aiter); PyObject_GC_Del(aw); } @@ -1081,7 +1211,817 @@ return NULL; } Py_INCREF(aiter); - aw->aw_aiter = aiter; + aw->ags_aiter = aiter; _PyObject_GC_TRACK(aw); return (PyObject *)aw; } + + +/* ========= Asynchronous Generators ========= */ + + +typedef enum { + AWAITABLE_STATE_INIT, /* new awaitable, has not yet been iterated */ + AWAITABLE_STATE_ITER, /* being iterated */ + AWAITABLE_STATE_CLOSED, /* closed */ +} AwaitableState; + + +typedef struct { + PyObject_HEAD + PyAsyncGenObject *ags_gen; + + /* Can be NULL, when in the __anext__() mode + (equivalent of "asend(None)") */ + PyObject *ags_sendval; + + AwaitableState ags_state; +} PyAsyncGenASend; + + +typedef struct { + PyObject_HEAD + PyAsyncGenObject *agt_gen; + + /* Can be NULL, when in the "aclose()" mode + (equivalent of "athrow(GeneratorExit)") */ + PyObject *agt_args; + + AwaitableState agt_state; +} PyAsyncGenAThrow; + + +typedef struct { + PyObject_HEAD + PyObject *agw_val; +} _PyAsyncGenWrappedValue; + + +#ifndef _PyAsyncGen_MAXFREELIST +#define _PyAsyncGen_MAXFREELIST 80 +#endif + +/* Freelists boost performance 6-10%; they also reduce memory + fragmentation, as _PyAsyncGenWrappedValue and PyAsyncGenASend + are short-living objects that are instantiated for every + __anext__ call. +*/ + +static _PyAsyncGenWrappedValue *ag_value_freelist[_PyAsyncGen_MAXFREELIST]; +static int ag_value_freelist_free = 0; + +static PyAsyncGenASend *ag_asend_freelist[_PyAsyncGen_MAXFREELIST]; +static int ag_asend_freelist_free = 0; + +#define _PyAsyncGenWrappedValue_CheckExact(o) \ + (Py_TYPE(o) == &_PyAsyncGenWrappedValue_Type) + +#define PyAsyncGenASend_CheckExact(o) \ + (Py_TYPE(o) == &_PyAsyncGenASend_Type) + + +static int +async_gen_traverse(PyAsyncGenObject *gen, visitproc visit, void *arg) +{ + Py_VISIT(gen->ag_finalizer); + return gen_traverse((PyGenObject*)gen, visit, arg); +} + + +static PyObject * +async_gen_repr(PyAsyncGenObject *o) +{ + return PyUnicode_FromFormat("", + o->ag_qualname, o); +} + + +static int +async_gen_init_hooks(PyAsyncGenObject *o) +{ + PyThreadState *tstate; + PyObject *finalizer; + PyObject *firstiter; + + if (o->ag_hooks_inited) { + return 0; + } + + o->ag_hooks_inited = 1; + + tstate = PyThreadState_GET(); + + finalizer = tstate->async_gen_finalizer; + if (finalizer) { + Py_INCREF(finalizer); + o->ag_finalizer = finalizer; + } + + firstiter = tstate->async_gen_firstiter; + if (firstiter) { + PyObject *res; + + Py_INCREF(firstiter); + res = PyObject_CallFunction(firstiter, "O", o); + Py_DECREF(firstiter); + if (res == NULL) { + return 1; + } + Py_DECREF(res); + } + + return 0; +} + + +static PyObject * +async_gen_anext(PyAsyncGenObject *o) +{ + if (async_gen_init_hooks(o)) { + return NULL; + } + return async_gen_asend_new(o, NULL); +} + + +static PyObject * +async_gen_asend(PyAsyncGenObject *o, PyObject *arg) +{ + if (async_gen_init_hooks(o)) { + return NULL; + } + return async_gen_asend_new(o, arg); +} + + +static PyObject * +async_gen_aclose(PyAsyncGenObject *o, PyObject *arg) +{ + if (async_gen_init_hooks(o)) { + return NULL; + } + return async_gen_athrow_new(o, NULL); +} + +static PyObject * +async_gen_athrow(PyAsyncGenObject *o, PyObject *args) +{ + if (async_gen_init_hooks(o)) { + return NULL; + } + return async_gen_athrow_new(o, args); +} + + +static PyGetSetDef async_gen_getsetlist[] = { + {"__name__", (getter)gen_get_name, (setter)gen_set_name, + PyDoc_STR("name of the async generator")}, + {"__qualname__", (getter)gen_get_qualname, (setter)gen_set_qualname, + PyDoc_STR("qualified name of the async generator")}, + {"ag_await", (getter)coro_get_cr_await, NULL, + PyDoc_STR("object being awaited on, or None")}, + {NULL} /* Sentinel */ +}; + +static PyMemberDef async_gen_memberlist[] = { + {"ag_frame", T_OBJECT, offsetof(PyAsyncGenObject, ag_frame), READONLY}, + {"ag_running", T_BOOL, offsetof(PyAsyncGenObject, ag_running), READONLY}, + {"ag_code", T_OBJECT, offsetof(PyAsyncGenObject, ag_code), READONLY}, + {NULL} /* Sentinel */ +}; + +PyDoc_STRVAR(async_aclose_doc, +"aclose() -> raise GeneratorExit inside generator."); + +PyDoc_STRVAR(async_asend_doc, +"asend(v) -> send 'v' in generator."); + +PyDoc_STRVAR(async_athrow_doc, +"athrow(typ[,val[,tb]]) -> raise exception in generator."); + +static PyMethodDef async_gen_methods[] = { + {"asend", (PyCFunction)async_gen_asend, METH_O, async_asend_doc}, + {"athrow",(PyCFunction)async_gen_athrow, METH_VARARGS, async_athrow_doc}, + {"aclose", (PyCFunction)async_gen_aclose, METH_NOARGS, async_aclose_doc}, + {NULL, NULL} /* Sentinel */ +}; + + +static PyAsyncMethods async_gen_as_async = { + 0, /* am_await */ + PyObject_SelfIter, /* am_aiter */ + (unaryfunc)async_gen_anext /* am_anext */ +}; + + +PyTypeObject PyAsyncGen_Type = { + PyVarObject_HEAD_INIT(&PyType_Type, 0) + "async_generator", /* tp_name */ + sizeof(PyAsyncGenObject), /* tp_basicsize */ + 0, /* tp_itemsize */ + /* methods */ + (destructor)gen_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + &async_gen_as_async, /* tp_as_async */ + (reprfunc)async_gen_repr, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_HAVE_FINALIZE, /* tp_flags */ + 0, /* tp_doc */ + (traverseproc)async_gen_traverse, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + offsetof(PyAsyncGenObject, ag_weakreflist), /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + async_gen_methods, /* tp_methods */ + async_gen_memberlist, /* tp_members */ + async_gen_getsetlist, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + 0, /* tp_new */ + 0, /* tp_free */ + 0, /* tp_is_gc */ + 0, /* tp_bases */ + 0, /* tp_mro */ + 0, /* tp_cache */ + 0, /* tp_subclasses */ + 0, /* tp_weaklist */ + 0, /* tp_del */ + 0, /* tp_version_tag */ + _PyGen_Finalize, /* tp_finalize */ +}; + + +PyObject * +PyAsyncGen_New(PyFrameObject *f, PyObject *name, PyObject *qualname) +{ + PyAsyncGenObject *o; + o = (PyAsyncGenObject *)gen_new_with_qualname( + &PyAsyncGen_Type, f, name, qualname); + if (o == NULL) { + return NULL; + } + o->ag_finalizer = NULL; + o->ag_closed = 0; + o->ag_hooks_inited = 0; + return (PyObject*)o; +} + + +int +PyAsyncGen_ClearFreeLists(void) +{ + int ret = ag_value_freelist_free + ag_asend_freelist_free; + + while (ag_value_freelist_free) { + _PyAsyncGenWrappedValue *o; + o = ag_value_freelist[--ag_value_freelist_free]; + assert(_PyAsyncGenWrappedValue_CheckExact(o)); + PyObject_Del(o); + } + + while (ag_asend_freelist_free) { + PyAsyncGenASend *o; + o = ag_asend_freelist[--ag_asend_freelist_free]; + assert(Py_TYPE(o) == &_PyAsyncGenASend_Type); + PyObject_Del(o); + } + + return ret; +} + +void +PyAsyncGen_Fini(void) +{ + PyAsyncGen_ClearFreeLists(); +} + + +static PyObject * +async_gen_unwrap_value(PyAsyncGenObject *gen, PyObject *result) +{ + if (result == NULL) { + if (!PyErr_Occurred()) { + PyErr_SetNone(PyExc_StopAsyncIteration); + } + + if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration) + || PyErr_ExceptionMatches(PyExc_GeneratorExit) + ) { + gen->ag_closed = 1; + } + + return NULL; + } + + if (_PyAsyncGenWrappedValue_CheckExact(result)) { + /* async yield */ + PyObject *e = PyObject_CallFunctionObjArgs( + PyExc_StopIteration, + ((_PyAsyncGenWrappedValue*)result)->agw_val, + NULL); + Py_DECREF(result); + if (e == NULL) { + return NULL; + } + PyErr_SetObject(PyExc_StopIteration, e); + Py_DECREF(e); + return NULL; + } + + return result; +} + + +/* ---------- Async Generator ASend Awaitable ------------ */ + + +static void +async_gen_asend_dealloc(PyAsyncGenASend *o) +{ + Py_CLEAR(o->ags_gen); + Py_CLEAR(o->ags_sendval); + if (ag_asend_freelist_free < _PyAsyncGen_MAXFREELIST) { + assert(PyAsyncGenASend_CheckExact(o)); + ag_asend_freelist[ag_asend_freelist_free++] = o; + } else { + PyObject_Del(o); + } +} + + +static PyObject * +async_gen_asend_send(PyAsyncGenASend *o, PyObject *arg) +{ + PyObject *result; + + if (o->ags_state == AWAITABLE_STATE_CLOSED) { + PyErr_SetNone(PyExc_StopIteration); + return NULL; + } + + if (o->ags_state == AWAITABLE_STATE_INIT) { + if (arg == NULL || arg == Py_None) { + arg = o->ags_sendval; + } + o->ags_state = AWAITABLE_STATE_ITER; + } + + result = gen_send_ex((PyGenObject*)o->ags_gen, arg, 0, 0); + result = async_gen_unwrap_value(o->ags_gen, result); + + if (result == NULL) { + o->ags_state = AWAITABLE_STATE_CLOSED; + } + + return result; +} + + +static PyObject * +async_gen_asend_iternext(PyAsyncGenASend *o) +{ + return async_gen_asend_send(o, NULL); +} + + +static PyObject * +async_gen_asend_throw(PyAsyncGenASend *o, PyObject *args) +{ + PyObject *result; + + if (o->ags_state == AWAITABLE_STATE_CLOSED) { + PyErr_SetNone(PyExc_StopIteration); + return NULL; + } + + result = gen_throw((PyGenObject*)o->ags_gen, args); + result = async_gen_unwrap_value(o->ags_gen, result); + + if (result == NULL) { + o->ags_state = AWAITABLE_STATE_CLOSED; + } + + return result; +} + + +static PyObject * +async_gen_asend_close(PyAsyncGenASend *o, PyObject *args) +{ + o->ags_state = AWAITABLE_STATE_CLOSED; + Py_RETURN_NONE; +} + + +static PyMethodDef async_gen_asend_methods[] = { + {"send", (PyCFunction)async_gen_asend_send, METH_O, send_doc}, + {"throw", (PyCFunction)async_gen_asend_throw, METH_VARARGS, throw_doc}, + {"close", (PyCFunction)async_gen_asend_close, METH_NOARGS, close_doc}, + {NULL, NULL} /* Sentinel */ +}; + + +static PyAsyncMethods async_gen_asend_as_async = { + PyObject_SelfIter, /* am_await */ + 0, /* am_aiter */ + 0 /* am_anext */ +}; + + +PyTypeObject _PyAsyncGenASend_Type = { + PyVarObject_HEAD_INIT(&PyType_Type, 0) + "async_generator_asend", /* tp_name */ + sizeof(PyAsyncGenASend), /* tp_basicsize */ + 0, /* tp_itemsize */ + /* methods */ + (destructor)async_gen_asend_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + &async_gen_asend_as_async, /* tp_as_async */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /* tp_flags */ + 0, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + PyObject_SelfIter, /* tp_iter */ + (iternextfunc)async_gen_asend_iternext, /* tp_iternext */ + async_gen_asend_methods, /* tp_methods */ + 0, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + 0, /* tp_new */ +}; + + +static PyObject * +async_gen_asend_new(PyAsyncGenObject *gen, PyObject *sendval) +{ + PyAsyncGenASend *o; + if (ag_asend_freelist_free) { + ag_asend_freelist_free--; + o = ag_asend_freelist[ag_asend_freelist_free]; + _Py_NewReference((PyObject *)o); + } else { + o = PyObject_New(PyAsyncGenASend, &_PyAsyncGenASend_Type); + if (o == NULL) { + return NULL; + } + } + + Py_INCREF(gen); + o->ags_gen = gen; + + Py_XINCREF(sendval); + o->ags_sendval = sendval; + + o->ags_state = AWAITABLE_STATE_INIT; + return (PyObject*)o; +} + + +/* ---------- Async Generator Value Wrapper ------------ */ + + +static void +async_gen_wrapped_val_dealloc(_PyAsyncGenWrappedValue *o) +{ + Py_CLEAR(o->agw_val); + if (ag_value_freelist_free < _PyAsyncGen_MAXFREELIST) { + assert(_PyAsyncGenWrappedValue_CheckExact(o)); + ag_value_freelist[ag_value_freelist_free++] = o; + } else { + PyObject_Del(o); + } +} + + +PyTypeObject _PyAsyncGenWrappedValue_Type = { + PyVarObject_HEAD_INIT(&PyType_Type, 0) + "async_generator_wrapped_value", /* tp_name */ + sizeof(_PyAsyncGenWrappedValue), /* tp_basicsize */ + 0, /* tp_itemsize */ + /* methods */ + (destructor)async_gen_wrapped_val_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_as_async */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /* tp_flags */ + 0, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + 0, /* tp_methods */ + 0, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + 0, /* tp_new */ +}; + + +PyObject * +_PyAsyncGenValueWrapperNew(PyObject *val) +{ + _PyAsyncGenWrappedValue *o; + assert(val); + + if (ag_value_freelist_free) { + ag_value_freelist_free--; + o = ag_value_freelist[ag_value_freelist_free]; + assert(_PyAsyncGenWrappedValue_CheckExact(o)); + _Py_NewReference((PyObject*)o); + } else { + o = PyObject_New(_PyAsyncGenWrappedValue, &_PyAsyncGenWrappedValue_Type); + if (o == NULL) { + return NULL; + } + } + o->agw_val = val; + Py_INCREF(val); + return (PyObject*)o; +} + + +/* ---------- Async Generator AThrow awaitable ------------ */ + + +static void +async_gen_athrow_dealloc(PyAsyncGenAThrow *o) +{ + Py_CLEAR(o->agt_gen); + Py_CLEAR(o->agt_args); + PyObject_Del(o); +} + + +static PyObject * +async_gen_athrow_send(PyAsyncGenAThrow *o, PyObject *arg) +{ + PyGenObject *gen = (PyGenObject*)o->agt_gen; + PyFrameObject *f = gen->gi_frame; + PyObject *retval; + + if (f == NULL || f->f_stacktop == NULL || + o->agt_state == AWAITABLE_STATE_CLOSED) { + PyErr_SetNone(PyExc_StopIteration); + return NULL; + } + + if (o->agt_state == AWAITABLE_STATE_INIT) { + if (o->agt_gen->ag_closed) { + PyErr_SetNone(PyExc_StopIteration); + return NULL; + } + + if (arg != Py_None) { + PyErr_SetString(PyExc_RuntimeError, NON_INIT_CORO_MSG); + return NULL; + } + + o->agt_state = AWAITABLE_STATE_ITER; + + if (o->agt_args == NULL) { + /* aclose() mode */ + o->agt_gen->ag_closed = 1; + + retval = _gen_throw((PyGenObject *)gen, + 0, /* Do not close generator when + PyExc_GeneratorExit is passed */ + PyExc_GeneratorExit, NULL, NULL); + + if (retval && _PyAsyncGenWrappedValue_CheckExact(retval)) { + Py_DECREF(retval); + goto yield_close; + } + } else { + PyObject *typ; + PyObject *tb = NULL; + PyObject *val = NULL; + + if (!PyArg_UnpackTuple(o->agt_args, "athrow", 1, 3, + &typ, &val, &tb)) { + return NULL; + } + + retval = _gen_throw((PyGenObject *)gen, + 0, /* Do not close generator when + PyExc_GeneratorExit is passed */ + typ, val, tb); + retval = async_gen_unwrap_value(o->agt_gen, retval); + } + if (retval == NULL) { + goto check_error; + } + return retval; + } + + assert(o->agt_state == AWAITABLE_STATE_ITER); + + retval = gen_send_ex((PyGenObject *)gen, arg, 0, 0); + if (o->agt_args) { + return async_gen_unwrap_value(o->agt_gen, retval); + } else { + /* aclose() mode */ + if (retval) { + if (_PyAsyncGenWrappedValue_CheckExact(retval)) { + Py_DECREF(retval); + goto yield_close; + } + else { + return retval; + } + } + else { + goto check_error; + } + } + +yield_close: + PyErr_SetString( + PyExc_RuntimeError, ASYNC_GEN_IGNORED_EXIT_MSG); + return NULL; + +check_error: + if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration) + || PyErr_ExceptionMatches(PyExc_GeneratorExit) + ) { + o->agt_state = AWAITABLE_STATE_CLOSED; + PyErr_Clear(); /* ignore these errors */ + PyErr_SetNone(PyExc_StopIteration); + } + return NULL; +} + + +static PyObject * +async_gen_athrow_throw(PyAsyncGenAThrow *o, PyObject *args) +{ + PyObject *retval; + + if (o->agt_state == AWAITABLE_STATE_INIT) { + PyErr_SetString(PyExc_RuntimeError, NON_INIT_CORO_MSG); + return NULL; + } + + if (o->agt_state == AWAITABLE_STATE_CLOSED) { + PyErr_SetNone(PyExc_StopIteration); + return NULL; + } + + retval = gen_throw((PyGenObject*)o->agt_gen, args); + if (o->agt_args) { + return async_gen_unwrap_value(o->agt_gen, retval); + } else { + /* aclose() mode */ + if (retval && _PyAsyncGenWrappedValue_CheckExact(retval)) { + Py_DECREF(retval); + PyErr_SetString(PyExc_RuntimeError, ASYNC_GEN_IGNORED_EXIT_MSG); + return NULL; + } + return retval; + } +} + + +static PyObject * +async_gen_athrow_iternext(PyAsyncGenAThrow *o) +{ + return async_gen_athrow_send(o, Py_None); +} + + +static PyObject * +async_gen_athrow_close(PyAsyncGenAThrow *o, PyObject *args) +{ + o->agt_state = AWAITABLE_STATE_CLOSED; + Py_RETURN_NONE; +} + + +static PyMethodDef async_gen_athrow_methods[] = { + {"send", (PyCFunction)async_gen_athrow_send, METH_O, send_doc}, + {"throw", (PyCFunction)async_gen_athrow_throw, METH_VARARGS, throw_doc}, + {"close", (PyCFunction)async_gen_athrow_close, METH_NOARGS, close_doc}, + {NULL, NULL} /* Sentinel */ +}; + + +static PyAsyncMethods async_gen_athrow_as_async = { + PyObject_SelfIter, /* am_await */ + 0, /* am_aiter */ + 0 /* am_anext */ +}; + + +PyTypeObject _PyAsyncGenAThrow_Type = { + PyVarObject_HEAD_INIT(&PyType_Type, 0) + "async_generator_athrow", /* tp_name */ + sizeof(PyAsyncGenAThrow), /* tp_basicsize */ + 0, /* tp_itemsize */ + /* methods */ + (destructor)async_gen_athrow_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + &async_gen_athrow_as_async, /* tp_as_async */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /* tp_flags */ + 0, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + PyObject_SelfIter, /* tp_iter */ + (iternextfunc)async_gen_athrow_iternext, /* tp_iternext */ + async_gen_athrow_methods, /* tp_methods */ + 0, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + 0, /* tp_new */ +}; + + +static PyObject * +async_gen_athrow_new(PyAsyncGenObject *gen, PyObject *args) +{ + PyAsyncGenAThrow *o; + o = PyObject_New(PyAsyncGenAThrow, &_PyAsyncGenAThrow_Type); + if (o == NULL) { + return NULL; + } + o->agt_gen = gen; + o->agt_args = args; + o->agt_state = AWAITABLE_STATE_INIT; + Py_INCREF(gen); + Py_XINCREF(args); + return (PyObject*)o; +} diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1204,7 +1204,7 @@ f->f_stacktop = NULL; /* remains NULL unless yield suspends frame */ f->f_executing = 1; - if (co->co_flags & (CO_GENERATOR | CO_COROUTINE)) { + if (co->co_flags & (CO_GENERATOR | CO_COROUTINE | CO_ASYNC_GENERATOR)) { if (!throwflag && f->f_exc_type != NULL && f->f_exc_type != Py_None) { /* We were in an except handler when we left, restore the exception state which was put aside @@ -2083,36 +2083,45 @@ PyObject *aiter = TOP(); PyTypeObject *type = Py_TYPE(aiter); - if (type->tp_as_async != NULL) - getter = type->tp_as_async->am_anext; - - if (getter != NULL) { - next_iter = (*getter)(aiter); - if (next_iter == NULL) { + if (PyAsyncGen_CheckExact(aiter)) { + awaitable = type->tp_as_async->am_anext(aiter); + if (awaitable == NULL) { goto error; } + } else { + if (type->tp_as_async != NULL){ + getter = type->tp_as_async->am_anext; + } + + if (getter != NULL) { + next_iter = (*getter)(aiter); + if (next_iter == NULL) { + goto error; + } + } + else { + PyErr_Format( + PyExc_TypeError, + "'async for' requires an iterator with " + "__anext__ method, got %.100s", + type->tp_name); + goto error; + } + + awaitable = _PyCoro_GetAwaitableIter(next_iter); + if (awaitable == NULL) { + PyErr_Format( + PyExc_TypeError, + "'async for' received an invalid object " + "from __anext__: %.100s", + Py_TYPE(next_iter)->tp_name); + + Py_DECREF(next_iter); + goto error; + } else { + Py_DECREF(next_iter); + } } - else { - PyErr_Format( - PyExc_TypeError, - "'async for' requires an iterator with " - "__anext__ method, got %.100s", - type->tp_name); - goto error; - } - - awaitable = _PyCoro_GetAwaitableIter(next_iter); - if (awaitable == NULL) { - PyErr_Format( - PyExc_TypeError, - "'async for' received an invalid object " - "from __anext__: %.100s", - Py_TYPE(next_iter)->tp_name); - - Py_DECREF(next_iter); - goto error; - } else - Py_DECREF(next_iter); PUSH(awaitable); PREDICT(LOAD_CONST); @@ -2187,6 +2196,17 @@ TARGET(YIELD_VALUE) { retval = POP(); + + if (co->co_flags & CO_ASYNC_GENERATOR) { + PyObject *w = _PyAsyncGenValueWrapperNew(retval); + Py_DECREF(retval); + if (w == NULL) { + retval = NULL; + goto error; + } + retval = w; + } + f->f_stacktop = stack_pointer; why = WHY_YIELD; goto fast_yield; @@ -3712,7 +3732,7 @@ assert((retval != NULL) ^ (PyErr_Occurred() != NULL)); fast_yield: - if (co->co_flags & (CO_GENERATOR | CO_COROUTINE)) { + if (co->co_flags & (CO_GENERATOR | CO_COROUTINE | CO_ASYNC_GENERATOR)) { /* The purpose of this block is to put aside the generator's exception state and restore that of the calling frame. If the current @@ -4156,8 +4176,8 @@ freevars[PyTuple_GET_SIZE(co->co_cellvars) + i] = o; } - /* Handle generator/coroutine */ - if (co->co_flags & (CO_GENERATOR | CO_COROUTINE)) { + /* Handle generator/coroutine/asynchronous generator */ + if (co->co_flags & (CO_GENERATOR | CO_COROUTINE | CO_ASYNC_GENERATOR)) { PyObject *gen; PyObject *coro_wrapper = tstate->coroutine_wrapper; int is_coro = co->co_flags & CO_COROUTINE; @@ -4182,6 +4202,8 @@ * and return that as the value. */ if (is_coro) { gen = PyCoro_New(f, name, qualname); + } else if (co->co_flags & CO_ASYNC_GENERATOR) { + gen = PyAsyncGen_New(f, name, qualname); } else { gen = PyGen_NewWithQualName(f, name, qualname); } @@ -4660,6 +4682,38 @@ return tstate->coroutine_wrapper; } +void +_PyEval_SetAsyncGenFirstiter(PyObject *firstiter) +{ + PyThreadState *tstate = PyThreadState_GET(); + + Py_XINCREF(firstiter); + Py_XSETREF(tstate->async_gen_firstiter, firstiter); +} + +PyObject * +_PyEval_GetAsyncGenFirstiter(void) +{ + PyThreadState *tstate = PyThreadState_GET(); + return tstate->async_gen_firstiter; +} + +void +_PyEval_SetAsyncGenFinalizer(PyObject *finalizer) +{ + PyThreadState *tstate = PyThreadState_GET(); + + Py_XINCREF(finalizer); + Py_XSETREF(tstate->async_gen_finalizer, finalizer); +} + +PyObject * +_PyEval_GetAsyncGenFinalizer(void) +{ + PyThreadState *tstate = PyThreadState_GET(); + return tstate->async_gen_finalizer; +} + PyObject * PyEval_GetBuiltins(void) { diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -1886,8 +1886,6 @@ return 0; } - if (is_async) - co->co_flags |= CO_COROUTINE; compiler_make_closure(c, co, funcflags, qualname); Py_DECREF(qualname); Py_DECREF(co); @@ -2801,6 +2799,9 @@ if (c->u->u_ste->ste_type != FunctionBlock) return compiler_error(c, "'return' outside function"); if (s->v.Return.value) { + if (c->u->u_ste->ste_coroutine && c->u->u_ste->ste_generator) + return compiler_error( + c, "'return' with value in async generator"); VISIT(c, expr, s->v.Return.value); } else @@ -4115,8 +4116,6 @@ case Yield_kind: if (c->u->u_ste->ste_type != FunctionBlock) return compiler_error(c, "'yield' outside function"); - if (c->u->u_scope_type == COMPILER_SCOPE_ASYNC_FUNCTION) - return compiler_error(c, "'yield' inside async function"); if (e->v.Yield.value) { VISIT(c, expr, e->v.Yield.value); } @@ -4992,8 +4991,12 @@ flags |= CO_NEWLOCALS | CO_OPTIMIZED; if (ste->ste_nested) flags |= CO_NESTED; - if (ste->ste_generator) + if (ste->ste_generator && !ste->ste_coroutine) flags |= CO_GENERATOR; + if (!ste->ste_generator && ste->ste_coroutine) + flags |= CO_COROUTINE; + if (ste->ste_generator && ste->ste_coroutine) + flags |= CO_ASYNC_GENERATOR; if (ste->ste_varargs) flags |= CO_VARARGS; if (ste->ste_varkeywords) diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -694,6 +694,7 @@ _PyGC_Fini(); _PyRandom_Fini(); _PyArg_Fini(); + PyAsyncGen_Fini(); /* Cleanup Unicode implementation */ _PyUnicode_Fini(); diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -229,6 +229,9 @@ tstate->in_coroutine_wrapper = 0; tstate->co_extra_user_count = 0; + tstate->async_gen_firstiter = NULL; + tstate->async_gen_finalizer = NULL; + if (init) _PyThreadState_Init(tstate); @@ -408,6 +411,8 @@ Py_CLEAR(tstate->c_traceobj); Py_CLEAR(tstate->coroutine_wrapper); + Py_CLEAR(tstate->async_gen_firstiter); + Py_CLEAR(tstate->async_gen_finalizer); } diff --git a/Python/symtable.c b/Python/symtable.c --- a/Python/symtable.c +++ b/Python/symtable.c @@ -63,6 +63,7 @@ ste->ste_nested = 1; ste->ste_child_free = 0; ste->ste_generator = 0; + ste->ste_coroutine = 0; ste->ste_returns_value = 0; ste->ste_needs_class_closure = 0; @@ -378,7 +379,7 @@ PyLong_AsLong(PyTuple_GET_ITEM(data, 2))); return 0; - } + } } PyErr_SetString(PyExc_RuntimeError, "BUG: internal directive bookkeeping broken"); @@ -1397,6 +1398,7 @@ FunctionBlock, (void *)s, s->lineno, s->col_offset)) VISIT_QUIT(st, 0); + st->st_cur->ste_coroutine = 1; VISIT(st, arguments, s->v.AsyncFunctionDef.args); VISIT_SEQ(st, stmt, s->v.AsyncFunctionDef.body); if (!symtable_exit_block(st, s)) @@ -1492,7 +1494,7 @@ break; case Await_kind: VISIT(st, expr, e->v.Await.value); - st->st_cur->ste_generator = 1; + st->st_cur->ste_coroutine = 1; break; case Compare_kind: VISIT(st, expr, e->v.Compare.left); diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -717,6 +717,113 @@ ); +static PyTypeObject AsyncGenHooksType; + +PyDoc_STRVAR(asyncgen_hooks_doc, +"asyncgen_hooks\n\ +\n\ +A struct sequence providing information about asynhronous\n\ +generators hooks. The attributes are read only."); + +static PyStructSequence_Field asyncgen_hooks_fields[] = { + {"firstiter", "Hook to intercept first iteration"}, + {"finalizer", "Hook to intercept finalization"}, + {0} +}; + +static PyStructSequence_Desc asyncgen_hooks_desc = { + "asyncgen_hooks", /* name */ + asyncgen_hooks_doc, /* doc */ + asyncgen_hooks_fields , /* fields */ + 2 +}; + + +static PyObject * +sys_set_asyncgen_hooks(PyObject *self, PyObject *args, PyObject *kw) +{ + static char *keywords[] = {"firstiter", "finalizer", NULL}; + PyObject *firstiter = NULL; + PyObject *finalizer = NULL; + + if (!PyArg_ParseTupleAndKeywords( + args, kw, "|OO", keywords, + &firstiter, &finalizer)) { + return NULL; + } + + if (finalizer && finalizer != Py_None) { + if (!PyCallable_Check(finalizer)) { + PyErr_Format(PyExc_TypeError, + "callable finalizer expected, got %.50s", + Py_TYPE(finalizer)->tp_name); + return NULL; + } + _PyEval_SetAsyncGenFinalizer(finalizer); + } + else if (finalizer == Py_None) { + _PyEval_SetAsyncGenFinalizer(NULL); + } + + if (firstiter && firstiter != Py_None) { + if (!PyCallable_Check(firstiter)) { + PyErr_Format(PyExc_TypeError, + "callable firstiter expected, got %.50s", + Py_TYPE(firstiter)->tp_name); + return NULL; + } + _PyEval_SetAsyncGenFirstiter(firstiter); + } + else if (firstiter == Py_None) { + _PyEval_SetAsyncGenFirstiter(NULL); + } + + Py_RETURN_NONE; +} + +PyDoc_STRVAR(set_asyncgen_hooks_doc, +"set_asyncgen_hooks(*, firstiter=None, finalizer=None)\n\ +\n\ +Set a finalizer for async generators objects." +); + +static PyObject * +sys_get_asyncgen_hooks(PyObject *self, PyObject *args) +{ + PyObject *res; + PyObject *firstiter = _PyEval_GetAsyncGenFirstiter(); + PyObject *finalizer = _PyEval_GetAsyncGenFinalizer(); + + res = PyStructSequence_New(&AsyncGenHooksType); + if (res == NULL) { + return NULL; + } + + if (firstiter == NULL) { + firstiter = Py_None; + } + + if (finalizer == NULL) { + finalizer = Py_None; + } + + Py_INCREF(firstiter); + PyStructSequence_SET_ITEM(res, 0, firstiter); + + Py_INCREF(finalizer); + PyStructSequence_SET_ITEM(res, 1, finalizer); + + return res; +} + +PyDoc_STRVAR(get_asyncgen_hooks_doc, +"get_asyncgen_hooks()\n\ +\n\ +Return a namedtuple of installed asynchronous generators hooks \ +(firstiter, finalizer)." +); + + static PyTypeObject Hash_InfoType; PyDoc_STRVAR(hash_info_doc, @@ -1315,6 +1422,10 @@ set_coroutine_wrapper_doc}, {"get_coroutine_wrapper", sys_get_coroutine_wrapper, METH_NOARGS, get_coroutine_wrapper_doc}, + {"set_asyncgen_hooks", sys_set_asyncgen_hooks, + METH_VARARGS | METH_KEYWORDS, set_asyncgen_hooks_doc}, + {"get_asyncgen_hooks", sys_get_asyncgen_hooks, METH_NOARGS, + get_asyncgen_hooks_doc}, {NULL, NULL} /* sentinel */ }; @@ -1950,6 +2061,14 @@ SET_SYS_FROM_STRING("thread_info", PyThread_GetInfo()); #endif + /* initialize asyncgen_hooks */ + if (AsyncGenHooksType.tp_name == NULL) { + if (PyStructSequence_InitType2( + &AsyncGenHooksType, &asyncgen_hooks_desc) < 0) { + return NULL; + } + } + #undef SET_SYS_FROM_STRING #undef SET_SYS_FROM_STRING_BORROW if (PyErr_Occurred()) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 02:38:22 2016 From: python-checkins at python.org (yury.selivanov) Date: Fri, 09 Sep 2016 06:38:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_ceval=3A_tighten_the_code_?= =?utf-8?q?of_STORE=5FANNOTATION?= Message-ID: <20160909063822.48739.97825.25D21EB3@psf.io> https://hg.python.org/cpython/rev/28923a58a17e changeset: 103393:28923a58a17e user: Yury Selivanov date: Thu Sep 08 23:38:21 2016 -0700 summary: ceval: tighten the code of STORE_ANNOTATION files: Python/ceval.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1921,11 +1921,10 @@ err = PyObject_SetItem(ann_dict, name, ann); } Py_DECREF(ann_dict); + Py_DECREF(ann); if (err != 0) { - Py_DECREF(ann); goto error; } - Py_DECREF(ann); DISPATCH(); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 03:02:52 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 09 Sep 2016 07:02:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327106=3A_Add_test?= =?utf-8?q?_for_configparser=2E=5F=5Fall=5F=5F?= Message-ID: <20160909070252.18872.78001.A55D3B13@psf.io> https://hg.python.org/cpython/rev/739528288996 changeset: 103394:739528288996 user: Martin Panter date: Fri Sep 09 06:46:48 2016 +0000 summary: Issue #27106: Add test for configparser.__all__ Patch by Jacek Ko?odziej. The Error class is deliberately omitted because it is a generic name and of limited use. files: Lib/test/test_configparser.py | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py --- a/Lib/test/test_configparser.py +++ b/Lib/test/test_configparser.py @@ -2052,5 +2052,11 @@ self.assertEqual(cfg['two'].getlen('one'), 5) +class MiscTestCase(unittest.TestCase): + def test__all__(self): + blacklist = {"Error"} + support.check__all__(self, configparser, blacklist=blacklist) + + if __name__ == '__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 03:05:59 2016 From: python-checkins at python.org (yury.selivanov) Date: Fri, 09 Sep 2016 07:05:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328003=3A_Fix_a_co?= =?utf-8?q?mpiler_warning?= Message-ID: <20160909070559.87341.59844.907BDDBB@psf.io> https://hg.python.org/cpython/rev/910e293663cb changeset: 103395:910e293663cb user: Yury Selivanov date: Fri Sep 09 00:05:42 2016 -0700 summary: Issue #28003: Fix a compiler warning files: Python/sysmodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -1422,7 +1422,7 @@ set_coroutine_wrapper_doc}, {"get_coroutine_wrapper", sys_get_coroutine_wrapper, METH_NOARGS, get_coroutine_wrapper_doc}, - {"set_asyncgen_hooks", sys_set_asyncgen_hooks, + {"set_asyncgen_hooks", (PyCFunction)sys_set_asyncgen_hooks, METH_VARARGS | METH_KEYWORDS, set_asyncgen_hooks_doc}, {"get_asyncgen_hooks", sys_get_asyncgen_hooks, METH_NOARGS, get_asyncgen_hooks_doc}, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 05:55:40 2016 From: python-checkins at python.org (martin.panter) Date: Fri, 09 Sep 2016 09:55:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327364=3A_Raw_stri?= =?utf-8?q?ngs_to_avoid_deprecated_escaping_in_com2ann=2Epy?= Message-ID: <20160909095534.48417.89667.2ECCC10F@psf.io> https://hg.python.org/cpython/rev/98a57845c8cc changeset: 103396:98a57845c8cc user: Martin Panter date: Fri Sep 09 07:38:50 2016 +0000 summary: Issue #27364: Raw strings to avoid deprecated escaping in com2ann.py files: Tools/parser/com2ann.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Tools/parser/com2ann.py b/Tools/parser/com2ann.py --- a/Tools/parser/com2ann.py +++ b/Tools/parser/com2ann.py @@ -10,8 +10,8 @@ __all__ = ['com2ann', 'TYPE_COM'] -TYPE_COM = re.compile('\s*#\s*type\s*:.*$', flags=re.DOTALL) -TRAIL_OR_COM = re.compile('\s*$|\s*#.*$', flags=re.DOTALL) +TYPE_COM = re.compile(r'\s*#\s*type\s*:.*$', flags=re.DOTALL) +TRAIL_OR_COM = re.compile(r'\s*$|\s*#.*$', flags=re.DOTALL) class _Data: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 11:35:13 2016 From: python-checkins at python.org (donald.stufft) Date: Fri, 09 Sep 2016 15:35:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Upgrade_setupt?= =?utf-8?q?ools_to_27=2E1=2E1?= Message-ID: <20160909153510.87341.53113.201B288B@psf.io> https://hg.python.org/cpython/rev/02ee6e457061 changeset: 103397:02ee6e457061 branch: 3.4 parent: 103109:556a11c11edd user: Donald Stufft date: Fri Sep 09 11:35:02 2016 -0400 summary: Upgrade setuptools to 27.1.1 files: Lib/ensurepip/__init__.py | 2 +- Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl | Bin 3 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,7 +8,7 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "25.2.0" +_SETUPTOOLS_VERSION = "27.1.1" _PIP_VERSION = "8.1.2" diff --git a/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl deleted file mode 100644 index 02c8ce8737bea57315cfff3b6b3b1a49eadacc6e..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8e4ddc4f1ad0a77ec09744858abee256263a2664 GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 11:37:08 2016 From: python-checkins at python.org (donald.stufft) Date: Fri, 09 Sep 2016 15:37:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20160909153704.48439.28669.3F382EFA@psf.io> https://hg.python.org/cpython/rev/f8340e2991e9 changeset: 103399:f8340e2991e9 parent: 103396:98a57845c8cc parent: 103398:d1ff6a039ab3 user: Donald Stufft date: Fri Sep 09 11:36:59 2016 -0400 summary: Merge 3.5 files: Lib/ensurepip/__init__.py | 2 +- Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl | Bin 3 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,7 +8,7 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "25.2.0" +_SETUPTOOLS_VERSION = "27.1.1" _PIP_VERSION = "8.1.2" diff --git a/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl deleted file mode 100644 index 02c8ce8737bea57315cfff3b6b3b1a49eadacc6e..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8e4ddc4f1ad0a77ec09744858abee256263a2664 GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 11:37:09 2016 From: python-checkins at python.org (donald.stufft) Date: Fri, 09 Sep 2016 15:37:09 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4?= Message-ID: <20160909153704.21237.92801.C9996B89@psf.io> https://hg.python.org/cpython/rev/d1ff6a039ab3 changeset: 103398:d1ff6a039ab3 branch: 3.5 parent: 103386:468961cea562 parent: 103397:02ee6e457061 user: Donald Stufft date: Fri Sep 09 11:35:43 2016 -0400 summary: Merge 3.4 files: Lib/ensurepip/__init__.py | 2 +- Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl | Bin 3 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,7 +8,7 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "25.2.0" +_SETUPTOOLS_VERSION = "27.1.1" _PIP_VERSION = "8.1.2" diff --git a/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl deleted file mode 100644 index 02c8ce8737bea57315cfff3b6b3b1a49eadacc6e..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8e4ddc4f1ad0a77ec09744858abee256263a2664 GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 11:57:11 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 15:57:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Revert_=2327959=3A_ImportE?= =?utf-8?q?rror_within_an_encoding_module_should_also_skip_the?= Message-ID: <20160909155708.2706.87923.EB094EBE@psf.io> https://hg.python.org/cpython/rev/4e80a157ea66 changeset: 103400:4e80a157ea66 user: Steve Dower date: Fri Sep 09 08:56:37 2016 -0700 summary: Revert #27959: ImportError within an encoding module should also skip the encoding files: Lib/encodings/__init__.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py --- a/Lib/encodings/__init__.py +++ b/Lib/encodings/__init__.py @@ -98,9 +98,10 @@ # module with side-effects that is not in the 'encodings' package. mod = __import__('encodings.' + modname, fromlist=_import_tail, level=0) - except ModuleNotFoundError as ex: - if ex.name != 'encodings.' + modname: - raise + except ImportError: + # ImportError may occur because 'encodings.(modname)' does not exist, + # or because it imports a name that does not exist (see mbcs and oem) + pass else: break else: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:03:25 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 16:03:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327781=3A_Fixes_un?= =?utf-8?q?initialized_fd_when_!MS=5FWINDOWS_and_!HAVE=5FOPENAT?= Message-ID: <20160909160324.48515.90546.C2B82A9A@psf.io> https://hg.python.org/cpython/rev/801634d3c105 changeset: 103401:801634d3c105 user: Steve Dower date: Fri Sep 09 09:03:15 2016 -0700 summary: Issue #27781: Fixes uninitialized fd when !MS_WINDOWS and !HAVE_OPENAT files: Modules/posixmodule.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -7477,13 +7477,14 @@ Py_BEGIN_ALLOW_THREADS #ifdef MS_WINDOWS fd = _wopen(path->wide, flags, mode); -#endif +#else #ifdef HAVE_OPENAT if (dir_fd != DEFAULT_DIR_FD) fd = openat(dir_fd, path->narrow, flags, mode); else +#endif /* HAVE_OPENAT */ fd = open(path->narrow, flags, mode); -#endif +#endif /* !MS_WINDOWS */ Py_END_ALLOW_THREADS } while (fd < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); _Py_END_SUPPRESS_IPH -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:06:47 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 09 Sep 2016 16:06:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328038=3A_Remove_T?= =?utf-8?q?ools/parser/com2ann=2Epy_and_its_unit_test=2E?= Message-ID: <20160909160646.8758.32714.B0AABB0E@psf.io> https://hg.python.org/cpython/rev/35447332ab19 changeset: 103402:35447332ab19 user: Guido van Rossum date: Fri Sep 09 09:06:11 2016 -0700 summary: Issue #28038: Remove Tools/parser/com2ann.py and its unit test. Development is moving to https://github.com/ilevkivskyi/com2ann files: Lib/test/test_tools/test_com2ann.py | 260 ------------- Tools/parser/com2ann.py | 308 ---------------- 2 files changed, 0 insertions(+), 568 deletions(-) diff --git a/Lib/test/test_tools/test_com2ann.py b/Lib/test/test_tools/test_com2ann.py deleted file mode 100644 --- a/Lib/test/test_tools/test_com2ann.py +++ /dev/null @@ -1,260 +0,0 @@ -"""Tests for the com2ann.py script in the Tools/parser directory.""" - -import unittest -import test.support -import os -import re - -from test.test_tools import basepath, toolsdir, skip_if_missing - -skip_if_missing() - -parser_path = os.path.join(toolsdir, "parser") - -with test.support.DirsOnSysPath(parser_path): - from com2ann import * - -class BaseTestCase(unittest.TestCase): - - def check(self, code, expected, n=False, e=False): - self.assertEqual(com2ann(code, - drop_None=n, drop_Ellipsis=e, silent=True), - expected) - -class SimpleTestCase(BaseTestCase): - # Tests for basic conversions - - def test_basics(self): - self.check("z = 5", "z = 5") - self.check("z: int = 5", "z: int = 5") - self.check("z = 5 # type: int", "z: int = 5") - self.check("z = 5 # type: int # comment", - "z: int = 5 # comment") - - def test_type_ignore(self): - self.check("foobar = foobaz() #type: ignore", - "foobar = foobaz() #type: ignore") - self.check("a = 42 #type: ignore #comment", - "a = 42 #type: ignore #comment") - - def test_complete_tuple(self): - self.check("t = 1, 2, 3 # type: Tuple[int, ...]", - "t: Tuple[int, ...] = (1, 2, 3)") - self.check("t = 1, # type: Tuple[int]", - "t: Tuple[int] = (1,)") - self.check("t = (1, 2, 3) # type: Tuple[int, ...]", - "t: Tuple[int, ...] = (1, 2, 3)") - - def test_drop_None(self): - self.check("x = None # type: int", - "x: int", True) - self.check("x = None # type: int # another", - "x: int # another", True) - self.check("x = None # type: int # None", - "x: int # None", True) - - def test_drop_Ellipsis(self): - self.check("x = ... # type: int", - "x: int", False, True) - self.check("x = ... # type: int # another", - "x: int # another", False, True) - self.check("x = ... # type: int # ...", - "x: int # ...", False, True) - - def test_newline(self): - self.check("z = 5 # type: int\r\n", "z: int = 5\r\n") - self.check("z = 5 # type: int # comment\x85", - "z: int = 5 # comment\x85") - - def test_wrong(self): - self.check("#type : str", "#type : str") - self.check("x==y #type: bool", "x==y #type: bool") - - def test_pattern(self): - for line in ["#type: int", " # type: str[:] # com"]: - self.assertTrue(re.search(TYPE_COM, line)) - for line in ["", "#", "# comment", "#type", "type int:"]: - self.assertFalse(re.search(TYPE_COM, line)) - -class BigTestCase(BaseTestCase): - # Tests for really crazy formatting, to be sure - # that script works reasonably in extreme situations - - def test_crazy(self): - self.maxDiff = None - self.check(crazy_code, big_result, False, False) - self.check(crazy_code, big_result_ne, True, True) - -crazy_code = """\ -# -*- coding: utf-8 -*- # this should not be spoiled -''' -Docstring here -''' - -import testmod -x = 5 #type : int # this one is OK -ttt \\ - = \\ - 1.0, \\ - 2.0, \\ - 3.0, #type: Tuple[float, float, float] -with foo(x==1) as f: #type: str - print(f) - -for i, j in my_inter(x=1): # type: ignore - i + j # type: int # what about this - -x = y = z = 1 # type: int -x, y, z = [], [], [] # type: (List[int], List[int], List[str]) -class C: - - - l[f(x - =1)] = [ - - 1, - 2, - ] # type: List[int] - - - (C.x[1]) = \\ - 42 == 5# type: bool -lst[...] = \\ - ((\\ -...)) # type: int # comment .. - -y = ... # type: int # comment ... -z = ... -#type: int - - -#DONE placement of annotation after target rather than before = - -TD.x[1] \\ - = 0 == 5# type: bool - -TD.y[1] =5 == 5# type: bool # one more here -F[G(x == y, - -# hm... - - z)]\\ - = None # type: OMG[int] # comment: None -x = None#type:int #comment : None""" - -big_result = """\ -# -*- coding: utf-8 -*- # this should not be spoiled -''' -Docstring here -''' - -import testmod -x: int = 5 # this one is OK -ttt: Tuple[float, float, float] \\ - = \\ - (1.0, \\ - 2.0, \\ - 3.0,) -with foo(x==1) as f: #type: str - print(f) - -for i, j in my_inter(x=1): # type: ignore - i + j # type: int # what about this - -x = y = z = 1 # type: int -x, y, z = [], [], [] # type: (List[int], List[int], List[str]) -class C: - - - l[f(x - =1)]: List[int] = [ - - 1, - 2, - ] - - - (C.x[1]): bool = \\ - 42 == 5 -lst[...]: int = \\ - ((\\ -...)) # comment .. - -y: int = ... # comment ... -z = ... -#type: int - - -#DONE placement of annotation after target rather than before = - -TD.x[1]: bool \\ - = 0 == 5 - -TD.y[1]: bool =5 == 5 # one more here -F[G(x == y, - -# hm... - - z)]: OMG[int]\\ - = None # comment: None -x: int = None #comment : None""" - -big_result_ne = """\ -# -*- coding: utf-8 -*- # this should not be spoiled -''' -Docstring here -''' - -import testmod -x: int = 5 # this one is OK -ttt: Tuple[float, float, float] \\ - = \\ - (1.0, \\ - 2.0, \\ - 3.0,) -with foo(x==1) as f: #type: str - print(f) - -for i, j in my_inter(x=1): # type: ignore - i + j # type: int # what about this - -x = y = z = 1 # type: int -x, y, z = [], [], [] # type: (List[int], List[int], List[str]) -class C: - - - l[f(x - =1)]: List[int] = [ - - 1, - 2, - ] - - - (C.x[1]): bool = \\ - 42 == 5 -lst[...]: int \\ - \\ - # comment .. - -y: int # comment ... -z = ... -#type: int - - -#DONE placement of annotation after target rather than before = - -TD.x[1]: bool \\ - = 0 == 5 - -TD.y[1]: bool =5 == 5 # one more here -F[G(x == y, - -# hm... - - z)]: OMG[int]\\ - # comment: None -x: int #comment : None""" - -if __name__ == '__main__': - unittest.main() diff --git a/Tools/parser/com2ann.py b/Tools/parser/com2ann.py deleted file mode 100644 --- a/Tools/parser/com2ann.py +++ /dev/null @@ -1,308 +0,0 @@ -"""Helper module to tranlate 3.5 type comments to 3.6 variable annotations.""" -import re -import os -import ast -import argparse -import tokenize -from collections import defaultdict -from textwrap import dedent -from io import BytesIO - -__all__ = ['com2ann', 'TYPE_COM'] - -TYPE_COM = re.compile(r'\s*#\s*type\s*:.*$', flags=re.DOTALL) -TRAIL_OR_COM = re.compile(r'\s*$|\s*#.*$', flags=re.DOTALL) - - -class _Data: - """Internal class describing global data on file.""" - def __init__(self, lines, tokens): - self.lines = lines - self.tokens = tokens - ttab = defaultdict(list) # maps line number to token numbers - for i, tok in enumerate(tokens): - ttab[tok.start[0]].append(i) - self.ttab = ttab - self.success = [] # list of lines where type comments where processed - self.fail = [] # list of lines where type comments where rejected - - -def skip_blank(d, lno): - while d.lines[lno].strip() == '': - lno += 1 - return lno - - -def find_start(d, lcom): - """Find first char of the assignment target.""" - i = d.ttab[lcom + 1][-2] # index of type comment token in tokens list - while ((d.tokens[i].exact_type != tokenize.NEWLINE) and - (d.tokens[i].exact_type != tokenize.ENCODING)): - i -= 1 - lno = d.tokens[i].start[0] - return skip_blank(d, lno) - - -def check_target(stmt): - if len(stmt.body): - assign = stmt.body[0] - else: - return False - if isinstance(assign, ast.Assign) and len(assign.targets) == 1: - targ = assign.targets[0] - else: - return False - if (isinstance(targ, ast.Name) or isinstance(targ, ast.Attribute) - or isinstance(targ, ast.Subscript)): - return True - return False - - -def find_eq(d, lstart): - """Find equal sign starting from lstart taking care about d[f(x=1)] = 5.""" - col = pars = 0 - lno = lstart - while d.lines[lno][col] != '=' or pars != 0: - ch = d.lines[lno][col] - if ch in '([{': - pars += 1 - elif ch in ')]}': - pars -= 1 - if ch == '#' or col == len(d.lines[lno])-1: - lno = skip_blank(d, lno+1) - col = 0 - else: - col += 1 - return lno, col - - -def find_val(d, poseq): - """Find position of first char of assignment value starting from poseq.""" - lno, col = poseq - while (d.lines[lno][col].isspace() or d.lines[lno][col] in '=\\'): - if col == len(d.lines[lno])-1: - lno += 1 - col = 0 - else: - col += 1 - return lno, col - - -def find_targ(d, poseq): - """Find position of last char of target (annotation goes here).""" - lno, col = poseq - while (d.lines[lno][col].isspace() or d.lines[lno][col] in '=\\'): - if col == 0: - lno -= 1 - col = len(d.lines[lno])-1 - else: - col -= 1 - return lno, col+1 - - -def trim(new_lines, string, ltarg, poseq, lcom, ccom): - """Remove None or Ellipsis from assignment value. - - Also remove parens if one has (None), (...) etc. - string -- 'None' or '...' - ltarg -- line where last char of target is located - poseq -- position of equal sign - lcom, ccom -- position of type comment - """ - nopars = lambda s: s.replace('(', '').replace(')', '') - leq, ceq = poseq - end = ccom if leq == lcom else len(new_lines[leq]) - subline = new_lines[leq][:ceq] - if leq == ltarg: - subline = subline.rstrip() - new_lines[leq] = subline + (new_lines[leq][end:] if leq == lcom - else new_lines[leq][ceq+1:end]) - - for lno in range(leq+1,lcom): - new_lines[lno] = nopars(new_lines[lno]) - - if lcom != leq: - subline = nopars(new_lines[lcom][:ccom]).replace(string, '') - if (not subline.isspace()): - subline = subline.rstrip() - new_lines[lcom] = subline + new_lines[lcom][ccom:] - - -def _com2ann(d, drop_None, drop_Ellipsis): - new_lines = d.lines[:] - for lcom, line in enumerate(d.lines): - match = re.search(TYPE_COM, line) - if match: - # strip " # type : annotation \n" -> "annotation \n" - tp = match.group().lstrip()[1:].lstrip()[4:].lstrip()[1:].lstrip() - submatch = re.search(TRAIL_OR_COM, tp) - subcom = '' - if submatch and submatch.group(): - subcom = submatch.group() - tp = tp[:submatch.start()] - if tp == 'ignore': - continue - ccom = match.start() - if not any(d.tokens[i].exact_type == tokenize.COMMENT - for i in d.ttab[lcom + 1]): - d.fail.append(lcom) - continue # type comment inside string - lstart = find_start(d, lcom) - stmt_str = dedent(''.join(d.lines[lstart:lcom+1])) - try: - stmt = ast.parse(stmt_str) - except SyntaxError: - d.fail.append(lcom) - continue # for or with statements - if not check_target(stmt): - d.fail.append(lcom) - continue - - d.success.append(lcom) - val = stmt.body[0].value - - # writing output now - poseq = find_eq(d, lstart) - lval, cval = find_val(d, poseq) - ltarg, ctarg = find_targ(d, poseq) - - op_par = '' - cl_par = '' - if isinstance(val, ast.Tuple): - if d.lines[lval][cval] != '(': - op_par = '(' - cl_par = ')' - # write the comment first - new_lines[lcom] = d.lines[lcom][:ccom].rstrip() + cl_par + subcom - ccom = len(d.lines[lcom][:ccom].rstrip()) - - string = False - if isinstance(val, ast.Tuple): - # t = 1, 2 -> t = (1, 2); only latter is allowed with annotation - free_place = int(new_lines[lval][cval-2:cval] == ' ') - new_lines[lval] = (new_lines[lval][:cval-free_place] + - op_par + new_lines[lval][cval:]) - elif isinstance(val, ast.Ellipsis) and drop_Ellipsis: - string = '...' - elif (isinstance(val, ast.NameConstant) and - val.value is None and drop_None): - string = 'None' - if string: - trim(new_lines, string, ltarg, poseq, lcom, ccom) - - # finally write an annotation - new_lines[ltarg] = (new_lines[ltarg][:ctarg] + - ': ' + tp + new_lines[ltarg][ctarg:]) - return ''.join(new_lines) - - -def com2ann(code, *, drop_None=False, drop_Ellipsis=False, silent=False): - """Translate type comments to type annotations in code. - - Take code as string and return this string where:: - - variable = value # type: annotation # real comment - - is translated to:: - - variable: annotation = value # real comment - - For unsupported syntax cases, the type comments are - left intact. If drop_None is True or if drop_Ellipsis - is True translate correcpondingly:: - - variable = None # type: annotation - variable = ... # type: annotation - - into:: - - variable: annotation - - The tool tries to preserve code formatting as much as - possible, but an exact translation is not guarateed. - A summary of translated comments id printed by default. - """ - try: - ast.parse(code) # we want to work only with file without syntax errors - except SyntaxError: - return None - lines = code.splitlines(keepends=True) - rl = BytesIO(code.encode('utf-8')).readline - tokens = list(tokenize.tokenize(rl)) - - data = _Data(lines, tokens) - new_code = _com2ann(data, drop_None, drop_Ellipsis) - - if not silent: - if data.success: - print('Comments translated on lines:', - ', '.join(str(lno+1) for lno in data.success)) - if data.fail: - print('Comments rejected on lines:', - ', '.join(str(lno+1) for lno in data.fail)) - if not data.success and not data.fail: - print('No type comments found') - - return new_code - - -def translate_file(infile, outfile, dnone, dell, silent): - try: - descr = tokenize.open(infile) - except SyntaxError: - print("Cannot open", infile) - return - with descr as f: - code = f.read() - enc = f.encoding - if not silent: - print('File:', infile) - new_code = com2ann(code, drop_None=dnone, - drop_Ellipsis=dell, - silent=silent) - if new_code is None: - print("SyntaxError in", infile) - return - with open(outfile, 'wb') as f: - f.write((new_code).encode(enc)) - - -if __name__ == '__main__': - - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("-o", "--outfile", - help="output file, will be overwritten if exists,\n" - "defaults to input file") - parser.add_argument("infile", - help="input file or directory for translation, must\n" - "contain no syntax errors, for directory\n" - "the outfile is ignored and translation is\n" - "made in place") - parser.add_argument("-s", "--silent", - help="Do not print summary for line numbers of\n" - "translated and rejected comments", - action="store_true") - parser.add_argument("-n", "--drop-none", - help="drop any None as assignment value during\n" - "translation if it is annotated by a type coment", - action="store_true") - parser.add_argument("-e", "--drop-ellipsis", - help="drop any Ellipsis (...) as assignment value during\n" - "translation if it is annotated by a type coment", - action="store_true") - args = parser.parse_args() - if args.outfile is None: - args.outfile = args.infile - - if os.path.isfile(args.infile): - translate_file(args.infile, args.outfile, - args.drop_none, args.drop_ellipsis, args.silent) - else: - for root, dirs, files in os.walk(args.infile): - for afile in files: - _, ext = os.path.splitext(afile) - if ext == '.py' or ext == '.pyi': - fname = os.path.join(root, afile) - translate_file(fname, fname, - args.drop_none, args.drop_ellipsis, - args.silent) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:10:14 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 09 Sep 2016 16:10:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_duplicate_entry_for?= =?utf-8?q?_Ivan_L=2E?= Message-ID: <20160909161006.2012.33114.4C5B0A67@psf.io> https://hg.python.org/cpython/rev/57afe873e5bf changeset: 103403:57afe873e5bf user: Guido van Rossum date: Fri Sep 09 09:09:19 2016 -0700 summary: Remove duplicate entry for Ivan L. files: Misc/ACKS | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -468,7 +468,6 @@ Robin Friedrich Bradley Froehle Ivan Frohne -Ivan Levkivskyi Matthias Fuchs Jim Fulton Tadayoshi Funaba -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:14:17 2016 From: python-checkins at python.org (donald.stufft) Date: Fri, 09 Sep 2016 16:14:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E4=29=3A_Upgrade_setupt?= =?utf-8?q?ools_to_27=2E1=2E2?= Message-ID: <20160909161414.1800.91810.8FA925E0@psf.io> https://hg.python.org/cpython/rev/c853260f6223 changeset: 103404:c853260f6223 branch: 3.4 parent: 103397:02ee6e457061 user: Donald Stufft date: Fri Sep 09 12:08:53 2016 -0400 summary: Upgrade setuptools to 27.1.2 files: Lib/ensurepip/__init__.py | 2 +- Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl | Bin 3 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,7 +8,7 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "27.1.1" +_SETUPTOOLS_VERSION = "27.1.2" _PIP_VERSION = "8.1.2" diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl deleted file mode 100644 index 8e4ddc4f1ad0a77ec09744858abee256263a2664..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..de213bb72715caa07d12e94f97033110c1767e8d GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:15:14 2016 From: python-checkins at python.org (donald.stufft) Date: Fri, 09 Sep 2016 16:15:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41?= Message-ID: <20160909161512.59651.43302.36059672@psf.io> https://hg.python.org/cpython/rev/fdb3537e633a changeset: 103406:fdb3537e633a parent: 103403:57afe873e5bf parent: 103405:ff942f8a8f49 user: Donald Stufft date: Fri Sep 09 12:15:07 2016 -0400 summary: Merge 3.5 files: Lib/ensurepip/__init__.py | 2 +- Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl | Bin 3 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,7 +8,7 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "27.1.1" +_SETUPTOOLS_VERSION = "27.1.2" _PIP_VERSION = "8.1.2" diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl deleted file mode 100644 index 8e4ddc4f1ad0a77ec09744858abee256263a2664..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..de213bb72715caa07d12e94f97033110c1767e8d GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:15:16 2016 From: python-checkins at python.org (donald.stufft) Date: Fri, 09 Sep 2016 16:15:16 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAobWVyZ2UgMy40IC0+IDMuNSk6?= =?utf-8?q?_Merge_3=2E4?= Message-ID: <20160909161512.19423.58669.6C2302EC@psf.io> https://hg.python.org/cpython/rev/ff942f8a8f49 changeset: 103405:ff942f8a8f49 branch: 3.5 parent: 103398:d1ff6a039ab3 parent: 103404:c853260f6223 user: Donald Stufft date: Fri Sep 09 12:14:43 2016 -0400 summary: Merge 3.4 files: Lib/ensurepip/__init__.py | 2 +- Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl | Bin 3 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -8,7 +8,7 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "27.1.1" +_SETUPTOOLS_VERSION = "27.1.2" _PIP_VERSION = "8.1.2" diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.1-py2.py3-none-any.whl deleted file mode 100644 index 8e4ddc4f1ad0a77ec09744858abee256263a2664..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..de213bb72715caa07d12e94f97033110c1767e8d GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:16:30 2016 From: python-checkins at python.org (donald.stufft) Date: Fri, 09 Sep 2016 16:16:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Upgrade_setupt?= =?utf-8?q?ools_to_27=2E1=2E2?= Message-ID: <20160909161627.21076.94871.566336A6@psf.io> https://hg.python.org/cpython/rev/47c1ace69d02 changeset: 103407:47c1ace69d02 branch: 2.7 parent: 103391:3cce329c20e6 user: Donald Stufft date: Fri Sep 09 12:16:12 2016 -0400 summary: Upgrade setuptools to 27.1.2 files: Lib/ensurepip/__init__.py | 2 +- Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl | Bin Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl | Bin 3 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py --- a/Lib/ensurepip/__init__.py +++ b/Lib/ensurepip/__init__.py @@ -12,7 +12,7 @@ __all__ = ["version", "bootstrap"] -_SETUPTOOLS_VERSION = "25.2.0" +_SETUPTOOLS_VERSION = "27.1.2" _PIP_VERSION = "8.1.2" diff --git a/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-25.2.0-py2.py3-none-any.whl deleted file mode 100644 index 02c8ce8737bea57315cfff3b6b3b1a49eadacc6e..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch [stripped] diff --git a/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-27.1.2-py2.py3-none-any.whl new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..de213bb72715caa07d12e94f97033110c1767e8d GIT binary patch [stripped] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:21:16 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 16:21:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixes_expected_error_when_?= =?utf-8?q?getting_encoding_while_shutting_down=2E?= Message-ID: <20160909162114.69700.7895.BFE157A6@psf.io> https://hg.python.org/cpython/rev/8fa335f0eabf changeset: 103409:8fa335f0eabf user: Steve Dower date: Fri Sep 09 09:21:01 2016 -0700 summary: Fixes expected error when getting encoding while shutting down. files: Lib/test/test_io.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -3276,7 +3276,7 @@ class PyTextIOWrapperTest(TextIOWrapperTest): io = pyio - shutdown_error = "ImportError: sys.meta_path is None, Python is likely shutting down" + shutdown_error = "LookupError: unknown encoding: ascii" class IncrementalNewlineDecoderTest(unittest.TestCase): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:23:11 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 16:23:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Changes_pyvenv=2Ecfg_trick?= =?utf-8?q?_into_an_actual_sys=2Epath_file=2E?= Message-ID: <20160909161830.13046.69255.FF720A51@psf.io> https://hg.python.org/cpython/rev/03517dd54977 changeset: 103408:03517dd54977 parent: 103406:fdb3537e633a user: Steve Dower date: Fri Sep 09 09:17:35 2016 -0700 summary: Changes pyvenv.cfg trick into an actual sys.path file. files: Doc/using/windows.rst | 53 ++++--- Lib/site.py | 6 - PC/getpathp.c | 181 +++++++++++++++--------- PCbuild/pythoncore.vcxproj | 2 +- Tools/msi/make_zip.py | 11 +- 5 files changed, 154 insertions(+), 99 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -29,13 +29,13 @@ As specified in :pep:`11`, a Python release only supports a Windows platform while Microsoft considers the platform under extended support. This means that -Python 3.5 supports Windows Vista and newer. If you require Windows XP support +Python 3.6 supports Windows Vista and newer. If you require Windows XP support then please install Python 3.4. Installation Steps ------------------ -Four Python 3.5 installers are available for download - two each for the 32-bit +Four Python 3.6 installers are available for download - two each for the 32-bit and 64-bit versions of the interpreter. The *web installer* is a small initial download, and it will automatically download the required components as necessary. The *offline installer* includes the components necessary for a @@ -193,13 +193,13 @@ For example, to silently install a default, system-wide Python installation, you could use the following command (from an elevated command prompt):: - python-3.5.0.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0 + python-3.6.0.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0 To allow users to easily install a personal copy of Python without the test suite, you could provide a shortcut with the following command. This will display a simplified initial page and disallow customization:: - python-3.5.0.exe InstallAllUsers=0 Include_launcher=0 Include_test=0 + python-3.6.0.exe InstallAllUsers=0 Include_launcher=0 Include_test=0 SimpleInstall=1 SimpleInstallDescription="Just for me, no test suite." (Note that omitting the launcher also omits file associations, and is only @@ -234,13 +234,13 @@ useful to have a locally cached copy. Execute the following command from Command Prompt to download all possible -required files. Remember to substitute ``python-3.5.0.exe`` for the actual +required files. Remember to substitute ``python-3.6.0.exe`` for the actual name of your installer, and to create layouts in their own directories to avoid collisions between files with the same name. :: - python-3.5.0.exe /layout [optional target directory] + python-3.6.0.exe /layout [optional target directory] You may also specify the ``/quiet`` option to hide the progress display. @@ -345,7 +345,7 @@ To temporarily set environment variables, open Command Prompt and use the :command:`set` command:: - C:\>set PATH=C:\Program Files\Python 3.5;%PATH% + C:\>set PATH=C:\Program Files\Python 3.6;%PATH% C:\>set PYTHONPATH=%PYTHONPATH%;C:\My_python_lib C:\>python @@ -401,10 +401,10 @@ Besides using the automatically created start menu entry for the Python interpreter, you might want to start Python in the command prompt. The -installer for Python 3.5 and later has an option to set that up for you. +installer for Python 3.6 has an option to set that up for you. -On the first page of the installer, an option labelled "Add Python 3.5 to -PATH" can be selected to have the installer add the install location into the +On the first page of the installer, an option labelled "Add Python to PATH" +may be selected to have the installer add the install location into the :envvar:`PATH`. The location of the :file:`Scripts\\` folder is also added. This allows you to type :command:`python` to run the interpreter, and :command:`pip` for the package installer. Thus, you can also execute your @@ -418,7 +418,7 @@ example variable could look like this (assuming the first two entries already existed):: - C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files\Python 3.5 + C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files\Python 3.6 .. _launcher: @@ -720,7 +720,15 @@ :file:`C:\\Python\\Lib\\` and third-party modules should be stored in :file:`C:\\Python\\Lib\\site-packages\\`. -This is how :data:`sys.path` is populated on Windows: +To completely override :data:`sys.path`, create a text file named ``'sys.path'`` +containing a list of paths alongside the Python executable. This will ignore all +registry settings and environment variables, enable isolated mode, disable +importing :mod:`site`, and fill :data:`sys.path` with exactly the paths listed +in the file. Paths may be absolute or relative to the directory containing the +file. + +When the ``'sys.path'`` file is missing, this is how :data:`sys.path` is +populated on Windows: * An empty entry is added at the start, which corresponds to the current directory. @@ -755,10 +763,6 @@ path is used instead of the path to the main executable when deducing the home location. -* If ``applocal`` is set to true, the ``home`` property or the main executable - is always used as the home path, and all environment variables or registry - values affecting the path are ignored. The landmark file is not checked. - The end result of all this is: * When running :file:`python.exe`, or any other .exe in the main Python @@ -777,13 +781,11 @@ For those who want to bundle Python into their application or distribution, the following advice will prevent conflicts with other installations: -* Include a ``pyvenv.cfg`` file alongside your executable containing - ``applocal = true``. This will ensure that your own directory will be used to - resolve paths even if you have included the standard library in a ZIP file. - It will also ignore user site-packages and other paths listed in the - registry. +* Include a ``sys.path`` file alongside your executable containing the + directories to include. This will ignore user site-packages and other paths + listed in the registry or in environment variables. -* If you are loading :file:`python3.dll` or :file:`python35.dll` in your own +* If you are loading :file:`python3.dll` or :file:`python36.dll` in your own executable, explicitly call :c:func:`Py_SetPath` or (at least) :c:func:`Py_SetProgramName` before :c:func:`Py_Initialize`. @@ -801,6 +803,11 @@ the first suggestion is the best, as the other may still be susceptible to non-standard paths in the registry and user site-packages. +.. versionchanged:: 3.6 + + Adds ``sys.path`` file support and removes ``applocal`` option from + ``pyvenv.cfg``. + Additional modules ================== @@ -900,7 +907,7 @@ When extracted, the embedded distribution is (almost) fully isolated from the user's system, including environment variables, system registry settings, and installed packages. The standard library is included as pre-compiled and -optimized ``.pyc`` files in a ZIP, and ``python3.dll``, ``python35.dll``, +optimized ``.pyc`` files in a ZIP, and ``python3.dll``, ``python36.dll``, ``python.exe`` and ``pythonw.exe`` are all provided. Tcl/tk (including all dependants, such as Idle), pip and the Python documentation are not included. diff --git a/Lib/site.py b/Lib/site.py --- a/Lib/site.py +++ b/Lib/site.py @@ -463,12 +463,6 @@ system_site = value.lower() elif key == 'home': sys._home = value - elif key == 'applocal' and value.lower() == 'true': - # App-local installs use the exe_dir as prefix, - # not one level higher, and do not use system - # site packages. - site_prefix = exe_dir - system_site = 'false' sys.prefix = sys.exec_prefix = site_prefix diff --git a/PC/getpathp.c b/PC/getpathp.c --- a/PC/getpathp.c +++ b/PC/getpathp.c @@ -6,7 +6,8 @@ PATH RULES FOR WINDOWS: This describes how sys.path is formed on Windows. It describes the functionality, not the implementation (ie, the order in which these - are actually fetched is different) + are actually fetched is different). The presence of a sys.path file + alongside the program overrides these rules - see below. * Python always adds an empty entry at the start, which corresponds to the current directory. @@ -36,6 +37,12 @@ used (eg. .\Lib;.\plat-win, etc) + If a sys.path file exists adjacent to python.exe, it must contain a + list of paths to add to sys.path, one per line (like a .pth file but without + the ability to execute arbitrary code). Each path is relative to the + directory containing the file. No other paths are added to the search path, + and the registry finder is not enabled. + The end result of all this is: * When running python.exe, or any other .exe in the main Python directory (either an installed version, or directly from the PCbuild directory), @@ -52,7 +59,10 @@ some default, but relative, paths. * An embedding application can use Py_SetPath() to override all of - these authomatic path computations. + these automatic path computations. + + * An isolation install of Python can disable all implicit paths by + providing a sys.path file. ---------------------------------------------------------------- */ @@ -61,9 +71,12 @@ #include "osdefs.h" #include -#ifdef MS_WINDOWS +#ifndef MS_WINDOWS +#error getpathp.c should only be built on Windows +#endif + #include -#endif +#include #ifdef HAVE_SYS_TYPES_H #include @@ -163,24 +176,30 @@ than MAXPATHLEN characters at exit. If stuff is too long, only as much of stuff as fits will be appended. */ + +static int _PathCchCombineEx_Initialized = 0; +typedef HRESULT(__stdcall *PPathCchCombineEx)(PWSTR pszPathOut, size_t cchPathOut, PCWSTR pszPathIn, PCWSTR pszMore, unsigned long dwFlags); +static PPathCchCombineEx _PathCchCombineEx; + static void join(wchar_t *buffer, const wchar_t *stuff) { - size_t n; - if (is_sep(stuff[0]) || - (wcsnlen_s(stuff, 4) >= 3 && stuff[1] == ':' && is_sep(stuff[2]))) { - if (wcscpy_s(buffer, MAXPATHLEN+1, stuff) != 0) - Py_FatalError("buffer overflow in getpathp.c's join()"); - return; + if (_PathCchCombineEx_Initialized == 0) { + HMODULE pathapi = LoadLibraryW(L"api-ms-win-core-path-l1-1-0.dll"); + if (pathapi) + _PathCchCombineEx = (PPathCchCombineEx)GetProcAddress(pathapi, "PathCchCombineEx"); + else + _PathCchCombineEx = NULL; + _PathCchCombineEx_Initialized = 1; } - n = wcsnlen_s(buffer, MAXPATHLEN+1); - if (n > 0 && !is_sep(buffer[n - 1]) && n < MAXPATHLEN) { - buffer[n] = SEP; - buffer[n + 1] = '\0'; + if (_PathCchCombineEx) { + if (FAILED(_PathCchCombineEx(buffer, MAXPATHLEN+1, buffer, stuff, 0))) + Py_FatalError("buffer overflow in getpathp.c's join()"); + } else { + if (!PathCombineW(buffer, NULL, stuff)) + Py_FatalError("buffer overflow in getpathp.c's join()"); } - if (wcscat_s(buffer, MAXPATHLEN+1, stuff) != 0) - Py_FatalError("buffer overflow in getpathp.c's join()"); } /* gotlandmark only called by search_for_prefix, which ensures @@ -214,7 +233,6 @@ return 0; } -#ifdef MS_WINDOWS #ifdef Py_ENABLE_SHARED /* a string loaded from the DLL at startup.*/ @@ -369,7 +387,6 @@ return retval; } #endif /* Py_ENABLE_SHARED */ -#endif /* MS_WINDOWS */ static void get_progpath(void) @@ -378,7 +395,6 @@ wchar_t *path = _wgetenv(L"PATH"); wchar_t *prog = Py_GetProgramName(); -#ifdef MS_WINDOWS #ifdef Py_ENABLE_SHARED extern HANDLE PyWin_DLLhModule; /* static init of progpath ensures final char remains \0 */ @@ -390,7 +406,6 @@ #endif if (GetModuleFileNameW(NULL, progpath, MAXPATHLEN)) return; -#endif if (prog == NULL || *prog == '\0') prog = L"python"; @@ -483,6 +498,67 @@ return result; } +static int +read_sys_path_file(const wchar_t *path, const wchar_t *prefix) +{ + FILE *sp_file = _Py_wfopen(path, L"r"); + if (sp_file == NULL) + return -1; + + size_t bufsiz = MAXPATHLEN; + size_t prefixlen = wcslen(prefix); + + wchar_t *buf = (wchar_t*)PyMem_RawMalloc(bufsiz * sizeof(wchar_t)); + buf[0] = '\0'; + + while (!feof(sp_file)) { + char line[MAXPATHLEN + 1]; + char *p = fgets(line, MAXPATHLEN + 1, sp_file); + if (!p) + break; + + DWORD n = strlen(line); + if (n == 0 || p[n - 1] != '\n') + break; + if (n > 2 && p[n - 1] == '\r') + --n; + + DWORD wn = MultiByteToWideChar(CP_UTF8, 0, line, n - 1, NULL, 0); + wchar_t *wline = (wchar_t*)PyMem_RawMalloc((wn + 1) * sizeof(wchar_t)); + wn = MultiByteToWideChar(CP_UTF8, 0, line, n - 1, wline, wn); + wline[wn] = '\0'; + + while (wn + prefixlen + 4 > bufsiz) { + bufsiz += MAXPATHLEN; + buf = (wchar_t*)PyMem_RawRealloc(buf, (bufsiz + 1) * sizeof(wchar_t)); + if (!buf) { + PyMem_RawFree(wline); + goto error; + } + } + + if (buf[0]) + wcscat_s(buf, bufsiz, L";"); + wchar_t *b = &buf[wcslen(buf)]; + + wcscat_s(buf, bufsiz, prefix); + join(b, wline); + + PyMem_RawFree(wline); + } + + module_search_path = buf; + + fclose(sp_file); + return 0; + +error: + PyMem_RawFree(buf); + fclose(sp_file); + return -1; +} + + static void calculate_path(void) { @@ -492,32 +568,34 @@ wchar_t *pythonhome = Py_GetPythonHome(); wchar_t *envpath = NULL; -#ifdef MS_WINDOWS int skiphome, skipdefault; wchar_t *machinepath = NULL; wchar_t *userpath = NULL; wchar_t zip_path[MAXPATHLEN+1]; - int applocal = 0; if (!Py_IgnoreEnvironmentFlag) { envpath = _wgetenv(L"PYTHONPATH"); } -#else - char *_envpath = Py_GETENV("PYTHONPATH"); - wchar_t wenvpath[MAXPATHLEN+1]; - if (_envpath) { - size_t r = mbstowcs(wenvpath, _envpath, MAXPATHLEN+1); - envpath = wenvpath; - if (r == (size_t)-1 || r >= MAXPATHLEN) - envpath = NULL; - } -#endif get_progpath(); /* progpath guaranteed \0 terminated in MAXPATH+1 bytes. */ wcscpy_s(argv0_path, MAXPATHLEN+1, progpath); reduce(argv0_path); + /* Search for a sys.path file */ + { + wchar_t spbuffer[MAXPATHLEN+1]; + + wcscpy_s(spbuffer, MAXPATHLEN+1, argv0_path); + join(spbuffer, L"sys.path"); + if (exists(spbuffer) && read_sys_path_file(spbuffer, argv0_path) == 0) { + wcscpy_s(prefix, MAXPATHLEN + 1, argv0_path); + Py_IsolatedFlag = 1; + Py_NoSiteFlag = 1; + return; + } + } + /* Search for an environment configuration file, first in the executable's directory and then in the parent directory. If found, open it for use when searching for prefixes. @@ -543,17 +621,6 @@ } } if (env_file != NULL) { - /* Look for an 'applocal' variable and, if true, ignore all registry - * keys and environment variables, but retain the default paths - * (DLLs, Lib) and the zip file. Setting pythonhome here suppresses - * the search for LANDMARK below and overrides %PYTHONHOME%. - */ - if (find_env_config_value(env_file, L"applocal", tmpbuffer) && - (applocal = (wcsicmp(tmpbuffer, L"true") == 0))) { - envpath = NULL; - pythonhome = argv0_path; - } - /* Look for a 'home' variable and set argv0_path to it, if found */ if (find_env_config_value(env_file, L"home", tmpbuffer)) { wcscpy_s(argv0_path, MAXPATHLEN+1, tmpbuffer); @@ -576,7 +643,6 @@ envpath = NULL; -#ifdef MS_WINDOWS /* Calculate zip archive path from DLL or exe path */ if (wcscpy_s(zip_path, MAXPATHLEN+1, dllpath[0] ? dllpath : progpath)) /* exceeded buffer length - ignore zip_path */ @@ -590,16 +656,13 @@ skiphome = pythonhome==NULL ? 0 : 1; #ifdef Py_ENABLE_SHARED - if (!applocal) { - machinepath = getpythonregpath(HKEY_LOCAL_MACHINE, skiphome); - userpath = getpythonregpath(HKEY_CURRENT_USER, skiphome); - } + machinepath = getpythonregpath(HKEY_LOCAL_MACHINE, skiphome); + userpath = getpythonregpath(HKEY_CURRENT_USER, skiphome); #endif /* We only use the default relative PYTHONPATH if we havent anything better to use! */ skipdefault = envpath!=NULL || pythonhome!=NULL || \ machinepath!=NULL || userpath!=NULL; -#endif /* We need to construct a path from the following parts. (1) the PYTHONPATH environment variable, if set; @@ -612,7 +675,6 @@ Extra rules: - If PYTHONHOME is set (in any way) item (3) is ignored. - If registry values are used, (4) and (5) are ignored. - - If applocal is set, (1), (3), and registry values are ignored */ /* Calculate size of return buffer */ @@ -629,13 +691,11 @@ bufsz = 0; bufsz += wcslen(PYTHONPATH) + 1; bufsz += wcslen(argv0_path) + 1; -#ifdef MS_WINDOWS - if (!applocal && userpath) + if (userpath) bufsz += wcslen(userpath) + 1; - if (!applocal && machinepath) + if (machinepath) bufsz += wcslen(machinepath) + 1; bufsz += wcslen(zip_path) + 1; -#endif if (envpath != NULL) bufsz += wcslen(envpath) + 1; @@ -651,10 +711,8 @@ fprintf(stderr, "Using default static path.\n"); module_search_path = PYTHONPATH; } -#ifdef MS_WINDOWS PyMem_RawFree(machinepath); PyMem_RawFree(userpath); -#endif /* MS_WINDOWS */ return; } @@ -664,7 +722,6 @@ buf = wcschr(buf, L'\0'); *buf++ = DELIM; } -#ifdef MS_WINDOWS if (zip_path[0]) { if (wcscpy_s(buf, bufsz - (buf - module_search_path), zip_path)) Py_FatalError("buffer overflow in getpathp.c's calculate_path()"); @@ -692,15 +749,7 @@ buf = wcschr(buf, L'\0'); *buf++ = DELIM; } - } -#else - if (pythonhome == NULL) { - wcscpy(buf, PYTHONPATH); - buf = wcschr(buf, L'\0'); - *buf++ = DELIM; - } -#endif /* MS_WINDOWS */ - else { + } else { wchar_t *p = PYTHONPATH; wchar_t *q; size_t n; diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -69,7 +69,7 @@ _USRDLL;Py_BUILD_CORE;Py_ENABLE_SHARED;MS_DLL_ID="$(SysWinVer)";%(PreprocessorDefinitions) - ws2_32.lib;%(AdditionalDependencies) + shlwapi.lib;ws2_32.lib;%(AdditionalDependencies) 0x1e000000 diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -46,6 +46,10 @@ 'python3stub', } +EXCLUDED_FILES = { + 'pyshellext', +} + def is_not_debug(p): if DEBUG_RE.search(p.name): return False @@ -53,7 +57,7 @@ if TKTCL_RE.search(p.name): return False - return p.stem.lower() not in DEBUG_FILES + return p.stem.lower() not in DEBUG_FILES and p.stem.lower() not in EXCLUDED_FILES def is_not_debug_or_python(p): return is_not_debug(p) and not PYTHON_DLL_RE.search(p.name) @@ -209,8 +213,9 @@ copied = copy_to_layout(temp / t.rstrip('/'), rglob(s, p, c)) print('Copied {} files'.format(copied)) - with open(str(temp / 'pyvenv.cfg'), 'w') as f: - print('applocal = true', file=f) + with open(str(temp / 'sys.path'), 'w') as f: + print('python{0.major}{0.minor}.zip'.format(sys.version_info), file=f) + print('.', file=f) if out: total = copy_to_layout(out, rglob(temp, '**/*', None)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:33:49 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 16:33:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_credit_Raymond?= Message-ID: <20160909163347.8696.92679.5EFA942D@psf.io> https://hg.python.org/cpython/rev/b75ea284d981 changeset: 103410:b75ea284d981 user: Benjamin Peterson date: Fri Sep 09 09:33:23 2016 -0700 summary: credit Raymond files: Doc/whatsnew/3.6.rst | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -404,7 +404,9 @@ * :func:`dict` now uses a "compact" representation `pioneered by PyPy `_. :pep:`468` (Preserving the order of ``**kwargs`` in a function.) is - implemented by this. (Contributed by INADA Naoki in :issue:`27350`.) + implemented by this. (Contributed by INADA Naoki in :issue:`27350`. Idea + `originally suggested by Raymond Hettinger + `_.) * Long sequences of repeated traceback lines are now abbreviated as ``"[Previous line repeated {count} more times]"`` (see -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:41:04 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 16:41:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Switch_to_using_=7Cversion?= =?utf-8?q?=7C_substitition_in_Windows_docs=2E?= Message-ID: <20160909164102.8596.97895.DF16F4DF@psf.io> https://hg.python.org/cpython/rev/7e2726acd3f1 changeset: 103411:7e2726acd3f1 user: Steve Dower date: Fri Sep 09 09:40:06 2016 -0700 summary: Switch to using |version| substitition in Windows docs. files: Doc/using/windows.rst | 18 +++++++++--------- 1 files changed, 9 insertions(+), 9 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -29,15 +29,15 @@ As specified in :pep:`11`, a Python release only supports a Windows platform while Microsoft considers the platform under extended support. This means that -Python 3.6 supports Windows Vista and newer. If you require Windows XP support -then please install Python 3.4. +Python |version| supports Windows Vista and newer. If you require Windows XP +support then please install Python 3.4. Installation Steps ------------------ -Four Python 3.6 installers are available for download - two each for the 32-bit -and 64-bit versions of the interpreter. The *web installer* is a small initial -download, and it will automatically download the required components as +Four Python |version| installers are available for download - two each for the +32-bit and 64-bit versions of the interpreter. The *web installer* is a small +initial download, and it will automatically download the required components as necessary. The *offline installer* includes the components necessary for a default installation and only requires an internet connection for optional features. See :ref:`install-layout-option` for other ways to avoid downloading @@ -401,7 +401,7 @@ Besides using the automatically created start menu entry for the Python interpreter, you might want to start Python in the command prompt. The -installer for Python 3.6 has an option to set that up for you. +installer has an option to set that up for you. On the first page of the installer, an option labelled "Add Python to PATH" may be selected to have the installer add the install location into the @@ -458,9 +458,9 @@ started - it can be exited as normal, and any additional command-line arguments specified will be sent directly to Python. -If you have multiple versions of Python installed (e.g., 2.7 and 3.6) you -will have noticed that Python 3.6 was started - to launch Python 2.7, try the -command: +If you have multiple versions of Python installed (e.g., 2.7 and |version|) you +will have noticed that Python |version| was started - to launch Python 2.7, try +the command: :: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:47:23 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 16:47:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2NTEz?= =?utf-8?q?=3A_Fixes_platform_module_detection_of_Windows_Server?= Message-ID: <20160909164722.8602.62665.02595783@psf.io> https://hg.python.org/cpython/rev/aeb39d4475c5 changeset: 103412:aeb39d4475c5 branch: 3.5 parent: 103405:ff942f8a8f49 user: Steve Dower date: Fri Sep 09 09:46:56 2016 -0700 summary: Issue #26513: Fixes platform module detection of Windows Server files: Lib/platform.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -586,7 +586,7 @@ csd = 'SP' + csd[13:] # VER_NT_SERVER = 3 - if getattr(winver, 'product', None) == 3: + if getattr(winver, 'product_type', None) == 3: release = (_WIN32_SERVER_RELEASES.get((maj, min)) or _WIN32_SERVER_RELEASES.get((maj, None)) or release) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:47:31 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 16:47:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326513=3A_Fixes_platform_module_detection_of_Win?= =?utf-8?q?dows_Server?= Message-ID: <20160909164722.2012.42755.C3676C13@psf.io> https://hg.python.org/cpython/rev/67c50dd3fcea changeset: 103413:67c50dd3fcea parent: 103411:7e2726acd3f1 parent: 103412:aeb39d4475c5 user: Steve Dower date: Fri Sep 09 09:47:09 2016 -0700 summary: Issue #26513: Fixes platform module detection of Windows Server files: Lib/platform.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -585,7 +585,7 @@ csd = 'SP' + csd[13:] # VER_NT_SERVER = 3 - if getattr(winver, 'product', None) == 3: + if getattr(winver, 'product_type', None) == 3: release = (_WIN32_SERVER_RELEASES.get((maj, min)) or _WIN32_SERVER_RELEASES.get((maj, None)) or release) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:53:18 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 09 Sep 2016 16:53:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Move_news_items_for_PEP_52?= =?utf-8?q?6_and_525_to_the_top_of_their_section=2E?= Message-ID: <20160909165315.21048.79118.64B7615D@psf.io> https://hg.python.org/cpython/rev/f6a0b80ade26 changeset: 103414:f6a0b80ade26 user: Guido van Rossum date: Fri Sep 09 09:29:42 2016 -0700 summary: Move news items for PEP 526 and 525 to the top of their section. (News items should be ordered newest-first within their section.) files: Misc/NEWS | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,11 @@ Core and Builtins ----------------- +- Issue #28003: Implement PEP 525 -- Asynchronous Generators. + +- Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. + Patch by Ivan Levkivskyi. + - Issue #26058: Add a new private version to the builtin dict type, incremented at each dictionary creation and at each dictionary change. Implementation of the PEP 509. @@ -100,10 +105,6 @@ In a brand new thread, raise a RuntimeError since there is no active exception to reraise. Patch written by Xiang Zhang. -- Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. - Patch by Ivan Levkivskyi. - -- Issue #28003: Implement PEP 525 -- Asynchronous Generators. Library -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 12:53:18 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 09 Sep 2016 16:53:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327999=3A_Make_=22?= =?utf-8?q?global_after_use=22_a_SyntaxError=2C_and_ditto_for_nonlocal=2E?= Message-ID: <20160909165315.21136.26750.46DADC30@psf.io> https://hg.python.org/cpython/rev/804b71d43c85 changeset: 103415:804b71d43c85 user: Guido van Rossum date: Fri Sep 09 09:36:26 2016 -0700 summary: Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi. files: Doc/reference/simple_stmts.rst | 5 +- Lib/test/test_syntax.py | 18 +++- Misc/NEWS | 3 + Python/symtable.c | 104 +++++++------------- 4 files changed, 59 insertions(+), 71 deletions(-) diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -903,11 +903,12 @@ Names listed in a :keyword:`global` statement must not be defined as formal parameters or in a :keyword:`for` loop control target, :keyword:`class` -definition, function definition, or :keyword:`import` statement. +definition, function definition, :keyword:`import` statement, or variable +annotation. .. impl-detail:: - The current implementation does not enforce the two restrictions, but + The current implementation does not enforce some of these restriction, but programs should not abuse this freedom, as future implementations may enforce them or silently change the meaning of the program. diff --git a/Lib/test/test_syntax.py b/Lib/test/test_syntax.py --- a/Lib/test/test_syntax.py +++ b/Lib/test/test_syntax.py @@ -366,7 +366,23 @@ ... SyntaxError: too many statically nested blocks -Misuse of the nonlocal statement can lead to a few unique syntax errors. +Misuse of the nonlocal and global statement can lead to a few unique syntax errors. + + >>> def f(): + ... x = 1 + ... global x + Traceback (most recent call last): + ... + SyntaxError: name 'x' is assigned to before global declaration + + >>> def f(): + ... x = 1 + ... def g(): + ... print(x) + ... nonlocal x + Traceback (most recent call last): + ... + SyntaxError: name 'x' is used prior to nonlocal declaration >>> def f(x): ... nonlocal x diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. + Patch by Ivan Levkivskyi. + - Issue #28003: Implement PEP 525 -- Asynchronous Generators. - Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. diff --git a/Python/symtable.c b/Python/symtable.c --- a/Python/symtable.c +++ b/Python/symtable.c @@ -6,16 +6,22 @@ /* error strings used for warnings */ #define GLOBAL_AFTER_ASSIGN \ -"name '%.400s' is assigned to before global declaration" +"name '%U' is assigned to before global declaration" #define NONLOCAL_AFTER_ASSIGN \ -"name '%.400s' is assigned to before nonlocal declaration" +"name '%U' is assigned to before nonlocal declaration" #define GLOBAL_AFTER_USE \ -"name '%.400s' is used prior to global declaration" +"name '%U' is used prior to global declaration" #define NONLOCAL_AFTER_USE \ -"name '%.400s' is used prior to nonlocal declaration" +"name '%U' is used prior to nonlocal declaration" + +#define GLOBAL_ANNOT \ +"annotated name '%U' can't be global" + +#define NONLOCAL_ANNOT \ +"annotated name '%U' can't be nonlocal" #define IMPORT_STAR_WARNING "import * only allowed at module level" @@ -161,7 +167,6 @@ }; static int symtable_analyze(struct symtable *st); -static int symtable_warn(struct symtable *st, const char *msg, int lineno); static int symtable_enter_block(struct symtable *st, identifier name, _Py_block_ty block, void *ast, int lineno, int col_offset); @@ -907,27 +912,6 @@ return r; } - -static int -symtable_warn(struct symtable *st, const char *msg, int lineno) -{ - PyObject *message = PyUnicode_FromString(msg); - if (message == NULL) - return 0; - if (PyErr_WarnExplicitObject(PyExc_SyntaxWarning, message, st->st_filename, - lineno, NULL, NULL) < 0) { - Py_DECREF(message); - if (PyErr_ExceptionMatches(PyExc_SyntaxWarning)) { - PyErr_SetString(PyExc_SyntaxError, msg); - PyErr_SyntaxLocationObject(st->st_filename, st->st_cur->ste_lineno, - st->st_cur->ste_col_offset); - } - return 0; - } - Py_DECREF(message); - return 1; -} - /* symtable_enter_block() gets a reference via ste_new. This reference is released when the block is exited, via the DECREF in symtable_exit_block(). @@ -1212,9 +1196,8 @@ if ((cur & (DEF_GLOBAL | DEF_NONLOCAL)) && s->v.AnnAssign.simple) { PyErr_Format(PyExc_SyntaxError, - "annotated name '%U' can't be %s", - e_name->v.Name.id, - cur & DEF_GLOBAL ? "global" : "nonlocal"); + cur & DEF_GLOBAL ? GLOBAL_ANNOT : NONLOCAL_ANNOT, + e_name->v.Name.id); PyErr_SyntaxLocationObject(st->st_filename, s->lineno, s->col_offset); @@ -1297,31 +1280,24 @@ long cur = symtable_lookup(st, name); if (cur < 0) VISIT_QUIT(st, 0); - if (cur & DEF_ANNOT) { + if (cur & (DEF_LOCAL | USE | DEF_ANNOT)) { + char* msg; + if (cur & DEF_ANNOT) { + msg = GLOBAL_ANNOT; + } + if (cur & DEF_LOCAL) { + msg = GLOBAL_AFTER_ASSIGN; + } + else { + msg = GLOBAL_AFTER_USE; + } PyErr_Format(PyExc_SyntaxError, - "annotated name '%U' can't be global", - name); + msg, name); PyErr_SyntaxLocationObject(st->st_filename, s->lineno, s->col_offset); VISIT_QUIT(st, 0); } - if (cur & (DEF_LOCAL | USE)) { - char buf[256]; - char *c_name = _PyUnicode_AsString(name); - if (!c_name) - return 0; - if (cur & DEF_LOCAL) - PyOS_snprintf(buf, sizeof(buf), - GLOBAL_AFTER_ASSIGN, - c_name); - else - PyOS_snprintf(buf, sizeof(buf), - GLOBAL_AFTER_USE, - c_name); - if (!symtable_warn(st, buf, s->lineno)) - VISIT_QUIT(st, 0); - } if (!symtable_add_def(st, name, DEF_GLOBAL)) VISIT_QUIT(st, 0); if (!symtable_record_directive(st, name, s)) @@ -1337,31 +1313,23 @@ long cur = symtable_lookup(st, name); if (cur < 0) VISIT_QUIT(st, 0); - if (cur & DEF_ANNOT) { - PyErr_Format(PyExc_SyntaxError, - "annotated name '%U' can't be nonlocal", - name); + if (cur & (DEF_LOCAL | USE | DEF_ANNOT)) { + char* msg; + if (cur & DEF_ANNOT) { + msg = NONLOCAL_ANNOT; + } + if (cur & DEF_LOCAL) { + msg = NONLOCAL_AFTER_ASSIGN; + } + else { + msg = NONLOCAL_AFTER_USE; + } + PyErr_Format(PyExc_SyntaxError, msg, name); PyErr_SyntaxLocationObject(st->st_filename, s->lineno, s->col_offset); VISIT_QUIT(st, 0); } - if (cur & (DEF_LOCAL | USE)) { - char buf[256]; - char *c_name = _PyUnicode_AsString(name); - if (!c_name) - return 0; - if (cur & DEF_LOCAL) - PyOS_snprintf(buf, sizeof(buf), - NONLOCAL_AFTER_ASSIGN, - c_name); - else - PyOS_snprintf(buf, sizeof(buf), - NONLOCAL_AFTER_USE, - c_name); - if (!symtable_warn(st, buf, s->lineno)) - VISIT_QUIT(st, 0); - } if (!symtable_add_def(st, name, DEF_NONLOCAL)) VISIT_QUIT(st, 0); if (!symtable_record_directive(st, name, s)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 13:00:03 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 09 Sep 2016 17:00:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_a_few_big-ticket_items?= =?utf-8?q?_to_What=27s_new_in_3=2E6=2E?= Message-ID: <20160909170001.13175.58639.3AD8DCB5@psf.io> https://hg.python.org/cpython/rev/06d2274689c8 changeset: 103416:06d2274689c8 user: Guido van Rossum date: Fri Sep 09 09:59:34 2016 -0700 summary: Add a few big-ticket items to What's new in 3.6. files: Doc/whatsnew/3.6.rst | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -64,8 +64,20 @@ New syntax features: +* A ``global`` or ``nonlocal`` statement must now textually appear + before the first use of the affected name in the same scope. + Previously this was a SyntaxWarning. + * PEP 498: :ref:`Formatted string literals ` +* PEP 515: Underscores in Numeric Literals + +* PEP 526: Syntax for Variable Annotations + +* PEP 525: Asynchronous Generators + +* PEP 530: Asynchronous Comprehensions + Standard library improvements: Security improvements: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 13:23:29 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 17:23:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_compile_with_-std=3Dc99_in?= =?utf-8?q?stead_of_-std=3Dgnu99=3B_use_kiddie-gloves_with?= Message-ID: <20160909172324.19423.77577.AB35C656@psf.io> https://hg.python.org/cpython/rev/91017e2202ae changeset: 103417:91017e2202ae user: Benjamin Peterson date: Fri Sep 09 10:22:45 2016 -0700 summary: compile with -std=c99 instead of -std=gnu99; use kiddie-gloves with bluetooth/bluetooh.h (#28017) files: configure | 24 ++++++++++++++++++++---- configure.ac | 13 +++++++++---- 2 files changed, 29 insertions(+), 8 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -6855,9 +6855,7 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) - # GNU dialect of C99, enables GNU extensions like __attribute__. GNU99 - # is required by bluetooth.h on big endian machines. - CFLAGS_NODIST="$CFLAGS_NODIST -std=gnu99" + CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable @@ -7617,7 +7615,7 @@ sys/stat.h sys/syscall.h sys/sys_domain.h sys/termio.h sys/time.h \ sys/times.h sys/types.h sys/uio.h sys/un.h sys/utsname.h sys/wait.h pty.h \ libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \ -bluetooth/bluetooth.h linux/tipc.h linux/random.h spawn.h util.h alloca.h endian.h \ +linux/tipc.h linux/random.h spawn.h util.h alloca.h endian.h \ sys/endian.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` @@ -7840,6 +7838,24 @@ fi +# bluetooth/bluetooth.h has been known to not compile with -std=c99. +# http://permalink.gmane.org/gmane.linux.bluez.kernel/22294 +SAVE_CFLAGS=$CFLAGS +CFLAGS="-std=c99 $CFLAGS" +for ac_header in bluetooth/bluetooth.h +do : + ac_fn_c_check_header_mongrel "$LINENO" "bluetooth/bluetooth.h" "ac_cv_header_bluetooth_bluetooth_h" "$ac_includes_default" +if test "x$ac_cv_header_bluetooth_bluetooth_h" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_BLUETOOTH_BLUETOOTH_H 1 +_ACEOF + +fi + +done + +CFLAGS=$SAVE_CFLAGS + # On Darwin (OS X) net/if.h requires sys/socket.h to be imported first. for ac_header in net/if.h do : diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1520,9 +1520,7 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) - # GNU dialect of C99, enables GNU extensions like __attribute__. GNU99 - # is required by bluetooth.h on big endian machines. - CFLAGS_NODIST="$CFLAGS_NODIST -std=gnu99" + CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable @@ -1991,11 +1989,18 @@ sys/stat.h sys/syscall.h sys/sys_domain.h sys/termio.h sys/time.h \ sys/times.h sys/types.h sys/uio.h sys/un.h sys/utsname.h sys/wait.h pty.h \ libutil.h sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \ -bluetooth/bluetooth.h linux/tipc.h linux/random.h spawn.h util.h alloca.h endian.h \ +linux/tipc.h linux/random.h spawn.h util.h alloca.h endian.h \ sys/endian.h) AC_HEADER_DIRENT AC_HEADER_MAJOR +# bluetooth/bluetooth.h has been known to not compile with -std=c99. +# http://permalink.gmane.org/gmane.linux.bluez.kernel/22294 +SAVE_CFLAGS=$CFLAGS +CFLAGS="-std=c99 $CFLAGS" +AC_CHECK_HEADERS(bluetooth/bluetooth.h) +CFLAGS=$SAVE_CFLAGS + # On Darwin (OS X) net/if.h requires sys/socket.h to be imported first. AC_CHECK_HEADERS([net/if.h], [], [], [#include -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 13:36:12 2016 From: python-checkins at python.org (yury.selivanov) Date: Fri, 09 Sep 2016 17:36:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328008=3A_Implemen?= =?utf-8?q?t_PEP_530_--_asynchronous_comprehensions=2E?= Message-ID: <20160909173609.3168.29295.0A44425B@psf.io> https://hg.python.org/cpython/rev/cf91a929b81a changeset: 103418:cf91a929b81a user: Yury Selivanov date: Fri Sep 09 10:36:01 2016 -0700 summary: Issue #28008: Implement PEP 530 -- asynchronous comprehensions. files: Grammar/Grammar | 2 +- Include/Python-ast.h | 5 +- Lib/lib2to3/Grammar.txt | 2 +- Lib/lib2to3/tests/test_parser.py | 18 + Lib/test/badsyntax_async1.py | 2 - Lib/test/badsyntax_async2.py | 2 - Lib/test/badsyntax_async3.py | 2 - Lib/test/badsyntax_async4.py | 2 - Lib/test/badsyntax_async5.py | 2 - Lib/test/badsyntax_async7.py | 2 - Lib/test/badsyntax_async8.py | 2 - Lib/test/test_ast.py | 35 +- Lib/test/test_coroutines.py | 324 +++++++++++++++++- Misc/NEWS | 2 +- Parser/Python.asdl | 2 +- Python/Python-ast.c | 27 +- Python/ast.c | 35 +- Python/compile.c | 220 +++++++++++- Python/graminit.c | 41 +- Python/symtable.c | 6 + 20 files changed, 616 insertions(+), 117 deletions(-) diff --git a/Grammar/Grammar b/Grammar/Grammar --- a/Grammar/Grammar +++ b/Grammar/Grammar @@ -146,7 +146,7 @@ '*' test ) comp_iter: comp_for | comp_if -comp_for: 'for' exprlist 'in' or_test [comp_iter] +comp_for: [ASYNC] 'for' exprlist 'in' or_test [comp_iter] comp_if: 'if' test_nocond [comp_iter] # not used in grammar, but may appear in "node" passed from Parser to Compiler diff --git a/Include/Python-ast.h b/Include/Python-ast.h --- a/Include/Python-ast.h +++ b/Include/Python-ast.h @@ -389,6 +389,7 @@ expr_ty target; expr_ty iter; asdl_seq *ifs; + int is_async; }; enum _excepthandler_kind {ExceptHandler_kind=1}; @@ -609,9 +610,9 @@ slice_ty _Py_ExtSlice(asdl_seq * dims, PyArena *arena); #define Index(a0, a1) _Py_Index(a0, a1) slice_ty _Py_Index(expr_ty value, PyArena *arena); -#define comprehension(a0, a1, a2, a3) _Py_comprehension(a0, a1, a2, a3) +#define comprehension(a0, a1, a2, a3, a4) _Py_comprehension(a0, a1, a2, a3, a4) comprehension_ty _Py_comprehension(expr_ty target, expr_ty iter, asdl_seq * - ifs, PyArena *arena); + ifs, int is_async, PyArena *arena); #define ExceptHandler(a0, a1, a2, a3, a4, a5) _Py_ExceptHandler(a0, a1, a2, a3, a4, a5) excepthandler_ty _Py_ExceptHandler(expr_ty type, identifier name, asdl_seq * body, int lineno, int col_offset, PyArena diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -150,7 +150,7 @@ argument: test [comp_for] | test '=' test # Really [keyword '='] test comp_iter: comp_for | comp_if -comp_for: 'for' exprlist 'in' testlist_safe [comp_iter] +comp_for: [ASYNC] 'for' exprlist 'in' testlist_safe [comp_iter] comp_if: 'if' old_test [comp_iter] testlist1: test (',' test)* diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -134,6 +134,24 @@ """) self.validate("""async def foo(): + [i async for i in b] + """) + + self.validate("""async def foo(): + {i for i in b + async for i in a if await i + for b in i} + """) + + self.validate("""async def foo(): + [await i for i in b if await c] + """) + + self.validate("""async def foo(): + [ i for i in b if c] + """) + + self.validate("""async def foo(): def foo(): pass diff --git a/Lib/test/badsyntax_async1.py b/Lib/test/badsyntax_async1.py deleted file mode 100644 --- a/Lib/test/badsyntax_async1.py +++ /dev/null @@ -1,2 +0,0 @@ -async def foo(a=await something()): - pass diff --git a/Lib/test/badsyntax_async2.py b/Lib/test/badsyntax_async2.py deleted file mode 100644 --- a/Lib/test/badsyntax_async2.py +++ /dev/null @@ -1,2 +0,0 @@ -async def foo(a=await something()): - pass diff --git a/Lib/test/badsyntax_async3.py b/Lib/test/badsyntax_async3.py deleted file mode 100644 --- a/Lib/test/badsyntax_async3.py +++ /dev/null @@ -1,2 +0,0 @@ -async def foo(): - [i async for i in els] diff --git a/Lib/test/badsyntax_async4.py b/Lib/test/badsyntax_async4.py deleted file mode 100644 --- a/Lib/test/badsyntax_async4.py +++ /dev/null @@ -1,2 +0,0 @@ -async def foo(): - await diff --git a/Lib/test/badsyntax_async5.py b/Lib/test/badsyntax_async5.py deleted file mode 100644 --- a/Lib/test/badsyntax_async5.py +++ /dev/null @@ -1,2 +0,0 @@ -def foo(): - await something() diff --git a/Lib/test/badsyntax_async7.py b/Lib/test/badsyntax_async7.py deleted file mode 100644 --- a/Lib/test/badsyntax_async7.py +++ /dev/null @@ -1,2 +0,0 @@ -async def foo(): - yield from [] diff --git a/Lib/test/badsyntax_async8.py b/Lib/test/badsyntax_async8.py deleted file mode 100644 --- a/Lib/test/badsyntax_async8.py +++ /dev/null @@ -1,2 +0,0 @@ -async def foo(): - await await fut diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -116,6 +116,8 @@ # PEP 448: Additional Unpacking Generalizations "{**{1:2}, 2:3}", "{*{1, 2}, 3}", + # Asynchronous comprehensions + "async def f():\n [i async for b in c]", ] # These are compiled through "single" @@ -809,21 +811,21 @@ def _check_comprehension(self, fac): self.expr(fac([]), "comprehension with no generators") g = ast.comprehension(ast.Name("x", ast.Load()), - ast.Name("x", ast.Load()), []) + ast.Name("x", ast.Load()), [], 0) self.expr(fac([g]), "must have Store context") g = ast.comprehension(ast.Name("x", ast.Store()), - ast.Name("x", ast.Store()), []) + ast.Name("x", ast.Store()), [], 0) self.expr(fac([g]), "must have Load context") x = ast.Name("x", ast.Store()) y = ast.Name("y", ast.Load()) - g = ast.comprehension(x, y, [None]) + g = ast.comprehension(x, y, [None], 0) self.expr(fac([g]), "None disallowed") - g = ast.comprehension(x, y, [ast.Name("x", ast.Store())]) + g = ast.comprehension(x, y, [ast.Name("x", ast.Store())], 0) self.expr(fac([g]), "must have Load context") def _simple_comp(self, fac): g = ast.comprehension(ast.Name("x", ast.Store()), - ast.Name("x", ast.Load()), []) + ast.Name("x", ast.Load()), [], 0) self.expr(fac(ast.Name("x", ast.Store()), [g]), "must have Load context") def wrap(gens): @@ -841,7 +843,7 @@ def test_dictcomp(self): g = ast.comprehension(ast.Name("y", ast.Store()), - ast.Name("p", ast.Load()), []) + ast.Name("p", ast.Load()), [], 0) c = ast.DictComp(ast.Name("x", ast.Store()), ast.Name("y", ast.Load()), [g]) self.expr(c, "must have Load context") @@ -1111,19 +1113,20 @@ ('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Break', (1, 11))], [])]), ('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Continue', (1, 11))], [])]), ('Module', [('For', (1, 0), ('Tuple', (1, 4), [('Name', (1, 4), 'a', ('Store',)), ('Name', (1, 6), 'b', ('Store',))], ('Store',)), ('Name', (1, 11), 'c', ('Load',)), [('Pass', (1, 14))], [])]), -('Module', [('Expr', (1, 0), ('ListComp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), -('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [])]))]), -('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 12), [('Name', (1, 12), 'a', ('Store',)), ('Name', (1, 14), 'b', ('Store',))], ('Store',)), ('Name', (1, 20), 'c', ('Load',)), [])]))]), -('Module', [('Expr', (1, 0), ('GeneratorExp', (2, 4), ('Tuple', (3, 4), [('Name', (3, 4), 'Aa', ('Load',)), ('Name', (5, 7), 'Bb', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (8, 4), [('Name', (8, 4), 'Aa', ('Store',)), ('Name', (10, 4), 'Bb', ('Store',))], ('Store',)), ('Name', (10, 10), 'Cc', ('Load',)), [])]))]), -('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Name', (1, 11), 'w', ('Store',)), ('Name', (1, 16), 'x', ('Load',)), []), ('comprehension', ('Name', (1, 22), 'm', ('Store',)), ('Name', (1, 27), 'p', ('Load',)), [('Name', (1, 32), 'g', ('Load',))])]))]), -('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [])]))]), -('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))])]))]), -('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [])]))]), +('Module', [('Expr', (1, 0), ('ListComp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [], 0)]))]), +('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'a', ('Store',)), ('Name', (1, 13), 'b', ('Store',))], ('Store',)), ('Name', (1, 18), 'c', ('Load',)), [], 0)]))]), +('Module', [('Expr', (1, 0), ('GeneratorExp', (1, 1), ('Tuple', (1, 2), [('Name', (1, 2), 'a', ('Load',)), ('Name', (1, 4), 'b', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (1, 12), [('Name', (1, 12), 'a', ('Store',)), ('Name', (1, 14), 'b', ('Store',))], ('Store',)), ('Name', (1, 20), 'c', ('Load',)), [], 0)]))]), +('Module', [('Expr', (1, 0), ('GeneratorExp', (2, 4), ('Tuple', (3, 4), [('Name', (3, 4), 'Aa', ('Load',)), ('Name', (5, 7), 'Bb', ('Load',))], ('Load',)), [('comprehension', ('Tuple', (8, 4), [('Name', (8, 4), 'Aa', ('Store',)), ('Name', (10, 4), 'Bb', ('Store',))], ('Store',)), ('Name', (10, 10), 'Cc', ('Load',)), [], 0)]))]), +('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Name', (1, 11), 'w', ('Store',)), ('Name', (1, 16), 'x', ('Load',)), [], 0), ('comprehension', ('Name', (1, 22), 'm', ('Store',)), ('Name', (1, 27), 'p', ('Load',)), [('Name', (1, 32), 'g', ('Load',))], 0)]))]), +('Module', [('Expr', (1, 0), ('DictComp', (1, 0), ('Name', (1, 1), 'a', ('Load',)), ('Name', (1, 5), 'b', ('Load',)), [('comprehension', ('Tuple', (1, 11), [('Name', (1, 11), 'v', ('Store',)), ('Name', (1, 13), 'w', ('Store',))], ('Store',)), ('Name', (1, 18), 'x', ('Load',)), [], 0)]))]), +('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 12), 'x', ('Load',)), [('Name', (1, 17), 'g', ('Load',))], 0)]))]), +('Module', [('Expr', (1, 0), ('SetComp', (1, 0), ('Name', (1, 1), 'r', ('Load',)), [('comprehension', ('Tuple', (1, 7), [('Name', (1, 7), 'l', ('Store',)), ('Name', (1, 9), 'm', ('Store',))], ('Store',)), ('Name', (1, 14), 'x', ('Load',)), [], 0)]))]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('Await', (2, 1), ('Call', (2, 7), ('Name', (2, 7), 'something', ('Load',)), [], [])))], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncFor', (2, 7), ('Name', (2, 11), 'e', ('Store',)), ('Name', (2, 16), 'i', ('Load',)), [('Expr', (2, 19), ('Num', (2, 19), 1))], [('Expr', (3, 7), ('Num', (3, 7), 2))])], [], None)]), ('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('AsyncWith', (2, 7), [('withitem', ('Name', (2, 12), 'a', ('Load',)), ('Name', (2, 17), 'b', ('Store',)))], [('Expr', (2, 20), ('Num', (2, 20), 1))])], [], None)]), ('Module', [('Expr', (1, 0), ('Dict', (1, 0), [None, ('Num', (1, 10), 2)], [('Dict', (1, 3), [('Num', (1, 4), 1)], [('Num', (1, 6), 2)]), ('Num', (1, 12), 3)]))]), ('Module', [('Expr', (1, 0), ('Set', (1, 0), [('Starred', (1, 1), ('Set', (1, 2), [('Num', (1, 3), 1), ('Num', (1, 6), 2)]), ('Load',)), ('Num', (1, 10), 3)]))]), +('Module', [('AsyncFunctionDef', (1, 6), 'f', ('arguments', [], None, [], [], None, []), [('Expr', (2, 1), ('ListComp', (2, 2), ('Name', (2, 2), 'i', ('Load',)), [('comprehension', ('Name', (2, 14), 'b', ('Store',)), ('Name', (2, 19), 'c', ('Load',)), [], 1)]))], [], None)]), ] single_results = [ ('Interactive', [('Expr', (1, 0), ('BinOp', (1, 0), ('Num', (1, 0), 1), ('Add',), ('Num', (1, 2), 2)))]), @@ -1138,8 +1141,8 @@ ('Expression', ('Dict', (1, 0), [], [])), ('Expression', ('Set', (1, 0), [('NameConstant', (1, 1), None)])), ('Expression', ('Dict', (1, 0), [('Num', (2, 6), 1)], [('Num', (4, 10), 2)])), -('Expression', ('ListComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])), -('Expression', ('GeneratorExp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))])])), +('Expression', ('ListComp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))], 0)])), +('Expression', ('GeneratorExp', (1, 1), ('Name', (1, 1), 'a', ('Load',)), [('comprehension', ('Name', (1, 7), 'b', ('Store',)), ('Name', (1, 12), 'c', ('Load',)), [('Name', (1, 17), 'd', ('Load',))], 0)])), ('Expression', ('Compare', (1, 0), ('Num', (1, 0), 1), [('Lt',), ('Lt',)], [('Num', (1, 4), 2), ('Num', (1, 8), 3)])), ('Expression', ('Call', (1, 0), ('Name', (1, 0), 'f', ('Load',)), [('Num', (1, 2), 1), ('Num', (1, 4), 2), ('Starred', (1, 10), ('Name', (1, 11), 'd', ('Load',)), ('Load',))], [('keyword', 'c', ('Num', (1, 8), 3)), ('keyword', None, ('Name', (1, 15), 'e', ('Load',)))])), ('Expression', ('Num', (1, 0), 10)), diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py --- a/Lib/test/test_coroutines.py +++ b/Lib/test/test_coroutines.py @@ -69,49 +69,130 @@ class AsyncBadSyntaxTest(unittest.TestCase): def test_badsyntax_1(self): - with self.assertRaisesRegex(SyntaxError, "'await' outside"): - import test.badsyntax_async1 + samples = [ + """def foo(): + await something() + """, - def test_badsyntax_2(self): - with self.assertRaisesRegex(SyntaxError, "'await' outside"): - import test.badsyntax_async2 + """await something()""", - def test_badsyntax_3(self): - with self.assertRaisesRegex(SyntaxError, 'invalid syntax'): - import test.badsyntax_async3 + """async def foo(): + yield from [] + """, - def test_badsyntax_4(self): - with self.assertRaisesRegex(SyntaxError, 'invalid syntax'): - import test.badsyntax_async4 + """async def foo(): + await await fut + """, - def test_badsyntax_5(self): - with self.assertRaisesRegex(SyntaxError, 'invalid syntax'): - import test.badsyntax_async5 + """async def foo(a=await something()): + pass + """, - def test_badsyntax_7(self): - with self.assertRaisesRegex( - SyntaxError, "'yield from' inside async function"): + """async def foo(a:await something()): + pass + """, - import test.badsyntax_async7 + """async def foo(): + def bar(): + [i async for i in els] + """, - def test_badsyntax_8(self): - with self.assertRaisesRegex(SyntaxError, 'invalid syntax'): - import test.badsyntax_async8 + """async def foo(): + def bar(): + [await i for i in els] + """, - def test_badsyntax_9(self): - ns = {} - for comp in {'(await a for a in b)', - '[await a for a in b]', - '{await a for a in b}', - '{await a: c for a in b}'}: + """async def foo(): + def bar(): + [i for i in els + async for b in els] + """, - with self.assertRaisesRegex(SyntaxError, 'await.*in comprehen'): - exec('async def f():\n\t{}'.format(comp), ns, ns) + """async def foo(): + def bar(): + [i for i in els + for c in b + async for b in els] + """, - def test_badsyntax_10(self): - # Tests for issue 24619 + """async def foo(): + def bar(): + [i for i in els + async for b in els + for c in b] + """, - samples = [ + """async def foo(): + def bar(): + [i for i in els + for b in await els] + """, + + """async def foo(): + def bar(): + [i for i in els + for b in els + if await b] + """, + + """async def foo(): + def bar(): + [i for i in await els] + """, + + """async def foo(): + def bar(): + [i for i in els if await i] + """, + + """def bar(): + [i async for i in els] + """, + + """def bar(): + [await i for i in els] + """, + + """def bar(): + [i for i in els + async for b in els] + """, + + """def bar(): + [i for i in els + for c in b + async for b in els] + """, + + """def bar(): + [i for i in els + async for b in els + for c in b] + """, + + """def bar(): + [i for i in els + for b in await els] + """, + + """def bar(): + [i for i in els + for b in els + if await b] + """, + + """def bar(): + [i for i in await els] + """, + + """def bar(): + [i for i in els if await i] + """, + + """async def foo(): + await + """, + """async def foo(): def bar(): pass await = 1 @@ -1531,6 +1612,185 @@ warnings.simplefilter("error") run_async(foo()) + def test_comp_1(self): + async def f(i): + return i + + async def run_list(): + return [await c for c in [f(1), f(41)]] + + async def run_set(): + return {await c for c in [f(1), f(41)]} + + async def run_dict1(): + return {await c: 'a' for c in [f(1), f(41)]} + + async def run_dict2(): + return {i: await c for i, c in enumerate([f(1), f(41)])} + + self.assertEqual(run_async(run_list()), ([], [1, 41])) + self.assertEqual(run_async(run_set()), ([], {1, 41})) + self.assertEqual(run_async(run_dict1()), ([], {1: 'a', 41: 'a'})) + self.assertEqual(run_async(run_dict2()), ([], {0: 1, 1: 41})) + + def test_comp_2(self): + async def f(i): + return i + + async def run_list(): + return [s for c in [f(''), f('abc'), f(''), f(['de', 'fg'])] + for s in await c] + + self.assertEqual( + run_async(run_list()), + ([], ['a', 'b', 'c', 'de', 'fg'])) + + async def run_set(): + return {d + for c in [f([f([10, 30]), + f([20])])] + for s in await c + for d in await s} + + self.assertEqual( + run_async(run_set()), + ([], {10, 20, 30})) + + async def run_set2(): + return {await s + for c in [f([f(10), f(20)])] + for s in await c} + + self.assertEqual( + run_async(run_set2()), + ([], {10, 20})) + + def test_comp_3(self): + async def f(it): + for i in it: + yield i + + async def run_list(): + return [i + 1 async for i in f([10, 20])] + self.assertEqual( + run_async(run_list()), + ([], [11, 21])) + + async def run_set(): + return {i + 1 async for i in f([10, 20])} + self.assertEqual( + run_async(run_set()), + ([], {11, 21})) + + async def run_dict(): + return {i + 1: i + 2 async for i in f([10, 20])} + self.assertEqual( + run_async(run_dict()), + ([], {11: 12, 21: 22})) + + async def run_gen(): + gen = (i + 1 async for i in f([10, 20])) + return [g + 100 async for g in gen] + self.assertEqual( + run_async(run_gen()), + ([], [111, 121])) + + def test_comp_4(self): + async def f(it): + for i in it: + yield i + + async def run_list(): + return [i + 1 async for i in f([10, 20]) if i > 10] + self.assertEqual( + run_async(run_list()), + ([], [21])) + + async def run_set(): + return {i + 1 async for i in f([10, 20]) if i > 10} + self.assertEqual( + run_async(run_set()), + ([], {21})) + + async def run_dict(): + return {i + 1: i + 2 async for i in f([10, 20]) if i > 10} + self.assertEqual( + run_async(run_dict()), + ([], {21: 22})) + + async def run_gen(): + gen = (i + 1 async for i in f([10, 20]) if i > 10) + return [g + 100 async for g in gen] + self.assertEqual( + run_async(run_gen()), + ([], [121])) + + def test_comp_5(self): + async def f(it): + for i in it: + yield i + + async def run_list(): + return [i + 1 for pair in ([10, 20], [30, 40]) if pair[0] > 10 + async for i in f(pair) if i > 30] + self.assertEqual( + run_async(run_list()), + ([], [41])) + + def test_comp_6(self): + async def f(it): + for i in it: + yield i + + async def run_list(): + return [i + 1 async for seq in f([(10, 20), (30,)]) + for i in seq] + + self.assertEqual( + run_async(run_list()), + ([], [11, 21, 31])) + + def test_comp_7(self): + async def f(): + yield 1 + yield 2 + raise Exception('aaa') + + async def run_list(): + return [i async for i in f()] + + with self.assertRaisesRegex(Exception, 'aaa'): + run_async(run_list()) + + def test_comp_8(self): + async def f(): + return [i for i in [1, 2, 3]] + + self.assertEqual( + run_async(f()), + ([], [1, 2, 3])) + + def test_comp_9(self): + async def gen(): + yield 1 + yield 2 + async def f(): + l = [i async for i in gen()] + return [i for i in l] + + self.assertEqual( + run_async(f()), + ([], [1, 2])) + + def test_comp_10(self): + async def f(): + xx = {i for i in [1, 2, 3]} + return {x: x for x in xx} + + self.assertEqual( + run_async(f()), + ([], {1: 1, 2: 2, 3: 3})) + def test_copy(self): async def func(): pass coro = func() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -108,7 +108,7 @@ In a brand new thread, raise a RuntimeError since there is no active exception to reraise. Patch written by Xiang Zhang. - +- Issue #28008: Implement PEP 530 -- asynchronous comprehensions. Library ------- diff --git a/Parser/Python.asdl b/Parser/Python.asdl --- a/Parser/Python.asdl +++ b/Parser/Python.asdl @@ -110,7 +110,7 @@ cmpop = Eq | NotEq | Lt | LtE | Gt | GtE | Is | IsNot | In | NotIn - comprehension = (expr target, expr iter, expr* ifs) + comprehension = (expr target, expr iter, expr* ifs, int is_async) excepthandler = ExceptHandler(expr? type, identifier? name, stmt* body) attributes (int lineno, int col_offset) diff --git a/Python/Python-ast.c b/Python/Python-ast.c --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -435,10 +435,12 @@ static PyTypeObject *comprehension_type; static PyObject* ast2obj_comprehension(void*); _Py_IDENTIFIER(ifs); +_Py_IDENTIFIER(is_async); static char *comprehension_fields[]={ "target", "iter", "ifs", + "is_async", }; static PyTypeObject *excepthandler_type; static char *excepthandler_attributes[] = { @@ -1148,7 +1150,7 @@ NotIn_singleton = PyType_GenericNew(NotIn_type, NULL, NULL); if (!NotIn_singleton) return 0; comprehension_type = make_type("comprehension", &AST_type, - comprehension_fields, 3); + comprehension_fields, 4); if (!comprehension_type) return 0; if (!add_attributes(comprehension_type, NULL, 0)) return 0; excepthandler_type = make_type("excepthandler", &AST_type, NULL, 0); @@ -2445,7 +2447,8 @@ } comprehension_ty -comprehension(expr_ty target, expr_ty iter, asdl_seq * ifs, PyArena *arena) +comprehension(expr_ty target, expr_ty iter, asdl_seq * ifs, int is_async, + PyArena *arena) { comprehension_ty p; if (!target) { @@ -2464,6 +2467,7 @@ p->target = target; p->iter = iter; p->ifs = ifs; + p->is_async = is_async; return p; } @@ -3722,6 +3726,11 @@ if (_PyObject_SetAttrId(result, &PyId_ifs, value) == -1) goto failed; Py_DECREF(value); + value = ast2obj_int(o->is_async); + if (!value) goto failed; + if (_PyObject_SetAttrId(result, &PyId_is_async, value) == -1) + goto failed; + Py_DECREF(value); return result; failed: Py_XDECREF(value); @@ -7146,6 +7155,7 @@ expr_ty target; expr_ty iter; asdl_seq* ifs; + int is_async; if (_PyObject_HasAttrId(obj, &PyId_target)) { int res; @@ -7193,7 +7203,18 @@ PyErr_SetString(PyExc_TypeError, "required field \"ifs\" missing from comprehension"); return 1; } - *out = comprehension(target, iter, ifs, arena); + if (_PyObject_HasAttrId(obj, &PyId_is_async)) { + int res; + tmp = _PyObject_GetAttrId(obj, &PyId_is_async); + if (tmp == NULL) goto failed; + res = obj2ast_int(tmp, &is_async, arena); + if (res != 0) goto failed; + Py_CLEAR(tmp); + } else { + PyErr_SetString(PyExc_TypeError, "required field \"is_async\" missing from comprehension"); + return 1; + } + *out = comprehension(target, iter, ifs, is_async, arena); return 0; failed: Py_XDECREF(tmp); diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -1747,14 +1747,21 @@ count_comp_fors(struct compiling *c, const node *n) { int n_fors = 0; + int is_async; count_comp_for: + is_async = 0; n_fors++; REQ(n, comp_for); - if (NCH(n) == 5) - n = CHILD(n, 4); - else + if (TYPE(CHILD(n, 0)) == ASYNC) { + is_async = 1; + } + if (NCH(n) == (5 + is_async)) { + n = CHILD(n, 4 + is_async); + } + else { return n_fors; + } count_comp_iter: REQ(n, comp_iter); n = CHILD(n, 0); @@ -1817,14 +1824,19 @@ asdl_seq *t; expr_ty expression, first; node *for_ch; + int is_async = 0; REQ(n, comp_for); - for_ch = CHILD(n, 1); + if (TYPE(CHILD(n, 0)) == ASYNC) { + is_async = 1; + } + + for_ch = CHILD(n, 1 + is_async); t = ast_for_exprlist(c, for_ch, Store); if (!t) return NULL; - expression = ast_for_expr(c, CHILD(n, 3)); + expression = ast_for_expr(c, CHILD(n, 3 + is_async)); if (!expression) return NULL; @@ -1832,19 +1844,20 @@ (x for x, in ...) has 1 element in t, but still requires a Tuple. */ first = (expr_ty)asdl_seq_GET(t, 0); if (NCH(for_ch) == 1) - comp = comprehension(first, expression, NULL, c->c_arena); + comp = comprehension(first, expression, NULL, + is_async, c->c_arena); else - comp = comprehension(Tuple(t, Store, first->lineno, first->col_offset, - c->c_arena), - expression, NULL, c->c_arena); + comp = comprehension(Tuple(t, Store, first->lineno, + first->col_offset, c->c_arena), + expression, NULL, is_async, c->c_arena); if (!comp) return NULL; - if (NCH(n) == 5) { + if (NCH(n) == (5 + is_async)) { int j, n_ifs; asdl_seq *ifs; - n = CHILD(n, 4); + n = CHILD(n, 4 + is_async); n_ifs = count_comp_ifs(c, n); if (n_ifs == -1) return NULL; diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -202,6 +202,16 @@ static int compiler_try_except(struct compiler *, stmt_ty); static int compiler_set_qualname(struct compiler *); +static int compiler_sync_comprehension_generator( + struct compiler *c, + asdl_seq *generators, int gen_index, + expr_ty elt, expr_ty val, int type); + +static int compiler_async_comprehension_generator( + struct compiler *c, + asdl_seq *generators, int gen_index, + expr_ty elt, expr_ty val, int type); + static PyCodeObject *assemble(struct compiler *, int addNone); static PyObject *__doc__; @@ -2165,14 +2175,14 @@ static int compiler_async_for(struct compiler *c, stmt_ty s) { - static PyObject *stopiter_error = NULL; + _Py_IDENTIFIER(StopAsyncIteration); + basicblock *try, *except, *end, *after_try, *try_cleanup, *after_loop, *after_loop_else; - if (stopiter_error == NULL) { - stopiter_error = PyUnicode_InternFromString("StopAsyncIteration"); - if (stopiter_error == NULL) - return 0; + PyObject *stop_aiter_error = _PyUnicode_FromId(&PyId_StopAsyncIteration); + if (stop_aiter_error == NULL) { + return 0; } try = compiler_new_block(c); @@ -2214,7 +2224,7 @@ compiler_use_next_block(c, except); ADDOP(c, DUP_TOP); - ADDOP_O(c, LOAD_GLOBAL, stopiter_error, names); + ADDOP_O(c, LOAD_GLOBAL, stop_aiter_error, names); ADDOP_I(c, COMPARE_OP, PyCmp_EXC_MATCH); ADDOP_JABS(c, POP_JUMP_IF_FALSE, try_cleanup); @@ -3627,11 +3637,28 @@ - iterate over the generator sequence instead of using recursion */ + static int compiler_comprehension_generator(struct compiler *c, asdl_seq *generators, int gen_index, expr_ty elt, expr_ty val, int type) { + comprehension_ty gen; + gen = (comprehension_ty)asdl_seq_GET(generators, gen_index); + if (gen->is_async) { + return compiler_async_comprehension_generator( + c, generators, gen_index, elt, val, type); + } else { + return compiler_sync_comprehension_generator( + c, generators, gen_index, elt, val, type); + } +} + +static int +compiler_sync_comprehension_generator(struct compiler *c, + asdl_seq *generators, int gen_index, + expr_ty elt, expr_ty val, int type) +{ /* generate code for the iterator, then each of the ifs, and then write to the element */ @@ -3718,20 +3745,167 @@ } static int +compiler_async_comprehension_generator(struct compiler *c, + asdl_seq *generators, int gen_index, + expr_ty elt, expr_ty val, int type) +{ + _Py_IDENTIFIER(StopAsyncIteration); + + comprehension_ty gen; + basicblock *anchor, *skip, *if_cleanup, *try, + *after_try, *except, *try_cleanup; + Py_ssize_t i, n; + + PyObject *stop_aiter_error = _PyUnicode_FromId(&PyId_StopAsyncIteration); + if (stop_aiter_error == NULL) { + return 0; + } + + try = compiler_new_block(c); + after_try = compiler_new_block(c); + try_cleanup = compiler_new_block(c); + except = compiler_new_block(c); + skip = compiler_new_block(c); + if_cleanup = compiler_new_block(c); + anchor = compiler_new_block(c); + + if (skip == NULL || if_cleanup == NULL || anchor == NULL || + try == NULL || after_try == NULL || + except == NULL || after_try == NULL) { + return 0; + } + + gen = (comprehension_ty)asdl_seq_GET(generators, gen_index); + + if (gen_index == 0) { + /* Receive outermost iter as an implicit argument */ + c->u->u_argcount = 1; + ADDOP_I(c, LOAD_FAST, 0); + } + else { + /* Sub-iter - calculate on the fly */ + VISIT(c, expr, gen->iter); + ADDOP(c, GET_AITER); + ADDOP_O(c, LOAD_CONST, Py_None, consts); + ADDOP(c, YIELD_FROM); + } + + compiler_use_next_block(c, try); + + + ADDOP_JREL(c, SETUP_EXCEPT, except); + if (!compiler_push_fblock(c, EXCEPT, try)) + return 0; + + ADDOP(c, GET_ANEXT); + ADDOP_O(c, LOAD_CONST, Py_None, consts); + ADDOP(c, YIELD_FROM); + VISIT(c, expr, gen->target); + ADDOP(c, POP_BLOCK); + compiler_pop_fblock(c, EXCEPT, try); + ADDOP_JREL(c, JUMP_FORWARD, after_try); + + + compiler_use_next_block(c, except); + ADDOP(c, DUP_TOP); + ADDOP_O(c, LOAD_GLOBAL, stop_aiter_error, names); + ADDOP_I(c, COMPARE_OP, PyCmp_EXC_MATCH); + ADDOP_JABS(c, POP_JUMP_IF_FALSE, try_cleanup); + + ADDOP(c, POP_TOP); + ADDOP(c, POP_TOP); + ADDOP(c, POP_TOP); + ADDOP(c, POP_EXCEPT); /* for SETUP_EXCEPT */ + ADDOP_JABS(c, JUMP_ABSOLUTE, anchor); + + + compiler_use_next_block(c, try_cleanup); + ADDOP(c, END_FINALLY); + + compiler_use_next_block(c, after_try); + + n = asdl_seq_LEN(gen->ifs); + for (i = 0; i < n; i++) { + expr_ty e = (expr_ty)asdl_seq_GET(gen->ifs, i); + VISIT(c, expr, e); + ADDOP_JABS(c, POP_JUMP_IF_FALSE, if_cleanup); + NEXT_BLOCK(c); + } + + if (++gen_index < asdl_seq_LEN(generators)) + if (!compiler_comprehension_generator(c, + generators, gen_index, + elt, val, type)) + return 0; + + /* only append after the last for generator */ + if (gen_index >= asdl_seq_LEN(generators)) { + /* comprehension specific code */ + switch (type) { + case COMP_GENEXP: + VISIT(c, expr, elt); + ADDOP(c, YIELD_VALUE); + ADDOP(c, POP_TOP); + break; + case COMP_LISTCOMP: + VISIT(c, expr, elt); + ADDOP_I(c, LIST_APPEND, gen_index + 1); + break; + case COMP_SETCOMP: + VISIT(c, expr, elt); + ADDOP_I(c, SET_ADD, gen_index + 1); + break; + case COMP_DICTCOMP: + /* With 'd[k] = v', v is evaluated before k, so we do + the same. */ + VISIT(c, expr, val); + VISIT(c, expr, elt); + ADDOP_I(c, MAP_ADD, gen_index + 1); + break; + default: + return 0; + } + + compiler_use_next_block(c, skip); + } + compiler_use_next_block(c, if_cleanup); + ADDOP_JABS(c, JUMP_ABSOLUTE, try); + compiler_use_next_block(c, anchor); + ADDOP(c, POP_TOP); + + return 1; +} + +static int compiler_comprehension(struct compiler *c, expr_ty e, int type, identifier name, asdl_seq *generators, expr_ty elt, expr_ty val) { PyCodeObject *co = NULL; - expr_ty outermost_iter; + comprehension_ty outermost; PyObject *qualname = NULL; - - outermost_iter = ((comprehension_ty) - asdl_seq_GET(generators, 0))->iter; + int is_async_function = c->u->u_ste->ste_coroutine; + int is_async_generator = 0; + + outermost = (comprehension_ty) asdl_seq_GET(generators, 0); if (!compiler_enter_scope(c, name, COMPILER_SCOPE_COMPREHENSION, (void *)e, e->lineno)) + { goto error; + } + + is_async_generator = c->u->u_ste->ste_coroutine; + + if (is_async_generator && !is_async_function) { + if (e->lineno > c->u->u_lineno) { + c->u->u_lineno = e->lineno; + c->u->u_lineno_set = 0; + } + compiler_error(c, "asynchronous comprehension outside of " + "an asynchronous function"); + goto error_in_scope; + } if (type != COMP_GENEXP) { int op; @@ -3774,9 +3948,24 @@ Py_DECREF(qualname); Py_DECREF(co); - VISIT(c, expr, outermost_iter); - ADDOP(c, GET_ITER); + VISIT(c, expr, outermost->iter); + + if (outermost->is_async) { + ADDOP(c, GET_AITER); + ADDOP_O(c, LOAD_CONST, Py_None, consts); + ADDOP(c, YIELD_FROM); + } else { + ADDOP(c, GET_ITER); + } + ADDOP_I(c, CALL_FUNCTION, 1); + + if (is_async_generator && type != COMP_GENEXP) { + ADDOP(c, GET_AWAITABLE); + ADDOP_O(c, LOAD_CONST, Py_None, consts); + ADDOP(c, YIELD_FROM); + } + return 1; error_in_scope: compiler_exit_scope(c); @@ -4140,11 +4329,8 @@ if (c->u->u_ste->ste_type != FunctionBlock) return compiler_error(c, "'await' outside function"); - if (c->u->u_scope_type == COMPILER_SCOPE_COMPREHENSION) - return compiler_error( - c, "'await' expressions in comprehensions are not supported"); - - if (c->u->u_scope_type != COMPILER_SCOPE_ASYNC_FUNCTION) + if (c->u->u_scope_type != COMPILER_SCOPE_ASYNC_FUNCTION && + c->u->u_scope_type != COMPILER_SCOPE_COMPREHENSION) return compiler_error(c, "'await' outside async function"); VISIT(c, expr, e->v.Await.value); diff --git a/Python/graminit.c b/Python/graminit.c --- a/Python/graminit.c +++ b/Python/graminit.c @@ -1812,32 +1812,37 @@ {2, arcs_80_0}, {1, arcs_80_1}, }; -static arc arcs_81_0[1] = { - {101, 1}, +static arc arcs_81_0[2] = { + {21, 1}, + {101, 2}, }; static arc arcs_81_1[1] = { - {66, 2}, + {101, 2}, }; static arc arcs_81_2[1] = { - {102, 3}, + {66, 3}, }; static arc arcs_81_3[1] = { - {112, 4}, -}; -static arc arcs_81_4[2] = { - {171, 5}, - {0, 4}, -}; -static arc arcs_81_5[1] = { + {102, 4}, +}; +static arc arcs_81_4[1] = { + {112, 5}, +}; +static arc arcs_81_5[2] = { + {171, 6}, {0, 5}, }; -static state states_81[6] = { - {1, arcs_81_0}, +static arc arcs_81_6[1] = { + {0, 6}, +}; +static state states_81[7] = { + {2, arcs_81_0}, {1, arcs_81_1}, {1, arcs_81_2}, {1, arcs_81_3}, - {2, arcs_81_4}, - {1, arcs_81_5}, + {1, arcs_81_4}, + {2, arcs_81_5}, + {1, arcs_81_6}, }; static arc arcs_82_0[1] = { {97, 1}, @@ -2060,9 +2065,9 @@ {335, "argument", 0, 4, states_79, "\000\040\200\000\006\000\000\000\000\000\010\000\000\000\020\002\000\300\220\050\037\000"}, {336, "comp_iter", 0, 2, states_80, - "\000\000\000\000\000\000\000\000\000\000\000\000\042\000\000\000\000\000\000\000\000\000"}, - {337, "comp_for", 0, 6, states_81, - "\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000"}, + "\000\000\040\000\000\000\000\000\000\000\000\000\042\000\000\000\000\000\000\000\000\000"}, + {337, "comp_for", 0, 7, states_81, + "\000\000\040\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000"}, {338, "comp_if", 0, 4, states_82, "\000\000\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000"}, {339, "encoding_decl", 0, 2, states_83, diff --git a/Python/symtable.c b/Python/symtable.c --- a/Python/symtable.c +++ b/Python/symtable.c @@ -1682,6 +1682,9 @@ VISIT(st, expr, lc->target); VISIT(st, expr, lc->iter); VISIT_SEQ(st, expr, lc->ifs); + if (lc->is_async) { + st->st_cur->ste_coroutine = 1; + } return 1; } @@ -1734,6 +1737,9 @@ return 0; } st->st_cur->ste_generator = is_generator; + if (outermost->is_async) { + st->st_cur->ste_coroutine = 1; + } /* Outermost iter is received as an argument */ if (!symtable_implicit_arg(st, 0)) { symtable_exit_block(st, (void *)e); -- Repository URL: https://hg.python.org/cpython From lp_benchmark_robot at intel.com Fri Sep 9 14:05:45 2016 From: lp_benchmark_robot at intel.com (lp_benchmark_robot at intel.com) Date: Fri, 9 Sep 2016 19:05:45 +0100 Subject: [Python-checkins] GOOD Benchmark Results for Python 2.7 2016-09-09 Message-ID: Results for project Python 2.7, build date 2016-09-09 02:47:00 +0000 commit: 7e89469e4342 previous commit: 2d6dd8402d77 revision date: 2016-09-08 22:44:44 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dc from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.14% 0.32% 4.77% 3.32% :-) pybench 0.19% -0.21% 6.10% 4.18% :-( regex_v8 0.69% -0.29% -2.34% 10.71% :-) nbody 0.11% -0.79% 6.08% 7.01% :-) json_dump_v2 0.27% 1.43% 2.11% 9.64% :-| normal_startup 0.42% -0.55% -1.06% 2.60% :-) ssbench 0.20% -0.43% 2.04% 1.99% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/good-benchmark-results-for-python-2-7-2016-09-09/ Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Sep 9 14:06:49 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 18:06:49 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2MTMy?= =?utf-8?q?=3A_Only_adds_manifest_to_executables_and_main_DLL=2E?= Message-ID: <20160909180648.48546.70670.3A404EE9@psf.io> https://hg.python.org/cpython/rev/313ae9926889 changeset: 103419:313ae9926889 branch: 2.7 parent: 103407:47c1ace69d02 user: Steve Dower date: Fri Sep 09 11:05:46 2016 -0700 summary: Issue #26132: Only adds manifest to executables and main DLL. files: PCbuild/pyproject.props | 7 ++----- PCbuild/python.vcxproj | 3 +++ PCbuild/pythoncore.vcxproj | 3 +++ PCbuild/pythonw.vcxproj | 6 ++++++ 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/PCbuild/pyproject.props b/PCbuild/pyproject.props --- a/PCbuild/pyproject.props +++ b/PCbuild/pyproject.props @@ -8,12 +8,9 @@ $(SolutionDir)obj\$(ArchName)\$(ProjectName)\ $(ProjectName) $(TargetName)$(PyDebugExt) - false - false - + false + false - true - true true true false diff --git a/PCbuild/python.vcxproj b/PCbuild/python.vcxproj --- a/PCbuild/python.vcxproj +++ b/PCbuild/python.vcxproj @@ -55,6 +55,9 @@ <_ProjectFileVersion>10.0.30319.1 + + true + true diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -59,6 +59,9 @@ <_ProjectFileVersion>10.0.30319.1 $(PyDllName) + + true + true Link diff --git a/PCbuild/pythonw.vcxproj b/PCbuild/pythonw.vcxproj --- a/PCbuild/pythonw.vcxproj +++ b/PCbuild/pythonw.vcxproj @@ -44,6 +44,9 @@ Application false + + true + true @@ -55,6 +58,9 @@ <_ProjectFileVersion>10.0.30319.1 + + true + true -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:06:49 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 18:06:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Removes_file_f?= =?utf-8?q?rom_installer=2E?= Message-ID: <20160909180648.2439.24735.E6DBFF10@psf.io> https://hg.python.org/cpython/rev/dbaf2cbf8f6f changeset: 103420:dbaf2cbf8f6f branch: 2.7 user: Steve Dower date: Fri Sep 09 11:05:58 2016 -0700 summary: Removes file from installer. files: Tools/msi/msi.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Tools/msi/msi.py b/Tools/msi/msi.py --- a/Tools/msi/msi.py +++ b/Tools/msi/msi.py @@ -1053,7 +1053,6 @@ lib.add_file("sgml_input.html") lib.add_file("testtar.tar") lib.add_file("test_difflib_expect.html") - lib.add_file("check_soundcard.vbs") lib.add_file("empty.vbs") lib.add_file("Sine-1000Hz-300ms.aif") lib.add_file("revocation.crl") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:07:39 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 18:07:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_Lib/=5Fsysconfigdat?= =?utf-8?q?a=2Epy_from_=2Egitignore?= Message-ID: <20160909180739.36323.75402.73B54F29@psf.io> https://hg.python.org/cpython/rev/49a5d2b0e5c1 changeset: 103421:49a5d2b0e5c1 parent: 103418:cf91a929b81a user: Zachary Ware date: Fri Sep 09 11:07:23 2016 -0700 summary: Remove Lib/_sysconfigdata.py from .gitignore files: .gitignore | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -21,7 +21,6 @@ Lib/distutils/command/*.pdb Lib/lib2to3/*.pickle Lib/test/data/* -Lib/_sysconfigdata.py Lib/plat-mac/errors.rsrc.df.rsrc Makefile Makefile.pre -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:11:49 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 09 Sep 2016 18:11:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Mention_that_the_order-pre?= =?utf-8?q?serving_aspect_of_the_new_dict?= Message-ID: <20160909181149.18843.29404.D157F6DD@psf.io> https://hg.python.org/cpython/rev/47526892cacf changeset: 103422:47526892cacf user: Brett Cannon date: Fri Sep 09 11:11:45 2016 -0700 summary: Mention that the order-preserving aspect of the new dict implementation is an implementation detail (and why that is so). files: Doc/whatsnew/3.6.rst | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -416,7 +416,14 @@ * :func:`dict` now uses a "compact" representation `pioneered by PyPy `_. :pep:`468` (Preserving the order of ``**kwargs`` in a function.) is - implemented by this. (Contributed by INADA Naoki in :issue:`27350`. Idea + implemented by this. The order-preserving aspect of this new + implementation is considered an implementation detail and should + not be relied upon (this may change in the future, but it is desired + to have this new dict implementation in the language for a few + releases before changing the language spec to mandate + order-preserving semantics for all current and future Python + implementations). + (Contributed by INADA Naoki in :issue:`27350`. Idea `originally suggested by Raymond Hettinger `_.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:15:06 2016 From: python-checkins at python.org (yury.selivanov) Date: Fri, 09 Sep 2016 18:15:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_tests=3A_use_subTest_in_te?= =?utf-8?q?st=5Funparse=2Etest=5Ffiles?= Message-ID: <20160909181505.87575.57545.F163500A@psf.io> https://hg.python.org/cpython/rev/842c01f2f062 changeset: 103423:842c01f2f062 user: Yury Selivanov date: Fri Sep 09 11:14:59 2016 -0700 summary: tests: use subTest in test_unparse.test_files files: Lib/test/test_tools/test_unparse.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_tools/test_unparse.py b/Lib/test/test_tools/test_unparse.py --- a/Lib/test/test_tools/test_unparse.py +++ b/Lib/test/test_tools/test_unparse.py @@ -293,8 +293,9 @@ print(f'Skipping {filename}: see issue 27921') continue - source = read_pyfile(filename) - self.check_roundtrip(source) + with self.subTest(filename=filename): + source = read_pyfile(filename) + self.check_roundtrip(source) if __name__ == '__main__': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:19:21 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 09 Sep 2016 18:19:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Mention_how_requiring_orde?= =?utf-8?q?red_dicts_breaks_backwards-compatibility=2E?= Message-ID: <20160909181921.59651.55346.8DE8F274@psf.io> https://hg.python.org/cpython/rev/41087fc6962e changeset: 103424:41087fc6962e parent: 103422:47526892cacf user: Brett Cannon date: Fri Sep 09 11:18:21 2016 -0700 summary: Mention how requiring ordered dicts breaks backwards-compatibility. files: Doc/whatsnew/3.6.rst | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -422,7 +422,9 @@ to have this new dict implementation in the language for a few releases before changing the language spec to mandate order-preserving semantics for all current and future Python - implementations). + implementations; this also helps preserve backwards-compatibility + with older versions of the language where random iteration order is + still in effect, e.g. Python 3.5). (Contributed by INADA Naoki in :issue:`27350`. Idea `originally suggested by Raymond Hettinger `_.) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:19:22 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 09 Sep 2016 18:19:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge?= Message-ID: <20160909181921.21157.26321.EE93E110@psf.io> https://hg.python.org/cpython/rev/847eed281646 changeset: 103425:847eed281646 parent: 103424:41087fc6962e parent: 103423:842c01f2f062 user: Brett Cannon date: Fri Sep 09 11:19:16 2016 -0700 summary: Merge files: Lib/test/test_tools/test_unparse.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_tools/test_unparse.py b/Lib/test/test_tools/test_unparse.py --- a/Lib/test/test_tools/test_unparse.py +++ b/Lib/test/test_tools/test_unparse.py @@ -293,8 +293,9 @@ print(f'Skipping {filename}: see issue 27921') continue - source = read_pyfile(filename) - self.check_roundtrip(source) + with self.subTest(filename=filename): + source = read_pyfile(filename) + self.check_roundtrip(source) if __name__ == '__main__': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:22:42 2016 From: python-checkins at python.org (eric.snow) Date: Fri, 09 Sep 2016 18:22:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Doc_updates_for_PEPs_520_a?= =?utf-8?q?nd_468=2E?= Message-ID: <20160909182241.1774.16523.51AE2524@psf.io> https://hg.python.org/cpython/rev/07fa802cafae changeset: 103426:07fa802cafae user: Eric Snow date: Fri Sep 09 11:22:14 2016 -0700 summary: Doc updates for PEPs 520 and 468. files: Doc/reference/datamodel.rst | 6 ++-- Doc/whatsnew/3.6.rst | 28 ++++++++++++++++-------- 2 files changed, 22 insertions(+), 12 deletions(-) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1801,9 +1801,9 @@ in the local namespace as the defined class. When a new class is created by ``type.__new__``, the object provided as the -namespace parameter is copied to a standard Python dictionary and the original -object is discarded. The new copy becomes the :attr:`~object.__dict__` attribute -of the class object. +namespace parameter is copied to a new ordered mapping and the original +object is discarded. The new copy is wrapped in a read-only proxy, which +becomes the :attr:`~object.__dict__` attribute of the class object. .. seealso:: diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -116,7 +116,10 @@ :pep:`4XX` - Python Virtual Environments PEP written by Carl Meyer -.. XXX PEP 520: :ref:`Preserving Class Attribute Definition Order` +* PEP 520: :ref:`Preserving Class Attribute Definition Order` + +* PEP 468: :ref:`Preserving Keyword Argument Order` + New Features ============ @@ -380,17 +383,10 @@ Attributes in a class definition body have a natural ordering: the same order in which the names appear in the source. This order is now -preserved in the new class's ``__definition_order__`` attribute. It is -a tuple of the attribute names, in the order in which they appear in -the class definition body. - -For types that don't have a definition (e.g. builtins), or the attribute -order could not be determined, ``__definition_order__`` is ``None``. +preserved in the new class's ``__dict__`` attribute. Also, the effective default class *execution* namespace (returned from ``type.__prepare__()``) is now an insertion-order-preserving mapping. -For CPython, it is now ``collections.OrderedDict``. Note that the -class namespace, ``cls.__dict__``, is unchanged. .. seealso:: @@ -398,6 +394,20 @@ PEP written and implemented by Eric Snow. +.. _whatsnew-kwargs: + +PEP 468: Preserving Keyword Argument Order +========================================== + +``**kwargs`` in a function signature is now guaranteed to be an +insertion-order-preserving mapping. + +.. seealso:: + + :pep:`468` - Preserving Keyword Argument Order + PEP written and implemented by Eric Snow. + + PEP 509: Add a private version to dict -------------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:38:43 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 18:38:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_Lib/test/test=5Fpep?= =?utf-8?q?247=2Epy?= Message-ID: <20160909183843.18872.67167.07F82844@psf.io> https://hg.python.org/cpython/rev/c477ea0d15ba changeset: 103427:c477ea0d15ba user: Zachary Ware date: Fri Sep 09 11:38:38 2016 -0700 summary: Remove Lib/test/test_pep247.py This test file is a holdover from the days before hashlib, and doesn't seem to have anything of value in it. files: Lib/test/test_pep247.py | 66 ----------------------------- 1 files changed, 0 insertions(+), 66 deletions(-) diff --git a/Lib/test/test_pep247.py b/Lib/test/test_pep247.py deleted file mode 100644 --- a/Lib/test/test_pep247.py +++ /dev/null @@ -1,66 +0,0 @@ -""" -Test suite to check compliance with PEP 247, the standard API -for hashing algorithms -""" - -import hmac -import unittest -from hashlib import md5, sha1, sha224, sha256, sha384, sha512 - -class Pep247Test(unittest.TestCase): - - def check_module(self, module, key=None): - self.assertTrue(hasattr(module, 'digest_size')) - self.assertTrue(module.digest_size is None or module.digest_size > 0) - self.check_object(module.new, module.digest_size, key) - - def check_object(self, cls, digest_size, key, digestmod=None): - if key is not None: - if digestmod is None: - digestmod = md5 - obj1 = cls(key, digestmod=digestmod) - obj2 = cls(key, b'string', digestmod=digestmod) - h1 = cls(key, b'string', digestmod=digestmod).digest() - obj3 = cls(key, digestmod=digestmod) - obj3.update(b'string') - h2 = obj3.digest() - else: - obj1 = cls() - obj2 = cls(b'string') - h1 = cls(b'string').digest() - obj3 = cls() - obj3.update(b'string') - h2 = obj3.digest() - self.assertEqual(h1, h2) - self.assertTrue(hasattr(obj1, 'digest_size')) - - if digest_size is not None: - self.assertEqual(obj1.digest_size, digest_size) - - self.assertEqual(obj1.digest_size, len(h1)) - obj1.update(b'string') - obj_copy = obj1.copy() - self.assertEqual(obj1.digest(), obj_copy.digest()) - self.assertEqual(obj1.hexdigest(), obj_copy.hexdigest()) - - digest, hexdigest = obj1.digest(), obj1.hexdigest() - hd2 = "" - for byte in digest: - hd2 += '%02x' % byte - self.assertEqual(hd2, hexdigest) - - def test_md5(self): - self.check_object(md5, None, None) - - def test_sha(self): - self.check_object(sha1, None, None) - self.check_object(sha224, None, None) - self.check_object(sha256, None, None) - self.check_object(sha384, None, None) - self.check_object(sha512, None, None) - - def test_hmac(self): - self.check_module(hmac, key=b'abc') - -if __name__ == '__main__': - unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:39:44 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 18:39:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_unused_osx10=2E5_sd?= =?utf-8?q?k_check?= Message-ID: <20160909183944.2723.14825.782DFC80@psf.io> https://hg.python.org/cpython/rev/b366a4d5860a changeset: 103428:b366a4d5860a user: Benjamin Peterson date: Fri Sep 09 11:37:58 2016 -0700 summary: remove unused osx10.5 sdk check files: configure | 31 ------------------------------- configure.ac | 10 ---------- pyconfig.h.in | 3 --- 3 files changed, 0 insertions(+), 44 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -10806,37 +10806,6 @@ fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for OSX 10.5 SDK or later" >&5 -$as_echo_n "checking for OSX 10.5 SDK or later... " >&6; } -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - - #include -int -main () -{ -FSIORefNum fRef = 0 - ; - return 0; -} - -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - - -$as_echo "#define HAVE_OSX105_SDK 1" >>confdefs.h - - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } - -else - - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - # Check for --with-doc-strings { $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-doc-strings" >&5 $as_echo_n "checking for --with-doc-strings... " >&6; } diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3203,16 +3203,6 @@ AC_MSG_RESULT(no) ]) -AC_MSG_CHECKING(for OSX 10.5 SDK or later) -AC_COMPILE_IFELSE([ - AC_LANG_PROGRAM([[#include ]], [[FSIORefNum fRef = 0]]) -],[ - AC_DEFINE(HAVE_OSX105_SDK, 1, [Define if compiling using MacOS X 10.5 SDK or later.]) - AC_MSG_RESULT(yes) -],[ - AC_MSG_RESULT(no) -]) - # Check for --with-doc-strings AC_MSG_CHECKING(for --with-doc-strings) AC_ARG_WITH(doc-strings, diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -649,9 +649,6 @@ /* Define to 1 if you have the `openpty' function. */ #undef HAVE_OPENPTY -/* Define if compiling using MacOS X 10.5 SDK or later. */ -#undef HAVE_OSX105_SDK - /* Define to 1 if you have the `pathconf' function. */ #undef HAVE_PATHCONF -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:44:47 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 18:44:47 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2NjE5?= =?utf-8?q?=3A_Improves_error_message_when_installing_on_out-of-date_Windo?= =?utf-8?q?ws?= Message-ID: <20160909184447.2613.19211.E1E5E04C@psf.io> https://hg.python.org/cpython/rev/a95e2a43b09b changeset: 103429:a95e2a43b09b branch: 3.5 parent: 103412:aeb39d4475c5 user: Steve Dower date: Fri Sep 09 11:41:28 2016 -0700 summary: Issue #26619: Improves error message when installing on out-of-date Windows Server files: Tools/msi/bundle/Default.wxl | 10 + Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 58 ++++++--- 2 files changed, 50 insertions(+), 18 deletions(-) diff --git a/Tools/msi/bundle/Default.wxl b/Tools/msi/bundle/Default.wxl --- a/Tools/msi/bundle/Default.wxl +++ b/Tools/msi/bundle/Default.wxl @@ -132,4 +132,14 @@ Windows Vista or later is required to install and use [WixBundleName]. Visit <a href="https://www.python.org/">python.org</a> to download Python 3.4. + + Windows Server 2008 R2 Service Pack 1 and all applicable updates are required to install [WixBundleName]. + +Please <a href="https://www.bing.com/search?q=how%20to%20install%20windows%20server%202008%20r2%20service%20pack%201">update your machine</a> and then restart the installation. + Windows Server 2008 Service Pack 2 and all applicable updates are required to install [WixBundleName]. + +Please <a href="https://www.bing.com/search?q=how%20to%20install%20windows%20server%202008%20service%20pack%202">update your machine</a> and then restart the installation. + Windows Server 2008 SP2 or later is required to install and use [WixBundleName]. + +Visit <a href="https://www.python.org/">python.org</a> to download Python 3.4. diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -3032,24 +3032,46 @@ void ValidateOperatingSystem() { LOC_STRING *pLocString = nullptr; - if (IsWindows7SP1OrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows 7 SP1 or later"); - return; - } else if (IsWindows7OrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows 7 RTM"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 1 is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureWin7MissingSP1)", &pLocString); - } else if (IsWindowsVistaSP2OrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows Vista SP2"); - return; - } else if (IsWindowsVistaOrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Vista RTM or SP1"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 2 is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureVistaMissingSP2)", &pLocString); - } else { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows XP or earlier"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows Vista SP2 or later is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureXPOrEarlier)", &pLocString); + if (IsWindowsServer()) { + if (IsWindowsVersionOrGreater(6, 1, 1)) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows Server 2008 R2 or later"); + return; + } else if (IsWindowsVersionOrGreater(6, 1, 0)) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2008 R2"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 1 is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureWS2K8R2MissingSP1)", &pLocString); + } else if (IsWindowsVersionOrGreater(6, 0, 2)) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows Server 2008 SP2 or later"); + return; + } else if (IsWindowsVersionOrGreater(6, 0, 0)) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2008"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 2 is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureWS2K8MissingSP2)", &pLocString); + } else { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2003 or earlier"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows Server 2008 SP2 or later is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureWS2K3OrEarlier)", &pLocString); + } + } else { + if (IsWindows7SP1OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows 7 SP1 or later"); + return; + } else if (IsWindows7OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows 7 RTM"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 1 is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureWin7MissingSP1)", &pLocString); + } else if (IsWindowsVistaSP2OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows Vista SP2"); + return; + } else if (IsWindowsVistaOrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Vista RTM or SP1"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 2 is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureVistaMissingSP2)", &pLocString); + } else { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows XP or earlier"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows Vista SP2 or later is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureXPOrEarlier)", &pLocString); + } } if (pLocString && pLocString->wzText) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:44:47 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 18:44:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326619=3A_Improves_error_message_when_installing?= =?utf-8?q?_on_out-of-date_Windows?= Message-ID: <20160909184447.18843.43559.5C44C276@psf.io> https://hg.python.org/cpython/rev/fbb4aa1fcc47 changeset: 103430:fbb4aa1fcc47 parent: 103428:b366a4d5860a parent: 103429:a95e2a43b09b user: Steve Dower date: Fri Sep 09 11:44:26 2016 -0700 summary: Issue #26619: Improves error message when installing on out-of-date Windows Server files: Tools/msi/bundle/Default.wxl | 10 + Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 58 ++++++--- 2 files changed, 50 insertions(+), 18 deletions(-) diff --git a/Tools/msi/bundle/Default.wxl b/Tools/msi/bundle/Default.wxl --- a/Tools/msi/bundle/Default.wxl +++ b/Tools/msi/bundle/Default.wxl @@ -133,6 +133,16 @@ Visit <a href="https://www.python.org/">python.org</a> to download Python 3.4. + Windows Server 2008 R2 Service Pack 1 and all applicable updates are required to install [WixBundleName]. + +Please <a href="https://www.bing.com/search?q=how%20to%20install%20windows%20server%202008%20r2%20service%20pack%201">update your machine</a> and then restart the installation. + Windows Server 2008 Service Pack 2 and all applicable updates are required to install [WixBundleName]. + +Please <a href="https://www.bing.com/search?q=how%20to%20install%20windows%20server%202008%20service%20pack%202">update your machine</a> and then restart the installation. + Windows Server 2008 SP2 or later is required to install and use [WixBundleName]. + +Visit <a href="https://www.python.org/">python.org</a> to download Python 3.4. + Disable path length limit Changes your machine configuration to allow programs, including Python, to bypass the 260 character "MAX_PATH" limitation. diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -3017,24 +3017,46 @@ void ValidateOperatingSystem() { LOC_STRING *pLocString = nullptr; - if (IsWindows7SP1OrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows 7 SP1 or later"); - return; - } else if (IsWindows7OrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows 7 RTM"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 1 is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureWin7MissingSP1)", &pLocString); - } else if (IsWindowsVistaSP2OrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows Vista SP2"); - return; - } else if (IsWindowsVistaOrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Vista RTM or SP1"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 2 is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureVistaMissingSP2)", &pLocString); - } else { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows XP or earlier"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows Vista SP2 or later is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureXPOrEarlier)", &pLocString); + if (IsWindowsServer()) { + if (IsWindowsVersionOrGreater(6, 1, 1)) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows Server 2008 R2 or later"); + return; + } else if (IsWindowsVersionOrGreater(6, 1, 0)) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2008 R2"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 1 is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureWS2K8R2MissingSP1)", &pLocString); + } else if (IsWindowsVersionOrGreater(6, 0, 2)) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows Server 2008 SP2 or later"); + return; + } else if (IsWindowsVersionOrGreater(6, 0, 0)) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2008"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 2 is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureWS2K8MissingSP2)", &pLocString); + } else { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2003 or earlier"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows Server 2008 SP2 or later is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureWS2K3OrEarlier)", &pLocString); + } + } else { + if (IsWindows7SP1OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows 7 SP1 or later"); + return; + } else if (IsWindows7OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows 7 RTM"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 1 is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureWin7MissingSP1)", &pLocString); + } else if (IsWindowsVistaSP2OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows Vista SP2"); + return; + } else if (IsWindowsVistaOrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Vista RTM or SP1"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 2 is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureVistaMissingSP2)", &pLocString); + } else { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows XP or earlier"); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows Vista SP2 or later is required to continue installation"); + LocGetString(_wixLoc, L"#(loc.FailureXPOrEarlier)", &pLocString); + } } if (pLocString && pLocString->wzText) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:48:02 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 18:48:02 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQ2xvc2VzICMyMjcz?= =?utf-8?q?1=3A_Documents_change_of_console_mode=2E?= Message-ID: <20160909184802.2012.49997.E8981338@psf.io> https://hg.python.org/cpython/rev/b31e803d5ad3 changeset: 103431:b31e803d5ad3 branch: 3.5 parent: 103429:a95e2a43b09b user: Steve Dower date: Fri Sep 09 11:46:37 2016 -0700 summary: Closes #22731: Documents change of console mode. files: Doc/c-api/init.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -37,6 +37,10 @@ (without calling :c:func:`Py_Finalize` first). There is no return value; it is a fatal error if the initialization fails. + .. note:: + On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``, which will + also affect non-Python uses of the console using the C Runtime. + .. c:function:: void Py_InitializeEx(int initsigs) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:48:02 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 18:48:02 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2322731=3A_Documents_change_of_console_mode=2E?= Message-ID: <20160909184802.59525.28044.177283A1@psf.io> https://hg.python.org/cpython/rev/e93a8088a1f9 changeset: 103432:e93a8088a1f9 parent: 103430:fbb4aa1fcc47 parent: 103431:b31e803d5ad3 user: Steve Dower date: Fri Sep 09 11:47:02 2016 -0700 summary: Closes #22731: Documents change of console mode. files: Doc/c-api/init.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -37,6 +37,10 @@ (without calling :c:func:`Py_FinalizeEx` first). There is no return value; it is a fatal error if the initialization fails. + .. note:: + On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``, which will + also affect non-Python uses of the console using the C Runtime. + .. c:function:: void Py_InitializeEx(int initsigs) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:48:46 2016 From: python-checkins at python.org (yury.selivanov) Date: Fri, 09 Sep 2016 18:48:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328008=3A_Fix_test?= =?utf-8?q?=5Funparse?= Message-ID: <20160909184845.87475.16539.5BBA2528@psf.io> https://hg.python.org/cpython/rev/223c134af5a1 changeset: 103433:223c134af5a1 user: Yury Selivanov date: Fri Sep 09 11:48:39 2016 -0700 summary: Issue #28008: Fix test_unparse files: Tools/parser/unparse.py | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Tools/parser/unparse.py b/Tools/parser/unparse.py --- a/Tools/parser/unparse.py +++ b/Tools/parser/unparse.py @@ -444,7 +444,10 @@ self.write("}") def _comprehension(self, t): - self.write(" for ") + if t.is_async: + self.write(" async for ") + else: + self.write(" for ") self.dispatch(t.target) self.write(" in ") self.dispatch(t.iter) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 14:54:45 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 09 Sep 2016 18:54:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328049=3A_Add_docu?= =?utf-8?q?mentation_for_typing=2EAwaitable_and_friends=2E?= Message-ID: <20160909185445.69891.13651.C206D1EB@psf.io> https://hg.python.org/cpython/rev/eba19d4b2944 changeset: 103434:eba19d4b2944 user: Guido van Rossum date: Fri Sep 09 11:46:34 2016 -0700 summary: Issue #28049: Add documentation for typing.Awaitable and friends. By Michael Lee. files: Doc/library/typing.rst | 14 +++++++++++++- 1 files changed, 13 insertions(+), 1 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -646,6 +646,18 @@ A generic version of :class:`collections.abc.ValuesView`. +.. class:: Awaitable(Generic[T_co]) + + A generic version of :class:`collections.abc.Awaitable`. + +.. class:: AsyncIterable(Generic[T_co]) + + A generic version of :class:`collections.abc.AsyncIterable`. + +.. class:: AsyncIterator(AsyncIterable[T_co]) + + A generic version of :class:`collections.abc.AsyncIterator`. + .. class:: ContextManager(Generic[T_co]) A generic version of :class:`contextlib.AbstractContextManager`. @@ -684,7 +696,7 @@ start += 1 Alternatively, annotate your generator as having a return type of - ``Iterator[YieldType]``:: + either ``Iterable[YieldType]`` or ``Iterator[YieldType]``:: def infinite_stream(start: int) -> Iterator[int]: while True: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:00:29 2016 From: python-checkins at python.org (r.david.murray) Date: Fri, 09 Sep 2016 19:00:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzI4MDQ3OiBGaXgg?= =?utf-8?q?calculation_of_base64_line_length=2E?= Message-ID: <20160909190029.8596.60344.22F4988D@psf.io> https://hg.python.org/cpython/rev/99db6a25444b changeset: 103435:99db6a25444b branch: 3.5 parent: 103431:b31e803d5ad3 user: R David Murray date: Fri Sep 09 15:00:09 2016 -0400 summary: #28047: Fix calculation of base64 line length. This is buggy in the old email code as well, but it doesn't affect anything there because only the default line length is ever used there. files: Lib/email/contentmanager.py | 9 +++-- Lib/test/test_email/__init__.py | 12 ++++++- Lib/test/test_email/test_inversion.py | 23 ++++++++++++++- Misc/NEWS | 3 + 4 files changed, 40 insertions(+), 7 deletions(-) diff --git a/Lib/email/contentmanager.py b/Lib/email/contentmanager.py --- a/Lib/email/contentmanager.py +++ b/Lib/email/contentmanager.py @@ -126,12 +126,13 @@ msg.set_param(key, value) -# XXX: This is a cleaned-up version of base64mime.body_encode. It would -# be nice to drop both this and quoprimime.body_encode in favor of -# enhanced binascii routines that accepted a max_line_length parameter. +# XXX: This is a cleaned-up version of base64mime.body_encode (including a bug +# fix in the calculation of unencoded_bytes_per_line). It would be nice to +# drop both this and quoprimime.body_encode in favor of enhanced binascii +# routines that accepted a max_line_length parameter. def _encode_base64(data, max_line_length): encoded_lines = [] - unencoded_bytes_per_line = max_line_length * 3 // 4 + unencoded_bytes_per_line = max_line_length // 4 * 3 for i in range(0, len(data), unencoded_bytes_per_line): thisline = data[i:i+unencoded_bytes_per_line] encoded_lines.append(binascii.b2a_base64(thisline).decode('ascii')) diff --git a/Lib/test/test_email/__init__.py b/Lib/test/test_email/__init__.py --- a/Lib/test/test_email/__init__.py +++ b/Lib/test/test_email/__init__.py @@ -121,6 +121,10 @@ Note: if and only if the generated test name is a valid identifier can it be used to select the test individually from the unittest command line. + The values in the params dict can be a single value, a tuple, or a + dict. If a single value of a tuple, it is passed to the test function + as positional arguments. If a dict, it is a passed via **kw. + """ paramdicts = {} testers = collections.defaultdict(list) @@ -149,8 +153,12 @@ if name.startswith(paramsname): testnameroot = 'test_' + name[len(paramsname):] for paramname, params in paramsdict.items(): - test = (lambda self, name=name, params=params: - getattr(self, name)(*params)) + if hasattr(params, 'keys'): + test = (lambda self, name=name, params=params: + getattr(self, name)(**params)) + else: + test = (lambda self, name=name, params=params: + getattr(self, name)(*params)) testname = testnameroot + '_' + paramname test.__name__ = testname testfuncs[testname] = test diff --git a/Lib/test/test_email/test_inversion.py b/Lib/test/test_email/test_inversion.py --- a/Lib/test/test_email/test_inversion.py +++ b/Lib/test/test_email/test_inversion.py @@ -7,6 +7,7 @@ import io import unittest from email import policy, message_from_bytes +from email.message import EmailMessage from email.generator import BytesGenerator from test.test_email import TestEmailBase, parameterize @@ -23,7 +24,10 @@ @parameterize -class TestInversion(TestEmailBase, unittest.TestCase): +class TestInversion(TestEmailBase): + + policy = policy.default + message = EmailMessage def msg_as_input(self, msg): m = message_from_bytes(msg, policy=policy.SMTP) @@ -44,6 +48,23 @@ } + payload_params = { + 'plain_text': dict(payload='This is a test\n'*20), + 'base64_text': dict(payload=(('xy a'*40+'\n')*5), cte='base64'), + 'qp_text': dict(payload=(('xy a'*40+'\n')*5), cte='quoted-printable'), + } + + def payload_as_body(self, payload, **kw): + msg = self._make_message() + msg['From'] = 'foo' + msg['To'] = 'bar' + msg['Subject'] = 'payload round trip test' + msg.set_content(payload, **kw) + b = bytes(msg) + msg2 = message_from_bytes(b, policy=self.policy) + self.assertEqual(bytes(msg2), b) + self.assertEqual(msg2.get_content(), payload) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -62,6 +62,9 @@ Library ------- +- Issue #28047: Fixed calculation of line length used for the base64 CTE + in the new email policies. + - Issue #27445: Don't pass str(_charset) to MIMEText.set_payload(). Patch by Claude Paroz. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:01:34 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 19:01:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_--with=28out=29-sig?= =?utf-8?q?nal-module=2C_since_the_signal_module_is_non-optional?= Message-ID: <20160909190132.3168.76185.5063F28B@psf.io> https://hg.python.org/cpython/rev/d5bb5ad5a108 changeset: 103436:d5bb5ad5a108 parent: 103434:eba19d4b2944 user: Benjamin Peterson date: Fri Sep 09 12:01:10 2016 -0700 summary: remove --with(out)-signal-module, since the signal module is non-optional files: Makefile.pre.in | 7 +----- Modules/Setup.config.in | 3 -- Modules/Setup.dist | 1 + configure | 30 ----------------------------- configure.ac | 20 ------------------- 5 files changed, 2 insertions(+), 59 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -254,9 +254,6 @@ Modules/main.o \ Modules/gcmodule.o -# Used of signalmodule.o is not available -SIGNAL_OBJS= @SIGNAL_OBJS@ - IO_H= Modules/_io/_iomodule.h IO_OBJS= \ @@ -448,7 +445,6 @@ $(OBJECT_OBJS) \ $(PYTHON_OBJS) \ $(MODULE_OBJS) \ - $(SIGNAL_OBJS) \ $(MODOBJS) LIBRARY_OBJS= \ @@ -598,7 +594,7 @@ $(AR) $(ARFLAGS) $@ $(PARSER_OBJS) $(AR) $(ARFLAGS) $@ $(OBJECT_OBJS) $(AR) $(ARFLAGS) $@ $(PYTHON_OBJS) Python/frozen.o - $(AR) $(ARFLAGS) $@ $(MODULE_OBJS) $(SIGNAL_OBJS) + $(AR) $(ARFLAGS) $@ $(MODULE_OBJS) $(AR) $(ARFLAGS) $@ $(MODOBJS) $(RANLIB) $@ @@ -718,7 +714,6 @@ $(OBJECT_OBJS) \ $(PYTHON_OBJS) \ $(MODULE_OBJS) \ - $(SIGNAL_OBJS) \ $(MODOBJS) \ $(srcdir)/Modules/getbuildinfo.c $(CC) -c $(PY_CORE_CFLAGS) \ diff --git a/Modules/Setup.config.in b/Modules/Setup.config.in --- a/Modules/Setup.config.in +++ b/Modules/Setup.config.in @@ -6,8 +6,5 @@ # Threading @USE_THREAD_MODULE at _thread _threadmodule.c -# The signal module - at USE_SIGNAL_MODULE@_signal signalmodule.c - # The rest of the modules previously listed in this file are built # by the setup.py script in Python 2.1 and later. diff --git a/Modules/Setup.dist b/Modules/Setup.dist --- a/Modules/Setup.dist +++ b/Modules/Setup.dist @@ -117,6 +117,7 @@ _collections _collectionsmodule.c # Container types itertools itertoolsmodule.c # Functions creating iterators for efficient looping atexit atexitmodule.c # Register functions to be run at interpreter-shutdown +_signal signalmodule.c _stat _stat.c # stat.h interface time timemodule.c # -lm # time operations and variables diff --git a/configure b/configure --- a/configure +++ b/configure @@ -645,8 +645,6 @@ THREADOBJ LDLAST USE_THREAD_MODULE -SIGNAL_OBJS -USE_SIGNAL_MODULE TCLTK_LIBS TCLTK_INCLUDES LIBFFI_INCLUDEDIR @@ -828,7 +826,6 @@ with_tcltk_includes with_tcltk_libs with_dbmliborder -with_signal_module with_threads with_thread enable_ipv6 @@ -1532,7 +1529,6 @@ order to check db backends for dbm. Valid value is a colon separated string with the backend names `ndbm', `gdbm' and `bdb'. - --with-signal-module disable/enable signal module --with(out)-threads[=DIRECTORY] disable/enable thread support --with(out)-thread[=DIRECTORY] @@ -9964,32 +9960,6 @@ { $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_dbmliborder" >&5 $as_echo "$with_dbmliborder" >&6; } -# Determine if signalmodule should be used. - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-signal-module" >&5 -$as_echo_n "checking for --with-signal-module... " >&6; } - -# Check whether --with-signal-module was given. -if test "${with_signal_module+set}" = set; then : - withval=$with_signal_module; -fi - - -if test -z "$with_signal_module" -then with_signal_module="yes" -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_signal_module" >&5 -$as_echo "$with_signal_module" >&6; } - -if test "${with_signal_module}" = "yes"; then - USE_SIGNAL_MODULE="" - SIGNAL_OBJS="" -else - USE_SIGNAL_MODULE="#" - SIGNAL_OBJS="Parser/intrcheck.o Python/sigcheck.o" -fi - # This is used to generate Setup.config USE_THREAD_MODULE="" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -2814,26 +2814,6 @@ fi]) AC_MSG_RESULT($with_dbmliborder) -# Determine if signalmodule should be used. -AC_SUBST(USE_SIGNAL_MODULE) -AC_SUBST(SIGNAL_OBJS) -AC_MSG_CHECKING(for --with-signal-module) -AC_ARG_WITH(signal-module, - AS_HELP_STRING([--with-signal-module], [disable/enable signal module])) - -if test -z "$with_signal_module" -then with_signal_module="yes" -fi -AC_MSG_RESULT($with_signal_module) - -if test "${with_signal_module}" = "yes"; then - USE_SIGNAL_MODULE="" - SIGNAL_OBJS="" -else - USE_SIGNAL_MODULE="#" - SIGNAL_OBJS="Parser/intrcheck.o Python/sigcheck.o" -fi - # This is used to generate Setup.config AC_SUBST(USE_THREAD_MODULE) USE_THREAD_MODULE="" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:04:32 2016 From: python-checkins at python.org (eric.snow) Date: Fri, 09 Sep 2016 19:04:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327576=3A_Fix_call?= =?utf-8?b?IG9yZGVyIGluIE9yZGVyZWREaWN0Ll9faW5pdF9fKCku?= Message-ID: <20160909190431.59448.919.843C3B2B@psf.io> https://hg.python.org/cpython/rev/70758c12e888 changeset: 103437:70758c12e888 user: Eric Snow date: Fri Sep 09 11:59:08 2016 -0700 summary: Issue #27576: Fix call order in OrderedDict.__init__(). files: Lib/test/test_ordered_dict.py | 13 +++++++++++++ Misc/NEWS | 2 ++ Objects/odictobject.c | 17 +++++++++++++++-- 3 files changed, 30 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_ordered_dict.py b/Lib/test/test_ordered_dict.py --- a/Lib/test/test_ordered_dict.py +++ b/Lib/test/test_ordered_dict.py @@ -98,6 +98,19 @@ self.assertRaises(TypeError, OrderedDict().update, (), ()) self.assertRaises(TypeError, OrderedDict.update) + def test_init_calls(self): + calls = [] + class Spam: + def keys(self): + calls.append('keys') + return () + def items(self): + calls.append('items') + return () + + self.OrderedDict(Spam()) + self.assertEqual(calls, ['keys']) + def test_fromkeys(self): OrderedDict = self.OrderedDict od = OrderedDict.fromkeys('abc') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -113,6 +113,8 @@ Library ------- +- Issue #27576: Fix call order in OrderedDict.__init__(). + - email.generator.DecodedGenerator now supports the policy keyword. - Issue #28027: Remove undocumented modules from ``Lib/plat-*``: IN, CDROM, diff --git a/Objects/odictobject.c b/Objects/odictobject.c --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -2356,8 +2356,7 @@ PyObject *other = PyTuple_GET_ITEM(args, 0); /* borrowed reference */ assert(other != NULL); Py_INCREF(other); - if (PyDict_CheckExact(other) || - _PyObject_HasAttrId(other, &PyId_items)) { /* never fails */ + if PyDict_CheckExact(other) { PyObject *items; if (PyDict_CheckExact(other)) items = PyDict_Items(other); @@ -2400,6 +2399,20 @@ if (res != 0 || PyErr_Occurred()) return NULL; } + else if (_PyObject_HasAttrId(other, &PyId_items)) { /* never fails */ + PyObject *items; + if (PyDict_CheckExact(other)) + items = PyDict_Items(other); + else + items = _PyObject_CallMethodId(other, &PyId_items, NULL); + Py_DECREF(other); + if (items == NULL) + return NULL; + res = mutablemapping_add_pairs(self, items); + Py_DECREF(items); + if (res == -1) + return NULL; + } else { res = mutablemapping_add_pairs(self, other); Py_DECREF(other); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:10:04 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 19:10:04 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI0NTk0?= =?utf-8?q?=3A_Validates_persist_parameter_when_opening_MSI_database?= Message-ID: <20160909191003.13175.81186.AA98F17F@psf.io> https://hg.python.org/cpython/rev/fa89e107f43d changeset: 103439:fa89e107f43d branch: 3.5 parent: 103435:99db6a25444b user: Steve Dower date: Fri Sep 09 11:56:34 2016 -0700 summary: Issue #24594: Validates persist parameter when opening MSI database files: Misc/NEWS | 2 ++ PC/_msi.c | 20 +++++++++++++++++--- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -62,6 +62,8 @@ Library ------- +- Issue #24594: Validates persist parameter when opening MSI database + - Issue #28047: Fixed calculation of line length used for the base64 CTE in the new email policies. diff --git a/PC/_msi.c b/PC/_msi.c --- a/PC/_msi.c +++ b/PC/_msi.c @@ -955,6 +955,17 @@ 0, /*tp_is_gc*/ }; +#define Py_NOT_PERSIST(x, flag) \ + (x != (int)(flag) && \ + x != ((int)(flag) | MSIDBOPEN_PATCHFILE)) + +#define Py_INVALID_PERSIST(x) \ + (Py_NOT_PERSIST(x, MSIDBOPEN_READONLY) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_TRANSACT) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_DIRECT) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_CREATE) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_CREATEDIRECT)) + static PyObject* msiopendb(PyObject *obj, PyObject *args) { int status; @@ -962,11 +973,14 @@ int persist; MSIHANDLE h; msiobj *result; - if (!PyArg_ParseTuple(args, "si:MSIOpenDatabase", &path, &persist)) return NULL; - - status = MsiOpenDatabase(path, (LPCSTR)persist, &h); + /* We need to validate that persist is a valid MSIDBOPEN_* value. Otherwise, + MsiOpenDatabase may treat the value as a pointer, leading to unexpected + behavior. */ + if (Py_INVALID_PERSIST(persist)) + return msierror(ERROR_INVALID_PARAMETER); + status = MsiOpenDatabase(path, (LPCSTR)persist, &h); if (status != ERROR_SUCCESS) return msierror(status); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:10:07 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 19:10:07 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI0NTk0?= =?utf-8?q?=3A_Validates_persist_parameter_when_opening_MSI_database?= Message-ID: <20160909191003.59475.17057.A5DCCD72@psf.io> https://hg.python.org/cpython/rev/e524d5dc8767 changeset: 103438:e524d5dc8767 branch: 2.7 parent: 103420:dbaf2cbf8f6f user: Steve Dower date: Fri Sep 09 11:56:34 2016 -0700 summary: Issue #24594: Validates persist parameter when opening MSI database files: Misc/NEWS | 2 ++ PC/_msi.c | 20 +++++++++++++++++--- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -42,6 +42,8 @@ Library ------- +- Issue #24594: Validates persist parameter when opening MSI database + - Issue #27570: Avoid zero-length memcpy() etc calls with null source pointers in the "ctypes" and "array" modules. diff --git a/PC/_msi.c b/PC/_msi.c --- a/PC/_msi.c +++ b/PC/_msi.c @@ -938,6 +938,17 @@ 0, /*tp_is_gc*/ }; +#define Py_NOT_PERSIST(x, flag) \ + (x != (int)(flag) && \ + x != ((int)(flag) | MSIDBOPEN_PATCHFILE)) + +#define Py_INVALID_PERSIST(x) \ + (Py_NOT_PERSIST(x, MSIDBOPEN_READONLY) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_TRANSACT) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_DIRECT) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_CREATE) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_CREATEDIRECT)) + static PyObject* msiopendb(PyObject *obj, PyObject *args) { int status; @@ -945,11 +956,14 @@ int persist; MSIHANDLE h; msiobj *result; - if (!PyArg_ParseTuple(args, "si:MSIOpenDatabase", &path, &persist)) return NULL; - - status = MsiOpenDatabase(path, (LPCSTR)persist, &h); + /* We need to validate that persist is a valid MSIDBOPEN_* value. Otherwise, + MsiOpenDatabase may treat the value as a pointer, leading to unexpected + behavior. */ + if (Py_INVALID_PERSIST(persist)) + return msierror(ERROR_INVALID_PARAMETER); + status = MsiOpenDatabase(path, (LPCSTR)persist, &h); if (status != ERROR_SUCCESS) return msierror(status); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:10:07 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 19:10:07 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_with_3=2E5?= Message-ID: <20160909191004.69446.65774.D4EDAC6E@psf.io> https://hg.python.org/cpython/rev/da4c4979fc19 changeset: 103440:da4c4979fc19 parent: 103437:70758c12e888 parent: 103439:fa89e107f43d user: Steve Dower date: Fri Sep 09 12:09:07 2016 -0700 summary: Merge with 3.5 files: Lib/email/contentmanager.py | 9 +++-- Lib/test/test_email/__init__.py | 12 ++++++- Lib/test/test_email/test_inversion.py | 23 ++++++++++++++- Misc/NEWS | 5 +++ PC/_msi.c | 20 +++++++++++- 5 files changed, 59 insertions(+), 10 deletions(-) diff --git a/Lib/email/contentmanager.py b/Lib/email/contentmanager.py --- a/Lib/email/contentmanager.py +++ b/Lib/email/contentmanager.py @@ -126,12 +126,13 @@ msg.set_param(key, value) -# XXX: This is a cleaned-up version of base64mime.body_encode. It would -# be nice to drop both this and quoprimime.body_encode in favor of -# enhanced binascii routines that accepted a max_line_length parameter. +# XXX: This is a cleaned-up version of base64mime.body_encode (including a bug +# fix in the calculation of unencoded_bytes_per_line). It would be nice to +# drop both this and quoprimime.body_encode in favor of enhanced binascii +# routines that accepted a max_line_length parameter. def _encode_base64(data, max_line_length): encoded_lines = [] - unencoded_bytes_per_line = max_line_length * 3 // 4 + unencoded_bytes_per_line = max_line_length // 4 * 3 for i in range(0, len(data), unencoded_bytes_per_line): thisline = data[i:i+unencoded_bytes_per_line] encoded_lines.append(binascii.b2a_base64(thisline).decode('ascii')) diff --git a/Lib/test/test_email/__init__.py b/Lib/test/test_email/__init__.py --- a/Lib/test/test_email/__init__.py +++ b/Lib/test/test_email/__init__.py @@ -120,6 +120,10 @@ Note: if and only if the generated test name is a valid identifier can it be used to select the test individually from the unittest command line. + The values in the params dict can be a single value, a tuple, or a + dict. If a single value of a tuple, it is passed to the test function + as positional arguments. If a dict, it is a passed via **kw. + """ paramdicts = {} testers = collections.defaultdict(list) @@ -148,8 +152,12 @@ if name.startswith(paramsname): testnameroot = 'test_' + name[len(paramsname):] for paramname, params in paramsdict.items(): - test = (lambda self, name=name, params=params: - getattr(self, name)(*params)) + if hasattr(params, 'keys'): + test = (lambda self, name=name, params=params: + getattr(self, name)(**params)) + else: + test = (lambda self, name=name, params=params: + getattr(self, name)(*params)) testname = testnameroot + '_' + paramname test.__name__ = testname testfuncs[testname] = test diff --git a/Lib/test/test_email/test_inversion.py b/Lib/test/test_email/test_inversion.py --- a/Lib/test/test_email/test_inversion.py +++ b/Lib/test/test_email/test_inversion.py @@ -7,6 +7,7 @@ import io import unittest from email import policy, message_from_bytes +from email.message import EmailMessage from email.generator import BytesGenerator from test.test_email import TestEmailBase, parameterize @@ -23,7 +24,10 @@ @parameterize -class TestInversion(TestEmailBase, unittest.TestCase): +class TestInversion(TestEmailBase): + + policy = policy.default + message = EmailMessage def msg_as_input(self, msg): m = message_from_bytes(msg, policy=policy.SMTP) @@ -44,6 +48,23 @@ } + payload_params = { + 'plain_text': dict(payload='This is a test\n'*20), + 'base64_text': dict(payload=(('xy a'*40+'\n')*5), cte='base64'), + 'qp_text': dict(payload=(('xy a'*40+'\n')*5), cte='quoted-printable'), + } + + def payload_as_body(self, payload, **kw): + msg = self._make_message() + msg['From'] = 'foo' + msg['To'] = 'bar' + msg['Subject'] = 'payload round trip test' + msg.set_content(payload, **kw) + b = bytes(msg) + msg2 = message_from_bytes(b, policy=self.policy) + self.assertEqual(bytes(msg2), b) + self.assertEqual(msg2.get_content(), payload) + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -113,6 +113,11 @@ Library ------- +- Issue #24594: Validates persist parameter when opening MSI database + +- Issue #28047: Fixed calculation of line length used for the base64 CTE + in the new email policies. + - Issue #27576: Fix call order in OrderedDict.__init__(). - email.generator.DecodedGenerator now supports the policy keyword. diff --git a/PC/_msi.c b/PC/_msi.c --- a/PC/_msi.c +++ b/PC/_msi.c @@ -955,6 +955,17 @@ 0, /*tp_is_gc*/ }; +#define Py_NOT_PERSIST(x, flag) \ + (x != (int)(flag) && \ + x != ((int)(flag) | MSIDBOPEN_PATCHFILE)) + +#define Py_INVALID_PERSIST(x) \ + (Py_NOT_PERSIST(x, MSIDBOPEN_READONLY) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_TRANSACT) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_DIRECT) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_CREATE) && \ + Py_NOT_PERSIST(x, MSIDBOPEN_CREATEDIRECT)) + static PyObject* msiopendb(PyObject *obj, PyObject *args) { int status; @@ -962,11 +973,14 @@ int persist; MSIHANDLE h; msiobj *result; - if (!PyArg_ParseTuple(args, "si:MSIOpenDatabase", &path, &persist)) return NULL; - - status = MsiOpenDatabase(path, (LPCSTR)persist, &h); + /* We need to validate that persist is a valid MSIDBOPEN_* value. Otherwise, + MsiOpenDatabase may treat the value as a pointer, leading to unexpected + behavior. */ + if (Py_INVALID_PERSIST(persist)) + return msierror(ERROR_INVALID_PARAMETER); + status = MsiOpenDatabase(path, (LPCSTR)persist, &h); if (status != ERROR_SUCCESS) return msierror(status); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:24:13 2016 From: python-checkins at python.org (gregory.p.smith) Date: Fri, 09 Sep 2016 19:24:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Fix_make_buildbottest_to_not_re-trigger_a_profile-opt_bu?= =?utf-8?q?ild=2E_issue28035=2E?= Message-ID: <20160909192412.36396.63280.3FEC27EA@psf.io> https://hg.python.org/cpython/rev/914a81781291 changeset: 103442:914a81781291 parent: 103440:da4c4979fc19 parent: 103441:852b0f536791 user: Gregory P. Smith [Google Inc.] date: Fri Sep 09 12:23:05 2016 -0700 summary: Fix make buildbottest to not re-trigger a profile-opt build. issue28035. files: Makefile.pre.in | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -993,7 +993,7 @@ # Like testall, but with only one pass and without multiple processes. # Run an optional script to include information about the build environment. -buildbottest: all platform +buildbottest: build_all platform - at if which pybuildbot.identify >/dev/null 2>&1; then \ pybuildbot.identify "CC='$(CC)'" "CXX='$(CXX)'"; \ fi -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:24:13 2016 From: python-checkins at python.org (gregory.p.smith) Date: Fri, 09 Sep 2016 19:24:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Fix_make_build?= =?utf-8?q?bottest_to_not_re-trigger_a_profile-opt_build=2E_issue28035=2E?= Message-ID: <20160909192412.3168.16642.64B265B5@psf.io> https://hg.python.org/cpython/rev/852b0f536791 changeset: 103441:852b0f536791 branch: 3.5 parent: 103439:fa89e107f43d user: Gregory P. Smith [Google Inc.] date: Fri Sep 09 12:22:49 2016 -0700 summary: Fix make buildbottest to not re-trigger a profile-opt build. issue28035. files: Makefile.pre.in | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1011,7 +1011,7 @@ # Like testall, but with only one pass and without multiple processes. # Run an optional script to include information about the build environment. -buildbottest: all platform +buildbottest: build_all platform - at if which pybuildbot.identify >/dev/null 2>&1; then \ pybuildbot.identify "CC='$(CC)'" "CXX='$(CXX)'"; \ fi -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:25:50 2016 From: python-checkins at python.org (gregory.p.smith) Date: Fri, 09 Sep 2016 19:25:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Fix_make_build?= =?utf-8?q?bottest_to_not_re-trigger_a_profile-opt_build=2E_issue28035=2E?= Message-ID: <20160909192548.69487.62651.FE6735FA@psf.io> https://hg.python.org/cpython/rev/951f0de11a01 changeset: 103443:951f0de11a01 branch: 2.7 parent: 103438:e524d5dc8767 user: Gregory P. Smith [Google Inc.] date: Fri Sep 09 12:25:27 2016 -0700 summary: Fix make buildbottest to not re-trigger a profile-opt build. issue28035. files: Makefile.pre.in | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -883,7 +883,7 @@ # Like testall, but with a single pass only # run an optional script to include some information about the build environment -buildbottest: all platform +buildbottest: build_all platform - at if which pybuildbot.identify >/dev/null 2>&1; then \ pybuildbot.identify "CC='$(CC)'" "CXX='$(CXX)'"; \ fi -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:28:39 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 09 Sep 2016 19:28:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Rework_CALL=5FFUNCTION*_op?= =?utf-8?q?codes?= Message-ID: <20160909192838.1841.16118.AF8A7A6F@psf.io> https://hg.python.org/cpython/rev/a77756e480c2 changeset: 103444:a77756e480c2 parent: 103442:914a81781291 user: Victor Stinner date: Fri Sep 09 10:17:08 2016 -0700 summary: Rework CALL_FUNCTION* opcodes Issue #27213: Rework CALL_FUNCTION* opcodes to produce shorter and more efficient bytecode: * CALL_FUNCTION now only accepts position arguments * CALL_FUNCTION_KW accepts position arguments and keyword arguments, but keys of keyword arguments are packed into a constant tuple. * CALL_FUNCTION_EX is the most generic, it expects a tuple and a dict for positional and keyword arguments. CALL_FUNCTION_VAR and CALL_FUNCTION_VAR_KW opcodes have been removed. 2 tests of test_traceback are currently broken: skip test, the issue #28050 was created to track the issue. Patch by Demur Rumed, design by Serhiy Storchaka, reviewed by Serhiy Storchaka and Victor Stinner. files: Include/opcode.h | 3 +- Lib/dis.py | 2 +- Lib/importlib/_bootstrap_external.py | 3 +- Lib/opcode.py | 13 +- Lib/test/test_dis.py | 36 +- Lib/test/test_extcall.py | 18 +- Lib/test/test_traceback.py | 1 + Python/ceval.c | 526 +- Python/compile.c | 165 +- Python/importlib.h | 2894 ++++++------ Python/importlib_external.h | 3207 +++++++------ Python/opcode_targets.h | 4 +- 12 files changed, 3383 insertions(+), 3489 deletions(-) diff --git a/Include/opcode.h b/Include/opcode.h --- a/Include/opcode.h +++ b/Include/opcode.h @@ -108,9 +108,8 @@ #define LOAD_DEREF 136 #define STORE_DEREF 137 #define DELETE_DEREF 138 -#define CALL_FUNCTION_VAR 140 #define CALL_FUNCTION_KW 141 -#define CALL_FUNCTION_VAR_KW 142 +#define CALL_FUNCTION_EX 142 #define SETUP_WITH 143 #define EXTENDED_ARG 144 #define LIST_APPEND 145 diff --git a/Lib/dis.py b/Lib/dis.py --- a/Lib/dis.py +++ b/Lib/dis.py @@ -314,7 +314,7 @@ argrepr = argval elif op in hasfree: argval, argrepr = _get_name_info(arg, cells) - elif op in hasnargs: + elif op in hasnargs: # unused argrepr = "%d positional, %d keyword pair" % (arg%256, arg//256) yield Instruction(opname[op], op, arg, argval, argrepr, diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -236,6 +236,7 @@ # Python 3.6b1 3373 (add BUILD_STRING opcode #27078) # Python 3.6b1 3375 (add SETUP_ANNOTATIONS and STORE_ANNOTATION opcodes # #27985) +# Python 3.6a1 3376 (simplify CALL_FUNCTIONs & BUILD_MAP_UNPACK_WITH_CALL) # # MAGIC must change whenever the bytecode emitted by the compiler may no # longer be understood by older implementations of the eval loop (usually @@ -244,7 +245,7 @@ # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # in PC/launcher.c must also be updated. -MAGIC_NUMBER = (3375).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3376).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Lib/opcode.py b/Lib/opcode.py --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -31,7 +31,7 @@ haslocal = [] hascompare = [] hasfree = [] -hasnargs = [] +hasnargs = [] # unused opmap = {} opname = ['<%r>' % (op,) for op in range(256)] @@ -172,8 +172,7 @@ name_op('STORE_ANNOTATION', 127) # Index in name list def_op('RAISE_VARARGS', 130) # Number of raise arguments (1, 2, or 3) -def_op('CALL_FUNCTION', 131) # #args + (#kwargs << 8) -hasnargs.append(131) +def_op('CALL_FUNCTION', 131) # #args def_op('MAKE_FUNCTION', 132) # Flags def_op('BUILD_SLICE', 133) # Number of items def_op('LOAD_CLOSURE', 135) @@ -185,12 +184,8 @@ def_op('DELETE_DEREF', 138) hasfree.append(138) -def_op('CALL_FUNCTION_VAR', 140) # #args + (#kwargs << 8) -hasnargs.append(140) -def_op('CALL_FUNCTION_KW', 141) # #args + (#kwargs << 8) -hasnargs.append(141) -def_op('CALL_FUNCTION_VAR_KW', 142) # #args + (#kwargs << 8) -hasnargs.append(142) +def_op('CALL_FUNCTION_KW', 141) # #args + #kwargs +def_op('CALL_FUNCTION_EX', 142) # Flags jrel_op('SETUP_WITH', 143) diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -96,7 +96,7 @@ dis_f = """\ %3d 0 LOAD_GLOBAL 0 (print) 2 LOAD_FAST 0 (a) - 4 CALL_FUNCTION 1 (1 positional, 0 keyword pair) + 4 CALL_FUNCTION 1 6 POP_TOP %3d 8 LOAD_CONST 1 (1) @@ -108,7 +108,7 @@ dis_f_co_code = """\ 0 LOAD_GLOBAL 0 (0) 2 LOAD_FAST 0 (0) - 4 CALL_FUNCTION 1 (1 positional, 0 keyword pair) + 4 CALL_FUNCTION 1 6 POP_TOP 8 LOAD_CONST 1 (1) 10 RETURN_VALUE @@ -126,7 +126,7 @@ 4 LOAD_CONST 1 (1) %3d 6 LOAD_CONST 2 (10) - 8 CALL_FUNCTION 2 (2 positional, 0 keyword pair) + 8 CALL_FUNCTION 2 10 GET_ITER >> 12 FOR_ITER 4 (to 18) 14 STORE_FAST 0 (res) @@ -154,11 +154,11 @@ 10 MAKE_FUNCTION 0 12 LOAD_FAST 0 (x) 14 GET_ITER - 16 CALL_FUNCTION 1 (1 positional, 0 keyword pair) + 16 CALL_FUNCTION 1 %3d 18 LOAD_CONST 4 (1) 20 BINARY_ADD - 22 CALL_FUNCTION 1 (1 positional, 0 keyword pair) + 22 CALL_FUNCTION 1 24 RAISE_VARARGS 1 %3d >> 26 LOAD_CONST 0 (None) @@ -224,14 +224,14 @@ 3 10 LOAD_NAME 2 (fun) 12 LOAD_CONST 0 (1) - 14 CALL_FUNCTION 1 (1 positional, 0 keyword pair) + 14 CALL_FUNCTION 1 16 STORE_ANNOTATION 3 (y) 4 18 LOAD_CONST 0 (1) 20 LOAD_NAME 4 (lst) 22 LOAD_NAME 2 (fun) 24 LOAD_CONST 1 (0) - 26 CALL_FUNCTION 1 (1 positional, 0 keyword pair) + 26 CALL_FUNCTION 1 28 STORE_SUBSCR 30 LOAD_NAME 1 (int) 32 POP_TOP @@ -698,7 +698,7 @@ Instruction(opname='BUILD_LIST', opcode=103, arg=0, argval=0, argrepr='', offset=26, starts_line=None, is_jump_target=False), Instruction(opname='BUILD_MAP', opcode=105, arg=0, argval=0, argrepr='', offset=28, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval='Hello world!', argrepr="'Hello world!'", offset=30, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=7, argval=7, argrepr='7 positional, 0 keyword pair', offset=32, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=7, argval=7, argrepr='', offset=32, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=34, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='f', argrepr='f', offset=36, starts_line=8, is_jump_target=False), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=38, starts_line=None, is_jump_target=False), @@ -720,7 +720,7 @@ Instruction(opname='LOAD_DEREF', opcode=136, arg=3, argval='b', argrepr='b', offset=24, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_DEREF', opcode=136, arg=0, argval='c', argrepr='c', offset=26, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_DEREF', opcode=136, arg=1, argval='d', argrepr='d', offset=28, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=4, argval=4, argrepr='4 positional, 0 keyword pair', offset=30, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=4, argval=4, argrepr='', offset=30, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=32, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='inner', argrepr='inner', offset=34, starts_line=6, is_jump_target=False), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=36, starts_line=None, is_jump_target=False), @@ -734,7 +734,7 @@ Instruction(opname='LOAD_DEREF', opcode=136, arg=3, argval='d', argrepr='d', offset=8, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='e', argrepr='e', offset=10, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_FAST', opcode=124, arg=1, argval='f', argrepr='f', offset=12, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=6, argval=6, argrepr='6 positional, 0 keyword pair', offset=14, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=6, argval=6, argrepr='', offset=14, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=16, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=18, starts_line=None, is_jump_target=False), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=20, starts_line=None, is_jump_target=False), @@ -744,13 +744,13 @@ Instruction(opname='SETUP_LOOP', opcode=120, arg=52, argval=54, argrepr='to 54', offset=0, starts_line=3, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='range', argrepr='range', offset=2, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval=10, argrepr='10', offset=4, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='1 positional, 0 keyword pair', offset=6, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=6, starts_line=None, is_jump_target=False), Instruction(opname='GET_ITER', opcode=68, arg=None, argval=None, argrepr='', offset=8, starts_line=None, is_jump_target=False), Instruction(opname='FOR_ITER', opcode=93, arg=32, argval=44, argrepr='to 44', offset=10, starts_line=None, is_jump_target=True), Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=12, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=14, starts_line=4, is_jump_target=False), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=16, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='1 positional, 0 keyword pair', offset=18, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=18, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=20, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=22, starts_line=5, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=24, starts_line=None, is_jump_target=False), @@ -766,14 +766,14 @@ Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=44, starts_line=None, is_jump_target=True), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=46, starts_line=10, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='I can haz else clause?', argrepr="'I can haz else clause?'", offset=48, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='1 positional, 0 keyword pair', offset=50, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=50, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=52, starts_line=None, is_jump_target=False), Instruction(opname='SETUP_LOOP', opcode=120, arg=52, argval=108, argrepr='to 108', offset=54, starts_line=11, is_jump_target=True), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=56, starts_line=None, is_jump_target=True), Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=98, argval=98, argrepr='', offset=58, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=60, starts_line=12, is_jump_target=False), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=62, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='1 positional, 0 keyword pair', offset=64, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=64, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=66, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=68, starts_line=13, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=70, starts_line=None, is_jump_target=False), @@ -793,7 +793,7 @@ Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=98, starts_line=None, is_jump_target=True), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=100, starts_line=19, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=102, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='1 positional, 0 keyword pair', offset=104, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=104, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=106, starts_line=None, is_jump_target=False), Instruction(opname='SETUP_FINALLY', opcode=122, arg=70, argval=180, argrepr='to 180', offset=108, starts_line=20, is_jump_target=True), Instruction(opname='SETUP_EXCEPT', opcode=121, arg=12, argval=124, argrepr='to 124', offset=110, starts_line=None, is_jump_target=False), @@ -812,7 +812,7 @@ Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=136, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=138, starts_line=23, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=140, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='1 positional, 0 keyword pair', offset=142, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=142, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=144, starts_line=None, is_jump_target=False), Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=146, starts_line=None, is_jump_target=False), Instruction(opname='JUMP_FORWARD', opcode=110, arg=26, argval=176, argrepr='to 176', offset=148, starts_line=None, is_jump_target=False), @@ -822,7 +822,7 @@ Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=156, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=158, starts_line=26, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Never reach this', argrepr="'Never reach this'", offset=160, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='1 positional, 0 keyword pair', offset=162, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=162, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=False), Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=166, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=168, starts_line=None, is_jump_target=False), @@ -833,7 +833,7 @@ Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=178, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=180, starts_line=28, is_jump_target=True), Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=182, starts_line=None, is_jump_target=False), - Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='1 positional, 0 keyword pair', offset=184, starts_line=None, is_jump_target=False), + Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=184, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False), Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=190, starts_line=None, is_jump_target=False), diff --git a/Lib/test/test_extcall.py b/Lib/test/test_extcall.py --- a/Lib/test/test_extcall.py +++ b/Lib/test/test_extcall.py @@ -118,7 +118,7 @@ >>> g(*Nothing()) Traceback (most recent call last): ... - TypeError: g() argument after * must be an iterable, not Nothing + TypeError: 'Nothing' object is not iterable >>> class Nothing: ... def __len__(self): return 5 @@ -127,7 +127,7 @@ >>> g(*Nothing()) Traceback (most recent call last): ... - TypeError: g() argument after * must be an iterable, not Nothing + TypeError: 'Nothing' object is not iterable >>> class Nothing(): ... def __len__(self): return 5 @@ -231,34 +231,32 @@ >>> h(*h) Traceback (most recent call last): ... - TypeError: h() argument after * must be an iterable, not function + TypeError: 'function' object is not iterable >>> dir(*h) Traceback (most recent call last): ... - TypeError: dir() argument after * must be an iterable, not function + TypeError: 'function' object is not iterable >>> None(*h) Traceback (most recent call last): ... - TypeError: NoneType object argument after * must be an iterable, \ -not function + TypeError: 'function' object is not iterable >>> h(**h) Traceback (most recent call last): ... - TypeError: h() argument after ** must be a mapping, not function + TypeError: 'function' object is not a mapping >>> dir(**h) Traceback (most recent call last): ... - TypeError: dir() argument after ** must be a mapping, not function + TypeError: 'function' object is not a mapping >>> None(**h) Traceback (most recent call last): ... - TypeError: NoneType object argument after ** must be a mapping, \ -not function + TypeError: 'function' object is not a mapping >>> dir(b=1, **{'b': 1}) Traceback (most recent call last): diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -304,6 +304,7 @@ ]) # issue 26823 - Shrink recursive tracebacks + @unittest.skipIf(True, "FIXME: test broken, see issue #28050") def _check_recursive_traceback_display(self, render_exc): # Always show full diffs when this test fails # Note that rearranging things may require adjusting diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -109,19 +109,15 @@ /* Forward declarations */ #ifdef WITH_TSC -static PyObject * call_function(PyObject ***, int, uint64*, uint64*); +static PyObject * call_function(PyObject ***, Py_ssize_t, PyObject *, uint64*, uint64*); #else -static PyObject * call_function(PyObject ***, int); +static PyObject * call_function(PyObject ***, Py_ssize_t, PyObject *); #endif -static PyObject * fast_function(PyObject *, PyObject **, Py_ssize_t, Py_ssize_t); -static PyObject * do_call(PyObject *, PyObject ***, Py_ssize_t, Py_ssize_t); -static PyObject * ext_do_call(PyObject *, PyObject ***, int, Py_ssize_t, Py_ssize_t); -static PyObject * update_keyword_args(PyObject *, Py_ssize_t, PyObject ***, - PyObject *); -static PyObject * update_star_args(Py_ssize_t, Py_ssize_t, PyObject *, PyObject ***); +static PyObject * fast_function(PyObject *, PyObject ***, Py_ssize_t, PyObject *); +static PyObject * do_call(PyObject *, PyObject ***, Py_ssize_t, PyObject *); +static PyObject * do_call_core(PyObject *, PyObject *, PyObject *); +static PyObject * create_keyword_args(PyObject *, PyObject ***, PyObject *); static PyObject * load_args(PyObject ***, Py_ssize_t); -#define CALL_FLAG_VAR 1 -#define CALL_FLAG_KW 2 #ifdef LLTRACE static int lltrace; @@ -2659,8 +2655,14 @@ TARGET(BUILD_LIST_UNPACK) { int convert_to_tuple = opcode == BUILD_TUPLE_UNPACK; Py_ssize_t i; - PyObject *sum = PyList_New(0); + PyObject *sum; PyObject *return_value; + + if (convert_to_tuple && oparg == 1 && PyTuple_CheckExact(TOP())) { + DISPATCH(); + } + + sum = PyList_New(0); if (sum == NULL) goto error; @@ -2847,29 +2849,25 @@ TARGET(BUILD_MAP_UNPACK_WITH_CALL) TARGET(BUILD_MAP_UNPACK) { int with_call = opcode == BUILD_MAP_UNPACK_WITH_CALL; - int num_maps; - int function_location; Py_ssize_t i; - PyObject *sum = PyDict_New(); + PyObject *sum; + + if (with_call && oparg == 1 && PyDict_CheckExact(TOP())) { + DISPATCH(); + } + + sum = PyDict_New(); if (sum == NULL) goto error; - if (with_call) { - num_maps = oparg & 0xff; - function_location = (oparg>>8) & 0xff; - } - else { - num_maps = oparg; - } - - for (i = num_maps; i > 0; i--) { + + for (i = oparg; i > 0; i--) { PyObject *arg = PEEK(i); - if (with_call) { + if (with_call && PyDict_Size(sum)) { PyObject *intersection = _PyDictView_Intersect(sum, arg); if (intersection == NULL) { if (PyErr_ExceptionMatches(PyExc_AttributeError)) { - PyObject *func = ( - PEEK(function_location + num_maps)); + PyObject *func = PEEK(2 + oparg); PyErr_Format(PyExc_TypeError, "%.200s%.200s argument after ** " "must be a mapping, not %.200s", @@ -2884,7 +2882,7 @@ if (PySet_GET_SIZE(intersection)) { Py_ssize_t idx = 0; PyObject *key; - PyObject *func = PEEK(function_location + num_maps); + PyObject *func = PEEK(2 + oparg); Py_hash_t hash; _PySet_NextEntry(intersection, &idx, &key, &hash); if (!PyUnicode_Check(key)) { @@ -2918,7 +2916,7 @@ } } - while (num_maps--) + while (oparg--) Py_DECREF(POP()); PUSH(sum); DISPATCH(); @@ -3409,63 +3407,62 @@ PCALL(PCALL_ALL); sp = stack_pointer; #ifdef WITH_TSC - res = call_function(&sp, oparg, &intr0, &intr1); + res = call_function(&sp, oparg, NULL, &intr0, &intr1); #else - res = call_function(&sp, oparg); + res = call_function(&sp, oparg, NULL); #endif stack_pointer = sp; PUSH(res); - if (res == NULL) + if (res == NULL) { goto error; + } DISPATCH(); } - TARGET(CALL_FUNCTION_VAR) - TARGET(CALL_FUNCTION_KW) - TARGET(CALL_FUNCTION_VAR_KW) { - Py_ssize_t nargs = oparg & 0xff; - Py_ssize_t nkwargs = (oparg>>8) & 0xff; - int flags = (opcode - CALL_FUNCTION) & 3; - Py_ssize_t n; - PyObject **pfunc, *func, **sp, *res; + TARGET(CALL_FUNCTION_KW) { + PyObject **sp, *res, *names; + + names = POP(); + assert(PyTuple_CheckExact(names) && PyTuple_GET_SIZE(names) <= oparg); PCALL(PCALL_ALL); - - n = nargs + 2 * nkwargs; - if (flags & CALL_FLAG_VAR) { - n++; + sp = stack_pointer; +#ifdef WITH_TSC + res = call_function(&sp, oparg, names, &intr0, &intr1); +#else + res = call_function(&sp, oparg, names); +#endif + stack_pointer = sp; + PUSH(res); + Py_DECREF(names); + + if (res == NULL) { + goto error; } - if (flags & CALL_FLAG_KW) { - n++; + DISPATCH(); + } + + TARGET(CALL_FUNCTION_EX) { + PyObject *func, *callargs, *kwargs = NULL, *result; + PCALL(PCALL_ALL); + if (oparg & 0x01) { + kwargs = POP(); + assert(PyDict_CheckExact(kwargs)); } - pfunc = stack_pointer - n - 1; - func = *pfunc; - - if (PyMethod_Check(func) - && PyMethod_GET_SELF(func) != NULL) { - PyObject *self = PyMethod_GET_SELF(func); - Py_INCREF(self); - func = PyMethod_GET_FUNCTION(func); - Py_INCREF(func); - Py_SETREF(*pfunc, self); - nargs++; + callargs = POP(); + assert(PyTuple_CheckExact(callargs)); + func = TOP(); + + READ_TIMESTAMP(intr0); + result = do_call_core(func, callargs, kwargs); + READ_TIMESTAMP(intr1); + Py_DECREF(func); + Py_DECREF(callargs); + Py_XDECREF(kwargs); + + SET_TOP(result); + if (result == NULL) { + goto error; } - else { - Py_INCREF(func); - } - sp = stack_pointer; - READ_TIMESTAMP(intr0); - res = ext_do_call(func, &sp, flags, nargs, nkwargs); - READ_TIMESTAMP(intr1); - stack_pointer = sp; - Py_DECREF(func); - - while (stack_pointer > pfunc) { - PyObject *o = POP(); - Py_DECREF(o); - } - PUSH(res); - if (res == NULL) - goto error; DISPATCH(); } @@ -3950,7 +3947,6 @@ Py_DECREF(kwonly_sig); } - /* This is gonna seem *real weird*, but if you put some other code between PyEval_EvalFrame() and PyEval_EvalCodeEx() you will need to adjust the test in the if statements in Misc/gdbinit (pystack and pystackv). */ @@ -3959,6 +3955,7 @@ _PyEval_EvalCodeWithName(PyObject *_co, PyObject *globals, PyObject *locals, PyObject **args, Py_ssize_t argcount, PyObject **kws, Py_ssize_t kwcount, + PyObject *kwnames, PyObject **kwstack, PyObject **defs, Py_ssize_t defcount, PyObject *kwdefs, PyObject *closure, PyObject *name, PyObject *qualname) @@ -3972,6 +3969,7 @@ const Py_ssize_t total_args = co->co_argcount + co->co_kwonlyargcount; Py_ssize_t i, n; PyObject *kwdict; + Py_ssize_t kwcount2 = kwnames == NULL ? 0 : PyTuple_GET_SIZE(kwnames); assert((kwcount == 0) || (kws != NULL)); @@ -4033,7 +4031,7 @@ } } - /* Handle keyword arguments (passed as an array of (key, value)) */ + /* Handle keyword arguments passed as an array of (key, value) pairs */ for (i = 0; i < kwcount; i++) { PyObject **co_varnames; PyObject *keyword = kws[2*i]; @@ -4092,6 +4090,61 @@ SETLOCAL(j, value); } + /* Handle keyword arguments passed as keys tuple + values array */ + for (i = 0; i < kwcount2; i++) { + PyObject **co_varnames; + PyObject *keyword = PyTuple_GET_ITEM(kwnames, i); + PyObject *value = kwstack[i]; + int j; + if (keyword == NULL || !PyUnicode_Check(keyword)) { + PyErr_Format(PyExc_TypeError, + "%U() keywords must be strings", + co->co_name); + goto fail; + } + /* Speed hack: do raw pointer compares. As names are + normally interned this should almost always hit. */ + co_varnames = ((PyTupleObject *)(co->co_varnames))->ob_item; + for (j = 0; j < total_args; j++) { + PyObject *nm = co_varnames[j]; + if (nm == keyword) + goto kw_found2; + } + /* Slow fallback, just in case */ + for (j = 0; j < total_args; j++) { + PyObject *nm = co_varnames[j]; + int cmp = PyObject_RichCompareBool( + keyword, nm, Py_EQ); + if (cmp > 0) + goto kw_found2; + else if (cmp < 0) + goto fail; + } + if (j >= total_args && kwdict == NULL) { + PyErr_Format(PyExc_TypeError, + "%U() got an unexpected " + "keyword argument '%S'", + co->co_name, + keyword); + goto fail; + } + if (PyDict_SetItem(kwdict, keyword, value) == -1) { + goto fail; + } + continue; + kw_found2: + if (GETLOCAL(j) != NULL) { + PyErr_Format(PyExc_TypeError, + "%U() got multiple " + "values for argument '%S'", + co->co_name, + keyword); + goto fail; + } + Py_INCREF(value); + SETLOCAL(j, value); + } + /* Check the number of positional arguments */ if (argcount > co->co_argcount && !(co->co_flags & CO_VARARGS)) { too_many_positional(co, argcount, defcount, fastlocals); @@ -4244,6 +4297,7 @@ return _PyEval_EvalCodeWithName(_co, globals, locals, args, argcount, kws, kwcount, + NULL, NULL, defs, defcount, kwdefs, closure, NULL, NULL); @@ -4886,28 +4940,27 @@ } static PyObject * -call_function(PyObject ***pp_stack, int oparg +call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *names #ifdef WITH_TSC , uint64* pintr0, uint64* pintr1 #endif ) { - Py_ssize_t nargs = oparg & 0xff; - Py_ssize_t nkwargs = (oparg>>8) & 0xff; - int n = nargs + 2 * nkwargs; - PyObject **pfunc = (*pp_stack) - n - 1; + PyObject **pfunc = (*pp_stack) - oparg - 1; PyObject *func = *pfunc; PyObject *x, *w; + Py_ssize_t nk = names == NULL ? 0 : PyTuple_GET_SIZE(names); + Py_ssize_t nargs = oparg - nk; /* Always dispatch PyCFunction first, because these are presumed to be the most frequent callable object. */ - if (PyCFunction_Check(func) && nkwargs == 0) { + if (PyCFunction_Check(func)) { int flags = PyCFunction_GET_FLAGS(func); PyThreadState *tstate = PyThreadState_GET(); PCALL(PCALL_CFUNCTION); - if (flags & (METH_NOARGS | METH_O)) { + if (names == NULL && flags & (METH_NOARGS | METH_O)) { PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); if (flags & METH_NOARGS && nargs == 0) { @@ -4928,48 +4981,57 @@ } } else { - PyObject *callargs; + PyObject *callargs, *kwdict = NULL; + if (names != NULL) { + kwdict = create_keyword_args(names, pp_stack, func); + if (kwdict == NULL) { + x = NULL; + goto cfuncerror; + } + } callargs = load_args(pp_stack, nargs); if (callargs != NULL) { READ_TIMESTAMP(*pintr0); - C_TRACE(x, PyCFunction_Call(func,callargs,NULL)); + C_TRACE(x, PyCFunction_Call(func, callargs, kwdict)); READ_TIMESTAMP(*pintr1); - Py_XDECREF(callargs); + Py_DECREF(callargs); } else { x = NULL; } + Py_XDECREF(kwdict); } } else { - if (PyMethod_Check(func) && PyMethod_GET_SELF(func) != NULL) { - /* optimize access to bound methods */ - PyObject *self = PyMethod_GET_SELF(func); - PCALL(PCALL_METHOD); - PCALL(PCALL_BOUND_METHOD); - Py_INCREF(self); - func = PyMethod_GET_FUNCTION(func); - Py_INCREF(func); - Py_SETREF(*pfunc, self); - nargs++; - n++; - } - else { - Py_INCREF(func); - } - READ_TIMESTAMP(*pintr0); - if (PyFunction_Check(func)) { - x = fast_function(func, (*pp_stack) - n, nargs, nkwargs); - } - else { - x = do_call(func, pp_stack, nargs, nkwargs); - } - READ_TIMESTAMP(*pintr1); - Py_DECREF(func); - - assert((x != NULL) ^ (PyErr_Occurred() != NULL)); + if (PyMethod_Check(func) && PyMethod_GET_SELF(func) != NULL) { + /* optimize access to bound methods */ + PyObject *self = PyMethod_GET_SELF(func); + PCALL(PCALL_METHOD); + PCALL(PCALL_BOUND_METHOD); + Py_INCREF(self); + func = PyMethod_GET_FUNCTION(func); + Py_INCREF(func); + Py_SETREF(*pfunc, self); + nargs++; + } + else { + Py_INCREF(func); + } + + READ_TIMESTAMP(*pintr0); + if (PyFunction_Check(func)) { + x = fast_function(func, pp_stack, nargs, names); + } else { + x = do_call(func, pp_stack, nargs, names); + } + READ_TIMESTAMP(*pintr1); + + Py_DECREF(func); } +cfuncerror: + assert((x != NULL) ^ (PyErr_Occurred() != NULL)); + /* Clear the stack of the function object. Also removes the arguments in case they weren't consumed already (fast_function() and err_args() leave them on the stack). @@ -4980,7 +5042,6 @@ PCALL(PCALL_POP); } - assert((x != NULL) ^ (PyErr_Occurred() != NULL)); return x; } @@ -5033,19 +5094,16 @@ /* Similar to _PyFunction_FastCall() but keywords are passed a (key, value) pairs in stack */ static PyObject * -fast_function(PyObject *func, PyObject **stack, Py_ssize_t nargs, Py_ssize_t nkwargs) +fast_function(PyObject *func, PyObject ***pp_stack, Py_ssize_t nargs, PyObject *names) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); PyObject *kwdefs, *closure, *name, *qualname; PyObject **d; - int nd; - - assert(func != NULL); - assert(nargs >= 0); - assert(nkwargs >= 0); - assert((nargs == 0 && nkwargs == 0) || stack != NULL); + Py_ssize_t nkwargs = names == NULL ? 0 : PyTuple_GET_SIZE(names); + Py_ssize_t nd; + PyObject **stack = (*pp_stack)-nargs-nkwargs; PCALL(PCALL_FUNCTION); PCALL(PCALL_FAST_FUNCTION); @@ -5081,8 +5139,9 @@ } return _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, stack, nargs, - stack + nargs, nkwargs, - d, nd, kwdefs, + NULL, 0, + names, stack + nargs, + d, (int)nd, kwdefs, closure, name, qualname); } @@ -5166,6 +5225,7 @@ result = _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, args, nargs, k, nk, + NULL, NULL, d, nd, kwdefs, closure, name, qualname); Py_XDECREF(kwtuple); @@ -5173,22 +5233,17 @@ } static PyObject * -update_keyword_args(PyObject *orig_kwdict, Py_ssize_t nk, PyObject ***pp_stack, +create_keyword_args(PyObject *names, PyObject ***pp_stack, PyObject *func) { - PyObject *kwdict = NULL; - if (orig_kwdict == NULL) - kwdict = PyDict_New(); - else { - kwdict = PyDict_Copy(orig_kwdict); - Py_DECREF(orig_kwdict); - } + Py_ssize_t nk = PyTuple_GET_SIZE(names); + PyObject *kwdict = _PyDict_NewPresized(nk); if (kwdict == NULL) return NULL; while (--nk >= 0) { int err; PyObject *value = EXT_POP(*pp_stack); - PyObject *key = EXT_POP(*pp_stack); + PyObject *key = PyTuple_GET_ITEM(names, nk); if (PyDict_GetItem(kwdict, key) != NULL) { PyErr_Format(PyExc_TypeError, "%.200s%s got multiple values " @@ -5196,13 +5251,11 @@ PyEval_GetFuncName(func), PyEval_GetFuncDesc(func), key); - Py_DECREF(key); Py_DECREF(value); Py_DECREF(kwdict); return NULL; } err = PyDict_SetItem(kwdict, key, value); - Py_DECREF(key); Py_DECREF(value); if (err) { Py_DECREF(kwdict); @@ -5213,77 +5266,51 @@ } static PyObject * -update_star_args(Py_ssize_t nstack, Py_ssize_t nstar, PyObject *stararg, - PyObject ***pp_stack) +load_args(PyObject ***pp_stack, Py_ssize_t nargs) { - PyObject *callargs, *w; - - if (!nstack) { - if (!stararg) { - /* There are no positional arguments on the stack and there is no - sequence to be unpacked. */ - return PyTuple_New(0); - } - if (PyTuple_CheckExact(stararg)) { - /* No arguments are passed on the stack and the sequence is not a - tuple subclass so we can just pass the stararg tuple directly - to the function. */ - Py_INCREF(stararg); - return stararg; - } - } - - callargs = PyTuple_New(nstack + nstar); - if (callargs == NULL) { + PyObject *args = PyTuple_New(nargs); + + if (args == NULL) { return NULL; } - if (nstar) { - Py_ssize_t i; - for (i = 0; i < nstar; i++) { - PyObject *arg = PyTuple_GET_ITEM(stararg, i); - Py_INCREF(arg); - PyTuple_SET_ITEM(callargs, nstack + i, arg); - } - } - - while (--nstack >= 0) { - w = EXT_POP(*pp_stack); - PyTuple_SET_ITEM(callargs, nstack, w); - } - return callargs; -} - -static PyObject * -load_args(PyObject ***pp_stack, Py_ssize_t na) -{ - PyObject *args = PyTuple_New(na); - PyObject *w; - - if (args == NULL) - return NULL; - while (--na >= 0) { - w = EXT_POP(*pp_stack); - PyTuple_SET_ITEM(args, na, w); + while (--nargs >= 0) { + PyObject *arg= EXT_POP(*pp_stack); + PyTuple_SET_ITEM(args, nargs, arg); } return args; } static PyObject * -do_call(PyObject *func, PyObject ***pp_stack, Py_ssize_t nargs, Py_ssize_t nkwargs) +do_call(PyObject *func, PyObject ***pp_stack, Py_ssize_t nargs, PyObject *kwnames) { - PyObject *callargs = NULL; - PyObject *kwdict = NULL; - PyObject *result = NULL; - - if (nkwargs > 0) { - kwdict = update_keyword_args(NULL, nkwargs, pp_stack, func); - if (kwdict == NULL) - goto call_fail; + PyObject *callargs, *kwdict, *result; + + if (kwnames != NULL) { + kwdict = create_keyword_args(kwnames, pp_stack, func); + if (kwdict == NULL) { + return NULL; + } } + else { + kwdict = NULL; + } + callargs = load_args(pp_stack, nargs); - if (callargs == NULL) - goto call_fail; + if (callargs == NULL) { + Py_XDECREF(kwdict); + return NULL; + } + + result = do_call_core(func, callargs, kwdict); + Py_XDECREF(callargs); + Py_XDECREF(kwdict); + return result; +} + +static PyObject * +do_call_core(PyObject *func, PyObject *callargs, PyObject *kwdict) +{ #ifdef CALL_PROFILE /* At this point, we have to look at the type of func to update the call stats properly. Do it here so as to avoid @@ -5300,125 +5327,16 @@ else PCALL(PCALL_OTHER); #endif + if (PyCFunction_Check(func)) { + PyObject *result; PyThreadState *tstate = PyThreadState_GET(); C_TRACE(result, PyCFunction_Call(func, callargs, kwdict)); - } - else - result = PyObject_Call(func, callargs, kwdict); -call_fail: - Py_XDECREF(callargs); - Py_XDECREF(kwdict); - return result; -} - -static PyObject * -ext_do_call(PyObject *func, PyObject ***pp_stack, int flags, - Py_ssize_t nargs, Py_ssize_t nkwargs) -{ - Py_ssize_t nstar; - PyObject *callargs = NULL; - PyObject *stararg = NULL; - PyObject *kwdict = NULL; - PyObject *result = NULL; - - if (flags & CALL_FLAG_KW) { - kwdict = EXT_POP(*pp_stack); - if (!PyDict_CheckExact(kwdict)) { - PyObject *d; - d = PyDict_New(); - if (d == NULL) - goto ext_call_fail; - if (PyDict_Update(d, kwdict) != 0) { - Py_DECREF(d); - /* PyDict_Update raises attribute - * error (percolated from an attempt - * to get 'keys' attribute) instead of - * a type error if its second argument - * is not a mapping. - */ - if (PyErr_ExceptionMatches(PyExc_AttributeError)) { - PyErr_Format(PyExc_TypeError, - "%.200s%.200s argument after ** " - "must be a mapping, not %.200s", - PyEval_GetFuncName(func), - PyEval_GetFuncDesc(func), - kwdict->ob_type->tp_name); - } - goto ext_call_fail; - } - Py_DECREF(kwdict); - kwdict = d; - } - } - - if (nkwargs > 0) { - kwdict = update_keyword_args(kwdict, nkwargs, pp_stack, func); - if (kwdict == NULL) - goto ext_call_fail; - } - - if (flags & CALL_FLAG_VAR) { - stararg = EXT_POP(*pp_stack); - if (!PyTuple_Check(stararg)) { - PyObject *t = NULL; - if (Py_TYPE(stararg)->tp_iter == NULL && - !PySequence_Check(stararg)) { - PyErr_Format(PyExc_TypeError, - "%.200s%.200s argument after * " - "must be an iterable, not %.200s", - PyEval_GetFuncName(func), - PyEval_GetFuncDesc(func), - stararg->ob_type->tp_name); - goto ext_call_fail; - } - t = PySequence_Tuple(stararg); - if (t == NULL) { - goto ext_call_fail; - } - Py_DECREF(stararg); - stararg = t; - } - nstar = PyTuple_GET_SIZE(stararg); + return result; } else { - nstar = 0; + return PyObject_Call(func, callargs, kwdict); } - - callargs = update_star_args(nargs, nstar, stararg, pp_stack); - if (callargs == NULL) { - goto ext_call_fail; - } - -#ifdef CALL_PROFILE - /* At this point, we have to look at the type of func to - update the call stats properly. Do it here so as to avoid - exposing the call stats machinery outside ceval.c - */ - if (PyFunction_Check(func)) - PCALL(PCALL_FUNCTION); - else if (PyMethod_Check(func)) - PCALL(PCALL_METHOD); - else if (PyType_Check(func)) - PCALL(PCALL_TYPE); - else if (PyCFunction_Check(func)) - PCALL(PCALL_CFUNCTION); - else - PCALL(PCALL_OTHER); -#endif - if (PyCFunction_Check(func)) { - PyThreadState *tstate = PyThreadState_GET(); - C_TRACE(result, PyCFunction_Call(func, callargs, kwdict)); - } - else { - result = PyObject_Call(func, callargs, kwdict); - } - -ext_call_fail: - Py_XDECREF(callargs); - Py_XDECREF(kwdict); - Py_XDECREF(stararg); - return result; } /* Extract a slice index from a PyLong or an object with the diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -991,7 +991,7 @@ case BUILD_MAP_UNPACK: return 1 - oparg; case BUILD_MAP_UNPACK_WITH_CALL: - return 1 - (oparg & 0xFF); + return 1 - oparg; case BUILD_MAP: return 1 - 2*oparg; case BUILD_CONST_KEY_MAP: @@ -1038,15 +1038,12 @@ case RAISE_VARARGS: return -oparg; -#define NARGS(o) (((o) % 256) + 2*(((o) / 256) % 256)) case CALL_FUNCTION: - return -NARGS(oparg); - case CALL_FUNCTION_VAR: + return -oparg; case CALL_FUNCTION_KW: - return -NARGS(oparg)-1; - case CALL_FUNCTION_VAR_KW: - return -NARGS(oparg)-2; -#undef NARGS + return -oparg-1; + case CALL_FUNCTION_EX: + return - ((oparg & 0x01) != 0) - ((oparg & 0x02) != 0); case MAKE_FUNCTION: return -1 - ((oparg & 0x01) != 0) - ((oparg & 0x02) != 0) - ((oparg & 0x04) != 0) - ((oparg & 0x08) != 0); @@ -3500,22 +3497,29 @@ asdl_seq *args, asdl_seq *keywords) { - int code = 0; - Py_ssize_t nelts, i, nseen; - int nkw; + Py_ssize_t i, nseen, nelts, nkwelts; + int musttupleunpack = 0, mustdictunpack = 0; /* the number of tuples and dictionaries on the stack */ Py_ssize_t nsubargs = 0, nsubkwargs = 0; - nkw = 0; - nseen = 0; /* the number of positional arguments on the stack */ nelts = asdl_seq_LEN(args); + nkwelts = asdl_seq_LEN(keywords); + + for (i = 0; i < nkwelts; i++) { + keyword_ty kw = asdl_seq_GET(keywords, i); + if (kw->arg == NULL) { + mustdictunpack = 1; + break; + } + } + + nseen = n; /* the number of positional arguments on the stack */ for (i = 0; i < nelts; i++) { expr_ty elt = asdl_seq_GET(args, i); if (elt->kind == Starred_kind) { /* A star-arg. If we've seen positional arguments, - pack the positional arguments into a - tuple. */ + pack the positional arguments into a tuple. */ if (nseen) { ADDOP_I(c, BUILD_TUPLE, nseen); nseen = 0; @@ -3523,102 +3527,80 @@ } VISIT(c, expr, elt->v.Starred.value); nsubargs++; + musttupleunpack = 1; } - else if (nsubargs) { - /* We've seen star-args already, so we - count towards items-to-pack-into-tuple. */ + else { VISIT(c, expr, elt); nseen++; } - else { - /* Positional arguments before star-arguments - are left on the stack. */ - VISIT(c, expr, elt); - n++; + } + + /* Same dance again for keyword arguments */ + if (musttupleunpack || mustdictunpack) { + if (nseen) { + /* Pack up any trailing positional arguments. */ + ADDOP_I(c, BUILD_TUPLE, nseen); + nsubargs++; } - } - if (nseen) { - /* Pack up any trailing positional arguments. */ - ADDOP_I(c, BUILD_TUPLE, nseen); - nsubargs++; - } - if (nsubargs) { - code |= 1; - if (nsubargs > 1) { + if (musttupleunpack || nsubargs > 1) { /* If we ended up with more than one stararg, we need to concatenate them into a single sequence. */ - ADDOP_I(c, BUILD_LIST_UNPACK, nsubargs); + ADDOP_I(c, BUILD_TUPLE_UNPACK, nsubargs); } - } - - /* Same dance again for keyword arguments */ - nseen = 0; /* the number of keyword arguments on the stack following */ - nelts = asdl_seq_LEN(keywords); - for (i = 0; i < nelts; i++) { - keyword_ty kw = asdl_seq_GET(keywords, i); - if (kw->arg == NULL) { - /* A keyword argument unpacking. */ - if (nseen) { - if (nsubkwargs) { + else if (nsubargs == 0) { + ADDOP_I(c, BUILD_TUPLE, 0); + } + nseen = 0; /* the number of keyword arguments on the stack following */ + for (i = 0; i < nkwelts; i++) { + keyword_ty kw = asdl_seq_GET(keywords, i); + if (kw->arg == NULL) { + /* A keyword argument unpacking. */ + if (nseen) { if (!compiler_subkwargs(c, keywords, i - nseen, i)) return 0; nsubkwargs++; + nseen = 0; } - else { - Py_ssize_t j; - for (j = 0; j < nseen; j++) { - VISIT(c, keyword, asdl_seq_GET(keywords, j)); - } - nkw = nseen; - } - nseen = 0; + VISIT(c, expr, kw->value); + nsubkwargs++; } - VISIT(c, expr, kw->value); - nsubkwargs++; + else { + nseen++; + } } - else { - nseen++; - } - } - if (nseen) { - if (nsubkwargs) { + if (nseen) { /* Pack up any trailing keyword arguments. */ - if (!compiler_subkwargs(c, keywords, nelts - nseen, nelts)) + if (!compiler_subkwargs(c, keywords, nkwelts - nseen, nkwelts)) return 0; nsubkwargs++; } - else { - VISIT_SEQ(c, keyword, keywords); - nkw = nseen; + if (mustdictunpack || nsubkwargs > 1) { + /* Pack it all up */ + ADDOP_I(c, BUILD_MAP_UNPACK_WITH_CALL, nsubkwargs); } - } - if (nsubkwargs) { - code |= 2; - if (nsubkwargs > 1) { - /* Pack it all up */ - int function_pos = n + (code & 1) + 2 * nkw + 1; - ADDOP_I(c, BUILD_MAP_UNPACK_WITH_CALL, nsubkwargs | (function_pos << 8)); + ADDOP_I(c, CALL_FUNCTION_EX, nsubkwargs > 0); + return 1; + } + else if (nkwelts) { + PyObject *names; + VISIT_SEQ(c, keyword, keywords); + names = PyTuple_New(nkwelts); + if (names == NULL) { + return 0; } - } - assert(n < 1<<8); - assert(nkw < 1<<24); - n |= nkw << 8; - - switch (code) { - case 0: - ADDOP_I(c, CALL_FUNCTION, n); - break; - case 1: - ADDOP_I(c, CALL_FUNCTION_VAR, n); - break; - case 2: - ADDOP_I(c, CALL_FUNCTION_KW, n); - break; - case 3: - ADDOP_I(c, CALL_FUNCTION_VAR_KW, n); - break; - } - return 1; + for (i = 0; i < nkwelts; i++) { + keyword_ty kw = asdl_seq_GET(keywords, i); + Py_INCREF(kw->arg); + PyTuple_SET_ITEM(names, i, kw->arg); + } + ADDOP_N(c, LOAD_CONST, names, consts); + ADDOP_I(c, CALL_FUNCTION_KW, n + nelts + nkwelts); + return 1; + } + else { + ADDOP_I(c, CALL_FUNCTION, n + nelts); + return 1; + } } @@ -4040,7 +4022,6 @@ static int compiler_visit_keyword(struct compiler *c, keyword_ty k) { - ADDOP_O(c, LOAD_CONST, k->arg, consts); VISIT(c, expr, k->value); return 1; } diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -139,16 +139,16 @@ 114,4,0,0,0,114,6,0,0,0,218,10,95,112,97,116, 104,95,106,111,105,110,57,0,0,0,115,4,0,0,0,0, 2,10,1,114,30,0,0,0,99,1,0,0,0,0,0,0, - 0,5,0,0,0,5,0,0,0,67,0,0,0,115,98,0, + 0,5,0,0,0,5,0,0,0,67,0,0,0,115,96,0, 0,0,116,0,116,1,131,1,100,1,107,2,114,36,124,0, 106,2,116,3,131,1,92,3,125,1,125,2,125,3,124,1, - 124,3,102,2,83,0,120,52,116,4,124,0,131,1,68,0, - 93,40,125,4,124,4,116,1,107,6,114,46,124,0,106,5, - 124,4,100,2,100,1,144,1,131,1,92,2,125,1,125,3, - 124,1,124,3,102,2,83,0,113,46,87,0,100,3,124,0, - 102,2,83,0,41,4,122,32,82,101,112,108,97,99,101,109, - 101,110,116,32,102,111,114,32,111,115,46,112,97,116,104,46, - 115,112,108,105,116,40,41,46,233,1,0,0,0,90,8,109, + 124,3,102,2,83,0,120,50,116,4,124,0,131,1,68,0, + 93,38,125,4,124,4,116,1,107,6,114,46,124,0,106,5, + 124,4,100,1,100,2,141,2,92,2,125,1,125,3,124,1, + 124,3,102,2,83,0,113,46,87,0,100,3,124,0,102,2, + 83,0,41,4,122,32,82,101,112,108,97,99,101,109,101,110, + 116,32,102,111,114,32,111,115,46,112,97,116,104,46,115,112, + 108,105,116,40,41,46,233,1,0,0,0,41,1,90,8,109, 97,120,115,112,108,105,116,218,0,41,6,218,3,108,101,110, 114,23,0,0,0,218,10,114,112,97,114,116,105,116,105,111, 110,114,27,0,0,0,218,8,114,101,118,101,114,115,101,100, @@ -157,7 +157,7 @@ 114,18,0,0,0,114,4,0,0,0,114,4,0,0,0,114, 6,0,0,0,218,11,95,112,97,116,104,95,115,112,108,105, 116,63,0,0,0,115,16,0,0,0,0,2,12,1,16,1, - 8,1,14,1,8,1,20,1,12,1,114,40,0,0,0,99, + 8,1,14,1,8,1,18,1,12,1,114,40,0,0,0,99, 1,0,0,0,0,0,0,0,1,0,0,0,2,0,0,0, 67,0,0,0,115,10,0,0,0,116,0,106,1,124,0,131, 1,83,0,41,1,122,126,83,116,97,116,32,116,104,101,32, @@ -242,7 +242,7 @@ 101,95,97,116,111,109,105,99,106,0,0,0,115,26,0,0, 0,0,5,16,1,6,1,26,1,2,3,14,1,20,1,16, 1,14,1,2,1,14,1,14,1,6,1,114,58,0,0,0, - 105,47,13,0,0,233,2,0,0,0,114,15,0,0,0,115, + 105,48,13,0,0,233,2,0,0,0,114,15,0,0,0,115, 2,0,0,0,13,10,90,11,95,95,112,121,99,97,99,104, 101,95,95,122,4,111,112,116,45,122,3,46,112,121,122,4, 46,112,121,99,78,41,1,218,12,111,112,116,105,109,105,122, @@ -346,7 +346,7 @@ 103,90,15,97,108,109,111,115,116,95,102,105,108,101,110,97, 109,101,114,4,0,0,0,114,4,0,0,0,114,6,0,0, 0,218,17,99,97,99,104,101,95,102,114,111,109,95,115,111, - 117,114,99,101,3,1,0,0,115,48,0,0,0,0,18,8, + 117,114,99,101,4,1,0,0,115,48,0,0,0,0,18,8, 1,6,1,6,1,8,1,4,1,8,1,12,1,10,1,12, 1,16,1,8,1,8,1,8,1,24,1,8,1,12,1,6, 2,8,1,8,1,8,1,8,1,14,1,14,1,114,83,0, @@ -420,7 +420,7 @@ 112,116,95,108,101,118,101,108,90,13,98,97,115,101,95,102, 105,108,101,110,97,109,101,114,4,0,0,0,114,4,0,0, 0,114,6,0,0,0,218,17,115,111,117,114,99,101,95,102, - 114,111,109,95,99,97,99,104,101,48,1,0,0,115,46,0, + 114,111,109,95,99,97,99,104,101,49,1,0,0,115,46,0, 0,0,0,9,12,1,8,1,10,1,12,1,12,1,8,1, 6,1,10,1,10,1,8,1,6,1,10,1,8,1,16,1, 10,1,6,1,8,1,16,1,8,1,6,1,8,1,14,1, @@ -456,7 +456,7 @@ 115,105,111,110,218,11,115,111,117,114,99,101,95,112,97,116, 104,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, 218,15,95,103,101,116,95,115,111,117,114,99,101,102,105,108, - 101,82,1,0,0,115,20,0,0,0,0,7,12,1,4,1, + 101,83,1,0,0,115,20,0,0,0,0,7,12,1,4,1, 16,1,26,1,4,1,2,1,12,1,18,1,18,1,114,95, 0,0,0,99,1,0,0,0,0,0,0,0,1,0,0,0, 11,0,0,0,67,0,0,0,115,74,0,0,0,124,0,106, @@ -469,7 +469,7 @@ 0,0,114,83,0,0,0,114,70,0,0,0,114,78,0,0, 0,41,1,218,8,102,105,108,101,110,97,109,101,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,11,95,103, - 101,116,95,99,97,99,104,101,100,101,1,0,0,115,16,0, + 101,116,95,99,97,99,104,101,100,102,1,0,0,115,16,0, 0,0,0,1,14,1,2,1,8,1,14,1,8,1,14,1, 6,2,114,99,0,0,0,99,1,0,0,0,0,0,0,0, 2,0,0,0,11,0,0,0,67,0,0,0,115,52,0,0, @@ -483,7 +483,7 @@ 0,233,128,0,0,0,41,3,114,41,0,0,0,114,43,0, 0,0,114,42,0,0,0,41,2,114,37,0,0,0,114,44, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,10,95,99,97,108,99,95,109,111,100,101,113,1, + 0,0,218,10,95,99,97,108,99,95,109,111,100,101,114,1, 0,0,115,12,0,0,0,0,2,2,1,14,1,14,1,10, 3,8,1,114,101,0,0,0,99,1,0,0,0,0,0,0, 0,3,0,0,0,11,0,0,0,3,0,0,0,115,68,0, @@ -508,252 +508,253 @@ 97,105,108,115,32,116,104,101,110,32,73,109,112,111,114,116, 69,114,114,111,114,32,105,115,32,114,97,105,115,101,100,46, 10,10,32,32,32,32,78,99,2,0,0,0,0,0,0,0, - 4,0,0,0,5,0,0,0,31,0,0,0,115,64,0,0, + 4,0,0,0,4,0,0,0,31,0,0,0,115,68,0,0, 0,124,1,100,0,107,8,114,16,124,0,106,0,125,1,110, - 34,124,0,106,0,124,1,107,3,114,50,116,1,100,1,124, - 0,106,0,124,1,102,2,22,0,100,2,124,1,144,1,131, - 1,130,1,136,0,124,0,124,1,124,2,124,3,142,2,83, - 0,41,3,78,122,30,108,111,97,100,101,114,32,102,111,114, - 32,37,115,32,99,97,110,110,111,116,32,104,97,110,100,108, - 101,32,37,115,218,4,110,97,109,101,41,2,114,102,0,0, - 0,218,11,73,109,112,111,114,116,69,114,114,111,114,41,4, - 218,4,115,101,108,102,114,102,0,0,0,218,4,97,114,103, - 115,90,6,107,119,97,114,103,115,41,1,218,6,109,101,116, - 104,111,100,114,4,0,0,0,114,6,0,0,0,218,19,95, - 99,104,101,99,107,95,110,97,109,101,95,119,114,97,112,112, - 101,114,133,1,0,0,115,12,0,0,0,0,1,8,1,8, - 1,10,1,4,1,20,1,122,40,95,99,104,101,99,107,95, - 110,97,109,101,46,60,108,111,99,97,108,115,62,46,95,99, - 104,101,99,107,95,110,97,109,101,95,119,114,97,112,112,101, - 114,99,2,0,0,0,0,0,0,0,3,0,0,0,7,0, - 0,0,83,0,0,0,115,60,0,0,0,120,40,100,5,68, - 0,93,32,125,2,116,0,124,1,124,2,131,2,114,6,116, - 1,124,0,124,2,116,2,124,1,124,2,131,2,131,3,1, - 0,113,6,87,0,124,0,106,3,106,4,124,1,106,3,131, - 1,1,0,100,0,83,0,41,6,78,218,10,95,95,109,111, - 100,117,108,101,95,95,218,8,95,95,110,97,109,101,95,95, - 218,12,95,95,113,117,97,108,110,97,109,101,95,95,218,7, - 95,95,100,111,99,95,95,41,4,122,10,95,95,109,111,100, - 117,108,101,95,95,122,8,95,95,110,97,109,101,95,95,122, - 12,95,95,113,117,97,108,110,97,109,101,95,95,122,7,95, - 95,100,111,99,95,95,41,5,218,7,104,97,115,97,116,116, - 114,218,7,115,101,116,97,116,116,114,218,7,103,101,116,97, - 116,116,114,218,8,95,95,100,105,99,116,95,95,218,6,117, - 112,100,97,116,101,41,3,90,3,110,101,119,90,3,111,108, - 100,114,55,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,218,5,95,119,114,97,112,144,1,0,0, - 115,8,0,0,0,0,1,10,1,10,1,22,1,122,26,95, + 32,124,0,106,0,124,1,107,3,114,48,116,1,100,1,124, + 0,106,0,124,1,102,2,22,0,124,1,100,2,141,2,130, + 1,136,0,124,0,124,1,102,2,124,2,152,2,124,3,151, + 1,142,1,83,0,41,3,78,122,30,108,111,97,100,101,114, + 32,102,111,114,32,37,115,32,99,97,110,110,111,116,32,104, + 97,110,100,108,101,32,37,115,41,1,218,4,110,97,109,101, + 41,2,114,102,0,0,0,218,11,73,109,112,111,114,116,69, + 114,114,111,114,41,4,218,4,115,101,108,102,114,102,0,0, + 0,218,4,97,114,103,115,90,6,107,119,97,114,103,115,41, + 1,218,6,109,101,116,104,111,100,114,4,0,0,0,114,6, + 0,0,0,218,19,95,99,104,101,99,107,95,110,97,109,101, + 95,119,114,97,112,112,101,114,134,1,0,0,115,12,0,0, + 0,0,1,8,1,8,1,10,1,4,1,18,1,122,40,95, 99,104,101,99,107,95,110,97,109,101,46,60,108,111,99,97, - 108,115,62,46,95,119,114,97,112,41,1,78,41,3,218,10, - 95,98,111,111,116,115,116,114,97,112,114,117,0,0,0,218, - 9,78,97,109,101,69,114,114,111,114,41,3,114,106,0,0, - 0,114,107,0,0,0,114,117,0,0,0,114,4,0,0,0, - 41,1,114,106,0,0,0,114,6,0,0,0,218,11,95,99, - 104,101,99,107,95,110,97,109,101,125,1,0,0,115,14,0, - 0,0,0,8,14,7,2,1,10,1,14,2,14,5,10,1, - 114,120,0,0,0,99,2,0,0,0,0,0,0,0,5,0, - 0,0,4,0,0,0,67,0,0,0,115,60,0,0,0,124, - 0,106,0,124,1,131,1,92,2,125,2,125,3,124,2,100, - 1,107,8,114,56,116,1,124,3,131,1,114,56,100,2,125, - 4,116,2,106,3,124,4,106,4,124,3,100,3,25,0,131, - 1,116,5,131,2,1,0,124,2,83,0,41,4,122,155,84, - 114,121,32,116,111,32,102,105,110,100,32,97,32,108,111,97, - 100,101,114,32,102,111,114,32,116,104,101,32,115,112,101,99, - 105,102,105,101,100,32,109,111,100,117,108,101,32,98,121,32, - 100,101,108,101,103,97,116,105,110,103,32,116,111,10,32,32, - 32,32,115,101,108,102,46,102,105,110,100,95,108,111,97,100, - 101,114,40,41,46,10,10,32,32,32,32,84,104,105,115,32, - 109,101,116,104,111,100,32,105,115,32,100,101,112,114,101,99, - 97,116,101,100,32,105,110,32,102,97,118,111,114,32,111,102, - 32,102,105,110,100,101,114,46,102,105,110,100,95,115,112,101, - 99,40,41,46,10,10,32,32,32,32,78,122,44,78,111,116, - 32,105,109,112,111,114,116,105,110,103,32,100,105,114,101,99, - 116,111,114,121,32,123,125,58,32,109,105,115,115,105,110,103, - 32,95,95,105,110,105,116,95,95,114,62,0,0,0,41,6, - 218,11,102,105,110,100,95,108,111,97,100,101,114,114,33,0, - 0,0,114,63,0,0,0,114,64,0,0,0,114,50,0,0, - 0,218,13,73,109,112,111,114,116,87,97,114,110,105,110,103, - 41,5,114,104,0,0,0,218,8,102,117,108,108,110,97,109, - 101,218,6,108,111,97,100,101,114,218,8,112,111,114,116,105, - 111,110,115,218,3,109,115,103,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,218,17,95,102,105,110,100,95,109, - 111,100,117,108,101,95,115,104,105,109,153,1,0,0,115,10, - 0,0,0,0,10,14,1,16,1,4,1,22,1,114,127,0, - 0,0,99,4,0,0,0,0,0,0,0,11,0,0,0,19, - 0,0,0,67,0,0,0,115,128,1,0,0,105,0,125,4, - 124,2,100,1,107,9,114,22,124,2,124,4,100,2,60,0, - 110,4,100,3,125,2,124,3,100,1,107,9,114,42,124,3, - 124,4,100,4,60,0,124,0,100,1,100,5,133,2,25,0, - 125,5,124,0,100,5,100,6,133,2,25,0,125,6,124,0, - 100,6,100,7,133,2,25,0,125,7,124,5,116,0,107,3, - 114,122,100,8,106,1,124,2,124,5,131,2,125,8,116,2, - 106,3,100,9,124,8,131,2,1,0,116,4,124,8,124,4, - 141,1,130,1,110,86,116,5,124,6,131,1,100,5,107,3, - 114,166,100,10,106,1,124,2,131,1,125,8,116,2,106,3, - 100,9,124,8,131,2,1,0,116,6,124,8,131,1,130,1, - 110,42,116,5,124,7,131,1,100,5,107,3,114,208,100,11, - 106,1,124,2,131,1,125,8,116,2,106,3,100,9,124,8, - 131,2,1,0,116,6,124,8,131,1,130,1,124,1,100,1, - 107,9,144,1,114,116,121,16,116,7,124,1,100,12,25,0, - 131,1,125,9,87,0,110,20,4,0,116,8,107,10,114,254, - 1,0,1,0,1,0,89,0,110,48,88,0,116,9,124,6, - 131,1,124,9,107,3,144,1,114,46,100,13,106,1,124,2, - 131,1,125,8,116,2,106,3,100,9,124,8,131,2,1,0, - 116,4,124,8,124,4,141,1,130,1,121,16,124,1,100,14, + 108,115,62,46,95,99,104,101,99,107,95,110,97,109,101,95, + 119,114,97,112,112,101,114,99,2,0,0,0,0,0,0,0, + 3,0,0,0,7,0,0,0,83,0,0,0,115,60,0,0, + 0,120,40,100,5,68,0,93,32,125,2,116,0,124,1,124, + 2,131,2,114,6,116,1,124,0,124,2,116,2,124,1,124, + 2,131,2,131,3,1,0,113,6,87,0,124,0,106,3,106, + 4,124,1,106,3,131,1,1,0,100,0,83,0,41,6,78, + 218,10,95,95,109,111,100,117,108,101,95,95,218,8,95,95, + 110,97,109,101,95,95,218,12,95,95,113,117,97,108,110,97, + 109,101,95,95,218,7,95,95,100,111,99,95,95,41,4,122, + 10,95,95,109,111,100,117,108,101,95,95,122,8,95,95,110, + 97,109,101,95,95,122,12,95,95,113,117,97,108,110,97,109, + 101,95,95,122,7,95,95,100,111,99,95,95,41,5,218,7, + 104,97,115,97,116,116,114,218,7,115,101,116,97,116,116,114, + 218,7,103,101,116,97,116,116,114,218,8,95,95,100,105,99, + 116,95,95,218,6,117,112,100,97,116,101,41,3,90,3,110, + 101,119,90,3,111,108,100,114,55,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,5,95,119,114, + 97,112,145,1,0,0,115,8,0,0,0,0,1,10,1,10, + 1,22,1,122,26,95,99,104,101,99,107,95,110,97,109,101, + 46,60,108,111,99,97,108,115,62,46,95,119,114,97,112,41, + 1,78,41,3,218,10,95,98,111,111,116,115,116,114,97,112, + 114,117,0,0,0,218,9,78,97,109,101,69,114,114,111,114, + 41,3,114,106,0,0,0,114,107,0,0,0,114,117,0,0, + 0,114,4,0,0,0,41,1,114,106,0,0,0,114,6,0, + 0,0,218,11,95,99,104,101,99,107,95,110,97,109,101,126, + 1,0,0,115,14,0,0,0,0,8,14,7,2,1,10,1, + 14,2,14,5,10,1,114,120,0,0,0,99,2,0,0,0, + 0,0,0,0,5,0,0,0,4,0,0,0,67,0,0,0, + 115,60,0,0,0,124,0,106,0,124,1,131,1,92,2,125, + 2,125,3,124,2,100,1,107,8,114,56,116,1,124,3,131, + 1,114,56,100,2,125,4,116,2,106,3,124,4,106,4,124, + 3,100,3,25,0,131,1,116,5,131,2,1,0,124,2,83, + 0,41,4,122,155,84,114,121,32,116,111,32,102,105,110,100, + 32,97,32,108,111,97,100,101,114,32,102,111,114,32,116,104, + 101,32,115,112,101,99,105,102,105,101,100,32,109,111,100,117, + 108,101,32,98,121,32,100,101,108,101,103,97,116,105,110,103, + 32,116,111,10,32,32,32,32,115,101,108,102,46,102,105,110, + 100,95,108,111,97,100,101,114,40,41,46,10,10,32,32,32, + 32,84,104,105,115,32,109,101,116,104,111,100,32,105,115,32, + 100,101,112,114,101,99,97,116,101,100,32,105,110,32,102,97, + 118,111,114,32,111,102,32,102,105,110,100,101,114,46,102,105, + 110,100,95,115,112,101,99,40,41,46,10,10,32,32,32,32, + 78,122,44,78,111,116,32,105,109,112,111,114,116,105,110,103, + 32,100,105,114,101,99,116,111,114,121,32,123,125,58,32,109, + 105,115,115,105,110,103,32,95,95,105,110,105,116,95,95,114, + 62,0,0,0,41,6,218,11,102,105,110,100,95,108,111,97, + 100,101,114,114,33,0,0,0,114,63,0,0,0,114,64,0, + 0,0,114,50,0,0,0,218,13,73,109,112,111,114,116,87, + 97,114,110,105,110,103,41,5,114,104,0,0,0,218,8,102, + 117,108,108,110,97,109,101,218,6,108,111,97,100,101,114,218, + 8,112,111,114,116,105,111,110,115,218,3,109,115,103,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,17,95, + 102,105,110,100,95,109,111,100,117,108,101,95,115,104,105,109, + 154,1,0,0,115,10,0,0,0,0,10,14,1,16,1,4, + 1,22,1,114,127,0,0,0,99,4,0,0,0,0,0,0, + 0,11,0,0,0,22,0,0,0,67,0,0,0,115,142,1, + 0,0,105,0,125,4,124,2,100,1,107,9,114,22,124,2, + 124,4,100,2,60,0,110,4,100,3,125,2,124,3,100,1, + 107,9,114,42,124,3,124,4,100,4,60,0,124,0,100,1, + 100,5,133,2,25,0,125,5,124,0,100,5,100,6,133,2, + 25,0,125,6,124,0,100,6,100,7,133,2,25,0,125,7, + 124,5,116,0,107,3,114,126,100,8,106,1,124,2,124,5, + 131,2,125,8,116,2,106,3,100,9,124,8,131,2,1,0, + 116,4,124,8,102,1,124,4,151,1,142,1,130,1,110,86, + 116,5,124,6,131,1,100,5,107,3,114,170,100,10,106,1, + 124,2,131,1,125,8,116,2,106,3,100,9,124,8,131,2, + 1,0,116,6,124,8,131,1,130,1,110,42,116,5,124,7, + 131,1,100,5,107,3,114,212,100,11,106,1,124,2,131,1, + 125,8,116,2,106,3,100,9,124,8,131,2,1,0,116,6, + 124,8,131,1,130,1,124,1,100,1,107,9,144,1,114,130, + 121,16,116,7,124,1,100,12,25,0,131,1,125,9,87,0, + 110,22,4,0,116,8,107,10,144,1,114,4,1,0,1,0, + 1,0,89,0,110,52,88,0,116,9,124,6,131,1,124,9, + 107,3,144,1,114,56,100,13,106,1,124,2,131,1,125,8, + 116,2,106,3,100,9,124,8,131,2,1,0,116,4,124,8, + 102,1,124,4,151,1,142,1,130,1,121,16,124,1,100,14, 25,0,100,15,64,0,125,10,87,0,110,22,4,0,116,8, - 107,10,144,1,114,84,1,0,1,0,1,0,89,0,110,32, - 88,0,116,9,124,7,131,1,124,10,107,3,144,1,114,116, - 116,4,100,13,106,1,124,2,131,1,124,4,141,1,130,1, - 124,0,100,7,100,1,133,2,25,0,83,0,41,16,97,122, - 1,0,0,86,97,108,105,100,97,116,101,32,116,104,101,32, - 104,101,97,100,101,114,32,111,102,32,116,104,101,32,112,97, - 115,115,101,100,45,105,110,32,98,121,116,101,99,111,100,101, - 32,97,103,97,105,110,115,116,32,115,111,117,114,99,101,95, - 115,116,97,116,115,32,40,105,102,10,32,32,32,32,103,105, - 118,101,110,41,32,97,110,100,32,114,101,116,117,114,110,105, - 110,103,32,116,104,101,32,98,121,116,101,99,111,100,101,32, - 116,104,97,116,32,99,97,110,32,98,101,32,99,111,109,112, - 105,108,101,100,32,98,121,32,99,111,109,112,105,108,101,40, - 41,46,10,10,32,32,32,32,65,108,108,32,111,116,104,101, - 114,32,97,114,103,117,109,101,110,116,115,32,97,114,101,32, - 117,115,101,100,32,116,111,32,101,110,104,97,110,99,101,32, - 101,114,114,111,114,32,114,101,112,111,114,116,105,110,103,46, - 10,10,32,32,32,32,73,109,112,111,114,116,69,114,114,111, - 114,32,105,115,32,114,97,105,115,101,100,32,119,104,101,110, - 32,116,104,101,32,109,97,103,105,99,32,110,117,109,98,101, - 114,32,105,115,32,105,110,99,111,114,114,101,99,116,32,111, - 114,32,116,104,101,32,98,121,116,101,99,111,100,101,32,105, - 115,10,32,32,32,32,102,111,117,110,100,32,116,111,32,98, - 101,32,115,116,97,108,101,46,32,69,79,70,69,114,114,111, - 114,32,105,115,32,114,97,105,115,101,100,32,119,104,101,110, - 32,116,104,101,32,100,97,116,97,32,105,115,32,102,111,117, - 110,100,32,116,111,32,98,101,10,32,32,32,32,116,114,117, - 110,99,97,116,101,100,46,10,10,32,32,32,32,78,114,102, - 0,0,0,122,10,60,98,121,116,101,99,111,100,101,62,114, - 37,0,0,0,114,14,0,0,0,233,8,0,0,0,233,12, - 0,0,0,122,30,98,97,100,32,109,97,103,105,99,32,110, - 117,109,98,101,114,32,105,110,32,123,33,114,125,58,32,123, - 33,114,125,122,2,123,125,122,43,114,101,97,99,104,101,100, - 32,69,79,70,32,119,104,105,108,101,32,114,101,97,100,105, - 110,103,32,116,105,109,101,115,116,97,109,112,32,105,110,32, - 123,33,114,125,122,48,114,101,97,99,104,101,100,32,69,79, - 70,32,119,104,105,108,101,32,114,101,97,100,105,110,103,32, - 115,105,122,101,32,111,102,32,115,111,117,114,99,101,32,105, - 110,32,123,33,114,125,218,5,109,116,105,109,101,122,26,98, - 121,116,101,99,111,100,101,32,105,115,32,115,116,97,108,101, - 32,102,111,114,32,123,33,114,125,218,4,115,105,122,101,108, - 3,0,0,0,255,127,255,127,3,0,41,10,218,12,77,65, - 71,73,67,95,78,85,77,66,69,82,114,50,0,0,0,114, - 118,0,0,0,218,16,95,118,101,114,98,111,115,101,95,109, - 101,115,115,97,103,101,114,103,0,0,0,114,33,0,0,0, - 218,8,69,79,70,69,114,114,111,114,114,16,0,0,0,218, - 8,75,101,121,69,114,114,111,114,114,21,0,0,0,41,11, - 114,56,0,0,0,218,12,115,111,117,114,99,101,95,115,116, - 97,116,115,114,102,0,0,0,114,37,0,0,0,90,11,101, - 120,99,95,100,101,116,97,105,108,115,90,5,109,97,103,105, - 99,90,13,114,97,119,95,116,105,109,101,115,116,97,109,112, - 90,8,114,97,119,95,115,105,122,101,114,79,0,0,0,218, - 12,115,111,117,114,99,101,95,109,116,105,109,101,218,11,115, - 111,117,114,99,101,95,115,105,122,101,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,218,25,95,118,97,108,105, - 100,97,116,101,95,98,121,116,101,99,111,100,101,95,104,101, - 97,100,101,114,170,1,0,0,115,76,0,0,0,0,11,4, - 1,8,1,10,3,4,1,8,1,8,1,12,1,12,1,12, - 1,8,1,12,1,12,1,12,1,12,1,10,1,12,1,10, - 1,12,1,10,1,12,1,8,1,10,1,2,1,16,1,14, - 1,6,2,14,1,10,1,12,1,10,1,2,1,16,1,16, - 1,6,2,14,1,10,1,6,1,114,139,0,0,0,99,4, - 0,0,0,0,0,0,0,5,0,0,0,6,0,0,0,67, - 0,0,0,115,86,0,0,0,116,0,106,1,124,0,131,1, - 125,4,116,2,124,4,116,3,131,2,114,58,116,4,106,5, - 100,1,124,2,131,2,1,0,124,3,100,2,107,9,114,52, - 116,6,106,7,124,4,124,3,131,2,1,0,124,4,83,0, - 110,24,116,8,100,3,106,9,124,2,131,1,100,4,124,1, - 100,5,124,2,144,2,131,1,130,1,100,2,83,0,41,6, + 107,10,144,1,114,94,1,0,1,0,1,0,89,0,110,36, + 88,0,116,9,124,7,131,1,124,10,107,3,144,1,114,130, + 116,4,100,13,106,1,124,2,131,1,102,1,124,4,151,1, + 142,1,130,1,124,0,100,7,100,1,133,2,25,0,83,0, + 41,16,97,122,1,0,0,86,97,108,105,100,97,116,101,32, + 116,104,101,32,104,101,97,100,101,114,32,111,102,32,116,104, + 101,32,112,97,115,115,101,100,45,105,110,32,98,121,116,101, + 99,111,100,101,32,97,103,97,105,110,115,116,32,115,111,117, + 114,99,101,95,115,116,97,116,115,32,40,105,102,10,32,32, + 32,32,103,105,118,101,110,41,32,97,110,100,32,114,101,116, + 117,114,110,105,110,103,32,116,104,101,32,98,121,116,101,99, + 111,100,101,32,116,104,97,116,32,99,97,110,32,98,101,32, + 99,111,109,112,105,108,101,100,32,98,121,32,99,111,109,112, + 105,108,101,40,41,46,10,10,32,32,32,32,65,108,108,32, + 111,116,104,101,114,32,97,114,103,117,109,101,110,116,115,32, + 97,114,101,32,117,115,101,100,32,116,111,32,101,110,104,97, + 110,99,101,32,101,114,114,111,114,32,114,101,112,111,114,116, + 105,110,103,46,10,10,32,32,32,32,73,109,112,111,114,116, + 69,114,114,111,114,32,105,115,32,114,97,105,115,101,100,32, + 119,104,101,110,32,116,104,101,32,109,97,103,105,99,32,110, + 117,109,98,101,114,32,105,115,32,105,110,99,111,114,114,101, + 99,116,32,111,114,32,116,104,101,32,98,121,116,101,99,111, + 100,101,32,105,115,10,32,32,32,32,102,111,117,110,100,32, + 116,111,32,98,101,32,115,116,97,108,101,46,32,69,79,70, + 69,114,114,111,114,32,105,115,32,114,97,105,115,101,100,32, + 119,104,101,110,32,116,104,101,32,100,97,116,97,32,105,115, + 32,102,111,117,110,100,32,116,111,32,98,101,10,32,32,32, + 32,116,114,117,110,99,97,116,101,100,46,10,10,32,32,32, + 32,78,114,102,0,0,0,122,10,60,98,121,116,101,99,111, + 100,101,62,114,37,0,0,0,114,14,0,0,0,233,8,0, + 0,0,233,12,0,0,0,122,30,98,97,100,32,109,97,103, + 105,99,32,110,117,109,98,101,114,32,105,110,32,123,33,114, + 125,58,32,123,33,114,125,122,2,123,125,122,43,114,101,97, + 99,104,101,100,32,69,79,70,32,119,104,105,108,101,32,114, + 101,97,100,105,110,103,32,116,105,109,101,115,116,97,109,112, + 32,105,110,32,123,33,114,125,122,48,114,101,97,99,104,101, + 100,32,69,79,70,32,119,104,105,108,101,32,114,101,97,100, + 105,110,103,32,115,105,122,101,32,111,102,32,115,111,117,114, + 99,101,32,105,110,32,123,33,114,125,218,5,109,116,105,109, + 101,122,26,98,121,116,101,99,111,100,101,32,105,115,32,115, + 116,97,108,101,32,102,111,114,32,123,33,114,125,218,4,115, + 105,122,101,108,3,0,0,0,255,127,255,127,3,0,41,10, + 218,12,77,65,71,73,67,95,78,85,77,66,69,82,114,50, + 0,0,0,114,118,0,0,0,218,16,95,118,101,114,98,111, + 115,101,95,109,101,115,115,97,103,101,114,103,0,0,0,114, + 33,0,0,0,218,8,69,79,70,69,114,114,111,114,114,16, + 0,0,0,218,8,75,101,121,69,114,114,111,114,114,21,0, + 0,0,41,11,114,56,0,0,0,218,12,115,111,117,114,99, + 101,95,115,116,97,116,115,114,102,0,0,0,114,37,0,0, + 0,90,11,101,120,99,95,100,101,116,97,105,108,115,90,5, + 109,97,103,105,99,90,13,114,97,119,95,116,105,109,101,115, + 116,97,109,112,90,8,114,97,119,95,115,105,122,101,114,79, + 0,0,0,218,12,115,111,117,114,99,101,95,109,116,105,109, + 101,218,11,115,111,117,114,99,101,95,115,105,122,101,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,25,95, + 118,97,108,105,100,97,116,101,95,98,121,116,101,99,111,100, + 101,95,104,101,97,100,101,114,171,1,0,0,115,76,0,0, + 0,0,11,4,1,8,1,10,3,4,1,8,1,8,1,12, + 1,12,1,12,1,8,1,12,1,12,1,16,1,12,1,10, + 1,12,1,10,1,12,1,10,1,12,1,8,1,10,1,2, + 1,16,1,16,1,6,2,14,1,10,1,12,1,14,1,2, + 1,16,1,16,1,6,2,14,1,12,1,8,1,114,139,0, + 0,0,99,4,0,0,0,0,0,0,0,5,0,0,0,5, + 0,0,0,67,0,0,0,115,82,0,0,0,116,0,106,1, + 124,0,131,1,125,4,116,2,124,4,116,3,131,2,114,58, + 116,4,106,5,100,1,124,2,131,2,1,0,124,3,100,2, + 107,9,114,52,116,6,106,7,124,4,124,3,131,2,1,0, + 124,4,83,0,110,20,116,8,100,3,106,9,124,2,131,1, + 124,1,124,2,100,4,141,3,130,1,100,2,83,0,41,5, 122,60,67,111,109,112,105,108,101,32,98,121,116,101,99,111, 100,101,32,97,115,32,114,101,116,117,114,110,101,100,32,98, 121,32,95,118,97,108,105,100,97,116,101,95,98,121,116,101, 99,111,100,101,95,104,101,97,100,101,114,40,41,46,122,21, 99,111,100,101,32,111,98,106,101,99,116,32,102,114,111,109, 32,123,33,114,125,78,122,23,78,111,110,45,99,111,100,101, - 32,111,98,106,101,99,116,32,105,110,32,123,33,114,125,114, - 102,0,0,0,114,37,0,0,0,41,10,218,7,109,97,114, - 115,104,97,108,90,5,108,111,97,100,115,218,10,105,115,105, - 110,115,116,97,110,99,101,218,10,95,99,111,100,101,95,116, - 121,112,101,114,118,0,0,0,114,133,0,0,0,218,4,95, - 105,109,112,90,16,95,102,105,120,95,99,111,95,102,105,108, - 101,110,97,109,101,114,103,0,0,0,114,50,0,0,0,41, - 5,114,56,0,0,0,114,102,0,0,0,114,93,0,0,0, - 114,94,0,0,0,218,4,99,111,100,101,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,17,95,99,111,109, - 112,105,108,101,95,98,121,116,101,99,111,100,101,225,1,0, - 0,115,16,0,0,0,0,2,10,1,10,1,12,1,8,1, - 12,1,6,2,12,1,114,145,0,0,0,114,62,0,0,0, - 99,3,0,0,0,0,0,0,0,4,0,0,0,3,0,0, - 0,67,0,0,0,115,56,0,0,0,116,0,116,1,131,1, - 125,3,124,3,106,2,116,3,124,1,131,1,131,1,1,0, - 124,3,106,2,116,3,124,2,131,1,131,1,1,0,124,3, - 106,2,116,4,106,5,124,0,131,1,131,1,1,0,124,3, - 83,0,41,1,122,80,67,111,109,112,105,108,101,32,97,32, - 99,111,100,101,32,111,98,106,101,99,116,32,105,110,116,111, - 32,98,121,116,101,99,111,100,101,32,102,111,114,32,119,114, - 105,116,105,110,103,32,111,117,116,32,116,111,32,97,32,98, - 121,116,101,45,99,111,109,112,105,108,101,100,10,32,32,32, - 32,102,105,108,101,46,41,6,218,9,98,121,116,101,97,114, - 114,97,121,114,132,0,0,0,218,6,101,120,116,101,110,100, - 114,19,0,0,0,114,140,0,0,0,90,5,100,117,109,112, - 115,41,4,114,144,0,0,0,114,130,0,0,0,114,138,0, - 0,0,114,56,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,218,17,95,99,111,100,101,95,116,111, - 95,98,121,116,101,99,111,100,101,237,1,0,0,115,10,0, - 0,0,0,3,8,1,14,1,14,1,16,1,114,148,0,0, - 0,99,1,0,0,0,0,0,0,0,5,0,0,0,4,0, - 0,0,67,0,0,0,115,62,0,0,0,100,1,100,2,108, - 0,125,1,116,1,106,2,124,0,131,1,106,3,125,2,124, - 1,106,4,124,2,131,1,125,3,116,1,106,5,100,2,100, - 3,131,2,125,4,124,4,106,6,124,0,106,6,124,3,100, - 1,25,0,131,1,131,1,83,0,41,4,122,121,68,101,99, - 111,100,101,32,98,121,116,101,115,32,114,101,112,114,101,115, - 101,110,116,105,110,103,32,115,111,117,114,99,101,32,99,111, - 100,101,32,97,110,100,32,114,101,116,117,114,110,32,116,104, - 101,32,115,116,114,105,110,103,46,10,10,32,32,32,32,85, - 110,105,118,101,114,115,97,108,32,110,101,119,108,105,110,101, - 32,115,117,112,112,111,114,116,32,105,115,32,117,115,101,100, - 32,105,110,32,116,104,101,32,100,101,99,111,100,105,110,103, - 46,10,32,32,32,32,114,62,0,0,0,78,84,41,7,218, - 8,116,111,107,101,110,105,122,101,114,52,0,0,0,90,7, - 66,121,116,101,115,73,79,90,8,114,101,97,100,108,105,110, - 101,90,15,100,101,116,101,99,116,95,101,110,99,111,100,105, - 110,103,90,25,73,110,99,114,101,109,101,110,116,97,108,78, - 101,119,108,105,110,101,68,101,99,111,100,101,114,218,6,100, - 101,99,111,100,101,41,5,218,12,115,111,117,114,99,101,95, - 98,121,116,101,115,114,149,0,0,0,90,21,115,111,117,114, - 99,101,95,98,121,116,101,115,95,114,101,97,100,108,105,110, - 101,218,8,101,110,99,111,100,105,110,103,90,15,110,101,119, - 108,105,110,101,95,100,101,99,111,100,101,114,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,13,100,101,99, - 111,100,101,95,115,111,117,114,99,101,247,1,0,0,115,10, - 0,0,0,0,5,8,1,12,1,10,1,12,1,114,153,0, - 0,0,41,2,114,124,0,0,0,218,26,115,117,98,109,111, - 100,117,108,101,95,115,101,97,114,99,104,95,108,111,99,97, - 116,105,111,110,115,99,2,0,0,0,2,0,0,0,9,0, - 0,0,19,0,0,0,67,0,0,0,115,20,1,0,0,124, - 1,100,1,107,8,114,60,100,2,125,1,116,0,124,2,100, - 3,131,2,114,70,121,14,124,2,106,1,124,0,131,1,125, - 1,87,0,113,70,4,0,116,2,107,10,114,56,1,0,1, - 0,1,0,89,0,113,70,88,0,110,10,116,3,106,4,124, - 1,131,1,125,1,116,5,106,6,124,0,124,2,100,4,124, - 1,144,1,131,2,125,4,100,5,124,4,95,7,124,2,100, - 1,107,8,114,158,120,54,116,8,131,0,68,0,93,40,92, + 32,111,98,106,101,99,116,32,105,110,32,123,33,114,125,41, + 2,114,102,0,0,0,114,37,0,0,0,41,10,218,7,109, + 97,114,115,104,97,108,90,5,108,111,97,100,115,218,10,105, + 115,105,110,115,116,97,110,99,101,218,10,95,99,111,100,101, + 95,116,121,112,101,114,118,0,0,0,114,133,0,0,0,218, + 4,95,105,109,112,90,16,95,102,105,120,95,99,111,95,102, + 105,108,101,110,97,109,101,114,103,0,0,0,114,50,0,0, + 0,41,5,114,56,0,0,0,114,102,0,0,0,114,93,0, + 0,0,114,94,0,0,0,218,4,99,111,100,101,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,218,17,95,99, + 111,109,112,105,108,101,95,98,121,116,101,99,111,100,101,226, + 1,0,0,115,16,0,0,0,0,2,10,1,10,1,12,1, + 8,1,12,1,6,2,10,1,114,145,0,0,0,114,62,0, + 0,0,99,3,0,0,0,0,0,0,0,4,0,0,0,3, + 0,0,0,67,0,0,0,115,56,0,0,0,116,0,116,1, + 131,1,125,3,124,3,106,2,116,3,124,1,131,1,131,1, + 1,0,124,3,106,2,116,3,124,2,131,1,131,1,1,0, + 124,3,106,2,116,4,106,5,124,0,131,1,131,1,1,0, + 124,3,83,0,41,1,122,80,67,111,109,112,105,108,101,32, + 97,32,99,111,100,101,32,111,98,106,101,99,116,32,105,110, + 116,111,32,98,121,116,101,99,111,100,101,32,102,111,114,32, + 119,114,105,116,105,110,103,32,111,117,116,32,116,111,32,97, + 32,98,121,116,101,45,99,111,109,112,105,108,101,100,10,32, + 32,32,32,102,105,108,101,46,41,6,218,9,98,121,116,101, + 97,114,114,97,121,114,132,0,0,0,218,6,101,120,116,101, + 110,100,114,19,0,0,0,114,140,0,0,0,90,5,100,117, + 109,112,115,41,4,114,144,0,0,0,114,130,0,0,0,114, + 138,0,0,0,114,56,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,218,17,95,99,111,100,101,95, + 116,111,95,98,121,116,101,99,111,100,101,238,1,0,0,115, + 10,0,0,0,0,3,8,1,14,1,14,1,16,1,114,148, + 0,0,0,99,1,0,0,0,0,0,0,0,5,0,0,0, + 4,0,0,0,67,0,0,0,115,62,0,0,0,100,1,100, + 2,108,0,125,1,116,1,106,2,124,0,131,1,106,3,125, + 2,124,1,106,4,124,2,131,1,125,3,116,1,106,5,100, + 2,100,3,131,2,125,4,124,4,106,6,124,0,106,6,124, + 3,100,1,25,0,131,1,131,1,83,0,41,4,122,121,68, + 101,99,111,100,101,32,98,121,116,101,115,32,114,101,112,114, + 101,115,101,110,116,105,110,103,32,115,111,117,114,99,101,32, + 99,111,100,101,32,97,110,100,32,114,101,116,117,114,110,32, + 116,104,101,32,115,116,114,105,110,103,46,10,10,32,32,32, + 32,85,110,105,118,101,114,115,97,108,32,110,101,119,108,105, + 110,101,32,115,117,112,112,111,114,116,32,105,115,32,117,115, + 101,100,32,105,110,32,116,104,101,32,100,101,99,111,100,105, + 110,103,46,10,32,32,32,32,114,62,0,0,0,78,84,41, + 7,218,8,116,111,107,101,110,105,122,101,114,52,0,0,0, + 90,7,66,121,116,101,115,73,79,90,8,114,101,97,100,108, + 105,110,101,90,15,100,101,116,101,99,116,95,101,110,99,111, + 100,105,110,103,90,25,73,110,99,114,101,109,101,110,116,97, + 108,78,101,119,108,105,110,101,68,101,99,111,100,101,114,218, + 6,100,101,99,111,100,101,41,5,218,12,115,111,117,114,99, + 101,95,98,121,116,101,115,114,149,0,0,0,90,21,115,111, + 117,114,99,101,95,98,121,116,101,115,95,114,101,97,100,108, + 105,110,101,218,8,101,110,99,111,100,105,110,103,90,15,110, + 101,119,108,105,110,101,95,100,101,99,111,100,101,114,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,13,100, + 101,99,111,100,101,95,115,111,117,114,99,101,248,1,0,0, + 115,10,0,0,0,0,5,8,1,12,1,10,1,12,1,114, + 153,0,0,0,41,2,114,124,0,0,0,218,26,115,117,98, + 109,111,100,117,108,101,95,115,101,97,114,99,104,95,108,111, + 99,97,116,105,111,110,115,99,2,0,0,0,2,0,0,0, + 9,0,0,0,19,0,0,0,67,0,0,0,115,18,1,0, + 0,124,1,100,1,107,8,114,60,100,2,125,1,116,0,124, + 2,100,3,131,2,114,70,121,14,124,2,106,1,124,0,131, + 1,125,1,87,0,113,70,4,0,116,2,107,10,114,56,1, + 0,1,0,1,0,89,0,113,70,88,0,110,10,116,3,106, + 4,124,1,131,1,125,1,116,5,106,6,124,0,124,2,124, + 1,100,4,141,3,125,4,100,5,124,4,95,7,124,2,100, + 1,107,8,114,156,120,54,116,8,131,0,68,0,93,40,92, 2,125,5,125,6,124,1,106,9,116,10,124,6,131,1,131, - 1,114,110,124,5,124,0,124,1,131,2,125,2,124,2,124, - 4,95,11,80,0,113,110,87,0,100,1,83,0,124,3,116, - 12,107,8,114,224,116,0,124,2,100,6,131,2,114,230,121, + 1,114,108,124,5,124,0,124,1,131,2,125,2,124,2,124, + 4,95,11,80,0,113,108,87,0,100,1,83,0,124,3,116, + 12,107,8,114,222,116,0,124,2,100,6,131,2,114,228,121, 14,124,2,106,13,124,0,131,1,125,7,87,0,110,20,4, - 0,116,2,107,10,114,210,1,0,1,0,1,0,89,0,113, - 230,88,0,124,7,114,230,103,0,124,4,95,14,110,6,124, + 0,116,2,107,10,114,208,1,0,1,0,1,0,89,0,113, + 228,88,0,124,7,114,228,103,0,124,4,95,14,110,6,124, 3,124,4,95,14,124,4,106,14,103,0,107,2,144,1,114, - 16,124,1,144,1,114,16,116,15,124,1,131,1,100,7,25, + 14,124,1,144,1,114,14,116,15,124,1,131,1,100,7,25, 0,125,8,124,4,106,14,106,16,124,8,131,1,1,0,124, 4,83,0,41,8,97,61,1,0,0,82,101,116,117,114,110, 32,97,32,109,111,100,117,108,101,32,115,112,101,99,32,98, @@ -777,75 +778,75 @@ 121,32,95,95,105,110,105,116,95,95,40,41,32,97,114,103, 46,10,10,32,32,32,32,78,122,9,60,117,110,107,110,111, 119,110,62,218,12,103,101,116,95,102,105,108,101,110,97,109, - 101,218,6,111,114,105,103,105,110,84,218,10,105,115,95,112, - 97,99,107,97,103,101,114,62,0,0,0,41,17,114,112,0, - 0,0,114,155,0,0,0,114,103,0,0,0,114,3,0,0, - 0,114,67,0,0,0,114,118,0,0,0,218,10,77,111,100, - 117,108,101,83,112,101,99,90,13,95,115,101,116,95,102,105, - 108,101,97,116,116,114,218,27,95,103,101,116,95,115,117,112, - 112,111,114,116,101,100,95,102,105,108,101,95,108,111,97,100, - 101,114,115,114,96,0,0,0,114,97,0,0,0,114,124,0, - 0,0,218,9,95,80,79,80,85,76,65,84,69,114,157,0, - 0,0,114,154,0,0,0,114,40,0,0,0,218,6,97,112, - 112,101,110,100,41,9,114,102,0,0,0,90,8,108,111,99, - 97,116,105,111,110,114,124,0,0,0,114,154,0,0,0,218, - 4,115,112,101,99,218,12,108,111,97,100,101,114,95,99,108, - 97,115,115,218,8,115,117,102,102,105,120,101,115,114,157,0, - 0,0,90,7,100,105,114,110,97,109,101,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,23,115,112,101,99, - 95,102,114,111,109,95,102,105,108,101,95,108,111,99,97,116, - 105,111,110,8,2,0,0,115,62,0,0,0,0,12,8,4, - 4,1,10,2,2,1,14,1,14,1,8,2,10,8,18,1, - 6,3,8,1,16,1,14,1,10,1,6,1,6,2,4,3, - 8,2,10,1,2,1,14,1,14,1,6,2,4,1,8,2, - 6,1,12,1,6,1,12,1,12,2,114,165,0,0,0,99, - 0,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0, - 64,0,0,0,115,80,0,0,0,101,0,90,1,100,0,90, - 2,100,1,90,3,100,2,90,4,100,3,90,5,100,4,90, - 6,101,7,100,5,100,6,132,0,131,1,90,8,101,7,100, - 7,100,8,132,0,131,1,90,9,101,7,100,14,100,10,100, - 11,132,1,131,1,90,10,101,7,100,15,100,12,100,13,132, - 1,131,1,90,11,100,9,83,0,41,16,218,21,87,105,110, - 100,111,119,115,82,101,103,105,115,116,114,121,70,105,110,100, - 101,114,122,62,77,101,116,97,32,112,97,116,104,32,102,105, - 110,100,101,114,32,102,111,114,32,109,111,100,117,108,101,115, - 32,100,101,99,108,97,114,101,100,32,105,110,32,116,104,101, - 32,87,105,110,100,111,119,115,32,114,101,103,105,115,116,114, - 121,46,122,59,83,111,102,116,119,97,114,101,92,80,121,116, - 104,111,110,92,80,121,116,104,111,110,67,111,114,101,92,123, - 115,121,115,95,118,101,114,115,105,111,110,125,92,77,111,100, - 117,108,101,115,92,123,102,117,108,108,110,97,109,101,125,122, - 65,83,111,102,116,119,97,114,101,92,80,121,116,104,111,110, - 92,80,121,116,104,111,110,67,111,114,101,92,123,115,121,115, - 95,118,101,114,115,105,111,110,125,92,77,111,100,117,108,101, - 115,92,123,102,117,108,108,110,97,109,101,125,92,68,101,98, - 117,103,70,99,2,0,0,0,0,0,0,0,2,0,0,0, - 11,0,0,0,67,0,0,0,115,50,0,0,0,121,14,116, - 0,106,1,116,0,106,2,124,1,131,2,83,0,4,0,116, - 3,107,10,114,44,1,0,1,0,1,0,116,0,106,1,116, - 0,106,4,124,1,131,2,83,0,88,0,100,0,83,0,41, - 1,78,41,5,218,7,95,119,105,110,114,101,103,90,7,79, - 112,101,110,75,101,121,90,17,72,75,69,89,95,67,85,82, - 82,69,78,84,95,85,83,69,82,114,42,0,0,0,90,18, - 72,75,69,89,95,76,79,67,65,76,95,77,65,67,72,73, - 78,69,41,2,218,3,99,108,115,114,5,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,14,95, - 111,112,101,110,95,114,101,103,105,115,116,114,121,88,2,0, - 0,115,8,0,0,0,0,2,2,1,14,1,14,1,122,36, - 87,105,110,100,111,119,115,82,101,103,105,115,116,114,121,70, - 105,110,100,101,114,46,95,111,112,101,110,95,114,101,103,105, - 115,116,114,121,99,2,0,0,0,0,0,0,0,6,0,0, - 0,16,0,0,0,67,0,0,0,115,116,0,0,0,124,0, - 106,0,114,14,124,0,106,1,125,2,110,6,124,0,106,2, - 125,2,124,2,106,3,100,1,124,1,100,2,100,3,116,4, - 106,5,100,0,100,4,133,2,25,0,22,0,144,2,131,0, - 125,3,121,38,124,0,106,6,124,3,131,1,143,18,125,4, - 116,7,106,8,124,4,100,5,131,2,125,5,87,0,100,0, - 81,0,82,0,88,0,87,0,110,20,4,0,116,9,107,10, - 114,110,1,0,1,0,1,0,100,0,83,0,88,0,124,5, - 83,0,41,6,78,114,123,0,0,0,90,11,115,121,115,95, - 118,101,114,115,105,111,110,122,5,37,100,46,37,100,114,59, - 0,0,0,114,32,0,0,0,41,10,218,11,68,69,66,85, + 101,41,1,218,6,111,114,105,103,105,110,84,218,10,105,115, + 95,112,97,99,107,97,103,101,114,62,0,0,0,41,17,114, + 112,0,0,0,114,155,0,0,0,114,103,0,0,0,114,3, + 0,0,0,114,67,0,0,0,114,118,0,0,0,218,10,77, + 111,100,117,108,101,83,112,101,99,90,13,95,115,101,116,95, + 102,105,108,101,97,116,116,114,218,27,95,103,101,116,95,115, + 117,112,112,111,114,116,101,100,95,102,105,108,101,95,108,111, + 97,100,101,114,115,114,96,0,0,0,114,97,0,0,0,114, + 124,0,0,0,218,9,95,80,79,80,85,76,65,84,69,114, + 157,0,0,0,114,154,0,0,0,114,40,0,0,0,218,6, + 97,112,112,101,110,100,41,9,114,102,0,0,0,90,8,108, + 111,99,97,116,105,111,110,114,124,0,0,0,114,154,0,0, + 0,218,4,115,112,101,99,218,12,108,111,97,100,101,114,95, + 99,108,97,115,115,218,8,115,117,102,102,105,120,101,115,114, + 157,0,0,0,90,7,100,105,114,110,97,109,101,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,218,23,115,112, + 101,99,95,102,114,111,109,95,102,105,108,101,95,108,111,99, + 97,116,105,111,110,9,2,0,0,115,62,0,0,0,0,12, + 8,4,4,1,10,2,2,1,14,1,14,1,8,2,10,8, + 16,1,6,3,8,1,16,1,14,1,10,1,6,1,6,2, + 4,3,8,2,10,1,2,1,14,1,14,1,6,2,4,1, + 8,2,6,1,12,1,6,1,12,1,12,2,114,165,0,0, + 0,99,0,0,0,0,0,0,0,0,0,0,0,0,4,0, + 0,0,64,0,0,0,115,80,0,0,0,101,0,90,1,100, + 0,90,2,100,1,90,3,100,2,90,4,100,3,90,5,100, + 4,90,6,101,7,100,5,100,6,132,0,131,1,90,8,101, + 7,100,7,100,8,132,0,131,1,90,9,101,7,100,14,100, + 10,100,11,132,1,131,1,90,10,101,7,100,15,100,12,100, + 13,132,1,131,1,90,11,100,9,83,0,41,16,218,21,87, + 105,110,100,111,119,115,82,101,103,105,115,116,114,121,70,105, + 110,100,101,114,122,62,77,101,116,97,32,112,97,116,104,32, + 102,105,110,100,101,114,32,102,111,114,32,109,111,100,117,108, + 101,115,32,100,101,99,108,97,114,101,100,32,105,110,32,116, + 104,101,32,87,105,110,100,111,119,115,32,114,101,103,105,115, + 116,114,121,46,122,59,83,111,102,116,119,97,114,101,92,80, + 121,116,104,111,110,92,80,121,116,104,111,110,67,111,114,101, + 92,123,115,121,115,95,118,101,114,115,105,111,110,125,92,77, + 111,100,117,108,101,115,92,123,102,117,108,108,110,97,109,101, + 125,122,65,83,111,102,116,119,97,114,101,92,80,121,116,104, + 111,110,92,80,121,116,104,111,110,67,111,114,101,92,123,115, + 121,115,95,118,101,114,115,105,111,110,125,92,77,111,100,117, + 108,101,115,92,123,102,117,108,108,110,97,109,101,125,92,68, + 101,98,117,103,70,99,2,0,0,0,0,0,0,0,2,0, + 0,0,11,0,0,0,67,0,0,0,115,50,0,0,0,121, + 14,116,0,106,1,116,0,106,2,124,1,131,2,83,0,4, + 0,116,3,107,10,114,44,1,0,1,0,1,0,116,0,106, + 1,116,0,106,4,124,1,131,2,83,0,88,0,100,0,83, + 0,41,1,78,41,5,218,7,95,119,105,110,114,101,103,90, + 7,79,112,101,110,75,101,121,90,17,72,75,69,89,95,67, + 85,82,82,69,78,84,95,85,83,69,82,114,42,0,0,0, + 90,18,72,75,69,89,95,76,79,67,65,76,95,77,65,67, + 72,73,78,69,41,2,218,3,99,108,115,114,5,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, + 14,95,111,112,101,110,95,114,101,103,105,115,116,114,121,89, + 2,0,0,115,8,0,0,0,0,2,2,1,14,1,14,1, + 122,36,87,105,110,100,111,119,115,82,101,103,105,115,116,114, + 121,70,105,110,100,101,114,46,95,111,112,101,110,95,114,101, + 103,105,115,116,114,121,99,2,0,0,0,0,0,0,0,6, + 0,0,0,16,0,0,0,67,0,0,0,115,112,0,0,0, + 124,0,106,0,114,14,124,0,106,1,125,2,110,6,124,0, + 106,2,125,2,124,2,106,3,124,1,100,1,116,4,106,5, + 100,0,100,2,133,2,25,0,22,0,100,3,141,2,125,3, + 121,38,124,0,106,6,124,3,131,1,143,18,125,4,116,7, + 106,8,124,4,100,4,131,2,125,5,87,0,100,0,81,0, + 82,0,88,0,87,0,110,20,4,0,116,9,107,10,114,106, + 1,0,1,0,1,0,100,0,83,0,88,0,124,5,83,0, + 41,5,78,122,5,37,100,46,37,100,114,59,0,0,0,41, + 2,114,123,0,0,0,90,11,115,121,115,95,118,101,114,115, + 105,111,110,114,32,0,0,0,41,10,218,11,68,69,66,85, 71,95,66,85,73,76,68,218,18,82,69,71,73,83,84,82, 89,95,75,69,89,95,68,69,66,85,71,218,12,82,69,71, 73,83,84,82,89,95,75,69,89,114,50,0,0,0,114,8, @@ -856,21 +857,21 @@ 114,121,95,107,101,121,114,5,0,0,0,90,4,104,107,101, 121,218,8,102,105,108,101,112,97,116,104,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,16,95,115,101,97, - 114,99,104,95,114,101,103,105,115,116,114,121,95,2,0,0, - 115,22,0,0,0,0,2,6,1,8,2,6,1,10,1,22, + 114,99,104,95,114,101,103,105,115,116,114,121,96,2,0,0, + 115,22,0,0,0,0,2,6,1,8,2,6,1,6,1,22, 1,2,1,12,1,26,1,14,1,6,1,122,38,87,105,110, 100,111,119,115,82,101,103,105,115,116,114,121,70,105,110,100, 101,114,46,95,115,101,97,114,99,104,95,114,101,103,105,115, 116,114,121,78,99,4,0,0,0,0,0,0,0,8,0,0, - 0,14,0,0,0,67,0,0,0,115,122,0,0,0,124,0, + 0,14,0,0,0,67,0,0,0,115,120,0,0,0,124,0, 106,0,124,1,131,1,125,4,124,4,100,0,107,8,114,22, 100,0,83,0,121,12,116,1,124,4,131,1,1,0,87,0, 110,20,4,0,116,2,107,10,114,54,1,0,1,0,1,0, - 100,0,83,0,88,0,120,60,116,3,131,0,68,0,93,50, + 100,0,83,0,88,0,120,58,116,3,131,0,68,0,93,48, 92,2,125,5,125,6,124,4,106,4,116,5,124,6,131,1, 131,1,114,64,116,6,106,7,124,1,124,5,124,1,124,4, - 131,2,100,1,124,4,144,1,131,2,125,7,124,7,83,0, - 113,64,87,0,100,0,83,0,41,2,78,114,156,0,0,0, + 131,2,124,4,100,1,141,3,125,7,124,7,83,0,113,64, + 87,0,100,0,83,0,41,2,78,41,1,114,156,0,0,0, 41,8,114,175,0,0,0,114,41,0,0,0,114,42,0,0, 0,114,159,0,0,0,114,96,0,0,0,114,97,0,0,0, 114,118,0,0,0,218,16,115,112,101,99,95,102,114,111,109, @@ -878,9 +879,9 @@ 0,0,0,114,37,0,0,0,218,6,116,97,114,103,101,116, 114,174,0,0,0,114,124,0,0,0,114,164,0,0,0,114, 162,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,9,102,105,110,100,95,115,112,101,99,110,2, + 0,0,0,218,9,102,105,110,100,95,115,112,101,99,111,2, 0,0,115,26,0,0,0,0,2,10,1,8,1,4,1,2, - 1,12,1,14,1,6,1,16,1,14,1,6,1,10,1,8, + 1,12,1,14,1,6,1,16,1,14,1,6,1,8,1,8, 1,122,31,87,105,110,100,111,119,115,82,101,103,105,115,116, 114,121,70,105,110,100,101,114,46,102,105,110,100,95,115,112, 101,99,99,3,0,0,0,0,0,0,0,4,0,0,0,3, @@ -897,7 +898,7 @@ 78,41,2,114,178,0,0,0,114,124,0,0,0,41,4,114, 168,0,0,0,114,123,0,0,0,114,37,0,0,0,114,162, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,102,105,110,100,95,109,111,100,117,108,101,126, + 0,0,218,11,102,105,110,100,95,109,111,100,117,108,101,127, 2,0,0,115,8,0,0,0,0,7,12,1,8,1,8,2, 122,33,87,105,110,100,111,119,115,82,101,103,105,115,116,114, 121,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, @@ -907,7 +908,7 @@ 11,99,108,97,115,115,109,101,116,104,111,100,114,169,0,0, 0,114,175,0,0,0,114,178,0,0,0,114,179,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,166,0,0,0,76,2,0,0,115,20,0, + 6,0,0,0,114,166,0,0,0,77,2,0,0,115,20,0, 0,0,8,2,4,3,4,3,4,2,4,2,12,7,12,15, 2,1,12,15,2,1,114,166,0,0,0,99,0,0,0,0, 0,0,0,0,0,0,0,0,2,0,0,0,64,0,0,0, @@ -942,7 +943,7 @@ 0,114,123,0,0,0,114,98,0,0,0,90,13,102,105,108, 101,110,97,109,101,95,98,97,115,101,90,9,116,97,105,108, 95,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,157,0,0,0,145,2,0,0,115,8,0, + 6,0,0,0,114,157,0,0,0,146,2,0,0,115,8,0, 0,0,0,3,18,1,16,1,14,1,122,24,95,76,111,97, 100,101,114,66,97,115,105,99,115,46,105,115,95,112,97,99, 107,97,103,101,99,2,0,0,0,0,0,0,0,2,0,0, @@ -953,7 +954,7 @@ 78,114,4,0,0,0,41,2,114,104,0,0,0,114,162,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, 0,218,13,99,114,101,97,116,101,95,109,111,100,117,108,101, - 153,2,0,0,115,0,0,0,0,122,27,95,76,111,97,100, + 154,2,0,0,115,0,0,0,0,122,27,95,76,111,97,100, 101,114,66,97,115,105,99,115,46,99,114,101,97,116,101,95, 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,3, 0,0,0,4,0,0,0,67,0,0,0,115,56,0,0,0, @@ -972,7 +973,7 @@ 100,218,4,101,120,101,99,114,115,0,0,0,41,3,114,104, 0,0,0,218,6,109,111,100,117,108,101,114,144,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 11,101,120,101,99,95,109,111,100,117,108,101,156,2,0,0, + 11,101,120,101,99,95,109,111,100,117,108,101,157,2,0,0, 115,10,0,0,0,0,2,12,1,8,1,6,1,10,1,122, 25,95,76,111,97,100,101,114,66,97,115,105,99,115,46,101, 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, @@ -984,13 +985,13 @@ 117,108,101,95,115,104,105,109,41,2,114,104,0,0,0,114, 123,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, 0,0,0,218,11,108,111,97,100,95,109,111,100,117,108,101, - 164,2,0,0,115,2,0,0,0,0,2,122,25,95,76,111, + 165,2,0,0,115,2,0,0,0,0,2,122,25,95,76,111, 97,100,101,114,66,97,115,105,99,115,46,108,111,97,100,95, 109,111,100,117,108,101,78,41,8,114,109,0,0,0,114,108, 0,0,0,114,110,0,0,0,114,111,0,0,0,114,157,0, 0,0,114,183,0,0,0,114,188,0,0,0,114,190,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,181,0,0,0,140,2,0,0,115,10, + 114,6,0,0,0,114,181,0,0,0,141,2,0,0,115,10, 0,0,0,8,3,4,2,8,8,8,3,8,8,114,181,0, 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,3, 0,0,0,64,0,0,0,115,74,0,0,0,101,0,90,1, @@ -1016,7 +1017,7 @@ 1,218,7,73,79,69,114,114,111,114,41,2,114,104,0,0, 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, 114,6,0,0,0,218,10,112,97,116,104,95,109,116,105,109, - 101,171,2,0,0,115,2,0,0,0,0,6,122,23,83,111, + 101,172,2,0,0,115,2,0,0,0,0,6,122,23,83,111, 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, 109,116,105,109,101,99,2,0,0,0,0,0,0,0,2,0, 0,0,3,0,0,0,67,0,0,0,115,14,0,0,0,100, @@ -1051,7 +1052,7 @@ 0,0,0,41,1,114,193,0,0,0,41,2,114,104,0,0, 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, 114,6,0,0,0,218,10,112,97,116,104,95,115,116,97,116, - 115,179,2,0,0,115,2,0,0,0,0,11,122,23,83,111, + 115,180,2,0,0,115,2,0,0,0,0,11,122,23,83,111, 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, 115,116,97,116,115,99,4,0,0,0,0,0,0,0,4,0, 0,0,3,0,0,0,67,0,0,0,115,12,0,0,0,124, @@ -1074,7 +1075,7 @@ 4,114,104,0,0,0,114,94,0,0,0,90,10,99,97,99, 104,101,95,112,97,116,104,114,56,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,15,95,99,97, - 99,104,101,95,98,121,116,101,99,111,100,101,192,2,0,0, + 99,104,101,95,98,121,116,101,99,111,100,101,193,2,0,0, 115,2,0,0,0,0,8,122,28,83,111,117,114,99,101,76, 111,97,100,101,114,46,95,99,97,99,104,101,95,98,121,116, 101,99,111,100,101,99,3,0,0,0,0,0,0,0,3,0, @@ -1091,1298 +1092,1298 @@ 108,101,115,46,10,32,32,32,32,32,32,32,32,78,114,4, 0,0,0,41,3,114,104,0,0,0,114,37,0,0,0,114, 56,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,195,0,0,0,202,2,0,0,115,0,0,0, + 0,0,0,114,195,0,0,0,203,2,0,0,115,0,0,0, 0,122,21,83,111,117,114,99,101,76,111,97,100,101,114,46, 115,101,116,95,100,97,116,97,99,2,0,0,0,0,0,0, - 0,5,0,0,0,16,0,0,0,67,0,0,0,115,84,0, + 0,5,0,0,0,16,0,0,0,67,0,0,0,115,82,0, 0,0,124,0,106,0,124,1,131,1,125,2,121,14,124,0, - 106,1,124,2,131,1,125,3,87,0,110,50,4,0,116,2, - 107,10,114,74,1,0,125,4,1,0,122,22,116,3,100,1, - 100,2,124,1,144,1,131,1,124,4,130,2,87,0,89,0, - 100,3,100,3,125,4,126,4,88,0,110,2,88,0,116,4, - 124,3,131,1,83,0,41,4,122,52,67,111,110,99,114,101, - 116,101,32,105,109,112,108,101,109,101,110,116,97,116,105,111, - 110,32,111,102,32,73,110,115,112,101,99,116,76,111,97,100, - 101,114,46,103,101,116,95,115,111,117,114,99,101,46,122,39, - 115,111,117,114,99,101,32,110,111,116,32,97,118,97,105,108, - 97,98,108,101,32,116,104,114,111,117,103,104,32,103,101,116, - 95,100,97,116,97,40,41,114,102,0,0,0,78,41,5,114, + 106,1,124,2,131,1,125,3,87,0,110,48,4,0,116,2, + 107,10,114,72,1,0,125,4,1,0,122,20,116,3,100,1, + 124,1,100,2,141,2,124,4,130,2,87,0,89,0,100,3, + 100,3,125,4,126,4,88,0,110,2,88,0,116,4,124,3, + 131,1,83,0,41,4,122,52,67,111,110,99,114,101,116,101, + 32,105,109,112,108,101,109,101,110,116,97,116,105,111,110,32, + 111,102,32,73,110,115,112,101,99,116,76,111,97,100,101,114, + 46,103,101,116,95,115,111,117,114,99,101,46,122,39,115,111, + 117,114,99,101,32,110,111,116,32,97,118,97,105,108,97,98, + 108,101,32,116,104,114,111,117,103,104,32,103,101,116,95,100, + 97,116,97,40,41,41,1,114,102,0,0,0,78,41,5,114, 155,0,0,0,218,8,103,101,116,95,100,97,116,97,114,42, 0,0,0,114,103,0,0,0,114,153,0,0,0,41,5,114, 104,0,0,0,114,123,0,0,0,114,37,0,0,0,114,151, 0,0,0,218,3,101,120,99,114,4,0,0,0,114,4,0, 0,0,114,6,0,0,0,218,10,103,101,116,95,115,111,117, - 114,99,101,209,2,0,0,115,14,0,0,0,0,2,10,1, - 2,1,14,1,16,1,6,1,28,1,122,23,83,111,117,114, + 114,99,101,210,2,0,0,115,14,0,0,0,0,2,10,1, + 2,1,14,1,16,1,4,1,28,1,122,23,83,111,117,114, 99,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, 114,99,101,114,31,0,0,0,41,1,218,9,95,111,112,116, 105,109,105,122,101,99,3,0,0,0,1,0,0,0,4,0, - 0,0,9,0,0,0,67,0,0,0,115,26,0,0,0,116, - 0,106,1,116,2,124,1,124,2,100,1,100,2,100,3,100, - 4,124,3,144,2,131,4,83,0,41,5,122,130,82,101,116, - 117,114,110,32,116,104,101,32,99,111,100,101,32,111,98,106, - 101,99,116,32,99,111,109,112,105,108,101,100,32,102,114,111, - 109,32,115,111,117,114,99,101,46,10,10,32,32,32,32,32, - 32,32,32,84,104,101,32,39,100,97,116,97,39,32,97,114, - 103,117,109,101,110,116,32,99,97,110,32,98,101,32,97,110, - 121,32,111,98,106,101,99,116,32,116,121,112,101,32,116,104, - 97,116,32,99,111,109,112,105,108,101,40,41,32,115,117,112, - 112,111,114,116,115,46,10,32,32,32,32,32,32,32,32,114, - 186,0,0,0,218,12,100,111,110,116,95,105,110,104,101,114, - 105,116,84,114,72,0,0,0,41,3,114,118,0,0,0,114, - 185,0,0,0,218,7,99,111,109,112,105,108,101,41,4,114, - 104,0,0,0,114,56,0,0,0,114,37,0,0,0,114,200, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,14,115,111,117,114,99,101,95,116,111,95,99,111, - 100,101,219,2,0,0,115,4,0,0,0,0,5,14,1,122, - 27,83,111,117,114,99,101,76,111,97,100,101,114,46,115,111, - 117,114,99,101,95,116,111,95,99,111,100,101,99,2,0,0, - 0,0,0,0,0,10,0,0,0,43,0,0,0,67,0,0, - 0,115,106,1,0,0,124,0,106,0,124,1,131,1,125,2, - 100,1,125,3,121,12,116,1,124,2,131,1,125,4,87,0, - 110,24,4,0,116,2,107,10,114,50,1,0,1,0,1,0, - 100,1,125,4,89,0,110,174,88,0,121,14,124,0,106,3, - 124,2,131,1,125,5,87,0,110,20,4,0,116,4,107,10, - 114,86,1,0,1,0,1,0,89,0,110,138,88,0,116,5, - 124,5,100,2,25,0,131,1,125,3,121,14,124,0,106,6, - 124,4,131,1,125,6,87,0,110,20,4,0,116,7,107,10, - 114,134,1,0,1,0,1,0,89,0,110,90,88,0,121,26, - 116,8,124,6,100,3,124,5,100,4,124,1,100,5,124,4, - 144,3,131,1,125,7,87,0,110,24,4,0,116,9,116,10, - 102,2,107,10,114,186,1,0,1,0,1,0,89,0,110,38, - 88,0,116,11,106,12,100,6,124,4,124,2,131,3,1,0, - 116,13,124,7,100,4,124,1,100,7,124,4,100,8,124,2, - 144,3,131,1,83,0,124,0,106,6,124,2,131,1,125,8, - 124,0,106,14,124,8,124,2,131,2,125,9,116,11,106,12, - 100,9,124,2,131,2,1,0,116,15,106,16,12,0,144,1, - 114,102,124,4,100,1,107,9,144,1,114,102,124,3,100,1, - 107,9,144,1,114,102,116,17,124,9,124,3,116,18,124,8, - 131,1,131,3,125,6,121,30,124,0,106,19,124,2,124,4, - 124,6,131,3,1,0,116,11,106,12,100,10,124,4,131,2, - 1,0,87,0,110,22,4,0,116,2,107,10,144,1,114,100, - 1,0,1,0,1,0,89,0,110,2,88,0,124,9,83,0, - 41,11,122,190,67,111,110,99,114,101,116,101,32,105,109,112, - 108,101,109,101,110,116,97,116,105,111,110,32,111,102,32,73, - 110,115,112,101,99,116,76,111,97,100,101,114,46,103,101,116, - 95,99,111,100,101,46,10,10,32,32,32,32,32,32,32,32, - 82,101,97,100,105,110,103,32,111,102,32,98,121,116,101,99, - 111,100,101,32,114,101,113,117,105,114,101,115,32,112,97,116, - 104,95,115,116,97,116,115,32,116,111,32,98,101,32,105,109, - 112,108,101,109,101,110,116,101,100,46,32,84,111,32,119,114, - 105,116,101,10,32,32,32,32,32,32,32,32,98,121,116,101, - 99,111,100,101,44,32,115,101,116,95,100,97,116,97,32,109, - 117,115,116,32,97,108,115,111,32,98,101,32,105,109,112,108, - 101,109,101,110,116,101,100,46,10,10,32,32,32,32,32,32, - 32,32,78,114,130,0,0,0,114,136,0,0,0,114,102,0, - 0,0,114,37,0,0,0,122,13,123,125,32,109,97,116,99, - 104,101,115,32,123,125,114,93,0,0,0,114,94,0,0,0, - 122,19,99,111,100,101,32,111,98,106,101,99,116,32,102,114, - 111,109,32,123,125,122,10,119,114,111,116,101,32,123,33,114, - 125,41,20,114,155,0,0,0,114,83,0,0,0,114,70,0, - 0,0,114,194,0,0,0,114,192,0,0,0,114,16,0,0, - 0,114,197,0,0,0,114,42,0,0,0,114,139,0,0,0, - 114,103,0,0,0,114,134,0,0,0,114,118,0,0,0,114, - 133,0,0,0,114,145,0,0,0,114,203,0,0,0,114,8, - 0,0,0,218,19,100,111,110,116,95,119,114,105,116,101,95, - 98,121,116,101,99,111,100,101,114,148,0,0,0,114,33,0, - 0,0,114,196,0,0,0,41,10,114,104,0,0,0,114,123, - 0,0,0,114,94,0,0,0,114,137,0,0,0,114,93,0, - 0,0,218,2,115,116,114,56,0,0,0,218,10,98,121,116, - 101,115,95,100,97,116,97,114,151,0,0,0,90,11,99,111, - 100,101,95,111,98,106,101,99,116,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,184,0,0,0,227,2,0, - 0,115,78,0,0,0,0,7,10,1,4,1,2,1,12,1, - 14,1,10,2,2,1,14,1,14,1,6,2,12,1,2,1, - 14,1,14,1,6,2,2,1,6,1,8,1,12,1,18,1, - 6,2,8,1,6,1,10,1,4,1,8,1,10,1,12,1, - 12,1,20,1,10,1,6,1,10,1,2,1,14,1,16,1, - 16,1,6,1,122,21,83,111,117,114,99,101,76,111,97,100, - 101,114,46,103,101,116,95,99,111,100,101,78,114,91,0,0, - 0,41,10,114,109,0,0,0,114,108,0,0,0,114,110,0, - 0,0,114,193,0,0,0,114,194,0,0,0,114,196,0,0, - 0,114,195,0,0,0,114,199,0,0,0,114,203,0,0,0, - 114,184,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,191,0,0,0,169,2, - 0,0,115,14,0,0,0,8,2,8,8,8,13,8,10,8, - 7,8,10,14,8,114,191,0,0,0,99,0,0,0,0,0, - 0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,115, - 76,0,0,0,101,0,90,1,100,0,90,2,100,1,90,3, - 100,2,100,3,132,0,90,4,100,4,100,5,132,0,90,5, - 100,6,100,7,132,0,90,6,101,7,135,0,102,1,100,8, - 100,9,132,8,131,1,90,8,101,7,100,10,100,11,132,0, - 131,1,90,9,100,12,100,13,132,0,90,10,135,0,83,0, - 41,14,218,10,70,105,108,101,76,111,97,100,101,114,122,103, - 66,97,115,101,32,102,105,108,101,32,108,111,97,100,101,114, - 32,99,108,97,115,115,32,119,104,105,99,104,32,105,109,112, - 108,101,109,101,110,116,115,32,116,104,101,32,108,111,97,100, - 101,114,32,112,114,111,116,111,99,111,108,32,109,101,116,104, - 111,100,115,32,116,104,97,116,10,32,32,32,32,114,101,113, - 117,105,114,101,32,102,105,108,101,32,115,121,115,116,101,109, - 32,117,115,97,103,101,46,99,3,0,0,0,0,0,0,0, - 3,0,0,0,2,0,0,0,67,0,0,0,115,16,0,0, - 0,124,1,124,0,95,0,124,2,124,0,95,1,100,1,83, - 0,41,2,122,75,67,97,99,104,101,32,116,104,101,32,109, - 111,100,117,108,101,32,110,97,109,101,32,97,110,100,32,116, - 104,101,32,112,97,116,104,32,116,111,32,116,104,101,32,102, - 105,108,101,32,102,111,117,110,100,32,98,121,32,116,104,101, - 10,32,32,32,32,32,32,32,32,102,105,110,100,101,114,46, - 78,41,2,114,102,0,0,0,114,37,0,0,0,41,3,114, - 104,0,0,0,114,123,0,0,0,114,37,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,182,0, - 0,0,28,3,0,0,115,4,0,0,0,0,3,6,1,122, - 19,70,105,108,101,76,111,97,100,101,114,46,95,95,105,110, - 105,116,95,95,99,2,0,0,0,0,0,0,0,2,0,0, - 0,2,0,0,0,67,0,0,0,115,24,0,0,0,124,0, - 106,0,124,1,106,0,107,2,111,22,124,0,106,1,124,1, - 106,1,107,2,83,0,41,1,78,41,2,218,9,95,95,99, - 108,97,115,115,95,95,114,115,0,0,0,41,2,114,104,0, - 0,0,218,5,111,116,104,101,114,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,6,95,95,101,113,95,95, - 34,3,0,0,115,4,0,0,0,0,1,12,1,122,17,70, - 105,108,101,76,111,97,100,101,114,46,95,95,101,113,95,95, - 99,1,0,0,0,0,0,0,0,1,0,0,0,3,0,0, - 0,67,0,0,0,115,20,0,0,0,116,0,124,0,106,1, - 131,1,116,0,124,0,106,2,131,1,65,0,83,0,41,1, - 78,41,3,218,4,104,97,115,104,114,102,0,0,0,114,37, - 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,218,8,95,95,104,97,115, - 104,95,95,38,3,0,0,115,2,0,0,0,0,1,122,19, - 70,105,108,101,76,111,97,100,101,114,46,95,95,104,97,115, - 104,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, - 3,0,0,0,3,0,0,0,115,16,0,0,0,116,0,116, - 1,124,0,131,2,106,2,124,1,131,1,83,0,41,1,122, - 100,76,111,97,100,32,97,32,109,111,100,117,108,101,32,102, - 114,111,109,32,97,32,102,105,108,101,46,10,10,32,32,32, - 32,32,32,32,32,84,104,105,115,32,109,101,116,104,111,100, - 32,105,115,32,100,101,112,114,101,99,97,116,101,100,46,32, - 32,85,115,101,32,101,120,101,99,95,109,111,100,117,108,101, - 40,41,32,105,110,115,116,101,97,100,46,10,10,32,32,32, - 32,32,32,32,32,41,3,218,5,115,117,112,101,114,114,207, - 0,0,0,114,190,0,0,0,41,2,114,104,0,0,0,114, - 123,0,0,0,41,1,114,208,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,190,0,0,0,41,3,0,0,115,2, - 0,0,0,0,10,122,22,70,105,108,101,76,111,97,100,101, - 114,46,108,111,97,100,95,109,111,100,117,108,101,99,2,0, - 0,0,0,0,0,0,2,0,0,0,1,0,0,0,67,0, - 0,0,115,6,0,0,0,124,0,106,0,83,0,41,1,122, - 58,82,101,116,117,114,110,32,116,104,101,32,112,97,116,104, - 32,116,111,32,116,104,101,32,115,111,117,114,99,101,32,102, - 105,108,101,32,97,115,32,102,111,117,110,100,32,98,121,32, - 116,104,101,32,102,105,110,100,101,114,46,41,1,114,37,0, - 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,155,0, - 0,0,53,3,0,0,115,2,0,0,0,0,3,122,23,70, - 105,108,101,76,111,97,100,101,114,46,103,101,116,95,102,105, - 108,101,110,97,109,101,99,2,0,0,0,0,0,0,0,3, - 0,0,0,9,0,0,0,67,0,0,0,115,32,0,0,0, - 116,0,106,1,124,1,100,1,131,2,143,10,125,2,124,2, - 106,2,131,0,83,0,81,0,82,0,88,0,100,2,83,0, - 41,3,122,39,82,101,116,117,114,110,32,116,104,101,32,100, - 97,116,97,32,102,114,111,109,32,112,97,116,104,32,97,115, - 32,114,97,119,32,98,121,116,101,115,46,218,1,114,78,41, - 3,114,52,0,0,0,114,53,0,0,0,90,4,114,101,97, - 100,41,3,114,104,0,0,0,114,37,0,0,0,114,57,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,197,0,0,0,58,3,0,0,115,4,0,0,0,0, - 2,14,1,122,19,70,105,108,101,76,111,97,100,101,114,46, - 103,101,116,95,100,97,116,97,41,11,114,109,0,0,0,114, - 108,0,0,0,114,110,0,0,0,114,111,0,0,0,114,182, - 0,0,0,114,210,0,0,0,114,212,0,0,0,114,120,0, - 0,0,114,190,0,0,0,114,155,0,0,0,114,197,0,0, - 0,114,4,0,0,0,114,4,0,0,0,41,1,114,208,0, - 0,0,114,6,0,0,0,114,207,0,0,0,23,3,0,0, - 115,14,0,0,0,8,3,4,2,8,6,8,4,8,3,16, - 12,12,5,114,207,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,3,0,0,0,64,0,0,0,115,46,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, - 100,7,156,1,100,8,100,9,132,2,90,6,100,10,83,0, - 41,11,218,16,83,111,117,114,99,101,70,105,108,101,76,111, - 97,100,101,114,122,62,67,111,110,99,114,101,116,101,32,105, + 0,0,8,0,0,0,67,0,0,0,115,22,0,0,0,116, + 0,106,1,116,2,124,1,124,2,100,1,100,2,124,3,100, + 3,141,6,83,0,41,4,122,130,82,101,116,117,114,110,32, + 116,104,101,32,99,111,100,101,32,111,98,106,101,99,116,32, + 99,111,109,112,105,108,101,100,32,102,114,111,109,32,115,111, + 117,114,99,101,46,10,10,32,32,32,32,32,32,32,32,84, + 104,101,32,39,100,97,116,97,39,32,97,114,103,117,109,101, + 110,116,32,99,97,110,32,98,101,32,97,110,121,32,111,98, + 106,101,99,116,32,116,121,112,101,32,116,104,97,116,32,99, + 111,109,112,105,108,101,40,41,32,115,117,112,112,111,114,116, + 115,46,10,32,32,32,32,32,32,32,32,114,186,0,0,0, + 84,41,2,218,12,100,111,110,116,95,105,110,104,101,114,105, + 116,114,72,0,0,0,41,3,114,118,0,0,0,114,185,0, + 0,0,218,7,99,111,109,112,105,108,101,41,4,114,104,0, + 0,0,114,56,0,0,0,114,37,0,0,0,114,200,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, + 218,14,115,111,117,114,99,101,95,116,111,95,99,111,100,101, + 220,2,0,0,115,4,0,0,0,0,5,12,1,122,27,83, + 111,117,114,99,101,76,111,97,100,101,114,46,115,111,117,114, + 99,101,95,116,111,95,99,111,100,101,99,2,0,0,0,0, + 0,0,0,10,0,0,0,43,0,0,0,67,0,0,0,115, + 94,1,0,0,124,0,106,0,124,1,131,1,125,2,100,1, + 125,3,121,12,116,1,124,2,131,1,125,4,87,0,110,24, + 4,0,116,2,107,10,114,50,1,0,1,0,1,0,100,1, + 125,4,89,0,110,162,88,0,121,14,124,0,106,3,124,2, + 131,1,125,5,87,0,110,20,4,0,116,4,107,10,114,86, + 1,0,1,0,1,0,89,0,110,126,88,0,116,5,124,5, + 100,2,25,0,131,1,125,3,121,14,124,0,106,6,124,4, + 131,1,125,6,87,0,110,20,4,0,116,7,107,10,114,134, + 1,0,1,0,1,0,89,0,110,78,88,0,121,20,116,8, + 124,6,124,5,124,1,124,4,100,3,141,4,125,7,87,0, + 110,24,4,0,116,9,116,10,102,2,107,10,114,180,1,0, + 1,0,1,0,89,0,110,32,88,0,116,11,106,12,100,4, + 124,4,124,2,131,3,1,0,116,13,124,7,124,1,124,4, + 124,2,100,5,141,4,83,0,124,0,106,6,124,2,131,1, + 125,8,124,0,106,14,124,8,124,2,131,2,125,9,116,11, + 106,12,100,6,124,2,131,2,1,0,116,15,106,16,12,0, + 144,1,114,90,124,4,100,1,107,9,144,1,114,90,124,3, + 100,1,107,9,144,1,114,90,116,17,124,9,124,3,116,18, + 124,8,131,1,131,3,125,6,121,30,124,0,106,19,124,2, + 124,4,124,6,131,3,1,0,116,11,106,12,100,7,124,4, + 131,2,1,0,87,0,110,22,4,0,116,2,107,10,144,1, + 114,88,1,0,1,0,1,0,89,0,110,2,88,0,124,9, + 83,0,41,8,122,190,67,111,110,99,114,101,116,101,32,105, 109,112,108,101,109,101,110,116,97,116,105,111,110,32,111,102, - 32,83,111,117,114,99,101,76,111,97,100,101,114,32,117,115, - 105,110,103,32,116,104,101,32,102,105,108,101,32,115,121,115, - 116,101,109,46,99,2,0,0,0,0,0,0,0,3,0,0, - 0,3,0,0,0,67,0,0,0,115,22,0,0,0,116,0, - 124,1,131,1,125,2,124,2,106,1,124,2,106,2,100,1, - 156,2,83,0,41,2,122,33,82,101,116,117,114,110,32,116, - 104,101,32,109,101,116,97,100,97,116,97,32,102,111,114,32, - 116,104,101,32,112,97,116,104,46,41,2,122,5,109,116,105, - 109,101,122,4,115,105,122,101,41,3,114,41,0,0,0,218, - 8,115,116,95,109,116,105,109,101,90,7,115,116,95,115,105, - 122,101,41,3,114,104,0,0,0,114,37,0,0,0,114,205, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,194,0,0,0,68,3,0,0,115,4,0,0,0, - 0,2,8,1,122,27,83,111,117,114,99,101,70,105,108,101, - 76,111,97,100,101,114,46,112,97,116,104,95,115,116,97,116, - 115,99,4,0,0,0,0,0,0,0,5,0,0,0,5,0, - 0,0,67,0,0,0,115,26,0,0,0,116,0,124,1,131, - 1,125,4,124,0,106,1,124,2,124,3,100,1,124,4,144, - 1,131,2,83,0,41,2,78,218,5,95,109,111,100,101,41, - 2,114,101,0,0,0,114,195,0,0,0,41,5,114,104,0, - 0,0,114,94,0,0,0,114,93,0,0,0,114,56,0,0, - 0,114,44,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,196,0,0,0,73,3,0,0,115,4, - 0,0,0,0,2,8,1,122,32,83,111,117,114,99,101,70, - 105,108,101,76,111,97,100,101,114,46,95,99,97,99,104,101, - 95,98,121,116,101,99,111,100,101,105,182,1,0,0,41,1, - 114,217,0,0,0,99,3,0,0,0,1,0,0,0,9,0, - 0,0,17,0,0,0,67,0,0,0,115,250,0,0,0,116, - 0,124,1,131,1,92,2,125,4,125,5,103,0,125,6,120, - 40,124,4,114,56,116,1,124,4,131,1,12,0,114,56,116, - 0,124,4,131,1,92,2,125,4,125,7,124,6,106,2,124, - 7,131,1,1,0,113,18,87,0,120,108,116,3,124,6,131, - 1,68,0,93,96,125,7,116,4,124,4,124,7,131,2,125, - 4,121,14,116,5,106,6,124,4,131,1,1,0,87,0,113, - 68,4,0,116,7,107,10,114,118,1,0,1,0,1,0,119, - 68,89,0,113,68,4,0,116,8,107,10,114,162,1,0,125, - 8,1,0,122,18,116,9,106,10,100,1,124,4,124,8,131, - 3,1,0,100,2,83,0,100,2,125,8,126,8,88,0,113, - 68,88,0,113,68,87,0,121,28,116,11,124,1,124,2,124, - 3,131,3,1,0,116,9,106,10,100,3,124,1,131,2,1, - 0,87,0,110,48,4,0,116,8,107,10,114,244,1,0,125, - 8,1,0,122,20,116,9,106,10,100,1,124,1,124,8,131, - 3,1,0,87,0,89,0,100,2,100,2,125,8,126,8,88, - 0,110,2,88,0,100,2,83,0,41,4,122,27,87,114,105, - 116,101,32,98,121,116,101,115,32,100,97,116,97,32,116,111, - 32,97,32,102,105,108,101,46,122,27,99,111,117,108,100,32, - 110,111,116,32,99,114,101,97,116,101,32,123,33,114,125,58, - 32,123,33,114,125,78,122,12,99,114,101,97,116,101,100,32, - 123,33,114,125,41,12,114,40,0,0,0,114,48,0,0,0, - 114,161,0,0,0,114,35,0,0,0,114,30,0,0,0,114, - 3,0,0,0,90,5,109,107,100,105,114,218,15,70,105,108, - 101,69,120,105,115,116,115,69,114,114,111,114,114,42,0,0, - 0,114,118,0,0,0,114,133,0,0,0,114,58,0,0,0, - 41,9,114,104,0,0,0,114,37,0,0,0,114,56,0,0, - 0,114,217,0,0,0,218,6,112,97,114,101,110,116,114,98, - 0,0,0,114,29,0,0,0,114,25,0,0,0,114,198,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,195,0,0,0,78,3,0,0,115,42,0,0,0,0, - 2,12,1,4,2,16,1,12,1,14,2,14,1,10,1,2, - 1,14,1,14,2,6,1,16,3,6,1,8,1,20,1,2, - 1,12,1,16,1,16,2,8,1,122,25,83,111,117,114,99, - 101,70,105,108,101,76,111,97,100,101,114,46,115,101,116,95, - 100,97,116,97,78,41,7,114,109,0,0,0,114,108,0,0, - 0,114,110,0,0,0,114,111,0,0,0,114,194,0,0,0, - 114,196,0,0,0,114,195,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,215, - 0,0,0,64,3,0,0,115,8,0,0,0,8,2,4,2, - 8,5,8,5,114,215,0,0,0,99,0,0,0,0,0,0, - 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,32, - 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, - 2,100,3,132,0,90,4,100,4,100,5,132,0,90,5,100, - 6,83,0,41,7,218,20,83,111,117,114,99,101,108,101,115, - 115,70,105,108,101,76,111,97,100,101,114,122,45,76,111,97, - 100,101,114,32,119,104,105,99,104,32,104,97,110,100,108,101, - 115,32,115,111,117,114,99,101,108,101,115,115,32,102,105,108, - 101,32,105,109,112,111,114,116,115,46,99,2,0,0,0,0, - 0,0,0,5,0,0,0,6,0,0,0,67,0,0,0,115, - 56,0,0,0,124,0,106,0,124,1,131,1,125,2,124,0, - 106,1,124,2,131,1,125,3,116,2,124,3,100,1,124,1, - 100,2,124,2,144,2,131,1,125,4,116,3,124,4,100,1, - 124,1,100,3,124,2,144,2,131,1,83,0,41,4,78,114, - 102,0,0,0,114,37,0,0,0,114,93,0,0,0,41,4, - 114,155,0,0,0,114,197,0,0,0,114,139,0,0,0,114, - 145,0,0,0,41,5,114,104,0,0,0,114,123,0,0,0, - 114,37,0,0,0,114,56,0,0,0,114,206,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,184, - 0,0,0,113,3,0,0,115,8,0,0,0,0,1,10,1, - 10,1,18,1,122,29,83,111,117,114,99,101,108,101,115,115, - 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, - 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, - 0,41,2,122,39,82,101,116,117,114,110,32,78,111,110,101, - 32,97,115,32,116,104,101,114,101,32,105,115,32,110,111,32, - 115,111,117,114,99,101,32,99,111,100,101,46,78,114,4,0, - 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, - 0,0,119,3,0,0,115,2,0,0,0,0,2,122,31,83, - 111,117,114,99,101,108,101,115,115,70,105,108,101,76,111,97, - 100,101,114,46,103,101,116,95,115,111,117,114,99,101,78,41, - 6,114,109,0,0,0,114,108,0,0,0,114,110,0,0,0, - 114,111,0,0,0,114,184,0,0,0,114,199,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,220,0,0,0,109,3,0,0,115,6,0,0, - 0,8,2,4,2,8,6,114,220,0,0,0,99,0,0,0, - 0,0,0,0,0,0,0,0,0,3,0,0,0,64,0,0, - 0,115,92,0,0,0,101,0,90,1,100,0,90,2,100,1, - 90,3,100,2,100,3,132,0,90,4,100,4,100,5,132,0, - 90,5,100,6,100,7,132,0,90,6,100,8,100,9,132,0, - 90,7,100,10,100,11,132,0,90,8,100,12,100,13,132,0, - 90,9,100,14,100,15,132,0,90,10,100,16,100,17,132,0, - 90,11,101,12,100,18,100,19,132,0,131,1,90,13,100,20, - 83,0,41,21,218,19,69,120,116,101,110,115,105,111,110,70, - 105,108,101,76,111,97,100,101,114,122,93,76,111,97,100,101, - 114,32,102,111,114,32,101,120,116,101,110,115,105,111,110,32, - 109,111,100,117,108,101,115,46,10,10,32,32,32,32,84,104, - 101,32,99,111,110,115,116,114,117,99,116,111,114,32,105,115, - 32,100,101,115,105,103,110,101,100,32,116,111,32,119,111,114, - 107,32,119,105,116,104,32,70,105,108,101,70,105,110,100,101, - 114,46,10,10,32,32,32,32,99,3,0,0,0,0,0,0, - 0,3,0,0,0,2,0,0,0,67,0,0,0,115,16,0, - 0,0,124,1,124,0,95,0,124,2,124,0,95,1,100,0, - 83,0,41,1,78,41,2,114,102,0,0,0,114,37,0,0, - 0,41,3,114,104,0,0,0,114,102,0,0,0,114,37,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,182,0,0,0,136,3,0,0,115,4,0,0,0,0, - 1,6,1,122,28,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,95,95,105,110,105,116,95, - 95,99,2,0,0,0,0,0,0,0,2,0,0,0,2,0, - 0,0,67,0,0,0,115,24,0,0,0,124,0,106,0,124, - 1,106,0,107,2,111,22,124,0,106,1,124,1,106,1,107, - 2,83,0,41,1,78,41,2,114,208,0,0,0,114,115,0, - 0,0,41,2,114,104,0,0,0,114,209,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,210,0, - 0,0,140,3,0,0,115,4,0,0,0,0,1,12,1,122, - 26,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, + 32,73,110,115,112,101,99,116,76,111,97,100,101,114,46,103, + 101,116,95,99,111,100,101,46,10,10,32,32,32,32,32,32, + 32,32,82,101,97,100,105,110,103,32,111,102,32,98,121,116, + 101,99,111,100,101,32,114,101,113,117,105,114,101,115,32,112, + 97,116,104,95,115,116,97,116,115,32,116,111,32,98,101,32, + 105,109,112,108,101,109,101,110,116,101,100,46,32,84,111,32, + 119,114,105,116,101,10,32,32,32,32,32,32,32,32,98,121, + 116,101,99,111,100,101,44,32,115,101,116,95,100,97,116,97, + 32,109,117,115,116,32,97,108,115,111,32,98,101,32,105,109, + 112,108,101,109,101,110,116,101,100,46,10,10,32,32,32,32, + 32,32,32,32,78,114,130,0,0,0,41,3,114,136,0,0, + 0,114,102,0,0,0,114,37,0,0,0,122,13,123,125,32, + 109,97,116,99,104,101,115,32,123,125,41,3,114,102,0,0, + 0,114,93,0,0,0,114,94,0,0,0,122,19,99,111,100, + 101,32,111,98,106,101,99,116,32,102,114,111,109,32,123,125, + 122,10,119,114,111,116,101,32,123,33,114,125,41,20,114,155, + 0,0,0,114,83,0,0,0,114,70,0,0,0,114,194,0, + 0,0,114,192,0,0,0,114,16,0,0,0,114,197,0,0, + 0,114,42,0,0,0,114,139,0,0,0,114,103,0,0,0, + 114,134,0,0,0,114,118,0,0,0,114,133,0,0,0,114, + 145,0,0,0,114,203,0,0,0,114,8,0,0,0,218,19, + 100,111,110,116,95,119,114,105,116,101,95,98,121,116,101,99, + 111,100,101,114,148,0,0,0,114,33,0,0,0,114,196,0, + 0,0,41,10,114,104,0,0,0,114,123,0,0,0,114,94, + 0,0,0,114,137,0,0,0,114,93,0,0,0,218,2,115, + 116,114,56,0,0,0,218,10,98,121,116,101,115,95,100,97, + 116,97,114,151,0,0,0,90,11,99,111,100,101,95,111,98, + 106,101,99,116,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,114,184,0,0,0,228,2,0,0,115,78,0,0, + 0,0,7,10,1,4,1,2,1,12,1,14,1,10,2,2, + 1,14,1,14,1,6,2,12,1,2,1,14,1,14,1,6, + 2,2,1,4,1,4,1,12,1,18,1,6,2,8,1,6, + 1,6,1,2,1,8,1,10,1,12,1,12,1,20,1,10, + 1,6,1,10,1,2,1,14,1,16,1,16,1,6,1,122, + 21,83,111,117,114,99,101,76,111,97,100,101,114,46,103,101, + 116,95,99,111,100,101,78,114,91,0,0,0,41,10,114,109, + 0,0,0,114,108,0,0,0,114,110,0,0,0,114,193,0, + 0,0,114,194,0,0,0,114,196,0,0,0,114,195,0,0, + 0,114,199,0,0,0,114,203,0,0,0,114,184,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,191,0,0,0,170,2,0,0,115,14,0, + 0,0,8,2,8,8,8,13,8,10,8,7,8,10,14,8, + 114,191,0,0,0,99,0,0,0,0,0,0,0,0,0,0, + 0,0,4,0,0,0,0,0,0,0,115,76,0,0,0,101, + 0,90,1,100,0,90,2,100,1,90,3,100,2,100,3,132, + 0,90,4,100,4,100,5,132,0,90,5,100,6,100,7,132, + 0,90,6,101,7,135,0,102,1,100,8,100,9,132,8,131, + 1,90,8,101,7,100,10,100,11,132,0,131,1,90,9,100, + 12,100,13,132,0,90,10,135,0,83,0,41,14,218,10,70, + 105,108,101,76,111,97,100,101,114,122,103,66,97,115,101,32, + 102,105,108,101,32,108,111,97,100,101,114,32,99,108,97,115, + 115,32,119,104,105,99,104,32,105,109,112,108,101,109,101,110, + 116,115,32,116,104,101,32,108,111,97,100,101,114,32,112,114, + 111,116,111,99,111,108,32,109,101,116,104,111,100,115,32,116, + 104,97,116,10,32,32,32,32,114,101,113,117,105,114,101,32, + 102,105,108,101,32,115,121,115,116,101,109,32,117,115,97,103, + 101,46,99,3,0,0,0,0,0,0,0,3,0,0,0,2, + 0,0,0,67,0,0,0,115,16,0,0,0,124,1,124,0, + 95,0,124,2,124,0,95,1,100,1,83,0,41,2,122,75, + 67,97,99,104,101,32,116,104,101,32,109,111,100,117,108,101, + 32,110,97,109,101,32,97,110,100,32,116,104,101,32,112,97, + 116,104,32,116,111,32,116,104,101,32,102,105,108,101,32,102, + 111,117,110,100,32,98,121,32,116,104,101,10,32,32,32,32, + 32,32,32,32,102,105,110,100,101,114,46,78,41,2,114,102, + 0,0,0,114,37,0,0,0,41,3,114,104,0,0,0,114, + 123,0,0,0,114,37,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,182,0,0,0,29,3,0, + 0,115,4,0,0,0,0,3,6,1,122,19,70,105,108,101, + 76,111,97,100,101,114,46,95,95,105,110,105,116,95,95,99, + 2,0,0,0,0,0,0,0,2,0,0,0,2,0,0,0, + 67,0,0,0,115,24,0,0,0,124,0,106,0,124,1,106, + 0,107,2,111,22,124,0,106,1,124,1,106,1,107,2,83, + 0,41,1,78,41,2,218,9,95,95,99,108,97,115,115,95, + 95,114,115,0,0,0,41,2,114,104,0,0,0,218,5,111, + 116,104,101,114,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,218,6,95,95,101,113,95,95,35,3,0,0,115, + 4,0,0,0,0,1,12,1,122,17,70,105,108,101,76,111, 97,100,101,114,46,95,95,101,113,95,95,99,1,0,0,0, 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, 115,20,0,0,0,116,0,124,0,106,1,131,1,116,0,124, - 0,106,2,131,1,65,0,83,0,41,1,78,41,3,114,211, - 0,0,0,114,102,0,0,0,114,37,0,0,0,41,1,114, - 104,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,212,0,0,0,144,3,0,0,115,2,0,0, - 0,0,1,122,28,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,95,95,104,97,115,104,95, - 95,99,2,0,0,0,0,0,0,0,3,0,0,0,4,0, - 0,0,67,0,0,0,115,36,0,0,0,116,0,106,1,116, - 2,106,3,124,1,131,2,125,2,116,0,106,4,100,1,124, - 1,106,5,124,0,106,6,131,3,1,0,124,2,83,0,41, - 2,122,38,67,114,101,97,116,101,32,97,110,32,117,110,105, - 116,105,97,108,105,122,101,100,32,101,120,116,101,110,115,105, - 111,110,32,109,111,100,117,108,101,122,38,101,120,116,101,110, - 115,105,111,110,32,109,111,100,117,108,101,32,123,33,114,125, - 32,108,111,97,100,101,100,32,102,114,111,109,32,123,33,114, - 125,41,7,114,118,0,0,0,114,185,0,0,0,114,143,0, - 0,0,90,14,99,114,101,97,116,101,95,100,121,110,97,109, + 0,106,2,131,1,65,0,83,0,41,1,78,41,3,218,4, + 104,97,115,104,114,102,0,0,0,114,37,0,0,0,41,1, + 114,104,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,218,8,95,95,104,97,115,104,95,95,39,3, + 0,0,115,2,0,0,0,0,1,122,19,70,105,108,101,76, + 111,97,100,101,114,46,95,95,104,97,115,104,95,95,99,2, + 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,3, + 0,0,0,115,16,0,0,0,116,0,116,1,124,0,131,2, + 106,2,124,1,131,1,83,0,41,1,122,100,76,111,97,100, + 32,97,32,109,111,100,117,108,101,32,102,114,111,109,32,97, + 32,102,105,108,101,46,10,10,32,32,32,32,32,32,32,32, + 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, + 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, + 101,120,101,99,95,109,111,100,117,108,101,40,41,32,105,110, + 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, + 41,3,218,5,115,117,112,101,114,114,207,0,0,0,114,190, + 0,0,0,41,2,114,104,0,0,0,114,123,0,0,0,41, + 1,114,208,0,0,0,114,4,0,0,0,114,6,0,0,0, + 114,190,0,0,0,42,3,0,0,115,2,0,0,0,0,10, + 122,22,70,105,108,101,76,111,97,100,101,114,46,108,111,97, + 100,95,109,111,100,117,108,101,99,2,0,0,0,0,0,0, + 0,2,0,0,0,1,0,0,0,67,0,0,0,115,6,0, + 0,0,124,0,106,0,83,0,41,1,122,58,82,101,116,117, + 114,110,32,116,104,101,32,112,97,116,104,32,116,111,32,116, + 104,101,32,115,111,117,114,99,101,32,102,105,108,101,32,97, + 115,32,102,111,117,110,100,32,98,121,32,116,104,101,32,102, + 105,110,100,101,114,46,41,1,114,37,0,0,0,41,2,114, + 104,0,0,0,114,123,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,155,0,0,0,54,3,0, + 0,115,2,0,0,0,0,3,122,23,70,105,108,101,76,111, + 97,100,101,114,46,103,101,116,95,102,105,108,101,110,97,109, + 101,99,2,0,0,0,0,0,0,0,3,0,0,0,9,0, + 0,0,67,0,0,0,115,32,0,0,0,116,0,106,1,124, + 1,100,1,131,2,143,10,125,2,124,2,106,2,131,0,83, + 0,81,0,82,0,88,0,100,2,83,0,41,3,122,39,82, + 101,116,117,114,110,32,116,104,101,32,100,97,116,97,32,102, + 114,111,109,32,112,97,116,104,32,97,115,32,114,97,119,32, + 98,121,116,101,115,46,218,1,114,78,41,3,114,52,0,0, + 0,114,53,0,0,0,90,4,114,101,97,100,41,3,114,104, + 0,0,0,114,37,0,0,0,114,57,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,197,0,0, + 0,59,3,0,0,115,4,0,0,0,0,2,14,1,122,19, + 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,100, + 97,116,97,41,11,114,109,0,0,0,114,108,0,0,0,114, + 110,0,0,0,114,111,0,0,0,114,182,0,0,0,114,210, + 0,0,0,114,212,0,0,0,114,120,0,0,0,114,190,0, + 0,0,114,155,0,0,0,114,197,0,0,0,114,4,0,0, + 0,114,4,0,0,0,41,1,114,208,0,0,0,114,6,0, + 0,0,114,207,0,0,0,24,3,0,0,115,14,0,0,0, + 8,3,4,2,8,6,8,4,8,3,16,12,12,5,114,207, + 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, + 3,0,0,0,64,0,0,0,115,46,0,0,0,101,0,90, + 1,100,0,90,2,100,1,90,3,100,2,100,3,132,0,90, + 4,100,4,100,5,132,0,90,5,100,6,100,7,156,1,100, + 8,100,9,132,2,90,6,100,10,83,0,41,11,218,16,83, + 111,117,114,99,101,70,105,108,101,76,111,97,100,101,114,122, + 62,67,111,110,99,114,101,116,101,32,105,109,112,108,101,109, + 101,110,116,97,116,105,111,110,32,111,102,32,83,111,117,114, + 99,101,76,111,97,100,101,114,32,117,115,105,110,103,32,116, + 104,101,32,102,105,108,101,32,115,121,115,116,101,109,46,99, + 2,0,0,0,0,0,0,0,3,0,0,0,3,0,0,0, + 67,0,0,0,115,22,0,0,0,116,0,124,1,131,1,125, + 2,124,2,106,1,124,2,106,2,100,1,156,2,83,0,41, + 2,122,33,82,101,116,117,114,110,32,116,104,101,32,109,101, + 116,97,100,97,116,97,32,102,111,114,32,116,104,101,32,112, + 97,116,104,46,41,2,122,5,109,116,105,109,101,122,4,115, + 105,122,101,41,3,114,41,0,0,0,218,8,115,116,95,109, + 116,105,109,101,90,7,115,116,95,115,105,122,101,41,3,114, + 104,0,0,0,114,37,0,0,0,114,205,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,194,0, + 0,0,69,3,0,0,115,4,0,0,0,0,2,8,1,122, + 27,83,111,117,114,99,101,70,105,108,101,76,111,97,100,101, + 114,46,112,97,116,104,95,115,116,97,116,115,99,4,0,0, + 0,0,0,0,0,5,0,0,0,5,0,0,0,67,0,0, + 0,115,24,0,0,0,116,0,124,1,131,1,125,4,124,0, + 106,1,124,2,124,3,124,4,100,1,141,3,83,0,41,2, + 78,41,1,218,5,95,109,111,100,101,41,2,114,101,0,0, + 0,114,195,0,0,0,41,5,114,104,0,0,0,114,94,0, + 0,0,114,93,0,0,0,114,56,0,0,0,114,44,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, + 114,196,0,0,0,74,3,0,0,115,4,0,0,0,0,2, + 8,1,122,32,83,111,117,114,99,101,70,105,108,101,76,111, + 97,100,101,114,46,95,99,97,99,104,101,95,98,121,116,101, + 99,111,100,101,105,182,1,0,0,41,1,114,217,0,0,0, + 99,3,0,0,0,1,0,0,0,9,0,0,0,17,0,0, + 0,67,0,0,0,115,250,0,0,0,116,0,124,1,131,1, + 92,2,125,4,125,5,103,0,125,6,120,40,124,4,114,56, + 116,1,124,4,131,1,12,0,114,56,116,0,124,4,131,1, + 92,2,125,4,125,7,124,6,106,2,124,7,131,1,1,0, + 113,18,87,0,120,108,116,3,124,6,131,1,68,0,93,96, + 125,7,116,4,124,4,124,7,131,2,125,4,121,14,116,5, + 106,6,124,4,131,1,1,0,87,0,113,68,4,0,116,7, + 107,10,114,118,1,0,1,0,1,0,119,68,89,0,113,68, + 4,0,116,8,107,10,114,162,1,0,125,8,1,0,122,18, + 116,9,106,10,100,1,124,4,124,8,131,3,1,0,100,2, + 83,0,100,2,125,8,126,8,88,0,113,68,88,0,113,68, + 87,0,121,28,116,11,124,1,124,2,124,3,131,3,1,0, + 116,9,106,10,100,3,124,1,131,2,1,0,87,0,110,48, + 4,0,116,8,107,10,114,244,1,0,125,8,1,0,122,20, + 116,9,106,10,100,1,124,1,124,8,131,3,1,0,87,0, + 89,0,100,2,100,2,125,8,126,8,88,0,110,2,88,0, + 100,2,83,0,41,4,122,27,87,114,105,116,101,32,98,121, + 116,101,115,32,100,97,116,97,32,116,111,32,97,32,102,105, + 108,101,46,122,27,99,111,117,108,100,32,110,111,116,32,99, + 114,101,97,116,101,32,123,33,114,125,58,32,123,33,114,125, + 78,122,12,99,114,101,97,116,101,100,32,123,33,114,125,41, + 12,114,40,0,0,0,114,48,0,0,0,114,161,0,0,0, + 114,35,0,0,0,114,30,0,0,0,114,3,0,0,0,90, + 5,109,107,100,105,114,218,15,70,105,108,101,69,120,105,115, + 116,115,69,114,114,111,114,114,42,0,0,0,114,118,0,0, + 0,114,133,0,0,0,114,58,0,0,0,41,9,114,104,0, + 0,0,114,37,0,0,0,114,56,0,0,0,114,217,0,0, + 0,218,6,112,97,114,101,110,116,114,98,0,0,0,114,29, + 0,0,0,114,25,0,0,0,114,198,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,195,0,0, + 0,79,3,0,0,115,42,0,0,0,0,2,12,1,4,2, + 16,1,12,1,14,2,14,1,10,1,2,1,14,1,14,2, + 6,1,16,3,6,1,8,1,20,1,2,1,12,1,16,1, + 16,2,8,1,122,25,83,111,117,114,99,101,70,105,108,101, + 76,111,97,100,101,114,46,115,101,116,95,100,97,116,97,78, + 41,7,114,109,0,0,0,114,108,0,0,0,114,110,0,0, + 0,114,111,0,0,0,114,194,0,0,0,114,196,0,0,0, + 114,195,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,215,0,0,0,65,3, + 0,0,115,8,0,0,0,8,2,4,2,8,5,8,5,114, + 215,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, + 0,2,0,0,0,64,0,0,0,115,32,0,0,0,101,0, + 90,1,100,0,90,2,100,1,90,3,100,2,100,3,132,0, + 90,4,100,4,100,5,132,0,90,5,100,6,83,0,41,7, + 218,20,83,111,117,114,99,101,108,101,115,115,70,105,108,101, + 76,111,97,100,101,114,122,45,76,111,97,100,101,114,32,119, + 104,105,99,104,32,104,97,110,100,108,101,115,32,115,111,117, + 114,99,101,108,101,115,115,32,102,105,108,101,32,105,109,112, + 111,114,116,115,46,99,2,0,0,0,0,0,0,0,5,0, + 0,0,5,0,0,0,67,0,0,0,115,48,0,0,0,124, + 0,106,0,124,1,131,1,125,2,124,0,106,1,124,2,131, + 1,125,3,116,2,124,3,124,1,124,2,100,1,141,3,125, + 4,116,3,124,4,124,1,124,2,100,2,141,3,83,0,41, + 3,78,41,2,114,102,0,0,0,114,37,0,0,0,41,2, + 114,102,0,0,0,114,93,0,0,0,41,4,114,155,0,0, + 0,114,197,0,0,0,114,139,0,0,0,114,145,0,0,0, + 41,5,114,104,0,0,0,114,123,0,0,0,114,37,0,0, + 0,114,56,0,0,0,114,206,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,114,184,0,0,0,114, + 3,0,0,115,8,0,0,0,0,1,10,1,10,1,14,1, + 122,29,83,111,117,114,99,101,108,101,115,115,70,105,108,101, + 76,111,97,100,101,114,46,103,101,116,95,99,111,100,101,99, + 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, + 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,122, + 39,82,101,116,117,114,110,32,78,111,110,101,32,97,115,32, + 116,104,101,114,101,32,105,115,32,110,111,32,115,111,117,114, + 99,101,32,99,111,100,101,46,78,114,4,0,0,0,41,2, + 114,104,0,0,0,114,123,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,199,0,0,0,120,3, + 0,0,115,2,0,0,0,0,2,122,31,83,111,117,114,99, + 101,108,101,115,115,70,105,108,101,76,111,97,100,101,114,46, + 103,101,116,95,115,111,117,114,99,101,78,41,6,114,109,0, + 0,0,114,108,0,0,0,114,110,0,0,0,114,111,0,0, + 0,114,184,0,0,0,114,199,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, + 220,0,0,0,110,3,0,0,115,6,0,0,0,8,2,4, + 2,8,6,114,220,0,0,0,99,0,0,0,0,0,0,0, + 0,0,0,0,0,3,0,0,0,64,0,0,0,115,92,0, + 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, + 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, + 100,7,132,0,90,6,100,8,100,9,132,0,90,7,100,10, + 100,11,132,0,90,8,100,12,100,13,132,0,90,9,100,14, + 100,15,132,0,90,10,100,16,100,17,132,0,90,11,101,12, + 100,18,100,19,132,0,131,1,90,13,100,20,83,0,41,21, + 218,19,69,120,116,101,110,115,105,111,110,70,105,108,101,76, + 111,97,100,101,114,122,93,76,111,97,100,101,114,32,102,111, + 114,32,101,120,116,101,110,115,105,111,110,32,109,111,100,117, + 108,101,115,46,10,10,32,32,32,32,84,104,101,32,99,111, + 110,115,116,114,117,99,116,111,114,32,105,115,32,100,101,115, + 105,103,110,101,100,32,116,111,32,119,111,114,107,32,119,105, + 116,104,32,70,105,108,101,70,105,110,100,101,114,46,10,10, + 32,32,32,32,99,3,0,0,0,0,0,0,0,3,0,0, + 0,2,0,0,0,67,0,0,0,115,16,0,0,0,124,1, + 124,0,95,0,124,2,124,0,95,1,100,0,83,0,41,1, + 78,41,2,114,102,0,0,0,114,37,0,0,0,41,3,114, + 104,0,0,0,114,102,0,0,0,114,37,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,182,0, + 0,0,137,3,0,0,115,4,0,0,0,0,1,6,1,122, + 28,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, + 97,100,101,114,46,95,95,105,110,105,116,95,95,99,2,0, + 0,0,0,0,0,0,2,0,0,0,2,0,0,0,67,0, + 0,0,115,24,0,0,0,124,0,106,0,124,1,106,0,107, + 2,111,22,124,0,106,1,124,1,106,1,107,2,83,0,41, + 1,78,41,2,114,208,0,0,0,114,115,0,0,0,41,2, + 114,104,0,0,0,114,209,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,210,0,0,0,141,3, + 0,0,115,4,0,0,0,0,1,12,1,122,26,69,120,116, + 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, + 46,95,95,101,113,95,95,99,1,0,0,0,0,0,0,0, + 1,0,0,0,3,0,0,0,67,0,0,0,115,20,0,0, + 0,116,0,124,0,106,1,131,1,116,0,124,0,106,2,131, + 1,65,0,83,0,41,1,78,41,3,114,211,0,0,0,114, + 102,0,0,0,114,37,0,0,0,41,1,114,104,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, + 212,0,0,0,145,3,0,0,115,2,0,0,0,0,1,122, + 28,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, + 97,100,101,114,46,95,95,104,97,115,104,95,95,99,2,0, + 0,0,0,0,0,0,3,0,0,0,4,0,0,0,67,0, + 0,0,115,36,0,0,0,116,0,106,1,116,2,106,3,124, + 1,131,2,125,2,116,0,106,4,100,1,124,1,106,5,124, + 0,106,6,131,3,1,0,124,2,83,0,41,2,122,38,67, + 114,101,97,116,101,32,97,110,32,117,110,105,116,105,97,108, + 105,122,101,100,32,101,120,116,101,110,115,105,111,110,32,109, + 111,100,117,108,101,122,38,101,120,116,101,110,115,105,111,110, + 32,109,111,100,117,108,101,32,123,33,114,125,32,108,111,97, + 100,101,100,32,102,114,111,109,32,123,33,114,125,41,7,114, + 118,0,0,0,114,185,0,0,0,114,143,0,0,0,90,14, + 99,114,101,97,116,101,95,100,121,110,97,109,105,99,114,133, + 0,0,0,114,102,0,0,0,114,37,0,0,0,41,3,114, + 104,0,0,0,114,162,0,0,0,114,187,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,183,0, + 0,0,148,3,0,0,115,10,0,0,0,0,2,4,1,10, + 1,6,1,12,1,122,33,69,120,116,101,110,115,105,111,110, + 70,105,108,101,76,111,97,100,101,114,46,99,114,101,97,116, + 101,95,109,111,100,117,108,101,99,2,0,0,0,0,0,0, + 0,2,0,0,0,4,0,0,0,67,0,0,0,115,36,0, + 0,0,116,0,106,1,116,2,106,3,124,1,131,2,1,0, + 116,0,106,4,100,1,124,0,106,5,124,0,106,6,131,3, + 1,0,100,2,83,0,41,3,122,30,73,110,105,116,105,97, + 108,105,122,101,32,97,110,32,101,120,116,101,110,115,105,111, + 110,32,109,111,100,117,108,101,122,40,101,120,116,101,110,115, + 105,111,110,32,109,111,100,117,108,101,32,123,33,114,125,32, + 101,120,101,99,117,116,101,100,32,102,114,111,109,32,123,33, + 114,125,78,41,7,114,118,0,0,0,114,185,0,0,0,114, + 143,0,0,0,90,12,101,120,101,99,95,100,121,110,97,109, 105,99,114,133,0,0,0,114,102,0,0,0,114,37,0,0, - 0,41,3,114,104,0,0,0,114,162,0,0,0,114,187,0, + 0,41,2,114,104,0,0,0,114,187,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,188,0,0, + 0,156,3,0,0,115,6,0,0,0,0,2,14,1,6,1, + 122,31,69,120,116,101,110,115,105,111,110,70,105,108,101,76, + 111,97,100,101,114,46,101,120,101,99,95,109,111,100,117,108, + 101,99,2,0,0,0,0,0,0,0,2,0,0,0,4,0, + 0,0,3,0,0,0,115,36,0,0,0,116,0,124,0,106, + 1,131,1,100,1,25,0,137,0,116,2,135,0,102,1,100, + 2,100,3,132,8,116,3,68,0,131,1,131,1,83,0,41, + 4,122,49,82,101,116,117,114,110,32,84,114,117,101,32,105, + 102,32,116,104,101,32,101,120,116,101,110,115,105,111,110,32, + 109,111,100,117,108,101,32,105,115,32,97,32,112,97,99,107, + 97,103,101,46,114,31,0,0,0,99,1,0,0,0,0,0, + 0,0,2,0,0,0,4,0,0,0,51,0,0,0,115,26, + 0,0,0,124,0,93,18,125,1,136,0,100,0,124,1,23, + 0,107,2,86,0,1,0,113,2,100,1,83,0,41,2,114, + 182,0,0,0,78,114,4,0,0,0,41,2,114,24,0,0, + 0,218,6,115,117,102,102,105,120,41,1,218,9,102,105,108, + 101,95,110,97,109,101,114,4,0,0,0,114,6,0,0,0, + 250,9,60,103,101,110,101,120,112,114,62,165,3,0,0,115, + 2,0,0,0,4,1,122,49,69,120,116,101,110,115,105,111, + 110,70,105,108,101,76,111,97,100,101,114,46,105,115,95,112, + 97,99,107,97,103,101,46,60,108,111,99,97,108,115,62,46, + 60,103,101,110,101,120,112,114,62,41,4,114,40,0,0,0, + 114,37,0,0,0,218,3,97,110,121,218,18,69,88,84,69, + 78,83,73,79,78,95,83,85,70,70,73,88,69,83,41,2, + 114,104,0,0,0,114,123,0,0,0,114,4,0,0,0,41, + 1,114,223,0,0,0,114,6,0,0,0,114,157,0,0,0, + 162,3,0,0,115,6,0,0,0,0,2,14,1,12,1,122, + 30,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, + 97,100,101,114,46,105,115,95,112,97,99,107,97,103,101,99, + 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, + 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,122, + 63,82,101,116,117,114,110,32,78,111,110,101,32,97,115,32, + 97,110,32,101,120,116,101,110,115,105,111,110,32,109,111,100, + 117,108,101,32,99,97,110,110,111,116,32,99,114,101,97,116, + 101,32,97,32,99,111,100,101,32,111,98,106,101,99,116,46, + 78,114,4,0,0,0,41,2,114,104,0,0,0,114,123,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,183,0,0,0,147,3,0,0,115,10,0,0,0,0, - 2,4,1,10,1,6,1,12,1,122,33,69,120,116,101,110, - 115,105,111,110,70,105,108,101,76,111,97,100,101,114,46,99, - 114,101,97,116,101,95,109,111,100,117,108,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,4,0,0,0,67,0,0, - 0,115,36,0,0,0,116,0,106,1,116,2,106,3,124,1, - 131,2,1,0,116,0,106,4,100,1,124,0,106,5,124,0, - 106,6,131,3,1,0,100,2,83,0,41,3,122,30,73,110, - 105,116,105,97,108,105,122,101,32,97,110,32,101,120,116,101, - 110,115,105,111,110,32,109,111,100,117,108,101,122,40,101,120, - 116,101,110,115,105,111,110,32,109,111,100,117,108,101,32,123, - 33,114,125,32,101,120,101,99,117,116,101,100,32,102,114,111, - 109,32,123,33,114,125,78,41,7,114,118,0,0,0,114,185, - 0,0,0,114,143,0,0,0,90,12,101,120,101,99,95,100, - 121,110,97,109,105,99,114,133,0,0,0,114,102,0,0,0, - 114,37,0,0,0,41,2,114,104,0,0,0,114,187,0,0, - 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 114,188,0,0,0,155,3,0,0,115,6,0,0,0,0,2, - 14,1,6,1,122,31,69,120,116,101,110,115,105,111,110,70, - 105,108,101,76,111,97,100,101,114,46,101,120,101,99,95,109, - 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, - 0,0,4,0,0,0,3,0,0,0,115,36,0,0,0,116, - 0,124,0,106,1,131,1,100,1,25,0,137,0,116,2,135, - 0,102,1,100,2,100,3,132,8,116,3,68,0,131,1,131, - 1,83,0,41,4,122,49,82,101,116,117,114,110,32,84,114, - 117,101,32,105,102,32,116,104,101,32,101,120,116,101,110,115, - 105,111,110,32,109,111,100,117,108,101,32,105,115,32,97,32, - 112,97,99,107,97,103,101,46,114,31,0,0,0,99,1,0, - 0,0,0,0,0,0,2,0,0,0,4,0,0,0,51,0, - 0,0,115,26,0,0,0,124,0,93,18,125,1,136,0,100, - 0,124,1,23,0,107,2,86,0,1,0,113,2,100,1,83, - 0,41,2,114,182,0,0,0,78,114,4,0,0,0,41,2, - 114,24,0,0,0,218,6,115,117,102,102,105,120,41,1,218, - 9,102,105,108,101,95,110,97,109,101,114,4,0,0,0,114, - 6,0,0,0,250,9,60,103,101,110,101,120,112,114,62,164, - 3,0,0,115,2,0,0,0,4,1,122,49,69,120,116,101, - 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, - 105,115,95,112,97,99,107,97,103,101,46,60,108,111,99,97, - 108,115,62,46,60,103,101,110,101,120,112,114,62,41,4,114, - 40,0,0,0,114,37,0,0,0,218,3,97,110,121,218,18, - 69,88,84,69,78,83,73,79,78,95,83,85,70,70,73,88, - 69,83,41,2,114,104,0,0,0,114,123,0,0,0,114,4, - 0,0,0,41,1,114,223,0,0,0,114,6,0,0,0,114, - 157,0,0,0,161,3,0,0,115,6,0,0,0,0,2,14, - 1,12,1,122,30,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, + 0,114,184,0,0,0,168,3,0,0,115,2,0,0,0,0, + 2,122,28,69,120,116,101,110,115,105,111,110,70,105,108,101, + 76,111,97,100,101,114,46,103,101,116,95,99,111,100,101,99, + 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, + 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,122, + 53,82,101,116,117,114,110,32,78,111,110,101,32,97,115,32, + 101,120,116,101,110,115,105,111,110,32,109,111,100,117,108,101, + 115,32,104,97,118,101,32,110,111,32,115,111,117,114,99,101, + 32,99,111,100,101,46,78,114,4,0,0,0,41,2,114,104, + 0,0,0,114,123,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,114,199,0,0,0,172,3,0,0, + 115,2,0,0,0,0,2,122,30,69,120,116,101,110,115,105, + 111,110,70,105,108,101,76,111,97,100,101,114,46,103,101,116, + 95,115,111,117,114,99,101,99,2,0,0,0,0,0,0,0, + 2,0,0,0,1,0,0,0,67,0,0,0,115,6,0,0, + 0,124,0,106,0,83,0,41,1,122,58,82,101,116,117,114, + 110,32,116,104,101,32,112,97,116,104,32,116,111,32,116,104, + 101,32,115,111,117,114,99,101,32,102,105,108,101,32,97,115, + 32,102,111,117,110,100,32,98,121,32,116,104,101,32,102,105, + 110,100,101,114,46,41,1,114,37,0,0,0,41,2,114,104, + 0,0,0,114,123,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,114,155,0,0,0,176,3,0,0, + 115,2,0,0,0,0,3,122,32,69,120,116,101,110,115,105, + 111,110,70,105,108,101,76,111,97,100,101,114,46,103,101,116, + 95,102,105,108,101,110,97,109,101,78,41,14,114,109,0,0, + 0,114,108,0,0,0,114,110,0,0,0,114,111,0,0,0, + 114,182,0,0,0,114,210,0,0,0,114,212,0,0,0,114, + 183,0,0,0,114,188,0,0,0,114,157,0,0,0,114,184, + 0,0,0,114,199,0,0,0,114,120,0,0,0,114,155,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,114,221,0,0,0,129,3,0,0,115, + 20,0,0,0,8,6,4,2,8,4,8,4,8,3,8,8, + 8,6,8,6,8,4,8,4,114,221,0,0,0,99,0,0, + 0,0,0,0,0,0,0,0,0,0,2,0,0,0,64,0, + 0,0,115,96,0,0,0,101,0,90,1,100,0,90,2,100, + 1,90,3,100,2,100,3,132,0,90,4,100,4,100,5,132, + 0,90,5,100,6,100,7,132,0,90,6,100,8,100,9,132, + 0,90,7,100,10,100,11,132,0,90,8,100,12,100,13,132, + 0,90,9,100,14,100,15,132,0,90,10,100,16,100,17,132, + 0,90,11,100,18,100,19,132,0,90,12,100,20,100,21,132, + 0,90,13,100,22,83,0,41,23,218,14,95,78,97,109,101, + 115,112,97,99,101,80,97,116,104,97,38,1,0,0,82,101, + 112,114,101,115,101,110,116,115,32,97,32,110,97,109,101,115, + 112,97,99,101,32,112,97,99,107,97,103,101,39,115,32,112, + 97,116,104,46,32,32,73,116,32,117,115,101,115,32,116,104, + 101,32,109,111,100,117,108,101,32,110,97,109,101,10,32,32, + 32,32,116,111,32,102,105,110,100,32,105,116,115,32,112,97, + 114,101,110,116,32,109,111,100,117,108,101,44,32,97,110,100, + 32,102,114,111,109,32,116,104,101,114,101,32,105,116,32,108, + 111,111,107,115,32,117,112,32,116,104,101,32,112,97,114,101, + 110,116,39,115,10,32,32,32,32,95,95,112,97,116,104,95, + 95,46,32,32,87,104,101,110,32,116,104,105,115,32,99,104, + 97,110,103,101,115,44,32,116,104,101,32,109,111,100,117,108, + 101,39,115,32,111,119,110,32,112,97,116,104,32,105,115,32, + 114,101,99,111,109,112,117,116,101,100,44,10,32,32,32,32, + 117,115,105,110,103,32,112,97,116,104,95,102,105,110,100,101, + 114,46,32,32,70,111,114,32,116,111,112,45,108,101,118,101, + 108,32,109,111,100,117,108,101,115,44,32,116,104,101,32,112, + 97,114,101,110,116,32,109,111,100,117,108,101,39,115,32,112, + 97,116,104,10,32,32,32,32,105,115,32,115,121,115,46,112, + 97,116,104,46,99,4,0,0,0,0,0,0,0,4,0,0, + 0,2,0,0,0,67,0,0,0,115,36,0,0,0,124,1, + 124,0,95,0,124,2,124,0,95,1,116,2,124,0,106,3, + 131,0,131,1,124,0,95,4,124,3,124,0,95,5,100,0, + 83,0,41,1,78,41,6,218,5,95,110,97,109,101,218,5, + 95,112,97,116,104,114,97,0,0,0,218,16,95,103,101,116, + 95,112,97,114,101,110,116,95,112,97,116,104,218,17,95,108, + 97,115,116,95,112,97,114,101,110,116,95,112,97,116,104,218, + 12,95,112,97,116,104,95,102,105,110,100,101,114,41,4,114, + 104,0,0,0,114,102,0,0,0,114,37,0,0,0,218,11, + 112,97,116,104,95,102,105,110,100,101,114,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,114,182,0,0,0,189, + 3,0,0,115,8,0,0,0,0,1,6,1,6,1,14,1, + 122,23,95,78,97,109,101,115,112,97,99,101,80,97,116,104, + 46,95,95,105,110,105,116,95,95,99,1,0,0,0,0,0, + 0,0,4,0,0,0,3,0,0,0,67,0,0,0,115,38, + 0,0,0,124,0,106,0,106,1,100,1,131,1,92,3,125, + 1,125,2,125,3,124,2,100,2,107,2,114,30,100,6,83, + 0,124,1,100,5,102,2,83,0,41,7,122,62,82,101,116, + 117,114,110,115,32,97,32,116,117,112,108,101,32,111,102,32, + 40,112,97,114,101,110,116,45,109,111,100,117,108,101,45,110, + 97,109,101,44,32,112,97,114,101,110,116,45,112,97,116,104, + 45,97,116,116,114,45,110,97,109,101,41,114,61,0,0,0, + 114,32,0,0,0,114,8,0,0,0,114,37,0,0,0,90, + 8,95,95,112,97,116,104,95,95,41,2,122,3,115,121,115, + 122,4,112,97,116,104,41,2,114,228,0,0,0,114,34,0, + 0,0,41,4,114,104,0,0,0,114,219,0,0,0,218,3, + 100,111,116,90,2,109,101,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,218,23,95,102,105,110,100,95,112,97, + 114,101,110,116,95,112,97,116,104,95,110,97,109,101,115,195, + 3,0,0,115,8,0,0,0,0,2,18,1,8,2,4,3, + 122,38,95,78,97,109,101,115,112,97,99,101,80,97,116,104, + 46,95,102,105,110,100,95,112,97,114,101,110,116,95,112,97, + 116,104,95,110,97,109,101,115,99,1,0,0,0,0,0,0, + 0,3,0,0,0,3,0,0,0,67,0,0,0,115,28,0, + 0,0,124,0,106,0,131,0,92,2,125,1,125,2,116,1, + 116,2,106,3,124,1,25,0,124,2,131,2,83,0,41,1, + 78,41,4,114,235,0,0,0,114,114,0,0,0,114,8,0, + 0,0,218,7,109,111,100,117,108,101,115,41,3,114,104,0, + 0,0,90,18,112,97,114,101,110,116,95,109,111,100,117,108, + 101,95,110,97,109,101,90,14,112,97,116,104,95,97,116,116, + 114,95,110,97,109,101,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,230,0,0,0,205,3,0,0,115,4, + 0,0,0,0,1,12,1,122,31,95,78,97,109,101,115,112, + 97,99,101,80,97,116,104,46,95,103,101,116,95,112,97,114, + 101,110,116,95,112,97,116,104,99,1,0,0,0,0,0,0, + 0,3,0,0,0,3,0,0,0,67,0,0,0,115,80,0, + 0,0,116,0,124,0,106,1,131,0,131,1,125,1,124,1, + 124,0,106,2,107,3,114,74,124,0,106,3,124,0,106,4, + 124,1,131,2,125,2,124,2,100,0,107,9,114,68,124,2, + 106,5,100,0,107,8,114,68,124,2,106,6,114,68,124,2, + 106,6,124,0,95,7,124,1,124,0,95,2,124,0,106,7, + 83,0,41,1,78,41,8,114,97,0,0,0,114,230,0,0, + 0,114,231,0,0,0,114,232,0,0,0,114,228,0,0,0, + 114,124,0,0,0,114,154,0,0,0,114,229,0,0,0,41, + 3,114,104,0,0,0,90,11,112,97,114,101,110,116,95,112, + 97,116,104,114,162,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,218,12,95,114,101,99,97,108,99, + 117,108,97,116,101,209,3,0,0,115,16,0,0,0,0,2, + 12,1,10,1,14,3,18,1,6,1,8,1,6,1,122,27, + 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, + 114,101,99,97,108,99,117,108,97,116,101,99,1,0,0,0, + 0,0,0,0,1,0,0,0,2,0,0,0,67,0,0,0, + 115,12,0,0,0,116,0,124,0,106,1,131,0,131,1,83, + 0,41,1,78,41,2,218,4,105,116,101,114,114,237,0,0, + 0,41,1,114,104,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,218,8,95,95,105,116,101,114,95, + 95,222,3,0,0,115,2,0,0,0,0,1,122,23,95,78, + 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,105, + 116,101,114,95,95,99,3,0,0,0,0,0,0,0,3,0, + 0,0,3,0,0,0,67,0,0,0,115,14,0,0,0,124, + 2,124,0,106,0,124,1,60,0,100,0,83,0,41,1,78, + 41,1,114,229,0,0,0,41,3,114,104,0,0,0,218,5, + 105,110,100,101,120,114,37,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,218,11,95,95,115,101,116, + 105,116,101,109,95,95,225,3,0,0,115,2,0,0,0,0, + 1,122,26,95,78,97,109,101,115,112,97,99,101,80,97,116, + 104,46,95,95,115,101,116,105,116,101,109,95,95,99,1,0, + 0,0,0,0,0,0,1,0,0,0,2,0,0,0,67,0, + 0,0,115,12,0,0,0,116,0,124,0,106,1,131,0,131, + 1,83,0,41,1,78,41,2,114,33,0,0,0,114,237,0, + 0,0,41,1,114,104,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,218,7,95,95,108,101,110,95, + 95,228,3,0,0,115,2,0,0,0,0,1,122,22,95,78, + 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,108, + 101,110,95,95,99,1,0,0,0,0,0,0,0,1,0,0, + 0,2,0,0,0,67,0,0,0,115,12,0,0,0,100,1, + 106,0,124,0,106,1,131,1,83,0,41,2,78,122,20,95, + 78,97,109,101,115,112,97,99,101,80,97,116,104,40,123,33, + 114,125,41,41,2,114,50,0,0,0,114,229,0,0,0,41, + 1,114,104,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,218,8,95,95,114,101,112,114,95,95,231, + 3,0,0,115,2,0,0,0,0,1,122,23,95,78,97,109, + 101,115,112,97,99,101,80,97,116,104,46,95,95,114,101,112, + 114,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, + 2,0,0,0,67,0,0,0,115,12,0,0,0,124,1,124, + 0,106,0,131,0,107,6,83,0,41,1,78,41,1,114,237, + 0,0,0,41,2,114,104,0,0,0,218,4,105,116,101,109, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, + 12,95,95,99,111,110,116,97,105,110,115,95,95,234,3,0, + 0,115,2,0,0,0,0,1,122,27,95,78,97,109,101,115, + 112,97,99,101,80,97,116,104,46,95,95,99,111,110,116,97, + 105,110,115,95,95,99,2,0,0,0,0,0,0,0,2,0, + 0,0,2,0,0,0,67,0,0,0,115,16,0,0,0,124, + 0,106,0,106,1,124,1,131,1,1,0,100,0,83,0,41, + 1,78,41,2,114,229,0,0,0,114,161,0,0,0,41,2, + 114,104,0,0,0,114,244,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,161,0,0,0,237,3, + 0,0,115,2,0,0,0,0,1,122,21,95,78,97,109,101, + 115,112,97,99,101,80,97,116,104,46,97,112,112,101,110,100, + 78,41,14,114,109,0,0,0,114,108,0,0,0,114,110,0, + 0,0,114,111,0,0,0,114,182,0,0,0,114,235,0,0, + 0,114,230,0,0,0,114,237,0,0,0,114,239,0,0,0, + 114,241,0,0,0,114,242,0,0,0,114,243,0,0,0,114, + 245,0,0,0,114,161,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,227,0, + 0,0,182,3,0,0,115,22,0,0,0,8,5,4,2,8, + 6,8,10,8,4,8,13,8,3,8,3,8,3,8,3,8, + 3,114,227,0,0,0,99,0,0,0,0,0,0,0,0,0, + 0,0,0,3,0,0,0,64,0,0,0,115,80,0,0,0, + 101,0,90,1,100,0,90,2,100,1,100,2,132,0,90,3, + 101,4,100,3,100,4,132,0,131,1,90,5,100,5,100,6, + 132,0,90,6,100,7,100,8,132,0,90,7,100,9,100,10, + 132,0,90,8,100,11,100,12,132,0,90,9,100,13,100,14, + 132,0,90,10,100,15,100,16,132,0,90,11,100,17,83,0, + 41,18,218,16,95,78,97,109,101,115,112,97,99,101,76,111, + 97,100,101,114,99,4,0,0,0,0,0,0,0,4,0,0, + 0,4,0,0,0,67,0,0,0,115,18,0,0,0,116,0, + 124,1,124,2,124,3,131,3,124,0,95,1,100,0,83,0, + 41,1,78,41,2,114,227,0,0,0,114,229,0,0,0,41, + 4,114,104,0,0,0,114,102,0,0,0,114,37,0,0,0, + 114,233,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,182,0,0,0,243,3,0,0,115,2,0, + 0,0,0,1,122,25,95,78,97,109,101,115,112,97,99,101, + 76,111,97,100,101,114,46,95,95,105,110,105,116,95,95,99, + 2,0,0,0,0,0,0,0,2,0,0,0,2,0,0,0, + 67,0,0,0,115,12,0,0,0,100,1,106,0,124,1,106, + 1,131,1,83,0,41,2,122,115,82,101,116,117,114,110,32, + 114,101,112,114,32,102,111,114,32,116,104,101,32,109,111,100, + 117,108,101,46,10,10,32,32,32,32,32,32,32,32,84,104, + 101,32,109,101,116,104,111,100,32,105,115,32,100,101,112,114, + 101,99,97,116,101,100,46,32,32,84,104,101,32,105,109,112, + 111,114,116,32,109,97,99,104,105,110,101,114,121,32,100,111, + 101,115,32,116,104,101,32,106,111,98,32,105,116,115,101,108, + 102,46,10,10,32,32,32,32,32,32,32,32,122,25,60,109, + 111,100,117,108,101,32,123,33,114,125,32,40,110,97,109,101, + 115,112,97,99,101,41,62,41,2,114,50,0,0,0,114,109, + 0,0,0,41,2,114,168,0,0,0,114,187,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,11, + 109,111,100,117,108,101,95,114,101,112,114,246,3,0,0,115, + 2,0,0,0,0,7,122,28,95,78,97,109,101,115,112,97, + 99,101,76,111,97,100,101,114,46,109,111,100,117,108,101,95, + 114,101,112,114,99,2,0,0,0,0,0,0,0,2,0,0, + 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, + 83,0,41,2,78,84,114,4,0,0,0,41,2,114,104,0, + 0,0,114,123,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,114,157,0,0,0,255,3,0,0,115, + 2,0,0,0,0,1,122,27,95,78,97,109,101,115,112,97, + 99,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, 97,103,101,99,2,0,0,0,0,0,0,0,2,0,0,0, 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, - 0,41,2,122,63,82,101,116,117,114,110,32,78,111,110,101, - 32,97,115,32,97,110,32,101,120,116,101,110,115,105,111,110, - 32,109,111,100,117,108,101,32,99,97,110,110,111,116,32,99, - 114,101,97,116,101,32,97,32,99,111,100,101,32,111,98,106, - 101,99,116,46,78,114,4,0,0,0,41,2,114,104,0,0, - 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,184,0,0,0,167,3,0,0,115,2, - 0,0,0,0,2,122,28,69,120,116,101,110,115,105,111,110, - 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, - 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, - 0,41,2,122,53,82,101,116,117,114,110,32,78,111,110,101, - 32,97,115,32,101,120,116,101,110,115,105,111,110,32,109,111, - 100,117,108,101,115,32,104,97,118,101,32,110,111,32,115,111, - 117,114,99,101,32,99,111,100,101,46,78,114,4,0,0,0, - 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,199,0,0,0, - 171,3,0,0,115,2,0,0,0,0,2,122,30,69,120,116, - 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, - 46,103,101,116,95,115,111,117,114,99,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,6,0,0,0,124,0,106,0,83,0,41,1,122,58,82, - 101,116,117,114,110,32,116,104,101,32,112,97,116,104,32,116, - 111,32,116,104,101,32,115,111,117,114,99,101,32,102,105,108, - 101,32,97,115,32,102,111,117,110,100,32,98,121,32,116,104, - 101,32,102,105,110,100,101,114,46,41,1,114,37,0,0,0, - 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,155,0,0,0, - 175,3,0,0,115,2,0,0,0,0,3,122,32,69,120,116, - 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, - 46,103,101,116,95,102,105,108,101,110,97,109,101,78,41,14, - 114,109,0,0,0,114,108,0,0,0,114,110,0,0,0,114, - 111,0,0,0,114,182,0,0,0,114,210,0,0,0,114,212, - 0,0,0,114,183,0,0,0,114,188,0,0,0,114,157,0, - 0,0,114,184,0,0,0,114,199,0,0,0,114,120,0,0, - 0,114,155,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,221,0,0,0,128, - 3,0,0,115,20,0,0,0,8,6,4,2,8,4,8,4, - 8,3,8,8,8,6,8,6,8,4,8,4,114,221,0,0, - 0,99,0,0,0,0,0,0,0,0,0,0,0,0,2,0, - 0,0,64,0,0,0,115,96,0,0,0,101,0,90,1,100, - 0,90,2,100,1,90,3,100,2,100,3,132,0,90,4,100, - 4,100,5,132,0,90,5,100,6,100,7,132,0,90,6,100, - 8,100,9,132,0,90,7,100,10,100,11,132,0,90,8,100, - 12,100,13,132,0,90,9,100,14,100,15,132,0,90,10,100, - 16,100,17,132,0,90,11,100,18,100,19,132,0,90,12,100, - 20,100,21,132,0,90,13,100,22,83,0,41,23,218,14,95, - 78,97,109,101,115,112,97,99,101,80,97,116,104,97,38,1, - 0,0,82,101,112,114,101,115,101,110,116,115,32,97,32,110, - 97,109,101,115,112,97,99,101,32,112,97,99,107,97,103,101, - 39,115,32,112,97,116,104,46,32,32,73,116,32,117,115,101, - 115,32,116,104,101,32,109,111,100,117,108,101,32,110,97,109, - 101,10,32,32,32,32,116,111,32,102,105,110,100,32,105,116, - 115,32,112,97,114,101,110,116,32,109,111,100,117,108,101,44, - 32,97,110,100,32,102,114,111,109,32,116,104,101,114,101,32, - 105,116,32,108,111,111,107,115,32,117,112,32,116,104,101,32, - 112,97,114,101,110,116,39,115,10,32,32,32,32,95,95,112, - 97,116,104,95,95,46,32,32,87,104,101,110,32,116,104,105, - 115,32,99,104,97,110,103,101,115,44,32,116,104,101,32,109, - 111,100,117,108,101,39,115,32,111,119,110,32,112,97,116,104, - 32,105,115,32,114,101,99,111,109,112,117,116,101,100,44,10, - 32,32,32,32,117,115,105,110,103,32,112,97,116,104,95,102, - 105,110,100,101,114,46,32,32,70,111,114,32,116,111,112,45, - 108,101,118,101,108,32,109,111,100,117,108,101,115,44,32,116, - 104,101,32,112,97,114,101,110,116,32,109,111,100,117,108,101, - 39,115,32,112,97,116,104,10,32,32,32,32,105,115,32,115, - 121,115,46,112,97,116,104,46,99,4,0,0,0,0,0,0, - 0,4,0,0,0,2,0,0,0,67,0,0,0,115,36,0, - 0,0,124,1,124,0,95,0,124,2,124,0,95,1,116,2, - 124,0,106,3,131,0,131,1,124,0,95,4,124,3,124,0, - 95,5,100,0,83,0,41,1,78,41,6,218,5,95,110,97, - 109,101,218,5,95,112,97,116,104,114,97,0,0,0,218,16, - 95,103,101,116,95,112,97,114,101,110,116,95,112,97,116,104, - 218,17,95,108,97,115,116,95,112,97,114,101,110,116,95,112, - 97,116,104,218,12,95,112,97,116,104,95,102,105,110,100,101, - 114,41,4,114,104,0,0,0,114,102,0,0,0,114,37,0, - 0,0,218,11,112,97,116,104,95,102,105,110,100,101,114,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,182, - 0,0,0,188,3,0,0,115,8,0,0,0,0,1,6,1, - 6,1,14,1,122,23,95,78,97,109,101,115,112,97,99,101, - 80,97,116,104,46,95,95,105,110,105,116,95,95,99,1,0, - 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, - 0,0,115,38,0,0,0,124,0,106,0,106,1,100,1,131, - 1,92,3,125,1,125,2,125,3,124,2,100,2,107,2,114, - 30,100,6,83,0,124,1,100,5,102,2,83,0,41,7,122, - 62,82,101,116,117,114,110,115,32,97,32,116,117,112,108,101, - 32,111,102,32,40,112,97,114,101,110,116,45,109,111,100,117, - 108,101,45,110,97,109,101,44,32,112,97,114,101,110,116,45, - 112,97,116,104,45,97,116,116,114,45,110,97,109,101,41,114, - 61,0,0,0,114,32,0,0,0,114,8,0,0,0,114,37, - 0,0,0,90,8,95,95,112,97,116,104,95,95,41,2,122, - 3,115,121,115,122,4,112,97,116,104,41,2,114,228,0,0, - 0,114,34,0,0,0,41,4,114,104,0,0,0,114,219,0, - 0,0,218,3,100,111,116,90,2,109,101,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,23,95,102,105,110, - 100,95,112,97,114,101,110,116,95,112,97,116,104,95,110,97, - 109,101,115,194,3,0,0,115,8,0,0,0,0,2,18,1, - 8,2,4,3,122,38,95,78,97,109,101,115,112,97,99,101, - 80,97,116,104,46,95,102,105,110,100,95,112,97,114,101,110, - 116,95,112,97,116,104,95,110,97,109,101,115,99,1,0,0, - 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, - 0,115,28,0,0,0,124,0,106,0,131,0,92,2,125,1, - 125,2,116,1,116,2,106,3,124,1,25,0,124,2,131,2, - 83,0,41,1,78,41,4,114,235,0,0,0,114,114,0,0, - 0,114,8,0,0,0,218,7,109,111,100,117,108,101,115,41, - 3,114,104,0,0,0,90,18,112,97,114,101,110,116,95,109, - 111,100,117,108,101,95,110,97,109,101,90,14,112,97,116,104, - 95,97,116,116,114,95,110,97,109,101,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,230,0,0,0,204,3, - 0,0,115,4,0,0,0,0,1,12,1,122,31,95,78,97, - 109,101,115,112,97,99,101,80,97,116,104,46,95,103,101,116, - 95,112,97,114,101,110,116,95,112,97,116,104,99,1,0,0, - 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, - 0,115,80,0,0,0,116,0,124,0,106,1,131,0,131,1, - 125,1,124,1,124,0,106,2,107,3,114,74,124,0,106,3, - 124,0,106,4,124,1,131,2,125,2,124,2,100,0,107,9, - 114,68,124,2,106,5,100,0,107,8,114,68,124,2,106,6, - 114,68,124,2,106,6,124,0,95,7,124,1,124,0,95,2, - 124,0,106,7,83,0,41,1,78,41,8,114,97,0,0,0, - 114,230,0,0,0,114,231,0,0,0,114,232,0,0,0,114, - 228,0,0,0,114,124,0,0,0,114,154,0,0,0,114,229, - 0,0,0,41,3,114,104,0,0,0,90,11,112,97,114,101, - 110,116,95,112,97,116,104,114,162,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,12,95,114,101, - 99,97,108,99,117,108,97,116,101,208,3,0,0,115,16,0, - 0,0,0,2,12,1,10,1,14,3,18,1,6,1,8,1, - 6,1,122,27,95,78,97,109,101,115,112,97,99,101,80,97, - 116,104,46,95,114,101,99,97,108,99,117,108,97,116,101,99, - 1,0,0,0,0,0,0,0,1,0,0,0,2,0,0,0, - 67,0,0,0,115,12,0,0,0,116,0,124,0,106,1,131, - 0,131,1,83,0,41,1,78,41,2,218,4,105,116,101,114, - 114,237,0,0,0,41,1,114,104,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,8,95,95,105, - 116,101,114,95,95,221,3,0,0,115,2,0,0,0,0,1, - 122,23,95,78,97,109,101,115,112,97,99,101,80,97,116,104, - 46,95,95,105,116,101,114,95,95,99,3,0,0,0,0,0, - 0,0,3,0,0,0,3,0,0,0,67,0,0,0,115,14, - 0,0,0,124,2,124,0,106,0,124,1,60,0,100,0,83, - 0,41,1,78,41,1,114,229,0,0,0,41,3,114,104,0, - 0,0,218,5,105,110,100,101,120,114,37,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,11,95, - 95,115,101,116,105,116,101,109,95,95,224,3,0,0,115,2, - 0,0,0,0,1,122,26,95,78,97,109,101,115,112,97,99, - 101,80,97,116,104,46,95,95,115,101,116,105,116,101,109,95, - 95,99,1,0,0,0,0,0,0,0,1,0,0,0,2,0, - 0,0,67,0,0,0,115,12,0,0,0,116,0,124,0,106, - 1,131,0,131,1,83,0,41,1,78,41,2,114,33,0,0, - 0,114,237,0,0,0,41,1,114,104,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,7,95,95, - 108,101,110,95,95,227,3,0,0,115,2,0,0,0,0,1, - 122,22,95,78,97,109,101,115,112,97,99,101,80,97,116,104, - 46,95,95,108,101,110,95,95,99,1,0,0,0,0,0,0, - 0,1,0,0,0,2,0,0,0,67,0,0,0,115,12,0, - 0,0,100,1,106,0,124,0,106,1,131,1,83,0,41,2, - 78,122,20,95,78,97,109,101,115,112,97,99,101,80,97,116, - 104,40,123,33,114,125,41,41,2,114,50,0,0,0,114,229, - 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,218,8,95,95,114,101,112, - 114,95,95,230,3,0,0,115,2,0,0,0,0,1,122,23, - 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, - 95,114,101,112,114,95,95,99,2,0,0,0,0,0,0,0, - 2,0,0,0,2,0,0,0,67,0,0,0,115,12,0,0, - 0,124,1,124,0,106,0,131,0,107,6,83,0,41,1,78, - 41,1,114,237,0,0,0,41,2,114,104,0,0,0,218,4, - 105,116,101,109,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,12,95,95,99,111,110,116,97,105,110,115,95, - 95,233,3,0,0,115,2,0,0,0,0,1,122,27,95,78, - 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,99, - 111,110,116,97,105,110,115,95,95,99,2,0,0,0,0,0, - 0,0,2,0,0,0,2,0,0,0,67,0,0,0,115,16, - 0,0,0,124,0,106,0,106,1,124,1,131,1,1,0,100, - 0,83,0,41,1,78,41,2,114,229,0,0,0,114,161,0, - 0,0,41,2,114,104,0,0,0,114,244,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,161,0, - 0,0,236,3,0,0,115,2,0,0,0,0,1,122,21,95, - 78,97,109,101,115,112,97,99,101,80,97,116,104,46,97,112, - 112,101,110,100,78,41,14,114,109,0,0,0,114,108,0,0, - 0,114,110,0,0,0,114,111,0,0,0,114,182,0,0,0, - 114,235,0,0,0,114,230,0,0,0,114,237,0,0,0,114, - 239,0,0,0,114,241,0,0,0,114,242,0,0,0,114,243, - 0,0,0,114,245,0,0,0,114,161,0,0,0,114,4,0, + 0,41,2,78,114,32,0,0,0,114,4,0,0,0,41,2, + 114,104,0,0,0,114,123,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,199,0,0,0,2,4, + 0,0,115,2,0,0,0,0,1,122,27,95,78,97,109,101, + 115,112,97,99,101,76,111,97,100,101,114,46,103,101,116,95, + 115,111,117,114,99,101,99,2,0,0,0,0,0,0,0,2, + 0,0,0,6,0,0,0,67,0,0,0,115,16,0,0,0, + 116,0,100,1,100,2,100,3,100,4,100,5,141,4,83,0, + 41,6,78,114,32,0,0,0,122,8,60,115,116,114,105,110, + 103,62,114,186,0,0,0,84,41,1,114,201,0,0,0,41, + 1,114,202,0,0,0,41,2,114,104,0,0,0,114,123,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,227,0,0,0,181,3,0,0,115,22,0,0,0,8, - 5,4,2,8,6,8,10,8,4,8,13,8,3,8,3,8, - 3,8,3,8,3,114,227,0,0,0,99,0,0,0,0,0, - 0,0,0,0,0,0,0,3,0,0,0,64,0,0,0,115, - 80,0,0,0,101,0,90,1,100,0,90,2,100,1,100,2, - 132,0,90,3,101,4,100,3,100,4,132,0,131,1,90,5, - 100,5,100,6,132,0,90,6,100,7,100,8,132,0,90,7, - 100,9,100,10,132,0,90,8,100,11,100,12,132,0,90,9, - 100,13,100,14,132,0,90,10,100,15,100,16,132,0,90,11, - 100,17,83,0,41,18,218,16,95,78,97,109,101,115,112,97, - 99,101,76,111,97,100,101,114,99,4,0,0,0,0,0,0, - 0,4,0,0,0,4,0,0,0,67,0,0,0,115,18,0, - 0,0,116,0,124,1,124,2,124,3,131,3,124,0,95,1, - 100,0,83,0,41,1,78,41,2,114,227,0,0,0,114,229, - 0,0,0,41,4,114,104,0,0,0,114,102,0,0,0,114, - 37,0,0,0,114,233,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,182,0,0,0,242,3,0, - 0,115,2,0,0,0,0,1,122,25,95,78,97,109,101,115, - 112,97,99,101,76,111,97,100,101,114,46,95,95,105,110,105, - 116,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, - 2,0,0,0,67,0,0,0,115,12,0,0,0,100,1,106, - 0,124,1,106,1,131,1,83,0,41,2,122,115,82,101,116, - 117,114,110,32,114,101,112,114,32,102,111,114,32,116,104,101, - 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, - 32,32,84,104,101,32,109,101,116,104,111,100,32,105,115,32, - 100,101,112,114,101,99,97,116,101,100,46,32,32,84,104,101, - 32,105,109,112,111,114,116,32,109,97,99,104,105,110,101,114, - 121,32,100,111,101,115,32,116,104,101,32,106,111,98,32,105, - 116,115,101,108,102,46,10,10,32,32,32,32,32,32,32,32, - 122,25,60,109,111,100,117,108,101,32,123,33,114,125,32,40, - 110,97,109,101,115,112,97,99,101,41,62,41,2,114,50,0, - 0,0,114,109,0,0,0,41,2,114,168,0,0,0,114,187, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,109,111,100,117,108,101,95,114,101,112,114,245, - 3,0,0,115,2,0,0,0,0,7,122,28,95,78,97,109, - 101,115,112,97,99,101,76,111,97,100,101,114,46,109,111,100, - 117,108,101,95,114,101,112,114,99,2,0,0,0,0,0,0, - 0,2,0,0,0,1,0,0,0,67,0,0,0,115,4,0, - 0,0,100,1,83,0,41,2,78,84,114,4,0,0,0,41, - 2,114,104,0,0,0,114,123,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,157,0,0,0,254, - 3,0,0,115,2,0,0,0,0,1,122,27,95,78,97,109, - 101,115,112,97,99,101,76,111,97,100,101,114,46,105,115,95, - 112,97,99,107,97,103,101,99,2,0,0,0,0,0,0,0, + 0,114,184,0,0,0,5,4,0,0,115,2,0,0,0,0, + 1,122,25,95,78,97,109,101,115,112,97,99,101,76,111,97, + 100,101,114,46,103,101,116,95,99,111,100,101,99,2,0,0, + 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, + 0,115,4,0,0,0,100,1,83,0,41,2,122,42,85,115, + 101,32,100,101,102,97,117,108,116,32,115,101,109,97,110,116, + 105,99,115,32,102,111,114,32,109,111,100,117,108,101,32,99, + 114,101,97,116,105,111,110,46,78,114,4,0,0,0,41,2, + 114,104,0,0,0,114,162,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,183,0,0,0,8,4, + 0,0,115,0,0,0,0,122,30,95,78,97,109,101,115,112, + 97,99,101,76,111,97,100,101,114,46,99,114,101,97,116,101, + 95,109,111,100,117,108,101,99,2,0,0,0,0,0,0,0, 2,0,0,0,1,0,0,0,67,0,0,0,115,4,0,0, - 0,100,1,83,0,41,2,78,114,32,0,0,0,114,4,0, - 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, - 0,0,1,4,0,0,115,2,0,0,0,0,1,122,27,95, + 0,100,0,83,0,41,1,78,114,4,0,0,0,41,2,114, + 104,0,0,0,114,187,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,188,0,0,0,11,4,0, + 0,115,2,0,0,0,0,1,122,28,95,78,97,109,101,115, + 112,97,99,101,76,111,97,100,101,114,46,101,120,101,99,95, + 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,2, + 0,0,0,3,0,0,0,67,0,0,0,115,26,0,0,0, + 116,0,106,1,100,1,124,0,106,2,131,2,1,0,116,0, + 106,3,124,0,124,1,131,2,83,0,41,2,122,98,76,111, + 97,100,32,97,32,110,97,109,101,115,112,97,99,101,32,109, + 111,100,117,108,101,46,10,10,32,32,32,32,32,32,32,32, + 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, + 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, + 101,120,101,99,95,109,111,100,117,108,101,40,41,32,105,110, + 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, + 122,38,110,97,109,101,115,112,97,99,101,32,109,111,100,117, + 108,101,32,108,111,97,100,101,100,32,119,105,116,104,32,112, + 97,116,104,32,123,33,114,125,41,4,114,118,0,0,0,114, + 133,0,0,0,114,229,0,0,0,114,189,0,0,0,41,2, + 114,104,0,0,0,114,123,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,190,0,0,0,14,4, + 0,0,115,6,0,0,0,0,7,6,1,8,1,122,28,95, 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, - 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, - 0,0,0,2,0,0,0,6,0,0,0,67,0,0,0,115, - 18,0,0,0,116,0,100,1,100,2,100,3,100,4,100,5, - 144,1,131,3,83,0,41,6,78,114,32,0,0,0,122,8, - 60,115,116,114,105,110,103,62,114,186,0,0,0,114,201,0, - 0,0,84,41,1,114,202,0,0,0,41,2,114,104,0,0, - 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,184,0,0,0,4,4,0,0,115,2, - 0,0,0,0,1,122,25,95,78,97,109,101,115,112,97,99, - 101,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, - 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, - 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, - 122,42,85,115,101,32,100,101,102,97,117,108,116,32,115,101, - 109,97,110,116,105,99,115,32,102,111,114,32,109,111,100,117, - 108,101,32,99,114,101,97,116,105,111,110,46,78,114,4,0, - 0,0,41,2,114,104,0,0,0,114,162,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,183,0, - 0,0,7,4,0,0,115,0,0,0,0,122,30,95,78,97, - 109,101,115,112,97,99,101,76,111,97,100,101,114,46,99,114, - 101,97,116,101,95,109,111,100,117,108,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,0,83,0,41,1,78,114,4,0,0, - 0,41,2,114,104,0,0,0,114,187,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,188,0,0, - 0,10,4,0,0,115,2,0,0,0,0,1,122,28,95,78, - 97,109,101,115,112,97,99,101,76,111,97,100,101,114,46,101, - 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, - 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, - 26,0,0,0,116,0,106,1,100,1,124,0,106,2,131,2, - 1,0,116,0,106,3,124,0,124,1,131,2,83,0,41,2, - 122,98,76,111,97,100,32,97,32,110,97,109,101,115,112,97, - 99,101,32,109,111,100,117,108,101,46,10,10,32,32,32,32, - 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, - 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, - 85,115,101,32,101,120,101,99,95,109,111,100,117,108,101,40, - 41,32,105,110,115,116,101,97,100,46,10,10,32,32,32,32, - 32,32,32,32,122,38,110,97,109,101,115,112,97,99,101,32, - 109,111,100,117,108,101,32,108,111,97,100,101,100,32,119,105, - 116,104,32,112,97,116,104,32,123,33,114,125,41,4,114,118, - 0,0,0,114,133,0,0,0,114,229,0,0,0,114,189,0, - 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,190,0, - 0,0,13,4,0,0,115,6,0,0,0,0,7,6,1,8, - 1,122,28,95,78,97,109,101,115,112,97,99,101,76,111,97, - 100,101,114,46,108,111,97,100,95,109,111,100,117,108,101,78, - 41,12,114,109,0,0,0,114,108,0,0,0,114,110,0,0, - 0,114,182,0,0,0,114,180,0,0,0,114,247,0,0,0, - 114,157,0,0,0,114,199,0,0,0,114,184,0,0,0,114, - 183,0,0,0,114,188,0,0,0,114,190,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,246,0,0,0,241,3,0,0,115,16,0,0,0, - 8,1,8,3,12,9,8,3,8,3,8,3,8,3,8,3, - 114,246,0,0,0,99,0,0,0,0,0,0,0,0,0,0, - 0,0,4,0,0,0,64,0,0,0,115,106,0,0,0,101, - 0,90,1,100,0,90,2,100,1,90,3,101,4,100,2,100, - 3,132,0,131,1,90,5,101,4,100,4,100,5,132,0,131, - 1,90,6,101,4,100,6,100,7,132,0,131,1,90,7,101, - 4,100,8,100,9,132,0,131,1,90,8,101,4,100,17,100, - 11,100,12,132,1,131,1,90,9,101,4,100,18,100,13,100, - 14,132,1,131,1,90,10,101,4,100,19,100,15,100,16,132, - 1,131,1,90,11,100,10,83,0,41,20,218,10,80,97,116, - 104,70,105,110,100,101,114,122,62,77,101,116,97,32,112,97, - 116,104,32,102,105,110,100,101,114,32,102,111,114,32,115,121, - 115,46,112,97,116,104,32,97,110,100,32,112,97,99,107,97, - 103,101,32,95,95,112,97,116,104,95,95,32,97,116,116,114, - 105,98,117,116,101,115,46,99,1,0,0,0,0,0,0,0, - 2,0,0,0,4,0,0,0,67,0,0,0,115,42,0,0, - 0,120,36,116,0,106,1,106,2,131,0,68,0,93,22,125, - 1,116,3,124,1,100,1,131,2,114,12,124,1,106,4,131, - 0,1,0,113,12,87,0,100,2,83,0,41,3,122,125,67, - 97,108,108,32,116,104,101,32,105,110,118,97,108,105,100,97, - 116,101,95,99,97,99,104,101,115,40,41,32,109,101,116,104, - 111,100,32,111,110,32,97,108,108,32,112,97,116,104,32,101, - 110,116,114,121,32,102,105,110,100,101,114,115,10,32,32,32, - 32,32,32,32,32,115,116,111,114,101,100,32,105,110,32,115, + 108,111,97,100,95,109,111,100,117,108,101,78,41,12,114,109, + 0,0,0,114,108,0,0,0,114,110,0,0,0,114,182,0, + 0,0,114,180,0,0,0,114,247,0,0,0,114,157,0,0, + 0,114,199,0,0,0,114,184,0,0,0,114,183,0,0,0, + 114,188,0,0,0,114,190,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,246, + 0,0,0,242,3,0,0,115,16,0,0,0,8,1,8,3, + 12,9,8,3,8,3,8,3,8,3,8,3,114,246,0,0, + 0,99,0,0,0,0,0,0,0,0,0,0,0,0,4,0, + 0,0,64,0,0,0,115,106,0,0,0,101,0,90,1,100, + 0,90,2,100,1,90,3,101,4,100,2,100,3,132,0,131, + 1,90,5,101,4,100,4,100,5,132,0,131,1,90,6,101, + 4,100,6,100,7,132,0,131,1,90,7,101,4,100,8,100, + 9,132,0,131,1,90,8,101,4,100,17,100,11,100,12,132, + 1,131,1,90,9,101,4,100,18,100,13,100,14,132,1,131, + 1,90,10,101,4,100,19,100,15,100,16,132,1,131,1,90, + 11,100,10,83,0,41,20,218,10,80,97,116,104,70,105,110, + 100,101,114,122,62,77,101,116,97,32,112,97,116,104,32,102, + 105,110,100,101,114,32,102,111,114,32,115,121,115,46,112,97, + 116,104,32,97,110,100,32,112,97,99,107,97,103,101,32,95, + 95,112,97,116,104,95,95,32,97,116,116,114,105,98,117,116, + 101,115,46,99,1,0,0,0,0,0,0,0,2,0,0,0, + 4,0,0,0,67,0,0,0,115,42,0,0,0,120,36,116, + 0,106,1,106,2,131,0,68,0,93,22,125,1,116,3,124, + 1,100,1,131,2,114,12,124,1,106,4,131,0,1,0,113, + 12,87,0,100,2,83,0,41,3,122,125,67,97,108,108,32, + 116,104,101,32,105,110,118,97,108,105,100,97,116,101,95,99, + 97,99,104,101,115,40,41,32,109,101,116,104,111,100,32,111, + 110,32,97,108,108,32,112,97,116,104,32,101,110,116,114,121, + 32,102,105,110,100,101,114,115,10,32,32,32,32,32,32,32, + 32,115,116,111,114,101,100,32,105,110,32,115,121,115,46,112, + 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, + 104,101,115,32,40,119,104,101,114,101,32,105,109,112,108,101, + 109,101,110,116,101,100,41,46,218,17,105,110,118,97,108,105, + 100,97,116,101,95,99,97,99,104,101,115,78,41,5,114,8, + 0,0,0,218,19,112,97,116,104,95,105,109,112,111,114,116, + 101,114,95,99,97,99,104,101,218,6,118,97,108,117,101,115, + 114,112,0,0,0,114,249,0,0,0,41,2,114,168,0,0, + 0,218,6,102,105,110,100,101,114,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,249,0,0,0,32,4,0, + 0,115,6,0,0,0,0,4,16,1,10,1,122,28,80,97, + 116,104,70,105,110,100,101,114,46,105,110,118,97,108,105,100, + 97,116,101,95,99,97,99,104,101,115,99,2,0,0,0,0, + 0,0,0,3,0,0,0,12,0,0,0,67,0,0,0,115, + 86,0,0,0,116,0,106,1,100,1,107,9,114,30,116,0, + 106,1,12,0,114,30,116,2,106,3,100,2,116,4,131,2, + 1,0,120,50,116,0,106,1,68,0,93,36,125,2,121,8, + 124,2,124,1,131,1,83,0,4,0,116,5,107,10,114,72, + 1,0,1,0,1,0,119,38,89,0,113,38,88,0,113,38, + 87,0,100,1,83,0,100,1,83,0,41,3,122,46,83,101, + 97,114,99,104,32,115,121,115,46,112,97,116,104,95,104,111, + 111,107,115,32,102,111,114,32,97,32,102,105,110,100,101,114, + 32,102,111,114,32,39,112,97,116,104,39,46,78,122,23,115, + 121,115,46,112,97,116,104,95,104,111,111,107,115,32,105,115, + 32,101,109,112,116,121,41,6,114,8,0,0,0,218,10,112, + 97,116,104,95,104,111,111,107,115,114,63,0,0,0,114,64, + 0,0,0,114,122,0,0,0,114,103,0,0,0,41,3,114, + 168,0,0,0,114,37,0,0,0,90,4,104,111,111,107,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,11, + 95,112,97,116,104,95,104,111,111,107,115,40,4,0,0,115, + 16,0,0,0,0,3,18,1,12,1,12,1,2,1,8,1, + 14,1,12,2,122,22,80,97,116,104,70,105,110,100,101,114, + 46,95,112,97,116,104,95,104,111,111,107,115,99,2,0,0, + 0,0,0,0,0,3,0,0,0,19,0,0,0,67,0,0, + 0,115,102,0,0,0,124,1,100,1,107,2,114,42,121,12, + 116,0,106,1,131,0,125,1,87,0,110,20,4,0,116,2, + 107,10,114,40,1,0,1,0,1,0,100,2,83,0,88,0, + 121,14,116,3,106,4,124,1,25,0,125,2,87,0,110,40, + 4,0,116,5,107,10,114,96,1,0,1,0,1,0,124,0, + 106,6,124,1,131,1,125,2,124,2,116,3,106,4,124,1, + 60,0,89,0,110,2,88,0,124,2,83,0,41,3,122,210, + 71,101,116,32,116,104,101,32,102,105,110,100,101,114,32,102, + 111,114,32,116,104,101,32,112,97,116,104,32,101,110,116,114, + 121,32,102,114,111,109,32,115,121,115,46,112,97,116,104,95, + 105,109,112,111,114,116,101,114,95,99,97,99,104,101,46,10, + 10,32,32,32,32,32,32,32,32,73,102,32,116,104,101,32, + 112,97,116,104,32,101,110,116,114,121,32,105,115,32,110,111, + 116,32,105,110,32,116,104,101,32,99,97,99,104,101,44,32, + 102,105,110,100,32,116,104,101,32,97,112,112,114,111,112,114, + 105,97,116,101,32,102,105,110,100,101,114,10,32,32,32,32, + 32,32,32,32,97,110,100,32,99,97,99,104,101,32,105,116, + 46,32,73,102,32,110,111,32,102,105,110,100,101,114,32,105, + 115,32,97,118,97,105,108,97,98,108,101,44,32,115,116,111, + 114,101,32,78,111,110,101,46,10,10,32,32,32,32,32,32, + 32,32,114,32,0,0,0,78,41,7,114,3,0,0,0,114, + 47,0,0,0,218,17,70,105,108,101,78,111,116,70,111,117, + 110,100,69,114,114,111,114,114,8,0,0,0,114,250,0,0, + 0,114,135,0,0,0,114,254,0,0,0,41,3,114,168,0, + 0,0,114,37,0,0,0,114,252,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,20,95,112,97, + 116,104,95,105,109,112,111,114,116,101,114,95,99,97,99,104, + 101,53,4,0,0,115,22,0,0,0,0,8,8,1,2,1, + 12,1,14,3,6,1,2,1,14,1,14,1,10,1,16,1, + 122,31,80,97,116,104,70,105,110,100,101,114,46,95,112,97, + 116,104,95,105,109,112,111,114,116,101,114,95,99,97,99,104, + 101,99,3,0,0,0,0,0,0,0,6,0,0,0,3,0, + 0,0,67,0,0,0,115,82,0,0,0,116,0,124,2,100, + 1,131,2,114,26,124,2,106,1,124,1,131,1,92,2,125, + 3,125,4,110,14,124,2,106,2,124,1,131,1,125,3,103, + 0,125,4,124,3,100,0,107,9,114,60,116,3,106,4,124, + 1,124,3,131,2,83,0,116,3,106,5,124,1,100,0,131, + 2,125,5,124,4,124,5,95,6,124,5,83,0,41,2,78, + 114,121,0,0,0,41,7,114,112,0,0,0,114,121,0,0, + 0,114,179,0,0,0,114,118,0,0,0,114,176,0,0,0, + 114,158,0,0,0,114,154,0,0,0,41,6,114,168,0,0, + 0,114,123,0,0,0,114,252,0,0,0,114,124,0,0,0, + 114,125,0,0,0,114,162,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,218,16,95,108,101,103,97, + 99,121,95,103,101,116,95,115,112,101,99,75,4,0,0,115, + 18,0,0,0,0,4,10,1,16,2,10,1,4,1,8,1, + 12,1,12,1,6,1,122,27,80,97,116,104,70,105,110,100, + 101,114,46,95,108,101,103,97,99,121,95,103,101,116,95,115, + 112,101,99,78,99,4,0,0,0,0,0,0,0,9,0,0, + 0,5,0,0,0,67,0,0,0,115,170,0,0,0,103,0, + 125,4,120,160,124,2,68,0,93,130,125,5,116,0,124,5, + 116,1,116,2,102,2,131,2,115,30,113,10,124,0,106,3, + 124,5,131,1,125,6,124,6,100,1,107,9,114,10,116,4, + 124,6,100,2,131,2,114,72,124,6,106,5,124,1,124,3, + 131,2,125,7,110,12,124,0,106,6,124,1,124,6,131,2, + 125,7,124,7,100,1,107,8,114,94,113,10,124,7,106,7, + 100,1,107,9,114,108,124,7,83,0,124,7,106,8,125,8, + 124,8,100,1,107,8,114,130,116,9,100,3,131,1,130,1, + 124,4,106,10,124,8,131,1,1,0,113,10,87,0,116,11, + 106,12,124,1,100,1,131,2,125,7,124,4,124,7,95,8, + 124,7,83,0,100,1,83,0,41,4,122,63,70,105,110,100, + 32,116,104,101,32,108,111,97,100,101,114,32,111,114,32,110, + 97,109,101,115,112,97,99,101,95,112,97,116,104,32,102,111, + 114,32,116,104,105,115,32,109,111,100,117,108,101,47,112,97, + 99,107,97,103,101,32,110,97,109,101,46,78,114,178,0,0, + 0,122,19,115,112,101,99,32,109,105,115,115,105,110,103,32, + 108,111,97,100,101,114,41,13,114,141,0,0,0,114,73,0, + 0,0,218,5,98,121,116,101,115,114,0,1,0,0,114,112, + 0,0,0,114,178,0,0,0,114,1,1,0,0,114,124,0, + 0,0,114,154,0,0,0,114,103,0,0,0,114,147,0,0, + 0,114,118,0,0,0,114,158,0,0,0,41,9,114,168,0, + 0,0,114,123,0,0,0,114,37,0,0,0,114,177,0,0, + 0,218,14,110,97,109,101,115,112,97,99,101,95,112,97,116, + 104,90,5,101,110,116,114,121,114,252,0,0,0,114,162,0, + 0,0,114,125,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,218,9,95,103,101,116,95,115,112,101, + 99,90,4,0,0,115,40,0,0,0,0,5,4,1,10,1, + 14,1,2,1,10,1,8,1,10,1,14,2,12,1,8,1, + 2,1,10,1,4,1,6,1,8,1,8,5,14,2,12,1, + 6,1,122,20,80,97,116,104,70,105,110,100,101,114,46,95, + 103,101,116,95,115,112,101,99,99,4,0,0,0,0,0,0, + 0,6,0,0,0,4,0,0,0,67,0,0,0,115,104,0, + 0,0,124,2,100,1,107,8,114,14,116,0,106,1,125,2, + 124,0,106,2,124,1,124,2,124,3,131,3,125,4,124,4, + 100,1,107,8,114,42,100,1,83,0,110,58,124,4,106,3, + 100,1,107,8,114,96,124,4,106,4,125,5,124,5,114,90, + 100,2,124,4,95,5,116,6,124,1,124,5,124,0,106,2, + 131,3,124,4,95,4,124,4,83,0,113,100,100,1,83,0, + 110,4,124,4,83,0,100,1,83,0,41,3,122,141,84,114, + 121,32,116,111,32,102,105,110,100,32,97,32,115,112,101,99, + 32,102,111,114,32,39,102,117,108,108,110,97,109,101,39,32, + 111,110,32,115,121,115,46,112,97,116,104,32,111,114,32,39, + 112,97,116,104,39,46,10,10,32,32,32,32,32,32,32,32, + 84,104,101,32,115,101,97,114,99,104,32,105,115,32,98,97, + 115,101,100,32,111,110,32,115,121,115,46,112,97,116,104,95, + 104,111,111,107,115,32,97,110,100,32,115,121,115,46,112,97, + 116,104,95,105,109,112,111,114,116,101,114,95,99,97,99,104, + 101,46,10,32,32,32,32,32,32,32,32,78,90,9,110,97, + 109,101,115,112,97,99,101,41,7,114,8,0,0,0,114,37, + 0,0,0,114,4,1,0,0,114,124,0,0,0,114,154,0, + 0,0,114,156,0,0,0,114,227,0,0,0,41,6,114,168, + 0,0,0,114,123,0,0,0,114,37,0,0,0,114,177,0, + 0,0,114,162,0,0,0,114,3,1,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,178,0,0,0, + 122,4,0,0,115,26,0,0,0,0,6,8,1,6,1,14, + 1,8,1,6,1,10,1,6,1,4,3,6,1,16,1,6, + 2,6,2,122,20,80,97,116,104,70,105,110,100,101,114,46, + 102,105,110,100,95,115,112,101,99,99,3,0,0,0,0,0, + 0,0,4,0,0,0,3,0,0,0,67,0,0,0,115,30, + 0,0,0,124,0,106,0,124,1,124,2,131,2,125,3,124, + 3,100,1,107,8,114,24,100,1,83,0,124,3,106,1,83, + 0,41,2,122,170,102,105,110,100,32,116,104,101,32,109,111, + 100,117,108,101,32,111,110,32,115,121,115,46,112,97,116,104, + 32,111,114,32,39,112,97,116,104,39,32,98,97,115,101,100, + 32,111,110,32,115,121,115,46,112,97,116,104,95,104,111,111, + 107,115,32,97,110,100,10,32,32,32,32,32,32,32,32,115, 121,115,46,112,97,116,104,95,105,109,112,111,114,116,101,114, - 95,99,97,99,104,101,115,32,40,119,104,101,114,101,32,105, - 109,112,108,101,109,101,110,116,101,100,41,46,218,17,105,110, - 118,97,108,105,100,97,116,101,95,99,97,99,104,101,115,78, - 41,5,114,8,0,0,0,218,19,112,97,116,104,95,105,109, - 112,111,114,116,101,114,95,99,97,99,104,101,218,6,118,97, - 108,117,101,115,114,112,0,0,0,114,249,0,0,0,41,2, - 114,168,0,0,0,218,6,102,105,110,100,101,114,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, - 0,31,4,0,0,115,6,0,0,0,0,4,16,1,10,1, - 122,28,80,97,116,104,70,105,110,100,101,114,46,105,110,118, - 97,108,105,100,97,116,101,95,99,97,99,104,101,115,99,2, - 0,0,0,0,0,0,0,3,0,0,0,12,0,0,0,67, - 0,0,0,115,86,0,0,0,116,0,106,1,100,1,107,9, - 114,30,116,0,106,1,12,0,114,30,116,2,106,3,100,2, - 116,4,131,2,1,0,120,50,116,0,106,1,68,0,93,36, - 125,2,121,8,124,2,124,1,131,1,83,0,4,0,116,5, - 107,10,114,72,1,0,1,0,1,0,119,38,89,0,113,38, - 88,0,113,38,87,0,100,1,83,0,100,1,83,0,41,3, - 122,46,83,101,97,114,99,104,32,115,121,115,46,112,97,116, - 104,95,104,111,111,107,115,32,102,111,114,32,97,32,102,105, - 110,100,101,114,32,102,111,114,32,39,112,97,116,104,39,46, - 78,122,23,115,121,115,46,112,97,116,104,95,104,111,111,107, - 115,32,105,115,32,101,109,112,116,121,41,6,114,8,0,0, - 0,218,10,112,97,116,104,95,104,111,111,107,115,114,63,0, - 0,0,114,64,0,0,0,114,122,0,0,0,114,103,0,0, - 0,41,3,114,168,0,0,0,114,37,0,0,0,90,4,104, - 111,111,107,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,95,112,97,116,104,95,104,111,111,107,115,39, - 4,0,0,115,16,0,0,0,0,3,18,1,12,1,12,1, - 2,1,8,1,14,1,12,2,122,22,80,97,116,104,70,105, - 110,100,101,114,46,95,112,97,116,104,95,104,111,111,107,115, - 99,2,0,0,0,0,0,0,0,3,0,0,0,19,0,0, - 0,67,0,0,0,115,102,0,0,0,124,1,100,1,107,2, - 114,42,121,12,116,0,106,1,131,0,125,1,87,0,110,20, - 4,0,116,2,107,10,114,40,1,0,1,0,1,0,100,2, - 83,0,88,0,121,14,116,3,106,4,124,1,25,0,125,2, - 87,0,110,40,4,0,116,5,107,10,114,96,1,0,1,0, - 1,0,124,0,106,6,124,1,131,1,125,2,124,2,116,3, - 106,4,124,1,60,0,89,0,110,2,88,0,124,2,83,0, - 41,3,122,210,71,101,116,32,116,104,101,32,102,105,110,100, - 101,114,32,102,111,114,32,116,104,101,32,112,97,116,104,32, - 101,110,116,114,121,32,102,114,111,109,32,115,121,115,46,112, - 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, - 104,101,46,10,10,32,32,32,32,32,32,32,32,73,102,32, - 116,104,101,32,112,97,116,104,32,101,110,116,114,121,32,105, - 115,32,110,111,116,32,105,110,32,116,104,101,32,99,97,99, - 104,101,44,32,102,105,110,100,32,116,104,101,32,97,112,112, - 114,111,112,114,105,97,116,101,32,102,105,110,100,101,114,10, - 32,32,32,32,32,32,32,32,97,110,100,32,99,97,99,104, - 101,32,105,116,46,32,73,102,32,110,111,32,102,105,110,100, - 101,114,32,105,115,32,97,118,97,105,108,97,98,108,101,44, - 32,115,116,111,114,101,32,78,111,110,101,46,10,10,32,32, - 32,32,32,32,32,32,114,32,0,0,0,78,41,7,114,3, - 0,0,0,114,47,0,0,0,218,17,70,105,108,101,78,111, - 116,70,111,117,110,100,69,114,114,111,114,114,8,0,0,0, - 114,250,0,0,0,114,135,0,0,0,114,254,0,0,0,41, - 3,114,168,0,0,0,114,37,0,0,0,114,252,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 20,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, - 99,97,99,104,101,52,4,0,0,115,22,0,0,0,0,8, - 8,1,2,1,12,1,14,3,6,1,2,1,14,1,14,1, - 10,1,16,1,122,31,80,97,116,104,70,105,110,100,101,114, - 46,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, - 99,97,99,104,101,99,3,0,0,0,0,0,0,0,6,0, - 0,0,3,0,0,0,67,0,0,0,115,82,0,0,0,116, - 0,124,2,100,1,131,2,114,26,124,2,106,1,124,1,131, - 1,92,2,125,3,125,4,110,14,124,2,106,2,124,1,131, - 1,125,3,103,0,125,4,124,3,100,0,107,9,114,60,116, - 3,106,4,124,1,124,3,131,2,83,0,116,3,106,5,124, - 1,100,0,131,2,125,5,124,4,124,5,95,6,124,5,83, - 0,41,2,78,114,121,0,0,0,41,7,114,112,0,0,0, - 114,121,0,0,0,114,179,0,0,0,114,118,0,0,0,114, - 176,0,0,0,114,158,0,0,0,114,154,0,0,0,41,6, - 114,168,0,0,0,114,123,0,0,0,114,252,0,0,0,114, - 124,0,0,0,114,125,0,0,0,114,162,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,16,95, - 108,101,103,97,99,121,95,103,101,116,95,115,112,101,99,74, - 4,0,0,115,18,0,0,0,0,4,10,1,16,2,10,1, - 4,1,8,1,12,1,12,1,6,1,122,27,80,97,116,104, - 70,105,110,100,101,114,46,95,108,101,103,97,99,121,95,103, - 101,116,95,115,112,101,99,78,99,4,0,0,0,0,0,0, - 0,9,0,0,0,5,0,0,0,67,0,0,0,115,170,0, - 0,0,103,0,125,4,120,160,124,2,68,0,93,130,125,5, - 116,0,124,5,116,1,116,2,102,2,131,2,115,30,113,10, - 124,0,106,3,124,5,131,1,125,6,124,6,100,1,107,9, - 114,10,116,4,124,6,100,2,131,2,114,72,124,6,106,5, - 124,1,124,3,131,2,125,7,110,12,124,0,106,6,124,1, - 124,6,131,2,125,7,124,7,100,1,107,8,114,94,113,10, - 124,7,106,7,100,1,107,9,114,108,124,7,83,0,124,7, - 106,8,125,8,124,8,100,1,107,8,114,130,116,9,100,3, - 131,1,130,1,124,4,106,10,124,8,131,1,1,0,113,10, - 87,0,116,11,106,12,124,1,100,1,131,2,125,7,124,4, - 124,7,95,8,124,7,83,0,100,1,83,0,41,4,122,63, - 70,105,110,100,32,116,104,101,32,108,111,97,100,101,114,32, - 111,114,32,110,97,109,101,115,112,97,99,101,95,112,97,116, - 104,32,102,111,114,32,116,104,105,115,32,109,111,100,117,108, - 101,47,112,97,99,107,97,103,101,32,110,97,109,101,46,78, - 114,178,0,0,0,122,19,115,112,101,99,32,109,105,115,115, - 105,110,103,32,108,111,97,100,101,114,41,13,114,141,0,0, - 0,114,73,0,0,0,218,5,98,121,116,101,115,114,0,1, - 0,0,114,112,0,0,0,114,178,0,0,0,114,1,1,0, - 0,114,124,0,0,0,114,154,0,0,0,114,103,0,0,0, - 114,147,0,0,0,114,118,0,0,0,114,158,0,0,0,41, - 9,114,168,0,0,0,114,123,0,0,0,114,37,0,0,0, - 114,177,0,0,0,218,14,110,97,109,101,115,112,97,99,101, - 95,112,97,116,104,90,5,101,110,116,114,121,114,252,0,0, - 0,114,162,0,0,0,114,125,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,9,95,103,101,116, - 95,115,112,101,99,89,4,0,0,115,40,0,0,0,0,5, - 4,1,10,1,14,1,2,1,10,1,8,1,10,1,14,2, - 12,1,8,1,2,1,10,1,4,1,6,1,8,1,8,5, - 14,2,12,1,6,1,122,20,80,97,116,104,70,105,110,100, - 101,114,46,95,103,101,116,95,115,112,101,99,99,4,0,0, - 0,0,0,0,0,6,0,0,0,4,0,0,0,67,0,0, - 0,115,104,0,0,0,124,2,100,1,107,8,114,14,116,0, - 106,1,125,2,124,0,106,2,124,1,124,2,124,3,131,3, - 125,4,124,4,100,1,107,8,114,42,100,1,83,0,110,58, - 124,4,106,3,100,1,107,8,114,96,124,4,106,4,125,5, - 124,5,114,90,100,2,124,4,95,5,116,6,124,1,124,5, - 124,0,106,2,131,3,124,4,95,4,124,4,83,0,113,100, - 100,1,83,0,110,4,124,4,83,0,100,1,83,0,41,3, - 122,141,84,114,121,32,116,111,32,102,105,110,100,32,97,32, - 115,112,101,99,32,102,111,114,32,39,102,117,108,108,110,97, - 109,101,39,32,111,110,32,115,121,115,46,112,97,116,104,32, - 111,114,32,39,112,97,116,104,39,46,10,10,32,32,32,32, - 32,32,32,32,84,104,101,32,115,101,97,114,99,104,32,105, - 115,32,98,97,115,101,100,32,111,110,32,115,121,115,46,112, - 97,116,104,95,104,111,111,107,115,32,97,110,100,32,115,121, - 115,46,112,97,116,104,95,105,109,112,111,114,116,101,114,95, - 99,97,99,104,101,46,10,32,32,32,32,32,32,32,32,78, - 90,9,110,97,109,101,115,112,97,99,101,41,7,114,8,0, - 0,0,114,37,0,0,0,114,4,1,0,0,114,124,0,0, - 0,114,154,0,0,0,114,156,0,0,0,114,227,0,0,0, - 41,6,114,168,0,0,0,114,123,0,0,0,114,37,0,0, - 0,114,177,0,0,0,114,162,0,0,0,114,3,1,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, - 178,0,0,0,121,4,0,0,115,26,0,0,0,0,6,8, - 1,6,1,14,1,8,1,6,1,10,1,6,1,4,3,6, - 1,16,1,6,2,6,2,122,20,80,97,116,104,70,105,110, - 100,101,114,46,102,105,110,100,95,115,112,101,99,99,3,0, - 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, - 0,0,115,30,0,0,0,124,0,106,0,124,1,124,2,131, - 2,125,3,124,3,100,1,107,8,114,24,100,1,83,0,124, - 3,106,1,83,0,41,2,122,170,102,105,110,100,32,116,104, - 101,32,109,111,100,117,108,101,32,111,110,32,115,121,115,46, - 112,97,116,104,32,111,114,32,39,112,97,116,104,39,32,98, - 97,115,101,100,32,111,110,32,115,121,115,46,112,97,116,104, - 95,104,111,111,107,115,32,97,110,100,10,32,32,32,32,32, - 32,32,32,115,121,115,46,112,97,116,104,95,105,109,112,111, - 114,116,101,114,95,99,97,99,104,101,46,10,10,32,32,32, + 95,99,97,99,104,101,46,10,10,32,32,32,32,32,32,32, + 32,84,104,105,115,32,109,101,116,104,111,100,32,105,115,32, + 100,101,112,114,101,99,97,116,101,100,46,32,32,85,115,101, + 32,102,105,110,100,95,115,112,101,99,40,41,32,105,110,115, + 116,101,97,100,46,10,10,32,32,32,32,32,32,32,32,78, + 41,2,114,178,0,0,0,114,124,0,0,0,41,4,114,168, + 0,0,0,114,123,0,0,0,114,37,0,0,0,114,162,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,179,0,0,0,146,4,0,0,115,8,0,0,0,0, + 8,12,1,8,1,4,1,122,22,80,97,116,104,70,105,110, + 100,101,114,46,102,105,110,100,95,109,111,100,117,108,101,41, + 1,78,41,2,78,78,41,1,78,41,12,114,109,0,0,0, + 114,108,0,0,0,114,110,0,0,0,114,111,0,0,0,114, + 180,0,0,0,114,249,0,0,0,114,254,0,0,0,114,0, + 1,0,0,114,1,1,0,0,114,4,1,0,0,114,178,0, + 0,0,114,179,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,248,0,0,0, + 28,4,0,0,115,22,0,0,0,8,2,4,2,12,8,12, + 13,12,22,12,15,2,1,12,31,2,1,12,23,2,1,114, + 248,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, + 0,3,0,0,0,64,0,0,0,115,90,0,0,0,101,0, + 90,1,100,0,90,2,100,1,90,3,100,2,100,3,132,0, + 90,4,100,4,100,5,132,0,90,5,101,6,90,7,100,6, + 100,7,132,0,90,8,100,8,100,9,132,0,90,9,100,19, + 100,11,100,12,132,1,90,10,100,13,100,14,132,0,90,11, + 101,12,100,15,100,16,132,0,131,1,90,13,100,17,100,18, + 132,0,90,14,100,10,83,0,41,20,218,10,70,105,108,101, + 70,105,110,100,101,114,122,172,70,105,108,101,45,98,97,115, + 101,100,32,102,105,110,100,101,114,46,10,10,32,32,32,32, + 73,110,116,101,114,97,99,116,105,111,110,115,32,119,105,116, + 104,32,116,104,101,32,102,105,108,101,32,115,121,115,116,101, + 109,32,97,114,101,32,99,97,99,104,101,100,32,102,111,114, + 32,112,101,114,102,111,114,109,97,110,99,101,44,32,98,101, + 105,110,103,10,32,32,32,32,114,101,102,114,101,115,104,101, + 100,32,119,104,101,110,32,116,104,101,32,100,105,114,101,99, + 116,111,114,121,32,116,104,101,32,102,105,110,100,101,114,32, + 105,115,32,104,97,110,100,108,105,110,103,32,104,97,115,32, + 98,101,101,110,32,109,111,100,105,102,105,101,100,46,10,10, + 32,32,32,32,99,2,0,0,0,0,0,0,0,5,0,0, + 0,5,0,0,0,7,0,0,0,115,88,0,0,0,103,0, + 125,3,120,40,124,2,68,0,93,32,92,2,137,0,125,4, + 124,3,106,0,135,0,102,1,100,1,100,2,132,8,124,4, + 68,0,131,1,131,1,1,0,113,10,87,0,124,3,124,0, + 95,1,124,1,112,58,100,3,124,0,95,2,100,6,124,0, + 95,3,116,4,131,0,124,0,95,5,116,4,131,0,124,0, + 95,6,100,5,83,0,41,7,122,154,73,110,105,116,105,97, + 108,105,122,101,32,119,105,116,104,32,116,104,101,32,112,97, + 116,104,32,116,111,32,115,101,97,114,99,104,32,111,110,32, + 97,110,100,32,97,32,118,97,114,105,97,98,108,101,32,110, + 117,109,98,101,114,32,111,102,10,32,32,32,32,32,32,32, + 32,50,45,116,117,112,108,101,115,32,99,111,110,116,97,105, + 110,105,110,103,32,116,104,101,32,108,111,97,100,101,114,32, + 97,110,100,32,116,104,101,32,102,105,108,101,32,115,117,102, + 102,105,120,101,115,32,116,104,101,32,108,111,97,100,101,114, + 10,32,32,32,32,32,32,32,32,114,101,99,111,103,110,105, + 122,101,115,46,99,1,0,0,0,0,0,0,0,2,0,0, + 0,3,0,0,0,51,0,0,0,115,22,0,0,0,124,0, + 93,14,125,1,124,1,136,0,102,2,86,0,1,0,113,2, + 100,0,83,0,41,1,78,114,4,0,0,0,41,2,114,24, + 0,0,0,114,222,0,0,0,41,1,114,124,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,224,0,0,0,175,4, + 0,0,115,2,0,0,0,4,0,122,38,70,105,108,101,70, + 105,110,100,101,114,46,95,95,105,110,105,116,95,95,46,60, + 108,111,99,97,108,115,62,46,60,103,101,110,101,120,112,114, + 62,114,61,0,0,0,114,31,0,0,0,78,114,91,0,0, + 0,41,7,114,147,0,0,0,218,8,95,108,111,97,100,101, + 114,115,114,37,0,0,0,218,11,95,112,97,116,104,95,109, + 116,105,109,101,218,3,115,101,116,218,11,95,112,97,116,104, + 95,99,97,99,104,101,218,19,95,114,101,108,97,120,101,100, + 95,112,97,116,104,95,99,97,99,104,101,41,5,114,104,0, + 0,0,114,37,0,0,0,218,14,108,111,97,100,101,114,95, + 100,101,116,97,105,108,115,90,7,108,111,97,100,101,114,115, + 114,164,0,0,0,114,4,0,0,0,41,1,114,124,0,0, + 0,114,6,0,0,0,114,182,0,0,0,169,4,0,0,115, + 16,0,0,0,0,4,4,1,14,1,28,1,6,2,10,1, + 6,1,8,1,122,19,70,105,108,101,70,105,110,100,101,114, + 46,95,95,105,110,105,116,95,95,99,1,0,0,0,0,0, + 0,0,1,0,0,0,2,0,0,0,67,0,0,0,115,10, + 0,0,0,100,3,124,0,95,0,100,2,83,0,41,4,122, + 31,73,110,118,97,108,105,100,97,116,101,32,116,104,101,32, + 100,105,114,101,99,116,111,114,121,32,109,116,105,109,101,46, + 114,31,0,0,0,78,114,91,0,0,0,41,1,114,7,1, + 0,0,41,1,114,104,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,249,0,0,0,183,4,0, + 0,115,2,0,0,0,0,2,122,28,70,105,108,101,70,105, + 110,100,101,114,46,105,110,118,97,108,105,100,97,116,101,95, + 99,97,99,104,101,115,99,2,0,0,0,0,0,0,0,3, + 0,0,0,2,0,0,0,67,0,0,0,115,42,0,0,0, + 124,0,106,0,124,1,131,1,125,2,124,2,100,1,107,8, + 114,26,100,1,103,0,102,2,83,0,124,2,106,1,124,2, + 106,2,112,38,103,0,102,2,83,0,41,2,122,197,84,114, + 121,32,116,111,32,102,105,110,100,32,97,32,108,111,97,100, + 101,114,32,102,111,114,32,116,104,101,32,115,112,101,99,105, + 102,105,101,100,32,109,111,100,117,108,101,44,32,111,114,32, + 116,104,101,32,110,97,109,101,115,112,97,99,101,10,32,32, + 32,32,32,32,32,32,112,97,99,107,97,103,101,32,112,111, + 114,116,105,111,110,115,46,32,82,101,116,117,114,110,115,32, + 40,108,111,97,100,101,114,44,32,108,105,115,116,45,111,102, + 45,112,111,114,116,105,111,110,115,41,46,10,10,32,32,32, 32,32,32,32,32,84,104,105,115,32,109,101,116,104,111,100, 32,105,115,32,100,101,112,114,101,99,97,116,101,100,46,32, 32,85,115,101,32,102,105,110,100,95,115,112,101,99,40,41, 32,105,110,115,116,101,97,100,46,10,10,32,32,32,32,32, - 32,32,32,78,41,2,114,178,0,0,0,114,124,0,0,0, - 41,4,114,168,0,0,0,114,123,0,0,0,114,37,0,0, + 32,32,32,78,41,3,114,178,0,0,0,114,124,0,0,0, + 114,154,0,0,0,41,3,114,104,0,0,0,114,123,0,0, 0,114,162,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,179,0,0,0,145,4,0,0,115,8, - 0,0,0,0,8,12,1,8,1,4,1,122,22,80,97,116, - 104,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, - 117,108,101,41,1,78,41,2,78,78,41,1,78,41,12,114, - 109,0,0,0,114,108,0,0,0,114,110,0,0,0,114,111, - 0,0,0,114,180,0,0,0,114,249,0,0,0,114,254,0, - 0,0,114,0,1,0,0,114,1,1,0,0,114,4,1,0, - 0,114,178,0,0,0,114,179,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, - 248,0,0,0,27,4,0,0,115,22,0,0,0,8,2,4, - 2,12,8,12,13,12,22,12,15,2,1,12,31,2,1,12, - 23,2,1,114,248,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,3,0,0,0,64,0,0,0,115,90,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 100,3,132,0,90,4,100,4,100,5,132,0,90,5,101,6, - 90,7,100,6,100,7,132,0,90,8,100,8,100,9,132,0, - 90,9,100,19,100,11,100,12,132,1,90,10,100,13,100,14, - 132,0,90,11,101,12,100,15,100,16,132,0,131,1,90,13, - 100,17,100,18,132,0,90,14,100,10,83,0,41,20,218,10, - 70,105,108,101,70,105,110,100,101,114,122,172,70,105,108,101, - 45,98,97,115,101,100,32,102,105,110,100,101,114,46,10,10, - 32,32,32,32,73,110,116,101,114,97,99,116,105,111,110,115, - 32,119,105,116,104,32,116,104,101,32,102,105,108,101,32,115, - 121,115,116,101,109,32,97,114,101,32,99,97,99,104,101,100, - 32,102,111,114,32,112,101,114,102,111,114,109,97,110,99,101, - 44,32,98,101,105,110,103,10,32,32,32,32,114,101,102,114, - 101,115,104,101,100,32,119,104,101,110,32,116,104,101,32,100, - 105,114,101,99,116,111,114,121,32,116,104,101,32,102,105,110, - 100,101,114,32,105,115,32,104,97,110,100,108,105,110,103,32, - 104,97,115,32,98,101,101,110,32,109,111,100,105,102,105,101, - 100,46,10,10,32,32,32,32,99,2,0,0,0,0,0,0, - 0,5,0,0,0,5,0,0,0,7,0,0,0,115,88,0, - 0,0,103,0,125,3,120,40,124,2,68,0,93,32,92,2, - 137,0,125,4,124,3,106,0,135,0,102,1,100,1,100,2, - 132,8,124,4,68,0,131,1,131,1,1,0,113,10,87,0, - 124,3,124,0,95,1,124,1,112,58,100,3,124,0,95,2, - 100,6,124,0,95,3,116,4,131,0,124,0,95,5,116,4, - 131,0,124,0,95,6,100,5,83,0,41,7,122,154,73,110, - 105,116,105,97,108,105,122,101,32,119,105,116,104,32,116,104, - 101,32,112,97,116,104,32,116,111,32,115,101,97,114,99,104, - 32,111,110,32,97,110,100,32,97,32,118,97,114,105,97,98, - 108,101,32,110,117,109,98,101,114,32,111,102,10,32,32,32, - 32,32,32,32,32,50,45,116,117,112,108,101,115,32,99,111, - 110,116,97,105,110,105,110,103,32,116,104,101,32,108,111,97, - 100,101,114,32,97,110,100,32,116,104,101,32,102,105,108,101, - 32,115,117,102,102,105,120,101,115,32,116,104,101,32,108,111, - 97,100,101,114,10,32,32,32,32,32,32,32,32,114,101,99, - 111,103,110,105,122,101,115,46,99,1,0,0,0,0,0,0, - 0,2,0,0,0,3,0,0,0,51,0,0,0,115,22,0, - 0,0,124,0,93,14,125,1,124,1,136,0,102,2,86,0, - 1,0,113,2,100,0,83,0,41,1,78,114,4,0,0,0, - 41,2,114,24,0,0,0,114,222,0,0,0,41,1,114,124, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,224,0, - 0,0,174,4,0,0,115,2,0,0,0,4,0,122,38,70, - 105,108,101,70,105,110,100,101,114,46,95,95,105,110,105,116, - 95,95,46,60,108,111,99,97,108,115,62,46,60,103,101,110, - 101,120,112,114,62,114,61,0,0,0,114,31,0,0,0,78, - 114,91,0,0,0,41,7,114,147,0,0,0,218,8,95,108, - 111,97,100,101,114,115,114,37,0,0,0,218,11,95,112,97, - 116,104,95,109,116,105,109,101,218,3,115,101,116,218,11,95, - 112,97,116,104,95,99,97,99,104,101,218,19,95,114,101,108, - 97,120,101,100,95,112,97,116,104,95,99,97,99,104,101,41, - 5,114,104,0,0,0,114,37,0,0,0,218,14,108,111,97, - 100,101,114,95,100,101,116,97,105,108,115,90,7,108,111,97, - 100,101,114,115,114,164,0,0,0,114,4,0,0,0,41,1, - 114,124,0,0,0,114,6,0,0,0,114,182,0,0,0,168, - 4,0,0,115,16,0,0,0,0,4,4,1,14,1,28,1, - 6,2,10,1,6,1,8,1,122,19,70,105,108,101,70,105, - 110,100,101,114,46,95,95,105,110,105,116,95,95,99,1,0, - 0,0,0,0,0,0,1,0,0,0,2,0,0,0,67,0, - 0,0,115,10,0,0,0,100,3,124,0,95,0,100,2,83, - 0,41,4,122,31,73,110,118,97,108,105,100,97,116,101,32, - 116,104,101,32,100,105,114,101,99,116,111,114,121,32,109,116, - 105,109,101,46,114,31,0,0,0,78,114,91,0,0,0,41, - 1,114,7,1,0,0,41,1,114,104,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, - 0,182,4,0,0,115,2,0,0,0,0,2,122,28,70,105, - 108,101,70,105,110,100,101,114,46,105,110,118,97,108,105,100, - 97,116,101,95,99,97,99,104,101,115,99,2,0,0,0,0, - 0,0,0,3,0,0,0,2,0,0,0,67,0,0,0,115, - 42,0,0,0,124,0,106,0,124,1,131,1,125,2,124,2, - 100,1,107,8,114,26,100,1,103,0,102,2,83,0,124,2, - 106,1,124,2,106,2,112,38,103,0,102,2,83,0,41,2, - 122,197,84,114,121,32,116,111,32,102,105,110,100,32,97,32, - 108,111,97,100,101,114,32,102,111,114,32,116,104,101,32,115, - 112,101,99,105,102,105,101,100,32,109,111,100,117,108,101,44, - 32,111,114,32,116,104,101,32,110,97,109,101,115,112,97,99, - 101,10,32,32,32,32,32,32,32,32,112,97,99,107,97,103, - 101,32,112,111,114,116,105,111,110,115,46,32,82,101,116,117, - 114,110,115,32,40,108,111,97,100,101,114,44,32,108,105,115, - 116,45,111,102,45,112,111,114,116,105,111,110,115,41,46,10, - 10,32,32,32,32,32,32,32,32,84,104,105,115,32,109,101, - 116,104,111,100,32,105,115,32,100,101,112,114,101,99,97,116, - 101,100,46,32,32,85,115,101,32,102,105,110,100,95,115,112, - 101,99,40,41,32,105,110,115,116,101,97,100,46,10,10,32, - 32,32,32,32,32,32,32,78,41,3,114,178,0,0,0,114, - 124,0,0,0,114,154,0,0,0,41,3,114,104,0,0,0, - 114,123,0,0,0,114,162,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,121,0,0,0,188,4, - 0,0,115,8,0,0,0,0,7,10,1,8,1,8,1,122, - 22,70,105,108,101,70,105,110,100,101,114,46,102,105,110,100, - 95,108,111,97,100,101,114,99,6,0,0,0,0,0,0,0, - 7,0,0,0,7,0,0,0,67,0,0,0,115,30,0,0, - 0,124,1,124,2,124,3,131,2,125,6,116,0,124,2,124, - 3,100,1,124,6,100,2,124,4,144,2,131,2,83,0,41, - 3,78,114,124,0,0,0,114,154,0,0,0,41,1,114,165, - 0,0,0,41,7,114,104,0,0,0,114,163,0,0,0,114, - 123,0,0,0,114,37,0,0,0,90,4,115,109,115,108,114, - 177,0,0,0,114,124,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,4,1,0,0,200,4,0, - 0,115,6,0,0,0,0,1,10,1,12,1,122,20,70,105, - 108,101,70,105,110,100,101,114,46,95,103,101,116,95,115,112, - 101,99,78,99,3,0,0,0,0,0,0,0,14,0,0,0, - 15,0,0,0,67,0,0,0,115,100,1,0,0,100,1,125, - 3,124,1,106,0,100,2,131,1,100,3,25,0,125,4,121, - 24,116,1,124,0,106,2,112,34,116,3,106,4,131,0,131, - 1,106,5,125,5,87,0,110,24,4,0,116,6,107,10,114, - 66,1,0,1,0,1,0,100,10,125,5,89,0,110,2,88, - 0,124,5,124,0,106,7,107,3,114,92,124,0,106,8,131, - 0,1,0,124,5,124,0,95,7,116,9,131,0,114,114,124, - 0,106,10,125,6,124,4,106,11,131,0,125,7,110,10,124, - 0,106,12,125,6,124,4,125,7,124,7,124,6,107,6,114, - 218,116,13,124,0,106,2,124,4,131,2,125,8,120,72,124, - 0,106,14,68,0,93,54,92,2,125,9,125,10,100,5,124, - 9,23,0,125,11,116,13,124,8,124,11,131,2,125,12,116, - 15,124,12,131,1,114,152,124,0,106,16,124,10,124,1,124, - 12,124,8,103,1,124,2,131,5,83,0,113,152,87,0,116, - 17,124,8,131,1,125,3,120,90,124,0,106,14,68,0,93, - 80,92,2,125,9,125,10,116,13,124,0,106,2,124,4,124, - 9,23,0,131,2,125,12,116,18,106,19,100,6,124,12,100, - 7,100,3,144,1,131,2,1,0,124,7,124,9,23,0,124, - 6,107,6,114,226,116,15,124,12,131,1,114,226,124,0,106, - 16,124,10,124,1,124,12,100,8,124,2,131,5,83,0,113, - 226,87,0,124,3,144,1,114,96,116,18,106,19,100,9,124, - 8,131,2,1,0,116,18,106,20,124,1,100,8,131,2,125, - 13,124,8,103,1,124,13,95,21,124,13,83,0,100,8,83, - 0,41,11,122,111,84,114,121,32,116,111,32,102,105,110,100, - 32,97,32,115,112,101,99,32,102,111,114,32,116,104,101,32, - 115,112,101,99,105,102,105,101,100,32,109,111,100,117,108,101, - 46,10,10,32,32,32,32,32,32,32,32,82,101,116,117,114, - 110,115,32,116,104,101,32,109,97,116,99,104,105,110,103,32, - 115,112,101,99,44,32,111,114,32,78,111,110,101,32,105,102, - 32,110,111,116,32,102,111,117,110,100,46,10,32,32,32,32, - 32,32,32,32,70,114,61,0,0,0,114,59,0,0,0,114, - 31,0,0,0,114,182,0,0,0,122,9,116,114,121,105,110, - 103,32,123,125,90,9,118,101,114,98,111,115,105,116,121,78, - 122,25,112,111,115,115,105,98,108,101,32,110,97,109,101,115, - 112,97,99,101,32,102,111,114,32,123,125,114,91,0,0,0, - 41,22,114,34,0,0,0,114,41,0,0,0,114,37,0,0, - 0,114,3,0,0,0,114,47,0,0,0,114,216,0,0,0, - 114,42,0,0,0,114,7,1,0,0,218,11,95,102,105,108, - 108,95,99,97,99,104,101,114,7,0,0,0,114,10,1,0, - 0,114,92,0,0,0,114,9,1,0,0,114,30,0,0,0, - 114,6,1,0,0,114,46,0,0,0,114,4,1,0,0,114, - 48,0,0,0,114,118,0,0,0,114,133,0,0,0,114,158, - 0,0,0,114,154,0,0,0,41,14,114,104,0,0,0,114, - 123,0,0,0,114,177,0,0,0,90,12,105,115,95,110,97, - 109,101,115,112,97,99,101,90,11,116,97,105,108,95,109,111, - 100,117,108,101,114,130,0,0,0,90,5,99,97,99,104,101, - 90,12,99,97,99,104,101,95,109,111,100,117,108,101,90,9, - 98,97,115,101,95,112,97,116,104,114,222,0,0,0,114,163, - 0,0,0,90,13,105,110,105,116,95,102,105,108,101,110,97, - 109,101,90,9,102,117,108,108,95,112,97,116,104,114,162,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,178,0,0,0,205,4,0,0,115,70,0,0,0,0, - 5,4,1,14,1,2,1,24,1,14,1,10,1,10,1,8, - 1,6,2,6,1,6,1,10,2,6,1,4,2,8,1,12, - 1,16,1,8,1,10,1,8,1,24,4,8,2,16,1,16, - 1,18,1,12,1,8,1,10,1,12,1,6,1,12,1,12, - 1,8,1,4,1,122,20,70,105,108,101,70,105,110,100,101, - 114,46,102,105,110,100,95,115,112,101,99,99,1,0,0,0, - 0,0,0,0,9,0,0,0,13,0,0,0,67,0,0,0, - 115,194,0,0,0,124,0,106,0,125,1,121,22,116,1,106, - 2,124,1,112,22,116,1,106,3,131,0,131,1,125,2,87, - 0,110,30,4,0,116,4,116,5,116,6,102,3,107,10,114, - 58,1,0,1,0,1,0,103,0,125,2,89,0,110,2,88, - 0,116,7,106,8,106,9,100,1,131,1,115,84,116,10,124, - 2,131,1,124,0,95,11,110,78,116,10,131,0,125,3,120, - 64,124,2,68,0,93,56,125,4,124,4,106,12,100,2,131, - 1,92,3,125,5,125,6,125,7,124,6,114,138,100,3,106, - 13,124,5,124,7,106,14,131,0,131,2,125,8,110,4,124, - 5,125,8,124,3,106,15,124,8,131,1,1,0,113,96,87, - 0,124,3,124,0,95,11,116,7,106,8,106,9,116,16,131, - 1,114,190,100,4,100,5,132,0,124,2,68,0,131,1,124, - 0,95,17,100,6,83,0,41,7,122,68,70,105,108,108,32, - 116,104,101,32,99,97,99,104,101,32,111,102,32,112,111,116, - 101,110,116,105,97,108,32,109,111,100,117,108,101,115,32,97, - 110,100,32,112,97,99,107,97,103,101,115,32,102,111,114,32, - 116,104,105,115,32,100,105,114,101,99,116,111,114,121,46,114, - 0,0,0,0,114,61,0,0,0,122,5,123,125,46,123,125, - 99,1,0,0,0,0,0,0,0,2,0,0,0,3,0,0, - 0,83,0,0,0,115,20,0,0,0,104,0,124,0,93,12, - 125,1,124,1,106,0,131,0,146,2,113,4,83,0,114,4, - 0,0,0,41,1,114,92,0,0,0,41,2,114,24,0,0, - 0,90,2,102,110,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,250,9,60,115,101,116,99,111,109,112,62,26, - 5,0,0,115,2,0,0,0,6,0,122,41,70,105,108,101, - 70,105,110,100,101,114,46,95,102,105,108,108,95,99,97,99, - 104,101,46,60,108,111,99,97,108,115,62,46,60,115,101,116, - 99,111,109,112,62,78,41,18,114,37,0,0,0,114,3,0, - 0,0,90,7,108,105,115,116,100,105,114,114,47,0,0,0, - 114,255,0,0,0,218,15,80,101,114,109,105,115,115,105,111, - 110,69,114,114,111,114,218,18,78,111,116,65,68,105,114,101, - 99,116,111,114,121,69,114,114,111,114,114,8,0,0,0,114, - 9,0,0,0,114,10,0,0,0,114,8,1,0,0,114,9, - 1,0,0,114,87,0,0,0,114,50,0,0,0,114,92,0, - 0,0,218,3,97,100,100,114,11,0,0,0,114,10,1,0, - 0,41,9,114,104,0,0,0,114,37,0,0,0,90,8,99, - 111,110,116,101,110,116,115,90,21,108,111,119,101,114,95,115, - 117,102,102,105,120,95,99,111,110,116,101,110,116,115,114,244, - 0,0,0,114,102,0,0,0,114,234,0,0,0,114,222,0, - 0,0,90,8,110,101,119,95,110,97,109,101,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,12,1,0,0, - 253,4,0,0,115,34,0,0,0,0,2,6,1,2,1,22, - 1,20,3,10,3,12,1,12,7,6,1,10,1,16,1,4, - 1,18,2,4,1,14,1,6,1,12,1,122,22,70,105,108, - 101,70,105,110,100,101,114,46,95,102,105,108,108,95,99,97, - 99,104,101,99,1,0,0,0,0,0,0,0,3,0,0,0, - 3,0,0,0,7,0,0,0,115,18,0,0,0,135,0,135, - 1,102,2,100,1,100,2,132,8,125,2,124,2,83,0,41, - 3,97,20,1,0,0,65,32,99,108,97,115,115,32,109,101, - 116,104,111,100,32,119,104,105,99,104,32,114,101,116,117,114, - 110,115,32,97,32,99,108,111,115,117,114,101,32,116,111,32, - 117,115,101,32,111,110,32,115,121,115,46,112,97,116,104,95, - 104,111,111,107,10,32,32,32,32,32,32,32,32,119,104,105, - 99,104,32,119,105,108,108,32,114,101,116,117,114,110,32,97, - 110,32,105,110,115,116,97,110,99,101,32,117,115,105,110,103, - 32,116,104,101,32,115,112,101,99,105,102,105,101,100,32,108, - 111,97,100,101,114,115,32,97,110,100,32,116,104,101,32,112, - 97,116,104,10,32,32,32,32,32,32,32,32,99,97,108,108, - 101,100,32,111,110,32,116,104,101,32,99,108,111,115,117,114, - 101,46,10,10,32,32,32,32,32,32,32,32,73,102,32,116, - 104,101,32,112,97,116,104,32,99,97,108,108,101,100,32,111, - 110,32,116,104,101,32,99,108,111,115,117,114,101,32,105,115, - 32,110,111,116,32,97,32,100,105,114,101,99,116,111,114,121, - 44,32,73,109,112,111,114,116,69,114,114,111,114,32,105,115, - 10,32,32,32,32,32,32,32,32,114,97,105,115,101,100,46, - 10,10,32,32,32,32,32,32,32,32,99,1,0,0,0,0, - 0,0,0,1,0,0,0,4,0,0,0,19,0,0,0,115, - 32,0,0,0,116,0,124,0,131,1,115,22,116,1,100,1, - 100,2,124,0,144,1,131,1,130,1,136,0,124,0,136,1, - 140,1,83,0,41,3,122,45,80,97,116,104,32,104,111,111, - 107,32,102,111,114,32,105,109,112,111,114,116,108,105,98,46, - 109,97,99,104,105,110,101,114,121,46,70,105,108,101,70,105, - 110,100,101,114,46,122,30,111,110,108,121,32,100,105,114,101, - 99,116,111,114,105,101,115,32,97,114,101,32,115,117,112,112, - 111,114,116,101,100,114,37,0,0,0,41,2,114,48,0,0, - 0,114,103,0,0,0,41,1,114,37,0,0,0,41,2,114, - 168,0,0,0,114,11,1,0,0,114,4,0,0,0,114,6, - 0,0,0,218,24,112,97,116,104,95,104,111,111,107,95,102, - 111,114,95,70,105,108,101,70,105,110,100,101,114,38,5,0, - 0,115,6,0,0,0,0,2,8,1,14,1,122,54,70,105, - 108,101,70,105,110,100,101,114,46,112,97,116,104,95,104,111, - 111,107,46,60,108,111,99,97,108,115,62,46,112,97,116,104, - 95,104,111,111,107,95,102,111,114,95,70,105,108,101,70,105, - 110,100,101,114,114,4,0,0,0,41,3,114,168,0,0,0, - 114,11,1,0,0,114,17,1,0,0,114,4,0,0,0,41, - 2,114,168,0,0,0,114,11,1,0,0,114,6,0,0,0, - 218,9,112,97,116,104,95,104,111,111,107,28,5,0,0,115, - 4,0,0,0,0,10,14,6,122,20,70,105,108,101,70,105, - 110,100,101,114,46,112,97,116,104,95,104,111,111,107,99,1, - 0,0,0,0,0,0,0,1,0,0,0,2,0,0,0,67, - 0,0,0,115,12,0,0,0,100,1,106,0,124,0,106,1, - 131,1,83,0,41,2,78,122,16,70,105,108,101,70,105,110, - 100,101,114,40,123,33,114,125,41,41,2,114,50,0,0,0, - 114,37,0,0,0,41,1,114,104,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,243,0,0,0, - 46,5,0,0,115,2,0,0,0,0,1,122,19,70,105,108, - 101,70,105,110,100,101,114,46,95,95,114,101,112,114,95,95, - 41,1,78,41,15,114,109,0,0,0,114,108,0,0,0,114, - 110,0,0,0,114,111,0,0,0,114,182,0,0,0,114,249, - 0,0,0,114,127,0,0,0,114,179,0,0,0,114,121,0, - 0,0,114,4,1,0,0,114,178,0,0,0,114,12,1,0, - 0,114,180,0,0,0,114,18,1,0,0,114,243,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,5,1,0,0,159,4,0,0,115,20,0, - 0,0,8,7,4,2,8,14,8,4,4,2,8,12,8,5, - 10,48,8,31,12,18,114,5,1,0,0,99,4,0,0,0, - 0,0,0,0,6,0,0,0,11,0,0,0,67,0,0,0, - 115,148,0,0,0,124,0,106,0,100,1,131,1,125,4,124, - 0,106,0,100,2,131,1,125,5,124,4,115,66,124,5,114, - 36,124,5,106,1,125,4,110,30,124,2,124,3,107,2,114, - 56,116,2,124,1,124,2,131,2,125,4,110,10,116,3,124, - 1,124,2,131,2,125,4,124,5,115,86,116,4,124,1,124, - 2,100,3,124,4,144,1,131,2,125,5,121,36,124,5,124, - 0,100,2,60,0,124,4,124,0,100,1,60,0,124,2,124, - 0,100,4,60,0,124,3,124,0,100,5,60,0,87,0,110, - 20,4,0,116,5,107,10,114,142,1,0,1,0,1,0,89, - 0,110,2,88,0,100,0,83,0,41,6,78,218,10,95,95, - 108,111,97,100,101,114,95,95,218,8,95,95,115,112,101,99, - 95,95,114,124,0,0,0,90,8,95,95,102,105,108,101,95, - 95,90,10,95,95,99,97,99,104,101,100,95,95,41,6,218, - 3,103,101,116,114,124,0,0,0,114,220,0,0,0,114,215, - 0,0,0,114,165,0,0,0,218,9,69,120,99,101,112,116, - 105,111,110,41,6,90,2,110,115,114,102,0,0,0,90,8, - 112,97,116,104,110,97,109,101,90,9,99,112,97,116,104,110, - 97,109,101,114,124,0,0,0,114,162,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,14,95,102, - 105,120,95,117,112,95,109,111,100,117,108,101,52,5,0,0, - 115,34,0,0,0,0,2,10,1,10,1,4,1,4,1,8, - 1,8,1,12,2,10,1,4,1,16,1,2,1,8,1,8, - 1,8,1,12,1,14,2,114,23,1,0,0,99,0,0,0, - 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, - 0,115,38,0,0,0,116,0,116,1,106,2,131,0,102,2, - 125,0,116,3,116,4,102,2,125,1,116,5,116,6,102,2, - 125,2,124,0,124,1,124,2,103,3,83,0,41,1,122,95, - 82,101,116,117,114,110,115,32,97,32,108,105,115,116,32,111, - 102,32,102,105,108,101,45,98,97,115,101,100,32,109,111,100, - 117,108,101,32,108,111,97,100,101,114,115,46,10,10,32,32, - 32,32,69,97,99,104,32,105,116,101,109,32,105,115,32,97, - 32,116,117,112,108,101,32,40,108,111,97,100,101,114,44,32, - 115,117,102,102,105,120,101,115,41,46,10,32,32,32,32,41, - 7,114,221,0,0,0,114,143,0,0,0,218,18,101,120,116, - 101,110,115,105,111,110,95,115,117,102,102,105,120,101,115,114, - 215,0,0,0,114,88,0,0,0,114,220,0,0,0,114,78, - 0,0,0,41,3,90,10,101,120,116,101,110,115,105,111,110, - 115,90,6,115,111,117,114,99,101,90,8,98,121,116,101,99, - 111,100,101,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,159,0,0,0,75,5,0,0,115,8,0,0,0, - 0,5,12,1,8,1,8,1,114,159,0,0,0,99,1,0, - 0,0,0,0,0,0,12,0,0,0,12,0,0,0,67,0, - 0,0,115,188,1,0,0,124,0,97,0,116,0,106,1,97, - 1,116,0,106,2,97,2,116,1,106,3,116,4,25,0,125, - 1,120,56,100,26,68,0,93,48,125,2,124,2,116,1,106, - 3,107,7,114,58,116,0,106,5,124,2,131,1,125,3,110, - 10,116,1,106,3,124,2,25,0,125,3,116,6,124,1,124, - 2,124,3,131,3,1,0,113,32,87,0,100,5,100,6,103, - 1,102,2,100,7,100,8,100,6,103,2,102,2,102,2,125, - 4,120,118,124,4,68,0,93,102,92,2,125,5,125,6,116, - 7,100,9,100,10,132,0,124,6,68,0,131,1,131,1,115, - 142,116,8,130,1,124,6,100,11,25,0,125,7,124,5,116, - 1,106,3,107,6,114,174,116,1,106,3,124,5,25,0,125, - 8,80,0,113,112,121,16,116,0,106,5,124,5,131,1,125, - 8,80,0,87,0,113,112,4,0,116,9,107,10,114,212,1, - 0,1,0,1,0,119,112,89,0,113,112,88,0,113,112,87, - 0,116,9,100,12,131,1,130,1,116,6,124,1,100,13,124, - 8,131,3,1,0,116,6,124,1,100,14,124,7,131,3,1, - 0,116,6,124,1,100,15,100,16,106,10,124,6,131,1,131, - 3,1,0,121,14,116,0,106,5,100,17,131,1,125,9,87, - 0,110,26,4,0,116,9,107,10,144,1,114,52,1,0,1, - 0,1,0,100,18,125,9,89,0,110,2,88,0,116,6,124, - 1,100,17,124,9,131,3,1,0,116,0,106,5,100,19,131, - 1,125,10,116,6,124,1,100,19,124,10,131,3,1,0,124, - 5,100,7,107,2,144,1,114,120,116,0,106,5,100,20,131, - 1,125,11,116,6,124,1,100,21,124,11,131,3,1,0,116, - 6,124,1,100,22,116,11,131,0,131,3,1,0,116,12,106, - 13,116,2,106,14,131,0,131,1,1,0,124,5,100,7,107, - 2,144,1,114,184,116,15,106,16,100,23,131,1,1,0,100, - 24,116,12,107,6,144,1,114,184,100,25,116,17,95,18,100, - 18,83,0,41,27,122,205,83,101,116,117,112,32,116,104,101, - 32,112,97,116,104,45,98,97,115,101,100,32,105,109,112,111, - 114,116,101,114,115,32,102,111,114,32,105,109,112,111,114,116, - 108,105,98,32,98,121,32,105,109,112,111,114,116,105,110,103, - 32,110,101,101,100,101,100,10,32,32,32,32,98,117,105,108, - 116,45,105,110,32,109,111,100,117,108,101,115,32,97,110,100, - 32,105,110,106,101,99,116,105,110,103,32,116,104,101,109,32, - 105,110,116,111,32,116,104,101,32,103,108,111,98,97,108,32, - 110,97,109,101,115,112,97,99,101,46,10,10,32,32,32,32, - 79,116,104,101,114,32,99,111,109,112,111,110,101,110,116,115, - 32,97,114,101,32,101,120,116,114,97,99,116,101,100,32,102, - 114,111,109,32,116,104,101,32,99,111,114,101,32,98,111,111, - 116,115,116,114,97,112,32,109,111,100,117,108,101,46,10,10, - 32,32,32,32,114,52,0,0,0,114,63,0,0,0,218,8, - 98,117,105,108,116,105,110,115,114,140,0,0,0,90,5,112, - 111,115,105,120,250,1,47,218,2,110,116,250,1,92,99,1, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,115, - 0,0,0,115,26,0,0,0,124,0,93,18,125,1,116,0, - 124,1,131,1,100,0,107,2,86,0,1,0,113,2,100,1, - 83,0,41,2,114,31,0,0,0,78,41,1,114,33,0,0, - 0,41,2,114,24,0,0,0,114,81,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,224,0,0, - 0,111,5,0,0,115,2,0,0,0,4,0,122,25,95,115, - 101,116,117,112,46,60,108,111,99,97,108,115,62,46,60,103, - 101,110,101,120,112,114,62,114,62,0,0,0,122,30,105,109, - 112,111,114,116,108,105,98,32,114,101,113,117,105,114,101,115, - 32,112,111,115,105,120,32,111,114,32,110,116,114,3,0,0, - 0,114,27,0,0,0,114,23,0,0,0,114,32,0,0,0, - 90,7,95,116,104,114,101,97,100,78,90,8,95,119,101,97, - 107,114,101,102,90,6,119,105,110,114,101,103,114,167,0,0, - 0,114,7,0,0,0,122,4,46,112,121,119,122,6,95,100, - 46,112,121,100,84,41,4,122,3,95,105,111,122,9,95,119, - 97,114,110,105,110,103,115,122,8,98,117,105,108,116,105,110, - 115,122,7,109,97,114,115,104,97,108,41,19,114,118,0,0, - 0,114,8,0,0,0,114,143,0,0,0,114,236,0,0,0, - 114,109,0,0,0,90,18,95,98,117,105,108,116,105,110,95, - 102,114,111,109,95,110,97,109,101,114,113,0,0,0,218,3, - 97,108,108,218,14,65,115,115,101,114,116,105,111,110,69,114, - 114,111,114,114,103,0,0,0,114,28,0,0,0,114,13,0, - 0,0,114,226,0,0,0,114,147,0,0,0,114,24,1,0, - 0,114,88,0,0,0,114,161,0,0,0,114,166,0,0,0, - 114,170,0,0,0,41,12,218,17,95,98,111,111,116,115,116, - 114,97,112,95,109,111,100,117,108,101,90,11,115,101,108,102, - 95,109,111,100,117,108,101,90,12,98,117,105,108,116,105,110, - 95,110,97,109,101,90,14,98,117,105,108,116,105,110,95,109, - 111,100,117,108,101,90,10,111,115,95,100,101,116,97,105,108, - 115,90,10,98,117,105,108,116,105,110,95,111,115,114,23,0, - 0,0,114,27,0,0,0,90,9,111,115,95,109,111,100,117, - 108,101,90,13,116,104,114,101,97,100,95,109,111,100,117,108, - 101,90,14,119,101,97,107,114,101,102,95,109,111,100,117,108, - 101,90,13,119,105,110,114,101,103,95,109,111,100,117,108,101, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 6,95,115,101,116,117,112,86,5,0,0,115,82,0,0,0, - 0,8,4,1,6,1,6,3,10,1,10,1,10,1,12,2, - 10,1,16,3,22,1,14,2,22,1,8,1,10,1,10,1, - 4,2,2,1,10,1,6,1,14,1,12,2,8,1,12,1, - 12,1,18,3,2,1,14,1,16,2,10,1,12,3,10,1, - 12,3,10,1,10,1,12,3,14,1,14,1,10,1,10,1, - 10,1,114,32,1,0,0,99,1,0,0,0,0,0,0,0, - 2,0,0,0,3,0,0,0,67,0,0,0,115,84,0,0, - 0,116,0,124,0,131,1,1,0,116,1,131,0,125,1,116, - 2,106,3,106,4,116,5,106,6,124,1,140,0,103,1,131, - 1,1,0,116,7,106,8,100,1,107,2,114,56,116,2,106, + 114,6,0,0,0,114,121,0,0,0,189,4,0,0,115,8, + 0,0,0,0,7,10,1,8,1,8,1,122,22,70,105,108, + 101,70,105,110,100,101,114,46,102,105,110,100,95,108,111,97, + 100,101,114,99,6,0,0,0,0,0,0,0,7,0,0,0, + 6,0,0,0,67,0,0,0,115,26,0,0,0,124,1,124, + 2,124,3,131,2,125,6,116,0,124,2,124,3,124,6,124, + 4,100,1,141,4,83,0,41,2,78,41,2,114,124,0,0, + 0,114,154,0,0,0,41,1,114,165,0,0,0,41,7,114, + 104,0,0,0,114,163,0,0,0,114,123,0,0,0,114,37, + 0,0,0,90,4,115,109,115,108,114,177,0,0,0,114,124, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,114,4,1,0,0,201,4,0,0,115,6,0,0,0, + 0,1,10,1,8,1,122,20,70,105,108,101,70,105,110,100, + 101,114,46,95,103,101,116,95,115,112,101,99,78,99,3,0, + 0,0,0,0,0,0,14,0,0,0,15,0,0,0,67,0, + 0,0,115,98,1,0,0,100,1,125,3,124,1,106,0,100, + 2,131,1,100,3,25,0,125,4,121,24,116,1,124,0,106, + 2,112,34,116,3,106,4,131,0,131,1,106,5,125,5,87, + 0,110,24,4,0,116,6,107,10,114,66,1,0,1,0,1, + 0,100,10,125,5,89,0,110,2,88,0,124,5,124,0,106, + 7,107,3,114,92,124,0,106,8,131,0,1,0,124,5,124, + 0,95,7,116,9,131,0,114,114,124,0,106,10,125,6,124, + 4,106,11,131,0,125,7,110,10,124,0,106,12,125,6,124, + 4,125,7,124,7,124,6,107,6,114,218,116,13,124,0,106, + 2,124,4,131,2,125,8,120,72,124,0,106,14,68,0,93, + 54,92,2,125,9,125,10,100,5,124,9,23,0,125,11,116, + 13,124,8,124,11,131,2,125,12,116,15,124,12,131,1,114, + 152,124,0,106,16,124,10,124,1,124,12,124,8,103,1,124, + 2,131,5,83,0,113,152,87,0,116,17,124,8,131,1,125, + 3,120,88,124,0,106,14,68,0,93,78,92,2,125,9,125, + 10,116,13,124,0,106,2,124,4,124,9,23,0,131,2,125, + 12,116,18,106,19,100,6,124,12,100,3,100,7,141,3,1, + 0,124,7,124,9,23,0,124,6,107,6,114,226,116,15,124, + 12,131,1,114,226,124,0,106,16,124,10,124,1,124,12,100, + 8,124,2,131,5,83,0,113,226,87,0,124,3,144,1,114, + 94,116,18,106,19,100,9,124,8,131,2,1,0,116,18,106, + 20,124,1,100,8,131,2,125,13,124,8,103,1,124,13,95, + 21,124,13,83,0,100,8,83,0,41,11,122,111,84,114,121, + 32,116,111,32,102,105,110,100,32,97,32,115,112,101,99,32, + 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, + 100,32,109,111,100,117,108,101,46,10,10,32,32,32,32,32, + 32,32,32,82,101,116,117,114,110,115,32,116,104,101,32,109, + 97,116,99,104,105,110,103,32,115,112,101,99,44,32,111,114, + 32,78,111,110,101,32,105,102,32,110,111,116,32,102,111,117, + 110,100,46,10,32,32,32,32,32,32,32,32,70,114,61,0, + 0,0,114,59,0,0,0,114,31,0,0,0,114,182,0,0, + 0,122,9,116,114,121,105,110,103,32,123,125,41,1,90,9, + 118,101,114,98,111,115,105,116,121,78,122,25,112,111,115,115, + 105,98,108,101,32,110,97,109,101,115,112,97,99,101,32,102, + 111,114,32,123,125,114,91,0,0,0,41,22,114,34,0,0, + 0,114,41,0,0,0,114,37,0,0,0,114,3,0,0,0, + 114,47,0,0,0,114,216,0,0,0,114,42,0,0,0,114, + 7,1,0,0,218,11,95,102,105,108,108,95,99,97,99,104, + 101,114,7,0,0,0,114,10,1,0,0,114,92,0,0,0, + 114,9,1,0,0,114,30,0,0,0,114,6,1,0,0,114, + 46,0,0,0,114,4,1,0,0,114,48,0,0,0,114,118, + 0,0,0,114,133,0,0,0,114,158,0,0,0,114,154,0, + 0,0,41,14,114,104,0,0,0,114,123,0,0,0,114,177, + 0,0,0,90,12,105,115,95,110,97,109,101,115,112,97,99, + 101,90,11,116,97,105,108,95,109,111,100,117,108,101,114,130, + 0,0,0,90,5,99,97,99,104,101,90,12,99,97,99,104, + 101,95,109,111,100,117,108,101,90,9,98,97,115,101,95,112, + 97,116,104,114,222,0,0,0,114,163,0,0,0,90,13,105, + 110,105,116,95,102,105,108,101,110,97,109,101,90,9,102,117, + 108,108,95,112,97,116,104,114,162,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,178,0,0,0, + 206,4,0,0,115,70,0,0,0,0,5,4,1,14,1,2, + 1,24,1,14,1,10,1,10,1,8,1,6,2,6,1,6, + 1,10,2,6,1,4,2,8,1,12,1,16,1,8,1,10, + 1,8,1,24,4,8,2,16,1,16,1,16,1,12,1,8, + 1,10,1,12,1,6,1,12,1,12,1,8,1,4,1,122, + 20,70,105,108,101,70,105,110,100,101,114,46,102,105,110,100, + 95,115,112,101,99,99,1,0,0,0,0,0,0,0,9,0, + 0,0,13,0,0,0,67,0,0,0,115,194,0,0,0,124, + 0,106,0,125,1,121,22,116,1,106,2,124,1,112,22,116, + 1,106,3,131,0,131,1,125,2,87,0,110,30,4,0,116, + 4,116,5,116,6,102,3,107,10,114,58,1,0,1,0,1, + 0,103,0,125,2,89,0,110,2,88,0,116,7,106,8,106, + 9,100,1,131,1,115,84,116,10,124,2,131,1,124,0,95, + 11,110,78,116,10,131,0,125,3,120,64,124,2,68,0,93, + 56,125,4,124,4,106,12,100,2,131,1,92,3,125,5,125, + 6,125,7,124,6,114,138,100,3,106,13,124,5,124,7,106, + 14,131,0,131,2,125,8,110,4,124,5,125,8,124,3,106, + 15,124,8,131,1,1,0,113,96,87,0,124,3,124,0,95, + 11,116,7,106,8,106,9,116,16,131,1,114,190,100,4,100, + 5,132,0,124,2,68,0,131,1,124,0,95,17,100,6,83, + 0,41,7,122,68,70,105,108,108,32,116,104,101,32,99,97, + 99,104,101,32,111,102,32,112,111,116,101,110,116,105,97,108, + 32,109,111,100,117,108,101,115,32,97,110,100,32,112,97,99, + 107,97,103,101,115,32,102,111,114,32,116,104,105,115,32,100, + 105,114,101,99,116,111,114,121,46,114,0,0,0,0,114,61, + 0,0,0,122,5,123,125,46,123,125,99,1,0,0,0,0, + 0,0,0,2,0,0,0,3,0,0,0,83,0,0,0,115, + 20,0,0,0,104,0,124,0,93,12,125,1,124,1,106,0, + 131,0,146,2,113,4,83,0,114,4,0,0,0,41,1,114, + 92,0,0,0,41,2,114,24,0,0,0,90,2,102,110,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,250,9, + 60,115,101,116,99,111,109,112,62,27,5,0,0,115,2,0, + 0,0,6,0,122,41,70,105,108,101,70,105,110,100,101,114, + 46,95,102,105,108,108,95,99,97,99,104,101,46,60,108,111, + 99,97,108,115,62,46,60,115,101,116,99,111,109,112,62,78, + 41,18,114,37,0,0,0,114,3,0,0,0,90,7,108,105, + 115,116,100,105,114,114,47,0,0,0,114,255,0,0,0,218, + 15,80,101,114,109,105,115,115,105,111,110,69,114,114,111,114, + 218,18,78,111,116,65,68,105,114,101,99,116,111,114,121,69, + 114,114,111,114,114,8,0,0,0,114,9,0,0,0,114,10, + 0,0,0,114,8,1,0,0,114,9,1,0,0,114,87,0, + 0,0,114,50,0,0,0,114,92,0,0,0,218,3,97,100, + 100,114,11,0,0,0,114,10,1,0,0,41,9,114,104,0, + 0,0,114,37,0,0,0,90,8,99,111,110,116,101,110,116, + 115,90,21,108,111,119,101,114,95,115,117,102,102,105,120,95, + 99,111,110,116,101,110,116,115,114,244,0,0,0,114,102,0, + 0,0,114,234,0,0,0,114,222,0,0,0,90,8,110,101, + 119,95,110,97,109,101,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,12,1,0,0,254,4,0,0,115,34, + 0,0,0,0,2,6,1,2,1,22,1,20,3,10,3,12, + 1,12,7,6,1,10,1,16,1,4,1,18,2,4,1,14, + 1,6,1,12,1,122,22,70,105,108,101,70,105,110,100,101, + 114,46,95,102,105,108,108,95,99,97,99,104,101,99,1,0, + 0,0,0,0,0,0,3,0,0,0,3,0,0,0,7,0, + 0,0,115,18,0,0,0,135,0,135,1,102,2,100,1,100, + 2,132,8,125,2,124,2,83,0,41,3,97,20,1,0,0, + 65,32,99,108,97,115,115,32,109,101,116,104,111,100,32,119, + 104,105,99,104,32,114,101,116,117,114,110,115,32,97,32,99, + 108,111,115,117,114,101,32,116,111,32,117,115,101,32,111,110, + 32,115,121,115,46,112,97,116,104,95,104,111,111,107,10,32, + 32,32,32,32,32,32,32,119,104,105,99,104,32,119,105,108, + 108,32,114,101,116,117,114,110,32,97,110,32,105,110,115,116, + 97,110,99,101,32,117,115,105,110,103,32,116,104,101,32,115, + 112,101,99,105,102,105,101,100,32,108,111,97,100,101,114,115, + 32,97,110,100,32,116,104,101,32,112,97,116,104,10,32,32, + 32,32,32,32,32,32,99,97,108,108,101,100,32,111,110,32, + 116,104,101,32,99,108,111,115,117,114,101,46,10,10,32,32, + 32,32,32,32,32,32,73,102,32,116,104,101,32,112,97,116, + 104,32,99,97,108,108,101,100,32,111,110,32,116,104,101,32, + 99,108,111,115,117,114,101,32,105,115,32,110,111,116,32,97, + 32,100,105,114,101,99,116,111,114,121,44,32,73,109,112,111, + 114,116,69,114,114,111,114,32,105,115,10,32,32,32,32,32, + 32,32,32,114,97,105,115,101,100,46,10,10,32,32,32,32, + 32,32,32,32,99,1,0,0,0,0,0,0,0,1,0,0, + 0,4,0,0,0,19,0,0,0,115,34,0,0,0,116,0, + 124,0,131,1,115,20,116,1,100,1,124,0,100,2,141,2, + 130,1,136,0,124,0,102,1,136,1,152,2,142,0,83,0, + 41,3,122,45,80,97,116,104,32,104,111,111,107,32,102,111, + 114,32,105,109,112,111,114,116,108,105,98,46,109,97,99,104, + 105,110,101,114,121,46,70,105,108,101,70,105,110,100,101,114, + 46,122,30,111,110,108,121,32,100,105,114,101,99,116,111,114, + 105,101,115,32,97,114,101,32,115,117,112,112,111,114,116,101, + 100,41,1,114,37,0,0,0,41,2,114,48,0,0,0,114, + 103,0,0,0,41,1,114,37,0,0,0,41,2,114,168,0, + 0,0,114,11,1,0,0,114,4,0,0,0,114,6,0,0, + 0,218,24,112,97,116,104,95,104,111,111,107,95,102,111,114, + 95,70,105,108,101,70,105,110,100,101,114,39,5,0,0,115, + 6,0,0,0,0,2,8,1,12,1,122,54,70,105,108,101, + 70,105,110,100,101,114,46,112,97,116,104,95,104,111,111,107, + 46,60,108,111,99,97,108,115,62,46,112,97,116,104,95,104, + 111,111,107,95,102,111,114,95,70,105,108,101,70,105,110,100, + 101,114,114,4,0,0,0,41,3,114,168,0,0,0,114,11, + 1,0,0,114,17,1,0,0,114,4,0,0,0,41,2,114, + 168,0,0,0,114,11,1,0,0,114,6,0,0,0,218,9, + 112,97,116,104,95,104,111,111,107,29,5,0,0,115,4,0, + 0,0,0,10,14,6,122,20,70,105,108,101,70,105,110,100, + 101,114,46,112,97,116,104,95,104,111,111,107,99,1,0,0, + 0,0,0,0,0,1,0,0,0,2,0,0,0,67,0,0, + 0,115,12,0,0,0,100,1,106,0,124,0,106,1,131,1, + 83,0,41,2,78,122,16,70,105,108,101,70,105,110,100,101, + 114,40,123,33,114,125,41,41,2,114,50,0,0,0,114,37, + 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,243,0,0,0,47,5, + 0,0,115,2,0,0,0,0,1,122,19,70,105,108,101,70, + 105,110,100,101,114,46,95,95,114,101,112,114,95,95,41,1, + 78,41,15,114,109,0,0,0,114,108,0,0,0,114,110,0, + 0,0,114,111,0,0,0,114,182,0,0,0,114,249,0,0, + 0,114,127,0,0,0,114,179,0,0,0,114,121,0,0,0, + 114,4,1,0,0,114,178,0,0,0,114,12,1,0,0,114, + 180,0,0,0,114,18,1,0,0,114,243,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,114,5,1,0,0,160,4,0,0,115,20,0,0,0, + 8,7,4,2,8,14,8,4,4,2,8,12,8,5,10,48, + 8,31,12,18,114,5,1,0,0,99,4,0,0,0,0,0, + 0,0,6,0,0,0,11,0,0,0,67,0,0,0,115,146, + 0,0,0,124,0,106,0,100,1,131,1,125,4,124,0,106, + 0,100,2,131,1,125,5,124,4,115,66,124,5,114,36,124, + 5,106,1,125,4,110,30,124,2,124,3,107,2,114,56,116, + 2,124,1,124,2,131,2,125,4,110,10,116,3,124,1,124, + 2,131,2,125,4,124,5,115,84,116,4,124,1,124,2,124, + 4,100,3,141,3,125,5,121,36,124,5,124,0,100,2,60, + 0,124,4,124,0,100,1,60,0,124,2,124,0,100,4,60, + 0,124,3,124,0,100,5,60,0,87,0,110,20,4,0,116, + 5,107,10,114,140,1,0,1,0,1,0,89,0,110,2,88, + 0,100,0,83,0,41,6,78,218,10,95,95,108,111,97,100, + 101,114,95,95,218,8,95,95,115,112,101,99,95,95,41,1, + 114,124,0,0,0,90,8,95,95,102,105,108,101,95,95,90, + 10,95,95,99,97,99,104,101,100,95,95,41,6,218,3,103, + 101,116,114,124,0,0,0,114,220,0,0,0,114,215,0,0, + 0,114,165,0,0,0,218,9,69,120,99,101,112,116,105,111, + 110,41,6,90,2,110,115,114,102,0,0,0,90,8,112,97, + 116,104,110,97,109,101,90,9,99,112,97,116,104,110,97,109, + 101,114,124,0,0,0,114,162,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,14,95,102,105,120, + 95,117,112,95,109,111,100,117,108,101,53,5,0,0,115,34, + 0,0,0,0,2,10,1,10,1,4,1,4,1,8,1,8, + 1,12,2,10,1,4,1,14,1,2,1,8,1,8,1,8, + 1,12,1,14,2,114,23,1,0,0,99,0,0,0,0,0, + 0,0,0,3,0,0,0,3,0,0,0,67,0,0,0,115, + 38,0,0,0,116,0,116,1,106,2,131,0,102,2,125,0, + 116,3,116,4,102,2,125,1,116,5,116,6,102,2,125,2, + 124,0,124,1,124,2,103,3,83,0,41,1,122,95,82,101, + 116,117,114,110,115,32,97,32,108,105,115,116,32,111,102,32, + 102,105,108,101,45,98,97,115,101,100,32,109,111,100,117,108, + 101,32,108,111,97,100,101,114,115,46,10,10,32,32,32,32, + 69,97,99,104,32,105,116,101,109,32,105,115,32,97,32,116, + 117,112,108,101,32,40,108,111,97,100,101,114,44,32,115,117, + 102,102,105,120,101,115,41,46,10,32,32,32,32,41,7,114, + 221,0,0,0,114,143,0,0,0,218,18,101,120,116,101,110, + 115,105,111,110,95,115,117,102,102,105,120,101,115,114,215,0, + 0,0,114,88,0,0,0,114,220,0,0,0,114,78,0,0, + 0,41,3,90,10,101,120,116,101,110,115,105,111,110,115,90, + 6,115,111,117,114,99,101,90,8,98,121,116,101,99,111,100, + 101,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, + 114,159,0,0,0,76,5,0,0,115,8,0,0,0,0,5, + 12,1,8,1,8,1,114,159,0,0,0,99,1,0,0,0, + 0,0,0,0,12,0,0,0,12,0,0,0,67,0,0,0, + 115,188,1,0,0,124,0,97,0,116,0,106,1,97,1,116, + 0,106,2,97,2,116,1,106,3,116,4,25,0,125,1,120, + 56,100,26,68,0,93,48,125,2,124,2,116,1,106,3,107, + 7,114,58,116,0,106,5,124,2,131,1,125,3,110,10,116, + 1,106,3,124,2,25,0,125,3,116,6,124,1,124,2,124, + 3,131,3,1,0,113,32,87,0,100,5,100,6,103,1,102, + 2,100,7,100,8,100,6,103,2,102,2,102,2,125,4,120, + 118,124,4,68,0,93,102,92,2,125,5,125,6,116,7,100, + 9,100,10,132,0,124,6,68,0,131,1,131,1,115,142,116, + 8,130,1,124,6,100,11,25,0,125,7,124,5,116,1,106, + 3,107,6,114,174,116,1,106,3,124,5,25,0,125,8,80, + 0,113,112,121,16,116,0,106,5,124,5,131,1,125,8,80, + 0,87,0,113,112,4,0,116,9,107,10,114,212,1,0,1, + 0,1,0,119,112,89,0,113,112,88,0,113,112,87,0,116, + 9,100,12,131,1,130,1,116,6,124,1,100,13,124,8,131, + 3,1,0,116,6,124,1,100,14,124,7,131,3,1,0,116, + 6,124,1,100,15,100,16,106,10,124,6,131,1,131,3,1, + 0,121,14,116,0,106,5,100,17,131,1,125,9,87,0,110, + 26,4,0,116,9,107,10,144,1,114,52,1,0,1,0,1, + 0,100,18,125,9,89,0,110,2,88,0,116,6,124,1,100, + 17,124,9,131,3,1,0,116,0,106,5,100,19,131,1,125, + 10,116,6,124,1,100,19,124,10,131,3,1,0,124,5,100, + 7,107,2,144,1,114,120,116,0,106,5,100,20,131,1,125, + 11,116,6,124,1,100,21,124,11,131,3,1,0,116,6,124, + 1,100,22,116,11,131,0,131,3,1,0,116,12,106,13,116, + 2,106,14,131,0,131,1,1,0,124,5,100,7,107,2,144, + 1,114,184,116,15,106,16,100,23,131,1,1,0,100,24,116, + 12,107,6,144,1,114,184,100,25,116,17,95,18,100,18,83, + 0,41,27,122,205,83,101,116,117,112,32,116,104,101,32,112, + 97,116,104,45,98,97,115,101,100,32,105,109,112,111,114,116, + 101,114,115,32,102,111,114,32,105,109,112,111,114,116,108,105, + 98,32,98,121,32,105,109,112,111,114,116,105,110,103,32,110, + 101,101,100,101,100,10,32,32,32,32,98,117,105,108,116,45, + 105,110,32,109,111,100,117,108,101,115,32,97,110,100,32,105, + 110,106,101,99,116,105,110,103,32,116,104,101,109,32,105,110, + 116,111,32,116,104,101,32,103,108,111,98,97,108,32,110,97, + 109,101,115,112,97,99,101,46,10,10,32,32,32,32,79,116, + 104,101,114,32,99,111,109,112,111,110,101,110,116,115,32,97, + 114,101,32,101,120,116,114,97,99,116,101,100,32,102,114,111, + 109,32,116,104,101,32,99,111,114,101,32,98,111,111,116,115, + 116,114,97,112,32,109,111,100,117,108,101,46,10,10,32,32, + 32,32,114,52,0,0,0,114,63,0,0,0,218,8,98,117, + 105,108,116,105,110,115,114,140,0,0,0,90,5,112,111,115, + 105,120,250,1,47,218,2,110,116,250,1,92,99,1,0,0, + 0,0,0,0,0,2,0,0,0,3,0,0,0,115,0,0, + 0,115,26,0,0,0,124,0,93,18,125,1,116,0,124,1, + 131,1,100,0,107,2,86,0,1,0,113,2,100,1,83,0, + 41,2,114,31,0,0,0,78,41,1,114,33,0,0,0,41, + 2,114,24,0,0,0,114,81,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,114,224,0,0,0,112, + 5,0,0,115,2,0,0,0,4,0,122,25,95,115,101,116, + 117,112,46,60,108,111,99,97,108,115,62,46,60,103,101,110, + 101,120,112,114,62,114,62,0,0,0,122,30,105,109,112,111, + 114,116,108,105,98,32,114,101,113,117,105,114,101,115,32,112, + 111,115,105,120,32,111,114,32,110,116,114,3,0,0,0,114, + 27,0,0,0,114,23,0,0,0,114,32,0,0,0,90,7, + 95,116,104,114,101,97,100,78,90,8,95,119,101,97,107,114, + 101,102,90,6,119,105,110,114,101,103,114,167,0,0,0,114, + 7,0,0,0,122,4,46,112,121,119,122,6,95,100,46,112, + 121,100,84,41,4,122,3,95,105,111,122,9,95,119,97,114, + 110,105,110,103,115,122,8,98,117,105,108,116,105,110,115,122, + 7,109,97,114,115,104,97,108,41,19,114,118,0,0,0,114, + 8,0,0,0,114,143,0,0,0,114,236,0,0,0,114,109, + 0,0,0,90,18,95,98,117,105,108,116,105,110,95,102,114, + 111,109,95,110,97,109,101,114,113,0,0,0,218,3,97,108, + 108,218,14,65,115,115,101,114,116,105,111,110,69,114,114,111, + 114,114,103,0,0,0,114,28,0,0,0,114,13,0,0,0, + 114,226,0,0,0,114,147,0,0,0,114,24,1,0,0,114, + 88,0,0,0,114,161,0,0,0,114,166,0,0,0,114,170, + 0,0,0,41,12,218,17,95,98,111,111,116,115,116,114,97, + 112,95,109,111,100,117,108,101,90,11,115,101,108,102,95,109, + 111,100,117,108,101,90,12,98,117,105,108,116,105,110,95,110, + 97,109,101,90,14,98,117,105,108,116,105,110,95,109,111,100, + 117,108,101,90,10,111,115,95,100,101,116,97,105,108,115,90, + 10,98,117,105,108,116,105,110,95,111,115,114,23,0,0,0, + 114,27,0,0,0,90,9,111,115,95,109,111,100,117,108,101, + 90,13,116,104,114,101,97,100,95,109,111,100,117,108,101,90, + 14,119,101,97,107,114,101,102,95,109,111,100,117,108,101,90, + 13,119,105,110,114,101,103,95,109,111,100,117,108,101,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,6,95, + 115,101,116,117,112,87,5,0,0,115,82,0,0,0,0,8, + 4,1,6,1,6,3,10,1,10,1,10,1,12,2,10,1, + 16,3,22,1,14,2,22,1,8,1,10,1,10,1,4,2, + 2,1,10,1,6,1,14,1,12,2,8,1,12,1,12,1, + 18,3,2,1,14,1,16,2,10,1,12,3,10,1,12,3, + 10,1,10,1,12,3,14,1,14,1,10,1,10,1,10,1, + 114,32,1,0,0,99,1,0,0,0,0,0,0,0,2,0, + 0,0,3,0,0,0,67,0,0,0,115,86,0,0,0,116, + 0,124,0,131,1,1,0,116,1,131,0,125,1,116,2,106, + 3,106,4,116,5,106,6,124,1,152,1,142,0,103,1,131, + 1,1,0,116,7,106,8,100,1,107,2,114,58,116,2,106, 9,106,10,116,11,131,1,1,0,116,2,106,9,106,10,116, 12,131,1,1,0,116,5,124,0,95,5,116,13,124,0,95, 13,100,2,83,0,41,3,122,41,73,110,115,116,97,108,108, @@ -2396,8 +2397,8 @@ 0,114,215,0,0,0,41,2,114,31,1,0,0,90,17,115, 117,112,112,111,114,116,101,100,95,108,111,97,100,101,114,115, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 8,95,105,110,115,116,97,108,108,154,5,0,0,115,16,0, - 0,0,0,2,8,1,6,1,20,1,10,1,12,1,12,4, + 8,95,105,110,115,116,97,108,108,155,5,0,0,115,16,0, + 0,0,0,2,8,1,6,1,22,1,10,1,12,1,12,4, 6,1,114,34,1,0,0,41,1,122,3,119,105,110,41,2, 114,1,0,0,0,114,2,0,0,0,41,1,114,49,0,0, 0,41,1,78,41,3,78,78,78,41,3,78,78,78,41,2, @@ -2430,7 +2431,7 @@ 0,0,0,114,4,0,0,0,114,6,0,0,0,218,8,60, 109,111,100,117,108,101,62,8,0,0,0,115,108,0,0,0, 4,16,4,1,4,1,2,1,6,3,8,17,8,5,8,5, - 8,6,8,12,8,10,8,9,8,5,8,7,10,22,10,119, + 8,6,8,12,8,10,8,9,8,5,8,7,10,22,10,120, 16,1,12,2,4,1,4,2,6,2,6,2,8,2,16,45, 8,34,8,19,8,12,8,12,8,28,8,17,10,55,10,12, 10,10,8,14,6,3,4,1,14,67,14,64,14,29,16,110, diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -139,9 +139,9 @@ &&TARGET_STORE_DEREF, &&TARGET_DELETE_DEREF, &&_unknown_opcode, - &&TARGET_CALL_FUNCTION_VAR, + &&_unknown_opcode, &&TARGET_CALL_FUNCTION_KW, - &&TARGET_CALL_FUNCTION_VAR_KW, + &&TARGET_CALL_FUNCTION_EX, &&TARGET_SETUP_WITH, &&TARGET_EXTENDED_ARG, &&TARGET_LIST_APPEND, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:31:26 2016 From: python-checkins at python.org (gregory.p.smith) Date: Fri, 09 Sep 2016 19:31:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Add_a_note_abo?= =?utf-8?q?ut_queue_not_being_safe_for_use_from_signal_handlers=2E?= Message-ID: <20160909193126.12514.83540.FA6E60F4@psf.io> https://hg.python.org/cpython/rev/137806ca59ce changeset: 103445:137806ca59ce branch: 3.5 parent: 103441:852b0f536791 user: Gregory P. Smith [Google Inc.] date: Fri Sep 09 12:30:34 2016 -0700 summary: Add a note about queue not being safe for use from signal handlers. issue14976. files: Doc/library/queue.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -186,6 +186,11 @@ t.join() +.. note:: + + The :mod:`queue` module is not safe for use from :mod:`signal` handlers as + it uses :mod:`threading` locks. + .. seealso:: Class :class:`multiprocessing.Queue` -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:31:26 2016 From: python-checkins at python.org (gregory.p.smith) Date: Fri, 09 Sep 2016 19:31:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Add_a_note_about_queue_not_being_safe_for_use_from_signa?= =?utf-8?q?l_handlers=2E?= Message-ID: <20160909193126.12852.20229.5A805917@psf.io> https://hg.python.org/cpython/rev/4e15e7618715 changeset: 103446:4e15e7618715 parent: 103444:a77756e480c2 parent: 103445:137806ca59ce user: Gregory P. Smith [Google Inc.] date: Fri Sep 09 12:31:05 2016 -0700 summary: Add a note about queue not being safe for use from signal handlers. issue14976. files: Doc/library/queue.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -189,6 +189,11 @@ t.join() +.. note:: + + The :mod:`queue` module is not safe for use from :mod:`signal` handlers as + it uses :mod:`threading` locks. + .. seealso:: Class :class:`multiprocessing.Queue` -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:43:37 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 19:43:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_unconvincing_use_of?= =?utf-8?q?_Py=5FLOCAL?= Message-ID: <20160909194335.21222.21226.AF23A85B@psf.io> https://hg.python.org/cpython/rev/330e0e28e9bd changeset: 103447:330e0e28e9bd user: Benjamin Peterson date: Fri Sep 09 12:42:51 2016 -0700 summary: remove unconvincing use of Py_LOCAL files: Objects/typeobject.c | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -549,7 +549,7 @@ static PyTypeObject *best_base(PyObject *); static int mro_internal(PyTypeObject *, PyObject **); -Py_LOCAL_INLINE(int) type_is_subtype_base_chain(PyTypeObject *, PyTypeObject *); +static int type_is_subtype_base_chain(PyTypeObject *, PyTypeObject *); static int compatible_for_assignment(PyTypeObject *, PyTypeObject *, const char *); static int add_subclass(PyTypeObject*, PyTypeObject*); static int add_all_subclasses(PyTypeObject *type, PyObject *bases); @@ -1333,7 +1333,7 @@ /* type test with subclassing support */ -Py_LOCAL_INLINE(int) +static int type_is_subtype_base_chain(PyTypeObject *a, PyTypeObject *b) { do { @@ -3805,7 +3805,7 @@ return PyImport_Import(copyreg_str); } -Py_LOCAL(PyObject *) +static PyObject * _PyType_GetSlotNames(PyTypeObject *cls) { PyObject *copyreg; @@ -3858,7 +3858,7 @@ return slotnames; } -Py_LOCAL(PyObject *) +static PyObject * _PyObject_GetState(PyObject *obj, int required) { PyObject *state; @@ -4004,7 +4004,7 @@ return state; } -Py_LOCAL(int) +static int _PyObject_GetNewArguments(PyObject *obj, PyObject **args, PyObject **kwargs) { PyObject *getnewargs, *getnewargs_ex; @@ -4100,7 +4100,7 @@ return 0; } -Py_LOCAL(int) +static int _PyObject_GetItemsIter(PyObject *obj, PyObject **listitems, PyObject **dictitems) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:47:15 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 19:47:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogbWVyZ2UgMy41ICgjMjgwNTEp?= Message-ID: <20160909194714.8490.81958.1CF76775@psf.io> https://hg.python.org/cpython/rev/ace662dbfdb6 changeset: 103449:ace662dbfdb6 parent: 103447:330e0e28e9bd parent: 103448:8e74e76deda1 user: Benjamin Peterson date: Fri Sep 09 12:46:49 2016 -0700 summary: merge 3.5 (#28051) files: Doc/whatsnew/3.5.rst | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -804,11 +804,12 @@ :func:`~asyncio.ensure_future`. (Contributed by Yury Selivanov.) -* New :meth:`loop.set_task_factory() ` - and :meth:`loop.set_task_factory() ` - methods to customize the task factory that - :meth:`loop.create_task() ` method uses. - (Contributed by Yury Selivanov.) +* New :meth:`loop.set_task_factory() + ` and + :meth:`loop.get_task_factory() ` + methods to customize the task factory that :meth:`loop.create_task() + ` method uses. (Contributed by Yury + Selivanov.) * New :meth:`Queue.join() ` and :meth:`Queue.task_done() ` queue methods. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:47:15 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 19:47:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_repair_errors_?= =?utf-8?b?aW4gKHNldHxnZXQpX3Rhc2tfZmFjdG9yeSBub3RlICgjMjgwNTEp?= Message-ID: <20160909194714.8701.42833.CBF3747A@psf.io> https://hg.python.org/cpython/rev/8e74e76deda1 changeset: 103448:8e74e76deda1 branch: 3.5 parent: 103445:137806ca59ce user: Benjamin Peterson date: Fri Sep 09 12:46:42 2016 -0700 summary: repair errors in (set|get)_task_factory note (#28051) files: Doc/whatsnew/3.5.rst | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -804,11 +804,12 @@ :func:`~asyncio.ensure_future`. (Contributed by Yury Selivanov.) -* New :meth:`loop.set_task_factory() ` - and :meth:`loop.set_task_factory() ` - methods to customize the task factory that - :meth:`loop.create_task() ` method uses. - (Contributed by Yury Selivanov.) +* New :meth:`loop.set_task_factory() + ` and + :meth:`loop.get_task_factory() ` + methods to customize the task factory that :meth:`loop.create_task() + ` method uses. (Contributed by Yury + Selivanov.) * New :meth:`Queue.join() ` and :meth:`Queue.task_done() ` queue methods. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:50:25 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 09 Sep 2016 19:50:25 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327213=3A_document?= =?utf-8?q?_changes_in_Misc/NEWS?= Message-ID: <20160909195025.12514.57381.02751F20@psf.io> https://hg.python.org/cpython/rev/854b08acca97 changeset: 103451:854b08acca97 user: Victor Stinner date: Fri Sep 09 12:43:42 2016 -0700 summary: Issue #27213: document changes in Misc/NEWS files: Misc/NEWS | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #27213: Rework CALL_FUNCTION* opcodes to produce shorter and more + efficient bytecode. Patch by Demur Rumed, design by Serhiy Storchaka, + reviewed by Serhiy Storchaka and Victor Stinner. + - Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:50:26 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 09 Sep 2016 19:50:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_=5FPyObject=5FFastCall?= =?utf-8?q?Keywords=28=29?= Message-ID: <20160909195025.48584.97036.40F5E9B5@psf.io> https://hg.python.org/cpython/rev/18d2d128cd1e changeset: 103450:18d2d128cd1e user: Victor Stinner date: Fri Sep 09 12:36:44 2016 -0700 summary: Add _PyObject_FastCallKeywords() Issue #27830: Add _PyObject_FastCallKeywords(): avoid the creation of a temporary dictionary for keyword arguments. Other changes: * Cleanup call_function() and fast_function() (ex: rename nk to nkwargs) * Remove now useless do_call(), replaced with _PyObject_FastCallKeywords() files: Include/abstract.h | 22 +++++++- Include/funcobject.h | 6 ++ Objects/abstract.c | 68 ++++++++++++++++++++++++ Python/ceval.c | 89 ++++++++++++------------------- 4 files changed, 130 insertions(+), 55 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -271,8 +271,8 @@ Py_ssize_t nargs); /* Call the callable object func with the "fast call" calling convention: - args is a C array for positional parameters (nargs is the number of - positional paramater), kwargs is a dictionary for keyword parameters. + args is a C array for positional arguments (nargs is the number of + positional arguments), kwargs is a dictionary for keyword arguments. If nargs is equal to zero, args can be NULL. kwargs can be NULL. nargs must be greater or equal to zero. @@ -283,6 +283,24 @@ PyObject **args, Py_ssize_t nargs, PyObject *kwargs); + /* Call the callable object func with the "fast call" calling convention: + args is a C array for positional arguments followed by values of + keyword arguments. Keys of keyword arguments are stored as a tuple + of strings in kwnames. nargs is the number of positional parameters at + the beginning of stack. The size of kwnames gives the number of keyword + values in the stack after positional arguments. + + If nargs is equal to zero and there is no keyword argument (kwnames is + NULL or its size is zero), args can be NULL. + + Return the result on success. Raise an exception and return NULL on + error. */ + PyAPI_FUNC(PyObject *) _PyObject_FastCallKeywords + (PyObject *func, + PyObject **args, + Py_ssize_t nargs, + PyObject *kwnames); + #define _PyObject_FastCall(func, args, nargs) \ _PyObject_FastCallDict((func), (args), (nargs), NULL) diff --git a/Include/funcobject.h b/Include/funcobject.h --- a/Include/funcobject.h +++ b/Include/funcobject.h @@ -64,6 +64,12 @@ PyObject **args, Py_ssize_t nargs, PyObject *kwargs); + +PyAPI_FUNC(PyObject *) _PyFunction_FastCallKeywords( + PyObject *func, + PyObject **stack, + Py_ssize_t nargs, + PyObject *kwnames); #endif /* Macros for direct access to these values. Type checks are *not* diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2366,6 +2366,74 @@ return result; } +static PyObject * +_PyStack_AsDict(PyObject **values, Py_ssize_t nkwargs, PyObject *kwnames, + PyObject *func) +{ + PyObject *kwdict; + Py_ssize_t i; + + kwdict = PyDict_New(); + if (kwdict == NULL) { + return NULL; + } + + for (i=0; i < nkwargs; i++) { + int err; + PyObject *key = PyTuple_GET_ITEM(kwnames, i); + PyObject *value = *values++; + + if (PyDict_GetItem(kwdict, key) != NULL) { + PyErr_Format(PyExc_TypeError, + "%.200s%s got multiple values " + "for keyword argument '%U'", + PyEval_GetFuncName(func), + PyEval_GetFuncDesc(func), + key); + Py_DECREF(kwdict); + return NULL; + } + + err = PyDict_SetItem(kwdict, key, value); + if (err) { + Py_DECREF(kwdict); + return NULL; + } + } + return kwdict; +} + +PyObject * +_PyObject_FastCallKeywords(PyObject *func, PyObject **stack, Py_ssize_t nargs, + PyObject *kwnames) +{ + PyObject *kwdict, *result; + Py_ssize_t nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames); + + assert(nargs >= 0); + assert(kwnames == NULL || PyTuple_CheckExact(kwnames)); + assert((nargs == 0 && nkwargs == 0) || stack != NULL); + + if (PyFunction_Check(func)) { + /* Fast-path: avoid temporary tuple or dict */ + return _PyFunction_FastCallKeywords(func, stack, nargs, kwnames); + } + + if (nkwargs > 0) { + kwdict = _PyStack_AsDict(stack + nargs, nkwargs, kwnames, func); + if (kwdict == NULL) { + return NULL; + } + } + else { + kwdict = NULL; + } + + result = _PyObject_FastCallDict(func, stack, nargs, kwdict); + Py_XDECREF(kwdict); + return result; +} + static PyObject* call_function_tail(PyObject *callable, PyObject *args) { diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -113,8 +113,7 @@ #else static PyObject * call_function(PyObject ***, Py_ssize_t, PyObject *); #endif -static PyObject * fast_function(PyObject *, PyObject ***, Py_ssize_t, PyObject *); -static PyObject * do_call(PyObject *, PyObject ***, Py_ssize_t, PyObject *); +static PyObject * fast_function(PyObject *, PyObject **, Py_ssize_t, PyObject *); static PyObject * do_call_core(PyObject *, PyObject *, PyObject *); static PyObject * create_keyword_args(PyObject *, PyObject ***, PyObject *); static PyObject * load_args(PyObject ***, Py_ssize_t); @@ -4940,7 +4939,7 @@ } static PyObject * -call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *names +call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *kwnames #ifdef WITH_TSC , uint64* pintr0, uint64* pintr1 #endif @@ -4949,8 +4948,8 @@ PyObject **pfunc = (*pp_stack) - oparg - 1; PyObject *func = *pfunc; PyObject *x, *w; - Py_ssize_t nk = names == NULL ? 0 : PyTuple_GET_SIZE(names); - Py_ssize_t nargs = oparg - nk; + Py_ssize_t nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames); + Py_ssize_t nargs = oparg - nkwargs; /* Always dispatch PyCFunction first, because these are presumed to be the most frequent callable object. @@ -4960,7 +4959,7 @@ PyThreadState *tstate = PyThreadState_GET(); PCALL(PCALL_CFUNCTION); - if (names == NULL && flags & (METH_NOARGS | METH_O)) { + if (kwnames == NULL && flags & (METH_NOARGS | METH_O)) { PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); if (flags & METH_NOARGS && nargs == 0) { @@ -4982,8 +4981,8 @@ } else { PyObject *callargs, *kwdict = NULL; - if (names != NULL) { - kwdict = create_keyword_args(names, pp_stack, func); + if (kwnames != NULL) { + kwdict = create_keyword_args(kwnames, pp_stack, func); if (kwdict == NULL) { x = NULL; goto cfuncerror; @@ -5003,6 +5002,9 @@ } } else { + Py_ssize_t nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames); + PyObject **stack; + if (PyMethod_Check(func) && PyMethod_GET_SELF(func) != NULL) { /* optimize access to bound methods */ PyObject *self = PyMethod_GET_SELF(func); @@ -5018,11 +5020,14 @@ Py_INCREF(func); } + stack = (*pp_stack) - nargs - nkwargs; + READ_TIMESTAMP(*pintr0); if (PyFunction_Check(func)) { - x = fast_function(func, pp_stack, nargs, names); - } else { - x = do_call(func, pp_stack, nargs, names); + x = fast_function(func, stack, nargs, kwnames); + } + else { + x = _PyObject_FastCallKeywords(func, stack, nargs, kwnames); } READ_TIMESTAMP(*pintr1); @@ -5055,8 +5060,8 @@ */ static PyObject* -_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t nargs, - PyObject *globals) +_PyFunction_FastCall(PyCodeObject *co, PyObject **args, Py_ssize_t nargs, + PyObject *globals) { PyFrameObject *f; PyThreadState *tstate = PyThreadState_GET(); @@ -5091,19 +5096,19 @@ return result; } -/* Similar to _PyFunction_FastCall() but keywords are passed a (key, value) - pairs in stack */ static PyObject * -fast_function(PyObject *func, PyObject ***pp_stack, Py_ssize_t nargs, PyObject *names) +fast_function(PyObject *func, PyObject **stack, + Py_ssize_t nargs, PyObject *kwnames) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); PyObject *kwdefs, *closure, *name, *qualname; PyObject **d; - Py_ssize_t nkwargs = names == NULL ? 0 : PyTuple_GET_SIZE(names); + Py_ssize_t nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames); Py_ssize_t nd; - PyObject **stack = (*pp_stack)-nargs-nkwargs; + + assert((nargs == 0 && nkwargs == 0) || stack != NULL); PCALL(PCALL_FUNCTION); PCALL(PCALL_FAST_FUNCTION); @@ -5112,15 +5117,14 @@ co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { if (argdefs == NULL && co->co_argcount == nargs) { - return _PyFunction_FastCallNoKw(co, stack, nargs, globals); + return _PyFunction_FastCall(co, stack, nargs, globals); } else if (nargs == 0 && argdefs != NULL && co->co_argcount == Py_SIZE(argdefs)) { /* function called with no arguments, but all parameters have a default value: use default values as arguments .*/ stack = &PyTuple_GET_ITEM(argdefs, 0); - return _PyFunction_FastCallNoKw(co, stack, Py_SIZE(argdefs), - globals); + return _PyFunction_FastCall(co, stack, Py_SIZE(argdefs), globals); } } @@ -5140,12 +5144,19 @@ return _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, stack, nargs, NULL, 0, - names, stack + nargs, + kwnames, stack + nargs, d, (int)nd, kwdefs, closure, name, qualname); } PyObject * +_PyFunction_FastCallKeywords(PyObject *func, PyObject **stack, + Py_ssize_t nargs, PyObject *kwnames) +{ + return fast_function(func, stack, nargs, kwnames); +} + +PyObject * _PyFunction_FastCallDict(PyObject *func, PyObject **args, Py_ssize_t nargs, PyObject *kwargs) { @@ -5172,15 +5183,14 @@ { /* Fast paths */ if (argdefs == NULL && co->co_argcount == nargs) { - return _PyFunction_FastCallNoKw(co, args, nargs, globals); + return _PyFunction_FastCall(co, args, nargs, globals); } else if (nargs == 0 && argdefs != NULL && co->co_argcount == Py_SIZE(argdefs)) { /* function called with no arguments, but all parameters have a default value: use default values as arguments .*/ args = &PyTuple_GET_ITEM(argdefs, 0); - return _PyFunction_FastCallNoKw(co, args, Py_SIZE(argdefs), - globals); + return _PyFunction_FastCall(co, args, Py_SIZE(argdefs), globals); } } @@ -5242,8 +5252,8 @@ return NULL; while (--nk >= 0) { int err; + PyObject *key = PyTuple_GET_ITEM(names, nk); PyObject *value = EXT_POP(*pp_stack); - PyObject *key = PyTuple_GET_ITEM(names, nk); if (PyDict_GetItem(kwdict, key) != NULL) { PyErr_Format(PyExc_TypeError, "%.200s%s got multiple values " @@ -5282,33 +5292,6 @@ } static PyObject * -do_call(PyObject *func, PyObject ***pp_stack, Py_ssize_t nargs, PyObject *kwnames) -{ - PyObject *callargs, *kwdict, *result; - - if (kwnames != NULL) { - kwdict = create_keyword_args(kwnames, pp_stack, func); - if (kwdict == NULL) { - return NULL; - } - } - else { - kwdict = NULL; - } - - callargs = load_args(pp_stack, nargs); - if (callargs == NULL) { - Py_XDECREF(kwdict); - return NULL; - } - - result = do_call_core(func, callargs, kwdict); - Py_XDECREF(callargs); - Py_XDECREF(kwdict); - return result; -} - -static PyObject * do_call_core(PyObject *func, PyObject *callargs, PyObject *kwdict) { #ifdef CALL_PROFILE -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:55:56 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 19:55:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogUmVuYW1lIHRlc3RfcGVwIyMj?= =?utf-8?q?=23=2Epy_files?= Message-ID: <20160909195556.59448.47711.1B4877CE@psf.io> https://hg.python.org/cpython/rev/7464142fc867 changeset: 103453:7464142fc867 user: Zachary Ware date: Fri Sep 09 12:55:37 2016 -0700 summary: Rename test_pep####.py files files: Lib/test/test_baseexception.py | 0 Lib/test/test_dict_version.py | 0 Lib/test/test_exception_hierarchy.py | 0 Lib/test/test_generator_stop.py | 0 Lib/test/test_tokenize.py | 11 ++++++----- Lib/test/test_unicode_file_functions.py | 0 Lib/test/test_unicode_identifiers.py | 0 Lib/test/test_utf8source.py | 0 Lib/test/test_yield_from.py | 0 9 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Lib/test/test_pep352.py b/Lib/test/test_baseexception.py rename from Lib/test/test_pep352.py rename to Lib/test/test_baseexception.py diff --git a/Lib/test/test_pep509.py b/Lib/test/test_dict_version.py rename from Lib/test/test_pep509.py rename to Lib/test/test_dict_version.py diff --git a/Lib/test/test_pep3151.py b/Lib/test/test_exception_hierarchy.py rename from Lib/test/test_pep3151.py rename to Lib/test/test_exception_hierarchy.py diff --git a/Lib/test/test_pep479.py b/Lib/test/test_generator_stop.py rename from Lib/test/test_pep479.py rename to Lib/test/test_generator_stop.py diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py --- a/Lib/test/test_tokenize.py +++ b/Lib/test/test_tokenize.py @@ -1529,12 +1529,13 @@ tempdir = os.path.dirname(fn) or os.curdir testfiles = glob.glob(os.path.join(tempdir, "test*.py")) - # Tokenize is broken on test_pep3131.py because regular expressions are - # broken on the obscure unicode identifiers in it. *sigh* - # With roundtrip extended to test the 5-tuple mode of untokenize, - # 7 more testfiles fail. Remove them also until the failure is diagnosed. + # Tokenize is broken on test_unicode_identifiers.py because regular + # expressions are broken on the obscure unicode identifiers in it. + # *sigh* With roundtrip extended to test the 5-tuple mode of + # untokenize, 7 more testfiles fail. Remove them also until the + # failure is diagnosed. - testfiles.remove(os.path.join(tempdir, "test_pep3131.py")) + testfiles.remove(os.path.join(tempdir, "test_unicode_identifiers.py")) for f in ('buffer', 'builtin', 'fileio', 'inspect', 'os', 'platform', 'sys'): testfiles.remove(os.path.join(tempdir, "test_%s.py") % f) diff --git a/Lib/test/test_pep277.py b/Lib/test/test_unicode_file_functions.py rename from Lib/test/test_pep277.py rename to Lib/test/test_unicode_file_functions.py diff --git a/Lib/test/test_pep3131.py b/Lib/test/test_unicode_identifiers.py rename from Lib/test/test_pep3131.py rename to Lib/test/test_unicode_identifiers.py diff --git a/Lib/test/test_pep3120.py b/Lib/test/test_utf8source.py rename from Lib/test/test_pep3120.py rename to Lib/test/test_utf8source.py diff --git a/Lib/test/test_pep380.py b/Lib/test/test_yield_from.py rename from Lib/test/test_pep380.py rename to Lib/test/test_yield_from.py -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 15:55:56 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 19:55:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_running_test=5Ftokeniz?= =?utf-8?q?e_directly?= Message-ID: <20160909195556.69808.84992.B8C3A453@psf.io> https://hg.python.org/cpython/rev/4ede709268a0 changeset: 103452:4ede709268a0 user: Zachary Ware date: Fri Sep 09 12:55:14 2016 -0700 summary: Fix running test_tokenize directly files: Lib/test/test_tokenize.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py --- a/Lib/test/test_tokenize.py +++ b/Lib/test/test_tokenize.py @@ -3,7 +3,7 @@ STRING, ENDMARKER, ENCODING, tok_name, detect_encoding, open as tokenize_open, Untokenizer) from io import BytesIO -from unittest import TestCase, mock +from unittest import TestCase, mock, main import os import token @@ -1564,4 +1564,4 @@ if __name__ == "__main__": - unittest.main() + main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:01:19 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 09 Sep 2016 20:01:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Rename_Future?= =?utf-8?q?=2E=5Fblocking_to_=5Fasyncio=5Ffuture=5Fblocking=2E?= Message-ID: <20160909200117.69577.82658.A7C50587@psf.io> https://hg.python.org/cpython/rev/ab1e10e3dc35 changeset: 103454:ab1e10e3dc35 branch: 3.5 parent: 103448:8e74e76deda1 user: Guido van Rossum date: Fri Sep 09 12:54:54 2016 -0700 summary: Rename Future._blocking to _asyncio_future_blocking. This is now an official "protected" API that can be used to write classes that are duck-type-compatible with Future without subclassing it. (For that purpose I also changed isinstance(result, Future) to check for this attribute instead.) Hopefully Amber Brown can use this to make Twisted.Deferred compatible with asyncio.Future. Tests and docs are TBD. files: Lib/asyncio/futures.py | 12 ++++++++++-- Lib/asyncio/tasks.py | 7 ++++--- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/Lib/asyncio/futures.py b/Lib/asyncio/futures.py --- a/Lib/asyncio/futures.py +++ b/Lib/asyncio/futures.py @@ -134,7 +134,15 @@ _loop = None _source_traceback = None - _blocking = False # proper use of future (yield vs yield from) + # This field is used for a dual purpose: + # - Its presence is a marker to declare that a class implements + # the Future protocol (i.e. is intended to be duck-type compatible). + # The value must also be not-None, to enable a subclass to declare + # that it is not compatible by setting this to None. + # - It is set by __iter__() below so that Task._step() can tell + # the difference between `yield from Future()` (correct) vs. + # `yield Future()` (incorrect). + _asyncio_future_blocking = False _log_traceback = False # Used for Python 3.4 and later _tb_logger = None # Used for Python 3.3 only @@ -357,7 +365,7 @@ def __iter__(self): if not self.done(): - self._blocking = True + self._asyncio_future_blocking = True yield self # This tells Task to wait for completion. assert self.done(), "yield from wasn't used with future" return self.result() # May raise too. diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -249,7 +249,8 @@ self.set_exception(exc) raise else: - if isinstance(result, futures.Future): + blocking = getattr(result, '_asyncio_future_blocking', None) + if blocking is not None: # Yielded Future must come from Future.__iter__(). if result._loop is not self._loop: self._loop.call_soon( @@ -257,8 +258,8 @@ RuntimeError( 'Task {!r} got Future {!r} attached to a ' 'different loop'.format(self, result))) - elif result._blocking: - result._blocking = False + elif blocking: + result._asyncio_future_blocking = False result.add_done_callback(self._wakeup) self._fut_waiter = result if self._must_cancel: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:01:19 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 09 Sep 2016 20:01:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Rename_Future=2E=5Fblocking_to_=5Fasyncio=5Ffuture=5Fblo?= =?utf-8?q?cking=2E?= Message-ID: <20160909200118.87391.12149.6994ACAB@psf.io> https://hg.python.org/cpython/rev/b2a5aed737bf changeset: 103455:b2a5aed737bf parent: 103453:7464142fc867 parent: 103454:ab1e10e3dc35 user: Guido van Rossum date: Fri Sep 09 12:58:15 2016 -0700 summary: Rename Future._blocking to _asyncio_future_blocking. This is now an official "protected" API that can be used to write classes that are duck-type-compatible with Future without subclassing it. (For that purpose I also changed isinstance(result, Future) to check for this attribute instead.) Hopefully Amber Brown can use this to make Twisted.Deferred compatible with asyncio.Future. Tests and docs are TBD. (Also there are more isinstance() checks to fix.) files: Lib/asyncio/futures.py | 12 ++++++++++-- Lib/asyncio/tasks.py | 7 ++++--- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/Lib/asyncio/futures.py b/Lib/asyncio/futures.py --- a/Lib/asyncio/futures.py +++ b/Lib/asyncio/futures.py @@ -134,7 +134,15 @@ _loop = None _source_traceback = None - _blocking = False # proper use of future (yield vs yield from) + # This field is used for a dual purpose: + # - Its presence is a marker to declare that a class implements + # the Future protocol (i.e. is intended to be duck-type compatible). + # The value must also be not-None, to enable a subclass to declare + # that it is not compatible by setting this to None. + # - It is set by __iter__() below so that Task._step() can tell + # the difference between `yield from Future()` (correct) vs. + # `yield Future()` (incorrect). + _asyncio_future_blocking = False _log_traceback = False # Used for Python 3.4 and later _tb_logger = None # Used for Python 3.3 only @@ -357,7 +365,7 @@ def __iter__(self): if not self.done(): - self._blocking = True + self._asyncio_future_blocking = True yield self # This tells Task to wait for completion. assert self.done(), "yield from wasn't used with future" return self.result() # May raise too. diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -249,7 +249,8 @@ self.set_exception(exc) raise else: - if isinstance(result, futures.Future): + blocking = getattr(result, '_asyncio_future_blocking', None) + if blocking is not None: # Yielded Future must come from Future.__iter__(). if result._loop is not self._loop: self._loop.call_soon( @@ -257,8 +258,8 @@ RuntimeError( 'Task {!r} got Future {!r} attached to a ' 'different loop'.format(self, result))) - elif result._blocking: - result._blocking = False + elif blocking: + result._asyncio_future_blocking = False result.add_done_callback(self._wakeup) self._fut_waiter = result if self._must_cancel: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:12:08 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 20:12:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2319489=3A_Merge_with_3=2E5?= Message-ID: <20160909201206.13175.93299.480612C8@psf.io> https://hg.python.org/cpython/rev/0f94a8fa5445 changeset: 103457:0f94a8fa5445 parent: 103455:b2a5aed737bf parent: 103456:274b25cd501f user: Zachary Ware date: Fri Sep 09 13:11:58 2016 -0700 summary: Closes #19489: Merge with 3.5 files: Doc/conf.py | 5 +- Doc/tools/pydoctheme/static/pydoctheme.css | 11 +++ Doc/tools/templates/customsourcelink.html | 10 +++ Doc/tools/templates/layout.html | 30 ++++++--- 4 files changed, 45 insertions(+), 11 deletions(-) diff --git a/Doc/conf.py b/Doc/conf.py --- a/Doc/conf.py +++ b/Doc/conf.py @@ -60,7 +60,10 @@ # Custom sidebar templates, filenames relative to this file. html_sidebars = { - 'index': 'indexsidebar.html', + # Defaults taken from http://www.sphinx-doc.org/en/stable/config.html#confval-html_sidebars + # Removes the quick search block + '**': ['localtoc.html', 'relations.html', 'customsourcelink.html'], + 'index': ['indexsidebar.html'], } # Additional templates that should be rendered to pages. diff --git a/Doc/tools/pydoctheme/static/pydoctheme.css b/Doc/tools/pydoctheme/static/pydoctheme.css --- a/Doc/tools/pydoctheme/static/pydoctheme.css +++ b/Doc/tools/pydoctheme/static/pydoctheme.css @@ -22,6 +22,16 @@ border-bottom: 1px solid #ccc; } +.inline-search { + display: inline; +} +form.inline-search input { + display: inline; +} +form.inline-search input[type="submit"] { + width: 30px; +} + div.sphinxsidebar { background-color: #eeeeee; border-radius: 5px; @@ -45,6 +55,7 @@ color: #0095C4; } +form.inline-search input, div.sphinxsidebar input { font-family: 'Lucida Grande',Arial,sans-serif; border: 1px solid #999999; diff --git a/Doc/tools/templates/customsourcelink.html b/Doc/tools/templates/customsourcelink.html new file mode 100644 --- /dev/null +++ b/Doc/tools/templates/customsourcelink.html @@ -0,0 +1,10 @@ +{%- if show_source and has_source and sourcename %} + +{%- endif %} diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -12,8 +12,28 @@ {%- endif %} {% endblock %} +{%- macro searchbox() %} +{# modified from sphinx/themes/basic/searchbox.html #} + + +{%- endmacro %} {% block relbar1 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} {% block relbar2 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} +{% block relbaritems %} + {%- if pagename != "search" and builder != "singlehtml" %} +
  • + {{ searchbox() }} + {{ reldelim2 }} +
  • + {%- endif %} +{% endblock %} {% block extrahead %} {% if not embedded %}{% endif %} @@ -90,13 +110,3 @@ {% trans sphinx_version=sphinx_version|e %}Created using Sphinx {{ sphinx_version }}.{% endtrans %} {% endblock %} -{% block sidebarsourcelink %} -{%- if show_source and has_source and sourcename %} -

    {{ _('This Page') }}

    - -{%- endif %} -{% endblock %} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:12:08 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 20:12:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE5NDg5?= =?utf-8?q?=3A_Move_the_search_box_from_sidebar_to_header_and_footer=2E?= Message-ID: <20160909201206.21095.71229.C1D141FA@psf.io> https://hg.python.org/cpython/rev/274b25cd501f changeset: 103456:274b25cd501f branch: 3.5 parent: 103454:ab1e10e3dc35 user: Zachary Ware date: Fri Sep 09 13:11:27 2016 -0700 summary: Issue #19489: Move the search box from sidebar to header and footer. files: Doc/conf.py | 5 +- Doc/tools/pydoctheme/static/pydoctheme.css | 11 +++ Doc/tools/templates/customsourcelink.html | 10 +++ Doc/tools/templates/layout.html | 30 ++++++--- 4 files changed, 45 insertions(+), 11 deletions(-) diff --git a/Doc/conf.py b/Doc/conf.py --- a/Doc/conf.py +++ b/Doc/conf.py @@ -60,7 +60,10 @@ # Custom sidebar templates, filenames relative to this file. html_sidebars = { - 'index': 'indexsidebar.html', + # Defaults taken from http://www.sphinx-doc.org/en/stable/config.html#confval-html_sidebars + # Removes the quick search block + '**': ['localtoc.html', 'relations.html', 'customsourcelink.html'], + 'index': ['indexsidebar.html'], } # Additional templates that should be rendered to pages. diff --git a/Doc/tools/pydoctheme/static/pydoctheme.css b/Doc/tools/pydoctheme/static/pydoctheme.css --- a/Doc/tools/pydoctheme/static/pydoctheme.css +++ b/Doc/tools/pydoctheme/static/pydoctheme.css @@ -22,6 +22,16 @@ border-bottom: 1px solid #ccc; } +.inline-search { + display: inline; +} +form.inline-search input { + display: inline; +} +form.inline-search input[type="submit"] { + width: 30px; +} + div.sphinxsidebar { background-color: #eeeeee; border-radius: 5px; @@ -45,6 +55,7 @@ color: #0095C4; } +form.inline-search input, div.sphinxsidebar input { font-family: 'Lucida Grande',Arial,sans-serif; border: 1px solid #999999; diff --git a/Doc/tools/templates/customsourcelink.html b/Doc/tools/templates/customsourcelink.html new file mode 100644 --- /dev/null +++ b/Doc/tools/templates/customsourcelink.html @@ -0,0 +1,10 @@ +{%- if show_source and has_source and sourcename %} + +{%- endif %} diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -12,8 +12,28 @@ {%- endif %} {% endblock %} +{%- macro searchbox() %} +{# modified from sphinx/themes/basic/searchbox.html #} + + +{%- endmacro %} {% block relbar1 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} {% block relbar2 %} {% if builder != 'qthelp' %} {{ relbar() }} {% endif %} {% endblock %} +{% block relbaritems %} + {%- if pagename != "search" and builder != "singlehtml" %} +
  • + {{ searchbox() }} + {{ reldelim2 }} +
  • + {%- endif %} +{% endblock %} {% block extrahead %} {% if not embedded %}{% endif %} @@ -90,13 +110,3 @@ {% trans sphinx_version=sphinx_version|e %}Created using Sphinx {{ sphinx_version }}.{% endtrans %} {% endblock %} -{% block sidebarsourcelink %} -{%- if show_source and has_source and sourcename %} -

    {{ _('This Page') }}

    - -{%- endif %} -{% endblock %} -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:15:57 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 20:15:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2319489=3A_Merge_with_3=2E5?= Message-ID: <20160909201556.21157.1403.B2CC6842@psf.io> https://hg.python.org/cpython/rev/086c3e7a7955 changeset: 103459:086c3e7a7955 parent: 103457:0f94a8fa5445 parent: 103458:8927417c5e88 user: Zachary Ware date: Fri Sep 09 13:15:47 2016 -0700 summary: Issue #19489: Merge with 3.5 files: Misc/ACKS | 1 + Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -59,6 +59,7 @@ Jeffrey Armstrong Jason Asbahr David Ascher +Ammar Askar Chris AtLee Aymeric Augustin John Aycock diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -773,6 +773,9 @@ Documentation ------------- +- Issue #19489: Moved the search box from the sidebar to the header and footer + of each page. Patch by Ammar Askar. + - Issue #27285: Update documentation to reflect the deprecation of ``pyvenv`` and normalize on the term "virtual environment". Patch by Steve Piercy. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:15:57 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 20:15:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzE5NDg5?= =?utf-8?q?=3A_Add_NEWS_and_ACKS?= Message-ID: <20160909201556.87425.89010.F5BBE684@psf.io> https://hg.python.org/cpython/rev/8927417c5e88 changeset: 103458:8927417c5e88 branch: 3.5 parent: 103456:274b25cd501f user: Zachary Ware date: Fri Sep 09 13:14:42 2016 -0700 summary: Issue #19489: Add NEWS and ACKS files: Misc/ACKS | 1 + Misc/NEWS | 3 +++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -59,6 +59,7 @@ Jeffrey Armstrong Jason Asbahr David Ascher +Ammar Askar Chris AtLee Aymeric Augustin John Aycock diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -928,6 +928,9 @@ Documentation ------------- +- Issue #19489: Moved the search box from the sidebar to the header and footer + of each page. Patch by Ammar Askar. + - Issue #24136: Document the new PEP 448 unpacking syntax of 3.5. - Issue #26736: Used HTTPS for external links in the documentation if possible. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:19:19 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 20:19:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2327314=3A_Fixes_l?= =?utf-8?q?auncher_installer_upgrade_table=2E?= Message-ID: <20160909201919.21311.80120.23F49017@psf.io> https://hg.python.org/cpython/rev/85460db4c310 changeset: 103460:85460db4c310 user: Steve Dower date: Fri Sep 09 13:19:09 2016 -0700 summary: Closes #27314: Fixes launcher installer upgrade table. files: Tools/msi/common.wxs | 2 ++ Tools/msi/launcher/launcher.wixproj | 2 +- Tools/msi/launcher/launcher.wxs | 11 +++++++---- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/Tools/msi/common.wxs b/Tools/msi/common.wxs --- a/Tools/msi/common.wxs +++ b/Tools/msi/common.wxs @@ -20,10 +20,12 @@ + + diff --git a/Tools/msi/launcher/launcher.wixproj b/Tools/msi/launcher/launcher.wixproj --- a/Tools/msi/launcher/launcher.wixproj +++ b/Tools/msi/launcher/launcher.wixproj @@ -5,7 +5,7 @@ 2.0 launcher Package - UpgradeCode=1B68A0EC-4DD3-5134-840E-73854B0863F1;$(DefineConstants) + UpgradeCode=1B68A0EC-4DD3-5134-840E-73854B0863F1;SuppressUpgradeTable=1;$(DefineConstants) true ICE80
    diff --git a/Tools/msi/launcher/launcher.wxs b/Tools/msi/launcher/launcher.wxs --- a/Tools/msi/launcher/launcher.wxs +++ b/Tools/msi/launcher/launcher.wxs @@ -29,18 +29,21 @@ NOT Installed AND NOT ALLUSERS=1 NOT Installed AND ALLUSERS=1 - UPGRADE or REMOVE_350_LAUNCHER + UPGRADE or REMOVE_350_LAUNCHER or REMOVE_360A1_LAUNCHER + + + + + - - + - Installed OR NOT BLOCK_360A1_LAUNCHER -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:25:56 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 20:25:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_tix_deprecation_to_wha?= =?utf-8?q?tsnew?= Message-ID: <20160909202556.48584.5986.FE9B3315@psf.io> https://hg.python.org/cpython/rev/74850afe99be changeset: 103461:74850afe99be user: Zachary Ware date: Fri Sep 09 13:25:44 2016 -0700 summary: Add tix deprecation to whatsnew files: Doc/whatsnew/3.6.rst | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -981,6 +981,9 @@ been deprecated in previous versions of Python in favour of :meth:`importlib.abc.Loader.exec_module`. +* The :mod:`tkinter.tix` module is now deprecated. :mod:`tkinter` users should + use :mod:`tkinter.ttk` instead. + Deprecated functions and types of the C API ------------------------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:27:00 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 20:27:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_unused_suspicious_r?= =?utf-8?q?ules?= Message-ID: <20160909202655.36191.8188.ABEA9785@psf.io> https://hg.python.org/cpython/rev/351ff418be21 changeset: 103462:351ff418be21 user: Zachary Ware date: Fri Sep 09 13:26:47 2016 -0700 summary: Remove unused suspicious rules files: Doc/tools/susp-ignored.csv | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -276,9 +276,6 @@ whatsnew/3.2,,:gz,">>> with tarfile.open(name='myarchive.tar.gz', mode='w:gz') as tf:" whatsnew/3.2,,:location,zope9-location = ${zope9:location} whatsnew/3.2,,:prefix,zope-conf = ${custom:prefix}/etc/zope.conf -whatsnew/changelog,,:version,import sys; I = version[:version.index(' ')] -whatsnew/changelog,,:gz,": TarFile opened with external fileobj and ""w:gz"" mode didn't" -whatsnew/changelog,,::,": Use ""127.0.0.1"" or ""::1"" instead of ""localhost"" as much as" library/tarfile,149,:xz,'x:xz' library/xml.etree.elementtree,290,:sometag,prefix:sometag library/xml.etree.elementtree,301,:fictional," Results for project Python default, build date 2016-09-09 18:40:42 +0000 commit: c477ea0d15ba previous commit: 344f44bd793f revision date: 2016-09-09 18:38:38 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.22% -0.51% 9.80% 13.19% :-) pybench 0.19% -1.82% 5.73% 7.07% :-( regex_v8 2.65% -0.70% -3.37% 4.05% :-) nbody 0.11% 2.71% 1.58% 5.83% :-( json_dump_v2 0.28% -2.26% -5.19% 11.84% :-) normal_startup 1.24% 4.48% 2.89% 6.46% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-python-default-2016-09-09/ Note: Benchmark results are measured in seconds. Subject Label Legend: Attributes are determined based on the performance evolution of the workloads compared to the previous measurement iteration. NEUTRAL: performance did not change by more than 1% for any workload GOOD: performance improved by more than 1% for at least one workload and there is no regression greater than 1% BAD: performance dropped by more than 1% for at least one workload and there is no improvement greater than 1% UGLY: performance improved by more than 1% for at least one workload and also dropped by more than 1% for at least one workload Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. From python-checkins at python.org Fri Sep 9 16:41:43 2016 From: python-checkins at python.org (eric.snow) Date: Fri, 09 Sep 2016 20:41:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324320=3A_Drop_an_?= =?utf-8?q?old_setuptools-induced_hack=2E?= Message-ID: <20160909204143.48584.83777.3D77CF5D@psf.io> https://hg.python.org/cpython/rev/821663556d87 changeset: 103463:821663556d87 user: Eric Snow date: Fri Sep 09 13:30:54 2016 -0700 summary: Issue #24320: Drop an old setuptools-induced hack. files: Lib/importlib/_bootstrap_external.py | 5 - Python/importlib_external.h | 105 +++++++------- 2 files changed, 52 insertions(+), 58 deletions(-) diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -1440,8 +1440,3 @@ if _os.__name__ == 'nt': sys.meta_path.append(WindowsRegistryFinder) sys.meta_path.append(PathFinder) - - # XXX We expose a couple of classes in _bootstrap for the sake of - # a setuptools bug (https://bitbucket.org/pypa/setuptools/issue/378). - _bootstrap_module.FileFinder = FileFinder - _bootstrap_module.SourceFileLoader = SourceFileLoader diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -2380,61 +2380,60 @@ 18,3,2,1,14,1,16,2,10,1,12,3,10,1,12,3, 10,1,10,1,12,3,14,1,14,1,10,1,10,1,10,1, 114,32,1,0,0,99,1,0,0,0,0,0,0,0,2,0, - 0,0,3,0,0,0,67,0,0,0,115,86,0,0,0,116, + 0,0,3,0,0,0,67,0,0,0,115,74,0,0,0,116, 0,124,0,131,1,1,0,116,1,131,0,125,1,116,2,106, 3,106,4,116,5,106,6,124,1,152,1,142,0,103,1,131, 1,1,0,116,7,106,8,100,1,107,2,114,58,116,2,106, 9,106,10,116,11,131,1,1,0,116,2,106,9,106,10,116, - 12,131,1,1,0,116,5,124,0,95,5,116,13,124,0,95, - 13,100,2,83,0,41,3,122,41,73,110,115,116,97,108,108, - 32,116,104,101,32,112,97,116,104,45,98,97,115,101,100,32, - 105,109,112,111,114,116,32,99,111,109,112,111,110,101,110,116, - 115,46,114,27,1,0,0,78,41,14,114,32,1,0,0,114, - 159,0,0,0,114,8,0,0,0,114,253,0,0,0,114,147, - 0,0,0,114,5,1,0,0,114,18,1,0,0,114,3,0, - 0,0,114,109,0,0,0,218,9,109,101,116,97,95,112,97, - 116,104,114,161,0,0,0,114,166,0,0,0,114,248,0,0, - 0,114,215,0,0,0,41,2,114,31,1,0,0,90,17,115, - 117,112,112,111,114,116,101,100,95,108,111,97,100,101,114,115, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 8,95,105,110,115,116,97,108,108,155,5,0,0,115,16,0, - 0,0,0,2,8,1,6,1,22,1,10,1,12,1,12,4, - 6,1,114,34,1,0,0,41,1,122,3,119,105,110,41,2, - 114,1,0,0,0,114,2,0,0,0,41,1,114,49,0,0, - 0,41,1,78,41,3,78,78,78,41,3,78,78,78,41,2, - 114,62,0,0,0,114,62,0,0,0,41,1,78,41,1,78, - 41,58,114,111,0,0,0,114,12,0,0,0,90,37,95,67, - 65,83,69,95,73,78,83,69,78,83,73,84,73,86,69,95, - 80,76,65,84,70,79,82,77,83,95,66,89,84,69,83,95, - 75,69,89,114,11,0,0,0,114,13,0,0,0,114,19,0, - 0,0,114,21,0,0,0,114,30,0,0,0,114,40,0,0, - 0,114,41,0,0,0,114,45,0,0,0,114,46,0,0,0, - 114,48,0,0,0,114,58,0,0,0,218,4,116,121,112,101, - 218,8,95,95,99,111,100,101,95,95,114,142,0,0,0,114, - 17,0,0,0,114,132,0,0,0,114,16,0,0,0,114,20, - 0,0,0,90,17,95,82,65,87,95,77,65,71,73,67,95, - 78,85,77,66,69,82,114,77,0,0,0,114,76,0,0,0, - 114,88,0,0,0,114,78,0,0,0,90,23,68,69,66,85, - 71,95,66,89,84,69,67,79,68,69,95,83,85,70,70,73, - 88,69,83,90,27,79,80,84,73,77,73,90,69,68,95,66, - 89,84,69,67,79,68,69,95,83,85,70,70,73,88,69,83, - 114,83,0,0,0,114,89,0,0,0,114,95,0,0,0,114, - 99,0,0,0,114,101,0,0,0,114,120,0,0,0,114,127, - 0,0,0,114,139,0,0,0,114,145,0,0,0,114,148,0, - 0,0,114,153,0,0,0,218,6,111,98,106,101,99,116,114, - 160,0,0,0,114,165,0,0,0,114,166,0,0,0,114,181, - 0,0,0,114,191,0,0,0,114,207,0,0,0,114,215,0, - 0,0,114,220,0,0,0,114,226,0,0,0,114,221,0,0, - 0,114,227,0,0,0,114,246,0,0,0,114,248,0,0,0, - 114,5,1,0,0,114,23,1,0,0,114,159,0,0,0,114, - 32,1,0,0,114,34,1,0,0,114,4,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,8,60, - 109,111,100,117,108,101,62,8,0,0,0,115,108,0,0,0, - 4,16,4,1,4,1,2,1,6,3,8,17,8,5,8,5, - 8,6,8,12,8,10,8,9,8,5,8,7,10,22,10,120, - 16,1,12,2,4,1,4,2,6,2,6,2,8,2,16,45, - 8,34,8,19,8,12,8,12,8,28,8,17,10,55,10,12, - 10,10,8,14,6,3,4,1,14,67,14,64,14,29,16,110, - 14,41,18,45,18,16,4,3,18,53,14,60,14,42,14,127, - 0,5,14,127,0,22,10,23,8,11,8,68, + 12,131,1,1,0,100,2,83,0,41,3,122,41,73,110,115, + 116,97,108,108,32,116,104,101,32,112,97,116,104,45,98,97, + 115,101,100,32,105,109,112,111,114,116,32,99,111,109,112,111, + 110,101,110,116,115,46,114,27,1,0,0,78,41,13,114,32, + 1,0,0,114,159,0,0,0,114,8,0,0,0,114,253,0, + 0,0,114,147,0,0,0,114,5,1,0,0,114,18,1,0, + 0,114,3,0,0,0,114,109,0,0,0,218,9,109,101,116, + 97,95,112,97,116,104,114,161,0,0,0,114,166,0,0,0, + 114,248,0,0,0,41,2,114,31,1,0,0,90,17,115,117, + 112,112,111,114,116,101,100,95,108,111,97,100,101,114,115,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,8, + 95,105,110,115,116,97,108,108,155,5,0,0,115,12,0,0, + 0,0,2,8,1,6,1,22,1,10,1,12,1,114,34,1, + 0,0,41,1,122,3,119,105,110,41,2,114,1,0,0,0, + 114,2,0,0,0,41,1,114,49,0,0,0,41,1,78,41, + 3,78,78,78,41,3,78,78,78,41,2,114,62,0,0,0, + 114,62,0,0,0,41,1,78,41,1,78,41,58,114,111,0, + 0,0,114,12,0,0,0,90,37,95,67,65,83,69,95,73, + 78,83,69,78,83,73,84,73,86,69,95,80,76,65,84,70, + 79,82,77,83,95,66,89,84,69,83,95,75,69,89,114,11, + 0,0,0,114,13,0,0,0,114,19,0,0,0,114,21,0, + 0,0,114,30,0,0,0,114,40,0,0,0,114,41,0,0, + 0,114,45,0,0,0,114,46,0,0,0,114,48,0,0,0, + 114,58,0,0,0,218,4,116,121,112,101,218,8,95,95,99, + 111,100,101,95,95,114,142,0,0,0,114,17,0,0,0,114, + 132,0,0,0,114,16,0,0,0,114,20,0,0,0,90,17, + 95,82,65,87,95,77,65,71,73,67,95,78,85,77,66,69, + 82,114,77,0,0,0,114,76,0,0,0,114,88,0,0,0, + 114,78,0,0,0,90,23,68,69,66,85,71,95,66,89,84, + 69,67,79,68,69,95,83,85,70,70,73,88,69,83,90,27, + 79,80,84,73,77,73,90,69,68,95,66,89,84,69,67,79, + 68,69,95,83,85,70,70,73,88,69,83,114,83,0,0,0, + 114,89,0,0,0,114,95,0,0,0,114,99,0,0,0,114, + 101,0,0,0,114,120,0,0,0,114,127,0,0,0,114,139, + 0,0,0,114,145,0,0,0,114,148,0,0,0,114,153,0, + 0,0,218,6,111,98,106,101,99,116,114,160,0,0,0,114, + 165,0,0,0,114,166,0,0,0,114,181,0,0,0,114,191, + 0,0,0,114,207,0,0,0,114,215,0,0,0,114,220,0, + 0,0,114,226,0,0,0,114,221,0,0,0,114,227,0,0, + 0,114,246,0,0,0,114,248,0,0,0,114,5,1,0,0, + 114,23,1,0,0,114,159,0,0,0,114,32,1,0,0,114, + 34,1,0,0,114,4,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,218,8,60,109,111,100,117,108, + 101,62,8,0,0,0,115,108,0,0,0,4,16,4,1,4, + 1,2,1,6,3,8,17,8,5,8,5,8,6,8,12,8, + 10,8,9,8,5,8,7,10,22,10,120,16,1,12,2,4, + 1,4,2,6,2,6,2,8,2,16,45,8,34,8,19,8, + 12,8,12,8,28,8,17,10,55,10,12,10,10,8,14,6, + 3,4,1,14,67,14,64,14,29,16,110,14,41,18,45,18, + 16,4,3,18,53,14,60,14,42,14,127,0,5,14,127,0, + 22,10,23,8,11,8,68, }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 16:55:01 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 20:55:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_all_usage_of_Py=5FL?= =?utf-8?q?OCAL?= Message-ID: <20160909205456.2574.32766.A89C4EC5@psf.io> https://hg.python.org/cpython/rev/5d8b3cda3559 changeset: 103464:5d8b3cda3559 user: Benjamin Peterson date: Fri Sep 09 13:54:34 2016 -0700 summary: remove all usage of Py_LOCAL files: Objects/bytes_methods.c | 4 +- Objects/bytesobject.c | 2 +- Objects/stringlib/transmogrify.h | 22 ++++++++++---------- Objects/unicodeobject.c | 2 +- 4 files changed, 15 insertions(+), 15 deletions(-) diff --git a/Objects/bytes_methods.c b/Objects/bytes_methods.c --- a/Objects/bytes_methods.c +++ b/Objects/bytes_methods.c @@ -670,7 +670,7 @@ * against substr, using the start and end arguments. Returns * -1 on error, 0 if not found and 1 if found. */ -Py_LOCAL(int) +static int tailmatch(const char *str, Py_ssize_t len, PyObject *substr, Py_ssize_t start, Py_ssize_t end, int direction) { @@ -716,7 +716,7 @@ return 0; } -Py_LOCAL(PyObject *) +static PyObject * _Py_bytes_tailmatch(const char *str, Py_ssize_t len, const char *function_name, PyObject *args, int direction) diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1500,7 +1500,7 @@ return PyLong_FromLong((unsigned char)a->ob_sval[i]); } -Py_LOCAL(int) +static int bytes_compare_eq(PyBytesObject *a, PyBytesObject *b) { int cmp; diff --git a/Objects/stringlib/transmogrify.h b/Objects/stringlib/transmogrify.h --- a/Objects/stringlib/transmogrify.h +++ b/Objects/stringlib/transmogrify.h @@ -5,7 +5,7 @@ /* the more complicated methods. parts of these should be pulled out into the shared code in bytes_methods.c to cut down on duplicate code bloat. */ -Py_LOCAL_INLINE(PyObject *) +static inline PyObject * return_self(PyObject *self) { #if !STRINGLIB_MUTABLE @@ -90,7 +90,7 @@ return NULL; } -Py_LOCAL_INLINE(PyObject *) +static inline PyObject * pad(PyObject *self, Py_ssize_t left, Py_ssize_t right, char fill) { PyObject *u; @@ -212,7 +212,7 @@ ((char *)memchr((const void *)(target), c, target_len)) -Py_LOCAL_INLINE(Py_ssize_t) +static Py_ssize_t countchar(const char *target, Py_ssize_t target_len, char c, Py_ssize_t maxcount) { @@ -233,7 +233,7 @@ /* Algorithms for different cases of string replacement */ /* len(self)>=1, from="", len(to)>=1, maxcount>=1 */ -Py_LOCAL(PyObject *) +static PyObject * stringlib_replace_interleave(PyObject *self, const char *to_s, Py_ssize_t to_len, Py_ssize_t maxcount) @@ -304,7 +304,7 @@ /* Special case for deleting a single character */ /* len(self)>=1, len(from)==1, to="", maxcount>=1 */ -Py_LOCAL(PyObject *) +static PyObject * stringlib_replace_delete_single_character(PyObject *self, char from_c, Py_ssize_t maxcount) { @@ -348,7 +348,7 @@ /* len(self)>=1, len(from)>=2, to="", maxcount>=1 */ -Py_LOCAL(PyObject *) +static PyObject * stringlib_replace_delete_substring(PyObject *self, const char *from_s, Py_ssize_t from_len, Py_ssize_t maxcount) @@ -400,7 +400,7 @@ } /* len(self)>=1, len(from)==len(to)==1, maxcount>=1 */ -Py_LOCAL(PyObject *) +static PyObject * stringlib_replace_single_character_in_place(PyObject *self, char from_c, char to_c, Py_ssize_t maxcount) @@ -447,7 +447,7 @@ } /* len(self)>=1, len(from)==len(to)>=2, maxcount>=1 */ -Py_LOCAL(PyObject *) +static PyObject * stringlib_replace_substring_in_place(PyObject *self, const char *from_s, Py_ssize_t from_len, const char *to_s, Py_ssize_t to_len, @@ -499,7 +499,7 @@ } /* len(self)>=1, len(from)==1, len(to)>=2, maxcount>=1 */ -Py_LOCAL(PyObject *) +static PyObject * stringlib_replace_single_character(PyObject *self, char from_c, const char *to_s, Py_ssize_t to_len, @@ -563,7 +563,7 @@ } /* len(self)>=1, len(from)>=2, len(to)>=2, maxcount>=1 */ -Py_LOCAL(PyObject *) +static PyObject * stringlib_replace_substring(PyObject *self, const char *from_s, Py_ssize_t from_len, const char *to_s, Py_ssize_t to_len, @@ -632,7 +632,7 @@ } -Py_LOCAL(PyObject *) +static PyObject * stringlib_replace(PyObject *self, const char *from_s, Py_ssize_t from_len, const char *to_s, Py_ssize_t to_len, diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -10936,7 +10936,7 @@ #undef COMPARE } -Py_LOCAL(int) +static int unicode_compare_eq(PyObject *str1, PyObject *str2) { int kind; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:21:33 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 21:21:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Prevent_PGO_build_for_x86_?= =?utf-8?q?releases=2E?= Message-ID: <20160909212132.8732.4313.91B80A7B@psf.io> https://hg.python.org/cpython/rev/5259831f23cb changeset: 103465:5259831f23cb user: Steve Dower date: Fri Sep 09 14:21:24 2016 -0700 summary: Prevent PGO build for x86 releases. files: Tools/msi/buildrelease.bat | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat --- a/Tools/msi/buildrelease.bat +++ b/Tools/msi/buildrelease.bat @@ -106,6 +106,7 @@ if "%1" EQU "x86" ( call "%PCBUILD%env.bat" x86 + set PGO= set BUILD=%PCBUILD%win32\ set BUILD_PLAT=Win32 set OUTDIR_PLAT=win32 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:22:53 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 21:22:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327874=3A_Allows_u?= =?utf-8?q?se_of_pythonXX=2Ezip_file_as_landmark_on_Windows?= Message-ID: <20160909212253.2824.83799.0E09210D@psf.io> https://hg.python.org/cpython/rev/672c5fe7372c changeset: 103466:672c5fe7372c user: Steve Dower date: Fri Sep 09 14:22:43 2016 -0700 summary: Issue #27874: Allows use of pythonXX.zip file as landmark on Windows files: PC/getpathp.c | 35 ++++++++++++++++++++--------------- 1 files changed, 20 insertions(+), 15 deletions(-) diff --git a/PC/getpathp.c b/PC/getpathp.c --- a/PC/getpathp.c +++ b/PC/getpathp.c @@ -24,7 +24,7 @@ * We attempt to locate the "Python Home" - if the PYTHONHOME env var is set, we believe it. Otherwise, we use the path of our host .EXE's - to try and locate our "landmark" (lib\\os.py) and deduce our home. + to try and locate on of our "landmarks" and deduce our home. - If we DO have a Python Home: The relevant sub-directories (Lib, plat-win, etc) are based on the Python Home - If we DO NOT have a Python Home, the core Python Path is @@ -207,7 +207,7 @@ 'landmark' can not overflow prefix if too long. */ static int -gotlandmark(wchar_t *landmark) +gotlandmark(const wchar_t *landmark) { int ok; Py_ssize_t n = wcsnlen_s(prefix, MAXPATHLEN); @@ -221,7 +221,7 @@ /* assumes argv0_path is MAXPATHLEN+1 bytes long, already \0 term'd. assumption provided by only caller, calculate_path() */ static int -search_for_prefix(wchar_t *argv0_path, wchar_t *landmark) +search_for_prefix(wchar_t *argv0_path, const wchar_t *landmark) { /* Search from argv0_path, until landmark is found */ wcscpy_s(prefix, MAXPATHLEN + 1, argv0_path); @@ -630,8 +630,24 @@ } } + /* Calculate zip archive path from DLL or exe path */ + if (wcscpy_s(zip_path, MAXPATHLEN + 1, dllpath[0] ? dllpath : progpath)) { + /* exceeded buffer length - ignore zip_path */ + zip_path[0] = '\0'; + } else { + wchar_t *dot = wcsrchr(zip_path, '.'); + if (!dot || wcscpy_s(dot, MAXPATHLEN + 1 - (dot - zip_path), L".zip")) { + /* exceeded buffer length - ignore zip_path */ + zip_path[0] = L'\0'; + } + } + if (pythonhome == NULL || *pythonhome == '\0') { - if (search_for_prefix(argv0_path, LANDMARK)) + if (zip_path[0] && exists(zip_path)) { + wcscpy_s(prefix, MAXPATHLEN+1, zip_path); + reduce(prefix); + pythonhome = prefix; + } else if (search_for_prefix(argv0_path, LANDMARK)) pythonhome = prefix; else pythonhome = NULL; @@ -643,17 +659,6 @@ envpath = NULL; - /* Calculate zip archive path from DLL or exe path */ - if (wcscpy_s(zip_path, MAXPATHLEN+1, dllpath[0] ? dllpath : progpath)) - /* exceeded buffer length - ignore zip_path */ - zip_path[0] = '\0'; - else { - wchar_t *dot = wcsrchr(zip_path, '.'); - if (!dot || wcscpy_s(dot, MAXPATHLEN+1 - (dot - zip_path), L".zip")) - /* exceeded buffer length - ignore zip_path */ - zip_path[0] = L'\0'; - } - skiphome = pythonhome==NULL ? 0 : 1; #ifdef Py_ENABLE_SHARED machinepath = getpythonregpath(HKEY_LOCAL_MACHINE, skiphome); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:23:12 2016 From: python-checkins at python.org (victor.stinner) Date: Fri, 09 Sep 2016 21:23:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327810=3A_Add_=5FP?= =?utf-8?q?yCFunction=5FFastCallKeywords=28=29?= Message-ID: <20160909212311.69577.93453.5A3D0C0E@psf.io> https://hg.python.org/cpython/rev/a25c39873d93 changeset: 103467:a25c39873d93 user: Victor Stinner date: Fri Sep 09 14:07:44 2016 -0700 summary: Issue #27810: Add _PyCFunction_FastCallKeywords() Use _PyCFunction_FastCallKeywords() in ceval.c: it allows to remove a lot of code from ceval.c which was only used to call C functions. files: Include/abstract.h | 9 +- Include/methodobject.h | 5 + Objects/abstract.c | 7 +- Objects/methodobject.c | 26 ++++ Python/ceval.c | 170 +++++----------------------- 5 files changed, 75 insertions(+), 142 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -267,9 +267,16 @@ PyObject *args, PyObject *kwargs); #ifndef Py_LIMITED_API - PyAPI_FUNC(PyObject*) _PyStack_AsTuple(PyObject **stack, + PyAPI_FUNC(PyObject*) _PyStack_AsTuple( + PyObject **stack, Py_ssize_t nargs); + PyAPI_FUNC(PyObject *) _PyStack_AsDict( + PyObject **values, + Py_ssize_t nkwargs, + PyObject *kwnames, + PyObject *func); + /* Call the callable object func with the "fast call" calling convention: args is a C array for positional arguments (nargs is the number of positional arguments), kwargs is a dictionary for keyword arguments. diff --git a/Include/methodobject.h b/Include/methodobject.h --- a/Include/methodobject.h +++ b/Include/methodobject.h @@ -42,6 +42,11 @@ PyObject **args, Py_ssize_t nargs, PyObject *kwargs); + +PyAPI_FUNC(PyObject *) _PyCFunction_FastCallKeywords(PyObject *func, + PyObject **stack, + Py_ssize_t nargs, + PyObject *kwnames); #endif struct PyMethodDef { diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2366,7 +2366,7 @@ return result; } -static PyObject * +PyObject * _PyStack_AsDict(PyObject **values, Py_ssize_t nkwargs, PyObject *kwnames, PyObject *func) { @@ -2415,10 +2415,13 @@ assert((nargs == 0 && nkwargs == 0) || stack != NULL); if (PyFunction_Check(func)) { - /* Fast-path: avoid temporary tuple or dict */ return _PyFunction_FastCallKeywords(func, stack, nargs, kwnames); } + if (PyCFunction_Check(func)) { + return _PyCFunction_FastCallKeywords(func, stack, nargs, kwnames); + } + if (nkwargs > 0) { kwdict = _PyStack_AsDict(stack + nargs, nkwargs, kwnames, func); if (kwdict == NULL) { diff --git a/Objects/methodobject.c b/Objects/methodobject.c --- a/Objects/methodobject.c +++ b/Objects/methodobject.c @@ -155,6 +155,7 @@ PyObject *result; int flags; + assert(PyCFunction_Check(func)); assert(func != NULL); assert(nargs >= 0); assert(nargs == 0 || args != NULL); @@ -243,6 +244,31 @@ return result; } +PyObject * +_PyCFunction_FastCallKeywords(PyObject *func, PyObject **stack, + Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *kwdict, *result; + Py_ssize_t nkwargs; + + assert(PyCFunction_Check(func)); + + nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames); + if (nkwargs > 0) { + kwdict = _PyStack_AsDict(stack + nargs, nkwargs, kwnames, func); + if (kwdict == NULL) { + return NULL; + } + } + else { + kwdict = NULL; + } + + result = _PyCFunction_FastCallDict(func, stack, nargs, kwdict); + Py_XDECREF(kwdict); + return result; +} + /* Methods (the standard built-in methods, that is) */ static void diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -115,8 +115,6 @@ #endif static PyObject * fast_function(PyObject *, PyObject **, Py_ssize_t, PyObject *); static PyObject * do_call_core(PyObject *, PyObject *, PyObject *); -static PyObject * create_keyword_args(PyObject *, PyObject ***, PyObject *); -static PyObject * load_args(PyObject ***, Py_ssize_t); #ifdef LLTRACE static int lltrace; @@ -4892,21 +4890,6 @@ return " object"; } -static void -err_args(PyObject *func, int flags, Py_ssize_t nargs) -{ - if (flags & METH_NOARGS) - PyErr_Format(PyExc_TypeError, - "%.200s() takes no arguments (%zd given)", - ((PyCFunctionObject *)func)->m_ml->ml_name, - nargs); - else - PyErr_Format(PyExc_TypeError, - "%.200s() takes exactly one argument (%zd given)", - ((PyCFunctionObject *)func)->m_ml->ml_name, - nargs); -} - #define C_TRACE(x, call) \ if (tstate->use_tracing && tstate->c_profilefunc) { \ if (call_trace(tstate->c_profilefunc, tstate->c_profileobj, \ @@ -4950,91 +4933,49 @@ PyObject *x, *w; Py_ssize_t nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames); Py_ssize_t nargs = oparg - nkwargs; + PyObject **stack; /* Always dispatch PyCFunction first, because these are presumed to be the most frequent callable object. */ if (PyCFunction_Check(func)) { - int flags = PyCFunction_GET_FLAGS(func); PyThreadState *tstate = PyThreadState_GET(); PCALL(PCALL_CFUNCTION); - if (kwnames == NULL && flags & (METH_NOARGS | METH_O)) { - PyCFunction meth = PyCFunction_GET_FUNCTION(func); - PyObject *self = PyCFunction_GET_SELF(func); - if (flags & METH_NOARGS && nargs == 0) { - C_TRACE(x, (*meth)(self,NULL)); - - x = _Py_CheckFunctionResult(func, x, NULL); - } - else if (flags & METH_O && nargs == 1) { - PyObject *arg = EXT_POP(*pp_stack); - C_TRACE(x, (*meth)(self,arg)); - Py_DECREF(arg); - - x = _Py_CheckFunctionResult(func, x, NULL); - } - else { - err_args(func, flags, nargs); - x = NULL; - } - } - else { - PyObject *callargs, *kwdict = NULL; - if (kwnames != NULL) { - kwdict = create_keyword_args(kwnames, pp_stack, func); - if (kwdict == NULL) { - x = NULL; - goto cfuncerror; - } - } - callargs = load_args(pp_stack, nargs); - if (callargs != NULL) { - READ_TIMESTAMP(*pintr0); - C_TRACE(x, PyCFunction_Call(func, callargs, kwdict)); - READ_TIMESTAMP(*pintr1); - Py_DECREF(callargs); - } - else { - x = NULL; - } - Py_XDECREF(kwdict); - } + + stack = (*pp_stack) - nargs - nkwargs; + C_TRACE(x, _PyCFunction_FastCallKeywords(func, stack, nargs, kwnames)); } else { - Py_ssize_t nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames); - PyObject **stack; - - if (PyMethod_Check(func) && PyMethod_GET_SELF(func) != NULL) { - /* optimize access to bound methods */ - PyObject *self = PyMethod_GET_SELF(func); - PCALL(PCALL_METHOD); - PCALL(PCALL_BOUND_METHOD); - Py_INCREF(self); - func = PyMethod_GET_FUNCTION(func); - Py_INCREF(func); - Py_SETREF(*pfunc, self); - nargs++; - } - else { - Py_INCREF(func); - } - - stack = (*pp_stack) - nargs - nkwargs; - - READ_TIMESTAMP(*pintr0); - if (PyFunction_Check(func)) { - x = fast_function(func, stack, nargs, kwnames); - } - else { - x = _PyObject_FastCallKeywords(func, stack, nargs, kwnames); - } - READ_TIMESTAMP(*pintr1); - - Py_DECREF(func); + if (PyMethod_Check(func) && PyMethod_GET_SELF(func) != NULL) { + /* optimize access to bound methods */ + PyObject *self = PyMethod_GET_SELF(func); + PCALL(PCALL_METHOD); + PCALL(PCALL_BOUND_METHOD); + Py_INCREF(self); + func = PyMethod_GET_FUNCTION(func); + Py_INCREF(func); + Py_SETREF(*pfunc, self); + nargs++; + } + else { + Py_INCREF(func); + } + + stack = (*pp_stack) - nargs - nkwargs; + + READ_TIMESTAMP(*pintr0); + if (PyFunction_Check(func)) { + x = fast_function(func, stack, nargs, kwnames); + } + else { + x = _PyObject_FastCallKeywords(func, stack, nargs, kwnames); + } + READ_TIMESTAMP(*pintr1); + + Py_DECREF(func); } -cfuncerror: assert((x != NULL) ^ (PyErr_Occurred() != NULL)); /* Clear the stack of the function object. Also removes @@ -5243,55 +5184,6 @@ } static PyObject * -create_keyword_args(PyObject *names, PyObject ***pp_stack, - PyObject *func) -{ - Py_ssize_t nk = PyTuple_GET_SIZE(names); - PyObject *kwdict = _PyDict_NewPresized(nk); - if (kwdict == NULL) - return NULL; - while (--nk >= 0) { - int err; - PyObject *key = PyTuple_GET_ITEM(names, nk); - PyObject *value = EXT_POP(*pp_stack); - if (PyDict_GetItem(kwdict, key) != NULL) { - PyErr_Format(PyExc_TypeError, - "%.200s%s got multiple values " - "for keyword argument '%U'", - PyEval_GetFuncName(func), - PyEval_GetFuncDesc(func), - key); - Py_DECREF(value); - Py_DECREF(kwdict); - return NULL; - } - err = PyDict_SetItem(kwdict, key, value); - Py_DECREF(value); - if (err) { - Py_DECREF(kwdict); - return NULL; - } - } - return kwdict; -} - -static PyObject * -load_args(PyObject ***pp_stack, Py_ssize_t nargs) -{ - PyObject *args = PyTuple_New(nargs); - - if (args == NULL) { - return NULL; - } - - while (--nargs >= 0) { - PyObject *arg= EXT_POP(*pp_stack); - PyTuple_SET_ITEM(args, nargs, arg); - } - return args; -} - -static PyObject * do_call_core(PyObject *func, PyObject *callargs, PyObject *kwdict) { #ifdef CALL_PROFILE -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:27:14 2016 From: python-checkins at python.org (guido.van.rossum) Date: Fri, 09 Sep 2016 21:27:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Merge_asyncio_?= =?utf-8?q?upstream=2E?= Message-ID: <20160909212714.21136.66249.126EB6C6@psf.io> https://hg.python.org/cpython/rev/ab3d9bdb69d1 changeset: 103468:ab3d9bdb69d1 branch: 3.5 parent: 103458:8927417c5e88 user: Guido van Rossum date: Fri Sep 09 14:26:31 2016 -0700 summary: Merge asyncio upstream. files: Lib/asyncio/base_events.py | 2 +- Lib/asyncio/coroutines.py | 4 +- Lib/asyncio/futures.py | 24 +++- Lib/asyncio/tasks.py | 8 +- Lib/test/test_asyncio/test_events.py | 2 +- Lib/test/test_asyncio/test_futures.py | 68 +++++++++++++++ 6 files changed, 94 insertions(+), 14 deletions(-) diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -363,7 +363,7 @@ """ self._check_closed() - new_task = not isinstance(future, futures.Future) + new_task = not futures.isfuture(future) future = tasks.ensure_future(future, loop=self) if new_task: # An exception is raised if the future didn't complete, so there diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py --- a/Lib/asyncio/coroutines.py +++ b/Lib/asyncio/coroutines.py @@ -204,8 +204,8 @@ @functools.wraps(func) def coro(*args, **kw): res = func(*args, **kw) - if isinstance(res, futures.Future) or inspect.isgenerator(res) or \ - isinstance(res, CoroWrapper): + if (futures.isfuture(res) or inspect.isgenerator(res) or + isinstance(res, CoroWrapper)): res = yield from res elif _AwaitableABC is not None: # If 'func' returns an Awaitable (new in 3.5) we diff --git a/Lib/asyncio/futures.py b/Lib/asyncio/futures.py --- a/Lib/asyncio/futures.py +++ b/Lib/asyncio/futures.py @@ -110,6 +110,16 @@ self.loop.call_exception_handler({'message': msg}) +def isfuture(obj): + """Check for a Future. + + This returns True when obj is a Future instance or is advertising + itself as duck-type compatible by setting _asyncio_future_blocking. + See comment in Future for more details. + """ + return getattr(obj, '_asyncio_future_blocking', None) is not None + + class Future: """This class is *almost* compatible with concurrent.futures.Future. @@ -423,15 +433,17 @@ If destination is cancelled, source gets cancelled too. Compatible with both asyncio.Future and concurrent.futures.Future. """ - if not isinstance(source, (Future, concurrent.futures.Future)): + if not isfuture(source) and not isinstance(source, + concurrent.futures.Future): raise TypeError('A future is required for source argument') - if not isinstance(destination, (Future, concurrent.futures.Future)): + if not isfuture(destination) and not isinstance(destination, + concurrent.futures.Future): raise TypeError('A future is required for destination argument') - source_loop = source._loop if isinstance(source, Future) else None - dest_loop = destination._loop if isinstance(destination, Future) else None + source_loop = source._loop if isfuture(source) else None + dest_loop = destination._loop if isfuture(destination) else None def _set_state(future, other): - if isinstance(future, Future): + if isfuture(future): _copy_future_state(other, future) else: _set_concurrent_future_state(future, other) @@ -455,7 +467,7 @@ def wrap_future(future, *, loop=None): """Wrap concurrent.futures.Future object.""" - if isinstance(future, Future): + if isfuture(future): return future assert isinstance(future, concurrent.futures.Future), \ 'concurrent.futures.Future is expected, got {!r}'.format(future) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -333,7 +333,7 @@ Note: This does not raise TimeoutError! Futures that aren't done when the timeout occurs are returned in the second set. """ - if isinstance(fs, futures.Future) or coroutines.iscoroutine(fs): + if futures.isfuture(fs) or coroutines.iscoroutine(fs): raise TypeError("expect a list of futures, not %s" % type(fs).__name__) if not fs: raise ValueError('Set of coroutines/Futures is empty.') @@ -462,7 +462,7 @@ Note: The futures 'f' are not necessarily members of fs. """ - if isinstance(fs, futures.Future) or coroutines.iscoroutine(fs): + if futures.isfuture(fs) or coroutines.iscoroutine(fs): raise TypeError("expect a list of futures, not %s" % type(fs).__name__) loop = loop if loop is not None else events.get_event_loop() todo = {ensure_future(f, loop=loop) for f in set(fs)} @@ -538,7 +538,7 @@ If the argument is a Future, it is returned directly. """ - if isinstance(coro_or_future, futures.Future): + if futures.isfuture(coro_or_future): if loop is not None and loop is not coro_or_future._loop: raise ValueError('loop argument must agree with Future') return coro_or_future @@ -614,7 +614,7 @@ arg_to_fut = {} for arg in set(coros_or_futures): - if not isinstance(arg, futures.Future): + if not futures.isfuture(arg): fut = ensure_future(arg, loop=loop) if loop is None: loop = fut._loop diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -793,7 +793,7 @@ loop.connect_accepted_socket( (lambda : proto), conn, ssl=server_ssl)) loop.run_forever() - conn.close() + proto.transport.close() lsock.close() thread.join(1) diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py --- a/Lib/test/test_asyncio/test_futures.py +++ b/Lib/test/test_asyncio/test_futures.py @@ -25,6 +25,74 @@ pass +class DuckFuture: + # Class that does not inherit from Future but aims to be duck-type + # compatible with it. + + _asyncio_future_blocking = False + __cancelled = False + __result = None + __exception = None + + def cancel(self): + if self.done(): + return False + self.__cancelled = True + return True + + def cancelled(self): + return self.__cancelled + + def done(self): + return (self.__cancelled + or self.__result is not None + or self.__exception is not None) + + def result(self): + assert not self.cancelled() + if self.__exception is not None: + raise self.__exception + return self.__result + + def exception(self): + assert not self.cancelled() + return self.__exception + + def set_result(self, result): + assert not self.done() + assert result is not None + self.__result = result + + def set_exception(self, exception): + assert not self.done() + assert exception is not None + self.__exception = exception + + def __iter__(self): + if not self.done(): + self._asyncio_future_blocking = True + yield self + assert self.done() + return self.result() + + +class DuckTests(test_utils.TestCase): + + def setUp(self): + self.loop = self.new_test_loop() + self.addCleanup(self.loop.close) + + def test_wrap_future(self): + f = DuckFuture() + g = asyncio.wrap_future(f) + assert g is f + + def test_ensure_future(self): + f = DuckFuture() + g = asyncio.ensure_future(f) + assert g is f + + class FutureTests(test_utils.TestCase): def setUp(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:48:16 2016 From: python-checkins at python.org (gregory.p.smith) Date: Fri, 09 Sep 2016 21:48:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue27985_-_fix_the_incor?= =?utf-8?q?rect_duplicate_class_name_in_the_lib2to3?= Message-ID: <20160909214813.1702.33395.B5AE68DD@psf.io> https://hg.python.org/cpython/rev/ef3d30cc6b4f changeset: 103469:ef3d30cc6b4f parent: 103467:a25c39873d93 user: Gregory P. Smith date: Fri Sep 09 14:48:08 2016 -0700 summary: issue27985 - fix the incorrect duplicate class name in the lib2to3 test. call it TestVarAnnotations instead. files: Lib/lib2to3/tests/test_parser.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -256,7 +256,7 @@ # Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.test_var_annot -class TestFunctionAnnotations(GrammarTest): +class TestVarAnnotations(GrammarTest): def test_1(self): self.validate("var1: int = 5") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:55:19 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 09 Sep 2016 21:55:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325856=3A_The_=5F?= =?utf-8?q?=5Fmodule=5F=5F_attribute_of_extension_classes_and_functions?= Message-ID: <20160909215519.21363.28684.3A6A54CB@psf.io> https://hg.python.org/cpython/rev/861ddad3e0c1 changeset: 103470:861ddad3e0c1 parent: 103467:a25c39873d93 user: Serhiy Storchaka date: Sat Sep 10 00:53:02 2016 +0300 summary: Issue #25856: The __module__ attribute of extension classes and functions now is interned. This leads to more compact pickle data with protocol 4. files: Misc/NEWS | 3 +++ Objects/typeobject.c | 29 ++++++++++++++++------------- 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #25856: The __module__ attribute of extension classes and functions + now is interned. This leads to more compact pickle data with protocol 4. + - Issue #27213: Rework CALL_FUNCTION* opcodes to produce shorter and more efficient bytecode. Patch by Demur Rumed, design by Serhiy Storchaka, reviewed by Serhiy Storchaka and Victor Stinner. diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -454,27 +454,30 @@ static PyObject * type_module(PyTypeObject *type, void *context) { - char *s; + PyObject *mod; if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) { - PyObject *mod = _PyDict_GetItemId(type->tp_dict, &PyId___module__); - if (!mod) { + mod = _PyDict_GetItemId(type->tp_dict, &PyId___module__); + if (mod == NULL) { PyErr_Format(PyExc_AttributeError, "__module__"); - return 0; + return NULL; } Py_INCREF(mod); - return mod; } else { - PyObject *name; - s = strrchr(type->tp_name, '.'); - if (s != NULL) - return PyUnicode_FromStringAndSize( + const char *s = strrchr(type->tp_name, '.'); + if (s != NULL) { + mod = PyUnicode_FromStringAndSize( type->tp_name, (Py_ssize_t)(s - type->tp_name)); - name = _PyUnicode_FromId(&PyId_builtins); - Py_XINCREF(name); - return name; - } + if (mod != NULL) + PyUnicode_InternInPlace(&mod); + } + else { + mod = _PyUnicode_FromId(&PyId_builtins); + Py_XINCREF(mod); + } + } + return mod; } static int -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:55:20 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 09 Sep 2016 21:55:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_heads?= Message-ID: <20160909215519.69700.72808.60897F53@psf.io> https://hg.python.org/cpython/rev/66afc449efa9 changeset: 103471:66afc449efa9 parent: 103470:861ddad3e0c1 parent: 103469:ef3d30cc6b4f user: Serhiy Storchaka date: Sat Sep 10 00:55:01 2016 +0300 summary: Merge heads files: Lib/lib2to3/tests/test_parser.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -256,7 +256,7 @@ # Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.test_var_annot -class TestFunctionAnnotations(GrammarTest): +class TestVarAnnotations(GrammarTest): def test_1(self): self.validate("var1: int = 5") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:57:16 2016 From: python-checkins at python.org (brett.cannon) Date: Fri, 09 Sep 2016 21:57:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326331=3A_Implemen?= =?utf-8?q?t_the_parsing_part_of_PEP_515=2E?= Message-ID: <20160909215716.8568.9143.5E67574F@psf.io> https://hg.python.org/cpython/rev/8a881dafe335 changeset: 103472:8a881dafe335 user: Brett Cannon date: Fri Sep 09 14:57:09 2016 -0700 summary: Issue #26331: Implement the parsing part of PEP 515. Thanks to Georg Brandl for the patch. files: Doc/library/decimal.rst | 10 +- Doc/library/functions.rst | 16 +- Doc/reference/lexical_analysis.rst | 45 ++- Doc/whatsnew/3.6.rst | 23 + Include/pystrtod.h | 4 + Lib/_pydecimal.py | 10 +- Lib/test/test_complex.py | 14 + Lib/test/test_decimal.py | 10 + Lib/test/test_float.py | 24 +- Lib/test/test_grammar.py | 89 ++++++ Lib/test/test_int.py | 21 + Lib/test/test_tokenize.py | 30 +- Lib/test/test_types.py | 1 + Lib/tokenize.py | 17 +- Misc/NEWS | 6 +- Modules/_decimal/_decimal.c | 12 +- Objects/complexobject.c | 63 ++- Objects/floatobject.c | 59 ++- Objects/longobject.c | 169 +++++++++-- Parser/tokenizer.c | 232 +++++++++++----- Python/ast.c | 27 +- Python/pystrtod.c | 66 ++++ 22 files changed, 743 insertions(+), 205 deletions(-) diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -345,7 +345,7 @@ *value* can be an integer, string, tuple, :class:`float`, or another :class:`Decimal` object. If no *value* is given, returns ``Decimal('0')``. If *value* is a string, it should conform to the decimal numeric string syntax after leading - and trailing whitespace characters are removed:: + and trailing whitespace characters, as well as underscores throughout, are removed:: sign ::= '+' | '-' digit ::= '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9' @@ -394,6 +394,10 @@ :class:`float` arguments raise an exception if the :exc:`FloatOperation` trap is set. By default the trap is off. + .. versionchanged:: 3.6 + Underscores are allowed for grouping, as with integral and floating-point + literals in code. + Decimal floating point objects share many properties with the other built-in numeric types such as :class:`float` and :class:`int`. All of the usual math operations and special methods apply. Likewise, decimal objects can be @@ -1075,8 +1079,8 @@ Decimal('4.44') This method implements the to-number operation of the IBM specification. - If the argument is a string, no leading or trailing whitespace is - permitted. + If the argument is a string, no leading or trailing whitespace or + underscores are permitted. .. method:: create_decimal_from_float(f) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -271,6 +271,9 @@ The complex type is described in :ref:`typesnumeric`. + .. versionchanged:: 3.6 + Grouping digits with underscores as in code literals is allowed. + .. function:: delattr(object, name) @@ -531,11 +534,14 @@ The float type is described in :ref:`typesnumeric`. - .. index:: - single: __format__ - single: string; format() (built-in function) + .. versionchanged:: 3.6 + Grouping digits with underscores as in code literals is allowed. +.. index:: + single: __format__ + single: string; format() (built-in function) + .. function:: format(value[, format_spec]) Convert a *value* to a "formatted" representation, as controlled by @@ -702,6 +708,10 @@ :meth:`base.__int__ ` instead of :meth:`base.__index__ `. + .. versionchanged:: 3.6 + Grouping digits with underscores as in code literals is allowed. + + .. function:: isinstance(object, classinfo) Return true if the *object* argument is an instance of the *classinfo* diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -721,20 +721,24 @@ Integer literals are described by the following lexical definitions: .. productionlist:: - integer: `decimalinteger` | `octinteger` | `hexinteger` | `bininteger` - decimalinteger: `nonzerodigit` `digit`* | "0"+ + integer: `decinteger` | `bininteger` | `octinteger` | `hexinteger` + decinteger: `nonzerodigit` (["_"] `digit`)* | "0"+ (["_"] "0")* + bininteger: "0" ("b" | "B") (["_"] `bindigit`)+ + octinteger: "0" ("o" | "O") (["_"] `octdigit`)+ + hexinteger: "0" ("x" | "X") (["_"] `hexdigit`)+ nonzerodigit: "1"..."9" digit: "0"..."9" - octinteger: "0" ("o" | "O") `octdigit`+ - hexinteger: "0" ("x" | "X") `hexdigit`+ - bininteger: "0" ("b" | "B") `bindigit`+ + bindigit: "0" | "1" octdigit: "0"..."7" hexdigit: `digit` | "a"..."f" | "A"..."F" - bindigit: "0" | "1" There is no limit for the length of integer literals apart from what can be stored in available memory. +Underscores are ignored for determining the numeric value of the literal. They +can be used to group digits for enhanced readability. One underscore can occur +between digits, and after base specifiers like ``0x``. + Note that leading zeros in a non-zero decimal number are not allowed. This is for disambiguation with C-style octal literals, which Python used before version 3.0. @@ -743,6 +747,10 @@ 7 2147483647 0o177 0b100110111 3 79228162514264337593543950336 0o377 0xdeadbeef + 100_000_000_000 0b_1110_0101 + +.. versionchanged:: 3.6 + Underscores are now allowed for grouping purposes in literals. .. _floating: @@ -754,23 +762,28 @@ .. productionlist:: floatnumber: `pointfloat` | `exponentfloat` - pointfloat: [`intpart`] `fraction` | `intpart` "." - exponentfloat: (`intpart` | `pointfloat`) `exponent` - intpart: `digit`+ - fraction: "." `digit`+ - exponent: ("e" | "E") ["+" | "-"] `digit`+ + pointfloat: [`digitpart`] `fraction` | `digitpart` "." + exponentfloat: (`digitpart` | `pointfloat`) `exponent` + digitpart: `digit` (["_"] `digit`)* + fraction: "." `digitpart` + exponent: ("e" | "E") ["+" | "-"] `digitpart` Note that the integer and exponent parts are always interpreted using radix 10. For example, ``077e010`` is legal, and denotes the same number as ``77e10``. The -allowed range of floating point literals is implementation-dependent. Some -examples of floating point literals:: +allowed range of floating point literals is implementation-dependent. As in +integer literals, underscores are supported for digit grouping. - 3.14 10. .001 1e100 3.14e-10 0e0 +Some examples of floating point literals:: + + 3.14 10. .001 1e100 3.14e-10 0e0 3.14_15_93 Note that numeric literals do not include a sign; a phrase like ``-1`` is actually an expression composed of the unary operator ``-`` and the literal ``1``. +.. versionchanged:: 3.6 + Underscores are now allowed for grouping purposes in literals. + .. _imaginary: @@ -780,7 +793,7 @@ Imaginary literals are described by the following lexical definitions: .. productionlist:: - imagnumber: (`floatnumber` | `intpart`) ("j" | "J") + imagnumber: (`floatnumber` | `digitpart`) ("j" | "J") An imaginary literal yields a complex number with a real part of 0.0. Complex numbers are represented as a pair of floating point numbers and have the same @@ -788,7 +801,7 @@ part, add a floating point number to it, e.g., ``(3+4j)``. Some examples of imaginary literals:: - 3.14j 10.j 10j .001j 1e100j 3.14e-10j + 3.14j 10.j 10j .001j 1e100j 3.14e-10j 3.14_15_93j .. _operators: diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -124,6 +124,29 @@ New Features ============ +.. _pep-515: + +PEP 515: Underscores in Numeric Literals +======================================== + +Prior to PEP 515, there was no support for writing long numeric +literals with some form of separator to improve readability. For +instance, how big is ``1000000000000000```? With :pep:`515`, though, +you can use underscores to separate digits as desired to make numeric +literals easier to read: ``1_000_000_000_000_000``. Underscores can be +used with other numeric literals beyond integers, e.g. +``0x_FF_FF_FF_FF``. + +Single underscores are allowed between digits and after any base +specifier. More than a single underscore in a row, leading, or +trailing underscores are not allowed. + +.. seealso:: + + :pep:`523` - Underscores in Numeric Literals + PEP written by Georg Brandl & Serhiy Storchaka. + + .. _pep-523: PEP 523: Adding a frame evaluation API to CPython diff --git a/Include/pystrtod.h b/Include/pystrtod.h --- a/Include/pystrtod.h +++ b/Include/pystrtod.h @@ -19,6 +19,10 @@ int *type); #ifndef Py_LIMITED_API +PyAPI_FUNC(PyObject *) _Py_string_to_number_with_underscores( + const char *str, Py_ssize_t len, const char *what, PyObject *obj, void *arg, + PyObject *(*innerfunc)(const char *, Py_ssize_t, void *)); + PyAPI_FUNC(double) _Py_parse_inf_or_nan(const char *p, char **endptr); #endif diff --git a/Lib/_pydecimal.py b/Lib/_pydecimal.py --- a/Lib/_pydecimal.py +++ b/Lib/_pydecimal.py @@ -589,7 +589,7 @@ # From a string # REs insist on real strings, so we can too. if isinstance(value, str): - m = _parser(value.strip()) + m = _parser(value.strip().replace("_", "")) if m is None: if context is None: context = getcontext() @@ -4125,7 +4125,7 @@ This will make it round up for that operation. """ rounding = self.rounding - self.rounding= type + self.rounding = type return rounding def create_decimal(self, num='0'): @@ -4134,10 +4134,10 @@ This method implements the to-number operation of the IBM Decimal specification.""" - if isinstance(num, str) and num != num.strip(): + if isinstance(num, str) and (num != num.strip() or '_' in num): return self._raise_error(ConversionSyntax, - "no trailing or leading whitespace is " - "permitted.") + "trailing or leading whitespace and " + "underscores are not permitted.") d = Decimal(num, context=self) if d._isnan() and len(d._int) > self.prec - self.clamp: diff --git a/Lib/test/test_complex.py b/Lib/test/test_complex.py --- a/Lib/test/test_complex.py +++ b/Lib/test/test_complex.py @@ -1,5 +1,7 @@ import unittest from test import support +from test.test_grammar import (VALID_UNDERSCORE_LITERALS, + INVALID_UNDERSCORE_LITERALS) from random import random from math import atan2, isnan, copysign @@ -377,6 +379,18 @@ self.assertAlmostEqual(complex(complex1(1j)), 2j) self.assertRaises(TypeError, complex, complex2(1j)) + def test_underscores(self): + # check underscores + for lit in VALID_UNDERSCORE_LITERALS: + if not any(ch in lit for ch in 'xXoObB'): + self.assertEqual(complex(lit), eval(lit)) + self.assertEqual(complex(lit), complex(lit.replace('_', ''))) + for lit in INVALID_UNDERSCORE_LITERALS: + if lit in ('0_7', '09_99'): # octals are not recognized here + continue + if not any(ch in lit for ch in 'xXoObB'): + self.assertRaises(ValueError, complex, lit) + def test_hash(self): for x in range(-30, 30): self.assertEqual(hash(x), hash(complex(x, 0))) diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -554,6 +554,10 @@ self.assertEqual(str(Decimal(' -7.89')), '-7.89') self.assertEqual(str(Decimal(" 3.45679 ")), '3.45679') + # underscores + self.assertEqual(str(Decimal('1_3.3e4_0')), '1.33E+41') + self.assertEqual(str(Decimal('1_0_0_0')), '1000') + # unicode whitespace for lead in ["", ' ', '\u00a0', '\u205f']: for trail in ["", ' ', '\u00a0', '\u205f']: @@ -578,6 +582,9 @@ # embedded NUL self.assertRaises(InvalidOperation, Decimal, "12\u00003") + # underscores don't prevent errors + self.assertRaises(InvalidOperation, Decimal, "1_2_\u00003") + @cpython_only def test_from_legacy_strings(self): import _testcapi @@ -772,6 +779,9 @@ self.assertRaises(InvalidOperation, nc.create_decimal, "xyz") self.assertRaises(ValueError, nc.create_decimal, (1, "xyz", -25)) self.assertRaises(TypeError, nc.create_decimal, "1234", "5678") + # no whitespace and underscore stripping is done with this method + self.assertRaises(InvalidOperation, nc.create_decimal, " 1234") + self.assertRaises(InvalidOperation, nc.create_decimal, "12_34") # too many NaN payload digits nc.prec = 3 diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py --- a/Lib/test/test_float.py +++ b/Lib/test/test_float.py @@ -1,4 +1,3 @@ - import fractions import operator import os @@ -9,6 +8,8 @@ import unittest from test import support +from test.test_grammar import (VALID_UNDERSCORE_LITERALS, + INVALID_UNDERSCORE_LITERALS) from math import isinf, isnan, copysign, ldexp INF = float("inf") @@ -60,6 +61,27 @@ float(b'.' + b'1'*1000) float('.' + '1'*1000) + def test_underscores(self): + for lit in VALID_UNDERSCORE_LITERALS: + if not any(ch in lit for ch in 'jJxXoObB'): + self.assertEqual(float(lit), eval(lit)) + self.assertEqual(float(lit), float(lit.replace('_', ''))) + for lit in INVALID_UNDERSCORE_LITERALS: + if lit in ('0_7', '09_99'): # octals are not recognized here + continue + if not any(ch in lit for ch in 'jJxXoObB'): + self.assertRaises(ValueError, float, lit) + # Additional test cases; nan and inf are never valid as literals, + # only in the float() constructor, but we don't allow underscores + # in or around them. + self.assertRaises(ValueError, float, '_NaN') + self.assertRaises(ValueError, float, 'Na_N') + self.assertRaises(ValueError, float, 'IN_F') + self.assertRaises(ValueError, float, '-_INF') + self.assertRaises(ValueError, float, '-INF_') + # Check that we handle bytes values correctly. + self.assertRaises(ValueError, float, b'0_.\xff9') + def test_non_numeric_input_types(self): # Test possible non-numeric types for the argument x, including # subclasses of the explicitly documented accepted types. diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -16,6 +16,87 @@ from test import ann_module2 import test +# These are shared with test_tokenize and other test modules. +# +# Note: since several test cases filter out floats by looking for "e" and ".", +# don't add hexadecimal literals that contain "e" or "E". +VALID_UNDERSCORE_LITERALS = [ + '0_0_0', + '4_2', + '1_0000_0000', + '0b1001_0100', + '0xffff_ffff', + '0o5_7_7', + '1_00_00.5', + '1_00_00.5e5', + '1_00_00e5_1', + '1e1_0', + '.1_4', + '.1_4e1', + '0b_0', + '0x_f', + '0o_5', + '1_00_00j', + '1_00_00.5j', + '1_00_00e5_1j', + '.1_4j', + '(1_2.5+3_3j)', + '(.5_6j)', +] +INVALID_UNDERSCORE_LITERALS = [ + # Trailing underscores: + '0_', + '42_', + '1.4j_', + '0x_', + '0b1_', + '0xf_', + '0o5_', + '0 if 1_Else 1', + # Underscores in the base selector: + '0_b0', + '0_xf', + '0_o5', + # Old-style octal, still disallowed: + '0_7', + '09_99', + # Multiple consecutive underscores: + '4_______2', + '0.1__4', + '0.1__4j', + '0b1001__0100', + '0xffff__ffff', + '0x___', + '0o5__77', + '1e1__0', + '1e1__0j', + # Underscore right before a dot: + '1_.4', + '1_.4j', + # Underscore right after a dot: + '1._4', + '1._4j', + '._5', + '._5j', + # Underscore right after a sign: + '1.0e+_1', + '1.0e+_1j', + # Underscore right before j: + '1.4_j', + '1.4e5_j', + # Underscore right before e: + '1_e1', + '1.4_e1', + '1.4_e1j', + # Underscore right after e: + '1e_1', + '1.4e_1', + '1.4e_1j', + # Complex cases with parens: + '(1+1.5_j_)', + '(1+1.5_j)', +] + class TokenTests(unittest.TestCase): @@ -95,6 +176,14 @@ self.assertEqual(1 if 0else 0, 0) self.assertRaises(SyntaxError, eval, "0 if 1Else 0") + def test_underscore_literals(self): + for lit in VALID_UNDERSCORE_LITERALS: + self.assertEqual(eval(lit), eval(lit.replace('_', ''))) + for lit in INVALID_UNDERSCORE_LITERALS: + self.assertRaises(SyntaxError, eval, lit) + # Sanity check: no literal begins with an underscore + self.assertRaises(NameError, eval, "_0") + def test_string_literals(self): x = ''; y = ""; self.assertTrue(len(x) == 0 and x == y) x = '\''; y = "'"; self.assertTrue(len(x) == 1 and x == y and ord(x) == 39) diff --git a/Lib/test/test_int.py b/Lib/test/test_int.py --- a/Lib/test/test_int.py +++ b/Lib/test/test_int.py @@ -2,6 +2,8 @@ import unittest from test import support +from test.test_grammar import (VALID_UNDERSCORE_LITERALS, + INVALID_UNDERSCORE_LITERALS) L = [ ('0', 0), @@ -212,6 +214,25 @@ self.assertEqual(int('2br45qc', 35), 4294967297) self.assertEqual(int('1z141z5', 36), 4294967297) + def test_underscores(self): + for lit in VALID_UNDERSCORE_LITERALS: + if any(ch in lit for ch in '.eEjJ'): + continue + self.assertEqual(int(lit, 0), eval(lit)) + self.assertEqual(int(lit, 0), int(lit.replace('_', ''), 0)) + for lit in INVALID_UNDERSCORE_LITERALS: + if any(ch in lit for ch in '.eEjJ'): + continue + self.assertRaises(ValueError, int, lit, 0) + # Additional test cases with bases != 0, only for the constructor: + self.assertEqual(int("1_00", 3), 9) + self.assertEqual(int("0_100"), 100) # not valid as a literal! + self.assertEqual(int(b"1_00"), 100) # byte underscore + self.assertRaises(ValueError, int, "_100") + self.assertRaises(ValueError, int, "+_100") + self.assertRaises(ValueError, int, "1__00") + self.assertRaises(ValueError, int, "100_") + @support.cpython_only def test_small_ints(self): # Bug #3236: Return small longs from PyLong_FromString diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py --- a/Lib/test/test_tokenize.py +++ b/Lib/test/test_tokenize.py @@ -3,7 +3,9 @@ STRING, ENDMARKER, ENCODING, tok_name, detect_encoding, open as tokenize_open, Untokenizer) from io import BytesIO -from unittest import TestCase, mock, main +from unittest import TestCase, mock +from test.test_grammar import (VALID_UNDERSCORE_LITERALS, + INVALID_UNDERSCORE_LITERALS) import os import token @@ -185,6 +187,21 @@ NUMBER '3.14e159' (1, 4) (1, 12) """) + def test_underscore_literals(self): + def number_token(s): + f = BytesIO(s.encode('utf-8')) + for toktype, token, start, end, line in tokenize(f.readline): + if toktype == NUMBER: + return token + return 'invalid token' + for lit in VALID_UNDERSCORE_LITERALS: + if '(' in lit: + # this won't work with compound complex inputs + continue + self.assertEqual(number_token(lit), lit) + for lit in INVALID_UNDERSCORE_LITERALS: + self.assertNotEqual(number_token(lit), lit) + def test_string(self): # String literals self.check_tokenize("x = ''; y = \"\"", """\ @@ -1529,11 +1546,10 @@ tempdir = os.path.dirname(fn) or os.curdir testfiles = glob.glob(os.path.join(tempdir, "test*.py")) - # Tokenize is broken on test_unicode_identifiers.py because regular - # expressions are broken on the obscure unicode identifiers in it. - # *sigh* With roundtrip extended to test the 5-tuple mode of - # untokenize, 7 more testfiles fail. Remove them also until the - # failure is diagnosed. + # Tokenize is broken on test_pep3131.py because regular expressions are + # broken on the obscure unicode identifiers in it. *sigh* + # With roundtrip extended to test the 5-tuple mode of untokenize, + # 7 more testfiles fail. Remove them also until the failure is diagnosed. testfiles.remove(os.path.join(tempdir, "test_unicode_identifiers.py")) for f in ('buffer', 'builtin', 'fileio', 'inspect', 'os', 'platform', 'sys'): @@ -1565,4 +1581,4 @@ if __name__ == "__main__": - main() + unittest.main() diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -48,6 +48,7 @@ def test_float_constructor(self): self.assertRaises(ValueError, float, '') self.assertRaises(ValueError, float, '5\0') + self.assertRaises(ValueError, float, '5_5\0') def test_zero_division(self): try: 5.0 / 0.0 diff --git a/Lib/tokenize.py b/Lib/tokenize.py --- a/Lib/tokenize.py +++ b/Lib/tokenize.py @@ -120,16 +120,17 @@ Ignore = Whitespace + any(r'\\\r?\n' + Whitespace) + maybe(Comment) Name = r'\w+' -Hexnumber = r'0[xX][0-9a-fA-F]+' -Binnumber = r'0[bB][01]+' -Octnumber = r'0[oO][0-7]+' -Decnumber = r'(?:0+|[1-9][0-9]*)' +Hexnumber = r'0[xX](?:_?[0-9a-fA-F])+' +Binnumber = r'0[bB](?:_?[01])+' +Octnumber = r'0[oO](?:_?[0-7])+' +Decnumber = r'(?:0(?:_?0)*|[1-9](?:_?[0-9])*)' Intnumber = group(Hexnumber, Binnumber, Octnumber, Decnumber) -Exponent = r'[eE][-+]?[0-9]+' -Pointfloat = group(r'[0-9]+\.[0-9]*', r'\.[0-9]+') + maybe(Exponent) -Expfloat = r'[0-9]+' + Exponent +Exponent = r'[eE][-+]?[0-9](?:_?[0-9])*' +Pointfloat = group(r'[0-9](?:_?[0-9])*\.(?:[0-9](?:_?[0-9])*)?', + r'\.[0-9](?:_?[0-9])*') + maybe(Exponent) +Expfloat = r'[0-9](?:_?[0-9])*' + Exponent Floatnumber = group(Pointfloat, Expfloat) -Imagnumber = group(r'[0-9]+[jJ]', Floatnumber + r'[jJ]') +Imagnumber = group(r'[0-9](?:_?[0-9])*[jJ]', Floatnumber + r'[jJ]') Number = group(Imagnumber, Floatnumber, Intnumber) # Return the empty string, plus all of the valid string prefixes. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,6 +17,8 @@ efficient bytecode. Patch by Demur Rumed, design by Serhiy Storchaka, reviewed by Serhiy Storchaka and Victor Stinner. +- Issue #26331: Implement tokenizing support for PEP 515. Patch by Georg Brandl. + - Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi. @@ -2678,7 +2680,7 @@ - Issue #24774: Fix docstring in http.server.test. Patch from Chiu-Hsiang Hsu. - Issue #21159: Improve message in configparser.InterpolationMissingOptionError. - Patch from ??ukasz Langa. + Patch from ????ukasz Langa. - Issue #20362: Honour TestCase.longMessage correctly in assertRegex. Patch from Ilia Kurenkov. @@ -4606,7 +4608,7 @@ Based on patch by Martin Panter. - Issue #17293: uuid.getnode() now determines MAC address on AIX using netstat. - Based on patch by Aivars Kalv??ns. + Based on patch by Aivars Kalv????ns. - Issue #22769: Fixed ttk.Treeview.tag_has() when called without arguments. diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -1889,12 +1889,13 @@ /* Return the ASCII representation of a numeric Unicode string. The numeric string may contain ascii characters in the range [1, 127], any Unicode space and any unicode digit. If strip_ws is true, leading and trailing - whitespace is stripped. + whitespace is stripped. If ignore_underscores is true, underscores are + ignored. Return NULL if malloc fails and an empty string if invalid characters are found. */ static char * -numeric_as_ascii(const PyObject *u, int strip_ws) +numeric_as_ascii(const PyObject *u, int strip_ws, int ignore_underscores) { enum PyUnicode_Kind kind; void *data; @@ -1929,6 +1930,9 @@ for (; j < len; j++) { ch = PyUnicode_READ(kind, data, j); + if (ignore_underscores && ch == '_') { + continue; + } if (0 < ch && ch <= 127) { *cp++ = ch; continue; @@ -2011,7 +2015,7 @@ PyObject *dec; char *s; - s = numeric_as_ascii(u, 0); + s = numeric_as_ascii(u, 0, 0); if (s == NULL) { return NULL; } @@ -2031,7 +2035,7 @@ PyObject *dec; char *s; - s = numeric_as_ascii(u, 1); + s = numeric_as_ascii(u, 1, 1); if (s == NULL) { return NULL; } diff --git a/Objects/complexobject.c b/Objects/complexobject.c --- a/Objects/complexobject.c +++ b/Objects/complexobject.c @@ -759,29 +759,12 @@ }; static PyObject * -complex_subtype_from_string(PyTypeObject *type, PyObject *v) +complex_from_string_inner(const char *s, Py_ssize_t len, void *type) { - const char *s, *start; - char *end; double x=0.0, y=0.0, z; int got_bracket=0; - PyObject *s_buffer = NULL; - Py_ssize_t len; - - if (PyUnicode_Check(v)) { - s_buffer = _PyUnicode_TransformDecimalAndSpaceToASCII(v); - if (s_buffer == NULL) - return NULL; - s = PyUnicode_AsUTF8AndSize(s_buffer, &len); - if (s == NULL) - goto error; - } - else { - PyErr_Format(PyExc_TypeError, - "complex() argument must be a string or a number, not '%.200s'", - Py_TYPE(v)->tp_name); - return NULL; - } + const char *start; + char *end; /* position on first nonblank */ start = s; @@ -822,7 +805,7 @@ if (PyErr_ExceptionMatches(PyExc_ValueError)) PyErr_Clear(); else - goto error; + return NULL; } if (end != s) { /* all 4 forms starting with land here */ @@ -835,7 +818,7 @@ if (PyErr_ExceptionMatches(PyExc_ValueError)) PyErr_Clear(); else - goto error; + return NULL; } if (end != s) /* j */ @@ -890,18 +873,46 @@ if (s-start != len) goto parse_error; - Py_XDECREF(s_buffer); - return complex_subtype_from_doubles(type, x, y); + return complex_subtype_from_doubles((PyTypeObject *)type, x, y); parse_error: PyErr_SetString(PyExc_ValueError, "complex() arg is a malformed string"); - error: - Py_XDECREF(s_buffer); return NULL; } static PyObject * +complex_subtype_from_string(PyTypeObject *type, PyObject *v) +{ + const char *s; + PyObject *s_buffer = NULL, *result = NULL; + Py_ssize_t len; + + if (PyUnicode_Check(v)) { + s_buffer = _PyUnicode_TransformDecimalAndSpaceToASCII(v); + if (s_buffer == NULL) { + return NULL; + } + s = PyUnicode_AsUTF8AndSize(s_buffer, &len); + if (s == NULL) { + goto exit; + } + } + else { + PyErr_Format(PyExc_TypeError, + "complex() argument must be a string or a number, not '%.200s'", + Py_TYPE(v)->tp_name); + return NULL; + } + + result = _Py_string_to_number_with_underscores(s, len, "complex", v, type, + complex_from_string_inner); + exit: + Py_DECREF(s_buffer); + return result; +} + +static PyObject * complex_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { PyObject *r, *i, *tmp; diff --git a/Objects/floatobject.c b/Objects/floatobject.c --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -124,11 +124,43 @@ return (PyObject *) op; } +static PyObject * +float_from_string_inner(const char *s, Py_ssize_t len, void *obj) +{ + double x; + const char *end; + const char *last = s + len; + /* strip space */ + while (s < last && Py_ISSPACE(*s)) { + s++; + } + + while (s < last - 1 && Py_ISSPACE(last[-1])) { + last--; + } + + /* We don't care about overflow or underflow. If the platform + * supports them, infinities and signed zeroes (on underflow) are + * fine. */ + x = PyOS_string_to_double(s, (char **)&end, NULL); + if (end != last) { + PyErr_Format(PyExc_ValueError, + "could not convert string to float: " + "%R", obj); + return NULL; + } + else if (x == -1.0 && PyErr_Occurred()) { + return NULL; + } + else { + return PyFloat_FromDouble(x); + } +} + PyObject * PyFloat_FromString(PyObject *v) { - const char *s, *last, *end; - double x; + const char *s; PyObject *s_buffer = NULL; Py_ssize_t len; Py_buffer view = {NULL, NULL}; @@ -169,27 +201,8 @@ Py_TYPE(v)->tp_name); return NULL; } - last = s + len; - /* strip space */ - while (s < last && Py_ISSPACE(*s)) - s++; - while (s < last - 1 && Py_ISSPACE(last[-1])) - last--; - /* We don't care about overflow or underflow. If the platform - * supports them, infinities and signed zeroes (on underflow) are - * fine. */ - x = PyOS_string_to_double(s, (char **)&end, NULL); - if (end != last) { - PyErr_Format(PyExc_ValueError, - "could not convert string to float: " - "%R", v); - result = NULL; - } - else if (x == -1.0 && PyErr_Occurred()) - result = NULL; - else - result = PyFloat_FromDouble(x); - + result = _Py_string_to_number_with_underscores(s, len, "float", v, v, + float_from_string_inner); PyBuffer_Release(&view); Py_XDECREF(s_buffer); return result; diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -2004,12 +2004,18 @@ * non-digit (which may be *str!). A normalized int is returned. * The point to this routine is that it takes time linear in the number of * string characters. + * + * Return values: + * -1 on syntax error (exception needs to be set, *res is untouched) + * 0 else (exception may be set, in that case *res is set to NULL) */ -static PyLongObject * -long_from_binary_base(const char **str, int base) +static int +long_from_binary_base(const char **str, int base, PyLongObject **res) { const char *p = *str; const char *start = p; + char prev = 0; + int digits = 0; int bits_per_char; Py_ssize_t n; PyLongObject *z; @@ -2019,23 +2025,43 @@ assert(base >= 2 && base <= 32 && (base & (base - 1)) == 0); n = base; - for (bits_per_char = -1; n; ++bits_per_char) + for (bits_per_char = -1; n; ++bits_per_char) { n >>= 1; - /* n <- total # of bits needed, while setting p to end-of-string */ - while (_PyLong_DigitValue[Py_CHARMASK(*p)] < base) + } + /* count digits and set p to end-of-string */ + while (_PyLong_DigitValue[Py_CHARMASK(*p)] < base || *p == '_') { + if (*p == '_') { + if (prev == '_') { + *str = p - 1; + return -1; + } + } else { + ++digits; + } + prev = *p; ++p; + } + if (prev == '_') { + /* Trailing underscore not allowed. */ + *str = p - 1; + return -1; + } + *str = p; /* n <- # of Python digits needed, = ceiling(n/PyLong_SHIFT). */ - n = (p - start) * bits_per_char + PyLong_SHIFT - 1; + n = digits * bits_per_char + PyLong_SHIFT - 1; if (n / bits_per_char < p - start) { PyErr_SetString(PyExc_ValueError, "int string too large to convert"); - return NULL; + *res = NULL; + return 0; } n = n / PyLong_SHIFT; z = _PyLong_New(n); - if (z == NULL) - return NULL; + if (z == NULL) { + *res = NULL; + return 0; + } /* Read string from right, and fill in int from left; i.e., * from least to most significant in both. */ @@ -2043,7 +2069,11 @@ bits_in_accum = 0; pdigit = z->ob_digit; while (--p >= start) { - int k = (int)_PyLong_DigitValue[Py_CHARMASK(*p)]; + int k; + if (*p == '_') { + continue; + } + k = (int)_PyLong_DigitValue[Py_CHARMASK(*p)]; assert(k >= 0 && k < base); accum |= (twodigits)k << bits_in_accum; bits_in_accum += bits_per_char; @@ -2062,7 +2092,8 @@ } while (pdigit - z->ob_digit < n) *pdigit++ = 0; - return long_normalize(z); + *res = long_normalize(z); + return 0; } /* Parses an int from a bytestring. Leading and trailing whitespace will be @@ -2087,23 +2118,29 @@ "int() arg 2 must be >= 2 and <= 36"); return NULL; } - while (*str != '\0' && Py_ISSPACE(Py_CHARMASK(*str))) + while (*str != '\0' && Py_ISSPACE(Py_CHARMASK(*str))) { str++; - if (*str == '+') + } + if (*str == '+') { ++str; + } else if (*str == '-') { ++str; sign = -1; } if (base == 0) { - if (str[0] != '0') + if (str[0] != '0') { base = 10; - else if (str[1] == 'x' || str[1] == 'X') + } + else if (str[1] == 'x' || str[1] == 'X') { base = 16; - else if (str[1] == 'o' || str[1] == 'O') + } + else if (str[1] == 'o' || str[1] == 'O') { base = 8; - else if (str[1] == 'b' || str[1] == 'B') + } + else if (str[1] == 'b' || str[1] == 'B') { base = 2; + } else { /* "old" (C-style) octal literal, now invalid. it might still be zero though */ @@ -2114,12 +2151,26 @@ if (str[0] == '0' && ((base == 16 && (str[1] == 'x' || str[1] == 'X')) || (base == 8 && (str[1] == 'o' || str[1] == 'O')) || - (base == 2 && (str[1] == 'b' || str[1] == 'B')))) + (base == 2 && (str[1] == 'b' || str[1] == 'B')))) { str += 2; + /* One underscore allowed here. */ + if (*str == '_') { + ++str; + } + } + if (str[0] == '_') { + /* May not start with underscores. */ + goto onError; + } start = str; - if ((base & (base - 1)) == 0) - z = long_from_binary_base(&str, base); + if ((base & (base - 1)) == 0) { + int res = long_from_binary_base(&str, base, &z); + if (res < 0) { + /* Syntax error. */ + goto onError; + } + } else { /*** Binary bases can be converted in time linear in the number of digits, because @@ -2208,11 +2259,13 @@ ***/ twodigits c; /* current input character */ Py_ssize_t size_z; + int digits = 0; int i; int convwidth; twodigits convmultmax, convmult; digit *pz, *pzstop; - const char* scan; + const char *scan, *lastdigit; + char prev = 0; static double log_base_BASE[37] = {0.0e0,}; static int convwidth_base[37] = {0,}; @@ -2226,8 +2279,9 @@ log((double)PyLong_BASE)); for (;;) { twodigits next = convmax * base; - if (next > PyLong_BASE) + if (next > PyLong_BASE) { break; + } convmax = next; ++i; } @@ -2238,21 +2292,43 @@ /* Find length of the string of numeric characters. */ scan = str; - while (_PyLong_DigitValue[Py_CHARMASK(*scan)] < base) + lastdigit = str; + + while (_PyLong_DigitValue[Py_CHARMASK(*scan)] < base || *scan == '_') { + if (*scan == '_') { + if (prev == '_') { + /* Only one underscore allowed. */ + str = lastdigit + 1; + goto onError; + } + } + else { + ++digits; + lastdigit = scan; + } + prev = *scan; ++scan; + } + if (prev == '_') { + /* Trailing underscore not allowed. */ + /* Set error pointer to first underscore. */ + str = lastdigit + 1; + goto onError; + } /* Create an int object that can contain the largest possible * integer with this base and length. Note that there's no * need to initialize z->ob_digit -- no slot is read up before * being stored into. */ - size_z = (Py_ssize_t)((scan - str) * log_base_BASE[base]) + 1; + size_z = (Py_ssize_t)(digits * log_base_BASE[base]) + 1; /* Uncomment next line to test exceedingly rare copy code */ /* size_z = 1; */ assert(size_z > 0); z = _PyLong_New(size_z); - if (z == NULL) + if (z == NULL) { return NULL; + } Py_SIZE(z) = 0; /* `convwidth` consecutive input digits are treated as a single @@ -2263,9 +2339,17 @@ /* Work ;-) */ while (str < scan) { + if (*str == '_') { + str++; + continue; + } /* grab up to convwidth digits from the input string */ c = (digit)_PyLong_DigitValue[Py_CHARMASK(*str++)]; - for (i = 1; i < convwidth && str != scan; ++i, ++str) { + for (i = 1; i < convwidth && str != scan; ++str) { + if (*str == '_') { + continue; + } + i++; c = (twodigits)(c * base + (int)_PyLong_DigitValue[Py_CHARMASK(*str)]); assert(c < PyLong_BASE); @@ -2277,8 +2361,9 @@ */ if (i != convwidth) { convmult = base; - for ( ; i > 1; --i) + for ( ; i > 1; --i) { convmult *= base; + } } /* Multiply z by convmult, and add c. */ @@ -2316,41 +2401,51 @@ } } } - if (z == NULL) + if (z == NULL) { return NULL; + } if (error_if_nonzero) { /* reset the base to 0, else the exception message doesn't make too much sense */ base = 0; - if (Py_SIZE(z) != 0) + if (Py_SIZE(z) != 0) { goto onError; + } /* there might still be other problems, therefore base remains zero here for the same reason */ } - if (str == start) + if (str == start) { goto onError; - if (sign < 0) + } + if (sign < 0) { Py_SIZE(z) = -(Py_SIZE(z)); - while (*str && Py_ISSPACE(Py_CHARMASK(*str))) + } + while (*str && Py_ISSPACE(Py_CHARMASK(*str))) { str++; - if (*str != '\0') + } + if (*str != '\0') { goto onError; + } long_normalize(z); z = maybe_small_long(z); - if (z == NULL) + if (z == NULL) { return NULL; - if (pend != NULL) + } + if (pend != NULL) { *pend = (char *)str; + } return (PyObject *) z; onError: - if (pend != NULL) + if (pend != NULL) { *pend = (char *)str; + } Py_XDECREF(z); slen = strlen(orig_str) < 200 ? strlen(orig_str) : 200; strobj = PyUnicode_FromStringAndSize(orig_str, slen); - if (strobj == NULL) + if (strobj == NULL) { return NULL; + } PyErr_Format(PyExc_ValueError, "invalid literal for int() with base %d: %.200R", base, strobj); diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1333,6 +1333,28 @@ } #endif +static int +tok_decimal_tail(struct tok_state *tok) +{ + int c; + + while (1) { + do { + c = tok_nextc(tok); + } while (isdigit(c)); + if (c != '_') { + break; + } + c = tok_nextc(tok); + if (!isdigit(c)) { + tok->done = E_TOKEN; + tok_backup(tok, c); + return 0; + } + } + return c; +} + /* Get next token, after space stripping etc. */ static int @@ -1353,17 +1375,20 @@ tok->atbol = 0; for (;;) { c = tok_nextc(tok); - if (c == ' ') + if (c == ' ') { col++, altcol++; + } else if (c == '\t') { col = (col/tok->tabsize + 1) * tok->tabsize; altcol = (altcol/tok->alttabsize + 1) * tok->alttabsize; } - else if (c == '\014') /* Control-L (formfeed) */ + else if (c == '\014') {/* Control-L (formfeed) */ col = altcol = 0; /* For Emacs users */ - else + } + else { break; + } } tok_backup(tok, c); if (c == '#' || c == '\n') { @@ -1372,10 +1397,12 @@ not passed to the parser as NEWLINE tokens, except *totally* empty lines in interactive mode, which signal the end of a command group. */ - if (col == 0 && c == '\n' && tok->prompt != NULL) + if (col == 0 && c == '\n' && tok->prompt != NULL) { blankline = 0; /* Let it through */ - else + } + else { blankline = 1; /* Ignore completely */ + } /* We can't jump back right here since we still may need to skip to the end of a comment */ } @@ -1383,8 +1410,9 @@ if (col == tok->indstack[tok->indent]) { /* No change */ if (altcol != tok->altindstack[tok->indent]) { - if (indenterror(tok)) + if (indenterror(tok)) { return ERRORTOKEN; + } } } else if (col > tok->indstack[tok->indent]) { @@ -1395,8 +1423,9 @@ return ERRORTOKEN; } if (altcol <= tok->altindstack[tok->indent]) { - if (indenterror(tok)) + if (indenterror(tok)) { return ERRORTOKEN; + } } tok->pendin++; tok->indstack[++tok->indent] = col; @@ -1415,8 +1444,9 @@ return ERRORTOKEN; } if (altcol != tok->altindstack[tok->indent]) { - if (indenterror(tok)) + if (indenterror(tok)) { return ERRORTOKEN; + } } } } @@ -1462,9 +1492,11 @@ tok->start = tok->cur - 1; /* Skip comment */ - if (c == '#') - while (c != EOF && c != '\n') + if (c == '#') { + while (c != EOF && c != '\n') { c = tok_nextc(tok); + } + } /* Check for EOF and errors now */ if (c == EOF) { @@ -1481,27 +1513,35 @@ saw_b = 1; /* Since this is a backwards compatibility support literal we don't want to support it in arbitrary order like byte literals. */ - else if (!(saw_b || saw_u || saw_r || saw_f) && (c == 'u' || c == 'U')) + else if (!(saw_b || saw_u || saw_r || saw_f) + && (c == 'u'|| c == 'U')) { saw_u = 1; + } /* ur"" and ru"" are not supported */ - else if (!(saw_r || saw_u) && (c == 'r' || c == 'R')) + else if (!(saw_r || saw_u) && (c == 'r' || c == 'R')) { saw_r = 1; - else if (!(saw_f || saw_b || saw_u) && (c == 'f' || c == 'F')) + } + else if (!(saw_f || saw_b || saw_u) && (c == 'f' || c == 'F')) { saw_f = 1; - else + } + else { break; + } c = tok_nextc(tok); - if (c == '"' || c == '\'') + if (c == '"' || c == '\'') { goto letter_quote; + } } while (is_potential_identifier_char(c)) { - if (c >= 128) + if (c >= 128) { nonascii = 1; + } c = tok_nextc(tok); } tok_backup(tok, c); - if (nonascii && !verify_identifier(tok)) + if (nonascii && !verify_identifier(tok)) { return ERRORTOKEN; + } *p_start = tok->start; *p_end = tok->cur; @@ -1510,10 +1550,12 @@ /* Current token length is 5. */ if (tok->async_def) { /* We're inside an 'async def' function. */ - if (memcmp(tok->start, "async", 5) == 0) + if (memcmp(tok->start, "async", 5) == 0) { return ASYNC; - if (memcmp(tok->start, "await", 5) == 0) + } + if (memcmp(tok->start, "await", 5) == 0) { return AWAIT; + } } else if (memcmp(tok->start, "async", 5) == 0) { /* The current token is 'async'. @@ -1546,8 +1588,9 @@ /* Newline */ if (c == '\n') { tok->atbol = 1; - if (blankline || tok->level > 0) + if (blankline || tok->level > 0) { goto nextline; + } *p_start = tok->start; *p_end = tok->cur - 1; /* Leave '\n' out of the string */ tok->cont_line = 0; @@ -1570,11 +1613,13 @@ *p_start = tok->start; *p_end = tok->cur; return ELLIPSIS; - } else { + } + else { tok_backup(tok, c); } tok_backup(tok, '.'); - } else { + } + else { tok_backup(tok, c); } *p_start = tok->start; @@ -1588,59 +1633,93 @@ /* Hex, octal or binary -- maybe. */ c = tok_nextc(tok); if (c == 'x' || c == 'X') { - /* Hex */ c = tok_nextc(tok); - if (!isxdigit(c)) { - tok->done = E_TOKEN; - tok_backup(tok, c); - return ERRORTOKEN; - } do { - c = tok_nextc(tok); - } while (isxdigit(c)); + if (c == '_') { + c = tok_nextc(tok); + } + if (!isxdigit(c)) { + tok->done = E_TOKEN; + tok_backup(tok, c); + return ERRORTOKEN; + } + do { + c = tok_nextc(tok); + } while (isxdigit(c)); + } while (c == '_'); } else if (c == 'o' || c == 'O') { /* Octal */ c = tok_nextc(tok); - if (c < '0' || c >= '8') { - tok->done = E_TOKEN; - tok_backup(tok, c); - return ERRORTOKEN; - } do { - c = tok_nextc(tok); - } while ('0' <= c && c < '8'); + if (c == '_') { + c = tok_nextc(tok); + } + if (c < '0' || c >= '8') { + tok->done = E_TOKEN; + tok_backup(tok, c); + return ERRORTOKEN; + } + do { + c = tok_nextc(tok); + } while ('0' <= c && c < '8'); + } while (c == '_'); } else if (c == 'b' || c == 'B') { /* Binary */ c = tok_nextc(tok); - if (c != '0' && c != '1') { - tok->done = E_TOKEN; - tok_backup(tok, c); - return ERRORTOKEN; - } do { - c = tok_nextc(tok); - } while (c == '0' || c == '1'); + if (c == '_') { + c = tok_nextc(tok); + } + if (c != '0' && c != '1') { + tok->done = E_TOKEN; + tok_backup(tok, c); + return ERRORTOKEN; + } + do { + c = tok_nextc(tok); + } while (c == '0' || c == '1'); + } while (c == '_'); } else { int nonzero = 0; /* maybe old-style octal; c is first char of it */ /* in any case, allow '0' as a literal */ - while (c == '0') - c = tok_nextc(tok); - while (isdigit(c)) { - nonzero = 1; + while (1) { + if (c == '_') { + c = tok_nextc(tok); + if (!isdigit(c)) { + tok->done = E_TOKEN; + tok_backup(tok, c); + return ERRORTOKEN; + } + } + if (c != '0') { + break; + } c = tok_nextc(tok); } - if (c == '.') + if (isdigit(c)) { + nonzero = 1; + c = tok_decimal_tail(tok); + if (c == 0) { + return ERRORTOKEN; + } + } + if (c == '.') { + c = tok_nextc(tok); goto fraction; - else if (c == 'e' || c == 'E') + } + else if (c == 'e' || c == 'E') { goto exponent; - else if (c == 'j' || c == 'J') + } + else if (c == 'j' || c == 'J') { goto imaginary; + } else if (nonzero) { + /* Old-style octal: now disallowed. */ tok->done = E_TOKEN; tok_backup(tok, c); return ERRORTOKEN; @@ -1649,17 +1728,22 @@ } else { /* Decimal */ - do { - c = tok_nextc(tok); - } while (isdigit(c)); + c = tok_decimal_tail(tok); + if (c == 0) { + return ERRORTOKEN; + } { /* Accept floating point numbers. */ if (c == '.') { + c = tok_nextc(tok); fraction: /* Fraction */ - do { - c = tok_nextc(tok); - } while (isdigit(c)); + if (isdigit(c)) { + c = tok_decimal_tail(tok); + if (c == 0) { + return ERRORTOKEN; + } + } } if (c == 'e' || c == 'E') { int e; @@ -1681,14 +1765,16 @@ *p_end = tok->cur; return NUMBER; } - do { - c = tok_nextc(tok); - } while (isdigit(c)); + c = tok_decimal_tail(tok); + if (c == 0) { + return ERRORTOKEN; + } } - if (c == 'j' || c == 'J') + if (c == 'j' || c == 'J') { /* Imaginary part */ imaginary: c = tok_nextc(tok); + } } } tok_backup(tok, c); @@ -1708,22 +1794,27 @@ c = tok_nextc(tok); if (c == quote) { c = tok_nextc(tok); - if (c == quote) + if (c == quote) { quote_size = 3; - else + } + else { end_quote_size = 1; /* empty string found */ + } } - if (c != quote) + if (c != quote) { tok_backup(tok, c); + } /* Get rest of string */ while (end_quote_size != quote_size) { c = tok_nextc(tok); if (c == EOF) { - if (quote_size == 3) + if (quote_size == 3) { tok->done = E_EOFS; - else + } + else { tok->done = E_EOLS; + } tok->cur = tok->inp; return ERRORTOKEN; } @@ -1732,12 +1823,14 @@ tok->cur = tok->inp; return ERRORTOKEN; } - if (c == quote) + if (c == quote) { end_quote_size += 1; + } else { end_quote_size = 0; - if (c == '\\') + if (c == '\\') { tok_nextc(tok); /* skip escaped char */ + } } } @@ -1767,7 +1860,8 @@ int token3 = PyToken_ThreeChars(c, c2, c3); if (token3 != OP) { token = token3; - } else { + } + else { tok_backup(tok, c3); } *p_start = tok->start; diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -4018,7 +4018,7 @@ } static PyObject * -parsenumber(struct compiling *c, const char *s) +parsenumber_raw(struct compiling *c, const char *s) { const char *end; long x; @@ -4061,6 +4061,31 @@ } static PyObject * +parsenumber(struct compiling *c, const char *s) +{ + char *dup, *end; + PyObject *res = NULL; + + assert(s != NULL); + + if (strchr(s, '_') == NULL) { + return parsenumber_raw(c, s); + } + /* Create a duplicate without underscores. */ + dup = PyMem_Malloc(strlen(s) + 1); + end = dup; + for (; *s; s++) { + if (*s != '_') { + *end++ = *s; + } + } + *end = '\0'; + res = parsenumber_raw(c, dup); + PyMem_Free(dup); + return res; +} + +static PyObject * decode_utf8(struct compiling *c, const char **sPtr, const char *end) { const char *s, *t; diff --git a/Python/pystrtod.c b/Python/pystrtod.c --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -370,6 +370,72 @@ return result; } +/* Remove underscores that follow the underscore placement rule from + the string and then call the `innerfunc` function on the result. + It should return a new object or NULL on exception. + + `what` is used for the error message emitted when underscores are detected + that don't follow the rule. `arg` is an opaque pointer passed to the inner + function. + + This is used to implement underscore-agnostic conversion for floats + and complex numbers. +*/ +PyObject * +_Py_string_to_number_with_underscores( + const char *s, Py_ssize_t orig_len, const char *what, PyObject *obj, void *arg, + PyObject *(*innerfunc)(const char *, Py_ssize_t, void *)) +{ + char prev; + const char *p, *last; + char *dup, *end; + PyObject *result; + + if (strchr(s, '_') == NULL) { + return innerfunc(s, orig_len, arg); + } + + dup = PyMem_Malloc(orig_len + 1); + end = dup; + prev = '\0'; + last = s + orig_len; + for (p = s; *p; p++) { + if (*p == '_') { + /* Underscores are only allowed after digits. */ + if (!(prev >= '0' && prev <= '9')) { + goto error; + } + } + else { + *end++ = *p; + /* Underscores are only allowed before digits. */ + if (prev == '_' && !(*p >= '0' && *p <= '9')) { + goto error; + } + } + prev = *p; + } + /* Underscores are not allowed at the end. */ + if (prev == '_') { + goto error; + } + /* No embedded NULs allowed. */ + if (p != last) { + goto error; + } + *end = '\0'; + result = innerfunc(dup, end - dup, arg); + PyMem_Free(dup); + return result; + + error: + PyMem_Free(dup); + PyErr_Format(PyExc_ValueError, + "could not convert string to %s: " + "%R", what, obj); + return NULL; +} + #ifdef PY_NO_SHORT_FLOAT_REPR /* Given a string that may have a decimal point in the current -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:57:48 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 21:57:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324186=3A_Reenable?= =?utf-8?q?_optimised_OpenSSL_function?= Message-ID: <20160909215748.59404.73485.8944F5A7@psf.io> https://hg.python.org/cpython/rev/6a8a6e26a1bc changeset: 103473:6a8a6e26a1bc user: Steve Dower date: Fri Sep 09 14:57:39 2016 -0700 summary: Issue #24186: Reenable optimised OpenSSL function files: PCbuild/openssl.props | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/PCbuild/openssl.props b/PCbuild/openssl.props --- a/PCbuild/openssl.props +++ b/PCbuild/openssl.props @@ -19,6 +19,7 @@ + @@ -38,7 +39,6 @@ - -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 17:59:08 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 21:59:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_ceval_timestamp_sup?= =?utf-8?q?port?= Message-ID: <20160909215908.19368.53011.FC69E5D7@psf.io> https://hg.python.org/cpython/rev/2310a5b75fae changeset: 103474:2310a5b75fae user: Benjamin Peterson date: Fri Sep 09 14:57:58 2016 -0700 summary: remove ceval timestamp support files: Doc/library/sys.rst | 12 -- Include/pystate.h | 3 - Misc/SpecialBuilds.txt | 26 ---- Python/ceval.c | 154 +---------------------------- Python/pystate.c | 3 - Python/sysmodule.c | 30 ----- configure | 25 ---- configure.ac | 13 -- pyconfig.h.in | 3 - 9 files changed, 1 insertions(+), 268 deletions(-) diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1104,18 +1104,6 @@ thus may not be available in all Python implementations. -.. function:: settscdump(on_flag) - - Activate dumping of VM measurements using the Pentium timestamp counter, if - *on_flag* is true. Deactivate these dumps if *on_flag* is off. The function is - available only if Python was compiled with ``--with-tsc``. To understand - the output of this dump, read :file:`Python/ceval.c` in the Python sources. - - .. impl-detail:: - This function is intimately bound to CPython implementation details and - thus not likely to be implemented elsewhere. - - .. function:: set_coroutine_wrapper(wrapper) Allows intercepting creation of :term:`coroutine` objects (only ones that diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -43,9 +43,6 @@ #ifdef HAVE_DLOPEN int dlopenflags; #endif -#ifdef WITH_TSC - int tscdump; -#endif PyObject *builtins_copy; PyObject *import_func; diff --git a/Misc/SpecialBuilds.txt b/Misc/SpecialBuilds.txt --- a/Misc/SpecialBuilds.txt +++ b/Misc/SpecialBuilds.txt @@ -234,29 +234,3 @@ number of function calls made. It keeps detailed statistics about what kind of object was called and whether the call hit any of the special fast paths in the code. - - -WITH_TSC --------- - -Super-lowlevel profiling of the interpreter. When enabled, the sys module grows -a new function: - -settscdump(bool) - If true, tell the Python interpreter to dump VM measurements to stderr. If - false, turn off dump. The measurements are based on the processor's - time-stamp counter. - -This build option requires a small amount of platform specific code. Currently -this code is present for linux/x86 and any PowerPC platform that uses GCC -(i.e. OS X and linux/ppc). - -On the PowerPC the rate at which the time base register is incremented is not -defined by the architecture specification, so you'll need to find the manual for -your specific processor. For the 750CX, 750CXe and 750FX (all sold as the G3) -we find: - - The time base counter is clocked at a frequency that is one-fourth that of - the bus clock. - -This build is enabled by the --with-tsc flag to configure. diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -20,82 +20,6 @@ #include -#ifndef WITH_TSC - -#define READ_TIMESTAMP(var) - -#else - -typedef unsigned long long uint64; - -/* PowerPC support. - "__ppc__" appears to be the preprocessor definition to detect on OS X, whereas - "__powerpc__" appears to be the correct one for Linux with GCC -*/ -#if defined(__ppc__) || defined (__powerpc__) - -#define READ_TIMESTAMP(var) ppc_getcounter(&var) - -static void -ppc_getcounter(uint64 *v) -{ - unsigned long tbu, tb, tbu2; - - loop: - asm volatile ("mftbu %0" : "=r" (tbu) ); - asm volatile ("mftb %0" : "=r" (tb) ); - asm volatile ("mftbu %0" : "=r" (tbu2)); - if (__builtin_expect(tbu != tbu2, 0)) goto loop; - - /* The slightly peculiar way of writing the next lines is - compiled better by GCC than any other way I tried. */ - ((long*)(v))[0] = tbu; - ((long*)(v))[1] = tb; -} - -#elif defined(__i386__) - -/* this is for linux/x86 (and probably any other GCC/x86 combo) */ - -#define READ_TIMESTAMP(val) \ - __asm__ __volatile__("rdtsc" : "=A" (val)) - -#elif defined(__x86_64__) - -/* for gcc/x86_64, the "A" constraint in DI mode means *either* rax *or* rdx; - not edx:eax as it does for i386. Since rdtsc puts its result in edx:eax - even in 64-bit mode, we need to use "a" and "d" for the lower and upper - 32-bit pieces of the result. */ - -#define READ_TIMESTAMP(val) do { \ - unsigned int h, l; \ - __asm__ __volatile__("rdtsc" : "=a" (l), "=d" (h)); \ - (val) = ((uint64)l) | (((uint64)h) << 32); \ - } while(0) - - -#else - -#error "Don't know how to implement timestamp counter for this architecture" - -#endif - -void dump_tsc(int opcode, int ticked, uint64 inst0, uint64 inst1, - uint64 loop0, uint64 loop1, uint64 intr0, uint64 intr1) -{ - uint64 intr, inst, loop; - PyThreadState *tstate = PyThreadState_Get(); - if (!tstate->interp->tscdump) - return; - intr = intr1 - intr0; - inst = inst1 - inst0 - intr; - loop = loop1 - loop0 - intr; - fprintf(stderr, "opcode=%03d t=%d inst=%06lld loop=%06lld\n", - opcode, ticked, inst, loop); -} - -#endif - /* Turn this on if your compiler chokes on the big switch: */ /* #define CASE_TOO_BIG 1 */ @@ -108,11 +32,7 @@ typedef PyObject *(*callproc)(PyObject *, PyObject *, PyObject *); /* Forward declarations */ -#ifdef WITH_TSC -static PyObject * call_function(PyObject ***, Py_ssize_t, PyObject *, uint64*, uint64*); -#else static PyObject * call_function(PyObject ***, Py_ssize_t, PyObject *); -#endif static PyObject * fast_function(PyObject *, PyObject **, Py_ssize_t, PyObject *); static PyObject * do_call_core(PyObject *, PyObject *, PyObject *); @@ -938,46 +858,6 @@ #define GETITEM(v, i) PyTuple_GetItem((v), (i)) #endif -#ifdef WITH_TSC -/* Use Pentium timestamp counter to mark certain events: - inst0 -- beginning of switch statement for opcode dispatch - inst1 -- end of switch statement (may be skipped) - loop0 -- the top of the mainloop - loop1 -- place where control returns again to top of mainloop - (may be skipped) - intr1 -- beginning of long interruption - intr2 -- end of long interruption - - Many opcodes call out to helper C functions. In some cases, the - time in those functions should be counted towards the time for the - opcode, but not in all cases. For example, a CALL_FUNCTION opcode - calls another Python function; there's no point in charge all the - bytecode executed by the called function to the caller. - - It's hard to make a useful judgement statically. In the presence - of operator overloading, it's impossible to tell if a call will - execute new Python code or not. - - It's a case-by-case judgement. I'll use intr1 for the following - cases: - - IMPORT_STAR - IMPORT_FROM - CALL_FUNCTION (and friends) - - */ - uint64 inst0, inst1, loop0, loop1, intr0 = 0, intr1 = 0; - int ticked = 0; - - READ_TIMESTAMP(inst0); - READ_TIMESTAMP(inst1); - READ_TIMESTAMP(loop0); - READ_TIMESTAMP(loop1); - - /* shut up the compiler */ - opcode = 0; -#endif - /* Code access macros */ #ifdef WORDS_BIGENDIAN @@ -1225,23 +1105,6 @@ #endif for (;;) { -#ifdef WITH_TSC - if (inst1 == 0) { - /* Almost surely, the opcode executed a break - or a continue, preventing inst1 from being set - on the way out of the loop. - */ - READ_TIMESTAMP(inst1); - loop1 = inst1; - } - dump_tsc(opcode, ticked, inst0, inst1, loop0, loop1, - intr0, intr1); - ticked = 0; - inst1 = 0; - intr0 = 0; - intr1 = 0; - READ_TIMESTAMP(loop0); -#endif assert(stack_pointer >= f->f_valuestack); /* else underflow */ assert(STACK_LEVEL() <= co->co_stacksize); /* else overflow */ assert(!PyErr_Occurred()); @@ -1260,9 +1123,6 @@ a try: finally: block uninterruptible. */ goto fast_next_opcode; } -#ifdef WITH_TSC - ticked = 1; -#endif if (_Py_atomic_load_relaxed(&pendingcalls_to_do)) { if (Py_MakePendingCalls() < 0) goto error; @@ -3403,11 +3263,7 @@ PyObject **sp, *res; PCALL(PCALL_ALL); sp = stack_pointer; -#ifdef WITH_TSC - res = call_function(&sp, oparg, NULL, &intr0, &intr1); -#else res = call_function(&sp, oparg, NULL); -#endif stack_pointer = sp; PUSH(res); if (res == NULL) { @@ -3423,11 +3279,7 @@ assert(PyTuple_CheckExact(names) && PyTuple_GET_SIZE(names) <= oparg); PCALL(PCALL_ALL); sp = stack_pointer; -#ifdef WITH_TSC - res = call_function(&sp, oparg, names, &intr0, &intr1); -#else res = call_function(&sp, oparg, names); -#endif stack_pointer = sp; PUSH(res); Py_DECREF(names); @@ -4922,11 +4774,7 @@ } static PyObject * -call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *kwnames -#ifdef WITH_TSC - , uint64* pintr0, uint64* pintr1 -#endif - ) +call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *kwnames) { PyObject **pfunc = (*pp_stack) - oparg - 1; PyObject *func = *pfunc; diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -99,9 +99,6 @@ interp->dlopenflags = RTLD_LAZY; #endif #endif -#ifdef WITH_TSC - interp->tscdump = 0; -#endif HEAD_LOCK(); interp->next = interp_head; diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -609,33 +609,6 @@ #endif /* WITH_THREAD */ -#ifdef WITH_TSC -static PyObject * -sys_settscdump(PyObject *self, PyObject *args) -{ - int bool; - PyThreadState *tstate = PyThreadState_Get(); - - if (!PyArg_ParseTuple(args, "i:settscdump", &bool)) - return NULL; - if (bool) - tstate->interp->tscdump = 1; - else - tstate->interp->tscdump = 0; - Py_INCREF(Py_None); - return Py_None; - -} - -PyDoc_STRVAR(settscdump_doc, -"settscdump(bool)\n\ -\n\ -If true, tell the Python interpreter to dump VM measurements to\n\ -stderr. If false, turn off dump. The measurements are based on the\n\ -processor's time-stamp counter." -); -#endif /* TSC */ - static PyObject * sys_setrecursionlimit(PyObject *self, PyObject *args) { @@ -1410,9 +1383,6 @@ {"getprofile", sys_getprofile, METH_NOARGS, getprofile_doc}, {"setrecursionlimit", sys_setrecursionlimit, METH_VARARGS, setrecursionlimit_doc}, -#ifdef WITH_TSC - {"settscdump", sys_settscdump, METH_VARARGS, settscdump_doc}, -#endif {"settrace", sys_settrace, METH_O, settrace_doc}, {"gettrace", sys_gettrace, METH_NOARGS, gettrace_doc}, {"call_tracing", sys_call_tracing, METH_VARARGS, call_tracing_doc}, diff --git a/configure b/configure --- a/configure +++ b/configure @@ -830,7 +830,6 @@ with_thread enable_ipv6 with_doc_strings -with_tsc with_pymalloc with_valgrind with_fpectl @@ -1534,7 +1533,6 @@ --with(out)-thread[=DIRECTORY] deprecated; use --with(out)-threads --with(out)-doc-strings disable/enable documentation strings - --with(out)-tsc enable/disable timestamp counter profile --with(out)-pymalloc disable/enable specialized mallocs --with-valgrind Enable Valgrind support --with-fpectl enable SIGFPE catching @@ -10798,29 +10796,6 @@ { $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_doc_strings" >&5 $as_echo "$with_doc_strings" >&6; } -# Check if eval loop should use timestamp counter profiling -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-tsc" >&5 -$as_echo_n "checking for --with-tsc... " >&6; } - -# Check whether --with-tsc was given. -if test "${with_tsc+set}" = set; then : - withval=$with_tsc; -if test "$withval" != no -then - -$as_echo "#define WITH_TSC 1" >>confdefs.h - - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } -else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - # Check for Python-specific malloc support { $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-pymalloc" >&5 $as_echo_n "checking for --with-pymalloc... " >&6; } diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3198,19 +3198,6 @@ fi AC_MSG_RESULT($with_doc_strings) -# Check if eval loop should use timestamp counter profiling -AC_MSG_CHECKING(for --with-tsc) -AC_ARG_WITH(tsc, - AS_HELP_STRING([--with(out)-tsc],[enable/disable timestamp counter profile]),[ -if test "$withval" != no -then - AC_DEFINE(WITH_TSC, 1, - [Define to profile with the Pentium timestamp counter]) - AC_MSG_RESULT(yes) -else AC_MSG_RESULT(no) -fi], -[AC_MSG_RESULT(no)]) - # Check for Python-specific malloc support AC_MSG_CHECKING(for --with-pymalloc) AC_ARG_WITH(pymalloc, diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -1388,9 +1388,6 @@ /* Define if you want to compile in rudimentary thread support */ #undef WITH_THREAD -/* Define to profile with the Pentium timestamp counter */ -#undef WITH_TSC - /* Define if you want pymalloc to be disabled when running under valgrind */ #undef WITH_VALGRIND -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:02:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 22:02:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_READ=5FTIMESTAMP_ma?= =?utf-8?q?cro?= Message-ID: <20160909220232.1920.89269.0C65C51F@psf.io> https://hg.python.org/cpython/rev/0fa2172f4260 changeset: 103475:0fa2172f4260 user: Benjamin Peterson date: Fri Sep 09 15:02:11 2016 -0700 summary: remove READ_TIMESTAMP macro files: Python/ceval.c | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1213,9 +1213,6 @@ } #endif - /* Main switch on opcode */ - READ_TIMESTAMP(inst0); - switch (opcode) { /* BEWARE! -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:03:41 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 22:03:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_remove_more_READ=5FTIMESTA?= =?utf-8?q?MP?= Message-ID: <20160909220341.21136.11450.18D8D6E6@psf.io> https://hg.python.org/cpython/rev/ac2378b0c116 changeset: 103476:ac2378b0c116 user: Benjamin Peterson date: Fri Sep 09 15:03:18 2016 -0700 summary: remove more READ_TIMESTAMP files: Python/ceval.c | 12 ------------ 1 files changed, 0 insertions(+), 12 deletions(-) diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2823,11 +2823,9 @@ PyObject *fromlist = POP(); PyObject *level = TOP(); PyObject *res; - READ_TIMESTAMP(intr0); res = import_name(f, name, fromlist, level); Py_DECREF(level); Py_DECREF(fromlist); - READ_TIMESTAMP(intr1); SET_TOP(res); if (res == NULL) goto error; @@ -2846,9 +2844,7 @@ "no locals found during 'import *'"); goto error; } - READ_TIMESTAMP(intr0); err = import_all_from(locals, from); - READ_TIMESTAMP(intr1); PyFrame_LocalsToFast(f, 0); Py_DECREF(from); if (err != 0) @@ -2860,9 +2856,7 @@ PyObject *name = GETITEM(names, oparg); PyObject *from = TOP(); PyObject *res; - READ_TIMESTAMP(intr0); res = import_from(from, name); - READ_TIMESTAMP(intr1); PUSH(res); if (res == NULL) goto error; @@ -3298,9 +3292,7 @@ assert(PyTuple_CheckExact(callargs)); func = TOP(); - READ_TIMESTAMP(intr0); result = do_call_core(func, callargs, kwargs); - READ_TIMESTAMP(intr1); Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); @@ -3451,7 +3443,6 @@ assert(0); error: - READ_TIMESTAMP(inst1); assert(why == WHY_NOT); why = WHY_EXCEPTION; @@ -3555,7 +3546,6 @@ if (why != WHY_NOT) break; - READ_TIMESTAMP(loop1); assert(!PyErr_Occurred()); @@ -4809,14 +4799,12 @@ stack = (*pp_stack) - nargs - nkwargs; - READ_TIMESTAMP(*pintr0); if (PyFunction_Check(func)) { x = fast_function(func, stack, nargs, kwnames); } else { x = _PyObject_FastCallKeywords(func, stack, nargs, kwnames); } - READ_TIMESTAMP(*pintr1); Py_DECREF(func); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:07:55 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 22:07:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Adds_documentation_for_pyt?= =?utf-8?q?honXX=2Ezip_as_a_landmark=2E?= Message-ID: <20160909220755.2613.91429.C9B84E57@psf.io> https://hg.python.org/cpython/rev/4e1316149051 changeset: 103477:4e1316149051 user: Steve Dower date: Fri Sep 09 15:07:46 2016 -0700 summary: Adds documentation for pythonXX.zip as a landmark. files: Doc/using/windows.rst | 21 +++++++++++++-------- 1 files changed, 13 insertions(+), 8 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -747,10 +747,11 @@ * If the environment variable :envvar:`PYTHONHOME` is set, it is assumed as "Python Home". Otherwise, the path of the main Python executable is used to - locate a "landmark file" (``Lib\os.py``) to deduce the "Python Home". If a - Python home is found, the relevant sub-directories added to :data:`sys.path` - (``Lib``, ``plat-win``, etc) are based on that folder. Otherwise, the core - Python path is constructed from the PythonPath stored in the registry. + locate a "landmark file" (either ``Lib\os.py`` or ``pythonXY.zip``) to deduce + the "Python Home". If a Python home is found, the relevant sub-directories + added to :data:`sys.path` (``Lib``, ``plat-win``, etc) are based on that + folder. Otherwise, the core Python path is constructed from the PythonPath + stored in the registry. * If the Python Home cannot be located, no :envvar:`PYTHONPATH` is specified in the environment, and no registry entries can be found, a default path with @@ -795,7 +796,8 @@ * If you cannot use the previous suggestions (for example, you are a distribution that allows people to run :file:`python.exe` directly), ensure that the landmark file (:file:`Lib\\os.py`) exists in your install directory. - (Note that it will not be detected inside a ZIP file.) + (Note that it will not be detected inside a ZIP file, but a correctly named + ZIP file will be detected instead.) These will ensure that the files in a system-wide installation will not take precedence over the copy of the standard library bundled with your application. @@ -803,10 +805,13 @@ the first suggestion is the best, as the other may still be susceptible to non-standard paths in the registry and user site-packages. -.. versionchanged:: 3.6 +.. versionchanged:: + 3.6 - Adds ``sys.path`` file support and removes ``applocal`` option from - ``pyvenv.cfg``. + * Adds ``sys.path`` file support and removes ``applocal`` option from + ``pyvenv.cfg``. + * Adds ``pythonXX.zip`` as a potential landmark when directly adjacent + to the executable. Additional modules ================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:09:58 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 22:09:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Adds_search_path_changes_t?= =?utf-8?q?o_whatsnew/3=2E6=2Erst?= Message-ID: <20160909220958.19592.7944.ACE008A7@psf.io> https://hg.python.org/cpython/rev/f04ae295fbca changeset: 103478:f04ae295fbca user: Steve Dower date: Fri Sep 09 15:09:30 2016 -0700 summary: Adds search path changes to whatsnew/3.6.rst files: Doc/whatsnew/3.6.rst | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -101,6 +101,11 @@ which means that when the 260 character path limit may no longer apply. See :ref:`removing the MAX_PATH limitation ` for details. +* A ``sys.path`` file can be added to force isolated mode and fully specify + all search paths to avoid registry and environment lookup. + +* A ``python36.zip`` file now works as a landmark to infer :envvar:`PYTHONHOME` + .. PEP-sized items next. .. _pep-4XX: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:15:11 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 09 Sep 2016 22:15:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=23433028=3A_Added_s?= =?utf-8?q?upport_of_modifier_spans_in_regular_expressions=2E?= Message-ID: <20160909221510.59726.48176.B9ED4EED@psf.io> https://hg.python.org/cpython/rev/ce5a834978ac changeset: 103479:ce5a834978ac parent: 103471:66afc449efa9 user: Serhiy Storchaka date: Sat Sep 10 00:57:55 2016 +0300 summary: Issue #433028: Added support of modifier spans in regular expressions. files: Doc/library/re.rst | 10 ++ Doc/whatsnew/3.6.rst | 9 ++ Lib/re.py | 2 +- Lib/sre_compile.py | 69 ++++++++++-------- Lib/sre_parse.py | 114 ++++++++++++++++++++++-------- Lib/test/test_re.py | 40 +++++++++- Misc/NEWS | 2 + 7 files changed, 180 insertions(+), 66 deletions(-) diff --git a/Doc/library/re.rst b/Doc/library/re.rst --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -237,6 +237,16 @@ *cannot* be retrieved after performing a match or referenced later in the pattern. +``(?imsx-imsx:...)`` + (Zero or more letters from the set ``'i'``, ``'m'``, ``'s'``, ``'x'``, + optionally followed by ``'-'`` followed by one or more letters from the + same set.) The letters set or removes the corresponding flags: + :const:`re.I` (ignore case), :const:`re.M` (multi-line), :const:`re.S` + (dot matches all), and :const:`re.X` (verbose), for the part of the + expression. (The flags are described in :ref:`contents-of-module-re`.) + + .. versionadded: 3.7 + ``(?P...)`` Similar to regular parentheses, but the substring matched by the group is accessible via the symbolic group name *name*. Group names must be valid diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -645,6 +645,15 @@ Storchaka in :issue:`24164`.) +re +-- + +Added support of modifier spans in regular expressions. Examples: +``'(?i:p)ython'`` matches ``'python'`` and ``'Python'``, but not ``'PYTHON'``; +``'(?i)g(?-i:v)r'`` matches ``'GvR'`` and ``'gvr'``, but not ``'GVR'``. +(Contributed by Serhiy Storchaka in :issue:`433028`.) + + readline -------- diff --git a/Lib/re.py b/Lib/re.py --- a/Lib/re.py +++ b/Lib/re.py @@ -352,7 +352,7 @@ for phrase, action in lexicon: gid = s.opengroup() p.append(sre_parse.SubPattern(s, [ - (SUBPATTERN, (gid, sre_parse.parse(phrase, flags))), + (SUBPATTERN, (gid, 0, 0, sre_parse.parse(phrase, flags))), ])) s.closegroup(gid, p[-1]) p = sre_parse.SubPattern(s, [(BRANCH, (None, p))]) diff --git a/Lib/sre_compile.py b/Lib/sre_compile.py --- a/Lib/sre_compile.py +++ b/Lib/sre_compile.py @@ -71,7 +71,8 @@ ASSERT_CODES = _ASSERT_CODES if (flags & SRE_FLAG_IGNORECASE and not (flags & SRE_FLAG_LOCALE) and - flags & SRE_FLAG_UNICODE): + flags & SRE_FLAG_UNICODE and + not (flags & SRE_FLAG_ASCII)): fixes = _ignorecase_fixes else: fixes = None @@ -137,14 +138,15 @@ else: emit(MIN_UNTIL) elif op is SUBPATTERN: - if av[0]: + group, add_flags, del_flags, p = av + if group: emit(MARK) - emit((av[0]-1)*2) - # _compile_info(code, av[1], flags) - _compile(code, av[1], flags) - if av[0]: + emit((group-1)*2) + # _compile_info(code, p, (flags | add_flags) & ~del_flags) + _compile(code, p, (flags | add_flags) & ~del_flags) + if group: emit(MARK) - emit((av[0]-1)*2+1) + emit((group-1)*2+1) elif op in SUCCESS_CODES: emit(op) elif op in ASSERT_CODES: @@ -172,7 +174,7 @@ av = AT_MULTILINE.get(av, av) if flags & SRE_FLAG_LOCALE: av = AT_LOCALE.get(av, av) - elif flags & SRE_FLAG_UNICODE: + elif (flags & SRE_FLAG_UNICODE) and not (flags & SRE_FLAG_ASCII): av = AT_UNICODE.get(av, av) emit(av) elif op is BRANCH: @@ -193,7 +195,7 @@ emit(op) if flags & SRE_FLAG_LOCALE: av = CH_LOCALE[av] - elif flags & SRE_FLAG_UNICODE: + elif (flags & SRE_FLAG_UNICODE) and not (flags & SRE_FLAG_ASCII): av = CH_UNICODE[av] emit(av) elif op is GROUPREF: @@ -237,7 +239,7 @@ elif op is CATEGORY: if flags & SRE_FLAG_LOCALE: emit(CH_LOCALE[av]) - elif flags & SRE_FLAG_UNICODE: + elif (flags & SRE_FLAG_UNICODE) and not (flags & SRE_FLAG_ASCII): emit(CH_UNICODE[av]) else: emit(av) @@ -414,14 +416,16 @@ prefix = [] prefixappend = prefix.append prefix_skip = None - got_all = True for op, av in pattern.data: if op is LITERAL: prefixappend(av) elif op is SUBPATTERN: - prefix1, prefix_skip1, got_all = _get_literal_prefix(av[1]) + group, add_flags, del_flags, p = av + if add_flags & SRE_FLAG_IGNORECASE: + break + prefix1, prefix_skip1, got_all = _get_literal_prefix(p) if prefix_skip is None: - if av[0] is not None: + if group is not None: prefix_skip = len(prefix) elif prefix_skip1 is not None: prefix_skip = len(prefix) + prefix_skip1 @@ -429,32 +433,35 @@ if not got_all: break else: - got_all = False break - return prefix, prefix_skip, got_all + else: + return prefix, prefix_skip, True + return prefix, prefix_skip, False def _get_charset_prefix(pattern): charset = [] # not used charsetappend = charset.append if pattern.data: op, av = pattern.data[0] - if op is SUBPATTERN and av[1]: - op, av = av[1][0] - if op is LITERAL: - charsetappend((op, av)) - elif op is BRANCH: - c = [] - cappend = c.append - for p in av[1]: - if not p: - break - op, av = p[0] - if op is LITERAL: - cappend((op, av)) + if op is SUBPATTERN: + group, add_flags, del_flags, p = av + if p and not (add_flags & SRE_FLAG_IGNORECASE): + op, av = p[0] + if op is LITERAL: + charsetappend((op, av)) + elif op is BRANCH: + c = [] + cappend = c.append + for p in av[1]: + if not p: + break + op, av = p[0] + if op is LITERAL: + cappend((op, av)) + else: + break else: - break - else: - charset = c + charset = c elif op is BRANCH: c = [] cappend = c.append diff --git a/Lib/sre_parse.py b/Lib/sre_parse.py --- a/Lib/sre_parse.py +++ b/Lib/sre_parse.py @@ -65,6 +65,12 @@ "u": SRE_FLAG_UNICODE, } +GLOBAL_FLAGS = (SRE_FLAG_ASCII | SRE_FLAG_LOCALE | SRE_FLAG_UNICODE | + SRE_FLAG_DEBUG | SRE_FLAG_TEMPLATE) + +class Verbose(Exception): + pass + class Pattern: # master pattern object. keeps track of global attributes def __init__(self): @@ -184,7 +190,7 @@ lo = lo + i hi = hi + j elif op is SUBPATTERN: - i, j = av[1].getwidth() + i, j = av[-1].getwidth() lo = lo + i hi = hi + j elif op in _REPEATCODES: @@ -395,7 +401,7 @@ pass raise source.error("bad escape %s" % escape, len(escape)) -def _parse_sub(source, state, nested=True): +def _parse_sub(source, state, verbose, nested=True): # parse an alternation: a|b|c items = [] @@ -403,7 +409,7 @@ sourcematch = source.match start = source.tell() while True: - itemsappend(_parse(source, state)) + itemsappend(_parse(source, state, verbose)) if not sourcematch("|"): break @@ -445,10 +451,10 @@ subpattern.append((BRANCH, (None, items))) return subpattern -def _parse_sub_cond(source, state, condgroup): - item_yes = _parse(source, state) +def _parse_sub_cond(source, state, condgroup, verbose): + item_yes = _parse(source, state, verbose) if source.match("|"): - item_no = _parse(source, state) + item_no = _parse(source, state, verbose) if source.next == "|": raise source.error("conditional backref with more than two branches") else: @@ -457,7 +463,7 @@ subpattern.append((GROUPREF_EXISTS, (condgroup, item_yes, item_no))) return subpattern -def _parse(source, state): +def _parse(source, state, verbose): # parse a simple pattern subpattern = SubPattern(state) @@ -467,7 +473,6 @@ sourcematch = source.match _len = len _ord = ord - verbose = state.flags & SRE_FLAG_VERBOSE while True: @@ -621,6 +626,8 @@ group = True name = None condgroup = None + add_flags = 0 + del_flags = 0 if sourcematch("?"): # options char = sourceget() @@ -682,7 +689,7 @@ lookbehindgroups = state.lookbehindgroups if lookbehindgroups is None: state.lookbehindgroups = state.groups - p = _parse_sub(source, state) + p = _parse_sub(source, state, verbose) if dir < 0: if lookbehindgroups is None: state.lookbehindgroups = None @@ -718,19 +725,13 @@ raise source.error("invalid group reference", len(condname) + 1) state.checklookbehindgroup(condgroup, source) - elif char in FLAGS: + elif char in FLAGS or char == "-": # flags - while True: - state.flags |= FLAGS[char] - char = sourceget() - if char is None: - raise source.error("missing )") - if char == ")": - break - if char not in FLAGS: - raise source.error("unknown flag", len(char)) - verbose = state.flags & SRE_FLAG_VERBOSE - continue + flags = _parse_flags(source, state, char) + if flags is None: # global flags + continue + add_flags, del_flags = flags + group = None else: raise source.error("unknown extension ?" + char, len(char) + 1) @@ -742,15 +743,17 @@ except error as err: raise source.error(err.msg, len(name) + 1) from None if condgroup: - p = _parse_sub_cond(source, state, condgroup) + p = _parse_sub_cond(source, state, condgroup, verbose) else: - p = _parse_sub(source, state) + sub_verbose = ((verbose or (add_flags & SRE_FLAG_VERBOSE)) and + not (del_flags & SRE_FLAG_VERBOSE)) + p = _parse_sub(source, state, sub_verbose) if not source.match(")"): raise source.error("missing ), unterminated subpattern", source.tell() - start) if group is not None: state.closegroup(group, p) - subpatternappend((SUBPATTERN, (group, p))) + subpatternappend((SUBPATTERN, (group, add_flags, del_flags, p))) elif this == "^": subpatternappend((AT, AT_BEGINNING)) @@ -763,6 +766,53 @@ return subpattern +def _parse_flags(source, state, char): + sourceget = source.get + add_flags = 0 + del_flags = 0 + if char != "-": + while True: + add_flags |= FLAGS[char] + char = sourceget() + if char is None: + raise source.error("missing -, : or )") + if char in ")-:": + break + if char not in FLAGS: + msg = "unknown flag" if char.isalpha() else "missing -, : or )" + raise source.error(msg, len(char)) + if char == ")": + if ((add_flags & SRE_FLAG_VERBOSE) and + not (state.flags & SRE_FLAG_VERBOSE)): + raise Verbose + state.flags |= add_flags + return None + if add_flags & GLOBAL_FLAGS: + raise source.error("bad inline flags: cannot turn on global flag", 1) + if char == "-": + char = sourceget() + if char is None: + raise source.error("missing flag") + if char not in FLAGS: + msg = "unknown flag" if char.isalpha() else "missing flag" + raise source.error(msg, len(char)) + while True: + del_flags |= FLAGS[char] + char = sourceget() + if char is None: + raise source.error("missing :") + if char == ":": + break + if char not in FLAGS: + msg = "unknown flag" if char.isalpha() else "missing :" + raise source.error(msg, len(char)) + assert char == ":" + if del_flags & GLOBAL_FLAGS: + raise source.error("bad inline flags: cannot turn off global flag", 1) + if add_flags & del_flags: + raise source.error("bad inline flags: flag turned on and off", 1) + return add_flags, del_flags + def fix_flags(src, flags): # Check and fix flags according to the type of pattern (str or bytes) if isinstance(src, str): @@ -789,18 +839,22 @@ pattern.flags = flags pattern.str = str - p = _parse_sub(source, pattern, 0) + try: + p = _parse_sub(source, pattern, flags & SRE_FLAG_VERBOSE, False) + except Verbose: + # the VERBOSE flag was switched on inside the pattern. to be + # on the safe side, we'll parse the whole thing again... + pattern = Pattern() + pattern.flags = flags | SRE_FLAG_VERBOSE + pattern.str = str + p = _parse_sub(source, pattern, True, False) + p.pattern.flags = fix_flags(str, p.pattern.flags) if source.next is not None: assert source.next == ")" raise source.error("unbalanced parenthesis") - if not (flags & SRE_FLAG_VERBOSE) and p.pattern.flags & SRE_FLAG_VERBOSE: - # the VERBOSE flag was switched on inside the pattern. to be - # on the safe side, we'll parse the whole thing again... - return parse(str, p.pattern.flags) - if flags & SRE_FLAG_DEBUG: p.dump() diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -1376,6 +1376,38 @@ self.assertRaises(ValueError, re.compile, b'(?a)', re.LOCALE) self.assertRaises(ValueError, re.compile, b'(?aL)') + def test_scoped_flags(self): + self.assertTrue(re.match(r'(?i:a)b', 'Ab')) + self.assertIsNone(re.match(r'(?i:a)b', 'aB')) + self.assertIsNone(re.match(r'(?-i:a)b', 'Ab', re.IGNORECASE)) + self.assertTrue(re.match(r'(?-i:a)b', 'aB', re.IGNORECASE)) + self.assertIsNone(re.match(r'(?i:(?-i:a)b)', 'Ab')) + self.assertTrue(re.match(r'(?i:(?-i:a)b)', 'aB')) + + self.assertTrue(re.match(r'(?x: a) b', 'a b')) + self.assertIsNone(re.match(r'(?x: a) b', ' a b')) + self.assertTrue(re.match(r'(?-x: a) b', ' ab', re.VERBOSE)) + self.assertIsNone(re.match(r'(?-x: a) b', 'ab', re.VERBOSE)) + + self.checkPatternError(r'(?a:\w)', + 'bad inline flags: cannot turn on global flag', 3) + self.checkPatternError(r'(?a)(?-a:\w)', + 'bad inline flags: cannot turn off global flag', 8) + self.checkPatternError(r'(?i-i:a)', + 'bad inline flags: flag turned on and off', 5) + + self.checkPatternError(r'(?-', 'missing flag', 3) + self.checkPatternError(r'(?-+', 'missing flag', 3) + self.checkPatternError(r'(?-z', 'unknown flag', 3) + self.checkPatternError(r'(?-i', 'missing :', 4) + self.checkPatternError(r'(?-i)', 'missing :', 4) + self.checkPatternError(r'(?-i+', 'missing :', 4) + self.checkPatternError(r'(?-iz', 'unknown flag', 4) + self.checkPatternError(r'(?i:', 'missing ), unterminated subpattern', 0) + self.checkPatternError(r'(?i', 'missing -, : or )', 3) + self.checkPatternError(r'(?i+', 'missing -, : or )', 3) + self.checkPatternError(r'(?iz', 'unknown flag', 3) + def test_bug_6509(self): # Replacement strings of both types must parse properly. # all strings @@ -1538,9 +1570,9 @@ with captured_stdout() as out: re.compile(pat, re.DEBUG) dump = '''\ -SUBPATTERN 1 +SUBPATTERN 1 0 0 LITERAL 46 -SUBPATTERN None +SUBPATTERN None 0 0 BRANCH IN LITERAL 99 @@ -1548,7 +1580,7 @@ OR LITERAL 112 LITERAL 121 -SUBPATTERN None +SUBPATTERN None 0 0 GROUPREF_EXISTS 1 AT AT_END ELSE @@ -1664,7 +1696,7 @@ self.checkPatternError(r'(?P', 'unexpected end of pattern', 3) self.checkPatternError(r'(?z)', 'unknown extension ?z', 1) self.checkPatternError(r'(?iz)', 'unknown flag', 3) - self.checkPatternError(r'(?i', 'missing )', 3) + self.checkPatternError(r'(?i', 'missing -, : or )', 3) self.checkPatternError(r'(?#abc', 'missing ), unterminated comment', 0) self.checkPatternError(r'(?<', 'unexpected end of pattern', 3) self.checkPatternError(r'(?<>)', 'unknown extension ?<>', 1) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -120,6 +120,8 @@ Library ------- +- Issue #433028: Added support of modifier spans in regular expressions. + - Issue #24594: Validates persist parameter when opening MSI database - Issue #28047: Fixed calculation of line length used for the base64 CTE -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:15:14 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Fri, 09 Sep 2016 22:15:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_heads?= Message-ID: <20160909221510.1774.24921.F1545AE9@psf.io> https://hg.python.org/cpython/rev/461fe9ee4038 changeset: 103480:461fe9ee4038 parent: 103479:ce5a834978ac parent: 103478:f04ae295fbca user: Serhiy Storchaka date: Sat Sep 10 01:14:38 2016 +0300 summary: Merge heads files: Doc/library/decimal.rst | 10 +- Doc/library/functions.rst | 16 +- Doc/library/sys.rst | 12 - Doc/reference/lexical_analysis.rst | 45 ++- Doc/using/windows.rst | 21 +- Doc/whatsnew/3.6.rst | 28 ++ Include/pystate.h | 3 - Include/pystrtod.h | 4 + Lib/_pydecimal.py | 10 +- Lib/test/test_complex.py | 14 + Lib/test/test_decimal.py | 10 + Lib/test/test_float.py | 24 +- Lib/test/test_grammar.py | 89 ++++++ Lib/test/test_int.py | 21 + Lib/test/test_tokenize.py | 30 +- Lib/test/test_types.py | 1 + Lib/tokenize.py | 17 +- Misc/NEWS | 6 +- Misc/SpecialBuilds.txt | 26 - Modules/_decimal/_decimal.c | 12 +- Objects/complexobject.c | 63 ++- Objects/floatobject.c | 59 ++- Objects/longobject.c | 169 +++++++++-- PCbuild/openssl.props | 2 +- Parser/tokenizer.c | 232 +++++++++++----- Python/ast.c | 27 +- Python/ceval.c | 169 +------------ Python/pystate.c | 3 - Python/pystrtod.c | 66 ++++ Python/sysmodule.c | 30 -- configure | 25 - configure.ac | 13 - pyconfig.h.in | 3 - 33 files changed, 763 insertions(+), 497 deletions(-) diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -345,7 +345,7 @@ *value* can be an integer, string, tuple, :class:`float`, or another :class:`Decimal` object. If no *value* is given, returns ``Decimal('0')``. If *value* is a string, it should conform to the decimal numeric string syntax after leading - and trailing whitespace characters are removed:: + and trailing whitespace characters, as well as underscores throughout, are removed:: sign ::= '+' | '-' digit ::= '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9' @@ -394,6 +394,10 @@ :class:`float` arguments raise an exception if the :exc:`FloatOperation` trap is set. By default the trap is off. + .. versionchanged:: 3.6 + Underscores are allowed for grouping, as with integral and floating-point + literals in code. + Decimal floating point objects share many properties with the other built-in numeric types such as :class:`float` and :class:`int`. All of the usual math operations and special methods apply. Likewise, decimal objects can be @@ -1075,8 +1079,8 @@ Decimal('4.44') This method implements the to-number operation of the IBM specification. - If the argument is a string, no leading or trailing whitespace is - permitted. + If the argument is a string, no leading or trailing whitespace or + underscores are permitted. .. method:: create_decimal_from_float(f) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -271,6 +271,9 @@ The complex type is described in :ref:`typesnumeric`. + .. versionchanged:: 3.6 + Grouping digits with underscores as in code literals is allowed. + .. function:: delattr(object, name) @@ -531,11 +534,14 @@ The float type is described in :ref:`typesnumeric`. - .. index:: - single: __format__ - single: string; format() (built-in function) + .. versionchanged:: 3.6 + Grouping digits with underscores as in code literals is allowed. +.. index:: + single: __format__ + single: string; format() (built-in function) + .. function:: format(value[, format_spec]) Convert a *value* to a "formatted" representation, as controlled by @@ -702,6 +708,10 @@ :meth:`base.__int__ ` instead of :meth:`base.__index__ `. + .. versionchanged:: 3.6 + Grouping digits with underscores as in code literals is allowed. + + .. function:: isinstance(object, classinfo) Return true if the *object* argument is an instance of the *classinfo* diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1104,18 +1104,6 @@ thus may not be available in all Python implementations. -.. function:: settscdump(on_flag) - - Activate dumping of VM measurements using the Pentium timestamp counter, if - *on_flag* is true. Deactivate these dumps if *on_flag* is off. The function is - available only if Python was compiled with ``--with-tsc``. To understand - the output of this dump, read :file:`Python/ceval.c` in the Python sources. - - .. impl-detail:: - This function is intimately bound to CPython implementation details and - thus not likely to be implemented elsewhere. - - .. function:: set_coroutine_wrapper(wrapper) Allows intercepting creation of :term:`coroutine` objects (only ones that diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -721,20 +721,24 @@ Integer literals are described by the following lexical definitions: .. productionlist:: - integer: `decimalinteger` | `octinteger` | `hexinteger` | `bininteger` - decimalinteger: `nonzerodigit` `digit`* | "0"+ + integer: `decinteger` | `bininteger` | `octinteger` | `hexinteger` + decinteger: `nonzerodigit` (["_"] `digit`)* | "0"+ (["_"] "0")* + bininteger: "0" ("b" | "B") (["_"] `bindigit`)+ + octinteger: "0" ("o" | "O") (["_"] `octdigit`)+ + hexinteger: "0" ("x" | "X") (["_"] `hexdigit`)+ nonzerodigit: "1"..."9" digit: "0"..."9" - octinteger: "0" ("o" | "O") `octdigit`+ - hexinteger: "0" ("x" | "X") `hexdigit`+ - bininteger: "0" ("b" | "B") `bindigit`+ + bindigit: "0" | "1" octdigit: "0"..."7" hexdigit: `digit` | "a"..."f" | "A"..."F" - bindigit: "0" | "1" There is no limit for the length of integer literals apart from what can be stored in available memory. +Underscores are ignored for determining the numeric value of the literal. They +can be used to group digits for enhanced readability. One underscore can occur +between digits, and after base specifiers like ``0x``. + Note that leading zeros in a non-zero decimal number are not allowed. This is for disambiguation with C-style octal literals, which Python used before version 3.0. @@ -743,6 +747,10 @@ 7 2147483647 0o177 0b100110111 3 79228162514264337593543950336 0o377 0xdeadbeef + 100_000_000_000 0b_1110_0101 + +.. versionchanged:: 3.6 + Underscores are now allowed for grouping purposes in literals. .. _floating: @@ -754,23 +762,28 @@ .. productionlist:: floatnumber: `pointfloat` | `exponentfloat` - pointfloat: [`intpart`] `fraction` | `intpart` "." - exponentfloat: (`intpart` | `pointfloat`) `exponent` - intpart: `digit`+ - fraction: "." `digit`+ - exponent: ("e" | "E") ["+" | "-"] `digit`+ + pointfloat: [`digitpart`] `fraction` | `digitpart` "." + exponentfloat: (`digitpart` | `pointfloat`) `exponent` + digitpart: `digit` (["_"] `digit`)* + fraction: "." `digitpart` + exponent: ("e" | "E") ["+" | "-"] `digitpart` Note that the integer and exponent parts are always interpreted using radix 10. For example, ``077e010`` is legal, and denotes the same number as ``77e10``. The -allowed range of floating point literals is implementation-dependent. Some -examples of floating point literals:: +allowed range of floating point literals is implementation-dependent. As in +integer literals, underscores are supported for digit grouping. - 3.14 10. .001 1e100 3.14e-10 0e0 +Some examples of floating point literals:: + + 3.14 10. .001 1e100 3.14e-10 0e0 3.14_15_93 Note that numeric literals do not include a sign; a phrase like ``-1`` is actually an expression composed of the unary operator ``-`` and the literal ``1``. +.. versionchanged:: 3.6 + Underscores are now allowed for grouping purposes in literals. + .. _imaginary: @@ -780,7 +793,7 @@ Imaginary literals are described by the following lexical definitions: .. productionlist:: - imagnumber: (`floatnumber` | `intpart`) ("j" | "J") + imagnumber: (`floatnumber` | `digitpart`) ("j" | "J") An imaginary literal yields a complex number with a real part of 0.0. Complex numbers are represented as a pair of floating point numbers and have the same @@ -788,7 +801,7 @@ part, add a floating point number to it, e.g., ``(3+4j)``. Some examples of imaginary literals:: - 3.14j 10.j 10j .001j 1e100j 3.14e-10j + 3.14j 10.j 10j .001j 1e100j 3.14e-10j 3.14_15_93j .. _operators: diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -747,10 +747,11 @@ * If the environment variable :envvar:`PYTHONHOME` is set, it is assumed as "Python Home". Otherwise, the path of the main Python executable is used to - locate a "landmark file" (``Lib\os.py``) to deduce the "Python Home". If a - Python home is found, the relevant sub-directories added to :data:`sys.path` - (``Lib``, ``plat-win``, etc) are based on that folder. Otherwise, the core - Python path is constructed from the PythonPath stored in the registry. + locate a "landmark file" (either ``Lib\os.py`` or ``pythonXY.zip``) to deduce + the "Python Home". If a Python home is found, the relevant sub-directories + added to :data:`sys.path` (``Lib``, ``plat-win``, etc) are based on that + folder. Otherwise, the core Python path is constructed from the PythonPath + stored in the registry. * If the Python Home cannot be located, no :envvar:`PYTHONPATH` is specified in the environment, and no registry entries can be found, a default path with @@ -795,7 +796,8 @@ * If you cannot use the previous suggestions (for example, you are a distribution that allows people to run :file:`python.exe` directly), ensure that the landmark file (:file:`Lib\\os.py`) exists in your install directory. - (Note that it will not be detected inside a ZIP file.) + (Note that it will not be detected inside a ZIP file, but a correctly named + ZIP file will be detected instead.) These will ensure that the files in a system-wide installation will not take precedence over the copy of the standard library bundled with your application. @@ -803,10 +805,13 @@ the first suggestion is the best, as the other may still be susceptible to non-standard paths in the registry and user site-packages. -.. versionchanged:: 3.6 +.. versionchanged:: + 3.6 - Adds ``sys.path`` file support and removes ``applocal`` option from - ``pyvenv.cfg``. + * Adds ``sys.path`` file support and removes ``applocal`` option from + ``pyvenv.cfg``. + * Adds ``pythonXX.zip`` as a potential landmark when directly adjacent + to the executable. Additional modules ================== diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -101,6 +101,11 @@ which means that when the 260 character path limit may no longer apply. See :ref:`removing the MAX_PATH limitation ` for details. +* A ``sys.path`` file can be added to force isolated mode and fully specify + all search paths to avoid registry and environment lookup. + +* A ``python36.zip`` file now works as a landmark to infer :envvar:`PYTHONHOME` + .. PEP-sized items next. .. _pep-4XX: @@ -124,6 +129,29 @@ New Features ============ +.. _pep-515: + +PEP 515: Underscores in Numeric Literals +======================================== + +Prior to PEP 515, there was no support for writing long numeric +literals with some form of separator to improve readability. For +instance, how big is ``1000000000000000```? With :pep:`515`, though, +you can use underscores to separate digits as desired to make numeric +literals easier to read: ``1_000_000_000_000_000``. Underscores can be +used with other numeric literals beyond integers, e.g. +``0x_FF_FF_FF_FF``. + +Single underscores are allowed between digits and after any base +specifier. More than a single underscore in a row, leading, or +trailing underscores are not allowed. + +.. seealso:: + + :pep:`523` - Underscores in Numeric Literals + PEP written by Georg Brandl & Serhiy Storchaka. + + .. _pep-523: PEP 523: Adding a frame evaluation API to CPython diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -43,9 +43,6 @@ #ifdef HAVE_DLOPEN int dlopenflags; #endif -#ifdef WITH_TSC - int tscdump; -#endif PyObject *builtins_copy; PyObject *import_func; diff --git a/Include/pystrtod.h b/Include/pystrtod.h --- a/Include/pystrtod.h +++ b/Include/pystrtod.h @@ -19,6 +19,10 @@ int *type); #ifndef Py_LIMITED_API +PyAPI_FUNC(PyObject *) _Py_string_to_number_with_underscores( + const char *str, Py_ssize_t len, const char *what, PyObject *obj, void *arg, + PyObject *(*innerfunc)(const char *, Py_ssize_t, void *)); + PyAPI_FUNC(double) _Py_parse_inf_or_nan(const char *p, char **endptr); #endif diff --git a/Lib/_pydecimal.py b/Lib/_pydecimal.py --- a/Lib/_pydecimal.py +++ b/Lib/_pydecimal.py @@ -589,7 +589,7 @@ # From a string # REs insist on real strings, so we can too. if isinstance(value, str): - m = _parser(value.strip()) + m = _parser(value.strip().replace("_", "")) if m is None: if context is None: context = getcontext() @@ -4125,7 +4125,7 @@ This will make it round up for that operation. """ rounding = self.rounding - self.rounding= type + self.rounding = type return rounding def create_decimal(self, num='0'): @@ -4134,10 +4134,10 @@ This method implements the to-number operation of the IBM Decimal specification.""" - if isinstance(num, str) and num != num.strip(): + if isinstance(num, str) and (num != num.strip() or '_' in num): return self._raise_error(ConversionSyntax, - "no trailing or leading whitespace is " - "permitted.") + "trailing or leading whitespace and " + "underscores are not permitted.") d = Decimal(num, context=self) if d._isnan() and len(d._int) > self.prec - self.clamp: diff --git a/Lib/test/test_complex.py b/Lib/test/test_complex.py --- a/Lib/test/test_complex.py +++ b/Lib/test/test_complex.py @@ -1,5 +1,7 @@ import unittest from test import support +from test.test_grammar import (VALID_UNDERSCORE_LITERALS, + INVALID_UNDERSCORE_LITERALS) from random import random from math import atan2, isnan, copysign @@ -377,6 +379,18 @@ self.assertAlmostEqual(complex(complex1(1j)), 2j) self.assertRaises(TypeError, complex, complex2(1j)) + def test_underscores(self): + # check underscores + for lit in VALID_UNDERSCORE_LITERALS: + if not any(ch in lit for ch in 'xXoObB'): + self.assertEqual(complex(lit), eval(lit)) + self.assertEqual(complex(lit), complex(lit.replace('_', ''))) + for lit in INVALID_UNDERSCORE_LITERALS: + if lit in ('0_7', '09_99'): # octals are not recognized here + continue + if not any(ch in lit for ch in 'xXoObB'): + self.assertRaises(ValueError, complex, lit) + def test_hash(self): for x in range(-30, 30): self.assertEqual(hash(x), hash(complex(x, 0))) diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -554,6 +554,10 @@ self.assertEqual(str(Decimal(' -7.89')), '-7.89') self.assertEqual(str(Decimal(" 3.45679 ")), '3.45679') + # underscores + self.assertEqual(str(Decimal('1_3.3e4_0')), '1.33E+41') + self.assertEqual(str(Decimal('1_0_0_0')), '1000') + # unicode whitespace for lead in ["", ' ', '\u00a0', '\u205f']: for trail in ["", ' ', '\u00a0', '\u205f']: @@ -578,6 +582,9 @@ # embedded NUL self.assertRaises(InvalidOperation, Decimal, "12\u00003") + # underscores don't prevent errors + self.assertRaises(InvalidOperation, Decimal, "1_2_\u00003") + @cpython_only def test_from_legacy_strings(self): import _testcapi @@ -772,6 +779,9 @@ self.assertRaises(InvalidOperation, nc.create_decimal, "xyz") self.assertRaises(ValueError, nc.create_decimal, (1, "xyz", -25)) self.assertRaises(TypeError, nc.create_decimal, "1234", "5678") + # no whitespace and underscore stripping is done with this method + self.assertRaises(InvalidOperation, nc.create_decimal, " 1234") + self.assertRaises(InvalidOperation, nc.create_decimal, "12_34") # too many NaN payload digits nc.prec = 3 diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py --- a/Lib/test/test_float.py +++ b/Lib/test/test_float.py @@ -1,4 +1,3 @@ - import fractions import operator import os @@ -9,6 +8,8 @@ import unittest from test import support +from test.test_grammar import (VALID_UNDERSCORE_LITERALS, + INVALID_UNDERSCORE_LITERALS) from math import isinf, isnan, copysign, ldexp INF = float("inf") @@ -60,6 +61,27 @@ float(b'.' + b'1'*1000) float('.' + '1'*1000) + def test_underscores(self): + for lit in VALID_UNDERSCORE_LITERALS: + if not any(ch in lit for ch in 'jJxXoObB'): + self.assertEqual(float(lit), eval(lit)) + self.assertEqual(float(lit), float(lit.replace('_', ''))) + for lit in INVALID_UNDERSCORE_LITERALS: + if lit in ('0_7', '09_99'): # octals are not recognized here + continue + if not any(ch in lit for ch in 'jJxXoObB'): + self.assertRaises(ValueError, float, lit) + # Additional test cases; nan and inf are never valid as literals, + # only in the float() constructor, but we don't allow underscores + # in or around them. + self.assertRaises(ValueError, float, '_NaN') + self.assertRaises(ValueError, float, 'Na_N') + self.assertRaises(ValueError, float, 'IN_F') + self.assertRaises(ValueError, float, '-_INF') + self.assertRaises(ValueError, float, '-INF_') + # Check that we handle bytes values correctly. + self.assertRaises(ValueError, float, b'0_.\xff9') + def test_non_numeric_input_types(self): # Test possible non-numeric types for the argument x, including # subclasses of the explicitly documented accepted types. diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -16,6 +16,87 @@ from test import ann_module2 import test +# These are shared with test_tokenize and other test modules. +# +# Note: since several test cases filter out floats by looking for "e" and ".", +# don't add hexadecimal literals that contain "e" or "E". +VALID_UNDERSCORE_LITERALS = [ + '0_0_0', + '4_2', + '1_0000_0000', + '0b1001_0100', + '0xffff_ffff', + '0o5_7_7', + '1_00_00.5', + '1_00_00.5e5', + '1_00_00e5_1', + '1e1_0', + '.1_4', + '.1_4e1', + '0b_0', + '0x_f', + '0o_5', + '1_00_00j', + '1_00_00.5j', + '1_00_00e5_1j', + '.1_4j', + '(1_2.5+3_3j)', + '(.5_6j)', +] +INVALID_UNDERSCORE_LITERALS = [ + # Trailing underscores: + '0_', + '42_', + '1.4j_', + '0x_', + '0b1_', + '0xf_', + '0o5_', + '0 if 1_Else 1', + # Underscores in the base selector: + '0_b0', + '0_xf', + '0_o5', + # Old-style octal, still disallowed: + '0_7', + '09_99', + # Multiple consecutive underscores: + '4_______2', + '0.1__4', + '0.1__4j', + '0b1001__0100', + '0xffff__ffff', + '0x___', + '0o5__77', + '1e1__0', + '1e1__0j', + # Underscore right before a dot: + '1_.4', + '1_.4j', + # Underscore right after a dot: + '1._4', + '1._4j', + '._5', + '._5j', + # Underscore right after a sign: + '1.0e+_1', + '1.0e+_1j', + # Underscore right before j: + '1.4_j', + '1.4e5_j', + # Underscore right before e: + '1_e1', + '1.4_e1', + '1.4_e1j', + # Underscore right after e: + '1e_1', + '1.4e_1', + '1.4e_1j', + # Complex cases with parens: + '(1+1.5_j_)', + '(1+1.5_j)', +] + class TokenTests(unittest.TestCase): @@ -95,6 +176,14 @@ self.assertEqual(1 if 0else 0, 0) self.assertRaises(SyntaxError, eval, "0 if 1Else 0") + def test_underscore_literals(self): + for lit in VALID_UNDERSCORE_LITERALS: + self.assertEqual(eval(lit), eval(lit.replace('_', ''))) + for lit in INVALID_UNDERSCORE_LITERALS: + self.assertRaises(SyntaxError, eval, lit) + # Sanity check: no literal begins with an underscore + self.assertRaises(NameError, eval, "_0") + def test_string_literals(self): x = ''; y = ""; self.assertTrue(len(x) == 0 and x == y) x = '\''; y = "'"; self.assertTrue(len(x) == 1 and x == y and ord(x) == 39) diff --git a/Lib/test/test_int.py b/Lib/test/test_int.py --- a/Lib/test/test_int.py +++ b/Lib/test/test_int.py @@ -2,6 +2,8 @@ import unittest from test import support +from test.test_grammar import (VALID_UNDERSCORE_LITERALS, + INVALID_UNDERSCORE_LITERALS) L = [ ('0', 0), @@ -212,6 +214,25 @@ self.assertEqual(int('2br45qc', 35), 4294967297) self.assertEqual(int('1z141z5', 36), 4294967297) + def test_underscores(self): + for lit in VALID_UNDERSCORE_LITERALS: + if any(ch in lit for ch in '.eEjJ'): + continue + self.assertEqual(int(lit, 0), eval(lit)) + self.assertEqual(int(lit, 0), int(lit.replace('_', ''), 0)) + for lit in INVALID_UNDERSCORE_LITERALS: + if any(ch in lit for ch in '.eEjJ'): + continue + self.assertRaises(ValueError, int, lit, 0) + # Additional test cases with bases != 0, only for the constructor: + self.assertEqual(int("1_00", 3), 9) + self.assertEqual(int("0_100"), 100) # not valid as a literal! + self.assertEqual(int(b"1_00"), 100) # byte underscore + self.assertRaises(ValueError, int, "_100") + self.assertRaises(ValueError, int, "+_100") + self.assertRaises(ValueError, int, "1__00") + self.assertRaises(ValueError, int, "100_") + @support.cpython_only def test_small_ints(self): # Bug #3236: Return small longs from PyLong_FromString diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py --- a/Lib/test/test_tokenize.py +++ b/Lib/test/test_tokenize.py @@ -3,7 +3,9 @@ STRING, ENDMARKER, ENCODING, tok_name, detect_encoding, open as tokenize_open, Untokenizer) from io import BytesIO -from unittest import TestCase, mock, main +from unittest import TestCase, mock +from test.test_grammar import (VALID_UNDERSCORE_LITERALS, + INVALID_UNDERSCORE_LITERALS) import os import token @@ -185,6 +187,21 @@ NUMBER '3.14e159' (1, 4) (1, 12) """) + def test_underscore_literals(self): + def number_token(s): + f = BytesIO(s.encode('utf-8')) + for toktype, token, start, end, line in tokenize(f.readline): + if toktype == NUMBER: + return token + return 'invalid token' + for lit in VALID_UNDERSCORE_LITERALS: + if '(' in lit: + # this won't work with compound complex inputs + continue + self.assertEqual(number_token(lit), lit) + for lit in INVALID_UNDERSCORE_LITERALS: + self.assertNotEqual(number_token(lit), lit) + def test_string(self): # String literals self.check_tokenize("x = ''; y = \"\"", """\ @@ -1529,11 +1546,10 @@ tempdir = os.path.dirname(fn) or os.curdir testfiles = glob.glob(os.path.join(tempdir, "test*.py")) - # Tokenize is broken on test_unicode_identifiers.py because regular - # expressions are broken on the obscure unicode identifiers in it. - # *sigh* With roundtrip extended to test the 5-tuple mode of - # untokenize, 7 more testfiles fail. Remove them also until the - # failure is diagnosed. + # Tokenize is broken on test_pep3131.py because regular expressions are + # broken on the obscure unicode identifiers in it. *sigh* + # With roundtrip extended to test the 5-tuple mode of untokenize, + # 7 more testfiles fail. Remove them also until the failure is diagnosed. testfiles.remove(os.path.join(tempdir, "test_unicode_identifiers.py")) for f in ('buffer', 'builtin', 'fileio', 'inspect', 'os', 'platform', 'sys'): @@ -1565,4 +1581,4 @@ if __name__ == "__main__": - main() + unittest.main() diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -48,6 +48,7 @@ def test_float_constructor(self): self.assertRaises(ValueError, float, '') self.assertRaises(ValueError, float, '5\0') + self.assertRaises(ValueError, float, '5_5\0') def test_zero_division(self): try: 5.0 / 0.0 diff --git a/Lib/tokenize.py b/Lib/tokenize.py --- a/Lib/tokenize.py +++ b/Lib/tokenize.py @@ -120,16 +120,17 @@ Ignore = Whitespace + any(r'\\\r?\n' + Whitespace) + maybe(Comment) Name = r'\w+' -Hexnumber = r'0[xX][0-9a-fA-F]+' -Binnumber = r'0[bB][01]+' -Octnumber = r'0[oO][0-7]+' -Decnumber = r'(?:0+|[1-9][0-9]*)' +Hexnumber = r'0[xX](?:_?[0-9a-fA-F])+' +Binnumber = r'0[bB](?:_?[01])+' +Octnumber = r'0[oO](?:_?[0-7])+' +Decnumber = r'(?:0(?:_?0)*|[1-9](?:_?[0-9])*)' Intnumber = group(Hexnumber, Binnumber, Octnumber, Decnumber) -Exponent = r'[eE][-+]?[0-9]+' -Pointfloat = group(r'[0-9]+\.[0-9]*', r'\.[0-9]+') + maybe(Exponent) -Expfloat = r'[0-9]+' + Exponent +Exponent = r'[eE][-+]?[0-9](?:_?[0-9])*' +Pointfloat = group(r'[0-9](?:_?[0-9])*\.(?:[0-9](?:_?[0-9])*)?', + r'\.[0-9](?:_?[0-9])*') + maybe(Exponent) +Expfloat = r'[0-9](?:_?[0-9])*' + Exponent Floatnumber = group(Pointfloat, Expfloat) -Imagnumber = group(r'[0-9]+[jJ]', Floatnumber + r'[jJ]') +Imagnumber = group(r'[0-9](?:_?[0-9])*[jJ]', Floatnumber + r'[jJ]') Number = group(Imagnumber, Floatnumber, Intnumber) # Return the empty string, plus all of the valid string prefixes. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -17,6 +17,8 @@ efficient bytecode. Patch by Demur Rumed, design by Serhiy Storchaka, reviewed by Serhiy Storchaka and Victor Stinner. +- Issue #26331: Implement tokenizing support for PEP 515. Patch by Georg Brandl. + - Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi. @@ -2680,7 +2682,7 @@ - Issue #24774: Fix docstring in http.server.test. Patch from Chiu-Hsiang Hsu. - Issue #21159: Improve message in configparser.InterpolationMissingOptionError. - Patch from ??ukasz Langa. + Patch from ????ukasz Langa. - Issue #20362: Honour TestCase.longMessage correctly in assertRegex. Patch from Ilia Kurenkov. @@ -4608,7 +4610,7 @@ Based on patch by Martin Panter. - Issue #17293: uuid.getnode() now determines MAC address on AIX using netstat. - Based on patch by Aivars Kalv??ns. + Based on patch by Aivars Kalv????ns. - Issue #22769: Fixed ttk.Treeview.tag_has() when called without arguments. diff --git a/Misc/SpecialBuilds.txt b/Misc/SpecialBuilds.txt --- a/Misc/SpecialBuilds.txt +++ b/Misc/SpecialBuilds.txt @@ -234,29 +234,3 @@ number of function calls made. It keeps detailed statistics about what kind of object was called and whether the call hit any of the special fast paths in the code. - - -WITH_TSC --------- - -Super-lowlevel profiling of the interpreter. When enabled, the sys module grows -a new function: - -settscdump(bool) - If true, tell the Python interpreter to dump VM measurements to stderr. If - false, turn off dump. The measurements are based on the processor's - time-stamp counter. - -This build option requires a small amount of platform specific code. Currently -this code is present for linux/x86 and any PowerPC platform that uses GCC -(i.e. OS X and linux/ppc). - -On the PowerPC the rate at which the time base register is incremented is not -defined by the architecture specification, so you'll need to find the manual for -your specific processor. For the 750CX, 750CXe and 750FX (all sold as the G3) -we find: - - The time base counter is clocked at a frequency that is one-fourth that of - the bus clock. - -This build is enabled by the --with-tsc flag to configure. diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c --- a/Modules/_decimal/_decimal.c +++ b/Modules/_decimal/_decimal.c @@ -1889,12 +1889,13 @@ /* Return the ASCII representation of a numeric Unicode string. The numeric string may contain ascii characters in the range [1, 127], any Unicode space and any unicode digit. If strip_ws is true, leading and trailing - whitespace is stripped. + whitespace is stripped. If ignore_underscores is true, underscores are + ignored. Return NULL if malloc fails and an empty string if invalid characters are found. */ static char * -numeric_as_ascii(const PyObject *u, int strip_ws) +numeric_as_ascii(const PyObject *u, int strip_ws, int ignore_underscores) { enum PyUnicode_Kind kind; void *data; @@ -1929,6 +1930,9 @@ for (; j < len; j++) { ch = PyUnicode_READ(kind, data, j); + if (ignore_underscores && ch == '_') { + continue; + } if (0 < ch && ch <= 127) { *cp++ = ch; continue; @@ -2011,7 +2015,7 @@ PyObject *dec; char *s; - s = numeric_as_ascii(u, 0); + s = numeric_as_ascii(u, 0, 0); if (s == NULL) { return NULL; } @@ -2031,7 +2035,7 @@ PyObject *dec; char *s; - s = numeric_as_ascii(u, 1); + s = numeric_as_ascii(u, 1, 1); if (s == NULL) { return NULL; } diff --git a/Objects/complexobject.c b/Objects/complexobject.c --- a/Objects/complexobject.c +++ b/Objects/complexobject.c @@ -759,29 +759,12 @@ }; static PyObject * -complex_subtype_from_string(PyTypeObject *type, PyObject *v) +complex_from_string_inner(const char *s, Py_ssize_t len, void *type) { - const char *s, *start; - char *end; double x=0.0, y=0.0, z; int got_bracket=0; - PyObject *s_buffer = NULL; - Py_ssize_t len; - - if (PyUnicode_Check(v)) { - s_buffer = _PyUnicode_TransformDecimalAndSpaceToASCII(v); - if (s_buffer == NULL) - return NULL; - s = PyUnicode_AsUTF8AndSize(s_buffer, &len); - if (s == NULL) - goto error; - } - else { - PyErr_Format(PyExc_TypeError, - "complex() argument must be a string or a number, not '%.200s'", - Py_TYPE(v)->tp_name); - return NULL; - } + const char *start; + char *end; /* position on first nonblank */ start = s; @@ -822,7 +805,7 @@ if (PyErr_ExceptionMatches(PyExc_ValueError)) PyErr_Clear(); else - goto error; + return NULL; } if (end != s) { /* all 4 forms starting with land here */ @@ -835,7 +818,7 @@ if (PyErr_ExceptionMatches(PyExc_ValueError)) PyErr_Clear(); else - goto error; + return NULL; } if (end != s) /* j */ @@ -890,18 +873,46 @@ if (s-start != len) goto parse_error; - Py_XDECREF(s_buffer); - return complex_subtype_from_doubles(type, x, y); + return complex_subtype_from_doubles((PyTypeObject *)type, x, y); parse_error: PyErr_SetString(PyExc_ValueError, "complex() arg is a malformed string"); - error: - Py_XDECREF(s_buffer); return NULL; } static PyObject * +complex_subtype_from_string(PyTypeObject *type, PyObject *v) +{ + const char *s; + PyObject *s_buffer = NULL, *result = NULL; + Py_ssize_t len; + + if (PyUnicode_Check(v)) { + s_buffer = _PyUnicode_TransformDecimalAndSpaceToASCII(v); + if (s_buffer == NULL) { + return NULL; + } + s = PyUnicode_AsUTF8AndSize(s_buffer, &len); + if (s == NULL) { + goto exit; + } + } + else { + PyErr_Format(PyExc_TypeError, + "complex() argument must be a string or a number, not '%.200s'", + Py_TYPE(v)->tp_name); + return NULL; + } + + result = _Py_string_to_number_with_underscores(s, len, "complex", v, type, + complex_from_string_inner); + exit: + Py_DECREF(s_buffer); + return result; +} + +static PyObject * complex_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { PyObject *r, *i, *tmp; diff --git a/Objects/floatobject.c b/Objects/floatobject.c --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -124,11 +124,43 @@ return (PyObject *) op; } +static PyObject * +float_from_string_inner(const char *s, Py_ssize_t len, void *obj) +{ + double x; + const char *end; + const char *last = s + len; + /* strip space */ + while (s < last && Py_ISSPACE(*s)) { + s++; + } + + while (s < last - 1 && Py_ISSPACE(last[-1])) { + last--; + } + + /* We don't care about overflow or underflow. If the platform + * supports them, infinities and signed zeroes (on underflow) are + * fine. */ + x = PyOS_string_to_double(s, (char **)&end, NULL); + if (end != last) { + PyErr_Format(PyExc_ValueError, + "could not convert string to float: " + "%R", obj); + return NULL; + } + else if (x == -1.0 && PyErr_Occurred()) { + return NULL; + } + else { + return PyFloat_FromDouble(x); + } +} + PyObject * PyFloat_FromString(PyObject *v) { - const char *s, *last, *end; - double x; + const char *s; PyObject *s_buffer = NULL; Py_ssize_t len; Py_buffer view = {NULL, NULL}; @@ -169,27 +201,8 @@ Py_TYPE(v)->tp_name); return NULL; } - last = s + len; - /* strip space */ - while (s < last && Py_ISSPACE(*s)) - s++; - while (s < last - 1 && Py_ISSPACE(last[-1])) - last--; - /* We don't care about overflow or underflow. If the platform - * supports them, infinities and signed zeroes (on underflow) are - * fine. */ - x = PyOS_string_to_double(s, (char **)&end, NULL); - if (end != last) { - PyErr_Format(PyExc_ValueError, - "could not convert string to float: " - "%R", v); - result = NULL; - } - else if (x == -1.0 && PyErr_Occurred()) - result = NULL; - else - result = PyFloat_FromDouble(x); - + result = _Py_string_to_number_with_underscores(s, len, "float", v, v, + float_from_string_inner); PyBuffer_Release(&view); Py_XDECREF(s_buffer); return result; diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -2004,12 +2004,18 @@ * non-digit (which may be *str!). A normalized int is returned. * The point to this routine is that it takes time linear in the number of * string characters. + * + * Return values: + * -1 on syntax error (exception needs to be set, *res is untouched) + * 0 else (exception may be set, in that case *res is set to NULL) */ -static PyLongObject * -long_from_binary_base(const char **str, int base) +static int +long_from_binary_base(const char **str, int base, PyLongObject **res) { const char *p = *str; const char *start = p; + char prev = 0; + int digits = 0; int bits_per_char; Py_ssize_t n; PyLongObject *z; @@ -2019,23 +2025,43 @@ assert(base >= 2 && base <= 32 && (base & (base - 1)) == 0); n = base; - for (bits_per_char = -1; n; ++bits_per_char) + for (bits_per_char = -1; n; ++bits_per_char) { n >>= 1; - /* n <- total # of bits needed, while setting p to end-of-string */ - while (_PyLong_DigitValue[Py_CHARMASK(*p)] < base) + } + /* count digits and set p to end-of-string */ + while (_PyLong_DigitValue[Py_CHARMASK(*p)] < base || *p == '_') { + if (*p == '_') { + if (prev == '_') { + *str = p - 1; + return -1; + } + } else { + ++digits; + } + prev = *p; ++p; + } + if (prev == '_') { + /* Trailing underscore not allowed. */ + *str = p - 1; + return -1; + } + *str = p; /* n <- # of Python digits needed, = ceiling(n/PyLong_SHIFT). */ - n = (p - start) * bits_per_char + PyLong_SHIFT - 1; + n = digits * bits_per_char + PyLong_SHIFT - 1; if (n / bits_per_char < p - start) { PyErr_SetString(PyExc_ValueError, "int string too large to convert"); - return NULL; + *res = NULL; + return 0; } n = n / PyLong_SHIFT; z = _PyLong_New(n); - if (z == NULL) - return NULL; + if (z == NULL) { + *res = NULL; + return 0; + } /* Read string from right, and fill in int from left; i.e., * from least to most significant in both. */ @@ -2043,7 +2069,11 @@ bits_in_accum = 0; pdigit = z->ob_digit; while (--p >= start) { - int k = (int)_PyLong_DigitValue[Py_CHARMASK(*p)]; + int k; + if (*p == '_') { + continue; + } + k = (int)_PyLong_DigitValue[Py_CHARMASK(*p)]; assert(k >= 0 && k < base); accum |= (twodigits)k << bits_in_accum; bits_in_accum += bits_per_char; @@ -2062,7 +2092,8 @@ } while (pdigit - z->ob_digit < n) *pdigit++ = 0; - return long_normalize(z); + *res = long_normalize(z); + return 0; } /* Parses an int from a bytestring. Leading and trailing whitespace will be @@ -2087,23 +2118,29 @@ "int() arg 2 must be >= 2 and <= 36"); return NULL; } - while (*str != '\0' && Py_ISSPACE(Py_CHARMASK(*str))) + while (*str != '\0' && Py_ISSPACE(Py_CHARMASK(*str))) { str++; - if (*str == '+') + } + if (*str == '+') { ++str; + } else if (*str == '-') { ++str; sign = -1; } if (base == 0) { - if (str[0] != '0') + if (str[0] != '0') { base = 10; - else if (str[1] == 'x' || str[1] == 'X') + } + else if (str[1] == 'x' || str[1] == 'X') { base = 16; - else if (str[1] == 'o' || str[1] == 'O') + } + else if (str[1] == 'o' || str[1] == 'O') { base = 8; - else if (str[1] == 'b' || str[1] == 'B') + } + else if (str[1] == 'b' || str[1] == 'B') { base = 2; + } else { /* "old" (C-style) octal literal, now invalid. it might still be zero though */ @@ -2114,12 +2151,26 @@ if (str[0] == '0' && ((base == 16 && (str[1] == 'x' || str[1] == 'X')) || (base == 8 && (str[1] == 'o' || str[1] == 'O')) || - (base == 2 && (str[1] == 'b' || str[1] == 'B')))) + (base == 2 && (str[1] == 'b' || str[1] == 'B')))) { str += 2; + /* One underscore allowed here. */ + if (*str == '_') { + ++str; + } + } + if (str[0] == '_') { + /* May not start with underscores. */ + goto onError; + } start = str; - if ((base & (base - 1)) == 0) - z = long_from_binary_base(&str, base); + if ((base & (base - 1)) == 0) { + int res = long_from_binary_base(&str, base, &z); + if (res < 0) { + /* Syntax error. */ + goto onError; + } + } else { /*** Binary bases can be converted in time linear in the number of digits, because @@ -2208,11 +2259,13 @@ ***/ twodigits c; /* current input character */ Py_ssize_t size_z; + int digits = 0; int i; int convwidth; twodigits convmultmax, convmult; digit *pz, *pzstop; - const char* scan; + const char *scan, *lastdigit; + char prev = 0; static double log_base_BASE[37] = {0.0e0,}; static int convwidth_base[37] = {0,}; @@ -2226,8 +2279,9 @@ log((double)PyLong_BASE)); for (;;) { twodigits next = convmax * base; - if (next > PyLong_BASE) + if (next > PyLong_BASE) { break; + } convmax = next; ++i; } @@ -2238,21 +2292,43 @@ /* Find length of the string of numeric characters. */ scan = str; - while (_PyLong_DigitValue[Py_CHARMASK(*scan)] < base) + lastdigit = str; + + while (_PyLong_DigitValue[Py_CHARMASK(*scan)] < base || *scan == '_') { + if (*scan == '_') { + if (prev == '_') { + /* Only one underscore allowed. */ + str = lastdigit + 1; + goto onError; + } + } + else { + ++digits; + lastdigit = scan; + } + prev = *scan; ++scan; + } + if (prev == '_') { + /* Trailing underscore not allowed. */ + /* Set error pointer to first underscore. */ + str = lastdigit + 1; + goto onError; + } /* Create an int object that can contain the largest possible * integer with this base and length. Note that there's no * need to initialize z->ob_digit -- no slot is read up before * being stored into. */ - size_z = (Py_ssize_t)((scan - str) * log_base_BASE[base]) + 1; + size_z = (Py_ssize_t)(digits * log_base_BASE[base]) + 1; /* Uncomment next line to test exceedingly rare copy code */ /* size_z = 1; */ assert(size_z > 0); z = _PyLong_New(size_z); - if (z == NULL) + if (z == NULL) { return NULL; + } Py_SIZE(z) = 0; /* `convwidth` consecutive input digits are treated as a single @@ -2263,9 +2339,17 @@ /* Work ;-) */ while (str < scan) { + if (*str == '_') { + str++; + continue; + } /* grab up to convwidth digits from the input string */ c = (digit)_PyLong_DigitValue[Py_CHARMASK(*str++)]; - for (i = 1; i < convwidth && str != scan; ++i, ++str) { + for (i = 1; i < convwidth && str != scan; ++str) { + if (*str == '_') { + continue; + } + i++; c = (twodigits)(c * base + (int)_PyLong_DigitValue[Py_CHARMASK(*str)]); assert(c < PyLong_BASE); @@ -2277,8 +2361,9 @@ */ if (i != convwidth) { convmult = base; - for ( ; i > 1; --i) + for ( ; i > 1; --i) { convmult *= base; + } } /* Multiply z by convmult, and add c. */ @@ -2316,41 +2401,51 @@ } } } - if (z == NULL) + if (z == NULL) { return NULL; + } if (error_if_nonzero) { /* reset the base to 0, else the exception message doesn't make too much sense */ base = 0; - if (Py_SIZE(z) != 0) + if (Py_SIZE(z) != 0) { goto onError; + } /* there might still be other problems, therefore base remains zero here for the same reason */ } - if (str == start) + if (str == start) { goto onError; - if (sign < 0) + } + if (sign < 0) { Py_SIZE(z) = -(Py_SIZE(z)); - while (*str && Py_ISSPACE(Py_CHARMASK(*str))) + } + while (*str && Py_ISSPACE(Py_CHARMASK(*str))) { str++; - if (*str != '\0') + } + if (*str != '\0') { goto onError; + } long_normalize(z); z = maybe_small_long(z); - if (z == NULL) + if (z == NULL) { return NULL; - if (pend != NULL) + } + if (pend != NULL) { *pend = (char *)str; + } return (PyObject *) z; onError: - if (pend != NULL) + if (pend != NULL) { *pend = (char *)str; + } Py_XDECREF(z); slen = strlen(orig_str) < 200 ? strlen(orig_str) : 200; strobj = PyUnicode_FromStringAndSize(orig_str, slen); - if (strobj == NULL) + if (strobj == NULL) { return NULL; + } PyErr_Format(PyExc_ValueError, "invalid literal for int() with base %d: %.200R", base, strobj); diff --git a/PCbuild/openssl.props b/PCbuild/openssl.props --- a/PCbuild/openssl.props +++ b/PCbuild/openssl.props @@ -19,6 +19,7 @@ + @@ -38,7 +39,6 @@ - diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1333,6 +1333,28 @@ } #endif +static int +tok_decimal_tail(struct tok_state *tok) +{ + int c; + + while (1) { + do { + c = tok_nextc(tok); + } while (isdigit(c)); + if (c != '_') { + break; + } + c = tok_nextc(tok); + if (!isdigit(c)) { + tok->done = E_TOKEN; + tok_backup(tok, c); + return 0; + } + } + return c; +} + /* Get next token, after space stripping etc. */ static int @@ -1353,17 +1375,20 @@ tok->atbol = 0; for (;;) { c = tok_nextc(tok); - if (c == ' ') + if (c == ' ') { col++, altcol++; + } else if (c == '\t') { col = (col/tok->tabsize + 1) * tok->tabsize; altcol = (altcol/tok->alttabsize + 1) * tok->alttabsize; } - else if (c == '\014') /* Control-L (formfeed) */ + else if (c == '\014') {/* Control-L (formfeed) */ col = altcol = 0; /* For Emacs users */ - else + } + else { break; + } } tok_backup(tok, c); if (c == '#' || c == '\n') { @@ -1372,10 +1397,12 @@ not passed to the parser as NEWLINE tokens, except *totally* empty lines in interactive mode, which signal the end of a command group. */ - if (col == 0 && c == '\n' && tok->prompt != NULL) + if (col == 0 && c == '\n' && tok->prompt != NULL) { blankline = 0; /* Let it through */ - else + } + else { blankline = 1; /* Ignore completely */ + } /* We can't jump back right here since we still may need to skip to the end of a comment */ } @@ -1383,8 +1410,9 @@ if (col == tok->indstack[tok->indent]) { /* No change */ if (altcol != tok->altindstack[tok->indent]) { - if (indenterror(tok)) + if (indenterror(tok)) { return ERRORTOKEN; + } } } else if (col > tok->indstack[tok->indent]) { @@ -1395,8 +1423,9 @@ return ERRORTOKEN; } if (altcol <= tok->altindstack[tok->indent]) { - if (indenterror(tok)) + if (indenterror(tok)) { return ERRORTOKEN; + } } tok->pendin++; tok->indstack[++tok->indent] = col; @@ -1415,8 +1444,9 @@ return ERRORTOKEN; } if (altcol != tok->altindstack[tok->indent]) { - if (indenterror(tok)) + if (indenterror(tok)) { return ERRORTOKEN; + } } } } @@ -1462,9 +1492,11 @@ tok->start = tok->cur - 1; /* Skip comment */ - if (c == '#') - while (c != EOF && c != '\n') + if (c == '#') { + while (c != EOF && c != '\n') { c = tok_nextc(tok); + } + } /* Check for EOF and errors now */ if (c == EOF) { @@ -1481,27 +1513,35 @@ saw_b = 1; /* Since this is a backwards compatibility support literal we don't want to support it in arbitrary order like byte literals. */ - else if (!(saw_b || saw_u || saw_r || saw_f) && (c == 'u' || c == 'U')) + else if (!(saw_b || saw_u || saw_r || saw_f) + && (c == 'u'|| c == 'U')) { saw_u = 1; + } /* ur"" and ru"" are not supported */ - else if (!(saw_r || saw_u) && (c == 'r' || c == 'R')) + else if (!(saw_r || saw_u) && (c == 'r' || c == 'R')) { saw_r = 1; - else if (!(saw_f || saw_b || saw_u) && (c == 'f' || c == 'F')) + } + else if (!(saw_f || saw_b || saw_u) && (c == 'f' || c == 'F')) { saw_f = 1; - else + } + else { break; + } c = tok_nextc(tok); - if (c == '"' || c == '\'') + if (c == '"' || c == '\'') { goto letter_quote; + } } while (is_potential_identifier_char(c)) { - if (c >= 128) + if (c >= 128) { nonascii = 1; + } c = tok_nextc(tok); } tok_backup(tok, c); - if (nonascii && !verify_identifier(tok)) + if (nonascii && !verify_identifier(tok)) { return ERRORTOKEN; + } *p_start = tok->start; *p_end = tok->cur; @@ -1510,10 +1550,12 @@ /* Current token length is 5. */ if (tok->async_def) { /* We're inside an 'async def' function. */ - if (memcmp(tok->start, "async", 5) == 0) + if (memcmp(tok->start, "async", 5) == 0) { return ASYNC; - if (memcmp(tok->start, "await", 5) == 0) + } + if (memcmp(tok->start, "await", 5) == 0) { return AWAIT; + } } else if (memcmp(tok->start, "async", 5) == 0) { /* The current token is 'async'. @@ -1546,8 +1588,9 @@ /* Newline */ if (c == '\n') { tok->atbol = 1; - if (blankline || tok->level > 0) + if (blankline || tok->level > 0) { goto nextline; + } *p_start = tok->start; *p_end = tok->cur - 1; /* Leave '\n' out of the string */ tok->cont_line = 0; @@ -1570,11 +1613,13 @@ *p_start = tok->start; *p_end = tok->cur; return ELLIPSIS; - } else { + } + else { tok_backup(tok, c); } tok_backup(tok, '.'); - } else { + } + else { tok_backup(tok, c); } *p_start = tok->start; @@ -1588,59 +1633,93 @@ /* Hex, octal or binary -- maybe. */ c = tok_nextc(tok); if (c == 'x' || c == 'X') { - /* Hex */ c = tok_nextc(tok); - if (!isxdigit(c)) { - tok->done = E_TOKEN; - tok_backup(tok, c); - return ERRORTOKEN; - } do { - c = tok_nextc(tok); - } while (isxdigit(c)); + if (c == '_') { + c = tok_nextc(tok); + } + if (!isxdigit(c)) { + tok->done = E_TOKEN; + tok_backup(tok, c); + return ERRORTOKEN; + } + do { + c = tok_nextc(tok); + } while (isxdigit(c)); + } while (c == '_'); } else if (c == 'o' || c == 'O') { /* Octal */ c = tok_nextc(tok); - if (c < '0' || c >= '8') { - tok->done = E_TOKEN; - tok_backup(tok, c); - return ERRORTOKEN; - } do { - c = tok_nextc(tok); - } while ('0' <= c && c < '8'); + if (c == '_') { + c = tok_nextc(tok); + } + if (c < '0' || c >= '8') { + tok->done = E_TOKEN; + tok_backup(tok, c); + return ERRORTOKEN; + } + do { + c = tok_nextc(tok); + } while ('0' <= c && c < '8'); + } while (c == '_'); } else if (c == 'b' || c == 'B') { /* Binary */ c = tok_nextc(tok); - if (c != '0' && c != '1') { - tok->done = E_TOKEN; - tok_backup(tok, c); - return ERRORTOKEN; - } do { - c = tok_nextc(tok); - } while (c == '0' || c == '1'); + if (c == '_') { + c = tok_nextc(tok); + } + if (c != '0' && c != '1') { + tok->done = E_TOKEN; + tok_backup(tok, c); + return ERRORTOKEN; + } + do { + c = tok_nextc(tok); + } while (c == '0' || c == '1'); + } while (c == '_'); } else { int nonzero = 0; /* maybe old-style octal; c is first char of it */ /* in any case, allow '0' as a literal */ - while (c == '0') - c = tok_nextc(tok); - while (isdigit(c)) { - nonzero = 1; + while (1) { + if (c == '_') { + c = tok_nextc(tok); + if (!isdigit(c)) { + tok->done = E_TOKEN; + tok_backup(tok, c); + return ERRORTOKEN; + } + } + if (c != '0') { + break; + } c = tok_nextc(tok); } - if (c == '.') + if (isdigit(c)) { + nonzero = 1; + c = tok_decimal_tail(tok); + if (c == 0) { + return ERRORTOKEN; + } + } + if (c == '.') { + c = tok_nextc(tok); goto fraction; - else if (c == 'e' || c == 'E') + } + else if (c == 'e' || c == 'E') { goto exponent; - else if (c == 'j' || c == 'J') + } + else if (c == 'j' || c == 'J') { goto imaginary; + } else if (nonzero) { + /* Old-style octal: now disallowed. */ tok->done = E_TOKEN; tok_backup(tok, c); return ERRORTOKEN; @@ -1649,17 +1728,22 @@ } else { /* Decimal */ - do { - c = tok_nextc(tok); - } while (isdigit(c)); + c = tok_decimal_tail(tok); + if (c == 0) { + return ERRORTOKEN; + } { /* Accept floating point numbers. */ if (c == '.') { + c = tok_nextc(tok); fraction: /* Fraction */ - do { - c = tok_nextc(tok); - } while (isdigit(c)); + if (isdigit(c)) { + c = tok_decimal_tail(tok); + if (c == 0) { + return ERRORTOKEN; + } + } } if (c == 'e' || c == 'E') { int e; @@ -1681,14 +1765,16 @@ *p_end = tok->cur; return NUMBER; } - do { - c = tok_nextc(tok); - } while (isdigit(c)); + c = tok_decimal_tail(tok); + if (c == 0) { + return ERRORTOKEN; + } } - if (c == 'j' || c == 'J') + if (c == 'j' || c == 'J') { /* Imaginary part */ imaginary: c = tok_nextc(tok); + } } } tok_backup(tok, c); @@ -1708,22 +1794,27 @@ c = tok_nextc(tok); if (c == quote) { c = tok_nextc(tok); - if (c == quote) + if (c == quote) { quote_size = 3; - else + } + else { end_quote_size = 1; /* empty string found */ + } } - if (c != quote) + if (c != quote) { tok_backup(tok, c); + } /* Get rest of string */ while (end_quote_size != quote_size) { c = tok_nextc(tok); if (c == EOF) { - if (quote_size == 3) + if (quote_size == 3) { tok->done = E_EOFS; - else + } + else { tok->done = E_EOLS; + } tok->cur = tok->inp; return ERRORTOKEN; } @@ -1732,12 +1823,14 @@ tok->cur = tok->inp; return ERRORTOKEN; } - if (c == quote) + if (c == quote) { end_quote_size += 1; + } else { end_quote_size = 0; - if (c == '\\') + if (c == '\\') { tok_nextc(tok); /* skip escaped char */ + } } } @@ -1767,7 +1860,8 @@ int token3 = PyToken_ThreeChars(c, c2, c3); if (token3 != OP) { token = token3; - } else { + } + else { tok_backup(tok, c3); } *p_start = tok->start; diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -4018,7 +4018,7 @@ } static PyObject * -parsenumber(struct compiling *c, const char *s) +parsenumber_raw(struct compiling *c, const char *s) { const char *end; long x; @@ -4061,6 +4061,31 @@ } static PyObject * +parsenumber(struct compiling *c, const char *s) +{ + char *dup, *end; + PyObject *res = NULL; + + assert(s != NULL); + + if (strchr(s, '_') == NULL) { + return parsenumber_raw(c, s); + } + /* Create a duplicate without underscores. */ + dup = PyMem_Malloc(strlen(s) + 1); + end = dup; + for (; *s; s++) { + if (*s != '_') { + *end++ = *s; + } + } + *end = '\0'; + res = parsenumber_raw(c, dup); + PyMem_Free(dup); + return res; +} + +static PyObject * decode_utf8(struct compiling *c, const char **sPtr, const char *end) { const char *s, *t; diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -20,82 +20,6 @@ #include -#ifndef WITH_TSC - -#define READ_TIMESTAMP(var) - -#else - -typedef unsigned long long uint64; - -/* PowerPC support. - "__ppc__" appears to be the preprocessor definition to detect on OS X, whereas - "__powerpc__" appears to be the correct one for Linux with GCC -*/ -#if defined(__ppc__) || defined (__powerpc__) - -#define READ_TIMESTAMP(var) ppc_getcounter(&var) - -static void -ppc_getcounter(uint64 *v) -{ - unsigned long tbu, tb, tbu2; - - loop: - asm volatile ("mftbu %0" : "=r" (tbu) ); - asm volatile ("mftb %0" : "=r" (tb) ); - asm volatile ("mftbu %0" : "=r" (tbu2)); - if (__builtin_expect(tbu != tbu2, 0)) goto loop; - - /* The slightly peculiar way of writing the next lines is - compiled better by GCC than any other way I tried. */ - ((long*)(v))[0] = tbu; - ((long*)(v))[1] = tb; -} - -#elif defined(__i386__) - -/* this is for linux/x86 (and probably any other GCC/x86 combo) */ - -#define READ_TIMESTAMP(val) \ - __asm__ __volatile__("rdtsc" : "=A" (val)) - -#elif defined(__x86_64__) - -/* for gcc/x86_64, the "A" constraint in DI mode means *either* rax *or* rdx; - not edx:eax as it does for i386. Since rdtsc puts its result in edx:eax - even in 64-bit mode, we need to use "a" and "d" for the lower and upper - 32-bit pieces of the result. */ - -#define READ_TIMESTAMP(val) do { \ - unsigned int h, l; \ - __asm__ __volatile__("rdtsc" : "=a" (l), "=d" (h)); \ - (val) = ((uint64)l) | (((uint64)h) << 32); \ - } while(0) - - -#else - -#error "Don't know how to implement timestamp counter for this architecture" - -#endif - -void dump_tsc(int opcode, int ticked, uint64 inst0, uint64 inst1, - uint64 loop0, uint64 loop1, uint64 intr0, uint64 intr1) -{ - uint64 intr, inst, loop; - PyThreadState *tstate = PyThreadState_Get(); - if (!tstate->interp->tscdump) - return; - intr = intr1 - intr0; - inst = inst1 - inst0 - intr; - loop = loop1 - loop0 - intr; - fprintf(stderr, "opcode=%03d t=%d inst=%06lld loop=%06lld\n", - opcode, ticked, inst, loop); -} - -#endif - /* Turn this on if your compiler chokes on the big switch: */ /* #define CASE_TOO_BIG 1 */ @@ -108,11 +32,7 @@ typedef PyObject *(*callproc)(PyObject *, PyObject *, PyObject *); /* Forward declarations */ -#ifdef WITH_TSC -static PyObject * call_function(PyObject ***, Py_ssize_t, PyObject *, uint64*, uint64*); -#else static PyObject * call_function(PyObject ***, Py_ssize_t, PyObject *); -#endif static PyObject * fast_function(PyObject *, PyObject **, Py_ssize_t, PyObject *); static PyObject * do_call_core(PyObject *, PyObject *, PyObject *); @@ -938,46 +858,6 @@ #define GETITEM(v, i) PyTuple_GetItem((v), (i)) #endif -#ifdef WITH_TSC -/* Use Pentium timestamp counter to mark certain events: - inst0 -- beginning of switch statement for opcode dispatch - inst1 -- end of switch statement (may be skipped) - loop0 -- the top of the mainloop - loop1 -- place where control returns again to top of mainloop - (may be skipped) - intr1 -- beginning of long interruption - intr2 -- end of long interruption - - Many opcodes call out to helper C functions. In some cases, the - time in those functions should be counted towards the time for the - opcode, but not in all cases. For example, a CALL_FUNCTION opcode - calls another Python function; there's no point in charge all the - bytecode executed by the called function to the caller. - - It's hard to make a useful judgement statically. In the presence - of operator overloading, it's impossible to tell if a call will - execute new Python code or not. - - It's a case-by-case judgement. I'll use intr1 for the following - cases: - - IMPORT_STAR - IMPORT_FROM - CALL_FUNCTION (and friends) - - */ - uint64 inst0, inst1, loop0, loop1, intr0 = 0, intr1 = 0; - int ticked = 0; - - READ_TIMESTAMP(inst0); - READ_TIMESTAMP(inst1); - READ_TIMESTAMP(loop0); - READ_TIMESTAMP(loop1); - - /* shut up the compiler */ - opcode = 0; -#endif - /* Code access macros */ #ifdef WORDS_BIGENDIAN @@ -1225,23 +1105,6 @@ #endif for (;;) { -#ifdef WITH_TSC - if (inst1 == 0) { - /* Almost surely, the opcode executed a break - or a continue, preventing inst1 from being set - on the way out of the loop. - */ - READ_TIMESTAMP(inst1); - loop1 = inst1; - } - dump_tsc(opcode, ticked, inst0, inst1, loop0, loop1, - intr0, intr1); - ticked = 0; - inst1 = 0; - intr0 = 0; - intr1 = 0; - READ_TIMESTAMP(loop0); -#endif assert(stack_pointer >= f->f_valuestack); /* else underflow */ assert(STACK_LEVEL() <= co->co_stacksize); /* else overflow */ assert(!PyErr_Occurred()); @@ -1260,9 +1123,6 @@ a try: finally: block uninterruptible. */ goto fast_next_opcode; } -#ifdef WITH_TSC - ticked = 1; -#endif if (_Py_atomic_load_relaxed(&pendingcalls_to_do)) { if (Py_MakePendingCalls() < 0) goto error; @@ -1353,9 +1213,6 @@ } #endif - /* Main switch on opcode */ - READ_TIMESTAMP(inst0); - switch (opcode) { /* BEWARE! @@ -2966,11 +2823,9 @@ PyObject *fromlist = POP(); PyObject *level = TOP(); PyObject *res; - READ_TIMESTAMP(intr0); res = import_name(f, name, fromlist, level); Py_DECREF(level); Py_DECREF(fromlist); - READ_TIMESTAMP(intr1); SET_TOP(res); if (res == NULL) goto error; @@ -2989,9 +2844,7 @@ "no locals found during 'import *'"); goto error; } - READ_TIMESTAMP(intr0); err = import_all_from(locals, from); - READ_TIMESTAMP(intr1); PyFrame_LocalsToFast(f, 0); Py_DECREF(from); if (err != 0) @@ -3003,9 +2856,7 @@ PyObject *name = GETITEM(names, oparg); PyObject *from = TOP(); PyObject *res; - READ_TIMESTAMP(intr0); res = import_from(from, name); - READ_TIMESTAMP(intr1); PUSH(res); if (res == NULL) goto error; @@ -3403,11 +3254,7 @@ PyObject **sp, *res; PCALL(PCALL_ALL); sp = stack_pointer; -#ifdef WITH_TSC - res = call_function(&sp, oparg, NULL, &intr0, &intr1); -#else res = call_function(&sp, oparg, NULL); -#endif stack_pointer = sp; PUSH(res); if (res == NULL) { @@ -3423,11 +3270,7 @@ assert(PyTuple_CheckExact(names) && PyTuple_GET_SIZE(names) <= oparg); PCALL(PCALL_ALL); sp = stack_pointer; -#ifdef WITH_TSC - res = call_function(&sp, oparg, names, &intr0, &intr1); -#else res = call_function(&sp, oparg, names); -#endif stack_pointer = sp; PUSH(res); Py_DECREF(names); @@ -3449,9 +3292,7 @@ assert(PyTuple_CheckExact(callargs)); func = TOP(); - READ_TIMESTAMP(intr0); result = do_call_core(func, callargs, kwargs); - READ_TIMESTAMP(intr1); Py_DECREF(func); Py_DECREF(callargs); Py_XDECREF(kwargs); @@ -3602,7 +3443,6 @@ assert(0); error: - READ_TIMESTAMP(inst1); assert(why == WHY_NOT); why = WHY_EXCEPTION; @@ -3706,7 +3546,6 @@ if (why != WHY_NOT) break; - READ_TIMESTAMP(loop1); assert(!PyErr_Occurred()); @@ -4922,11 +4761,7 @@ } static PyObject * -call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *kwnames -#ifdef WITH_TSC - , uint64* pintr0, uint64* pintr1 -#endif - ) +call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *kwnames) { PyObject **pfunc = (*pp_stack) - oparg - 1; PyObject *func = *pfunc; @@ -4964,14 +4799,12 @@ stack = (*pp_stack) - nargs - nkwargs; - READ_TIMESTAMP(*pintr0); if (PyFunction_Check(func)) { x = fast_function(func, stack, nargs, kwnames); } else { x = _PyObject_FastCallKeywords(func, stack, nargs, kwnames); } - READ_TIMESTAMP(*pintr1); Py_DECREF(func); } diff --git a/Python/pystate.c b/Python/pystate.c --- a/Python/pystate.c +++ b/Python/pystate.c @@ -99,9 +99,6 @@ interp->dlopenflags = RTLD_LAZY; #endif #endif -#ifdef WITH_TSC - interp->tscdump = 0; -#endif HEAD_LOCK(); interp->next = interp_head; diff --git a/Python/pystrtod.c b/Python/pystrtod.c --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -370,6 +370,72 @@ return result; } +/* Remove underscores that follow the underscore placement rule from + the string and then call the `innerfunc` function on the result. + It should return a new object or NULL on exception. + + `what` is used for the error message emitted when underscores are detected + that don't follow the rule. `arg` is an opaque pointer passed to the inner + function. + + This is used to implement underscore-agnostic conversion for floats + and complex numbers. +*/ +PyObject * +_Py_string_to_number_with_underscores( + const char *s, Py_ssize_t orig_len, const char *what, PyObject *obj, void *arg, + PyObject *(*innerfunc)(const char *, Py_ssize_t, void *)) +{ + char prev; + const char *p, *last; + char *dup, *end; + PyObject *result; + + if (strchr(s, '_') == NULL) { + return innerfunc(s, orig_len, arg); + } + + dup = PyMem_Malloc(orig_len + 1); + end = dup; + prev = '\0'; + last = s + orig_len; + for (p = s; *p; p++) { + if (*p == '_') { + /* Underscores are only allowed after digits. */ + if (!(prev >= '0' && prev <= '9')) { + goto error; + } + } + else { + *end++ = *p; + /* Underscores are only allowed before digits. */ + if (prev == '_' && !(*p >= '0' && *p <= '9')) { + goto error; + } + } + prev = *p; + } + /* Underscores are not allowed at the end. */ + if (prev == '_') { + goto error; + } + /* No embedded NULs allowed. */ + if (p != last) { + goto error; + } + *end = '\0'; + result = innerfunc(dup, end - dup, arg); + PyMem_Free(dup); + return result; + + error: + PyMem_Free(dup); + PyErr_Format(PyExc_ValueError, + "could not convert string to %s: " + "%R", what, obj); + return NULL; +} + #ifdef PY_NO_SHORT_FLOAT_REPR /* Given a string that may have a decimal point in the current diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -609,33 +609,6 @@ #endif /* WITH_THREAD */ -#ifdef WITH_TSC -static PyObject * -sys_settscdump(PyObject *self, PyObject *args) -{ - int bool; - PyThreadState *tstate = PyThreadState_Get(); - - if (!PyArg_ParseTuple(args, "i:settscdump", &bool)) - return NULL; - if (bool) - tstate->interp->tscdump = 1; - else - tstate->interp->tscdump = 0; - Py_INCREF(Py_None); - return Py_None; - -} - -PyDoc_STRVAR(settscdump_doc, -"settscdump(bool)\n\ -\n\ -If true, tell the Python interpreter to dump VM measurements to\n\ -stderr. If false, turn off dump. The measurements are based on the\n\ -processor's time-stamp counter." -); -#endif /* TSC */ - static PyObject * sys_setrecursionlimit(PyObject *self, PyObject *args) { @@ -1410,9 +1383,6 @@ {"getprofile", sys_getprofile, METH_NOARGS, getprofile_doc}, {"setrecursionlimit", sys_setrecursionlimit, METH_VARARGS, setrecursionlimit_doc}, -#ifdef WITH_TSC - {"settscdump", sys_settscdump, METH_VARARGS, settscdump_doc}, -#endif {"settrace", sys_settrace, METH_O, settrace_doc}, {"gettrace", sys_gettrace, METH_NOARGS, gettrace_doc}, {"call_tracing", sys_call_tracing, METH_VARARGS, call_tracing_doc}, diff --git a/configure b/configure --- a/configure +++ b/configure @@ -830,7 +830,6 @@ with_thread enable_ipv6 with_doc_strings -with_tsc with_pymalloc with_valgrind with_fpectl @@ -1534,7 +1533,6 @@ --with(out)-thread[=DIRECTORY] deprecated; use --with(out)-threads --with(out)-doc-strings disable/enable documentation strings - --with(out)-tsc enable/disable timestamp counter profile --with(out)-pymalloc disable/enable specialized mallocs --with-valgrind Enable Valgrind support --with-fpectl enable SIGFPE catching @@ -10798,29 +10796,6 @@ { $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_doc_strings" >&5 $as_echo "$with_doc_strings" >&6; } -# Check if eval loop should use timestamp counter profiling -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-tsc" >&5 -$as_echo_n "checking for --with-tsc... " >&6; } - -# Check whether --with-tsc was given. -if test "${with_tsc+set}" = set; then : - withval=$with_tsc; -if test "$withval" != no -then - -$as_echo "#define WITH_TSC 1" >>confdefs.h - - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } -else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - # Check for Python-specific malloc support { $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-pymalloc" >&5 $as_echo_n "checking for --with-pymalloc... " >&6; } diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3198,19 +3198,6 @@ fi AC_MSG_RESULT($with_doc_strings) -# Check if eval loop should use timestamp counter profiling -AC_MSG_CHECKING(for --with-tsc) -AC_ARG_WITH(tsc, - AS_HELP_STRING([--with(out)-tsc],[enable/disable timestamp counter profile]),[ -if test "$withval" != no -then - AC_DEFINE(WITH_TSC, 1, - [Define to profile with the Pentium timestamp counter]) - AC_MSG_RESULT(yes) -else AC_MSG_RESULT(no) -fi], -[AC_MSG_RESULT(no)]) - # Check for Python-specific malloc support AC_MSG_CHECKING(for --with-pymalloc) AC_ARG_WITH(pymalloc, diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -1388,9 +1388,6 @@ /* Define if you want to compile in rudimentary thread support */ #undef WITH_THREAD -/* Define to profile with the Pentium timestamp counter */ -#undef WITH_TSC - /* Define if you want pymalloc to be disabled when running under valgrind */ #undef WITH_VALGRIND -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:15:47 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 22:15:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_repair_reST?= Message-ID: <20160909221546.36557.92952.9AAA4EEC@psf.io> https://hg.python.org/cpython/rev/18e6983f20c9 changeset: 103481:18e6983f20c9 user: Benjamin Peterson date: Fri Sep 09 15:14:56 2016 -0700 summary: repair reST files: Misc/NEWS | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -996,10 +996,10 @@ - Issue #27156: Remove obsolete code not used by IDLE. Replacements: 1. help.txt, replaced by help.html, is out-of-date and should not be used. Its dedicated viewer has be replaced by the html viewer in help.py. - 2. ?`import idlever; I = idlever.IDLE_VERSION`? is the same as - ?`import sys; I = version[:version.index(' ')]`? - 3. After ?`ob = stackviewer.VariablesTreeItem(*args)`?, - ?`ob.keys() == list(ob.object.keys)`?. + 2. ``import idlever; I = idlever.IDLE_VERSION`` is the same as + ``import sys; I = version[:version.index(' ')]`` + 3. After ``ob = stackviewer.VariablesTreeItem(*args)``, + ``ob.keys() == list(ob.object.keys)``. 4. In macosc, runningAsOSXAPP == isAquaTk; idCarbonAquaTk == isCarbonTk - Issue #27117: Make colorizer htest and turtledemo work with dark themes. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:19:57 2016 From: python-checkins at python.org (christian.heimes) Date: Fri, 09 Sep 2016 22:19:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328025=3A_Convert_?= =?utf-8?q?all_ssl_module_constants_to_IntEnum_and_IntFlags=2E?= Message-ID: <20160909221957.1874.53414.13C62ED1@psf.io> https://hg.python.org/cpython/rev/c32e9f9b00f7 changeset: 103482:c32e9f9b00f7 user: Christian Heimes date: Sat Sep 10 00:19:35 2016 +0200 summary: Issue #28025: Convert all ssl module constants to IntEnum and IntFlags. files: Doc/library/ssl.rst | 51 +++++++++++++++++++++ Lib/ssl.py | 80 +++++++++++++++++++++++++------- Misc/NEWS | 3 + 3 files changed, 115 insertions(+), 19 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -515,6 +515,10 @@ Constants ^^^^^^^^^ + All constants are now :class:`enum.IntEnum` or :class:`enum.IntFlag` collections. + + .. versionadded:: 3.6 + .. data:: CERT_NONE Possible value for :attr:`SSLContext.verify_mode`, or the ``cert_reqs`` @@ -548,6 +552,12 @@ be passed, either to :meth:`SSLContext.load_verify_locations` or as a value of the ``ca_certs`` parameter to :func:`wrap_socket`. +.. class:: VerifyMode + + :class:`enum.IntEnum` collection of CERT_* constants. + + .. versionadded:: 3.6 + .. data:: VERIFY_DEFAULT Possible value for :attr:`SSLContext.verify_flags`. In this mode, certificate @@ -588,6 +598,12 @@ .. versionadded:: 3.4.4 +.. class:: VerifyFlags + + :class:`enum.IntFlag` collection of VERIFY_* constants. + + .. versionadded:: 3.6 + .. data:: PROTOCOL_TLS Selects the highest protocol version that both the client and server support. @@ -757,6 +773,12 @@ .. versionadded:: 3.3 +.. class:: Options + + :class:`enum.IntFlag` collection of OP_* constants. + + .. versionadded:: 3.6 + .. data:: HAS_ALPN Whether the OpenSSL library has built-in support for the *Application-Layer @@ -839,6 +861,12 @@ .. versionadded:: 3.4 +.. class:: AlertDescription + + :class:`enum.IntEnum` collection of ALERT_DESCRIPTION_* constants. + + .. versionadded:: 3.6 + .. data:: Purpose.SERVER_AUTH Option for :func:`create_default_context` and @@ -857,6 +885,12 @@ .. versionadded:: 3.4 +.. class:: SSLErrorNumber + + :class:`enum.IntEnum` collection of SSL_ERROR_* constants. + + .. versionadded:: 3.6 + SSL Sockets ----------- @@ -1540,6 +1574,12 @@ to set options, not to clear them. Attempting to clear an option (by resetting the corresponding bits) will raise a ``ValueError``. + .. versionchanged:: 3.6 + :attr:`SSLContext.options` returns :class:`Options` flags: + + >>> ssl.create_default_context().options + + .. attribute:: SSLContext.protocol The protocol version chosen when constructing the context. This attribute @@ -1554,12 +1594,23 @@ .. versionadded:: 3.4 + .. versionchanged:: 3.6 + :attr:`SSLContext.verify_flags` returns :class:`VerifyFlags` flags: + + >>> ssl.create_default_context().verify_flags + + .. attribute:: SSLContext.verify_mode Whether to try to verify other peers' certificates and how to behave if verification fails. This attribute must be one of :data:`CERT_NONE`, :data:`CERT_OPTIONAL` or :data:`CERT_REQUIRED`. + .. versionchanged:: 3.6 + :attr:`SSLContext.verify_mode` returns :class:`VerifyMode` enum: + + >>> ssl.create_default_context().verify_mode + .. index:: single: certificates diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -94,7 +94,7 @@ import sys import os from collections import namedtuple -from enum import Enum as _Enum, IntEnum as _IntEnum +from enum import Enum as _Enum, IntEnum as _IntEnum, IntFlag as _IntFlag import _ssl # if we can't import it, let the error propagate @@ -104,7 +104,6 @@ SSLError, SSLZeroReturnError, SSLWantReadError, SSLWantWriteError, SSLSyscallError, SSLEOFError, ) -from _ssl import CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED from _ssl import txt2obj as _txt2obj, nid2obj as _nid2obj from _ssl import RAND_status, RAND_add, RAND_bytes, RAND_pseudo_bytes try: @@ -113,32 +112,47 @@ # LibreSSL does not provide RAND_egd pass -def _import_symbols(prefix): - for n in dir(_ssl): - if n.startswith(prefix): - globals()[n] = getattr(_ssl, n) - -_import_symbols('OP_') -_import_symbols('ALERT_DESCRIPTION_') -_import_symbols('SSL_ERROR_') -_import_symbols('VERIFY_') from _ssl import HAS_SNI, HAS_ECDH, HAS_NPN, HAS_ALPN - from _ssl import _OPENSSL_API_VERSION + _IntEnum._convert( - '_SSLMethod', __name__, - lambda name: name.startswith('PROTOCOL_') and name != 'PROTOCOL_SSLv23', - source=_ssl) + '_SSLMethod', __name__, + lambda name: name.startswith('PROTOCOL_') and name != 'PROTOCOL_SSLv23', + source=_ssl) + +_IntFlag._convert( + 'Options', __name__, + lambda name: name.startswith('OP_'), + source=_ssl) + +_IntEnum._convert( + 'AlertDescription', __name__, + lambda name: name.startswith('ALERT_DESCRIPTION_'), + source=_ssl) + +_IntEnum._convert( + 'SSLErrorNumber', __name__, + lambda name: name.startswith('SSL_ERROR_'), + source=_ssl) + +_IntFlag._convert( + 'VerifyFlags', __name__, + lambda name: name.startswith('VERIFY_'), + source=_ssl) + +_IntEnum._convert( + 'VerifyMode', __name__, + lambda name: name.startswith('CERT_'), + source=_ssl) + PROTOCOL_SSLv23 = _SSLMethod.PROTOCOL_SSLv23 = _SSLMethod.PROTOCOL_TLS _PROTOCOL_NAMES = {value: name for name, value in _SSLMethod.__members__.items()} -try: - _SSLv2_IF_EXISTS = PROTOCOL_SSLv2 -except NameError: - _SSLv2_IF_EXISTS = None +_SSLv2_IF_EXISTS = getattr(_SSLMethod, 'PROTOCOL_SSLv2', None) + if sys.platform == "win32": from _ssl import enum_certificates, enum_crls @@ -434,6 +448,34 @@ self._load_windows_store_certs(storename, purpose) self.set_default_verify_paths() + @property + def options(self): + return Options(super().options) + + @options.setter + def options(self, value): + super(SSLContext, SSLContext).options.__set__(self, value) + + @property + def verify_flags(self): + return VerifyFlags(super().verify_flags) + + @verify_flags.setter + def verify_flags(self, value): + super(SSLContext, SSLContext).verify_flags.__set__(self, value) + + @property + def verify_mode(self): + value = super().verify_mode + try: + return VerifyMode(value) + except ValueError: + return value + + @verify_mode.setter + def verify_mode(self, value): + super(SSLContext, SSLContext).verify_mode.__set__(self, value) + def create_default_context(purpose=Purpose.SERVER_AUTH, *, cafile=None, capath=None, cadata=None): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -122,6 +122,9 @@ Library ------- +- Issue #28025: Convert all ssl module constants to IntEnum and IntFlags. + SSLContext properties now return flags and enums. + - Issue #433028: Added support of modifier spans in regular expressions. - Issue #24594: Validates persist parameter when opening MSI database -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:22:22 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 22:22:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_links_from_whatsnew_to?= =?utf-8?q?_Windows_docs=2E?= Message-ID: <20160909222222.3168.85192.4F2D4E24@psf.io> https://hg.python.org/cpython/rev/391ffd449a47 changeset: 103483:391ffd449a47 user: Steve Dower date: Fri Sep 09 15:22:13 2016 -0700 summary: Add links from whatsnew to Windows docs. files: Doc/using/windows.rst | 2 +- Doc/whatsnew/3.6.rst | 7 +++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -709,7 +709,7 @@ -.. finding_modules: +.. _finding_modules: Finding modules =============== diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -102,9 +102,12 @@ See :ref:`removing the MAX_PATH limitation ` for details. * A ``sys.path`` file can be added to force isolated mode and fully specify - all search paths to avoid registry and environment lookup. + all search paths to avoid registry and environment lookup. See + :ref:`the documentation ` for more information. -* A ``python36.zip`` file now works as a landmark to infer :envvar:`PYTHONHOME` +* A ``python36.zip`` file now works as a landmark to infer + :envvar:`PYTHONHOME`. See :ref:`the documentation ` for + more information. .. PEP-sized items next. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:24:27 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 22:24:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Ensures_buildbots_don=27t_?= =?utf-8?q?have_zip_files_in_build_directory=2E?= Message-ID: <20160909222426.13656.85059.3408F704@psf.io> https://hg.python.org/cpython/rev/b97ad2f4c6b2 changeset: 103484:b97ad2f4c6b2 user: Steve Dower date: Fri Sep 09 15:24:11 2016 -0700 summary: Ensures buildbots don't have zip files in build directory. files: Tools/buildbot/clean.bat | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Tools/buildbot/clean.bat b/Tools/buildbot/clean.bat --- a/Tools/buildbot/clean.bat +++ b/Tools/buildbot/clean.bat @@ -14,3 +14,4 @@ echo Deleting test leftovers ... rmdir /s /q "%root%\build" +del /s "%pcbuild%\python*.zip" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:34:12 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 22:34:12 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Adds_temporary_validation_?= =?utf-8?q?code_to_buildbot_script?= Message-ID: <20160909223412.8471.26237.267CD9DD@psf.io> https://hg.python.org/cpython/rev/808b4b8c6fe8 changeset: 103485:808b4b8c6fe8 user: Steve Dower date: Fri Sep 09 15:33:42 2016 -0700 summary: Adds temporary validation code to buildbot script files: Tools/buildbot/test.bat | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -16,4 +16,11 @@ if NOT "%1"=="" (set regrtest_args=%regrtest_args% %1) & shift & goto CheckOpts echo on +rem Start temporary diagnostic code +set +echo All zip files +dir /s/b "%here%..\..\PCbuild\*.zip" +echo. +rem End temporary diagnostic code + call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW --slowest --timeout=900 %regrtest_args% -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:35:40 2016 From: python-checkins at python.org (benjamin.peterson) Date: Fri, 09 Sep 2016 22:35:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_repair_versionadded_direct?= =?utf-8?q?ive?= Message-ID: <20160909223538.87325.45947.BF27009F@psf.io> https://hg.python.org/cpython/rev/c635297e6642 changeset: 103486:c635297e6642 user: Benjamin Peterson date: Fri Sep 09 15:34:58 2016 -0700 summary: repair versionadded directive files: Doc/library/re.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/re.rst b/Doc/library/re.rst --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -245,7 +245,7 @@ (dot matches all), and :const:`re.X` (verbose), for the part of the expression. (The flags are described in :ref:`contents-of-module-re`.) - .. versionadded: 3.7 + .. versionadded:: 3.7 ``(?P...)`` Similar to regular parentheses, but the substring matched by the group is -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:35:54 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 22:35:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_outdated_buildbot_s?= =?utf-8?q?cripts?= Message-ID: <20160909223553.8602.43501.988E7216@psf.io> https://hg.python.org/cpython/rev/d85bc2d9fc3c changeset: 103487:d85bc2d9fc3c user: Zachary Ware date: Fri Sep 09 15:35:38 2016 -0700 summary: Remove outdated buildbot scripts files: Tools/buildbot/build-amd64.bat | 5 ----- Tools/buildbot/clean-amd64.bat | 5 ----- Tools/buildbot/external-amd64.bat | 3 --- Tools/buildbot/test-amd64.bat | 6 ------ 4 files changed, 0 insertions(+), 19 deletions(-) diff --git a/Tools/buildbot/build-amd64.bat b/Tools/buildbot/build-amd64.bat deleted file mode 100644 --- a/Tools/buildbot/build-amd64.bat +++ /dev/null @@ -1,5 +0,0 @@ - at rem Formerly used by the buildbot "compile" step. - at echo This script is no longer used and may be removed in the future. - at echo To get the same effect as this script, use - at echo PCbuild\build.bat -d -e -k -p x64 -call "%~dp0build.bat" -p x64 %* diff --git a/Tools/buildbot/clean-amd64.bat b/Tools/buildbot/clean-amd64.bat deleted file mode 100644 --- a/Tools/buildbot/clean-amd64.bat +++ /dev/null @@ -1,5 +0,0 @@ - at rem Formerly used by the buildbot "clean" step. - at echo This script is no longer used and may be removed in the future. - at echo To get the same effect as this script, use `clean.bat` from this - at echo directory and pass `-p x64` as two arguments. -call "%~dp0clean.bat" -p x64 %* diff --git a/Tools/buildbot/external-amd64.bat b/Tools/buildbot/external-amd64.bat deleted file mode 100644 --- a/Tools/buildbot/external-amd64.bat +++ /dev/null @@ -1,3 +0,0 @@ - at echo This script is no longer used and may be removed in the future. - at echo Please use PCbuild\get_externals.bat instead. -@"%~dp0..\..\PCbuild\get_externals.bat" %* diff --git a/Tools/buildbot/test-amd64.bat b/Tools/buildbot/test-amd64.bat deleted file mode 100644 --- a/Tools/buildbot/test-amd64.bat +++ /dev/null @@ -1,6 +0,0 @@ - at rem Formerly used by the buildbot "test" step. - at echo This script is no longer used and may be removed in the future. - at echo To get the same effect as this script, use - at echo PCbuild\rt.bat -q -d -x64 -uall -rwW - at echo or use `test.bat` in this directory and pass `-x64` as an argument. -call "%~dp0test.bat" -x64 %* -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:39:23 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 22:39:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Expands_buildbot_validatio?= =?utf-8?q?n_code?= Message-ID: <20160909223919.8490.6354.A55BFCC5@psf.io> https://hg.python.org/cpython/rev/50129f5ab5b4 changeset: 103488:50129f5ab5b4 user: Steve Dower date: Fri Sep 09 15:39:11 2016 -0700 summary: Expands buildbot validation code files: Tools/buildbot/test.bat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -19,7 +19,7 @@ rem Start temporary diagnostic code set echo All zip files -dir /s/b "%here%..\..\PCbuild\*.zip" +dir /s/b "%here%..\..\PCbuild\*" echo. rem End temporary diagnostic code -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:40:33 2016 From: python-checkins at python.org (r.david.murray) Date: Fri, 09 Sep 2016 22:40:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2320476=3A_add_a_message?= =?utf-8?q?=5Ffactory_policy_attribute_to_email=2E?= Message-ID: <20160909224033.2666.68738.601B15AC@psf.io> https://hg.python.org/cpython/rev/9ba8f4be0651 changeset: 103489:9ba8f4be0651 user: R David Murray date: Fri Sep 09 18:39:18 2016 -0400 summary: #20476: add a message_factory policy attribute to email. files: Doc/library/email.parser.rst | 20 ++- Doc/library/email.policy.rst | 11 ++ Doc/whatsnew/3.6.rst | 7 + Lib/email/_policybase.py | 4 + Lib/email/feedparser.py | 9 +- Lib/email/message.py | 8 +- Lib/email/policy.py | 2 + Lib/test/test_email/test_parser.py | 91 ++++++++++++----- Lib/test/test_email/test_policy.py | 42 ++++--- 9 files changed, 128 insertions(+), 66 deletions(-) diff --git a/Doc/library/email.parser.rst b/Doc/library/email.parser.rst --- a/Doc/library/email.parser.rst +++ b/Doc/library/email.parser.rst @@ -73,8 +73,9 @@ .. class:: BytesFeedParser(_factory=None, *, policy=policy.compat32) Create a :class:`BytesFeedParser` instance. Optional *_factory* is a - no-argument callable; if not specified determine the default based on the - *policy*. Call *_factory* whenever a new message object is needed. + no-argument callable; if not specified use the + :attr:`~email.policy.Policy.message_factory` from the *policy*. Call + *_factory* whenever a new message object is needed. If *policy* is specified use the rules it specifies to update the representation of the message. If *policy* is not set, use the @@ -91,6 +92,7 @@ .. versionadded:: 3.2 .. versionchanged:: 3.3 Added the *policy* keyword. + .. versionchanged:: 3.6 _factory defaults to the policy ``message_factory``. .. method:: feed(data) @@ -146,6 +148,7 @@ .. versionchanged:: 3.3 Removed the *strict* argument that was deprecated in 2.4. Added the *policy* keyword. + .. versionchanged:: 3.6 _class defaults to the policy ``message_factory``. .. method:: parse(fp, headersonly=False) @@ -194,6 +197,7 @@ .. versionchanged:: 3.3 Removed the *strict* argument. Added the *policy* keyword. + .. versionchanged:: 3.6 _class defaults to the policy ``message_factory``. .. method:: parse(fp, headersonly=False) @@ -230,8 +234,7 @@ .. currentmodule:: email -.. function:: message_from_bytes(s, _class=None, *, \ - policy=policy.compat32) +.. function:: message_from_bytes(s, _class=None, *, policy=policy.compat32) Return a message object structure from a :term:`bytes-like object`. This is equivalent to ``BytesParser().parsebytes(s)``. Optional *_class* and @@ -243,7 +246,7 @@ Removed the *strict* argument. Added the *policy* keyword. -.. function:: message_from_binary_file(fp, _class=None, *, \ +.. function:: message_from_binary_file(fp, _class=None, *, policy=policy.compat32) Return a message object structure tree from an open binary :term:`file @@ -256,8 +259,7 @@ Removed the *strict* argument. Added the *policy* keyword. -.. function:: message_from_string(s, _class=None, *, \ - policy=policy.compat32) +.. function:: message_from_string(s, _class=None, *, policy=policy.compat32) Return a message object structure from a string. This is equivalent to ``Parser().parsestr(s)``. *_class* and *policy* are interpreted as @@ -267,8 +269,7 @@ Removed the *strict* argument. Added the *policy* keyword. -.. function:: message_from_file(fp, _class=None, *, \ - policy=policy.compat32) +.. function:: message_from_file(fp, _class=None, *, policy=policy.compat32) Return a message object structure tree from an open :term:`file object`. This is equivalent to ``Parser().parse(fp)``. *_class* and *policy* are @@ -276,6 +277,7 @@ .. versionchanged:: 3.3 Removed the *strict* argument. Added the *policy* keyword. + .. versionchanged:: 3.6 _class defaults to the policy ``message_factory``. Here's an example of how you might use :func:`message_from_bytes` at an diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -221,6 +221,14 @@ The *mangle_from_* parameter. + .. attribute:: message_factory + + A factory function for constructing a new empty message object. Used + by the parser when building messages. Defaults to + :class:`~email.message.Message`. + + .. versionadded:: 3.6 + The following :class:`Policy` method is intended to be called by code using the email library to create policy instances with custom settings: @@ -368,6 +376,9 @@ on the type of the field. The parsing and folding algorithm fully implement :rfc:`2047` and :rfc:`5322`. + The default value for the :attr:`~email.policy.Policy.message_factory` + attribute is :class:`~email.message.EmailMessage`. + In addition to the settable attributes listed above that apply to all policies, this policy adds the following additional attributes: diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -598,6 +598,13 @@ The :class:`~email.generator.DecodedGenerator` now supports the *policy* keyword. +There is a new :mod:`~email.policy` attribute, +:attr:`~email.policy.Policy.message_factory`, that controls what class is used +by default when the parser creates new message objects. For the +:attr:`email.policy.compat32` policy this is :class:`~email.message.Message`, +for the new policies it is :class:`~email.message.EmailMessage`. +(Contributed by R. David Murray in :issue:`20476`.) + encodings --------- diff --git a/Lib/email/_policybase.py b/Lib/email/_policybase.py --- a/Lib/email/_policybase.py +++ b/Lib/email/_policybase.py @@ -154,6 +154,8 @@ them. This is used when the message is being serialized by a generator. Default: True. + message_factory -- the class to use to create new message objects. + """ raise_on_defect = False @@ -161,6 +163,8 @@ cte_type = '8bit' max_line_length = 78 mangle_from_ = False + # XXX To avoid circular imports, this is set in email.message. + message_factory = None def handle_defect(self, obj, defect): """Based on policy, either raise defect or call register_defect. diff --git a/Lib/email/feedparser.py b/Lib/email/feedparser.py --- a/Lib/email/feedparser.py +++ b/Lib/email/feedparser.py @@ -24,7 +24,6 @@ import re from email import errors -from email import message from email._policybase import compat32 from collections import deque from io import StringIO @@ -148,13 +147,7 @@ self.policy = policy self._old_style_factory = False if _factory is None: - # What this should be: - #self._factory = policy.default_message_factory - # but, because we are post 3.4 feature freeze, fix with temp hack: - if self.policy is compat32: - self._factory = message.Message - else: - self._factory = message.EmailMessage + self._factory = policy.message_factory else: self._factory = _factory try: diff --git a/Lib/email/message.py b/Lib/email/message.py --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -4,18 +4,17 @@ """Basic message object for the email package object model.""" -__all__ = ['Message'] +__all__ = ['Message', 'EmailMessage'] import re import uu import quopri -import warnings from io import BytesIO, StringIO # Intrapackage imports from email import utils from email import errors -from email._policybase import compat32 +from email._policybase import Policy, compat32 from email import charset as _charset from email._encoded_words import decode_b Charset = _charset.Charset @@ -1163,3 +1162,6 @@ super().set_content(*args, **kw) if 'MIME-Version' not in self: self['MIME-Version'] = '1.0' + +# Set message_factory on Policy here to avoid a circular import. +Policy.message_factory = Message diff --git a/Lib/email/policy.py b/Lib/email/policy.py --- a/Lib/email/policy.py +++ b/Lib/email/policy.py @@ -7,6 +7,7 @@ from email.utils import _has_surrogates from email.headerregistry import HeaderRegistry as HeaderRegistry from email.contentmanager import raw_data_manager +from email.message import EmailMessage __all__ = [ 'Compat32', @@ -82,6 +83,7 @@ """ + message_factory = EmailMessage utf8 = False refold_source = 'long' header_factory = HeaderRegistry() diff --git a/Lib/test/test_email/test_parser.py b/Lib/test/test_email/test_parser.py --- a/Lib/test/test_email/test_parser.py +++ b/Lib/test/test_email/test_parser.py @@ -1,7 +1,7 @@ import io import email import unittest -from email.message import Message +from email.message import Message, EmailMessage from email.policy import default from test.test_email import TestEmailBase @@ -39,38 +39,71 @@ # The unicode line splitter splits on unicode linebreaks, which are # more numerous than allowed by the email RFCs; make sure we are only # splitting on those two. - msg = self.parser( - "Next-Line: not\x85broken\r\n" - "Null: not\x00broken\r\n" - "Vertical-Tab: not\vbroken\r\n" - "Form-Feed: not\fbroken\r\n" - "File-Separator: not\x1Cbroken\r\n" - "Group-Separator: not\x1Dbroken\r\n" - "Record-Separator: not\x1Ebroken\r\n" - "Line-Separator: not\u2028broken\r\n" - "Paragraph-Separator: not\u2029broken\r\n" - "\r\n", - policy=default, - ) - self.assertEqual(msg.items(), [ - ("Next-Line", "not\x85broken"), - ("Null", "not\x00broken"), - ("Vertical-Tab", "not\vbroken"), - ("Form-Feed", "not\fbroken"), - ("File-Separator", "not\x1Cbroken"), - ("Group-Separator", "not\x1Dbroken"), - ("Record-Separator", "not\x1Ebroken"), - ("Line-Separator", "not\u2028broken"), - ("Paragraph-Separator", "not\u2029broken"), - ]) - self.assertEqual(msg.get_payload(), "") + for parser in self.parsers: + with self.subTest(parser=parser.__name__): + msg = parser( + "Next-Line: not\x85broken\r\n" + "Null: not\x00broken\r\n" + "Vertical-Tab: not\vbroken\r\n" + "Form-Feed: not\fbroken\r\n" + "File-Separator: not\x1Cbroken\r\n" + "Group-Separator: not\x1Dbroken\r\n" + "Record-Separator: not\x1Ebroken\r\n" + "Line-Separator: not\u2028broken\r\n" + "Paragraph-Separator: not\u2029broken\r\n" + "\r\n", + policy=default, + ) + self.assertEqual(msg.items(), [ + ("Next-Line", "not\x85broken"), + ("Null", "not\x00broken"), + ("Vertical-Tab", "not\vbroken"), + ("Form-Feed", "not\fbroken"), + ("File-Separator", "not\x1Cbroken"), + ("Group-Separator", "not\x1Dbroken"), + ("Record-Separator", "not\x1Ebroken"), + ("Line-Separator", "not\u2028broken"), + ("Paragraph-Separator", "not\u2029broken"), + ]) + self.assertEqual(msg.get_payload(), "") + + class MyMessage(EmailMessage): + pass + + def test_custom_message_factory_on_policy(self): + for parser in self.parsers: + with self.subTest(parser=parser.__name__): + MyPolicy = default.clone(message_factory=self.MyMessage) + msg = parser("To: foo\n\ntest", policy=MyPolicy) + self.assertIsInstance(msg, self.MyMessage) + + def test_factory_arg_overrides_policy(self): + for parser in self.parsers: + with self.subTest(parser=parser.__name__): + MyPolicy = default.clone(message_factory=self.MyMessage) + msg = parser("To: foo\n\ntest", Message, policy=MyPolicy) + self.assertNotIsInstance(msg, self.MyMessage) + self.assertIsInstance(msg, Message) + +# Play some games to get nice output in subTest. This code could be clearer +# if staticmethod supported __name__. + +def message_from_file(s, *args, **kw): + f = io.StringIO(s) + return email.message_from_file(f, *args, **kw) class TestParser(TestParserBase, TestEmailBase): - parser = staticmethod(email.message_from_string) + parsers = (email.message_from_string, message_from_file) + +def message_from_bytes(s, *args, **kw): + return email.message_from_bytes(s.encode(), *args, **kw) + +def message_from_binary_file(s, *args, **kw): + f = io.BytesIO(s.encode()) + return email.message_from_binary_file(f, *args, **kw) class TestBytesParser(TestParserBase, TestEmailBase): - def parser(self, s, *args, **kw): - return email.message_from_bytes(s.encode(), *args, **kw) + parsers = (message_from_bytes, message_from_binary_file) if __name__ == '__main__': diff --git a/Lib/test/test_email/test_policy.py b/Lib/test/test_email/test_policy.py --- a/Lib/test/test_email/test_policy.py +++ b/Lib/test/test_email/test_policy.py @@ -5,6 +5,7 @@ import email.policy import email.parser import email.generator +import email.message from email import headerregistry def make_defaults(base_defaults, differences): @@ -23,6 +24,7 @@ 'cte_type': '8bit', 'raise_on_defect': False, 'mangle_from_': True, + 'message_factory': email.message.Message, } # These default values are the ones set on email.policy.default. # If any of these defaults change, the docs must be updated. @@ -34,6 +36,7 @@ 'refold_source': 'long', 'content_manager': email.policy.EmailPolicy.content_manager, 'mangle_from_': False, + 'message_factory': email.message.EmailMessage, }) # For each policy under test, we give here what we expect the defaults to @@ -62,20 +65,22 @@ def test_defaults(self): for policy, expected in self.policies.items(): for attr, value in expected.items(): - self.assertEqual(getattr(policy, attr), value, - ("change {} docs/docstrings if defaults have " - "changed").format(policy)) + with self.subTest(policy=policy, attr=attr): + self.assertEqual(getattr(policy, attr), value, + ("change {} docs/docstrings if defaults have " + "changed").format(policy)) def test_all_attributes_covered(self): for policy, expected in self.policies.items(): for attr in dir(policy): - if (attr.startswith('_') or - isinstance(getattr(email.policy.EmailPolicy, attr), - types.FunctionType)): - continue - else: - self.assertIn(attr, expected, - "{} is not fully tested".format(attr)) + with self.subTest(policy=policy, attr=attr): + if (attr.startswith('_') or + isinstance(getattr(email.policy.EmailPolicy, attr), + types.FunctionType)): + continue + else: + self.assertIn(attr, expected, + "{} is not fully tested".format(attr)) def test_abc(self): with self.assertRaises(TypeError) as cm: @@ -237,6 +242,9 @@ # wins), but that the order still works (right overrides left). +class TestException(Exception): + pass + class TestPolicyPropagation(unittest.TestCase): # The abstract methods are used by the parser but not by the wrapper @@ -244,40 +252,40 @@ # policy was actually propagated all the way to feedparser. class MyPolicy(email.policy.Policy): def badmethod(self, *args, **kw): - raise Exception("test") + raise TestException("test") fold = fold_binary = header_fetch_parser = badmethod header_source_parse = header_store_parse = badmethod def test_message_from_string(self): - with self.assertRaisesRegex(Exception, "^test$"): + with self.assertRaisesRegex(TestException, "^test$"): email.message_from_string("Subject: test\n\n", policy=self.MyPolicy) def test_message_from_bytes(self): - with self.assertRaisesRegex(Exception, "^test$"): + with self.assertRaisesRegex(TestException, "^test$"): email.message_from_bytes(b"Subject: test\n\n", policy=self.MyPolicy) def test_message_from_file(self): f = io.StringIO('Subject: test\n\n') - with self.assertRaisesRegex(Exception, "^test$"): + with self.assertRaisesRegex(TestException, "^test$"): email.message_from_file(f, policy=self.MyPolicy) def test_message_from_binary_file(self): f = io.BytesIO(b'Subject: test\n\n') - with self.assertRaisesRegex(Exception, "^test$"): + with self.assertRaisesRegex(TestException, "^test$"): email.message_from_binary_file(f, policy=self.MyPolicy) # These are redundant, but we need them for black-box completeness. def test_parser(self): p = email.parser.Parser(policy=self.MyPolicy) - with self.assertRaisesRegex(Exception, "^test$"): + with self.assertRaisesRegex(TestException, "^test$"): p.parsestr('Subject: test\n\n') def test_bytes_parser(self): p = email.parser.BytesParser(policy=self.MyPolicy) - with self.assertRaisesRegex(Exception, "^test$"): + with self.assertRaisesRegex(TestException, "^test$"): p.parsebytes(b'Subject: test\n\n') # Now that we've established that all the parse methods get the -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:42:15 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 22:42:15 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_another_useless_bui?= =?utf-8?q?ldbot_script?= Message-ID: <20160909224215.2637.85919.8DFA8E78@psf.io> https://hg.python.org/cpython/rev/20b62fff3ffe changeset: 103490:20b62fff3ffe user: Zachary Ware date: Fri Sep 09 15:42:06 2016 -0700 summary: Remove another useless buildbot script files: Tools/buildbot/external.bat | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Tools/buildbot/external.bat b/Tools/buildbot/external.bat deleted file mode 100644 --- a/Tools/buildbot/external.bat +++ /dev/null @@ -1,3 +0,0 @@ - at echo This script is no longer used and may be removed in the future. - at echo Please use PCbuild\get_externals.bat instead. -@"%~dp0..\..\PCbuild\get_externals.bat" %* -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:45:55 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 22:45:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_buildbot_diagnostic?= =?utf-8?q?_code=2E?= Message-ID: <20160909224555.48584.51517.FDEC15C4@psf.io> https://hg.python.org/cpython/rev/8e2f7b896a44 changeset: 103491:8e2f7b896a44 user: Steve Dower date: Fri Sep 09 15:45:47 2016 -0700 summary: Remove buildbot diagnostic code. files: Tools/buildbot/test.bat | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -16,11 +16,4 @@ if NOT "%1"=="" (set regrtest_args=%regrtest_args% %1) & shift & goto CheckOpts echo on -rem Start temporary diagnostic code -set -echo All zip files -dir /s/b "%here%..\..\PCbuild\*" -echo. -rem End temporary diagnostic code - call "%here%..\..\PCbuild\rt.bat" %rt_opts% -uall -rwW --slowest --timeout=900 %regrtest_args% -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:46:26 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 22:46:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_suspicious_markup?= Message-ID: <20160909224626.3168.3883.F219CC65@psf.io> https://hg.python.org/cpython/rev/856cca48bc73 changeset: 103492:856cca48bc73 user: Zachary Ware date: Fri Sep 09 15:46:14 2016 -0700 summary: Fix suspicious markup files: Doc/tools/susp-ignored.csv | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -297,3 +297,7 @@ whatsnew/3.5,,::,>>> addr6 = ipaddress.IPv6Address('::1') whatsnew/3.5,,:root,ERROR:root:exception whatsnew/3.5,,:exception,ERROR:root:exception +whatsnew/3.6,140,`,1000000000000000` +whatsnew/changelog,998,:version,import sys; I = version[:version.index(' ')] +whatsnew/changelog,6416,:gz,": TarFile opened with external fileobj and ""w:gz"" mode didn't" +whatsnew/changelog,8023,::,": Use ""127.0.0.1"" or ""::1"" instead of ""localhost"" as much as" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:47:18 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 22:47:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_We=27re_not_that_far_in_th?= =?utf-8?q?e_future_yet?= Message-ID: <20160909224718.2591.58279.5AD64AA2@psf.io> https://hg.python.org/cpython/rev/cd7012df4d8d changeset: 103493:cd7012df4d8d user: Zachary Ware date: Fri Sep 09 15:47:05 2016 -0700 summary: We're not that far in the future yet files: Doc/library/re.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/re.rst b/Doc/library/re.rst --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -245,7 +245,7 @@ (dot matches all), and :const:`re.X` (verbose), for the part of the expression. (The flags are described in :ref:`contents-of-module-re`.) - .. versionadded:: 3.7 + .. versionadded:: 3.6 ``(?P...)`` Similar to regular parentheses, but the substring matched by the group is -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:54:06 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 22:54:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_call_to_PathCombineW?= =?utf-8?q?=2E?= Message-ID: <20160909225406.12434.81914.34FAFE36@psf.io> https://hg.python.org/cpython/rev/bd980225bacc changeset: 103494:bd980225bacc user: Steve Dower date: Fri Sep 09 15:53:58 2016 -0700 summary: Fix call to PathCombineW. files: PC/getpathp.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/PC/getpathp.c b/PC/getpathp.c --- a/PC/getpathp.c +++ b/PC/getpathp.c @@ -197,7 +197,7 @@ if (FAILED(_PathCchCombineEx(buffer, MAXPATHLEN+1, buffer, stuff, 0))) Py_FatalError("buffer overflow in getpathp.c's join()"); } else { - if (!PathCombineW(buffer, NULL, stuff)) + if (!PathCombineW(buffer, buffer, stuff)) Py_FatalError("buffer overflow in getpathp.c's join()"); } } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:59:23 2016 From: python-checkins at python.org (raymond.hettinger) Date: Fri, 09 Sep 2016 22:59:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgMTQ5NzY6?= =?utf-8?q?__Note_that_the_queue_module_is_not_designed_to_protect_against?= Message-ID: <20160909225922.2071.2326.81E6F421@psf.io> https://hg.python.org/cpython/rev/8c00cbbd3ff9 changeset: 103495:8c00cbbd3ff9 branch: 3.5 parent: 103468:ab3d9bdb69d1 user: Raymond Hettinger date: Fri Sep 09 15:57:13 2016 -0700 summary: Issue 14976: Note that the queue module is not designed to protect against reentrancy files: Doc/library/queue.rst | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -22,6 +22,8 @@ the entries are kept sorted (using the :mod:`heapq` module) and the lowest valued entry is retrieved first. +Internally, the module uses locks to temporarily block competing threads; +however, it is not designed to handle reentrancy within a thread. The :mod:`queue` module defines the following classes and exceptions: @@ -186,11 +188,6 @@ t.join() -.. note:: - - The :mod:`queue` module is not safe for use from :mod:`signal` handlers as - it uses :mod:`threading` locks. - .. seealso:: Class :class:`multiprocessing.Queue` -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 18:59:24 2016 From: python-checkins at python.org (raymond.hettinger) Date: Fri, 09 Sep 2016 22:59:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20160909225922.48656.42589.D868924F@psf.io> https://hg.python.org/cpython/rev/5fe118913bf8 changeset: 103496:5fe118913bf8 parent: 103494:bd980225bacc parent: 103495:8c00cbbd3ff9 user: Raymond Hettinger date: Fri Sep 09 15:58:00 2016 -0700 summary: merge files: Doc/library/queue.rst | 7 +- Lib/asyncio/base_events.py | 2 +- Lib/asyncio/coroutines.py | 4 +- Lib/asyncio/futures.py | 24 +++- Lib/asyncio/tasks.py | 8 +- Lib/test/test_asyncio/test_events.py | 2 +- Lib/test/test_asyncio/test_futures.py | 68 +++++++++++++++ 7 files changed, 96 insertions(+), 19 deletions(-) diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -23,6 +23,8 @@ the entries are kept sorted (using the :mod:`heapq` module) and the lowest valued entry is retrieved first. +Internally, the module uses locks to temporarily block competing threads; +however, it is not designed to handle reentrancy within a thread. The :mod:`queue` module defines the following classes and exceptions: @@ -189,11 +191,6 @@ t.join() -.. note:: - - The :mod:`queue` module is not safe for use from :mod:`signal` handlers as - it uses :mod:`threading` locks. - .. seealso:: Class :class:`multiprocessing.Queue` diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -414,7 +414,7 @@ """ self._check_closed() - new_task = not isinstance(future, futures.Future) + new_task = not futures.isfuture(future) future = tasks.ensure_future(future, loop=self) if new_task: # An exception is raised if the future didn't complete, so there diff --git a/Lib/asyncio/coroutines.py b/Lib/asyncio/coroutines.py --- a/Lib/asyncio/coroutines.py +++ b/Lib/asyncio/coroutines.py @@ -204,8 +204,8 @@ @functools.wraps(func) def coro(*args, **kw): res = func(*args, **kw) - if isinstance(res, futures.Future) or inspect.isgenerator(res) or \ - isinstance(res, CoroWrapper): + if (futures.isfuture(res) or inspect.isgenerator(res) or + isinstance(res, CoroWrapper)): res = yield from res elif _AwaitableABC is not None: # If 'func' returns an Awaitable (new in 3.5) we diff --git a/Lib/asyncio/futures.py b/Lib/asyncio/futures.py --- a/Lib/asyncio/futures.py +++ b/Lib/asyncio/futures.py @@ -110,6 +110,16 @@ self.loop.call_exception_handler({'message': msg}) +def isfuture(obj): + """Check for a Future. + + This returns True when obj is a Future instance or is advertising + itself as duck-type compatible by setting _asyncio_future_blocking. + See comment in Future for more details. + """ + return getattr(obj, '_asyncio_future_blocking', None) is not None + + class Future: """This class is *almost* compatible with concurrent.futures.Future. @@ -423,15 +433,17 @@ If destination is cancelled, source gets cancelled too. Compatible with both asyncio.Future and concurrent.futures.Future. """ - if not isinstance(source, (Future, concurrent.futures.Future)): + if not isfuture(source) and not isinstance(source, + concurrent.futures.Future): raise TypeError('A future is required for source argument') - if not isinstance(destination, (Future, concurrent.futures.Future)): + if not isfuture(destination) and not isinstance(destination, + concurrent.futures.Future): raise TypeError('A future is required for destination argument') - source_loop = source._loop if isinstance(source, Future) else None - dest_loop = destination._loop if isinstance(destination, Future) else None + source_loop = source._loop if isfuture(source) else None + dest_loop = destination._loop if isfuture(destination) else None def _set_state(future, other): - if isinstance(future, Future): + if isfuture(future): _copy_future_state(other, future) else: _set_concurrent_future_state(future, other) @@ -455,7 +467,7 @@ def wrap_future(future, *, loop=None): """Wrap concurrent.futures.Future object.""" - if isinstance(future, Future): + if isfuture(future): return future assert isinstance(future, concurrent.futures.Future), \ 'concurrent.futures.Future is expected, got {!r}'.format(future) diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -333,7 +333,7 @@ Note: This does not raise TimeoutError! Futures that aren't done when the timeout occurs are returned in the second set. """ - if isinstance(fs, futures.Future) or coroutines.iscoroutine(fs): + if futures.isfuture(fs) or coroutines.iscoroutine(fs): raise TypeError("expect a list of futures, not %s" % type(fs).__name__) if not fs: raise ValueError('Set of coroutines/Futures is empty.') @@ -462,7 +462,7 @@ Note: The futures 'f' are not necessarily members of fs. """ - if isinstance(fs, futures.Future) or coroutines.iscoroutine(fs): + if futures.isfuture(fs) or coroutines.iscoroutine(fs): raise TypeError("expect a list of futures, not %s" % type(fs).__name__) loop = loop if loop is not None else events.get_event_loop() todo = {ensure_future(f, loop=loop) for f in set(fs)} @@ -538,7 +538,7 @@ If the argument is a Future, it is returned directly. """ - if isinstance(coro_or_future, futures.Future): + if futures.isfuture(coro_or_future): if loop is not None and loop is not coro_or_future._loop: raise ValueError('loop argument must agree with Future') return coro_or_future @@ -614,7 +614,7 @@ arg_to_fut = {} for arg in set(coros_or_futures): - if not isinstance(arg, futures.Future): + if not futures.isfuture(arg): fut = ensure_future(arg, loop=loop) if loop is None: loop = fut._loop diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -793,7 +793,7 @@ loop.connect_accepted_socket( (lambda : proto), conn, ssl=server_ssl)) loop.run_forever() - conn.close() + proto.transport.close() lsock.close() thread.join(1) diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py --- a/Lib/test/test_asyncio/test_futures.py +++ b/Lib/test/test_asyncio/test_futures.py @@ -25,6 +25,74 @@ pass +class DuckFuture: + # Class that does not inherit from Future but aims to be duck-type + # compatible with it. + + _asyncio_future_blocking = False + __cancelled = False + __result = None + __exception = None + + def cancel(self): + if self.done(): + return False + self.__cancelled = True + return True + + def cancelled(self): + return self.__cancelled + + def done(self): + return (self.__cancelled + or self.__result is not None + or self.__exception is not None) + + def result(self): + assert not self.cancelled() + if self.__exception is not None: + raise self.__exception + return self.__result + + def exception(self): + assert not self.cancelled() + return self.__exception + + def set_result(self, result): + assert not self.done() + assert result is not None + self.__result = result + + def set_exception(self, exception): + assert not self.done() + assert exception is not None + self.__exception = exception + + def __iter__(self): + if not self.done(): + self._asyncio_future_blocking = True + yield self + assert self.done() + return self.result() + + +class DuckTests(test_utils.TestCase): + + def setUp(self): + self.loop = self.new_test_loop() + self.addCleanup(self.loop.close) + + def test_wrap_future(self): + f = DuckFuture() + g = asyncio.wrap_future(f) + assert g is f + + def test_ensure_future(self): + f = DuckFuture() + g = asyncio.ensure_future(f) + assert g is f + + class FutureTests(test_utils.TestCase): def setUp(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 19:03:27 2016 From: python-checkins at python.org (davin.potts) Date: Fri, 09 Sep 2016 23:03:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328053=3A_Applying?= =?utf-8?q?_refactorings=2C_docs_and_other_cleanup_to_follow=2E?= Message-ID: <20160909230327.87626.25243.BEB17E3C@psf.io> https://hg.python.org/cpython/rev/7381b1b50e00 changeset: 103497:7381b1b50e00 user: Davin Potts date: Fri Sep 09 18:03:10 2016 -0500 summary: Issue #28053: Applying refactorings, docs and other cleanup to follow. files: Lib/multiprocessing/connection.py | 8 +- Lib/multiprocessing/context.py | 13 +++- Lib/multiprocessing/forkserver.py | 2 +- Lib/multiprocessing/heap.py | 5 +- Lib/multiprocessing/managers.py | 5 +- Lib/multiprocessing/popen_forkserver.py | 7 +- Lib/multiprocessing/popen_spawn_posix.py | 7 +- Lib/multiprocessing/popen_spawn_win32.py | 9 +- Lib/multiprocessing/queues.py | 10 +- Lib/multiprocessing/reduction.py | 34 ++++++++++++ Lib/multiprocessing/resource_sharer.py | 2 +- Lib/multiprocessing/sharedctypes.py | 6 +- Lib/multiprocessing/spawn.py | 9 +- 13 files changed, 77 insertions(+), 40 deletions(-) diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py --- a/Lib/multiprocessing/connection.py +++ b/Lib/multiprocessing/connection.py @@ -20,11 +20,11 @@ import _multiprocessing -from . import reduction from . import util from . import AuthenticationError, BufferTooShort -from .reduction import ForkingPickler +from .context import reduction +_ForkingPickler = reduction.ForkingPickler try: import _winapi @@ -203,7 +203,7 @@ """Send a (picklable) object""" self._check_closed() self._check_writable() - self._send_bytes(ForkingPickler.dumps(obj)) + self._send_bytes(_ForkingPickler.dumps(obj)) def recv_bytes(self, maxlength=None): """ @@ -248,7 +248,7 @@ self._check_closed() self._check_readable() buf = self._recv_bytes() - return ForkingPickler.loads(buf.getbuffer()) + return _ForkingPickler.loads(buf.getbuffer()) def poll(self, timeout=0.0): """Whether there is any input available to be read""" diff --git a/Lib/multiprocessing/context.py b/Lib/multiprocessing/context.py --- a/Lib/multiprocessing/context.py +++ b/Lib/multiprocessing/context.py @@ -3,6 +3,7 @@ import threading from . import process +from . import reduction __all__ = [] # things are copied from here to __init__.py @@ -198,6 +199,16 @@ def set_start_method(self, method=None): raise ValueError('cannot set start method of concrete context') + @property + def reducer(self): + '''Controls how objects will be reduced to a form that can be + shared with other processes.''' + return globals().get('reduction') + + @reducer.setter + def reducer(self, reduction): + globals()['reduction'] = reduction + def _check_available(self): pass @@ -245,7 +256,6 @@ if sys.platform == 'win32': return ['spawn'] else: - from . import reduction if reduction.HAVE_SEND_HANDLE: return ['fork', 'spawn', 'forkserver'] else: @@ -292,7 +302,6 @@ _name = 'forkserver' Process = ForkServerProcess def _check_available(self): - from . import reduction if not reduction.HAVE_SEND_HANDLE: raise ValueError('forkserver start method not available') diff --git a/Lib/multiprocessing/forkserver.py b/Lib/multiprocessing/forkserver.py --- a/Lib/multiprocessing/forkserver.py +++ b/Lib/multiprocessing/forkserver.py @@ -9,7 +9,7 @@ from . import connection from . import process -from . import reduction +from .context import reduction from . import semaphore_tracker from . import spawn from . import util diff --git a/Lib/multiprocessing/heap.py b/Lib/multiprocessing/heap.py --- a/Lib/multiprocessing/heap.py +++ b/Lib/multiprocessing/heap.py @@ -14,8 +14,7 @@ import tempfile import threading -from . import context -from . import reduction +from .context import reduction, assert_spawning from . import util __all__ = ['BufferWrapper'] @@ -48,7 +47,7 @@ self._state = (self.size, self.name) def __getstate__(self): - context.assert_spawning(self) + assert_spawning(self) return self._state def __setstate__(self, state): diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -23,10 +23,9 @@ from traceback import format_exc from . import connection -from . import context +from .context import reduction, get_spawning_popen from . import pool from . import process -from . import reduction from . import util from . import get_context @@ -833,7 +832,7 @@ def __reduce__(self): kwds = {} - if context.get_spawning_popen() is not None: + if get_spawning_popen() is not None: kwds['authkey'] = self._authkey if getattr(self, '_isauto', False): diff --git a/Lib/multiprocessing/popen_forkserver.py b/Lib/multiprocessing/popen_forkserver.py --- a/Lib/multiprocessing/popen_forkserver.py +++ b/Lib/multiprocessing/popen_forkserver.py @@ -1,10 +1,9 @@ import io import os -from . import reduction +from .context import reduction, set_spawning_popen if not reduction.HAVE_SEND_HANDLE: raise ImportError('No support for sending fds between processes') -from . import context from . import forkserver from . import popen_fork from . import spawn @@ -42,12 +41,12 @@ def _launch(self, process_obj): prep_data = spawn.get_preparation_data(process_obj._name) buf = io.BytesIO() - context.set_spawning_popen(self) + set_spawning_popen(self) try: reduction.dump(prep_data, buf) reduction.dump(process_obj, buf) finally: - context.set_spawning_popen(None) + set_spawning_popen(None) self.sentinel, w = forkserver.connect_to_new_process(self._fds) util.Finalize(self, os.close, (self.sentinel,)) diff --git a/Lib/multiprocessing/popen_spawn_posix.py b/Lib/multiprocessing/popen_spawn_posix.py --- a/Lib/multiprocessing/popen_spawn_posix.py +++ b/Lib/multiprocessing/popen_spawn_posix.py @@ -1,9 +1,8 @@ import io import os -from . import context +from .context import reduction, set_spawning_popen from . import popen_fork -from . import reduction from . import spawn from . import util @@ -42,12 +41,12 @@ self._fds.append(tracker_fd) prep_data = spawn.get_preparation_data(process_obj._name) fp = io.BytesIO() - context.set_spawning_popen(self) + set_spawning_popen(self) try: reduction.dump(prep_data, fp) reduction.dump(process_obj, fp) finally: - context.set_spawning_popen(None) + set_spawning_popen(None) parent_r = child_w = child_r = parent_w = None try: diff --git a/Lib/multiprocessing/popen_spawn_win32.py b/Lib/multiprocessing/popen_spawn_win32.py --- a/Lib/multiprocessing/popen_spawn_win32.py +++ b/Lib/multiprocessing/popen_spawn_win32.py @@ -4,9 +4,8 @@ import sys import _winapi -from . import context +from .context import reduction, get_spawning_popen, set_spawning_popen from . import spawn -from . import reduction from . import util __all__ = ['Popen'] @@ -60,15 +59,15 @@ util.Finalize(self, _winapi.CloseHandle, (self.sentinel,)) # send information to child - context.set_spawning_popen(self) + set_spawning_popen(self) try: reduction.dump(prep_data, to_child) reduction.dump(process_obj, to_child) finally: - context.set_spawning_popen(None) + set_spawning_popen(None) def duplicate_for_child(self, handle): - assert self is context.get_spawning_popen() + assert self is get_spawning_popen() return reduction.duplicate(handle, self.sentinel) def wait(self, timeout=None): diff --git a/Lib/multiprocessing/queues.py b/Lib/multiprocessing/queues.py --- a/Lib/multiprocessing/queues.py +++ b/Lib/multiprocessing/queues.py @@ -23,9 +23,9 @@ from . import connection from . import context +_ForkingPickler = context.reduction.ForkingPickler from .util import debug, info, Finalize, register_after_fork, is_exiting -from .reduction import ForkingPickler # # Queue type using a pipe, buffer and thread @@ -110,7 +110,7 @@ finally: self._rlock.release() # unserialize the data after having released the lock - return ForkingPickler.loads(res) + return _ForkingPickler.loads(res) def qsize(self): # Raises NotImplementedError on Mac OSX because of broken sem_getvalue() @@ -238,7 +238,7 @@ return # serialize the data before acquiring the lock - obj = ForkingPickler.dumps(obj) + obj = _ForkingPickler.dumps(obj) if wacquire is None: send_bytes(obj) else: @@ -342,11 +342,11 @@ with self._rlock: res = self._reader.recv_bytes() # unserialize the data after having released the lock - return ForkingPickler.loads(res) + return _ForkingPickler.loads(res) def put(self, obj): # serialize the data before acquiring the lock - obj = ForkingPickler.dumps(obj) + obj = _ForkingPickler.dumps(obj) if self._wlock is None: # writes to a message oriented win32 pipe are atomic self._writer.send_bytes(obj) diff --git a/Lib/multiprocessing/reduction.py b/Lib/multiprocessing/reduction.py --- a/Lib/multiprocessing/reduction.py +++ b/Lib/multiprocessing/reduction.py @@ -7,6 +7,7 @@ # Licensed to PSF under a Contributor Agreement. # +from abc import ABCMeta, abstractmethod import copyreg import functools import io @@ -238,3 +239,36 @@ fd = df.detach() return socket.socket(family, type, proto, fileno=fd) register(socket.socket, _reduce_socket) + + +class AbstractReducer(metaclass=ABCMeta): + '''Abstract base class for use in implementing a Reduction class + suitable for use in replacing the standard reduction mechanism + used in multiprocessing.''' + ForkingPickler = ForkingPickler + register = register + dump = dump + send_handle = send_handle + recv_handle = recv_handle + + if sys.platform == 'win32': + steal_handle = steal_handle + duplicate = duplicate + DupHandle = DupHandle + else: + sendfds = sendfds + recvfds = recvfds + DupFd = DupFd + + _reduce_method = _reduce_method + _reduce_method_descriptor = _reduce_method_descriptor + _rebuild_partial = _rebuild_partial + _reduce_socket = _reduce_socket + _rebuild_socket = _rebuild_socket + + def __init__(self, *args): + register(type(_C().f), _reduce_method) + register(type(list.append), _reduce_method_descriptor) + register(type(int.__add__), _reduce_method_descriptor) + register(functools.partial, _reduce_partial) + register(socket.socket, _reduce_socket) diff --git a/Lib/multiprocessing/resource_sharer.py b/Lib/multiprocessing/resource_sharer.py --- a/Lib/multiprocessing/resource_sharer.py +++ b/Lib/multiprocessing/resource_sharer.py @@ -15,7 +15,7 @@ import threading from . import process -from . import reduction +from .context import reduction from . import util __all__ = ['stop'] diff --git a/Lib/multiprocessing/sharedctypes.py b/Lib/multiprocessing/sharedctypes.py --- a/Lib/multiprocessing/sharedctypes.py +++ b/Lib/multiprocessing/sharedctypes.py @@ -13,8 +13,8 @@ from . import heap from . import get_context -from .context import assert_spawning -from .reduction import ForkingPickler +from .context import reduction, assert_spawning +_ForkingPickler = reduction.ForkingPickler __all__ = ['RawValue', 'RawArray', 'Value', 'Array', 'copy', 'synchronized'] @@ -134,7 +134,7 @@ def rebuild_ctype(type_, wrapper, length): if length is not None: type_ = type_ * length - ForkingPickler.register(type_, reduce_ctype) + _ForkingPickler.register(type_, reduce_ctype) buf = wrapper.create_memoryview() obj = type_.from_buffer(buf) obj._wrapper = wrapper diff --git a/Lib/multiprocessing/spawn.py b/Lib/multiprocessing/spawn.py --- a/Lib/multiprocessing/spawn.py +++ b/Lib/multiprocessing/spawn.py @@ -9,13 +9,13 @@ # import os -import pickle import sys import runpy import types from . import get_start_method, set_start_method from . import process +from .context import reduction from . import util __all__ = ['_main', 'freeze_support', 'set_executable', 'get_executable', @@ -96,8 +96,7 @@ assert is_forking(sys.argv) if sys.platform == 'win32': import msvcrt - from .reduction import steal_handle - new_handle = steal_handle(parent_pid, pipe_handle) + new_handle = reduction.steal_handle(parent_pid, pipe_handle) fd = msvcrt.open_osfhandle(new_handle, os.O_RDONLY) else: from . import semaphore_tracker @@ -111,9 +110,9 @@ with os.fdopen(fd, 'rb', closefd=True) as from_parent: process.current_process()._inheriting = True try: - preparation_data = pickle.load(from_parent) + preparation_data = reduction.pickle.load(from_parent) prepare(preparation_data) - self = pickle.load(from_parent) + self = reduction.pickle.load(from_parent) finally: del process.current_process()._inheriting return self._bootstrap() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 19:15:13 2016 From: python-checkins at python.org (zach.ware) Date: Fri, 09 Sep 2016 23:15:13 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Rename_test=5Fstrlit_-=3E_?= =?utf-8?q?test=5Fstring=5Fliterals?= Message-ID: <20160909231512.2591.50476.E7B150CE@psf.io> https://hg.python.org/cpython/rev/741a87d212da changeset: 103498:741a87d212da user: Zachary Ware date: Fri Sep 09 16:15:03 2016 -0700 summary: Rename test_strlit -> test_string_literals files: Lib/test/test_string_literals.py | 0 1 files changed, 0 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_strlit.py b/Lib/test/test_string_literals.py rename from Lib/test/test_strlit.py rename to Lib/test/test_string_literals.py -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 19:24:33 2016 From: python-checkins at python.org (raymond.hettinger) Date: Fri, 09 Sep 2016 23:24:33 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIyNDUw?= =?utf-8?q?=3A_Use_=22Accept=3A_*/*=22_in_the_default_headers_for_urllib?= Message-ID: <20160909232433.21095.56628.25306B46@psf.io> https://hg.python.org/cpython/rev/e84105b48436 changeset: 103499:e84105b48436 branch: 2.7 parent: 103443:951f0de11a01 user: Raymond Hettinger date: Fri Sep 09 16:23:06 2016 -0700 summary: Issue #22450: Use "Accept: */*" in the default headers for urllib files: Lib/urllib.py | 2 +- Misc/NEWS | 4 ++++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/urllib.py b/Lib/urllib.py --- a/Lib/urllib.py +++ b/Lib/urllib.py @@ -138,7 +138,7 @@ self.key_file = x509.get('key_file') self.cert_file = x509.get('cert_file') self.context = context - self.addheaders = [('User-Agent', self.version)] + self.addheaders = [('User-Agent', self.version), ('Accept', '*/*')] self.__tempfiles = [] self.__unlink = os.unlink # See cleanup() self.tempcache = None diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -47,6 +47,10 @@ - Issue #27570: Avoid zero-length memcpy() etc calls with null source pointers in the "ctypes" and "array" modules. +- Issue #22450: urllib now includes an "Accept: */*" header among the + default headers. This makes the results of REST API requests more + consistent and predictable especially when proxy servers are involved. + - lib2to3.pgen3.driver.load_grammar() now creates a stable cache file between runs given the same Grammar.txt input regardless of the hash randomization setting. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 19:39:56 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 23:39:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325144=3A_Ensures_TargetDir_is_set_before_contin?= =?utf-8?q?uing_with_custom_install=2E?= Message-ID: <20160909233956.1774.65148.8719E36C@psf.io> https://hg.python.org/cpython/rev/024f3312ccee changeset: 103501:024f3312ccee parent: 103498:741a87d212da parent: 103500:16d652760a06 user: Steve Dower date: Fri Sep 09 16:39:36 2016 -0700 summary: Issue #25144: Ensures TargetDir is set before continuing with custom install. files: Misc/NEWS | 3 +++ Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 3 +++ 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -333,6 +333,9 @@ Windows ------- +- Issue #25144: Ensures TargetDir is set before continuing with custom + install. + - Issue #1602: Windows console doesn't input or print Unicode (PEP 528) - Issue #27781: Change file system encoding on Windows to UTF-8 (PEP 529) diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -323,6 +323,9 @@ case ID_CUSTOM_INSTALL_BUTTON: SavePageSettings(); + hr = EnsureTargetDir(); + ExitOnFailure(hr, L"Failed to set TargetDir"); + hr = BalGetStringVariable(L"TargetDir", &targetDir); if (SUCCEEDED(hr)) { // TODO: Check whether directory exists and contains another installation -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 19:39:56 2016 From: python-checkins at python.org (steve.dower) Date: Fri, 09 Sep 2016 23:39:56 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MTQ0?= =?utf-8?q?=3A_Ensures_TargetDir_is_set_before_continuing_with_custom_inst?= =?utf-8?q?all=2E?= Message-ID: <20160909233956.48560.11852.1C915671@psf.io> https://hg.python.org/cpython/rev/16d652760a06 changeset: 103500:16d652760a06 branch: 3.5 parent: 103495:8c00cbbd3ff9 user: Steve Dower date: Fri Sep 09 16:37:53 2016 -0700 summary: Issue #25144: Ensures TargetDir is set before continuing with custom install. files: Misc/NEWS | 3 +++ Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp | 3 +++ 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -275,6 +275,9 @@ Windows ------- +- Issue #25144: Ensures TargetDir is set before continuing with custom + install. + - Issue #27469: Adds a shell extension to the launcher so that drag and drop works correctly. diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -327,6 +327,9 @@ case ID_CUSTOM_INSTALL_BUTTON: SavePageSettings(); + hr = EnsureTargetDir(); + ExitOnFailure(hr, L"Failed to set TargetDir"); + hr = BalGetStringVariable(L"TargetDir", &targetDir); if (SUCCEEDED(hr)) { // TODO: Check whether directory exists and contains another installation -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 19:45:23 2016 From: python-checkins at python.org (raymond.hettinger) Date: Fri, 09 Sep 2016 23:45:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIyNDUw?= =?utf-8?q?=3A_Use_=22Accept=3A_*/*=22_in_the_default_headers_for_urllib?= =?utf-8?q?=2Erequest?= Message-ID: <20160909234523.1920.38591.1A0B0CD0@psf.io> https://hg.python.org/cpython/rev/00da8bfa2a60 changeset: 103502:00da8bfa2a60 branch: 3.5 parent: 103500:16d652760a06 user: Raymond Hettinger date: Fri Sep 09 16:43:48 2016 -0700 summary: Issue #22450: Use "Accept: */*" in the default headers for urllib.request files: Lib/urllib/request.py | 2 +- Misc/NEWS | 4 ++++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -1625,7 +1625,7 @@ self.proxies = proxies self.key_file = x509.get('key_file') self.cert_file = x509.get('cert_file') - self.addheaders = [('User-Agent', self.version)] + self.addheaders = [('User-Agent', self.version), ('Accept', '*/*')] self.__tempfiles = [] self.__unlink = os.unlink # See cleanup() self.tempcache = None diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -70,6 +70,10 @@ - Issue #27445: Don't pass str(_charset) to MIMEText.set_payload(). Patch by Claude Paroz. +- Issue #22450: urllib now includes an "Accept: */*" header among the + default headers. This makes the results of REST API requests more + consistent and predictable especially when proxy servers are involved. + - lib2to3.pgen3.driver.load_grammar() now creates a stable cache file between runs given the same Grammar.txt input regardless of the hash randomization setting. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 19:45:24 2016 From: python-checkins at python.org (raymond.hettinger) Date: Fri, 09 Sep 2016 23:45:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Merge?= Message-ID: <20160909234523.87704.40663.3C3E03B5@psf.io> https://hg.python.org/cpython/rev/7f36c432bfae changeset: 103503:7f36c432bfae parent: 103501:024f3312ccee user: Raymond Hettinger date: Fri Sep 09 16:44:53 2016 -0700 summary: Merge files: Lib/urllib/request.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -1686,7 +1686,7 @@ self.proxies = proxies self.key_file = x509.get('key_file') self.cert_file = x509.get('cert_file') - self.addheaders = [('User-Agent', self.version)] + self.addheaders = [('User-Agent', self.version), ('Accept', '*/*')] self.__tempfiles = [] self.__unlink = os.unlink # See cleanup() self.tempcache = None -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:01:33 2016 From: python-checkins at python.org (zach.ware) Date: Sat, 10 Sep 2016 00:01:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2327976=3A_Depreca?= =?utf-8?q?te_bundled_full_copy_of_libffi?= Message-ID: <20160910000132.59726.47993.1197F727@psf.io> https://hg.python.org/cpython/rev/3e02187e50df changeset: 103504:3e02187e50df user: Zachary Ware date: Fri Sep 09 17:01:21 2016 -0700 summary: Closes #27976: Deprecate bundled full copy of libffi Builds on non-OSX UNIX now default to using the system libffi, and warn if the bundled copy is used. files: Doc/whatsnew/3.6.rst | 10 ++++++++++ Misc/NEWS | 3 +++ configure | 24 ++++++++++++++++++++---- configure.ac | 22 +++++++++++++++++++--- setup.py | 10 +++++++--- 5 files changed, 59 insertions(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1010,6 +1010,16 @@ Deprecated ========== +Deprecated Build Options +------------------------ + +The ``--with-system-ffi`` configure flag is now on by default on non-OSX UNIX +platforms. It may be disabled by using ``--without-system-ffi``, but using the +flag is deprecated and will not be accepted in Python 3.7. OSX is unaffected +by this change. Note that many OS distributors already use the +``--with-system-ffi`` flag when building their system Python. + + New Keywords ------------ diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -311,6 +311,9 @@ Build ----- +- Issue #27976: Deprecate building _ctypes with the bundled copy of libffi on + non-OSX UNIX platforms. + - Issue #27983: Cause lack of llvm-profdata tool when using clang as required for PGO linking to be a configure time error rather than make time when --with-optimizations is enabled. Also improve our diff --git a/configure b/configure --- a/configure +++ b/configure @@ -9851,10 +9851,26 @@ # Check whether --with-system_ffi was given. if test "${with_system_ffi+set}" = set; then : withval=$with_system_ffi; -else - with_system_ffi="no" -fi - +fi + + +case "$with_system_ffi" in + "") + case $ac_sys_system in + Darwin) + with_system_ffi="no" + ;; + *) + with_system_ffi="yes" + ;; + esac + ;; + yes|no) + ;; + *) + as_fn_error $? "--with-system-ffi accepts no arguments" "$LINENO" 5 + ;; +esac if test "$with_system_ffi" = "yes" && test -n "$PKG_CONFIG"; then LIBFFI_INCLUDEDIR="`"$PKG_CONFIG" libffi --cflags-only-I 2>/dev/null | sed -e 's/^-I//;s/ *$//'`" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -2737,9 +2737,25 @@ # Check for use of the system libffi library AC_MSG_CHECKING(for --with-system-ffi) AC_ARG_WITH(system_ffi, - AS_HELP_STRING([--with-system-ffi], [build _ctypes module using an installed ffi library]), - [], - [with_system_ffi="no"]) + AS_HELP_STRING([--with-system-ffi], [build _ctypes module using an installed ffi library]),,,) + +case "$with_system_ffi" in + "") + case $ac_sys_system in + Darwin) + with_system_ffi="no" + ;; + *) + with_system_ffi="yes" + ;; + esac + ;; + yes|no) + ;; + *) + AC_MSG_ERROR([--with-system-ffi accepts no arguments]) + ;; +esac if test "$with_system_ffi" = "yes" && test -n "$PKG_CONFIG"; then LIBFFI_INCLUDEDIR="`"$PKG_CONFIG" libffi --cflags-only-I 2>/dev/null | sed -e 's/^-I//;s/ *$//'`" diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -1911,6 +1911,9 @@ if host_platform == 'darwin': return self.configure_ctypes_darwin(ext) + print('warning: building with the bundled copy of libffi is' + ' deprecated on this platform. It will not be' + ' distributed with Python 3.7') srcdir = sysconfig.get_config_var('srcdir') ffi_builddir = os.path.join(self.build_temp, 'libffi') ffi_srcdir = os.path.abspath(os.path.join(srcdir, 'Modules', @@ -2007,13 +2010,14 @@ libraries=math_libs) self.extensions.extend([ext, ext_test]) - if not '--with-system-ffi' in sysconfig.get_config_var("CONFIG_ARGS"): - return - if host_platform == 'darwin': + if '--with-system-ffi' not in sysconfig.get_config_var("CONFIG_ARGS"): + return # OS X 10.5 comes with libffi.dylib; the include files are # in /usr/include/ffi inc_dirs.append('/usr/include/ffi') + elif '--without-system-ffi' in sysconfig.get_config_var("CONFIG_ARGS"): + return ffi_inc = [sysconfig.get_config_var("LIBFFI_INCLUDEDIR")] if not ffi_inc or ffi_inc[0] == '': -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:04:08 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 10 Sep 2016 00:04:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_2to3=27s_fix=5Fcall?= =?utf-8?q?able=2E=2E=2E_We_reintroduced_the_callable_built-in?= Message-ID: <20160910000408.13637.80202.8CA57AE3@psf.io> https://hg.python.org/cpython/rev/c89692446258 changeset: 103505:c89692446258 user: Gregory P. Smith date: Fri Sep 09 17:03:58 2016 -0700 summary: Remove 2to3's fix_callable... We reintroduced the callable built-in pretty early on in the 3.x series (3.1 or 3.2?). files: Lib/lib2to3/fixes/fix_callable.py | 37 ------- Lib/lib2to3/tests/test_fixers.py | 92 ------------------- 2 files changed, 0 insertions(+), 129 deletions(-) diff --git a/Lib/lib2to3/fixes/fix_callable.py b/Lib/lib2to3/fixes/fix_callable.py deleted file mode 100644 --- a/Lib/lib2to3/fixes/fix_callable.py +++ /dev/null @@ -1,37 +0,0 @@ -# Copyright 2007 Google, Inc. All Rights Reserved. -# Licensed to PSF under a Contributor Agreement. - -"""Fixer for callable(). - -This converts callable(obj) into isinstance(obj, collections.Callable), adding a -collections import if needed.""" - -# Local imports -from lib2to3 import fixer_base -from lib2to3.fixer_util import Call, Name, String, Attr, touch_import - -class FixCallable(fixer_base.BaseFix): - BM_compatible = True - - order = "pre" - - # Ignore callable(*args) or use of keywords. - # Either could be a hint that the builtin callable() is not being used. - PATTERN = """ - power< 'callable' - trailer< lpar='(' - ( not(arglist | argument) any ','> ) - rpar=')' > - after=any* - > - """ - - def transform(self, node, results): - func = results['func'] - - touch_import(None, 'collections', node=node) - - args = [func.clone(), String(', ')] - args.extend(Attr(Name('collections'), Name('Callable'))) - return Call(Name('isinstance'), args, prefix=node.prefix) diff --git a/Lib/lib2to3/tests/test_fixers.py b/Lib/lib2to3/tests/test_fixers.py --- a/Lib/lib2to3/tests/test_fixers.py +++ b/Lib/lib2to3/tests/test_fixers.py @@ -2919,98 +2919,6 @@ a = f + r"""r'\\\u20ac\U0001d121\\u20ac'""" self.check(b, a) -class Test_callable(FixerTestCase): - fixer = "callable" - - def test_prefix_preservation(self): - b = """callable( x)""" - a = """import collections\nisinstance( x, collections.Callable)""" - self.check(b, a) - - b = """if callable(x): pass""" - a = """import collections -if isinstance(x, collections.Callable): pass""" - self.check(b, a) - - def test_callable_call(self): - b = """callable(x)""" - a = """import collections\nisinstance(x, collections.Callable)""" - self.check(b, a) - - def test_global_import(self): - b = """ -def spam(foo): - callable(foo)"""[1:] - a = """ -import collections -def spam(foo): - isinstance(foo, collections.Callable)"""[1:] - self.check(b, a) - - b = """ -import collections -def spam(foo): - callable(foo)"""[1:] - # same output if it was already imported - self.check(b, a) - - b = """ -from collections import * -def spam(foo): - callable(foo)"""[1:] - a = """ -from collections import * -import collections -def spam(foo): - isinstance(foo, collections.Callable)"""[1:] - self.check(b, a) - - b = """ -do_stuff() -do_some_other_stuff() -assert callable(do_stuff)"""[1:] - a = """ -import collections -do_stuff() -do_some_other_stuff() -assert isinstance(do_stuff, collections.Callable)"""[1:] - self.check(b, a) - - b = """ -if isinstance(do_stuff, Callable): - assert callable(do_stuff) - do_stuff(do_stuff) - if not callable(do_stuff): - exit(1) - else: - assert callable(do_stuff) -else: - assert not callable(do_stuff)"""[1:] - a = """ -import collections -if isinstance(do_stuff, Callable): - assert isinstance(do_stuff, collections.Callable) - do_stuff(do_stuff) - if not isinstance(do_stuff, collections.Callable): - exit(1) - else: - assert isinstance(do_stuff, collections.Callable) -else: - assert not isinstance(do_stuff, collections.Callable)"""[1:] - self.check(b, a) - - def test_callable_should_not_change(self): - a = """callable(*x)""" - self.unchanged(a) - - a = """callable(x, y)""" - self.unchanged(a) - - a = """callable(x, kw=y)""" - self.unchanged(a) - - a = """callable()""" - self.unchanged(a) class Test_filter(FixerTestCase): fixer = "filter" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:09:58 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 10 Sep 2016 00:09:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge=3A_=2314977=3A_Make_mailcap_respect_the_order_of_t?= =?utf-8?q?he_lines_in_the_mailcap_file=2E?= Message-ID: <20160910000957.19326.5175.376E46BA@psf.io> https://hg.python.org/cpython/rev/efd692c86429 changeset: 103507:efd692c86429 parent: 103505:c89692446258 parent: 103506:f1bf0abcca0c user: R David Murray date: Fri Sep 09 20:09:43 2016 -0400 summary: Merge: #14977: Make mailcap respect the order of the lines in the mailcap file. files: Lib/mailcap.py | 28 ++++++++++- Lib/test/mailcap.txt | 2 +- Lib/test/test_mailcap.py | 69 ++++++++++++++++++--------- Misc/ACKS | 1 + Misc/NEWS | 7 ++ 5 files changed, 80 insertions(+), 27 deletions(-) diff --git a/Lib/mailcap.py b/Lib/mailcap.py --- a/Lib/mailcap.py +++ b/Lib/mailcap.py @@ -1,9 +1,19 @@ """Mailcap file handling. See RFC 1524.""" import os +import warnings __all__ = ["getcaps","findmatch"] + +def lineno_sort_key(entry): + # Sort in ascending order, with unspecified entries at the end + if 'lineno' in entry: + return 0, entry['lineno'] + else: + return 1, 0 + + # Part 1: top-level interface. def getcaps(): @@ -17,13 +27,14 @@ """ caps = {} + lineno = 0 for mailcap in listmailcapfiles(): try: fp = open(mailcap, 'r') except OSError: continue with fp: - morecaps = readmailcapfile(fp) + morecaps, lineno = _readmailcapfile(fp, lineno) for key, value in morecaps.items(): if not key in caps: caps[key] = value @@ -49,8 +60,15 @@ # Part 2: the parser. +def readmailcapfile(fp): + """Read a mailcap file and return a dictionary keyed by MIME type.""" + warnings.warn('readmailcapfile is deprecated, use getcaps instead', + DeprecationWarning, 2) + caps, _ = _readmailcapfile(fp, None) + return caps -def readmailcapfile(fp): + +def _readmailcapfile(fp, lineno): """Read a mailcap file and return a dictionary keyed by MIME type. Each MIME type is mapped to an entry consisting of a list of @@ -76,6 +94,9 @@ key, fields = parseline(line) if not (key and fields): continue + if lineno is not None: + fields['lineno'] = lineno + lineno += 1 # Normalize the key types = key.split('/') for j in range(len(types)): @@ -86,7 +107,7 @@ caps[key].append(fields) else: caps[key] = [fields] - return caps + return caps, lineno def parseline(line): """Parse one entry in a mailcap file and return a dictionary. @@ -165,6 +186,7 @@ entries = entries + caps[MIMEtype] if key is not None: entries = [e for e in entries if key in e] + entries = sorted(entries, key=lineno_sort_key) return entries def subst(field, MIMEtype, filename, plist=[]): diff --git a/Lib/test/mailcap.txt b/Lib/test/mailcap.txt --- a/Lib/test/mailcap.txt +++ b/Lib/test/mailcap.txt @@ -35,5 +35,5 @@ text/richtext; shownonascii iso-8859-8 -e richtext -p %s; test=test "`echo \ %{charset} | tr '[A-Z]' '[a-z]'`" = iso-8859-8; copiousoutput -video/mpeg; mpeg_play %s video/*; animate %s +video/mpeg; mpeg_play %s \ No newline at end of file diff --git a/Lib/test/test_mailcap.py b/Lib/test/test_mailcap.py --- a/Lib/test/test_mailcap.py +++ b/Lib/test/test_mailcap.py @@ -1,5 +1,6 @@ import mailcap import os +import copy import test.support import unittest @@ -13,43 +14,55 @@ [{'compose': 'moviemaker %s', 'x11-bitmap': '"/usr/lib/Zmail/bitmaps/movie.xbm"', 'description': '"Movie"', - 'view': 'movieplayer %s'}], + 'view': 'movieplayer %s', + 'lineno': 4}], 'application/*': [{'copiousoutput': '', - 'view': 'echo "This is \\"%t\\" but is 50 \\% Greek to me" \\; cat %s'}], + 'view': 'echo "This is \\"%t\\" but is 50 \\% Greek to me" \\; cat %s', + 'lineno': 5}], 'audio/basic': [{'edit': 'audiocompose %s', 'compose': 'audiocompose %s', 'description': '"An audio fragment"', - 'view': 'showaudio %s'}], + 'view': 'showaudio %s', + 'lineno': 6}], 'video/mpeg': - [{'view': 'mpeg_play %s'}], + [{'view': 'mpeg_play %s', 'lineno': 13}], 'application/postscript': - [{'needsterminal': '', 'view': 'ps-to-terminal %s'}, - {'compose': 'idraw %s', 'view': 'ps-to-terminal %s'}], + [{'needsterminal': '', 'view': 'ps-to-terminal %s', 'lineno': 1}, + {'compose': 'idraw %s', 'view': 'ps-to-terminal %s', 'lineno': 2}], 'application/x-dvi': - [{'view': 'xdvi %s'}], + [{'view': 'xdvi %s', 'lineno': 3}], 'message/external-body': [{'composetyped': 'extcompose %s', 'description': '"A reference to data stored in an external location"', 'needsterminal': '', - 'view': 'showexternal %s %{access-type} %{name} %{site} %{directory} %{mode} %{server}'}], + 'view': 'showexternal %s %{access-type} %{name} %{site} %{directory} %{mode} %{server}', + 'lineno': 10}], 'text/richtext': [{'test': 'test "`echo %{charset} | tr \'[A-Z]\' \'[a-z]\'`" = iso-8859-8', 'copiousoutput': '', - 'view': 'shownonascii iso-8859-8 -e richtext -p %s'}], + 'view': 'shownonascii iso-8859-8 -e richtext -p %s', + 'lineno': 11}], 'image/x-xwindowdump': - [{'view': 'display %s'}], + [{'view': 'display %s', 'lineno': 9}], 'audio/*': - [{'view': '/usr/local/bin/showaudio %t'}], + [{'view': '/usr/local/bin/showaudio %t', 'lineno': 7}], 'video/*': - [{'view': 'animate %s'}], + [{'view': 'animate %s', 'lineno': 12}], 'application/frame': - [{'print': '"cat %s | lp"', 'view': 'showframe %s'}], + [{'print': '"cat %s | lp"', 'view': 'showframe %s', 'lineno': 0}], 'image/rgb': - [{'view': 'display %s'}] + [{'view': 'display %s', 'lineno': 8}] } +# For backwards compatibility, readmailcapfile() and lookup() still support +# the old version of mailcapdict without line numbers. +MAILCAPDICT_DEPRECATED = copy.deepcopy(MAILCAPDICT) +for entry_list in MAILCAPDICT_DEPRECATED.values(): + for entry in entry_list: + entry.pop('lineno') + class HelperFunctionTest(unittest.TestCase): @@ -75,12 +88,14 @@ def test_readmailcapfile(self): # Test readmailcapfile() using test file. It should match MAILCAPDICT. with open(MAILCAPFILE, 'r') as mcf: - d = mailcap.readmailcapfile(mcf) - self.assertDictEqual(d, MAILCAPDICT) + with self.assertWarns(DeprecationWarning): + d = mailcap.readmailcapfile(mcf) + self.assertDictEqual(d, MAILCAPDICT_DEPRECATED) def test_lookup(self): # Test without key - expected = [{'view': 'mpeg_play %s'}, {'view': 'animate %s'}] + expected = [{'view': 'animate %s', 'lineno': 12}, + {'view': 'mpeg_play %s', 'lineno': 13}] actual = mailcap.lookup(MAILCAPDICT, 'video/mpeg') self.assertListEqual(expected, actual) @@ -89,10 +104,16 @@ expected = [{'edit': 'audiocompose %s', 'compose': 'audiocompose %s', 'description': '"An audio fragment"', - 'view': 'showaudio %s'}] + 'view': 'showaudio %s', + 'lineno': 6}] actual = mailcap.lookup(MAILCAPDICT, 'audio/basic', key) self.assertListEqual(expected, actual) + # Test on user-defined dicts without line numbers + expected = [{'view': 'mpeg_play %s'}, {'view': 'animate %s'}] + actual = mailcap.lookup(MAILCAPDICT_DEPRECATED, 'video/mpeg') + self.assertListEqual(expected, actual) + def test_subst(self): plist = ['id=1', 'number=2', 'total=3'] # test case: ([field, MIMEtype, filename, plist=[]], ) @@ -151,14 +172,16 @@ 'edit': 'audiocompose %s', 'compose': 'audiocompose %s', 'description': '"An audio fragment"', - 'view': 'showaudio %s' + 'view': 'showaudio %s', + 'lineno': 6 } - audio_entry = {"view": "/usr/local/bin/showaudio %t"} - video_entry = {'view': 'animate %s'} + audio_entry = {"view": "/usr/local/bin/showaudio %t", 'lineno': 7} + video_entry = {'view': 'animate %s', 'lineno': 12} message_entry = { 'composetyped': 'extcompose %s', 'description': '"A reference to data stored in an external location"', 'needsterminal': '', - 'view': 'showexternal %s %{access-type} %{name} %{site} %{directory} %{mode} %{server}' + 'view': 'showexternal %s %{access-type} %{name} %{site} %{directory} %{mode} %{server}', + 'lineno': 10, } # test case: (findmatch args, findmatch keyword args, expected output) @@ -168,7 +191,7 @@ cases = [ ([{}, "video/mpeg"], {}, (None, None)), ([c, "foo/bar"], {}, (None, None)), - ([c, "video/mpeg"], {}, ('mpeg_play /dev/null', {'view': 'mpeg_play %s'})), + ([c, "video/mpeg"], {}, ('animate /dev/null', video_entry)), ([c, "audio/basic", "edit"], {}, ("audiocompose /dev/null", audio_basic_entry)), ([c, "audio/basic", "compose"], {}, ("audiocompose /dev/null", audio_basic_entry)), ([c, "audio/basic", "description"], {}, ('"An audio fragment"', audio_basic_entry)), diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -843,6 +843,7 @@ Chris Lawrence Mark Lawrence Chris Laws +Michael Lazar Brian Leair Mathieu Leduc-Hamel Amandine Lee diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -122,6 +122,9 @@ Library ------- +- Issue #14977: mailcap now respects the order of the lines in the mailcap + files ("first match"), as required by RFC 1542. Patch by Michael Lazar. + - Issue #28025: Convert all ssl module constants to IntEnum and IntFlags. SSLContext properties now return flags and enums. @@ -145,6 +148,10 @@ - Issue #24277: The new email API is no longer provisional, and the docs have been reorganized and rewritten to emphasize the new API. +- Issue #22450: urllib now includes an "Accept: */*" header among the + default headers. This makes the results of REST API requests more + consistent and predictable especially when proxy servers are involved. + - lib2to3.pgen3.driver.load_grammar() now creates a stable cache file between runs given the same Grammar.txt input regardless of the hash randomization setting. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:09:59 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 10 Sep 2016 00:09:59 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzE0OTc3OiBNYWtl?= =?utf-8?q?_mailcap_respect_the_order_of_the_lines_in_the_mailcap_file=2E?= Message-ID: <20160910000957.87446.16124.9D72FA19@psf.io> https://hg.python.org/cpython/rev/f1bf0abcca0c changeset: 103506:f1bf0abcca0c branch: 3.5 parent: 103502:00da8bfa2a60 user: R David Murray date: Fri Sep 09 20:04:23 2016 -0400 summary: #14977: Make mailcap respect the order of the lines in the mailcap file. This is required by RFC 1542, so despite the subtle behavior change we are treating it as a bug. Patch by Michael Lazar. files: Lib/mailcap.py | 28 ++++++++++- Lib/test/mailcap.txt | 2 +- Lib/test/test_mailcap.py | 69 ++++++++++++++++++--------- Misc/ACKS | 1 + Misc/NEWS | 3 + 5 files changed, 76 insertions(+), 27 deletions(-) diff --git a/Lib/mailcap.py b/Lib/mailcap.py --- a/Lib/mailcap.py +++ b/Lib/mailcap.py @@ -1,9 +1,19 @@ """Mailcap file handling. See RFC 1524.""" import os +import warnings __all__ = ["getcaps","findmatch"] + +def lineno_sort_key(entry): + # Sort in ascending order, with unspecified entries at the end + if 'lineno' in entry: + return 0, entry['lineno'] + else: + return 1, 0 + + # Part 1: top-level interface. def getcaps(): @@ -17,13 +27,14 @@ """ caps = {} + lineno = 0 for mailcap in listmailcapfiles(): try: fp = open(mailcap, 'r') except OSError: continue with fp: - morecaps = readmailcapfile(fp) + morecaps, lineno = _readmailcapfile(fp, lineno) for key, value in morecaps.items(): if not key in caps: caps[key] = value @@ -49,8 +60,15 @@ # Part 2: the parser. +def readmailcapfile(fp): + """Read a mailcap file and return a dictionary keyed by MIME type.""" + warnings.warn('readmailcapfile is deprecated, use getcaps instead', + DeprecationWarning, 2) + caps, _ = _readmailcapfile(fp, None) + return caps -def readmailcapfile(fp): + +def _readmailcapfile(fp, lineno): """Read a mailcap file and return a dictionary keyed by MIME type. Each MIME type is mapped to an entry consisting of a list of @@ -76,6 +94,9 @@ key, fields = parseline(line) if not (key and fields): continue + if lineno is not None: + fields['lineno'] = lineno + lineno += 1 # Normalize the key types = key.split('/') for j in range(len(types)): @@ -86,7 +107,7 @@ caps[key].append(fields) else: caps[key] = [fields] - return caps + return caps, lineno def parseline(line): """Parse one entry in a mailcap file and return a dictionary. @@ -165,6 +186,7 @@ entries = entries + caps[MIMEtype] if key is not None: entries = [e for e in entries if key in e] + entries = sorted(entries, key=lineno_sort_key) return entries def subst(field, MIMEtype, filename, plist=[]): diff --git a/Lib/test/mailcap.txt b/Lib/test/mailcap.txt --- a/Lib/test/mailcap.txt +++ b/Lib/test/mailcap.txt @@ -35,5 +35,5 @@ text/richtext; shownonascii iso-8859-8 -e richtext -p %s; test=test "`echo \ %{charset} | tr '[A-Z]' '[a-z]'`" = iso-8859-8; copiousoutput -video/mpeg; mpeg_play %s video/*; animate %s +video/mpeg; mpeg_play %s \ No newline at end of file diff --git a/Lib/test/test_mailcap.py b/Lib/test/test_mailcap.py --- a/Lib/test/test_mailcap.py +++ b/Lib/test/test_mailcap.py @@ -1,6 +1,7 @@ import mailcap import os import shutil +import copy import test.support import unittest @@ -14,43 +15,55 @@ [{'compose': 'moviemaker %s', 'x11-bitmap': '"/usr/lib/Zmail/bitmaps/movie.xbm"', 'description': '"Movie"', - 'view': 'movieplayer %s'}], + 'view': 'movieplayer %s', + 'lineno': 4}], 'application/*': [{'copiousoutput': '', - 'view': 'echo "This is \\"%t\\" but is 50 \\% Greek to me" \\; cat %s'}], + 'view': 'echo "This is \\"%t\\" but is 50 \\% Greek to me" \\; cat %s', + 'lineno': 5}], 'audio/basic': [{'edit': 'audiocompose %s', 'compose': 'audiocompose %s', 'description': '"An audio fragment"', - 'view': 'showaudio %s'}], + 'view': 'showaudio %s', + 'lineno': 6}], 'video/mpeg': - [{'view': 'mpeg_play %s'}], + [{'view': 'mpeg_play %s', 'lineno': 13}], 'application/postscript': - [{'needsterminal': '', 'view': 'ps-to-terminal %s'}, - {'compose': 'idraw %s', 'view': 'ps-to-terminal %s'}], + [{'needsterminal': '', 'view': 'ps-to-terminal %s', 'lineno': 1}, + {'compose': 'idraw %s', 'view': 'ps-to-terminal %s', 'lineno': 2}], 'application/x-dvi': - [{'view': 'xdvi %s'}], + [{'view': 'xdvi %s', 'lineno': 3}], 'message/external-body': [{'composetyped': 'extcompose %s', 'description': '"A reference to data stored in an external location"', 'needsterminal': '', - 'view': 'showexternal %s %{access-type} %{name} %{site} %{directory} %{mode} %{server}'}], + 'view': 'showexternal %s %{access-type} %{name} %{site} %{directory} %{mode} %{server}', + 'lineno': 10}], 'text/richtext': [{'test': 'test "`echo %{charset} | tr \'[A-Z]\' \'[a-z]\'`" = iso-8859-8', 'copiousoutput': '', - 'view': 'shownonascii iso-8859-8 -e richtext -p %s'}], + 'view': 'shownonascii iso-8859-8 -e richtext -p %s', + 'lineno': 11}], 'image/x-xwindowdump': - [{'view': 'display %s'}], + [{'view': 'display %s', 'lineno': 9}], 'audio/*': - [{'view': '/usr/local/bin/showaudio %t'}], + [{'view': '/usr/local/bin/showaudio %t', 'lineno': 7}], 'video/*': - [{'view': 'animate %s'}], + [{'view': 'animate %s', 'lineno': 12}], 'application/frame': - [{'print': '"cat %s | lp"', 'view': 'showframe %s'}], + [{'print': '"cat %s | lp"', 'view': 'showframe %s', 'lineno': 0}], 'image/rgb': - [{'view': 'display %s'}] + [{'view': 'display %s', 'lineno': 8}] } +# For backwards compatibility, readmailcapfile() and lookup() still support +# the old version of mailcapdict without line numbers. +MAILCAPDICT_DEPRECATED = copy.deepcopy(MAILCAPDICT) +for entry_list in MAILCAPDICT_DEPRECATED.values(): + for entry in entry_list: + entry.pop('lineno') + class HelperFunctionTest(unittest.TestCase): @@ -76,12 +89,14 @@ def test_readmailcapfile(self): # Test readmailcapfile() using test file. It should match MAILCAPDICT. with open(MAILCAPFILE, 'r') as mcf: - d = mailcap.readmailcapfile(mcf) - self.assertDictEqual(d, MAILCAPDICT) + with self.assertWarns(DeprecationWarning): + d = mailcap.readmailcapfile(mcf) + self.assertDictEqual(d, MAILCAPDICT_DEPRECATED) def test_lookup(self): # Test without key - expected = [{'view': 'mpeg_play %s'}, {'view': 'animate %s'}] + expected = [{'view': 'animate %s', 'lineno': 12}, + {'view': 'mpeg_play %s', 'lineno': 13}] actual = mailcap.lookup(MAILCAPDICT, 'video/mpeg') self.assertListEqual(expected, actual) @@ -90,10 +105,16 @@ expected = [{'edit': 'audiocompose %s', 'compose': 'audiocompose %s', 'description': '"An audio fragment"', - 'view': 'showaudio %s'}] + 'view': 'showaudio %s', + 'lineno': 6}] actual = mailcap.lookup(MAILCAPDICT, 'audio/basic', key) self.assertListEqual(expected, actual) + # Test on user-defined dicts without line numbers + expected = [{'view': 'mpeg_play %s'}, {'view': 'animate %s'}] + actual = mailcap.lookup(MAILCAPDICT_DEPRECATED, 'video/mpeg') + self.assertListEqual(expected, actual) + def test_subst(self): plist = ['id=1', 'number=2', 'total=3'] # test case: ([field, MIMEtype, filename, plist=[]], ) @@ -152,14 +173,16 @@ 'edit': 'audiocompose %s', 'compose': 'audiocompose %s', 'description': '"An audio fragment"', - 'view': 'showaudio %s' + 'view': 'showaudio %s', + 'lineno': 6 } - audio_entry = {"view": "/usr/local/bin/showaudio %t"} - video_entry = {'view': 'animate %s'} + audio_entry = {"view": "/usr/local/bin/showaudio %t", 'lineno': 7} + video_entry = {'view': 'animate %s', 'lineno': 12} message_entry = { 'composetyped': 'extcompose %s', 'description': '"A reference to data stored in an external location"', 'needsterminal': '', - 'view': 'showexternal %s %{access-type} %{name} %{site} %{directory} %{mode} %{server}' + 'view': 'showexternal %s %{access-type} %{name} %{site} %{directory} %{mode} %{server}', + 'lineno': 10, } # test case: (findmatch args, findmatch keyword args, expected output) @@ -169,7 +192,7 @@ cases = [ ([{}, "video/mpeg"], {}, (None, None)), ([c, "foo/bar"], {}, (None, None)), - ([c, "video/mpeg"], {}, ('mpeg_play /dev/null', {'view': 'mpeg_play %s'})), + ([c, "video/mpeg"], {}, ('animate /dev/null', video_entry)), ([c, "audio/basic", "edit"], {}, ("audiocompose /dev/null", audio_basic_entry)), ([c, "audio/basic", "compose"], {}, ("audiocompose /dev/null", audio_basic_entry)), ([c, "audio/basic", "description"], {}, ('"An audio fragment"', audio_basic_entry)), diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -833,6 +833,7 @@ Chris Lawrence Mark Lawrence Chris Laws +Michael Lazar Brian Leair Mathieu Leduc-Hamel Amandine Lee diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -62,6 +62,9 @@ Library ------- +- Issue #14977: mailcap now respects the order of the lines in the mailcap + files ("first match"), as required by RFC 1542. Patch by Michael Lazar. + - Issue #24594: Validates persist parameter when opening MSI database - Issue #28047: Fixed calculation of line length used for the base64 CTE -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:33:59 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 00:33:59 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1NzU4?= =?utf-8?q?=3A_Prevents_zipimport_from_unnecessarily_encoding_a_filename_?= =?utf-8?q?=28patch?= Message-ID: <20160910003359.12414.38901.974360C2@psf.io> https://hg.python.org/cpython/rev/663a62bcf9c9 changeset: 103508:663a62bcf9c9 branch: 3.5 parent: 103506:f1bf0abcca0c user: Steve Dower date: Fri Sep 09 17:27:33 2016 -0700 summary: Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch by Eryk Sun) files: Lib/test/test_zipimport.py | 2 +- Misc/NEWS | 3 +++ Modules/zipimport.c | 14 ++++---------- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -596,7 +596,7 @@ z.writestr(zinfo, test_src) z.close() try: - zipimport.zipimporter(filename) + zipimport.zipimporter(filename).load_module(TESTMOD) finally: os.remove(filename) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #25758: Prevents zipimport from unnecessarily encoding a filename + (patch by Eryk Sun) + - Issue #27812: Properly clear out a generator's frame's backreference to the generator to prevent crashes in frame.clear(). diff --git a/Modules/zipimport.c b/Modules/zipimport.c --- a/Modules/zipimport.c +++ b/Modules/zipimport.c @@ -1362,22 +1362,16 @@ static PyObject * compile_source(PyObject *pathname, PyObject *source) { - PyObject *code, *fixed_source, *pathbytes; - - pathbytes = PyUnicode_EncodeFSDefault(pathname); - if (pathbytes == NULL) - return NULL; + PyObject *code, *fixed_source; fixed_source = normalize_line_endings(source); if (fixed_source == NULL) { - Py_DECREF(pathbytes); return NULL; } - code = Py_CompileString(PyBytes_AsString(fixed_source), - PyBytes_AsString(pathbytes), - Py_file_input); - Py_DECREF(pathbytes); + code = Py_CompileStringObject(PyBytes_AsString(fixed_source), + pathname, Py_file_input, NULL, 1); + Py_DECREF(fixed_source); return code; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:33:59 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 00:33:59 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325758=3A_Prevents_zipimport_from_unnecessarily_?= =?utf-8?q?encoding_a_filename_=28patch?= Message-ID: <20160910003359.21038.60778.E7B5817E@psf.io> https://hg.python.org/cpython/rev/ead30e7262d5 changeset: 103509:ead30e7262d5 parent: 103507:efd692c86429 parent: 103508:663a62bcf9c9 user: Steve Dower date: Fri Sep 09 17:33:37 2016 -0700 summary: Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch by Eryk Sun) files: Lib/test/test_zipimport.py | 2 +- Misc/NEWS | 3 +++ Modules/zipimport.c | 14 ++++---------- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -615,7 +615,7 @@ z.writestr(zinfo, test_src) z.close() try: - zipimport.zipimporter(filename) + zipimport.zipimporter(filename).load_module(TESTMOD) finally: os.remove(filename) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #25758: Prevents zipimport from unnecessarily encoding a filename + (patch by Eryk Sun) + - Issue #25856: The __module__ attribute of extension classes and functions now is interned. This leads to more compact pickle data with protocol 4. diff --git a/Modules/zipimport.c b/Modules/zipimport.c --- a/Modules/zipimport.c +++ b/Modules/zipimport.c @@ -1362,22 +1362,16 @@ static PyObject * compile_source(PyObject *pathname, PyObject *source) { - PyObject *code, *fixed_source, *pathbytes; - - pathbytes = PyUnicode_EncodeFSDefault(pathname); - if (pathbytes == NULL) - return NULL; + PyObject *code, *fixed_source; fixed_source = normalize_line_endings(source); if (fixed_source == NULL) { - Py_DECREF(pathbytes); return NULL; } - code = Py_CompileString(PyBytes_AsString(fixed_source), - PyBytes_AsString(pathbytes), - Py_file_input); - Py_DECREF(pathbytes); + code = Py_CompileStringObject(PyBytes_AsString(fixed_source), + pathname, Py_file_input, NULL, 1); + Py_DECREF(fixed_source); return code; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:38:36 2016 From: python-checkins at python.org (zach.ware) Date: Sat, 10 Sep 2016 00:38:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Remove_line_numbers_from_s?= =?utf-8?q?uspicious_rules?= Message-ID: <20160910003836.2071.11503.5E9D6DE0@psf.io> https://hg.python.org/cpython/rev/ab4f8d053d94 changeset: 103510:ab4f8d053d94 user: Zachary Ware date: Fri Sep 09 17:38:28 2016 -0700 summary: Remove line numbers from suspicious rules files: Doc/tools/susp-ignored.csv | 32 +++++++++++++------------- 1 files changed, 16 insertions(+), 16 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -243,7 +243,7 @@ using/cmdline,,:message,action:message:category:module:line using/cmdline,,:module,action:message:category:module:line using/unix,,:Packaging,https://en.opensuse.org/Portal:Packaging -whatsnew/2.0,418,:len, +whatsnew/2.0,,:len, whatsnew/2.3,,::, whatsnew/2.3,,:config, whatsnew/2.3,,:Critical, @@ -276,20 +276,20 @@ whatsnew/3.2,,:gz,">>> with tarfile.open(name='myarchive.tar.gz', mode='w:gz') as tf:" whatsnew/3.2,,:location,zope9-location = ${zope9:location} whatsnew/3.2,,:prefix,zope-conf = ${custom:prefix}/etc/zope.conf -library/tarfile,149,:xz,'x:xz' -library/xml.etree.elementtree,290,:sometag,prefix:sometag -library/xml.etree.elementtree,301,:fictional,"Lancelot -library/xml.etree.elementtree,301,:character,Archie Leach -library/xml.etree.elementtree,301,:character,Sir Robin -library/xml.etree.elementtree,301,:character,Gunther -library/xml.etree.elementtree,301,:character,Commander Clement +library/tarfile,,:xz,'x:xz' +library/xml.etree.elementtree,,:sometag,prefix:sometag +library/xml.etree.elementtree,,:fictional,"Lancelot +library/xml.etree.elementtree,,:character,Archie Leach +library/xml.etree.elementtree,,:character,Sir Robin +library/xml.etree.elementtree,,:character,Gunther +library/xml.etree.elementtree,,:character,Commander Clement library/xml.etree.elementtree,,:actor,"for actor in root.findall('real_person:actor', ns):" library/xml.etree.elementtree,,:name,"name = actor.find('real_person:name', ns)" library/xml.etree.elementtree,,:character,"for char in actor.findall('role:character', ns):" -library/zipapp,31,:main,"$ python -m zipapp myapp -m ""myapp:main""" -library/zipapp,82,:fn,"argument should have the form ""pkg.mod:fn"", where ""pkg.mod"" is a" -library/zipapp,155,:callable,"""pkg.module:callable"" and the archive will be run by importing" +library/zipapp,,:main,"$ python -m zipapp myapp -m ""myapp:main""" +library/zipapp,,:fn,"argument should have the form ""pkg.mod:fn"", where ""pkg.mod"" is a" +library/zipapp,,:callable,"""pkg.module:callable"" and the archive will be run by importing" library/stdtypes,,::,>>> m[::2].tolist() library/sys,,`,# ``wrapper`` creates a ``wrap(coro)`` coroutine: whatsnew/3.5,,:root,'WARNING:root:warning\n' @@ -297,7 +297,7 @@ whatsnew/3.5,,::,>>> addr6 = ipaddress.IPv6Address('::1') whatsnew/3.5,,:root,ERROR:root:exception whatsnew/3.5,,:exception,ERROR:root:exception -whatsnew/3.6,140,`,1000000000000000` -whatsnew/changelog,998,:version,import sys; I = version[:version.index(' ')] -whatsnew/changelog,6416,:gz,": TarFile opened with external fileobj and ""w:gz"" mode didn't" -whatsnew/changelog,8023,::,": Use ""127.0.0.1"" or ""::1"" instead of ""localhost"" as much as" +whatsnew/3.6,,`,1000000000000000` +whatsnew/changelog,,:version,import sys; I = version[:version.index(' ')] +whatsnew/changelog,,:gz,": TarFile opened with external fileobj and ""w:gz"" mode didn't" +whatsnew/changelog,,::,": Use ""127.0.0.1"" or ""::1"" instead of ""localhost"" as much as" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:39:44 2016 From: python-checkins at python.org (lukasz.langa) Date: Sat, 10 Sep 2016 00:39:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_DTrace_support=3A_function?= =?utf-8?q?_calls=2C_GC_activity=2C_line_execution?= Message-ID: <20160910003944.21157.44894.B526EF06@psf.io> https://hg.python.org/cpython/rev/d622dbd71f2b changeset: 103511:d622dbd71f2b user: ?ukasz Langa date: Fri Sep 09 17:37:37 2016 -0700 summary: DTrace support: function calls, GC activity, line execution Tested on macOS 10.11 dtrace, Ubuntu 16.04 SystemTap, and libbcc. Largely based by an initial patch by Jes?s Cea Avi?n, with some influence from Dave Malcolm's SystemTap patch and Nikhil Benesch's unification patch. Things deliberately left out for simplicity: - ustack helpers, I have no way of testing them at this point since they are Solaris-specific - PyFrameObject * in function__entry/function__return, this is SystemTap-specific - SPARC support - dynamic tracing - sys module dtrace facility introspection All of those might be added later. files: .hgignore | 1 + Doc/howto/index.rst | 1 + Doc/howto/instrumentation.rst | 411 ++++++++++ Doc/whatsnew/3.6.rst | 25 + Include/pydtrace.d | 19 + Include/pydtrace.h | 47 + Lib/test/dtracedata/assert_usable.d | 5 + Lib/test/dtracedata/assert_usable.stp | 5 + Lib/test/dtracedata/call_stack.d | 31 + Lib/test/dtracedata/call_stack.d.expected | 18 + Lib/test/dtracedata/call_stack.py | 30 + Lib/test/dtracedata/call_stack.stp | 41 + Lib/test/dtracedata/call_stack.stp.expected | 14 + Lib/test/dtracedata/gc.d | 18 + Lib/test/dtracedata/gc.d.expected | 8 + Lib/test/dtracedata/gc.py | 13 + Lib/test/dtracedata/gc.stp | 26 + Lib/test/dtracedata/gc.stp.expected | 8 + Lib/test/dtracedata/instance.py | 24 + Lib/test/dtracedata/line.d | 7 + Lib/test/dtracedata/line.d.expected | 20 + Lib/test/dtracedata/line.py | 17 + Lib/test/test_dtrace.py | 178 ++++ Makefile.pre.in | 36 +- Misc/ACKS | 1 + Misc/NEWS | 2 + Modules/gcmodule.c | 8 + Python/ceval.c | 78 +- configure | 186 ++++- configure.ac | 42 + pyconfig.h.in | 3 + 31 files changed, 1305 insertions(+), 18 deletions(-) diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -56,6 +56,7 @@ *.profclang? *.profraw *.dyn +Include/pydtrace_probes.h Lib/distutils/command/*.pdb Lib/lib2to3/*.pickle Lib/test/data/* diff --git a/Doc/howto/index.rst b/Doc/howto/index.rst --- a/Doc/howto/index.rst +++ b/Doc/howto/index.rst @@ -28,4 +28,5 @@ argparse.rst ipaddress.rst clinic.rst + instrumentation.rst diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst new file mode 100644 --- /dev/null +++ b/Doc/howto/instrumentation.rst @@ -0,0 +1,411 @@ +.. _instrumentation: + +=============================================== +Instrumenting CPython with DTrace and SystemTap +=============================================== + +:author: David Malcolm +:author: ?ukasz Langa + +DTrace and SystemTap are monitoring tools, each providing a way to inspect +what the processes on a computer system are doing. They both use +domain-specific languages allowing a user to write scripts which: + + - filter which processes are to be observed + - gather data from the processes of interest + - generate reports on the data + +As of Python 3.6, CPython can be built with embedded "markers", also +known as "probes", that can be observed by a DTrace or SystemTap script, +making it easier to monitor what the CPython processes on a system are +doing. + +.. I'm using ".. code-block:: c" for SystemTap scripts, as "c" is syntactically + the closest match that Sphinx supports + +.. impl-detail:: + + DTrace markers are implementation details of the CPython interpreter. + No guarantees are made about probe compatibility between versions of + CPython. DTrace scripts can stop working or work incorrectly without + warning when changing CPython versions. + + +Enabling the static markers +--------------------------- + +macOS comes with built-in support for DTrace. On Linux, in order to +build CPython with the embedded markers for SystemTap, the SystemTap +development tools must be installed. + +On a Linux machine, this can be done via:: + + yum install systemtap-sdt-devel + +or:: + + sudo apt-get install systemtap-sdt-dev + + +CPython must then be configured `--with-dtrace`:: + + checking for --with-dtrace... yes + +On macOS, you can list available DTrace probes by running a Python +process in the background and listing all probes made available by the +Python provider:: + + $ python3.6 -q & + $ sudo dtrace -l -P python$! # or: dtrace -l -m python3.6 + + ID PROVIDER MODULE FUNCTION NAME + 29564 python18035 python3.6 _PyEval_EvalFrameDefault function-entry + 29565 python18035 python3.6 dtrace_function_entry function-entry + 29566 python18035 python3.6 _PyEval_EvalFrameDefault function-return + 29567 python18035 python3.6 dtrace_function_return function-return + 29568 python18035 python3.6 collect gc-done + 29569 python18035 python3.6 collect gc-start + 29570 python18035 python3.6 _PyEval_EvalFrameDefault line + 29571 python18035 python3.6 maybe_dtrace_line line + +On Linux, you can verify if the SystemTap static markers are present in +the built binary by seeing if it contains a ".note.stapsdt" section. + +.. code-block:: bash + + $ readelf -S ./python | grep .note.stapsdt + [30] .note.stapsdt NOTE 0000000000000000 00308d78 + +If you've built Python as a shared library (with --enable-shared), you +need to look instead within the shared library. For example: + +.. code-block:: bash + + $ readelf -S libpython3.3dm.so.1.0 | grep .note.stapsdt + [29] .note.stapsdt NOTE 0000000000000000 00365b68 + +Sufficiently modern readelf can print the metadata: + +.. code-block:: bash + + $ readelf -n ./python + + Displaying notes found at file offset 0x00000254 with length 0x00000020: + Owner Data size Description + GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag) + OS: Linux, ABI: 2.6.32 + + Displaying notes found at file offset 0x00000274 with length 0x00000024: + Owner Data size Description + GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring) + Build ID: df924a2b08a7e89f6e11251d4602022977af2670 + + Displaying notes found at file offset 0x002d6c30 with length 0x00000144: + Owner Data size Description + stapsdt 0x00000031 NT_STAPSDT (SystemTap probe descriptors) + Provider: python + Name: gc__start + Location: 0x00000000004371c3, Base: 0x0000000000630ce2, Semaphore: 0x00000000008d6bf6 + Arguments: -4@%ebx + stapsdt 0x00000030 NT_STAPSDT (SystemTap probe descriptors) + Provider: python + Name: gc__done + Location: 0x00000000004374e1, Base: 0x0000000000630ce2, Semaphore: 0x00000000008d6bf8 + Arguments: -8@%rax + stapsdt 0x00000045 NT_STAPSDT (SystemTap probe descriptors) + Provider: python + Name: function__entry + Location: 0x000000000053db6c, Base: 0x0000000000630ce2, Semaphore: 0x00000000008d6be8 + Arguments: 8@%rbp 8@%r12 -4@%eax + stapsdt 0x00000046 NT_STAPSDT (SystemTap probe descriptors) + Provider: python + Name: function__return + Location: 0x000000000053dba8, Base: 0x0000000000630ce2, Semaphore: 0x00000000008d6bea + Arguments: 8@%rbp 8@%r12 -4@%eax + +The above metadata contains information for SystemTap describing how it +can patch strategically-placed machine code instructions to enable the +tracing hooks used by a SystemTap script. + + +Static DTrace probes +-------------------- + +The following example DTrace script can be used to show the call/return +hierarchy of a Python script, only tracing within the invocation of +a function called "start". In other words, import-time function +invocations are not going to be listed: + +.. code-block:: c + + self int indent; + + python$target:::function-entry + /copyinstr(arg1) == "start"/ + { + self->trace = 1; + } + + python$target:::function-entry + /self->trace/ + { + printf("%d\t%*s:", timestamp, 15, probename); + printf("%*s", self->indent, ""); + printf("%s:%s:%d\n", basename(copyinstr(arg0)), copyinstr(arg1), arg2); + self->indent++; + } + + python$target:::function-return + /self->trace/ + { + self->indent--; + printf("%d\t%*s:", timestamp, 15, probename); + printf("%*s", self->indent, ""); + printf("%s:%s:%d\n", basename(copyinstr(arg0)), copyinstr(arg1), arg2); + } + + python$target:::function-return + /copyinstr(arg1) == "start"/ + { + self->trace = 0; + } + +It can be invoked like this: + +.. code-block:: bash + + $ sudo dtrace -q -s call_stack.d -c "python3.6 script.py" + +The output looks like this:: + + 156641360502280 function-entry:call_stack.py:start:23 + 156641360518804 function-entry: call_stack.py:function_1:1 + 156641360532797 function-entry: call_stack.py:function_3:9 + 156641360546807 function-return: call_stack.py:function_3:10 + 156641360563367 function-return: call_stack.py:function_1:2 + 156641360578365 function-entry: call_stack.py:function_2:5 + 156641360591757 function-entry: call_stack.py:function_1:1 + 156641360605556 function-entry: call_stack.py:function_3:9 + 156641360617482 function-return: call_stack.py:function_3:10 + 156641360629814 function-return: call_stack.py:function_1:2 + 156641360642285 function-return: call_stack.py:function_2:6 + 156641360656770 function-entry: call_stack.py:function_3:9 + 156641360669707 function-return: call_stack.py:function_3:10 + 156641360687853 function-entry: call_stack.py:function_4:13 + 156641360700719 function-return: call_stack.py:function_4:14 + 156641360719640 function-entry: call_stack.py:function_5:18 + 156641360732567 function-return: call_stack.py:function_5:21 + 156641360747370 function-return:call_stack.py:start:28 + + +Static SystemTap markers +------------------------ + +The low-level way to use the SystemTap integration is to use the static +markers directly. This requires you to explicitly state the binary file +containing them. + +For example, this SystemTap script can be used to show the call/return +hierarchy of a Python script: + +.. code-block:: c + + probe process('python').mark("function__entry") { + filename = user_string($arg1); + funcname = user_string($arg2); + lineno = $arg3; + + printf("%s => %s in %s:%d\\n", + thread_indent(1), funcname, filename, lineno); + } + + probe process('python').mark("function__return") { + filename = user_string($arg1); + funcname = user_string($arg2); + lineno = $arg3; + + printf("%s <= %s in %s:%d\\n", + thread_indent(-1), funcname, filename, lineno); + } + +It can be invoked like this: + +.. code-block:: bash + + $ stap \ + show-call-hierarchy.stp \ + -c ./python test.py + +The output looks like this:: + + 11408 python(8274): => __contains__ in Lib/_abcoll.py:362 + 11414 python(8274): => __getitem__ in Lib/os.py:425 + 11418 python(8274): => encode in Lib/os.py:490 + 11424 python(8274): <= encode in Lib/os.py:493 + 11428 python(8274): <= __getitem__ in Lib/os.py:426 + 11433 python(8274): <= __contains__ in Lib/_abcoll.py:366 + +where the columns are: + + - time in microseconds since start of script + + - name of executable + + - PID of process + +and the remainder indicates the call/return hierarchy as the script executes. + +For a `--enable-shared` build of CPython, the markers are contained within the +libpython shared library, and the probe's dotted path needs to reflect this. For +example, this line from the above example:: + + probe process('python').mark("function__entry") { + +should instead read:: + + probe process('python').library("libpython3.6dm.so.1.0").mark("function__entry") { + +(assuming a debug build of CPython 3.6) + + +Available static markers +------------------------ + +.. I'm reusing the "c:function" type for markers + +.. c:function:: function__entry(str filename, str funcname, int lineno) + + This marker indicates that execution of a Python function has begun. + It is only triggered for pure-Python (bytecode) functions. + + The filename, function name, and line number are provided back to the + tracing script as positional arguments, which must be accessed using + `$arg1`, `$arg2`, `$arg3`: + + * `$arg1` : `(const char *)` filename, accessible using `user_string($arg1)` + + * `$arg2` : `(const char *)` function name, accessible using + `user_string($arg2)` + + * `$arg3` : `int` line number + +.. c:function:: function__return(str filename, str funcname, int lineno) + + This marker is the converse of `function__entry`, and indicates that + execution of a Python function has ended (either via ``return``, or + via an exception). It is only triggered for pure-Python (bytecode) + functions. + + The arguments are the same as for `function__entry` + +.. c:function:: line(str filename, str funcname, int lineno) + + This marker indicates a Python line is about to be executed. It is + the equivalent of line-by-line tracing with a Python profiler. It is + not triggered within C functions. + + The arguments are the same as for `function__entry`. + +.. c:function:: gc__start(int generation) + + Fires when the Python interpreter starts a garbage collection cycle. + `arg0` is the generation to scan, like :func:`gc.collect()`. + +.. c:function:: gc__done(long collected) + + Fires when the Python interpreter finishes a garbage collection + cycle. `arg0` is the number of collected objects. + + +SystemTap Tapsets +----------------- + +The higher-level way to use the SystemTap integration is to use a "tapset": +SystemTap's equivalent of a library, which hides some of the lower-level +details of the static markers. + +Here is a tapset file, based on a non-shared build of CPython: + +.. code-block:: c + + /* + Provide a higher-level wrapping around the function__entry and + function__return markers: + \*/ + probe python.function.entry = process("python").mark("function__entry") + { + filename = user_string($arg1); + funcname = user_string($arg2); + lineno = $arg3; + frameptr = $arg4 + } + probe python.function.return = process("python").mark("function__return") + { + filename = user_string($arg1); + funcname = user_string($arg2); + lineno = $arg3; + frameptr = $arg4 + } + +If this file is installed in SystemTap's tapset directory (e.g. +`/usr/share/systemtap/tapset`), then these additional probepoints become +available: + +.. c:function:: python.function.entry(str filename, str funcname, int lineno, frameptr) + + This probe point indicates that execution of a Python function has begun. + It is only triggered for pure-python (bytecode) functions. + +.. c:function:: python.function.return(str filename, str funcname, int lineno, frameptr) + + This probe point is the converse of `python.function.return`, and indicates + that execution of a Python function has ended (either via ``return``, or + via an exception). It is only triggered for pure-python (bytecode) functions. + + +Examples +-------- +This SystemTap script uses the tapset above to more cleanly implement the +example given above of tracing the Python function-call hierarchy, without +needing to directly name the static markers: + +.. code-block:: c + + probe python.function.entry + { + printf("%s => %s in %s:%d\n", + thread_indent(1), funcname, filename, lineno); + } + + probe python.function.return + { + printf("%s <= %s in %s:%d\n", + thread_indent(-1), funcname, filename, lineno); + } + + +The following script uses the tapset above to provide a top-like view of all +running CPython code, showing the top 20 most frequently-entered bytecode +frames, each second, across the whole system: + +.. code-block:: c + + global fn_calls; + + probe python.function.entry + { + fn_calls[pid(), filename, funcname, lineno] += 1; + } + + probe timer.ms(1000) { + printf("\033[2J\033[1;1H") /* clear screen \*/ + printf("%6s %80s %6s %30s %6s\n", + "PID", "FILENAME", "LINE", "FUNCTION", "CALLS") + foreach ([pid, filename, funcname, lineno] in fn_calls- limit 20) { + printf("%6d %80s %6d %30s %6d\n", + pid, filename, lineno, funcname, + fn_calls[pid, filename, funcname, lineno]); + } + delete fn_calls; + } + diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -407,6 +407,31 @@ (Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.) +DTrace and SystemTap probing support +------------------------------------ + +Python can now be built ``--with-dtrace`` which enables static markers +for the following events in the interpreter: + +* function call/return + +* garbage collection started/finished + +* line of code executed. + +This can be used to instrument running interpreters in production, +without the need to recompile specific debug builds or providing +application-specific profiling/debugging code. + +More details in:ref:`instrumentation`. + +The current implementation is tested on Linux and macOS. Additional +markers may be added in the future. + +(Contributed by ?ukasz Langa in :issue:`21590`, based on patches by +Jes?s Cea Avi?n, David Malcolm, and Nikhil Benesch.) + + .. _whatsnew-deforder: PEP 520: Preserving Class Attribute Definition Order diff --git a/Include/pydtrace.d b/Include/pydtrace.d new file mode 100644 --- /dev/null +++ b/Include/pydtrace.d @@ -0,0 +1,19 @@ +/* Python DTrace provider */ + +provider python { + probe function__entry(const char *, const char *, int); + probe function__return(const char *, const char *, int); + probe instance__new__start(const char *, const char *); + probe instance__new__done(const char *, const char *); + probe instance__delete__start(const char *, const char *); + probe instance__delete__done(const char *, const char *); + probe line(const char *, const char *, int); + probe gc__start(int); + probe gc__done(long); +}; + +#pragma D attributes Evolving/Evolving/Common provider python provider +#pragma D attributes Evolving/Evolving/Common provider python module +#pragma D attributes Evolving/Evolving/Common provider python function +#pragma D attributes Evolving/Evolving/Common provider python name +#pragma D attributes Evolving/Evolving/Common provider python args diff --git a/Include/pydtrace.h b/Include/pydtrace.h new file mode 100644 --- /dev/null +++ b/Include/pydtrace.h @@ -0,0 +1,47 @@ +/* Static DTrace probes interface */ + +#ifndef Py_DTRACE_H +#define Py_DTRACE_H + +#ifdef WITH_DTRACE + +#include "pydtrace_probes.h" + +/* pydtrace_probes.h, on systems with DTrace, is auto-generated to include + `PyDTrace_{PROBE}` and `PyDTrace_{PROBE}_ENABLED()` macros for every probe + defined in pydtrace_provider.d. + + Calling these functions must be guarded by a `PyDTrace_{PROBE}_ENABLED()` + check to minimize performance impact when probing is off. For example: + + if (PyDTrace_FUNCTION_ENTRY_ENABLED()) + PyDTrace_FUNCTION_ENTRY(f); +*/ + +#else + +/* Without DTrace, compile to nothing. */ + +#define PyDTrace_LINE(arg0, arg1, arg2, arg3) do ; while (0) +#define PyDTrace_FUNCTION_ENTRY(arg0, arg1, arg2) do ; while (0) +#define PyDTrace_FUNCTION_RETURN(arg0, arg1, arg2) do ; while (0) +#define PyDTrace_GC_START(arg0) do ; while (0) +#define PyDTrace_GC_DONE(arg0) do ; while (0) +#define PyDTrace_INSTANCE_NEW_START(arg0) do ; while (0) +#define PyDTrace_INSTANCE_NEW_DONE(arg0) do ; while (0) +#define PyDTrace_INSTANCE_DELETE_START(arg0) do ; while (0) +#define PyDTrace_INSTANCE_DELETE_DONE(arg0) do ; while (0) + +#define PyDTrace_LINE_ENABLED() (0) +#define PyDTrace_FUNCTION_ENTRY_ENABLED() (0) +#define PyDTrace_FUNCTION_RETURN_ENABLED() (0) +#define PyDTrace_GC_START_ENABLED() (0) +#define PyDTrace_GC_DONE_ENABLED() (0) +#define PyDTrace_INSTANCE_NEW_START_ENABLED() (0) +#define PyDTrace_INSTANCE_NEW_DONE_ENABLED() (0) +#define PyDTrace_INSTANCE_DELETE_START_ENABLED() (0) +#define PyDTrace_INSTANCE_DELETE_DONE_ENABLED() (0) + +#endif /* !WITH_DTRACE */ + +#endif /* !Py_DTRACE_H */ diff --git a/Lib/test/dtracedata/assert_usable.d b/Lib/test/dtracedata/assert_usable.d new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/assert_usable.d @@ -0,0 +1,5 @@ +BEGIN +{ + printf("probe: success\n"); + exit(0); +} diff --git a/Lib/test/dtracedata/assert_usable.stp b/Lib/test/dtracedata/assert_usable.stp new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/assert_usable.stp @@ -0,0 +1,5 @@ +probe begin +{ + println("probe: success") + exit () +} diff --git a/Lib/test/dtracedata/call_stack.d b/Lib/test/dtracedata/call_stack.d new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/call_stack.d @@ -0,0 +1,31 @@ +self int indent; + +python$target:::function-entry +/copyinstr(arg1) == "start"/ +{ + self->trace = 1; +} + +python$target:::function-entry +/self->trace/ +{ + printf("%d\t%*s:", timestamp, 15, probename); + printf("%*s", self->indent, ""); + printf("%s:%s:%d\n", basename(copyinstr(arg0)), copyinstr(arg1), arg2); + self->indent++; +} + +python$target:::function-return +/self->trace/ +{ + self->indent--; + printf("%d\t%*s:", timestamp, 15, probename); + printf("%*s", self->indent, ""); + printf("%s:%s:%d\n", basename(copyinstr(arg0)), copyinstr(arg1), arg2); +} + +python$target:::function-return +/copyinstr(arg1) == "start"/ +{ + self->trace = 0; +} diff --git a/Lib/test/dtracedata/call_stack.d.expected b/Lib/test/dtracedata/call_stack.d.expected new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/call_stack.d.expected @@ -0,0 +1,18 @@ + function-entry:call_stack.py:start:23 + function-entry: call_stack.py:function_1:1 + function-entry: call_stack.py:function_3:9 +function-return: call_stack.py:function_3:10 +function-return: call_stack.py:function_1:2 + function-entry: call_stack.py:function_2:5 + function-entry: call_stack.py:function_1:1 + function-entry: call_stack.py:function_3:9 +function-return: call_stack.py:function_3:10 +function-return: call_stack.py:function_1:2 +function-return: call_stack.py:function_2:6 + function-entry: call_stack.py:function_3:9 +function-return: call_stack.py:function_3:10 + function-entry: call_stack.py:function_4:13 +function-return: call_stack.py:function_4:14 + function-entry: call_stack.py:function_5:18 +function-return: call_stack.py:function_5:21 +function-return:call_stack.py:start:28 diff --git a/Lib/test/dtracedata/call_stack.py b/Lib/test/dtracedata/call_stack.py new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/call_stack.py @@ -0,0 +1,30 @@ +def function_1(): + function_3(1, 2) + +# Check stacktrace +def function_2(): + function_1() + +# CALL_FUNCTION_VAR +def function_3(dummy, dummy2): + pass + +# CALL_FUNCTION_KW +def function_4(**dummy): + return 1 + return 2 # unreachable + +# CALL_FUNCTION_VAR_KW +def function_5(dummy, dummy2, **dummy3): + if False: + return 7 + return 8 + +def start(): + function_1() + function_2() + function_3(1, 2) + function_4(test=42) + function_5(*(1, 2), **{"test": 42}) + +start() diff --git a/Lib/test/dtracedata/call_stack.stp b/Lib/test/dtracedata/call_stack.stp new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/call_stack.stp @@ -0,0 +1,41 @@ +global tracing + +function basename:string(path:string) +{ + last_token = token = tokenize(path, "/"); + while (token != "") { + last_token = token; + token = tokenize("", "/"); + } + return last_token; +} + +probe process.mark("function__entry") +{ + funcname = user_string($arg2); + + if (funcname == "start") { + tracing = 1; + } +} + +probe process.mark("function__entry"), process.mark("function__return") +{ + filename = user_string($arg1); + funcname = user_string($arg2); + lineno = $arg3; + + if (tracing) { + printf("%d\t%s:%s:%s:%d\n", gettimeofday_us(), $$name, + basename(filename), funcname, lineno); + } +} + +probe process.mark("function__return") +{ + funcname = user_string($arg2); + + if (funcname == "start") { + tracing = 0; + } +} diff --git a/Lib/test/dtracedata/call_stack.stp.expected b/Lib/test/dtracedata/call_stack.stp.expected new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/call_stack.stp.expected @@ -0,0 +1,14 @@ +function__entry:call_stack.py:start:23 +function__entry:call_stack.py:function_1:1 +function__return:call_stack.py:function_1:2 +function__entry:call_stack.py:function_2:5 +function__entry:call_stack.py:function_1:1 +function__return:call_stack.py:function_1:2 +function__return:call_stack.py:function_2:6 +function__entry:call_stack.py:function_3:9 +function__return:call_stack.py:function_3:10 +function__entry:call_stack.py:function_4:13 +function__return:call_stack.py:function_4:14 +function__entry:call_stack.py:function_5:18 +function__return:call_stack.py:function_5:21 +function__return:call_stack.py:start:28 diff --git a/Lib/test/dtracedata/gc.d b/Lib/test/dtracedata/gc.d new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/gc.d @@ -0,0 +1,18 @@ +python$target:::function-entry +/copyinstr(arg1) == "start"/ +{ + self->trace = 1; +} + +python$target:::gc-start, +python$target:::gc-done +/self->trace/ +{ + printf("%d\t%s:%ld\n", timestamp, probename, arg0); +} + +python$target:::function-return +/copyinstr(arg1) == "start"/ +{ + self->trace = 0; +} diff --git a/Lib/test/dtracedata/gc.d.expected b/Lib/test/dtracedata/gc.d.expected new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/gc.d.expected @@ -0,0 +1,8 @@ +gc-start:0 +gc-done:0 +gc-start:1 +gc-done:0 +gc-start:2 +gc-done:0 +gc-start:2 +gc-done:1 diff --git a/Lib/test/dtracedata/gc.py b/Lib/test/dtracedata/gc.py new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/gc.py @@ -0,0 +1,13 @@ +import gc + +def start(): + gc.collect(0) + gc.collect(1) + gc.collect(2) + l = [] + l.append(l) + del l + gc.collect(2) + +gc.collect() +start() diff --git a/Lib/test/dtracedata/gc.stp b/Lib/test/dtracedata/gc.stp new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/gc.stp @@ -0,0 +1,26 @@ +global tracing + +probe process.mark("function__entry") +{ + funcname = user_string($arg2); + + if (funcname == "start") { + tracing = 1; + } +} + +probe process.mark("gc__start"), process.mark("gc__done") +{ + if (tracing) { + printf("%d\t%s:%ld\n", gettimeofday_us(), $$name, $arg1); + } +} + +probe process.mark("function__return") +{ + funcname = user_string($arg2); + + if (funcname == "start") { + tracing = 0; + } +} diff --git a/Lib/test/dtracedata/gc.stp.expected b/Lib/test/dtracedata/gc.stp.expected new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/gc.stp.expected @@ -0,0 +1,8 @@ +gc__start:0 +gc__done:0 +gc__start:1 +gc__done:0 +gc__start:2 +gc__done:0 +gc__start:2 +gc__done:1 diff --git a/Lib/test/dtracedata/instance.py b/Lib/test/dtracedata/instance.py new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/instance.py @@ -0,0 +1,24 @@ +import gc + +class old_style_class(): + pass +class new_style_class(object): + pass + +a = old_style_class() +del a +gc.collect() +b = new_style_class() +del b +gc.collect() + +a = old_style_class() +del old_style_class +gc.collect() +b = new_style_class() +del new_style_class +gc.collect() +del a +gc.collect() +del b +gc.collect() diff --git a/Lib/test/dtracedata/line.d b/Lib/test/dtracedata/line.d new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/line.d @@ -0,0 +1,7 @@ +python$target:::line +/(copyinstr(arg1)=="test_line")/ +{ + printf("%d\t%s:%s:%s:%d\n", timestamp, + probename, basename(copyinstr(arg0)), + copyinstr(arg1), arg2); +} diff --git a/Lib/test/dtracedata/line.d.expected b/Lib/test/dtracedata/line.d.expected new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/line.d.expected @@ -0,0 +1,20 @@ +line:line.py:test_line:2 +line:line.py:test_line:3 +line:line.py:test_line:4 +line:line.py:test_line:5 +line:line.py:test_line:6 +line:line.py:test_line:7 +line:line.py:test_line:8 +line:line.py:test_line:9 +line:line.py:test_line:10 +line:line.py:test_line:11 +line:line.py:test_line:4 +line:line.py:test_line:5 +line:line.py:test_line:6 +line:line.py:test_line:7 +line:line.py:test_line:8 +line:line.py:test_line:10 +line:line.py:test_line:11 +line:line.py:test_line:4 +line:line.py:test_line:12 +line:line.py:test_line:13 diff --git a/Lib/test/dtracedata/line.py b/Lib/test/dtracedata/line.py new file mode 100644 --- /dev/null +++ b/Lib/test/dtracedata/line.py @@ -0,0 +1,17 @@ +def test_line(): + a = 1 + print('# Preamble', a) + for i in range(2): + a = i + b = i+2 + c = i+3 + if c < 4: + a = c + d = a + b +c + print('#', a, b, c, d) + a = 1 + print('# Epilogue', a) + + +if __name__ == '__main__': + test_line() diff --git a/Lib/test/test_dtrace.py b/Lib/test/test_dtrace.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_dtrace.py @@ -0,0 +1,178 @@ +import dis +import os.path +import re +import subprocess +import sys +import types +import unittest + +from test.support import findfile, run_unittest + + +def abspath(filename): + return os.path.abspath(findfile(filename, subdir="dtracedata")) + + +def normalize_trace_output(output): + """Normalize DTrace output for comparison. + + DTrace keeps a per-CPU buffer, and when showing the fired probes, buffers + are concatenated. So if the operating system moves our thread around, the + straight result can be "non-causal". So we add timestamps to the probe + firing, sort by that field, then strip it from the output""" + + # When compiling with '--with-pydebug', strip '[# refs]' debug output. + output = re.sub(r"\[[0-9]+ refs\]", "", output) + try: + result = [ + row.split("\t") + for row in output.splitlines() + if row and not row.startswith('#') + ] + result.sort(key=lambda row: int(row[0])) + result = [row[1] for row in result] + return "\n".join(result) + except (IndexError, ValueError): + raise AssertionError( + "tracer produced unparseable output:\n{}".format(output) + ) + + +class TraceBackend: + EXTENSION = None + COMMAND = None + COMMAND_ARGS = [] + + def run_case(self, name, optimize_python=None): + actual_output = normalize_trace_output(self.trace_python( + script_file=abspath(name + self.EXTENSION), + python_file=abspath(name + ".py"), + optimize_python=optimize_python)) + + with open(abspath(name + self.EXTENSION + ".expected")) as f: + expected_output = f.read().rstrip() + + return (expected_output, actual_output) + + def generate_trace_command(self, script_file, subcommand=None): + command = self.COMMAND + [script_file] + if subcommand: + command += ["-c", subcommand] + return command + + def trace(self, script_file, subcommand=None): + command = self.generate_trace_command(script_file, subcommand) + stdout, _ = subprocess.Popen(command, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + universal_newlines=True).communicate() + return stdout + + def trace_python(self, script_file, python_file, optimize_python=None): + python_flags = [] + if optimize_python: + python_flags.extend(["-O"] * optimize_python) + subcommand = " ".join([sys.executable] + python_flags + [python_file]) + return self.trace(script_file, subcommand) + + def assert_usable(self): + try: + output = self.trace(abspath("assert_usable" + self.EXTENSION)) + output = output.strip() + except FileNotFoundError as fnfe: + output = str(fnfe) + if output != "probe: success": + raise unittest.SkipTest( + "{}(1) failed: {}".format(self.COMMAND[0], output) + ) + + +class DTraceBackend(TraceBackend): + EXTENSION = ".d" + COMMAND = ["dtrace", "-q", "-s"] + + +class SystemTapBackend(TraceBackend): + EXTENSION = ".stp" + COMMAND = ["stap", "-g"] + + +class TraceTests(unittest.TestCase): + # unittest.TestCase options + maxDiff = None + + # TraceTests options + backend = None + optimize_python = 0 + + @classmethod + def setUpClass(self): + self.backend.assert_usable() + + def run_case(self, name): + actual_output, expected_output = self.backend.run_case( + name, optimize_python=self.optimize_python) + self.assertEqual(actual_output, expected_output) + + def test_function_entry_return(self): + self.run_case("call_stack") + + def test_verify_call_opcodes(self): + """Ensure our call stack test hits all function call opcodes""" + + opcodes = set(["CALL_FUNCTION", "CALL_FUNCTION_EX", "CALL_FUNCTION_KW"]) + + with open(abspath("call_stack.py")) as f: + code_string = f.read() + + def get_function_instructions(funcname): + # Recompile with appropriate optimization setting + code = compile(source=code_string, + filename="", + mode="exec", + optimize=self.optimize_python) + + for c in code.co_consts: + if isinstance(c, types.CodeType) and c.co_name == funcname: + return dis.get_instructions(c) + return [] + + for instruction in get_function_instructions('start'): + opcodes.discard(instruction.opname) + + self.assertEqual(set(), opcodes) + + def test_gc(self): + self.run_case("gc") + + def test_line(self): + self.run_case("line") + + +class DTraceNormalTests(TraceTests): + backend = DTraceBackend() + optimize_python = 0 + + +class DTraceOptimizedTests(TraceTests): + backend = DTraceBackend() + optimize_python = 2 + + +class SystemTapNormalTests(TraceTests): + backend = SystemTapBackend() + optimize_python = 0 + + +class SystemTapOptimizedTests(TraceTests): + backend = SystemTapBackend() + optimize_python = 2 + + +def test_main(): + run_unittest(DTraceNormalTests, DTraceOptimizedTests, SystemTapNormalTests, + SystemTapOptimizedTests) + + +if __name__ == '__main__': + test_main() diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -48,6 +48,10 @@ LLVM_PROF_MERGER=@LLVM_PROF_MERGER@ LLVM_PROF_FILE=@LLVM_PROF_FILE@ LLVM_PROF_ERR=@LLVM_PROF_ERR@ +DTRACE= @DTRACE@ +DFLAGS= @DFLAGS@ +DTRACE_HEADERS= @DTRACE_HEADERS@ +DTRACE_OBJS= @DTRACE_OBJS@ GNULD= @GNULD@ @@ -315,7 +319,7 @@ OPCODE_H_SCRIPT= $(srcdir)/Tools/scripts/generate_opcode_h.py OPCODE_H= $(OPCODE_H_DIR)/opcode.h OPCODE_H_GEN= $(PYTHON_FOR_GEN) $(OPCODE_H_SCRIPT) $(srcdir)/Lib/opcode.py $(OPCODE_H) -# + ########################################################################## # AST AST_H_DIR= Include @@ -391,7 +395,8 @@ Python/$(DYNLOADFILE) \ $(LIBOBJS) \ $(MACHDEP_OBJS) \ - $(THREADOBJ) + $(THREADOBJ) \ + $(DTRACE_OBJS) ########################################################################## @@ -451,6 +456,15 @@ $(LIBRARY_OBJS_OMIT_FROZEN) \ Python/frozen.o +########################################################################## +# DTrace + +# On some systems, object files that reference DTrace probes need to be modified +# in-place by dtrace(1). +DTRACE_DEPS = \ + Python/ceval.o +# XXX: should gcmodule, etc. be here, too? + ######################################################################### # Rules @@ -852,6 +866,18 @@ Python/frozen.o: Python/importlib.h Python/importlib_external.h +# Generate DTrace probe macros, then rename them (PYTHON_ -> PyDTrace_) to +# follow our naming conventions. dtrace(1) uses the output filename to generate +# an include guard, so we can't use a pipeline to transform its output. +Include/pydtrace_probes.h: $(srcdir)/Include/pydtrace.d + $(DTRACE) $(DFLAGS) -o $@ -h -s $< + : sed in-place edit with POSIX-only tools + sed 's/PYTHON_/PyDTrace_/' $@ > $@.tmp + mv $@.tmp $@ + +Python/pydtrace.o: $(srcdir)/Include/pydtrace.d $(DTRACE_DEPS) + $(DTRACE) $(DFLAGS) -o $@ -G -s $< $(DTRACE_DEPS) + Objects/typeobject.o: Objects/typeslots.inc Objects/typeslots.inc: $(srcdir)/Include/typeslots.h $(srcdir)/Objects/typeslots.py $(PYTHON_FOR_GEN) $(srcdir)/Objects/typeslots.py < $(srcdir)/Include/typeslots.h Objects/typeslots.inc @@ -918,6 +944,7 @@ $(srcdir)/Include/pycapsule.h \ $(srcdir)/Include/pyctype.h \ $(srcdir)/Include/pydebug.h \ + $(srcdir)/Include/pydtrace.h \ $(srcdir)/Include/pyerrors.h \ $(srcdir)/Include/pyfpe.h \ $(srcdir)/Include/pyhash.h \ @@ -949,7 +976,8 @@ $(srcdir)/Include/weakrefobject.h \ pyconfig.h \ $(PARSER_HEADERS) \ - $(AST_H) + $(AST_H) \ + $(DTRACE_HEADERS) $(LIBRARY_OBJS) $(MODOBJS) Programs/python.o: $(PYTHON_HEADERS) @@ -1158,6 +1186,7 @@ test/audiodata \ test/capath test/data \ test/cjkencodings test/decimaltestdata test/xmltestdata \ + test/dtracedata \ test/eintrdata \ test/imghdrdata \ test/libregrtest \ @@ -1569,6 +1598,7 @@ -rm -f Lib/lib2to3/*Grammar*.pickle -rm -f Programs/_testembed Programs/_freeze_importlib -find build -type f -a ! -name '*.gc??' -exec rm -f {} ';' + -rm -f Include/pydtrace_probes.h profile-removal: find . -name '*.gc??' -exec rm -f {} ';' diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -120,6 +120,7 @@ Thomas Bellman Alexander ?????? Belopolsky Eli Bendersky +Nikhil Benesch David Benjamin Oscar Benjamin Andrew Bennetts diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -329,6 +329,8 @@ make time when --with-optimizations is enabled. Also improve our ability to find the llvm-profdata tool on MacOS and some Linuxes. +- Issue #21590: Support for DTrace and SystemTap probes. + - Issue #26307: The profile-opt build now applys PGO to the built-in modules. - Issue #26539: Add the --with-optimizations flag to turn on LTO and PGO build diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c --- a/Modules/gcmodule.c +++ b/Modules/gcmodule.c @@ -25,6 +25,7 @@ #include "Python.h" #include "frameobject.h" /* for PyFrame_ClearFreeList */ +#include "pydtrace.h" #include "pytime.h" /* for _PyTime_GetMonotonicClock() */ /* Get an object's GC head */ @@ -925,6 +926,9 @@ PySys_WriteStderr("\n"); } + if (PyDTrace_GC_START_ENABLED()) + PyDTrace_GC_START(generation); + /* update collection and allocation counters */ if (generation+1 < NUM_GENERATIONS) generations[generation+1].count += 1; @@ -1069,6 +1073,10 @@ stats->collections++; stats->collected += m; stats->uncollectable += n; + + if (PyDTrace_GC_DONE_ENABLED()) + PyDTrace_GC_DONE(n+m); + return n+m; } diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -15,6 +15,7 @@ #include "dictobject.h" #include "frameobject.h" #include "opcode.h" +#include "pydtrace.h" #include "setobject.h" #include "structmember.h" @@ -50,6 +51,9 @@ PyThreadState *, PyFrameObject *); static int maybe_call_line_trace(Py_tracefunc, PyObject *, PyThreadState *, PyFrameObject *, int *, int *, int *); +static void maybe_dtrace_line(PyFrameObject *, int *, int *, int *); +static void dtrace_function_entry(PyFrameObject *); +static void dtrace_function_return(PyFrameObject *); static PyObject * cmp_outcome(int, PyObject *, PyObject *); static PyObject * import_name(PyFrameObject *, PyObject *, PyObject *, PyObject *); @@ -822,7 +826,7 @@ #ifdef LLTRACE #define FAST_DISPATCH() \ { \ - if (!lltrace && !_Py_TracingPossible) { \ + if (!lltrace && !_Py_TracingPossible && !PyDTrace_LINE_ENABLED()) { \ f->f_lasti = INSTR_OFFSET(); \ NEXTOPARG(); \ goto *opcode_targets[opcode]; \ @@ -832,7 +836,7 @@ #else #define FAST_DISPATCH() \ { \ - if (!_Py_TracingPossible) { \ + if (!_Py_TracingPossible && !PyDTrace_LINE_ENABLED()) { \ f->f_lasti = INSTR_OFFSET(); \ NEXTOPARG(); \ goto *opcode_targets[opcode]; \ @@ -1042,6 +1046,9 @@ } } + if (PyDTrace_FUNCTION_ENTRY_ENABLED()) + dtrace_function_entry(f); + co = f->f_code; names = co->co_names; consts = co->co_consts; @@ -1162,6 +1169,9 @@ fast_next_opcode: f->f_lasti = INSTR_OFFSET(); + if (PyDTrace_LINE_ENABLED()) + maybe_dtrace_line(f, &instr_lb, &instr_ub, &instr_prev); + /* line-by-line tracing support */ if (_Py_TracingPossible && @@ -3620,6 +3630,8 @@ /* pop frame */ exit_eval_frame: + if (PyDTrace_FUNCTION_RETURN_ENABLED()) + dtrace_function_return(f); Py_LeaveRecursiveCall(); f->f_executing = 0; tstate->frame = f->f_back; @@ -5415,3 +5427,65 @@ tstate->co_extra_freefuncs[new_index] = free; return new_index; } + +static void +dtrace_function_entry(PyFrameObject *f) +{ + char* filename; + char* funcname; + int lineno; + + filename = PyUnicode_AsUTF8(f->f_code->co_filename); + funcname = PyUnicode_AsUTF8(f->f_code->co_name); + lineno = PyCode_Addr2Line(f->f_code, f->f_lasti); + + PyDTrace_FUNCTION_ENTRY(filename, funcname, lineno); +} + +static void +dtrace_function_return(PyFrameObject *f) +{ + char* filename; + char* funcname; + int lineno; + + filename = PyUnicode_AsUTF8(f->f_code->co_filename); + funcname = PyUnicode_AsUTF8(f->f_code->co_name); + lineno = PyCode_Addr2Line(f->f_code, f->f_lasti); + + PyDTrace_FUNCTION_RETURN(filename, funcname, lineno); +} + +/* DTrace equivalent of maybe_call_line_trace. */ +static void +maybe_dtrace_line(PyFrameObject *frame, + int *instr_lb, int *instr_ub, int *instr_prev) +{ + int line = frame->f_lineno; + char *co_filename, *co_name; + + /* If the last instruction executed isn't in the current + instruction window, reset the window. + */ + if (frame->f_lasti < *instr_lb || frame->f_lasti >= *instr_ub) { + PyAddrPair bounds; + line = _PyCode_CheckLineNumber(frame->f_code, frame->f_lasti, + &bounds); + *instr_lb = bounds.ap_lower; + *instr_ub = bounds.ap_upper; + } + /* If the last instruction falls at the start of a line or if + it represents a jump backwards, update the frame's line + number and call the trace function. */ + if (frame->f_lasti == *instr_lb || frame->f_lasti < *instr_prev) { + frame->f_lineno = line; + co_filename = PyUnicode_AsUTF8(frame->f_code->co_filename); + if (!co_filename) + co_filename = "?"; + co_name = PyUnicode_AsUTF8(frame->f_code->co_name); + if (!co_name) + co_name = "?"; + PyDTrace_LINE(co_filename, co_name, line); + } + *instr_prev = frame->f_lasti; +} diff --git a/configure b/configure --- a/configure +++ b/configure @@ -642,6 +642,10 @@ MACHDEP_OBJS DYNLOADFILE DLINCLDIR +DTRACE_OBJS +DTRACE_HEADERS +DFLAGS +DTRACE THREADOBJ LDLAST USE_THREAD_MODULE @@ -713,6 +717,7 @@ ac_ct_CXX MAINCC CXX +SED GREP CPP OBJEXT @@ -780,7 +785,6 @@ docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir @@ -832,6 +836,7 @@ with_doc_strings with_pymalloc with_valgrind +with_dtrace with_fpectl with_libm with_libc @@ -890,7 +895,6 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1143,15 +1147,6 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1289,7 +1284,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1442,7 +1437,6 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -1535,6 +1529,7 @@ --with(out)-doc-strings disable/enable documentation strings --with(out)-pymalloc disable/enable specialized mallocs --with-valgrind Enable Valgrind support + --with(out)-dtrace disable/enable DTrace support --with-fpectl enable SIGFPE catching --with-libm=STRING math library --with-libc=STRING C library @@ -4581,6 +4576,75 @@ GREP="$ac_cv_path_GREP" +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for a sed that does not truncate output" >&5 +$as_echo_n "checking for a sed that does not truncate output... " >&6; } +if ${ac_cv_path_SED+:} false; then : + $as_echo_n "(cached) " >&6 +else + ac_script=s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/ + for ac_i in 1 2 3 4 5 6 7; do + ac_script="$ac_script$as_nl$ac_script" + done + echo "$ac_script" 2>/dev/null | sed 99q >conftest.sed + { ac_script=; unset ac_script;} + if test -z "$SED"; then + ac_path_SED_found=false + # Loop through the user's path and test for each of PROGNAME-LIST + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_prog in sed gsed; do + for ac_exec_ext in '' $ac_executable_extensions; do + ac_path_SED="$as_dir/$ac_prog$ac_exec_ext" + as_fn_executable_p "$ac_path_SED" || continue +# Check for GNU ac_path_SED and select it if it is found. + # Check for GNU $ac_path_SED +case `"$ac_path_SED" --version 2>&1` in +*GNU*) + ac_cv_path_SED="$ac_path_SED" ac_path_SED_found=:;; +*) + ac_count=0 + $as_echo_n 0123456789 >"conftest.in" + while : + do + cat "conftest.in" "conftest.in" >"conftest.tmp" + mv "conftest.tmp" "conftest.in" + cp "conftest.in" "conftest.nl" + $as_echo '' >> "conftest.nl" + "$ac_path_SED" -f conftest.sed < "conftest.nl" >"conftest.out" 2>/dev/null || break + diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break + as_fn_arith $ac_count + 1 && ac_count=$as_val + if test $ac_count -gt ${ac_path_SED_max-0}; then + # Best one so far, save it but keep looking for a better one + ac_cv_path_SED="$ac_path_SED" + ac_path_SED_max=$ac_count + fi + # 10*(2^10) chars as input seems more than enough + test $ac_count -gt 10 && break + done + rm -f conftest.in conftest.tmp conftest.nl conftest.out;; +esac + + $ac_path_SED_found && break 3 + done + done + done +IFS=$as_save_IFS + if test -z "$ac_cv_path_SED"; then + as_fn_error $? "no acceptable sed could be found in \$PATH" "$LINENO" 5 + fi +else + ac_cv_path_SED=$SED +fi + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_SED" >&5 +$as_echo "$ac_cv_path_SED" >&6; } + SED="$ac_cv_path_SED" + rm -f conftest.sed + @@ -10864,6 +10928,102 @@ OPT="-DDYNAMIC_ANNOTATIONS_ENABLED=1 $OPT" fi +# Check for DTrace support +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-dtrace" >&5 +$as_echo_n "checking for --with-dtrace... " >&6; } + +# Check whether --with-dtrace was given. +if test "${with_dtrace+set}" = set; then : + withval=$with_dtrace; +else + with_dtrace=no +fi + +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_dtrace" >&5 +$as_echo "$with_dtrace" >&6; } + + + + + +DTRACE= +DFLAGS= +DTRACE_HEADERS= +DTRACE_OBJS= + +if test "$with_dtrace" = "yes" +then + # Extract the first word of "dtrace", so it can be a program name with args. +set dummy dtrace; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_DTRACE+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $DTRACE in + [\\/]* | ?:[\\/]*) + ac_cv_path_DTRACE="$DTRACE" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_DTRACE="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + test -z "$ac_cv_path_DTRACE" && ac_cv_path_DTRACE="not found" + ;; +esac +fi +DTRACE=$ac_cv_path_DTRACE +if test -n "$DTRACE"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DTRACE" >&5 +$as_echo "$DTRACE" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + + if test "$DTRACE" = "not found"; then + as_fn_error $? "dtrace command not found on \$PATH" "$LINENO" 5 + fi + +$as_echo "#define WITH_DTRACE 1" >>confdefs.h + + DTRACE_HEADERS="Include/pydtrace_probes.h" + + # On OS X, DTrace providers do not need to be explicitly compiled and + # linked into the binary. Correspondingly, dtrace(1) is missing the ELF + # generation flag '-G'. We check for presence of this flag, rather than + # hardcoding support by OS, in the interest of robustness. + { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether DTrace probes require linking" >&5 +$as_echo_n "checking whether DTrace probes require linking... " >&6; } +if ${ac_cv_dtrace_link+:} false; then : + $as_echo_n "(cached) " >&6 +else + ac_cv_dtrace_link=no + echo 'BEGIN' > conftest.d + "$DTRACE" -G -s conftest.d -o conftest.o > /dev/null 2>&1 && \ + ac_cv_dtrace_link=yes + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_dtrace_link" >&5 +$as_echo "$ac_cv_dtrace_link" >&6; } + if test "$ac_cv_dtrace_link" = "yes"; then + DTRACE_OBJS="Python/pydtrace.o" + fi +fi + # -I${DLINCLDIR} is added to the compile rule for importdl.o DLINCLDIR=. diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -694,6 +694,7 @@ AC_PROG_CC AC_PROG_CPP AC_PROG_GREP +AC_PROG_SED AC_SUBST(CXX) AC_SUBST(MAINCC) @@ -3245,6 +3246,47 @@ OPT="-DDYNAMIC_ANNOTATIONS_ENABLED=1 $OPT" fi +# Check for DTrace support +AC_MSG_CHECKING(for --with-dtrace) +AC_ARG_WITH(dtrace, + AC_HELP_STRING(--with(out)-dtrace, [disable/enable DTrace support]),, + with_dtrace=no) +AC_MSG_RESULT($with_dtrace) + +AC_SUBST(DTRACE) +AC_SUBST(DFLAGS) +AC_SUBST(DTRACE_HEADERS) +AC_SUBST(DTRACE_OBJS) +DTRACE= +DFLAGS= +DTRACE_HEADERS= +DTRACE_OBJS= + +if test "$with_dtrace" = "yes" +then + AC_PATH_PROG(DTRACE, [dtrace], [not found]) + if test "$DTRACE" = "not found"; then + AC_MSG_ERROR([dtrace command not found on \$PATH]) + fi + AC_DEFINE(WITH_DTRACE, 1, [Define if you want to compile in DTrace support]) + DTRACE_HEADERS="Include/pydtrace_probes.h" + + # On OS X, DTrace providers do not need to be explicitly compiled and + # linked into the binary. Correspondingly, dtrace(1) is missing the ELF + # generation flag '-G'. We check for presence of this flag, rather than + # hardcoding support by OS, in the interest of robustness. + AC_CACHE_CHECK([whether DTrace probes require linking], + [ac_cv_dtrace_link], [dnl + ac_cv_dtrace_link=no + echo 'BEGIN' > conftest.d + "$DTRACE" -G -s conftest.d -o conftest.o > /dev/null 2>&1 && \ + ac_cv_dtrace_link=yes + ]) + if test "$ac_cv_dtrace_link" = "yes"; then + DTRACE_OBJS="Python/pydtrace.o" + fi +fi + # -I${DLINCLDIR} is added to the compile rule for importdl.o AC_SUBST(DLINCLDIR) DLINCLDIR=. diff --git a/pyconfig.h.in b/pyconfig.h.in --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -1370,6 +1370,9 @@ /* Define if you want documentation strings in extension modules */ #undef WITH_DOC_STRINGS +/* Define if you want to compile in DTrace support */ +#undef WITH_DTRACE + /* Define if you want to use the new-style (Openstep, Rhapsody, MacOS) dynamic linker (dyld) instead of the old-style (NextStep) dynamic linker (rld). Dyld is necessary to support frameworks. */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:46:29 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 10 Sep 2016 00:46:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_dummy_macro?= Message-ID: <20160910004629.59383.85263.9BA523B4@psf.io> https://hg.python.org/cpython/rev/640c123f7461 changeset: 103512:640c123f7461 user: Benjamin Peterson date: Fri Sep 09 17:46:24 2016 -0700 summary: fix dummy macro files: Include/pydtrace.h | 2 +- Modules/_datetimemodule.c | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Include/pydtrace.h b/Include/pydtrace.h --- a/Include/pydtrace.h +++ b/Include/pydtrace.h @@ -22,7 +22,7 @@ /* Without DTrace, compile to nothing. */ -#define PyDTrace_LINE(arg0, arg1, arg2, arg3) do ; while (0) +#define PyDTrace_LINE(arg0, arg1, arg2) do ; while (0) #define PyDTrace_FUNCTION_ENTRY(arg0, arg1, arg2) do ; while (0) #define PyDTrace_FUNCTION_RETURN(arg0, arg1, arg2) do ; while (0) #define PyDTrace_GC_START(arg0) do ; while (0) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -2841,9 +2841,10 @@ static Py_hash_t date_hash(PyDateTime_Date *self) { - if (self->hashcode == -1) + if (self->hashcode == -1) { self->hashcode = generic_hash( (unsigned char *)self->data, _PyDateTime_DATE_DATASIZE); + } return self->hashcode; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 20:47:50 2016 From: python-checkins at python.org (zach.ware) Date: Sat, 10 Sep 2016 00:47:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Actually_fix_suspicious_ma?= =?utf-8?q?rkup=2C_I_ignored_it_too_readily?= Message-ID: <20160910004750.36170.31086.A0F4BC1A@psf.io> https://hg.python.org/cpython/rev/55d0860f5bb8 changeset: 103513:55d0860f5bb8 user: Zachary Ware date: Fri Sep 09 17:47:38 2016 -0700 summary: Actually fix suspicious markup, I ignored it too readily files: Doc/tools/susp-ignored.csv | 1 - Doc/whatsnew/3.6.rst | 2 +- 2 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -297,7 +297,6 @@ whatsnew/3.5,,::,>>> addr6 = ipaddress.IPv6Address('::1') whatsnew/3.5,,:root,ERROR:root:exception whatsnew/3.5,,:exception,ERROR:root:exception -whatsnew/3.6,,`,1000000000000000` whatsnew/changelog,,:version,import sys; I = version[:version.index(' ')] whatsnew/changelog,,:gz,": TarFile opened with external fileobj and ""w:gz"" mode didn't" whatsnew/changelog,,::,": Use ""127.0.0.1"" or ""::1"" instead of ""localhost"" as much as" diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -139,7 +139,7 @@ Prior to PEP 515, there was no support for writing long numeric literals with some form of separator to improve readability. For -instance, how big is ``1000000000000000```? With :pep:`515`, though, +instance, how big is ``1000000000000000``? With :pep:`515`, though, you can use underscores to separate digits as desired to make numeric literals easier to read: ``1_000_000_000_000_000``. Underscores can be used with other numeric literals beyond integers, e.g. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:00:04 2016 From: python-checkins at python.org (zach.ware) Date: Sat, 10 Sep 2016 01:00:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328046=3A_Remove_p?= =?utf-8?q?latform-specific_directories_from_sys=2Epath?= Message-ID: <20160910010004.19090.6420.7CD02168@psf.io> https://hg.python.org/cpython/rev/90396ec9a2f8 changeset: 103514:90396ec9a2f8 user: Zachary Ware date: Fri Sep 09 17:59:49 2016 -0700 summary: Issue #28046: Remove platform-specific directories from sys.path files: .gitignore | 1 - .hgignore | 1 - Lib/sysconfig.py | 13 +++++++++++-- Mac/BuildScript/build-installer.py | 3 ++- Makefile.pre.in | 18 +++++------------- Misc/NEWS | 2 ++ PC/getpathp.c | 4 ++-- Tools/msi/make_zip.py | 2 -- configure | 9 +-------- configure.ac | 8 +------- 10 files changed, 24 insertions(+), 37 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -21,7 +21,6 @@ Lib/distutils/command/*.pdb Lib/lib2to3/*.pickle Lib/test/data/* -Lib/plat-mac/errors.rsrc.df.rsrc Makefile Makefile.pre Misc/python.pc diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -26,7 +26,6 @@ python-config.py$ reflog.txt$ tags$ -Lib/plat-mac/errors.rsrc.df.rsrc Misc/python.pc Misc/python-config.sh$ Modules/Setup$ diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -341,6 +341,15 @@ config_dir_name += '-%s' % sys.implementation._multiarch return os.path.join(get_path('stdlib'), config_dir_name, 'Makefile') + +def _get_sysconfigdata_name(): + return '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( + abi=sys.abiflags, + platform=sys.platform, + multiarch=getattr(sys.implementation, '_multiarch', ''), + ) + + def _generate_posix_vars(): """Generate the Python module containing build-time variables.""" import pprint @@ -381,7 +390,7 @@ # _sysconfigdata module manually and populate it with the build vars. # This is more than sufficient for ensuring the subsequent call to # get_platform() succeeds. - name = '_sysconfigdata_' + sys.abiflags + name = _get_sysconfigdata_name() if 'darwin' in sys.platform: import types module = types.ModuleType(name) @@ -407,7 +416,7 @@ def _init_posix(vars): """Initialize the module as appropriate for POSIX systems.""" # _sysconfigdata is generated at build time, see _generate_posix_vars() - name = '_sysconfigdata_' + sys.abiflags + name = _get_sysconfigdata_name() _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) build_time_vars = _temp.build_time_vars vars.update(build_time_vars) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -1292,7 +1292,8 @@ import pprint if getVersionMajorMinor() >= (3, 6): - path = os.path.join(path_to_lib, 'plat-darwin', '_sysconfigdata_m.py') + # XXX this is extra-fragile + path = os.path.join(path_to_lib, '_sysconfigdata_m_darwin_darwin.py') else: path = os.path.join(path_to_lib, '_sysconfigdata.py') fp = open(path, 'r') diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1178,8 +1178,6 @@ (cd $(DESTDIR)$(MANDIR)/man1; $(LN) -s python$(VERSION).1 python3.1) # Install the library -PLATDIR= @PLATDIR@ -MACHDEPS= $(PLATDIR) XMLLIBSUBDIRS= xml xml/dom xml/etree xml/parsers xml/sax LIBSUBDIRS= tkinter tkinter/test tkinter/test/test_tkinter \ tkinter/test/test_ttk site-packages test \ @@ -1238,8 +1236,8 @@ multiprocessing multiprocessing/dummy \ unittest unittest/test unittest/test/testmock \ venv venv/scripts venv/scripts/posix \ - curses pydoc_data $(MACHDEPS) -libinstall: build_all $(srcdir)/Lib/$(PLATDIR) $(srcdir)/Modules/xxmodule.c + curses pydoc_data +libinstall: build_all $(srcdir)/Modules/xxmodule.c @for i in $(SCRIPTDIR) $(LIBDEST); \ do \ if test ! -d $(DESTDIR)$$i; then \ @@ -1294,10 +1292,10 @@ esac; \ done; \ done - $(INSTALL_DATA) `cat pybuilddir.txt`/_sysconfigdata_$(ABIFLAGS).py \ - $(DESTDIR)$(LIBDEST)/$(PLATDIR); \ + $(INSTALL_DATA) `cat pybuilddir.txt`/_sysconfigdata_$(ABIFLAGS)_$(MACHDEP)_$(MULTIARCH).py \ + $(DESTDIR)$(LIBDEST); \ echo $(INSTALL_DATA) `cat pybuilddir.txt`/_sysconfigdata_$(ABIFLAGS).py \ - $(LIBDEST)/$(PLATDIR) + $(LIBDEST) $(INSTALL_DATA) $(srcdir)/LICENSE $(DESTDIR)$(LIBDEST)/LICENSE.txt if test -d $(DESTDIR)$(LIBDEST)/distutils/tests; then \ $(INSTALL_DATA) $(srcdir)/Modules/xxmodule.c \ @@ -1335,9 +1333,6 @@ -PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \ $(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/PatternGrammar.txt -$(srcdir)/Lib/$(PLATDIR): - mkdir $(srcdir)/Lib/$(PLATDIR) - python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh # Substitution happens here, as the completely-expanded BINDIR # is not available in configure @@ -1614,9 +1609,6 @@ -rm -rf build platform -rm -rf $(PYTHONFRAMEWORKDIR) -rm -f python-config.py python-config - if [ -n "$(MULTIARCH)" ]; then \ - rm -rf $(srcdir)/Lib/$(PLATDIR); \ - fi # Make things extra clean, before making a distribution: # remove all generated files, even Makefile[.pre] diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #28046: Remove platform-specific directories from sys.path. + - Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch by Eryk Sun) diff --git a/PC/getpathp.c b/PC/getpathp.c --- a/PC/getpathp.c +++ b/PC/getpathp.c @@ -26,7 +26,7 @@ is set, we believe it. Otherwise, we use the path of our host .EXE's to try and locate on of our "landmarks" and deduce our home. - If we DO have a Python Home: The relevant sub-directories (Lib, - plat-win, etc) are based on the Python Home + DLLs, etc) are based on the Python Home - If we DO NOT have a Python Home, the core Python Path is loaded from the registry. This is the main PythonPath key, and both HKLM and HKCU are combined to form the path) @@ -34,7 +34,7 @@ * Iff - we can not locate the Python Home, have not had a PYTHONPATH specified, and can't locate any Registry entries (ie, we have _nothing_ we can assume is a good path), a default path with relative entries is - used (eg. .\Lib;.\plat-win, etc) + used (eg. .\Lib;.\DLLs, etc) If a sys.path file exists adjacent to python.exe, it must contain a diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -67,8 +67,6 @@ if p.is_dir(): if name in EXCLUDE_FROM_LIBRARY: return False - if name.startswith('plat-'): - return False if name == 'test' and p.parts[-2].lower() == 'lib': return False if name in {'test', 'tests'} and p.parts[-3].lower() == 'lib': diff --git a/configure b/configure --- a/configure +++ b/configure @@ -712,7 +712,6 @@ NO_AS_NEEDED MULTIARCH_CPPFLAGS PLATFORM_TRIPLET -PLATDIR MULTIARCH ac_ct_CXX MAINCC @@ -2929,7 +2928,7 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $interp" >&5 $as_echo "$interp" >&6; } - PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp + PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib '$interp fi # Used to comment out stuff for rebuilding generated files GENERATED_COMMENT='#' @@ -5361,12 +5360,6 @@ elif test x$PLATFORM_TRIPLET != x && test x$MULTIARCH = x; then MULTIARCH=$PLATFORM_TRIPLET fi -if test x$PLATFORM_TRIPLET = x; then - PLATDIR=plat-$MACHDEP -else - PLATDIR=plat-$PLATFORM_TRIPLET -fi - if test x$MULTIARCH != x; then MULTIARCH_CPPFLAGS="-DMULTIARCH=\\\"$MULTIARCH\\\"" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -78,7 +78,7 @@ AC_MSG_ERROR([python$PACKAGE_VERSION interpreter not found]) fi AC_MSG_RESULT($interp) - PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp + PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib '$interp fi # Used to comment out stuff for rebuilding generated files GENERATED_COMMENT='#' @@ -910,12 +910,6 @@ elif test x$PLATFORM_TRIPLET != x && test x$MULTIARCH = x; then MULTIARCH=$PLATFORM_TRIPLET fi -if test x$PLATFORM_TRIPLET = x; then - PLATDIR=plat-$MACHDEP -else - PLATDIR=plat-$PLATFORM_TRIPLET -fi -AC_SUBST(PLATDIR) AC_SUBST(PLATFORM_TRIPLET) if test x$MULTIARCH != x; then MULTIARCH_CPPFLAGS="-DMULTIARCH=\\\"$MULTIARCH\\\"" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:05:23 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 01:05:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327932=3A_Fixes_memory_leak_in_platform=2Ewin32?= =?utf-8?b?X3Zlcigp?= Message-ID: <20160910010523.36379.54847.E6F92F20@psf.io> https://hg.python.org/cpython/rev/31b7eaff5588 changeset: 103516:31b7eaff5588 parent: 103513:55d0860f5bb8 parent: 103515:384c178cf823 user: Steve Dower date: Fri Sep 09 18:04:26 2016 -0700 summary: Issue #27932: Fixes memory leak in platform.win32_ver() files: Lib/platform.py | 64 +++++++++++++++++++----------------- Misc/NEWS | 2 + 2 files changed, 36 insertions(+), 30 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -497,57 +497,61 @@ (6, None): "post2012ServerR2", } +if sys.platform == 'win32': + import ctypes + import ctypes.wintypes + + class VS_FIXEDFILEINFO(ctypes.Structure): + _fields_ = [ + ("dwSignature", ctypes.wintypes.DWORD), + ("dwStrucVersion", ctypes.wintypes.DWORD), + ("dwFileVersionMS", ctypes.wintypes.DWORD), + ("dwFileVersionLS", ctypes.wintypes.DWORD), + ("dwProductVersionMS", ctypes.wintypes.DWORD), + ("dwProductVersionLS", ctypes.wintypes.DWORD), + ("dwFileFlagsMask", ctypes.wintypes.DWORD), + ("dwFileFlags", ctypes.wintypes.DWORD), + ("dwFileOS", ctypes.wintypes.DWORD), + ("dwFileType", ctypes.wintypes.DWORD), + ("dwFileSubtype", ctypes.wintypes.DWORD), + ("dwFileDateMS", ctypes.wintypes.DWORD), + ("dwFileDateLS", ctypes.wintypes.DWORD), + ] + + P_VS_FIXEDFILEINFO = ctypes.POINTER(VS_FIXEDFILEINFO) + def _get_real_winver(maj, min, build): if maj < 6 or (maj == 6 and min < 2): return maj, min, build - from ctypes import (c_buffer, POINTER, byref, create_unicode_buffer, - Structure, WinDLL) - from ctypes.wintypes import DWORD, HANDLE - - class VS_FIXEDFILEINFO(Structure): - _fields_ = [ - ("dwSignature", DWORD), - ("dwStrucVersion", DWORD), - ("dwFileVersionMS", DWORD), - ("dwFileVersionLS", DWORD), - ("dwProductVersionMS", DWORD), - ("dwProductVersionLS", DWORD), - ("dwFileFlagsMask", DWORD), - ("dwFileFlags", DWORD), - ("dwFileOS", DWORD), - ("dwFileType", DWORD), - ("dwFileSubtype", DWORD), - ("dwFileDateMS", DWORD), - ("dwFileDateLS", DWORD), - ] - - kernel32 = WinDLL('kernel32') - version = WinDLL('version') - + kernel32 = ctypes.WinDLL('kernel32') # We will immediately double the length up to MAX_PATH, but the # path may be longer, so we retry until the returned string is # shorter than our buffer. name_len = actual_len = 130 while actual_len == name_len: name_len *= 2 - name = create_unicode_buffer(name_len) - actual_len = kernel32.GetModuleFileNameW(HANDLE(kernel32._handle), - name, len(name)) + name = ctypes.create_unicode_buffer(name_len) + actual_len = kernel32.GetModuleFileNameW( + ctypes.wintypes.HANDLE(kernel32._handle), + name, len(name) + ) if not actual_len: return maj, min, build + version = ctypes.WinDLL('version') size = version.GetFileVersionInfoSizeW(name, None) if not size: return maj, min, build - ver_block = c_buffer(size) + ver_block = ctypes.c_buffer(size) if (not version.GetFileVersionInfoW(name, None, size, ver_block) or not ver_block): return maj, min, build - pvi = POINTER(VS_FIXEDFILEINFO)() - if not version.VerQueryValueW(ver_block, "", byref(pvi), byref(DWORD())): + pvi = P_VS_FIXEDFILEINFO() + if not version.VerQueryValueW(ver_block, "", + ctypes.byref(pvi), ctypes.byref(ctypes.wintypes.DWORD())): return maj, min, build maj = pvi.contents.dwProductVersionMS >> 16 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -125,6 +125,8 @@ Library ------- +- Issue #27932: Fixes memory leak in platform.win32_ver() + - Issue #14977: mailcap now respects the order of the lines in the mailcap files ("first match"), as required by RFC 1542. Patch by Michael Lazar. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:05:23 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 01:05:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTMy?= =?utf-8?q?=3A_Fixes_memory_leak_in_platform=2Ewin32=5Fver=28=29?= Message-ID: <20160910010522.69652.7869.44FD4617@psf.io> https://hg.python.org/cpython/rev/384c178cf823 changeset: 103515:384c178cf823 branch: 3.5 parent: 103508:663a62bcf9c9 user: Steve Dower date: Fri Sep 09 18:01:25 2016 -0700 summary: Issue #27932: Fixes memory leak in platform.win32_ver() files: Lib/platform.py | 64 +++++++++++++++++++----------------- Misc/NEWS | 2 + 2 files changed, 36 insertions(+), 30 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -498,57 +498,61 @@ (6, None): "post2012ServerR2", } +if sys.platform == 'win32': + import ctypes + import ctypes.wintypes + + class VS_FIXEDFILEINFO(ctypes.Structure): + _fields_ = [ + ("dwSignature", ctypes.wintypes.DWORD), + ("dwStrucVersion", ctypes.wintypes.DWORD), + ("dwFileVersionMS", ctypes.wintypes.DWORD), + ("dwFileVersionLS", ctypes.wintypes.DWORD), + ("dwProductVersionMS", ctypes.wintypes.DWORD), + ("dwProductVersionLS", ctypes.wintypes.DWORD), + ("dwFileFlagsMask", ctypes.wintypes.DWORD), + ("dwFileFlags", ctypes.wintypes.DWORD), + ("dwFileOS", ctypes.wintypes.DWORD), + ("dwFileType", ctypes.wintypes.DWORD), + ("dwFileSubtype", ctypes.wintypes.DWORD), + ("dwFileDateMS", ctypes.wintypes.DWORD), + ("dwFileDateLS", ctypes.wintypes.DWORD), + ] + + P_VS_FIXEDFILEINFO = ctypes.POINTER(VS_FIXEDFILEINFO) + def _get_real_winver(maj, min, build): if maj < 6 or (maj == 6 and min < 2): return maj, min, build - from ctypes import (c_buffer, POINTER, byref, create_unicode_buffer, - Structure, WinDLL) - from ctypes.wintypes import DWORD, HANDLE - - class VS_FIXEDFILEINFO(Structure): - _fields_ = [ - ("dwSignature", DWORD), - ("dwStrucVersion", DWORD), - ("dwFileVersionMS", DWORD), - ("dwFileVersionLS", DWORD), - ("dwProductVersionMS", DWORD), - ("dwProductVersionLS", DWORD), - ("dwFileFlagsMask", DWORD), - ("dwFileFlags", DWORD), - ("dwFileOS", DWORD), - ("dwFileType", DWORD), - ("dwFileSubtype", DWORD), - ("dwFileDateMS", DWORD), - ("dwFileDateLS", DWORD), - ] - - kernel32 = WinDLL('kernel32') - version = WinDLL('version') - + kernel32 = ctypes.WinDLL('kernel32') # We will immediately double the length up to MAX_PATH, but the # path may be longer, so we retry until the returned string is # shorter than our buffer. name_len = actual_len = 130 while actual_len == name_len: name_len *= 2 - name = create_unicode_buffer(name_len) - actual_len = kernel32.GetModuleFileNameW(HANDLE(kernel32._handle), - name, len(name)) + name = ctypes.create_unicode_buffer(name_len) + actual_len = kernel32.GetModuleFileNameW( + ctypes.wintypes.HANDLE(kernel32._handle), + name, len(name) + ) if not actual_len: return maj, min, build + version = ctypes.WinDLL('version') size = version.GetFileVersionInfoSizeW(name, None) if not size: return maj, min, build - ver_block = c_buffer(size) + ver_block = ctypes.c_buffer(size) if (not version.GetFileVersionInfoW(name, None, size, ver_block) or not ver_block): return maj, min, build - pvi = POINTER(VS_FIXEDFILEINFO)() - if not version.VerQueryValueW(ver_block, "", byref(pvi), byref(DWORD())): + pvi = P_VS_FIXEDFILEINFO() + if not version.VerQueryValueW(ver_block, "", + ctypes.byref(pvi), ctypes.byref(ctypes.wintypes.DWORD())): return maj, min, build maj = pvi.contents.dwProductVersionMS >> 16 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -65,6 +65,8 @@ Library ------- +- Issue #27932: Fixes memory leak in platform.win32_ver() + - Issue #14977: mailcap now respects the order of the lines in the mailcap files ("first match"), as required by RFC 1542. Patch by Michael Lazar. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:05:23 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 01:05:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_Zach?= Message-ID: <20160910010523.13046.1763.A6EFB6AC@psf.io> https://hg.python.org/cpython/rev/8c2924eb97a8 changeset: 103517:8c2924eb97a8 parent: 103516:31b7eaff5588 parent: 103514:90396ec9a2f8 user: Steve Dower date: Fri Sep 09 18:05:07 2016 -0700 summary: Merge from Zach files: .gitignore | 1 - .hgignore | 1 - Lib/sysconfig.py | 13 +++++++++++-- Mac/BuildScript/build-installer.py | 3 ++- Makefile.pre.in | 18 +++++------------- Misc/NEWS | 2 ++ PC/getpathp.c | 4 ++-- Tools/msi/make_zip.py | 2 -- configure | 9 +-------- configure.ac | 8 +------- 10 files changed, 24 insertions(+), 37 deletions(-) diff --git a/.gitignore b/.gitignore --- a/.gitignore +++ b/.gitignore @@ -21,7 +21,6 @@ Lib/distutils/command/*.pdb Lib/lib2to3/*.pickle Lib/test/data/* -Lib/plat-mac/errors.rsrc.df.rsrc Makefile Makefile.pre Misc/python.pc diff --git a/.hgignore b/.hgignore --- a/.hgignore +++ b/.hgignore @@ -26,7 +26,6 @@ python-config.py$ reflog.txt$ tags$ -Lib/plat-mac/errors.rsrc.df.rsrc Misc/python.pc Misc/python-config.sh$ Modules/Setup$ diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -341,6 +341,15 @@ config_dir_name += '-%s' % sys.implementation._multiarch return os.path.join(get_path('stdlib'), config_dir_name, 'Makefile') + +def _get_sysconfigdata_name(): + return '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( + abi=sys.abiflags, + platform=sys.platform, + multiarch=getattr(sys.implementation, '_multiarch', ''), + ) + + def _generate_posix_vars(): """Generate the Python module containing build-time variables.""" import pprint @@ -381,7 +390,7 @@ # _sysconfigdata module manually and populate it with the build vars. # This is more than sufficient for ensuring the subsequent call to # get_platform() succeeds. - name = '_sysconfigdata_' + sys.abiflags + name = _get_sysconfigdata_name() if 'darwin' in sys.platform: import types module = types.ModuleType(name) @@ -407,7 +416,7 @@ def _init_posix(vars): """Initialize the module as appropriate for POSIX systems.""" # _sysconfigdata is generated at build time, see _generate_posix_vars() - name = '_sysconfigdata_' + sys.abiflags + name = _get_sysconfigdata_name() _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) build_time_vars = _temp.build_time_vars vars.update(build_time_vars) diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -1292,7 +1292,8 @@ import pprint if getVersionMajorMinor() >= (3, 6): - path = os.path.join(path_to_lib, 'plat-darwin', '_sysconfigdata_m.py') + # XXX this is extra-fragile + path = os.path.join(path_to_lib, '_sysconfigdata_m_darwin_darwin.py') else: path = os.path.join(path_to_lib, '_sysconfigdata.py') fp = open(path, 'r') diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1178,8 +1178,6 @@ (cd $(DESTDIR)$(MANDIR)/man1; $(LN) -s python$(VERSION).1 python3.1) # Install the library -PLATDIR= @PLATDIR@ -MACHDEPS= $(PLATDIR) XMLLIBSUBDIRS= xml xml/dom xml/etree xml/parsers xml/sax LIBSUBDIRS= tkinter tkinter/test tkinter/test/test_tkinter \ tkinter/test/test_ttk site-packages test \ @@ -1238,8 +1236,8 @@ multiprocessing multiprocessing/dummy \ unittest unittest/test unittest/test/testmock \ venv venv/scripts venv/scripts/posix \ - curses pydoc_data $(MACHDEPS) -libinstall: build_all $(srcdir)/Lib/$(PLATDIR) $(srcdir)/Modules/xxmodule.c + curses pydoc_data +libinstall: build_all $(srcdir)/Modules/xxmodule.c @for i in $(SCRIPTDIR) $(LIBDEST); \ do \ if test ! -d $(DESTDIR)$$i; then \ @@ -1294,10 +1292,10 @@ esac; \ done; \ done - $(INSTALL_DATA) `cat pybuilddir.txt`/_sysconfigdata_$(ABIFLAGS).py \ - $(DESTDIR)$(LIBDEST)/$(PLATDIR); \ + $(INSTALL_DATA) `cat pybuilddir.txt`/_sysconfigdata_$(ABIFLAGS)_$(MACHDEP)_$(MULTIARCH).py \ + $(DESTDIR)$(LIBDEST); \ echo $(INSTALL_DATA) `cat pybuilddir.txt`/_sysconfigdata_$(ABIFLAGS).py \ - $(LIBDEST)/$(PLATDIR) + $(LIBDEST) $(INSTALL_DATA) $(srcdir)/LICENSE $(DESTDIR)$(LIBDEST)/LICENSE.txt if test -d $(DESTDIR)$(LIBDEST)/distutils/tests; then \ $(INSTALL_DATA) $(srcdir)/Modules/xxmodule.c \ @@ -1335,9 +1333,6 @@ -PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \ $(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/PatternGrammar.txt -$(srcdir)/Lib/$(PLATDIR): - mkdir $(srcdir)/Lib/$(PLATDIR) - python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh # Substitution happens here, as the completely-expanded BINDIR # is not available in configure @@ -1614,9 +1609,6 @@ -rm -rf build platform -rm -rf $(PYTHONFRAMEWORKDIR) -rm -f python-config.py python-config - if [ -n "$(MULTIARCH)" ]; then \ - rm -rf $(srcdir)/Lib/$(PLATDIR); \ - fi # Make things extra clean, before making a distribution: # remove all generated files, even Makefile[.pre] diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,8 @@ Core and Builtins ----------------- +- Issue #28046: Remove platform-specific directories from sys.path. + - Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch by Eryk Sun) diff --git a/PC/getpathp.c b/PC/getpathp.c --- a/PC/getpathp.c +++ b/PC/getpathp.c @@ -26,7 +26,7 @@ is set, we believe it. Otherwise, we use the path of our host .EXE's to try and locate on of our "landmarks" and deduce our home. - If we DO have a Python Home: The relevant sub-directories (Lib, - plat-win, etc) are based on the Python Home + DLLs, etc) are based on the Python Home - If we DO NOT have a Python Home, the core Python Path is loaded from the registry. This is the main PythonPath key, and both HKLM and HKCU are combined to form the path) @@ -34,7 +34,7 @@ * Iff - we can not locate the Python Home, have not had a PYTHONPATH specified, and can't locate any Registry entries (ie, we have _nothing_ we can assume is a good path), a default path with relative entries is - used (eg. .\Lib;.\plat-win, etc) + used (eg. .\Lib;.\DLLs, etc) If a sys.path file exists adjacent to python.exe, it must contain a diff --git a/Tools/msi/make_zip.py b/Tools/msi/make_zip.py --- a/Tools/msi/make_zip.py +++ b/Tools/msi/make_zip.py @@ -67,8 +67,6 @@ if p.is_dir(): if name in EXCLUDE_FROM_LIBRARY: return False - if name.startswith('plat-'): - return False if name == 'test' and p.parts[-2].lower() == 'lib': return False if name in {'test', 'tests'} and p.parts[-3].lower() == 'lib': diff --git a/configure b/configure --- a/configure +++ b/configure @@ -712,7 +712,6 @@ NO_AS_NEEDED MULTIARCH_CPPFLAGS PLATFORM_TRIPLET -PLATDIR MULTIARCH ac_ct_CXX MAINCC @@ -2929,7 +2928,7 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $interp" >&5 $as_echo "$interp" >&6; } - PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp + PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib '$interp fi # Used to comment out stuff for rebuilding generated files GENERATED_COMMENT='#' @@ -5361,12 +5360,6 @@ elif test x$PLATFORM_TRIPLET != x && test x$MULTIARCH = x; then MULTIARCH=$PLATFORM_TRIPLET fi -if test x$PLATFORM_TRIPLET = x; then - PLATDIR=plat-$MACHDEP -else - PLATDIR=plat-$PLATFORM_TRIPLET -fi - if test x$MULTIARCH != x; then MULTIARCH_CPPFLAGS="-DMULTIARCH=\\\"$MULTIARCH\\\"" diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -78,7 +78,7 @@ AC_MSG_ERROR([python$PACKAGE_VERSION interpreter not found]) fi AC_MSG_RESULT($interp) - PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp + PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib '$interp fi # Used to comment out stuff for rebuilding generated files GENERATED_COMMENT='#' @@ -910,12 +910,6 @@ elif test x$PLATFORM_TRIPLET != x && test x$MULTIARCH = x; then MULTIARCH=$PLATFORM_TRIPLET fi -if test x$PLATFORM_TRIPLET = x; then - PLATDIR=plat-$MACHDEP -else - PLATDIR=plat-$PLATFORM_TRIPLET -fi -AC_SUBST(PLATDIR) AC_SUBST(PLATFORM_TRIPLET) if test x$MULTIARCH != x; then MULTIARCH_CPPFLAGS="-DMULTIARCH=\\\"$MULTIARCH\\\"" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:10:21 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 10 Sep 2016 01:10:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_dummy_dtrace_probes_are_a_?= =?utf-8?q?good_place_to_use_inline_functions?= Message-ID: <20160910011019.13065.99104.9EDB6F55@psf.io> https://hg.python.org/cpython/rev/2f77a9f0b9d6 changeset: 103518:2f77a9f0b9d6 user: Benjamin Peterson date: Fri Sep 09 18:09:52 2016 -0700 summary: dummy dtrace probes are a good place to use inline functions files: Include/pydtrace.h | 36 +++++++++++++++++----------------- 1 files changed, 18 insertions(+), 18 deletions(-) diff --git a/Include/pydtrace.h b/Include/pydtrace.h --- a/Include/pydtrace.h +++ b/Include/pydtrace.h @@ -22,25 +22,25 @@ /* Without DTrace, compile to nothing. */ -#define PyDTrace_LINE(arg0, arg1, arg2) do ; while (0) -#define PyDTrace_FUNCTION_ENTRY(arg0, arg1, arg2) do ; while (0) -#define PyDTrace_FUNCTION_RETURN(arg0, arg1, arg2) do ; while (0) -#define PyDTrace_GC_START(arg0) do ; while (0) -#define PyDTrace_GC_DONE(arg0) do ; while (0) -#define PyDTrace_INSTANCE_NEW_START(arg0) do ; while (0) -#define PyDTrace_INSTANCE_NEW_DONE(arg0) do ; while (0) -#define PyDTrace_INSTANCE_DELETE_START(arg0) do ; while (0) -#define PyDTrace_INSTANCE_DELETE_DONE(arg0) do ; while (0) +inline void PyDTrace_LINE(const char *arg0, const char *arg1, int arg2) {} +inline void PyDTrace_FUNCTION_ENTRY(const char *arg0, const char *arg1, int arg2) {} +inline void PyDTrace_FUNCTION_RETURN(const char *arg0, const char *arg1, int arg2) {} +inline void PyDTrace_GC_START(int arg0) {} +inline void PyDTrace_GC_DONE(int arg0) {} +inline void PyDTrace_INSTANCE_NEW_START(int arg0) {} +inline void PyDTrace_INSTANCE_NEW_DONE(int arg0) {} +inline void PyDTrace_INSTANCE_DELETE_START(int arg0) {} +inline void PyDTrace_INSTANCE_DELETE_DONE(int arg0) {} -#define PyDTrace_LINE_ENABLED() (0) -#define PyDTrace_FUNCTION_ENTRY_ENABLED() (0) -#define PyDTrace_FUNCTION_RETURN_ENABLED() (0) -#define PyDTrace_GC_START_ENABLED() (0) -#define PyDTrace_GC_DONE_ENABLED() (0) -#define PyDTrace_INSTANCE_NEW_START_ENABLED() (0) -#define PyDTrace_INSTANCE_NEW_DONE_ENABLED() (0) -#define PyDTrace_INSTANCE_DELETE_START_ENABLED() (0) -#define PyDTrace_INSTANCE_DELETE_DONE_ENABLED() (0) +inline int PyDTrace_LINE_ENABLED(void) { return 0; } +inline int PyDTrace_FUNCTION_ENTRY_ENABLED(void) { return 0; } +inline int PyDTrace_FUNCTION_RETURN_ENABLED(void) { return 0; } +inline int PyDTrace_GC_START_ENABLED(void) { return 0; } +inline int PyDTrace_GC_DONE_ENABLED(void) { return 0; } +inline int PyDTrace_INSTANCE_NEW_START_ENABLED(void) { return 0; } +inline int PyDTrace_INSTANCE_NEW_DONE_ENABLED(void) { return 0; } +inline int PyDTrace_INSTANCE_DELETE_START_ENABLED(void) { return 0; } +inline int PyDTrace_INSTANCE_DELETE_DONE_ENABLED(void) { return 0; } #endif /* !WITH_DTRACE */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:14:58 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 10 Sep 2016 01:14:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge_=283=2E6_already_has_this=29?= Message-ID: <20160910011458.48546.25450.9511AD2D@psf.io> https://hg.python.org/cpython/rev/3eb20d3df15f changeset: 103520:3eb20d3df15f parent: 103518:2f77a9f0b9d6 parent: 103519:2184d3c94a82 user: Gregory P. Smith date: Fri Sep 09 18:14:52 2016 -0700 summary: merge (3.6 already has this) files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:14:58 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 10 Sep 2016 01:14:58 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogcmVtb3ZlIGZpeF9j?= =?utf-8?q?allable_-_callable=28=29_was_readded_many_releases_ago=2E?= Message-ID: <20160910011458.1755.53599.724E0085@psf.io> https://hg.python.org/cpython/rev/2184d3c94a82 changeset: 103519:2184d3c94a82 branch: 3.5 parent: 103515:384c178cf823 user: Gregory P. Smith date: Fri Sep 09 18:14:33 2016 -0700 summary: remove fix_callable - callable() was readded many releases ago. files: Lib/lib2to3/fixes/fix_callable.py | 37 ------- Lib/lib2to3/tests/test_fixers.py | 92 ------------------- 2 files changed, 0 insertions(+), 129 deletions(-) diff --git a/Lib/lib2to3/fixes/fix_callable.py b/Lib/lib2to3/fixes/fix_callable.py deleted file mode 100644 --- a/Lib/lib2to3/fixes/fix_callable.py +++ /dev/null @@ -1,37 +0,0 @@ -# Copyright 2007 Google, Inc. All Rights Reserved. -# Licensed to PSF under a Contributor Agreement. - -"""Fixer for callable(). - -This converts callable(obj) into isinstance(obj, collections.Callable), adding a -collections import if needed.""" - -# Local imports -from lib2to3 import fixer_base -from lib2to3.fixer_util import Call, Name, String, Attr, touch_import - -class FixCallable(fixer_base.BaseFix): - BM_compatible = True - - order = "pre" - - # Ignore callable(*args) or use of keywords. - # Either could be a hint that the builtin callable() is not being used. - PATTERN = """ - power< 'callable' - trailer< lpar='(' - ( not(arglist | argument) any ','> ) - rpar=')' > - after=any* - > - """ - - def transform(self, node, results): - func = results['func'] - - touch_import(None, 'collections', node=node) - - args = [func.clone(), String(', ')] - args.extend(Attr(Name('collections'), Name('Callable'))) - return Call(Name('isinstance'), args, prefix=node.prefix) diff --git a/Lib/lib2to3/tests/test_fixers.py b/Lib/lib2to3/tests/test_fixers.py --- a/Lib/lib2to3/tests/test_fixers.py +++ b/Lib/lib2to3/tests/test_fixers.py @@ -2920,98 +2920,6 @@ a = f + r"""r'\\\u20ac\U0001d121\\u20ac'""" self.check(b, a) -class Test_callable(FixerTestCase): - fixer = "callable" - - def test_prefix_preservation(self): - b = """callable( x)""" - a = """import collections\nisinstance( x, collections.Callable)""" - self.check(b, a) - - b = """if callable(x): pass""" - a = """import collections -if isinstance(x, collections.Callable): pass""" - self.check(b, a) - - def test_callable_call(self): - b = """callable(x)""" - a = """import collections\nisinstance(x, collections.Callable)""" - self.check(b, a) - - def test_global_import(self): - b = """ -def spam(foo): - callable(foo)"""[1:] - a = """ -import collections -def spam(foo): - isinstance(foo, collections.Callable)"""[1:] - self.check(b, a) - - b = """ -import collections -def spam(foo): - callable(foo)"""[1:] - # same output if it was already imported - self.check(b, a) - - b = """ -from collections import * -def spam(foo): - callable(foo)"""[1:] - a = """ -from collections import * -import collections -def spam(foo): - isinstance(foo, collections.Callable)"""[1:] - self.check(b, a) - - b = """ -do_stuff() -do_some_other_stuff() -assert callable(do_stuff)"""[1:] - a = """ -import collections -do_stuff() -do_some_other_stuff() -assert isinstance(do_stuff, collections.Callable)"""[1:] - self.check(b, a) - - b = """ -if isinstance(do_stuff, Callable): - assert callable(do_stuff) - do_stuff(do_stuff) - if not callable(do_stuff): - exit(1) - else: - assert callable(do_stuff) -else: - assert not callable(do_stuff)"""[1:] - a = """ -import collections -if isinstance(do_stuff, Callable): - assert isinstance(do_stuff, collections.Callable) - do_stuff(do_stuff) - if not isinstance(do_stuff, collections.Callable): - exit(1) - else: - assert isinstance(do_stuff, collections.Callable) -else: - assert not isinstance(do_stuff, collections.Callable)"""[1:] - self.check(b, a) - - def test_callable_should_not_change(self): - a = """callable(*x)""" - self.unchanged(a) - - a = """callable(x, y)""" - self.unchanged(a) - - a = """callable(x, kw=y)""" - self.unchanged(a) - - a = """callable()""" - self.unchanged(a) class Test_filter(FixerTestCase): fixer = "filter" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:20:11 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 10 Sep 2016 01:20:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1OTY5?= =?utf-8?q?=3A_Update_the_lib2to3_grammar_to_handle_the_unpacking?= Message-ID: <20160910012011.18827.93015.34E5673B@psf.io> https://hg.python.org/cpython/rev/1206c64de875 changeset: 103521:1206c64de875 branch: 3.5 parent: 103519:2184d3c94a82 user: Gregory P. Smith [Google Inc.] date: Fri Sep 09 18:18:52 2016 -0700 summary: Issue #25969: Update the lib2to3 grammar to handle the unpacking generalizations added in 3.5. files: Lib/lib2to3/Grammar.txt | 23 +++++++++--- Lib/lib2to3/fixes/fix_apply.py | 11 ++++++ Lib/lib2to3/fixes/fix_intern.py | 10 +++++ Lib/lib2to3/fixes/fix_reload.py | 11 ++++++ Lib/lib2to3/tests/test_fixers.py | 4 ++ Lib/lib2to3/tests/test_parser.py | 35 ++++++++++++++++++++ Misc/NEWS | 3 + 7 files changed, 91 insertions(+), 6 deletions(-) diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -138,15 +138,26 @@ sliceop: ':' [test] exprlist: (expr|star_expr) (',' (expr|star_expr))* [','] testlist: test (',' test)* [','] -dictsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [','])) | - (test (comp_for | (',' test)* [','])) ) +dictsetmaker: ( ((test ':' test | '**' expr) + (comp_for | (',' (test ':' test | '**' expr))* [','])) | + ((test | star_expr) + (comp_for | (',' (test | star_expr))* [','])) ) classdef: 'class' NAME ['(' [arglist] ')'] ':' suite -arglist: (argument ',')* (argument [','] - |'*' test (',' argument)* [',' '**' test] - |'**' test) -argument: test [comp_for] | test '=' test # Really [keyword '='] test +arglist: argument (',' argument)* [','] + +# "test '=' test" is really "keyword '=' test", but we have no such token. +# These need to be in a single rule to avoid grammar that is ambiguous +# to our LL(1) parser. Even though 'test' includes '*expr' in star_expr, +# we explicitly match '*' here, too, to give it proper precedence. +# Illegal combinations and orderings are blocked in ast.c: +# multiple (test comp_for) arguements are blocked; keyword unpackings +# that precede iterable unpackings are blocked; etc. +argument: ( test [comp_for] | + test '=' test | + '**' expr | + star_expr ) comp_iter: comp_for | comp_if comp_for: 'for' exprlist 'in' testlist_safe [comp_iter] diff --git a/Lib/lib2to3/fixes/fix_apply.py b/Lib/lib2to3/fixes/fix_apply.py --- a/Lib/lib2to3/fixes/fix_apply.py +++ b/Lib/lib2to3/fixes/fix_apply.py @@ -34,6 +34,17 @@ func = results["func"] args = results["args"] kwds = results.get("kwds") + # I feel like we should be able to express this logic in the + # PATTERN above but I don't know how to do it so... + if args: + if args.type == self.syms.star_expr: + return # Make no change. + if (args.type == self.syms.argument and + args.children[0].value == '**'): + return # Make no change. + if kwds and (kwds.type == self.syms.argument and + kwds.children[0].value == '**'): + return # Make no change. prefix = node.prefix func = func.clone() if (func.type not in (token.NAME, syms.atom) and diff --git a/Lib/lib2to3/fixes/fix_intern.py b/Lib/lib2to3/fixes/fix_intern.py --- a/Lib/lib2to3/fixes/fix_intern.py +++ b/Lib/lib2to3/fixes/fix_intern.py @@ -25,6 +25,16 @@ """ def transform(self, node, results): + if results: + # I feel like we should be able to express this logic in the + # PATTERN above but I don't know how to do it so... + obj = results['obj'] + if obj: + if obj.type == self.syms.star_expr: + return # Make no change. + if (obj.type == self.syms.argument and + obj.children[0].value == '**'): + return # Make no change. names = ('sys', 'intern') new = ImportAndCall(node, results, names) touch_import(None, 'sys', node) diff --git a/Lib/lib2to3/fixes/fix_reload.py b/Lib/lib2to3/fixes/fix_reload.py --- a/Lib/lib2to3/fixes/fix_reload.py +++ b/Lib/lib2to3/fixes/fix_reload.py @@ -22,6 +22,17 @@ """ def transform(self, node, results): + if results: + # I feel like we should be able to express this logic in the + # PATTERN above but I don't know how to do it so... + obj = results['obj'] + print('obj:', repr(obj)) + if obj: + if obj.type == self.syms.star_expr: + return # Make no change. + if (obj.type == self.syms.argument and + obj.children[0].value == '**'): + return # Make no change. names = ('imp', 'reload') new = ImportAndCall(node, results, names) touch_import(None, 'imp', node) diff --git a/Lib/lib2to3/tests/test_fixers.py b/Lib/lib2to3/tests/test_fixers.py --- a/Lib/lib2to3/tests/test_fixers.py +++ b/Lib/lib2to3/tests/test_fixers.py @@ -260,6 +260,10 @@ s = """apply(f, *args)""" self.unchanged(s) + def test_unchanged_6b(self): + s = """apply(f, **kwds)""" + self.unchanged(s) + def test_unchanged_7(self): s = """apply(func=f, args=args, kwds=kwds)""" self.unchanged(s) diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -208,6 +208,41 @@ self.invalid_syntax("raise E from") +# Modelled after Lib/test/test_grammar.py:TokenTests.test_funcdef issue2292 +# and Lib/test/text_parser.py test_list_displays, test_set_displays, +# test_dict_displays, test_argument_unpacking, ... changes. +class TestUnpackingGeneralizations(GrammarTest): + def test_mid_positional_star(self): + self.validate("""func(1, *(2, 3), 4)""") + + def test_double_star_dict_literal(self): + self.validate("""func(**{'eggs':'scrambled', 'spam':'fried'})""") + + def test_double_star_dict_literal_after_keywords(self): + self.validate("""func(spam='fried', **{'eggs':'scrambled'})""") + + def test_list_display(self): + self.validate("""[*{2}, 3, *[4]]""") + + def test_set_display(self): + self.validate("""{*{2}, 3, *[4]}""") + + def test_dict_display_1(self): + self.validate("""{**{}}""") + + def test_dict_display_2(self): + self.validate("""{**{}, 3:4, **{5:6, 7:8}}""") + + def test_argument_unpacking_1(self): + self.validate("""f(a, *b, *c, d)""") + + def test_argument_unpacking_2(self): + self.validate("""f(**a, **b)""") + + def test_argument_unpacking_3(self): + self.validate("""f(2, *a, *b, **b, **c, **d)""") + + # Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.testFuncdef class TestFunctionAnnotations(GrammarTest): def test_1(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -65,6 +65,9 @@ Library ------- +- Issue #25969: Update the lib2to3 grammar to handle the unpacking + generalizations added in 3.5. + - Issue #27932: Fixes memory leak in platform.win32_ver() - Issue #14977: mailcap now respects the order of the lines in the mailcap -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:20:11 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 10 Sep 2016 01:20:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325969=3A_Update_the_lib2to3_grammar_to_handle_t?= =?utf-8?q?he_unpacking?= Message-ID: <20160910012011.12492.50011.9AD7A7D6@psf.io> https://hg.python.org/cpython/rev/2460b30c1985 changeset: 103522:2460b30c1985 parent: 103520:3eb20d3df15f parent: 103521:1206c64de875 user: Gregory P. Smith [Google Inc.] date: Fri Sep 09 18:19:51 2016 -0700 summary: Issue #25969: Update the lib2to3 grammar to handle the unpacking generalizations added in 3.5. files: Lib/lib2to3/Grammar.txt | 23 +++++++++--- Lib/lib2to3/fixes/fix_apply.py | 11 ++++++ Lib/lib2to3/fixes/fix_intern.py | 10 +++++ Lib/lib2to3/fixes/fix_reload.py | 11 ++++++ Lib/lib2to3/tests/test_fixers.py | 4 ++ Lib/lib2to3/tests/test_parser.py | 35 ++++++++++++++++++++ Misc/NEWS | 3 + 7 files changed, 91 insertions(+), 6 deletions(-) diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -139,15 +139,26 @@ sliceop: ':' [test] exprlist: (expr|star_expr) (',' (expr|star_expr))* [','] testlist: test (',' test)* [','] -dictsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [','])) | - (test (comp_for | (',' test)* [','])) ) +dictsetmaker: ( ((test ':' test | '**' expr) + (comp_for | (',' (test ':' test | '**' expr))* [','])) | + ((test | star_expr) + (comp_for | (',' (test | star_expr))* [','])) ) classdef: 'class' NAME ['(' [arglist] ')'] ':' suite -arglist: (argument ',')* (argument [','] - |'*' test (',' argument)* [',' '**' test] - |'**' test) -argument: test [comp_for] | test '=' test # Really [keyword '='] test +arglist: argument (',' argument)* [','] + +# "test '=' test" is really "keyword '=' test", but we have no such token. +# These need to be in a single rule to avoid grammar that is ambiguous +# to our LL(1) parser. Even though 'test' includes '*expr' in star_expr, +# we explicitly match '*' here, too, to give it proper precedence. +# Illegal combinations and orderings are blocked in ast.c: +# multiple (test comp_for) arguements are blocked; keyword unpackings +# that precede iterable unpackings are blocked; etc. +argument: ( test [comp_for] | + test '=' test | + '**' expr | + star_expr ) comp_iter: comp_for | comp_if comp_for: [ASYNC] 'for' exprlist 'in' testlist_safe [comp_iter] diff --git a/Lib/lib2to3/fixes/fix_apply.py b/Lib/lib2to3/fixes/fix_apply.py --- a/Lib/lib2to3/fixes/fix_apply.py +++ b/Lib/lib2to3/fixes/fix_apply.py @@ -34,6 +34,17 @@ func = results["func"] args = results["args"] kwds = results.get("kwds") + # I feel like we should be able to express this logic in the + # PATTERN above but I don't know how to do it so... + if args: + if args.type == self.syms.star_expr: + return # Make no change. + if (args.type == self.syms.argument and + args.children[0].value == '**'): + return # Make no change. + if kwds and (kwds.type == self.syms.argument and + kwds.children[0].value == '**'): + return # Make no change. prefix = node.prefix func = func.clone() if (func.type not in (token.NAME, syms.atom) and diff --git a/Lib/lib2to3/fixes/fix_intern.py b/Lib/lib2to3/fixes/fix_intern.py --- a/Lib/lib2to3/fixes/fix_intern.py +++ b/Lib/lib2to3/fixes/fix_intern.py @@ -25,6 +25,16 @@ """ def transform(self, node, results): + if results: + # I feel like we should be able to express this logic in the + # PATTERN above but I don't know how to do it so... + obj = results['obj'] + if obj: + if obj.type == self.syms.star_expr: + return # Make no change. + if (obj.type == self.syms.argument and + obj.children[0].value == '**'): + return # Make no change. names = ('sys', 'intern') new = ImportAndCall(node, results, names) touch_import(None, 'sys', node) diff --git a/Lib/lib2to3/fixes/fix_reload.py b/Lib/lib2to3/fixes/fix_reload.py --- a/Lib/lib2to3/fixes/fix_reload.py +++ b/Lib/lib2to3/fixes/fix_reload.py @@ -22,6 +22,17 @@ """ def transform(self, node, results): + if results: + # I feel like we should be able to express this logic in the + # PATTERN above but I don't know how to do it so... + obj = results['obj'] + print('obj:', repr(obj)) + if obj: + if obj.type == self.syms.star_expr: + return # Make no change. + if (obj.type == self.syms.argument and + obj.children[0].value == '**'): + return # Make no change. names = ('imp', 'reload') new = ImportAndCall(node, results, names) touch_import(None, 'imp', node) diff --git a/Lib/lib2to3/tests/test_fixers.py b/Lib/lib2to3/tests/test_fixers.py --- a/Lib/lib2to3/tests/test_fixers.py +++ b/Lib/lib2to3/tests/test_fixers.py @@ -259,6 +259,10 @@ s = """apply(f, *args)""" self.unchanged(s) + def test_unchanged_6b(self): + s = """apply(f, **kwds)""" + self.unchanged(s) + def test_unchanged_7(self): s = """apply(func=f, args=args, kwds=kwds)""" self.unchanged(s) diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -226,6 +226,41 @@ self.invalid_syntax("raise E from") +# Modelled after Lib/test/test_grammar.py:TokenTests.test_funcdef issue2292 +# and Lib/test/text_parser.py test_list_displays, test_set_displays, +# test_dict_displays, test_argument_unpacking, ... changes. +class TestUnpackingGeneralizations(GrammarTest): + def test_mid_positional_star(self): + self.validate("""func(1, *(2, 3), 4)""") + + def test_double_star_dict_literal(self): + self.validate("""func(**{'eggs':'scrambled', 'spam':'fried'})""") + + def test_double_star_dict_literal_after_keywords(self): + self.validate("""func(spam='fried', **{'eggs':'scrambled'})""") + + def test_list_display(self): + self.validate("""[*{2}, 3, *[4]]""") + + def test_set_display(self): + self.validate("""{*{2}, 3, *[4]}""") + + def test_dict_display_1(self): + self.validate("""{**{}}""") + + def test_dict_display_2(self): + self.validate("""{**{}, 3:4, **{5:6, 7:8}}""") + + def test_argument_unpacking_1(self): + self.validate("""f(a, *b, *c, d)""") + + def test_argument_unpacking_2(self): + self.validate("""f(**a, **b)""") + + def test_argument_unpacking_3(self): + self.validate("""f(2, *a, *b, **b, **c, **d)""") + + # Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.testFuncdef class TestFunctionAnnotations(GrammarTest): def test_1(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -127,6 +127,9 @@ Library ------- +- Issue #25969: Update the lib2to3 grammar to handle the unpacking + generalizations added in 3.5. + - Issue #27932: Fixes memory leak in platform.win32_ver() - Issue #14977: mailcap now respects the order of the lines in the mailcap -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:25:39 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 01:25:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NzA1?= =?utf-8?q?=3A_Update_message_in_validate=5Fucrtbase=2Epy?= Message-ID: <20160910012539.12514.83061.26433A1C@psf.io> https://hg.python.org/cpython/rev/7f7a994bbfbc changeset: 103523:7f7a994bbfbc branch: 3.5 parent: 103521:1206c64de875 user: Steve Dower date: Fri Sep 09 18:21:15 2016 -0700 summary: Issue #27705: Update message in validate_ucrtbase.py files: Misc/NEWS | 2 ++ PC/validate_ucrtbase.py | 5 +++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -301,6 +301,8 @@ Build ----- +- Issue #27705: Update message in validate_ucrtbase.py + - Issue #27983: Cause lack of llvm-profdata tool when using clang as required for PGO linking to be a configure time error rather than make time when --with-optimizations is enabled. Also improve our diff --git a/PC/validate_ucrtbase.py b/PC/validate_ucrtbase.py --- a/PC/validate_ucrtbase.py +++ b/PC/validate_ucrtbase.py @@ -82,7 +82,8 @@ if ver < (10, 0, 10586): print('WARN: ucrtbased contains known issues. ' - 'Please update Visual Studio or the Windows SDK.') + 'Please update the Windows 10 SDK.') print('See:') - print(' http://bugs.python.org/issue26624') + print(' http://bugs.python.org/issue27705') + print(' https://developer.microsoft.com/en-US/windows/downloads/windows-10-sdk') sys.exit(1) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:25:44 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 01:25:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327705=3A_Update_message_in_validate=5Fucrtbase?= =?utf-8?b?LnB5?= Message-ID: <20160910012544.59448.73738.8C68E915@psf.io> https://hg.python.org/cpython/rev/a791a54c0a1c changeset: 103524:a791a54c0a1c parent: 103522:2460b30c1985 parent: 103523:7f7a994bbfbc user: Steve Dower date: Fri Sep 09 18:25:29 2016 -0700 summary: Issue #27705: Update message in validate_ucrtbase.py files: Misc/NEWS | 2 ++ PC/validate_ucrtbase.py | 5 +++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -328,6 +328,8 @@ Build ----- +- Issue #27705: Update message in validate_ucrtbase.py + - Issue #27976: Deprecate building _ctypes with the bundled copy of libffi on non-OSX UNIX platforms. diff --git a/PC/validate_ucrtbase.py b/PC/validate_ucrtbase.py --- a/PC/validate_ucrtbase.py +++ b/PC/validate_ucrtbase.py @@ -82,7 +82,8 @@ if ver < (10, 0, 10586): print('WARN: ucrtbased contains known issues. ' - 'Please update Visual Studio or the Windows SDK.') + 'Please update the Windows 10 SDK.') print('See:') - print(' http://bugs.python.org/issue26624') + print(' http://bugs.python.org/issue27705') + print(' https://developer.microsoft.com/en-US/windows/downloads/windows-10-sdk') sys.exit(1) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:33:51 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 10 Sep 2016 01:33:51 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogcmVtb3ZlIGZpeF9j?= =?utf-8?q?allable=2C_callable=28=29_was_readded_long_ago_in_3=2Ex=2E?= Message-ID: <20160910013350.2071.3284.D0ED4C52@psf.io> https://hg.python.org/cpython/rev/af4529e3594d changeset: 103526:af4529e3594d branch: 2.7 user: Gregory P. Smith [Google Inc.] date: Fri Sep 09 18:33:32 2016 -0700 summary: remove fix_callable, callable() was readded long ago in 3.x. files: Lib/lib2to3/fixes/fix_callable.py | 37 ------------------- 1 files changed, 0 insertions(+), 37 deletions(-) diff --git a/Lib/lib2to3/fixes/fix_callable.py b/Lib/lib2to3/fixes/fix_callable.py deleted file mode 100644 --- a/Lib/lib2to3/fixes/fix_callable.py +++ /dev/null @@ -1,37 +0,0 @@ -# Copyright 2007 Google, Inc. All Rights Reserved. -# Licensed to PSF under a Contributor Agreement. - -"""Fixer for callable(). - -This converts callable(obj) into isinstance(obj, collections.Callable), adding a -collections import if needed.""" - -# Local imports -from lib2to3 import fixer_base -from lib2to3.fixer_util import Call, Name, String, Attr, touch_import - -class FixCallable(fixer_base.BaseFix): - BM_compatible = True - - order = "pre" - - # Ignore callable(*args) or use of keywords. - # Either could be a hint that the builtin callable() is not being used. - PATTERN = """ - power< 'callable' - trailer< lpar='(' - ( not(arglist | argument) any ','> ) - rpar=')' > - after=any* - > - """ - - def transform(self, node, results): - func = results['func'] - - touch_import(None, u'collections', node=node) - - args = [func.clone(), String(u', ')] - args.extend(Attr(Name(u'collections'), Name(u'Callable'))) - return Call(Name(u'isinstance'), args, prefix=node.prefix) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:33:50 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sat, 10 Sep 2016 01:33:50 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI1OTY5?= =?utf-8?q?=3A_Update_the_lib2to3_grammar_to_handle_the_unpacking?= Message-ID: <20160910013350.59502.31101.CC2A76A9@psf.io> https://hg.python.org/cpython/rev/8344cf7eebf8 changeset: 103525:8344cf7eebf8 branch: 2.7 parent: 103499:e84105b48436 user: Gregory P. Smith [Google Inc.] date: Fri Sep 09 18:32:52 2016 -0700 summary: Issue #25969: Update the lib2to3 grammar to handle the unpacking generalizations added in 3.5. files: Lib/lib2to3/Grammar.txt | 23 +++- Lib/lib2to3/fixes/fix_apply.py | 11 ++ Lib/lib2to3/fixes/fix_intern.py | 10 ++ Lib/lib2to3/tests/test_fixers.py | 96 +------------------- Lib/lib2to3/tests/test_parser.py | 35 +++++++ Misc/NEWS | 3 + 6 files changed, 80 insertions(+), 98 deletions(-) diff --git a/Lib/lib2to3/Grammar.txt b/Lib/lib2to3/Grammar.txt --- a/Lib/lib2to3/Grammar.txt +++ b/Lib/lib2to3/Grammar.txt @@ -136,15 +136,26 @@ sliceop: ':' [test] exprlist: (expr|star_expr) (',' (expr|star_expr))* [','] testlist: test (',' test)* [','] -dictsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [','])) | - (test (comp_for | (',' test)* [','])) ) +dictsetmaker: ( ((test ':' test | '**' expr) + (comp_for | (',' (test ':' test | '**' expr))* [','])) | + ((test | star_expr) + (comp_for | (',' (test | star_expr))* [','])) ) classdef: 'class' NAME ['(' [arglist] ')'] ':' suite -arglist: (argument ',')* (argument [','] - |'*' test (',' argument)* [',' '**' test] - |'**' test) -argument: test [comp_for] | test '=' test # Really [keyword '='] test +arglist: argument (',' argument)* [','] + +# "test '=' test" is really "keyword '=' test", but we have no such token. +# These need to be in a single rule to avoid grammar that is ambiguous +# to our LL(1) parser. Even though 'test' includes '*expr' in star_expr, +# we explicitly match '*' here, too, to give it proper precedence. +# Illegal combinations and orderings are blocked in ast.c: +# multiple (test comp_for) arguements are blocked; keyword unpackings +# that precede iterable unpackings are blocked; etc. +argument: ( test [comp_for] | + test '=' test | + '**' expr | + star_expr ) comp_iter: comp_for | comp_if comp_for: 'for' exprlist 'in' testlist_safe [comp_iter] diff --git a/Lib/lib2to3/fixes/fix_apply.py b/Lib/lib2to3/fixes/fix_apply.py --- a/Lib/lib2to3/fixes/fix_apply.py +++ b/Lib/lib2to3/fixes/fix_apply.py @@ -34,6 +34,17 @@ func = results["func"] args = results["args"] kwds = results.get("kwds") + # I feel like we should be able to express this logic in the + # PATTERN above but I don't know how to do it so... + if args: + if args.type == self.syms.star_expr: + return # Make no change. + if (args.type == self.syms.argument and + args.children[0].value == '**'): + return # Make no change. + if kwds and (kwds.type == self.syms.argument and + kwds.children[0].value == '**'): + return # Make no change. prefix = node.prefix func = func.clone() if (func.type not in (token.NAME, syms.atom) and diff --git a/Lib/lib2to3/fixes/fix_intern.py b/Lib/lib2to3/fixes/fix_intern.py --- a/Lib/lib2to3/fixes/fix_intern.py +++ b/Lib/lib2to3/fixes/fix_intern.py @@ -26,6 +26,16 @@ """ def transform(self, node, results): + if results: + # I feel like we should be able to express this logic in the + # PATTERN above but I don't know how to do it so... + obj = results['obj'] + if obj: + if obj.type == self.syms.star_expr: + return # Make no change. + if (obj.type == self.syms.argument and + obj.children[0].value == '**'): + return # Make no change. syms = self.syms obj = results["obj"].clone() if obj.type == syms.arglist: diff --git a/Lib/lib2to3/tests/test_fixers.py b/Lib/lib2to3/tests/test_fixers.py --- a/Lib/lib2to3/tests/test_fixers.py +++ b/Lib/lib2to3/tests/test_fixers.py @@ -260,6 +260,10 @@ s = """apply(f, *args)""" self.unchanged(s) + def test_unchanged_6b(self): + s = """apply(f, **kwds)""" + self.unchanged(s) + def test_unchanged_7(self): s = """apply(func=f, args=args, kwds=kwds)""" self.unchanged(s) @@ -2861,98 +2865,6 @@ a = f + """r'\\\\\\u20ac\\U0001d121\\\\u20ac'""" self.check(b, a) -class Test_callable(FixerTestCase): - fixer = "callable" - - def test_prefix_preservation(self): - b = """callable( x)""" - a = """import collections\nisinstance( x, collections.Callable)""" - self.check(b, a) - - b = """if callable(x): pass""" - a = """import collections -if isinstance(x, collections.Callable): pass""" - self.check(b, a) - - def test_callable_call(self): - b = """callable(x)""" - a = """import collections\nisinstance(x, collections.Callable)""" - self.check(b, a) - - def test_global_import(self): - b = """ -def spam(foo): - callable(foo)"""[1:] - a = """ -import collections -def spam(foo): - isinstance(foo, collections.Callable)"""[1:] - self.check(b, a) - - b = """ -import collections -def spam(foo): - callable(foo)"""[1:] - # same output if it was already imported - self.check(b, a) - - b = """ -from collections import * -def spam(foo): - callable(foo)"""[1:] - a = """ -from collections import * -import collections -def spam(foo): - isinstance(foo, collections.Callable)"""[1:] - self.check(b, a) - - b = """ -do_stuff() -do_some_other_stuff() -assert callable(do_stuff)"""[1:] - a = """ -import collections -do_stuff() -do_some_other_stuff() -assert isinstance(do_stuff, collections.Callable)"""[1:] - self.check(b, a) - - b = """ -if isinstance(do_stuff, Callable): - assert callable(do_stuff) - do_stuff(do_stuff) - if not callable(do_stuff): - exit(1) - else: - assert callable(do_stuff) -else: - assert not callable(do_stuff)"""[1:] - a = """ -import collections -if isinstance(do_stuff, Callable): - assert isinstance(do_stuff, collections.Callable) - do_stuff(do_stuff) - if not isinstance(do_stuff, collections.Callable): - exit(1) - else: - assert isinstance(do_stuff, collections.Callable) -else: - assert not isinstance(do_stuff, collections.Callable)"""[1:] - self.check(b, a) - - def test_callable_should_not_change(self): - a = """callable(*x)""" - self.unchanged(a) - - a = """callable(x, y)""" - self.unchanged(a) - - a = """callable(x, kw=y)""" - self.unchanged(a) - - a = """callable()""" - self.unchanged(a) class Test_filter(FixerTestCase): fixer = "filter" diff --git a/Lib/lib2to3/tests/test_parser.py b/Lib/lib2to3/tests/test_parser.py --- a/Lib/lib2to3/tests/test_parser.py +++ b/Lib/lib2to3/tests/test_parser.py @@ -154,6 +154,41 @@ self.invalid_syntax("raise E from") +# Modelled after Lib/test/test_grammar.py:TokenTests.test_funcdef issue2292 +# and Lib/test/text_parser.py test_list_displays, test_set_displays, +# test_dict_displays, test_argument_unpacking, ... changes. +class TestUnpackingGeneralizations(GrammarTest): + def test_mid_positional_star(self): + self.validate("""func(1, *(2, 3), 4)""") + + def test_double_star_dict_literal(self): + self.validate("""func(**{'eggs':'scrambled', 'spam':'fried'})""") + + def test_double_star_dict_literal_after_keywords(self): + self.validate("""func(spam='fried', **{'eggs':'scrambled'})""") + + def test_list_display(self): + self.validate("""[*{2}, 3, *[4]]""") + + def test_set_display(self): + self.validate("""{*{2}, 3, *[4]}""") + + def test_dict_display_1(self): + self.validate("""{**{}}""") + + def test_dict_display_2(self): + self.validate("""{**{}, 3:4, **{5:6, 7:8}}""") + + def test_argument_unpacking_1(self): + self.validate("""f(a, *b, *c, d)""") + + def test_argument_unpacking_2(self): + self.validate("""f(**a, **b)""") + + def test_argument_unpacking_3(self): + self.validate("""f(2, *a, *b, **b, **c, **d)""") + + # Adaptated from Python 3's Lib/test/test_grammar.py:GrammarTests.testFuncdef class TestFunctionAnnotations(GrammarTest): def test_1(self): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -42,6 +42,9 @@ Library ------- +- Issue #25969: Update the lib2to3 grammar to handle the unpacking + generalizations added in 3.5. + - Issue #24594: Validates persist parameter when opening MSI database - Issue #27570: Avoid zero-length memcpy() etc calls with null source -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:36:39 2016 From: python-checkins at python.org (zach.ware) Date: Sat, 10 Sep 2016 01:36:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328046=3A_Fix_dist?= =?utf-8?q?utils?= Message-ID: <20160910013639.69808.46888.0A799052@psf.io> https://hg.python.org/cpython/rev/2bfe63a3eb5c changeset: 103527:2bfe63a3eb5c parent: 103524:a791a54c0a1c user: Zachary Ware date: Fri Sep 09 18:29:10 2016 -0700 summary: Issue #28046: Fix distutils Why do we have two sysconfig modules again? files: Lib/distutils/sysconfig.py | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py --- a/Lib/distutils/sysconfig.py +++ b/Lib/distutils/sysconfig.py @@ -418,7 +418,11 @@ def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" # _sysconfigdata is generated at build time, see the sysconfig module - name = '_sysconfigdata_' + sys.abiflags + name = '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( + abi=sys.abiflags, + platform=sys.platform, + multiarch=getattr(sys.implementation, '_multiarch', ''), + ) _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) build_time_vars = _temp.build_time_vars global _config_vars -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:42:08 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 01:42:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NTY2?= =?utf-8?q?=3A_Fix_clean_target_in_freeze_makefile_=28patch_by_Lisa_Roach?= =?utf-8?q?=29?= Message-ID: <20160910014208.2439.43672.A914D63C@psf.io> https://hg.python.org/cpython/rev/3ffff303df95 changeset: 103529:3ffff303df95 branch: 3.5 parent: 103523:7f7a994bbfbc user: Steve Dower date: Fri Sep 09 18:38:20 2016 -0700 summary: Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach) files: Misc/NEWS | 2 ++ Tools/freeze/winmakemakefile.py | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -301,6 +301,8 @@ Build ----- +- Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach) + - Issue #27705: Update message in validate_ucrtbase.py - Issue #27983: Cause lack of llvm-profdata tool when using clang as diff --git a/Tools/freeze/winmakemakefile.py b/Tools/freeze/winmakemakefile.py --- a/Tools/freeze/winmakemakefile.py +++ b/Tools/freeze/winmakemakefile.py @@ -144,5 +144,5 @@ print("<<") print() print("clean:") - print("\t-rm -f *.obj") - print("\t-rm -f $(target).exe") + print("\t-del /f *.obj") + print("\t-del /f $(target).exe") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:42:08 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 01:42:08 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3NTY2?= =?utf-8?q?=3A_Fix_clean_target_in_freeze_makefile_=28patch_by_Lisa_Roach?= =?utf-8?q?=29?= Message-ID: <20160910014208.2071.86031.52C9E35D@psf.io> https://hg.python.org/cpython/rev/e96eb2bd0d5e changeset: 103528:e96eb2bd0d5e branch: 2.7 parent: 103526:af4529e3594d user: Steve Dower date: Fri Sep 09 18:38:10 2016 -0700 summary: Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach) files: Misc/NEWS | 2 ++ Tools/freeze/winmakemakefile.py | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -170,6 +170,8 @@ Build ----- +- Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach) + - Issue #27983: Cause lack of llvm-profdata tool when using clang as required for PGO linking to be a configure time error rather than make time when --with-optimizations is enabled. Also improve our diff --git a/Tools/freeze/winmakemakefile.py b/Tools/freeze/winmakemakefile.py --- a/Tools/freeze/winmakemakefile.py +++ b/Tools/freeze/winmakemakefile.py @@ -143,5 +143,5 @@ print "<<" print print "clean:" - print "\t-rm -f *.obj" - print "\t-rm -f $(target).exe" + print "\t-del /f *.obj" + print "\t-del /f $(target).exe" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:42:08 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 01:42:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327566=3A_Fix_clean_target_in_freeze_makefile_?= =?utf-8?q?=28patch_by_Lisa_Roach=29?= Message-ID: <20160910014208.48739.73845.B15C81A5@psf.io> https://hg.python.org/cpython/rev/6a2d95630a7c changeset: 103530:6a2d95630a7c parent: 103527:2bfe63a3eb5c parent: 103529:3ffff303df95 user: Steve Dower date: Fri Sep 09 18:41:56 2016 -0700 summary: Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach) files: Misc/NEWS | 2 ++ Tools/freeze/winmakemakefile.py | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -328,6 +328,8 @@ Build ----- +- Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach) + - Issue #27705: Update message in validate_ucrtbase.py - Issue #27976: Deprecate building _ctypes with the bundled copy of libffi on diff --git a/Tools/freeze/winmakemakefile.py b/Tools/freeze/winmakemakefile.py --- a/Tools/freeze/winmakemakefile.py +++ b/Tools/freeze/winmakemakefile.py @@ -144,5 +144,5 @@ print("<<") print() print("clean:") - print("\t-rm -f *.obj") - print("\t-rm -f $(target).exe") + print("\t-del /f *.obj") + print("\t-del /f $(target).exe") -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 21:56:23 2016 From: python-checkins at python.org (eric.smith) Date: Sat, 10 Sep 2016 01:56:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27948=3A_Allow_backs?= =?utf-8?q?lashes_in_the_literal_string_portion_of_f-strings=2C_but?= Message-ID: <20160910015623.87446.2880.E59CF46E@psf.io> https://hg.python.org/cpython/rev/b3ac1e154cdd changeset: 103531:b3ac1e154cdd user: Eric V. Smith date: Fri Sep 09 21:56:20 2016 -0400 summary: Issue 27948: Allow backslashes in the literal string portion of f-strings, but not in the expressions. Also, require expressions to begin and end with literal curly braces. files: Lib/http/client.py | 2 +- Lib/test/libregrtest/save_env.py | 2 +- Lib/test/test_faulthandler.py | 4 +- Lib/test/test_fstring.py | 120 ++- Lib/test/test_tools/test_unparse.py | 10 +- Lib/test/test_traceback.py | 28 +- Lib/traceback.py | 4 +- Misc/NEWS | 12 +- Python/ast.c | 496 +++++++--------- 9 files changed, 329 insertions(+), 349 deletions(-) diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -1060,7 +1060,7 @@ if encode_chunked and self._http_vsn == 11: # chunked encoding - chunk = f'{len(chunk):X}''\r\n'.encode('ascii') + chunk \ + chunk = f'{len(chunk):X}\r\n'.encode('ascii') + chunk \ + b'\r\n' self.send(chunk) diff --git a/Lib/test/libregrtest/save_env.py b/Lib/test/libregrtest/save_env.py --- a/Lib/test/libregrtest/save_env.py +++ b/Lib/test/libregrtest/save_env.py @@ -280,6 +280,6 @@ print(f"Warning -- {name} was modified by {self.testname}", file=sys.stderr, flush=True) if self.verbose > 1: - print(f" Before: {original}""\n"f" After: {current} ", + print(f" Before: {original}\n After: {current} ", file=sys.stderr, flush=True) return False diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py --- a/Lib/test/test_faulthandler.py +++ b/Lib/test/test_faulthandler.py @@ -735,11 +735,11 @@ ('EXCEPTION_INT_DIVIDE_BY_ZERO', 'int divide by zero'), ('EXCEPTION_STACK_OVERFLOW', 'stack overflow'), ): - self.check_windows_exception(""" + self.check_windows_exception(f""" import faulthandler faulthandler.enable() faulthandler._raise_exception(faulthandler._{exc}) - """.format(exc=exc), + """, 3, name) diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -119,6 +119,14 @@ self.assertEqual(f'a}}', 'a}') self.assertEqual(f'}}b', '}b') self.assertEqual(f'a}}b', 'a}b') + self.assertEqual(f'{{}}', '{}') + self.assertEqual(f'a{{}}', 'a{}') + self.assertEqual(f'{{b}}', '{b}') + self.assertEqual(f'{{}}c', '{}c') + self.assertEqual(f'a{{b}}', 'a{b}') + self.assertEqual(f'a{{}}c', 'a{}c') + self.assertEqual(f'{{b}}c', '{b}c') + self.assertEqual(f'a{{b}}c', 'a{b}c') self.assertEqual(f'{{{10}', '{10') self.assertEqual(f'}}{10}', '}10') @@ -302,56 +310,79 @@ ["f'{\n}'", ]) - def test_no_backslashes(self): - # See issue 27921 + def test_backslashes_in_string_part(self): + self.assertEqual(f'\t', '\t') + self.assertEqual(r'\t', '\\t') + self.assertEqual(rf'\t', '\\t') + self.assertEqual(f'{2}\t', '2\t') + self.assertEqual(f'{2}\t{3}', '2\t3') + self.assertEqual(f'\t{3}', '\t3') - # These should work, but currently don't - self.assertAllRaise(SyntaxError, 'backslashes not allowed', - [r"f'\t'", - r"f'{2}\t'", - r"f'{2}\t{3}'", - r"f'\t{3}'", + self.assertEqual(f'\u0394', '\u0394') + self.assertEqual(r'\u0394', '\\u0394') + self.assertEqual(rf'\u0394', '\\u0394') + self.assertEqual(f'{2}\u0394', '2\u0394') + self.assertEqual(f'{2}\u0394{3}', '2\u03943') + self.assertEqual(f'\u0394{3}', '\u03943') - r"f'\N{GREEK CAPITAL LETTER DELTA}'", - r"f'{2}\N{GREEK CAPITAL LETTER DELTA}'", - r"f'{2}\N{GREEK CAPITAL LETTER DELTA}{3}'", - r"f'\N{GREEK CAPITAL LETTER DELTA}{3}'", + self.assertEqual(f'\U00000394', '\u0394') + self.assertEqual(r'\U00000394', '\\U00000394') + self.assertEqual(rf'\U00000394', '\\U00000394') + self.assertEqual(f'{2}\U00000394', '2\u0394') + self.assertEqual(f'{2}\U00000394{3}', '2\u03943') + self.assertEqual(f'\U00000394{3}', '\u03943') - r"f'\u0394'", - r"f'{2}\u0394'", - r"f'{2}\u0394{3}'", - r"f'\u0394{3}'", + self.assertEqual(f'\N{GREEK CAPITAL LETTER DELTA}', '\u0394') + self.assertEqual(f'{2}\N{GREEK CAPITAL LETTER DELTA}', '2\u0394') + self.assertEqual(f'{2}\N{GREEK CAPITAL LETTER DELTA}{3}', '2\u03943') + self.assertEqual(f'\N{GREEK CAPITAL LETTER DELTA}{3}', '\u03943') + self.assertEqual(f'2\N{GREEK CAPITAL LETTER DELTA}', '2\u0394') + self.assertEqual(f'2\N{GREEK CAPITAL LETTER DELTA}3', '2\u03943') + self.assertEqual(f'\N{GREEK CAPITAL LETTER DELTA}3', '\u03943') - r"f'\U00000394'", - r"f'{2}\U00000394'", - r"f'{2}\U00000394{3}'", - r"f'\U00000394{3}'", + self.assertEqual(f'\x20', ' ') + self.assertEqual(r'\x20', '\\x20') + self.assertEqual(rf'\x20', '\\x20') + self.assertEqual(f'{2}\x20', '2 ') + self.assertEqual(f'{2}\x20{3}', '2 3') + self.assertEqual(f'\x20{3}', ' 3') - r"f'\x20'", - r"f'{2}\x20'", - r"f'{2}\x20{3}'", - r"f'\x20{3}'", + self.assertEqual(f'2\x20', '2 ') + self.assertEqual(f'2\x203', '2 3') + self.assertEqual(f'\x203', ' 3') - r"f'2\x20'", - r"f'2\x203'", - r"f'2\x203'", + def test_misformed_unicode_character_name(self): + # These test are needed because unicode names are parsed + # differently inside f-strings. + self.assertAllRaise(SyntaxError, r"\(unicode error\) 'unicodeescape' codec can't decode bytes in position .*: malformed \\N character escape", + [r"f'\N'", + r"f'\N{'", + r"f'\N{GREEK CAPITAL LETTER DELTA'", + + # Here are the non-f-string versions, + # which should give the same errors. + r"'\N'", + r"'\N{'", + r"'\N{GREEK CAPITAL LETTER DELTA'", ]) - # And these don't work now, and shouldn't work in the future. - self.assertAllRaise(SyntaxError, 'backslashes not allowed', + def test_no_backslashes_in_expression_part(self): + self.assertAllRaise(SyntaxError, 'f-string expression part cannot include a backslash', [r"f'{\'a\'}'", r"f'{\t3}'", + r"f'{\}'", + r"rf'{\'a\'}'", + r"rf'{\t3}'", + r"rf'{\}'", + r"""rf'{"\N{LEFT CURLY BRACKET}"}'""", ]) - # add this when backslashes are allowed again. see issue 27921 - # these test will be needed because unicode names will be parsed - # differently once backslashes are allowed inside expressions - ## def test_misformed_unicode_character_name(self): - ## self.assertAllRaise(SyntaxError, 'xx', - ## [r"f'\N'", - ## [r"f'\N{'", - ## [r"f'\N{GREEK CAPITAL LETTER DELTA'", - ## ]) + def test_no_escapes_for_braces(self): + # \x7b is '{'. Make sure it doesn't start an expression. + self.assertEqual(f'\x7b2}}', '{2}') + self.assertEqual(f'\x7b2', '{2') + self.assertEqual(f'\u007b2', '{2') + self.assertEqual(f'\N{LEFT CURLY BRACKET}2\N{RIGHT CURLY BRACKET}', '{2}') def test_newlines_in_expressions(self): self.assertEqual(f'{0}', '0') @@ -509,6 +540,14 @@ "ruf''", "FUR''", "Fur''", + "fb''", + "fB''", + "Fb''", + "FB''", + "bf''", + "bF''", + "Bf''", + "BF''", ]) def test_leading_trailing_spaces(self): @@ -551,8 +590,8 @@ self.assertAllRaise(SyntaxError, 'f-string: invalid conversion character', ["f'{3!g}'", "f'{3!A}'", - "f'{3!A}'", - "f'{3!A}'", + "f'{3!3}'", + "f'{3!G}'", "f'{3!!}'", "f'{3!:}'", "f'{3! s}'", # no space before conversion char @@ -601,6 +640,7 @@ "f'{3!s:3'", "f'x{'", "f'x{x'", + "f'{x'", "f'{3:s'", "f'{{{'", "f'{{}}{'", diff --git a/Lib/test/test_tools/test_unparse.py b/Lib/test/test_tools/test_unparse.py --- a/Lib/test/test_tools/test_unparse.py +++ b/Lib/test/test_tools/test_unparse.py @@ -285,12 +285,12 @@ if test.support.verbose: print('Testing %s' % filename) - # it's very much a hack that I'm skipping these files, but - # I can't figure out why they fail. I'll fix it when I - # address issue #27948. - if os.path.basename(filename) in ('test_fstring.py', 'test_traceback.py'): + # Some f-strings are not correctly round-tripped by + # Tools/parser/unparse.py. See issue 28002 for details. + # We need to skip files that contain such f-strings. + if os.path.basename(filename) in ('test_fstring.py', ): if test.support.verbose: - print(f'Skipping {filename}: see issue 27921') + print(f'Skipping {filename}: see issue 28002') continue with self.subTest(filename=filename): diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -326,13 +326,13 @@ lineno_f = f.__code__.co_firstlineno result_f = ( 'Traceback (most recent call last):\n' - f' File "{__file__}", line {lineno_f+5}, in _check_recursive_traceback_display''\n' + f' File "{__file__}", line {lineno_f+5}, in _check_recursive_traceback_display\n' ' f()\n' - f' File "{__file__}", line {lineno_f+1}, in f''\n' + f' File "{__file__}", line {lineno_f+1}, in f\n' ' f()\n' - f' File "{__file__}", line {lineno_f+1}, in f''\n' + f' File "{__file__}", line {lineno_f+1}, in f\n' ' f()\n' - f' File "{__file__}", line {lineno_f+1}, in f''\n' + f' File "{__file__}", line {lineno_f+1}, in f\n' ' f()\n' # XXX: The following line changes depending on whether the tests # are run through the interactive interpreter or with -m @@ -371,20 +371,20 @@ lineno_g = g.__code__.co_firstlineno result_g = ( - f' File "{__file__}", line {lineno_g+2}, in g''\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' ' return g(count-1)\n' - f' File "{__file__}", line {lineno_g+2}, in g''\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' ' return g(count-1)\n' - f' File "{__file__}", line {lineno_g+2}, in g''\n' + f' File "{__file__}", line {lineno_g+2}, in g\n' ' return g(count-1)\n' ' [Previous line repeated 6 more times]\n' - f' File "{__file__}", line {lineno_g+3}, in g''\n' + f' File "{__file__}", line {lineno_g+3}, in g\n' ' raise ValueError\n' 'ValueError\n' ) tb_line = ( 'Traceback (most recent call last):\n' - f' File "{__file__}", line {lineno_g+7}, in _check_recursive_traceback_display''\n' + f' File "{__file__}", line {lineno_g+7}, in _check_recursive_traceback_display\n' ' g()\n' ) expected = (tb_line + result_g).splitlines() @@ -408,16 +408,16 @@ lineno_h = h.__code__.co_firstlineno result_h = ( 'Traceback (most recent call last):\n' - f' File "{__file__}", line {lineno_h+7}, in _check_recursive_traceback_display''\n' + f' File "{__file__}", line {lineno_h+7}, in _check_recursive_traceback_display\n' ' h()\n' - f' File "{__file__}", line {lineno_h+2}, in h''\n' + f' File "{__file__}", line {lineno_h+2}, in h\n' ' return h(count-1)\n' - f' File "{__file__}", line {lineno_h+2}, in h''\n' + f' File "{__file__}", line {lineno_h+2}, in h\n' ' return h(count-1)\n' - f' File "{__file__}", line {lineno_h+2}, in h''\n' + f' File "{__file__}", line {lineno_h+2}, in h\n' ' return h(count-1)\n' ' [Previous line repeated 6 more times]\n' - f' File "{__file__}", line {lineno_h+3}, in h''\n' + f' File "{__file__}", line {lineno_h+3}, in h\n' ' g()\n' ) expected = (result_h + result_g).splitlines() diff --git a/Lib/traceback.py b/Lib/traceback.py --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -402,7 +402,7 @@ count += 1 else: if count > 3: - result.append(f' [Previous line repeated {count-3} more times]'+'\n') + result.append(f' [Previous line repeated {count-3} more times]\n') last_file = frame.filename last_line = frame.lineno last_name = frame.name @@ -419,7 +419,7 @@ row.append(' {name} = {value}\n'.format(name=name, value=value)) result.append(''.join(row)) if count > 3: - result.append(f' [Previous line repeated {count-3} more times]'+'\n') + result.append(f' [Previous line repeated {count-3} more times]\n') return result diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,13 @@ Core and Builtins ----------------- +- Issue #27948: In f-strings, only allow backslashes inside the braces + (where the expressions are). This is a breaking change from the 3.6 + alpha releases, where backslashes are allowed anywhere in an + f-string. Also, require that expressions inside f-strings be + enclosed within literal braces, and not escapes like + f'\x7b"hi"\x7d'. + - Issue #28046: Remove platform-specific directories from sys.path. - Issue #25758: Prevents zipimport from unnecessarily encoding a filename @@ -56,11 +63,6 @@ - Issue #27355: Removed support for Windows CE. It was never finished, and Windows CE is no longer a relevant platform for Python. -- Issue #27921: Disallow backslashes in f-strings. This is a temporary - restriction: in beta 2, backslashes will only be disallowed inside - the braces (where the expressions are). This is a breaking change - from the 3.6 alpha releases. - - Implement PEP 523. - Issue #27870: A left shift of zero by a large integer no longer attempts diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -4155,141 +4155,74 @@ return v; } -/* Compile this expression in to an expr_ty. We know that we can - temporarily modify the character before the start of this string - (it's '{'), and we know we can temporarily modify the character - after this string (it is a '}'). Leverage this to create a - sub-string with enough room for us to add parens around the - expression. This is to allow strings with embedded newlines, for - example. */ +/* Compile this expression in to an expr_ty. Add parens around the + expression, in order to allow leading spaces in the expression. */ static expr_ty -fstring_compile_expr(PyObject *str, Py_ssize_t expr_start, - Py_ssize_t expr_end, struct compiling *c, const node *n) +fstring_compile_expr(const char *expr_start, const char *expr_end, + struct compiling *c, const node *n) { + int all_whitespace = 1; + int kind; + void *data; PyCompilerFlags cf; mod_ty mod; - char *utf_expr; + char *str; + PyObject *o; + Py_ssize_t len; Py_ssize_t i; - Py_UCS4 end_ch = -1; - int all_whitespace; - PyObject *sub = NULL; - - /* We only decref sub if we allocated it with a PyUnicode_Substring. - decref_sub records that. */ - int decref_sub = 0; - - assert(str); - - assert(expr_start >= 0 && expr_start < PyUnicode_GET_LENGTH(str)); - assert(expr_end >= 0 && expr_end < PyUnicode_GET_LENGTH(str)); + assert(expr_end >= expr_start); - - /* There has to be at least one character on each side of the - expression inside this str. This will have been caught before - we're called. */ - assert(expr_start >= 1); - assert(expr_end <= PyUnicode_GET_LENGTH(str)-1); - - /* If the substring is all whitespace, it's an error. We need to - catch this here, and not when we call PyParser_ASTFromString, - because turning the expression '' in to '()' would go from - being invalid to valid. */ - /* Note that this code says an empty string is all - whitespace. That's important. There's a test for it: f'{}'. */ - all_whitespace = 1; - for (i = expr_start; i < expr_end; i++) { - if (!Py_UNICODE_ISSPACE(PyUnicode_READ_CHAR(str, i))) { + assert(*(expr_start-1) == '{'); + assert(*expr_end == '}' || *expr_end == '!' || *expr_end == ':'); + + /* We know there are no escapes here, because backslashes are not allowed, + and we know it's utf-8 encoded (per PEP 263). But, in order to check + that each char is not whitespace, we need to decode it to unicode. + Which is unfortunate, but such is life. */ + + /* If the substring is all whitespace, it's an error. We need to catch + this here, and not when we call PyParser_ASTFromString, because turning + the expression '' in to '()' would go from being invalid to valid. */ + /* Note that this code says an empty string is all whitespace. That's + important. There's a test for it: f'{}'. */ + o = PyUnicode_DecodeUTF8(expr_start, expr_end-expr_start, NULL); + if (o == NULL) + return NULL; + len = PyUnicode_GET_LENGTH(o); + kind = PyUnicode_KIND(o); + data = PyUnicode_DATA(o); + for (i = 0; i < len; i++) { + if (!Py_UNICODE_ISSPACE(PyUnicode_READ(kind, data, i))) { all_whitespace = 0; break; } } + Py_DECREF(o); if (all_whitespace) { ast_error(c, n, "f-string: empty expression not allowed"); - goto error; + return NULL; } - /* If the substring will be the entire source string, we can't use - PyUnicode_Substring, since it will return another reference to - our original string. Because we're modifying the string in - place, that's a no-no. So, detect that case and just use our - string directly. */ - - if (expr_start-1 == 0 && expr_end+1 == PyUnicode_GET_LENGTH(str)) { - /* If str is well formed, then the first and last chars must - be '{' and '}', respectively. But, if there's a syntax - error, for example f'{3!', then the last char won't be a - closing brace. So, remember the last character we read in - order for us to restore it. */ - end_ch = PyUnicode_ReadChar(str, expr_end-expr_start+1); - assert(end_ch != (Py_UCS4)-1); - - /* In all cases, however, start_ch must be '{'. */ - assert(PyUnicode_ReadChar(str, 0) == '{'); - - sub = str; - } else { - /* Create a substring object. It must be a new object, with - refcount==1, so that we can modify it. */ - sub = PyUnicode_Substring(str, expr_start-1, expr_end+1); - if (!sub) - goto error; - assert(sub != str); /* Make sure it's a new string. */ - decref_sub = 1; /* Remember to deallocate it on error. */ - } - - /* Put () around the expression. */ - if (PyUnicode_WriteChar(sub, 0, '(') < 0 || - PyUnicode_WriteChar(sub, expr_end-expr_start+1, ')') < 0) - goto error; - - /* No need to free the memory returned here: it's managed by the - string. */ - utf_expr = PyUnicode_AsUTF8(sub); - if (!utf_expr) - goto error; + /* Reuse len to be the length of the utf-8 input string. */ + len = expr_end - expr_start; + /* Allocate 3 extra bytes: open paren, close paren, null byte. */ + str = PyMem_RawMalloc(len + 3); + if (str == NULL) + return NULL; + + str[0] = '('; + memcpy(str+1, expr_start, len); + str[len+1] = ')'; + str[len+2] = 0; cf.cf_flags = PyCF_ONLY_AST; - mod = PyParser_ASTFromString(utf_expr, "", + mod = PyParser_ASTFromString(str, "", Py_eval_input, &cf, c->c_arena); + PyMem_RawFree(str); if (!mod) - goto error; - - if (sub != str) - /* Clear instead of decref in case we ever modify this code to change - the error handling: this is safest because the XDECREF won't try - and decref it when it's NULL. */ - /* No need to restore the chars in sub, since we know it's getting - ready to get deleted (refcount must be 1, since we got a new string - in PyUnicode_Substring). */ - Py_CLEAR(sub); - else { - assert(!decref_sub); - assert(end_ch != (Py_UCS4)-1); - /* Restore str, which we earlier modified directly. */ - if (PyUnicode_WriteChar(str, 0, '{') < 0 || - PyUnicode_WriteChar(str, expr_end-expr_start+1, end_ch) < 0) - goto error; - } + return NULL; return mod->v.Expression.body; - -error: - /* Only decref sub if it was the result of a call to SubString. */ - if (decref_sub) - Py_XDECREF(sub); - - if (end_ch != (Py_UCS4)-1) { - /* We only get here if we modified str. Make sure that's the - case: str will be equal to sub. */ - if (str == sub) { - /* Don't check the error, because we've already set the - error state (that's why we're in 'error', after - all). */ - PyUnicode_WriteChar(str, 0, '{'); - PyUnicode_WriteChar(str, expr_end-expr_start+1, end_ch); - } - } - return NULL; } /* Return -1 on error. @@ -4301,35 +4234,38 @@ doubled braces. */ static int -fstring_find_literal(PyObject *str, Py_ssize_t *ofs, PyObject **literal, - int recurse_lvl, struct compiling *c, const node *n) +fstring_find_literal(const char **str, const char *end, int raw, + PyObject **literal, int recurse_lvl, + struct compiling *c, const node *n) { - /* Get any literal string. It ends when we hit an un-doubled brace, or the - end of the string. */ - - Py_ssize_t literal_start, literal_end; + /* Get any literal string. It ends when we hit an un-doubled left + brace (which isn't part of a unicode name escape such as + "\N{EULER CONSTANT}"), or the end of the string. */ + + const char *literal_start = *str; + const char *literal_end; + int in_named_escape = 0; int result = 0; - enum PyUnicode_Kind kind = PyUnicode_KIND(str); - void *data = PyUnicode_DATA(str); - assert(*literal == NULL); - - literal_start = *ofs; - for (; *ofs < PyUnicode_GET_LENGTH(str); *ofs += 1) { - Py_UCS4 ch = PyUnicode_READ(kind, data, *ofs); - if (ch == '{' || ch == '}') { + for (; *str < end; (*str)++) { + char ch = **str; + if (!in_named_escape && ch == '{' && (*str)-literal_start >= 2 && + *(*str-2) == '\\' && *(*str-1) == 'N') { + in_named_escape = 1; + } else if (in_named_escape && ch == '}') { + in_named_escape = 0; + } else if (ch == '{' || ch == '}') { /* Check for doubled braces, but only at the top level. If we checked at every level, then f'{0:{3}}' would fail with the two closing braces. */ if (recurse_lvl == 0) { - if (*ofs + 1 < PyUnicode_GET_LENGTH(str) && - PyUnicode_READ(kind, data, *ofs + 1) == ch) { + if (*str+1 < end && *(*str+1) == ch) { /* We're going to tell the caller that the literal ends here, but that they should continue scanning. But also skip over the second brace when we resume scanning. */ - literal_end = *ofs + 1; - *ofs += 2; + literal_end = *str+1; + *str += 2; result = 1; goto done; } @@ -4341,34 +4277,36 @@ return -1; } } - /* We're either at a '{', which means we're starting another expression; or a '}', which means we're at the end of this f-string (for a nested format_spec). */ break; } } - literal_end = *ofs; - - assert(*ofs == PyUnicode_GET_LENGTH(str) || - PyUnicode_READ(kind, data, *ofs) == '{' || - PyUnicode_READ(kind, data, *ofs) == '}'); + literal_end = *str; + assert(*str <= end); + assert(*str == end || **str == '{' || **str == '}'); done: if (literal_start != literal_end) { - *literal = PyUnicode_Substring(str, literal_start, literal_end); + if (raw) + *literal = PyUnicode_DecodeUTF8Stateful(literal_start, + literal_end-literal_start, + NULL, NULL); + else + *literal = decode_unicode_with_escapes(c, literal_start, + literal_end-literal_start); if (!*literal) return -1; } - return result; } /* Forward declaration because parsing is recursive. */ static expr_ty -fstring_parse(PyObject *str, Py_ssize_t *ofs, int recurse_lvl, +fstring_parse(const char **str, const char *end, int raw, int recurse_lvl, struct compiling *c, const node *n); -/* Parse the f-string str, starting at ofs. We know *ofs starts an +/* Parse the f-string at *str, ending at end. We know *str starts an expression (so it must be a '{'). Returns the FormattedValue node, which includes the expression, conversion character, and format_spec expression. @@ -4379,23 +4317,20 @@ find the end of all valid ones. Any errors inside the expression will be caught when we parse it later. */ static int -fstring_find_expr(PyObject *str, Py_ssize_t *ofs, int recurse_lvl, +fstring_find_expr(const char **str, const char *end, int raw, int recurse_lvl, expr_ty *expression, struct compiling *c, const node *n) { /* Return -1 on error, else 0. */ - Py_ssize_t expr_start; - Py_ssize_t expr_end; + const char *expr_start; + const char *expr_end; expr_ty simple_expression; expr_ty format_spec = NULL; /* Optional format specifier. */ - Py_UCS4 conversion = -1; /* The conversion char. -1 if not specified. */ - - enum PyUnicode_Kind kind = PyUnicode_KIND(str); - void *data = PyUnicode_DATA(str); + char conversion = -1; /* The conversion char. -1 if not specified. */ /* 0 if we're not in a string, else the quote char we're trying to match (single or double quote). */ - Py_UCS4 quote_char = 0; + char quote_char = 0; /* If we're inside a string, 1=normal, 3=triple-quoted. */ int string_type = 0; @@ -4412,22 +4347,30 @@ /* The first char must be a left brace, or we wouldn't have gotten here. Skip over it. */ - assert(PyUnicode_READ(kind, data, *ofs) == '{'); - *ofs += 1; - - expr_start = *ofs; - for (; *ofs < PyUnicode_GET_LENGTH(str); *ofs += 1) { - Py_UCS4 ch; + assert(**str == '{'); + *str += 1; + + expr_start = *str; + for (; *str < end; (*str)++) { + char ch; /* Loop invariants. */ assert(nested_depth >= 0); - assert(*ofs >= expr_start); + assert(*str >= expr_start && *str < end); if (quote_char) assert(string_type == 1 || string_type == 3); else assert(string_type == 0); - ch = PyUnicode_READ(kind, data, *ofs); + ch = **str; + /* Nowhere inside an expression is a backslash allowed. */ + if (ch == '\\') { + /* Error: can't include a backslash character, inside + parens or strings or not. */ + ast_error(c, n, "f-string expression part " + "cannot include a backslash"); + return -1; + } if (quote_char) { /* We're inside a string. See if we're at the end. */ /* This code needs to implement the same non-error logic @@ -4443,11 +4386,9 @@ /* Does this match the string_type (single or triple quoted)? */ if (string_type == 3) { - if (*ofs+2 < PyUnicode_GET_LENGTH(str) && - PyUnicode_READ(kind, data, *ofs+1) == ch && - PyUnicode_READ(kind, data, *ofs+2) == ch) { + if (*str+2 < end && *(*str+1) == ch && *(*str+2) == ch) { /* We're at the end of a triple quoted string. */ - *ofs += 2; + *str += 2; string_type = 0; quote_char = 0; continue; @@ -4459,21 +4400,11 @@ continue; } } - /* We're inside a string, and not finished with the - string. If this is a backslash, skip the next char (it - might be an end quote that needs skipping). Otherwise, - just consume this character normally. */ - if (ch == '\\' && *ofs+1 < PyUnicode_GET_LENGTH(str)) { - /* Just skip the next char, whatever it is. */ - *ofs += 1; - } } else if (ch == '\'' || ch == '"') { /* Is this a triple quoted string? */ - if (*ofs+2 < PyUnicode_GET_LENGTH(str) && - PyUnicode_READ(kind, data, *ofs+1) == ch && - PyUnicode_READ(kind, data, *ofs+2) == ch) { + if (*str+2 < end && *(*str+1) == ch && *(*str+2) == ch) { string_type = 3; - *ofs += 2; + *str += 2; } else { /* Start of a normal string. */ string_type = 1; @@ -4495,18 +4426,17 @@ /* First, test for the special case of "!=". Since '=' is not an allowed conversion character, nothing is lost in this test. */ - if (ch == '!' && *ofs+1 < PyUnicode_GET_LENGTH(str) && - PyUnicode_READ(kind, data, *ofs+1) == '=') + if (ch == '!' && *str+1 < end && *(*str+1) == '=') { /* This isn't a conversion character, just continue. */ continue; - + } /* Normal way out of this loop. */ break; } else { /* Just consume this char and loop around. */ } } - expr_end = *ofs; + expr_end = *str; /* If we leave this loop in a string or with mismatched parens, we don't care. We'll get a syntax error when compiling the expression. But, we can produce a better error message, so @@ -4520,24 +4450,24 @@ return -1; } - if (*ofs >= PyUnicode_GET_LENGTH(str)) + if (*str >= end) goto unexpected_end_of_string; /* Compile the expression as soon as possible, so we show errors related to the expression before errors related to the conversion or format_spec. */ - simple_expression = fstring_compile_expr(str, expr_start, expr_end, c, n); + simple_expression = fstring_compile_expr(expr_start, expr_end, c, n); if (!simple_expression) return -1; /* Check for a conversion char, if present. */ - if (PyUnicode_READ(kind, data, *ofs) == '!') { - *ofs += 1; - if (*ofs >= PyUnicode_GET_LENGTH(str)) + if (**str == '!') { + *str += 1; + if (*str >= end) goto unexpected_end_of_string; - conversion = PyUnicode_READ(kind, data, *ofs); - *ofs += 1; + conversion = **str; + *str += 1; /* Validate the conversion. */ if (!(conversion == 's' || conversion == 'r' @@ -4549,30 +4479,29 @@ } /* Check for the format spec, if present. */ - if (*ofs >= PyUnicode_GET_LENGTH(str)) + if (*str >= end) goto unexpected_end_of_string; - if (PyUnicode_READ(kind, data, *ofs) == ':') { - *ofs += 1; - if (*ofs >= PyUnicode_GET_LENGTH(str)) + if (**str == ':') { + *str += 1; + if (*str >= end) goto unexpected_end_of_string; /* Parse the format spec. */ - format_spec = fstring_parse(str, ofs, recurse_lvl+1, c, n); + format_spec = fstring_parse(str, end, raw, recurse_lvl+1, c, n); if (!format_spec) return -1; } - if (*ofs >= PyUnicode_GET_LENGTH(str) || - PyUnicode_READ(kind, data, *ofs) != '}') + if (*str >= end || **str != '}') goto unexpected_end_of_string; /* We're at a right brace. Consume it. */ - assert(*ofs < PyUnicode_GET_LENGTH(str)); - assert(PyUnicode_READ(kind, data, *ofs) == '}'); - *ofs += 1; - - /* And now create the FormattedValue node that represents this entire - expression with the conversion and format spec. */ + assert(*str < end); + assert(**str == '}'); + *str += 1; + + /* And now create the FormattedValue node that represents this + entire expression with the conversion and format spec. */ *expression = FormattedValue(simple_expression, (int)conversion, format_spec, LINENO(n), n->n_col_offset, c->c_arena); @@ -4610,8 +4539,9 @@ we're finished. */ static int -fstring_find_literal_and_expr(PyObject *str, Py_ssize_t *ofs, int recurse_lvl, - PyObject **literal, expr_ty *expression, +fstring_find_literal_and_expr(const char **str, const char *end, int raw, + int recurse_lvl, PyObject **literal, + expr_ty *expression, struct compiling *c, const node *n) { int result; @@ -4619,7 +4549,7 @@ assert(*literal == NULL && *expression == NULL); /* Get any literal string. */ - result = fstring_find_literal(str, ofs, literal, recurse_lvl, c, n); + result = fstring_find_literal(str, end, raw, literal, recurse_lvl, c, n); if (result < 0) goto error; @@ -4629,10 +4559,7 @@ /* We have a literal, but don't look at the expression. */ return 1; - assert(*ofs <= PyUnicode_GET_LENGTH(str)); - - if (*ofs >= PyUnicode_GET_LENGTH(str) || - PyUnicode_READ_CHAR(str, *ofs) == '}') + if (*str >= end || **str == '}') /* We're at the end of the string or the end of a nested f-string: no expression. The top-level error case where we expect to be at the end of the string but we're at a '}' is @@ -4640,10 +4567,9 @@ return 0; /* We must now be the start of an expression, on a '{'. */ - assert(*ofs < PyUnicode_GET_LENGTH(str) && - PyUnicode_READ_CHAR(str, *ofs) == '{'); - - if (fstring_find_expr(str, ofs, recurse_lvl, expression, c, n) < 0) + assert(**str == '{'); + + if (fstring_find_expr(str, end, raw, recurse_lvl, expression, c, n) < 0) goto error; return 0; @@ -4852,13 +4778,11 @@ return 0; } -/* Parse an f-string. The f-string is in str, starting at ofs, with no 'f' - or quotes. str is not decref'd, since we don't know if it's used elsewhere. - And if we're only looking at a part of a string, then decref'ing is - definitely not the right thing to do! */ +/* Parse an f-string. The f-string is in *str to end, with no + 'f' or quotes. */ static int -FstringParser_ConcatFstring(FstringParser *state, PyObject *str, - Py_ssize_t *ofs, int recurse_lvl, +FstringParser_ConcatFstring(FstringParser *state, const char **str, + const char *end, int raw, int recurse_lvl, struct compiling *c, const node *n) { FstringParser_check_invariants(state); @@ -4872,7 +4796,7 @@ expression, literal will be NULL. If we're at the end of the f-string, expression will be NULL (unless result == 1, see below). */ - int result = fstring_find_literal_and_expr(str, ofs, recurse_lvl, + int result = fstring_find_literal_and_expr(str, end, raw, recurse_lvl, &literal, &expression, c, n); if (result < 0) @@ -4925,16 +4849,14 @@ return -1; } - assert(*ofs <= PyUnicode_GET_LENGTH(str)); - /* If recurse_lvl is zero, then we must be at the end of the string. Otherwise, we must be at a right brace. */ - if (recurse_lvl == 0 && *ofs < PyUnicode_GET_LENGTH(str)) { + if (recurse_lvl == 0 && *str < end-1) { ast_error(c, n, "f-string: unexpected end of string"); return -1; } - if (recurse_lvl != 0 && PyUnicode_READ_CHAR(str, *ofs) != '}') { + if (recurse_lvl != 0 && **str != '}') { ast_error(c, n, "f-string: expecting '}'"); return -1; } @@ -4991,17 +4913,17 @@ return NULL; } -/* Given an f-string (with no 'f' or quotes) that's in str starting at - ofs, parse it into an expr_ty. Return NULL on error. Does not - decref str. */ +/* Given an f-string (with no 'f' or quotes) that's in *str and ends + at end, parse it into an expr_ty. Return NULL on error. Adjust + str to point past the parsed portion. */ static expr_ty -fstring_parse(PyObject *str, Py_ssize_t *ofs, int recurse_lvl, +fstring_parse(const char **str, const char *end, int raw, int recurse_lvl, struct compiling *c, const node *n) { FstringParser state; FstringParser_Init(&state); - if (FstringParser_ConcatFstring(&state, str, ofs, recurse_lvl, + if (FstringParser_ConcatFstring(&state, str, end, raw, recurse_lvl, c, n) < 0) { FstringParser_Dealloc(&state); return NULL; @@ -5012,19 +4934,25 @@ /* n is a Python string literal, including the bracketing quote characters, and r, b, u, &/or f prefixes (if any), and embedded - escape sequences (if any). parsestr parses it, and returns the + escape sequences (if any). parsestr parses it, and sets *result to decoded Python string object. If the string is an f-string, set - *fmode and return the unparsed string object. + *fstr and *fstrlen to the unparsed string object. Return 0 if no + errors occurred. */ -static PyObject * -parsestr(struct compiling *c, const node *n, int *bytesmode, int *fmode) +static int +parsestr(struct compiling *c, const node *n, int *bytesmode, int *rawmode, + PyObject **result, const char **fstr, Py_ssize_t *fstrlen) { size_t len; const char *s = STR(n); int quote = Py_CHARMASK(*s); - int rawmode = 0; + int fmode = 0; + *bytesmode = 0; + *rawmode = 0; + *result = NULL; + *fstr = NULL; if (Py_ISALPHA(quote)) { - while (!*bytesmode || !rawmode) { + while (!*bytesmode || !*rawmode) { if (quote == 'b' || quote == 'B') { quote = *++s; *bytesmode = 1; @@ -5034,24 +4962,24 @@ } else if (quote == 'r' || quote == 'R') { quote = *++s; - rawmode = 1; + *rawmode = 1; } else if (quote == 'f' || quote == 'F') { quote = *++s; - *fmode = 1; + fmode = 1; } else { break; } } } - if (*fmode && *bytesmode) { + if (fmode && *bytesmode) { PyErr_BadInternalCall(); - return NULL; + return -1; } if (quote != '\'' && quote != '\"') { PyErr_BadInternalCall(); - return NULL; + return -1; } /* Skip the leading quote char. */ s++; @@ -5059,12 +4987,12 @@ if (len > INT_MAX) { PyErr_SetString(PyExc_OverflowError, "string to parse is too long"); - return NULL; + return -1; } if (s[--len] != quote) { /* Last quote char must match the first. */ PyErr_BadInternalCall(); - return NULL; + return -1; } if (len >= 4 && s[0] == quote && s[1] == quote) { /* A triple quoted string. We've already skipped one quote at @@ -5075,21 +5003,21 @@ /* And check that the last two match. */ if (s[--len] != quote || s[--len] != quote) { PyErr_BadInternalCall(); - return NULL; + return -1; } } - /* Temporary hack: if this is an f-string, no backslashes are allowed. */ - /* See issue 27921. */ - if (*fmode && strchr(s, '\\') != NULL) { - /* Syntax error. At a later date fix this so it only checks for - backslashes within the braces. */ - ast_error(c, n, "backslashes not allowed in f-strings"); - return NULL; + if (fmode) { + /* Just return the bytes. The caller will parse the resulting + string. */ + *fstr = s; + *fstrlen = len; + return 0; } + /* Not an f-string. */ /* Avoid invoking escape decoding routines if possible. */ - rawmode = rawmode || strchr(s, '\\') == NULL; + *rawmode = *rawmode || strchr(s, '\\') == NULL; if (*bytesmode) { /* Disallow non-ASCII characters. */ const char *ch; @@ -5097,19 +5025,20 @@ if (Py_CHARMASK(*ch) >= 0x80) { ast_error(c, n, "bytes can only contain ASCII " "literal characters."); - return NULL; + return -1; } } - if (rawmode) - return PyBytes_FromStringAndSize(s, len); + if (*rawmode) + *result = PyBytes_FromStringAndSize(s, len); else - return PyBytes_DecodeEscape(s, len, NULL, /* ignored */ 0, NULL); + *result = PyBytes_DecodeEscape(s, len, NULL, /* ignored */ 0, NULL); } else { - if (rawmode) - return PyUnicode_DecodeUTF8Stateful(s, len, NULL, NULL); + if (*rawmode) + *result = PyUnicode_DecodeUTF8Stateful(s, len, NULL, NULL); else - return decode_unicode_with_escapes(c, s, len); + *result = decode_unicode_with_escapes(c, s, len); } + return *result == NULL ? -1 : 0; } /* Accepts a STRING+ atom, and produces an expr_ty node. Run through @@ -5131,13 +5060,15 @@ FstringParser_Init(&state); for (i = 0; i < NCH(n); i++) { - int this_bytesmode = 0; - int this_fmode = 0; + int this_bytesmode; + int this_rawmode; PyObject *s; + const char *fstr; + Py_ssize_t fstrlen = -1; /* Silence a compiler warning. */ REQ(CHILD(n, i), STRING); - s = parsestr(c, CHILD(n, i), &this_bytesmode, &this_fmode); - if (!s) + if (parsestr(c, CHILD(n, i), &this_bytesmode, &this_rawmode, &s, + &fstr, &fstrlen) != 0) goto error; /* Check that we're not mixing bytes with unicode. */ @@ -5148,29 +5079,36 @@ } bytesmode = this_bytesmode; - assert(bytesmode ? PyBytes_CheckExact(s) : PyUnicode_CheckExact(s)); - - if (bytesmode) { - /* For bytes, concat as we go. */ - if (i == 0) { - /* First time, just remember this value. */ - bytes_str = s; - } else { - PyBytes_ConcatAndDel(&bytes_str, s); - if (!bytes_str) - goto error; - } - } else if (this_fmode) { - /* This is an f-string. Concatenate and decref it. */ - Py_ssize_t ofs = 0; - int result = FstringParser_ConcatFstring(&state, s, &ofs, 0, c, n); - Py_DECREF(s); + if (fstr != NULL) { + int result; + assert(s == NULL && !bytesmode); + /* This is an f-string. Parse and concatenate it. */ + result = FstringParser_ConcatFstring(&state, &fstr, fstr+fstrlen, + this_rawmode, 0, c, n); if (result < 0) goto error; } else { - /* This is a regular string. Concatenate it. */ - if (FstringParser_ConcatAndDel(&state, s) < 0) - goto error; + assert(bytesmode ? PyBytes_CheckExact(s) : + PyUnicode_CheckExact(s)); + + /* A string or byte string. */ + assert(s != NULL && fstr == NULL); + if (bytesmode) { + /* For bytes, concat as we go. */ + if (i == 0) { + /* First time, just remember this value. */ + bytes_str = s; + } else { + PyBytes_ConcatAndDel(&bytes_str, s); + if (!bytes_str) + goto error; + } + } else { + assert(s != NULL && fstr == NULL); + /* This is a regular string. Concatenate it. */ + if (FstringParser_ConcatAndDel(&state, s) < 0) + goto error; + } } } if (bytesmode) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 22:31:34 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 10 Sep 2016 02:31:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_just_start_with_an_int_rat?= =?utf-8?q?her_than_casting?= Message-ID: <20160910023134.13637.37988.E4E95A35@psf.io> https://hg.python.org/cpython/rev/1dd0e5b8f8ef changeset: 103532:1dd0e5b8f8ef user: Benjamin Peterson date: Fri Sep 09 19:31:12 2016 -0700 summary: just start with an int rather than casting files: Python/ast.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -4326,7 +4326,7 @@ const char *expr_end; expr_ty simple_expression; expr_ty format_spec = NULL; /* Optional format specifier. */ - char conversion = -1; /* The conversion char. -1 if not specified. */ + int conversion = -1; /* The conversion char. -1 if not specified. */ /* 0 if we're not in a string, else the quote char we're trying to match (single or double quote). */ @@ -4502,7 +4502,7 @@ /* And now create the FormattedValue node that represents this entire expression with the conversion and format spec. */ - *expression = FormattedValue(simple_expression, (int)conversion, + *expression = FormattedValue(simple_expression, conversion, format_spec, LINENO(n), n->n_col_offset, c->c_arena); if (!*expression) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 22:49:10 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 10 Sep 2016 02:49:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_add_dtrace_inline_stubs?= Message-ID: <20160910024910.19368.26678.D70852D9@psf.io> https://hg.python.org/cpython/rev/63ae310b60ff changeset: 103533:63ae310b60ff user: Benjamin Peterson date: Fri Sep 09 19:48:47 2016 -0700 summary: add dtrace inline stubs files: Makefile.pre.in | 1 + Python/dtrace_stubs.c | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+), 0 deletions(-) diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -355,6 +355,7 @@ Python/compile.o \ Python/codecs.o \ Python/dynamic_annotations.o \ + Python/dtrace_stubs.o \ Python/errors.o \ Python/frozenmain.o \ Python/future.o \ diff --git a/Python/dtrace_stubs.c b/Python/dtrace_stubs.c new file mode 100644 --- /dev/null +++ b/Python/dtrace_stubs.c @@ -0,0 +1,24 @@ +#include +#include "pydtrace.h" + +#ifndef WITH_DTRACE +extern inline void PyDTrace_LINE(const char *arg0, const char *arg1, int arg2); +extern inline void PyDTrace_FUNCTION_ENTRY(const char *arg0, const char *arg1, int arg2); +extern inline void PyDTrace_FUNCTION_RETURN(const char *arg0, const char *arg1, int arg2); +extern inline void PyDTrace_GC_START(int arg0); +extern inline void PyDTrace_GC_DONE(int arg0); +extern inline void PyDTrace_INSTANCE_NEW_START(int arg0); +extern inline void PyDTrace_INSTANCE_NEW_DONE(int arg0); +extern inline void PyDTrace_INSTANCE_DELETE_START(int arg0); +extern inline void PyDTrace_INSTANCE_DELETE_DONE(int arg0); + +extern inline int PyDTrace_LINE_ENABLED(void); +extern inline int PyDTrace_FUNCTION_ENTRY_ENABLED(void); +extern inline int PyDTrace_FUNCTION_RETURN_ENABLED(void); +extern inline int PyDTrace_GC_START_ENABLED(void); +extern inline int PyDTrace_GC_DONE_ENABLED(void); +extern inline int PyDTrace_INSTANCE_NEW_START_ENABLED(void); +extern inline int PyDTrace_INSTANCE_NEW_DONE_ENABLED(void); +extern inline int PyDTrace_INSTANCE_DELETE_START_ENABLED(void); +extern inline int PyDTrace_INSTANCE_DELETE_DONE_ENABLED(void); +#endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 22:50:48 2016 From: python-checkins at python.org (lukasz.langa) Date: Sat, 10 Sep 2016 02:50:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327199=3A_TarFile_?= =?utf-8?q?expose_copyfileobj_bufsize_to_improve_throughput?= Message-ID: <20160910025048.1874.55562.A67E26D6@psf.io> https://hg.python.org/cpython/rev/0bac85e355b5 changeset: 103534:0bac85e355b5 user: ?ukasz Langa date: Fri Sep 09 19:48:14 2016 -0700 summary: Issue #27199: TarFile expose copyfileobj bufsize to improve throughput Patch by Jason Fried. files: Lib/tarfile.py | 33 ++++++++++++++++++--------------- Misc/NEWS | 3 +++ 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/Lib/tarfile.py b/Lib/tarfile.py --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -228,21 +228,21 @@ signed_chksum = 256 + sum(struct.unpack_from("148b8x356b", buf)) return unsigned_chksum, signed_chksum -def copyfileobj(src, dst, length=None, exception=OSError): +def copyfileobj(src, dst, length=None, exception=OSError, bufsize=None): """Copy length bytes from fileobj src to fileobj dst. If length is None, copy the entire content. """ + bufsize = bufsize or 16 * 1024 if length == 0: return if length is None: - shutil.copyfileobj(src, dst) + shutil.copyfileobj(src, dst, bufsize) return - BUFSIZE = 16 * 1024 - blocks, remainder = divmod(length, BUFSIZE) + blocks, remainder = divmod(length, bufsize) for b in range(blocks): - buf = src.read(BUFSIZE) - if len(buf) < BUFSIZE: + buf = src.read(bufsize) + if len(buf) < bufsize: raise exception("unexpected end of data") dst.write(buf) @@ -1403,7 +1403,8 @@ def __init__(self, name=None, mode="r", fileobj=None, format=None, tarinfo=None, dereference=None, ignore_zeros=None, encoding=None, - errors="surrogateescape", pax_headers=None, debug=None, errorlevel=None): + errors="surrogateescape", pax_headers=None, debug=None, + errorlevel=None, copybufsize=None): """Open an (uncompressed) tar archive `name'. `mode' is either 'r' to read from an existing archive, 'a' to append data to an existing file or 'w' to create a new file overwriting an existing one. `mode' @@ -1459,6 +1460,7 @@ self.errorlevel = errorlevel # Init datastructures. + self.copybufsize = copybufsize self.closed = False self.members = [] # list of members as TarInfo objects self._loaded = False # flag if all members have been read @@ -1558,7 +1560,7 @@ saved_pos = fileobj.tell() try: return func(name, "r", fileobj, **kwargs) - except (ReadError, CompressionError) as e: + except (ReadError, CompressionError): if fileobj is not None: fileobj.seek(saved_pos) continue @@ -1963,10 +1965,10 @@ buf = tarinfo.tobuf(self.format, self.encoding, self.errors) self.fileobj.write(buf) self.offset += len(buf) - + bufsize=self.copybufsize # If there's data to follow, append it. if fileobj is not None: - copyfileobj(fileobj, self.fileobj, tarinfo.size) + copyfileobj(fileobj, self.fileobj, tarinfo.size, bufsize=bufsize) blocks, remainder = divmod(tarinfo.size, BLOCKSIZE) if remainder > 0: self.fileobj.write(NUL * (BLOCKSIZE - remainder)) @@ -2148,15 +2150,16 @@ """ source = self.fileobj source.seek(tarinfo.offset_data) + bufsize = self.copybufsize with bltn_open(targetpath, "wb") as target: if tarinfo.sparse is not None: for offset, size in tarinfo.sparse: target.seek(offset) - copyfileobj(source, target, size, ReadError) + copyfileobj(source, target, size, ReadError, bufsize) target.seek(tarinfo.size) target.truncate() else: - copyfileobj(source, target, tarinfo.size, ReadError) + copyfileobj(source, target, tarinfo.size, ReadError, bufsize) def makeunknown(self, tarinfo, targetpath): """Make a file from a TarInfo object with an unknown type @@ -2235,7 +2238,7 @@ os.lchown(targetpath, u, g) else: os.chown(targetpath, u, g) - except OSError as e: + except OSError: raise ExtractError("could not change owner") def chmod(self, tarinfo, targetpath): @@ -2244,7 +2247,7 @@ if hasattr(os, 'chmod'): try: os.chmod(targetpath, tarinfo.mode) - except OSError as e: + except OSError: raise ExtractError("could not change mode") def utime(self, tarinfo, targetpath): @@ -2254,7 +2257,7 @@ return try: os.utime(targetpath, (tarinfo.mtime, tarinfo.mtime)) - except OSError as e: + except OSError: raise ExtractError("could not change modification time") #-------------------------------------------------------------------------- diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27199: In tarfile, expose copyfileobj bufsize to improve throughput. + Patch by Jason Fried. + - Issue #27948: In f-strings, only allow backslashes inside the braces (where the expressions are). This is a breaking change from the 3.6 alpha releases, where backslashes are allowed anywhere in an -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 22:53:03 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 10 Sep 2016 02:53:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_compile_dtrace_stubs?= Message-ID: <20160910025303.18872.81861.28FD3D43@psf.io> https://hg.python.org/cpython/rev/93ee4a615bd3 changeset: 103535:93ee4a615bd3 user: Benjamin Peterson date: Fri Sep 09 19:52:23 2016 -0700 summary: compile dtrace stubs files: PCbuild/pythoncore.vcxproj | 3 ++- PCbuild/pythoncore.vcxproj.filters | 5 ++++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -1,4 +1,4 @@ -? + @@ -356,6 +356,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -1,4 +1,4 @@ -? + @@ -848,6 +848,9 @@ Python + + Python + Python -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:06:58 2016 From: python-checkins at python.org (eric.smith) Date: Sat, 10 Sep 2016 03:06:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27080=3A_PEP_515=3A_?= =?utf-8?q?add_=27=5F=27_formatting_option=2E?= Message-ID: <20160910030658.1800.64390.77155F43@psf.io> https://hg.python.org/cpython/rev/99abb731ea7a changeset: 103536:99abb731ea7a user: Eric V. Smith date: Fri Sep 09 23:06:47 2016 -0400 summary: Issue 27080: PEP 515: add '_' formatting option. files: Doc/library/string.rst | 12 +++- Lib/test/test_long.py | 28 ++++++++++ Misc/NEWS | 3 + Python/formatter_unicode.c | 72 ++++++++++++++++++------- 4 files changed, 93 insertions(+), 22 deletions(-) diff --git a/Doc/library/string.rst b/Doc/library/string.rst --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -300,7 +300,7 @@ The general form of a *standard format specifier* is: .. productionlist:: sf - format_spec: [[`fill`]`align`][`sign`][#][0][`width`][,][.`precision`][`type`] + format_spec: [[`fill`]`align`][`sign`][#][0][`width`][,][_][.`precision`][`type`] fill: align: "<" | ">" | "=" | "^" sign: "+" | "-" | " " @@ -378,6 +378,16 @@ .. versionchanged:: 3.1 Added the ``','`` option (see also :pep:`378`). +The ``'_'`` option signals the use of an underscore for a thousands +separator for floating point presentation types and for integer +presentation type ``'d'``. For integer presentation types ``'b'``, +``'o'``, ``'x'``, and ``'X'``, underscores will be inserted every 4 +digits. For other presentation types, specifying this option is an +error. + +.. versionchanged:: 3.6 + Added the ``'_'`` option (see also :pep:`515`). + *width* is a decimal integer defining the minimum field width. If not specified, then the field width will be determined by the content. diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -621,6 +621,8 @@ def test__format__(self): self.assertEqual(format(123456789, 'd'), '123456789') self.assertEqual(format(123456789, 'd'), '123456789') + self.assertEqual(format(123456789, ','), '123,456,789') + self.assertEqual(format(123456789, '_'), '123_456_789') # sign and aligning are interdependent self.assertEqual(format(1, "-"), '1') @@ -649,8 +651,25 @@ self.assertEqual(format(int('be', 16), "X"), "BE") self.assertEqual(format(-int('be', 16), "x"), "-be") self.assertEqual(format(-int('be', 16), "X"), "-BE") + self.assertRaises(ValueError, format, 1234567890, ',x') + self.assertEqual(format(1234567890, '_x'), '4996_02d2') + self.assertEqual(format(1234567890, '_X'), '4996_02D2') # octal + self.assertEqual(format(3, "o"), "3") + self.assertEqual(format(-3, "o"), "-3") + self.assertEqual(format(1234, "o"), "2322") + self.assertEqual(format(-1234, "o"), "-2322") + self.assertEqual(format(1234, "-o"), "2322") + self.assertEqual(format(-1234, "-o"), "-2322") + self.assertEqual(format(1234, " o"), " 2322") + self.assertEqual(format(-1234, " o"), "-2322") + self.assertEqual(format(1234, "+o"), "+2322") + self.assertEqual(format(-1234, "+o"), "-2322") + self.assertRaises(ValueError, format, 1234567890, ',o') + self.assertEqual(format(1234567890, '_o'), '111_4540_1322') + + # binary self.assertEqual(format(3, "b"), "11") self.assertEqual(format(-3, "b"), "-11") self.assertEqual(format(1234, "b"), "10011010010") @@ -661,12 +680,21 @@ self.assertEqual(format(-1234, " b"), "-10011010010") self.assertEqual(format(1234, "+b"), "+10011010010") self.assertEqual(format(-1234, "+b"), "-10011010010") + self.assertRaises(ValueError, format, 1234567890, ',b') + self.assertEqual(format(12345, '_b'), '11_0000_0011_1001') # make sure these are errors self.assertRaises(ValueError, format, 3, "1.3") # precision disallowed + self.assertRaises(ValueError, format, 3, "_c") # underscore, + self.assertRaises(ValueError, format, 3, ",c") # comma, and self.assertRaises(ValueError, format, 3, "+c") # sign not allowed # with 'c' + self.assertRaisesRegex(ValueError, 'Cannot specify both', format, 3, '_,') + self.assertRaisesRegex(ValueError, 'Cannot specify both', format, 3, ',_') + self.assertRaisesRegex(ValueError, 'Cannot specify both', format, 3, '_,d') + self.assertRaisesRegex(ValueError, 'Cannot specify both', format, 3, ',_d') + # ensure that only int and float type specifiers work for format_spec in ([chr(x) for x in range(ord('a'), ord('z')+1)] + [chr(x) for x in range(ord('A'), ord('Z')+1)]): diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #27080: Implement formatting support for PEP 515. Initial patch + by Chris Angelico. + - Issue #27199: In tarfile, expose copyfileobj bufsize to improve throughput. Patch by Jason Fried. diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -32,14 +32,20 @@ { if (presentation_type > 32 && presentation_type < 128) PyErr_Format(PyExc_ValueError, - "Cannot specify ',' with '%c'.", + "Cannot specify ',' or '_' with '%c'.", (char)presentation_type); else PyErr_Format(PyExc_ValueError, - "Cannot specify ',' with '\\x%x'.", + "Cannot specify ',' or '_' with '\\x%x'.", (unsigned int)presentation_type); } +static void +invalid_comma_and_underscore() +{ + PyErr_Format(PyExc_ValueError, "Cannot specify both ',' and '_'."); +} + /* get_integer consumes 0 or more decimal digit characters from an input string, updates *result with the corresponding positive @@ -108,6 +114,12 @@ } } +/* Locale type codes. LT_NO_LOCALE must be zero. */ +#define LT_NO_LOCALE 0 +#define LT_DEFAULT_LOCALE 1 +#define LT_UNDERSCORE_LOCALE 2 +#define LT_UNDER_FOUR_LOCALE 3 +#define LT_CURRENT_LOCALE 4 typedef struct { Py_UCS4 fill_char; @@ -223,9 +235,22 @@ /* Comma signifies add thousands separators */ if (end-pos && READ_spec(pos) == ',') { - format->thousands_separators = 1; + format->thousands_separators = LT_DEFAULT_LOCALE; ++pos; } + /* Underscore signifies add thousands separators */ + if (end-pos && READ_spec(pos) == '_') { + if (format->thousands_separators != 0) { + invalid_comma_and_underscore(); + return 0; + } + format->thousands_separators = LT_UNDERSCORE_LOCALE; + ++pos; + } + if (end-pos && READ_spec(pos) == ',') { + invalid_comma_and_underscore(); + return 0; + } /* Parse field precision */ if (end-pos && READ_spec(pos) == '.') { @@ -275,6 +300,16 @@ case '\0': /* These are allowed. See PEP 378.*/ break; + case 'b': + case 'o': + case 'x': + case 'X': + /* Underscores are allowed in bin/oct/hex. See PEP 515. */ + if (format->thousands_separators == LT_UNDERSCORE_LOCALE) { + /* Every four digits, not every three, in bin/oct/hex. */ + format->thousands_separators = LT_UNDER_FOUR_LOCALE; + break; + } default: invalid_comma_type(format->type); return 0; @@ -351,11 +386,6 @@ /*********** common routines for numeric formatting *********************/ /************************************************************************/ -/* Locale type codes. */ -#define LT_CURRENT_LOCALE 0 -#define LT_DEFAULT_LOCALE 1 -#define LT_NO_LOCALE 2 - /* Locale info needed for formatting integers and the part of floats before and including the decimal. Note that locales only support 8-bit chars, not unicode. */ @@ -667,8 +697,8 @@ /* Find the decimal point character(s?), thousands_separator(s?), and grouping description, either for the current locale if type is - LT_CURRENT_LOCALE, a hard-coded locale if LT_DEFAULT_LOCALE, or - none if LT_NO_LOCALE. */ + LT_CURRENT_LOCALE, a hard-coded locale if LT_DEFAULT_LOCALE or + LT_UNDERSCORE_LOCALE/LT_UNDER_FOUR_LOCALE, or none if LT_NO_LOCALE. */ static int get_locale_info(int type, LocaleInfo *locale_info) { @@ -691,16 +721,22 @@ break; } case LT_DEFAULT_LOCALE: + case LT_UNDERSCORE_LOCALE: + case LT_UNDER_FOUR_LOCALE: locale_info->decimal_point = PyUnicode_FromOrdinal('.'); - locale_info->thousands_sep = PyUnicode_FromOrdinal(','); + locale_info->thousands_sep = PyUnicode_FromOrdinal( + type == LT_DEFAULT_LOCALE ? ',' : '_'); if (!locale_info->decimal_point || !locale_info->thousands_sep) { Py_XDECREF(locale_info->decimal_point); Py_XDECREF(locale_info->thousands_sep); return -1; } - locale_info->grouping = "\3"; /* Group every 3 characters. The + if (type != LT_UNDER_FOUR_LOCALE) + locale_info->grouping = "\3"; /* Group every 3 characters. The (implicit) trailing 0 means repeat infinitely. */ + else + locale_info->grouping = "\4"; /* Bin/oct/hex group every four. */ break; case LT_NO_LOCALE: locale_info->decimal_point = PyUnicode_FromOrdinal('.'); @@ -952,9 +988,7 @@ /* Determine the grouping, separator, and decimal point, if any. */ if (get_locale_info(format->type == 'n' ? LT_CURRENT_LOCALE : - (format->thousands_separators ? - LT_DEFAULT_LOCALE : - LT_NO_LOCALE), + format->thousands_separators, &locale) == -1) goto done; @@ -1099,9 +1133,7 @@ /* Determine the grouping, separator, and decimal point, if any. */ if (get_locale_info(format->type == 'n' ? LT_CURRENT_LOCALE : - (format->thousands_separators ? - LT_DEFAULT_LOCALE : - LT_NO_LOCALE), + format->thousands_separators, &locale) == -1) goto done; @@ -1277,9 +1309,7 @@ /* Determine the grouping, separator, and decimal point, if any. */ if (get_locale_info(format->type == 'n' ? LT_CURRENT_LOCALE : - (format->thousands_separators ? - LT_DEFAULT_LOCALE : - LT_NO_LOCALE), + format->thousands_separators, &locale) == -1) goto done; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:12:12 2016 From: python-checkins at python.org (eric.smith) Date: Sat, 10 Sep 2016 03:12:12 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbjogSW1wcm92ZWQgJywnIGFuZCAn?= =?utf-8?q?=5F=27_specification_in_format_mini-language=2E?= Message-ID: <20160910031212.87391.92194.336B94F2@psf.io> https://hg.python.org/cpython/rev/954e137b53d2 changeset: 103537:954e137b53d2 user: Eric V. Smith date: Fri Sep 09 23:12:02 2016 -0400 summary: Improved ',' and '_' specification in format mini-language. files: Doc/library/string.rst | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Doc/library/string.rst b/Doc/library/string.rst --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -300,11 +300,12 @@ The general form of a *standard format specifier* is: .. productionlist:: sf - format_spec: [[`fill`]`align`][`sign`][#][0][`width`][,][_][.`precision`][`type`] + format_spec: [[`fill`]`align`][`sign`][#][0][`width`][`option`][.`precision`][`type`] fill: align: "<" | ">" | "=" | "^" sign: "+" | "-" | " " width: `integer` + option: "_" | "," precision: `integer` type: "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:13:10 2016 From: python-checkins at python.org (eric.smith) Date: Sat, 10 Sep 2016 03:13:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Further_improved_=27=2C=27?= =?utf-8?q?_and_=27=5F=27_specification_in_format_mini-language=2E?= Message-ID: <20160910031310.48610.40109.5347C090@psf.io> https://hg.python.org/cpython/rev/0dca54cc03dd changeset: 103538:0dca54cc03dd user: Eric V. Smith date: Fri Sep 09 23:13:01 2016 -0400 summary: Further improved ',' and '_' specification in format mini-language. files: Doc/library/string.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/string.rst b/Doc/library/string.rst --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -300,12 +300,12 @@ The general form of a *standard format specifier* is: .. productionlist:: sf - format_spec: [[`fill`]`align`][`sign`][#][0][`width`][`option`][.`precision`][`type`] + format_spec: [[`fill`]`align`][`sign`][#][0][`width`][`grouping_option`][.`precision`][`type`] fill: align: "<" | ">" | "=" | "^" sign: "+" | "-" | " " width: `integer` - option: "_" | "," + grouping_option: "_" | "," precision: `integer` type: "b" | "c" | "d" | "e" | "E" | "f" | "F" | "g" | "G" | "n" | "o" | "s" | "x" | "X" | "%" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:15:54 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 10 Sep 2016 03:15:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_make_invalid=5Fcomma=5Fand?= =?utf-8?q?=5Funderscore_a_real_prototype?= Message-ID: <20160910031554.36195.16737.ED9DDCC8@psf.io> https://hg.python.org/cpython/rev/1e7b636b6009 changeset: 103539:1e7b636b6009 user: Benjamin Peterson date: Fri Sep 09 20:14:05 2016 -0700 summary: make invalid_comma_and_underscore a real prototype files: Python/formatter_unicode.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -41,7 +41,7 @@ } static void -invalid_comma_and_underscore() +invalid_comma_and_underscore(void) { PyErr_Format(PyExc_ValueError, "Cannot specify both ',' and '_'."); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:22:39 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 03:22:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_SystemError_in_compact?= =?utf-8?q?_dict?= Message-ID: <20160910032239.2591.84870.0B3068C5@psf.io> https://hg.python.org/cpython/rev/4a5b61b0d090 changeset: 103540:4a5b61b0d090 user: Victor Stinner date: Fri Sep 09 19:28:36 2016 -0700 summary: Fix SystemError in compact dict Issue #28040: Fix _PyDict_DelItem_KnownHash() and _PyDict_Pop(): convert splitted table to combined table to be able to delete the item. Write an unit test for the issue. Patch by INADA Naoki. files: Lib/test/test_dict.py | 69 +++++++++++++++++++++++++++++++ Objects/dictobject.c | 52 ++++++++++++++-------- 2 files changed, 102 insertions(+), 19 deletions(-) diff --git a/Lib/test/test_dict.py b/Lib/test/test_dict.py --- a/Lib/test/test_dict.py +++ b/Lib/test/test_dict.py @@ -4,6 +4,7 @@ import pickle import random import string +import sys import unittest import weakref from test import support @@ -838,6 +839,74 @@ pass self._tracked(MyDict()) + def make_shared_key_dict(self, n): + class C: + pass + + dicts = [] + for i in range(n): + a = C() + a.x, a.y, a.z = 1, 2, 3 + dicts.append(a.__dict__) + + return dicts + + @support.cpython_only + def test_splittable_del(self): + """split table must be combined when del d[k]""" + a, b = self.make_shared_key_dict(2) + + orig_size = sys.getsizeof(a) + + del a['y'] # split table is combined + with self.assertRaises(KeyError): + del a['y'] + + self.assertGreater(sys.getsizeof(a), orig_size) + self.assertEqual(list(a), ['x', 'z']) + self.assertEqual(list(b), ['x', 'y', 'z']) + + # Two dicts have different insertion order. + a['y'] = 42 + self.assertEqual(list(a), ['x', 'z', 'y']) + self.assertEqual(list(b), ['x', 'y', 'z']) + + @support.cpython_only + def test_splittable_pop(self): + """split table must be combined when d.pop(k)""" + a, b = self.make_shared_key_dict(2) + + orig_size = sys.getsizeof(a) + + a.pop('y') # split table is combined + with self.assertRaises(KeyError): + a.pop('y') + + self.assertGreater(sys.getsizeof(a), orig_size) + self.assertEqual(list(a), ['x', 'z']) + self.assertEqual(list(b), ['x', 'y', 'z']) + + # Two dicts have different insertion order. + a['y'] = 42 + self.assertEqual(list(a), ['x', 'z', 'y']) + self.assertEqual(list(b), ['x', 'y', 'z']) + + @support.cpython_only + def test_splittable_popitem(self): + """split table must be combined when d.popitem()""" + a, b = self.make_shared_key_dict(2) + + orig_size = sys.getsizeof(a) + + item = a.popitem() # split table is combined + self.assertEqual(item, ('z', 3)) + with self.assertRaises(KeyError): + del a['z'] + + self.assertGreater(sys.getsizeof(a), orig_size) + self.assertEqual(list(a), ['x', 'y']) + self.assertEqual(list(b), ['x', 'y', 'z']) + def test_iterator_pickling(self): for proto in range(pickle.HIGHEST_PROTOCOL + 1): data = {1:"a", 2:"b", 3:"c"} diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -1546,21 +1546,27 @@ return -1; } assert(dk_get_index(mp->ma_keys, hashpos) == ix); + + // Split table doesn't allow deletion. Combine it. + if (_PyDict_HasSplitTable(mp)) { + if (dictresize(mp, DK_SIZE(mp->ma_keys))) { + return -1; + } + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, &hashpos); + assert(ix >= 0); + } + old_value = *value_addr; + assert(old_value != NULL); *value_addr = NULL; mp->ma_used--; mp->ma_version_tag = DICT_NEXT_VERSION(); - if (_PyDict_HasSplitTable(mp)) { - mp->ma_keys->dk_usable = 0; - } - else { - ep = &DK_ENTRIES(mp->ma_keys)[ix]; - dk_set_index(mp->ma_keys, hashpos, DKIX_DUMMY); - ENSURE_ALLOWS_DELETIONS(mp); - old_key = ep->me_key; - ep->me_key = NULL; - Py_DECREF(old_key); - } + ep = &DK_ENTRIES(mp->ma_keys)[ix]; + dk_set_index(mp->ma_keys, hashpos, DKIX_DUMMY); + ENSURE_ALLOWS_DELETIONS(mp); + old_key = ep->me_key; + ep->me_key = NULL; + Py_DECREF(old_key); Py_DECREF(old_value); return 0; } @@ -1725,18 +1731,26 @@ return NULL; } + // Split table doesn't allow deletion. Combine it. + if (_PyDict_HasSplitTable(mp)) { + if (dictresize(mp, DK_SIZE(mp->ma_keys))) { + return NULL; + } + ix = (mp->ma_keys->dk_lookup)(mp, key, hash, &value_addr, &hashpos); + assert(ix >= 0); + } + old_value = *value_addr; + assert(old_value != NULL); *value_addr = NULL; mp->ma_used--; mp->ma_version_tag = DICT_NEXT_VERSION(); - if (!_PyDict_HasSplitTable(mp)) { - dk_set_index(mp->ma_keys, hashpos, DKIX_DUMMY); - ep = &DK_ENTRIES(mp->ma_keys)[ix]; - ENSURE_ALLOWS_DELETIONS(mp); - old_key = ep->me_key; - ep->me_key = NULL; - Py_DECREF(old_key); - } + dk_set_index(mp->ma_keys, hashpos, DKIX_DUMMY); + ep = &DK_ENTRIES(mp->ma_keys)[ix]; + ENSURE_ALLOWS_DELETIONS(mp); + old_key = ep->me_key; + ep->me_key = NULL; + Py_DECREF(old_key); return old_value; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:22:39 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 03:22:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_METH=5FFASTCALL_callin?= =?utf-8?q?g_convention?= Message-ID: <20160910032239.1874.53899.94A058DC@psf.io> https://hg.python.org/cpython/rev/86b0f5a900c7 changeset: 103541:86b0f5a900c7 user: Victor Stinner date: Fri Sep 09 17:40:22 2016 -0700 summary: Add METH_FASTCALL calling convention Issue #27810: Add a new calling convention for C functions: PyObject* func(PyObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames); Where args is a C array of positional arguments followed by values of keyword arguments. nargs is the number of positional arguments, kwnames are keys of keyword arguments. kwnames can be NULL. files: Include/abstract.h | 16 ++++++++ Include/methodobject.h | 4 ++ Objects/abstract.c | 56 ++++++++++++++++++++++++++++++ Objects/methodobject.c | 24 ++++++++++++ Python/getargs.c | 3 +- 5 files changed, 102 insertions(+), 1 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -277,6 +277,22 @@ PyObject *kwnames, PyObject *func); + /* Convert (args, nargs, kwargs) into a (stack, nargs, kwnames). + + Return a new stack which should be released by PyMem_Free(), or return + args unchanged if kwargs is NULL or an empty dictionary. + + The stack uses borrowed references. + + The type of keyword keys is not checked, these checks should be done + later (ex: _PyArg_ParseStack). */ + PyAPI_FUNC(PyObject **) _PyStack_UnpackDict( + PyObject **args, + Py_ssize_t nargs, + PyObject *kwargs, + PyObject **kwnames, + PyObject *func); + /* Call the callable object func with the "fast call" calling convention: args is a C array for positional arguments (nargs is the number of positional arguments), kwargs is a dictionary for keyword arguments. diff --git a/Include/methodobject.h b/Include/methodobject.h --- a/Include/methodobject.h +++ b/Include/methodobject.h @@ -16,6 +16,8 @@ #define PyCFunction_Check(op) (Py_TYPE(op) == &PyCFunction_Type) typedef PyObject *(*PyCFunction)(PyObject *, PyObject *); +typedef PyObject *(*_PyCFunctionFast) (PyObject *self, PyObject **args, + Py_ssize_t nargs, PyObject *kwnames); typedef PyObject *(*PyCFunctionWithKeywords)(PyObject *, PyObject *, PyObject *); typedef PyObject *(*PyNoArgsFunction)(PyObject *); @@ -83,6 +85,8 @@ #define METH_COEXIST 0x0040 +#define METH_FASTCALL 0x0080 + #ifndef Py_LIMITED_API typedef struct { PyObject_HEAD diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2403,6 +2403,62 @@ return kwdict; } +PyObject ** +_PyStack_UnpackDict(PyObject **args, Py_ssize_t nargs, PyObject *kwargs, + PyObject **p_kwnames, PyObject *func) +{ + PyObject **stack, **kwstack; + Py_ssize_t nkwargs; + Py_ssize_t pos, i; + PyObject *key, *value; + PyObject *kwnames; + + assert(nargs >= 0); + assert(kwargs == NULL || PyDict_CheckExact(kwargs)); + + nkwargs = (kwargs != NULL) ? PyDict_Size(kwargs) : 0; + if (!nkwargs) { + *p_kwnames = NULL; + return args; + } + + if ((size_t)nargs > PY_SSIZE_T_MAX / sizeof(stack[0]) - (size_t)nkwargs) { + PyErr_NoMemory(); + return NULL; + } + + stack = PyMem_Malloc((nargs + nkwargs) * sizeof(stack[0])); + if (stack == NULL) { + PyErr_NoMemory(); + return NULL; + } + + kwnames = PyTuple_New(nkwargs); + if (kwnames == NULL) { + PyMem_Free(stack); + return NULL; + } + + /* Copy position arguments (borrowed references) */ + Py_MEMCPY(stack, args, nargs * sizeof(stack[0])); + + kwstack = stack + nargs; + pos = i = 0; + /* This loop doesn't support lookup function mutating the dictionary + to change its size. It's a deliberate choice for speed, this function is + called in the performance critical hot code. */ + while (PyDict_Next(kwargs, &pos, &key, &value)) { + Py_INCREF(key); + PyTuple_SET_ITEM(kwnames, i, key); + /* The stack contains borrowed references */ + kwstack[i] = value; + i++; + } + + *p_kwnames = kwnames; + return stack; +} + PyObject * _PyObject_FastCallKeywords(PyObject *func, PyObject **stack, Py_ssize_t nargs, PyObject *kwnames) diff --git a/Objects/methodobject.c b/Objects/methodobject.c --- a/Objects/methodobject.c +++ b/Objects/methodobject.c @@ -97,6 +97,11 @@ if (flags == (METH_VARARGS | METH_KEYWORDS)) { res = (*(PyCFunctionWithKeywords)meth)(self, args, kwds); } + else if (flags == METH_FASTCALL) { + PyObject **stack = &PyTuple_GET_ITEM(args, 0); + Py_ssize_t nargs = PyTuple_GET_SIZE(args); + res = _PyCFunction_FastCallDict(func, stack, nargs, kwds); + } else { if (kwds != NULL && PyDict_Size(kwds) != 0) { PyErr_Format(PyExc_TypeError, "%.200s() takes no keyword arguments", @@ -232,6 +237,25 @@ break; } + case METH_FASTCALL: + { + PyObject **stack; + PyObject *kwnames; + _PyCFunctionFast fastmeth = (_PyCFunctionFast)meth; + + stack = _PyStack_UnpackDict(args, nargs, kwargs, &kwnames, func_obj); + if (stack == NULL) { + return NULL; + } + + result = (*fastmeth) (self, stack, nargs, kwnames); + if (stack != args) { + PyMem_Free(stack); + } + Py_XDECREF(kwnames); + break; + } + default: PyErr_SetString(PyExc_SystemError, "Bad call flags in PyCFunction_Call. " diff --git a/Python/getargs.c b/Python/getargs.c --- a/Python/getargs.c +++ b/Python/getargs.c @@ -1992,8 +1992,9 @@ return cleanreturn(0, &freelist); } } - else if (i < nargs) + else if (i < nargs) { current_arg = PyTuple_GET_ITEM(args, i); + } if (current_arg) { msg = convertitem(current_arg, &format, p_va, flags, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:22:40 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 03:22:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Emit_METH=5FFASTCALL_code_?= =?utf-8?q?in_Argument_Clinic?= Message-ID: <20160910032239.59651.47806.FF46E1CD@psf.io> https://hg.python.org/cpython/rev/633f850038c3 changeset: 103542:633f850038c3 user: Victor Stinner date: Fri Sep 09 17:40:38 2016 -0700 summary: Emit METH_FASTCALL code in Argument Clinic Issue #27810: * Modify vgetargskeywordsfast() to work on a C array of PyObject* rather than working on a tuple directly. * Add _PyArg_ParseStack() * Argument Clinic now emits code using the new METH_FASTCALL calling convention files: Include/modsupport.h | 3 + Python/getargs.c | 184 ++++++++++++++++++++++++---- Tools/clinic/clinic.py | 18 ++ 3 files changed, 178 insertions(+), 27 deletions(-) diff --git a/Include/modsupport.h b/Include/modsupport.h --- a/Include/modsupport.h +++ b/Include/modsupport.h @@ -58,10 +58,13 @@ } _PyArg_Parser; #ifdef PY_SSIZE_T_CLEAN #define _PyArg_ParseTupleAndKeywordsFast _PyArg_ParseTupleAndKeywordsFast_SizeT +#define _PyArg_ParseStack _PyArg_ParseStack_SizeT #define _PyArg_VaParseTupleAndKeywordsFast _PyArg_VaParseTupleAndKeywordsFast_SizeT #endif PyAPI_FUNC(int) _PyArg_ParseTupleAndKeywordsFast(PyObject *, PyObject *, struct _PyArg_Parser *, ...); +PyAPI_FUNC(int) _PyArg_ParseStack(PyObject **args, Py_ssize_t nargs, PyObject *kwnames, + struct _PyArg_Parser *, ...); PyAPI_FUNC(int) _PyArg_VaParseTupleAndKeywordsFast(PyObject *, PyObject *, struct _PyArg_Parser *, va_list); void _PyArg_Fini(void); diff --git a/Python/getargs.c b/Python/getargs.c --- a/Python/getargs.c +++ b/Python/getargs.c @@ -79,6 +79,10 @@ const char *, char **, va_list *, int); static int vgetargskeywordsfast(PyObject *, PyObject *, struct _PyArg_Parser *, va_list *, int); +static int vgetargskeywordsfast_impl(PyObject **args, Py_ssize_t nargs, + PyObject *keywords, PyObject *kwnames, + struct _PyArg_Parser *parser, + va_list *p_va, int flags); static const char *skipitem(const char **, va_list *, int); int @@ -1469,6 +1473,46 @@ return retval; } +int +_PyArg_ParseStack(PyObject **args, Py_ssize_t nargs, PyObject *kwnames, + struct _PyArg_Parser *parser, ...) +{ + int retval; + va_list va; + + if ((kwnames != NULL && !PyTuple_Check(kwnames)) || + parser == NULL) + { + PyErr_BadInternalCall(); + return 0; + } + + va_start(va, parser); + retval = vgetargskeywordsfast_impl(args, nargs, NULL, kwnames, parser, &va, 0); + va_end(va); + return retval; +} + +int +_PyArg_ParseStack_SizeT(PyObject **args, Py_ssize_t nargs, PyObject *kwnames, + struct _PyArg_Parser *parser, ...) +{ + int retval; + va_list va; + + if ((kwnames != NULL && !PyTuple_Check(kwnames)) || + parser == NULL) + { + PyErr_BadInternalCall(); + return 0; + } + + va_start(va, parser); + retval = vgetargskeywordsfast_impl(args, nargs, NULL, kwnames, parser, &va, FLAG_SIZE_T); + va_end(va); + return retval; +} + int _PyArg_VaParseTupleAndKeywordsFast(PyObject *args, PyObject *keywords, @@ -1899,10 +1943,37 @@ Py_CLEAR(parser->kwtuple); } +static PyObject* +find_keyword(PyObject *kwnames, PyObject **kwstack, PyObject *key) +{ + Py_ssize_t i, nkwargs; + + nkwargs = PyTuple_GET_SIZE(kwnames); + for (i=0; i < nkwargs; i++) { + PyObject *kwname = PyTuple_GET_ITEM(kwnames, i); + + /* ptr==ptr should match in most cases since keyword keys + should be interned strings */ + if (kwname == key) { + return kwstack[i]; + } + if (!PyUnicode_Check(kwname)) { + /* ignore non-string keyword keys: + an error will be raised above */ + continue; + } + if (_PyUnicode_EQ(kwname, key)) { + return kwstack[i]; + } + } + return NULL; +} + static int -vgetargskeywordsfast(PyObject *args, PyObject *keywords, - struct _PyArg_Parser *parser, - va_list *p_va, int flags) +vgetargskeywordsfast_impl(PyObject **args, Py_ssize_t nargs, + PyObject *keywords, PyObject *kwnames, + struct _PyArg_Parser *parser, + va_list *p_va, int flags) { PyObject *kwtuple; char msgbuf[512]; @@ -1911,17 +1982,20 @@ const char *msg; PyObject *keyword; int i, pos, len; - Py_ssize_t nargs, nkeywords; + Py_ssize_t nkeywords; PyObject *current_arg; freelistentry_t static_entries[STATIC_FREELIST_ENTRIES]; freelist_t freelist; + PyObject **kwstack = NULL; freelist.entries = static_entries; freelist.first_available = 0; freelist.entries_malloced = 0; - assert(args != NULL && PyTuple_Check(args)); assert(keywords == NULL || PyDict_Check(keywords)); + assert(kwnames == NULL || PyTuple_Check(kwnames)); + assert((keywords != NULL || kwnames != NULL) + || (keywords == NULL && kwnames == NULL)); assert(parser != NULL); assert(p_va != NULL); @@ -1942,8 +2016,16 @@ freelist.entries_malloced = 1; } - nargs = PyTuple_GET_SIZE(args); - nkeywords = (keywords == NULL) ? 0 : PyDict_Size(keywords); + if (keywords != NULL) { + nkeywords = PyDict_Size(keywords); + } + else if (kwnames != NULL) { + nkeywords = PyTuple_GET_SIZE(kwnames); + kwstack = args + nargs; + } + else { + nkeywords = 0; + } if (nargs + nkeywords > len) { PyErr_Format(PyExc_TypeError, "%s%s takes at most %d argument%s (%zd given)", @@ -1976,9 +2058,14 @@ current_arg = NULL; if (nkeywords && i >= pos) { - current_arg = PyDict_GetItem(keywords, keyword); - if (!current_arg && PyErr_Occurred()) { - return cleanreturn(0, &freelist); + if (keywords != NULL) { + current_arg = PyDict_GetItem(keywords, keyword); + if (!current_arg && PyErr_Occurred()) { + return cleanreturn(0, &freelist); + } + } + else { + current_arg = find_keyword(kwnames, kwstack, keyword); } } if (current_arg) { @@ -1993,7 +2080,7 @@ } } else if (i < nargs) { - current_arg = PyTuple_GET_ITEM(args, i); + current_arg = args[i]; } if (current_arg) { @@ -2040,24 +2127,52 @@ /* make sure there are no extraneous keyword arguments */ if (nkeywords > 0) { - PyObject *key, *value; - Py_ssize_t pos = 0; - while (PyDict_Next(keywords, &pos, &key, &value)) { - int match; - if (!PyUnicode_Check(key)) { - PyErr_SetString(PyExc_TypeError, - "keywords must be strings"); - return cleanreturn(0, &freelist); + if (keywords != NULL) { + PyObject *key, *value; + Py_ssize_t pos = 0; + while (PyDict_Next(keywords, &pos, &key, &value)) { + int match; + if (!PyUnicode_Check(key)) { + PyErr_SetString(PyExc_TypeError, + "keywords must be strings"); + return cleanreturn(0, &freelist); + } + match = PySequence_Contains(kwtuple, key); + if (match <= 0) { + if (!match) { + PyErr_Format(PyExc_TypeError, + "'%U' is an invalid keyword " + "argument for this function", + key); + } + return cleanreturn(0, &freelist); + } } - match = PySequence_Contains(kwtuple, key); - if (match <= 0) { - if (!match) { - PyErr_Format(PyExc_TypeError, - "'%U' is an invalid keyword " - "argument for this function", - key); + } + else { + Py_ssize_t j, nkwargs; + + nkwargs = PyTuple_GET_SIZE(kwnames); + for (j=0; j < nkwargs; j++) { + PyObject *key = PyTuple_GET_ITEM(kwnames, j); + int match; + + if (!PyUnicode_Check(key)) { + PyErr_SetString(PyExc_TypeError, + "keywords must be strings"); + return cleanreturn(0, &freelist); } - return cleanreturn(0, &freelist); + + match = PySequence_Contains(kwtuple, key); + if (match <= 0) { + if (!match) { + PyErr_Format(PyExc_TypeError, + "'%U' is an invalid keyword " + "argument for this function", + key); + } + return cleanreturn(0, &freelist); + } } } } @@ -2065,6 +2180,21 @@ return cleanreturn(1, &freelist); } +static int +vgetargskeywordsfast(PyObject *args, PyObject *keywords, + struct _PyArg_Parser *parser, va_list *p_va, int flags) +{ + PyObject **stack; + Py_ssize_t nargs; + + assert(args != NULL && PyTuple_Check(args)); + + stack = &PyTuple_GET_ITEM(args, 0); + nargs = PyTuple_GET_SIZE(args); + return vgetargskeywordsfast_impl(stack, nargs, keywords, NULL, + parser, p_va, flags); +} + static const char * skipitem(const char **p_format, va_list *p_va, int flags) diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -705,6 +705,11 @@ {c_basename}({self_type}{self_name}, PyObject *args, PyObject *kwargs) """) + parser_prototype_fastcall = normalize_snippet(""" + static PyObject * + {c_basename}({self_type}{self_name}, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) + """) + parser_prototype_varargs = normalize_snippet(""" static PyObject * {c_basename}({self_type}{self_name}, PyObject *args) @@ -845,6 +850,19 @@ }} """, indent=4)) + elif not new_or_init: + flags = "METH_FASTCALL" + + parser_prototype = parser_prototype_fastcall + + body = normalize_snippet(""" + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, + {parse_arguments})) {{ + goto exit; + }} + """, indent=4) + parser_definition = parser_body(parser_prototype, body) + parser_definition = insert_keywords(parser_definition) else: # positional-or-keyword arguments flags = "METH_VARARGS|METH_KEYWORDS" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:22:40 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 03:22:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327810=3A_Rerun_Ar?= =?utf-8?q?gument_Clinic_on_all_modules?= Message-ID: <20160910032240.1920.82953.B306E4A0@psf.io> https://hg.python.org/cpython/rev/97a68adbe826 changeset: 103543:97a68adbe826 user: Victor Stinner date: Fri Sep 09 20:00:13 2016 -0700 summary: Issue #27810: Rerun Argument Clinic on all modules files: Modules/clinic/_bz2module.c.h | 8 +- Modules/clinic/_codecsmodule.c.h | 14 +- Modules/clinic/_datetimemodule.c.h | 8 +- Modules/clinic/_elementtree.c.h | 38 +- Modules/clinic/_hashopenssl.c.h | 8 +- Modules/clinic/_lzmamodule.c.h | 8 +- Modules/clinic/_pickle.c.h | 26 +- Modules/clinic/_sre.c.h | 92 ++-- Modules/clinic/_ssl.c.h | 50 +- Modules/clinic/_winapi.c.h | 20 +- Modules/clinic/binascii.c.h | 20 +- Modules/clinic/cmathmodule.c.h | 8 +- Modules/clinic/grpmodule.c.h | 14 +- Modules/clinic/md5module.c.h | 8 +- Modules/clinic/posixmodule.c.h | 383 ++++++++-------- Modules/clinic/pyexpat.c.h | 8 +- Modules/clinic/sha1module.c.h | 8 +- Modules/clinic/sha256module.c.h | 14 +- Modules/clinic/sha512module.c.h | 14 +- Modules/clinic/zlibmodule.c.h | 32 +- Objects/clinic/bytearrayobject.c.h | 32 +- Objects/clinic/bytesobject.c.h | 32 +- PC/clinic/winreg.c.h | 32 +- PC/clinic/winsound.c.h | 20 +- Python/clinic/bltinmodule.c.h | 8 +- 25 files changed, 453 insertions(+), 452 deletions(-) diff --git a/Modules/clinic/_bz2module.c.h b/Modules/clinic/_bz2module.c.h --- a/Modules/clinic/_bz2module.c.h +++ b/Modules/clinic/_bz2module.c.h @@ -115,14 +115,14 @@ "the unused_data attribute."); #define _BZ2_BZ2DECOMPRESSOR_DECOMPRESS_METHODDEF \ - {"decompress", (PyCFunction)_bz2_BZ2Decompressor_decompress, METH_VARARGS|METH_KEYWORDS, _bz2_BZ2Decompressor_decompress__doc__}, + {"decompress", (PyCFunction)_bz2_BZ2Decompressor_decompress, METH_FASTCALL, _bz2_BZ2Decompressor_decompress__doc__}, static PyObject * _bz2_BZ2Decompressor_decompress_impl(BZ2Decompressor *self, Py_buffer *data, Py_ssize_t max_length); static PyObject * -_bz2_BZ2Decompressor_decompress(BZ2Decompressor *self, PyObject *args, PyObject *kwargs) +_bz2_BZ2Decompressor_decompress(BZ2Decompressor *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"data", "max_length", NULL}; @@ -130,7 +130,7 @@ Py_buffer data = {NULL, NULL}; Py_ssize_t max_length = -1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &data, &max_length)) { goto exit; } @@ -174,4 +174,4 @@ exit: return return_value; } -/*[clinic end generated code: output=40e5ef049f9e719b input=a9049054013a1b77]*/ +/*[clinic end generated code: output=7e57af0b368d3e55 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_codecsmodule.c.h b/Modules/clinic/_codecsmodule.c.h --- a/Modules/clinic/_codecsmodule.c.h +++ b/Modules/clinic/_codecsmodule.c.h @@ -55,14 +55,14 @@ "codecs.register_error that can handle ValueErrors."); #define _CODECS_ENCODE_METHODDEF \ - {"encode", (PyCFunction)_codecs_encode, METH_VARARGS|METH_KEYWORDS, _codecs_encode__doc__}, + {"encode", (PyCFunction)_codecs_encode, METH_FASTCALL, _codecs_encode__doc__}, static PyObject * _codecs_encode_impl(PyObject *module, PyObject *obj, const char *encoding, const char *errors); static PyObject * -_codecs_encode(PyObject *module, PyObject *args, PyObject *kwargs) +_codecs_encode(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"obj", "encoding", "errors", NULL}; @@ -71,7 +71,7 @@ const char *encoding = NULL; const char *errors = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &obj, &encoding, &errors)) { goto exit; } @@ -94,14 +94,14 @@ "codecs.register_error that can handle ValueErrors."); #define _CODECS_DECODE_METHODDEF \ - {"decode", (PyCFunction)_codecs_decode, METH_VARARGS|METH_KEYWORDS, _codecs_decode__doc__}, + {"decode", (PyCFunction)_codecs_decode, METH_FASTCALL, _codecs_decode__doc__}, static PyObject * _codecs_decode_impl(PyObject *module, PyObject *obj, const char *encoding, const char *errors); static PyObject * -_codecs_decode(PyObject *module, PyObject *args, PyObject *kwargs) +_codecs_decode(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"obj", "encoding", "errors", NULL}; @@ -110,7 +110,7 @@ const char *encoding = NULL; const char *errors = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &obj, &encoding, &errors)) { goto exit; } @@ -1536,4 +1536,4 @@ #ifndef _CODECS_CODE_PAGE_ENCODE_METHODDEF #define _CODECS_CODE_PAGE_ENCODE_METHODDEF #endif /* !defined(_CODECS_CODE_PAGE_ENCODE_METHODDEF) */ -/*[clinic end generated code: output=ebe313ab417b17bb input=a9049054013a1b77]*/ +/*[clinic end generated code: output=6d6afcabde10ed79 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_datetimemodule.c.h b/Modules/clinic/_datetimemodule.c.h --- a/Modules/clinic/_datetimemodule.c.h +++ b/Modules/clinic/_datetimemodule.c.h @@ -14,20 +14,20 @@ "If no tz is specified, uses local timezone."); #define DATETIME_DATETIME_NOW_METHODDEF \ - {"now", (PyCFunction)datetime_datetime_now, METH_VARARGS|METH_KEYWORDS|METH_CLASS, datetime_datetime_now__doc__}, + {"now", (PyCFunction)datetime_datetime_now, METH_FASTCALL|METH_CLASS, datetime_datetime_now__doc__}, static PyObject * datetime_datetime_now_impl(PyTypeObject *type, PyObject *tz); static PyObject * -datetime_datetime_now(PyTypeObject *type, PyObject *args, PyObject *kwargs) +datetime_datetime_now(PyTypeObject *type, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"tz", NULL}; static _PyArg_Parser _parser = {"|O:now", _keywords, 0}; PyObject *tz = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &tz)) { goto exit; } @@ -36,4 +36,4 @@ exit: return return_value; } -/*[clinic end generated code: output=61f85af5637df8b5 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=8aaac0705add61ca input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_elementtree.c.h b/Modules/clinic/_elementtree.c.h --- a/Modules/clinic/_elementtree.c.h +++ b/Modules/clinic/_elementtree.c.h @@ -136,14 +136,14 @@ "\n"); #define _ELEMENTTREE_ELEMENT_FIND_METHODDEF \ - {"find", (PyCFunction)_elementtree_Element_find, METH_VARARGS|METH_KEYWORDS, _elementtree_Element_find__doc__}, + {"find", (PyCFunction)_elementtree_Element_find, METH_FASTCALL, _elementtree_Element_find__doc__}, static PyObject * _elementtree_Element_find_impl(ElementObject *self, PyObject *path, PyObject *namespaces); static PyObject * -_elementtree_Element_find(ElementObject *self, PyObject *args, PyObject *kwargs) +_elementtree_Element_find(ElementObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "namespaces", NULL}; @@ -151,7 +151,7 @@ PyObject *path; PyObject *namespaces = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &path, &namespaces)) { goto exit; } @@ -167,7 +167,7 @@ "\n"); #define _ELEMENTTREE_ELEMENT_FINDTEXT_METHODDEF \ - {"findtext", (PyCFunction)_elementtree_Element_findtext, METH_VARARGS|METH_KEYWORDS, _elementtree_Element_findtext__doc__}, + {"findtext", (PyCFunction)_elementtree_Element_findtext, METH_FASTCALL, _elementtree_Element_findtext__doc__}, static PyObject * _elementtree_Element_findtext_impl(ElementObject *self, PyObject *path, @@ -175,7 +175,7 @@ PyObject *namespaces); static PyObject * -_elementtree_Element_findtext(ElementObject *self, PyObject *args, PyObject *kwargs) +_elementtree_Element_findtext(ElementObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "default", "namespaces", NULL}; @@ -184,7 +184,7 @@ PyObject *default_value = Py_None; PyObject *namespaces = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &path, &default_value, &namespaces)) { goto exit; } @@ -200,14 +200,14 @@ "\n"); #define _ELEMENTTREE_ELEMENT_FINDALL_METHODDEF \ - {"findall", (PyCFunction)_elementtree_Element_findall, METH_VARARGS|METH_KEYWORDS, _elementtree_Element_findall__doc__}, + {"findall", (PyCFunction)_elementtree_Element_findall, METH_FASTCALL, _elementtree_Element_findall__doc__}, static PyObject * _elementtree_Element_findall_impl(ElementObject *self, PyObject *path, PyObject *namespaces); static PyObject * -_elementtree_Element_findall(ElementObject *self, PyObject *args, PyObject *kwargs) +_elementtree_Element_findall(ElementObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "namespaces", NULL}; @@ -215,7 +215,7 @@ PyObject *path; PyObject *namespaces = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &path, &namespaces)) { goto exit; } @@ -231,14 +231,14 @@ "\n"); #define _ELEMENTTREE_ELEMENT_ITERFIND_METHODDEF \ - {"iterfind", (PyCFunction)_elementtree_Element_iterfind, METH_VARARGS|METH_KEYWORDS, _elementtree_Element_iterfind__doc__}, + {"iterfind", (PyCFunction)_elementtree_Element_iterfind, METH_FASTCALL, _elementtree_Element_iterfind__doc__}, static PyObject * _elementtree_Element_iterfind_impl(ElementObject *self, PyObject *path, PyObject *namespaces); static PyObject * -_elementtree_Element_iterfind(ElementObject *self, PyObject *args, PyObject *kwargs) +_elementtree_Element_iterfind(ElementObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "namespaces", NULL}; @@ -246,7 +246,7 @@ PyObject *path; PyObject *namespaces = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &path, &namespaces)) { goto exit; } @@ -262,14 +262,14 @@ "\n"); #define _ELEMENTTREE_ELEMENT_GET_METHODDEF \ - {"get", (PyCFunction)_elementtree_Element_get, METH_VARARGS|METH_KEYWORDS, _elementtree_Element_get__doc__}, + {"get", (PyCFunction)_elementtree_Element_get, METH_FASTCALL, _elementtree_Element_get__doc__}, static PyObject * _elementtree_Element_get_impl(ElementObject *self, PyObject *key, PyObject *default_value); static PyObject * -_elementtree_Element_get(ElementObject *self, PyObject *args, PyObject *kwargs) +_elementtree_Element_get(ElementObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"key", "default", NULL}; @@ -277,7 +277,7 @@ PyObject *key; PyObject *default_value = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &key, &default_value)) { goto exit; } @@ -310,20 +310,20 @@ "\n"); #define _ELEMENTTREE_ELEMENT_ITER_METHODDEF \ - {"iter", (PyCFunction)_elementtree_Element_iter, METH_VARARGS|METH_KEYWORDS, _elementtree_Element_iter__doc__}, + {"iter", (PyCFunction)_elementtree_Element_iter, METH_FASTCALL, _elementtree_Element_iter__doc__}, static PyObject * _elementtree_Element_iter_impl(ElementObject *self, PyObject *tag); static PyObject * -_elementtree_Element_iter(ElementObject *self, PyObject *args, PyObject *kwargs) +_elementtree_Element_iter(ElementObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"tag", NULL}; static _PyArg_Parser _parser = {"|O:iter", _keywords, 0}; PyObject *tag = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &tag)) { goto exit; } @@ -702,4 +702,4 @@ exit: return return_value; } -/*[clinic end generated code: output=4c5e94c28a009ce6 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=b4a571a98ced3163 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_hashopenssl.c.h b/Modules/clinic/_hashopenssl.c.h --- a/Modules/clinic/_hashopenssl.c.h +++ b/Modules/clinic/_hashopenssl.c.h @@ -12,7 +12,7 @@ "scrypt password-based key derivation function."); #define _HASHLIB_SCRYPT_METHODDEF \ - {"scrypt", (PyCFunction)_hashlib_scrypt, METH_VARARGS|METH_KEYWORDS, _hashlib_scrypt__doc__}, + {"scrypt", (PyCFunction)_hashlib_scrypt, METH_FASTCALL, _hashlib_scrypt__doc__}, static PyObject * _hashlib_scrypt_impl(PyObject *module, Py_buffer *password, Py_buffer *salt, @@ -20,7 +20,7 @@ long maxmem, long dklen); static PyObject * -_hashlib_scrypt(PyObject *module, PyObject *args, PyObject *kwargs) +_hashlib_scrypt(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"password", "salt", "n", "r", "p", "maxmem", "dklen", NULL}; @@ -33,7 +33,7 @@ long maxmem = 0; long dklen = 64; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &password, &salt, &PyLong_Type, &n_obj, &PyLong_Type, &r_obj, &PyLong_Type, &p_obj, &maxmem, &dklen)) { goto exit; } @@ -57,4 +57,4 @@ #ifndef _HASHLIB_SCRYPT_METHODDEF #define _HASHLIB_SCRYPT_METHODDEF #endif /* !defined(_HASHLIB_SCRYPT_METHODDEF) */ -/*[clinic end generated code: output=8c5386789f77430a input=a9049054013a1b77]*/ +/*[clinic end generated code: output=118cd7036fa0fb52 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_lzmamodule.c.h b/Modules/clinic/_lzmamodule.c.h --- a/Modules/clinic/_lzmamodule.c.h +++ b/Modules/clinic/_lzmamodule.c.h @@ -81,14 +81,14 @@ "the unused_data attribute."); #define _LZMA_LZMADECOMPRESSOR_DECOMPRESS_METHODDEF \ - {"decompress", (PyCFunction)_lzma_LZMADecompressor_decompress, METH_VARARGS|METH_KEYWORDS, _lzma_LZMADecompressor_decompress__doc__}, + {"decompress", (PyCFunction)_lzma_LZMADecompressor_decompress, METH_FASTCALL, _lzma_LZMADecompressor_decompress__doc__}, static PyObject * _lzma_LZMADecompressor_decompress_impl(Decompressor *self, Py_buffer *data, Py_ssize_t max_length); static PyObject * -_lzma_LZMADecompressor_decompress(Decompressor *self, PyObject *args, PyObject *kwargs) +_lzma_LZMADecompressor_decompress(Decompressor *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"data", "max_length", NULL}; @@ -96,7 +96,7 @@ Py_buffer data = {NULL, NULL}; Py_ssize_t max_length = -1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &data, &max_length)) { goto exit; } @@ -256,4 +256,4 @@ return return_value; } -/*[clinic end generated code: output=9434583fe111c771 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=f27abae460122706 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_pickle.c.h b/Modules/clinic/_pickle.c.h --- a/Modules/clinic/_pickle.c.h +++ b/Modules/clinic/_pickle.c.h @@ -384,14 +384,14 @@ "2, so that the pickle data stream is readable with Python 2."); #define _PICKLE_DUMP_METHODDEF \ - {"dump", (PyCFunction)_pickle_dump, METH_VARARGS|METH_KEYWORDS, _pickle_dump__doc__}, + {"dump", (PyCFunction)_pickle_dump, METH_FASTCALL, _pickle_dump__doc__}, static PyObject * _pickle_dump_impl(PyObject *module, PyObject *obj, PyObject *file, PyObject *protocol, int fix_imports); static PyObject * -_pickle_dump(PyObject *module, PyObject *args, PyObject *kwargs) +_pickle_dump(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"obj", "file", "protocol", "fix_imports", NULL}; @@ -401,7 +401,7 @@ PyObject *protocol = NULL; int fix_imports = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &obj, &file, &protocol, &fix_imports)) { goto exit; } @@ -430,14 +430,14 @@ "Python 2, so that the pickle data stream is readable with Python 2."); #define _PICKLE_DUMPS_METHODDEF \ - {"dumps", (PyCFunction)_pickle_dumps, METH_VARARGS|METH_KEYWORDS, _pickle_dumps__doc__}, + {"dumps", (PyCFunction)_pickle_dumps, METH_FASTCALL, _pickle_dumps__doc__}, static PyObject * _pickle_dumps_impl(PyObject *module, PyObject *obj, PyObject *protocol, int fix_imports); static PyObject * -_pickle_dumps(PyObject *module, PyObject *args, PyObject *kwargs) +_pickle_dumps(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"obj", "protocol", "fix_imports", NULL}; @@ -446,7 +446,7 @@ PyObject *protocol = NULL; int fix_imports = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &obj, &protocol, &fix_imports)) { goto exit; } @@ -486,14 +486,14 @@ "string instances as bytes objects."); #define _PICKLE_LOAD_METHODDEF \ - {"load", (PyCFunction)_pickle_load, METH_VARARGS|METH_KEYWORDS, _pickle_load__doc__}, + {"load", (PyCFunction)_pickle_load, METH_FASTCALL, _pickle_load__doc__}, static PyObject * _pickle_load_impl(PyObject *module, PyObject *file, int fix_imports, const char *encoding, const char *errors); static PyObject * -_pickle_load(PyObject *module, PyObject *args, PyObject *kwargs) +_pickle_load(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"file", "fix_imports", "encoding", "errors", NULL}; @@ -503,7 +503,7 @@ const char *encoding = "ASCII"; const char *errors = "strict"; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &file, &fix_imports, &encoding, &errors)) { goto exit; } @@ -534,14 +534,14 @@ "string instances as bytes objects."); #define _PICKLE_LOADS_METHODDEF \ - {"loads", (PyCFunction)_pickle_loads, METH_VARARGS|METH_KEYWORDS, _pickle_loads__doc__}, + {"loads", (PyCFunction)_pickle_loads, METH_FASTCALL, _pickle_loads__doc__}, static PyObject * _pickle_loads_impl(PyObject *module, PyObject *data, int fix_imports, const char *encoding, const char *errors); static PyObject * -_pickle_loads(PyObject *module, PyObject *args, PyObject *kwargs) +_pickle_loads(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"data", "fix_imports", "encoding", "errors", NULL}; @@ -551,7 +551,7 @@ const char *encoding = "ASCII"; const char *errors = "strict"; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &data, &fix_imports, &encoding, &errors)) { goto exit; } @@ -560,4 +560,4 @@ exit: return return_value; } -/*[clinic end generated code: output=50f9127109673c98 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=82be137b3c09cb9f input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_sre.c.h b/Modules/clinic/_sre.c.h --- a/Modules/clinic/_sre.c.h +++ b/Modules/clinic/_sre.c.h @@ -69,7 +69,7 @@ "Matches zero or more characters at the beginning of the string."); #define _SRE_SRE_PATTERN_MATCH_METHODDEF \ - {"match", (PyCFunction)_sre_SRE_Pattern_match, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern_match__doc__}, + {"match", (PyCFunction)_sre_SRE_Pattern_match, METH_FASTCALL, _sre_SRE_Pattern_match__doc__}, static PyObject * _sre_SRE_Pattern_match_impl(PatternObject *self, PyObject *string, @@ -77,7 +77,7 @@ PyObject *pattern); static PyObject * -_sre_SRE_Pattern_match(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern_match(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", "pos", "endpos", "pattern", NULL}; @@ -87,7 +87,7 @@ Py_ssize_t endpos = PY_SSIZE_T_MAX; PyObject *pattern = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string, &pos, &endpos, &pattern)) { goto exit; } @@ -105,7 +105,7 @@ "Matches against all of the string"); #define _SRE_SRE_PATTERN_FULLMATCH_METHODDEF \ - {"fullmatch", (PyCFunction)_sre_SRE_Pattern_fullmatch, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern_fullmatch__doc__}, + {"fullmatch", (PyCFunction)_sre_SRE_Pattern_fullmatch, METH_FASTCALL, _sre_SRE_Pattern_fullmatch__doc__}, static PyObject * _sre_SRE_Pattern_fullmatch_impl(PatternObject *self, PyObject *string, @@ -113,7 +113,7 @@ PyObject *pattern); static PyObject * -_sre_SRE_Pattern_fullmatch(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern_fullmatch(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", "pos", "endpos", "pattern", NULL}; @@ -123,7 +123,7 @@ Py_ssize_t endpos = PY_SSIZE_T_MAX; PyObject *pattern = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string, &pos, &endpos, &pattern)) { goto exit; } @@ -143,7 +143,7 @@ "Return None if no position in the string matches."); #define _SRE_SRE_PATTERN_SEARCH_METHODDEF \ - {"search", (PyCFunction)_sre_SRE_Pattern_search, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern_search__doc__}, + {"search", (PyCFunction)_sre_SRE_Pattern_search, METH_FASTCALL, _sre_SRE_Pattern_search__doc__}, static PyObject * _sre_SRE_Pattern_search_impl(PatternObject *self, PyObject *string, @@ -151,7 +151,7 @@ PyObject *pattern); static PyObject * -_sre_SRE_Pattern_search(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern_search(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", "pos", "endpos", "pattern", NULL}; @@ -161,7 +161,7 @@ Py_ssize_t endpos = PY_SSIZE_T_MAX; PyObject *pattern = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string, &pos, &endpos, &pattern)) { goto exit; } @@ -179,7 +179,7 @@ "Return a list of all non-overlapping matches of pattern in string."); #define _SRE_SRE_PATTERN_FINDALL_METHODDEF \ - {"findall", (PyCFunction)_sre_SRE_Pattern_findall, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern_findall__doc__}, + {"findall", (PyCFunction)_sre_SRE_Pattern_findall, METH_FASTCALL, _sre_SRE_Pattern_findall__doc__}, static PyObject * _sre_SRE_Pattern_findall_impl(PatternObject *self, PyObject *string, @@ -187,7 +187,7 @@ PyObject *source); static PyObject * -_sre_SRE_Pattern_findall(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern_findall(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", "pos", "endpos", "source", NULL}; @@ -197,7 +197,7 @@ Py_ssize_t endpos = PY_SSIZE_T_MAX; PyObject *source = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string, &pos, &endpos, &source)) { goto exit; } @@ -216,14 +216,14 @@ "For each match, the iterator returns a match object."); #define _SRE_SRE_PATTERN_FINDITER_METHODDEF \ - {"finditer", (PyCFunction)_sre_SRE_Pattern_finditer, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern_finditer__doc__}, + {"finditer", (PyCFunction)_sre_SRE_Pattern_finditer, METH_FASTCALL, _sre_SRE_Pattern_finditer__doc__}, static PyObject * _sre_SRE_Pattern_finditer_impl(PatternObject *self, PyObject *string, Py_ssize_t pos, Py_ssize_t endpos); static PyObject * -_sre_SRE_Pattern_finditer(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern_finditer(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", "pos", "endpos", NULL}; @@ -232,7 +232,7 @@ Py_ssize_t pos = 0; Py_ssize_t endpos = PY_SSIZE_T_MAX; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string, &pos, &endpos)) { goto exit; } @@ -248,14 +248,14 @@ "\n"); #define _SRE_SRE_PATTERN_SCANNER_METHODDEF \ - {"scanner", (PyCFunction)_sre_SRE_Pattern_scanner, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern_scanner__doc__}, + {"scanner", (PyCFunction)_sre_SRE_Pattern_scanner, METH_FASTCALL, _sre_SRE_Pattern_scanner__doc__}, static PyObject * _sre_SRE_Pattern_scanner_impl(PatternObject *self, PyObject *string, Py_ssize_t pos, Py_ssize_t endpos); static PyObject * -_sre_SRE_Pattern_scanner(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern_scanner(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", "pos", "endpos", NULL}; @@ -264,7 +264,7 @@ Py_ssize_t pos = 0; Py_ssize_t endpos = PY_SSIZE_T_MAX; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string, &pos, &endpos)) { goto exit; } @@ -281,14 +281,14 @@ "Split string by the occurrences of pattern."); #define _SRE_SRE_PATTERN_SPLIT_METHODDEF \ - {"split", (PyCFunction)_sre_SRE_Pattern_split, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern_split__doc__}, + {"split", (PyCFunction)_sre_SRE_Pattern_split, METH_FASTCALL, _sre_SRE_Pattern_split__doc__}, static PyObject * _sre_SRE_Pattern_split_impl(PatternObject *self, PyObject *string, Py_ssize_t maxsplit, PyObject *source); static PyObject * -_sre_SRE_Pattern_split(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern_split(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", "maxsplit", "source", NULL}; @@ -297,7 +297,7 @@ Py_ssize_t maxsplit = 0; PyObject *source = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string, &maxsplit, &source)) { goto exit; } @@ -314,14 +314,14 @@ "Return the string obtained by replacing the leftmost non-overlapping occurrences of pattern in string by the replacement repl."); #define _SRE_SRE_PATTERN_SUB_METHODDEF \ - {"sub", (PyCFunction)_sre_SRE_Pattern_sub, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern_sub__doc__}, + {"sub", (PyCFunction)_sre_SRE_Pattern_sub, METH_FASTCALL, _sre_SRE_Pattern_sub__doc__}, static PyObject * _sre_SRE_Pattern_sub_impl(PatternObject *self, PyObject *repl, PyObject *string, Py_ssize_t count); static PyObject * -_sre_SRE_Pattern_sub(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern_sub(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"repl", "string", "count", NULL}; @@ -330,7 +330,7 @@ PyObject *string; Py_ssize_t count = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &repl, &string, &count)) { goto exit; } @@ -347,14 +347,14 @@ "Return the tuple (new_string, number_of_subs_made) found by replacing the leftmost non-overlapping occurrences of pattern with the replacement repl."); #define _SRE_SRE_PATTERN_SUBN_METHODDEF \ - {"subn", (PyCFunction)_sre_SRE_Pattern_subn, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern_subn__doc__}, + {"subn", (PyCFunction)_sre_SRE_Pattern_subn, METH_FASTCALL, _sre_SRE_Pattern_subn__doc__}, static PyObject * _sre_SRE_Pattern_subn_impl(PatternObject *self, PyObject *repl, PyObject *string, Py_ssize_t count); static PyObject * -_sre_SRE_Pattern_subn(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern_subn(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"repl", "string", "count", NULL}; @@ -363,7 +363,7 @@ PyObject *string; Py_ssize_t count = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &repl, &string, &count)) { goto exit; } @@ -396,20 +396,20 @@ "\n"); #define _SRE_SRE_PATTERN___DEEPCOPY___METHODDEF \ - {"__deepcopy__", (PyCFunction)_sre_SRE_Pattern___deepcopy__, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Pattern___deepcopy____doc__}, + {"__deepcopy__", (PyCFunction)_sre_SRE_Pattern___deepcopy__, METH_FASTCALL, _sre_SRE_Pattern___deepcopy____doc__}, static PyObject * _sre_SRE_Pattern___deepcopy___impl(PatternObject *self, PyObject *memo); static PyObject * -_sre_SRE_Pattern___deepcopy__(PatternObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Pattern___deepcopy__(PatternObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"memo", NULL}; static _PyArg_Parser _parser = {"O:__deepcopy__", _keywords, 0}; PyObject *memo; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &memo)) { goto exit; } @@ -426,7 +426,7 @@ "\n"); #define _SRE_COMPILE_METHODDEF \ - {"compile", (PyCFunction)_sre_compile, METH_VARARGS|METH_KEYWORDS, _sre_compile__doc__}, + {"compile", (PyCFunction)_sre_compile, METH_FASTCALL, _sre_compile__doc__}, static PyObject * _sre_compile_impl(PyObject *module, PyObject *pattern, int flags, @@ -434,7 +434,7 @@ PyObject *indexgroup); static PyObject * -_sre_compile(PyObject *module, PyObject *args, PyObject *kwargs) +_sre_compile(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"pattern", "flags", "code", "groups", "groupindex", "indexgroup", NULL}; @@ -446,7 +446,7 @@ PyObject *groupindex; PyObject *indexgroup; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &pattern, &flags, &PyList_Type, &code, &groups, &groupindex, &indexgroup)) { goto exit; } @@ -463,20 +463,20 @@ "Return the string obtained by doing backslash substitution on the string template, as done by the sub() method."); #define _SRE_SRE_MATCH_EXPAND_METHODDEF \ - {"expand", (PyCFunction)_sre_SRE_Match_expand, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Match_expand__doc__}, + {"expand", (PyCFunction)_sre_SRE_Match_expand, METH_FASTCALL, _sre_SRE_Match_expand__doc__}, static PyObject * _sre_SRE_Match_expand_impl(MatchObject *self, PyObject *template); static PyObject * -_sre_SRE_Match_expand(MatchObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Match_expand(MatchObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"template", NULL}; static _PyArg_Parser _parser = {"O:expand", _keywords, 0}; PyObject *template; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &template)) { goto exit; } @@ -496,20 +496,20 @@ " Is used for groups that did not participate in the match."); #define _SRE_SRE_MATCH_GROUPS_METHODDEF \ - {"groups", (PyCFunction)_sre_SRE_Match_groups, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Match_groups__doc__}, + {"groups", (PyCFunction)_sre_SRE_Match_groups, METH_FASTCALL, _sre_SRE_Match_groups__doc__}, static PyObject * _sre_SRE_Match_groups_impl(MatchObject *self, PyObject *default_value); static PyObject * -_sre_SRE_Match_groups(MatchObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Match_groups(MatchObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"default", NULL}; static _PyArg_Parser _parser = {"|O:groups", _keywords, 0}; PyObject *default_value = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &default_value)) { goto exit; } @@ -529,20 +529,20 @@ " Is used for groups that did not participate in the match."); #define _SRE_SRE_MATCH_GROUPDICT_METHODDEF \ - {"groupdict", (PyCFunction)_sre_SRE_Match_groupdict, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Match_groupdict__doc__}, + {"groupdict", (PyCFunction)_sre_SRE_Match_groupdict, METH_FASTCALL, _sre_SRE_Match_groupdict__doc__}, static PyObject * _sre_SRE_Match_groupdict_impl(MatchObject *self, PyObject *default_value); static PyObject * -_sre_SRE_Match_groupdict(MatchObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Match_groupdict(MatchObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"default", NULL}; static _PyArg_Parser _parser = {"|O:groupdict", _keywords, 0}; PyObject *default_value = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &default_value)) { goto exit; } @@ -672,20 +672,20 @@ "\n"); #define _SRE_SRE_MATCH___DEEPCOPY___METHODDEF \ - {"__deepcopy__", (PyCFunction)_sre_SRE_Match___deepcopy__, METH_VARARGS|METH_KEYWORDS, _sre_SRE_Match___deepcopy____doc__}, + {"__deepcopy__", (PyCFunction)_sre_SRE_Match___deepcopy__, METH_FASTCALL, _sre_SRE_Match___deepcopy____doc__}, static PyObject * _sre_SRE_Match___deepcopy___impl(MatchObject *self, PyObject *memo); static PyObject * -_sre_SRE_Match___deepcopy__(MatchObject *self, PyObject *args, PyObject *kwargs) +_sre_SRE_Match___deepcopy__(MatchObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"memo", NULL}; static _PyArg_Parser _parser = {"O:__deepcopy__", _keywords, 0}; PyObject *memo; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &memo)) { goto exit; } @@ -728,4 +728,4 @@ { return _sre_SRE_Scanner_search_impl(self); } -/*[clinic end generated code: output=2cbc2b1482738e54 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=a4a246bca1963bc9 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_ssl.c.h b/Modules/clinic/_ssl.c.h --- a/Modules/clinic/_ssl.c.h +++ b/Modules/clinic/_ssl.c.h @@ -469,14 +469,14 @@ "\n"); #define _SSL__SSLCONTEXT_LOAD_CERT_CHAIN_METHODDEF \ - {"load_cert_chain", (PyCFunction)_ssl__SSLContext_load_cert_chain, METH_VARARGS|METH_KEYWORDS, _ssl__SSLContext_load_cert_chain__doc__}, + {"load_cert_chain", (PyCFunction)_ssl__SSLContext_load_cert_chain, METH_FASTCALL, _ssl__SSLContext_load_cert_chain__doc__}, static PyObject * _ssl__SSLContext_load_cert_chain_impl(PySSLContext *self, PyObject *certfile, PyObject *keyfile, PyObject *password); static PyObject * -_ssl__SSLContext_load_cert_chain(PySSLContext *self, PyObject *args, PyObject *kwargs) +_ssl__SSLContext_load_cert_chain(PySSLContext *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"certfile", "keyfile", "password", NULL}; @@ -485,7 +485,7 @@ PyObject *keyfile = NULL; PyObject *password = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &certfile, &keyfile, &password)) { goto exit; } @@ -501,7 +501,7 @@ "\n"); #define _SSL__SSLCONTEXT_LOAD_VERIFY_LOCATIONS_METHODDEF \ - {"load_verify_locations", (PyCFunction)_ssl__SSLContext_load_verify_locations, METH_VARARGS|METH_KEYWORDS, _ssl__SSLContext_load_verify_locations__doc__}, + {"load_verify_locations", (PyCFunction)_ssl__SSLContext_load_verify_locations, METH_FASTCALL, _ssl__SSLContext_load_verify_locations__doc__}, static PyObject * _ssl__SSLContext_load_verify_locations_impl(PySSLContext *self, @@ -510,7 +510,7 @@ PyObject *cadata); static PyObject * -_ssl__SSLContext_load_verify_locations(PySSLContext *self, PyObject *args, PyObject *kwargs) +_ssl__SSLContext_load_verify_locations(PySSLContext *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"cafile", "capath", "cadata", NULL}; @@ -519,7 +519,7 @@ PyObject *capath = NULL; PyObject *cadata = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &cafile, &capath, &cadata)) { goto exit; } @@ -543,14 +543,14 @@ "\n"); #define _SSL__SSLCONTEXT__WRAP_SOCKET_METHODDEF \ - {"_wrap_socket", (PyCFunction)_ssl__SSLContext__wrap_socket, METH_VARARGS|METH_KEYWORDS, _ssl__SSLContext__wrap_socket__doc__}, + {"_wrap_socket", (PyCFunction)_ssl__SSLContext__wrap_socket, METH_FASTCALL, _ssl__SSLContext__wrap_socket__doc__}, static PyObject * _ssl__SSLContext__wrap_socket_impl(PySSLContext *self, PyObject *sock, int server_side, PyObject *hostname_obj); static PyObject * -_ssl__SSLContext__wrap_socket(PySSLContext *self, PyObject *args, PyObject *kwargs) +_ssl__SSLContext__wrap_socket(PySSLContext *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"sock", "server_side", "server_hostname", NULL}; @@ -559,7 +559,7 @@ int server_side; PyObject *hostname_obj = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, PySocketModule.Sock_Type, &sock, &server_side, &hostname_obj)) { goto exit; } @@ -576,7 +576,7 @@ "\n"); #define _SSL__SSLCONTEXT__WRAP_BIO_METHODDEF \ - {"_wrap_bio", (PyCFunction)_ssl__SSLContext__wrap_bio, METH_VARARGS|METH_KEYWORDS, _ssl__SSLContext__wrap_bio__doc__}, + {"_wrap_bio", (PyCFunction)_ssl__SSLContext__wrap_bio, METH_FASTCALL, _ssl__SSLContext__wrap_bio__doc__}, static PyObject * _ssl__SSLContext__wrap_bio_impl(PySSLContext *self, PySSLMemoryBIO *incoming, @@ -584,7 +584,7 @@ PyObject *hostname_obj); static PyObject * -_ssl__SSLContext__wrap_bio(PySSLContext *self, PyObject *args, PyObject *kwargs) +_ssl__SSLContext__wrap_bio(PySSLContext *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"incoming", "outgoing", "server_side", "server_hostname", NULL}; @@ -594,7 +594,7 @@ int server_side; PyObject *hostname_obj = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &PySSLMemoryBIO_Type, &incoming, &PySSLMemoryBIO_Type, &outgoing, &server_side, &hostname_obj)) { goto exit; } @@ -700,20 +700,20 @@ "been used at least once."); #define _SSL__SSLCONTEXT_GET_CA_CERTS_METHODDEF \ - {"get_ca_certs", (PyCFunction)_ssl__SSLContext_get_ca_certs, METH_VARARGS|METH_KEYWORDS, _ssl__SSLContext_get_ca_certs__doc__}, + {"get_ca_certs", (PyCFunction)_ssl__SSLContext_get_ca_certs, METH_FASTCALL, _ssl__SSLContext_get_ca_certs__doc__}, static PyObject * _ssl__SSLContext_get_ca_certs_impl(PySSLContext *self, int binary_form); static PyObject * -_ssl__SSLContext_get_ca_certs(PySSLContext *self, PyObject *args, PyObject *kwargs) +_ssl__SSLContext_get_ca_certs(PySSLContext *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"binary_form", NULL}; static _PyArg_Parser _parser = {"|p:get_ca_certs", _keywords, 0}; int binary_form = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &binary_form)) { goto exit; } @@ -1011,13 +1011,13 @@ "long name are also matched."); #define _SSL_TXT2OBJ_METHODDEF \ - {"txt2obj", (PyCFunction)_ssl_txt2obj, METH_VARARGS|METH_KEYWORDS, _ssl_txt2obj__doc__}, + {"txt2obj", (PyCFunction)_ssl_txt2obj, METH_FASTCALL, _ssl_txt2obj__doc__}, static PyObject * _ssl_txt2obj_impl(PyObject *module, const char *txt, int name); static PyObject * -_ssl_txt2obj(PyObject *module, PyObject *args, PyObject *kwargs) +_ssl_txt2obj(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"txt", "name", NULL}; @@ -1025,7 +1025,7 @@ const char *txt; int name = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &txt, &name)) { goto exit; } @@ -1077,20 +1077,20 @@ "a set of OIDs or the boolean True."); #define _SSL_ENUM_CERTIFICATES_METHODDEF \ - {"enum_certificates", (PyCFunction)_ssl_enum_certificates, METH_VARARGS|METH_KEYWORDS, _ssl_enum_certificates__doc__}, + {"enum_certificates", (PyCFunction)_ssl_enum_certificates, METH_FASTCALL, _ssl_enum_certificates__doc__}, static PyObject * _ssl_enum_certificates_impl(PyObject *module, const char *store_name); static PyObject * -_ssl_enum_certificates(PyObject *module, PyObject *args, PyObject *kwargs) +_ssl_enum_certificates(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"store_name", NULL}; static _PyArg_Parser _parser = {"s:enum_certificates", _keywords, 0}; const char *store_name; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &store_name)) { goto exit; } @@ -1116,20 +1116,20 @@ "X509_ASN_ENCODING or PKCS_7_ASN_ENCODING."); #define _SSL_ENUM_CRLS_METHODDEF \ - {"enum_crls", (PyCFunction)_ssl_enum_crls, METH_VARARGS|METH_KEYWORDS, _ssl_enum_crls__doc__}, + {"enum_crls", (PyCFunction)_ssl_enum_crls, METH_FASTCALL, _ssl_enum_crls__doc__}, static PyObject * _ssl_enum_crls_impl(PyObject *module, const char *store_name); static PyObject * -_ssl_enum_crls(PyObject *module, PyObject *args, PyObject *kwargs) +_ssl_enum_crls(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"store_name", NULL}; static _PyArg_Parser _parser = {"s:enum_crls", _keywords, 0}; const char *store_name; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &store_name)) { goto exit; } @@ -1168,4 +1168,4 @@ #ifndef _SSL_ENUM_CRLS_METHODDEF #define _SSL_ENUM_CRLS_METHODDEF #endif /* !defined(_SSL_ENUM_CRLS_METHODDEF) */ -/*[clinic end generated code: output=2e7907a7d8f97ccf input=a9049054013a1b77]*/ +/*[clinic end generated code: output=a859b21fe68a6115 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/_winapi.c.h b/Modules/clinic/_winapi.c.h --- a/Modules/clinic/_winapi.c.h +++ b/Modules/clinic/_winapi.c.h @@ -95,14 +95,14 @@ "\n"); #define _WINAPI_CONNECTNAMEDPIPE_METHODDEF \ - {"ConnectNamedPipe", (PyCFunction)_winapi_ConnectNamedPipe, METH_VARARGS|METH_KEYWORDS, _winapi_ConnectNamedPipe__doc__}, + {"ConnectNamedPipe", (PyCFunction)_winapi_ConnectNamedPipe, METH_FASTCALL, _winapi_ConnectNamedPipe__doc__}, static PyObject * _winapi_ConnectNamedPipe_impl(PyObject *module, HANDLE handle, int use_overlapped); static PyObject * -_winapi_ConnectNamedPipe(PyObject *module, PyObject *args, PyObject *kwargs) +_winapi_ConnectNamedPipe(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"handle", "overlapped", NULL}; @@ -110,7 +110,7 @@ HANDLE handle; int use_overlapped = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &handle, &use_overlapped)) { goto exit; } @@ -670,14 +670,14 @@ "\n"); #define _WINAPI_READFILE_METHODDEF \ - {"ReadFile", (PyCFunction)_winapi_ReadFile, METH_VARARGS|METH_KEYWORDS, _winapi_ReadFile__doc__}, + {"ReadFile", (PyCFunction)_winapi_ReadFile, METH_FASTCALL, _winapi_ReadFile__doc__}, static PyObject * _winapi_ReadFile_impl(PyObject *module, HANDLE handle, int size, int use_overlapped); static PyObject * -_winapi_ReadFile(PyObject *module, PyObject *args, PyObject *kwargs) +_winapi_ReadFile(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"handle", "size", "overlapped", NULL}; @@ -686,7 +686,7 @@ int size; int use_overlapped = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &handle, &size, &use_overlapped)) { goto exit; } @@ -864,14 +864,14 @@ "\n"); #define _WINAPI_WRITEFILE_METHODDEF \ - {"WriteFile", (PyCFunction)_winapi_WriteFile, METH_VARARGS|METH_KEYWORDS, _winapi_WriteFile__doc__}, + {"WriteFile", (PyCFunction)_winapi_WriteFile, METH_FASTCALL, _winapi_WriteFile__doc__}, static PyObject * _winapi_WriteFile_impl(PyObject *module, HANDLE handle, PyObject *buffer, int use_overlapped); static PyObject * -_winapi_WriteFile(PyObject *module, PyObject *args, PyObject *kwargs) +_winapi_WriteFile(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"handle", "buffer", "overlapped", NULL}; @@ -880,7 +880,7 @@ PyObject *buffer; int use_overlapped = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &handle, &buffer, &use_overlapped)) { goto exit; } @@ -889,4 +889,4 @@ exit: return return_value; } -/*[clinic end generated code: output=4bfccfb32ab726e8 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=46d6382a6662c4a9 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/binascii.c.h b/Modules/clinic/binascii.c.h --- a/Modules/clinic/binascii.c.h +++ b/Modules/clinic/binascii.c.h @@ -103,13 +103,13 @@ "Base64-code line of data."); #define BINASCII_B2A_BASE64_METHODDEF \ - {"b2a_base64", (PyCFunction)binascii_b2a_base64, METH_VARARGS|METH_KEYWORDS, binascii_b2a_base64__doc__}, + {"b2a_base64", (PyCFunction)binascii_b2a_base64, METH_FASTCALL, binascii_b2a_base64__doc__}, static PyObject * binascii_b2a_base64_impl(PyObject *module, Py_buffer *data, int newline); static PyObject * -binascii_b2a_base64(PyObject *module, PyObject *args, PyObject *kwargs) +binascii_b2a_base64(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"data", "newline", NULL}; @@ -117,7 +117,7 @@ Py_buffer data = {NULL, NULL}; int newline = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &data, &newline)) { goto exit; } @@ -480,13 +480,13 @@ "Decode a string of qp-encoded data."); #define BINASCII_A2B_QP_METHODDEF \ - {"a2b_qp", (PyCFunction)binascii_a2b_qp, METH_VARARGS|METH_KEYWORDS, binascii_a2b_qp__doc__}, + {"a2b_qp", (PyCFunction)binascii_a2b_qp, METH_FASTCALL, binascii_a2b_qp__doc__}, static PyObject * binascii_a2b_qp_impl(PyObject *module, Py_buffer *data, int header); static PyObject * -binascii_a2b_qp(PyObject *module, PyObject *args, PyObject *kwargs) +binascii_a2b_qp(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"data", "header", NULL}; @@ -494,7 +494,7 @@ Py_buffer data = {NULL, NULL}; int header = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, ascii_buffer_converter, &data, &header)) { goto exit; } @@ -519,14 +519,14 @@ "are both encoded. When quotetabs is set, space and tabs are encoded."); #define BINASCII_B2A_QP_METHODDEF \ - {"b2a_qp", (PyCFunction)binascii_b2a_qp, METH_VARARGS|METH_KEYWORDS, binascii_b2a_qp__doc__}, + {"b2a_qp", (PyCFunction)binascii_b2a_qp, METH_FASTCALL, binascii_b2a_qp__doc__}, static PyObject * binascii_b2a_qp_impl(PyObject *module, Py_buffer *data, int quotetabs, int istext, int header); static PyObject * -binascii_b2a_qp(PyObject *module, PyObject *args, PyObject *kwargs) +binascii_b2a_qp(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"data", "quotetabs", "istext", "header", NULL}; @@ -536,7 +536,7 @@ int istext = 1; int header = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &data, "etabs, &istext, &header)) { goto exit; } @@ -550,4 +550,4 @@ return return_value; } -/*[clinic end generated code: output=12611b05d8bf4a9c input=a9049054013a1b77]*/ +/*[clinic end generated code: output=1f8d6e48f75f6d1e input=a9049054013a1b77]*/ diff --git a/Modules/clinic/cmathmodule.c.h b/Modules/clinic/cmathmodule.c.h --- a/Modules/clinic/cmathmodule.c.h +++ b/Modules/clinic/cmathmodule.c.h @@ -851,14 +851,14 @@ "not close to anything, even itself. inf and -inf are only close to themselves."); #define CMATH_ISCLOSE_METHODDEF \ - {"isclose", (PyCFunction)cmath_isclose, METH_VARARGS|METH_KEYWORDS, cmath_isclose__doc__}, + {"isclose", (PyCFunction)cmath_isclose, METH_FASTCALL, cmath_isclose__doc__}, static int cmath_isclose_impl(PyObject *module, Py_complex a, Py_complex b, double rel_tol, double abs_tol); static PyObject * -cmath_isclose(PyObject *module, PyObject *args, PyObject *kwargs) +cmath_isclose(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"a", "b", "rel_tol", "abs_tol", NULL}; @@ -869,7 +869,7 @@ double abs_tol = 0.0; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &a, &b, &rel_tol, &abs_tol)) { goto exit; } @@ -882,4 +882,4 @@ exit: return return_value; } -/*[clinic end generated code: output=aa2e77ca9fc26928 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=978f59702b41655f input=a9049054013a1b77]*/ diff --git a/Modules/clinic/grpmodule.c.h b/Modules/clinic/grpmodule.c.h --- a/Modules/clinic/grpmodule.c.h +++ b/Modules/clinic/grpmodule.c.h @@ -11,20 +11,20 @@ "If id is not valid, raise KeyError."); #define GRP_GETGRGID_METHODDEF \ - {"getgrgid", (PyCFunction)grp_getgrgid, METH_VARARGS|METH_KEYWORDS, grp_getgrgid__doc__}, + {"getgrgid", (PyCFunction)grp_getgrgid, METH_FASTCALL, grp_getgrgid__doc__}, static PyObject * grp_getgrgid_impl(PyObject *module, PyObject *id); static PyObject * -grp_getgrgid(PyObject *module, PyObject *args, PyObject *kwargs) +grp_getgrgid(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"id", NULL}; static _PyArg_Parser _parser = {"O:getgrgid", _keywords, 0}; PyObject *id; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &id)) { goto exit; } @@ -43,20 +43,20 @@ "If name is not valid, raise KeyError."); #define GRP_GETGRNAM_METHODDEF \ - {"getgrnam", (PyCFunction)grp_getgrnam, METH_VARARGS|METH_KEYWORDS, grp_getgrnam__doc__}, + {"getgrnam", (PyCFunction)grp_getgrnam, METH_FASTCALL, grp_getgrnam__doc__}, static PyObject * grp_getgrnam_impl(PyObject *module, PyObject *name); static PyObject * -grp_getgrnam(PyObject *module, PyObject *args, PyObject *kwargs) +grp_getgrnam(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"name", NULL}; static _PyArg_Parser _parser = {"U:getgrnam", _keywords, 0}; PyObject *name; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &name)) { goto exit; } @@ -86,4 +86,4 @@ { return grp_getgrall_impl(module); } -/*[clinic end generated code: output=c06081097b7fffe7 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=d6417ae0a7298e0e input=a9049054013a1b77]*/ diff --git a/Modules/clinic/md5module.c.h b/Modules/clinic/md5module.c.h --- a/Modules/clinic/md5module.c.h +++ b/Modules/clinic/md5module.c.h @@ -72,20 +72,20 @@ "Return a new MD5 hash object; optionally initialized with a string."); #define _MD5_MD5_METHODDEF \ - {"md5", (PyCFunction)_md5_md5, METH_VARARGS|METH_KEYWORDS, _md5_md5__doc__}, + {"md5", (PyCFunction)_md5_md5, METH_FASTCALL, _md5_md5__doc__}, static PyObject * _md5_md5_impl(PyObject *module, PyObject *string); static PyObject * -_md5_md5(PyObject *module, PyObject *args, PyObject *kwargs) +_md5_md5(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", NULL}; static _PyArg_Parser _parser = {"|O:md5", _keywords, 0}; PyObject *string = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string)) { goto exit; } @@ -94,4 +94,4 @@ exit: return return_value; } -/*[clinic end generated code: output=f86fc2f3f21831e2 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=54cd50db050f2589 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -27,13 +27,13 @@ " an open file descriptor."); #define OS_STAT_METHODDEF \ - {"stat", (PyCFunction)os_stat, METH_VARARGS|METH_KEYWORDS, os_stat__doc__}, + {"stat", (PyCFunction)os_stat, METH_FASTCALL, os_stat__doc__}, static PyObject * os_stat_impl(PyObject *module, path_t *path, int dir_fd, int follow_symlinks); static PyObject * -os_stat(PyObject *module, PyObject *args, PyObject *kwargs) +os_stat(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "dir_fd", "follow_symlinks", NULL}; @@ -42,7 +42,7 @@ int dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, FSTATAT_DIR_FD_CONVERTER, &dir_fd, &follow_symlinks)) { goto exit; } @@ -65,13 +65,13 @@ "Equivalent to stat(path, follow_symlinks=False)."); #define OS_LSTAT_METHODDEF \ - {"lstat", (PyCFunction)os_lstat, METH_VARARGS|METH_KEYWORDS, os_lstat__doc__}, + {"lstat", (PyCFunction)os_lstat, METH_FASTCALL, os_lstat__doc__}, static PyObject * os_lstat_impl(PyObject *module, path_t *path, int dir_fd); static PyObject * -os_lstat(PyObject *module, PyObject *args, PyObject *kwargs) +os_lstat(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "dir_fd", NULL}; @@ -79,7 +79,7 @@ path_t path = PATH_T_INITIALIZE("lstat", "path", 0, 0); int dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, FSTATAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -125,14 +125,14 @@ " has the specified access to the path."); #define OS_ACCESS_METHODDEF \ - {"access", (PyCFunction)os_access, METH_VARARGS|METH_KEYWORDS, os_access__doc__}, + {"access", (PyCFunction)os_access, METH_FASTCALL, os_access__doc__}, static int os_access_impl(PyObject *module, path_t *path, int mode, int dir_fd, int effective_ids, int follow_symlinks); static PyObject * -os_access(PyObject *module, PyObject *args, PyObject *kwargs) +os_access(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "mode", "dir_fd", "effective_ids", "follow_symlinks", NULL}; @@ -144,7 +144,7 @@ int follow_symlinks = 1; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &mode, FACCESSAT_DIR_FD_CONVERTER, &dir_fd, &effective_ids, &follow_symlinks)) { goto exit; } @@ -233,20 +233,20 @@ " If this functionality is unavailable, using it raises an exception."); #define OS_CHDIR_METHODDEF \ - {"chdir", (PyCFunction)os_chdir, METH_VARARGS|METH_KEYWORDS, os_chdir__doc__}, + {"chdir", (PyCFunction)os_chdir, METH_FASTCALL, os_chdir__doc__}, static PyObject * os_chdir_impl(PyObject *module, path_t *path); static PyObject * -os_chdir(PyObject *module, PyObject *args, PyObject *kwargs) +os_chdir(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", NULL}; static _PyArg_Parser _parser = {"O&:chdir", _keywords, 0}; path_t path = PATH_T_INITIALIZE("chdir", "path", 0, PATH_HAVE_FCHDIR); - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path)) { goto exit; } @@ -271,20 +271,20 @@ "Equivalent to os.chdir(fd)."); #define OS_FCHDIR_METHODDEF \ - {"fchdir", (PyCFunction)os_fchdir, METH_VARARGS|METH_KEYWORDS, os_fchdir__doc__}, + {"fchdir", (PyCFunction)os_fchdir, METH_FASTCALL, os_fchdir__doc__}, static PyObject * os_fchdir_impl(PyObject *module, int fd); static PyObject * -os_fchdir(PyObject *module, PyObject *args, PyObject *kwargs) +os_fchdir(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"fd", NULL}; static _PyArg_Parser _parser = {"O&:fchdir", _keywords, 0}; int fd; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, fildes_converter, &fd)) { goto exit; } @@ -323,14 +323,14 @@ " If they are unavailable, using them will raise a NotImplementedError."); #define OS_CHMOD_METHODDEF \ - {"chmod", (PyCFunction)os_chmod, METH_VARARGS|METH_KEYWORDS, os_chmod__doc__}, + {"chmod", (PyCFunction)os_chmod, METH_FASTCALL, os_chmod__doc__}, static PyObject * os_chmod_impl(PyObject *module, path_t *path, int mode, int dir_fd, int follow_symlinks); static PyObject * -os_chmod(PyObject *module, PyObject *args, PyObject *kwargs) +os_chmod(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "mode", "dir_fd", "follow_symlinks", NULL}; @@ -340,7 +340,7 @@ int dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &mode, FCHMODAT_DIR_FD_CONVERTER, &dir_fd, &follow_symlinks)) { goto exit; } @@ -364,13 +364,13 @@ "Equivalent to os.chmod(fd, mode)."); #define OS_FCHMOD_METHODDEF \ - {"fchmod", (PyCFunction)os_fchmod, METH_VARARGS|METH_KEYWORDS, os_fchmod__doc__}, + {"fchmod", (PyCFunction)os_fchmod, METH_FASTCALL, os_fchmod__doc__}, static PyObject * os_fchmod_impl(PyObject *module, int fd, int mode); static PyObject * -os_fchmod(PyObject *module, PyObject *args, PyObject *kwargs) +os_fchmod(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"fd", "mode", NULL}; @@ -378,7 +378,7 @@ int fd; int mode; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &fd, &mode)) { goto exit; } @@ -402,13 +402,13 @@ "Equivalent to chmod(path, mode, follow_symlinks=False).\""); #define OS_LCHMOD_METHODDEF \ - {"lchmod", (PyCFunction)os_lchmod, METH_VARARGS|METH_KEYWORDS, os_lchmod__doc__}, + {"lchmod", (PyCFunction)os_lchmod, METH_FASTCALL, os_lchmod__doc__}, static PyObject * os_lchmod_impl(PyObject *module, path_t *path, int mode); static PyObject * -os_lchmod(PyObject *module, PyObject *args, PyObject *kwargs) +os_lchmod(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "mode", NULL}; @@ -416,7 +416,7 @@ path_t path = PATH_T_INITIALIZE("lchmod", "path", 0, 0); int mode; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &mode)) { goto exit; } @@ -446,14 +446,14 @@ "unavailable, using it will raise a NotImplementedError."); #define OS_CHFLAGS_METHODDEF \ - {"chflags", (PyCFunction)os_chflags, METH_VARARGS|METH_KEYWORDS, os_chflags__doc__}, + {"chflags", (PyCFunction)os_chflags, METH_FASTCALL, os_chflags__doc__}, static PyObject * os_chflags_impl(PyObject *module, path_t *path, unsigned long flags, int follow_symlinks); static PyObject * -os_chflags(PyObject *module, PyObject *args, PyObject *kwargs) +os_chflags(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "flags", "follow_symlinks", NULL}; @@ -462,7 +462,7 @@ unsigned long flags; int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &flags, &follow_symlinks)) { goto exit; } @@ -489,13 +489,13 @@ "Equivalent to chflags(path, flags, follow_symlinks=False)."); #define OS_LCHFLAGS_METHODDEF \ - {"lchflags", (PyCFunction)os_lchflags, METH_VARARGS|METH_KEYWORDS, os_lchflags__doc__}, + {"lchflags", (PyCFunction)os_lchflags, METH_FASTCALL, os_lchflags__doc__}, static PyObject * os_lchflags_impl(PyObject *module, path_t *path, unsigned long flags); static PyObject * -os_lchflags(PyObject *module, PyObject *args, PyObject *kwargs) +os_lchflags(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "flags", NULL}; @@ -503,7 +503,7 @@ path_t path = PATH_T_INITIALIZE("lchflags", "path", 0, 0); unsigned long flags; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &flags)) { goto exit; } @@ -527,20 +527,20 @@ "Change root directory to path."); #define OS_CHROOT_METHODDEF \ - {"chroot", (PyCFunction)os_chroot, METH_VARARGS|METH_KEYWORDS, os_chroot__doc__}, + {"chroot", (PyCFunction)os_chroot, METH_FASTCALL, os_chroot__doc__}, static PyObject * os_chroot_impl(PyObject *module, path_t *path); static PyObject * -os_chroot(PyObject *module, PyObject *args, PyObject *kwargs) +os_chroot(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", NULL}; static _PyArg_Parser _parser = {"O&:chroot", _keywords, 0}; path_t path = PATH_T_INITIALIZE("chroot", "path", 0, 0); - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path)) { goto exit; } @@ -564,20 +564,20 @@ "Force write of fd to disk."); #define OS_FSYNC_METHODDEF \ - {"fsync", (PyCFunction)os_fsync, METH_VARARGS|METH_KEYWORDS, os_fsync__doc__}, + {"fsync", (PyCFunction)os_fsync, METH_FASTCALL, os_fsync__doc__}, static PyObject * os_fsync_impl(PyObject *module, int fd); static PyObject * -os_fsync(PyObject *module, PyObject *args, PyObject *kwargs) +os_fsync(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"fd", NULL}; static _PyArg_Parser _parser = {"O&:fsync", _keywords, 0}; int fd; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, fildes_converter, &fd)) { goto exit; } @@ -620,20 +620,20 @@ "Force write of fd to disk without forcing update of metadata."); #define OS_FDATASYNC_METHODDEF \ - {"fdatasync", (PyCFunction)os_fdatasync, METH_VARARGS|METH_KEYWORDS, os_fdatasync__doc__}, + {"fdatasync", (PyCFunction)os_fdatasync, METH_FASTCALL, os_fdatasync__doc__}, static PyObject * os_fdatasync_impl(PyObject *module, int fd); static PyObject * -os_fdatasync(PyObject *module, PyObject *args, PyObject *kwargs) +os_fdatasync(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"fd", NULL}; static _PyArg_Parser _parser = {"O&:fdatasync", _keywords, 0}; int fd; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, fildes_converter, &fd)) { goto exit; } @@ -678,14 +678,14 @@ " If they are unavailable, using them will raise a NotImplementedError."); #define OS_CHOWN_METHODDEF \ - {"chown", (PyCFunction)os_chown, METH_VARARGS|METH_KEYWORDS, os_chown__doc__}, + {"chown", (PyCFunction)os_chown, METH_FASTCALL, os_chown__doc__}, static PyObject * os_chown_impl(PyObject *module, path_t *path, uid_t uid, gid_t gid, int dir_fd, int follow_symlinks); static PyObject * -os_chown(PyObject *module, PyObject *args, PyObject *kwargs) +os_chown(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "uid", "gid", "dir_fd", "follow_symlinks", NULL}; @@ -696,7 +696,7 @@ int dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, _Py_Uid_Converter, &uid, _Py_Gid_Converter, &gid, FCHOWNAT_DIR_FD_CONVERTER, &dir_fd, &follow_symlinks)) { goto exit; } @@ -722,13 +722,13 @@ "Equivalent to os.chown(fd, uid, gid)."); #define OS_FCHOWN_METHODDEF \ - {"fchown", (PyCFunction)os_fchown, METH_VARARGS|METH_KEYWORDS, os_fchown__doc__}, + {"fchown", (PyCFunction)os_fchown, METH_FASTCALL, os_fchown__doc__}, static PyObject * os_fchown_impl(PyObject *module, int fd, uid_t uid, gid_t gid); static PyObject * -os_fchown(PyObject *module, PyObject *args, PyObject *kwargs) +os_fchown(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"fd", "uid", "gid", NULL}; @@ -737,7 +737,7 @@ uid_t uid; gid_t gid; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &fd, _Py_Uid_Converter, &uid, _Py_Gid_Converter, &gid)) { goto exit; } @@ -761,13 +761,13 @@ "Equivalent to os.chown(path, uid, gid, follow_symlinks=False)."); #define OS_LCHOWN_METHODDEF \ - {"lchown", (PyCFunction)os_lchown, METH_VARARGS|METH_KEYWORDS, os_lchown__doc__}, + {"lchown", (PyCFunction)os_lchown, METH_FASTCALL, os_lchown__doc__}, static PyObject * os_lchown_impl(PyObject *module, path_t *path, uid_t uid, gid_t gid); static PyObject * -os_lchown(PyObject *module, PyObject *args, PyObject *kwargs) +os_lchown(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "uid", "gid", NULL}; @@ -776,7 +776,7 @@ uid_t uid; gid_t gid; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, _Py_Uid_Converter, &uid, _Py_Gid_Converter, &gid)) { goto exit; } @@ -847,14 +847,14 @@ " NotImplementedError."); #define OS_LINK_METHODDEF \ - {"link", (PyCFunction)os_link, METH_VARARGS|METH_KEYWORDS, os_link__doc__}, + {"link", (PyCFunction)os_link, METH_FASTCALL, os_link__doc__}, static PyObject * os_link_impl(PyObject *module, path_t *src, path_t *dst, int src_dir_fd, int dst_dir_fd, int follow_symlinks); static PyObject * -os_link(PyObject *module, PyObject *args, PyObject *kwargs) +os_link(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"src", "dst", "src_dir_fd", "dst_dir_fd", "follow_symlinks", NULL}; @@ -865,7 +865,7 @@ int dst_dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &src, path_converter, &dst, dir_fd_converter, &src_dir_fd, dir_fd_converter, &dst_dir_fd, &follow_symlinks)) { goto exit; } @@ -900,20 +900,20 @@ "entries \'.\' and \'..\' even if they are present in the directory."); #define OS_LISTDIR_METHODDEF \ - {"listdir", (PyCFunction)os_listdir, METH_VARARGS|METH_KEYWORDS, os_listdir__doc__}, + {"listdir", (PyCFunction)os_listdir, METH_FASTCALL, os_listdir__doc__}, static PyObject * os_listdir_impl(PyObject *module, path_t *path); static PyObject * -os_listdir(PyObject *module, PyObject *args, PyObject *kwargs) +os_listdir(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", NULL}; static _PyArg_Parser _parser = {"|O&:listdir", _keywords, 0}; path_t path = PATH_T_INITIALIZE("listdir", "path", 1, PATH_HAVE_FDOPENDIR); - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path)) { goto exit; } @@ -1032,20 +1032,20 @@ "A helper function for ismount on Win32."); #define OS__GETVOLUMEPATHNAME_METHODDEF \ - {"_getvolumepathname", (PyCFunction)os__getvolumepathname, METH_VARARGS|METH_KEYWORDS, os__getvolumepathname__doc__}, + {"_getvolumepathname", (PyCFunction)os__getvolumepathname, METH_FASTCALL, os__getvolumepathname__doc__}, static PyObject * os__getvolumepathname_impl(PyObject *module, PyObject *path); static PyObject * -os__getvolumepathname(PyObject *module, PyObject *args, PyObject *kwargs) +os__getvolumepathname(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", NULL}; static _PyArg_Parser _parser = {"U:_getvolumepathname", _keywords, 0}; PyObject *path; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &path)) { goto exit; } @@ -1071,13 +1071,13 @@ "The mode argument is ignored on Windows."); #define OS_MKDIR_METHODDEF \ - {"mkdir", (PyCFunction)os_mkdir, METH_VARARGS|METH_KEYWORDS, os_mkdir__doc__}, + {"mkdir", (PyCFunction)os_mkdir, METH_FASTCALL, os_mkdir__doc__}, static PyObject * os_mkdir_impl(PyObject *module, path_t *path, int mode, int dir_fd); static PyObject * -os_mkdir(PyObject *module, PyObject *args, PyObject *kwargs) +os_mkdir(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "mode", "dir_fd", NULL}; @@ -1086,7 +1086,7 @@ int mode = 511; int dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &mode, MKDIRAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -1139,13 +1139,13 @@ "Return program scheduling priority."); #define OS_GETPRIORITY_METHODDEF \ - {"getpriority", (PyCFunction)os_getpriority, METH_VARARGS|METH_KEYWORDS, os_getpriority__doc__}, + {"getpriority", (PyCFunction)os_getpriority, METH_FASTCALL, os_getpriority__doc__}, static PyObject * os_getpriority_impl(PyObject *module, int which, int who); static PyObject * -os_getpriority(PyObject *module, PyObject *args, PyObject *kwargs) +os_getpriority(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"which", "who", NULL}; @@ -1153,7 +1153,7 @@ int which; int who; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &which, &who)) { goto exit; } @@ -1174,13 +1174,13 @@ "Set program scheduling priority."); #define OS_SETPRIORITY_METHODDEF \ - {"setpriority", (PyCFunction)os_setpriority, METH_VARARGS|METH_KEYWORDS, os_setpriority__doc__}, + {"setpriority", (PyCFunction)os_setpriority, METH_FASTCALL, os_setpriority__doc__}, static PyObject * os_setpriority_impl(PyObject *module, int which, int who, int priority); static PyObject * -os_setpriority(PyObject *module, PyObject *args, PyObject *kwargs) +os_setpriority(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"which", "who", "priority", NULL}; @@ -1189,7 +1189,7 @@ int who; int priority; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &which, &who, &priority)) { goto exit; } @@ -1214,14 +1214,14 @@ " If they are unavailable, using them will raise a NotImplementedError."); #define OS_RENAME_METHODDEF \ - {"rename", (PyCFunction)os_rename, METH_VARARGS|METH_KEYWORDS, os_rename__doc__}, + {"rename", (PyCFunction)os_rename, METH_FASTCALL, os_rename__doc__}, static PyObject * os_rename_impl(PyObject *module, path_t *src, path_t *dst, int src_dir_fd, int dst_dir_fd); static PyObject * -os_rename(PyObject *module, PyObject *args, PyObject *kwargs) +os_rename(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"src", "dst", "src_dir_fd", "dst_dir_fd", NULL}; @@ -1231,7 +1231,7 @@ int src_dir_fd = DEFAULT_DIR_FD; int dst_dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &src, path_converter, &dst, dir_fd_converter, &src_dir_fd, dir_fd_converter, &dst_dir_fd)) { goto exit; } @@ -1259,14 +1259,14 @@ " If they are unavailable, using them will raise a NotImplementedError.\""); #define OS_REPLACE_METHODDEF \ - {"replace", (PyCFunction)os_replace, METH_VARARGS|METH_KEYWORDS, os_replace__doc__}, + {"replace", (PyCFunction)os_replace, METH_FASTCALL, os_replace__doc__}, static PyObject * os_replace_impl(PyObject *module, path_t *src, path_t *dst, int src_dir_fd, int dst_dir_fd); static PyObject * -os_replace(PyObject *module, PyObject *args, PyObject *kwargs) +os_replace(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"src", "dst", "src_dir_fd", "dst_dir_fd", NULL}; @@ -1276,7 +1276,7 @@ int src_dir_fd = DEFAULT_DIR_FD; int dst_dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &src, path_converter, &dst, dir_fd_converter, &src_dir_fd, dir_fd_converter, &dst_dir_fd)) { goto exit; } @@ -1303,13 +1303,13 @@ " If it is unavailable, using it will raise a NotImplementedError."); #define OS_RMDIR_METHODDEF \ - {"rmdir", (PyCFunction)os_rmdir, METH_VARARGS|METH_KEYWORDS, os_rmdir__doc__}, + {"rmdir", (PyCFunction)os_rmdir, METH_FASTCALL, os_rmdir__doc__}, static PyObject * os_rmdir_impl(PyObject *module, path_t *path, int dir_fd); static PyObject * -os_rmdir(PyObject *module, PyObject *args, PyObject *kwargs) +os_rmdir(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "dir_fd", NULL}; @@ -1317,7 +1317,7 @@ path_t path = PATH_T_INITIALIZE("rmdir", "path", 0, 0); int dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, UNLINKAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -1339,13 +1339,13 @@ "Execute the command in a subshell."); #define OS_SYSTEM_METHODDEF \ - {"system", (PyCFunction)os_system, METH_VARARGS|METH_KEYWORDS, os_system__doc__}, + {"system", (PyCFunction)os_system, METH_FASTCALL, os_system__doc__}, static long os_system_impl(PyObject *module, Py_UNICODE *command); static PyObject * -os_system(PyObject *module, PyObject *args, PyObject *kwargs) +os_system(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"command", NULL}; @@ -1353,7 +1353,7 @@ Py_UNICODE *command; long _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &command)) { goto exit; } @@ -1378,13 +1378,13 @@ "Execute the command in a subshell."); #define OS_SYSTEM_METHODDEF \ - {"system", (PyCFunction)os_system, METH_VARARGS|METH_KEYWORDS, os_system__doc__}, + {"system", (PyCFunction)os_system, METH_FASTCALL, os_system__doc__}, static long os_system_impl(PyObject *module, PyObject *command); static PyObject * -os_system(PyObject *module, PyObject *args, PyObject *kwargs) +os_system(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"command", NULL}; @@ -1392,7 +1392,7 @@ PyObject *command = NULL; long _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, PyUnicode_FSConverter, &command)) { goto exit; } @@ -1450,13 +1450,13 @@ " If it is unavailable, using it will raise a NotImplementedError."); #define OS_UNLINK_METHODDEF \ - {"unlink", (PyCFunction)os_unlink, METH_VARARGS|METH_KEYWORDS, os_unlink__doc__}, + {"unlink", (PyCFunction)os_unlink, METH_FASTCALL, os_unlink__doc__}, static PyObject * os_unlink_impl(PyObject *module, path_t *path, int dir_fd); static PyObject * -os_unlink(PyObject *module, PyObject *args, PyObject *kwargs) +os_unlink(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "dir_fd", NULL}; @@ -1464,7 +1464,7 @@ path_t path = PATH_T_INITIALIZE("unlink", "path", 0, 0); int dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, UNLINKAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -1489,13 +1489,13 @@ " If it is unavailable, using it will raise a NotImplementedError."); #define OS_REMOVE_METHODDEF \ - {"remove", (PyCFunction)os_remove, METH_VARARGS|METH_KEYWORDS, os_remove__doc__}, + {"remove", (PyCFunction)os_remove, METH_FASTCALL, os_remove__doc__}, static PyObject * os_remove_impl(PyObject *module, path_t *path, int dir_fd); static PyObject * -os_remove(PyObject *module, PyObject *args, PyObject *kwargs) +os_remove(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "dir_fd", NULL}; @@ -1503,7 +1503,7 @@ path_t path = PATH_T_INITIALIZE("remove", "path", 0, 0); int dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, UNLINKAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -1571,14 +1571,14 @@ " If they are unavailable, using them will raise a NotImplementedError."); #define OS_UTIME_METHODDEF \ - {"utime", (PyCFunction)os_utime, METH_VARARGS|METH_KEYWORDS, os_utime__doc__}, + {"utime", (PyCFunction)os_utime, METH_FASTCALL, os_utime__doc__}, static PyObject * os_utime_impl(PyObject *module, path_t *path, PyObject *times, PyObject *ns, int dir_fd, int follow_symlinks); static PyObject * -os_utime(PyObject *module, PyObject *args, PyObject *kwargs) +os_utime(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "times", "ns", "dir_fd", "follow_symlinks", NULL}; @@ -1589,7 +1589,7 @@ int dir_fd = DEFAULT_DIR_FD; int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, ×, &ns, FUTIMENSAT_DIR_FD_CONVERTER, &dir_fd, &follow_symlinks)) { goto exit; } @@ -1609,20 +1609,20 @@ "Exit to the system with specified status, without normal exit processing."); #define OS__EXIT_METHODDEF \ - {"_exit", (PyCFunction)os__exit, METH_VARARGS|METH_KEYWORDS, os__exit__doc__}, + {"_exit", (PyCFunction)os__exit, METH_FASTCALL, os__exit__doc__}, static PyObject * os__exit_impl(PyObject *module, int status); static PyObject * -os__exit(PyObject *module, PyObject *args, PyObject *kwargs) +os__exit(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"status", NULL}; static _PyArg_Parser _parser = {"i:_exit", _keywords, 0}; int status; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &status)) { goto exit; } @@ -1689,13 +1689,13 @@ " Dictionary of strings mapping to strings."); #define OS_EXECVE_METHODDEF \ - {"execve", (PyCFunction)os_execve, METH_VARARGS|METH_KEYWORDS, os_execve__doc__}, + {"execve", (PyCFunction)os_execve, METH_FASTCALL, os_execve__doc__}, static PyObject * os_execve_impl(PyObject *module, path_t *path, PyObject *argv, PyObject *env); static PyObject * -os_execve(PyObject *module, PyObject *args, PyObject *kwargs) +os_execve(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "argv", "env", NULL}; @@ -1704,7 +1704,7 @@ PyObject *argv; PyObject *env; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &argv, &env)) { goto exit; } @@ -1868,20 +1868,20 @@ "Get the maximum scheduling priority for policy."); #define OS_SCHED_GET_PRIORITY_MAX_METHODDEF \ - {"sched_get_priority_max", (PyCFunction)os_sched_get_priority_max, METH_VARARGS|METH_KEYWORDS, os_sched_get_priority_max__doc__}, + {"sched_get_priority_max", (PyCFunction)os_sched_get_priority_max, METH_FASTCALL, os_sched_get_priority_max__doc__}, static PyObject * os_sched_get_priority_max_impl(PyObject *module, int policy); static PyObject * -os_sched_get_priority_max(PyObject *module, PyObject *args, PyObject *kwargs) +os_sched_get_priority_max(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"policy", NULL}; static _PyArg_Parser _parser = {"i:sched_get_priority_max", _keywords, 0}; int policy; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &policy)) { goto exit; } @@ -1902,20 +1902,20 @@ "Get the minimum scheduling priority for policy."); #define OS_SCHED_GET_PRIORITY_MIN_METHODDEF \ - {"sched_get_priority_min", (PyCFunction)os_sched_get_priority_min, METH_VARARGS|METH_KEYWORDS, os_sched_get_priority_min__doc__}, + {"sched_get_priority_min", (PyCFunction)os_sched_get_priority_min, METH_FASTCALL, os_sched_get_priority_min__doc__}, static PyObject * os_sched_get_priority_min_impl(PyObject *module, int policy); static PyObject * -os_sched_get_priority_min(PyObject *module, PyObject *args, PyObject *kwargs) +os_sched_get_priority_min(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"policy", NULL}; static _PyArg_Parser _parser = {"i:sched_get_priority_min", _keywords, 0}; int policy; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &policy)) { goto exit; } @@ -2398,20 +2398,20 @@ "Call the system call getpgid(), and return the result."); #define OS_GETPGID_METHODDEF \ - {"getpgid", (PyCFunction)os_getpgid, METH_VARARGS|METH_KEYWORDS, os_getpgid__doc__}, + {"getpgid", (PyCFunction)os_getpgid, METH_FASTCALL, os_getpgid__doc__}, static PyObject * os_getpgid_impl(PyObject *module, pid_t pid); static PyObject * -os_getpgid(PyObject *module, PyObject *args, PyObject *kwargs) +os_getpgid(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"pid", NULL}; static _PyArg_Parser _parser = {"" _Py_PARSE_PID ":getpgid", _keywords, 0}; pid_t pid; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &pid)) { goto exit; } @@ -2848,20 +2848,20 @@ " (pid, status, rusage)"); #define OS_WAIT3_METHODDEF \ - {"wait3", (PyCFunction)os_wait3, METH_VARARGS|METH_KEYWORDS, os_wait3__doc__}, + {"wait3", (PyCFunction)os_wait3, METH_FASTCALL, os_wait3__doc__}, static PyObject * os_wait3_impl(PyObject *module, int options); static PyObject * -os_wait3(PyObject *module, PyObject *args, PyObject *kwargs) +os_wait3(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"options", NULL}; static _PyArg_Parser _parser = {"i:wait3", _keywords, 0}; int options; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &options)) { goto exit; } @@ -2885,13 +2885,13 @@ " (pid, status, rusage)"); #define OS_WAIT4_METHODDEF \ - {"wait4", (PyCFunction)os_wait4, METH_VARARGS|METH_KEYWORDS, os_wait4__doc__}, + {"wait4", (PyCFunction)os_wait4, METH_FASTCALL, os_wait4__doc__}, static PyObject * os_wait4_impl(PyObject *module, pid_t pid, int options); static PyObject * -os_wait4(PyObject *module, PyObject *args, PyObject *kwargs) +os_wait4(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"pid", "options", NULL}; @@ -2899,7 +2899,7 @@ pid_t pid; int options; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &pid, &options)) { goto exit; } @@ -3076,14 +3076,14 @@ " If it is unavailable, using it will raise a NotImplementedError."); #define OS_SYMLINK_METHODDEF \ - {"symlink", (PyCFunction)os_symlink, METH_VARARGS|METH_KEYWORDS, os_symlink__doc__}, + {"symlink", (PyCFunction)os_symlink, METH_FASTCALL, os_symlink__doc__}, static PyObject * os_symlink_impl(PyObject *module, path_t *src, path_t *dst, int target_is_directory, int dir_fd); static PyObject * -os_symlink(PyObject *module, PyObject *args, PyObject *kwargs) +os_symlink(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"src", "dst", "target_is_directory", "dir_fd", NULL}; @@ -3093,7 +3093,7 @@ int target_is_directory = 0; int dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &src, path_converter, &dst, &target_is_directory, SYMLINKAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -3298,13 +3298,13 @@ " If it is unavailable, using it will raise a NotImplementedError."); #define OS_OPEN_METHODDEF \ - {"open", (PyCFunction)os_open, METH_VARARGS|METH_KEYWORDS, os_open__doc__}, + {"open", (PyCFunction)os_open, METH_FASTCALL, os_open__doc__}, static int os_open_impl(PyObject *module, path_t *path, int flags, int mode, int dir_fd); static PyObject * -os_open(PyObject *module, PyObject *args, PyObject *kwargs) +os_open(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "flags", "mode", "dir_fd", NULL}; @@ -3315,7 +3315,7 @@ int dir_fd = DEFAULT_DIR_FD; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &flags, &mode, OPENAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -3339,20 +3339,20 @@ "Close a file descriptor."); #define OS_CLOSE_METHODDEF \ - {"close", (PyCFunction)os_close, METH_VARARGS|METH_KEYWORDS, os_close__doc__}, + {"close", (PyCFunction)os_close, METH_FASTCALL, os_close__doc__}, static PyObject * os_close_impl(PyObject *module, int fd); static PyObject * -os_close(PyObject *module, PyObject *args, PyObject *kwargs) +os_close(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"fd", NULL}; static _PyArg_Parser _parser = {"i:close", _keywords, 0}; int fd; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &fd)) { goto exit; } @@ -3430,13 +3430,13 @@ "Duplicate file descriptor."); #define OS_DUP2_METHODDEF \ - {"dup2", (PyCFunction)os_dup2, METH_VARARGS|METH_KEYWORDS, os_dup2__doc__}, + {"dup2", (PyCFunction)os_dup2, METH_FASTCALL, os_dup2__doc__}, static PyObject * os_dup2_impl(PyObject *module, int fd, int fd2, int inheritable); static PyObject * -os_dup2(PyObject *module, PyObject *args, PyObject *kwargs) +os_dup2(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"fd", "fd2", "inheritable", NULL}; @@ -3445,7 +3445,7 @@ int fd2; int inheritable = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &fd, &fd2, &inheritable)) { goto exit; } @@ -3695,20 +3695,20 @@ "Equivalent to os.stat(fd)."); #define OS_FSTAT_METHODDEF \ - {"fstat", (PyCFunction)os_fstat, METH_VARARGS|METH_KEYWORDS, os_fstat__doc__}, + {"fstat", (PyCFunction)os_fstat, METH_FASTCALL, os_fstat__doc__}, static PyObject * os_fstat_impl(PyObject *module, int fd); static PyObject * -os_fstat(PyObject *module, PyObject *args, PyObject *kwargs) +os_fstat(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"fd", NULL}; static _PyArg_Parser _parser = {"i:fstat", _keywords, 0}; int fd; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &fd)) { goto exit; } @@ -3918,13 +3918,13 @@ " If it is unavailable, using it will raise a NotImplementedError."); #define OS_MKFIFO_METHODDEF \ - {"mkfifo", (PyCFunction)os_mkfifo, METH_VARARGS|METH_KEYWORDS, os_mkfifo__doc__}, + {"mkfifo", (PyCFunction)os_mkfifo, METH_FASTCALL, os_mkfifo__doc__}, static PyObject * os_mkfifo_impl(PyObject *module, path_t *path, int mode, int dir_fd); static PyObject * -os_mkfifo(PyObject *module, PyObject *args, PyObject *kwargs) +os_mkfifo(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "mode", "dir_fd", NULL}; @@ -3933,7 +3933,7 @@ int mode = 438; int dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &mode, MKFIFOAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -3969,14 +3969,14 @@ " If it is unavailable, using it will raise a NotImplementedError."); #define OS_MKNOD_METHODDEF \ - {"mknod", (PyCFunction)os_mknod, METH_VARARGS|METH_KEYWORDS, os_mknod__doc__}, + {"mknod", (PyCFunction)os_mknod, METH_FASTCALL, os_mknod__doc__}, static PyObject * os_mknod_impl(PyObject *module, path_t *path, int mode, dev_t device, int dir_fd); static PyObject * -os_mknod(PyObject *module, PyObject *args, PyObject *kwargs) +os_mknod(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "mode", "device", "dir_fd", NULL}; @@ -3986,7 +3986,7 @@ dev_t device = 0; int dir_fd = DEFAULT_DIR_FD; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &mode, _Py_Dev_Converter, &device, MKNODAT_DIR_FD_CONVERTER, &dir_fd)) { goto exit; } @@ -4156,13 +4156,13 @@ " If this functionality is unavailable, using it raises an exception."); #define OS_TRUNCATE_METHODDEF \ - {"truncate", (PyCFunction)os_truncate, METH_VARARGS|METH_KEYWORDS, os_truncate__doc__}, + {"truncate", (PyCFunction)os_truncate, METH_FASTCALL, os_truncate__doc__}, static PyObject * os_truncate_impl(PyObject *module, path_t *path, Py_off_t length); static PyObject * -os_truncate(PyObject *module, PyObject *args, PyObject *kwargs) +os_truncate(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "length", NULL}; @@ -4170,7 +4170,7 @@ path_t path = PATH_T_INITIALIZE("truncate", "path", 0, PATH_HAVE_FTRUNCATE); Py_off_t length; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, Py_off_t_converter, &length)) { goto exit; } @@ -4447,13 +4447,13 @@ "job control stop."); #define OS_WIFCONTINUED_METHODDEF \ - {"WIFCONTINUED", (PyCFunction)os_WIFCONTINUED, METH_VARARGS|METH_KEYWORDS, os_WIFCONTINUED__doc__}, + {"WIFCONTINUED", (PyCFunction)os_WIFCONTINUED, METH_FASTCALL, os_WIFCONTINUED__doc__}, static int os_WIFCONTINUED_impl(PyObject *module, int status); static PyObject * -os_WIFCONTINUED(PyObject *module, PyObject *args, PyObject *kwargs) +os_WIFCONTINUED(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"status", NULL}; @@ -4461,7 +4461,7 @@ int status; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &status)) { goto exit; } @@ -4486,13 +4486,13 @@ "Return True if the process returning status was stopped."); #define OS_WIFSTOPPED_METHODDEF \ - {"WIFSTOPPED", (PyCFunction)os_WIFSTOPPED, METH_VARARGS|METH_KEYWORDS, os_WIFSTOPPED__doc__}, + {"WIFSTOPPED", (PyCFunction)os_WIFSTOPPED, METH_FASTCALL, os_WIFSTOPPED__doc__}, static int os_WIFSTOPPED_impl(PyObject *module, int status); static PyObject * -os_WIFSTOPPED(PyObject *module, PyObject *args, PyObject *kwargs) +os_WIFSTOPPED(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"status", NULL}; @@ -4500,7 +4500,7 @@ int status; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &status)) { goto exit; } @@ -4525,13 +4525,13 @@ "Return True if the process returning status was terminated by a signal."); #define OS_WIFSIGNALED_METHODDEF \ - {"WIFSIGNALED", (PyCFunction)os_WIFSIGNALED, METH_VARARGS|METH_KEYWORDS, os_WIFSIGNALED__doc__}, + {"WIFSIGNALED", (PyCFunction)os_WIFSIGNALED, METH_FASTCALL, os_WIFSIGNALED__doc__}, static int os_WIFSIGNALED_impl(PyObject *module, int status); static PyObject * -os_WIFSIGNALED(PyObject *module, PyObject *args, PyObject *kwargs) +os_WIFSIGNALED(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"status", NULL}; @@ -4539,7 +4539,7 @@ int status; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &status)) { goto exit; } @@ -4564,13 +4564,13 @@ "Return True if the process returning status exited via the exit() system call."); #define OS_WIFEXITED_METHODDEF \ - {"WIFEXITED", (PyCFunction)os_WIFEXITED, METH_VARARGS|METH_KEYWORDS, os_WIFEXITED__doc__}, + {"WIFEXITED", (PyCFunction)os_WIFEXITED, METH_FASTCALL, os_WIFEXITED__doc__}, static int os_WIFEXITED_impl(PyObject *module, int status); static PyObject * -os_WIFEXITED(PyObject *module, PyObject *args, PyObject *kwargs) +os_WIFEXITED(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"status", NULL}; @@ -4578,7 +4578,7 @@ int status; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &status)) { goto exit; } @@ -4603,13 +4603,13 @@ "Return the process return code from status."); #define OS_WEXITSTATUS_METHODDEF \ - {"WEXITSTATUS", (PyCFunction)os_WEXITSTATUS, METH_VARARGS|METH_KEYWORDS, os_WEXITSTATUS__doc__}, + {"WEXITSTATUS", (PyCFunction)os_WEXITSTATUS, METH_FASTCALL, os_WEXITSTATUS__doc__}, static int os_WEXITSTATUS_impl(PyObject *module, int status); static PyObject * -os_WEXITSTATUS(PyObject *module, PyObject *args, PyObject *kwargs) +os_WEXITSTATUS(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"status", NULL}; @@ -4617,7 +4617,7 @@ int status; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &status)) { goto exit; } @@ -4642,13 +4642,13 @@ "Return the signal that terminated the process that provided the status value."); #define OS_WTERMSIG_METHODDEF \ - {"WTERMSIG", (PyCFunction)os_WTERMSIG, METH_VARARGS|METH_KEYWORDS, os_WTERMSIG__doc__}, + {"WTERMSIG", (PyCFunction)os_WTERMSIG, METH_FASTCALL, os_WTERMSIG__doc__}, static int os_WTERMSIG_impl(PyObject *module, int status); static PyObject * -os_WTERMSIG(PyObject *module, PyObject *args, PyObject *kwargs) +os_WTERMSIG(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"status", NULL}; @@ -4656,7 +4656,7 @@ int status; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &status)) { goto exit; } @@ -4681,13 +4681,13 @@ "Return the signal that stopped the process that provided the status value."); #define OS_WSTOPSIG_METHODDEF \ - {"WSTOPSIG", (PyCFunction)os_WSTOPSIG, METH_VARARGS|METH_KEYWORDS, os_WSTOPSIG__doc__}, + {"WSTOPSIG", (PyCFunction)os_WSTOPSIG, METH_FASTCALL, os_WSTOPSIG__doc__}, static int os_WSTOPSIG_impl(PyObject *module, int status); static PyObject * -os_WSTOPSIG(PyObject *module, PyObject *args, PyObject *kwargs) +os_WSTOPSIG(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"status", NULL}; @@ -4695,7 +4695,7 @@ int status; int _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &status)) { goto exit; } @@ -4757,20 +4757,20 @@ " If this functionality is unavailable, using it raises an exception."); #define OS_STATVFS_METHODDEF \ - {"statvfs", (PyCFunction)os_statvfs, METH_VARARGS|METH_KEYWORDS, os_statvfs__doc__}, + {"statvfs", (PyCFunction)os_statvfs, METH_FASTCALL, os_statvfs__doc__}, static PyObject * os_statvfs_impl(PyObject *module, path_t *path); static PyObject * -os_statvfs(PyObject *module, PyObject *args, PyObject *kwargs) +os_statvfs(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", NULL}; static _PyArg_Parser _parser = {"O&:statvfs", _keywords, 0}; path_t path = PATH_T_INITIALIZE("statvfs", "path", 0, PATH_HAVE_FSTATVFS); - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path)) { goto exit; } @@ -4794,20 +4794,20 @@ "Return disk usage statistics about the given path as a (total, free) tuple."); #define OS__GETDISKUSAGE_METHODDEF \ - {"_getdiskusage", (PyCFunction)os__getdiskusage, METH_VARARGS|METH_KEYWORDS, os__getdiskusage__doc__}, + {"_getdiskusage", (PyCFunction)os__getdiskusage, METH_FASTCALL, os__getdiskusage__doc__}, static PyObject * os__getdiskusage_impl(PyObject *module, Py_UNICODE *path); static PyObject * -os__getdiskusage(PyObject *module, PyObject *args, PyObject *kwargs) +os__getdiskusage(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", NULL}; static _PyArg_Parser _parser = {"u:_getdiskusage", _keywords, 0}; Py_UNICODE *path; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &path)) { goto exit; } @@ -4872,13 +4872,13 @@ " If this functionality is unavailable, using it raises an exception."); #define OS_PATHCONF_METHODDEF \ - {"pathconf", (PyCFunction)os_pathconf, METH_VARARGS|METH_KEYWORDS, os_pathconf__doc__}, + {"pathconf", (PyCFunction)os_pathconf, METH_FASTCALL, os_pathconf__doc__}, static long os_pathconf_impl(PyObject *module, path_t *path, int name); static PyObject * -os_pathconf(PyObject *module, PyObject *args, PyObject *kwargs) +os_pathconf(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "name", NULL}; @@ -4887,7 +4887,7 @@ int name; long _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, conv_path_confname, &name)) { goto exit; } @@ -5020,20 +5020,21 @@ "the underlying Win32 ShellExecute function doesn\'t work if it is."); #define OS_STARTFILE_METHODDEF \ - {"startfile", (PyCFunction)os_startfile, METH_VARARGS|METH_KEYWORDS, os_startfile__doc__}, + {"startfile", (PyCFunction)os_startfile, METH_FASTCALL, os_startfile__doc__}, static PyObject * os_startfile_impl(PyObject *module, path_t *filepath, Py_UNICODE *operation); static PyObject * -os_startfile(PyObject *module, PyObject *args, PyObject *kwargs) +os_startfile(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; - static char *_keywords[] = {"filepath", "operation", NULL}; + static const char * const _keywords[] = {"filepath", "operation", NULL}; + static _PyArg_Parser _parser = {"O&|u:startfile", _keywords, 0}; path_t filepath = PATH_T_INITIALIZE("startfile", "filepath", 0, 0); Py_UNICODE *operation = NULL; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&|u:startfile", _keywords, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &filepath, &operation)) { goto exit; } @@ -5084,20 +5085,20 @@ "If the device is not a terminal, return None."); #define OS_DEVICE_ENCODING_METHODDEF \ - {"device_encoding", (PyCFunction)os_device_encoding, METH_VARARGS|METH_KEYWORDS, os_device_encoding__doc__}, + {"device_encoding", (PyCFunction)os_device_encoding, METH_FASTCALL, os_device_encoding__doc__}, static PyObject * os_device_encoding_impl(PyObject *module, int fd); static PyObject * -os_device_encoding(PyObject *module, PyObject *args, PyObject *kwargs) +os_device_encoding(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"fd", NULL}; static _PyArg_Parser _parser = {"i:device_encoding", _keywords, 0}; int fd; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &fd)) { goto exit; } @@ -5233,14 +5234,14 @@ " the link points to."); #define OS_GETXATTR_METHODDEF \ - {"getxattr", (PyCFunction)os_getxattr, METH_VARARGS|METH_KEYWORDS, os_getxattr__doc__}, + {"getxattr", (PyCFunction)os_getxattr, METH_FASTCALL, os_getxattr__doc__}, static PyObject * os_getxattr_impl(PyObject *module, path_t *path, path_t *attribute, int follow_symlinks); static PyObject * -os_getxattr(PyObject *module, PyObject *args, PyObject *kwargs) +os_getxattr(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "attribute", "follow_symlinks", NULL}; @@ -5249,7 +5250,7 @@ path_t attribute = PATH_T_INITIALIZE("getxattr", "attribute", 0, 0); int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, path_converter, &attribute, &follow_symlinks)) { goto exit; } @@ -5281,14 +5282,14 @@ " the link points to."); #define OS_SETXATTR_METHODDEF \ - {"setxattr", (PyCFunction)os_setxattr, METH_VARARGS|METH_KEYWORDS, os_setxattr__doc__}, + {"setxattr", (PyCFunction)os_setxattr, METH_FASTCALL, os_setxattr__doc__}, static PyObject * os_setxattr_impl(PyObject *module, path_t *path, path_t *attribute, Py_buffer *value, int flags, int follow_symlinks); static PyObject * -os_setxattr(PyObject *module, PyObject *args, PyObject *kwargs) +os_setxattr(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "attribute", "value", "flags", "follow_symlinks", NULL}; @@ -5299,7 +5300,7 @@ int flags = 0; int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, path_converter, &attribute, &value, &flags, &follow_symlinks)) { goto exit; } @@ -5334,14 +5335,14 @@ " the link points to."); #define OS_REMOVEXATTR_METHODDEF \ - {"removexattr", (PyCFunction)os_removexattr, METH_VARARGS|METH_KEYWORDS, os_removexattr__doc__}, + {"removexattr", (PyCFunction)os_removexattr, METH_FASTCALL, os_removexattr__doc__}, static PyObject * os_removexattr_impl(PyObject *module, path_t *path, path_t *attribute, int follow_symlinks); static PyObject * -os_removexattr(PyObject *module, PyObject *args, PyObject *kwargs) +os_removexattr(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "attribute", "follow_symlinks", NULL}; @@ -5350,7 +5351,7 @@ path_t attribute = PATH_T_INITIALIZE("removexattr", "attribute", 0, 0); int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, path_converter, &attribute, &follow_symlinks)) { goto exit; } @@ -5382,13 +5383,13 @@ " the link points to."); #define OS_LISTXATTR_METHODDEF \ - {"listxattr", (PyCFunction)os_listxattr, METH_VARARGS|METH_KEYWORDS, os_listxattr__doc__}, + {"listxattr", (PyCFunction)os_listxattr, METH_FASTCALL, os_listxattr__doc__}, static PyObject * os_listxattr_impl(PyObject *module, path_t *path, int follow_symlinks); static PyObject * -os_listxattr(PyObject *module, PyObject *args, PyObject *kwargs) +os_listxattr(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", "follow_symlinks", NULL}; @@ -5396,7 +5397,7 @@ path_t path = PATH_T_INITIALIZE("listxattr", "path", 1, 1); int follow_symlinks = 1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, path_converter, &path, &follow_symlinks)) { goto exit; } @@ -5602,20 +5603,20 @@ "types raise a TypeError."); #define OS_FSPATH_METHODDEF \ - {"fspath", (PyCFunction)os_fspath, METH_VARARGS|METH_KEYWORDS, os_fspath__doc__}, + {"fspath", (PyCFunction)os_fspath, METH_FASTCALL, os_fspath__doc__}, static PyObject * os_fspath_impl(PyObject *module, PyObject *path); static PyObject * -os_fspath(PyObject *module, PyObject *args, PyObject *kwargs) +os_fspath(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"path", NULL}; static _PyArg_Parser _parser = {"O:fspath", _keywords, 0}; PyObject *path; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &path)) { goto exit; } @@ -5634,13 +5635,13 @@ "Obtain a series of random bytes."); #define OS_GETRANDOM_METHODDEF \ - {"getrandom", (PyCFunction)os_getrandom, METH_VARARGS|METH_KEYWORDS, os_getrandom__doc__}, + {"getrandom", (PyCFunction)os_getrandom, METH_FASTCALL, os_getrandom__doc__}, static PyObject * os_getrandom_impl(PyObject *module, Py_ssize_t size, int flags); static PyObject * -os_getrandom(PyObject *module, PyObject *args, PyObject *kwargs) +os_getrandom(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"size", "flags", NULL}; @@ -5648,7 +5649,7 @@ Py_ssize_t size; int flags = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &size, &flags)) { goto exit; } @@ -6139,4 +6140,4 @@ #ifndef OS_GETRANDOM_METHODDEF #define OS_GETRANDOM_METHODDEF #endif /* !defined(OS_GETRANDOM_METHODDEF) */ -/*[clinic end generated code: output=fce51c7d432662c2 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=dfa6bc9d1f2db750 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/pyexpat.c.h b/Modules/clinic/pyexpat.c.h --- a/Modules/clinic/pyexpat.c.h +++ b/Modules/clinic/pyexpat.c.h @@ -233,14 +233,14 @@ "Return a new XML parser object."); #define PYEXPAT_PARSERCREATE_METHODDEF \ - {"ParserCreate", (PyCFunction)pyexpat_ParserCreate, METH_VARARGS|METH_KEYWORDS, pyexpat_ParserCreate__doc__}, + {"ParserCreate", (PyCFunction)pyexpat_ParserCreate, METH_FASTCALL, pyexpat_ParserCreate__doc__}, static PyObject * pyexpat_ParserCreate_impl(PyObject *module, const char *encoding, const char *namespace_separator, PyObject *intern); static PyObject * -pyexpat_ParserCreate(PyObject *module, PyObject *args, PyObject *kwargs) +pyexpat_ParserCreate(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"encoding", "namespace_separator", "intern", NULL}; @@ -249,7 +249,7 @@ const char *namespace_separator = NULL; PyObject *intern = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &encoding, &namespace_separator, &intern)) { goto exit; } @@ -289,4 +289,4 @@ #ifndef PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF #define PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF #endif /* !defined(PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF) */ -/*[clinic end generated code: output=93cfe662f2bc48e5 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=e889f7c6af6cc42f input=a9049054013a1b77]*/ diff --git a/Modules/clinic/sha1module.c.h b/Modules/clinic/sha1module.c.h --- a/Modules/clinic/sha1module.c.h +++ b/Modules/clinic/sha1module.c.h @@ -72,20 +72,20 @@ "Return a new SHA1 hash object; optionally initialized with a string."); #define _SHA1_SHA1_METHODDEF \ - {"sha1", (PyCFunction)_sha1_sha1, METH_VARARGS|METH_KEYWORDS, _sha1_sha1__doc__}, + {"sha1", (PyCFunction)_sha1_sha1, METH_FASTCALL, _sha1_sha1__doc__}, static PyObject * _sha1_sha1_impl(PyObject *module, PyObject *string); static PyObject * -_sha1_sha1(PyObject *module, PyObject *args, PyObject *kwargs) +_sha1_sha1(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", NULL}; static _PyArg_Parser _parser = {"|O:sha1", _keywords, 0}; PyObject *string = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string)) { goto exit; } @@ -94,4 +94,4 @@ exit: return return_value; } -/*[clinic end generated code: output=549a5d08c248337d input=a9049054013a1b77]*/ +/*[clinic end generated code: output=1430450f3f806895 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/sha256module.c.h b/Modules/clinic/sha256module.c.h --- a/Modules/clinic/sha256module.c.h +++ b/Modules/clinic/sha256module.c.h @@ -72,20 +72,20 @@ "Return a new SHA-256 hash object; optionally initialized with a string."); #define _SHA256_SHA256_METHODDEF \ - {"sha256", (PyCFunction)_sha256_sha256, METH_VARARGS|METH_KEYWORDS, _sha256_sha256__doc__}, + {"sha256", (PyCFunction)_sha256_sha256, METH_FASTCALL, _sha256_sha256__doc__}, static PyObject * _sha256_sha256_impl(PyObject *module, PyObject *string); static PyObject * -_sha256_sha256(PyObject *module, PyObject *args, PyObject *kwargs) +_sha256_sha256(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", NULL}; static _PyArg_Parser _parser = {"|O:sha256", _keywords, 0}; PyObject *string = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string)) { goto exit; } @@ -102,20 +102,20 @@ "Return a new SHA-224 hash object; optionally initialized with a string."); #define _SHA256_SHA224_METHODDEF \ - {"sha224", (PyCFunction)_sha256_sha224, METH_VARARGS|METH_KEYWORDS, _sha256_sha224__doc__}, + {"sha224", (PyCFunction)_sha256_sha224, METH_FASTCALL, _sha256_sha224__doc__}, static PyObject * _sha256_sha224_impl(PyObject *module, PyObject *string); static PyObject * -_sha256_sha224(PyObject *module, PyObject *args, PyObject *kwargs) +_sha256_sha224(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", NULL}; static _PyArg_Parser _parser = {"|O:sha224", _keywords, 0}; PyObject *string = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string)) { goto exit; } @@ -124,4 +124,4 @@ exit: return return_value; } -/*[clinic end generated code: output=a1296ba6d0780051 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=19439d70db7ead5c input=a9049054013a1b77]*/ diff --git a/Modules/clinic/sha512module.c.h b/Modules/clinic/sha512module.c.h --- a/Modules/clinic/sha512module.c.h +++ b/Modules/clinic/sha512module.c.h @@ -72,20 +72,20 @@ "Return a new SHA-512 hash object; optionally initialized with a string."); #define _SHA512_SHA512_METHODDEF \ - {"sha512", (PyCFunction)_sha512_sha512, METH_VARARGS|METH_KEYWORDS, _sha512_sha512__doc__}, + {"sha512", (PyCFunction)_sha512_sha512, METH_FASTCALL, _sha512_sha512__doc__}, static PyObject * _sha512_sha512_impl(PyObject *module, PyObject *string); static PyObject * -_sha512_sha512(PyObject *module, PyObject *args, PyObject *kwargs) +_sha512_sha512(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", NULL}; static _PyArg_Parser _parser = {"|O:sha512", _keywords, 0}; PyObject *string = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string)) { goto exit; } @@ -102,20 +102,20 @@ "Return a new SHA-384 hash object; optionally initialized with a string."); #define _SHA512_SHA384_METHODDEF \ - {"sha384", (PyCFunction)_sha512_sha384, METH_VARARGS|METH_KEYWORDS, _sha512_sha384__doc__}, + {"sha384", (PyCFunction)_sha512_sha384, METH_FASTCALL, _sha512_sha384__doc__}, static PyObject * _sha512_sha384_impl(PyObject *module, PyObject *string); static PyObject * -_sha512_sha384(PyObject *module, PyObject *args, PyObject *kwargs) +_sha512_sha384(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"string", NULL}; static _PyArg_Parser _parser = {"|O:sha384", _keywords, 0}; PyObject *string = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &string)) { goto exit; } @@ -124,4 +124,4 @@ exit: return return_value; } -/*[clinic end generated code: output=8f7f6603a9c4e910 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=18f15598c3487045 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/zlibmodule.c.h b/Modules/clinic/zlibmodule.c.h --- a/Modules/clinic/zlibmodule.c.h +++ b/Modules/clinic/zlibmodule.c.h @@ -14,13 +14,13 @@ " Compression level, in 0-9 or -1."); #define ZLIB_COMPRESS_METHODDEF \ - {"compress", (PyCFunction)zlib_compress, METH_VARARGS|METH_KEYWORDS, zlib_compress__doc__}, + {"compress", (PyCFunction)zlib_compress, METH_FASTCALL, zlib_compress__doc__}, static PyObject * zlib_compress_impl(PyObject *module, Py_buffer *data, int level); static PyObject * -zlib_compress(PyObject *module, PyObject *args, PyObject *kwargs) +zlib_compress(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"", "level", NULL}; @@ -28,7 +28,7 @@ Py_buffer data = {NULL, NULL}; int level = Z_DEFAULT_COMPRESSION; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &data, &level)) { goto exit; } @@ -57,14 +57,14 @@ " The initial output buffer size."); #define ZLIB_DECOMPRESS_METHODDEF \ - {"decompress", (PyCFunction)zlib_decompress, METH_VARARGS|METH_KEYWORDS, zlib_decompress__doc__}, + {"decompress", (PyCFunction)zlib_decompress, METH_FASTCALL, zlib_decompress__doc__}, static PyObject * zlib_decompress_impl(PyObject *module, Py_buffer *data, int wbits, Py_ssize_t bufsize); static PyObject * -zlib_decompress(PyObject *module, PyObject *args, PyObject *kwargs) +zlib_decompress(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"", "wbits", "bufsize", NULL}; @@ -73,7 +73,7 @@ int wbits = MAX_WBITS; Py_ssize_t bufsize = DEF_BUF_SIZE; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &data, &wbits, ssize_t_converter, &bufsize)) { goto exit; } @@ -119,14 +119,14 @@ " containing subsequences that are likely to occur in the input data."); #define ZLIB_COMPRESSOBJ_METHODDEF \ - {"compressobj", (PyCFunction)zlib_compressobj, METH_VARARGS|METH_KEYWORDS, zlib_compressobj__doc__}, + {"compressobj", (PyCFunction)zlib_compressobj, METH_FASTCALL, zlib_compressobj__doc__}, static PyObject * zlib_compressobj_impl(PyObject *module, int level, int method, int wbits, int memLevel, int strategy, Py_buffer *zdict); static PyObject * -zlib_compressobj(PyObject *module, PyObject *args, PyObject *kwargs) +zlib_compressobj(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"level", "method", "wbits", "memLevel", "strategy", "zdict", NULL}; @@ -138,7 +138,7 @@ int strategy = Z_DEFAULT_STRATEGY; Py_buffer zdict = {NULL, NULL}; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &level, &method, &wbits, &memLevel, &strategy, &zdict)) { goto exit; } @@ -166,13 +166,13 @@ " dictionary as used by the compressor that produced the input data."); #define ZLIB_DECOMPRESSOBJ_METHODDEF \ - {"decompressobj", (PyCFunction)zlib_decompressobj, METH_VARARGS|METH_KEYWORDS, zlib_decompressobj__doc__}, + {"decompressobj", (PyCFunction)zlib_decompressobj, METH_FASTCALL, zlib_decompressobj__doc__}, static PyObject * zlib_decompressobj_impl(PyObject *module, int wbits, PyObject *zdict); static PyObject * -zlib_decompressobj(PyObject *module, PyObject *args, PyObject *kwargs) +zlib_decompressobj(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"wbits", "zdict", NULL}; @@ -180,7 +180,7 @@ int wbits = MAX_WBITS; PyObject *zdict = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &wbits, &zdict)) { goto exit; } @@ -247,14 +247,14 @@ "Call the flush() method to clear these buffers."); #define ZLIB_DECOMPRESS_DECOMPRESS_METHODDEF \ - {"decompress", (PyCFunction)zlib_Decompress_decompress, METH_VARARGS|METH_KEYWORDS, zlib_Decompress_decompress__doc__}, + {"decompress", (PyCFunction)zlib_Decompress_decompress, METH_FASTCALL, zlib_Decompress_decompress__doc__}, static PyObject * zlib_Decompress_decompress_impl(compobject *self, Py_buffer *data, Py_ssize_t max_length); static PyObject * -zlib_Decompress_decompress(compobject *self, PyObject *args, PyObject *kwargs) +zlib_Decompress_decompress(compobject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"", "max_length", NULL}; @@ -262,7 +262,7 @@ Py_buffer data = {NULL, NULL}; Py_ssize_t max_length = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &data, ssize_t_converter, &max_length)) { goto exit; } @@ -467,4 +467,4 @@ #ifndef ZLIB_COMPRESS_COPY_METHODDEF #define ZLIB_COMPRESS_COPY_METHODDEF #endif /* !defined(ZLIB_COMPRESS_COPY_METHODDEF) */ -/*[clinic end generated code: output=48911ef429b65903 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=3a4e2bfe750423a3 input=a9049054013a1b77]*/ diff --git a/Objects/clinic/bytearrayobject.c.h b/Objects/clinic/bytearrayobject.c.h --- a/Objects/clinic/bytearrayobject.c.h +++ b/Objects/clinic/bytearrayobject.c.h @@ -51,14 +51,14 @@ "The remaining characters are mapped through the given translation table."); #define BYTEARRAY_TRANSLATE_METHODDEF \ - {"translate", (PyCFunction)bytearray_translate, METH_VARARGS|METH_KEYWORDS, bytearray_translate__doc__}, + {"translate", (PyCFunction)bytearray_translate, METH_FASTCALL, bytearray_translate__doc__}, static PyObject * bytearray_translate_impl(PyByteArrayObject *self, PyObject *table, PyObject *deletechars); static PyObject * -bytearray_translate(PyByteArrayObject *self, PyObject *args, PyObject *kwargs) +bytearray_translate(PyByteArrayObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"", "delete", NULL}; @@ -66,7 +66,7 @@ PyObject *table; PyObject *deletechars = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &table, &deletechars)) { goto exit; } @@ -181,14 +181,14 @@ " -1 (the default value) means no limit."); #define BYTEARRAY_SPLIT_METHODDEF \ - {"split", (PyCFunction)bytearray_split, METH_VARARGS|METH_KEYWORDS, bytearray_split__doc__}, + {"split", (PyCFunction)bytearray_split, METH_FASTCALL, bytearray_split__doc__}, static PyObject * bytearray_split_impl(PyByteArrayObject *self, PyObject *sep, Py_ssize_t maxsplit); static PyObject * -bytearray_split(PyByteArrayObject *self, PyObject *args, PyObject *kwargs) +bytearray_split(PyByteArrayObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"sep", "maxsplit", NULL}; @@ -196,7 +196,7 @@ PyObject *sep = Py_None; Py_ssize_t maxsplit = -1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &sep, &maxsplit)) { goto exit; } @@ -255,14 +255,14 @@ "Splitting is done starting at the end of the bytearray and working to the front."); #define BYTEARRAY_RSPLIT_METHODDEF \ - {"rsplit", (PyCFunction)bytearray_rsplit, METH_VARARGS|METH_KEYWORDS, bytearray_rsplit__doc__}, + {"rsplit", (PyCFunction)bytearray_rsplit, METH_FASTCALL, bytearray_rsplit__doc__}, static PyObject * bytearray_rsplit_impl(PyByteArrayObject *self, PyObject *sep, Py_ssize_t maxsplit); static PyObject * -bytearray_rsplit(PyByteArrayObject *self, PyObject *args, PyObject *kwargs) +bytearray_rsplit(PyByteArrayObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"sep", "maxsplit", NULL}; @@ -270,7 +270,7 @@ PyObject *sep = Py_None; Py_ssize_t maxsplit = -1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &sep, &maxsplit)) { goto exit; } @@ -547,14 +547,14 @@ " can handle UnicodeDecodeErrors."); #define BYTEARRAY_DECODE_METHODDEF \ - {"decode", (PyCFunction)bytearray_decode, METH_VARARGS|METH_KEYWORDS, bytearray_decode__doc__}, + {"decode", (PyCFunction)bytearray_decode, METH_FASTCALL, bytearray_decode__doc__}, static PyObject * bytearray_decode_impl(PyByteArrayObject *self, const char *encoding, const char *errors); static PyObject * -bytearray_decode(PyByteArrayObject *self, PyObject *args, PyObject *kwargs) +bytearray_decode(PyByteArrayObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"encoding", "errors", NULL}; @@ -562,7 +562,7 @@ const char *encoding = NULL; const char *errors = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &encoding, &errors)) { goto exit; } @@ -595,20 +595,20 @@ "true."); #define BYTEARRAY_SPLITLINES_METHODDEF \ - {"splitlines", (PyCFunction)bytearray_splitlines, METH_VARARGS|METH_KEYWORDS, bytearray_splitlines__doc__}, + {"splitlines", (PyCFunction)bytearray_splitlines, METH_FASTCALL, bytearray_splitlines__doc__}, static PyObject * bytearray_splitlines_impl(PyByteArrayObject *self, int keepends); static PyObject * -bytearray_splitlines(PyByteArrayObject *self, PyObject *args, PyObject *kwargs) +bytearray_splitlines(PyByteArrayObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"keepends", NULL}; static _PyArg_Parser _parser = {"|i:splitlines", _keywords, 0}; int keepends = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &keepends)) { goto exit; } @@ -711,4 +711,4 @@ { return bytearray_sizeof_impl(self); } -/*[clinic end generated code: output=59a0c86b29ff06d1 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=225342a680391b9c input=a9049054013a1b77]*/ diff --git a/Objects/clinic/bytesobject.c.h b/Objects/clinic/bytesobject.c.h --- a/Objects/clinic/bytesobject.c.h +++ b/Objects/clinic/bytesobject.c.h @@ -17,13 +17,13 @@ " -1 (the default value) means no limit."); #define BYTES_SPLIT_METHODDEF \ - {"split", (PyCFunction)bytes_split, METH_VARARGS|METH_KEYWORDS, bytes_split__doc__}, + {"split", (PyCFunction)bytes_split, METH_FASTCALL, bytes_split__doc__}, static PyObject * bytes_split_impl(PyBytesObject *self, PyObject *sep, Py_ssize_t maxsplit); static PyObject * -bytes_split(PyBytesObject *self, PyObject *args, PyObject *kwargs) +bytes_split(PyBytesObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"sep", "maxsplit", NULL}; @@ -31,7 +31,7 @@ PyObject *sep = Py_None; Py_ssize_t maxsplit = -1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &sep, &maxsplit)) { goto exit; } @@ -136,13 +136,13 @@ "Splitting is done starting at the end of the bytes and working to the front."); #define BYTES_RSPLIT_METHODDEF \ - {"rsplit", (PyCFunction)bytes_rsplit, METH_VARARGS|METH_KEYWORDS, bytes_rsplit__doc__}, + {"rsplit", (PyCFunction)bytes_rsplit, METH_FASTCALL, bytes_rsplit__doc__}, static PyObject * bytes_rsplit_impl(PyBytesObject *self, PyObject *sep, Py_ssize_t maxsplit); static PyObject * -bytes_rsplit(PyBytesObject *self, PyObject *args, PyObject *kwargs) +bytes_rsplit(PyBytesObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"sep", "maxsplit", NULL}; @@ -150,7 +150,7 @@ PyObject *sep = Py_None; Py_ssize_t maxsplit = -1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &sep, &maxsplit)) { goto exit; } @@ -281,14 +281,14 @@ "The remaining characters are mapped through the given translation table."); #define BYTES_TRANSLATE_METHODDEF \ - {"translate", (PyCFunction)bytes_translate, METH_VARARGS|METH_KEYWORDS, bytes_translate__doc__}, + {"translate", (PyCFunction)bytes_translate, METH_FASTCALL, bytes_translate__doc__}, static PyObject * bytes_translate_impl(PyBytesObject *self, PyObject *table, PyObject *deletechars); static PyObject * -bytes_translate(PyBytesObject *self, PyObject *args, PyObject *kwargs) +bytes_translate(PyBytesObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"", "delete", NULL}; @@ -296,7 +296,7 @@ PyObject *table; PyObject *deletechars = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &table, &deletechars)) { goto exit; } @@ -412,14 +412,14 @@ " can handle UnicodeDecodeErrors."); #define BYTES_DECODE_METHODDEF \ - {"decode", (PyCFunction)bytes_decode, METH_VARARGS|METH_KEYWORDS, bytes_decode__doc__}, + {"decode", (PyCFunction)bytes_decode, METH_FASTCALL, bytes_decode__doc__}, static PyObject * bytes_decode_impl(PyBytesObject *self, const char *encoding, const char *errors); static PyObject * -bytes_decode(PyBytesObject *self, PyObject *args, PyObject *kwargs) +bytes_decode(PyBytesObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"encoding", "errors", NULL}; @@ -427,7 +427,7 @@ const char *encoding = NULL; const char *errors = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &encoding, &errors)) { goto exit; } @@ -447,20 +447,20 @@ "true."); #define BYTES_SPLITLINES_METHODDEF \ - {"splitlines", (PyCFunction)bytes_splitlines, METH_VARARGS|METH_KEYWORDS, bytes_splitlines__doc__}, + {"splitlines", (PyCFunction)bytes_splitlines, METH_FASTCALL, bytes_splitlines__doc__}, static PyObject * bytes_splitlines_impl(PyBytesObject *self, int keepends); static PyObject * -bytes_splitlines(PyBytesObject *self, PyObject *args, PyObject *kwargs) +bytes_splitlines(PyBytesObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"keepends", NULL}; static _PyArg_Parser _parser = {"|i:splitlines", _keywords, 0}; int keepends = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &keepends)) { goto exit; } @@ -499,4 +499,4 @@ exit: return return_value; } -/*[clinic end generated code: output=5618c05c24c1e617 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=2dc3c93cfd2dc440 input=a9049054013a1b77]*/ diff --git a/PC/clinic/winreg.c.h b/PC/clinic/winreg.c.h --- a/PC/clinic/winreg.c.h +++ b/PC/clinic/winreg.c.h @@ -77,14 +77,14 @@ "\n"); #define WINREG_HKEYTYPE___EXIT___METHODDEF \ - {"__exit__", (PyCFunction)winreg_HKEYType___exit__, METH_VARARGS|METH_KEYWORDS, winreg_HKEYType___exit____doc__}, + {"__exit__", (PyCFunction)winreg_HKEYType___exit__, METH_FASTCALL, winreg_HKEYType___exit____doc__}, static PyObject * winreg_HKEYType___exit___impl(PyHKEYObject *self, PyObject *exc_type, PyObject *exc_value, PyObject *traceback); static PyObject * -winreg_HKEYType___exit__(PyHKEYObject *self, PyObject *args, PyObject *kwargs) +winreg_HKEYType___exit__(PyHKEYObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"exc_type", "exc_value", "traceback", NULL}; @@ -93,7 +93,7 @@ PyObject *exc_value; PyObject *traceback; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &exc_type, &exc_value, &traceback)) { goto exit; } @@ -235,14 +235,14 @@ "If the function fails, an OSError exception is raised."); #define WINREG_CREATEKEYEX_METHODDEF \ - {"CreateKeyEx", (PyCFunction)winreg_CreateKeyEx, METH_VARARGS|METH_KEYWORDS, winreg_CreateKeyEx__doc__}, + {"CreateKeyEx", (PyCFunction)winreg_CreateKeyEx, METH_FASTCALL, winreg_CreateKeyEx__doc__}, static HKEY winreg_CreateKeyEx_impl(PyObject *module, HKEY key, Py_UNICODE *sub_key, int reserved, REGSAM access); static PyObject * -winreg_CreateKeyEx(PyObject *module, PyObject *args, PyObject *kwargs) +winreg_CreateKeyEx(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"key", "sub_key", "reserved", "access", NULL}; @@ -253,7 +253,7 @@ REGSAM access = KEY_WRITE; HKEY _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, clinic_HKEY_converter, &key, &sub_key, &reserved, &access)) { goto exit; } @@ -334,14 +334,14 @@ "On unsupported Windows versions, NotImplementedError is raised."); #define WINREG_DELETEKEYEX_METHODDEF \ - {"DeleteKeyEx", (PyCFunction)winreg_DeleteKeyEx, METH_VARARGS|METH_KEYWORDS, winreg_DeleteKeyEx__doc__}, + {"DeleteKeyEx", (PyCFunction)winreg_DeleteKeyEx, METH_FASTCALL, winreg_DeleteKeyEx__doc__}, static PyObject * winreg_DeleteKeyEx_impl(PyObject *module, HKEY key, Py_UNICODE *sub_key, REGSAM access, int reserved); static PyObject * -winreg_DeleteKeyEx(PyObject *module, PyObject *args, PyObject *kwargs) +winreg_DeleteKeyEx(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"key", "sub_key", "access", "reserved", NULL}; @@ -351,7 +351,7 @@ REGSAM access = KEY_WOW64_64KEY; int reserved = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, clinic_HKEY_converter, &key, &sub_key, &access, &reserved)) { goto exit; } @@ -620,14 +620,14 @@ "If the function fails, an OSError exception is raised."); #define WINREG_OPENKEY_METHODDEF \ - {"OpenKey", (PyCFunction)winreg_OpenKey, METH_VARARGS|METH_KEYWORDS, winreg_OpenKey__doc__}, + {"OpenKey", (PyCFunction)winreg_OpenKey, METH_FASTCALL, winreg_OpenKey__doc__}, static HKEY winreg_OpenKey_impl(PyObject *module, HKEY key, Py_UNICODE *sub_key, int reserved, REGSAM access); static PyObject * -winreg_OpenKey(PyObject *module, PyObject *args, PyObject *kwargs) +winreg_OpenKey(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"key", "sub_key", "reserved", "access", NULL}; @@ -638,7 +638,7 @@ REGSAM access = KEY_READ; HKEY _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, clinic_HKEY_converter, &key, &sub_key, &reserved, &access)) { goto exit; } @@ -672,14 +672,14 @@ "If the function fails, an OSError exception is raised."); #define WINREG_OPENKEYEX_METHODDEF \ - {"OpenKeyEx", (PyCFunction)winreg_OpenKeyEx, METH_VARARGS|METH_KEYWORDS, winreg_OpenKeyEx__doc__}, + {"OpenKeyEx", (PyCFunction)winreg_OpenKeyEx, METH_FASTCALL, winreg_OpenKeyEx__doc__}, static HKEY winreg_OpenKeyEx_impl(PyObject *module, HKEY key, Py_UNICODE *sub_key, int reserved, REGSAM access); static PyObject * -winreg_OpenKeyEx(PyObject *module, PyObject *args, PyObject *kwargs) +winreg_OpenKeyEx(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"key", "sub_key", "reserved", "access", NULL}; @@ -690,7 +690,7 @@ REGSAM access = KEY_READ; HKEY _return_value; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, clinic_HKEY_converter, &key, &sub_key, &reserved, &access)) { goto exit; } @@ -1091,4 +1091,4 @@ exit: return return_value; } -/*[clinic end generated code: output=5b53d19cbe3f37cd input=a9049054013a1b77]*/ +/*[clinic end generated code: output=16dd06be6e14b86e input=a9049054013a1b77]*/ diff --git a/PC/clinic/winsound.c.h b/PC/clinic/winsound.c.h --- a/PC/clinic/winsound.c.h +++ b/PC/clinic/winsound.c.h @@ -14,13 +14,13 @@ " Flag values, ored together. See module documentation."); #define WINSOUND_PLAYSOUND_METHODDEF \ - {"PlaySound", (PyCFunction)winsound_PlaySound, METH_VARARGS|METH_KEYWORDS, winsound_PlaySound__doc__}, + {"PlaySound", (PyCFunction)winsound_PlaySound, METH_FASTCALL, winsound_PlaySound__doc__}, static PyObject * winsound_PlaySound_impl(PyObject *module, PyObject *sound, int flags); static PyObject * -winsound_PlaySound(PyObject *module, PyObject *args, PyObject *kwargs) +winsound_PlaySound(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"sound", "flags", NULL}; @@ -28,7 +28,7 @@ PyObject *sound; int flags; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &sound, &flags)) { goto exit; } @@ -51,13 +51,13 @@ " How long the sound should play, in milliseconds."); #define WINSOUND_BEEP_METHODDEF \ - {"Beep", (PyCFunction)winsound_Beep, METH_VARARGS|METH_KEYWORDS, winsound_Beep__doc__}, + {"Beep", (PyCFunction)winsound_Beep, METH_FASTCALL, winsound_Beep__doc__}, static PyObject * winsound_Beep_impl(PyObject *module, int frequency, int duration); static PyObject * -winsound_Beep(PyObject *module, PyObject *args, PyObject *kwargs) +winsound_Beep(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"frequency", "duration", NULL}; @@ -65,7 +65,7 @@ int frequency; int duration; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &frequency, &duration)) { goto exit; } @@ -84,20 +84,20 @@ "x defaults to MB_OK."); #define WINSOUND_MESSAGEBEEP_METHODDEF \ - {"MessageBeep", (PyCFunction)winsound_MessageBeep, METH_VARARGS|METH_KEYWORDS, winsound_MessageBeep__doc__}, + {"MessageBeep", (PyCFunction)winsound_MessageBeep, METH_FASTCALL, winsound_MessageBeep__doc__}, static PyObject * winsound_MessageBeep_impl(PyObject *module, int type); static PyObject * -winsound_MessageBeep(PyObject *module, PyObject *args, PyObject *kwargs) +winsound_MessageBeep(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"type", NULL}; static _PyArg_Parser _parser = {"|i:MessageBeep", _keywords, 0}; int type = MB_OK; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &type)) { goto exit; } @@ -106,4 +106,4 @@ exit: return return_value; } -/*[clinic end generated code: output=40b3d3ef2faefb15 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=bfe16b2b8b490cb1 input=a9049054013a1b77]*/ diff --git a/Python/clinic/bltinmodule.c.h b/Python/clinic/bltinmodule.c.h --- a/Python/clinic/bltinmodule.c.h +++ b/Python/clinic/bltinmodule.c.h @@ -148,7 +148,7 @@ "in addition to any features explicitly specified."); #define BUILTIN_COMPILE_METHODDEF \ - {"compile", (PyCFunction)builtin_compile, METH_VARARGS|METH_KEYWORDS, builtin_compile__doc__}, + {"compile", (PyCFunction)builtin_compile, METH_FASTCALL, builtin_compile__doc__}, static PyObject * builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename, @@ -156,7 +156,7 @@ int optimize); static PyObject * -builtin_compile(PyObject *module, PyObject *args, PyObject *kwargs) +builtin_compile(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"source", "filename", "mode", "flags", "dont_inherit", "optimize", NULL}; @@ -168,7 +168,7 @@ int dont_inherit = 0; int optimize = -1; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &source, PyUnicode_FSDecoder, &filename, &mode, &flags, &dont_inherit, &optimize)) { goto exit; } @@ -674,4 +674,4 @@ exit: return return_value; } -/*[clinic end generated code: output=790cb3d26531dfda input=a9049054013a1b77]*/ +/*[clinic end generated code: output=63483deb75805f7c input=a9049054013a1b77]*/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:24:21 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 03:24:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_dictobject=2Ec=3A_explain_?= =?utf-8?q?why_stringlib_is_used?= Message-ID: <20160910032420.2439.94350.289C8892@psf.io> https://hg.python.org/cpython/rev/307549843823 changeset: 103544:307549843823 user: Victor Stinner date: Fri Sep 09 20:22:59 2016 -0700 summary: dictobject.c: explain why stringlib is used files: Objects/dictobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -111,7 +111,7 @@ #include "Python.h" #include "dict-common.h" -#include "stringlib/eq.h" +#include "stringlib/eq.h" /* to get unicode_eq() */ /*[clinic input] class dict "PyDictObject *" "&PyDict_Type" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:45:45 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sat, 10 Sep 2016 03:45:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_export_of_size=5Ft_par?= =?utf-8?q?se_stack_function?= Message-ID: <20160910034544.8881.50465.230FC284@psf.io> https://hg.python.org/cpython/rev/3f22cc0ebf16 changeset: 103545:3f22cc0ebf16 user: Benjamin Peterson date: Fri Sep 09 20:45:06 2016 -0700 summary: fix export of size_t parse stack function files: Python/getargs.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Python/getargs.c b/Python/getargs.c --- a/Python/getargs.c +++ b/Python/getargs.c @@ -26,6 +26,8 @@ #ifdef HAVE_DECLSPEC_DLL /* Export functions */ PyAPI_FUNC(int) _PyArg_Parse_SizeT(PyObject *, const char *, ...); +PyAPI_FUNC(int) _PyArg_ParseStack_SizeT(PyObject **args, Py_ssize_t nargs, PyObject *kwnames, + struct _PyArg_Parser *parser, ...); PyAPI_FUNC(int) _PyArg_ParseTuple_SizeT(PyObject *, const char *, ...); PyAPI_FUNC(int) _PyArg_ParseTupleAndKeywords_SizeT(PyObject *, PyObject *, const char *, char **, ...); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Fri Sep 9 23:57:24 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 03:57:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327810=3A_Fix_geta?= =?utf-8?q?rgs=2Ec_compilation_on_Windows?= Message-ID: <20160910035723.36318.74406.0DE055EC@psf.io> https://hg.python.org/cpython/rev/3934e070c9db changeset: 103546:3934e070c9db user: Victor Stinner date: Fri Sep 09 20:56:52 2016 -0700 summary: Issue #27810: Fix getargs.c compilation on Windows files: Python/getargs.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Python/getargs.c b/Python/getargs.c --- a/Python/getargs.c +++ b/Python/getargs.c @@ -35,11 +35,12 @@ PyAPI_FUNC(int) _PyArg_VaParse_SizeT(PyObject *, const char *, va_list); PyAPI_FUNC(int) _PyArg_VaParseTupleAndKeywords_SizeT(PyObject *, PyObject *, const char *, char **, va_list); - PyAPI_FUNC(int) _PyArg_ParseTupleAndKeywordsFast_SizeT(PyObject *, PyObject *, struct _PyArg_Parser *, ...); PyAPI_FUNC(int) _PyArg_VaParseTupleAndKeywordsFast_SizeT(PyObject *, PyObject *, struct _PyArg_Parser *, va_list); +PyAPI_FUNC(int) _PyArg_ParseStack_SizeT(PyObject **args, Py_ssize_t nargs, + PyObject *kwnames, struct _PyArg_Parser *parser, ...); #endif #define FLAG_COMPAT 1 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 00:25:31 2016 From: python-checkins at python.org (r.david.murray) Date: Sat, 10 Sep 2016 04:25:31 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2320476=3A_Deal_with_the_?= =?utf-8?q?message=5Ffactory_circular_import_differently=2E?= Message-ID: <20160910042531.87626.66803.FB8C50E0@psf.io> https://hg.python.org/cpython/rev/8375b8d54bf7 changeset: 103547:8375b8d54bf7 user: R David Murray date: Sat Sep 10 00:22:25 2016 -0400 summary: #20476: Deal with the message_factory circular import differently. It turns out we can't depend on email.message getting imported every place message_factory is needed, so to avoid a circular import we need to special case Policy.message_factory=None in the parser instead of using monkey patching. I had a feeling that was a bad idea when I did it. files: Doc/library/email.policy.rst | 4 ++-- Lib/email/_policybase.py | 2 +- Lib/email/feedparser.py | 6 +++++- Lib/email/message.py | 3 --- Lib/test/test_email/test_policy.py | 2 +- 5 files changed, 9 insertions(+), 8 deletions(-) diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -224,8 +224,8 @@ .. attribute:: message_factory A factory function for constructing a new empty message object. Used - by the parser when building messages. Defaults to - :class:`~email.message.Message`. + by the parser when building messages. Defaults to ``None``, in + which case :class:`~email.message.Message` is used. .. versionadded:: 3.6 diff --git a/Lib/email/_policybase.py b/Lib/email/_policybase.py --- a/Lib/email/_policybase.py +++ b/Lib/email/_policybase.py @@ -155,6 +155,7 @@ serialized by a generator. Default: True. message_factory -- the class to use to create new message objects. + If the value is None, the default is Message. """ @@ -163,7 +164,6 @@ cte_type = '8bit' max_line_length = 78 mangle_from_ = False - # XXX To avoid circular imports, this is set in email.message. message_factory = None def handle_defect(self, obj, defect): diff --git a/Lib/email/feedparser.py b/Lib/email/feedparser.py --- a/Lib/email/feedparser.py +++ b/Lib/email/feedparser.py @@ -147,7 +147,11 @@ self.policy = policy self._old_style_factory = False if _factory is None: - self._factory = policy.message_factory + if policy.message_factory is None: + from email.message import Message + self._factory = Message + else: + self._factory = policy.message_factory else: self._factory = _factory try: diff --git a/Lib/email/message.py b/Lib/email/message.py --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -1162,6 +1162,3 @@ super().set_content(*args, **kw) if 'MIME-Version' not in self: self['MIME-Version'] = '1.0' - -# Set message_factory on Policy here to avoid a circular import. -Policy.message_factory = Message diff --git a/Lib/test/test_email/test_policy.py b/Lib/test/test_email/test_policy.py --- a/Lib/test/test_email/test_policy.py +++ b/Lib/test/test_email/test_policy.py @@ -24,7 +24,7 @@ 'cte_type': '8bit', 'raise_on_defect': False, 'mangle_from_': True, - 'message_factory': email.message.Message, + 'message_factory': None, } # These default values are the ones set on email.policy.default. # If any of these defaults change, the docs must be updated. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 00:39:50 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sat, 10 Sep 2016 04:39:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogVXNlIHJhdyBzdHJpbmcgZm9yIHJlZ2V4cCAoMy41LT4zLjYp?= Message-ID: <20160910043950.69891.3322.9E974AD5@psf.io> https://hg.python.org/cpython/rev/c73daf1dcc0c changeset: 103549:c73daf1dcc0c parent: 103547:8375b8d54bf7 parent: 103548:427ec71e8b0e user: Guido van Rossum date: Fri Sep 09 21:39:36 2016 -0700 summary: Use raw string for regexp (3.5->3.6) files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 00:39:50 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sat, 10 Sep 2016 04:39:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Use_raw_string?= =?utf-8?q?_for_regexp?= Message-ID: <20160910043950.19326.54197.D15BFCB2@psf.io> https://hg.python.org/cpython/rev/427ec71e8b0e changeset: 103548:427ec71e8b0e branch: 3.5 parent: 103529:3ffff303df95 user: Guido van Rossum date: Fri Sep 09 21:39:10 2016 -0700 summary: Use raw string for regexp files: Lib/test/test_asyncio/test_streams.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -831,7 +831,7 @@ stream._waiter = asyncio.Future(loop=self.loop) self.assertRegex( repr(stream), - ">") + r">") stream._waiter.set_result(None) self.loop.run_until_complete(stream._waiter) stream._waiter = None -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 00:49:00 2016 From: python-checkins at python.org (lukasz.langa) Date: Sat, 10 Sep 2016 04:49:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Don=27t_run_garbage_collec?= =?utf-8?q?tion_on_interpreter_exit_if_it_was_explicitly_disabled?= Message-ID: <20160910044900.69446.59582.9EE1A88C@psf.io> https://hg.python.org/cpython/rev/5c7eb6da72a3 changeset: 103550:5c7eb6da72a3 user: ?ukasz Langa date: Fri Sep 09 21:47:46 2016 -0700 summary: Don't run garbage collection on interpreter exit if it was explicitly disabled by the user. files: Include/objimpl.h | 3 ++- Modules/gcmodule.c | 9 +++++++++ Python/pylifecycle.c | 6 +++--- 3 files changed, 14 insertions(+), 4 deletions(-) diff --git a/Include/objimpl.h b/Include/objimpl.h --- a/Include/objimpl.h +++ b/Include/objimpl.h @@ -224,11 +224,12 @@ * ========================== */ -/* C equivalent of gc.collect(). */ +/* C equivalent of gc.collect() which ignores the state of gc.enabled. */ PyAPI_FUNC(Py_ssize_t) PyGC_Collect(void); #ifndef Py_LIMITED_API PyAPI_FUNC(Py_ssize_t) _PyGC_CollectNoFail(void); +PyAPI_FUNC(Py_ssize_t) _PyGC_CollectIfEnabled(void); #endif /* Test if a type has a GC head */ diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c --- a/Modules/gcmodule.c +++ b/Modules/gcmodule.c @@ -1597,6 +1597,15 @@ } Py_ssize_t +_PyGC_CollectIfEnabled(void) +{ + if (!enabled) + return 0; + + return PyGC_Collect(); +} + +Py_ssize_t _PyGC_CollectNoFail(void) { Py_ssize_t n; diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -600,12 +600,12 @@ * XXX but I'm unclear on exactly how that one happens. In any case, * XXX I haven't seen a real-life report of either of these. */ - PyGC_Collect(); + _PyGC_CollectIfEnabled(); #ifdef COUNT_ALLOCS /* With COUNT_ALLOCS, it helps to run GC multiple times: each collection might release some types from the type list, so they become garbage. */ - while (PyGC_Collect() > 0) + while (_PyGC_CollectIfEnabled() > 0) /* nothing */; #endif /* Destroy all modules */ @@ -632,7 +632,7 @@ * XXX Python code getting called. */ #if 0 - PyGC_Collect(); + _PyGC_CollectIfEnabled(); #endif /* Disable tracemalloc after all Python objects have been destroyed, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 00:52:33 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 04:52:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Try_to_fix_sizeof_unit_tes?= =?utf-8?q?ts_on_dict?= Message-ID: <20160910045233.19368.96182.757FCF02@psf.io> https://hg.python.org/cpython/rev/c3776dd858f0 changeset: 103551:c3776dd858f0 user: Victor Stinner date: Fri Sep 09 21:51:19 2016 -0700 summary: Try to fix sizeof unit tests on dict Issue #28056 and issue #26058. files: Lib/test/test_ordered_dict.py | 3 ++- Lib/test/test_sys.py | 6 +++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Lib/test/test_ordered_dict.py b/Lib/test/test_ordered_dict.py --- a/Lib/test/test_ordered_dict.py +++ b/Lib/test/test_ordered_dict.py @@ -668,7 +668,8 @@ size = support.calcobjsize check = self.check_sizeof - basicsize = size('n2P3PnPn2P') + 8 + calcsize('2nP2n') + basicsize = size('nQ2P' + '3PnPn2P') + calcsize('2nP2n') + entrysize = calcsize('n2P') p = calcsize('P') nodesize = calcsize('Pn2P') diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -937,9 +937,9 @@ # method-wrapper (descriptor object) check({}.__iter__, size('2P')) # dict - check({}, size('n2P') + 8 + calcsize('2nP2n') + 8 + (8*2//3)*calcsize('n2P')) + check({}, size('nQ2P') + calcsize('2nP2n') + 8 + (8*2//3)*calcsize('n2P')) longdict = {1:1, 2:2, 3:3, 4:4, 5:5, 6:6, 7:7, 8:8} - check(longdict, size('n2P') + 8 + calcsize('2nP2n') + 16 + (16*2//3)*calcsize('n2P')) + check(longdict, size('nQ2P') + calcsize('2nP2n') + 16 + (16*2//3)*calcsize('n2P')) # dictionary-keyview check({}.keys(), size('P')) # dictionary-valueview @@ -1103,7 +1103,7 @@ class newstyleclass(object): pass check(newstyleclass, s) # dict with shared keys - check(newstyleclass().__dict__, size('n2P' + '2nP2n') + 8) + check(newstyleclass().__dict__, size('nQ2P' + '2nP2n')) # unicode # each tuple contains a string and its expected character size # don't put any static strings here, as they may contain -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 01:27:16 2016 From: python-checkins at python.org (lukasz.langa) Date: Sat, 10 Sep 2016 05:27:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318401=3A_pdb_test?= =?utf-8?q?s_don=27t_read_=7E/=2Epdbrc_anymore?= Message-ID: <20160910052716.12434.65954.AAB94F2D@psf.io> https://hg.python.org/cpython/rev/ee0bbfd972de changeset: 103552:ee0bbfd972de user: ?ukasz Langa date: Fri Sep 09 22:21:17 2016 -0700 summary: Issue #18401: pdb tests don't read ~/.pdbrc anymore Patch by Martin Matusiak and Sam Kimbrel. files: Doc/library/pdb.rst | 8 +++- Lib/pdb.py | 22 +++++---- Lib/test/test_pdb.py | 74 +++++++++++++++++++++++-------- Misc/NEWS | 4 + 4 files changed, 78 insertions(+), 30 deletions(-) diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -144,7 +144,7 @@ access further features, you have to do this yourself: .. class:: Pdb(completekey='tab', stdin=None, stdout=None, skip=None, \ - nosigint=False) + nosigint=False, readrc=True) :class:`Pdb` is the debugger class. @@ -160,6 +160,9 @@ This allows you to break into the debugger again by pressing :kbd:`Ctrl-C`. If you want Pdb not to touch the SIGINT handler, set *nosigint* to true. + The *readrc* argument defaults to True and controls whether Pdb will load + .pdbrc files from the filesystem. + Example call to enable tracing with *skip*:: import pdb; pdb.Pdb(skip=['django.*']).set_trace() @@ -171,6 +174,9 @@ The *nosigint* argument. Previously, a SIGINT handler was never set by Pdb. + .. versionadded:: 3.5 + The *readrc* argument. + .. method:: run(statement, globals=None, locals=None) runeval(expression, globals=None, locals=None) runcall(function, *args, **kwds) diff --git a/Lib/pdb.py b/Lib/pdb.py --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -52,7 +52,8 @@ directory, it is read in and executed as if it had been typed at the debugger prompt. This is particularly useful for aliases. If both files exist, the one in the home directory is read first and aliases -defined there can be overridden by the local file. +defined there can be overridden by the local file. This behavior can be +disabled by passing the "readrc=False" argument to the Pdb constructor. Aside from aliases, the debugger is not directly programmable; but it is implemented as a class from which you can derive your own debugger @@ -135,7 +136,7 @@ class Pdb(bdb.Bdb, cmd.Cmd): def __init__(self, completekey='tab', stdin=None, stdout=None, skip=None, - nosigint=False): + nosigint=False, readrc=True): bdb.Bdb.__init__(self, skip=skip) cmd.Cmd.__init__(self, completekey, stdin, stdout) if stdout: @@ -158,18 +159,19 @@ # Read $HOME/.pdbrc and ./.pdbrc self.rcLines = [] - if 'HOME' in os.environ: - envHome = os.environ['HOME'] + if readrc: + if 'HOME' in os.environ: + envHome = os.environ['HOME'] + try: + with open(os.path.join(envHome, ".pdbrc")) as rcFile: + self.rcLines.extend(rcFile) + except OSError: + pass try: - with open(os.path.join(envHome, ".pdbrc")) as rcFile: + with open(".pdbrc") as rcFile: self.rcLines.extend(rcFile) except OSError: pass - try: - with open(".pdbrc") as rcFile: - self.rcLines.extend(rcFile) - except OSError: - pass self.commands = {} # associates a command list to breakpoint numbers self.commands_doprompt = {} # for each bp num, tells if the prompt diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1,8 +1,10 @@ # A test suite for pdb; not very comprehensive at the moment. import doctest +import os import pdb import sys +import tempfile import types import unittest import subprocess @@ -34,7 +36,7 @@ """This tests the custom displayhook for pdb. >>> def test_function(foo, bar): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... pass >>> with PdbTestInput([ @@ -74,7 +76,7 @@ ... return foo.upper() >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... ret = test_function_2('baz') ... print(ret) @@ -173,7 +175,7 @@ """Test basic commands related to breakpoints. >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... print(1) ... print(2) ... print(3) @@ -305,7 +307,7 @@ ... return foo >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... ret = test_function_2('baz') >>> with PdbTestInput([ # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE @@ -328,7 +330,7 @@ -> ret = test_function_2('baz') (Pdb) list 1 def test_function(): - 2 import pdb; pdb.Pdb(nosigint=True).set_trace() + 2 import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() 3 -> ret = test_function_2('baz') [EOF] (Pdb) step @@ -391,7 +393,7 @@ ... print('Exception!') >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... test_function_2() ... print('Not reached.') @@ -424,7 +426,7 @@ -> 1/0 (Pdb) list 1 def test_function(): - 2 import pdb; pdb.Pdb(nosigint=True).set_trace() + 2 import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() 3 -> test_function_2() 4 print('Not reached.') [EOF] @@ -448,7 +450,7 @@ >>> def skip_module(): ... import string - ... import pdb; pdb.Pdb(skip=['stri*'], nosigint=True).set_trace() + ... import pdb; pdb.Pdb(skip=['stri*'], nosigint=True, readrc=False).set_trace() ... string.capwords('FOO') >>> with PdbTestInput([ @@ -477,7 +479,7 @@ >>> def skip_module(): ... def callback(): ... return None - ... import pdb; pdb.Pdb(skip=['module_to_skip*'], nosigint=True).set_trace() + ... import pdb; pdb.Pdb(skip=['module_to_skip*'], nosigint=True, readrc=False).set_trace() ... mod.foo_pony(callback) >>> with PdbTestInput([ @@ -518,7 +520,7 @@ """Test that "continue" and "next" work properly in bottom frame (issue #5294). >>> def test_function(): - ... import pdb, sys; inst = pdb.Pdb(nosigint=True) + ... import pdb, sys; inst = pdb.Pdb(nosigint=True, readrc=False) ... inst.set_trace() ... inst.botframe = sys._getframe() # hackery to get the right botframe ... print(1) @@ -558,7 +560,7 @@ def pdb_invoke(method, arg): """Run pdb.method(arg).""" - getattr(pdb.Pdb(nosigint=True), method)(arg) + getattr(pdb.Pdb(nosigint=True, readrc=False), method)(arg) def test_pdb_run_with_incorrect_argument(): @@ -607,7 +609,7 @@ ... x = 2 >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... test_function_2() ... test_function_2() ... test_function_2() @@ -673,7 +675,7 @@ ... yield 2 >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... it = test_gen() ... try: ... if next(it) != 0: @@ -733,7 +735,7 @@ ... yield 2 >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... it = test_gen() ... try: ... if next(it) != 0: @@ -788,7 +790,7 @@ ... yield 2 >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... for i in test_gen(): ... print(i) ... print("finished") @@ -830,7 +832,7 @@ ... return 1 >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... for i in test_gen(): ... print('value', i) ... x = 123 @@ -875,7 +877,7 @@ ... return x >>> def test_function(): - ... import pdb; pdb.Pdb(nosigint=True).set_trace() + ... import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace() ... for i in test_gen(): ... print('value', i) ... x = 123 @@ -1025,7 +1027,7 @@ import pdb def start_pdb(): - pdb.Pdb().set_trace() + pdb.Pdb(readrc=False).set_trace() x = 1 y = 1 @@ -1054,13 +1056,47 @@ .format(expected, stdout)) + def test_readrc_kwarg(self): + save_home = os.environ['HOME'] + save_dir = os.getcwd() + script = """import pdb; pdb.Pdb(readrc=False).set_trace() + +print('hello') +""" + del os.environ['HOME'] + try: + with tempfile.TemporaryDirectory() as dirname: + os.chdir(dirname) + with open('.pdbrc', 'w') as f: + f.write("invalid\n") + + with open('main.py', 'w') as f: + f.write(script) + + cmd = [sys.executable, 'main.py'] + proc = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stdin=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + self.addCleanup(proc.stdout.close) + self.addCleanup(proc.stderr.close) + stdout, stderr = proc.communicate(b'q\n') + self.assertNotIn("NameError: name 'invalid' is not defined", + stdout.decode()) + + finally: + os.environ['HOME'] = save_home + os.chdir(save_dir) + def tearDown(self): support.unlink(support.TESTFN) def load_tests(*args): from test import test_pdb - suites = [unittest.makeSuite(PdbTestCase), doctest.DocTestSuite(test_pdb)] + suites = [unittest.makeSuite(PdbTestCase)] return unittest.TestSuite(suites) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -135,6 +135,10 @@ Library ------- +- Issue #18401: Pdb now supports the 'readrc' keyword argument to control + whether .pdbrc files should be read. Patch by Martin Matusiak and + Sam Kimbrel. + - Issue #25969: Update the lib2to3 grammar to handle the unpacking generalizations added in 3.5. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 01:57:04 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 05:57:04 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318401=3A_Fix_test?= =?utf-8?q?=5Fpdb_if_=24HOME_is_not_set?= Message-ID: <20160910055704.8490.91738.D984A718@psf.io> https://hg.python.org/cpython/rev/09c730db1aac changeset: 103553:09c730db1aac user: Victor Stinner date: Fri Sep 09 22:56:54 2016 -0700 summary: Issue #18401: Fix test_pdb if $HOME is not set HOME is not set on Windows for example. Use also textwrap.dedent() for the script. files: Lib/test/test_pdb.py | 16 ++++++++++------ 1 files changed, 10 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1057,14 +1057,17 @@ def test_readrc_kwarg(self): - save_home = os.environ['HOME'] + save_home = os.environ.get('HOME', None) save_dir = os.getcwd() - script = """import pdb; pdb.Pdb(readrc=False).set_trace() + script = textwrap.dedent(""" + import pdb; pdb.Pdb(readrc=False).set_trace() -print('hello') -""" - del os.environ['HOME'] + print('hello') + """) try: + if save_home is not None: + del os.environ['HOME'] + with tempfile.TemporaryDirectory() as dirname: os.chdir(dirname) with open('.pdbrc', 'w') as f: @@ -1087,7 +1090,8 @@ stdout.decode()) finally: - os.environ['HOME'] = save_home + if save_home is not None: + os.environ['HOME'] = save_home os.chdir(save_dir) def tearDown(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 02:14:51 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 06:14:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_check=5Fforce=5Fascii?= =?utf-8?b?KCk=?= Message-ID: <20160910061450.59726.10889.42B5762E@psf.io> https://hg.python.org/cpython/rev/3b185df3a3e2 changeset: 103554:3b185df3a3e2 user: Victor Stinner date: Fri Sep 09 23:11:52 2016 -0700 summary: Fix check_force_ascii() Issue #27938: Normalize aliases of the ASCII encoding, because _Py_normalize_encoding() now correctly normalize encoding names. files: Python/fileutils.c | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) diff --git a/Python/fileutils.c b/Python/fileutils.c --- a/Python/fileutils.c +++ b/Python/fileutils.c @@ -104,23 +104,24 @@ char *loc; #if defined(HAVE_LANGINFO_H) && defined(CODESET) char *codeset, **alias; - char encoding[100]; + char encoding[20]; /* longest name: "iso_646.irv_1991\0" */ int is_ascii; unsigned int i; char* ascii_aliases[] = { "ascii", + /* Aliases from Lib/encodings/aliases.py */ "646", - "ansi-x3.4-1968", - "ansi-x3-4-1968", - "ansi-x3.4-1986", + "ansi_x3.4_1968", + "ansi_x3.4_1986", + "ansi_x3_4_1968", "cp367", "csascii", "ibm367", - "iso646-us", - "iso-646.irv-1991", - "iso-ir-6", + "iso646_us", + "iso_646.irv_1991", + "iso_ir_6", "us", - "us-ascii", + "us_ascii", NULL }; #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 02:22:30 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 06:22:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2318401=3A_Fix_test?= =?utf-8?q?=5Fpdb_on_Windows?= Message-ID: <20160910062229.59601.38163.7BFCE936@psf.io> https://hg.python.org/cpython/rev/5aa77974dd56 changeset: 103555:5aa77974dd56 user: Victor Stinner date: Fri Sep 09 23:22:09 2016 -0700 summary: Issue #18401: Fix test_pdb on Windows * Use "with Popen" to cleanup properly the process * Use support.temp_cwd() to properly change the working directory * Use environ.pop() to cleanup the code files: Lib/test/test_pdb.py | 21 +++++++-------------- 1 files changed, 7 insertions(+), 14 deletions(-) diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -4,7 +4,6 @@ import os import pdb import sys -import tempfile import types import unittest import subprocess @@ -1057,19 +1056,15 @@ def test_readrc_kwarg(self): - save_home = os.environ.get('HOME', None) - save_dir = os.getcwd() script = textwrap.dedent(""" import pdb; pdb.Pdb(readrc=False).set_trace() print('hello') """) + + save_home = os.environ.pop('HOME', None) try: - if save_home is not None: - del os.environ['HOME'] - - with tempfile.TemporaryDirectory() as dirname: - os.chdir(dirname) + with support.temp_cwd(): with open('.pdbrc', 'w') as f: f.write("invalid\n") @@ -1083,16 +1078,14 @@ stdin=subprocess.PIPE, stderr=subprocess.PIPE, ) - self.addCleanup(proc.stdout.close) - self.addCleanup(proc.stderr.close) - stdout, stderr = proc.communicate(b'q\n') - self.assertNotIn("NameError: name 'invalid' is not defined", - stdout.decode()) + with proc: + stdout, stderr = proc.communicate(b'q\n') + self.assertNotIn("NameError: name 'invalid' is not defined", + stdout.decode()) finally: if save_home is not None: os.environ['HOME'] = save_home - os.chdir(save_dir) def tearDown(self): support.unlink(support.TESTFN) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 02:27:33 2016 From: python-checkins at python.org (berker.peksag) Date: Sat, 10 Sep 2016 06:27:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fix_Python_version_in_pdb?= =?utf-8?q?=2Erst?= Message-ID: <20160910062733.2286.46194.BD0A846C@psf.io> https://hg.python.org/cpython/rev/22103f83b975 changeset: 103556:22103f83b975 user: Berker Peksag date: Sat Sep 10 09:28:03 2016 +0300 summary: Fix Python version in pdb.rst files: Doc/library/pdb.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -160,7 +160,7 @@ This allows you to break into the debugger again by pressing :kbd:`Ctrl-C`. If you want Pdb not to touch the SIGINT handler, set *nosigint* to true. - The *readrc* argument defaults to True and controls whether Pdb will load + The *readrc* argument defaults to true and controls whether Pdb will load .pdbrc files from the filesystem. Example call to enable tracing with *skip*:: @@ -174,7 +174,7 @@ The *nosigint* argument. Previously, a SIGINT handler was never set by Pdb. - .. versionadded:: 3.5 + .. versionchanged:: 3.6 The *readrc* argument. .. method:: run(statement, globals=None, locals=None) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 02:54:40 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 10 Sep 2016 06:54:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI4MDE5?= =?utf-8?q?=3A_Backported_additional_tests_for_itertools=2Ecount=28=29=2E?= Message-ID: <20160910065438.19592.93581.549EE1C3@psf.io> https://hg.python.org/cpython/rev/74667320778e changeset: 103558:74667320778e branch: 2.7 parent: 103528:e96eb2bd0d5e user: Serhiy Storchaka date: Sat Sep 10 09:53:29 2016 +0300 summary: Issue #28019: Backported additional tests for itertools.count(). files: Lib/test/test_itertools.py | 29 +++++++++++++++++++++++-- 1 files changed, 26 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -351,8 +351,16 @@ self.assertEqual(take(2, zip('abc',count(-3))), [('a', -3), ('b', -2)]) self.assertRaises(TypeError, count, 2, 3, 4) self.assertRaises(TypeError, count, 'a') - self.assertEqual(list(islice(count(maxsize-5), 10)), range(maxsize-5, maxsize+5)) - self.assertEqual(list(islice(count(-maxsize-5), 10)), range(-maxsize-5, -maxsize+5)) + self.assertEqual(take(10, count(maxsize-5)), range(maxsize-5, maxsize+5)) + self.assertEqual(take(10, count(-maxsize-5)), range(-maxsize-5, -maxsize+5)) + self.assertEqual(take(3, count(3.25)), [3.25, 4.25, 5.25]) + self.assertEqual(take(3, count(3.25-4j)), [3.25-4j, 4.25-4j, 5.25-4j]) + self.assertEqual(take(3, count(Decimal('1.1'))), + [Decimal('1.1'), Decimal('2.1'), Decimal('3.1')]) + self.assertEqual(take(3, count(Fraction(2, 3))), + [Fraction(2, 3), Fraction(5, 3), Fraction(8, 3)]) + BIGINT = 1<<1000 + self.assertEqual(take(3, count(BIGINT)), [BIGINT, BIGINT+1, BIGINT+2]) c = count(3) self.assertEqual(repr(c), 'count(3)') c.next() @@ -360,8 +368,10 @@ c = count(-9) self.assertEqual(repr(c), 'count(-9)') c.next() + self.assertEqual(next(c), -8) self.assertEqual(repr(count(10.25)), 'count(10.25)') - self.assertEqual(c.next(), -8) + self.assertEqual(repr(count(10.0)), 'count(10.0)') + self.assertEqual(type(next(count(10.0))), float) for i in (-sys.maxint-5, -sys.maxint+5 ,-10, -1, 0, 10, sys.maxint-5, sys.maxint+5): # Test repr (ignoring the L in longs) r1 = repr(count(i)).replace('L', '') @@ -382,15 +392,24 @@ [('a', 2), ('b', 5), ('c', 8)]) self.assertEqual(zip('abc',count(step=-1)), [('a', 0), ('b', -1), ('c', -2)]) + self.assertRaises(TypeError, count, 'a', 'b') self.assertEqual(zip('abc',count(2,0)), [('a', 2), ('b', 2), ('c', 2)]) self.assertEqual(zip('abc',count(2,1)), [('a', 2), ('b', 3), ('c', 4)]) + self.assertEqual(zip('abc',count(2,3)), [('a', 2), ('b', 5), ('c', 8)]) + self.assertEqual(zip('abc',count(2,1L)), [('a', 2), ('b', 3), ('c', 4)]) + self.assertEqual(zip('abc',count(2,3L)), [('a', 2), ('b', 5), ('c', 8)]) self.assertEqual(take(20, count(maxsize-15, 3)), take(20, range(maxsize-15, maxsize+100, 3))) self.assertEqual(take(20, count(-maxsize-15, 3)), take(20, range(-maxsize-15,-maxsize+100, 3))) + self.assertEqual(take(3, count(10, maxsize+5)), + range(10, 10+3*(maxsize+5), maxsize+5)) + self.assertEqual(take(3, count(2, 1.25)), [2, 3.25, 4.5]) self.assertEqual(take(3, count(2, 3.25-4j)), [2, 5.25-4j, 8.5-8j]) self.assertEqual(take(3, count(Decimal('1.1'), Decimal('.1'))), [Decimal('1.1'), Decimal('1.2'), Decimal('1.3')]) self.assertEqual(take(3, count(Fraction(2,3), Fraction(1,7))), [Fraction(2,3), Fraction(17,21), Fraction(20,21)]) + BIGINT = 1<<1000 + self.assertEqual(take(3, count(step=BIGINT)), [0, BIGINT, 2*BIGINT]) self.assertEqual(repr(take(3, count(10, 2.5))), repr([10, 12.5, 15.0])) c = count(3, 5) self.assertEqual(repr(c), 'count(3, 5)') @@ -408,6 +427,10 @@ self.assertEqual(repr(count(10.5, 1.25)), 'count(10.5, 1.25)') self.assertEqual(repr(count(10.5, 1)), 'count(10.5)') # suppress step=1 when it's an int self.assertEqual(repr(count(10.5, 1.00)), 'count(10.5, 1.0)') # do show float values lilke 1.0 + self.assertEqual(repr(count(10, 1.00)), 'count(10, 1.0)') + c = count(10, 1.0) + self.assertEqual(type(next(c)), int) + self.assertEqual(type(next(c)), float) for i in (-sys.maxint-5, -sys.maxint+5 ,-10, -1, 0, 10, sys.maxint-5, sys.maxint+5): for j in (-sys.maxint-5, -sys.maxint+5 ,-10, -1, 0, 1, 10, sys.maxint-5, sys.maxint+5): # Test repr (ignoring the L in longs) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 02:54:40 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 10 Sep 2016 06:54:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4MDE5?= =?utf-8?q?=3A_itertools=2Ecount=28=29_no_longer_rounds_non-integer_step_i?= =?utf-8?q?n_range?= Message-ID: <20160910065438.21363.15688.EA47CA17@psf.io> https://hg.python.org/cpython/rev/b23f963a7b45 changeset: 103557:b23f963a7b45 branch: 3.5 parent: 103548:427ec71e8b0e user: Serhiy Storchaka date: Sat Sep 10 09:49:24 2016 +0300 summary: Issue #28019: itertools.count() no longer rounds non-integer step in range between 1.0 and 2.0 to 1. files: Lib/test/test_itertools.py | 28 ++++++++++++-- Misc/NEWS | 3 + Modules/itertoolsmodule.c | 47 ++++++++++++++++--------- 3 files changed, 56 insertions(+), 22 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -511,12 +511,18 @@ self.assertEqual(take(2, zip('abc',count(-3))), [('a', -3), ('b', -2)]) self.assertRaises(TypeError, count, 2, 3, 4) self.assertRaises(TypeError, count, 'a') - self.assertEqual(list(islice(count(maxsize-5), 10)), + self.assertEqual(take(10, count(maxsize-5)), list(range(maxsize-5, maxsize+5))) - self.assertEqual(list(islice(count(-maxsize-5), 10)), + self.assertEqual(take(10, count(-maxsize-5)), list(range(-maxsize-5, -maxsize+5))) - self.assertEqual(list(islice(count(10, maxsize+5), 3)), - list(range(10, 10+3*(maxsize+5), maxsize+5))) + self.assertEqual(take(3, count(3.25)), [3.25, 4.25, 5.25]) + self.assertEqual(take(3, count(3.25-4j)), [3.25-4j, 4.25-4j, 5.25-4j]) + self.assertEqual(take(3, count(Decimal('1.1'))), + [Decimal('1.1'), Decimal('2.1'), Decimal('3.1')]) + self.assertEqual(take(3, count(Fraction(2, 3))), + [Fraction(2, 3), Fraction(5, 3), Fraction(8, 3)]) + BIGINT = 1<<1000 + self.assertEqual(take(3, count(BIGINT)), [BIGINT, BIGINT+1, BIGINT+2]) c = count(3) self.assertEqual(repr(c), 'count(3)') next(c) @@ -524,8 +530,10 @@ c = count(-9) self.assertEqual(repr(c), 'count(-9)') next(c) + self.assertEqual(next(c), -8) self.assertEqual(repr(count(10.25)), 'count(10.25)') - self.assertEqual(next(c), -8) + self.assertEqual(repr(count(10.0)), 'count(10.0)') + self.assertEqual(type(next(count(10.0))), float) for i in (-sys.maxsize-5, -sys.maxsize+5 ,-10, -1, 0, 10, sys.maxsize-5, sys.maxsize+5): # Test repr r1 = repr(count(i)) @@ -549,16 +557,22 @@ [('a', 2), ('b', 5), ('c', 8)]) self.assertEqual(lzip('abc',count(step=-1)), [('a', 0), ('b', -1), ('c', -2)]) + self.assertRaises(TypeError, count, 'a', 'b') self.assertEqual(lzip('abc',count(2,0)), [('a', 2), ('b', 2), ('c', 2)]) self.assertEqual(lzip('abc',count(2,1)), [('a', 2), ('b', 3), ('c', 4)]) self.assertEqual(lzip('abc',count(2,3)), [('a', 2), ('b', 5), ('c', 8)]) self.assertEqual(take(20, count(maxsize-15, 3)), take(20, range(maxsize-15, maxsize+100, 3))) self.assertEqual(take(20, count(-maxsize-15, 3)), take(20, range(-maxsize-15,-maxsize+100, 3))) + self.assertEqual(take(3, count(10, maxsize+5)), + list(range(10, 10+3*(maxsize+5), maxsize+5))) + self.assertEqual(take(3, count(2, 1.25)), [2, 3.25, 4.5]) self.assertEqual(take(3, count(2, 3.25-4j)), [2, 5.25-4j, 8.5-8j]) self.assertEqual(take(3, count(Decimal('1.1'), Decimal('.1'))), [Decimal('1.1'), Decimal('1.2'), Decimal('1.3')]) self.assertEqual(take(3, count(Fraction(2,3), Fraction(1,7))), [Fraction(2,3), Fraction(17,21), Fraction(20,21)]) + BIGINT = 1<<1000 + self.assertEqual(take(3, count(step=BIGINT)), [0, BIGINT, 2*BIGINT]) self.assertEqual(repr(take(3, count(10, 2.5))), repr([10, 12.5, 15.0])) c = count(3, 5) self.assertEqual(repr(c), 'count(3, 5)') @@ -576,6 +590,10 @@ self.assertEqual(repr(count(10.5, 1.25)), 'count(10.5, 1.25)') self.assertEqual(repr(count(10.5, 1)), 'count(10.5)') # suppress step=1 when it's an int self.assertEqual(repr(count(10.5, 1.00)), 'count(10.5, 1.0)') # do show float values lilke 1.0 + self.assertEqual(repr(count(10, 1.00)), 'count(10, 1.0)') + c = count(10, 1.0) + self.assertEqual(type(next(c)), int) + self.assertEqual(type(next(c)), float) for i in (-sys.maxsize-5, -sys.maxsize+5 ,-10, -1, 0, 10, sys.maxsize-5, sys.maxsize+5): for j in (-sys.maxsize-5, -sys.maxsize+5 ,-10, -1, 0, 1, 10, sys.maxsize-5, sys.maxsize+5): # Test repr diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -65,6 +65,9 @@ Library ------- +- Issue #28019: itertools.count() no longer rounds non-integer step in range + between 1.0 and 2.0 to 1. + - Issue #25969: Update the lib2to3 grammar to handle the unpacking generalizations added in 3.5. diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -3905,7 +3905,7 @@ count_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { countobject *lz; - int slow_mode = 0; + int fast_mode; Py_ssize_t cnt = 0; PyObject *long_cnt = NULL; PyObject *long_step = NULL; @@ -3922,16 +3922,26 @@ return NULL; } + fast_mode = (long_cnt == NULL || PyLong_Check(long_cnt)) && + (long_step == NULL || PyLong_Check(long_step)); + + /* If not specified, start defaults to 0 */ if (long_cnt != NULL) { - cnt = PyLong_AsSsize_t(long_cnt); - if ((cnt == -1 && PyErr_Occurred()) || !PyLong_Check(long_cnt)) { - PyErr_Clear(); - slow_mode = 1; + if (fast_mode) { + assert(PyLong_Check(long_cnt)); + cnt = PyLong_AsSsize_t(long_cnt); + if (cnt == -1 && PyErr_Occurred()) { + PyErr_Clear(); + fast_mode = 0; + } } Py_INCREF(long_cnt); } else { cnt = 0; long_cnt = PyLong_FromLong(0); + if (long_cnt == NULL) { + return NULL; + } } /* If not specified, step defaults to 1 */ @@ -3947,21 +3957,24 @@ assert(long_cnt != NULL && long_step != NULL); /* Fast mode only works when the step is 1 */ - step = PyLong_AsLong(long_step); - if (step != 1) { - slow_mode = 1; - if (step == -1 && PyErr_Occurred()) - PyErr_Clear(); + if (fast_mode) { + assert(PyLong_Check(long_step)); + step = PyLong_AsLong(long_step); + if (step != 1) { + fast_mode = 0; + if (step == -1 && PyErr_Occurred()) + PyErr_Clear(); + } } - if (slow_mode) + if (fast_mode) + Py_CLEAR(long_cnt); + else cnt = PY_SSIZE_T_MAX; - else - Py_CLEAR(long_cnt); - - assert((cnt != PY_SSIZE_T_MAX && long_cnt == NULL && !slow_mode) || - (cnt == PY_SSIZE_T_MAX && long_cnt != NULL && slow_mode)); - assert(slow_mode || + + assert((cnt != PY_SSIZE_T_MAX && long_cnt == NULL && fast_mode) || + (cnt == PY_SSIZE_T_MAX && long_cnt != NULL && !fast_mode)); + assert(!fast_mode || (PyLong_Check(long_step) && PyLong_AS_LONG(long_step) == 1)); /* create countobject structure */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 02:54:40 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 10 Sep 2016 06:54:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2328019=3A_itertools=2Ecount=28=29_no_longer_roun?= =?utf-8?q?ds_non-integer_step_in_range?= Message-ID: <20160910065439.59448.40482.FFCFE79C@psf.io> https://hg.python.org/cpython/rev/51dfab4f28e7 changeset: 103559:51dfab4f28e7 parent: 103556:22103f83b975 parent: 103557:b23f963a7b45 user: Serhiy Storchaka date: Sat Sep 10 09:53:51 2016 +0300 summary: Issue #28019: itertools.count() no longer rounds non-integer step in range between 1.0 and 2.0 to 1. files: Lib/test/test_itertools.py | 28 ++++++++++++-- Misc/NEWS | 3 + Modules/itertoolsmodule.c | 47 ++++++++++++++++--------- 3 files changed, 56 insertions(+), 22 deletions(-) diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -510,12 +510,18 @@ self.assertEqual(take(2, zip('abc',count(-3))), [('a', -3), ('b', -2)]) self.assertRaises(TypeError, count, 2, 3, 4) self.assertRaises(TypeError, count, 'a') - self.assertEqual(list(islice(count(maxsize-5), 10)), + self.assertEqual(take(10, count(maxsize-5)), list(range(maxsize-5, maxsize+5))) - self.assertEqual(list(islice(count(-maxsize-5), 10)), + self.assertEqual(take(10, count(-maxsize-5)), list(range(-maxsize-5, -maxsize+5))) - self.assertEqual(list(islice(count(10, maxsize+5), 3)), - list(range(10, 10+3*(maxsize+5), maxsize+5))) + self.assertEqual(take(3, count(3.25)), [3.25, 4.25, 5.25]) + self.assertEqual(take(3, count(3.25-4j)), [3.25-4j, 4.25-4j, 5.25-4j]) + self.assertEqual(take(3, count(Decimal('1.1'))), + [Decimal('1.1'), Decimal('2.1'), Decimal('3.1')]) + self.assertEqual(take(3, count(Fraction(2, 3))), + [Fraction(2, 3), Fraction(5, 3), Fraction(8, 3)]) + BIGINT = 1<<1000 + self.assertEqual(take(3, count(BIGINT)), [BIGINT, BIGINT+1, BIGINT+2]) c = count(3) self.assertEqual(repr(c), 'count(3)') next(c) @@ -523,8 +529,10 @@ c = count(-9) self.assertEqual(repr(c), 'count(-9)') next(c) + self.assertEqual(next(c), -8) self.assertEqual(repr(count(10.25)), 'count(10.25)') - self.assertEqual(next(c), -8) + self.assertEqual(repr(count(10.0)), 'count(10.0)') + self.assertEqual(type(next(count(10.0))), float) for i in (-sys.maxsize-5, -sys.maxsize+5 ,-10, -1, 0, 10, sys.maxsize-5, sys.maxsize+5): # Test repr r1 = repr(count(i)) @@ -548,16 +556,22 @@ [('a', 2), ('b', 5), ('c', 8)]) self.assertEqual(lzip('abc',count(step=-1)), [('a', 0), ('b', -1), ('c', -2)]) + self.assertRaises(TypeError, count, 'a', 'b') self.assertEqual(lzip('abc',count(2,0)), [('a', 2), ('b', 2), ('c', 2)]) self.assertEqual(lzip('abc',count(2,1)), [('a', 2), ('b', 3), ('c', 4)]) self.assertEqual(lzip('abc',count(2,3)), [('a', 2), ('b', 5), ('c', 8)]) self.assertEqual(take(20, count(maxsize-15, 3)), take(20, range(maxsize-15, maxsize+100, 3))) self.assertEqual(take(20, count(-maxsize-15, 3)), take(20, range(-maxsize-15,-maxsize+100, 3))) + self.assertEqual(take(3, count(10, maxsize+5)), + list(range(10, 10+3*(maxsize+5), maxsize+5))) + self.assertEqual(take(3, count(2, 1.25)), [2, 3.25, 4.5]) self.assertEqual(take(3, count(2, 3.25-4j)), [2, 5.25-4j, 8.5-8j]) self.assertEqual(take(3, count(Decimal('1.1'), Decimal('.1'))), [Decimal('1.1'), Decimal('1.2'), Decimal('1.3')]) self.assertEqual(take(3, count(Fraction(2,3), Fraction(1,7))), [Fraction(2,3), Fraction(17,21), Fraction(20,21)]) + BIGINT = 1<<1000 + self.assertEqual(take(3, count(step=BIGINT)), [0, BIGINT, 2*BIGINT]) self.assertEqual(repr(take(3, count(10, 2.5))), repr([10, 12.5, 15.0])) c = count(3, 5) self.assertEqual(repr(c), 'count(3, 5)') @@ -575,6 +589,10 @@ self.assertEqual(repr(count(10.5, 1.25)), 'count(10.5, 1.25)') self.assertEqual(repr(count(10.5, 1)), 'count(10.5)') # suppress step=1 when it's an int self.assertEqual(repr(count(10.5, 1.00)), 'count(10.5, 1.0)') # do show float values lilke 1.0 + self.assertEqual(repr(count(10, 1.00)), 'count(10, 1.0)') + c = count(10, 1.0) + self.assertEqual(type(next(c)), int) + self.assertEqual(type(next(c)), float) for i in (-sys.maxsize-5, -sys.maxsize+5 ,-10, -1, 0, 10, sys.maxsize-5, sys.maxsize+5): for j in (-sys.maxsize-5, -sys.maxsize+5 ,-10, -1, 0, 1, 10, sys.maxsize-5, sys.maxsize+5): # Test repr diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -135,6 +135,9 @@ Library ------- +- Issue #28019: itertools.count() no longer rounds non-integer step in range + between 1.0 and 2.0 to 1. + - Issue #18401: Pdb now supports the 'readrc' keyword argument to control whether .pdbrc files should be read. Patch by Martin Matusiak and Sam Kimbrel. diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -3907,7 +3907,7 @@ count_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { countobject *lz; - int slow_mode = 0; + int fast_mode; Py_ssize_t cnt = 0; PyObject *long_cnt = NULL; PyObject *long_step = NULL; @@ -3924,16 +3924,26 @@ return NULL; } + fast_mode = (long_cnt == NULL || PyLong_Check(long_cnt)) && + (long_step == NULL || PyLong_Check(long_step)); + + /* If not specified, start defaults to 0 */ if (long_cnt != NULL) { - cnt = PyLong_AsSsize_t(long_cnt); - if ((cnt == -1 && PyErr_Occurred()) || !PyLong_Check(long_cnt)) { - PyErr_Clear(); - slow_mode = 1; + if (fast_mode) { + assert(PyLong_Check(long_cnt)); + cnt = PyLong_AsSsize_t(long_cnt); + if (cnt == -1 && PyErr_Occurred()) { + PyErr_Clear(); + fast_mode = 0; + } } Py_INCREF(long_cnt); } else { cnt = 0; long_cnt = PyLong_FromLong(0); + if (long_cnt == NULL) { + return NULL; + } } /* If not specified, step defaults to 1 */ @@ -3949,21 +3959,24 @@ assert(long_cnt != NULL && long_step != NULL); /* Fast mode only works when the step is 1 */ - step = PyLong_AsLong(long_step); - if (step != 1) { - slow_mode = 1; - if (step == -1 && PyErr_Occurred()) - PyErr_Clear(); + if (fast_mode) { + assert(PyLong_Check(long_step)); + step = PyLong_AsLong(long_step); + if (step != 1) { + fast_mode = 0; + if (step == -1 && PyErr_Occurred()) + PyErr_Clear(); + } } - if (slow_mode) + if (fast_mode) + Py_CLEAR(long_cnt); + else cnt = PY_SSIZE_T_MAX; - else - Py_CLEAR(long_cnt); - - assert((cnt != PY_SSIZE_T_MAX && long_cnt == NULL && !slow_mode) || - (cnt == PY_SSIZE_T_MAX && long_cnt != NULL && slow_mode)); - assert(slow_mode || + + assert((cnt != PY_SSIZE_T_MAX && long_cnt == NULL && fast_mode) || + (cnt == PY_SSIZE_T_MAX && long_cnt != NULL && !fast_mode)); + assert(!fast_mode || (PyLong_Check(long_step) && PyLong_AS_LONG(long_step) == 1)); /* create countobject structure */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 04:07:52 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 08:07:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Show_regrtest_env_changed_?= =?utf-8?q?warn_on_Windows_buildbot?= Message-ID: <20160910080751.18843.25145.CE21E95D@psf.io> https://hg.python.org/cpython/rev/491bbba73bca changeset: 103560:491bbba73bca user: Victor Stinner date: Sat Sep 10 04:07:38 2016 -0400 summary: Show regrtest env changed warn on Windows buildbot Issue #27829: don't pass --quiet option to regrtest to see "Warning -- xxx was modified by ..." warnings. files: Tools/buildbot/test.bat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -3,7 +3,7 @@ setlocal set here=%~dp0 -set rt_opts=-q -d +set rt_opts=-d set regrtest_args=-j1 :CheckOpts -- Repository URL: https://hg.python.org/cpython From tjreedy at udel.edu Sat Sep 10 03:15:15 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 10 Sep 2016 03:15:15 -0400 Subject: [Python-checkins] cpython: Issue #18401: Fix test_pdb if $HOME is not set In-Reply-To: <20160910055704.8490.91738.D984A718@psf.io> References: <20160910055704.8490.91738.D984A718@psf.io> Message-ID: On 9/10/2016 1:57 AM, victor.stinner wrote: > https://hg.python.org/cpython/rev/09c730db1aac > changeset: 103553:09c730db1aac > user: Victor Stinner > date: Fri Sep 09 22:56:54 2016 -0700 > summary: > Issue #18401: Fix test_pdb if $HOME is not set > > HOME is not set on Windows for example. On Windows, HOME is spelled USERPROFILE or ,HOMEDRIVE + HOMEPATH. These are set by default on Windows. The OS specific os.path.expanduser('~') deals with this difference. I don't know what you are testing exactly, but I believe cross-platform modules should use the latter instead of the *NIX-specific HOME. From python-checkins at python.org Sat Sep 10 04:28:39 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 08:28:39 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogcmVncnRlc3Q6IGFj?= =?utf-8?q?cept_options_after_test_names?= Message-ID: <20160910082838.36268.39055.D97B9072@psf.io> https://hg.python.org/cpython/rev/4df2d43e995d changeset: 103562:4df2d43e995d branch: 3.5 parent: 103557:b23f963a7b45 user: Victor Stinner date: Sat Sep 10 04:27:28 2016 -0400 summary: regrtest: accept options after test names files: Lib/test/regrtest.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -325,7 +325,7 @@ group.add_argument('-P', '--pgo', dest='pgo', action='store_true', help='enable Profile Guided Optimization training') - parser.add_argument('args', nargs=argparse.REMAINDER, + parser.add_argument('args', nargs='*', help=argparse.SUPPRESS) return parser -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 04:28:38 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 08:28:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_test=5Feintr=3A_Fix_Resour?= =?utf-8?q?ceWarning_warnings?= Message-ID: <20160910082838.48610.40774.ECD22E95@psf.io> https://hg.python.org/cpython/rev/08094079c36c changeset: 103561:08094079c36c user: Victor Stinner date: Sat Sep 10 04:19:48 2016 -0400 summary: test_eintr: Fix ResourceWarning warnings files: Lib/test/eintrdata/eintr_tester.py | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/Lib/test/eintrdata/eintr_tester.py b/Lib/test/eintrdata/eintr_tester.py --- a/Lib/test/eintrdata/eintr_tester.py +++ b/Lib/test/eintrdata/eintr_tester.py @@ -83,6 +83,9 @@ processes = [self.new_sleep_process() for _ in range(num)] for _ in range(num): wait_func() + # Call the Popen method to avoid a ResourceWarning + for proc in processes: + proc.wait() def test_wait(self): self._test_wait_multiple(os.wait) @@ -94,6 +97,8 @@ def _test_wait_single(self, wait_func): proc = self.new_sleep_process() wait_func(proc.pid) + # Call the Popen method to avoid a ResourceWarning + proc.wait() def test_waitpid(self): self._test_wait_single(lambda pid: os.waitpid(pid, 0)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 04:28:47 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 08:28:47 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChyZWdydGVzdCk=?= Message-ID: <20160910082839.21157.86059.34857E34@psf.io> https://hg.python.org/cpython/rev/b2802d3427a4 changeset: 103563:b2802d3427a4 parent: 103561:08094079c36c parent: 103562:4df2d43e995d user: Victor Stinner date: Sat Sep 10 04:27:56 2016 -0400 summary: Merge 3.5 (regrtest) files: Lib/test/libregrtest/cmdline.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/libregrtest/cmdline.py b/Lib/test/libregrtest/cmdline.py --- a/Lib/test/libregrtest/cmdline.py +++ b/Lib/test/libregrtest/cmdline.py @@ -242,7 +242,7 @@ group.add_argument('-P', '--pgo', dest='pgo', action='store_true', help='enable Profile Guided Optimization training') - parser.add_argument('args', nargs=argparse.REMAINDER, + parser.add_argument('args', nargs='*', help=argparse.SUPPRESS) return parser -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 06:00:14 2016 From: python-checkins at python.org (nick.coghlan) Date: Sat, 10 Sep 2016 10:00:14 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327137=3A_align_Py?= =?utf-8?q?thon_=26_C_implementations_of_functools=2Epartial?= Message-ID: <20160910100014.2686.59463.4D7B875C@psf.io> https://hg.python.org/cpython/rev/cdc91b6ae3b2 changeset: 103564:cdc91b6ae3b2 user: Nick Coghlan date: Sat Sep 10 20:00:02 2016 +1000 summary: Issue #27137: align Python & C implementations of functools.partial The pure Python fallback implementation of functools.partial now matches the behaviour of its accelerated C counterpart for subclassing, pickling and text representation purposes. Patch by Emanuel Barry and Serhiy Storchaka. files: Lib/functools.py | 90 ++++++++++-- Lib/test/test_functools.py | 171 ++++++++++++++---------- Misc/NEWS | 5 + Modules/_functoolsmodule.c | 2 +- 4 files changed, 179 insertions(+), 89 deletions(-) diff --git a/Lib/functools.py b/Lib/functools.py --- a/Lib/functools.py +++ b/Lib/functools.py @@ -21,6 +21,7 @@ from collections import namedtuple from types import MappingProxyType from weakref import WeakKeyDictionary +from reprlib import recursive_repr try: from _thread import RLock except ImportError: @@ -237,26 +238,83 @@ ################################################################################ # Purely functional, no descriptor behaviour -def partial(func, *args, **keywords): +class partial: """New function with partial application of the given arguments and keywords. """ - if hasattr(func, 'func'): - args = func.args + args - tmpkw = func.keywords.copy() - tmpkw.update(keywords) - keywords = tmpkw - del tmpkw - func = func.func - def newfunc(*fargs, **fkeywords): - newkeywords = keywords.copy() - newkeywords.update(fkeywords) - return func(*(args + fargs), **newkeywords) - newfunc.func = func - newfunc.args = args - newfunc.keywords = keywords - return newfunc + __slots__ = "func", "args", "keywords", "__dict__", "__weakref__" + + def __new__(*args, **keywords): + if not args: + raise TypeError("descriptor '__new__' of partial needs an argument") + if len(args) < 2: + raise TypeError("type 'partial' takes at least one argument") + cls, func, *args = args + if not callable(func): + raise TypeError("the first argument must be callable") + args = tuple(args) + + if hasattr(func, "func"): + args = func.args + args + tmpkw = func.keywords.copy() + tmpkw.update(keywords) + keywords = tmpkw + del tmpkw + func = func.func + + self = super(partial, cls).__new__(cls) + + self.func = func + self.args = args + self.keywords = keywords + return self + + def __call__(*args, **keywords): + if not args: + raise TypeError("descriptor '__call__' of partial needs an argument") + self, *args = args + newkeywords = self.keywords.copy() + newkeywords.update(keywords) + return self.func(*self.args, *args, **newkeywords) + + @recursive_repr() + def __repr__(self): + qualname = type(self).__qualname__ + args = [repr(self.func)] + args.extend(repr(x) for x in self.args) + args.extend(f"{k}={v!r}" for (k, v) in self.keywords.items()) + if type(self).__module__ == "functools": + return f"functools.{qualname}({', '.join(args)})" + return f"{qualname}({', '.join(args)})" + + def __reduce__(self): + return type(self), (self.func,), (self.func, self.args, + self.keywords or None, self.__dict__ or None) + + def __setstate__(self, state): + if not isinstance(state, tuple): + raise TypeError("argument to __setstate__ must be a tuple") + if len(state) != 4: + raise TypeError(f"expected 4 items in state, got {len(state)}") + func, args, kwds, namespace = state + if (not callable(func) or not isinstance(args, tuple) or + (kwds is not None and not isinstance(kwds, dict)) or + (namespace is not None and not isinstance(namespace, dict))): + raise TypeError("invalid partial state") + + args = tuple(args) # just in case it's a subclass + if kwds is None: + kwds = {} + elif type(kwds) is not dict: # XXX does it need to be *exactly* dict? + kwds = dict(kwds) + if namespace is None: + namespace = {} + + self.__dict__ = namespace + self.func = func + self.args = args + self.keywords = kwds try: from _functools import partial diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -8,6 +8,7 @@ from test import support import unittest from weakref import proxy +import contextlib try: import threading except ImportError: @@ -20,6 +21,14 @@ decimal = support.import_fresh_module('decimal', fresh=['_decimal']) + at contextlib.contextmanager +def replaced_module(name, replacement): + original_module = sys.modules[name] + sys.modules[name] = replacement + try: + yield + finally: + sys.modules[name] = original_module def capture(*args, **kw): """capture all positional and keyword arguments""" @@ -167,58 +176,35 @@ p2.new_attr = 'spam' self.assertEqual(p2.new_attr, 'spam') - - at unittest.skipUnless(c_functools, 'requires the C _functools module') -class TestPartialC(TestPartial, unittest.TestCase): - if c_functools: - partial = c_functools.partial - - def test_attributes_unwritable(self): - # attributes should not be writable - p = self.partial(capture, 1, 2, a=10, b=20) - self.assertRaises(AttributeError, setattr, p, 'func', map) - self.assertRaises(AttributeError, setattr, p, 'args', (1, 2)) - self.assertRaises(AttributeError, setattr, p, 'keywords', dict(a=1, b=2)) - - p = self.partial(hex) - try: - del p.__dict__ - except TypeError: - pass - else: - self.fail('partial object allowed __dict__ to be deleted') - def test_repr(self): args = (object(), object()) args_repr = ', '.join(repr(a) for a in args) kwargs = {'a': object(), 'b': object()} kwargs_reprs = ['a={a!r}, b={b!r}'.format_map(kwargs), 'b={b!r}, a={a!r}'.format_map(kwargs)] - if self.partial is c_functools.partial: + if self.partial in (c_functools.partial, py_functools.partial): name = 'functools.partial' else: name = self.partial.__name__ f = self.partial(capture) - self.assertEqual('{}({!r})'.format(name, capture), - repr(f)) + self.assertEqual(f'{name}({capture!r})', repr(f)) f = self.partial(capture, *args) - self.assertEqual('{}({!r}, {})'.format(name, capture, args_repr), - repr(f)) + self.assertEqual(f'{name}({capture!r}, {args_repr})', repr(f)) f = self.partial(capture, **kwargs) self.assertIn(repr(f), - ['{}({!r}, {})'.format(name, capture, kwargs_repr) + [f'{name}({capture!r}, {kwargs_repr})' for kwargs_repr in kwargs_reprs]) f = self.partial(capture, *args, **kwargs) self.assertIn(repr(f), - ['{}({!r}, {}, {})'.format(name, capture, args_repr, kwargs_repr) + [f'{name}({capture!r}, {args_repr}, {kwargs_repr})' for kwargs_repr in kwargs_reprs]) def test_recursive_repr(self): - if self.partial is c_functools.partial: + if self.partial in (c_functools.partial, py_functools.partial): name = 'functools.partial' else: name = self.partial.__name__ @@ -226,30 +212,31 @@ f = self.partial(capture) f.__setstate__((f, (), {}, {})) try: - self.assertEqual(repr(f), '%s(%s(...))' % (name, name)) + self.assertEqual(repr(f), '%s(...)' % (name,)) finally: f.__setstate__((capture, (), {}, {})) f = self.partial(capture) f.__setstate__((capture, (f,), {}, {})) try: - self.assertEqual(repr(f), '%s(%r, %s(...))' % (name, capture, name)) + self.assertEqual(repr(f), '%s(%r, ...)' % (name, capture,)) finally: f.__setstate__((capture, (), {}, {})) f = self.partial(capture) f.__setstate__((capture, (), {'a': f}, {})) try: - self.assertEqual(repr(f), '%s(%r, a=%s(...))' % (name, capture, name)) + self.assertEqual(repr(f), '%s(%r, a=...)' % (name, capture,)) finally: f.__setstate__((capture, (), {}, {})) def test_pickle(self): - f = self.partial(signature, ['asdf'], bar=[True]) - f.attr = [] - for proto in range(pickle.HIGHEST_PROTOCOL + 1): - f_copy = pickle.loads(pickle.dumps(f, proto)) - self.assertEqual(signature(f_copy), signature(f)) + with self.AllowPickle(): + f = self.partial(signature, ['asdf'], bar=[True]) + f.attr = [] + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + f_copy = pickle.loads(pickle.dumps(f, proto)) + self.assertEqual(signature(f_copy), signature(f)) def test_copy(self): f = self.partial(signature, ['asdf'], bar=[True]) @@ -274,11 +261,13 @@ def test_setstate(self): f = self.partial(signature) f.__setstate__((capture, (1,), dict(a=10), dict(attr=[]))) + self.assertEqual(signature(f), (capture, (1,), dict(a=10), dict(attr=[]))) self.assertEqual(f(2, b=20), ((1, 2), {'a': 10, 'b': 20})) f.__setstate__((capture, (1,), dict(a=10), None)) + self.assertEqual(signature(f), (capture, (1,), dict(a=10), {})) self.assertEqual(f(2, b=20), ((1, 2), {'a': 10, 'b': 20})) @@ -325,38 +314,39 @@ self.assertIs(type(r[0]), tuple) def test_recursive_pickle(self): - f = self.partial(capture) - f.__setstate__((f, (), {}, {})) - try: - for proto in range(pickle.HIGHEST_PROTOCOL + 1): - with self.assertRaises(RecursionError): - pickle.dumps(f, proto) - finally: - f.__setstate__((capture, (), {}, {})) + with self.AllowPickle(): + f = self.partial(capture) + f.__setstate__((f, (), {}, {})) + try: + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.assertRaises(RecursionError): + pickle.dumps(f, proto) + finally: + f.__setstate__((capture, (), {}, {})) - f = self.partial(capture) - f.__setstate__((capture, (f,), {}, {})) - try: - for proto in range(pickle.HIGHEST_PROTOCOL + 1): - f_copy = pickle.loads(pickle.dumps(f, proto)) - try: - self.assertIs(f_copy.args[0], f_copy) - finally: - f_copy.__setstate__((capture, (), {}, {})) - finally: - f.__setstate__((capture, (), {}, {})) + f = self.partial(capture) + f.__setstate__((capture, (f,), {}, {})) + try: + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + f_copy = pickle.loads(pickle.dumps(f, proto)) + try: + self.assertIs(f_copy.args[0], f_copy) + finally: + f_copy.__setstate__((capture, (), {}, {})) + finally: + f.__setstate__((capture, (), {}, {})) - f = self.partial(capture) - f.__setstate__((capture, (), {'a': f}, {})) - try: - for proto in range(pickle.HIGHEST_PROTOCOL + 1): - f_copy = pickle.loads(pickle.dumps(f, proto)) - try: - self.assertIs(f_copy.keywords['a'], f_copy) - finally: - f_copy.__setstate__((capture, (), {}, {})) - finally: - f.__setstate__((capture, (), {}, {})) + f = self.partial(capture) + f.__setstate__((capture, (), {'a': f}, {})) + try: + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + f_copy = pickle.loads(pickle.dumps(f, proto)) + try: + self.assertIs(f_copy.keywords['a'], f_copy) + finally: + f_copy.__setstate__((capture, (), {}, {})) + finally: + f.__setstate__((capture, (), {}, {})) # Issue 6083: Reference counting bug def test_setstate_refcount(self): @@ -375,24 +365,60 @@ f = self.partial(object) self.assertRaises(TypeError, f.__setstate__, BadSequence()) + at unittest.skipUnless(c_functools, 'requires the C _functools module') +class TestPartialC(TestPartial, unittest.TestCase): + if c_functools: + partial = c_functools.partial + + class AllowPickle: + def __enter__(self): + return self + def __exit__(self, type, value, tb): + return False + + def test_attributes_unwritable(self): + # attributes should not be writable + p = self.partial(capture, 1, 2, a=10, b=20) + self.assertRaises(AttributeError, setattr, p, 'func', map) + self.assertRaises(AttributeError, setattr, p, 'args', (1, 2)) + self.assertRaises(AttributeError, setattr, p, 'keywords', dict(a=1, b=2)) + + p = self.partial(hex) + try: + del p.__dict__ + except TypeError: + pass + else: + self.fail('partial object allowed __dict__ to be deleted') class TestPartialPy(TestPartial, unittest.TestCase): - partial = staticmethod(py_functools.partial) + partial = py_functools.partial + class AllowPickle: + def __init__(self): + self._cm = replaced_module("functools", py_functools) + def __enter__(self): + return self._cm.__enter__() + def __exit__(self, type, value, tb): + return self._cm.__exit__(type, value, tb) if c_functools: - class PartialSubclass(c_functools.partial): + class CPartialSubclass(c_functools.partial): pass +class PyPartialSubclass(py_functools.partial): + pass @unittest.skipUnless(c_functools, 'requires the C _functools module') class TestPartialCSubclass(TestPartialC): if c_functools: - partial = PartialSubclass + partial = CPartialSubclass # partial subclasses are not optimized for nested calls test_nested_optimization = None +class TestPartialPySubclass(TestPartialPy): + partial = PyPartialSubclass class TestPartialMethod(unittest.TestCase): @@ -683,9 +709,10 @@ self.assertEqual(wrapper.attr, 'This is a different test') self.assertEqual(wrapper.dict_attr, f.dict_attr) - + at unittest.skipUnless(c_functools, 'requires the C _functools module') class TestReduce(unittest.TestCase): - func = functools.reduce + if c_functools: + func = c_functools.reduce def test_reduce(self): class Squares: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -135,6 +135,11 @@ Library ------- +- Issue #27137: the pure Python fallback implementation of ``functools.partial`` + now matches the behaviour of its accelerated C counterpart for subclassing, + pickling and text representation purposes. Patch by Emanuel Barry and + Serhiy Storchaka. + - Issue #28019: itertools.count() no longer rounds non-integer step in range between 1.0 and 2.0 to 1. diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c --- a/Modules/_functoolsmodule.c +++ b/Modules/_functoolsmodule.c @@ -229,7 +229,7 @@ if (status != 0) { if (status < 0) return NULL; - return PyUnicode_FromFormat("%s(...)", Py_TYPE(pto)->tp_name); + return PyUnicode_FromString("..."); } arglist = PyUnicode_FromString(""); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 06:16:41 2016 From: python-checkins at python.org (nick.coghlan) Date: Sat, 10 Sep 2016 10:16:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2317909=3A_Accept_b?= =?utf-8?q?inary_input_in_json=2Eloads?= Message-ID: <20160910101641.87704.59048.546E1A18@psf.io> https://hg.python.org/cpython/rev/e9e1bf9ec2ac changeset: 103565:e9e1bf9ec2ac user: Nick Coghlan date: Sat Sep 10 20:16:18 2016 +1000 summary: Issue #17909: Accept binary input in json.loads json.loads (and hence json.load) now support binary input encoded as UTF-8, UTF-16 or UTF-32. Patch by Serhiy Storchaka. files: Doc/library/json.rst | 5 +- Doc/whatsnew/3.6.rst | 8 ++ Lib/json/__init__.py | 50 +++++++++++++++-- Lib/test/test_json/test_decode.py | 4 +- Lib/test/test_json/test_unicode.py | 16 ++++- Misc/NEWS | 3 + 6 files changed, 70 insertions(+), 16 deletions(-) diff --git a/Doc/library/json.rst b/Doc/library/json.rst --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -268,8 +268,9 @@ .. function:: loads(s, *, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw) - Deserialize *s* (a :class:`str` instance containing a JSON document) to a - Python object using this :ref:`conversion table `. + Deserialize *s* (a :class:`str`, :class:`bytes` or :class:`bytearray` + instance containing a JSON document) to a Python object using this + :ref:`conversion table `. The other arguments have the same meaning as in :func:`load`, except *encoding* which is ignored and deprecated. diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -680,6 +680,14 @@ :term:`path-like object`. +json +---- + +:func:`json.load` and :func:`json.loads` now support binary input. Encoded +JSON should be represented using either UTF-8, UTF-16, or UTF-32. +(Contributed by Serhiy Storchaka in :issue:`17909`.) + + os -- diff --git a/Lib/json/__init__.py b/Lib/json/__init__.py --- a/Lib/json/__init__.py +++ b/Lib/json/__init__.py @@ -105,6 +105,7 @@ from .decoder import JSONDecoder, JSONDecodeError from .encoder import JSONEncoder +import codecs _default_encoder = JSONEncoder( skipkeys=False, @@ -240,6 +241,35 @@ _default_decoder = JSONDecoder(object_hook=None, object_pairs_hook=None) +def detect_encoding(b): + bstartswith = b.startswith + if bstartswith((codecs.BOM_UTF32_BE, codecs.BOM_UTF32_LE)): + return 'utf-32' + if bstartswith((codecs.BOM_UTF16_BE, codecs.BOM_UTF16_LE)): + return 'utf-16' + if bstartswith(codecs.BOM_UTF8): + return 'utf-8-sig' + + if len(b) >= 4: + if not b[0]: + # 00 00 -- -- - utf-32-be + # 00 XX -- -- - utf-16-be + return 'utf-16-be' if b[1] else 'utf-32-be' + if not b[1]: + # XX 00 00 00 - utf-32-le + # XX 00 XX XX - utf-16-le + return 'utf-16-le' if b[2] or b[3] else 'utf-32-le' + elif len(b) == 2: + if not b[0]: + # 00 XX - utf-16-be + return 'utf-16-be' + if not b[1]: + # XX 00 - utf-16-le + return 'utf-16-le' + # default + return 'utf-8' + + def load(fp, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw): """Deserialize ``fp`` (a ``.read()``-supporting file-like object containing @@ -270,8 +300,8 @@ def loads(s, *, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw): - """Deserialize ``s`` (a ``str`` instance containing a JSON - document) to a Python object. + """Deserialize ``s`` (a ``str``, ``bytes`` or ``bytearray`` instance + containing a JSON document) to a Python object. ``object_hook`` is an optional function that will be called with the result of any object literal decode (a ``dict``). The return value of @@ -307,12 +337,16 @@ The ``encoding`` argument is ignored and deprecated. """ - if not isinstance(s, str): - raise TypeError('the JSON object must be str, not {!r}'.format( - s.__class__.__name__)) - if s.startswith(u'\ufeff'): - raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)", - s, 0) + if isinstance(s, str): + if s.startswith('\ufeff'): + raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)", + s, 0) + else: + if not isinstance(s, (bytes, bytearray)): + raise TypeError('the JSON object must be str, bytes or bytearray, ' + 'not {!r}'.format(s.__class__.__name__)) + s = s.decode(detect_encoding(s), 'surrogatepass') + if (cls is None and object_hook is None and parse_int is None and parse_float is None and parse_constant is None and object_pairs_hook is None and not kw): diff --git a/Lib/test/test_json/test_decode.py b/Lib/test/test_json/test_decode.py --- a/Lib/test/test_json/test_decode.py +++ b/Lib/test/test_json/test_decode.py @@ -72,10 +72,8 @@ def test_invalid_input_type(self): msg = 'the JSON object must be str' - for value in [1, 3.14, b'bytes', b'\xff\x00', [], {}, None]: + for value in [1, 3.14, [], {}, None]: self.assertRaisesRegex(TypeError, msg, self.loads, value) - with self.assertRaisesRegex(TypeError, msg): - self.json.load(BytesIO(b'[1,2,3]')) def test_string_with_utf8_bom(self): # see #18958 diff --git a/Lib/test/test_json/test_unicode.py b/Lib/test/test_json/test_unicode.py --- a/Lib/test/test_json/test_unicode.py +++ b/Lib/test/test_json/test_unicode.py @@ -1,3 +1,4 @@ +import codecs from collections import OrderedDict from test.test_json import PyTest, CTest @@ -52,9 +53,18 @@ self.assertRaises(TypeError, self.dumps, [b"hi"]) def test_bytes_decode(self): - self.assertRaises(TypeError, self.loads, b'"hi"') - self.assertRaises(TypeError, self.loads, b'["hi"]') - + for encoding, bom in [ + ('utf-8', codecs.BOM_UTF8), + ('utf-16be', codecs.BOM_UTF16_BE), + ('utf-16le', codecs.BOM_UTF16_LE), + ('utf-32be', codecs.BOM_UTF32_BE), + ('utf-32le', codecs.BOM_UTF32_LE), + ]: + data = ["a\xb5\u20ac\U0001d120"] + encoded = self.dumps(data).encode(encoding) + self.assertEqual(self.loads(bom + encoded), data) + self.assertEqual(self.loads(encoded), data) + self.assertRaises(UnicodeDecodeError, self.loads, b'["\x80"]') def test_object_pairs_hook_with_unicode(self): s = '{"xkd":1, "kcw":2, "art":3, "hxm":4, "qrt":5, "pad":6, "hoy":7}' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -135,6 +135,9 @@ Library ------- +- Issue #17909: ``json.load`` and ``json.loads`` now support binary input + encoded as UTF-8, UTF-16 or UTF-32. Patch by Serhiy Storchaka. + - Issue #27137: the pure Python fallback implementation of ``functools.partial`` now matches the behaviour of its accelerated C counterpart for subclassing, pickling and text representation purposes. Patch by Emanuel Barry and -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 06:25:38 2016 From: python-checkins at python.org (victor.stinner) Date: Sat, 10 Sep 2016 10:25:38 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_test=5Fplatform=3A_Save/re?= =?utf-8?q?store_os=2Eenviron_on_Windows?= Message-ID: <20160910102537.20991.85206.1F48A242@psf.io> https://hg.python.org/cpython/rev/3ded89cdea11 changeset: 103566:3ded89cdea11 user: Victor Stinner date: Sat Sep 10 06:24:47 2016 -0400 summary: test_platform: Save/restore os.environ on Windows files: Lib/test/test_platform.py | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -18,6 +18,12 @@ # On Windows, the EXE needs to know where pythonXY.dll is at so we have # to add the directory to the path. if sys.platform == "win32": + def restore_environ(old_env): + os.environ.clear() + os.environ.update(old_env) + + self.addCleanup(restore_environ, dict(os.environ)) + os.environ["Path"] = "{};{}".format( os.path.dirname(sys.executable), os.environ["Path"]) @@ -26,6 +32,7 @@ 'import platform; print(platform.architecture())'] p = subprocess.Popen(cmd, stdout=subprocess.PIPE) return p.communicate() + real = os.path.realpath(sys.executable) link = os.path.abspath(support.TESTFN) os.symlink(real, link) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 06:39:48 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 10 Sep 2016 10:39:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Correct_print?= =?utf-8?q?=28=29_calls_to_print_a_blank_line_in_various_test_cases?= Message-ID: <20160910103948.8829.51277.716BAE47@psf.io> https://hg.python.org/cpython/rev/13f3babb2547 changeset: 103567:13f3babb2547 branch: 2.7 parent: 103558:74667320778e user: Martin Panter date: Sat Sep 10 10:38:22 2016 +0000 summary: Correct print() calls to print a blank line in various test cases files: Lib/test/test_curses.py | 2 +- Lib/test/test_httpservers.py | 2 +- Lib/test/test_subprocess.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -63,7 +63,7 @@ def setUp(self): if verbose: # just to make the test output a little more readable - print() + print('') self.stdscr = curses.initscr() curses.savetty() diff --git a/Lib/test/test_httpservers.py b/Lib/test/test_httpservers.py --- a/Lib/test/test_httpservers.py +++ b/Lib/test/test_httpservers.py @@ -442,7 +442,7 @@ import os print("Content-type: text/html") -print() +print("") print(os.environ["%s"]) """ diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -426,7 +426,7 @@ def test_communicate_pipe_fd_leak(self): fd_directory = '/proc/%d/fd' % os.getpid() num_fds_before_popen = len(os.listdir(fd_directory)) - p = subprocess.Popen([sys.executable, "-c", "print()"], + p = subprocess.Popen([sys.executable, "-c", "print('')"], stdout=subprocess.PIPE) p.communicate() num_fds_after_communicate = len(os.listdir(fd_directory)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 06:39:48 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 10 Sep 2016 10:39:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Correct_spelli?= =?utf-8?q?ng_in_documentation_and_code_comments?= Message-ID: <20160910103948.8701.64037.1633E55C@psf.io> https://hg.python.org/cpython/rev/3219956eb703 changeset: 103568:3219956eb703 branch: 2.7 user: Martin Panter date: Sat Sep 10 10:38:28 2016 +0000 summary: Correct spelling in documentation and code comments files: Doc/whatsnew/2.2.rst | 4 ++-- Lib/lib-tk/turtle.py | 2 +- Lib/test/test_os.py | 2 +- Mac/Demo/textedit.html | 2 +- Mac/Modules/res/_Resmodule.c | 2 +- Mac/Modules/res/ressupport.py | 2 +- 6 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/whatsnew/2.2.rst b/Doc/whatsnew/2.2.rst --- a/Doc/whatsnew/2.2.rst +++ b/Doc/whatsnew/2.2.rst @@ -1159,7 +1159,7 @@ The main change is the possibility to build Python as a framework. This installs a self-contained Python installation plus the OSX framework "glue" into /Library/Frameworks/Python.framework (or - another location of choice). For now there is little immedeate added + another location of choice). For now there is little immediate added benefit to this (actually, there is the disadvantage that you have to change your PATH to be able to find Python), but it is the basis for creating a fullblown Python application, porting the MacPython IDE, @@ -1168,7 +1168,7 @@ The other change is that most MacPython toolbox modules, which interface to all the MacOS APIs such as windowing, quicktime, scripting, etc. have been ported. Again, most of these are not of - immedeate use, as they need a full application to be really useful, so + immediate use, as they need a full application to be really useful, so they have been commented out in setup.py. People wanting to experiment can uncomment them. Gestalt and Internet Config modules are enabled by default. diff --git a/Lib/lib-tk/turtle.py b/Lib/lib-tk/turtle.py --- a/Lib/lib-tk/turtle.py +++ b/Lib/lib-tk/turtle.py @@ -981,7 +981,7 @@ """Set turtle-mode ('standard', 'logo' or 'world') and perform reset. Optional argument: - mode -- on of the strings 'standard', 'logo' or 'world' + mode -- one of the strings 'standard', 'logo' or 'world' Mode 'standard' is compatible with turtle.py. Mode 'logo' is compatible with most Logo-Turtle-Graphics. diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -634,7 +634,7 @@ singles = ["fchdir", "fdopen", "dup", "fdatasync", "fstat", "fstatvfs", "fsync", "tcgetpgrp", "ttyname"] #singles.append("close") - #We omit close because it doesn'r raise an exception on some platforms + #We omit close because it doesn't raise an exception on some platforms def get_single(f): def helper(self): if hasattr(os, f): diff --git a/Mac/Demo/textedit.html b/Mac/Demo/textedit.html --- a/Mac/Demo/textedit.html +++ b/Mac/Demo/textedit.html @@ -24,7 +24,7 @@ to do your own initializations and override makeusermenus to create your menus (your menu callback routines may be here too, but this is by no means necessary). The event handling code can be overridden at various levels, from very low-level (the -dispatch method) to intermedeate level (do_keyDown, for instance) +dispatch method) to intermediate level (do_keyDown, for instance) to high-level (do_key). The application class knows about the Window objects you create, and will forward events to the appropriate window (So, normally you would have a do_key method in your window object, not your application object). diff --git a/Mac/Modules/res/_Resmodule.c b/Mac/Modules/res/_Resmodule.c --- a/Mac/Modules/res/_Resmodule.c +++ b/Mac/Modules/res/_Resmodule.c @@ -551,7 +551,7 @@ HLock(self->ob_itself); memcpy((char *)*self->ob_itself, data, size); HUnlock(self->ob_itself); - /* XXXX Should I do the Changed call immedeately? */ + /* XXXX Should I do the Changed call immediately? */ return 0; return 0; diff --git a/Mac/Modules/res/ressupport.py b/Mac/Modules/res/ressupport.py --- a/Mac/Modules/res/ressupport.py +++ b/Mac/Modules/res/ressupport.py @@ -125,7 +125,7 @@ HLock(self->ob_itself); memcpy((char *)*self->ob_itself, data, size); HUnlock(self->ob_itself); - /* XXXX Should I do the Changed call immedeately? */ + /* XXXX Should I do the Changed call immediately? */ return 0; """, 'The resource data' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 06:47:30 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 10 Sep 2016 10:47:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_spelling_fixes_from_3=2E5?= Message-ID: <20160910104730.19423.68563.58EC1EA1@psf.io> https://hg.python.org/cpython/rev/9c8160e08090 changeset: 103570:9c8160e08090 parent: 103566:3ded89cdea11 parent: 103569:f8e0ec74dca5 user: Martin Panter date: Sat Sep 10 10:44:12 2016 +0000 summary: Merge spelling fixes from 3.5 files: Doc/whatsnew/2.2.rst | 4 ++-- Lib/turtle.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/2.2.rst b/Doc/whatsnew/2.2.rst --- a/Doc/whatsnew/2.2.rst +++ b/Doc/whatsnew/2.2.rst @@ -1159,7 +1159,7 @@ The main change is the possibility to build Python as a framework. This installs a self-contained Python installation plus the OSX framework "glue" into /Library/Frameworks/Python.framework (or - another location of choice). For now there is little immedeate added + another location of choice). For now there is little immediate added benefit to this (actually, there is the disadvantage that you have to change your PATH to be able to find Python), but it is the basis for creating a fullblown Python application, porting the MacPython IDE, @@ -1168,7 +1168,7 @@ The other change is that most MacPython toolbox modules, which interface to all the MacOS APIs such as windowing, quicktime, scripting, etc. have been ported. Again, most of these are not of - immedeate use, as they need a full application to be really useful, so + immediate use, as they need a full application to be really useful, so they have been commented out in setup.py. People wanting to experiment can uncomment them. Gestalt and Internet Config modules are enabled by default. diff --git a/Lib/turtle.py b/Lib/turtle.py --- a/Lib/turtle.py +++ b/Lib/turtle.py @@ -1035,7 +1035,7 @@ """Set turtle-mode ('standard', 'logo' or 'world') and perform reset. Optional argument: - mode -- on of the strings 'standard', 'logo' or 'world' + mode -- one of the strings 'standard', 'logo' or 'world' Mode 'standard' is compatible with turtle.py. Mode 'logo' is compatible with most Logo-Turtle-Graphics. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 06:47:30 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 10 Sep 2016 10:47:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Correct_spelli?= =?utf-8?q?ng_in_documentation_and_code_comment?= Message-ID: <20160910104730.59448.63508.984833AC@psf.io> https://hg.python.org/cpython/rev/f8e0ec74dca5 changeset: 103569:f8e0ec74dca5 branch: 3.5 parent: 103562:4df2d43e995d user: Martin Panter date: Sat Sep 10 10:38:28 2016 +0000 summary: Correct spelling in documentation and code comment files: Doc/whatsnew/2.2.rst | 4 ++-- Lib/test/test_os.py | 2 +- Lib/turtle.py | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/2.2.rst b/Doc/whatsnew/2.2.rst --- a/Doc/whatsnew/2.2.rst +++ b/Doc/whatsnew/2.2.rst @@ -1159,7 +1159,7 @@ The main change is the possibility to build Python as a framework. This installs a self-contained Python installation plus the OSX framework "glue" into /Library/Frameworks/Python.framework (or - another location of choice). For now there is little immedeate added + another location of choice). For now there is little immediate added benefit to this (actually, there is the disadvantage that you have to change your PATH to be able to find Python), but it is the basis for creating a fullblown Python application, porting the MacPython IDE, @@ -1168,7 +1168,7 @@ The other change is that most MacPython toolbox modules, which interface to all the MacOS APIs such as windowing, quicktime, scripting, etc. have been ported. Again, most of these are not of - immedeate use, as they need a full application to be really useful, so + immediate use, as they need a full application to be really useful, so they have been commented out in setup.py. People wanting to experiment can uncomment them. Gestalt and Internet Config modules are enabled by default. diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -1471,7 +1471,7 @@ singles = ["fchdir", "dup", "fdopen", "fdatasync", "fstat", "fstatvfs", "fsync", "tcgetpgrp", "ttyname"] #singles.append("close") - #We omit close because it doesn'r raise an exception on some platforms + #We omit close because it doesn't raise an exception on some platforms def get_single(f): def helper(self): if hasattr(os, f): diff --git a/Lib/turtle.py b/Lib/turtle.py --- a/Lib/turtle.py +++ b/Lib/turtle.py @@ -1035,7 +1035,7 @@ """Set turtle-mode ('standard', 'logo' or 'world') and perform reset. Optional argument: - mode -- on of the strings 'standard', 'logo' or 'world' + mode -- one of the strings 'standard', 'logo' or 'world' Mode 'standard' is compatible with turtle.py. Mode 'logo' is compatible with most Logo-Turtle-Graphics. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 06:47:30 2016 From: python-checkins at python.org (martin.panter) Date: Sat, 10 Sep 2016 10:47:30 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_One_more_spelling_fix?= Message-ID: <20160910104730.36576.29635.7050397A@psf.io> https://hg.python.org/cpython/rev/a477ef882a16 changeset: 103571:a477ef882a16 user: Martin Panter date: Sat Sep 10 10:45:28 2016 +0000 summary: One more spelling fix files: PC/getpathp.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/PC/getpathp.c b/PC/getpathp.c --- a/PC/getpathp.c +++ b/PC/getpathp.c @@ -24,7 +24,7 @@ * We attempt to locate the "Python Home" - if the PYTHONHOME env var is set, we believe it. Otherwise, we use the path of our host .EXE's - to try and locate on of our "landmarks" and deduce our home. + to try and locate one of our "landmarks" and deduce our home. - If we DO have a Python Home: The relevant sub-directories (Lib, DLLs, etc) are based on the Python Home - If we DO NOT have a Python Home, the core Python Path is -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 09:56:45 2016 From: python-checkins at python.org (zach.ware) Date: Sat, 10 Sep 2016 13:56:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset_491bb?= =?utf-8?q?ba73bca?= Message-ID: <20160910135645.36379.16758.8742D2F3@psf.io> https://hg.python.org/cpython/rev/a14b86fc36f9 changeset: 103572:a14b86fc36f9 user: Zachary Ware date: Sat Sep 10 08:55:15 2016 -0500 summary: Backed out changeset 491bbba73bca This change didn't have the intended effect. files: Tools/buildbot/test.bat | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Tools/buildbot/test.bat b/Tools/buildbot/test.bat --- a/Tools/buildbot/test.bat +++ b/Tools/buildbot/test.bat @@ -3,7 +3,7 @@ setlocal set here=%~dp0 -set rt_opts=-d +set rt_opts=-q -d set regrtest_args=-j1 :CheckOpts -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 10:20:26 2016 From: python-checkins at python.org (xavier.degaye) Date: Sat, 10 Sep 2016 14:20:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328046=3A_Fix_get?= =?utf-8?q?=5Fsysconfigdata=5Fname=28=29=2E?= Message-ID: <20160910142026.59554.15927.5D0BD530@psf.io> https://hg.python.org/cpython/rev/1d71ce4531ac changeset: 103573:1d71ce4531ac user: Xavier de Gaye date: Sat Sep 10 16:19:45 2016 +0200 summary: Issue #28046: Fix get_sysconfigdata_name(). files: Lib/sysconfig.py | 21 ++++++++++++++------- 1 files changed, 14 insertions(+), 7 deletions(-) diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -342,12 +342,19 @@ return os.path.join(get_path('stdlib'), config_dir_name, 'Makefile') -def _get_sysconfigdata_name(): - return '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( - abi=sys.abiflags, - platform=sys.platform, - multiarch=getattr(sys.implementation, '_multiarch', ''), - ) +def _get_sysconfigdata_name(vars=None): + if vars is None: + return '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( + abi=sys.abiflags, + platform=sys.platform, + multiarch=getattr(sys.implementation, '_multiarch', ''), + ) + else: + return '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( + abi=vars['ABIFLAGS'], + platform=vars['MACHDEP'], + multiarch=vars.get('MULTIARCH', ''), + ) def _generate_posix_vars(): @@ -390,7 +397,7 @@ # _sysconfigdata module manually and populate it with the build vars. # This is more than sufficient for ensuring the subsequent call to # get_platform() succeeds. - name = _get_sysconfigdata_name() + name = _get_sysconfigdata_name(vars) if 'darwin' in sys.platform: import types module = types.ModuleType(name) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 14:32:36 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 10 Sep 2016 18:32:36 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2324693=3A_Changed_?= =?utf-8?q?some_RuntimeError=27s_in_the_zipfile_module_to_more?= Message-ID: <20160910183235.1800.14953.822B2FCF@psf.io> https://hg.python.org/cpython/rev/22112359abcf changeset: 103574:22112359abcf user: Serhiy Storchaka date: Sat Sep 10 21:28:07 2016 +0300 summary: Issue #24693: Changed some RuntimeError's in the zipfile module to more appropriate types. Improved some error messages and debugging output. files: Doc/library/zipfile.rst | 58 ++++++++++++++++++++------- Lib/test/test_zipfile.py | 40 +++++++++--------- Lib/zipfile.py | 56 +++++++++++++------------- Misc/NEWS | 3 + 4 files changed, 94 insertions(+), 63 deletions(-) diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst --- a/Doc/library/zipfile.rst +++ b/Doc/library/zipfile.rst @@ -147,10 +147,10 @@ *compression* is the ZIP compression method to use when writing the archive, and should be :const:`ZIP_STORED`, :const:`ZIP_DEFLATED`, :const:`ZIP_BZIP2` or :const:`ZIP_LZMA`; unrecognized - values will cause :exc:`RuntimeError` to be raised. If :const:`ZIP_DEFLATED`, + values will cause :exc:`NotImplementedError` to be raised. If :const:`ZIP_DEFLATED`, :const:`ZIP_BZIP2` or :const:`ZIP_LZMA` is specified but the corresponding module (:mod:`zlib`, :mod:`bz2` or :mod:`lzma`) is not available, :exc:`RuntimeError` - is also raised. The default is :const:`ZIP_STORED`. If *allowZip64* is + is raised. The default is :const:`ZIP_STORED`. If *allowZip64* is ``True`` (the default) zipfile will create ZIP files that use the ZIP64 extensions when the zipfile is larger than 2 GiB. If it is false :mod:`zipfile` will raise an exception when the ZIP file would require ZIP64 extensions. @@ -179,6 +179,10 @@ Added support for writing to unseekable streams. Added support for the ``'x'`` mode. + .. versionchanged:: 3.6 + Previously, a plain :exc:`RuntimeError` was raised for unrecognized + compression values. + .. method:: ZipFile.close() @@ -211,7 +215,6 @@ can be either the name of a file within the archive or a :class:`ZipInfo` object. The *mode* parameter, if included, must be ``'r'`` (the default) or ``'w'``. *pwd* is the password used to decrypt encrypted ZIP files. - Calling :meth:`.open` on a closed ZipFile will raise a :exc:`RuntimeError`. :meth:`~ZipFile.open` is also a context manager and therefore supports the :keyword:`with` statement:: @@ -230,7 +233,7 @@ With ``mode='w'``, a writable file handle is returned, which supports the :meth:`~io.BufferedIOBase.write` method. While a writable file handle is open, attempting to read or write other files in the ZIP file will raise a - :exc:`RuntimeError`. + :exc:`ValueError`. When writing a file, if the file size is not known in advance but may exceed 2 GiB, pass ``force_zip64=True`` to ensure that the header format is @@ -252,6 +255,11 @@ :meth:`open` can now be used to write files into the archive with the ``mode='w'`` option. + .. versionchanged:: 3.6 + Calling :meth:`.open` on a closed ZipFile will raise a :exc:`ValueError`. + Previously, a :exc:`RuntimeError` was raised. + + .. method:: ZipFile.extract(member, path=None, pwd=None) Extract a member from the archive to the current working directory; *member* @@ -272,6 +280,10 @@ characters (``:``, ``<``, ``>``, ``|``, ``"``, ``?``, and ``*``) replaced by underscore (``_``). + .. versionchanged:: 3.6 + Calling :meth:`extract` on a closed ZipFile will raise a + :exc:`ValueError`. Previously, a :exc:`RuntimeError` was raised. + .. method:: ZipFile.extractall(path=None, members=None, pwd=None) @@ -288,6 +300,10 @@ dots ``".."``. This module attempts to prevent that. See :meth:`extract` note. + .. versionchanged:: 3.6 + Calling :meth:`extractall` on a closed ZipFile will raise a + :exc:`ValueError`. Previously, a :exc:`RuntimeError` was raised. + .. method:: ZipFile.printdir() @@ -305,18 +321,24 @@ file in the archive, or a :class:`ZipInfo` object. The archive must be open for read or append. *pwd* is the password used for encrypted files and, if specified, it will override the default password set with :meth:`setpassword`. Calling - :meth:`read` on a closed ZipFile will raise a :exc:`RuntimeError`. Calling :meth:`read` on a ZipFile that uses a compression method other than :const:`ZIP_STORED`, :const:`ZIP_DEFLATED`, :const:`ZIP_BZIP2` or :const:`ZIP_LZMA` will raise a :exc:`NotImplementedError`. An error will also be raised if the corresponding compression module is not available. + .. versionchanged:: 3.6 + Calling :meth:`read` on a closed ZipFile will raise a :exc:`ValueError`. + Previously, a :exc:`RuntimeError` was raised. + .. method:: ZipFile.testzip() Read all the files in the archive and check their CRC's and file headers. - Return the name of the first bad file, or else return ``None``. Calling - :meth:`testzip` on a closed ZipFile will raise a :exc:`RuntimeError`. + Return the name of the first bad file, or else return ``None``. + + .. versionchanged:: 3.6 + Calling :meth:`testfile` on a closed ZipFile will raise a + :exc:`ValueError`. Previously, a :exc:`RuntimeError` was raised. .. method:: ZipFile.write(filename, arcname=None, compress_type=None) @@ -326,10 +348,7 @@ letter and with leading path separators removed). If given, *compress_type* overrides the value given for the *compression* parameter to the constructor for the new entry. - The archive must be open with mode ``'w'``, ``'x'`` or ``'a'`` -- calling - :meth:`write` on a ZipFile created with mode ``'r'`` will raise a - :exc:`RuntimeError`. Calling :meth:`write` on a closed ZipFile will raise a - :exc:`RuntimeError`. + The archive must be open with mode ``'w'``, ``'x'`` or ``'a'``. .. note:: @@ -348,16 +367,19 @@ If ``arcname`` (or ``filename``, if ``arcname`` is not given) contains a null byte, the name of the file in the archive will be truncated at the null byte. + .. versionchanged:: 3.6 + Calling :meth:`write` on a ZipFile created with mode ``'r'`` or + a closed ZipFile will raise a :exc:`ValueError`. Previously, + a :exc:`RuntimeError` was raised. + + .. method:: ZipFile.writestr(zinfo_or_arcname, data[, compress_type]) Write the string *data* to the archive; *zinfo_or_arcname* is either the file name it will be given in the archive, or a :class:`ZipInfo` instance. If it's an instance, at least the filename, date, and time must be given. If it's a name, the date and time is set to the current date and time. - The archive must be opened with mode ``'w'``, ``'x'`` or ``'a'`` -- calling - :meth:`writestr` on a ZipFile created with mode ``'r'`` will raise a - :exc:`RuntimeError`. Calling :meth:`writestr` on a closed ZipFile will - raise a :exc:`RuntimeError`. + The archive must be opened with mode ``'w'``, ``'x'`` or ``'a'``. If given, *compress_type* overrides the value given for the *compression* parameter to the constructor for the new entry, or in the *zinfo_or_arcname* @@ -373,6 +395,12 @@ .. versionchanged:: 3.2 The *compress_type* argument. + .. versionchanged:: 3.6 + Calling :meth:`writestr` on a ZipFile created with mode ``'r'`` or + a closed ZipFile will raise a :exc:`ValueError`. Previously, + a :exc:`RuntimeError` was raised. + + The following data attributes are also available: diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -449,15 +449,15 @@ def test_write_to_readonly(self): """Check that trying to call write() on a readonly ZipFile object - raises a RuntimeError.""" + raises a ValueError.""" with zipfile.ZipFile(TESTFN2, mode="w") as zipfp: zipfp.writestr("somefile.txt", "bogus") with zipfile.ZipFile(TESTFN2, mode="r") as zipfp: - self.assertRaises(RuntimeError, zipfp.write, TESTFN) + self.assertRaises(ValueError, zipfp.write, TESTFN) with zipfile.ZipFile(TESTFN2, mode="r") as zipfp: - with self.assertRaises(RuntimeError): + with self.assertRaises(ValueError): zipfp.open(TESTFN, mode='w') def test_add_file_before_1980(self): @@ -1210,27 +1210,27 @@ fp.write("short file") self.assertRaises(zipfile.BadZipFile, zipfile.ZipFile, TESTFN) - def test_closed_zip_raises_RuntimeError(self): + def test_closed_zip_raises_ValueError(self): """Verify that testzip() doesn't swallow inappropriate exceptions.""" data = io.BytesIO() with zipfile.ZipFile(data, mode="w") as zipf: zipf.writestr("foo.txt", "O, for a Muse of Fire!") # This is correct; calling .read on a closed ZipFile should raise - # a RuntimeError, and so should calling .testzip. An earlier + # a ValueError, and so should calling .testzip. An earlier # version of .testzip would swallow this exception (and any other) # and report that the first file in the archive was corrupt. - self.assertRaises(RuntimeError, zipf.read, "foo.txt") - self.assertRaises(RuntimeError, zipf.open, "foo.txt") - self.assertRaises(RuntimeError, zipf.testzip) - self.assertRaises(RuntimeError, zipf.writestr, "bogus.txt", "bogus") + self.assertRaises(ValueError, zipf.read, "foo.txt") + self.assertRaises(ValueError, zipf.open, "foo.txt") + self.assertRaises(ValueError, zipf.testzip) + self.assertRaises(ValueError, zipf.writestr, "bogus.txt", "bogus") with open(TESTFN, 'w') as f: f.write('zipfile test data') - self.assertRaises(RuntimeError, zipf.write, TESTFN) + self.assertRaises(ValueError, zipf.write, TESTFN) def test_bad_constructor_mode(self): """Check that bad modes passed to ZipFile constructor are caught.""" - self.assertRaises(RuntimeError, zipfile.ZipFile, TESTFN, "q") + self.assertRaises(ValueError, zipfile.ZipFile, TESTFN, "q") def test_bad_open_mode(self): """Check that bad modes passed to ZipFile.open are caught.""" @@ -1240,10 +1240,10 @@ with zipfile.ZipFile(TESTFN, mode="r") as zipf: # read the data to make sure the file is there zipf.read("foo.txt") - self.assertRaises(RuntimeError, zipf.open, "foo.txt", "q") + self.assertRaises(ValueError, zipf.open, "foo.txt", "q") # universal newlines support is removed - self.assertRaises(RuntimeError, zipf.open, "foo.txt", "U") - self.assertRaises(RuntimeError, zipf.open, "foo.txt", "rU") + self.assertRaises(ValueError, zipf.open, "foo.txt", "U") + self.assertRaises(ValueError, zipf.open, "foo.txt", "rU") def test_read0(self): """Check that calling read(0) on a ZipExtFile object returns an empty @@ -1266,7 +1266,7 @@ def test_bad_compression_mode(self): """Check that bad compression methods passed to ZipFile.open are caught.""" - self.assertRaises(RuntimeError, zipfile.ZipFile, TESTFN, "w", -1) + self.assertRaises(NotImplementedError, zipfile.ZipFile, TESTFN, "w", -1) def test_unsupported_compression(self): # data is declared as shrunk, but actually deflated @@ -1423,15 +1423,15 @@ with zipf.open('foo', mode='w') as w2: w2.write(msg1) with zipf.open('bar', mode='w') as w1: - with self.assertRaises(RuntimeError): + with self.assertRaises(ValueError): zipf.open('handle', mode='w') - with self.assertRaises(RuntimeError): + with self.assertRaises(ValueError): zipf.open('foo', mode='r') - with self.assertRaises(RuntimeError): + with self.assertRaises(ValueError): zipf.writestr('str', 'abcde') - with self.assertRaises(RuntimeError): + with self.assertRaises(ValueError): zipf.write(__file__, 'file') - with self.assertRaises(RuntimeError): + with self.assertRaises(ValueError): zipf.close() w1.write(msg2) with zipf.open('baz', mode='w') as w2: diff --git a/Lib/zipfile.py b/Lib/zipfile.py --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -449,7 +449,7 @@ elif ln == 0: counts = () else: - raise RuntimeError("Corrupt extra field %s"%(ln,)) + raise BadZipFile("Corrupt extra field %04x (size=%d)" % (tp, ln)) idx = 0 @@ -654,7 +654,7 @@ raise RuntimeError( "Compression requires the (missing) lzma module") else: - raise RuntimeError("That compression method is not supported") + raise NotImplementedError("That compression method is not supported") def _get_compressor(compress_type): @@ -697,7 +697,7 @@ def read(self, n=-1): with self._lock: if self._writing(): - raise RuntimeError("Can't read from the ZIP file while there " + raise ValueError("Can't read from the ZIP file while there " "is an open writing handle on it. " "Close the writing handle before trying to read.") self._file.seek(self._pos) @@ -1055,7 +1055,7 @@ """Open the ZIP file with mode read 'r', write 'w', exclusive create 'x', or append 'a'.""" if mode not in ('r', 'w', 'x', 'a'): - raise RuntimeError("ZipFile requires mode 'r', 'w', 'x', or 'a'") + raise ValueError("ZipFile requires mode 'r', 'w', 'x', or 'a'") _check_compression(compression) @@ -1129,7 +1129,7 @@ self._didModify = True self.start_dir = self.fp.tell() else: - raise RuntimeError("Mode must be 'r', 'w', 'x', or 'a'") + raise ValueError("Mode must be 'r', 'w', 'x', or 'a'") except: fp = self.fp self.fp = None @@ -1277,7 +1277,7 @@ def setpassword(self, pwd): """Set default password for encrypted files.""" if pwd and not isinstance(pwd, bytes): - raise TypeError("pwd: expected bytes, got %s" % type(pwd)) + raise TypeError("pwd: expected bytes, got %s" % type(pwd).__name__) if pwd: self.pwd = pwd else: @@ -1291,7 +1291,7 @@ @comment.setter def comment(self, comment): if not isinstance(comment, bytes): - raise TypeError("comment: expected bytes, got %s" % type(comment)) + raise TypeError("comment: expected bytes, got %s" % type(comment).__name__) # check for valid comment length if len(comment) > ZIP_MAX_COMMENT: import warnings @@ -1323,13 +1323,13 @@ instance for name, with zinfo.file_size set. """ if mode not in {"r", "w"}: - raise RuntimeError('open() requires mode "r" or "w"') + raise ValueError('open() requires mode "r" or "w"') if pwd and not isinstance(pwd, bytes): - raise TypeError("pwd: expected bytes, got %s" % type(pwd)) + raise TypeError("pwd: expected bytes, got %s" % type(pwd).__name__) if pwd and (mode == "w"): raise ValueError("pwd is only supported for reading files") if not self.fp: - raise RuntimeError( + raise ValueError( "Attempt to use ZIP archive that was already closed") # Make sure we have an info object @@ -1347,7 +1347,7 @@ return self._open_to_write(zinfo, force_zip64=force_zip64) if self._writing: - raise RuntimeError("Can't read from the ZIP file while there " + raise ValueError("Can't read from the ZIP file while there " "is an open writing handle on it. " "Close the writing handle before trying to read.") @@ -1394,7 +1394,7 @@ if not pwd: pwd = self.pwd if not pwd: - raise RuntimeError("File %s is encrypted, password " + raise RuntimeError("File %r is encrypted, password " "required for extraction" % name) zd = _ZipDecrypter(pwd) @@ -1412,7 +1412,7 @@ # compare against the CRC otherwise check_byte = (zinfo.CRC >> 24) & 0xff if h[11] != check_byte: - raise RuntimeError("Bad password for file", name) + raise RuntimeError("Bad password for file %r" % name) return ZipExtFile(zef_file, mode, zinfo, zd, True) except: @@ -1426,9 +1426,9 @@ "the ZIP file." ) if self._writing: - raise RuntimeError("Can't write to the ZIP file while there is " - "another write handle open on it. " - "Close the first handle before opening another.") + raise ValueError("Can't write to the ZIP file while there is " + "another write handle open on it. " + "Close the first handle before opening another.") # Sizes and CRC are overwritten with correct data after processing the file if not hasattr(zinfo, 'file_size'): @@ -1548,9 +1548,9 @@ import warnings warnings.warn('Duplicate name: %r' % zinfo.filename, stacklevel=3) if self.mode not in ('w', 'x', 'a'): - raise RuntimeError("write() requires mode 'w', 'x', or 'a'") + raise ValueError("write() requires mode 'w', 'x', or 'a'") if not self.fp: - raise RuntimeError( + raise ValueError( "Attempt to write ZIP archive that was already closed") _check_compression(zinfo.compress_type) if not self._allowZip64: @@ -1569,10 +1569,10 @@ """Put the bytes from filename into the archive under the name arcname.""" if not self.fp: - raise RuntimeError( + raise ValueError( "Attempt to write to ZIP archive that was already closed") if self._writing: - raise RuntimeError( + raise ValueError( "Can't write to ZIP archive while an open writing handle exists" ) @@ -1628,10 +1628,10 @@ zinfo = zinfo_or_arcname if not self.fp: - raise RuntimeError( + raise ValueError( "Attempt to write to ZIP archive that was already closed") if self._writing: - raise RuntimeError( + raise ValueError( "Can't write to ZIP archive while an open writing handle exists." ) @@ -1654,9 +1654,9 @@ return if self._writing: - raise RuntimeError("Can't close the ZIP file while there is " - "an open writing handle on it. " - "Close the writing handle before closing the zip.") + raise ValueError("Can't close the ZIP file while there is " + "an open writing handle on it. " + "Close the writing handle before closing the zip.") try: if self.mode in ('w', 'x', 'a') and self._didModify: # write ending records @@ -1803,7 +1803,7 @@ if filterfunc and not filterfunc(pathname): if self.debug: label = 'path' if os.path.isdir(pathname) else 'file' - print('%s "%s" skipped by filterfunc' % (label, pathname)) + print('%s %r skipped by filterfunc' % (label, pathname)) return dir, name = os.path.split(pathname) if os.path.isdir(pathname): @@ -1834,7 +1834,7 @@ elif ext == ".py": if filterfunc and not filterfunc(path): if self.debug: - print('file "%s" skipped by filterfunc' % path) + print('file %r skipped by filterfunc' % path) continue fname, arcname = self._get_codename(path[0:-3], basename) @@ -1851,7 +1851,7 @@ if ext == ".py": if filterfunc and not filterfunc(path): if self.debug: - print('file "%s" skipped by filterfunc' % path) + print('file %r skipped by filterfunc' % path) continue fname, arcname = self._get_codename(path[0:-3], basename) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -135,6 +135,9 @@ Library ------- +- Issue #24693: Changed some RuntimeError's in the zipfile module to more + appropriate types. Improved some error messages and debugging output. + - Issue #17909: ``json.load`` and ``json.loads`` now support binary input encoded as UTF-8, UTF-16 or UTF-32. Patch by Serhiy Storchaka. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 14:35:00 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 10 Sep 2016 18:35:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixed_compiler_warnings_in?= =?utf-8?q?_compact_dict_implementation_on_32-bit_platforms=2E?= Message-ID: <20160910183459.48656.26177.67081CF5@psf.io> https://hg.python.org/cpython/rev/b28b37de9470 changeset: 103575:b28b37de9470 user: Serhiy Storchaka date: Sat Sep 10 21:34:43 2016 +0300 summary: Fixed compiler warnings in compact dict implementation on 32-bit platforms. files: Objects/dictobject.c | 24 ++++++++++++------------ 1 files changed, 12 insertions(+), 12 deletions(-) diff --git a/Objects/dictobject.c b/Objects/dictobject.c --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -326,16 +326,16 @@ int16_t *indices = keys->dk_indices.as_2; ix = indices[i]; } - else if (s <= 0xffffffff) { - int32_t *indices = keys->dk_indices.as_4; - ix = indices[i]; - } #if SIZEOF_VOID_P > 4 - else { + else if (s > 0xffffffff) { int64_t *indices = keys->dk_indices.as_8; ix = indices[i]; } #endif + else { + int32_t *indices = keys->dk_indices.as_4; + ix = indices[i]; + } assert(ix >= DKIX_DUMMY); return ix; } @@ -358,17 +358,17 @@ assert(ix <= 0x7fff); indices[i] = (int16_t)ix; } - else if (s <= 0xffffffff) { +#if SIZEOF_VOID_P > 4 + else if (s > 0xffffffff) { + int64_t *indices = keys->dk_indices.as_8; + indices[i] = ix; + } +#endif + else { int32_t *indices = keys->dk_indices.as_4; assert(ix <= 0x7fffffff); indices[i] = (int32_t)ix; } -#if SIZEOF_VOID_P > 4 - else { - int64_t *indices = keys->dk_indices.as_8; - indices[i] = ix; - } -#endif } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 14:54:01 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 18:54:01 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTMy?= =?utf-8?q?=3A_Backs_out_change?= Message-ID: <20160910185400.12395.840.A13CA3A6@psf.io> https://hg.python.org/cpython/rev/dbfcb3b9c3c1 changeset: 103576:dbfcb3b9c3c1 branch: 3.5 parent: 103569:f8e0ec74dca5 user: Steve Dower date: Sat Sep 10 11:52:18 2016 -0700 summary: Issue #27932: Backs out change files: Lib/platform.py | 64 +++++++++++++++++------------------- Misc/NEWS | 2 - 2 files changed, 30 insertions(+), 36 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -498,61 +498,57 @@ (6, None): "post2012ServerR2", } -if sys.platform == 'win32': - import ctypes - import ctypes.wintypes - - class VS_FIXEDFILEINFO(ctypes.Structure): - _fields_ = [ - ("dwSignature", ctypes.wintypes.DWORD), - ("dwStrucVersion", ctypes.wintypes.DWORD), - ("dwFileVersionMS", ctypes.wintypes.DWORD), - ("dwFileVersionLS", ctypes.wintypes.DWORD), - ("dwProductVersionMS", ctypes.wintypes.DWORD), - ("dwProductVersionLS", ctypes.wintypes.DWORD), - ("dwFileFlagsMask", ctypes.wintypes.DWORD), - ("dwFileFlags", ctypes.wintypes.DWORD), - ("dwFileOS", ctypes.wintypes.DWORD), - ("dwFileType", ctypes.wintypes.DWORD), - ("dwFileSubtype", ctypes.wintypes.DWORD), - ("dwFileDateMS", ctypes.wintypes.DWORD), - ("dwFileDateLS", ctypes.wintypes.DWORD), - ] - - P_VS_FIXEDFILEINFO = ctypes.POINTER(VS_FIXEDFILEINFO) - def _get_real_winver(maj, min, build): if maj < 6 or (maj == 6 and min < 2): return maj, min, build - kernel32 = ctypes.WinDLL('kernel32') + from ctypes import (c_buffer, POINTER, byref, create_unicode_buffer, + Structure, WinDLL) + from ctypes.wintypes import DWORD, HANDLE + + class VS_FIXEDFILEINFO(Structure): + _fields_ = [ + ("dwSignature", DWORD), + ("dwStrucVersion", DWORD), + ("dwFileVersionMS", DWORD), + ("dwFileVersionLS", DWORD), + ("dwProductVersionMS", DWORD), + ("dwProductVersionLS", DWORD), + ("dwFileFlagsMask", DWORD), + ("dwFileFlags", DWORD), + ("dwFileOS", DWORD), + ("dwFileType", DWORD), + ("dwFileSubtype", DWORD), + ("dwFileDateMS", DWORD), + ("dwFileDateLS", DWORD), + ] + + kernel32 = WinDLL('kernel32') + version = WinDLL('version') + # We will immediately double the length up to MAX_PATH, but the # path may be longer, so we retry until the returned string is # shorter than our buffer. name_len = actual_len = 130 while actual_len == name_len: name_len *= 2 - name = ctypes.create_unicode_buffer(name_len) - actual_len = kernel32.GetModuleFileNameW( - ctypes.wintypes.HANDLE(kernel32._handle), - name, len(name) - ) + name = create_unicode_buffer(name_len) + actual_len = kernel32.GetModuleFileNameW(HANDLE(kernel32._handle), + name, len(name)) if not actual_len: return maj, min, build - version = ctypes.WinDLL('version') size = version.GetFileVersionInfoSizeW(name, None) if not size: return maj, min, build - ver_block = ctypes.c_buffer(size) + ver_block = c_buffer(size) if (not version.GetFileVersionInfoW(name, None, size, ver_block) or not ver_block): return maj, min, build - pvi = P_VS_FIXEDFILEINFO() - if not version.VerQueryValueW(ver_block, "", - ctypes.byref(pvi), ctypes.byref(ctypes.wintypes.DWORD())): + pvi = POINTER(VS_FIXEDFILEINFO)() + if not version.VerQueryValueW(ver_block, "", byref(pvi), byref(DWORD())): return maj, min, build maj = pvi.contents.dwProductVersionMS >> 16 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -71,8 +71,6 @@ - Issue #25969: Update the lib2to3 grammar to handle the unpacking generalizations added in 3.5. -- Issue #27932: Fixes memory leak in platform.win32_ver() - - Issue #14977: mailcap now respects the order of the lines in the mailcap files ("first match"), as required by RFC 1542. Patch by Michael Lazar. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 14:54:01 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 18:54:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_from_3=2E5?= Message-ID: <20160910185401.36557.43859.96AEB5E1@psf.io> https://hg.python.org/cpython/rev/b9bb77dd71b3 changeset: 103577:b9bb77dd71b3 parent: 103575:b28b37de9470 parent: 103576:dbfcb3b9c3c1 user: Steve Dower date: Sat Sep 10 11:53:34 2016 -0700 summary: Merge from 3.5 files: Lib/platform.py | 64 +++++++++++++++++------------------- Misc/NEWS | 2 - 2 files changed, 30 insertions(+), 36 deletions(-) diff --git a/Lib/platform.py b/Lib/platform.py --- a/Lib/platform.py +++ b/Lib/platform.py @@ -497,61 +497,57 @@ (6, None): "post2012ServerR2", } -if sys.platform == 'win32': - import ctypes - import ctypes.wintypes - - class VS_FIXEDFILEINFO(ctypes.Structure): - _fields_ = [ - ("dwSignature", ctypes.wintypes.DWORD), - ("dwStrucVersion", ctypes.wintypes.DWORD), - ("dwFileVersionMS", ctypes.wintypes.DWORD), - ("dwFileVersionLS", ctypes.wintypes.DWORD), - ("dwProductVersionMS", ctypes.wintypes.DWORD), - ("dwProductVersionLS", ctypes.wintypes.DWORD), - ("dwFileFlagsMask", ctypes.wintypes.DWORD), - ("dwFileFlags", ctypes.wintypes.DWORD), - ("dwFileOS", ctypes.wintypes.DWORD), - ("dwFileType", ctypes.wintypes.DWORD), - ("dwFileSubtype", ctypes.wintypes.DWORD), - ("dwFileDateMS", ctypes.wintypes.DWORD), - ("dwFileDateLS", ctypes.wintypes.DWORD), - ] - - P_VS_FIXEDFILEINFO = ctypes.POINTER(VS_FIXEDFILEINFO) - def _get_real_winver(maj, min, build): if maj < 6 or (maj == 6 and min < 2): return maj, min, build - kernel32 = ctypes.WinDLL('kernel32') + from ctypes import (c_buffer, POINTER, byref, create_unicode_buffer, + Structure, WinDLL) + from ctypes.wintypes import DWORD, HANDLE + + class VS_FIXEDFILEINFO(Structure): + _fields_ = [ + ("dwSignature", DWORD), + ("dwStrucVersion", DWORD), + ("dwFileVersionMS", DWORD), + ("dwFileVersionLS", DWORD), + ("dwProductVersionMS", DWORD), + ("dwProductVersionLS", DWORD), + ("dwFileFlagsMask", DWORD), + ("dwFileFlags", DWORD), + ("dwFileOS", DWORD), + ("dwFileType", DWORD), + ("dwFileSubtype", DWORD), + ("dwFileDateMS", DWORD), + ("dwFileDateLS", DWORD), + ] + + kernel32 = WinDLL('kernel32') + version = WinDLL('version') + # We will immediately double the length up to MAX_PATH, but the # path may be longer, so we retry until the returned string is # shorter than our buffer. name_len = actual_len = 130 while actual_len == name_len: name_len *= 2 - name = ctypes.create_unicode_buffer(name_len) - actual_len = kernel32.GetModuleFileNameW( - ctypes.wintypes.HANDLE(kernel32._handle), - name, len(name) - ) + name = create_unicode_buffer(name_len) + actual_len = kernel32.GetModuleFileNameW(HANDLE(kernel32._handle), + name, len(name)) if not actual_len: return maj, min, build - version = ctypes.WinDLL('version') size = version.GetFileVersionInfoSizeW(name, None) if not size: return maj, min, build - ver_block = ctypes.c_buffer(size) + ver_block = c_buffer(size) if (not version.GetFileVersionInfoW(name, None, size, ver_block) or not ver_block): return maj, min, build - pvi = P_VS_FIXEDFILEINFO() - if not version.VerQueryValueW(ver_block, "", - ctypes.byref(pvi), ctypes.byref(ctypes.wintypes.DWORD())): + pvi = POINTER(VS_FIXEDFILEINFO)() + if not version.VerQueryValueW(ver_block, "", byref(pvi), byref(DWORD())): return maj, min, build maj = pvi.contents.dwProductVersionMS >> 16 diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -156,8 +156,6 @@ - Issue #25969: Update the lib2to3 grammar to handle the unpacking generalizations added in 3.5. -- Issue #27932: Fixes memory leak in platform.win32_ver() - - Issue #14977: mailcap now respects the order of the lines in the mailcap files ("first match"), as required by RFC 1542. Patch by Michael Lazar. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 15:20:20 2016 From: python-checkins at python.org (mark.dickinson) Date: Sat, 10 Sep 2016 19:20:20 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI1MjIx?= =?utf-8?q?=3A_Fix_corrupted_result_from_PyLong=5FFromLong=280=29_when_Pyt?= =?utf-8?q?hon_is?= Message-ID: <20160910192020.1841.99593.7CB90ACA@psf.io> https://hg.python.org/cpython/rev/9eb0f7762999 changeset: 103578:9eb0f7762999 branch: 3.5 parent: 103576:dbfcb3b9c3c1 user: Mark Dickinson date: Sat Sep 10 20:17:36 2016 +0100 summary: Issue #25221: Fix corrupted result from PyLong_FromLong(0) when Python is compiled with NSMALLPOSINTS = 0. files: Misc/NEWS | 3 +++ Objects/longobject.c | 3 ++- 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #25221: Fix corrupted result from PyLong_FromLong(0) when + Python is compiled with NSMALLPOSINTS = 0. + - Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch by Eryk Sun) diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -234,7 +234,7 @@ unsigned long abs_ival; unsigned long t; /* unsigned so >> doesn't propagate sign bit */ int ndigits = 0; - int sign = 1; + int sign; CHECK_SMALL_INT(ival); @@ -246,6 +246,7 @@ } else { abs_ival = (unsigned long)ival; + sign = ival == 0 ? 0 : 1; } /* Fast path for single-digit ints */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 15:20:20 2016 From: python-checkins at python.org (mark.dickinson) Date: Sat, 10 Sep 2016 19:20:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2325221=3A_merge_from_3=2E5=2E?= Message-ID: <20160910192020.69446.48459.4D20C24B@psf.io> https://hg.python.org/cpython/rev/c7b48798dbaa changeset: 103579:c7b48798dbaa parent: 103577:b9bb77dd71b3 parent: 103578:9eb0f7762999 user: Mark Dickinson date: Sat Sep 10 20:20:08 2016 +0100 summary: Issue #25221: merge from 3.5. files: Misc/NEWS | 3 +++ Objects/longobject.c | 3 ++- 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #25221: Fix corrupted result from PyLong_FromLong(0) when Python + is compiled with NSMALLPOSINTS = 0. + - Issue #27080: Implement formatting support for PEP 515. Initial patch by Chris Angelico. diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -234,7 +234,7 @@ unsigned long abs_ival; unsigned long t; /* unsigned so >> doesn't propagate sign bit */ int ndigits = 0; - int sign = 1; + int sign; CHECK_SMALL_INT(ival); @@ -246,6 +246,7 @@ } else { abs_ival = (unsigned long)ival; + sign = ival == 0 ? 0 : 1; } /* Fast path for single-digit ints */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 15:25:23 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 19:25:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2328059=3A_Fixes_test=5Fplatform_to_set_PYTHONPA?= =?utf-8?q?TH_for_=2Epyd_files?= Message-ID: <20160910192523.21048.91668.546FAECA@psf.io> https://hg.python.org/cpython/rev/3ec4feb52a5b changeset: 103581:3ec4feb52a5b parent: 103579:c7b48798dbaa parent: 103580:94563ec74e1d user: Steve Dower date: Sat Sep 10 12:25:07 2016 -0700 summary: Closes #28059: Fixes test_platform to set PYTHONPATH for .pyd files files: Lib/test/test_platform.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -15,8 +15,8 @@ @support.skip_unless_symlink def test_architecture_via_symlink(self): # issue3762 - # On Windows, the EXE needs to know where pythonXY.dll is at so we have - # to add the directory to the path. + # On Windows, the EXE needs to know where pythonXY.dll and *.pyd is at + # so we add the directory to the path and PYTHONPATH. if sys.platform == "win32": def restore_environ(old_env): os.environ.clear() @@ -26,6 +26,7 @@ os.environ["Path"] = "{};{}".format( os.path.dirname(sys.executable), os.environ["Path"]) + os.environ["PYTHONPATH"] = os.path.dirname(sys.executable) def get(python): cmd = [python, '-c', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 15:25:23 2016 From: python-checkins at python.org (steve.dower) Date: Sat, 10 Sep 2016 19:25:23 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQ2xvc2VzICMyODA1?= =?utf-8?q?9=3A_Fixes_test=5Fplatform_to_set_PYTHONPATH_for_=2Epyd_files?= Message-ID: <20160910192523.12852.39642.5636EE9B@psf.io> https://hg.python.org/cpython/rev/94563ec74e1d changeset: 103580:94563ec74e1d branch: 3.5 parent: 103578:9eb0f7762999 user: Steve Dower date: Sat Sep 10 12:19:42 2016 -0700 summary: Closes #28059: Fixes test_platform to set PYTHONPATH for .pyd files files: Lib/test/test_platform.py | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -15,11 +15,18 @@ @support.skip_unless_symlink def test_architecture_via_symlink(self): # issue3762 - # On Windows, the EXE needs to know where pythonXY.dll is at so we have - # to add the directory to the path. + # On Windows, the EXE needs to know where pythonXY.dll and *.pyd is at + # so we add the directory to the path and PYTHONPATH. if sys.platform == "win32": + def restore_environ(old_env): + os.environ.clear() + os.environ.update(old_env) + + self.addCleanup(restore_environ, dict(os.environ)) + os.environ["Path"] = "{};{}".format( os.path.dirname(sys.executable), os.environ["Path"]) + os.environ["PYTHONPATH"] = os.path.dirname(sys.executable) def get(python): cmd = [python, '-c', -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 15:58:32 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Sat, 10 Sep 2016 19:58:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2328067=3A_Do_not_?= =?utf-8?q?call_localtime_=28gmtime=29_in_datetime_module=2E?= Message-ID: <20160910195832.20991.14724.96C8EFB0@psf.io> https://hg.python.org/cpython/rev/8a504694d92f changeset: 103582:8a504694d92f user: Alexander Belopolsky date: Sat Sep 10 15:58:31 2016 -0400 summary: Closes #28067: Do not call localtime (gmtime) in datetime module. files: Modules/_datetimemodule.c | 104 +++++++++++++++---------- 1 files changed, 61 insertions(+), 43 deletions(-) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -15,6 +15,12 @@ return result; return NULL; } +static struct tm *gmtime_r(const time_t *timep, struct tm *result) +{ + if (gmime_s(result, timep) == 0) + return result; + return NULL; +} #endif /* Differentiate between building the core module and building extension @@ -2517,14 +2523,13 @@ static PyObject * date_local_from_object(PyObject *cls, PyObject *obj) { - struct tm *tm; + struct tm tm; time_t t; if (_PyTime_ObjectToTime_t(obj, &t, _PyTime_ROUND_FLOOR) == -1) return NULL; - tm = localtime(&t); - if (tm == NULL) { + if (localtime_r(&t, &tm) == NULL) { /* unconvertible time */ #ifdef EINVAL if (errno == 0) @@ -2535,9 +2540,9 @@ } return PyObject_CallFunction(cls, "iii", - tm->tm_year + 1900, - tm->tm_mon + 1, - tm->tm_mday); + tm.tm_year + 1900, + tm.tm_mon + 1, + tm.tm_mday); } /* Return new date from current time. @@ -4194,8 +4199,8 @@ return self; } -/* TM_FUNC is the shared type of localtime() and gmtime(). */ -typedef struct tm *(*TM_FUNC)(const time_t *timer); +/* TM_FUNC is the shared type of localtime_r() and gmtime_r(). */ +typedef struct tm *(*TM_FUNC)(const time_t *timer, struct tm*); /* As of version 2015f max fold in IANA database is * 23 hours at 1969-09-30 13:00:00 in Kwajalein. */ @@ -4244,11 +4249,10 @@ datetime_from_timet_and_us(PyObject *cls, TM_FUNC f, time_t timet, int us, PyObject *tzinfo) { - struct tm *tm; + struct tm tm; int year, month, day, hour, minute, second, fold = 0; - tm = f(&timet); - if (tm == NULL) { + if (f(&timet, &tm) == NULL) { #ifdef EINVAL if (errno == 0) errno = EINVAL; @@ -4256,20 +4260,20 @@ return PyErr_SetFromErrno(PyExc_OSError); } - year = tm->tm_year + 1900; - month = tm->tm_mon + 1; - day = tm->tm_mday; - hour = tm->tm_hour; - minute = tm->tm_min; + year = tm.tm_year + 1900; + month = tm.tm_mon + 1; + day = tm.tm_mday; + hour = tm.tm_hour; + minute = tm.tm_min; /* The platform localtime/gmtime may insert leap seconds, - * indicated by tm->tm_sec > 59. We don't care about them, + * indicated by tm.tm_sec > 59. We don't care about them, * except to the extent that passing them on to the datetime * constructor would raise ValueError for a reason that * made no sense to the user. */ - second = Py_MIN(59, tm->tm_sec); - - if (tzinfo == Py_None && f == localtime) { + second = Py_MIN(59, tm.tm_sec); + + if (tzinfo == Py_None && f == localtime_r) { long long probe_seconds, result_seconds, transition; result_seconds = utc_to_seconds(year, month, day, @@ -4357,7 +4361,7 @@ return NULL; self = datetime_best_possible((PyObject *)type, - tz == Py_None ? localtime : gmtime, + tz == Py_None ? localtime_r : gmtime_r, tz); if (self != NULL && tz != Py_None) { /* Convert UTC to tzinfo's zone. */ @@ -4372,7 +4376,7 @@ static PyObject * datetime_utcnow(PyObject *cls, PyObject *dummy) { - return datetime_best_possible(cls, gmtime, Py_None); + return datetime_best_possible(cls, gmtime_r, Py_None); } /* Return new local datetime from timestamp (Python timestamp -- a double). */ @@ -4391,7 +4395,7 @@ return NULL; self = datetime_from_timestamp(cls, - tzinfo == Py_None ? localtime : gmtime, + tzinfo == Py_None ? localtime_r : gmtime_r, timestamp, tzinfo); if (self != NULL && tzinfo != Py_None) { @@ -4409,7 +4413,7 @@ PyObject *result = NULL; if (PyArg_ParseTuple(args, "O:utcfromtimestamp", ×tamp)) - result = datetime_from_timestamp(cls, gmtime, timestamp, + result = datetime_from_timestamp(cls, gmtime_r, timestamp, Py_None); return result; } @@ -5032,37 +5036,51 @@ { PyObject *result = NULL; PyObject *delta; - struct tm *local_time_tm; + struct tm local_time_tm; PyObject *nameo = NULL; const char *zone = NULL; - local_time_tm = localtime(×tamp); + if (localtime_r(×tamp, &local_time_tm) == NULL) { +#ifdef EINVAL + if (errno == 0) + errno = EINVAL; +#endif + PyErr_SetFromErrno(PyExc_OSError); + return NULL; + } #ifdef HAVE_STRUCT_TM_TM_ZONE - zone = local_time_tm->tm_zone; - delta = new_delta(0, local_time_tm->tm_gmtoff, 0, 1); + zone = local_time_tm.tm_zone; + delta = new_delta(0, local_time_tm.tm_gmtoff, 0, 1); #else /* HAVE_STRUCT_TM_TM_ZONE */ { PyObject *local_time, *utc_time; - struct tm *utc_time_tm; + struct tm utc_time_tm; char buf[100]; - strftime(buf, sizeof(buf), "%Z", local_time_tm); + strftime(buf, sizeof(buf), "%Z", &local_time_tm); zone = buf; - local_time = new_datetime(local_time_tm->tm_year + 1900, - local_time_tm->tm_mon + 1, - local_time_tm->tm_mday, - local_time_tm->tm_hour, - local_time_tm->tm_min, - local_time_tm->tm_sec, 0, Py_None, 0); + local_time = new_datetime(local_time_tm.tm_year + 1900, + local_time_tm.tm_mon + 1, + local_time_tm.tm_mday, + local_time_tm.tm_hour, + local_time_tm.tm_min, + local_time_tm.tm_sec, 0, Py_None, 0); if (local_time == NULL) { return NULL; } - utc_time_tm = gmtime(×tamp); - utc_time = new_datetime(utc_time_tm->tm_year + 1900, - utc_time_tm->tm_mon + 1, - utc_time_tm->tm_mday, - utc_time_tm->tm_hour, - utc_time_tm->tm_min, - utc_time_tm->tm_sec, 0, Py_None, 0); + if (gmtime(×tamp, &utc_time_tm) == NULL) { +#ifdef EINVAL + if (errno == 0) + errno = EINVAL; +#endif + PyErr_SetFromErrno(PyExc_OSError); + return NULL; + } + utc_time = new_datetime(utc_time_tm.tm_year + 1900, + utc_time_tm.tm_mon + 1, + utc_time_tm.tm_mday, + utc_time_tm.tm_hour, + utc_time_tm.tm_min, + utc_time_tm.tm_sec, 0, Py_None, 0); if (utc_time == NULL) { Py_DECREF(local_time); return NULL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 16:08:28 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Sat, 10 Sep 2016 20:08:28 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2328067=3A_Fixed_a_typo?= =?utf-8?q?=2E?= Message-ID: <20160910200828.69808.58231.F89BFE71@psf.io> https://hg.python.org/cpython/rev/5bdfe132e4ed changeset: 103583:5bdfe132e4ed user: Alexander Belopolsky date: Sat Sep 10 16:08:26 2016 -0400 summary: #28067: Fixed a typo. files: Modules/_datetimemodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -17,7 +17,7 @@ } static struct tm *gmtime_r(const time_t *timep, struct tm *result) { - if (gmime_s(result, timep) == 0) + if (gmtime_s(result, timep) == 0) return result; return NULL; } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 16:28:16 2016 From: python-checkins at python.org (terry.reedy) Date: Sat, 10 Sep 2016 20:28:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_IDLE_newx_item?= =?utf-8?q?s=2E?= Message-ID: <20160910202816.8517.73087.430CE489@psf.io> https://hg.python.org/cpython/rev/3ed2c0d55526 changeset: 103585:3ed2c0d55526 branch: 3.5 parent: 103580:94563ec74e1d user: Terry Jan Reedy date: Sat Sep 10 16:24:54 2016 -0400 summary: IDLE newx items. files: Lib/idlelib/NEWS.txt | 7 +++++++ Misc/NEWS | 7 +++++++ 2 files changed, 14 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,6 +2,13 @@ ========================= *Release date: 2017-01-01?* +- Issue #27922: Stop IDLE tests from 'flashing' gui widgets on the screen. + +- Add version to title of IDLE help window. + +- Issue #25564: In section on IDLE -- console differences, mention that + using exec means that __builtins__ is defined for each statement. + - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the -w option but without -jn. Fix warning from test_config. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -253,6 +253,13 @@ IDLE ---- +- Issue #27922: Stop IDLE tests from 'flashing' gui widgets on the screen. + +- Add version to title of IDLE help window. + +- Issue #25564: In section on IDLE -- console differences, mention that + using exec means that __builtins__ is defined for each statement. + - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the -w option but without -jn. Fix warning from test_config. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 16:28:16 2016 From: python-checkins at python.org (terry.reedy) Date: Sat, 10 Sep 2016 20:28:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_IDLE_newx_item?= =?utf-8?q?s=2E?= Message-ID: <20160910202816.19326.79821.2D298343@psf.io> https://hg.python.org/cpython/rev/168fe15798c6 changeset: 103584:168fe15798c6 branch: 2.7 parent: 103568:3219956eb703 user: Terry Jan Reedy date: Sat Sep 10 16:24:31 2016 -0400 summary: IDLE newx items. files: Lib/idlelib/NEWS.txt | 9 +++++++++ Misc/NEWS | 9 +++++++++ 2 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,6 +2,15 @@ ========================== *Release date: 2017-01-01?* +- Issue #27922: Stop IDLE tests from 'flashing' gui widgets on the screen. + +- Issue #17642: add larger font sizes for classroom projection. + +- Add version to title of IDLE help window. + +- Issue #25564: In section on IDLE -- console differences, mention that + using exec means that __builtins__ is defined for each statement. + - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the -w option but without -jn. Fix warning from test_config. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -142,6 +142,15 @@ IDLE ---- +- Issue #27922: Stop IDLE tests from 'flashing' gui widgets on the screen. + +- Issue #17642: add larger font sizes for classroom projection. + +- Add version to title of IDLE help window. + +- Issue #25564: In section on IDLE -- console differences, mention that + using exec means that __builtins__ is defined for each statement. + - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the -w option but without -jn. Fix warning from test_config. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 16:28:16 2016 From: python-checkins at python.org (terry.reedy) Date: Sat, 10 Sep 2016 20:28:16 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_IDLE_newx_items=2E_merge_from_3=2E5?= Message-ID: <20160910202816.13175.24850.80B1DE5B@psf.io> https://hg.python.org/cpython/rev/a4997914a011 changeset: 103586:a4997914a011 parent: 103583:5bdfe132e4ed parent: 103585:3ed2c0d55526 user: Terry Jan Reedy date: Sat Sep 10 16:28:01 2016 -0400 summary: IDLE newx items. merge from 3.5 files: Lib/idlelib/NEWS.txt | 16 +++++++++++++++- Misc/NEWS | 17 +++++++++++++++++ 2 files changed, 32 insertions(+), 1 deletions(-) diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -1,6 +1,20 @@ What's New in IDLE 3.6.0? =========================== -*Release date: 2016-09-??* +*Release date: 2016-12-16?* + +- Issue #27922: Stop IDLE tests from 'flashing' gui widgets on the screen. + +- Issue #27891: Consistently group and sort imports within idlelib modules. + +- Issue #17642: add larger font sizes for classroom projection. + +- Add version to title of IDLE help window. + +- Issue #25564: In section on IDLE -- console differences, mention that + using exec means that __builtins__ is defined for each statement. + +- Issue #27821: Fix 3.6.0a3 regression that prevented custom key sets + from being selected when no custom theme was defined. - Issue #27714: text_textview and test_autocomplete now pass when re-run in the same process. This occurs when test_idle fails when run with the diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -332,6 +332,23 @@ - Issue #21201: Improves readability of multiprocessing error message. Thanks to Wojciech Walczak for patch. +IDLE +---- + +- Issue #27922: Stop IDLE tests from 'flashing' gui widgets on the screen. + +- Issue #27891: Consistently group and sort imports within idlelib modules. + +- Issue #17642: add larger font sizes for classroom projection. + +- Add version to title of IDLE help window. + +- Issue #25564: In section on IDLE -- console differences, mention that + using exec means that __builtins__ is defined for each statement. + +- Issue #27821: Fix 3.6.0a3 regression that prevented custom key sets + from being selected when no custom theme was defined. + C API ----- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 16:43:54 2016 From: python-checkins at python.org (christian.heimes) Date: Sat, 10 Sep 2016 20:43:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_28043=3A_SSLContext_?= =?utf-8?q?has_improved_default_settings?= Message-ID: <20160910204354.19090.22572.7B21F9B2@psf.io> https://hg.python.org/cpython/rev/1b4c5d06c028 changeset: 103587:1b4c5d06c028 user: Christian Heimes date: Sat Sep 10 22:43:48 2016 +0200 summary: Issue 28043: SSLContext has improved default settings The options OP_NO_COMPRESSION, OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_DH_USE, OP_SINGLE_ECDH_USE, OP_NO_SSLv2 (except for PROTOCOL_SSLv2), and OP_NO_SSLv3 (except for PROTOCOL_SSLv3) are set by default. The initial cipher suite list contains only HIGH ciphers, no NULL ciphers and MD5 ciphers (except for PROTOCOL_SSLv2). files: Doc/library/ssl.rst | 9 ++++- Lib/ssl.py | 30 +++------------ Lib/test/test_ssl.py | 62 +++++++++++++++++-------------- Misc/NEWS | 4 ++ Modules/_ssl.c | 31 ++++++++++++++++ 5 files changed, 82 insertions(+), 54 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1191,7 +1191,14 @@ .. versionchanged:: 3.6 - :data:`PROTOCOL_TLS` is the default value. + The context is created with secure default values. The options + :data:`OP_NO_COMPRESSION`, :data:`OP_CIPHER_SERVER_PREFERENCE`, + :data:`OP_SINGLE_DH_USE`, :data:`OP_SINGLE_ECDH_USE`, + :data:`OP_NO_SSLv2` (except for :data:`PROTOCOL_SSLv2`), + and :data:`OP_NO_SSLv3` (except for :data:`PROTOCOL_SSLv3`) are + set by default. The initial cipher suite list contains only ``HIGH`` + ciphers, no ``NULL`` ciphers and no ``MD5`` ciphers (except for + :data:`PROTOCOL_SSLv2`). :class:`SSLContext` objects have the following methods and attributes: diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -488,32 +488,16 @@ if not isinstance(purpose, _ASN1Object): raise TypeError(purpose) + # SSLContext sets OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_COMPRESSION, + # OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_DH_USE and OP_SINGLE_ECDH_USE + # by default. context = SSLContext(PROTOCOL_TLS) - # SSLv2 considered harmful. - context.options |= OP_NO_SSLv2 - - # SSLv3 has problematic security and is only required for really old - # clients such as IE6 on Windows XP - context.options |= OP_NO_SSLv3 - - # disable compression to prevent CRIME attacks (OpenSSL 1.0+) - context.options |= getattr(_ssl, "OP_NO_COMPRESSION", 0) - if purpose == Purpose.SERVER_AUTH: # verify certs and host name in client mode context.verify_mode = CERT_REQUIRED context.check_hostname = True elif purpose == Purpose.CLIENT_AUTH: - # Prefer the server's ciphers by default so that we get stronger - # encryption - context.options |= getattr(_ssl, "OP_CIPHER_SERVER_PREFERENCE", 0) - - # Use single use keys in order to improve forward secrecy - context.options |= getattr(_ssl, "OP_SINGLE_DH_USE", 0) - context.options |= getattr(_ssl, "OP_SINGLE_ECDH_USE", 0) - - # disallow ciphers with known vulnerabilities context.set_ciphers(_RESTRICTED_SERVER_CIPHERS) if cafile or capath or cadata: @@ -539,12 +523,10 @@ if not isinstance(purpose, _ASN1Object): raise TypeError(purpose) + # SSLContext sets OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_COMPRESSION, + # OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_DH_USE and OP_SINGLE_ECDH_USE + # by default. context = SSLContext(protocol) - # SSLv2 considered harmful. - context.options |= OP_NO_SSLv2 - # SSLv3 has problematic security and is only required for really old - # clients such as IE6 on Windows XP - context.options |= OP_NO_SSLv3 if cert_reqs is not None: context.verify_mode = cert_reqs diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -80,6 +80,12 @@ DHFILE = data_file("dh1024.pem") BYTES_DHFILE = os.fsencode(DHFILE) +# Not defined in all versions of OpenSSL +OP_NO_COMPRESSION = getattr(ssl, "OP_NO_COMPRESSION", 0) +OP_SINGLE_DH_USE = getattr(ssl, "OP_SINGLE_DH_USE", 0) +OP_SINGLE_ECDH_USE = getattr(ssl, "OP_SINGLE_ECDH_USE", 0) +OP_CIPHER_SERVER_PREFERENCE = getattr(ssl, "OP_CIPHER_SERVER_PREFERENCE", 0) + def handle_error(prefix): exc_format = ' '.join(traceback.format_exception(*sys.exc_info())) @@ -870,8 +876,9 @@ ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) # OP_ALL | OP_NO_SSLv2 | OP_NO_SSLv3 is the default value default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3) - if not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0): - default |= ssl.OP_NO_COMPRESSION + # SSLContext also enables these by default + default |= (OP_NO_COMPRESSION | OP_CIPHER_SERVER_PREFERENCE | + OP_SINGLE_DH_USE | OP_SINGLE_ECDH_USE) self.assertEqual(default, ctx.options) ctx.options |= ssl.OP_NO_TLSv1 self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options) @@ -1236,16 +1243,29 @@ stats["x509"] += 1 self.assertEqual(ctx.cert_store_stats(), stats) + def _assert_context_options(self, ctx): + self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) + if OP_NO_COMPRESSION != 0: + self.assertEqual(ctx.options & OP_NO_COMPRESSION, + OP_NO_COMPRESSION) + if OP_SINGLE_DH_USE != 0: + self.assertEqual(ctx.options & OP_SINGLE_DH_USE, + OP_SINGLE_DH_USE) + if OP_SINGLE_ECDH_USE != 0: + self.assertEqual(ctx.options & OP_SINGLE_ECDH_USE, + OP_SINGLE_ECDH_USE) + if OP_CIPHER_SERVER_PREFERENCE != 0: + self.assertEqual(ctx.options & OP_CIPHER_SERVER_PREFERENCE, + OP_CIPHER_SERVER_PREFERENCE) + def test_create_default_context(self): ctx = ssl.create_default_context() + self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) self.assertTrue(ctx.check_hostname) - self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) - self.assertEqual( - ctx.options & getattr(ssl, "OP_NO_COMPRESSION", 0), - getattr(ssl, "OP_NO_COMPRESSION", 0), - ) + self._assert_context_options(ctx) + with open(SIGNING_CA) as f: cadata = f.read() @@ -1253,40 +1273,24 @@ cadata=cadata) self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) - self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) - self.assertEqual( - ctx.options & getattr(ssl, "OP_NO_COMPRESSION", 0), - getattr(ssl, "OP_NO_COMPRESSION", 0), - ) + self._assert_context_options(ctx) ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) self.assertEqual(ctx.verify_mode, ssl.CERT_NONE) - self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) - self.assertEqual( - ctx.options & getattr(ssl, "OP_NO_COMPRESSION", 0), - getattr(ssl, "OP_NO_COMPRESSION", 0), - ) - self.assertEqual( - ctx.options & getattr(ssl, "OP_SINGLE_DH_USE", 0), - getattr(ssl, "OP_SINGLE_DH_USE", 0), - ) - self.assertEqual( - ctx.options & getattr(ssl, "OP_SINGLE_ECDH_USE", 0), - getattr(ssl, "OP_SINGLE_ECDH_USE", 0), - ) + self._assert_context_options(ctx) def test__create_stdlib_context(self): ctx = ssl._create_stdlib_context() self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) self.assertEqual(ctx.verify_mode, ssl.CERT_NONE) self.assertFalse(ctx.check_hostname) - self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) + self._assert_context_options(ctx) ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1) self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLSv1) self.assertEqual(ctx.verify_mode, ssl.CERT_NONE) - self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) + self._assert_context_options(ctx) ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1, cert_reqs=ssl.CERT_REQUIRED, @@ -1294,12 +1298,12 @@ self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLSv1) self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) self.assertTrue(ctx.check_hostname) - self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) + self._assert_context_options(ctx) ctx = ssl._create_stdlib_context(purpose=ssl.Purpose.CLIENT_AUTH) self.assertEqual(ctx.protocol, ssl.PROTOCOL_SSLv23) self.assertEqual(ctx.verify_mode, ssl.CERT_NONE) - self.assertEqual(ctx.options & ssl.OP_NO_SSLv2, ssl.OP_NO_SSLv2) + self._assert_context_options(ctx) def test_check_hostname(self): ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -138,6 +138,10 @@ Library ------- +- Issue 28043: SSLContext has improved default settings: OP_NO_SSLv2, + OP_NO_SSLv3, OP_NO_COMPRESSION, OP_CIPHER_SERVER_PREFERENCE, + OP_SINGLE_DH_USE, OP_SINGLE_ECDH_USE and HIGH ciphers without MD5. + - Issue #24693: Changed some RuntimeError's in the zipfile module to more appropriate types. Improved some error messages and debugging output. diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2407,6 +2407,7 @@ PySSLContext *self; long options; SSL_CTX *ctx = NULL; + int result; #if defined(SSL_MODE_RELEASE_BUFFERS) unsigned long libver; #endif @@ -2470,8 +2471,38 @@ options |= SSL_OP_NO_SSLv2; if (proto_version != PY_SSL_VERSION_SSL3) options |= SSL_OP_NO_SSLv3; + /* Minimal security flags for server and client side context. + * Client sockets ignore server-side parameters. */ +#ifdef SSL_OP_NO_COMPRESSION + options |= SSL_OP_NO_COMPRESSION; +#endif +#ifdef SSL_OP_CIPHER_SERVER_PREFERENCE + options |= SSL_OP_CIPHER_SERVER_PREFERENCE; +#endif +#ifdef SSL_OP_SINGLE_DH_USE + options |= SSL_OP_SINGLE_DH_USE; +#endif +#ifdef SSL_OP_SINGLE_ECDH_USE + options |= SSL_OP_SINGLE_ECDH_USE; +#endif SSL_CTX_set_options(self->ctx, options); + /* A bare minimum cipher list without completly broken cipher suites. + * It's far from perfect but gives users a better head start. */ + if (proto_version != PY_SSL_VERSION_SSL2) { + result = SSL_CTX_set_cipher_list(ctx, "HIGH:!aNULL:!eNULL:!MD5"); + } else { + /* SSLv2 needs MD5 */ + result = SSL_CTX_set_cipher_list(ctx, "HIGH:!aNULL:!eNULL"); + } + if (result == 0) { + Py_DECREF(self); + ERR_clear_error(); + PyErr_SetString(PySSLErrorObject, + "No cipher can be selected."); + return NULL; + } + #if defined(SSL_MODE_RELEASE_BUFFERS) /* Set SSL_MODE_RELEASE_BUFFERS. This potentially greatly reduces memory usage for no cost at all. However, don't do this for OpenSSL versions -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 16:51:19 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Sat, 10 Sep 2016 20:51:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_=2328067=3A_Fixed_another_?= =?utf-8?q?typo=2E?= Message-ID: <20160910205119.59475.4871.9987B4A8@psf.io> https://hg.python.org/cpython/rev/c7e477fa9e09 changeset: 103588:c7e477fa9e09 user: Alexander Belopolsky date: Sat Sep 10 16:51:17 2016 -0400 summary: #28067: Fixed another typo. files: Modules/_datetimemodule.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -5067,7 +5067,7 @@ if (local_time == NULL) { return NULL; } - if (gmtime(×tamp, &utc_time_tm) == NULL) { + if (gmtime_r(×tamp, &utc_time_tm) == NULL) { #ifdef EINVAL if (errno == 0) errno = EINVAL; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 17:23:49 2016 From: python-checkins at python.org (christian.heimes) Date: Sat, 10 Sep 2016 21:23:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328022=3A_Deprecat?= =?utf-8?q?e_ssl-related_arguments_in_favor_of_SSLContext=2E?= Message-ID: <20160910212349.20970.44953.169A796D@psf.io> https://hg.python.org/cpython/rev/aed3c541b2f1 changeset: 103589:aed3c541b2f1 user: Christian Heimes date: Sat Sep 10 23:23:33 2016 +0200 summary: Issue #28022: Deprecate ssl-related arguments in favor of SSLContext. The deprecation include manual creation of SSLSocket and certfile/keyfile (or similar) in ftplib, httplib, imaplib, smtplib, poplib and urllib. ssl.wrap_socket() is not marked as deprecated yet. files: Doc/library/ftplib.rst | 7 + Doc/library/http.client.rst | 18 ++- Doc/library/imaplib.rst | 8 + Doc/library/poplib.rst | 7 + Doc/library/smtplib.rst | 8 + Doc/library/ssl.rst | 13 ++- Doc/library/urllib.request.rst | 6 + Lib/asyncio/test_utils.py | 8 +- Lib/ftplib.py | 4 + Lib/http/client.py | 6 + Lib/imaplib.py | 5 +- Lib/poplib.py | 4 + Lib/smtplib.py | 8 + Lib/ssl.py | 1 - Lib/test/test_ftplib.py | 10 +- Lib/test/test_imaplib.py | 6 +- Lib/test/test_nntplib.py | 6 +- Lib/test/test_poplib.py | 10 +- Lib/test/test_ssl.py | 87 +++++++++++------- Lib/test/test_urllib.py | 9 +- Lib/test/test_urllib2_localnet.py | 34 +++--- Lib/urllib/request.py | 3 + Misc/NEWS | 6 +- 23 files changed, 189 insertions(+), 85 deletions(-) diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -97,6 +97,13 @@ :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see :data:`ssl.HAS_SNI`). + .. deprecated:: 3.6 + + *keyfile* and *certfile* are deprecated in favor of *context*. + Please use :meth:`ssl.SSLContext.load_cert_chain` instead, or let + :func:`ssl.create_default_context` select the system's trusted CA + certificates for you. + Here's a sample session using the :class:`FTP_TLS` class:: >>> ftps = FTP_TLS('ftp.pureftpd.org') diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -69,13 +69,6 @@ must be a :class:`ssl.SSLContext` instance describing the various SSL options. - *key_file* and *cert_file* are deprecated, please use - :meth:`ssl.SSLContext.load_cert_chain` instead, or let - :func:`ssl.create_default_context` select the system's trusted CA - certificates for you. The *check_hostname* parameter is also deprecated; the - :attr:`ssl.SSLContext.check_hostname` attribute of *context* should be used - instead. - Please read :ref:`ssl-security` for more information on best practices. .. versionchanged:: 3.2 @@ -95,6 +88,17 @@ :func:`ssl._create_unverified_context` can be passed to the *context* parameter. + .. deprecated:: 3.6 + + *key_file* and *cert_file* are deprecated in favor of *context*. + Please use :meth:`ssl.SSLContext.load_cert_chain` instead, or let + :func:`ssl.create_default_context` select the system's trusted CA + certificates for you. + + The *check_hostname* parameter is also deprecated; the + :attr:`ssl.SSLContext.check_hostname` attribute of *context* should + be used instead. + .. class:: HTTPResponse(sock, debuglevel=0, method=None, url=None) diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -103,6 +103,14 @@ :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see :data:`ssl.HAS_SNI`). + .. deprecated:: 3.6 + + *keyfile* and *certfile* are deprecated in favor of *ssl_context*. + Please use :meth:`ssl.SSLContext.load_cert_chain` instead, or let + :func:`ssl.create_default_context` select the system's trusted CA + certificates for you. + + The second subclass allows for connections created by a child process: diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -62,6 +62,13 @@ :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see :data:`ssl.HAS_SNI`). + .. deprecated:: 3.6 + + *keyfile* and *certfile* are deprecated in favor of *context*. + Please use :meth:`ssl.SSLContext.load_cert_chain` instead, or let + :func:`ssl.create_default_context` select the system's trusted CA + certificates for you. + One exception is defined as an attribute of the :mod:`poplib` module: diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -95,6 +95,14 @@ :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see :data:`ssl.HAS_SNI`). + .. deprecated:: 3.6 + + *keyfile* and *certfile* are deprecated in favor of *context*. + Please use :meth:`ssl.SSLContext.load_cert_chain` instead, or let + :func:`ssl.create_default_context` select the system's trusted CA + certificates for you. + + .. class:: LMTP(host='', port=LMTP_PORT, local_hostname=None, source_address=None) The LMTP protocol, which is very similar to ESMTP, is heavily based on the diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -230,7 +230,6 @@ .. versionchanged:: 3.2 New optional argument *ciphers*. - Context creation ^^^^^^^^^^^^^^^^ @@ -925,7 +924,7 @@ :ref:`notes on non-blocking sockets `. Usually, :class:`SSLSocket` are not created directly, but using the - :func:`wrap_socket` function or the :meth:`SSLContext.wrap_socket` method. + the :meth:`SSLContext.wrap_socket` method. .. versionchanged:: 3.5 The :meth:`sendfile` method was added. @@ -935,6 +934,10 @@ are received or sent. The socket timeout is now to maximum total duration of the shutdown. + .. deprecated:: 3.6 + It is deprecated to create a :class:`SSLSocket` instance directly, use + :meth:`SSLContext.wrap_socket` to wrap a socket. + SSL sockets also have the following additional methods and attributes: @@ -955,6 +958,9 @@ The socket timeout is now to maximum total duration to read up to *len* bytes. + .. deprecated:: 3.6 + Use :meth:`~SSLSocket.recv` instead of :meth:`~SSLSocket.read`. + .. method:: SSLSocket.write(buf) Write *buf* to the SSL socket and return the number of bytes written. The @@ -970,6 +976,9 @@ The socket timeout is no more reset each time bytes are received or sent. The socket timeout is now to maximum total duration to write *buf*. + .. deprecated:: 3.6 + Use :meth:`~SSLSocket.send` instead of :meth:`~SSLSocket.write`. + .. note:: The :meth:`~SSLSocket.read` and :meth:`~SSLSocket.write` methods are the diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -111,6 +111,12 @@ .. versionchanged:: 3.4.3 *context* was added. + .. deprecated:: 3.6 + + *cafile*, *capath* and *cadefault* are deprecated in favor of *context*. + Please use :meth:`ssl.SSLContext.load_cert_chain` instead, or let + :func:`ssl.create_default_context` select the system's trusted CA + certificates for you. .. function:: install_opener(opener) diff --git a/Lib/asyncio/test_utils.py b/Lib/asyncio/test_utils.py --- a/Lib/asyncio/test_utils.py +++ b/Lib/asyncio/test_utils.py @@ -117,10 +117,10 @@ 'test', 'test_asyncio') keyfile = os.path.join(here, 'ssl_key.pem') certfile = os.path.join(here, 'ssl_cert.pem') - ssock = ssl.wrap_socket(request, - keyfile=keyfile, - certfile=certfile, - server_side=True) + context = ssl.SSLContext() + context.load_cert_chain(certfile, keyfile) + + ssock = context.wrap_socket(request, server_side=True) try: self.RequestHandlerClass(ssock, client_address, self) ssock.close() diff --git a/Lib/ftplib.py b/Lib/ftplib.py --- a/Lib/ftplib.py +++ b/Lib/ftplib.py @@ -728,6 +728,10 @@ if context is not None and certfile is not None: raise ValueError("context and certfile arguments are mutually " "exclusive") + if keyfile is not None or certfile is not None: + import warnings + warnings.warn("keyfile and certfile are deprecated, use a" + "custom context instead", DeprecationWarning, 2) self.keyfile = keyfile self.certfile = certfile if context is None: diff --git a/Lib/http/client.py b/Lib/http/client.py --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -1365,6 +1365,12 @@ check_hostname=None): super(HTTPSConnection, self).__init__(host, port, timeout, source_address) + if (key_file is not None or cert_file is not None or + check_hostname is not None): + import warnings + warnings.warn("key_file, cert_file and check_hostname are " + "deprecated, use a custom context instead.", + DeprecationWarning, 2) self.key_file = key_file self.cert_file = cert_file if context is None: diff --git a/Lib/imaplib.py b/Lib/imaplib.py --- a/Lib/imaplib.py +++ b/Lib/imaplib.py @@ -1267,7 +1267,10 @@ if ssl_context is not None and certfile is not None: raise ValueError("ssl_context and certfile arguments are mutually " "exclusive") - + if keyfile is not None or certfile is not None: + import warnings + warnings.warn("keyfile and certfile are deprecated, use a" + "custom ssl_context instead", DeprecationWarning, 2) self.keyfile = keyfile self.certfile = certfile if ssl_context is None: diff --git a/Lib/poplib.py b/Lib/poplib.py --- a/Lib/poplib.py +++ b/Lib/poplib.py @@ -431,6 +431,10 @@ if context is not None and certfile is not None: raise ValueError("context and certfile arguments are mutually " "exclusive") + if keyfile is not None or certfile is not None: + import warnings + warnings.warn("keyfile and certfile are deprecated, use a" + "custom context instead", DeprecationWarning, 2) self.keyfile = keyfile self.certfile = certfile if context is None: diff --git a/Lib/smtplib.py b/Lib/smtplib.py --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -759,6 +759,10 @@ if context is not None and certfile is not None: raise ValueError("context and certfile arguments are mutually " "exclusive") + if keyfile is not None or certfile is not None: + import warnings + warnings.warn("keyfile and certfile are deprecated, use a" + "custom context instead", DeprecationWarning, 2) if context is None: context = ssl._create_stdlib_context(certfile=certfile, keyfile=keyfile) @@ -1011,6 +1015,10 @@ if context is not None and certfile is not None: raise ValueError("context and certfile arguments are mutually " "exclusive") + if keyfile is not None or certfile is not None: + import warnings + warnings.warn("keyfile and certfile are deprecated, use a" + "custom context instead", DeprecationWarning, 2) self.keyfile = keyfile self.certfile = certfile if context is None: diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -1091,7 +1091,6 @@ do_handshake_on_connect=True, suppress_ragged_eofs=True, ciphers=None): - return SSLSocket(sock=sock, keyfile=keyfile, certfile=certfile, server_side=server_side, cert_reqs=cert_reqs, ssl_version=ssl_version, ca_certs=ca_certs, diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py --- a/Lib/test/test_ftplib.py +++ b/Lib/test/test_ftplib.py @@ -311,10 +311,12 @@ _ssl_closing = False def secure_connection(self): - socket = ssl.wrap_socket(self.socket, suppress_ragged_eofs=False, - certfile=CERTFILE, server_side=True, - do_handshake_on_connect=False, - ssl_version=ssl.PROTOCOL_SSLv23) + context = ssl.SSLContext() + context.load_cert_chain(CERTFILE) + socket = context.wrap_socket(self.socket, + suppress_ragged_eofs=False, + server_side=True, + do_handshake_on_connect=False) self.del_channel() self.set_socket(socket) self._ssl_accepting = True diff --git a/Lib/test/test_imaplib.py b/Lib/test/test_imaplib.py --- a/Lib/test/test_imaplib.py +++ b/Lib/test/test_imaplib.py @@ -77,9 +77,9 @@ def get_request(self): newsocket, fromaddr = self.socket.accept() - connstream = ssl.wrap_socket(newsocket, - server_side=True, - certfile=CERTFILE) + context = ssl.SSLContext() + context.load_cert_chain(CERTFILE) + connstream = context.wrap_socket(newsocket, server_side=True) return connstream, fromaddr IMAP4_SSL = imaplib.IMAP4_SSL diff --git a/Lib/test/test_nntplib.py b/Lib/test/test_nntplib.py --- a/Lib/test/test_nntplib.py +++ b/Lib/test/test_nntplib.py @@ -1542,8 +1542,10 @@ elif cmd == b'STARTTLS\r\n': reader.close() client.sendall(b'382 Begin TLS negotiation now\r\n') - client = ssl.wrap_socket( - client, server_side=True, certfile=certfile) + context = ssl.SSLContext() + context.load_cert_chain(certfile) + client = context.wrap_socket( + client, server_side=True) cleanup.enter_context(client) reader = cleanup.enter_context(client.makefile('rb')) elif cmd == b'QUIT\r\n': diff --git a/Lib/test/test_poplib.py b/Lib/test/test_poplib.py --- a/Lib/test/test_poplib.py +++ b/Lib/test/test_poplib.py @@ -152,10 +152,12 @@ def cmd_stls(self, arg): if self.tls_active is False: self.push('+OK Begin TLS negotiation') - tls_sock = ssl.wrap_socket(self.socket, certfile=CERTFILE, - server_side=True, - do_handshake_on_connect=False, - suppress_ragged_eofs=False) + context = ssl.SSLContext() + context.load_cert_chain(CERTFILE) + tls_sock = context.wrap_socket(self.socket, + server_side=True, + do_handshake_on_connect=False, + suppress_ragged_eofs=False) self.del_channel() self.set_socket(tls_sock) self.tls_active = True diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -143,6 +143,21 @@ needs_sni = unittest.skipUnless(ssl.HAS_SNI, "SNI support needed for this test") +def test_wrap_socket(sock, ssl_version=ssl.PROTOCOL_TLS, *, + cert_reqs=ssl.CERT_NONE, ca_certs=None, + ciphers=None, certfile=None, keyfile=None, + **kwargs): + context = ssl.SSLContext(ssl_version) + if cert_reqs is not None: + context.verify_mode = cert_reqs + if ca_certs is not None: + context.load_verify_locations(ca_certs) + if certfile is not None or keyfile is not None: + context.load_cert_chain(certfile, keyfile) + if ciphers is not None: + context.set_ciphers(ciphers) + return context.wrap_socket(sock, **kwargs) + class BasicSocketTests(unittest.TestCase): def test_constants(self): @@ -363,7 +378,7 @@ # Issue #7943: an SSL object doesn't create reference cycles with # itself. s = socket.socket(socket.AF_INET) - ss = ssl.wrap_socket(s) + ss = test_wrap_socket(s) wr = weakref.ref(ss) with support.check_warnings(("", ResourceWarning)): del ss @@ -373,7 +388,7 @@ # Methods on an unconnected SSLSocket propagate the original # OSError raise by the underlying socket object. s = socket.socket(socket.AF_INET) - with ssl.wrap_socket(s) as ss: + with test_wrap_socket(s) as ss: self.assertRaises(OSError, ss.recv, 1) self.assertRaises(OSError, ss.recv_into, bytearray(b'x')) self.assertRaises(OSError, ss.recvfrom, 1) @@ -387,10 +402,10 @@ for timeout in (None, 0.0, 5.0): s = socket.socket(socket.AF_INET) s.settimeout(timeout) - with ssl.wrap_socket(s) as ss: + with test_wrap_socket(s) as ss: self.assertEqual(timeout, ss.gettimeout()) - def test_errors(self): + def test_errors_sslwrap(self): sock = socket.socket() self.assertRaisesRegex(ValueError, "certfile must be specified", @@ -400,10 +415,10 @@ ssl.wrap_socket, sock, server_side=True) self.assertRaisesRegex(ValueError, "certfile must be specified for server-side operations", - ssl.wrap_socket, sock, server_side=True, certfile="") + ssl.wrap_socket, sock, server_side=True, certfile="") with ssl.wrap_socket(sock, server_side=True, certfile=CERTFILE) as s: self.assertRaisesRegex(ValueError, "can't connect in server-side mode", - s.connect, (HOST, 8080)) + s.connect, (HOST, 8080)) with self.assertRaises(OSError) as cm: with socket.socket() as sock: ssl.wrap_socket(sock, certfile=NONEXISTINGCERT) @@ -426,7 +441,7 @@ sock = socket.socket() self.addCleanup(sock.close) with self.assertRaises(ssl.SSLError): - ssl.wrap_socket(sock, + test_wrap_socket(sock, certfile=certfile, ssl_version=ssl.PROTOCOL_TLSv1) @@ -613,7 +628,7 @@ s.listen() c = socket.socket(socket.AF_INET) c.connect(s.getsockname()) - with ssl.wrap_socket(c, do_handshake_on_connect=False) as ss: + with test_wrap_socket(c, do_handshake_on_connect=False) as ss: with self.assertRaises(ValueError): ss.get_channel_binding("unknown-type") s.close() @@ -623,15 +638,15 @@ def test_tls_unique_channel_binding(self): # unconnected should return None for known type s = socket.socket(socket.AF_INET) - with ssl.wrap_socket(s) as ss: + with test_wrap_socket(s) as ss: self.assertIsNone(ss.get_channel_binding("tls-unique")) # the same for server-side s = socket.socket(socket.AF_INET) - with ssl.wrap_socket(s, server_side=True, certfile=CERTFILE) as ss: + with test_wrap_socket(s, server_side=True, certfile=CERTFILE) as ss: self.assertIsNone(ss.get_channel_binding("tls-unique")) def test_dealloc_warn(self): - ss = ssl.wrap_socket(socket.socket(socket.AF_INET)) + ss = test_wrap_socket(socket.socket(socket.AF_INET)) r = repr(ss) with self.assertWarns(ResourceWarning) as cm: ss = None @@ -750,7 +765,7 @@ s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) self.addCleanup(s.close) with self.assertRaises(NotImplementedError) as cx: - ssl.wrap_socket(s, cert_reqs=ssl.CERT_NONE) + test_wrap_socket(s, cert_reqs=ssl.CERT_NONE) self.assertEqual(str(cx.exception), "only stream sockets are supported") ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23) with self.assertRaises(NotImplementedError) as cx: @@ -826,7 +841,7 @@ server = socket.socket(socket.AF_INET) self.addCleanup(server.close) port = support.bind_port(server) # Reserve port but don't listen - s = ssl.wrap_socket(socket.socket(socket.AF_INET), + s = test_wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED) self.addCleanup(s.close) rc = s.connect_ex((HOST, port)) @@ -1444,13 +1459,13 @@ self.addCleanup(server.__exit__, None, None, None) def test_connect(self): - with ssl.wrap_socket(socket.socket(socket.AF_INET), + with test_wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE) as s: s.connect(self.server_addr) self.assertEqual({}, s.getpeercert()) # this should succeed because we specify the root cert - with ssl.wrap_socket(socket.socket(socket.AF_INET), + with test_wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, ca_certs=SIGNING_CA) as s: s.connect(self.server_addr) @@ -1460,7 +1475,7 @@ # This should fail because we have no verification certs. Connection # failure crashes ThreadedEchoServer, so run this in an independent # test method. - s = ssl.wrap_socket(socket.socket(socket.AF_INET), + s = test_wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED) self.addCleanup(s.close) self.assertRaisesRegex(ssl.SSLError, "certificate verify failed", @@ -1468,7 +1483,7 @@ def test_connect_ex(self): # Issue #11326: check connect_ex() implementation - s = ssl.wrap_socket(socket.socket(socket.AF_INET), + s = test_wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, ca_certs=SIGNING_CA) self.addCleanup(s.close) @@ -1478,7 +1493,7 @@ def test_non_blocking_connect_ex(self): # Issue #11326: non-blocking connect_ex() should allow handshake # to proceed after the socket gets ready. - s = ssl.wrap_socket(socket.socket(socket.AF_INET), + s = test_wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, ca_certs=SIGNING_CA, do_handshake_on_connect=False) @@ -1578,7 +1593,7 @@ # Issue #5238: creating a file-like object with makefile() shouldn't # delay closing the underlying "real socket" (here tested with its # file descriptor, hence skipping the test under Windows). - ss = ssl.wrap_socket(socket.socket(socket.AF_INET)) + ss = test_wrap_socket(socket.socket(socket.AF_INET)) ss.connect(self.server_addr) fd = ss.fileno() f = ss.makefile() @@ -1596,7 +1611,7 @@ s = socket.socket(socket.AF_INET) s.connect(self.server_addr) s.setblocking(False) - s = ssl.wrap_socket(s, + s = test_wrap_socket(s, cert_reqs=ssl.CERT_NONE, do_handshake_on_connect=False) self.addCleanup(s.close) @@ -1622,16 +1637,16 @@ _test_get_server_certificate_fail(self, *self.server_addr) def test_ciphers(self): - with ssl.wrap_socket(socket.socket(socket.AF_INET), + with test_wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE, ciphers="ALL") as s: s.connect(self.server_addr) - with ssl.wrap_socket(socket.socket(socket.AF_INET), + with test_wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_NONE, ciphers="DEFAULT") as s: s.connect(self.server_addr) # Error checking can happen at instantiation or when connecting with self.assertRaisesRegex(ssl.SSLError, "No cipher can be selected"): with socket.socket(socket.AF_INET) as sock: - s = ssl.wrap_socket(sock, + s = test_wrap_socket(sock, cert_reqs=ssl.CERT_NONE, ciphers="^$:,;?*'dorothyx") s.connect(self.server_addr) @@ -1749,7 +1764,7 @@ # Issue #12065: on a timeout, connect_ex() should return the original # errno (mimicking the behaviour of non-SSL sockets). with support.transient_internet(REMOTE_HOST): - s = ssl.wrap_socket(socket.socket(socket.AF_INET), + s = test_wrap_socket(socket.socket(socket.AF_INET), cert_reqs=ssl.CERT_REQUIRED, do_handshake_on_connect=False) self.addCleanup(s.close) @@ -2040,7 +2055,7 @@ class ConnectionHandler (asyncore.dispatcher_with_send): def __init__(self, conn, certfile): - self.socket = ssl.wrap_socket(conn, server_side=True, + self.socket = test_wrap_socket(conn, server_side=True, certfile=certfile, do_handshake_on_connect=False) asyncore.dispatcher_with_send.__init__(self, self.socket) @@ -2401,7 +2416,7 @@ connectionchatty=False) with server, \ socket.socket() as sock, \ - ssl.wrap_socket(sock, + test_wrap_socket(sock, certfile=certfile, ssl_version=ssl.PROTOCOL_TLSv1) as s: try: @@ -2448,7 +2463,7 @@ c.connect((HOST, port)) listener_gone.wait() try: - ssl_sock = ssl.wrap_socket(c) + ssl_sock = test_wrap_socket(c) except OSError: pass else: @@ -2638,7 +2653,7 @@ sys.stdout.write( " client: read %r from server, starting TLS...\n" % msg) - conn = ssl.wrap_socket(s, ssl_version=ssl.PROTOCOL_TLSv1) + conn = test_wrap_socket(s, ssl_version=ssl.PROTOCOL_TLSv1) wrapped = True elif indata == b"ENDTLS" and msg.startswith(b"ok"): # ENDTLS ok, switch back to clear text @@ -2699,7 +2714,7 @@ indata = b"FOO\n" server = AsyncoreEchoServer(CERTFILE) with server: - s = ssl.wrap_socket(socket.socket()) + s = test_wrap_socket(socket.socket()) s.connect(('127.0.0.1', server.port)) if support.verbose: sys.stdout.write( @@ -2732,7 +2747,7 @@ chatty=True, connectionchatty=False) with server: - s = ssl.wrap_socket(socket.socket(), + s = test_wrap_socket(socket.socket(), server_side=False, certfile=CERTFILE, ca_certs=CERTFILE, @@ -2856,7 +2871,7 @@ self.addCleanup(server.__exit__, None, None) s = socket.create_connection((HOST, server.port)) self.addCleanup(s.close) - s = ssl.wrap_socket(s, suppress_ragged_eofs=False) + s = test_wrap_socket(s, suppress_ragged_eofs=False) self.addCleanup(s.close) # recv/read(0) should return no data @@ -2878,7 +2893,7 @@ chatty=True, connectionchatty=False) with server: - s = ssl.wrap_socket(socket.socket(), + s = test_wrap_socket(socket.socket(), server_side=False, certfile=CERTFILE, ca_certs=CERTFILE, @@ -2932,12 +2947,12 @@ c.connect((host, port)) # Will attempt handshake and time out self.assertRaisesRegex(socket.timeout, "timed out", - ssl.wrap_socket, c) + test_wrap_socket, c) finally: c.close() try: c = socket.socket(socket.AF_INET) - c = ssl.wrap_socket(c) + c = test_wrap_socket(c) c.settimeout(0.2) # Will attempt handshake and time out self.assertRaisesRegex(socket.timeout, "timed out", @@ -3062,7 +3077,7 @@ chatty=True, connectionchatty=False) with server: - s = ssl.wrap_socket(socket.socket(), + s = test_wrap_socket(socket.socket(), server_side=False, certfile=CERTFILE, ca_certs=CERTFILE, @@ -3087,7 +3102,7 @@ s.close() # now, again - s = ssl.wrap_socket(socket.socket(), + s = test_wrap_socket(socket.socket(), server_side=False, certfile=CERTFILE, ca_certs=CERTFILE, diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py --- a/Lib/test/test_urllib.py +++ b/Lib/test/test_urllib.py @@ -469,10 +469,11 @@ @unittest.skipUnless(ssl, "ssl module required") def test_cafile_and_context(self): context = ssl.create_default_context() - with self.assertRaises(ValueError): - urllib.request.urlopen( - "https://localhost", cafile="/nonexistent/path", context=context - ) + with support.check_warnings(('', DeprecationWarning)): + with self.assertRaises(ValueError): + urllib.request.urlopen( + "https://localhost", cafile="/nonexistent/path", context=context + ) class urlopen_DataTests(unittest.TestCase): """Test urlopen() opening a data URL.""" diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py --- a/Lib/test/test_urllib2_localnet.py +++ b/Lib/test/test_urllib2_localnet.py @@ -548,26 +548,28 @@ def test_https_with_cafile(self): handler = self.start_https_server(certfile=CERT_localhost) - # Good cert - data = self.urlopen("https://localhost:%s/bizarre" % handler.port, - cafile=CERT_localhost) - self.assertEqual(data, b"we care a bit") - # Bad cert - with self.assertRaises(urllib.error.URLError) as cm: - self.urlopen("https://localhost:%s/bizarre" % handler.port, - cafile=CERT_fakehostname) - # Good cert, but mismatching hostname - handler = self.start_https_server(certfile=CERT_fakehostname) - with self.assertRaises(ssl.CertificateError) as cm: - self.urlopen("https://localhost:%s/bizarre" % handler.port, - cafile=CERT_fakehostname) + with support.check_warnings(('', DeprecationWarning)): + # Good cert + data = self.urlopen("https://localhost:%s/bizarre" % handler.port, + cafile=CERT_localhost) + self.assertEqual(data, b"we care a bit") + # Bad cert + with self.assertRaises(urllib.error.URLError) as cm: + self.urlopen("https://localhost:%s/bizarre" % handler.port, + cafile=CERT_fakehostname) + # Good cert, but mismatching hostname + handler = self.start_https_server(certfile=CERT_fakehostname) + with self.assertRaises(ssl.CertificateError) as cm: + self.urlopen("https://localhost:%s/bizarre" % handler.port, + cafile=CERT_fakehostname) def test_https_with_cadefault(self): handler = self.start_https_server(certfile=CERT_localhost) # Self-signed cert should fail verification with system certificate store - with self.assertRaises(urllib.error.URLError) as cm: - self.urlopen("https://localhost:%s/bizarre" % handler.port, - cadefault=True) + with support.check_warnings(('', DeprecationWarning)): + with self.assertRaises(urllib.error.URLError) as cm: + self.urlopen("https://localhost:%s/bizarre" % handler.port, + cadefault=True) def test_https_sni(self): if ssl is None: diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -198,6 +198,9 @@ ''' global _opener if cafile or capath or cadefault: + import warnings + warnings.warn("cafile, cpath and cadefault are deprecated, use a " + "custom context instead.", DeprecationWarning, 2) if context is not None: raise ValueError( "You can't pass both context and any of cafile, capath, and " diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -138,7 +138,11 @@ Library ------- -- Issue 28043: SSLContext has improved default settings: OP_NO_SSLv2, +- Issue #28022: Deprecate ssl-related arguments in favor of SSLContext. The + deprecation include manual creation of SSLSocket and certfile/keyfile + (or similar) in ftplib, httplib, imaplib, smtplib, poplib and urllib. + +- Issue #28043: SSLContext has improved default settings: OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_COMPRESSION, OP_CIPHER_SERVER_PREFERENCE, OP_SINGLE_DH_USE, OP_SINGLE_ECDH_USE and HIGH ciphers without MD5. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 17:45:00 2016 From: python-checkins at python.org (christian.heimes) Date: Sat, 10 Sep 2016 21:45:00 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2319500=3A_Add_clie?= =?utf-8?q?nt-side_SSL_session_resumption_to_the_ssl_module=2E?= Message-ID: <20160910214500.13088.5573.24B24EBD@psf.io> https://hg.python.org/cpython/rev/6f2644738876 changeset: 103590:6f2644738876 user: Christian Heimes date: Sat Sep 10 23:44:53 2016 +0200 summary: Issue #19500: Add client-side SSL session resumption to the ssl module. files: Doc/library/ssl.rst | 51 +++- Lib/ssl.py | 65 ++++- Lib/test/test_ssl.py | 112 +++++++++- Misc/NEWS | 2 + Modules/_ssl.c | 372 ++++++++++++++++++++++++++++++- 5 files changed, 582 insertions(+), 20 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -776,6 +776,10 @@ :class:`enum.IntFlag` collection of OP_* constants. +.. data:: OP_NO_TICKET + + Prevent client side from requesting a session ticket. + .. versionadded:: 3.6 .. data:: HAS_ALPN @@ -1176,6 +1180,19 @@ .. versionadded:: 3.2 +.. attribute:: SSLSocket.session + + The :class:`SSLSession` for this SSL connection. The session is available + for client and server side sockets after the TLS handshake has been + performed. For client sockets the session can be set before + :meth:`~SSLSocket.do_handshake` has been called to reuse a session. + + .. versionadded:: 3.6 + +.. attribute:: SSLSocket.session_reused + + .. versionadded:: 3.6 + SSL Contexts ------------ @@ -1509,7 +1526,7 @@ .. method:: SSLContext.wrap_socket(sock, server_side=False, \ do_handshake_on_connect=True, suppress_ragged_eofs=True, \ - server_hostname=None) + server_hostname=None, session=None) Wrap an existing Python socket *sock* and return an :class:`SSLSocket` object. *sock* must be a :data:`~socket.SOCK_STREAM` socket; other socket @@ -1526,19 +1543,27 @@ quite similarly to HTTP virtual hosts. Specifying *server_hostname* will raise a :exc:`ValueError` if *server_side* is true. + *session*, see :attr:`~SSLSocket.session`. + .. versionchanged:: 3.5 Always allow a server_hostname to be passed, even if OpenSSL does not have SNI. + .. versionchanged:: 3.6 + *session* argument was added. + .. method:: SSLContext.wrap_bio(incoming, outgoing, server_side=False, \ - server_hostname=None) + server_hostname=None, session=None) Create a new :class:`SSLObject` instance by wrapping the BIO objects *incoming* and *outgoing*. The SSL routines will read input data from the incoming BIO and write data to the outgoing BIO. - The *server_side* and *server_hostname* parameters have the same meaning as - in :meth:`SSLContext.wrap_socket`. + The *server_side*, *server_hostname* and *session* parameters have the + same meaning as in :meth:`SSLContext.wrap_socket`. + + .. versionchanged:: 3.6 + *session* argument was added. .. method:: SSLContext.session_stats() @@ -2045,6 +2070,8 @@ - :attr:`~SSLSocket.context` - :attr:`~SSLSocket.server_side` - :attr:`~SSLSocket.server_hostname` + - :attr:`~SSLSocket.session` + - :attr:`~SSLSocket.session_reused` - :meth:`~SSLSocket.read` - :meth:`~SSLSocket.write` - :meth:`~SSLSocket.getpeercert` @@ -2126,6 +2153,22 @@ become true after all data currently in the buffer has been read. +SSL session +----------- + +.. versionadded:: 3.6 + +.. class:: SSLSession + + Session object used by :attr:`~SSLSocket.session`. + + .. attribute:: id + .. attribute:: time + .. attribute:: timeout + .. attribute:: ticket_lifetime_hint + .. attribute:: has_ticket + + .. _ssl-security: Security considerations diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -99,7 +99,7 @@ import _ssl # if we can't import it, let the error propagate from _ssl import OPENSSL_VERSION_NUMBER, OPENSSL_VERSION_INFO, OPENSSL_VERSION -from _ssl import _SSLContext, MemoryBIO +from _ssl import _SSLContext, MemoryBIO, SSLSession from _ssl import ( SSLError, SSLZeroReturnError, SSLWantReadError, SSLWantWriteError, SSLSyscallError, SSLEOFError, @@ -391,18 +391,18 @@ def wrap_socket(self, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, - server_hostname=None): + server_hostname=None, session=None): return SSLSocket(sock=sock, server_side=server_side, do_handshake_on_connect=do_handshake_on_connect, suppress_ragged_eofs=suppress_ragged_eofs, server_hostname=server_hostname, - _context=self) + _context=self, _session=session) def wrap_bio(self, incoming, outgoing, server_side=False, - server_hostname=None): + server_hostname=None, session=None): sslobj = self._wrap_bio(incoming, outgoing, server_side=server_side, server_hostname=server_hostname) - return SSLObject(sslobj) + return SSLObject(sslobj, session=session) def set_npn_protocols(self, npn_protocols): protos = bytearray() @@ -572,10 +572,12 @@ * The ``do_handshake_on_connect`` and ``suppress_ragged_eofs`` machinery. """ - def __init__(self, sslobj, owner=None): + def __init__(self, sslobj, owner=None, session=None): self._sslobj = sslobj # Note: _sslobj takes a weak reference to owner self._sslobj.owner = owner or self + if session is not None: + self._sslobj.session = session @property def context(self): @@ -587,6 +589,20 @@ self._sslobj.context = ctx @property + def session(self): + """The SSLSession for client socket.""" + return self._sslobj.session + + @session.setter + def session(self, session): + self._sslobj.session = session + + @property + def session_reused(self): + """Was the client session reused during handshake""" + return self._sslobj.session_reused + + @property def server_side(self): """Whether this is a server-side socket.""" return self._sslobj.server_side @@ -703,7 +719,7 @@ family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None, suppress_ragged_eofs=True, npn_protocols=None, ciphers=None, server_hostname=None, - _context=None): + _context=None, _session=None): if _context: self._context = _context @@ -735,11 +751,16 @@ # mixed in. if sock.getsockopt(SOL_SOCKET, SO_TYPE) != SOCK_STREAM: raise NotImplementedError("only stream sockets are supported") - if server_side and server_hostname: - raise ValueError("server_hostname can only be specified " - "in client mode") + if server_side: + if server_hostname: + raise ValueError("server_hostname can only be specified " + "in client mode") + if _session is not None: + raise ValueError("session can only be specified in " + "client mode") if self._context.check_hostname and not server_hostname: raise ValueError("check_hostname requires server_hostname") + self._session = _session self.server_side = server_side self.server_hostname = server_hostname self.do_handshake_on_connect = do_handshake_on_connect @@ -775,7 +796,8 @@ try: sslobj = self._context._wrap_socket(self, server_side, server_hostname) - self._sslobj = SSLObject(sslobj, owner=self) + self._sslobj = SSLObject(sslobj, owner=self, + session=self._session) if do_handshake_on_connect: timeout = self.gettimeout() if timeout == 0.0: @@ -796,6 +818,24 @@ self._context = ctx self._sslobj.context = ctx + @property + def session(self): + """The SSLSession for client socket.""" + if self._sslobj is not None: + return self._sslobj.session + + @session.setter + def session(self, session): + self._session = session + if self._sslobj is not None: + self._sslobj.session = session + + @property + def session_reused(self): + """Was the client session reused during handshake""" + if self._sslobj is not None: + return self._sslobj.session_reused + def dup(self): raise NotImplemented("Can't dup() %s instances" % self.__class__.__name__) @@ -1028,7 +1068,8 @@ if self._connected: raise ValueError("attempt to connect already-connected SSLSocket!") sslobj = self.context._wrap_socket(self, False, self.server_hostname) - self._sslobj = SSLObject(sslobj, owner=self) + self._sslobj = SSLObject(sslobj, owner=self, + session=self._session) try: if connect_ex: rc = socket.connect_ex(self, addr) diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -2163,7 +2163,8 @@ self.server.close() def server_params_test(client_context, server_context, indata=b"FOO\n", - chatty=True, connectionchatty=False, sni_name=None): + chatty=True, connectionchatty=False, sni_name=None, + session=None): """ Launch a server, connect a client to it and try various reads and writes. @@ -2174,7 +2175,7 @@ connectionchatty=False) with server: with client_context.wrap_socket(socket.socket(), - server_hostname=sni_name) as s: + server_hostname=sni_name, session=session) as s: s.connect((HOST, server.port)) for arg in [indata, bytearray(indata), memoryview(indata)]: if connectionchatty: @@ -2202,6 +2203,8 @@ 'client_alpn_protocol': s.selected_alpn_protocol(), 'client_npn_protocol': s.selected_npn_protocol(), 'version': s.version(), + 'session_reused': s.session_reused, + 'session': s.session, }) s.close() stats['server_alpn_protocols'] = server.selected_alpn_protocols @@ -3412,6 +3415,111 @@ s.sendfile(file) self.assertEqual(s.recv(1024), TEST_DATA) + def test_session(self): + server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + server_context.load_cert_chain(SIGNED_CERTFILE) + client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1) + client_context.verify_mode = ssl.CERT_REQUIRED + client_context.load_verify_locations(SIGNING_CA) + + # first conncetion without session + stats = server_params_test(client_context, server_context) + session = stats['session'] + self.assertTrue(session.id) + self.assertGreater(session.time, 0) + self.assertGreater(session.timeout, 0) + self.assertTrue(session.has_ticket) + if ssl.OPENSSL_VERSION_INFO > (1, 0, 1): + self.assertGreater(session.ticket_lifetime_hint, 0) + self.assertFalse(stats['session_reused']) + sess_stat = server_context.session_stats() + self.assertEqual(sess_stat['accept'], 1) + self.assertEqual(sess_stat['hits'], 0) + + # reuse session + stats = server_params_test(client_context, server_context, session=session) + sess_stat = server_context.session_stats() + self.assertEqual(sess_stat['accept'], 2) + self.assertEqual(sess_stat['hits'], 1) + self.assertTrue(stats['session_reused']) + session2 = stats['session'] + self.assertEqual(session2.id, session.id) + self.assertEqual(session2, session) + self.assertIsNot(session2, session) + self.assertGreaterEqual(session2.time, session.time) + self.assertGreaterEqual(session2.timeout, session.timeout) + + # another one without session + stats = server_params_test(client_context, server_context) + self.assertFalse(stats['session_reused']) + session3 = stats['session'] + self.assertNotEqual(session3.id, session.id) + self.assertNotEqual(session3, session) + sess_stat = server_context.session_stats() + self.assertEqual(sess_stat['accept'], 3) + self.assertEqual(sess_stat['hits'], 1) + + # reuse session again + stats = server_params_test(client_context, server_context, session=session) + self.assertTrue(stats['session_reused']) + session4 = stats['session'] + self.assertEqual(session4.id, session.id) + self.assertEqual(session4, session) + self.assertGreaterEqual(session4.time, session.time) + self.assertGreaterEqual(session4.timeout, session.timeout) + sess_stat = server_context.session_stats() + self.assertEqual(sess_stat['accept'], 4) + self.assertEqual(sess_stat['hits'], 2) + + def test_session_handling(self): + context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) + context.verify_mode = ssl.CERT_REQUIRED + context.load_verify_locations(CERTFILE) + context.load_cert_chain(CERTFILE) + + context2 = ssl.SSLContext(ssl.PROTOCOL_SSLv23) + context2.verify_mode = ssl.CERT_REQUIRED + context2.load_verify_locations(CERTFILE) + context2.load_cert_chain(CERTFILE) + + server = ThreadedEchoServer(context=context, chatty=False) + with server: + with context.wrap_socket(socket.socket()) as s: + # session is None before handshake + self.assertEqual(s.session, None) + self.assertEqual(s.session_reused, None) + s.connect((HOST, server.port)) + session = s.session + self.assertTrue(session) + with self.assertRaises(TypeError) as e: + s.session = object + self.assertEqual(str(e.exception), 'Value is not a SSLSession.') + + with context.wrap_socket(socket.socket()) as s: + s.connect((HOST, server.port)) + # cannot set session after handshake + with self.assertRaises(ValueError) as e: + s.session = session + self.assertEqual(str(e.exception), + 'Cannot set session after handshake.') + + with context.wrap_socket(socket.socket()) as s: + # can set session before handshake and before the + # connection was established + s.session = session + s.connect((HOST, server.port)) + self.assertEqual(s.session.id, session.id) + self.assertEqual(s.session, session) + self.assertEqual(s.session_reused, True) + + with context2.wrap_socket(socket.socket()) as s: + # cannot re-use session with a different SSLContext + with self.assertRaises(ValueError) as e: + s.session = session + s.connect((HOST, server.port)) + self.assertEqual(str(e.exception), + 'Session refers to a different SSLContext.') + def test_main(verbose=False): if support.verbose: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -138,6 +138,8 @@ Library ------- +- Issue #19500: Add client-side SSL session resumption to the ssl module. + - Issue #28022: Deprecate ssl-related arguments in favor of SSLContext. The deprecation include manual creation of SSLSocket and certfile/keyfile (or similar) in ftplib, httplib, imaplib, smtplib, poplib and urllib. diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -187,6 +187,19 @@ { return store->param; } + +static int +SSL_SESSION_has_ticket(const SSL_SESSION *s) +{ + return (s->tlsext_ticklen > 0) ? 1 : 0; +} + +static unsigned long +SSL_SESSION_get_ticket_lifetime_hint(const SSL_SESSION *s) +{ + return s->tlsext_tick_lifetime_hint; +} + #endif /* OpenSSL < 1.1.0 or LibreSSL */ @@ -293,25 +306,35 @@ int eof_written; } PySSLMemoryBIO; +typedef struct { + PyObject_HEAD + SSL_SESSION *session; + PySSLContext *ctx; +} PySSLSession; + static PyTypeObject PySSLContext_Type; static PyTypeObject PySSLSocket_Type; static PyTypeObject PySSLMemoryBIO_Type; +static PyTypeObject PySSLSession_Type; /*[clinic input] module _ssl class _ssl._SSLContext "PySSLContext *" "&PySSLContext_Type" class _ssl._SSLSocket "PySSLSocket *" "&PySSLSocket_Type" class _ssl.MemoryBIO "PySSLMemoryBIO *" "&PySSLMemoryBIO_Type" +class _ssl.SSLSession "PySSLSession *" "&PySSLSession_Type" [clinic start generated code]*/ -/*[clinic end generated code: output=da39a3ee5e6b4b0d input=7bf7cb832638e2e1]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=bdc67fafeeaa8109]*/ #include "clinic/_ssl.c.h" static int PySSL_select(PySocketSockObject *s, int writing, _PyTime_t timeout); + #define PySSLContext_Check(v) (Py_TYPE(v) == &PySSLContext_Type) #define PySSLSocket_Check(v) (Py_TYPE(v) == &PySSLSocket_Type) #define PySSLMemoryBIO_Check(v) (Py_TYPE(v) == &PySSLMemoryBIO_Type) +#define PySSLSession_Check(v) (Py_TYPE(v) == &PySSLSession_Type) typedef enum { SOCKET_IS_NONBLOCKING, @@ -2325,6 +2348,152 @@ return retval; } +#ifdef OPENSSL_VERSION_1_1 + +static SSL_SESSION* +_ssl_session_dup(SSL_SESSION *session) { + SSL_SESSION *newsession = NULL; + int slen; + unsigned char *senc = NULL, *p; + const unsigned char *const_p; + + if (session == NULL) { + PyErr_SetString(PyExc_ValueError, "Invalid session"); + goto error; + } + + /* get length */ + slen = i2d_SSL_SESSION(session, NULL); + if (slen == 0 || slen > 0xFF00) { + PyErr_SetString(PyExc_ValueError, "i2d() failed."); + goto error; + } + if ((senc = PyMem_Malloc(slen)) == NULL) { + PyErr_NoMemory(); + goto error; + } + p = senc; + if (!i2d_SSL_SESSION(session, &p)) { + PyErr_SetString(PyExc_ValueError, "i2d() failed."); + goto error; + } + const_p = senc; + newsession = d2i_SSL_SESSION(NULL, &const_p, slen); + if (session == NULL) { + goto error; + } + PyMem_Free(senc); + return newsession; + error: + if (senc != NULL) { + PyMem_Free(senc); + } + return NULL; +} +#endif + +static PyObject * +PySSL_get_session(PySSLSocket *self, void *closure) { + /* get_session can return sessions from a server-side connection, + * it does not check for handshake done or client socket. */ + PySSLSession *pysess; + SSL_SESSION *session; + +#ifdef OPENSSL_VERSION_1_1 + /* duplicate session as workaround for session bug in OpenSSL 1.1.0, + * https://github.com/openssl/openssl/issues/1550 */ + session = SSL_get0_session(self->ssl); /* borrowed reference */ + if (session == NULL) { + Py_RETURN_NONE; + } + if ((session = _ssl_session_dup(session)) == NULL) { + return NULL; + } +#else + session = SSL_get1_session(self->ssl); + if (session == NULL) { + Py_RETURN_NONE; + } +#endif + + pysess = PyObject_New(PySSLSession, &PySSLSession_Type); + if (pysess == NULL) { + SSL_SESSION_free(session); + return NULL; + } + + assert(self->ctx); + pysess->ctx = self->ctx; + Py_INCREF(pysess->ctx); + pysess->session = session; + return (PyObject *)pysess; +} + +static int PySSL_set_session(PySSLSocket *self, PyObject *value, + void *closure) + { + PySSLSession *pysess; +#ifdef OPENSSL_VERSION_1_1 + SSL_SESSION *session; +#endif + int result; + + if (!PySSLSession_Check(value)) { + PyErr_SetString(PyExc_TypeError, "Value is not a SSLSession."); + return -1; + } + pysess = (PySSLSession *)value; + + if (self->ctx->ctx != pysess->ctx->ctx) { + PyErr_SetString(PyExc_ValueError, + "Session refers to a different SSLContext."); + return -1; + } + if (self->socket_type != PY_SSL_CLIENT) { + PyErr_SetString(PyExc_ValueError, + "Cannot set session for server-side SSLSocket."); + return -1; + } + if (self->handshake_done) { + PyErr_SetString(PyExc_ValueError, + "Cannot set session after handshake."); + return -1; + } +#ifdef OPENSSL_VERSION_1_1 + /* duplicate session */ + if ((session = _ssl_session_dup(pysess->session)) == NULL) { + return -1; + } + result = SSL_set_session(self->ssl, session); + /* free duplicate, SSL_set_session() bumps ref count */ + SSL_SESSION_free(session); +#else + result = SSL_set_session(self->ssl, pysess->session); +#endif + if (result == 0) { + _setSSLError(NULL, 0, __FILE__, __LINE__); + return -1; + } + return 0; +} + +PyDoc_STRVAR(PySSL_set_session_doc, +"_setter_session(session)\n\ +\ +Get / set SSLSession."); + +static PyObject * +PySSL_get_session_reused(PySSLSocket *self, void *closure) { + if (SSL_session_reused(self->ssl)) { + Py_RETURN_TRUE; + } else { + Py_RETURN_FALSE; + } +} + +PyDoc_STRVAR(PySSL_get_session_reused_doc, +"Was the client session reused during handshake?"); + static PyGetSetDef ssl_getsetlist[] = { {"context", (getter) PySSL_get_context, (setter) PySSL_set_context, PySSL_set_context_doc}, @@ -2334,6 +2503,10 @@ PySSL_get_server_hostname_doc}, {"owner", (getter) PySSL_get_owner, (setter) PySSL_set_owner, PySSL_get_owner_doc}, + {"session", (getter) PySSL_get_session, + (setter) PySSL_set_session, PySSL_set_session_doc}, + {"session_reused", (getter) PySSL_get_session_reused, NULL, + PySSL_get_session_reused_doc}, {NULL}, /* sentinel */ }; @@ -2618,7 +2791,7 @@ { SSL *ssl = NULL; STACK_OF(SSL_CIPHER) *sk = NULL; - SSL_CIPHER *cipher; + const SSL_CIPHER *cipher; int i=0; PyObject *result = NULL, *dct; @@ -4086,6 +4259,193 @@ }; +/* + * SSL Session object + */ + +static void +PySSLSession_dealloc(PySSLSession *self) +{ + Py_XDECREF(self->ctx); + if (self->session != NULL) { + SSL_SESSION_free(self->session); + } + PyObject_Del(self); +} + +static PyObject * +PySSLSession_richcompare(PyObject *left, PyObject *right, int op) +{ + int result; + + if (left == NULL || right == NULL) { + PyErr_BadInternalCall(); + return NULL; + } + + if (!PySSLSession_Check(left) || !PySSLSession_Check(right)) { + Py_RETURN_NOTIMPLEMENTED; + } + + if (left == right) { + result = 0; + } else { + const unsigned char *left_id, *right_id; + unsigned int left_len, right_len; + left_id = SSL_SESSION_get_id(((PySSLSession *)left)->session, + &left_len); + right_id = SSL_SESSION_get_id(((PySSLSession *)right)->session, + &right_len); + if (left_len == right_len) { + result = memcmp(left_id, right_id, left_len); + } else { + result = 1; + } + } + + switch (op) { + case Py_EQ: + if (result == 0) { + Py_RETURN_TRUE; + } else { + Py_RETURN_FALSE; + } + break; + case Py_NE: + if (result != 0) { + Py_RETURN_TRUE; + } else { + Py_RETURN_FALSE; + } + break; + case Py_LT: + case Py_LE: + case Py_GT: + case Py_GE: + Py_RETURN_NOTIMPLEMENTED; + break; + default: + PyErr_BadArgument(); + return NULL; + } +} + +static int +PySSLSession_traverse(PySSLSession *self, visitproc visit, void *arg) +{ + Py_VISIT(self->ctx); + return 0; +} + +static int +PySSLSession_clear(PySSLSession *self) +{ + Py_CLEAR(self->ctx); + return 0; +} + + +static PyObject * +PySSLSession_get_time(PySSLSession *self, void *closure) { + return PyLong_FromLong(SSL_SESSION_get_time(self->session)); +} + +PyDoc_STRVAR(PySSLSession_get_time_doc, +"Session creation time (seconds since epoch)."); + + +static PyObject * +PySSLSession_get_timeout(PySSLSession *self, void *closure) { + return PyLong_FromLong(SSL_SESSION_get_timeout(self->session)); +} + +PyDoc_STRVAR(PySSLSession_get_timeout_doc, +"Session timeout (delta in seconds)."); + + +static PyObject * +PySSLSession_get_ticket_lifetime_hint(PySSLSession *self, void *closure) { + unsigned long hint = SSL_SESSION_get_ticket_lifetime_hint(self->session); + return PyLong_FromUnsignedLong(hint); +} + +PyDoc_STRVAR(PySSLSession_get_ticket_lifetime_hint_doc, +"Ticket life time hint."); + + +static PyObject * +PySSLSession_get_session_id(PySSLSession *self, void *closure) { + const unsigned char *id; + unsigned int len; + id = SSL_SESSION_get_id(self->session, &len); + return PyBytes_FromStringAndSize((const char *)id, len); +} + +PyDoc_STRVAR(PySSLSession_get_session_id_doc, +"Session id"); + + +static PyObject * +PySSLSession_get_has_ticket(PySSLSession *self, void *closure) { + if (SSL_SESSION_has_ticket(self->session)) { + Py_RETURN_TRUE; + } else { + Py_RETURN_FALSE; + } +} + +PyDoc_STRVAR(PySSLSession_get_has_ticket_doc, +"Does the session contain a ticket?"); + + +static PyGetSetDef PySSLSession_getsetlist[] = { + {"has_ticket", (getter) PySSLSession_get_has_ticket, NULL, + PySSLSession_get_has_ticket_doc}, + {"id", (getter) PySSLSession_get_session_id, NULL, + PySSLSession_get_session_id_doc}, + {"ticket_lifetime_hint", (getter) PySSLSession_get_ticket_lifetime_hint, + NULL, PySSLSession_get_ticket_lifetime_hint_doc}, + {"time", (getter) PySSLSession_get_time, NULL, + PySSLSession_get_time_doc}, + {"timeout", (getter) PySSLSession_get_timeout, NULL, + PySSLSession_get_timeout_doc}, + {NULL}, /* sentinel */ +}; + +static PyTypeObject PySSLSession_Type = { + PyVarObject_HEAD_INIT(NULL, 0) + "_ssl.Session", /*tp_name*/ + sizeof(PySSLSession), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + (destructor)PySSLSession_dealloc, /*tp_dealloc*/ + 0, /*tp_print*/ + 0, /*tp_getattr*/ + 0, /*tp_setattr*/ + 0, /*tp_reserved*/ + 0, /*tp_repr*/ + 0, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + 0, /*tp_hash*/ + 0, /*tp_call*/ + 0, /*tp_str*/ + 0, /*tp_getattro*/ + 0, /*tp_setattro*/ + 0, /*tp_as_buffer*/ + Py_TPFLAGS_DEFAULT, /*tp_flags*/ + 0, /*tp_doc*/ + (traverseproc)PySSLSession_traverse, /*tp_traverse*/ + (inquiry)PySSLSession_clear, /*tp_clear*/ + PySSLSession_richcompare, /*tp_richcompare*/ + 0, /*tp_weaklistoffset*/ + 0, /*tp_iter*/ + 0, /*tp_iternext*/ + 0, /*tp_methods*/ + 0, /*tp_members*/ + PySSLSession_getsetlist, /*tp_getset*/ +}; + + /* helper routines for seeding the SSL PRNG */ /*[clinic input] _ssl.RAND_add @@ -4771,6 +5131,9 @@ return NULL; if (PyType_Ready(&PySSLMemoryBIO_Type) < 0) return NULL; + if (PyType_Ready(&PySSLSession_Type) < 0) + return NULL; + m = PyModule_Create(&_sslmodule); if (m == NULL) @@ -4842,6 +5205,10 @@ if (PyDict_SetItemString(d, "MemoryBIO", (PyObject *)&PySSLMemoryBIO_Type) != 0) return NULL; + if (PyDict_SetItemString(d, "SSLSession", + (PyObject *)&PySSLSession_Type) != 0) + return NULL; + PyModule_AddIntConstant(m, "SSL_ERROR_ZERO_RETURN", PY_SSL_ERROR_ZERO_RETURN); PyModule_AddIntConstant(m, "SSL_ERROR_WANT_READ", @@ -4968,6 +5335,7 @@ PyModule_AddIntConstant(m, "OP_CIPHER_SERVER_PREFERENCE", SSL_OP_CIPHER_SERVER_PREFERENCE); PyModule_AddIntConstant(m, "OP_SINGLE_DH_USE", SSL_OP_SINGLE_DH_USE); + PyModule_AddIntConstant(m, "OP_NO_TICKET", SSL_OP_NO_TICKET); #ifdef SSL_OP_SINGLE_ECDH_USE PyModule_AddIntConstant(m, "OP_SINGLE_ECDH_USE", SSL_OP_SINGLE_ECDH_USE); #endif -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 18:43:34 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 10 Sep 2016 22:43:34 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=282=2E7=29=3A_Backported_tes?= =?utf-8?q?ts_for_issue_=2328070=2E?= Message-ID: <20160910224333.8471.41558.AEB55207@psf.io> https://hg.python.org/cpython/rev/afc0d4478083 changeset: 103593:afc0d4478083 branch: 2.7 parent: 103584:168fe15798c6 user: Serhiy Storchaka date: Sun Sep 11 01:39:51 2016 +0300 summary: Backported tests for issue #28070. files: Lib/test/test_re.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -928,6 +928,9 @@ q = p.match(upper_char) self.assertTrue(q) + self.assertTrue(re.match('(?ixu) ' + upper_char, lower_char)) + self.assertTrue(re.match('(?ixu) ' + lower_char, upper_char)) + def test_dollar_matches_twice(self): "$ matches the end of string, and just before the terminating \n" pattern = re.compile('$') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 18:43:33 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 10 Sep 2016 22:43:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Backported_tes?= =?utf-8?q?ts_for_issue_=2328070=2E?= Message-ID: <20160910224333.12395.66351.F50119A5@psf.io> https://hg.python.org/cpython/rev/842e75f0e592 changeset: 103592:842e75f0e592 branch: 3.5 parent: 103585:3ed2c0d55526 user: Serhiy Storchaka date: Sun Sep 11 01:39:51 2016 +0300 summary: Backported tests for issue #28070. files: Lib/test/test_re.py | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -1270,6 +1270,9 @@ q = p.match(upper_char) self.assertTrue(q) + self.assertTrue(re.match('(?ixu) ' + upper_char, lower_char)) + self.assertTrue(re.match('(?ixu) ' + lower_char, upper_char)) + def test_dollar_matches_twice(self): "$ matches the end of string, and just before the terminating \n" pattern = re.compile('$') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 18:43:33 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 10 Sep 2016 22:43:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328070=3A_Fixed_pa?= =?utf-8?q?rsing_inline_verbose_flag_in_regular_expressions=2E?= Message-ID: <20160910224333.87525.70248.B7E208EA@psf.io> https://hg.python.org/cpython/rev/bee52e5f8fb1 changeset: 103591:bee52e5f8fb1 user: Serhiy Storchaka date: Sun Sep 11 01:39:01 2016 +0300 summary: Issue #28070: Fixed parsing inline verbose flag in regular expressions. files: Lib/sre_parse.py | 1 + Lib/test/test_re.py | 3 +++ Misc/NEWS | 2 ++ 3 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Lib/sre_parse.py b/Lib/sre_parse.py --- a/Lib/sre_parse.py +++ b/Lib/sre_parse.py @@ -847,6 +847,7 @@ pattern = Pattern() pattern.flags = flags | SRE_FLAG_VERBOSE pattern.str = str + source.seek(0) p = _parse_sub(source, pattern, True, False) p.pattern.flags = fix_flags(str, p.pattern.flags) diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -1276,6 +1276,9 @@ q = p.match(upper_char) self.assertTrue(q) + self.assertTrue(re.match('(?ixu) ' + upper_char, lower_char)) + self.assertTrue(re.match('(?ixu) ' + lower_char, upper_char)) + def test_dollar_matches_twice(self): "$ matches the end of string, and just before the terminating \n" pattern = re.compile('$') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -138,6 +138,8 @@ Library ------- +- Issue #28070: Fixed parsing inline verbose flag in regular expressions. + - Issue #19500: Add client-side SSL session resumption to the ssl module. - Issue #28022: Deprecate ssl-related arguments in favor of SSLContext. The -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 18:43:39 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sat, 10 Sep 2016 22:43:39 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Null_merge?= Message-ID: <20160910224338.13248.8715.8C218319@psf.io> https://hg.python.org/cpython/rev/208e1ed48cd3 changeset: 103594:208e1ed48cd3 parent: 103591:bee52e5f8fb1 parent: 103592:842e75f0e592 user: Serhiy Storchaka date: Sun Sep 11 01:41:01 2016 +0300 summary: Null merge files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 20:04:46 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 11 Sep 2016 00:04:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset_3934e?= =?utf-8?q?070c9db?= Message-ID: <20160911000446.2591.72832.EF1491DD@psf.io> https://hg.python.org/cpython/rev/2e7795b765cf changeset: 103595:2e7795b765cf user: Benjamin Peterson date: Sat Sep 10 17:04:36 2016 -0700 summary: Backed out changeset 3934e070c9db files: Python/getargs.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Python/getargs.c b/Python/getargs.c --- a/Python/getargs.c +++ b/Python/getargs.c @@ -35,12 +35,11 @@ PyAPI_FUNC(int) _PyArg_VaParse_SizeT(PyObject *, const char *, va_list); PyAPI_FUNC(int) _PyArg_VaParseTupleAndKeywords_SizeT(PyObject *, PyObject *, const char *, char **, va_list); + PyAPI_FUNC(int) _PyArg_ParseTupleAndKeywordsFast_SizeT(PyObject *, PyObject *, struct _PyArg_Parser *, ...); PyAPI_FUNC(int) _PyArg_VaParseTupleAndKeywordsFast_SizeT(PyObject *, PyObject *, struct _PyArg_Parser *, va_list); -PyAPI_FUNC(int) _PyArg_ParseStack_SizeT(PyObject **args, Py_ssize_t nargs, - PyObject *kwnames, struct _PyArg_Parser *parser, ...); #endif #define FLAG_COMPAT 1 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 20:24:33 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 11 Sep 2016 00:24:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_reST_is_not_markdown?= Message-ID: <20160911002432.2591.38079.DAD120FA@psf.io> https://hg.python.org/cpython/rev/590cff9c92f1 changeset: 103596:590cff9c92f1 user: Benjamin Peterson date: Sat Sep 10 17:24:25 2016 -0700 summary: reST is not markdown files: Doc/howto/instrumentation.rst | 34 +++++++++++----------- 1 files changed, 17 insertions(+), 17 deletions(-) diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst --- a/Doc/howto/instrumentation.rst +++ b/Doc/howto/instrumentation.rst @@ -280,23 +280,22 @@ The filename, function name, and line number are provided back to the tracing script as positional arguments, which must be accessed using - `$arg1`, `$arg2`, `$arg3`: + ``$arg1``, ``$arg2``, ``$arg3``: - * `$arg1` : `(const char *)` filename, accessible using `user_string($arg1)` + * ``$arg1`` : ``(const char *)`` filename, accessible using ``user_string($arg1)`` - * `$arg2` : `(const char *)` function name, accessible using - `user_string($arg2)` + * ``$arg2`` : ``(const char *)`` function name, accessible using + ``user_string($arg2)`` - * `$arg3` : `int` line number + * ``$arg3`` : ``int`` line number .. c:function:: function__return(str filename, str funcname, int lineno) - This marker is the converse of `function__entry`, and indicates that - execution of a Python function has ended (either via ``return``, or - via an exception). It is only triggered for pure-Python (bytecode) - functions. + This marker is the converse of :c:func:`function__entry`, and indicates that + execution of a Python function has ended (either via ``return``, or via an + exception). It is only triggered for pure-Python (bytecode) functions. - The arguments are the same as for `function__entry` + The arguments are the same as for :c:func:`function__entry` .. c:function:: line(str filename, str funcname, int lineno) @@ -304,17 +303,17 @@ the equivalent of line-by-line tracing with a Python profiler. It is not triggered within C functions. - The arguments are the same as for `function__entry`. + The arguments are the same as for :c:func:`function__entry`. .. c:function:: gc__start(int generation) Fires when the Python interpreter starts a garbage collection cycle. - `arg0` is the generation to scan, like :func:`gc.collect()`. + ``arg0`` is the generation to scan, like :func:`gc.collect()`. .. c:function:: gc__done(long collected) Fires when the Python interpreter finishes a garbage collection - cycle. `arg0` is the number of collected objects. + cycle. ``arg0`` is the number of collected objects. SystemTap Tapsets @@ -348,7 +347,7 @@ } If this file is installed in SystemTap's tapset directory (e.g. -`/usr/share/systemtap/tapset`), then these additional probepoints become +``/usr/share/systemtap/tapset``), then these additional probepoints become available: .. c:function:: python.function.entry(str filename, str funcname, int lineno, frameptr) @@ -358,9 +357,10 @@ .. c:function:: python.function.return(str filename, str funcname, int lineno, frameptr) - This probe point is the converse of `python.function.return`, and indicates - that execution of a Python function has ended (either via ``return``, or - via an exception). It is only triggered for pure-python (bytecode) functions. + This probe point is the converse of :c:func:`python.function.return`, and + indicates that execution of a Python function has ended (either via + ``return``, or via an exception). It is only triggered for pure-python + (bytecode) functions. Examples -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 20:34:21 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 11 Sep 2016 00:34:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_add_the_usual_extern_C_sil?= =?utf-8?q?liness_to_pydtrace=2Eh?= Message-ID: <20160911003421.2686.58562.9D7F5ADA@psf.io> https://hg.python.org/cpython/rev/52b2bbbe23ef changeset: 103597:52b2bbbe23ef user: Benjamin Peterson date: Sat Sep 10 17:34:15 2016 -0700 summary: add the usual extern C silliness to pydtrace.h files: Include/pydtrace.h | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Include/pydtrace.h b/Include/pydtrace.h --- a/Include/pydtrace.h +++ b/Include/pydtrace.h @@ -2,6 +2,9 @@ #ifndef Py_DTRACE_H #define Py_DTRACE_H +#ifdef __cplusplus +extern "C" { +#endif #ifdef WITH_DTRACE @@ -44,4 +47,7 @@ #endif /* !WITH_DTRACE */ +#ifdef __cplusplus +} +#endif #endif /* !Py_DTRACE_H */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 20:38:56 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 11 Sep 2016 00:38:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_fix_link_to_instrumentatio?= =?utf-8?q?n?= Message-ID: <20160911003856.19052.33861.5CE9C97E@psf.io> https://hg.python.org/cpython/rev/46b8d7aca720 changeset: 103598:46b8d7aca720 user: Benjamin Peterson date: Sat Sep 10 17:38:51 2016 -0700 summary: fix link to instrumentation files: Doc/whatsnew/3.6.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -423,7 +423,7 @@ without the need to recompile specific debug builds or providing application-specific profiling/debugging code. -More details in:ref:`instrumentation`. +More details in :ref:`instrumentation`. The current implementation is tested on Linux and macOS. Additional markers may be added in the future. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 20:45:44 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 11 Sep 2016 00:45:44 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_force_gcc_to_use_c99_inlin?= =?utf-8?q?e_semantics?= Message-ID: <20160911004544.59404.46942.FC5EC8E7@psf.io> https://hg.python.org/cpython/rev/8460a729e1de changeset: 103599:8460a729e1de user: Benjamin Peterson date: Sat Sep 10 17:45:33 2016 -0700 summary: force gcc to use c99 inline semantics files: configure | 16 ++++++++++++++-- configure.ac | 2 +- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -784,6 +784,7 @@ docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -894,6 +895,7 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1146,6 +1148,15 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1283,7 +1294,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1436,6 +1447,7 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -6906,7 +6918,7 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) - CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" + CFLAGS_NODIST="$CFLAGS_NODIST -std=c99 -fno-gnu89-inline" # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1515,7 +1515,7 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) - CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" + CFLAGS_NODIST="$CFLAGS_NODIST -std=c99 -fno-gnu89-inline" # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 20:53:19 2016 From: python-checkins at python.org (benjamin.peterson) Date: Sun, 11 Sep 2016 00:53:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Backed_out_changeset_8460a?= =?utf-8?q?729e1de?= Message-ID: <20160911005319.19406.57136.47B597A8@psf.io> https://hg.python.org/cpython/rev/c5f9e8f2d9df changeset: 103600:c5f9e8f2d9df user: Benjamin Peterson date: Sat Sep 10 17:53:13 2016 -0700 summary: Backed out changeset 8460a729e1de files: configure | 16 ++-------------- configure.ac | 2 +- 2 files changed, 3 insertions(+), 15 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -784,7 +784,6 @@ docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir @@ -895,7 +894,6 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1148,15 +1146,6 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1294,7 +1283,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1447,7 +1436,6 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -6918,7 +6906,7 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) - CFLAGS_NODIST="$CFLAGS_NODIST -std=c99 -fno-gnu89-inline" + CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1515,7 +1515,7 @@ # tweak BASECFLAGS based on compiler and platform case $GCC in yes) - CFLAGS_NODIST="$CFLAGS_NODIST -std=c99 -fno-gnu89-inline" + CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 21:49:40 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 01:49:40 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2328073=3A_Improve_wording_around_None=2E_Michael?= =?utf-8?b?IExlZS4gKDMuNS0+My42KQ==?= Message-ID: <20160911014940.2666.14991.887A1FDA@psf.io> https://hg.python.org/cpython/rev/7a4a16a880a4 changeset: 103602:7a4a16a880a4 parent: 103600:c5f9e8f2d9df parent: 103601:75514816741a user: Guido van Rossum date: Sat Sep 10 18:49:34 2016 -0700 summary: Issue #28073: Improve wording around None. Michael Lee. (3.5->3.6) files: Doc/library/typing.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -59,6 +59,9 @@ servers: List[Tuple[Tuple[str, int], Dict[str, str]]]) -> None: ... +Note that ``None`` as a type hint is a special case and is replaced by +``type(None)``. + NewType ------- @@ -148,7 +151,6 @@ It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis for the list of arguments in the type hint: ``Callable[..., ReturnType]``. -``None`` as a type hint is a special case and is replaced by ``type(None)``. Generics -------- @@ -443,7 +445,7 @@ Optional type. - ``Optional[X]`` is equivalent to ``Union[X, type(None)]``. + ``Optional[X]`` is equivalent to ``Union[X, None]``. Note that this is not the same concept as an optional argument, which is one that has a default. An optional argument with a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 21:49:40 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 01:49:40 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4MDcz?= =?utf-8?q?=3A_Improve_wording_around_None=2E_Michael_Lee=2E?= Message-ID: <20160911014940.69722.95096.F6373ABF@psf.io> https://hg.python.org/cpython/rev/75514816741a changeset: 103601:75514816741a branch: 3.5 parent: 103592:842e75f0e592 user: Guido van Rossum date: Sat Sep 10 18:49:14 2016 -0700 summary: Issue #28073: Improve wording around None. Michael Lee. files: Doc/library/typing.rst | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -59,6 +59,9 @@ servers: List[Tuple[Tuple[str, int], Dict[str, str]]]) -> None: ... +Note that ``None`` as a type hint is a special case and is replaced by +``type(None)``. + NewType ------- @@ -148,7 +151,6 @@ It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis for the list of arguments in the type hint: ``Callable[..., ReturnType]``. -``None`` as a type hint is a special case and is replaced by ``type(None)``. Generics -------- @@ -443,7 +445,7 @@ Optional type. - ``Optional[X]`` is equivalent to ``Union[X, type(None)]``. + ``Optional[X]`` is equivalent to ``Union[X, None]``. Note that this is not the same concept as an optional argument, which is one that has a default. An optional argument with a -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 22:05:53 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 02:05:53 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2MTQx?= =?utf-8?q?=3A_Update_docs_for_typing=2Epy=2E_Ivan_Levkivskyi=2E_=28Backpo?= =?utf-8?q?rt_from_the?= Message-ID: <20160911020553.2686.77555.A16B00D7@psf.io> https://hg.python.org/cpython/rev/1dc45beca118 changeset: 103604:1dc45beca118 branch: 3.5 parent: 103601:75514816741a user: Guido van Rossum date: Sat Sep 10 19:03:22 2016 -0700 summary: Issue #26141: Update docs for typing.py. Ivan Levkivskyi. (Backport from the 3.6 version) files: Doc/library/typing.rst | 127 +++++++++++++++++++++++++--- 1 files changed, 113 insertions(+), 14 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -62,10 +62,12 @@ Note that ``None`` as a type hint is a special case and is replaced by ``type(None)``. +.. _distinct: + NewType ------- -Use the ``NewType`` helper function to create distinct types:: +Use the :func:`NewType` helper function to create distinct types:: from typing import NewType @@ -103,7 +105,7 @@ This also means that it is not possible to create a subtype of ``Derived`` since it is an identity function at runtime, not an actual type. Similarly, it -is not possible to create another ``NewType`` based on a ``Derived`` type:: +is not possible to create another :func:`NewType` based on a ``Derived`` type:: from typing import NewType @@ -206,7 +208,7 @@ return self.value def log(self, message: str) -> None: - self.logger.info('{}: {}'.format(self.name, message)) + self.logger.info('%s: %s', self.name, message) ``Generic[T]`` as a base class defines that the class ``LoggedVar`` takes a single type parameter ``T`` . This also makes ``T`` valid as a type within the @@ -533,10 +535,10 @@ but should also allow constructor calls in subclasses that match the constructor calls in the indicated base class. How the type checker is required to handle this particular case may change in future revisions of - PEP 484. + :pep:`484`. - The only legal parameters for ``Type`` are classes, unions of classes, and - ``Any``. For example:: + The only legal parameters for :class:`Type` are classes, unions of classes, and + :class:`Any`. For example:: def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... @@ -545,11 +547,15 @@ .. class:: Iterable(Generic[T_co]) - A generic version of the :class:`collections.abc.Iterable`. + A generic version of :class:`collections.abc.Iterable`. .. class:: Iterator(Iterable[T_co]) - A generic version of the :class:`collections.abc.Iterator`. + A generic version of :class:`collections.abc.Iterator`. + +.. class:: Reversible(Iterable[T_co]) + + A generic version of :class:`collections.abc.Reversible`. .. class:: SupportsInt @@ -569,15 +575,18 @@ An ABC with one abstract method ``__round__`` that is covariant in its return type. -.. class:: Reversible - - An ABC with one abstract method ``__reversed__`` returning - an ``Iterator[T_co]``. - .. class:: Container(Generic[T_co]) A generic version of :class:`collections.abc.Container`. +.. class:: Hashable + + An alias to :class:`collections.abc.Hashable` + +.. class:: Sized + + An alias to :class:`collections.abc.Sized` + .. class:: AbstractSet(Sized, Iterable[T_co], Container[T_co]) A generic version of :class:`collections.abc.Set`. @@ -649,6 +658,18 @@ A generic version of :class:`collections.abc.ValuesView`. +.. class:: Awaitable(Generic[T_co]) + + A generic version of :class:`collections.abc.Awaitable`. + +.. class:: AsyncIterable(Generic[T_co]) + + A generic version of :class:`collections.abc.AsyncIterable`. + +.. class:: AsyncIterator(AsyncIterable[T_co]) + + A generic version of :class:`collections.abc.AsyncIterator`. + .. class:: Dict(dict, MutableMapping[KT, VT]) A generic version of :class:`dict`. @@ -657,6 +678,10 @@ def get_position_in_index(word_list: Dict[str, int], word: str) -> int: return word_list[word] +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) + + A generic version of :class:`collections.defaultdict` + .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) A generator can be annotated by the generic type @@ -681,7 +706,7 @@ start += 1 Alternatively, annotate your generator as having a return type of - ``Iterator[YieldType]``:: + either ``Iterable[YieldType]`` or ``Iterator[YieldType]``:: def infinite_stream(start: int) -> Iterator[int]: while True: @@ -752,6 +777,15 @@ are in the _fields attribute, which is part of the namedtuple API.) +.. function:: NewType(typ) + + A helper function to indicate a distinct types to a typechecker, + see :ref:`distinct`. At runtime it returns a function that returns + its argument. Usage:: + + UserId = NewType('UserId', int) + first_user = UserId(1) + .. function:: cast(typ, val) Cast a value to a type. @@ -769,6 +803,34 @@ forward references encoded as string literals, and if necessary adds ``Optional[t]`` if a default value equal to None is set. +.. decorator:: overload + + The ``@overload`` decorator allows describing functions and methods + that support multiple different combinations of argument types. A series + of ``@overload``-decorated definitions must be followed by exactly one + non-``@overload``-decorated definition (for the same function/method). + The ``@overload``-decorated definitions are for the benefit of the + type checker only, since they will be overwritten by the + non-``@overload``-decorated definition, while the latter is used at + runtime but should be ignored by a type checker. At runtime, calling + a ``@overload``-decorated function directly will raise + ``NotImplementedError``. An example of overload that gives a more + precise type than can be expressed using a union or a type variable:: + + @overload + def process(response: None) -> None: + ... + @overload + def process(response: int) -> Tuple[int, str]: + ... + @overload + def process(response: bytes) -> str: + ... + def process(response): + + + See :pep:`484` for details and comparison with other typing semantics. + .. decorator:: no_type_check(arg) Decorator to indicate that annotations are not type hints. @@ -785,3 +847,40 @@ This wraps the decorator with something that wraps the decorated function in :func:`no_type_check`. + +.. data:: ClassVar + + Special type construct to mark class variables. + + As introduced in :pep:`526`, a variable annotation wrapped in ClassVar + indicates that a given attribute is intended to be used as a class variable + and should not be set on instances of that class. Usage:: + + class Starship: + stats: ClassVar[Dict[str, int]] = {} # class variable + damage: int = 10 # instance variable + + ClassVar accepts only types and cannot be further subscribed. + + ClassVar is not a class itself, and should not + be used with isinstance() or issubclass(). Note that ClassVar + does not change Python runtime behavior, it can be used by + 3rd party type checkers, so that the following code will + flagged as an error by those:: + + enterprise_d = Starship(3000) + enterprise_d.stats = {} # Error, setting class variable on instance + Starship.stats = {} # This is OK + + .. versionadded:: 3.6 + +.. data:: TYPE_CHECKING + + A special constant that is assumed to be ``True`` by 3rd party static + type checkers. It is ``False`` at runtime. Usage:: + + if TYPE_CHECKING: + import expensive_mod + + def fun(): + local_var: expensive_mod.some_type = other_fun() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 22:05:53 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 02:05:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326141=3A_Update_d?= =?utf-8?q?ocs_for_typing=2Epy=2E_Ivan_Levkivskyi=2E?= Message-ID: <20160911020552.2286.5671.92978CA5@psf.io> https://hg.python.org/cpython/rev/7187d167abe8 changeset: 103603:7187d167abe8 user: Guido van Rossum date: Sat Sep 10 18:54:14 2016 -0700 summary: Issue #26141: Update docs for typing.py. Ivan Levkivskyi. files: Doc/library/typing.rst | 114 ++++++++++++++++++++++++++-- 1 files changed, 104 insertions(+), 10 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -10,7 +10,7 @@ -------------- -This module supports type hints as specified by :pep:`484`. The most +This module supports type hints as specified by :pep:`484` and :pep:`526`. The most fundamental support consists of the type :class:`Any`, :class:`Union`, :class:`Tuple`, :class:`Callable`, :class:`TypeVar`, and :class:`Generic`. For full specification please see :pep:`484`. For @@ -62,10 +62,12 @@ Note that ``None`` as a type hint is a special case and is replaced by ``type(None)``. +.. _distinct: + NewType ------- -Use the ``NewType`` helper function to create distinct types:: +Use the :func:`NewType` helper function to create distinct types:: from typing import NewType @@ -103,7 +105,7 @@ This also means that it is not possible to create a subtype of ``Derived`` since it is an identity function at runtime, not an actual type. Similarly, it -is not possible to create another ``NewType`` based on a ``Derived`` type:: +is not possible to create another :func:`NewType` based on a ``Derived`` type:: from typing import NewType @@ -533,10 +535,10 @@ but should also allow constructor calls in subclasses that match the constructor calls in the indicated base class. How the type checker is required to handle this particular case may change in future revisions of - PEP 484. + :pep:`484`. - The only legal parameters for ``Type`` are classes, unions of classes, and - ``Any``. For example:: + The only legal parameters for :class:`Type` are classes, unions of classes, and + :class:`Any`. For example:: def new_non_team_user(user_class: Type[Union[BaseUser, ProUser]]): ... @@ -577,7 +579,21 @@ A generic version of :class:`collections.abc.Container`. -.. class:: AbstractSet(Sized, Iterable[T_co], Container[T_co]) +.. class:: Hashable + + An alias to :class:`collections.abc.Hashable` + +.. class:: Sized + + An alias to :class:`collections.abc.Sized` + +.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) + + A generic version of :class:`collections.abc.Collection` + + .. versionadded:: 3.6 + +.. class:: AbstractSet(Sized, Collection[T_co]) A generic version of :class:`collections.abc.Set`. @@ -585,7 +601,7 @@ A generic version of :class:`collections.abc.MutableSet`. -.. class:: Mapping(Sized, Iterable[KT], Container[KT], Generic[VT_co]) +.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) A generic version of :class:`collections.abc.Mapping`. @@ -593,7 +609,7 @@ A generic version of :class:`collections.abc.MutableMapping`. -.. class:: Sequence(Sized, Reversible[T_co], Container[T_co]) +.. class:: Sequence(Reversible[T_co], Collection[T_co]) A generic version of :class:`collections.abc.Sequence`. @@ -674,6 +690,10 @@ def get_position_in_index(word_list: Dict[str, int], word: str) -> int: return word_list[word] +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) + + A generic version of :class:`collections.defaultdict` + .. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) A generator can be annotated by the generic type @@ -769,6 +789,15 @@ are in the _fields attribute, which is part of the namedtuple API.) +.. function:: NewType(typ) + + A helper function to indicate a distinct types to a typechecker, + see :ref:`distinct`. At runtime it returns a function that returns + its argument. Usage:: + + UserId = NewType('UserId', int) + first_user = UserId(1) + .. function:: cast(typ, val) Cast a value to a type. @@ -780,12 +809,40 @@ .. function:: get_type_hints(obj) - Return type hints for a function or method object. + Return type hints for a class, module, function or method object. This is often the same as ``obj.__annotations__``, but it handles forward references encoded as string literals, and if necessary adds ``Optional[t]`` if a default value equal to None is set. +.. decorator:: overload + + The ``@overload`` decorator allows describing functions and methods + that support multiple different combinations of argument types. A series + of ``@overload``-decorated definitions must be followed by exactly one + non-``@overload``-decorated definition (for the same function/method). + The ``@overload``-decorated definitions are for the benefit of the + type checker only, since they will be overwritten by the + non-``@overload``-decorated definition, while the latter is used at + runtime but should be ignored by a type checker. At runtime, calling + a ``@overload``-decorated function directly will raise + ``NotImplementedError``. An example of overload that gives a more + precise type than can be expressed using a union or a type variable:: + + @overload + def process(response: None) -> None: + ... + @overload + def process(response: int) -> Tuple[int, str]: + ... + @overload + def process(response: bytes) -> str: + ... + def process(response): + + + See :pep:`484` for details and comparison with other typing semantics. + .. decorator:: no_type_check(arg) Decorator to indicate that annotations are not type hints. @@ -802,3 +859,40 @@ This wraps the decorator with something that wraps the decorated function in :func:`no_type_check`. + +.. data:: ClassVar + + Special type construct to mark class variables. + + As introduced in :pep:`526`, a variable annotation wrapped in ClassVar + indicates that a given attribute is intended to be used as a class variable + and should not be set on instances of that class. Usage:: + + class Starship: + stats: ClassVar[Dict[str, int]] = {} # class variable + damage: int = 10 # instance variable + + ClassVar accepts only types and cannot be further subscribed. + + ClassVar is not a class itself, and should not + be used with isinstance() or issubclass(). Note that ClassVar + does not change Python runtime behavior, it can be used by + 3rd party type checkers, so that the following code will + flagged as an error by those:: + + enterprise_d = Starship(3000) + enterprise_d.stats = {} # Error, setting class variable on instance + Starship.stats = {} # This is OK + + .. versionadded:: 3.6 + +.. data:: TYPE_CHECKING + + A special constant that is assumed to be ``True`` by 3rd party static + type checkers. It is ``False`` at runtime. Usage:: + + if TYPE_CHECKING: + import expensive_mod + + def fun(): + local_var: expensive_mod.some_type = other_fun() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 22:05:53 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 02:05:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2326141=3A_Update_docs_for_typing=2Epy=2E_Ivan_Le?= =?utf-8?q?vkivskyi=2E_=28null_merge_3=2E5-=3E3=2E6=29?= Message-ID: <20160911020553.8602.4750.67903BD7@psf.io> https://hg.python.org/cpython/rev/960a73507d4d changeset: 103605:960a73507d4d parent: 103603:7187d167abe8 parent: 103604:1dc45beca118 user: Guido van Rossum date: Sat Sep 10 19:05:32 2016 -0700 summary: Issue #26141: Update docs for typing.py. Ivan Levkivskyi. (null merge 3.5->3.6) files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 23:40:53 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sun, 11 Sep 2016 03:40:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=283=2E5=29=3A_Remove_the_deb?= =?utf-8?q?ug_print_that_i_accidentally_left_in=2E?= Message-ID: <20160911034053.8517.15048.FCB441BE@psf.io> https://hg.python.org/cpython/rev/c6e45f1f76b5 changeset: 103606:c6e45f1f76b5 branch: 3.5 parent: 103604:1dc45beca118 user: Gregory P. Smith [Google Inc.] date: Sat Sep 10 20:39:36 2016 -0700 summary: Remove the debug print that i accidentally left in. files: Lib/lib2to3/fixes/fix_reload.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Lib/lib2to3/fixes/fix_reload.py b/Lib/lib2to3/fixes/fix_reload.py --- a/Lib/lib2to3/fixes/fix_reload.py +++ b/Lib/lib2to3/fixes/fix_reload.py @@ -26,7 +26,6 @@ # I feel like we should be able to express this logic in the # PATTERN above but I don't know how to do it so... obj = results['obj'] - print('obj:', repr(obj)) if obj: if obj.type == self.syms.star_expr: return # Make no change. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sat Sep 10 23:40:53 2016 From: python-checkins at python.org (gregory.p.smith) Date: Sun, 11 Sep 2016 03:40:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Remove_the_debug_print_that_i_accidentally_left_in=2E?= Message-ID: <20160911034053.13065.9517.933892DE@psf.io> https://hg.python.org/cpython/rev/038f6e2cdf2f changeset: 103607:038f6e2cdf2f parent: 103605:960a73507d4d parent: 103606:c6e45f1f76b5 user: Gregory P. Smith [Google Inc.] date: Sat Sep 10 20:39:50 2016 -0700 summary: Remove the debug print that i accidentally left in. files: Lib/lib2to3/fixes/fix_reload.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/Lib/lib2to3/fixes/fix_reload.py b/Lib/lib2to3/fixes/fix_reload.py --- a/Lib/lib2to3/fixes/fix_reload.py +++ b/Lib/lib2to3/fixes/fix_reload.py @@ -26,7 +26,6 @@ # I feel like we should be able to express this logic in the # PATTERN above but I don't know how to do it so... obj = results['obj'] - print('obj:', repr(obj)) if obj: if obj.type == self.syms.star_expr: return # Make no change. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 00:46:06 2016 From: python-checkins at python.org (nick.coghlan) Date: Sun, 11 Sep 2016 04:46:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323722=3A_Initiali?= =?utf-8?b?emUgX19jbGFzc19fIGZyb20gdHlwZS5fX25ld19fKCk=?= Message-ID: <20160911044606.69581.27872.86E86D4D@psf.io> https://hg.python.org/cpython/rev/feb1ae9d5381 changeset: 103608:feb1ae9d5381 user: Nick Coghlan date: Sun Sep 11 14:45:49 2016 +1000 summary: Issue #23722: Initialize __class__ from type.__new__() The __class__ cell used by zero-argument super() is now initialized from type.__new__ rather than __build_class__, so class methods relying on that will now work correctly when called from metaclass methods during class creation. Patch by Martin Teichmann. files: Lib/importlib/_bootstrap_external.py | 5 +- Lib/test/test_super.py | 81 + Misc/NEWS | 5 + Objects/typeobject.c | 13 +- PC/launcher.c | 2 +- Python/bltinmodule.c | 10 +- Python/compile.c | 15 +- Python/importlib_external.h | 2489 +++++++------ 8 files changed, 1360 insertions(+), 1260 deletions(-) diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -236,7 +236,8 @@ # Python 3.6b1 3373 (add BUILD_STRING opcode #27078) # Python 3.6b1 3375 (add SETUP_ANNOTATIONS and STORE_ANNOTATION opcodes # #27985) -# Python 3.6a1 3376 (simplify CALL_FUNCTIONs & BUILD_MAP_UNPACK_WITH_CALL) +# Python 3.6b1 3376 (simplify CALL_FUNCTIONs & BUILD_MAP_UNPACK_WITH_CALL) +# Python 3.6b1 3377 (set __class__ cell from type.__new__ #23722) # # MAGIC must change whenever the bytecode emitted by the compiler may no # longer be understood by older implementations of the eval loop (usually @@ -245,7 +246,7 @@ # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # in PC/launcher.c must also be updated. -MAGIC_NUMBER = (3376).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3377).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Lib/test/test_super.py b/Lib/test/test_super.py --- a/Lib/test/test_super.py +++ b/Lib/test/test_super.py @@ -143,6 +143,87 @@ return __class__ self.assertIs(X.f(), X) + def test___class___new(self): + test_class = None + + class Meta(type): + def __new__(cls, name, bases, namespace): + nonlocal test_class + self = super().__new__(cls, name, bases, namespace) + test_class = self.f() + return self + + class A(metaclass=Meta): + @staticmethod + def f(): + return __class__ + + self.assertIs(test_class, A) + + def test___class___delayed(self): + test_namespace = None + + class Meta(type): + def __new__(cls, name, bases, namespace): + nonlocal test_namespace + test_namespace = namespace + return None + + class A(metaclass=Meta): + @staticmethod + def f(): + return __class__ + + self.assertIs(A, None) + + B = type("B", (), test_namespace) + self.assertIs(B.f(), B) + + def test___class___mro(self): + test_class = None + + class Meta(type): + def mro(self): + # self.f() doesn't work yet... + self.__dict__["f"]() + return super().mro() + + class A(metaclass=Meta): + def f(): + nonlocal test_class + test_class = __class__ + + self.assertIs(test_class, A) + + def test___classcell___deleted(self): + class Meta(type): + def __new__(cls, name, bases, namespace): + del namespace['__classcell__'] + return super().__new__(cls, name, bases, namespace) + + class A(metaclass=Meta): + @staticmethod + def f(): + __class__ + + with self.assertRaises(NameError): + A.f() + + def test___classcell___reset(self): + class Meta(type): + def __new__(cls, name, bases, namespace): + namespace['__classcell__'] = 0 + return super().__new__(cls, name, bases, namespace) + + class A(metaclass=Meta): + @staticmethod + def f(): + __class__ + + with self.assertRaises(NameError): + A.f() + self.assertEqual(A.__classcell__, 0) + def test_obscure_super_errors(self): def f(): super() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,11 @@ Core and Builtins ----------------- +- Issue #23722: The __class__ cell used by zero-argument super() is now + initialized from type.__new__ rather than __build_class__, so class methods + relying on that will now work correctly when called from metaclass methods + during class creation. Patch by Martin Teichmann. + - Issue #25221: Fix corrupted result from PyLong_FromLong(0) when Python is compiled with NSMALLPOSINTS = 0. diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -2285,7 +2285,7 @@ type_new(PyTypeObject *metatype, PyObject *args, PyObject *kwds) { PyObject *name, *bases = NULL, *orig_dict, *dict = NULL; - PyObject *qualname, *slots = NULL, *tmp, *newslots; + PyObject *qualname, *slots = NULL, *tmp, *newslots, *cell; PyTypeObject *type = NULL, *base, *tmptype, *winner; PyHeapTypeObject *et; PyMemberDef *mp; @@ -2293,6 +2293,7 @@ int j, may_add_dict, may_add_weak, add_dict, add_weak; _Py_IDENTIFIER(__qualname__); _Py_IDENTIFIER(__slots__); + _Py_IDENTIFIER(__classcell__); assert(args != NULL && PyTuple_Check(args)); assert(kwds == NULL || PyDict_Check(kwds)); @@ -2559,7 +2560,7 @@ } et->ht_qualname = qualname ? qualname : et->ht_name; Py_INCREF(et->ht_qualname); - if (qualname != NULL && PyDict_DelItem(dict, PyId___qualname__.object) < 0) + if (qualname != NULL && _PyDict_DelItemId(dict, &PyId___qualname__) < 0) goto error; /* Set tp_doc to a copy of dict['__doc__'], if the latter is there @@ -2685,6 +2686,14 @@ else type->tp_free = PyObject_Del; + /* store type in class' cell */ + cell = _PyDict_GetItemId(dict, &PyId___classcell__); + if (cell != NULL && PyCell_Check(cell)) { + PyCell_Set(cell, (PyObject *) type); + _PyDict_DelItemId(dict, &PyId___classcell__); + PyErr_Clear(); + } + /* Initialize the rest */ if (PyType_Ready(type) < 0) goto error; diff --git a/PC/launcher.c b/PC/launcher.c --- a/PC/launcher.c +++ b/PC/launcher.c @@ -1089,7 +1089,7 @@ { 3190, 3230, L"3.3" }, { 3250, 3310, L"3.4" }, { 3320, 3351, L"3.5" }, - { 3360, 3375, L"3.6" }, + { 3360, 3379, L"3.6" }, { 0 } }; diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -54,7 +54,7 @@ static PyObject * builtin___build_class__(PyObject *self, PyObject *args, PyObject *kwds) { - PyObject *func, *name, *bases, *mkw, *meta, *winner, *prep, *ns, *cell; + PyObject *func, *name, *bases, *mkw, *meta, *winner, *prep, *ns, *none; PyObject *cls = NULL; Py_ssize_t nargs; int isclass = 0; /* initialize to prevent gcc warning */ @@ -167,15 +167,13 @@ Py_DECREF(bases); return NULL; } - cell = PyEval_EvalCodeEx(PyFunction_GET_CODE(func), PyFunction_GET_GLOBALS(func), ns, + none = PyEval_EvalCodeEx(PyFunction_GET_CODE(func), PyFunction_GET_GLOBALS(func), ns, NULL, 0, NULL, 0, NULL, 0, NULL, PyFunction_GET_CLOSURE(func)); - if (cell != NULL) { + if (none != NULL) { PyObject *margs[3] = {name, bases, ns}; cls = _PyObject_FastCallDict(meta, margs, 3, mkw); - if (cls != NULL && PyCell_Check(cell)) - PyCell_Set(cell, cls); - Py_DECREF(cell); + Py_DECREF(none); } Py_DECREF(ns); Py_DECREF(meta); diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -1968,7 +1968,7 @@ return 0; } if (c->u->u_ste->ste_needs_class_closure) { - /* return the (empty) __class__ cell */ + /* store __classcell__ into class namespace */ str = PyUnicode_InternFromString("__class__"); if (str == NULL) { compiler_exit_scope(c); @@ -1981,15 +1981,20 @@ return 0; } assert(i == 0); - /* Return the cell where to store __class__ */ + ADDOP_I(c, LOAD_CLOSURE, i); + str = PyUnicode_InternFromString("__classcell__"); + if (!str || !compiler_nameop(c, str, Store)) { + Py_XDECREF(str); + compiler_exit_scope(c); + return 0; + } + Py_DECREF(str); } else { + /* This happens when nobody references the cell. */ assert(PyDict_Size(c->u->u_cellvars) == 0); - /* This happens when nobody references the cell. Return None. */ - ADDOP_O(c, LOAD_CONST, Py_None, consts); } - ADDOP_IN_SCOPE(c, RETURN_VALUE); /* create the code object */ co = assemble(c, 1); } diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -242,7 +242,7 @@ 101,95,97,116,111,109,105,99,106,0,0,0,115,26,0,0, 0,0,5,16,1,6,1,26,1,2,3,14,1,20,1,16, 1,14,1,2,1,14,1,14,1,6,1,114,58,0,0,0, - 105,48,13,0,0,233,2,0,0,0,114,15,0,0,0,115, + 105,49,13,0,0,233,2,0,0,0,114,15,0,0,0,115, 2,0,0,0,13,10,90,11,95,95,112,121,99,97,99,104, 101,95,95,122,4,111,112,116,45,122,3,46,112,121,122,4, 46,112,121,99,78,41,1,218,12,111,112,116,105,109,105,122, @@ -346,7 +346,7 @@ 103,90,15,97,108,109,111,115,116,95,102,105,108,101,110,97, 109,101,114,4,0,0,0,114,4,0,0,0,114,6,0,0, 0,218,17,99,97,99,104,101,95,102,114,111,109,95,115,111, - 117,114,99,101,4,1,0,0,115,48,0,0,0,0,18,8, + 117,114,99,101,5,1,0,0,115,48,0,0,0,0,18,8, 1,6,1,6,1,8,1,4,1,8,1,12,1,10,1,12, 1,16,1,8,1,8,1,8,1,24,1,8,1,12,1,6, 2,8,1,8,1,8,1,8,1,14,1,14,1,114,83,0, @@ -420,7 +420,7 @@ 112,116,95,108,101,118,101,108,90,13,98,97,115,101,95,102, 105,108,101,110,97,109,101,114,4,0,0,0,114,4,0,0, 0,114,6,0,0,0,218,17,115,111,117,114,99,101,95,102, - 114,111,109,95,99,97,99,104,101,49,1,0,0,115,46,0, + 114,111,109,95,99,97,99,104,101,50,1,0,0,115,46,0, 0,0,0,9,12,1,8,1,10,1,12,1,12,1,8,1, 6,1,10,1,10,1,8,1,6,1,10,1,8,1,16,1, 10,1,6,1,8,1,16,1,8,1,6,1,8,1,14,1, @@ -456,7 +456,7 @@ 115,105,111,110,218,11,115,111,117,114,99,101,95,112,97,116, 104,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, 218,15,95,103,101,116,95,115,111,117,114,99,101,102,105,108, - 101,83,1,0,0,115,20,0,0,0,0,7,12,1,4,1, + 101,84,1,0,0,115,20,0,0,0,0,7,12,1,4,1, 16,1,26,1,4,1,2,1,12,1,18,1,18,1,114,95, 0,0,0,99,1,0,0,0,0,0,0,0,1,0,0,0, 11,0,0,0,67,0,0,0,115,74,0,0,0,124,0,106, @@ -469,7 +469,7 @@ 0,0,114,83,0,0,0,114,70,0,0,0,114,78,0,0, 0,41,1,218,8,102,105,108,101,110,97,109,101,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,11,95,103, - 101,116,95,99,97,99,104,101,100,102,1,0,0,115,16,0, + 101,116,95,99,97,99,104,101,100,103,1,0,0,115,16,0, 0,0,0,1,14,1,2,1,8,1,14,1,8,1,14,1, 6,2,114,99,0,0,0,99,1,0,0,0,0,0,0,0, 2,0,0,0,11,0,0,0,67,0,0,0,115,52,0,0, @@ -483,7 +483,7 @@ 0,233,128,0,0,0,41,3,114,41,0,0,0,114,43,0, 0,0,114,42,0,0,0,41,2,114,37,0,0,0,114,44, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,10,95,99,97,108,99,95,109,111,100,101,114,1, + 0,0,218,10,95,99,97,108,99,95,109,111,100,101,115,1, 0,0,115,12,0,0,0,0,2,2,1,14,1,14,1,10, 3,8,1,114,101,0,0,0,99,1,0,0,0,0,0,0, 0,3,0,0,0,11,0,0,0,3,0,0,0,115,68,0, @@ -521,7 +521,7 @@ 0,218,4,97,114,103,115,90,6,107,119,97,114,103,115,41, 1,218,6,109,101,116,104,111,100,114,4,0,0,0,114,6, 0,0,0,218,19,95,99,104,101,99,107,95,110,97,109,101, - 95,119,114,97,112,112,101,114,134,1,0,0,115,12,0,0, + 95,119,114,97,112,112,101,114,135,1,0,0,115,12,0,0, 0,0,1,8,1,8,1,10,1,4,1,18,1,122,40,95, 99,104,101,99,107,95,110,97,109,101,46,60,108,111,99,97, 108,115,62,46,95,99,104,101,99,107,95,110,97,109,101,95, @@ -542,14 +542,14 @@ 116,95,95,218,6,117,112,100,97,116,101,41,3,90,3,110, 101,119,90,3,111,108,100,114,55,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,5,95,119,114, - 97,112,145,1,0,0,115,8,0,0,0,0,1,10,1,10, + 97,112,146,1,0,0,115,8,0,0,0,0,1,10,1,10, 1,22,1,122,26,95,99,104,101,99,107,95,110,97,109,101, 46,60,108,111,99,97,108,115,62,46,95,119,114,97,112,41, 1,78,41,3,218,10,95,98,111,111,116,115,116,114,97,112, 114,117,0,0,0,218,9,78,97,109,101,69,114,114,111,114, 41,3,114,106,0,0,0,114,107,0,0,0,114,117,0,0, 0,114,4,0,0,0,41,1,114,106,0,0,0,114,6,0, - 0,0,218,11,95,99,104,101,99,107,95,110,97,109,101,126, + 0,0,218,11,95,99,104,101,99,107,95,110,97,109,101,127, 1,0,0,115,14,0,0,0,0,8,14,7,2,1,10,1, 14,2,14,5,10,1,114,120,0,0,0,99,2,0,0,0, 0,0,0,0,5,0,0,0,4,0,0,0,67,0,0,0, @@ -578,7 +578,7 @@ 8,112,111,114,116,105,111,110,115,218,3,109,115,103,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,218,17,95, 102,105,110,100,95,109,111,100,117,108,101,95,115,104,105,109, - 154,1,0,0,115,10,0,0,0,0,10,14,1,16,1,4, + 155,1,0,0,115,10,0,0,0,0,10,14,1,16,1,4, 1,22,1,114,127,0,0,0,99,4,0,0,0,0,0,0, 0,11,0,0,0,22,0,0,0,67,0,0,0,115,142,1, 0,0,105,0,125,4,124,2,100,1,107,9,114,22,124,2, @@ -658,7 +658,7 @@ 101,218,11,115,111,117,114,99,101,95,115,105,122,101,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,218,25,95, 118,97,108,105,100,97,116,101,95,98,121,116,101,99,111,100, - 101,95,104,101,97,100,101,114,171,1,0,0,115,76,0,0, + 101,95,104,101,97,100,101,114,172,1,0,0,115,76,0,0, 0,0,11,4,1,8,1,10,3,4,1,8,1,8,1,12, 1,12,1,12,1,8,1,12,1,12,1,16,1,12,1,10, 1,12,1,10,1,12,1,10,1,12,1,8,1,10,1,2, @@ -687,7 +687,7 @@ 0,41,5,114,56,0,0,0,114,102,0,0,0,114,93,0, 0,0,114,94,0,0,0,218,4,99,111,100,101,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,17,95,99, - 111,109,112,105,108,101,95,98,121,116,101,99,111,100,101,226, + 111,109,112,105,108,101,95,98,121,116,101,99,111,100,101,227, 1,0,0,115,16,0,0,0,0,2,10,1,10,1,12,1, 8,1,12,1,6,2,10,1,114,145,0,0,0,114,62,0, 0,0,99,3,0,0,0,0,0,0,0,4,0,0,0,3, @@ -706,7 +706,7 @@ 109,112,115,41,4,114,144,0,0,0,114,130,0,0,0,114, 138,0,0,0,114,56,0,0,0,114,4,0,0,0,114,4, 0,0,0,114,6,0,0,0,218,17,95,99,111,100,101,95, - 116,111,95,98,121,116,101,99,111,100,101,238,1,0,0,115, + 116,111,95,98,121,116,101,99,111,100,101,239,1,0,0,115, 10,0,0,0,0,3,8,1,14,1,14,1,16,1,114,148, 0,0,0,99,1,0,0,0,0,0,0,0,5,0,0,0, 4,0,0,0,67,0,0,0,115,62,0,0,0,100,1,100, @@ -733,7 +733,7 @@ 105,110,101,218,8,101,110,99,111,100,105,110,103,90,15,110, 101,119,108,105,110,101,95,100,101,99,111,100,101,114,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,218,13,100, - 101,99,111,100,101,95,115,111,117,114,99,101,248,1,0,0, + 101,99,111,100,101,95,115,111,117,114,99,101,249,1,0,0, 115,10,0,0,0,0,5,8,1,12,1,10,1,12,1,114, 153,0,0,0,41,2,114,124,0,0,0,218,26,115,117,98, 109,111,100,117,108,101,95,115,101,97,114,99,104,95,108,111, @@ -795,7 +795,7 @@ 157,0,0,0,90,7,100,105,114,110,97,109,101,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,218,23,115,112, 101,99,95,102,114,111,109,95,102,105,108,101,95,108,111,99, - 97,116,105,111,110,9,2,0,0,115,62,0,0,0,0,12, + 97,116,105,111,110,10,2,0,0,115,62,0,0,0,0,12, 8,4,4,1,10,2,2,1,14,1,14,1,8,2,10,8, 16,1,6,3,8,1,16,1,14,1,10,1,6,1,6,2, 4,3,8,2,10,1,2,1,14,1,14,1,6,2,4,1, @@ -831,7 +831,7 @@ 90,18,72,75,69,89,95,76,79,67,65,76,95,77,65,67, 72,73,78,69,41,2,218,3,99,108,115,114,5,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 14,95,111,112,101,110,95,114,101,103,105,115,116,114,121,89, + 14,95,111,112,101,110,95,114,101,103,105,115,116,114,121,90, 2,0,0,115,8,0,0,0,0,2,2,1,14,1,14,1, 122,36,87,105,110,100,111,119,115,82,101,103,105,115,116,114, 121,70,105,110,100,101,114,46,95,111,112,101,110,95,114,101, @@ -857,7 +857,7 @@ 114,121,95,107,101,121,114,5,0,0,0,90,4,104,107,101, 121,218,8,102,105,108,101,112,97,116,104,114,4,0,0,0, 114,4,0,0,0,114,6,0,0,0,218,16,95,115,101,97, - 114,99,104,95,114,101,103,105,115,116,114,121,96,2,0,0, + 114,99,104,95,114,101,103,105,115,116,114,121,97,2,0,0, 115,22,0,0,0,0,2,6,1,8,2,6,1,6,1,22, 1,2,1,12,1,26,1,14,1,6,1,122,38,87,105,110, 100,111,119,115,82,101,103,105,115,116,114,121,70,105,110,100, @@ -879,7 +879,7 @@ 0,0,0,114,37,0,0,0,218,6,116,97,114,103,101,116, 114,174,0,0,0,114,124,0,0,0,114,164,0,0,0,114, 162,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,9,102,105,110,100,95,115,112,101,99,111,2, + 0,0,0,218,9,102,105,110,100,95,115,112,101,99,112,2, 0,0,115,26,0,0,0,0,2,10,1,8,1,4,1,2, 1,12,1,14,1,6,1,16,1,14,1,6,1,8,1,8, 1,122,31,87,105,110,100,111,119,115,82,101,103,105,115,116, @@ -898,7 +898,7 @@ 78,41,2,114,178,0,0,0,114,124,0,0,0,41,4,114, 168,0,0,0,114,123,0,0,0,114,37,0,0,0,114,162, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,102,105,110,100,95,109,111,100,117,108,101,127, + 0,0,218,11,102,105,110,100,95,109,111,100,117,108,101,128, 2,0,0,115,8,0,0,0,0,7,12,1,8,1,8,2, 122,33,87,105,110,100,111,119,115,82,101,103,105,115,116,114, 121,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, @@ -908,7 +908,7 @@ 11,99,108,97,115,115,109,101,116,104,111,100,114,169,0,0, 0,114,175,0,0,0,114,178,0,0,0,114,179,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,166,0,0,0,77,2,0,0,115,20,0, + 6,0,0,0,114,166,0,0,0,78,2,0,0,115,20,0, 0,0,8,2,4,3,4,3,4,2,4,2,12,7,12,15, 2,1,12,15,2,1,114,166,0,0,0,99,0,0,0,0, 0,0,0,0,0,0,0,0,2,0,0,0,64,0,0,0, @@ -943,7 +943,7 @@ 0,114,123,0,0,0,114,98,0,0,0,90,13,102,105,108, 101,110,97,109,101,95,98,97,115,101,90,9,116,97,105,108, 95,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,157,0,0,0,146,2,0,0,115,8,0, + 6,0,0,0,114,157,0,0,0,147,2,0,0,115,8,0, 0,0,0,3,18,1,16,1,14,1,122,24,95,76,111,97, 100,101,114,66,97,115,105,99,115,46,105,115,95,112,97,99, 107,97,103,101,99,2,0,0,0,0,0,0,0,2,0,0, @@ -954,7 +954,7 @@ 78,114,4,0,0,0,41,2,114,104,0,0,0,114,162,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, 0,218,13,99,114,101,97,116,101,95,109,111,100,117,108,101, - 154,2,0,0,115,0,0,0,0,122,27,95,76,111,97,100, + 155,2,0,0,115,0,0,0,0,122,27,95,76,111,97,100, 101,114,66,97,115,105,99,115,46,99,114,101,97,116,101,95, 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,3, 0,0,0,4,0,0,0,67,0,0,0,115,56,0,0,0, @@ -973,7 +973,7 @@ 100,218,4,101,120,101,99,114,115,0,0,0,41,3,114,104, 0,0,0,218,6,109,111,100,117,108,101,114,144,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 11,101,120,101,99,95,109,111,100,117,108,101,157,2,0,0, + 11,101,120,101,99,95,109,111,100,117,108,101,158,2,0,0, 115,10,0,0,0,0,2,12,1,8,1,6,1,10,1,122, 25,95,76,111,97,100,101,114,66,97,115,105,99,115,46,101, 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, @@ -985,13 +985,13 @@ 117,108,101,95,115,104,105,109,41,2,114,104,0,0,0,114, 123,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, 0,0,0,218,11,108,111,97,100,95,109,111,100,117,108,101, - 165,2,0,0,115,2,0,0,0,0,2,122,25,95,76,111, + 166,2,0,0,115,2,0,0,0,0,2,122,25,95,76,111, 97,100,101,114,66,97,115,105,99,115,46,108,111,97,100,95, 109,111,100,117,108,101,78,41,8,114,109,0,0,0,114,108, 0,0,0,114,110,0,0,0,114,111,0,0,0,114,157,0, 0,0,114,183,0,0,0,114,188,0,0,0,114,190,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,181,0,0,0,141,2,0,0,115,10, + 114,6,0,0,0,114,181,0,0,0,142,2,0,0,115,10, 0,0,0,8,3,4,2,8,8,8,3,8,8,114,181,0, 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,3, 0,0,0,64,0,0,0,115,74,0,0,0,101,0,90,1, @@ -1017,7 +1017,7 @@ 1,218,7,73,79,69,114,114,111,114,41,2,114,104,0,0, 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, 114,6,0,0,0,218,10,112,97,116,104,95,109,116,105,109, - 101,172,2,0,0,115,2,0,0,0,0,6,122,23,83,111, + 101,173,2,0,0,115,2,0,0,0,0,6,122,23,83,111, 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, 109,116,105,109,101,99,2,0,0,0,0,0,0,0,2,0, 0,0,3,0,0,0,67,0,0,0,115,14,0,0,0,100, @@ -1052,7 +1052,7 @@ 0,0,0,41,1,114,193,0,0,0,41,2,114,104,0,0, 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, 114,6,0,0,0,218,10,112,97,116,104,95,115,116,97,116, - 115,180,2,0,0,115,2,0,0,0,0,11,122,23,83,111, + 115,181,2,0,0,115,2,0,0,0,0,11,122,23,83,111, 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, 115,116,97,116,115,99,4,0,0,0,0,0,0,0,4,0, 0,0,3,0,0,0,67,0,0,0,115,12,0,0,0,124, @@ -1075,7 +1075,7 @@ 4,114,104,0,0,0,114,94,0,0,0,90,10,99,97,99, 104,101,95,112,97,116,104,114,56,0,0,0,114,4,0,0, 0,114,4,0,0,0,114,6,0,0,0,218,15,95,99,97, - 99,104,101,95,98,121,116,101,99,111,100,101,193,2,0,0, + 99,104,101,95,98,121,116,101,99,111,100,101,194,2,0,0, 115,2,0,0,0,0,8,122,28,83,111,117,114,99,101,76, 111,97,100,101,114,46,95,99,97,99,104,101,95,98,121,116, 101,99,111,100,101,99,3,0,0,0,0,0,0,0,3,0, @@ -1092,7 +1092,7 @@ 108,101,115,46,10,32,32,32,32,32,32,32,32,78,114,4, 0,0,0,41,3,114,104,0,0,0,114,37,0,0,0,114, 56,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,195,0,0,0,203,2,0,0,115,0,0,0, + 0,0,0,114,195,0,0,0,204,2,0,0,115,0,0,0, 0,122,21,83,111,117,114,99,101,76,111,97,100,101,114,46, 115,101,116,95,100,97,116,97,99,2,0,0,0,0,0,0, 0,5,0,0,0,16,0,0,0,67,0,0,0,115,82,0, @@ -1113,7 +1113,7 @@ 104,0,0,0,114,123,0,0,0,114,37,0,0,0,114,151, 0,0,0,218,3,101,120,99,114,4,0,0,0,114,4,0, 0,0,114,6,0,0,0,218,10,103,101,116,95,115,111,117, - 114,99,101,210,2,0,0,115,14,0,0,0,0,2,10,1, + 114,99,101,211,2,0,0,115,14,0,0,0,0,2,10,1, 2,1,14,1,16,1,4,1,28,1,122,23,83,111,117,114, 99,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, 114,99,101,114,31,0,0,0,41,1,218,9,95,111,112,116, @@ -1135,7 +1135,7 @@ 0,0,114,56,0,0,0,114,37,0,0,0,114,200,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, 218,14,115,111,117,114,99,101,95,116,111,95,99,111,100,101, - 220,2,0,0,115,4,0,0,0,0,5,12,1,122,27,83, + 221,2,0,0,115,4,0,0,0,0,5,12,1,122,27,83, 111,117,114,99,101,76,111,97,100,101,114,46,115,111,117,114, 99,101,95,116,111,95,99,111,100,101,99,2,0,0,0,0, 0,0,0,10,0,0,0,43,0,0,0,67,0,0,0,115, @@ -1191,7 +1191,7 @@ 116,114,56,0,0,0,218,10,98,121,116,101,115,95,100,97, 116,97,114,151,0,0,0,90,11,99,111,100,101,95,111,98, 106,101,99,116,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,184,0,0,0,228,2,0,0,115,78,0,0, + 0,0,0,114,184,0,0,0,229,2,0,0,115,78,0,0, 0,0,7,10,1,4,1,2,1,12,1,14,1,10,2,2, 1,14,1,14,1,6,2,12,1,2,1,14,1,14,1,6, 2,2,1,4,1,4,1,12,1,18,1,6,2,8,1,6, @@ -1203,1237 +1203,1238 @@ 0,0,114,194,0,0,0,114,196,0,0,0,114,195,0,0, 0,114,199,0,0,0,114,203,0,0,0,114,184,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,191,0,0,0,170,2,0,0,115,14,0, + 6,0,0,0,114,191,0,0,0,171,2,0,0,115,14,0, 0,0,8,2,8,8,8,13,8,10,8,7,8,10,14,8, 114,191,0,0,0,99,0,0,0,0,0,0,0,0,0,0, - 0,0,4,0,0,0,0,0,0,0,115,76,0,0,0,101, + 0,0,4,0,0,0,0,0,0,0,115,80,0,0,0,101, 0,90,1,100,0,90,2,100,1,90,3,100,2,100,3,132, 0,90,4,100,4,100,5,132,0,90,5,100,6,100,7,132, 0,90,6,101,7,135,0,102,1,100,8,100,9,132,8,131, 1,90,8,101,7,100,10,100,11,132,0,131,1,90,9,100, - 12,100,13,132,0,90,10,135,0,83,0,41,14,218,10,70, - 105,108,101,76,111,97,100,101,114,122,103,66,97,115,101,32, - 102,105,108,101,32,108,111,97,100,101,114,32,99,108,97,115, - 115,32,119,104,105,99,104,32,105,109,112,108,101,109,101,110, - 116,115,32,116,104,101,32,108,111,97,100,101,114,32,112,114, - 111,116,111,99,111,108,32,109,101,116,104,111,100,115,32,116, - 104,97,116,10,32,32,32,32,114,101,113,117,105,114,101,32, - 102,105,108,101,32,115,121,115,116,101,109,32,117,115,97,103, - 101,46,99,3,0,0,0,0,0,0,0,3,0,0,0,2, - 0,0,0,67,0,0,0,115,16,0,0,0,124,1,124,0, - 95,0,124,2,124,0,95,1,100,1,83,0,41,2,122,75, - 67,97,99,104,101,32,116,104,101,32,109,111,100,117,108,101, - 32,110,97,109,101,32,97,110,100,32,116,104,101,32,112,97, - 116,104,32,116,111,32,116,104,101,32,102,105,108,101,32,102, - 111,117,110,100,32,98,121,32,116,104,101,10,32,32,32,32, - 32,32,32,32,102,105,110,100,101,114,46,78,41,2,114,102, - 0,0,0,114,37,0,0,0,41,3,114,104,0,0,0,114, - 123,0,0,0,114,37,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,182,0,0,0,29,3,0, - 0,115,4,0,0,0,0,3,6,1,122,19,70,105,108,101, - 76,111,97,100,101,114,46,95,95,105,110,105,116,95,95,99, - 2,0,0,0,0,0,0,0,2,0,0,0,2,0,0,0, - 67,0,0,0,115,24,0,0,0,124,0,106,0,124,1,106, - 0,107,2,111,22,124,0,106,1,124,1,106,1,107,2,83, - 0,41,1,78,41,2,218,9,95,95,99,108,97,115,115,95, - 95,114,115,0,0,0,41,2,114,104,0,0,0,218,5,111, - 116,104,101,114,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,6,95,95,101,113,95,95,35,3,0,0,115, - 4,0,0,0,0,1,12,1,122,17,70,105,108,101,76,111, + 12,100,13,132,0,90,10,135,0,90,11,100,14,83,0,41, + 15,218,10,70,105,108,101,76,111,97,100,101,114,122,103,66, + 97,115,101,32,102,105,108,101,32,108,111,97,100,101,114,32, + 99,108,97,115,115,32,119,104,105,99,104,32,105,109,112,108, + 101,109,101,110,116,115,32,116,104,101,32,108,111,97,100,101, + 114,32,112,114,111,116,111,99,111,108,32,109,101,116,104,111, + 100,115,32,116,104,97,116,10,32,32,32,32,114,101,113,117, + 105,114,101,32,102,105,108,101,32,115,121,115,116,101,109,32, + 117,115,97,103,101,46,99,3,0,0,0,0,0,0,0,3, + 0,0,0,2,0,0,0,67,0,0,0,115,16,0,0,0, + 124,1,124,0,95,0,124,2,124,0,95,1,100,1,83,0, + 41,2,122,75,67,97,99,104,101,32,116,104,101,32,109,111, + 100,117,108,101,32,110,97,109,101,32,97,110,100,32,116,104, + 101,32,112,97,116,104,32,116,111,32,116,104,101,32,102,105, + 108,101,32,102,111,117,110,100,32,98,121,32,116,104,101,10, + 32,32,32,32,32,32,32,32,102,105,110,100,101,114,46,78, + 41,2,114,102,0,0,0,114,37,0,0,0,41,3,114,104, + 0,0,0,114,123,0,0,0,114,37,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,182,0,0, + 0,30,3,0,0,115,4,0,0,0,0,3,6,1,122,19, + 70,105,108,101,76,111,97,100,101,114,46,95,95,105,110,105, + 116,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, + 2,0,0,0,67,0,0,0,115,24,0,0,0,124,0,106, + 0,124,1,106,0,107,2,111,22,124,0,106,1,124,1,106, + 1,107,2,83,0,41,1,78,41,2,218,9,95,95,99,108, + 97,115,115,95,95,114,115,0,0,0,41,2,114,104,0,0, + 0,218,5,111,116,104,101,114,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,218,6,95,95,101,113,95,95,36, + 3,0,0,115,4,0,0,0,0,1,12,1,122,17,70,105, + 108,101,76,111,97,100,101,114,46,95,95,101,113,95,95,99, + 1,0,0,0,0,0,0,0,1,0,0,0,3,0,0,0, + 67,0,0,0,115,20,0,0,0,116,0,124,0,106,1,131, + 1,116,0,124,0,106,2,131,1,65,0,83,0,41,1,78, + 41,3,218,4,104,97,115,104,114,102,0,0,0,114,37,0, + 0,0,41,1,114,104,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,218,8,95,95,104,97,115,104, + 95,95,40,3,0,0,115,2,0,0,0,0,1,122,19,70, + 105,108,101,76,111,97,100,101,114,46,95,95,104,97,115,104, + 95,95,99,2,0,0,0,0,0,0,0,2,0,0,0,3, + 0,0,0,3,0,0,0,115,16,0,0,0,116,0,116,1, + 124,0,131,2,106,2,124,1,131,1,83,0,41,1,122,100, + 76,111,97,100,32,97,32,109,111,100,117,108,101,32,102,114, + 111,109,32,97,32,102,105,108,101,46,10,10,32,32,32,32, + 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, + 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, + 85,115,101,32,101,120,101,99,95,109,111,100,117,108,101,40, + 41,32,105,110,115,116,101,97,100,46,10,10,32,32,32,32, + 32,32,32,32,41,3,218,5,115,117,112,101,114,114,207,0, + 0,0,114,190,0,0,0,41,2,114,104,0,0,0,114,123, + 0,0,0,41,1,114,208,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,190,0,0,0,43,3,0,0,115,2,0, + 0,0,0,10,122,22,70,105,108,101,76,111,97,100,101,114, + 46,108,111,97,100,95,109,111,100,117,108,101,99,2,0,0, + 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, + 0,115,6,0,0,0,124,0,106,0,83,0,41,1,122,58, + 82,101,116,117,114,110,32,116,104,101,32,112,97,116,104,32, + 116,111,32,116,104,101,32,115,111,117,114,99,101,32,102,105, + 108,101,32,97,115,32,102,111,117,110,100,32,98,121,32,116, + 104,101,32,102,105,110,100,101,114,46,41,1,114,37,0,0, + 0,41,2,114,104,0,0,0,114,123,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,155,0,0, + 0,55,3,0,0,115,2,0,0,0,0,3,122,23,70,105, + 108,101,76,111,97,100,101,114,46,103,101,116,95,102,105,108, + 101,110,97,109,101,99,2,0,0,0,0,0,0,0,3,0, + 0,0,9,0,0,0,67,0,0,0,115,32,0,0,0,116, + 0,106,1,124,1,100,1,131,2,143,10,125,2,124,2,106, + 2,131,0,83,0,81,0,82,0,88,0,100,2,83,0,41, + 3,122,39,82,101,116,117,114,110,32,116,104,101,32,100,97, + 116,97,32,102,114,111,109,32,112,97,116,104,32,97,115,32, + 114,97,119,32,98,121,116,101,115,46,218,1,114,78,41,3, + 114,52,0,0,0,114,53,0,0,0,90,4,114,101,97,100, + 41,3,114,104,0,0,0,114,37,0,0,0,114,57,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, + 114,197,0,0,0,60,3,0,0,115,4,0,0,0,0,2, + 14,1,122,19,70,105,108,101,76,111,97,100,101,114,46,103, + 101,116,95,100,97,116,97,78,41,12,114,109,0,0,0,114, + 108,0,0,0,114,110,0,0,0,114,111,0,0,0,114,182, + 0,0,0,114,210,0,0,0,114,212,0,0,0,114,120,0, + 0,0,114,190,0,0,0,114,155,0,0,0,114,197,0,0, + 0,90,13,95,95,99,108,97,115,115,99,101,108,108,95,95, + 114,4,0,0,0,114,4,0,0,0,41,1,114,208,0,0, + 0,114,6,0,0,0,114,207,0,0,0,25,3,0,0,115, + 14,0,0,0,8,3,4,2,8,6,8,4,8,3,16,12, + 12,5,114,207,0,0,0,99,0,0,0,0,0,0,0,0, + 0,0,0,0,3,0,0,0,64,0,0,0,115,46,0,0, + 0,101,0,90,1,100,0,90,2,100,1,90,3,100,2,100, + 3,132,0,90,4,100,4,100,5,132,0,90,5,100,6,100, + 7,156,1,100,8,100,9,132,2,90,6,100,10,83,0,41, + 11,218,16,83,111,117,114,99,101,70,105,108,101,76,111,97, + 100,101,114,122,62,67,111,110,99,114,101,116,101,32,105,109, + 112,108,101,109,101,110,116,97,116,105,111,110,32,111,102,32, + 83,111,117,114,99,101,76,111,97,100,101,114,32,117,115,105, + 110,103,32,116,104,101,32,102,105,108,101,32,115,121,115,116, + 101,109,46,99,2,0,0,0,0,0,0,0,3,0,0,0, + 3,0,0,0,67,0,0,0,115,22,0,0,0,116,0,124, + 1,131,1,125,2,124,2,106,1,124,2,106,2,100,1,156, + 2,83,0,41,2,122,33,82,101,116,117,114,110,32,116,104, + 101,32,109,101,116,97,100,97,116,97,32,102,111,114,32,116, + 104,101,32,112,97,116,104,46,41,2,122,5,109,116,105,109, + 101,122,4,115,105,122,101,41,3,114,41,0,0,0,218,8, + 115,116,95,109,116,105,109,101,90,7,115,116,95,115,105,122, + 101,41,3,114,104,0,0,0,114,37,0,0,0,114,205,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,194,0,0,0,70,3,0,0,115,4,0,0,0,0, + 2,8,1,122,27,83,111,117,114,99,101,70,105,108,101,76, + 111,97,100,101,114,46,112,97,116,104,95,115,116,97,116,115, + 99,4,0,0,0,0,0,0,0,5,0,0,0,5,0,0, + 0,67,0,0,0,115,24,0,0,0,116,0,124,1,131,1, + 125,4,124,0,106,1,124,2,124,3,124,4,100,1,141,3, + 83,0,41,2,78,41,1,218,5,95,109,111,100,101,41,2, + 114,101,0,0,0,114,195,0,0,0,41,5,114,104,0,0, + 0,114,94,0,0,0,114,93,0,0,0,114,56,0,0,0, + 114,44,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,196,0,0,0,75,3,0,0,115,4,0, + 0,0,0,2,8,1,122,32,83,111,117,114,99,101,70,105, + 108,101,76,111,97,100,101,114,46,95,99,97,99,104,101,95, + 98,121,116,101,99,111,100,101,105,182,1,0,0,41,1,114, + 217,0,0,0,99,3,0,0,0,1,0,0,0,9,0,0, + 0,17,0,0,0,67,0,0,0,115,250,0,0,0,116,0, + 124,1,131,1,92,2,125,4,125,5,103,0,125,6,120,40, + 124,4,114,56,116,1,124,4,131,1,12,0,114,56,116,0, + 124,4,131,1,92,2,125,4,125,7,124,6,106,2,124,7, + 131,1,1,0,113,18,87,0,120,108,116,3,124,6,131,1, + 68,0,93,96,125,7,116,4,124,4,124,7,131,2,125,4, + 121,14,116,5,106,6,124,4,131,1,1,0,87,0,113,68, + 4,0,116,7,107,10,114,118,1,0,1,0,1,0,119,68, + 89,0,113,68,4,0,116,8,107,10,114,162,1,0,125,8, + 1,0,122,18,116,9,106,10,100,1,124,4,124,8,131,3, + 1,0,100,2,83,0,100,2,125,8,126,8,88,0,113,68, + 88,0,113,68,87,0,121,28,116,11,124,1,124,2,124,3, + 131,3,1,0,116,9,106,10,100,3,124,1,131,2,1,0, + 87,0,110,48,4,0,116,8,107,10,114,244,1,0,125,8, + 1,0,122,20,116,9,106,10,100,1,124,1,124,8,131,3, + 1,0,87,0,89,0,100,2,100,2,125,8,126,8,88,0, + 110,2,88,0,100,2,83,0,41,4,122,27,87,114,105,116, + 101,32,98,121,116,101,115,32,100,97,116,97,32,116,111,32, + 97,32,102,105,108,101,46,122,27,99,111,117,108,100,32,110, + 111,116,32,99,114,101,97,116,101,32,123,33,114,125,58,32, + 123,33,114,125,78,122,12,99,114,101,97,116,101,100,32,123, + 33,114,125,41,12,114,40,0,0,0,114,48,0,0,0,114, + 161,0,0,0,114,35,0,0,0,114,30,0,0,0,114,3, + 0,0,0,90,5,109,107,100,105,114,218,15,70,105,108,101, + 69,120,105,115,116,115,69,114,114,111,114,114,42,0,0,0, + 114,118,0,0,0,114,133,0,0,0,114,58,0,0,0,41, + 9,114,104,0,0,0,114,37,0,0,0,114,56,0,0,0, + 114,217,0,0,0,218,6,112,97,114,101,110,116,114,98,0, + 0,0,114,29,0,0,0,114,25,0,0,0,114,198,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, + 114,195,0,0,0,80,3,0,0,115,42,0,0,0,0,2, + 12,1,4,2,16,1,12,1,14,2,14,1,10,1,2,1, + 14,1,14,2,6,1,16,3,6,1,8,1,20,1,2,1, + 12,1,16,1,16,2,8,1,122,25,83,111,117,114,99,101, + 70,105,108,101,76,111,97,100,101,114,46,115,101,116,95,100, + 97,116,97,78,41,7,114,109,0,0,0,114,108,0,0,0, + 114,110,0,0,0,114,111,0,0,0,114,194,0,0,0,114, + 196,0,0,0,114,195,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,215,0, + 0,0,66,3,0,0,115,8,0,0,0,8,2,4,2,8, + 5,8,5,114,215,0,0,0,99,0,0,0,0,0,0,0, + 0,0,0,0,0,2,0,0,0,64,0,0,0,115,32,0, + 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, + 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, + 83,0,41,7,218,20,83,111,117,114,99,101,108,101,115,115, + 70,105,108,101,76,111,97,100,101,114,122,45,76,111,97,100, + 101,114,32,119,104,105,99,104,32,104,97,110,100,108,101,115, + 32,115,111,117,114,99,101,108,101,115,115,32,102,105,108,101, + 32,105,109,112,111,114,116,115,46,99,2,0,0,0,0,0, + 0,0,5,0,0,0,5,0,0,0,67,0,0,0,115,48, + 0,0,0,124,0,106,0,124,1,131,1,125,2,124,0,106, + 1,124,2,131,1,125,3,116,2,124,3,124,1,124,2,100, + 1,141,3,125,4,116,3,124,4,124,1,124,2,100,2,141, + 3,83,0,41,3,78,41,2,114,102,0,0,0,114,37,0, + 0,0,41,2,114,102,0,0,0,114,93,0,0,0,41,4, + 114,155,0,0,0,114,197,0,0,0,114,139,0,0,0,114, + 145,0,0,0,41,5,114,104,0,0,0,114,123,0,0,0, + 114,37,0,0,0,114,56,0,0,0,114,206,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,184, + 0,0,0,115,3,0,0,115,8,0,0,0,0,1,10,1, + 10,1,14,1,122,29,83,111,117,114,99,101,108,101,115,115, + 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, + 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, + 0,41,2,122,39,82,101,116,117,114,110,32,78,111,110,101, + 32,97,115,32,116,104,101,114,101,32,105,115,32,110,111,32, + 115,111,117,114,99,101,32,99,111,100,101,46,78,114,4,0, + 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, + 0,0,121,3,0,0,115,2,0,0,0,0,2,122,31,83, + 111,117,114,99,101,108,101,115,115,70,105,108,101,76,111,97, + 100,101,114,46,103,101,116,95,115,111,117,114,99,101,78,41, + 6,114,109,0,0,0,114,108,0,0,0,114,110,0,0,0, + 114,111,0,0,0,114,184,0,0,0,114,199,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,114,220,0,0,0,111,3,0,0,115,6,0,0, + 0,8,2,4,2,8,6,114,220,0,0,0,99,0,0,0, + 0,0,0,0,0,0,0,0,0,3,0,0,0,64,0,0, + 0,115,92,0,0,0,101,0,90,1,100,0,90,2,100,1, + 90,3,100,2,100,3,132,0,90,4,100,4,100,5,132,0, + 90,5,100,6,100,7,132,0,90,6,100,8,100,9,132,0, + 90,7,100,10,100,11,132,0,90,8,100,12,100,13,132,0, + 90,9,100,14,100,15,132,0,90,10,100,16,100,17,132,0, + 90,11,101,12,100,18,100,19,132,0,131,1,90,13,100,20, + 83,0,41,21,218,19,69,120,116,101,110,115,105,111,110,70, + 105,108,101,76,111,97,100,101,114,122,93,76,111,97,100,101, + 114,32,102,111,114,32,101,120,116,101,110,115,105,111,110,32, + 109,111,100,117,108,101,115,46,10,10,32,32,32,32,84,104, + 101,32,99,111,110,115,116,114,117,99,116,111,114,32,105,115, + 32,100,101,115,105,103,110,101,100,32,116,111,32,119,111,114, + 107,32,119,105,116,104,32,70,105,108,101,70,105,110,100,101, + 114,46,10,10,32,32,32,32,99,3,0,0,0,0,0,0, + 0,3,0,0,0,2,0,0,0,67,0,0,0,115,16,0, + 0,0,124,1,124,0,95,0,124,2,124,0,95,1,100,0, + 83,0,41,1,78,41,2,114,102,0,0,0,114,37,0,0, + 0,41,3,114,104,0,0,0,114,102,0,0,0,114,37,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,182,0,0,0,138,3,0,0,115,4,0,0,0,0, + 1,6,1,122,28,69,120,116,101,110,115,105,111,110,70,105, + 108,101,76,111,97,100,101,114,46,95,95,105,110,105,116,95, + 95,99,2,0,0,0,0,0,0,0,2,0,0,0,2,0, + 0,0,67,0,0,0,115,24,0,0,0,124,0,106,0,124, + 1,106,0,107,2,111,22,124,0,106,1,124,1,106,1,107, + 2,83,0,41,1,78,41,2,114,208,0,0,0,114,115,0, + 0,0,41,2,114,104,0,0,0,114,209,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,210,0, + 0,0,142,3,0,0,115,4,0,0,0,0,1,12,1,122, + 26,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, 97,100,101,114,46,95,95,101,113,95,95,99,1,0,0,0, 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, 115,20,0,0,0,116,0,124,0,106,1,131,1,116,0,124, - 0,106,2,131,1,65,0,83,0,41,1,78,41,3,218,4, - 104,97,115,104,114,102,0,0,0,114,37,0,0,0,41,1, - 114,104,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,218,8,95,95,104,97,115,104,95,95,39,3, - 0,0,115,2,0,0,0,0,1,122,19,70,105,108,101,76, - 111,97,100,101,114,46,95,95,104,97,115,104,95,95,99,2, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,3, - 0,0,0,115,16,0,0,0,116,0,116,1,124,0,131,2, - 106,2,124,1,131,1,83,0,41,1,122,100,76,111,97,100, - 32,97,32,109,111,100,117,108,101,32,102,114,111,109,32,97, - 32,102,105,108,101,46,10,10,32,32,32,32,32,32,32,32, - 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, - 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, - 101,120,101,99,95,109,111,100,117,108,101,40,41,32,105,110, - 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, - 41,3,218,5,115,117,112,101,114,114,207,0,0,0,114,190, - 0,0,0,41,2,114,104,0,0,0,114,123,0,0,0,41, - 1,114,208,0,0,0,114,4,0,0,0,114,6,0,0,0, - 114,190,0,0,0,42,3,0,0,115,2,0,0,0,0,10, - 122,22,70,105,108,101,76,111,97,100,101,114,46,108,111,97, - 100,95,109,111,100,117,108,101,99,2,0,0,0,0,0,0, - 0,2,0,0,0,1,0,0,0,67,0,0,0,115,6,0, - 0,0,124,0,106,0,83,0,41,1,122,58,82,101,116,117, - 114,110,32,116,104,101,32,112,97,116,104,32,116,111,32,116, - 104,101,32,115,111,117,114,99,101,32,102,105,108,101,32,97, - 115,32,102,111,117,110,100,32,98,121,32,116,104,101,32,102, - 105,110,100,101,114,46,41,1,114,37,0,0,0,41,2,114, - 104,0,0,0,114,123,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,155,0,0,0,54,3,0, - 0,115,2,0,0,0,0,3,122,23,70,105,108,101,76,111, - 97,100,101,114,46,103,101,116,95,102,105,108,101,110,97,109, - 101,99,2,0,0,0,0,0,0,0,3,0,0,0,9,0, - 0,0,67,0,0,0,115,32,0,0,0,116,0,106,1,124, - 1,100,1,131,2,143,10,125,2,124,2,106,2,131,0,83, - 0,81,0,82,0,88,0,100,2,83,0,41,3,122,39,82, - 101,116,117,114,110,32,116,104,101,32,100,97,116,97,32,102, - 114,111,109,32,112,97,116,104,32,97,115,32,114,97,119,32, - 98,121,116,101,115,46,218,1,114,78,41,3,114,52,0,0, - 0,114,53,0,0,0,90,4,114,101,97,100,41,3,114,104, - 0,0,0,114,37,0,0,0,114,57,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,197,0,0, - 0,59,3,0,0,115,4,0,0,0,0,2,14,1,122,19, - 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,100, - 97,116,97,41,11,114,109,0,0,0,114,108,0,0,0,114, - 110,0,0,0,114,111,0,0,0,114,182,0,0,0,114,210, - 0,0,0,114,212,0,0,0,114,120,0,0,0,114,190,0, - 0,0,114,155,0,0,0,114,197,0,0,0,114,4,0,0, - 0,114,4,0,0,0,41,1,114,208,0,0,0,114,6,0, - 0,0,114,207,0,0,0,24,3,0,0,115,14,0,0,0, - 8,3,4,2,8,6,8,4,8,3,16,12,12,5,114,207, - 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, - 3,0,0,0,64,0,0,0,115,46,0,0,0,101,0,90, - 1,100,0,90,2,100,1,90,3,100,2,100,3,132,0,90, - 4,100,4,100,5,132,0,90,5,100,6,100,7,156,1,100, - 8,100,9,132,2,90,6,100,10,83,0,41,11,218,16,83, - 111,117,114,99,101,70,105,108,101,76,111,97,100,101,114,122, - 62,67,111,110,99,114,101,116,101,32,105,109,112,108,101,109, - 101,110,116,97,116,105,111,110,32,111,102,32,83,111,117,114, - 99,101,76,111,97,100,101,114,32,117,115,105,110,103,32,116, - 104,101,32,102,105,108,101,32,115,121,115,116,101,109,46,99, - 2,0,0,0,0,0,0,0,3,0,0,0,3,0,0,0, - 67,0,0,0,115,22,0,0,0,116,0,124,1,131,1,125, - 2,124,2,106,1,124,2,106,2,100,1,156,2,83,0,41, - 2,122,33,82,101,116,117,114,110,32,116,104,101,32,109,101, - 116,97,100,97,116,97,32,102,111,114,32,116,104,101,32,112, - 97,116,104,46,41,2,122,5,109,116,105,109,101,122,4,115, - 105,122,101,41,3,114,41,0,0,0,218,8,115,116,95,109, - 116,105,109,101,90,7,115,116,95,115,105,122,101,41,3,114, - 104,0,0,0,114,37,0,0,0,114,205,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,194,0, - 0,0,69,3,0,0,115,4,0,0,0,0,2,8,1,122, - 27,83,111,117,114,99,101,70,105,108,101,76,111,97,100,101, - 114,46,112,97,116,104,95,115,116,97,116,115,99,4,0,0, - 0,0,0,0,0,5,0,0,0,5,0,0,0,67,0,0, - 0,115,24,0,0,0,116,0,124,1,131,1,125,4,124,0, - 106,1,124,2,124,3,124,4,100,1,141,3,83,0,41,2, - 78,41,1,218,5,95,109,111,100,101,41,2,114,101,0,0, - 0,114,195,0,0,0,41,5,114,104,0,0,0,114,94,0, - 0,0,114,93,0,0,0,114,56,0,0,0,114,44,0,0, + 0,106,2,131,1,65,0,83,0,41,1,78,41,3,114,211, + 0,0,0,114,102,0,0,0,114,37,0,0,0,41,1,114, + 104,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,114,212,0,0,0,146,3,0,0,115,2,0,0, + 0,0,1,122,28,69,120,116,101,110,115,105,111,110,70,105, + 108,101,76,111,97,100,101,114,46,95,95,104,97,115,104,95, + 95,99,2,0,0,0,0,0,0,0,3,0,0,0,4,0, + 0,0,67,0,0,0,115,36,0,0,0,116,0,106,1,116, + 2,106,3,124,1,131,2,125,2,116,0,106,4,100,1,124, + 1,106,5,124,0,106,6,131,3,1,0,124,2,83,0,41, + 2,122,38,67,114,101,97,116,101,32,97,110,32,117,110,105, + 116,105,97,108,105,122,101,100,32,101,120,116,101,110,115,105, + 111,110,32,109,111,100,117,108,101,122,38,101,120,116,101,110, + 115,105,111,110,32,109,111,100,117,108,101,32,123,33,114,125, + 32,108,111,97,100,101,100,32,102,114,111,109,32,123,33,114, + 125,41,7,114,118,0,0,0,114,185,0,0,0,114,143,0, + 0,0,90,14,99,114,101,97,116,101,95,100,121,110,97,109, + 105,99,114,133,0,0,0,114,102,0,0,0,114,37,0,0, + 0,41,3,114,104,0,0,0,114,162,0,0,0,114,187,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,183,0,0,0,149,3,0,0,115,10,0,0,0,0, + 2,4,1,10,1,6,1,12,1,122,33,69,120,116,101,110, + 115,105,111,110,70,105,108,101,76,111,97,100,101,114,46,99, + 114,101,97,116,101,95,109,111,100,117,108,101,99,2,0,0, + 0,0,0,0,0,2,0,0,0,4,0,0,0,67,0,0, + 0,115,36,0,0,0,116,0,106,1,116,2,106,3,124,1, + 131,2,1,0,116,0,106,4,100,1,124,0,106,5,124,0, + 106,6,131,3,1,0,100,2,83,0,41,3,122,30,73,110, + 105,116,105,97,108,105,122,101,32,97,110,32,101,120,116,101, + 110,115,105,111,110,32,109,111,100,117,108,101,122,40,101,120, + 116,101,110,115,105,111,110,32,109,111,100,117,108,101,32,123, + 33,114,125,32,101,120,101,99,117,116,101,100,32,102,114,111, + 109,32,123,33,114,125,78,41,7,114,118,0,0,0,114,185, + 0,0,0,114,143,0,0,0,90,12,101,120,101,99,95,100, + 121,110,97,109,105,99,114,133,0,0,0,114,102,0,0,0, + 114,37,0,0,0,41,2,114,104,0,0,0,114,187,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 114,196,0,0,0,74,3,0,0,115,4,0,0,0,0,2, - 8,1,122,32,83,111,117,114,99,101,70,105,108,101,76,111, - 97,100,101,114,46,95,99,97,99,104,101,95,98,121,116,101, - 99,111,100,101,105,182,1,0,0,41,1,114,217,0,0,0, - 99,3,0,0,0,1,0,0,0,9,0,0,0,17,0,0, - 0,67,0,0,0,115,250,0,0,0,116,0,124,1,131,1, - 92,2,125,4,125,5,103,0,125,6,120,40,124,4,114,56, - 116,1,124,4,131,1,12,0,114,56,116,0,124,4,131,1, - 92,2,125,4,125,7,124,6,106,2,124,7,131,1,1,0, - 113,18,87,0,120,108,116,3,124,6,131,1,68,0,93,96, - 125,7,116,4,124,4,124,7,131,2,125,4,121,14,116,5, - 106,6,124,4,131,1,1,0,87,0,113,68,4,0,116,7, - 107,10,114,118,1,0,1,0,1,0,119,68,89,0,113,68, - 4,0,116,8,107,10,114,162,1,0,125,8,1,0,122,18, - 116,9,106,10,100,1,124,4,124,8,131,3,1,0,100,2, - 83,0,100,2,125,8,126,8,88,0,113,68,88,0,113,68, - 87,0,121,28,116,11,124,1,124,2,124,3,131,3,1,0, - 116,9,106,10,100,3,124,1,131,2,1,0,87,0,110,48, - 4,0,116,8,107,10,114,244,1,0,125,8,1,0,122,20, - 116,9,106,10,100,1,124,1,124,8,131,3,1,0,87,0, - 89,0,100,2,100,2,125,8,126,8,88,0,110,2,88,0, - 100,2,83,0,41,4,122,27,87,114,105,116,101,32,98,121, - 116,101,115,32,100,97,116,97,32,116,111,32,97,32,102,105, - 108,101,46,122,27,99,111,117,108,100,32,110,111,116,32,99, - 114,101,97,116,101,32,123,33,114,125,58,32,123,33,114,125, - 78,122,12,99,114,101,97,116,101,100,32,123,33,114,125,41, - 12,114,40,0,0,0,114,48,0,0,0,114,161,0,0,0, - 114,35,0,0,0,114,30,0,0,0,114,3,0,0,0,90, - 5,109,107,100,105,114,218,15,70,105,108,101,69,120,105,115, - 116,115,69,114,114,111,114,114,42,0,0,0,114,118,0,0, - 0,114,133,0,0,0,114,58,0,0,0,41,9,114,104,0, - 0,0,114,37,0,0,0,114,56,0,0,0,114,217,0,0, - 0,218,6,112,97,114,101,110,116,114,98,0,0,0,114,29, - 0,0,0,114,25,0,0,0,114,198,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,195,0,0, - 0,79,3,0,0,115,42,0,0,0,0,2,12,1,4,2, - 16,1,12,1,14,2,14,1,10,1,2,1,14,1,14,2, - 6,1,16,3,6,1,8,1,20,1,2,1,12,1,16,1, - 16,2,8,1,122,25,83,111,117,114,99,101,70,105,108,101, - 76,111,97,100,101,114,46,115,101,116,95,100,97,116,97,78, - 41,7,114,109,0,0,0,114,108,0,0,0,114,110,0,0, - 0,114,111,0,0,0,114,194,0,0,0,114,196,0,0,0, - 114,195,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,215,0,0,0,65,3, - 0,0,115,8,0,0,0,8,2,4,2,8,5,8,5,114, - 215,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, - 0,2,0,0,0,64,0,0,0,115,32,0,0,0,101,0, - 90,1,100,0,90,2,100,1,90,3,100,2,100,3,132,0, - 90,4,100,4,100,5,132,0,90,5,100,6,83,0,41,7, - 218,20,83,111,117,114,99,101,108,101,115,115,70,105,108,101, - 76,111,97,100,101,114,122,45,76,111,97,100,101,114,32,119, - 104,105,99,104,32,104,97,110,100,108,101,115,32,115,111,117, - 114,99,101,108,101,115,115,32,102,105,108,101,32,105,109,112, - 111,114,116,115,46,99,2,0,0,0,0,0,0,0,5,0, - 0,0,5,0,0,0,67,0,0,0,115,48,0,0,0,124, - 0,106,0,124,1,131,1,125,2,124,0,106,1,124,2,131, - 1,125,3,116,2,124,3,124,1,124,2,100,1,141,3,125, - 4,116,3,124,4,124,1,124,2,100,2,141,3,83,0,41, - 3,78,41,2,114,102,0,0,0,114,37,0,0,0,41,2, - 114,102,0,0,0,114,93,0,0,0,41,4,114,155,0,0, - 0,114,197,0,0,0,114,139,0,0,0,114,145,0,0,0, - 41,5,114,104,0,0,0,114,123,0,0,0,114,37,0,0, - 0,114,56,0,0,0,114,206,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,184,0,0,0,114, - 3,0,0,115,8,0,0,0,0,1,10,1,10,1,14,1, - 122,29,83,111,117,114,99,101,108,101,115,115,70,105,108,101, - 76,111,97,100,101,114,46,103,101,116,95,99,111,100,101,99, - 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, - 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,122, - 39,82,101,116,117,114,110,32,78,111,110,101,32,97,115,32, - 116,104,101,114,101,32,105,115,32,110,111,32,115,111,117,114, - 99,101,32,99,111,100,101,46,78,114,4,0,0,0,41,2, - 114,104,0,0,0,114,123,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,199,0,0,0,120,3, - 0,0,115,2,0,0,0,0,2,122,31,83,111,117,114,99, - 101,108,101,115,115,70,105,108,101,76,111,97,100,101,114,46, - 103,101,116,95,115,111,117,114,99,101,78,41,6,114,109,0, - 0,0,114,108,0,0,0,114,110,0,0,0,114,111,0,0, - 0,114,184,0,0,0,114,199,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, - 220,0,0,0,110,3,0,0,115,6,0,0,0,8,2,4, - 2,8,6,114,220,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,3,0,0,0,64,0,0,0,115,92,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, - 100,7,132,0,90,6,100,8,100,9,132,0,90,7,100,10, - 100,11,132,0,90,8,100,12,100,13,132,0,90,9,100,14, - 100,15,132,0,90,10,100,16,100,17,132,0,90,11,101,12, - 100,18,100,19,132,0,131,1,90,13,100,20,83,0,41,21, - 218,19,69,120,116,101,110,115,105,111,110,70,105,108,101,76, - 111,97,100,101,114,122,93,76,111,97,100,101,114,32,102,111, - 114,32,101,120,116,101,110,115,105,111,110,32,109,111,100,117, - 108,101,115,46,10,10,32,32,32,32,84,104,101,32,99,111, - 110,115,116,114,117,99,116,111,114,32,105,115,32,100,101,115, - 105,103,110,101,100,32,116,111,32,119,111,114,107,32,119,105, - 116,104,32,70,105,108,101,70,105,110,100,101,114,46,10,10, - 32,32,32,32,99,3,0,0,0,0,0,0,0,3,0,0, - 0,2,0,0,0,67,0,0,0,115,16,0,0,0,124,1, - 124,0,95,0,124,2,124,0,95,1,100,0,83,0,41,1, - 78,41,2,114,102,0,0,0,114,37,0,0,0,41,3,114, - 104,0,0,0,114,102,0,0,0,114,37,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,182,0, - 0,0,137,3,0,0,115,4,0,0,0,0,1,6,1,122, - 28,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, - 97,100,101,114,46,95,95,105,110,105,116,95,95,99,2,0, - 0,0,0,0,0,0,2,0,0,0,2,0,0,0,67,0, - 0,0,115,24,0,0,0,124,0,106,0,124,1,106,0,107, - 2,111,22,124,0,106,1,124,1,106,1,107,2,83,0,41, - 1,78,41,2,114,208,0,0,0,114,115,0,0,0,41,2, - 114,104,0,0,0,114,209,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,210,0,0,0,141,3, - 0,0,115,4,0,0,0,0,1,12,1,122,26,69,120,116, + 114,188,0,0,0,157,3,0,0,115,6,0,0,0,0,2, + 14,1,6,1,122,31,69,120,116,101,110,115,105,111,110,70, + 105,108,101,76,111,97,100,101,114,46,101,120,101,99,95,109, + 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, + 0,0,4,0,0,0,3,0,0,0,115,36,0,0,0,116, + 0,124,0,106,1,131,1,100,1,25,0,137,0,116,2,135, + 0,102,1,100,2,100,3,132,8,116,3,68,0,131,1,131, + 1,83,0,41,4,122,49,82,101,116,117,114,110,32,84,114, + 117,101,32,105,102,32,116,104,101,32,101,120,116,101,110,115, + 105,111,110,32,109,111,100,117,108,101,32,105,115,32,97,32, + 112,97,99,107,97,103,101,46,114,31,0,0,0,99,1,0, + 0,0,0,0,0,0,2,0,0,0,4,0,0,0,51,0, + 0,0,115,26,0,0,0,124,0,93,18,125,1,136,0,100, + 0,124,1,23,0,107,2,86,0,1,0,113,2,100,1,83, + 0,41,2,114,182,0,0,0,78,114,4,0,0,0,41,2, + 114,24,0,0,0,218,6,115,117,102,102,105,120,41,1,218, + 9,102,105,108,101,95,110,97,109,101,114,4,0,0,0,114, + 6,0,0,0,250,9,60,103,101,110,101,120,112,114,62,166, + 3,0,0,115,2,0,0,0,4,1,122,49,69,120,116,101, + 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, + 105,115,95,112,97,99,107,97,103,101,46,60,108,111,99,97, + 108,115,62,46,60,103,101,110,101,120,112,114,62,41,4,114, + 40,0,0,0,114,37,0,0,0,218,3,97,110,121,218,18, + 69,88,84,69,78,83,73,79,78,95,83,85,70,70,73,88, + 69,83,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,41,1,114,223,0,0,0,114,6,0,0,0,114, + 157,0,0,0,163,3,0,0,115,6,0,0,0,0,2,14, + 1,12,1,122,30,69,120,116,101,110,115,105,111,110,70,105, + 108,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, + 97,103,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, + 0,41,2,122,63,82,101,116,117,114,110,32,78,111,110,101, + 32,97,115,32,97,110,32,101,120,116,101,110,115,105,111,110, + 32,109,111,100,117,108,101,32,99,97,110,110,111,116,32,99, + 114,101,97,116,101,32,97,32,99,111,100,101,32,111,98,106, + 101,99,116,46,78,114,4,0,0,0,41,2,114,104,0,0, + 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,184,0,0,0,169,3,0,0,115,2, + 0,0,0,0,2,122,28,69,120,116,101,110,115,105,111,110, + 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, + 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, + 0,41,2,122,53,82,101,116,117,114,110,32,78,111,110,101, + 32,97,115,32,101,120,116,101,110,115,105,111,110,32,109,111, + 100,117,108,101,115,32,104,97,118,101,32,110,111,32,115,111, + 117,114,99,101,32,99,111,100,101,46,78,114,4,0,0,0, + 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,199,0,0,0, + 173,3,0,0,115,2,0,0,0,0,2,122,30,69,120,116, 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, - 46,95,95,101,113,95,95,99,1,0,0,0,0,0,0,0, - 1,0,0,0,3,0,0,0,67,0,0,0,115,20,0,0, - 0,116,0,124,0,106,1,131,1,116,0,124,0,106,2,131, - 1,65,0,83,0,41,1,78,41,3,114,211,0,0,0,114, - 102,0,0,0,114,37,0,0,0,41,1,114,104,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, - 212,0,0,0,145,3,0,0,115,2,0,0,0,0,1,122, - 28,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, - 97,100,101,114,46,95,95,104,97,115,104,95,95,99,2,0, - 0,0,0,0,0,0,3,0,0,0,4,0,0,0,67,0, - 0,0,115,36,0,0,0,116,0,106,1,116,2,106,3,124, - 1,131,2,125,2,116,0,106,4,100,1,124,1,106,5,124, - 0,106,6,131,3,1,0,124,2,83,0,41,2,122,38,67, - 114,101,97,116,101,32,97,110,32,117,110,105,116,105,97,108, - 105,122,101,100,32,101,120,116,101,110,115,105,111,110,32,109, - 111,100,117,108,101,122,38,101,120,116,101,110,115,105,111,110, - 32,109,111,100,117,108,101,32,123,33,114,125,32,108,111,97, - 100,101,100,32,102,114,111,109,32,123,33,114,125,41,7,114, - 118,0,0,0,114,185,0,0,0,114,143,0,0,0,90,14, - 99,114,101,97,116,101,95,100,121,110,97,109,105,99,114,133, - 0,0,0,114,102,0,0,0,114,37,0,0,0,41,3,114, - 104,0,0,0,114,162,0,0,0,114,187,0,0,0,114,4, + 46,103,101,116,95,115,111,117,114,99,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, + 115,6,0,0,0,124,0,106,0,83,0,41,1,122,58,82, + 101,116,117,114,110,32,116,104,101,32,112,97,116,104,32,116, + 111,32,116,104,101,32,115,111,117,114,99,101,32,102,105,108, + 101,32,97,115,32,102,111,117,110,100,32,98,121,32,116,104, + 101,32,102,105,110,100,101,114,46,41,1,114,37,0,0,0, + 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,155,0,0,0, + 177,3,0,0,115,2,0,0,0,0,3,122,32,69,120,116, + 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, + 46,103,101,116,95,102,105,108,101,110,97,109,101,78,41,14, + 114,109,0,0,0,114,108,0,0,0,114,110,0,0,0,114, + 111,0,0,0,114,182,0,0,0,114,210,0,0,0,114,212, + 0,0,0,114,183,0,0,0,114,188,0,0,0,114,157,0, + 0,0,114,184,0,0,0,114,199,0,0,0,114,120,0,0, + 0,114,155,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,114,221,0,0,0,130, + 3,0,0,115,20,0,0,0,8,6,4,2,8,4,8,4, + 8,3,8,8,8,6,8,6,8,4,8,4,114,221,0,0, + 0,99,0,0,0,0,0,0,0,0,0,0,0,0,2,0, + 0,0,64,0,0,0,115,96,0,0,0,101,0,90,1,100, + 0,90,2,100,1,90,3,100,2,100,3,132,0,90,4,100, + 4,100,5,132,0,90,5,100,6,100,7,132,0,90,6,100, + 8,100,9,132,0,90,7,100,10,100,11,132,0,90,8,100, + 12,100,13,132,0,90,9,100,14,100,15,132,0,90,10,100, + 16,100,17,132,0,90,11,100,18,100,19,132,0,90,12,100, + 20,100,21,132,0,90,13,100,22,83,0,41,23,218,14,95, + 78,97,109,101,115,112,97,99,101,80,97,116,104,97,38,1, + 0,0,82,101,112,114,101,115,101,110,116,115,32,97,32,110, + 97,109,101,115,112,97,99,101,32,112,97,99,107,97,103,101, + 39,115,32,112,97,116,104,46,32,32,73,116,32,117,115,101, + 115,32,116,104,101,32,109,111,100,117,108,101,32,110,97,109, + 101,10,32,32,32,32,116,111,32,102,105,110,100,32,105,116, + 115,32,112,97,114,101,110,116,32,109,111,100,117,108,101,44, + 32,97,110,100,32,102,114,111,109,32,116,104,101,114,101,32, + 105,116,32,108,111,111,107,115,32,117,112,32,116,104,101,32, + 112,97,114,101,110,116,39,115,10,32,32,32,32,95,95,112, + 97,116,104,95,95,46,32,32,87,104,101,110,32,116,104,105, + 115,32,99,104,97,110,103,101,115,44,32,116,104,101,32,109, + 111,100,117,108,101,39,115,32,111,119,110,32,112,97,116,104, + 32,105,115,32,114,101,99,111,109,112,117,116,101,100,44,10, + 32,32,32,32,117,115,105,110,103,32,112,97,116,104,95,102, + 105,110,100,101,114,46,32,32,70,111,114,32,116,111,112,45, + 108,101,118,101,108,32,109,111,100,117,108,101,115,44,32,116, + 104,101,32,112,97,114,101,110,116,32,109,111,100,117,108,101, + 39,115,32,112,97,116,104,10,32,32,32,32,105,115,32,115, + 121,115,46,112,97,116,104,46,99,4,0,0,0,0,0,0, + 0,4,0,0,0,2,0,0,0,67,0,0,0,115,36,0, + 0,0,124,1,124,0,95,0,124,2,124,0,95,1,116,2, + 124,0,106,3,131,0,131,1,124,0,95,4,124,3,124,0, + 95,5,100,0,83,0,41,1,78,41,6,218,5,95,110,97, + 109,101,218,5,95,112,97,116,104,114,97,0,0,0,218,16, + 95,103,101,116,95,112,97,114,101,110,116,95,112,97,116,104, + 218,17,95,108,97,115,116,95,112,97,114,101,110,116,95,112, + 97,116,104,218,12,95,112,97,116,104,95,102,105,110,100,101, + 114,41,4,114,104,0,0,0,114,102,0,0,0,114,37,0, + 0,0,218,11,112,97,116,104,95,102,105,110,100,101,114,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,182, + 0,0,0,190,3,0,0,115,8,0,0,0,0,1,6,1, + 6,1,14,1,122,23,95,78,97,109,101,115,112,97,99,101, + 80,97,116,104,46,95,95,105,110,105,116,95,95,99,1,0, + 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, + 0,0,115,38,0,0,0,124,0,106,0,106,1,100,1,131, + 1,92,3,125,1,125,2,125,3,124,2,100,2,107,2,114, + 30,100,6,83,0,124,1,100,5,102,2,83,0,41,7,122, + 62,82,101,116,117,114,110,115,32,97,32,116,117,112,108,101, + 32,111,102,32,40,112,97,114,101,110,116,45,109,111,100,117, + 108,101,45,110,97,109,101,44,32,112,97,114,101,110,116,45, + 112,97,116,104,45,97,116,116,114,45,110,97,109,101,41,114, + 61,0,0,0,114,32,0,0,0,114,8,0,0,0,114,37, + 0,0,0,90,8,95,95,112,97,116,104,95,95,41,2,122, + 3,115,121,115,122,4,112,97,116,104,41,2,114,228,0,0, + 0,114,34,0,0,0,41,4,114,104,0,0,0,114,219,0, + 0,0,218,3,100,111,116,90,2,109,101,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,23,95,102,105,110, + 100,95,112,97,114,101,110,116,95,112,97,116,104,95,110,97, + 109,101,115,196,3,0,0,115,8,0,0,0,0,2,18,1, + 8,2,4,3,122,38,95,78,97,109,101,115,112,97,99,101, + 80,97,116,104,46,95,102,105,110,100,95,112,97,114,101,110, + 116,95,112,97,116,104,95,110,97,109,101,115,99,1,0,0, + 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, + 0,115,28,0,0,0,124,0,106,0,131,0,92,2,125,1, + 125,2,116,1,116,2,106,3,124,1,25,0,124,2,131,2, + 83,0,41,1,78,41,4,114,235,0,0,0,114,114,0,0, + 0,114,8,0,0,0,218,7,109,111,100,117,108,101,115,41, + 3,114,104,0,0,0,90,18,112,97,114,101,110,116,95,109, + 111,100,117,108,101,95,110,97,109,101,90,14,112,97,116,104, + 95,97,116,116,114,95,110,97,109,101,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,230,0,0,0,206,3, + 0,0,115,4,0,0,0,0,1,12,1,122,31,95,78,97, + 109,101,115,112,97,99,101,80,97,116,104,46,95,103,101,116, + 95,112,97,114,101,110,116,95,112,97,116,104,99,1,0,0, + 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, + 0,115,80,0,0,0,116,0,124,0,106,1,131,0,131,1, + 125,1,124,1,124,0,106,2,107,3,114,74,124,0,106,3, + 124,0,106,4,124,1,131,2,125,2,124,2,100,0,107,9, + 114,68,124,2,106,5,100,0,107,8,114,68,124,2,106,6, + 114,68,124,2,106,6,124,0,95,7,124,1,124,0,95,2, + 124,0,106,7,83,0,41,1,78,41,8,114,97,0,0,0, + 114,230,0,0,0,114,231,0,0,0,114,232,0,0,0,114, + 228,0,0,0,114,124,0,0,0,114,154,0,0,0,114,229, + 0,0,0,41,3,114,104,0,0,0,90,11,112,97,114,101, + 110,116,95,112,97,116,104,114,162,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,12,95,114,101, + 99,97,108,99,117,108,97,116,101,210,3,0,0,115,16,0, + 0,0,0,2,12,1,10,1,14,3,18,1,6,1,8,1, + 6,1,122,27,95,78,97,109,101,115,112,97,99,101,80,97, + 116,104,46,95,114,101,99,97,108,99,117,108,97,116,101,99, + 1,0,0,0,0,0,0,0,1,0,0,0,2,0,0,0, + 67,0,0,0,115,12,0,0,0,116,0,124,0,106,1,131, + 0,131,1,83,0,41,1,78,41,2,218,4,105,116,101,114, + 114,237,0,0,0,41,1,114,104,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,8,95,95,105, + 116,101,114,95,95,223,3,0,0,115,2,0,0,0,0,1, + 122,23,95,78,97,109,101,115,112,97,99,101,80,97,116,104, + 46,95,95,105,116,101,114,95,95,99,3,0,0,0,0,0, + 0,0,3,0,0,0,3,0,0,0,67,0,0,0,115,14, + 0,0,0,124,2,124,0,106,0,124,1,60,0,100,0,83, + 0,41,1,78,41,1,114,229,0,0,0,41,3,114,104,0, + 0,0,218,5,105,110,100,101,120,114,37,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,11,95, + 95,115,101,116,105,116,101,109,95,95,226,3,0,0,115,2, + 0,0,0,0,1,122,26,95,78,97,109,101,115,112,97,99, + 101,80,97,116,104,46,95,95,115,101,116,105,116,101,109,95, + 95,99,1,0,0,0,0,0,0,0,1,0,0,0,2,0, + 0,0,67,0,0,0,115,12,0,0,0,116,0,124,0,106, + 1,131,0,131,1,83,0,41,1,78,41,2,114,33,0,0, + 0,114,237,0,0,0,41,1,114,104,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,218,7,95,95, + 108,101,110,95,95,229,3,0,0,115,2,0,0,0,0,1, + 122,22,95,78,97,109,101,115,112,97,99,101,80,97,116,104, + 46,95,95,108,101,110,95,95,99,1,0,0,0,0,0,0, + 0,1,0,0,0,2,0,0,0,67,0,0,0,115,12,0, + 0,0,100,1,106,0,124,0,106,1,131,1,83,0,41,2, + 78,122,20,95,78,97,109,101,115,112,97,99,101,80,97,116, + 104,40,123,33,114,125,41,41,2,114,50,0,0,0,114,229, + 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,218,8,95,95,114,101,112, + 114,95,95,232,3,0,0,115,2,0,0,0,0,1,122,23, + 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, + 95,114,101,112,114,95,95,99,2,0,0,0,0,0,0,0, + 2,0,0,0,2,0,0,0,67,0,0,0,115,12,0,0, + 0,124,1,124,0,106,0,131,0,107,6,83,0,41,1,78, + 41,1,114,237,0,0,0,41,2,114,104,0,0,0,218,4, + 105,116,101,109,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,218,12,95,95,99,111,110,116,97,105,110,115,95, + 95,235,3,0,0,115,2,0,0,0,0,1,122,27,95,78, + 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,99, + 111,110,116,97,105,110,115,95,95,99,2,0,0,0,0,0, + 0,0,2,0,0,0,2,0,0,0,67,0,0,0,115,16, + 0,0,0,124,0,106,0,106,1,124,1,131,1,1,0,100, + 0,83,0,41,1,78,41,2,114,229,0,0,0,114,161,0, + 0,0,41,2,114,104,0,0,0,114,244,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,161,0, + 0,0,238,3,0,0,115,2,0,0,0,0,1,122,21,95, + 78,97,109,101,115,112,97,99,101,80,97,116,104,46,97,112, + 112,101,110,100,78,41,14,114,109,0,0,0,114,108,0,0, + 0,114,110,0,0,0,114,111,0,0,0,114,182,0,0,0, + 114,235,0,0,0,114,230,0,0,0,114,237,0,0,0,114, + 239,0,0,0,114,241,0,0,0,114,242,0,0,0,114,243, + 0,0,0,114,245,0,0,0,114,161,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,227,0,0,0,183,3,0,0,115,22,0,0,0,8, + 5,4,2,8,6,8,10,8,4,8,13,8,3,8,3,8, + 3,8,3,8,3,114,227,0,0,0,99,0,0,0,0,0, + 0,0,0,0,0,0,0,3,0,0,0,64,0,0,0,115, + 80,0,0,0,101,0,90,1,100,0,90,2,100,1,100,2, + 132,0,90,3,101,4,100,3,100,4,132,0,131,1,90,5, + 100,5,100,6,132,0,90,6,100,7,100,8,132,0,90,7, + 100,9,100,10,132,0,90,8,100,11,100,12,132,0,90,9, + 100,13,100,14,132,0,90,10,100,15,100,16,132,0,90,11, + 100,17,83,0,41,18,218,16,95,78,97,109,101,115,112,97, + 99,101,76,111,97,100,101,114,99,4,0,0,0,0,0,0, + 0,4,0,0,0,4,0,0,0,67,0,0,0,115,18,0, + 0,0,116,0,124,1,124,2,124,3,131,3,124,0,95,1, + 100,0,83,0,41,1,78,41,2,114,227,0,0,0,114,229, + 0,0,0,41,4,114,104,0,0,0,114,102,0,0,0,114, + 37,0,0,0,114,233,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,182,0,0,0,244,3,0, + 0,115,2,0,0,0,0,1,122,25,95,78,97,109,101,115, + 112,97,99,101,76,111,97,100,101,114,46,95,95,105,110,105, + 116,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, + 2,0,0,0,67,0,0,0,115,12,0,0,0,100,1,106, + 0,124,1,106,1,131,1,83,0,41,2,122,115,82,101,116, + 117,114,110,32,114,101,112,114,32,102,111,114,32,116,104,101, + 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, + 32,32,84,104,101,32,109,101,116,104,111,100,32,105,115,32, + 100,101,112,114,101,99,97,116,101,100,46,32,32,84,104,101, + 32,105,109,112,111,114,116,32,109,97,99,104,105,110,101,114, + 121,32,100,111,101,115,32,116,104,101,32,106,111,98,32,105, + 116,115,101,108,102,46,10,10,32,32,32,32,32,32,32,32, + 122,25,60,109,111,100,117,108,101,32,123,33,114,125,32,40, + 110,97,109,101,115,112,97,99,101,41,62,41,2,114,50,0, + 0,0,114,109,0,0,0,41,2,114,168,0,0,0,114,187, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,11,109,111,100,117,108,101,95,114,101,112,114,247, + 3,0,0,115,2,0,0,0,0,7,122,28,95,78,97,109, + 101,115,112,97,99,101,76,111,97,100,101,114,46,109,111,100, + 117,108,101,95,114,101,112,114,99,2,0,0,0,0,0,0, + 0,2,0,0,0,1,0,0,0,67,0,0,0,115,4,0, + 0,0,100,1,83,0,41,2,78,84,114,4,0,0,0,41, + 2,114,104,0,0,0,114,123,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,114,157,0,0,0,0, + 4,0,0,115,2,0,0,0,0,1,122,27,95,78,97,109, + 101,115,112,97,99,101,76,111,97,100,101,114,46,105,115,95, + 112,97,99,107,97,103,101,99,2,0,0,0,0,0,0,0, + 2,0,0,0,1,0,0,0,67,0,0,0,115,4,0,0, + 0,100,1,83,0,41,2,78,114,32,0,0,0,114,4,0, + 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, + 0,0,3,4,0,0,115,2,0,0,0,0,1,122,27,95, + 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, + 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, + 0,0,0,2,0,0,0,6,0,0,0,67,0,0,0,115, + 16,0,0,0,116,0,100,1,100,2,100,3,100,4,100,5, + 141,4,83,0,41,6,78,114,32,0,0,0,122,8,60,115, + 116,114,105,110,103,62,114,186,0,0,0,84,41,1,114,201, + 0,0,0,41,1,114,202,0,0,0,41,2,114,104,0,0, + 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,184,0,0,0,6,4,0,0,115,2, + 0,0,0,0,1,122,25,95,78,97,109,101,115,112,97,99, + 101,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, + 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, + 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, + 122,42,85,115,101,32,100,101,102,97,117,108,116,32,115,101, + 109,97,110,116,105,99,115,32,102,111,114,32,109,111,100,117, + 108,101,32,99,114,101,97,116,105,111,110,46,78,114,4,0, + 0,0,41,2,114,104,0,0,0,114,162,0,0,0,114,4, 0,0,0,114,4,0,0,0,114,6,0,0,0,114,183,0, - 0,0,148,3,0,0,115,10,0,0,0,0,2,4,1,10, - 1,6,1,12,1,122,33,69,120,116,101,110,115,105,111,110, - 70,105,108,101,76,111,97,100,101,114,46,99,114,101,97,116, - 101,95,109,111,100,117,108,101,99,2,0,0,0,0,0,0, - 0,2,0,0,0,4,0,0,0,67,0,0,0,115,36,0, - 0,0,116,0,106,1,116,2,106,3,124,1,131,2,1,0, - 116,0,106,4,100,1,124,0,106,5,124,0,106,6,131,3, - 1,0,100,2,83,0,41,3,122,30,73,110,105,116,105,97, - 108,105,122,101,32,97,110,32,101,120,116,101,110,115,105,111, - 110,32,109,111,100,117,108,101,122,40,101,120,116,101,110,115, - 105,111,110,32,109,111,100,117,108,101,32,123,33,114,125,32, - 101,120,101,99,117,116,101,100,32,102,114,111,109,32,123,33, - 114,125,78,41,7,114,118,0,0,0,114,185,0,0,0,114, - 143,0,0,0,90,12,101,120,101,99,95,100,121,110,97,109, - 105,99,114,133,0,0,0,114,102,0,0,0,114,37,0,0, + 0,0,9,4,0,0,115,0,0,0,0,122,30,95,78,97, + 109,101,115,112,97,99,101,76,111,97,100,101,114,46,99,114, + 101,97,116,101,95,109,111,100,117,108,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, + 115,4,0,0,0,100,0,83,0,41,1,78,114,4,0,0, 0,41,2,114,104,0,0,0,114,187,0,0,0,114,4,0, 0,0,114,4,0,0,0,114,6,0,0,0,114,188,0,0, - 0,156,3,0,0,115,6,0,0,0,0,2,14,1,6,1, - 122,31,69,120,116,101,110,115,105,111,110,70,105,108,101,76, - 111,97,100,101,114,46,101,120,101,99,95,109,111,100,117,108, - 101,99,2,0,0,0,0,0,0,0,2,0,0,0,4,0, - 0,0,3,0,0,0,115,36,0,0,0,116,0,124,0,106, - 1,131,1,100,1,25,0,137,0,116,2,135,0,102,1,100, - 2,100,3,132,8,116,3,68,0,131,1,131,1,83,0,41, - 4,122,49,82,101,116,117,114,110,32,84,114,117,101,32,105, - 102,32,116,104,101,32,101,120,116,101,110,115,105,111,110,32, - 109,111,100,117,108,101,32,105,115,32,97,32,112,97,99,107, - 97,103,101,46,114,31,0,0,0,99,1,0,0,0,0,0, - 0,0,2,0,0,0,4,0,0,0,51,0,0,0,115,26, - 0,0,0,124,0,93,18,125,1,136,0,100,0,124,1,23, - 0,107,2,86,0,1,0,113,2,100,1,83,0,41,2,114, - 182,0,0,0,78,114,4,0,0,0,41,2,114,24,0,0, - 0,218,6,115,117,102,102,105,120,41,1,218,9,102,105,108, - 101,95,110,97,109,101,114,4,0,0,0,114,6,0,0,0, - 250,9,60,103,101,110,101,120,112,114,62,165,3,0,0,115, - 2,0,0,0,4,1,122,49,69,120,116,101,110,115,105,111, - 110,70,105,108,101,76,111,97,100,101,114,46,105,115,95,112, - 97,99,107,97,103,101,46,60,108,111,99,97,108,115,62,46, - 60,103,101,110,101,120,112,114,62,41,4,114,40,0,0,0, - 114,37,0,0,0,218,3,97,110,121,218,18,69,88,84,69, - 78,83,73,79,78,95,83,85,70,70,73,88,69,83,41,2, - 114,104,0,0,0,114,123,0,0,0,114,4,0,0,0,41, - 1,114,223,0,0,0,114,6,0,0,0,114,157,0,0,0, - 162,3,0,0,115,6,0,0,0,0,2,14,1,12,1,122, - 30,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, - 97,100,101,114,46,105,115,95,112,97,99,107,97,103,101,99, - 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, - 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,122, - 63,82,101,116,117,114,110,32,78,111,110,101,32,97,115,32, - 97,110,32,101,120,116,101,110,115,105,111,110,32,109,111,100, - 117,108,101,32,99,97,110,110,111,116,32,99,114,101,97,116, - 101,32,97,32,99,111,100,101,32,111,98,106,101,99,116,46, - 78,114,4,0,0,0,41,2,114,104,0,0,0,114,123,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,184,0,0,0,168,3,0,0,115,2,0,0,0,0, - 2,122,28,69,120,116,101,110,115,105,111,110,70,105,108,101, - 76,111,97,100,101,114,46,103,101,116,95,99,111,100,101,99, - 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, - 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,122, - 53,82,101,116,117,114,110,32,78,111,110,101,32,97,115,32, - 101,120,116,101,110,115,105,111,110,32,109,111,100,117,108,101, - 115,32,104,97,118,101,32,110,111,32,115,111,117,114,99,101, - 32,99,111,100,101,46,78,114,4,0,0,0,41,2,114,104, - 0,0,0,114,123,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,114,199,0,0,0,172,3,0,0, - 115,2,0,0,0,0,2,122,30,69,120,116,101,110,115,105, - 111,110,70,105,108,101,76,111,97,100,101,114,46,103,101,116, - 95,115,111,117,114,99,101,99,2,0,0,0,0,0,0,0, - 2,0,0,0,1,0,0,0,67,0,0,0,115,6,0,0, - 0,124,0,106,0,83,0,41,1,122,58,82,101,116,117,114, - 110,32,116,104,101,32,112,97,116,104,32,116,111,32,116,104, - 101,32,115,111,117,114,99,101,32,102,105,108,101,32,97,115, - 32,102,111,117,110,100,32,98,121,32,116,104,101,32,102,105, - 110,100,101,114,46,41,1,114,37,0,0,0,41,2,114,104, - 0,0,0,114,123,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,114,155,0,0,0,176,3,0,0, - 115,2,0,0,0,0,3,122,32,69,120,116,101,110,115,105, - 111,110,70,105,108,101,76,111,97,100,101,114,46,103,101,116, - 95,102,105,108,101,110,97,109,101,78,41,14,114,109,0,0, - 0,114,108,0,0,0,114,110,0,0,0,114,111,0,0,0, - 114,182,0,0,0,114,210,0,0,0,114,212,0,0,0,114, - 183,0,0,0,114,188,0,0,0,114,157,0,0,0,114,184, - 0,0,0,114,199,0,0,0,114,120,0,0,0,114,155,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,221,0,0,0,129,3,0,0,115, - 20,0,0,0,8,6,4,2,8,4,8,4,8,3,8,8, - 8,6,8,6,8,4,8,4,114,221,0,0,0,99,0,0, - 0,0,0,0,0,0,0,0,0,0,2,0,0,0,64,0, - 0,0,115,96,0,0,0,101,0,90,1,100,0,90,2,100, - 1,90,3,100,2,100,3,132,0,90,4,100,4,100,5,132, - 0,90,5,100,6,100,7,132,0,90,6,100,8,100,9,132, - 0,90,7,100,10,100,11,132,0,90,8,100,12,100,13,132, - 0,90,9,100,14,100,15,132,0,90,10,100,16,100,17,132, - 0,90,11,100,18,100,19,132,0,90,12,100,20,100,21,132, - 0,90,13,100,22,83,0,41,23,218,14,95,78,97,109,101, - 115,112,97,99,101,80,97,116,104,97,38,1,0,0,82,101, - 112,114,101,115,101,110,116,115,32,97,32,110,97,109,101,115, - 112,97,99,101,32,112,97,99,107,97,103,101,39,115,32,112, - 97,116,104,46,32,32,73,116,32,117,115,101,115,32,116,104, - 101,32,109,111,100,117,108,101,32,110,97,109,101,10,32,32, - 32,32,116,111,32,102,105,110,100,32,105,116,115,32,112,97, - 114,101,110,116,32,109,111,100,117,108,101,44,32,97,110,100, - 32,102,114,111,109,32,116,104,101,114,101,32,105,116,32,108, - 111,111,107,115,32,117,112,32,116,104,101,32,112,97,114,101, - 110,116,39,115,10,32,32,32,32,95,95,112,97,116,104,95, - 95,46,32,32,87,104,101,110,32,116,104,105,115,32,99,104, - 97,110,103,101,115,44,32,116,104,101,32,109,111,100,117,108, - 101,39,115,32,111,119,110,32,112,97,116,104,32,105,115,32, - 114,101,99,111,109,112,117,116,101,100,44,10,32,32,32,32, - 117,115,105,110,103,32,112,97,116,104,95,102,105,110,100,101, - 114,46,32,32,70,111,114,32,116,111,112,45,108,101,118,101, - 108,32,109,111,100,117,108,101,115,44,32,116,104,101,32,112, - 97,114,101,110,116,32,109,111,100,117,108,101,39,115,32,112, - 97,116,104,10,32,32,32,32,105,115,32,115,121,115,46,112, - 97,116,104,46,99,4,0,0,0,0,0,0,0,4,0,0, - 0,2,0,0,0,67,0,0,0,115,36,0,0,0,124,1, - 124,0,95,0,124,2,124,0,95,1,116,2,124,0,106,3, - 131,0,131,1,124,0,95,4,124,3,124,0,95,5,100,0, - 83,0,41,1,78,41,6,218,5,95,110,97,109,101,218,5, - 95,112,97,116,104,114,97,0,0,0,218,16,95,103,101,116, - 95,112,97,114,101,110,116,95,112,97,116,104,218,17,95,108, - 97,115,116,95,112,97,114,101,110,116,95,112,97,116,104,218, - 12,95,112,97,116,104,95,102,105,110,100,101,114,41,4,114, - 104,0,0,0,114,102,0,0,0,114,37,0,0,0,218,11, - 112,97,116,104,95,102,105,110,100,101,114,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,182,0,0,0,189, - 3,0,0,115,8,0,0,0,0,1,6,1,6,1,14,1, - 122,23,95,78,97,109,101,115,112,97,99,101,80,97,116,104, - 46,95,95,105,110,105,116,95,95,99,1,0,0,0,0,0, - 0,0,4,0,0,0,3,0,0,0,67,0,0,0,115,38, - 0,0,0,124,0,106,0,106,1,100,1,131,1,92,3,125, - 1,125,2,125,3,124,2,100,2,107,2,114,30,100,6,83, - 0,124,1,100,5,102,2,83,0,41,7,122,62,82,101,116, - 117,114,110,115,32,97,32,116,117,112,108,101,32,111,102,32, - 40,112,97,114,101,110,116,45,109,111,100,117,108,101,45,110, - 97,109,101,44,32,112,97,114,101,110,116,45,112,97,116,104, - 45,97,116,116,114,45,110,97,109,101,41,114,61,0,0,0, - 114,32,0,0,0,114,8,0,0,0,114,37,0,0,0,90, - 8,95,95,112,97,116,104,95,95,41,2,122,3,115,121,115, - 122,4,112,97,116,104,41,2,114,228,0,0,0,114,34,0, - 0,0,41,4,114,104,0,0,0,114,219,0,0,0,218,3, - 100,111,116,90,2,109,101,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,218,23,95,102,105,110,100,95,112,97, - 114,101,110,116,95,112,97,116,104,95,110,97,109,101,115,195, - 3,0,0,115,8,0,0,0,0,2,18,1,8,2,4,3, - 122,38,95,78,97,109,101,115,112,97,99,101,80,97,116,104, - 46,95,102,105,110,100,95,112,97,114,101,110,116,95,112,97, - 116,104,95,110,97,109,101,115,99,1,0,0,0,0,0,0, - 0,3,0,0,0,3,0,0,0,67,0,0,0,115,28,0, - 0,0,124,0,106,0,131,0,92,2,125,1,125,2,116,1, - 116,2,106,3,124,1,25,0,124,2,131,2,83,0,41,1, - 78,41,4,114,235,0,0,0,114,114,0,0,0,114,8,0, - 0,0,218,7,109,111,100,117,108,101,115,41,3,114,104,0, - 0,0,90,18,112,97,114,101,110,116,95,109,111,100,117,108, - 101,95,110,97,109,101,90,14,112,97,116,104,95,97,116,116, - 114,95,110,97,109,101,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,230,0,0,0,205,3,0,0,115,4, - 0,0,0,0,1,12,1,122,31,95,78,97,109,101,115,112, - 97,99,101,80,97,116,104,46,95,103,101,116,95,112,97,114, - 101,110,116,95,112,97,116,104,99,1,0,0,0,0,0,0, - 0,3,0,0,0,3,0,0,0,67,0,0,0,115,80,0, - 0,0,116,0,124,0,106,1,131,0,131,1,125,1,124,1, - 124,0,106,2,107,3,114,74,124,0,106,3,124,0,106,4, - 124,1,131,2,125,2,124,2,100,0,107,9,114,68,124,2, - 106,5,100,0,107,8,114,68,124,2,106,6,114,68,124,2, - 106,6,124,0,95,7,124,1,124,0,95,2,124,0,106,7, - 83,0,41,1,78,41,8,114,97,0,0,0,114,230,0,0, - 0,114,231,0,0,0,114,232,0,0,0,114,228,0,0,0, - 114,124,0,0,0,114,154,0,0,0,114,229,0,0,0,41, - 3,114,104,0,0,0,90,11,112,97,114,101,110,116,95,112, - 97,116,104,114,162,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,218,12,95,114,101,99,97,108,99, - 117,108,97,116,101,209,3,0,0,115,16,0,0,0,0,2, - 12,1,10,1,14,3,18,1,6,1,8,1,6,1,122,27, - 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, - 114,101,99,97,108,99,117,108,97,116,101,99,1,0,0,0, - 0,0,0,0,1,0,0,0,2,0,0,0,67,0,0,0, - 115,12,0,0,0,116,0,124,0,106,1,131,0,131,1,83, - 0,41,1,78,41,2,218,4,105,116,101,114,114,237,0,0, - 0,41,1,114,104,0,0,0,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,218,8,95,95,105,116,101,114,95, - 95,222,3,0,0,115,2,0,0,0,0,1,122,23,95,78, - 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,105, - 116,101,114,95,95,99,3,0,0,0,0,0,0,0,3,0, - 0,0,3,0,0,0,67,0,0,0,115,14,0,0,0,124, - 2,124,0,106,0,124,1,60,0,100,0,83,0,41,1,78, - 41,1,114,229,0,0,0,41,3,114,104,0,0,0,218,5, - 105,110,100,101,120,114,37,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,218,11,95,95,115,101,116, - 105,116,101,109,95,95,225,3,0,0,115,2,0,0,0,0, - 1,122,26,95,78,97,109,101,115,112,97,99,101,80,97,116, - 104,46,95,95,115,101,116,105,116,101,109,95,95,99,1,0, - 0,0,0,0,0,0,1,0,0,0,2,0,0,0,67,0, - 0,0,115,12,0,0,0,116,0,124,0,106,1,131,0,131, - 1,83,0,41,1,78,41,2,114,33,0,0,0,114,237,0, - 0,0,41,1,114,104,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,7,95,95,108,101,110,95, - 95,228,3,0,0,115,2,0,0,0,0,1,122,22,95,78, - 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,108, - 101,110,95,95,99,1,0,0,0,0,0,0,0,1,0,0, - 0,2,0,0,0,67,0,0,0,115,12,0,0,0,100,1, - 106,0,124,0,106,1,131,1,83,0,41,2,78,122,20,95, - 78,97,109,101,115,112,97,99,101,80,97,116,104,40,123,33, - 114,125,41,41,2,114,50,0,0,0,114,229,0,0,0,41, - 1,114,104,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,218,8,95,95,114,101,112,114,95,95,231, - 3,0,0,115,2,0,0,0,0,1,122,23,95,78,97,109, - 101,115,112,97,99,101,80,97,116,104,46,95,95,114,101,112, - 114,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, - 2,0,0,0,67,0,0,0,115,12,0,0,0,124,1,124, - 0,106,0,131,0,107,6,83,0,41,1,78,41,1,114,237, - 0,0,0,41,2,114,104,0,0,0,218,4,105,116,101,109, + 0,12,4,0,0,115,2,0,0,0,0,1,122,28,95,78, + 97,109,101,115,112,97,99,101,76,111,97,100,101,114,46,101, + 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, + 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, + 26,0,0,0,116,0,106,1,100,1,124,0,106,2,131,2, + 1,0,116,0,106,3,124,0,124,1,131,2,83,0,41,2, + 122,98,76,111,97,100,32,97,32,110,97,109,101,115,112,97, + 99,101,32,109,111,100,117,108,101,46,10,10,32,32,32,32, + 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, + 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, + 85,115,101,32,101,120,101,99,95,109,111,100,117,108,101,40, + 41,32,105,110,115,116,101,97,100,46,10,10,32,32,32,32, + 32,32,32,32,122,38,110,97,109,101,115,112,97,99,101,32, + 109,111,100,117,108,101,32,108,111,97,100,101,100,32,119,105, + 116,104,32,112,97,116,104,32,123,33,114,125,41,4,114,118, + 0,0,0,114,133,0,0,0,114,229,0,0,0,114,189,0, + 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,190,0, + 0,0,15,4,0,0,115,6,0,0,0,0,7,6,1,8, + 1,122,28,95,78,97,109,101,115,112,97,99,101,76,111,97, + 100,101,114,46,108,111,97,100,95,109,111,100,117,108,101,78, + 41,12,114,109,0,0,0,114,108,0,0,0,114,110,0,0, + 0,114,182,0,0,0,114,180,0,0,0,114,247,0,0,0, + 114,157,0,0,0,114,199,0,0,0,114,184,0,0,0,114, + 183,0,0,0,114,188,0,0,0,114,190,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,114,246,0,0,0,243,3,0,0,115,16,0,0,0, + 8,1,8,3,12,9,8,3,8,3,8,3,8,3,8,3, + 114,246,0,0,0,99,0,0,0,0,0,0,0,0,0,0, + 0,0,4,0,0,0,64,0,0,0,115,106,0,0,0,101, + 0,90,1,100,0,90,2,100,1,90,3,101,4,100,2,100, + 3,132,0,131,1,90,5,101,4,100,4,100,5,132,0,131, + 1,90,6,101,4,100,6,100,7,132,0,131,1,90,7,101, + 4,100,8,100,9,132,0,131,1,90,8,101,4,100,17,100, + 11,100,12,132,1,131,1,90,9,101,4,100,18,100,13,100, + 14,132,1,131,1,90,10,101,4,100,19,100,15,100,16,132, + 1,131,1,90,11,100,10,83,0,41,20,218,10,80,97,116, + 104,70,105,110,100,101,114,122,62,77,101,116,97,32,112,97, + 116,104,32,102,105,110,100,101,114,32,102,111,114,32,115,121, + 115,46,112,97,116,104,32,97,110,100,32,112,97,99,107,97, + 103,101,32,95,95,112,97,116,104,95,95,32,97,116,116,114, + 105,98,117,116,101,115,46,99,1,0,0,0,0,0,0,0, + 2,0,0,0,4,0,0,0,67,0,0,0,115,42,0,0, + 0,120,36,116,0,106,1,106,2,131,0,68,0,93,22,125, + 1,116,3,124,1,100,1,131,2,114,12,124,1,106,4,131, + 0,1,0,113,12,87,0,100,2,83,0,41,3,122,125,67, + 97,108,108,32,116,104,101,32,105,110,118,97,108,105,100,97, + 116,101,95,99,97,99,104,101,115,40,41,32,109,101,116,104, + 111,100,32,111,110,32,97,108,108,32,112,97,116,104,32,101, + 110,116,114,121,32,102,105,110,100,101,114,115,10,32,32,32, + 32,32,32,32,32,115,116,111,114,101,100,32,105,110,32,115, + 121,115,46,112,97,116,104,95,105,109,112,111,114,116,101,114, + 95,99,97,99,104,101,115,32,40,119,104,101,114,101,32,105, + 109,112,108,101,109,101,110,116,101,100,41,46,218,17,105,110, + 118,97,108,105,100,97,116,101,95,99,97,99,104,101,115,78, + 41,5,114,8,0,0,0,218,19,112,97,116,104,95,105,109, + 112,111,114,116,101,114,95,99,97,99,104,101,218,6,118,97, + 108,117,101,115,114,112,0,0,0,114,249,0,0,0,41,2, + 114,168,0,0,0,218,6,102,105,110,100,101,114,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, + 0,33,4,0,0,115,6,0,0,0,0,4,16,1,10,1, + 122,28,80,97,116,104,70,105,110,100,101,114,46,105,110,118, + 97,108,105,100,97,116,101,95,99,97,99,104,101,115,99,2, + 0,0,0,0,0,0,0,3,0,0,0,12,0,0,0,67, + 0,0,0,115,86,0,0,0,116,0,106,1,100,1,107,9, + 114,30,116,0,106,1,12,0,114,30,116,2,106,3,100,2, + 116,4,131,2,1,0,120,50,116,0,106,1,68,0,93,36, + 125,2,121,8,124,2,124,1,131,1,83,0,4,0,116,5, + 107,10,114,72,1,0,1,0,1,0,119,38,89,0,113,38, + 88,0,113,38,87,0,100,1,83,0,100,1,83,0,41,3, + 122,46,83,101,97,114,99,104,32,115,121,115,46,112,97,116, + 104,95,104,111,111,107,115,32,102,111,114,32,97,32,102,105, + 110,100,101,114,32,102,111,114,32,39,112,97,116,104,39,46, + 78,122,23,115,121,115,46,112,97,116,104,95,104,111,111,107, + 115,32,105,115,32,101,109,112,116,121,41,6,114,8,0,0, + 0,218,10,112,97,116,104,95,104,111,111,107,115,114,63,0, + 0,0,114,64,0,0,0,114,122,0,0,0,114,103,0,0, + 0,41,3,114,168,0,0,0,114,37,0,0,0,90,4,104, + 111,111,107,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,11,95,112,97,116,104,95,104,111,111,107,115,41, + 4,0,0,115,16,0,0,0,0,3,18,1,12,1,12,1, + 2,1,8,1,14,1,12,2,122,22,80,97,116,104,70,105, + 110,100,101,114,46,95,112,97,116,104,95,104,111,111,107,115, + 99,2,0,0,0,0,0,0,0,3,0,0,0,19,0,0, + 0,67,0,0,0,115,102,0,0,0,124,1,100,1,107,2, + 114,42,121,12,116,0,106,1,131,0,125,1,87,0,110,20, + 4,0,116,2,107,10,114,40,1,0,1,0,1,0,100,2, + 83,0,88,0,121,14,116,3,106,4,124,1,25,0,125,2, + 87,0,110,40,4,0,116,5,107,10,114,96,1,0,1,0, + 1,0,124,0,106,6,124,1,131,1,125,2,124,2,116,3, + 106,4,124,1,60,0,89,0,110,2,88,0,124,2,83,0, + 41,3,122,210,71,101,116,32,116,104,101,32,102,105,110,100, + 101,114,32,102,111,114,32,116,104,101,32,112,97,116,104,32, + 101,110,116,114,121,32,102,114,111,109,32,115,121,115,46,112, + 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, + 104,101,46,10,10,32,32,32,32,32,32,32,32,73,102,32, + 116,104,101,32,112,97,116,104,32,101,110,116,114,121,32,105, + 115,32,110,111,116,32,105,110,32,116,104,101,32,99,97,99, + 104,101,44,32,102,105,110,100,32,116,104,101,32,97,112,112, + 114,111,112,114,105,97,116,101,32,102,105,110,100,101,114,10, + 32,32,32,32,32,32,32,32,97,110,100,32,99,97,99,104, + 101,32,105,116,46,32,73,102,32,110,111,32,102,105,110,100, + 101,114,32,105,115,32,97,118,97,105,108,97,98,108,101,44, + 32,115,116,111,114,101,32,78,111,110,101,46,10,10,32,32, + 32,32,32,32,32,32,114,32,0,0,0,78,41,7,114,3, + 0,0,0,114,47,0,0,0,218,17,70,105,108,101,78,111, + 116,70,111,117,110,100,69,114,114,111,114,114,8,0,0,0, + 114,250,0,0,0,114,135,0,0,0,114,254,0,0,0,41, + 3,114,168,0,0,0,114,37,0,0,0,114,252,0,0,0, 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 12,95,95,99,111,110,116,97,105,110,115,95,95,234,3,0, - 0,115,2,0,0,0,0,1,122,27,95,78,97,109,101,115, - 112,97,99,101,80,97,116,104,46,95,95,99,111,110,116,97, - 105,110,115,95,95,99,2,0,0,0,0,0,0,0,2,0, - 0,0,2,0,0,0,67,0,0,0,115,16,0,0,0,124, - 0,106,0,106,1,124,1,131,1,1,0,100,0,83,0,41, - 1,78,41,2,114,229,0,0,0,114,161,0,0,0,41,2, - 114,104,0,0,0,114,244,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,161,0,0,0,237,3, - 0,0,115,2,0,0,0,0,1,122,21,95,78,97,109,101, - 115,112,97,99,101,80,97,116,104,46,97,112,112,101,110,100, - 78,41,14,114,109,0,0,0,114,108,0,0,0,114,110,0, - 0,0,114,111,0,0,0,114,182,0,0,0,114,235,0,0, - 0,114,230,0,0,0,114,237,0,0,0,114,239,0,0,0, - 114,241,0,0,0,114,242,0,0,0,114,243,0,0,0,114, - 245,0,0,0,114,161,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,227,0, - 0,0,182,3,0,0,115,22,0,0,0,8,5,4,2,8, - 6,8,10,8,4,8,13,8,3,8,3,8,3,8,3,8, - 3,114,227,0,0,0,99,0,0,0,0,0,0,0,0,0, - 0,0,0,3,0,0,0,64,0,0,0,115,80,0,0,0, - 101,0,90,1,100,0,90,2,100,1,100,2,132,0,90,3, - 101,4,100,3,100,4,132,0,131,1,90,5,100,5,100,6, - 132,0,90,6,100,7,100,8,132,0,90,7,100,9,100,10, - 132,0,90,8,100,11,100,12,132,0,90,9,100,13,100,14, - 132,0,90,10,100,15,100,16,132,0,90,11,100,17,83,0, - 41,18,218,16,95,78,97,109,101,115,112,97,99,101,76,111, - 97,100,101,114,99,4,0,0,0,0,0,0,0,4,0,0, - 0,4,0,0,0,67,0,0,0,115,18,0,0,0,116,0, - 124,1,124,2,124,3,131,3,124,0,95,1,100,0,83,0, - 41,1,78,41,2,114,227,0,0,0,114,229,0,0,0,41, - 4,114,104,0,0,0,114,102,0,0,0,114,37,0,0,0, - 114,233,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,182,0,0,0,243,3,0,0,115,2,0, - 0,0,0,1,122,25,95,78,97,109,101,115,112,97,99,101, - 76,111,97,100,101,114,46,95,95,105,110,105,116,95,95,99, - 2,0,0,0,0,0,0,0,2,0,0,0,2,0,0,0, - 67,0,0,0,115,12,0,0,0,100,1,106,0,124,1,106, - 1,131,1,83,0,41,2,122,115,82,101,116,117,114,110,32, - 114,101,112,114,32,102,111,114,32,116,104,101,32,109,111,100, - 117,108,101,46,10,10,32,32,32,32,32,32,32,32,84,104, - 101,32,109,101,116,104,111,100,32,105,115,32,100,101,112,114, - 101,99,97,116,101,100,46,32,32,84,104,101,32,105,109,112, - 111,114,116,32,109,97,99,104,105,110,101,114,121,32,100,111, - 101,115,32,116,104,101,32,106,111,98,32,105,116,115,101,108, - 102,46,10,10,32,32,32,32,32,32,32,32,122,25,60,109, - 111,100,117,108,101,32,123,33,114,125,32,40,110,97,109,101, - 115,112,97,99,101,41,62,41,2,114,50,0,0,0,114,109, - 0,0,0,41,2,114,168,0,0,0,114,187,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,11, - 109,111,100,117,108,101,95,114,101,112,114,246,3,0,0,115, - 2,0,0,0,0,7,122,28,95,78,97,109,101,115,112,97, - 99,101,76,111,97,100,101,114,46,109,111,100,117,108,101,95, - 114,101,112,114,99,2,0,0,0,0,0,0,0,2,0,0, - 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, - 83,0,41,2,78,84,114,4,0,0,0,41,2,114,104,0, - 0,0,114,123,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,157,0,0,0,255,3,0,0,115, - 2,0,0,0,0,1,122,27,95,78,97,109,101,115,112,97, - 99,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, - 97,103,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, - 0,41,2,78,114,32,0,0,0,114,4,0,0,0,41,2, - 114,104,0,0,0,114,123,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,199,0,0,0,2,4, - 0,0,115,2,0,0,0,0,1,122,27,95,78,97,109,101, - 115,112,97,99,101,76,111,97,100,101,114,46,103,101,116,95, - 115,111,117,114,99,101,99,2,0,0,0,0,0,0,0,2, - 0,0,0,6,0,0,0,67,0,0,0,115,16,0,0,0, - 116,0,100,1,100,2,100,3,100,4,100,5,141,4,83,0, - 41,6,78,114,32,0,0,0,122,8,60,115,116,114,105,110, - 103,62,114,186,0,0,0,84,41,1,114,201,0,0,0,41, - 1,114,202,0,0,0,41,2,114,104,0,0,0,114,123,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,184,0,0,0,5,4,0,0,115,2,0,0,0,0, - 1,122,25,95,78,97,109,101,115,112,97,99,101,76,111,97, - 100,101,114,46,103,101,116,95,99,111,100,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, - 0,115,4,0,0,0,100,1,83,0,41,2,122,42,85,115, - 101,32,100,101,102,97,117,108,116,32,115,101,109,97,110,116, - 105,99,115,32,102,111,114,32,109,111,100,117,108,101,32,99, - 114,101,97,116,105,111,110,46,78,114,4,0,0,0,41,2, - 114,104,0,0,0,114,162,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,183,0,0,0,8,4, - 0,0,115,0,0,0,0,122,30,95,78,97,109,101,115,112, - 97,99,101,76,111,97,100,101,114,46,99,114,101,97,116,101, - 95,109,111,100,117,108,101,99,2,0,0,0,0,0,0,0, - 2,0,0,0,1,0,0,0,67,0,0,0,115,4,0,0, - 0,100,0,83,0,41,1,78,114,4,0,0,0,41,2,114, - 104,0,0,0,114,187,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,188,0,0,0,11,4,0, - 0,115,2,0,0,0,0,1,122,28,95,78,97,109,101,115, - 112,97,99,101,76,111,97,100,101,114,46,101,120,101,99,95, - 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,2, - 0,0,0,3,0,0,0,67,0,0,0,115,26,0,0,0, - 116,0,106,1,100,1,124,0,106,2,131,2,1,0,116,0, - 106,3,124,0,124,1,131,2,83,0,41,2,122,98,76,111, - 97,100,32,97,32,110,97,109,101,115,112,97,99,101,32,109, - 111,100,117,108,101,46,10,10,32,32,32,32,32,32,32,32, - 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, - 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, - 101,120,101,99,95,109,111,100,117,108,101,40,41,32,105,110, - 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, - 122,38,110,97,109,101,115,112,97,99,101,32,109,111,100,117, - 108,101,32,108,111,97,100,101,100,32,119,105,116,104,32,112, - 97,116,104,32,123,33,114,125,41,4,114,118,0,0,0,114, - 133,0,0,0,114,229,0,0,0,114,189,0,0,0,41,2, - 114,104,0,0,0,114,123,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,190,0,0,0,14,4, - 0,0,115,6,0,0,0,0,7,6,1,8,1,122,28,95, - 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, - 108,111,97,100,95,109,111,100,117,108,101,78,41,12,114,109, - 0,0,0,114,108,0,0,0,114,110,0,0,0,114,182,0, - 0,0,114,180,0,0,0,114,247,0,0,0,114,157,0,0, - 0,114,199,0,0,0,114,184,0,0,0,114,183,0,0,0, - 114,188,0,0,0,114,190,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,246, - 0,0,0,242,3,0,0,115,16,0,0,0,8,1,8,3, - 12,9,8,3,8,3,8,3,8,3,8,3,114,246,0,0, - 0,99,0,0,0,0,0,0,0,0,0,0,0,0,4,0, - 0,0,64,0,0,0,115,106,0,0,0,101,0,90,1,100, - 0,90,2,100,1,90,3,101,4,100,2,100,3,132,0,131, - 1,90,5,101,4,100,4,100,5,132,0,131,1,90,6,101, - 4,100,6,100,7,132,0,131,1,90,7,101,4,100,8,100, - 9,132,0,131,1,90,8,101,4,100,17,100,11,100,12,132, - 1,131,1,90,9,101,4,100,18,100,13,100,14,132,1,131, - 1,90,10,101,4,100,19,100,15,100,16,132,1,131,1,90, - 11,100,10,83,0,41,20,218,10,80,97,116,104,70,105,110, - 100,101,114,122,62,77,101,116,97,32,112,97,116,104,32,102, - 105,110,100,101,114,32,102,111,114,32,115,121,115,46,112,97, - 116,104,32,97,110,100,32,112,97,99,107,97,103,101,32,95, - 95,112,97,116,104,95,95,32,97,116,116,114,105,98,117,116, - 101,115,46,99,1,0,0,0,0,0,0,0,2,0,0,0, - 4,0,0,0,67,0,0,0,115,42,0,0,0,120,36,116, - 0,106,1,106,2,131,0,68,0,93,22,125,1,116,3,124, - 1,100,1,131,2,114,12,124,1,106,4,131,0,1,0,113, - 12,87,0,100,2,83,0,41,3,122,125,67,97,108,108,32, - 116,104,101,32,105,110,118,97,108,105,100,97,116,101,95,99, - 97,99,104,101,115,40,41,32,109,101,116,104,111,100,32,111, - 110,32,97,108,108,32,112,97,116,104,32,101,110,116,114,121, - 32,102,105,110,100,101,114,115,10,32,32,32,32,32,32,32, - 32,115,116,111,114,101,100,32,105,110,32,115,121,115,46,112, - 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, - 104,101,115,32,40,119,104,101,114,101,32,105,109,112,108,101, - 109,101,110,116,101,100,41,46,218,17,105,110,118,97,108,105, - 100,97,116,101,95,99,97,99,104,101,115,78,41,5,114,8, - 0,0,0,218,19,112,97,116,104,95,105,109,112,111,114,116, - 101,114,95,99,97,99,104,101,218,6,118,97,108,117,101,115, - 114,112,0,0,0,114,249,0,0,0,41,2,114,168,0,0, - 0,218,6,102,105,110,100,101,114,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,249,0,0,0,32,4,0, - 0,115,6,0,0,0,0,4,16,1,10,1,122,28,80,97, - 116,104,70,105,110,100,101,114,46,105,110,118,97,108,105,100, - 97,116,101,95,99,97,99,104,101,115,99,2,0,0,0,0, - 0,0,0,3,0,0,0,12,0,0,0,67,0,0,0,115, - 86,0,0,0,116,0,106,1,100,1,107,9,114,30,116,0, - 106,1,12,0,114,30,116,2,106,3,100,2,116,4,131,2, - 1,0,120,50,116,0,106,1,68,0,93,36,125,2,121,8, - 124,2,124,1,131,1,83,0,4,0,116,5,107,10,114,72, - 1,0,1,0,1,0,119,38,89,0,113,38,88,0,113,38, - 87,0,100,1,83,0,100,1,83,0,41,3,122,46,83,101, - 97,114,99,104,32,115,121,115,46,112,97,116,104,95,104,111, - 111,107,115,32,102,111,114,32,97,32,102,105,110,100,101,114, - 32,102,111,114,32,39,112,97,116,104,39,46,78,122,23,115, - 121,115,46,112,97,116,104,95,104,111,111,107,115,32,105,115, - 32,101,109,112,116,121,41,6,114,8,0,0,0,218,10,112, - 97,116,104,95,104,111,111,107,115,114,63,0,0,0,114,64, - 0,0,0,114,122,0,0,0,114,103,0,0,0,41,3,114, - 168,0,0,0,114,37,0,0,0,90,4,104,111,111,107,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,11, - 95,112,97,116,104,95,104,111,111,107,115,40,4,0,0,115, - 16,0,0,0,0,3,18,1,12,1,12,1,2,1,8,1, - 14,1,12,2,122,22,80,97,116,104,70,105,110,100,101,114, - 46,95,112,97,116,104,95,104,111,111,107,115,99,2,0,0, - 0,0,0,0,0,3,0,0,0,19,0,0,0,67,0,0, - 0,115,102,0,0,0,124,1,100,1,107,2,114,42,121,12, - 116,0,106,1,131,0,125,1,87,0,110,20,4,0,116,2, - 107,10,114,40,1,0,1,0,1,0,100,2,83,0,88,0, - 121,14,116,3,106,4,124,1,25,0,125,2,87,0,110,40, - 4,0,116,5,107,10,114,96,1,0,1,0,1,0,124,0, - 106,6,124,1,131,1,125,2,124,2,116,3,106,4,124,1, - 60,0,89,0,110,2,88,0,124,2,83,0,41,3,122,210, - 71,101,116,32,116,104,101,32,102,105,110,100,101,114,32,102, - 111,114,32,116,104,101,32,112,97,116,104,32,101,110,116,114, - 121,32,102,114,111,109,32,115,121,115,46,112,97,116,104,95, - 105,109,112,111,114,116,101,114,95,99,97,99,104,101,46,10, - 10,32,32,32,32,32,32,32,32,73,102,32,116,104,101,32, - 112,97,116,104,32,101,110,116,114,121,32,105,115,32,110,111, - 116,32,105,110,32,116,104,101,32,99,97,99,104,101,44,32, - 102,105,110,100,32,116,104,101,32,97,112,112,114,111,112,114, - 105,97,116,101,32,102,105,110,100,101,114,10,32,32,32,32, - 32,32,32,32,97,110,100,32,99,97,99,104,101,32,105,116, - 46,32,73,102,32,110,111,32,102,105,110,100,101,114,32,105, - 115,32,97,118,97,105,108,97,98,108,101,44,32,115,116,111, - 114,101,32,78,111,110,101,46,10,10,32,32,32,32,32,32, - 32,32,114,32,0,0,0,78,41,7,114,3,0,0,0,114, - 47,0,0,0,218,17,70,105,108,101,78,111,116,70,111,117, - 110,100,69,114,114,111,114,114,8,0,0,0,114,250,0,0, - 0,114,135,0,0,0,114,254,0,0,0,41,3,114,168,0, - 0,0,114,37,0,0,0,114,252,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,20,95,112,97, - 116,104,95,105,109,112,111,114,116,101,114,95,99,97,99,104, - 101,53,4,0,0,115,22,0,0,0,0,8,8,1,2,1, - 12,1,14,3,6,1,2,1,14,1,14,1,10,1,16,1, - 122,31,80,97,116,104,70,105,110,100,101,114,46,95,112,97, - 116,104,95,105,109,112,111,114,116,101,114,95,99,97,99,104, - 101,99,3,0,0,0,0,0,0,0,6,0,0,0,3,0, - 0,0,67,0,0,0,115,82,0,0,0,116,0,124,2,100, - 1,131,2,114,26,124,2,106,1,124,1,131,1,92,2,125, - 3,125,4,110,14,124,2,106,2,124,1,131,1,125,3,103, - 0,125,4,124,3,100,0,107,9,114,60,116,3,106,4,124, - 1,124,3,131,2,83,0,116,3,106,5,124,1,100,0,131, - 2,125,5,124,4,124,5,95,6,124,5,83,0,41,2,78, - 114,121,0,0,0,41,7,114,112,0,0,0,114,121,0,0, - 0,114,179,0,0,0,114,118,0,0,0,114,176,0,0,0, - 114,158,0,0,0,114,154,0,0,0,41,6,114,168,0,0, - 0,114,123,0,0,0,114,252,0,0,0,114,124,0,0,0, - 114,125,0,0,0,114,162,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,218,16,95,108,101,103,97, - 99,121,95,103,101,116,95,115,112,101,99,75,4,0,0,115, - 18,0,0,0,0,4,10,1,16,2,10,1,4,1,8,1, - 12,1,12,1,6,1,122,27,80,97,116,104,70,105,110,100, - 101,114,46,95,108,101,103,97,99,121,95,103,101,116,95,115, - 112,101,99,78,99,4,0,0,0,0,0,0,0,9,0,0, - 0,5,0,0,0,67,0,0,0,115,170,0,0,0,103,0, - 125,4,120,160,124,2,68,0,93,130,125,5,116,0,124,5, - 116,1,116,2,102,2,131,2,115,30,113,10,124,0,106,3, - 124,5,131,1,125,6,124,6,100,1,107,9,114,10,116,4, - 124,6,100,2,131,2,114,72,124,6,106,5,124,1,124,3, - 131,2,125,7,110,12,124,0,106,6,124,1,124,6,131,2, - 125,7,124,7,100,1,107,8,114,94,113,10,124,7,106,7, - 100,1,107,9,114,108,124,7,83,0,124,7,106,8,125,8, - 124,8,100,1,107,8,114,130,116,9,100,3,131,1,130,1, - 124,4,106,10,124,8,131,1,1,0,113,10,87,0,116,11, - 106,12,124,1,100,1,131,2,125,7,124,4,124,7,95,8, - 124,7,83,0,100,1,83,0,41,4,122,63,70,105,110,100, - 32,116,104,101,32,108,111,97,100,101,114,32,111,114,32,110, - 97,109,101,115,112,97,99,101,95,112,97,116,104,32,102,111, - 114,32,116,104,105,115,32,109,111,100,117,108,101,47,112,97, - 99,107,97,103,101,32,110,97,109,101,46,78,114,178,0,0, - 0,122,19,115,112,101,99,32,109,105,115,115,105,110,103,32, - 108,111,97,100,101,114,41,13,114,141,0,0,0,114,73,0, - 0,0,218,5,98,121,116,101,115,114,0,1,0,0,114,112, - 0,0,0,114,178,0,0,0,114,1,1,0,0,114,124,0, - 0,0,114,154,0,0,0,114,103,0,0,0,114,147,0,0, - 0,114,118,0,0,0,114,158,0,0,0,41,9,114,168,0, - 0,0,114,123,0,0,0,114,37,0,0,0,114,177,0,0, - 0,218,14,110,97,109,101,115,112,97,99,101,95,112,97,116, - 104,90,5,101,110,116,114,121,114,252,0,0,0,114,162,0, - 0,0,114,125,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,218,9,95,103,101,116,95,115,112,101, - 99,90,4,0,0,115,40,0,0,0,0,5,4,1,10,1, - 14,1,2,1,10,1,8,1,10,1,14,2,12,1,8,1, - 2,1,10,1,4,1,6,1,8,1,8,5,14,2,12,1, - 6,1,122,20,80,97,116,104,70,105,110,100,101,114,46,95, - 103,101,116,95,115,112,101,99,99,4,0,0,0,0,0,0, - 0,6,0,0,0,4,0,0,0,67,0,0,0,115,104,0, - 0,0,124,2,100,1,107,8,114,14,116,0,106,1,125,2, - 124,0,106,2,124,1,124,2,124,3,131,3,125,4,124,4, - 100,1,107,8,114,42,100,1,83,0,110,58,124,4,106,3, - 100,1,107,8,114,96,124,4,106,4,125,5,124,5,114,90, - 100,2,124,4,95,5,116,6,124,1,124,5,124,0,106,2, - 131,3,124,4,95,4,124,4,83,0,113,100,100,1,83,0, - 110,4,124,4,83,0,100,1,83,0,41,3,122,141,84,114, - 121,32,116,111,32,102,105,110,100,32,97,32,115,112,101,99, - 32,102,111,114,32,39,102,117,108,108,110,97,109,101,39,32, - 111,110,32,115,121,115,46,112,97,116,104,32,111,114,32,39, - 112,97,116,104,39,46,10,10,32,32,32,32,32,32,32,32, - 84,104,101,32,115,101,97,114,99,104,32,105,115,32,98,97, - 115,101,100,32,111,110,32,115,121,115,46,112,97,116,104,95, - 104,111,111,107,115,32,97,110,100,32,115,121,115,46,112,97, - 116,104,95,105,109,112,111,114,116,101,114,95,99,97,99,104, - 101,46,10,32,32,32,32,32,32,32,32,78,90,9,110,97, - 109,101,115,112,97,99,101,41,7,114,8,0,0,0,114,37, - 0,0,0,114,4,1,0,0,114,124,0,0,0,114,154,0, - 0,0,114,156,0,0,0,114,227,0,0,0,41,6,114,168, - 0,0,0,114,123,0,0,0,114,37,0,0,0,114,177,0, - 0,0,114,162,0,0,0,114,3,1,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,178,0,0,0, - 122,4,0,0,115,26,0,0,0,0,6,8,1,6,1,14, - 1,8,1,6,1,10,1,6,1,4,3,6,1,16,1,6, - 2,6,2,122,20,80,97,116,104,70,105,110,100,101,114,46, - 102,105,110,100,95,115,112,101,99,99,3,0,0,0,0,0, - 0,0,4,0,0,0,3,0,0,0,67,0,0,0,115,30, - 0,0,0,124,0,106,0,124,1,124,2,131,2,125,3,124, - 3,100,1,107,8,114,24,100,1,83,0,124,3,106,1,83, - 0,41,2,122,170,102,105,110,100,32,116,104,101,32,109,111, - 100,117,108,101,32,111,110,32,115,121,115,46,112,97,116,104, - 32,111,114,32,39,112,97,116,104,39,32,98,97,115,101,100, - 32,111,110,32,115,121,115,46,112,97,116,104,95,104,111,111, - 107,115,32,97,110,100,10,32,32,32,32,32,32,32,32,115, - 121,115,46,112,97,116,104,95,105,109,112,111,114,116,101,114, - 95,99,97,99,104,101,46,10,10,32,32,32,32,32,32,32, - 32,84,104,105,115,32,109,101,116,104,111,100,32,105,115,32, - 100,101,112,114,101,99,97,116,101,100,46,32,32,85,115,101, - 32,102,105,110,100,95,115,112,101,99,40,41,32,105,110,115, - 116,101,97,100,46,10,10,32,32,32,32,32,32,32,32,78, - 41,2,114,178,0,0,0,114,124,0,0,0,41,4,114,168, - 0,0,0,114,123,0,0,0,114,37,0,0,0,114,162,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,179,0,0,0,146,4,0,0,115,8,0,0,0,0, - 8,12,1,8,1,4,1,122,22,80,97,116,104,70,105,110, - 100,101,114,46,102,105,110,100,95,109,111,100,117,108,101,41, - 1,78,41,2,78,78,41,1,78,41,12,114,109,0,0,0, - 114,108,0,0,0,114,110,0,0,0,114,111,0,0,0,114, - 180,0,0,0,114,249,0,0,0,114,254,0,0,0,114,0, - 1,0,0,114,1,1,0,0,114,4,1,0,0,114,178,0, - 0,0,114,179,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,248,0,0,0, - 28,4,0,0,115,22,0,0,0,8,2,4,2,12,8,12, - 13,12,22,12,15,2,1,12,31,2,1,12,23,2,1,114, - 248,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, - 0,3,0,0,0,64,0,0,0,115,90,0,0,0,101,0, - 90,1,100,0,90,2,100,1,90,3,100,2,100,3,132,0, - 90,4,100,4,100,5,132,0,90,5,101,6,90,7,100,6, - 100,7,132,0,90,8,100,8,100,9,132,0,90,9,100,19, - 100,11,100,12,132,1,90,10,100,13,100,14,132,0,90,11, - 101,12,100,15,100,16,132,0,131,1,90,13,100,17,100,18, - 132,0,90,14,100,10,83,0,41,20,218,10,70,105,108,101, - 70,105,110,100,101,114,122,172,70,105,108,101,45,98,97,115, - 101,100,32,102,105,110,100,101,114,46,10,10,32,32,32,32, - 73,110,116,101,114,97,99,116,105,111,110,115,32,119,105,116, - 104,32,116,104,101,32,102,105,108,101,32,115,121,115,116,101, - 109,32,97,114,101,32,99,97,99,104,101,100,32,102,111,114, - 32,112,101,114,102,111,114,109,97,110,99,101,44,32,98,101, - 105,110,103,10,32,32,32,32,114,101,102,114,101,115,104,101, - 100,32,119,104,101,110,32,116,104,101,32,100,105,114,101,99, - 116,111,114,121,32,116,104,101,32,102,105,110,100,101,114,32, - 105,115,32,104,97,110,100,108,105,110,103,32,104,97,115,32, - 98,101,101,110,32,109,111,100,105,102,105,101,100,46,10,10, - 32,32,32,32,99,2,0,0,0,0,0,0,0,5,0,0, - 0,5,0,0,0,7,0,0,0,115,88,0,0,0,103,0, - 125,3,120,40,124,2,68,0,93,32,92,2,137,0,125,4, - 124,3,106,0,135,0,102,1,100,1,100,2,132,8,124,4, - 68,0,131,1,131,1,1,0,113,10,87,0,124,3,124,0, - 95,1,124,1,112,58,100,3,124,0,95,2,100,6,124,0, - 95,3,116,4,131,0,124,0,95,5,116,4,131,0,124,0, - 95,6,100,5,83,0,41,7,122,154,73,110,105,116,105,97, - 108,105,122,101,32,119,105,116,104,32,116,104,101,32,112,97, - 116,104,32,116,111,32,115,101,97,114,99,104,32,111,110,32, - 97,110,100,32,97,32,118,97,114,105,97,98,108,101,32,110, - 117,109,98,101,114,32,111,102,10,32,32,32,32,32,32,32, - 32,50,45,116,117,112,108,101,115,32,99,111,110,116,97,105, - 110,105,110,103,32,116,104,101,32,108,111,97,100,101,114,32, - 97,110,100,32,116,104,101,32,102,105,108,101,32,115,117,102, - 102,105,120,101,115,32,116,104,101,32,108,111,97,100,101,114, - 10,32,32,32,32,32,32,32,32,114,101,99,111,103,110,105, - 122,101,115,46,99,1,0,0,0,0,0,0,0,2,0,0, - 0,3,0,0,0,51,0,0,0,115,22,0,0,0,124,0, - 93,14,125,1,124,1,136,0,102,2,86,0,1,0,113,2, - 100,0,83,0,41,1,78,114,4,0,0,0,41,2,114,24, - 0,0,0,114,222,0,0,0,41,1,114,124,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,224,0,0,0,175,4, - 0,0,115,2,0,0,0,4,0,122,38,70,105,108,101,70, - 105,110,100,101,114,46,95,95,105,110,105,116,95,95,46,60, - 108,111,99,97,108,115,62,46,60,103,101,110,101,120,112,114, - 62,114,61,0,0,0,114,31,0,0,0,78,114,91,0,0, - 0,41,7,114,147,0,0,0,218,8,95,108,111,97,100,101, - 114,115,114,37,0,0,0,218,11,95,112,97,116,104,95,109, - 116,105,109,101,218,3,115,101,116,218,11,95,112,97,116,104, - 95,99,97,99,104,101,218,19,95,114,101,108,97,120,101,100, - 95,112,97,116,104,95,99,97,99,104,101,41,5,114,104,0, - 0,0,114,37,0,0,0,218,14,108,111,97,100,101,114,95, - 100,101,116,97,105,108,115,90,7,108,111,97,100,101,114,115, - 114,164,0,0,0,114,4,0,0,0,41,1,114,124,0,0, - 0,114,6,0,0,0,114,182,0,0,0,169,4,0,0,115, - 16,0,0,0,0,4,4,1,14,1,28,1,6,2,10,1, - 6,1,8,1,122,19,70,105,108,101,70,105,110,100,101,114, - 46,95,95,105,110,105,116,95,95,99,1,0,0,0,0,0, - 0,0,1,0,0,0,2,0,0,0,67,0,0,0,115,10, - 0,0,0,100,3,124,0,95,0,100,2,83,0,41,4,122, - 31,73,110,118,97,108,105,100,97,116,101,32,116,104,101,32, - 100,105,114,101,99,116,111,114,121,32,109,116,105,109,101,46, - 114,31,0,0,0,78,114,91,0,0,0,41,1,114,7,1, - 0,0,41,1,114,104,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,249,0,0,0,183,4,0, - 0,115,2,0,0,0,0,2,122,28,70,105,108,101,70,105, - 110,100,101,114,46,105,110,118,97,108,105,100,97,116,101,95, - 99,97,99,104,101,115,99,2,0,0,0,0,0,0,0,3, - 0,0,0,2,0,0,0,67,0,0,0,115,42,0,0,0, - 124,0,106,0,124,1,131,1,125,2,124,2,100,1,107,8, - 114,26,100,1,103,0,102,2,83,0,124,2,106,1,124,2, - 106,2,112,38,103,0,102,2,83,0,41,2,122,197,84,114, - 121,32,116,111,32,102,105,110,100,32,97,32,108,111,97,100, - 101,114,32,102,111,114,32,116,104,101,32,115,112,101,99,105, - 102,105,101,100,32,109,111,100,117,108,101,44,32,111,114,32, - 116,104,101,32,110,97,109,101,115,112,97,99,101,10,32,32, - 32,32,32,32,32,32,112,97,99,107,97,103,101,32,112,111, - 114,116,105,111,110,115,46,32,82,101,116,117,114,110,115,32, - 40,108,111,97,100,101,114,44,32,108,105,115,116,45,111,102, - 45,112,111,114,116,105,111,110,115,41,46,10,10,32,32,32, + 20,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, + 99,97,99,104,101,54,4,0,0,115,22,0,0,0,0,8, + 8,1,2,1,12,1,14,3,6,1,2,1,14,1,14,1, + 10,1,16,1,122,31,80,97,116,104,70,105,110,100,101,114, + 46,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, + 99,97,99,104,101,99,3,0,0,0,0,0,0,0,6,0, + 0,0,3,0,0,0,67,0,0,0,115,82,0,0,0,116, + 0,124,2,100,1,131,2,114,26,124,2,106,1,124,1,131, + 1,92,2,125,3,125,4,110,14,124,2,106,2,124,1,131, + 1,125,3,103,0,125,4,124,3,100,0,107,9,114,60,116, + 3,106,4,124,1,124,3,131,2,83,0,116,3,106,5,124, + 1,100,0,131,2,125,5,124,4,124,5,95,6,124,5,83, + 0,41,2,78,114,121,0,0,0,41,7,114,112,0,0,0, + 114,121,0,0,0,114,179,0,0,0,114,118,0,0,0,114, + 176,0,0,0,114,158,0,0,0,114,154,0,0,0,41,6, + 114,168,0,0,0,114,123,0,0,0,114,252,0,0,0,114, + 124,0,0,0,114,125,0,0,0,114,162,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,16,95, + 108,101,103,97,99,121,95,103,101,116,95,115,112,101,99,76, + 4,0,0,115,18,0,0,0,0,4,10,1,16,2,10,1, + 4,1,8,1,12,1,12,1,6,1,122,27,80,97,116,104, + 70,105,110,100,101,114,46,95,108,101,103,97,99,121,95,103, + 101,116,95,115,112,101,99,78,99,4,0,0,0,0,0,0, + 0,9,0,0,0,5,0,0,0,67,0,0,0,115,170,0, + 0,0,103,0,125,4,120,160,124,2,68,0,93,130,125,5, + 116,0,124,5,116,1,116,2,102,2,131,2,115,30,113,10, + 124,0,106,3,124,5,131,1,125,6,124,6,100,1,107,9, + 114,10,116,4,124,6,100,2,131,2,114,72,124,6,106,5, + 124,1,124,3,131,2,125,7,110,12,124,0,106,6,124,1, + 124,6,131,2,125,7,124,7,100,1,107,8,114,94,113,10, + 124,7,106,7,100,1,107,9,114,108,124,7,83,0,124,7, + 106,8,125,8,124,8,100,1,107,8,114,130,116,9,100,3, + 131,1,130,1,124,4,106,10,124,8,131,1,1,0,113,10, + 87,0,116,11,106,12,124,1,100,1,131,2,125,7,124,4, + 124,7,95,8,124,7,83,0,100,1,83,0,41,4,122,63, + 70,105,110,100,32,116,104,101,32,108,111,97,100,101,114,32, + 111,114,32,110,97,109,101,115,112,97,99,101,95,112,97,116, + 104,32,102,111,114,32,116,104,105,115,32,109,111,100,117,108, + 101,47,112,97,99,107,97,103,101,32,110,97,109,101,46,78, + 114,178,0,0,0,122,19,115,112,101,99,32,109,105,115,115, + 105,110,103,32,108,111,97,100,101,114,41,13,114,141,0,0, + 0,114,73,0,0,0,218,5,98,121,116,101,115,114,0,1, + 0,0,114,112,0,0,0,114,178,0,0,0,114,1,1,0, + 0,114,124,0,0,0,114,154,0,0,0,114,103,0,0,0, + 114,147,0,0,0,114,118,0,0,0,114,158,0,0,0,41, + 9,114,168,0,0,0,114,123,0,0,0,114,37,0,0,0, + 114,177,0,0,0,218,14,110,97,109,101,115,112,97,99,101, + 95,112,97,116,104,90,5,101,110,116,114,121,114,252,0,0, + 0,114,162,0,0,0,114,125,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,9,95,103,101,116, + 95,115,112,101,99,91,4,0,0,115,40,0,0,0,0,5, + 4,1,10,1,14,1,2,1,10,1,8,1,10,1,14,2, + 12,1,8,1,2,1,10,1,4,1,6,1,8,1,8,5, + 14,2,12,1,6,1,122,20,80,97,116,104,70,105,110,100, + 101,114,46,95,103,101,116,95,115,112,101,99,99,4,0,0, + 0,0,0,0,0,6,0,0,0,4,0,0,0,67,0,0, + 0,115,104,0,0,0,124,2,100,1,107,8,114,14,116,0, + 106,1,125,2,124,0,106,2,124,1,124,2,124,3,131,3, + 125,4,124,4,100,1,107,8,114,42,100,1,83,0,110,58, + 124,4,106,3,100,1,107,8,114,96,124,4,106,4,125,5, + 124,5,114,90,100,2,124,4,95,5,116,6,124,1,124,5, + 124,0,106,2,131,3,124,4,95,4,124,4,83,0,113,100, + 100,1,83,0,110,4,124,4,83,0,100,1,83,0,41,3, + 122,141,84,114,121,32,116,111,32,102,105,110,100,32,97,32, + 115,112,101,99,32,102,111,114,32,39,102,117,108,108,110,97, + 109,101,39,32,111,110,32,115,121,115,46,112,97,116,104,32, + 111,114,32,39,112,97,116,104,39,46,10,10,32,32,32,32, + 32,32,32,32,84,104,101,32,115,101,97,114,99,104,32,105, + 115,32,98,97,115,101,100,32,111,110,32,115,121,115,46,112, + 97,116,104,95,104,111,111,107,115,32,97,110,100,32,115,121, + 115,46,112,97,116,104,95,105,109,112,111,114,116,101,114,95, + 99,97,99,104,101,46,10,32,32,32,32,32,32,32,32,78, + 90,9,110,97,109,101,115,112,97,99,101,41,7,114,8,0, + 0,0,114,37,0,0,0,114,4,1,0,0,114,124,0,0, + 0,114,154,0,0,0,114,156,0,0,0,114,227,0,0,0, + 41,6,114,168,0,0,0,114,123,0,0,0,114,37,0,0, + 0,114,177,0,0,0,114,162,0,0,0,114,3,1,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, + 178,0,0,0,123,4,0,0,115,26,0,0,0,0,6,8, + 1,6,1,14,1,8,1,6,1,10,1,6,1,4,3,6, + 1,16,1,6,2,6,2,122,20,80,97,116,104,70,105,110, + 100,101,114,46,102,105,110,100,95,115,112,101,99,99,3,0, + 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, + 0,0,115,30,0,0,0,124,0,106,0,124,1,124,2,131, + 2,125,3,124,3,100,1,107,8,114,24,100,1,83,0,124, + 3,106,1,83,0,41,2,122,170,102,105,110,100,32,116,104, + 101,32,109,111,100,117,108,101,32,111,110,32,115,121,115,46, + 112,97,116,104,32,111,114,32,39,112,97,116,104,39,32,98, + 97,115,101,100,32,111,110,32,115,121,115,46,112,97,116,104, + 95,104,111,111,107,115,32,97,110,100,10,32,32,32,32,32, + 32,32,32,115,121,115,46,112,97,116,104,95,105,109,112,111, + 114,116,101,114,95,99,97,99,104,101,46,10,10,32,32,32, 32,32,32,32,32,84,104,105,115,32,109,101,116,104,111,100, 32,105,115,32,100,101,112,114,101,99,97,116,101,100,46,32, 32,85,115,101,32,102,105,110,100,95,115,112,101,99,40,41, 32,105,110,115,116,101,97,100,46,10,10,32,32,32,32,32, - 32,32,32,78,41,3,114,178,0,0,0,114,124,0,0,0, - 114,154,0,0,0,41,3,114,104,0,0,0,114,123,0,0, + 32,32,32,78,41,2,114,178,0,0,0,114,124,0,0,0, + 41,4,114,168,0,0,0,114,123,0,0,0,114,37,0,0, 0,114,162,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,121,0,0,0,189,4,0,0,115,8, - 0,0,0,0,7,10,1,8,1,8,1,122,22,70,105,108, - 101,70,105,110,100,101,114,46,102,105,110,100,95,108,111,97, - 100,101,114,99,6,0,0,0,0,0,0,0,7,0,0,0, - 6,0,0,0,67,0,0,0,115,26,0,0,0,124,1,124, - 2,124,3,131,2,125,6,116,0,124,2,124,3,124,6,124, - 4,100,1,141,4,83,0,41,2,78,41,2,114,124,0,0, - 0,114,154,0,0,0,41,1,114,165,0,0,0,41,7,114, - 104,0,0,0,114,163,0,0,0,114,123,0,0,0,114,37, - 0,0,0,90,4,115,109,115,108,114,177,0,0,0,114,124, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,4,1,0,0,201,4,0,0,115,6,0,0,0, - 0,1,10,1,8,1,122,20,70,105,108,101,70,105,110,100, - 101,114,46,95,103,101,116,95,115,112,101,99,78,99,3,0, - 0,0,0,0,0,0,14,0,0,0,15,0,0,0,67,0, - 0,0,115,98,1,0,0,100,1,125,3,124,1,106,0,100, - 2,131,1,100,3,25,0,125,4,121,24,116,1,124,0,106, - 2,112,34,116,3,106,4,131,0,131,1,106,5,125,5,87, - 0,110,24,4,0,116,6,107,10,114,66,1,0,1,0,1, - 0,100,10,125,5,89,0,110,2,88,0,124,5,124,0,106, - 7,107,3,114,92,124,0,106,8,131,0,1,0,124,5,124, - 0,95,7,116,9,131,0,114,114,124,0,106,10,125,6,124, - 4,106,11,131,0,125,7,110,10,124,0,106,12,125,6,124, - 4,125,7,124,7,124,6,107,6,114,218,116,13,124,0,106, - 2,124,4,131,2,125,8,120,72,124,0,106,14,68,0,93, - 54,92,2,125,9,125,10,100,5,124,9,23,0,125,11,116, - 13,124,8,124,11,131,2,125,12,116,15,124,12,131,1,114, - 152,124,0,106,16,124,10,124,1,124,12,124,8,103,1,124, - 2,131,5,83,0,113,152,87,0,116,17,124,8,131,1,125, - 3,120,88,124,0,106,14,68,0,93,78,92,2,125,9,125, - 10,116,13,124,0,106,2,124,4,124,9,23,0,131,2,125, - 12,116,18,106,19,100,6,124,12,100,3,100,7,141,3,1, - 0,124,7,124,9,23,0,124,6,107,6,114,226,116,15,124, - 12,131,1,114,226,124,0,106,16,124,10,124,1,124,12,100, - 8,124,2,131,5,83,0,113,226,87,0,124,3,144,1,114, - 94,116,18,106,19,100,9,124,8,131,2,1,0,116,18,106, - 20,124,1,100,8,131,2,125,13,124,8,103,1,124,13,95, - 21,124,13,83,0,100,8,83,0,41,11,122,111,84,114,121, - 32,116,111,32,102,105,110,100,32,97,32,115,112,101,99,32, - 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, - 100,32,109,111,100,117,108,101,46,10,10,32,32,32,32,32, - 32,32,32,82,101,116,117,114,110,115,32,116,104,101,32,109, - 97,116,99,104,105,110,103,32,115,112,101,99,44,32,111,114, - 32,78,111,110,101,32,105,102,32,110,111,116,32,102,111,117, - 110,100,46,10,32,32,32,32,32,32,32,32,70,114,61,0, - 0,0,114,59,0,0,0,114,31,0,0,0,114,182,0,0, - 0,122,9,116,114,121,105,110,103,32,123,125,41,1,90,9, - 118,101,114,98,111,115,105,116,121,78,122,25,112,111,115,115, - 105,98,108,101,32,110,97,109,101,115,112,97,99,101,32,102, - 111,114,32,123,125,114,91,0,0,0,41,22,114,34,0,0, - 0,114,41,0,0,0,114,37,0,0,0,114,3,0,0,0, - 114,47,0,0,0,114,216,0,0,0,114,42,0,0,0,114, - 7,1,0,0,218,11,95,102,105,108,108,95,99,97,99,104, - 101,114,7,0,0,0,114,10,1,0,0,114,92,0,0,0, - 114,9,1,0,0,114,30,0,0,0,114,6,1,0,0,114, - 46,0,0,0,114,4,1,0,0,114,48,0,0,0,114,118, - 0,0,0,114,133,0,0,0,114,158,0,0,0,114,154,0, - 0,0,41,14,114,104,0,0,0,114,123,0,0,0,114,177, - 0,0,0,90,12,105,115,95,110,97,109,101,115,112,97,99, - 101,90,11,116,97,105,108,95,109,111,100,117,108,101,114,130, - 0,0,0,90,5,99,97,99,104,101,90,12,99,97,99,104, - 101,95,109,111,100,117,108,101,90,9,98,97,115,101,95,112, - 97,116,104,114,222,0,0,0,114,163,0,0,0,90,13,105, - 110,105,116,95,102,105,108,101,110,97,109,101,90,9,102,117, - 108,108,95,112,97,116,104,114,162,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,178,0,0,0, - 206,4,0,0,115,70,0,0,0,0,5,4,1,14,1,2, - 1,24,1,14,1,10,1,10,1,8,1,6,2,6,1,6, - 1,10,2,6,1,4,2,8,1,12,1,16,1,8,1,10, - 1,8,1,24,4,8,2,16,1,16,1,16,1,12,1,8, - 1,10,1,12,1,6,1,12,1,12,1,8,1,4,1,122, - 20,70,105,108,101,70,105,110,100,101,114,46,102,105,110,100, - 95,115,112,101,99,99,1,0,0,0,0,0,0,0,9,0, - 0,0,13,0,0,0,67,0,0,0,115,194,0,0,0,124, - 0,106,0,125,1,121,22,116,1,106,2,124,1,112,22,116, - 1,106,3,131,0,131,1,125,2,87,0,110,30,4,0,116, - 4,116,5,116,6,102,3,107,10,114,58,1,0,1,0,1, - 0,103,0,125,2,89,0,110,2,88,0,116,7,106,8,106, - 9,100,1,131,1,115,84,116,10,124,2,131,1,124,0,95, - 11,110,78,116,10,131,0,125,3,120,64,124,2,68,0,93, - 56,125,4,124,4,106,12,100,2,131,1,92,3,125,5,125, - 6,125,7,124,6,114,138,100,3,106,13,124,5,124,7,106, - 14,131,0,131,2,125,8,110,4,124,5,125,8,124,3,106, - 15,124,8,131,1,1,0,113,96,87,0,124,3,124,0,95, - 11,116,7,106,8,106,9,116,16,131,1,114,190,100,4,100, - 5,132,0,124,2,68,0,131,1,124,0,95,17,100,6,83, - 0,41,7,122,68,70,105,108,108,32,116,104,101,32,99,97, - 99,104,101,32,111,102,32,112,111,116,101,110,116,105,97,108, - 32,109,111,100,117,108,101,115,32,97,110,100,32,112,97,99, - 107,97,103,101,115,32,102,111,114,32,116,104,105,115,32,100, - 105,114,101,99,116,111,114,121,46,114,0,0,0,0,114,61, - 0,0,0,122,5,123,125,46,123,125,99,1,0,0,0,0, - 0,0,0,2,0,0,0,3,0,0,0,83,0,0,0,115, - 20,0,0,0,104,0,124,0,93,12,125,1,124,1,106,0, - 131,0,146,2,113,4,83,0,114,4,0,0,0,41,1,114, - 92,0,0,0,41,2,114,24,0,0,0,90,2,102,110,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,250,9, - 60,115,101,116,99,111,109,112,62,27,5,0,0,115,2,0, - 0,0,6,0,122,41,70,105,108,101,70,105,110,100,101,114, - 46,95,102,105,108,108,95,99,97,99,104,101,46,60,108,111, - 99,97,108,115,62,46,60,115,101,116,99,111,109,112,62,78, - 41,18,114,37,0,0,0,114,3,0,0,0,90,7,108,105, - 115,116,100,105,114,114,47,0,0,0,114,255,0,0,0,218, - 15,80,101,114,109,105,115,115,105,111,110,69,114,114,111,114, - 218,18,78,111,116,65,68,105,114,101,99,116,111,114,121,69, - 114,114,111,114,114,8,0,0,0,114,9,0,0,0,114,10, - 0,0,0,114,8,1,0,0,114,9,1,0,0,114,87,0, - 0,0,114,50,0,0,0,114,92,0,0,0,218,3,97,100, - 100,114,11,0,0,0,114,10,1,0,0,41,9,114,104,0, - 0,0,114,37,0,0,0,90,8,99,111,110,116,101,110,116, - 115,90,21,108,111,119,101,114,95,115,117,102,102,105,120,95, - 99,111,110,116,101,110,116,115,114,244,0,0,0,114,102,0, - 0,0,114,234,0,0,0,114,222,0,0,0,90,8,110,101, - 119,95,110,97,109,101,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,12,1,0,0,254,4,0,0,115,34, - 0,0,0,0,2,6,1,2,1,22,1,20,3,10,3,12, - 1,12,7,6,1,10,1,16,1,4,1,18,2,4,1,14, - 1,6,1,12,1,122,22,70,105,108,101,70,105,110,100,101, - 114,46,95,102,105,108,108,95,99,97,99,104,101,99,1,0, - 0,0,0,0,0,0,3,0,0,0,3,0,0,0,7,0, - 0,0,115,18,0,0,0,135,0,135,1,102,2,100,1,100, - 2,132,8,125,2,124,2,83,0,41,3,97,20,1,0,0, - 65,32,99,108,97,115,115,32,109,101,116,104,111,100,32,119, - 104,105,99,104,32,114,101,116,117,114,110,115,32,97,32,99, - 108,111,115,117,114,101,32,116,111,32,117,115,101,32,111,110, - 32,115,121,115,46,112,97,116,104,95,104,111,111,107,10,32, - 32,32,32,32,32,32,32,119,104,105,99,104,32,119,105,108, - 108,32,114,101,116,117,114,110,32,97,110,32,105,110,115,116, - 97,110,99,101,32,117,115,105,110,103,32,116,104,101,32,115, - 112,101,99,105,102,105,101,100,32,108,111,97,100,101,114,115, - 32,97,110,100,32,116,104,101,32,112,97,116,104,10,32,32, - 32,32,32,32,32,32,99,97,108,108,101,100,32,111,110,32, - 116,104,101,32,99,108,111,115,117,114,101,46,10,10,32,32, - 32,32,32,32,32,32,73,102,32,116,104,101,32,112,97,116, - 104,32,99,97,108,108,101,100,32,111,110,32,116,104,101,32, - 99,108,111,115,117,114,101,32,105,115,32,110,111,116,32,97, - 32,100,105,114,101,99,116,111,114,121,44,32,73,109,112,111, - 114,116,69,114,114,111,114,32,105,115,10,32,32,32,32,32, - 32,32,32,114,97,105,115,101,100,46,10,10,32,32,32,32, - 32,32,32,32,99,1,0,0,0,0,0,0,0,1,0,0, - 0,4,0,0,0,19,0,0,0,115,34,0,0,0,116,0, - 124,0,131,1,115,20,116,1,100,1,124,0,100,2,141,2, - 130,1,136,0,124,0,102,1,136,1,152,2,142,0,83,0, - 41,3,122,45,80,97,116,104,32,104,111,111,107,32,102,111, - 114,32,105,109,112,111,114,116,108,105,98,46,109,97,99,104, - 105,110,101,114,121,46,70,105,108,101,70,105,110,100,101,114, - 46,122,30,111,110,108,121,32,100,105,114,101,99,116,111,114, - 105,101,115,32,97,114,101,32,115,117,112,112,111,114,116,101, - 100,41,1,114,37,0,0,0,41,2,114,48,0,0,0,114, - 103,0,0,0,41,1,114,37,0,0,0,41,2,114,168,0, - 0,0,114,11,1,0,0,114,4,0,0,0,114,6,0,0, - 0,218,24,112,97,116,104,95,104,111,111,107,95,102,111,114, - 95,70,105,108,101,70,105,110,100,101,114,39,5,0,0,115, - 6,0,0,0,0,2,8,1,12,1,122,54,70,105,108,101, + 114,6,0,0,0,114,179,0,0,0,147,4,0,0,115,8, + 0,0,0,0,8,12,1,8,1,4,1,122,22,80,97,116, + 104,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, + 117,108,101,41,1,78,41,2,78,78,41,1,78,41,12,114, + 109,0,0,0,114,108,0,0,0,114,110,0,0,0,114,111, + 0,0,0,114,180,0,0,0,114,249,0,0,0,114,254,0, + 0,0,114,0,1,0,0,114,1,1,0,0,114,4,1,0, + 0,114,178,0,0,0,114,179,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, + 248,0,0,0,29,4,0,0,115,22,0,0,0,8,2,4, + 2,12,8,12,13,12,22,12,15,2,1,12,31,2,1,12, + 23,2,1,114,248,0,0,0,99,0,0,0,0,0,0,0, + 0,0,0,0,0,3,0,0,0,64,0,0,0,115,90,0, + 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, + 100,3,132,0,90,4,100,4,100,5,132,0,90,5,101,6, + 90,7,100,6,100,7,132,0,90,8,100,8,100,9,132,0, + 90,9,100,19,100,11,100,12,132,1,90,10,100,13,100,14, + 132,0,90,11,101,12,100,15,100,16,132,0,131,1,90,13, + 100,17,100,18,132,0,90,14,100,10,83,0,41,20,218,10, + 70,105,108,101,70,105,110,100,101,114,122,172,70,105,108,101, + 45,98,97,115,101,100,32,102,105,110,100,101,114,46,10,10, + 32,32,32,32,73,110,116,101,114,97,99,116,105,111,110,115, + 32,119,105,116,104,32,116,104,101,32,102,105,108,101,32,115, + 121,115,116,101,109,32,97,114,101,32,99,97,99,104,101,100, + 32,102,111,114,32,112,101,114,102,111,114,109,97,110,99,101, + 44,32,98,101,105,110,103,10,32,32,32,32,114,101,102,114, + 101,115,104,101,100,32,119,104,101,110,32,116,104,101,32,100, + 105,114,101,99,116,111,114,121,32,116,104,101,32,102,105,110, + 100,101,114,32,105,115,32,104,97,110,100,108,105,110,103,32, + 104,97,115,32,98,101,101,110,32,109,111,100,105,102,105,101, + 100,46,10,10,32,32,32,32,99,2,0,0,0,0,0,0, + 0,5,0,0,0,5,0,0,0,7,0,0,0,115,88,0, + 0,0,103,0,125,3,120,40,124,2,68,0,93,32,92,2, + 137,0,125,4,124,3,106,0,135,0,102,1,100,1,100,2, + 132,8,124,4,68,0,131,1,131,1,1,0,113,10,87,0, + 124,3,124,0,95,1,124,1,112,58,100,3,124,0,95,2, + 100,6,124,0,95,3,116,4,131,0,124,0,95,5,116,4, + 131,0,124,0,95,6,100,5,83,0,41,7,122,154,73,110, + 105,116,105,97,108,105,122,101,32,119,105,116,104,32,116,104, + 101,32,112,97,116,104,32,116,111,32,115,101,97,114,99,104, + 32,111,110,32,97,110,100,32,97,32,118,97,114,105,97,98, + 108,101,32,110,117,109,98,101,114,32,111,102,10,32,32,32, + 32,32,32,32,32,50,45,116,117,112,108,101,115,32,99,111, + 110,116,97,105,110,105,110,103,32,116,104,101,32,108,111,97, + 100,101,114,32,97,110,100,32,116,104,101,32,102,105,108,101, + 32,115,117,102,102,105,120,101,115,32,116,104,101,32,108,111, + 97,100,101,114,10,32,32,32,32,32,32,32,32,114,101,99, + 111,103,110,105,122,101,115,46,99,1,0,0,0,0,0,0, + 0,2,0,0,0,3,0,0,0,51,0,0,0,115,22,0, + 0,0,124,0,93,14,125,1,124,1,136,0,102,2,86,0, + 1,0,113,2,100,0,83,0,41,1,78,114,4,0,0,0, + 41,2,114,24,0,0,0,114,222,0,0,0,41,1,114,124, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,224,0, + 0,0,176,4,0,0,115,2,0,0,0,4,0,122,38,70, + 105,108,101,70,105,110,100,101,114,46,95,95,105,110,105,116, + 95,95,46,60,108,111,99,97,108,115,62,46,60,103,101,110, + 101,120,112,114,62,114,61,0,0,0,114,31,0,0,0,78, + 114,91,0,0,0,41,7,114,147,0,0,0,218,8,95,108, + 111,97,100,101,114,115,114,37,0,0,0,218,11,95,112,97, + 116,104,95,109,116,105,109,101,218,3,115,101,116,218,11,95, + 112,97,116,104,95,99,97,99,104,101,218,19,95,114,101,108, + 97,120,101,100,95,112,97,116,104,95,99,97,99,104,101,41, + 5,114,104,0,0,0,114,37,0,0,0,218,14,108,111,97, + 100,101,114,95,100,101,116,97,105,108,115,90,7,108,111,97, + 100,101,114,115,114,164,0,0,0,114,4,0,0,0,41,1, + 114,124,0,0,0,114,6,0,0,0,114,182,0,0,0,170, + 4,0,0,115,16,0,0,0,0,4,4,1,14,1,28,1, + 6,2,10,1,6,1,8,1,122,19,70,105,108,101,70,105, + 110,100,101,114,46,95,95,105,110,105,116,95,95,99,1,0, + 0,0,0,0,0,0,1,0,0,0,2,0,0,0,67,0, + 0,0,115,10,0,0,0,100,3,124,0,95,0,100,2,83, + 0,41,4,122,31,73,110,118,97,108,105,100,97,116,101,32, + 116,104,101,32,100,105,114,101,99,116,111,114,121,32,109,116, + 105,109,101,46,114,31,0,0,0,78,114,91,0,0,0,41, + 1,114,7,1,0,0,41,1,114,104,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, + 0,184,4,0,0,115,2,0,0,0,0,2,122,28,70,105, + 108,101,70,105,110,100,101,114,46,105,110,118,97,108,105,100, + 97,116,101,95,99,97,99,104,101,115,99,2,0,0,0,0, + 0,0,0,3,0,0,0,2,0,0,0,67,0,0,0,115, + 42,0,0,0,124,0,106,0,124,1,131,1,125,2,124,2, + 100,1,107,8,114,26,100,1,103,0,102,2,83,0,124,2, + 106,1,124,2,106,2,112,38,103,0,102,2,83,0,41,2, + 122,197,84,114,121,32,116,111,32,102,105,110,100,32,97,32, + 108,111,97,100,101,114,32,102,111,114,32,116,104,101,32,115, + 112,101,99,105,102,105,101,100,32,109,111,100,117,108,101,44, + 32,111,114,32,116,104,101,32,110,97,109,101,115,112,97,99, + 101,10,32,32,32,32,32,32,32,32,112,97,99,107,97,103, + 101,32,112,111,114,116,105,111,110,115,46,32,82,101,116,117, + 114,110,115,32,40,108,111,97,100,101,114,44,32,108,105,115, + 116,45,111,102,45,112,111,114,116,105,111,110,115,41,46,10, + 10,32,32,32,32,32,32,32,32,84,104,105,115,32,109,101, + 116,104,111,100,32,105,115,32,100,101,112,114,101,99,97,116, + 101,100,46,32,32,85,115,101,32,102,105,110,100,95,115,112, + 101,99,40,41,32,105,110,115,116,101,97,100,46,10,10,32, + 32,32,32,32,32,32,32,78,41,3,114,178,0,0,0,114, + 124,0,0,0,114,154,0,0,0,41,3,114,104,0,0,0, + 114,123,0,0,0,114,162,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,121,0,0,0,190,4, + 0,0,115,8,0,0,0,0,7,10,1,8,1,8,1,122, + 22,70,105,108,101,70,105,110,100,101,114,46,102,105,110,100, + 95,108,111,97,100,101,114,99,6,0,0,0,0,0,0,0, + 7,0,0,0,6,0,0,0,67,0,0,0,115,26,0,0, + 0,124,1,124,2,124,3,131,2,125,6,116,0,124,2,124, + 3,124,6,124,4,100,1,141,4,83,0,41,2,78,41,2, + 114,124,0,0,0,114,154,0,0,0,41,1,114,165,0,0, + 0,41,7,114,104,0,0,0,114,163,0,0,0,114,123,0, + 0,0,114,37,0,0,0,90,4,115,109,115,108,114,177,0, + 0,0,114,124,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,114,4,1,0,0,202,4,0,0,115, + 6,0,0,0,0,1,10,1,8,1,122,20,70,105,108,101, + 70,105,110,100,101,114,46,95,103,101,116,95,115,112,101,99, + 78,99,3,0,0,0,0,0,0,0,14,0,0,0,15,0, + 0,0,67,0,0,0,115,98,1,0,0,100,1,125,3,124, + 1,106,0,100,2,131,1,100,3,25,0,125,4,121,24,116, + 1,124,0,106,2,112,34,116,3,106,4,131,0,131,1,106, + 5,125,5,87,0,110,24,4,0,116,6,107,10,114,66,1, + 0,1,0,1,0,100,10,125,5,89,0,110,2,88,0,124, + 5,124,0,106,7,107,3,114,92,124,0,106,8,131,0,1, + 0,124,5,124,0,95,7,116,9,131,0,114,114,124,0,106, + 10,125,6,124,4,106,11,131,0,125,7,110,10,124,0,106, + 12,125,6,124,4,125,7,124,7,124,6,107,6,114,218,116, + 13,124,0,106,2,124,4,131,2,125,8,120,72,124,0,106, + 14,68,0,93,54,92,2,125,9,125,10,100,5,124,9,23, + 0,125,11,116,13,124,8,124,11,131,2,125,12,116,15,124, + 12,131,1,114,152,124,0,106,16,124,10,124,1,124,12,124, + 8,103,1,124,2,131,5,83,0,113,152,87,0,116,17,124, + 8,131,1,125,3,120,88,124,0,106,14,68,0,93,78,92, + 2,125,9,125,10,116,13,124,0,106,2,124,4,124,9,23, + 0,131,2,125,12,116,18,106,19,100,6,124,12,100,3,100, + 7,141,3,1,0,124,7,124,9,23,0,124,6,107,6,114, + 226,116,15,124,12,131,1,114,226,124,0,106,16,124,10,124, + 1,124,12,100,8,124,2,131,5,83,0,113,226,87,0,124, + 3,144,1,114,94,116,18,106,19,100,9,124,8,131,2,1, + 0,116,18,106,20,124,1,100,8,131,2,125,13,124,8,103, + 1,124,13,95,21,124,13,83,0,100,8,83,0,41,11,122, + 111,84,114,121,32,116,111,32,102,105,110,100,32,97,32,115, + 112,101,99,32,102,111,114,32,116,104,101,32,115,112,101,99, + 105,102,105,101,100,32,109,111,100,117,108,101,46,10,10,32, + 32,32,32,32,32,32,32,82,101,116,117,114,110,115,32,116, + 104,101,32,109,97,116,99,104,105,110,103,32,115,112,101,99, + 44,32,111,114,32,78,111,110,101,32,105,102,32,110,111,116, + 32,102,111,117,110,100,46,10,32,32,32,32,32,32,32,32, + 70,114,61,0,0,0,114,59,0,0,0,114,31,0,0,0, + 114,182,0,0,0,122,9,116,114,121,105,110,103,32,123,125, + 41,1,90,9,118,101,114,98,111,115,105,116,121,78,122,25, + 112,111,115,115,105,98,108,101,32,110,97,109,101,115,112,97, + 99,101,32,102,111,114,32,123,125,114,91,0,0,0,41,22, + 114,34,0,0,0,114,41,0,0,0,114,37,0,0,0,114, + 3,0,0,0,114,47,0,0,0,114,216,0,0,0,114,42, + 0,0,0,114,7,1,0,0,218,11,95,102,105,108,108,95, + 99,97,99,104,101,114,7,0,0,0,114,10,1,0,0,114, + 92,0,0,0,114,9,1,0,0,114,30,0,0,0,114,6, + 1,0,0,114,46,0,0,0,114,4,1,0,0,114,48,0, + 0,0,114,118,0,0,0,114,133,0,0,0,114,158,0,0, + 0,114,154,0,0,0,41,14,114,104,0,0,0,114,123,0, + 0,0,114,177,0,0,0,90,12,105,115,95,110,97,109,101, + 115,112,97,99,101,90,11,116,97,105,108,95,109,111,100,117, + 108,101,114,130,0,0,0,90,5,99,97,99,104,101,90,12, + 99,97,99,104,101,95,109,111,100,117,108,101,90,9,98,97, + 115,101,95,112,97,116,104,114,222,0,0,0,114,163,0,0, + 0,90,13,105,110,105,116,95,102,105,108,101,110,97,109,101, + 90,9,102,117,108,108,95,112,97,116,104,114,162,0,0,0, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, + 178,0,0,0,207,4,0,0,115,70,0,0,0,0,5,4, + 1,14,1,2,1,24,1,14,1,10,1,10,1,8,1,6, + 2,6,1,6,1,10,2,6,1,4,2,8,1,12,1,16, + 1,8,1,10,1,8,1,24,4,8,2,16,1,16,1,16, + 1,12,1,8,1,10,1,12,1,6,1,12,1,12,1,8, + 1,4,1,122,20,70,105,108,101,70,105,110,100,101,114,46, + 102,105,110,100,95,115,112,101,99,99,1,0,0,0,0,0, + 0,0,9,0,0,0,13,0,0,0,67,0,0,0,115,194, + 0,0,0,124,0,106,0,125,1,121,22,116,1,106,2,124, + 1,112,22,116,1,106,3,131,0,131,1,125,2,87,0,110, + 30,4,0,116,4,116,5,116,6,102,3,107,10,114,58,1, + 0,1,0,1,0,103,0,125,2,89,0,110,2,88,0,116, + 7,106,8,106,9,100,1,131,1,115,84,116,10,124,2,131, + 1,124,0,95,11,110,78,116,10,131,0,125,3,120,64,124, + 2,68,0,93,56,125,4,124,4,106,12,100,2,131,1,92, + 3,125,5,125,6,125,7,124,6,114,138,100,3,106,13,124, + 5,124,7,106,14,131,0,131,2,125,8,110,4,124,5,125, + 8,124,3,106,15,124,8,131,1,1,0,113,96,87,0,124, + 3,124,0,95,11,116,7,106,8,106,9,116,16,131,1,114, + 190,100,4,100,5,132,0,124,2,68,0,131,1,124,0,95, + 17,100,6,83,0,41,7,122,68,70,105,108,108,32,116,104, + 101,32,99,97,99,104,101,32,111,102,32,112,111,116,101,110, + 116,105,97,108,32,109,111,100,117,108,101,115,32,97,110,100, + 32,112,97,99,107,97,103,101,115,32,102,111,114,32,116,104, + 105,115,32,100,105,114,101,99,116,111,114,121,46,114,0,0, + 0,0,114,61,0,0,0,122,5,123,125,46,123,125,99,1, + 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,83, + 0,0,0,115,20,0,0,0,104,0,124,0,93,12,125,1, + 124,1,106,0,131,0,146,2,113,4,83,0,114,4,0,0, + 0,41,1,114,92,0,0,0,41,2,114,24,0,0,0,90, + 2,102,110,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,250,9,60,115,101,116,99,111,109,112,62,28,5,0, + 0,115,2,0,0,0,6,0,122,41,70,105,108,101,70,105, + 110,100,101,114,46,95,102,105,108,108,95,99,97,99,104,101, + 46,60,108,111,99,97,108,115,62,46,60,115,101,116,99,111, + 109,112,62,78,41,18,114,37,0,0,0,114,3,0,0,0, + 90,7,108,105,115,116,100,105,114,114,47,0,0,0,114,255, + 0,0,0,218,15,80,101,114,109,105,115,115,105,111,110,69, + 114,114,111,114,218,18,78,111,116,65,68,105,114,101,99,116, + 111,114,121,69,114,114,111,114,114,8,0,0,0,114,9,0, + 0,0,114,10,0,0,0,114,8,1,0,0,114,9,1,0, + 0,114,87,0,0,0,114,50,0,0,0,114,92,0,0,0, + 218,3,97,100,100,114,11,0,0,0,114,10,1,0,0,41, + 9,114,104,0,0,0,114,37,0,0,0,90,8,99,111,110, + 116,101,110,116,115,90,21,108,111,119,101,114,95,115,117,102, + 102,105,120,95,99,111,110,116,101,110,116,115,114,244,0,0, + 0,114,102,0,0,0,114,234,0,0,0,114,222,0,0,0, + 90,8,110,101,119,95,110,97,109,101,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,12,1,0,0,255,4, + 0,0,115,34,0,0,0,0,2,6,1,2,1,22,1,20, + 3,10,3,12,1,12,7,6,1,10,1,16,1,4,1,18, + 2,4,1,14,1,6,1,12,1,122,22,70,105,108,101,70, + 105,110,100,101,114,46,95,102,105,108,108,95,99,97,99,104, + 101,99,1,0,0,0,0,0,0,0,3,0,0,0,3,0, + 0,0,7,0,0,0,115,18,0,0,0,135,0,135,1,102, + 2,100,1,100,2,132,8,125,2,124,2,83,0,41,3,97, + 20,1,0,0,65,32,99,108,97,115,115,32,109,101,116,104, + 111,100,32,119,104,105,99,104,32,114,101,116,117,114,110,115, + 32,97,32,99,108,111,115,117,114,101,32,116,111,32,117,115, + 101,32,111,110,32,115,121,115,46,112,97,116,104,95,104,111, + 111,107,10,32,32,32,32,32,32,32,32,119,104,105,99,104, + 32,119,105,108,108,32,114,101,116,117,114,110,32,97,110,32, + 105,110,115,116,97,110,99,101,32,117,115,105,110,103,32,116, + 104,101,32,115,112,101,99,105,102,105,101,100,32,108,111,97, + 100,101,114,115,32,97,110,100,32,116,104,101,32,112,97,116, + 104,10,32,32,32,32,32,32,32,32,99,97,108,108,101,100, + 32,111,110,32,116,104,101,32,99,108,111,115,117,114,101,46, + 10,10,32,32,32,32,32,32,32,32,73,102,32,116,104,101, + 32,112,97,116,104,32,99,97,108,108,101,100,32,111,110,32, + 116,104,101,32,99,108,111,115,117,114,101,32,105,115,32,110, + 111,116,32,97,32,100,105,114,101,99,116,111,114,121,44,32, + 73,109,112,111,114,116,69,114,114,111,114,32,105,115,10,32, + 32,32,32,32,32,32,32,114,97,105,115,101,100,46,10,10, + 32,32,32,32,32,32,32,32,99,1,0,0,0,0,0,0, + 0,1,0,0,0,4,0,0,0,19,0,0,0,115,34,0, + 0,0,116,0,124,0,131,1,115,20,116,1,100,1,124,0, + 100,2,141,2,130,1,136,0,124,0,102,1,136,1,152,2, + 142,0,83,0,41,3,122,45,80,97,116,104,32,104,111,111, + 107,32,102,111,114,32,105,109,112,111,114,116,108,105,98,46, + 109,97,99,104,105,110,101,114,121,46,70,105,108,101,70,105, + 110,100,101,114,46,122,30,111,110,108,121,32,100,105,114,101, + 99,116,111,114,105,101,115,32,97,114,101,32,115,117,112,112, + 111,114,116,101,100,41,1,114,37,0,0,0,41,2,114,48, + 0,0,0,114,103,0,0,0,41,1,114,37,0,0,0,41, + 2,114,168,0,0,0,114,11,1,0,0,114,4,0,0,0, + 114,6,0,0,0,218,24,112,97,116,104,95,104,111,111,107, + 95,102,111,114,95,70,105,108,101,70,105,110,100,101,114,40, + 5,0,0,115,6,0,0,0,0,2,8,1,12,1,122,54, + 70,105,108,101,70,105,110,100,101,114,46,112,97,116,104,95, + 104,111,111,107,46,60,108,111,99,97,108,115,62,46,112,97, + 116,104,95,104,111,111,107,95,102,111,114,95,70,105,108,101, + 70,105,110,100,101,114,114,4,0,0,0,41,3,114,168,0, + 0,0,114,11,1,0,0,114,17,1,0,0,114,4,0,0, + 0,41,2,114,168,0,0,0,114,11,1,0,0,114,6,0, + 0,0,218,9,112,97,116,104,95,104,111,111,107,30,5,0, + 0,115,4,0,0,0,0,10,14,6,122,20,70,105,108,101, 70,105,110,100,101,114,46,112,97,116,104,95,104,111,111,107, - 46,60,108,111,99,97,108,115,62,46,112,97,116,104,95,104, - 111,111,107,95,102,111,114,95,70,105,108,101,70,105,110,100, - 101,114,114,4,0,0,0,41,3,114,168,0,0,0,114,11, - 1,0,0,114,17,1,0,0,114,4,0,0,0,41,2,114, - 168,0,0,0,114,11,1,0,0,114,6,0,0,0,218,9, - 112,97,116,104,95,104,111,111,107,29,5,0,0,115,4,0, - 0,0,0,10,14,6,122,20,70,105,108,101,70,105,110,100, - 101,114,46,112,97,116,104,95,104,111,111,107,99,1,0,0, - 0,0,0,0,0,1,0,0,0,2,0,0,0,67,0,0, - 0,115,12,0,0,0,100,1,106,0,124,0,106,1,131,1, - 83,0,41,2,78,122,16,70,105,108,101,70,105,110,100,101, - 114,40,123,33,114,125,41,41,2,114,50,0,0,0,114,37, - 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,243,0,0,0,47,5, - 0,0,115,2,0,0,0,0,1,122,19,70,105,108,101,70, - 105,110,100,101,114,46,95,95,114,101,112,114,95,95,41,1, - 78,41,15,114,109,0,0,0,114,108,0,0,0,114,110,0, - 0,0,114,111,0,0,0,114,182,0,0,0,114,249,0,0, - 0,114,127,0,0,0,114,179,0,0,0,114,121,0,0,0, - 114,4,1,0,0,114,178,0,0,0,114,12,1,0,0,114, - 180,0,0,0,114,18,1,0,0,114,243,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,5,1,0,0,160,4,0,0,115,20,0,0,0, - 8,7,4,2,8,14,8,4,4,2,8,12,8,5,10,48, - 8,31,12,18,114,5,1,0,0,99,4,0,0,0,0,0, - 0,0,6,0,0,0,11,0,0,0,67,0,0,0,115,146, - 0,0,0,124,0,106,0,100,1,131,1,125,4,124,0,106, - 0,100,2,131,1,125,5,124,4,115,66,124,5,114,36,124, - 5,106,1,125,4,110,30,124,2,124,3,107,2,114,56,116, - 2,124,1,124,2,131,2,125,4,110,10,116,3,124,1,124, - 2,131,2,125,4,124,5,115,84,116,4,124,1,124,2,124, - 4,100,3,141,3,125,5,121,36,124,5,124,0,100,2,60, - 0,124,4,124,0,100,1,60,0,124,2,124,0,100,4,60, - 0,124,3,124,0,100,5,60,0,87,0,110,20,4,0,116, - 5,107,10,114,140,1,0,1,0,1,0,89,0,110,2,88, - 0,100,0,83,0,41,6,78,218,10,95,95,108,111,97,100, - 101,114,95,95,218,8,95,95,115,112,101,99,95,95,41,1, - 114,124,0,0,0,90,8,95,95,102,105,108,101,95,95,90, - 10,95,95,99,97,99,104,101,100,95,95,41,6,218,3,103, - 101,116,114,124,0,0,0,114,220,0,0,0,114,215,0,0, - 0,114,165,0,0,0,218,9,69,120,99,101,112,116,105,111, - 110,41,6,90,2,110,115,114,102,0,0,0,90,8,112,97, - 116,104,110,97,109,101,90,9,99,112,97,116,104,110,97,109, - 101,114,124,0,0,0,114,162,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,14,95,102,105,120, - 95,117,112,95,109,111,100,117,108,101,53,5,0,0,115,34, - 0,0,0,0,2,10,1,10,1,4,1,4,1,8,1,8, - 1,12,2,10,1,4,1,14,1,2,1,8,1,8,1,8, - 1,12,1,14,2,114,23,1,0,0,99,0,0,0,0,0, - 0,0,0,3,0,0,0,3,0,0,0,67,0,0,0,115, - 38,0,0,0,116,0,116,1,106,2,131,0,102,2,125,0, - 116,3,116,4,102,2,125,1,116,5,116,6,102,2,125,2, - 124,0,124,1,124,2,103,3,83,0,41,1,122,95,82,101, - 116,117,114,110,115,32,97,32,108,105,115,116,32,111,102,32, - 102,105,108,101,45,98,97,115,101,100,32,109,111,100,117,108, - 101,32,108,111,97,100,101,114,115,46,10,10,32,32,32,32, - 69,97,99,104,32,105,116,101,109,32,105,115,32,97,32,116, - 117,112,108,101,32,40,108,111,97,100,101,114,44,32,115,117, - 102,102,105,120,101,115,41,46,10,32,32,32,32,41,7,114, - 221,0,0,0,114,143,0,0,0,218,18,101,120,116,101,110, - 115,105,111,110,95,115,117,102,102,105,120,101,115,114,215,0, - 0,0,114,88,0,0,0,114,220,0,0,0,114,78,0,0, - 0,41,3,90,10,101,120,116,101,110,115,105,111,110,115,90, - 6,115,111,117,114,99,101,90,8,98,121,116,101,99,111,100, - 101,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 114,159,0,0,0,76,5,0,0,115,8,0,0,0,0,5, - 12,1,8,1,8,1,114,159,0,0,0,99,1,0,0,0, - 0,0,0,0,12,0,0,0,12,0,0,0,67,0,0,0, - 115,188,1,0,0,124,0,97,0,116,0,106,1,97,1,116, - 0,106,2,97,2,116,1,106,3,116,4,25,0,125,1,120, - 56,100,26,68,0,93,48,125,2,124,2,116,1,106,3,107, - 7,114,58,116,0,106,5,124,2,131,1,125,3,110,10,116, - 1,106,3,124,2,25,0,125,3,116,6,124,1,124,2,124, - 3,131,3,1,0,113,32,87,0,100,5,100,6,103,1,102, - 2,100,7,100,8,100,6,103,2,102,2,102,2,125,4,120, - 118,124,4,68,0,93,102,92,2,125,5,125,6,116,7,100, - 9,100,10,132,0,124,6,68,0,131,1,131,1,115,142,116, - 8,130,1,124,6,100,11,25,0,125,7,124,5,116,1,106, - 3,107,6,114,174,116,1,106,3,124,5,25,0,125,8,80, - 0,113,112,121,16,116,0,106,5,124,5,131,1,125,8,80, - 0,87,0,113,112,4,0,116,9,107,10,114,212,1,0,1, - 0,1,0,119,112,89,0,113,112,88,0,113,112,87,0,116, - 9,100,12,131,1,130,1,116,6,124,1,100,13,124,8,131, - 3,1,0,116,6,124,1,100,14,124,7,131,3,1,0,116, - 6,124,1,100,15,100,16,106,10,124,6,131,1,131,3,1, - 0,121,14,116,0,106,5,100,17,131,1,125,9,87,0,110, - 26,4,0,116,9,107,10,144,1,114,52,1,0,1,0,1, - 0,100,18,125,9,89,0,110,2,88,0,116,6,124,1,100, - 17,124,9,131,3,1,0,116,0,106,5,100,19,131,1,125, - 10,116,6,124,1,100,19,124,10,131,3,1,0,124,5,100, - 7,107,2,144,1,114,120,116,0,106,5,100,20,131,1,125, - 11,116,6,124,1,100,21,124,11,131,3,1,0,116,6,124, - 1,100,22,116,11,131,0,131,3,1,0,116,12,106,13,116, - 2,106,14,131,0,131,1,1,0,124,5,100,7,107,2,144, - 1,114,184,116,15,106,16,100,23,131,1,1,0,100,24,116, - 12,107,6,144,1,114,184,100,25,116,17,95,18,100,18,83, - 0,41,27,122,205,83,101,116,117,112,32,116,104,101,32,112, - 97,116,104,45,98,97,115,101,100,32,105,109,112,111,114,116, - 101,114,115,32,102,111,114,32,105,109,112,111,114,116,108,105, - 98,32,98,121,32,105,109,112,111,114,116,105,110,103,32,110, - 101,101,100,101,100,10,32,32,32,32,98,117,105,108,116,45, - 105,110,32,109,111,100,117,108,101,115,32,97,110,100,32,105, - 110,106,101,99,116,105,110,103,32,116,104,101,109,32,105,110, - 116,111,32,116,104,101,32,103,108,111,98,97,108,32,110,97, - 109,101,115,112,97,99,101,46,10,10,32,32,32,32,79,116, - 104,101,114,32,99,111,109,112,111,110,101,110,116,115,32,97, - 114,101,32,101,120,116,114,97,99,116,101,100,32,102,114,111, - 109,32,116,104,101,32,99,111,114,101,32,98,111,111,116,115, - 116,114,97,112,32,109,111,100,117,108,101,46,10,10,32,32, - 32,32,114,52,0,0,0,114,63,0,0,0,218,8,98,117, - 105,108,116,105,110,115,114,140,0,0,0,90,5,112,111,115, - 105,120,250,1,47,218,2,110,116,250,1,92,99,1,0,0, - 0,0,0,0,0,2,0,0,0,3,0,0,0,115,0,0, - 0,115,26,0,0,0,124,0,93,18,125,1,116,0,124,1, - 131,1,100,0,107,2,86,0,1,0,113,2,100,1,83,0, - 41,2,114,31,0,0,0,78,41,1,114,33,0,0,0,41, - 2,114,24,0,0,0,114,81,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,224,0,0,0,112, - 5,0,0,115,2,0,0,0,4,0,122,25,95,115,101,116, - 117,112,46,60,108,111,99,97,108,115,62,46,60,103,101,110, - 101,120,112,114,62,114,62,0,0,0,122,30,105,109,112,111, - 114,116,108,105,98,32,114,101,113,117,105,114,101,115,32,112, - 111,115,105,120,32,111,114,32,110,116,114,3,0,0,0,114, - 27,0,0,0,114,23,0,0,0,114,32,0,0,0,90,7, - 95,116,104,114,101,97,100,78,90,8,95,119,101,97,107,114, - 101,102,90,6,119,105,110,114,101,103,114,167,0,0,0,114, - 7,0,0,0,122,4,46,112,121,119,122,6,95,100,46,112, - 121,100,84,41,4,122,3,95,105,111,122,9,95,119,97,114, - 110,105,110,103,115,122,8,98,117,105,108,116,105,110,115,122, - 7,109,97,114,115,104,97,108,41,19,114,118,0,0,0,114, - 8,0,0,0,114,143,0,0,0,114,236,0,0,0,114,109, - 0,0,0,90,18,95,98,117,105,108,116,105,110,95,102,114, - 111,109,95,110,97,109,101,114,113,0,0,0,218,3,97,108, - 108,218,14,65,115,115,101,114,116,105,111,110,69,114,114,111, - 114,114,103,0,0,0,114,28,0,0,0,114,13,0,0,0, - 114,226,0,0,0,114,147,0,0,0,114,24,1,0,0,114, - 88,0,0,0,114,161,0,0,0,114,166,0,0,0,114,170, - 0,0,0,41,12,218,17,95,98,111,111,116,115,116,114,97, - 112,95,109,111,100,117,108,101,90,11,115,101,108,102,95,109, - 111,100,117,108,101,90,12,98,117,105,108,116,105,110,95,110, - 97,109,101,90,14,98,117,105,108,116,105,110,95,109,111,100, - 117,108,101,90,10,111,115,95,100,101,116,97,105,108,115,90, - 10,98,117,105,108,116,105,110,95,111,115,114,23,0,0,0, - 114,27,0,0,0,90,9,111,115,95,109,111,100,117,108,101, - 90,13,116,104,114,101,97,100,95,109,111,100,117,108,101,90, - 14,119,101,97,107,114,101,102,95,109,111,100,117,108,101,90, - 13,119,105,110,114,101,103,95,109,111,100,117,108,101,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,6,95, - 115,101,116,117,112,87,5,0,0,115,82,0,0,0,0,8, - 4,1,6,1,6,3,10,1,10,1,10,1,12,2,10,1, - 16,3,22,1,14,2,22,1,8,1,10,1,10,1,4,2, - 2,1,10,1,6,1,14,1,12,2,8,1,12,1,12,1, - 18,3,2,1,14,1,16,2,10,1,12,3,10,1,12,3, - 10,1,10,1,12,3,14,1,14,1,10,1,10,1,10,1, - 114,32,1,0,0,99,1,0,0,0,0,0,0,0,2,0, - 0,0,3,0,0,0,67,0,0,0,115,74,0,0,0,116, - 0,124,0,131,1,1,0,116,1,131,0,125,1,116,2,106, - 3,106,4,116,5,106,6,124,1,152,1,142,0,103,1,131, - 1,1,0,116,7,106,8,100,1,107,2,114,58,116,2,106, - 9,106,10,116,11,131,1,1,0,116,2,106,9,106,10,116, - 12,131,1,1,0,100,2,83,0,41,3,122,41,73,110,115, - 116,97,108,108,32,116,104,101,32,112,97,116,104,45,98,97, - 115,101,100,32,105,109,112,111,114,116,32,99,111,109,112,111, - 110,101,110,116,115,46,114,27,1,0,0,78,41,13,114,32, - 1,0,0,114,159,0,0,0,114,8,0,0,0,114,253,0, - 0,0,114,147,0,0,0,114,5,1,0,0,114,18,1,0, - 0,114,3,0,0,0,114,109,0,0,0,218,9,109,101,116, - 97,95,112,97,116,104,114,161,0,0,0,114,166,0,0,0, - 114,248,0,0,0,41,2,114,31,1,0,0,90,17,115,117, - 112,112,111,114,116,101,100,95,108,111,97,100,101,114,115,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,8, - 95,105,110,115,116,97,108,108,155,5,0,0,115,12,0,0, - 0,0,2,8,1,6,1,22,1,10,1,12,1,114,34,1, - 0,0,41,1,122,3,119,105,110,41,2,114,1,0,0,0, - 114,2,0,0,0,41,1,114,49,0,0,0,41,1,78,41, - 3,78,78,78,41,3,78,78,78,41,2,114,62,0,0,0, - 114,62,0,0,0,41,1,78,41,1,78,41,58,114,111,0, - 0,0,114,12,0,0,0,90,37,95,67,65,83,69,95,73, - 78,83,69,78,83,73,84,73,86,69,95,80,76,65,84,70, - 79,82,77,83,95,66,89,84,69,83,95,75,69,89,114,11, - 0,0,0,114,13,0,0,0,114,19,0,0,0,114,21,0, - 0,0,114,30,0,0,0,114,40,0,0,0,114,41,0,0, - 0,114,45,0,0,0,114,46,0,0,0,114,48,0,0,0, - 114,58,0,0,0,218,4,116,121,112,101,218,8,95,95,99, - 111,100,101,95,95,114,142,0,0,0,114,17,0,0,0,114, - 132,0,0,0,114,16,0,0,0,114,20,0,0,0,90,17, - 95,82,65,87,95,77,65,71,73,67,95,78,85,77,66,69, - 82,114,77,0,0,0,114,76,0,0,0,114,88,0,0,0, - 114,78,0,0,0,90,23,68,69,66,85,71,95,66,89,84, - 69,67,79,68,69,95,83,85,70,70,73,88,69,83,90,27, - 79,80,84,73,77,73,90,69,68,95,66,89,84,69,67,79, - 68,69,95,83,85,70,70,73,88,69,83,114,83,0,0,0, - 114,89,0,0,0,114,95,0,0,0,114,99,0,0,0,114, - 101,0,0,0,114,120,0,0,0,114,127,0,0,0,114,139, - 0,0,0,114,145,0,0,0,114,148,0,0,0,114,153,0, - 0,0,218,6,111,98,106,101,99,116,114,160,0,0,0,114, - 165,0,0,0,114,166,0,0,0,114,181,0,0,0,114,191, - 0,0,0,114,207,0,0,0,114,215,0,0,0,114,220,0, - 0,0,114,226,0,0,0,114,221,0,0,0,114,227,0,0, - 0,114,246,0,0,0,114,248,0,0,0,114,5,1,0,0, - 114,23,1,0,0,114,159,0,0,0,114,32,1,0,0,114, - 34,1,0,0,114,4,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,8,60,109,111,100,117,108, - 101,62,8,0,0,0,115,108,0,0,0,4,16,4,1,4, - 1,2,1,6,3,8,17,8,5,8,5,8,6,8,12,8, - 10,8,9,8,5,8,7,10,22,10,120,16,1,12,2,4, - 1,4,2,6,2,6,2,8,2,16,45,8,34,8,19,8, - 12,8,12,8,28,8,17,10,55,10,12,10,10,8,14,6, - 3,4,1,14,67,14,64,14,29,16,110,14,41,18,45,18, - 16,4,3,18,53,14,60,14,42,14,127,0,5,14,127,0, - 22,10,23,8,11,8,68, + 99,1,0,0,0,0,0,0,0,1,0,0,0,2,0,0, + 0,67,0,0,0,115,12,0,0,0,100,1,106,0,124,0, + 106,1,131,1,83,0,41,2,78,122,16,70,105,108,101,70, + 105,110,100,101,114,40,123,33,114,125,41,41,2,114,50,0, + 0,0,114,37,0,0,0,41,1,114,104,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,243,0, + 0,0,48,5,0,0,115,2,0,0,0,0,1,122,19,70, + 105,108,101,70,105,110,100,101,114,46,95,95,114,101,112,114, + 95,95,41,1,78,41,15,114,109,0,0,0,114,108,0,0, + 0,114,110,0,0,0,114,111,0,0,0,114,182,0,0,0, + 114,249,0,0,0,114,127,0,0,0,114,179,0,0,0,114, + 121,0,0,0,114,4,1,0,0,114,178,0,0,0,114,12, + 1,0,0,114,180,0,0,0,114,18,1,0,0,114,243,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,114,5,1,0,0,161,4,0,0,115, + 20,0,0,0,8,7,4,2,8,14,8,4,4,2,8,12, + 8,5,10,48,8,31,12,18,114,5,1,0,0,99,4,0, + 0,0,0,0,0,0,6,0,0,0,11,0,0,0,67,0, + 0,0,115,146,0,0,0,124,0,106,0,100,1,131,1,125, + 4,124,0,106,0,100,2,131,1,125,5,124,4,115,66,124, + 5,114,36,124,5,106,1,125,4,110,30,124,2,124,3,107, + 2,114,56,116,2,124,1,124,2,131,2,125,4,110,10,116, + 3,124,1,124,2,131,2,125,4,124,5,115,84,116,4,124, + 1,124,2,124,4,100,3,141,3,125,5,121,36,124,5,124, + 0,100,2,60,0,124,4,124,0,100,1,60,0,124,2,124, + 0,100,4,60,0,124,3,124,0,100,5,60,0,87,0,110, + 20,4,0,116,5,107,10,114,140,1,0,1,0,1,0,89, + 0,110,2,88,0,100,0,83,0,41,6,78,218,10,95,95, + 108,111,97,100,101,114,95,95,218,8,95,95,115,112,101,99, + 95,95,41,1,114,124,0,0,0,90,8,95,95,102,105,108, + 101,95,95,90,10,95,95,99,97,99,104,101,100,95,95,41, + 6,218,3,103,101,116,114,124,0,0,0,114,220,0,0,0, + 114,215,0,0,0,114,165,0,0,0,218,9,69,120,99,101, + 112,116,105,111,110,41,6,90,2,110,115,114,102,0,0,0, + 90,8,112,97,116,104,110,97,109,101,90,9,99,112,97,116, + 104,110,97,109,101,114,124,0,0,0,114,162,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,14, + 95,102,105,120,95,117,112,95,109,111,100,117,108,101,54,5, + 0,0,115,34,0,0,0,0,2,10,1,10,1,4,1,4, + 1,8,1,8,1,12,2,10,1,4,1,14,1,2,1,8, + 1,8,1,8,1,12,1,14,2,114,23,1,0,0,99,0, + 0,0,0,0,0,0,0,3,0,0,0,3,0,0,0,67, + 0,0,0,115,38,0,0,0,116,0,116,1,106,2,131,0, + 102,2,125,0,116,3,116,4,102,2,125,1,116,5,116,6, + 102,2,125,2,124,0,124,1,124,2,103,3,83,0,41,1, + 122,95,82,101,116,117,114,110,115,32,97,32,108,105,115,116, + 32,111,102,32,102,105,108,101,45,98,97,115,101,100,32,109, + 111,100,117,108,101,32,108,111,97,100,101,114,115,46,10,10, + 32,32,32,32,69,97,99,104,32,105,116,101,109,32,105,115, + 32,97,32,116,117,112,108,101,32,40,108,111,97,100,101,114, + 44,32,115,117,102,102,105,120,101,115,41,46,10,32,32,32, + 32,41,7,114,221,0,0,0,114,143,0,0,0,218,18,101, + 120,116,101,110,115,105,111,110,95,115,117,102,102,105,120,101, + 115,114,215,0,0,0,114,88,0,0,0,114,220,0,0,0, + 114,78,0,0,0,41,3,90,10,101,120,116,101,110,115,105, + 111,110,115,90,6,115,111,117,114,99,101,90,8,98,121,116, + 101,99,111,100,101,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,159,0,0,0,77,5,0,0,115,8,0, + 0,0,0,5,12,1,8,1,8,1,114,159,0,0,0,99, + 1,0,0,0,0,0,0,0,12,0,0,0,12,0,0,0, + 67,0,0,0,115,188,1,0,0,124,0,97,0,116,0,106, + 1,97,1,116,0,106,2,97,2,116,1,106,3,116,4,25, + 0,125,1,120,56,100,26,68,0,93,48,125,2,124,2,116, + 1,106,3,107,7,114,58,116,0,106,5,124,2,131,1,125, + 3,110,10,116,1,106,3,124,2,25,0,125,3,116,6,124, + 1,124,2,124,3,131,3,1,0,113,32,87,0,100,5,100, + 6,103,1,102,2,100,7,100,8,100,6,103,2,102,2,102, + 2,125,4,120,118,124,4,68,0,93,102,92,2,125,5,125, + 6,116,7,100,9,100,10,132,0,124,6,68,0,131,1,131, + 1,115,142,116,8,130,1,124,6,100,11,25,0,125,7,124, + 5,116,1,106,3,107,6,114,174,116,1,106,3,124,5,25, + 0,125,8,80,0,113,112,121,16,116,0,106,5,124,5,131, + 1,125,8,80,0,87,0,113,112,4,0,116,9,107,10,114, + 212,1,0,1,0,1,0,119,112,89,0,113,112,88,0,113, + 112,87,0,116,9,100,12,131,1,130,1,116,6,124,1,100, + 13,124,8,131,3,1,0,116,6,124,1,100,14,124,7,131, + 3,1,0,116,6,124,1,100,15,100,16,106,10,124,6,131, + 1,131,3,1,0,121,14,116,0,106,5,100,17,131,1,125, + 9,87,0,110,26,4,0,116,9,107,10,144,1,114,52,1, + 0,1,0,1,0,100,18,125,9,89,0,110,2,88,0,116, + 6,124,1,100,17,124,9,131,3,1,0,116,0,106,5,100, + 19,131,1,125,10,116,6,124,1,100,19,124,10,131,3,1, + 0,124,5,100,7,107,2,144,1,114,120,116,0,106,5,100, + 20,131,1,125,11,116,6,124,1,100,21,124,11,131,3,1, + 0,116,6,124,1,100,22,116,11,131,0,131,3,1,0,116, + 12,106,13,116,2,106,14,131,0,131,1,1,0,124,5,100, + 7,107,2,144,1,114,184,116,15,106,16,100,23,131,1,1, + 0,100,24,116,12,107,6,144,1,114,184,100,25,116,17,95, + 18,100,18,83,0,41,27,122,205,83,101,116,117,112,32,116, + 104,101,32,112,97,116,104,45,98,97,115,101,100,32,105,109, + 112,111,114,116,101,114,115,32,102,111,114,32,105,109,112,111, + 114,116,108,105,98,32,98,121,32,105,109,112,111,114,116,105, + 110,103,32,110,101,101,100,101,100,10,32,32,32,32,98,117, + 105,108,116,45,105,110,32,109,111,100,117,108,101,115,32,97, + 110,100,32,105,110,106,101,99,116,105,110,103,32,116,104,101, + 109,32,105,110,116,111,32,116,104,101,32,103,108,111,98,97, + 108,32,110,97,109,101,115,112,97,99,101,46,10,10,32,32, + 32,32,79,116,104,101,114,32,99,111,109,112,111,110,101,110, + 116,115,32,97,114,101,32,101,120,116,114,97,99,116,101,100, + 32,102,114,111,109,32,116,104,101,32,99,111,114,101,32,98, + 111,111,116,115,116,114,97,112,32,109,111,100,117,108,101,46, + 10,10,32,32,32,32,114,52,0,0,0,114,63,0,0,0, + 218,8,98,117,105,108,116,105,110,115,114,140,0,0,0,90, + 5,112,111,115,105,120,250,1,47,218,2,110,116,250,1,92, + 99,1,0,0,0,0,0,0,0,2,0,0,0,3,0,0, + 0,115,0,0,0,115,26,0,0,0,124,0,93,18,125,1, + 116,0,124,1,131,1,100,0,107,2,86,0,1,0,113,2, + 100,1,83,0,41,2,114,31,0,0,0,78,41,1,114,33, + 0,0,0,41,2,114,24,0,0,0,114,81,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,224, + 0,0,0,113,5,0,0,115,2,0,0,0,4,0,122,25, + 95,115,101,116,117,112,46,60,108,111,99,97,108,115,62,46, + 60,103,101,110,101,120,112,114,62,114,62,0,0,0,122,30, + 105,109,112,111,114,116,108,105,98,32,114,101,113,117,105,114, + 101,115,32,112,111,115,105,120,32,111,114,32,110,116,114,3, + 0,0,0,114,27,0,0,0,114,23,0,0,0,114,32,0, + 0,0,90,7,95,116,104,114,101,97,100,78,90,8,95,119, + 101,97,107,114,101,102,90,6,119,105,110,114,101,103,114,167, + 0,0,0,114,7,0,0,0,122,4,46,112,121,119,122,6, + 95,100,46,112,121,100,84,41,4,122,3,95,105,111,122,9, + 95,119,97,114,110,105,110,103,115,122,8,98,117,105,108,116, + 105,110,115,122,7,109,97,114,115,104,97,108,41,19,114,118, + 0,0,0,114,8,0,0,0,114,143,0,0,0,114,236,0, + 0,0,114,109,0,0,0,90,18,95,98,117,105,108,116,105, + 110,95,102,114,111,109,95,110,97,109,101,114,113,0,0,0, + 218,3,97,108,108,218,14,65,115,115,101,114,116,105,111,110, + 69,114,114,111,114,114,103,0,0,0,114,28,0,0,0,114, + 13,0,0,0,114,226,0,0,0,114,147,0,0,0,114,24, + 1,0,0,114,88,0,0,0,114,161,0,0,0,114,166,0, + 0,0,114,170,0,0,0,41,12,218,17,95,98,111,111,116, + 115,116,114,97,112,95,109,111,100,117,108,101,90,11,115,101, + 108,102,95,109,111,100,117,108,101,90,12,98,117,105,108,116, + 105,110,95,110,97,109,101,90,14,98,117,105,108,116,105,110, + 95,109,111,100,117,108,101,90,10,111,115,95,100,101,116,97, + 105,108,115,90,10,98,117,105,108,116,105,110,95,111,115,114, + 23,0,0,0,114,27,0,0,0,90,9,111,115,95,109,111, + 100,117,108,101,90,13,116,104,114,101,97,100,95,109,111,100, + 117,108,101,90,14,119,101,97,107,114,101,102,95,109,111,100, + 117,108,101,90,13,119,105,110,114,101,103,95,109,111,100,117, + 108,101,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,218,6,95,115,101,116,117,112,88,5,0,0,115,82,0, + 0,0,0,8,4,1,6,1,6,3,10,1,10,1,10,1, + 12,2,10,1,16,3,22,1,14,2,22,1,8,1,10,1, + 10,1,4,2,2,1,10,1,6,1,14,1,12,2,8,1, + 12,1,12,1,18,3,2,1,14,1,16,2,10,1,12,3, + 10,1,12,3,10,1,10,1,12,3,14,1,14,1,10,1, + 10,1,10,1,114,32,1,0,0,99,1,0,0,0,0,0, + 0,0,2,0,0,0,3,0,0,0,67,0,0,0,115,74, + 0,0,0,116,0,124,0,131,1,1,0,116,1,131,0,125, + 1,116,2,106,3,106,4,116,5,106,6,124,1,152,1,142, + 0,103,1,131,1,1,0,116,7,106,8,100,1,107,2,114, + 58,116,2,106,9,106,10,116,11,131,1,1,0,116,2,106, + 9,106,10,116,12,131,1,1,0,100,2,83,0,41,3,122, + 41,73,110,115,116,97,108,108,32,116,104,101,32,112,97,116, + 104,45,98,97,115,101,100,32,105,109,112,111,114,116,32,99, + 111,109,112,111,110,101,110,116,115,46,114,27,1,0,0,78, + 41,13,114,32,1,0,0,114,159,0,0,0,114,8,0,0, + 0,114,253,0,0,0,114,147,0,0,0,114,5,1,0,0, + 114,18,1,0,0,114,3,0,0,0,114,109,0,0,0,218, + 9,109,101,116,97,95,112,97,116,104,114,161,0,0,0,114, + 166,0,0,0,114,248,0,0,0,41,2,114,31,1,0,0, + 90,17,115,117,112,112,111,114,116,101,100,95,108,111,97,100, + 101,114,115,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,8,95,105,110,115,116,97,108,108,156,5,0,0, + 115,12,0,0,0,0,2,8,1,6,1,22,1,10,1,12, + 1,114,34,1,0,0,41,1,122,3,119,105,110,41,2,114, + 1,0,0,0,114,2,0,0,0,41,1,114,49,0,0,0, + 41,1,78,41,3,78,78,78,41,3,78,78,78,41,2,114, + 62,0,0,0,114,62,0,0,0,41,1,78,41,1,78,41, + 58,114,111,0,0,0,114,12,0,0,0,90,37,95,67,65, + 83,69,95,73,78,83,69,78,83,73,84,73,86,69,95,80, + 76,65,84,70,79,82,77,83,95,66,89,84,69,83,95,75, + 69,89,114,11,0,0,0,114,13,0,0,0,114,19,0,0, + 0,114,21,0,0,0,114,30,0,0,0,114,40,0,0,0, + 114,41,0,0,0,114,45,0,0,0,114,46,0,0,0,114, + 48,0,0,0,114,58,0,0,0,218,4,116,121,112,101,218, + 8,95,95,99,111,100,101,95,95,114,142,0,0,0,114,17, + 0,0,0,114,132,0,0,0,114,16,0,0,0,114,20,0, + 0,0,90,17,95,82,65,87,95,77,65,71,73,67,95,78, + 85,77,66,69,82,114,77,0,0,0,114,76,0,0,0,114, + 88,0,0,0,114,78,0,0,0,90,23,68,69,66,85,71, + 95,66,89,84,69,67,79,68,69,95,83,85,70,70,73,88, + 69,83,90,27,79,80,84,73,77,73,90,69,68,95,66,89, + 84,69,67,79,68,69,95,83,85,70,70,73,88,69,83,114, + 83,0,0,0,114,89,0,0,0,114,95,0,0,0,114,99, + 0,0,0,114,101,0,0,0,114,120,0,0,0,114,127,0, + 0,0,114,139,0,0,0,114,145,0,0,0,114,148,0,0, + 0,114,153,0,0,0,218,6,111,98,106,101,99,116,114,160, + 0,0,0,114,165,0,0,0,114,166,0,0,0,114,181,0, + 0,0,114,191,0,0,0,114,207,0,0,0,114,215,0,0, + 0,114,220,0,0,0,114,226,0,0,0,114,221,0,0,0, + 114,227,0,0,0,114,246,0,0,0,114,248,0,0,0,114, + 5,1,0,0,114,23,1,0,0,114,159,0,0,0,114,32, + 1,0,0,114,34,1,0,0,114,4,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,218,8,60,109, + 111,100,117,108,101,62,8,0,0,0,115,108,0,0,0,4, + 16,4,1,4,1,2,1,6,3,8,17,8,5,8,5,8, + 6,8,12,8,10,8,9,8,5,8,7,10,22,10,121,16, + 1,12,2,4,1,4,2,6,2,6,2,8,2,16,45,8, + 34,8,19,8,12,8,12,8,28,8,17,10,55,10,12,10, + 10,8,14,6,3,4,1,14,67,14,64,14,29,16,110,14, + 41,18,45,18,16,4,3,18,53,14,60,14,42,14,127,0, + 5,14,127,0,22,10,23,8,11,8,68, }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 02:37:24 2016 From: python-checkins at python.org (ethan.furman) Date: Sun, 11 Sep 2016 06:37:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue23591=3A_add_auto=28?= =?utf-8?q?=29_for_auto-generating_Enum_member_values?= Message-ID: <20160911063724.48739.84917.D69C5F0A@psf.io> https://hg.python.org/cpython/rev/aad7443e62be changeset: 103609:aad7443e62be user: Ethan Furman date: Sat Sep 10 23:36:59 2016 -0700 summary: issue23591: add auto() for auto-generating Enum member values files: Doc/library/enum.rst | 99 +++++++++++++++++++++++++----- Lib/enum.py | 50 +++++++++++---- Lib/test/test_enum.py | 77 +++++++++++++++++++++++- 3 files changed, 195 insertions(+), 31 deletions(-) diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -25,7 +25,8 @@ This module defines four enumeration classes that can be used to define unique sets of names and values: :class:`Enum`, :class:`IntEnum`, and -:class:`IntFlags`. It also defines one decorator, :func:`unique`. +:class:`IntFlags`. It also defines one decorator, :func:`unique`, and one +helper, :class:`auto`. .. class:: Enum @@ -52,7 +53,11 @@ Enum class decorator that ensures only one name is bound to any one value. -.. versionadded:: 3.6 ``Flag``, ``IntFlag`` +.. class:: auto + + Instances are replaced with an appropriate value for Enum members. + +.. versionadded:: 3.6 ``Flag``, ``IntFlag``, ``auto`` Creating an Enum @@ -70,6 +75,13 @@ ... blue = 3 ... +.. note:: Enum member values + + Member values can be anything: :class:`int`, :class:`str`, etc.. If + the exact value is unimportant you may use :class:`auto` instances and an + appropriate value will be chosen for you. Care must be taken if you mix + :class:`auto` with other values. + .. note:: Nomenclature - The class :class:`Color` is an *enumeration* (or *enum*) @@ -225,6 +237,42 @@ ValueError: duplicate values found in : four -> three +Using automatic values +---------------------- + +If the exact value is unimportant you can use :class:`auto`:: + + >>> from enum import Enum, auto + >>> class Color(Enum): + ... red = auto() + ... blue = auto() + ... green = auto() + ... + >>> list(Color) + [, , ] + +The values are chosen by :func:`_generate_next_value_`, which can be +overridden:: + + >>> class AutoName(Enum): + ... def _generate_next_value_(name, start, count, last_values): + ... return name + ... + >>> class Ordinal(AutoName): + ... north = auto() + ... south = auto() + ... east = auto() + ... west = auto() + ... + >>> list(Ordinal) + [, , , ] + +.. note:: + + The goal of the default :meth:`_generate_next_value_` methods is to provide + the next :class:`int` in sequence with the last :class:`int` provided, but + the way it does this is an implementation detail and may change. + Iteration --------- @@ -597,7 +645,9 @@ The last variation is :class:`Flag`. Like :class:`IntFlag`, :class:`Flag` members can be combined using the bitwise operators (&, \|, ^, ~). Unlike :class:`IntFlag`, they cannot be combined with, nor compared against, any -other :class:`Flag` enumeration, nor :class:`int`. +other :class:`Flag` enumeration, nor :class:`int`. While it is possible to +specify the values directly it is recommended to use :class:`auto` as the +value and let :class:`Flag` select an appropriate value. .. versionadded:: 3.6 @@ -606,9 +656,9 @@ >>> from enum import Flag >>> class Color(Flag): - ... red = 1 - ... blue = 2 - ... green = 4 + ... red = auto() + ... blue = auto() + ... green = auto() ... >>> Color.red & Color.green @@ -619,21 +669,20 @@ while combinations of flags won't:: >>> class Color(Flag): - ... red = 1 - ... blue = 2 - ... green = 4 - ... white = 7 - ... # or - ... # white = red | blue | green + ... red = auto() + ... blue = auto() + ... green = auto() + ... white = red | blue | green + ... Giving a name to the "no flags set" condition does not change its boolean value:: >>> class Color(Flag): ... black = 0 - ... red = 1 - ... blue = 2 - ... green = 4 + ... red = auto() + ... blue = auto() + ... green = auto() ... >>> Color.black @@ -700,6 +749,7 @@ In many use-cases one doesn't care what the actual value of an enumeration is. There are several ways to define this type of simple enumeration: +- use instances of :class:`auto` for the value - use instances of :class:`object` as the value - use a descriptive string as the value - use a tuple as the value and a custom :meth:`__new__` to replace the @@ -718,6 +768,20 @@ ... +Using :class:`auto` +""""""""""""""""""" + +Using :class:`object` would look like:: + + >>> class Color(NoValue): + ... red = auto() + ... blue = auto() + ... green = auto() + ... + >>> Color.green + + + Using :class:`object` """"""""""""""""""""" @@ -930,8 +994,11 @@ overridden - ``_order_`` -- used in Python 2/3 code to ensure member order is consistent (class attribute, removed during class creation) +- ``_generate_next_value_`` -- used by the `Functional API`_ and by + :class:`auto` to get an appropriate value for an enum member; may be + overridden -.. versionadded:: 3.6 ``_missing_``, ``_order_`` +.. versionadded:: 3.6 ``_missing_``, ``_order_``, ``_generate_next_value_`` To help keep Python 2 / Python 3 code in sync an :attr:`_order_` attribute can be provided. It will be checked against the actual order of the enumeration diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -10,7 +10,11 @@ from collections import OrderedDict -__all__ = ['EnumMeta', 'Enum', 'IntEnum', 'Flag', 'IntFlag', 'unique'] +__all__ = [ + 'EnumMeta', + 'Enum', 'IntEnum', 'Flag', 'IntFlag', + 'auto', 'unique', + ] def _is_descriptor(obj): @@ -36,7 +40,6 @@ name[-2:-1] != '_' and len(name) > 2) - def _make_class_unpicklable(cls): """Make the given class un-picklable.""" def _break_on_call_reduce(self, proto): @@ -44,6 +47,12 @@ cls.__reduce_ex__ = _break_on_call_reduce cls.__module__ = '' +class auto: + """ + Instances are replaced with an appropriate value in Enum class suites. + """ + pass + class _EnumDict(dict): """Track enum member order and ensure member names are not reused. @@ -55,6 +64,7 @@ def __init__(self): super().__init__() self._member_names = [] + self._last_values = [] def __setitem__(self, key, value): """Changes anything not dundered or not a descriptor. @@ -71,6 +81,8 @@ '_generate_next_value_', '_missing_', ): raise ValueError('_names_ are reserved for future Enum use') + if key == '_generate_next_value_': + setattr(self, '_generate_next_value', value) elif _is_dunder(key): if key == '__order__': key = '_order_' @@ -81,11 +93,13 @@ if key in self: # enum overwriting a descriptor? raise TypeError('%r already defined as: %r' % (key, self[key])) + if isinstance(value, auto): + value = self._generate_next_value(key, 1, len(self._member_names), self._last_values[:]) self._member_names.append(key) + self._last_values.append(value) super().__setitem__(key, value) - # Dummy value for Enum as EnumMeta explicitly checks for it, but of course # until EnumMeta finishes running the first time the Enum class doesn't exist. # This is also why there are checks in EnumMeta like `if Enum is not None` @@ -366,10 +380,11 @@ names = names.replace(',', ' ').split() if isinstance(names, (tuple, list)) and isinstance(names[0], str): original_names, names = names, [] - last_value = None + last_values = [] for count, name in enumerate(original_names): - last_value = first_enum._generate_next_value_(name, start, count, last_value) - names.append((name, last_value)) + value = first_enum._generate_next_value_(name, start, count, last_values[:]) + last_values.append(value) + names.append((name, value)) # Here, names is either an iterable of (name, value) or a mapping. for item in names: @@ -514,11 +529,15 @@ # still not found -- try _missing_ hook return cls._missing_(value) - @staticmethod - def _generate_next_value_(name, start, count, last_value): - if not count: + def _generate_next_value_(name, start, count, last_values): + for last_value in reversed(last_values): + try: + return last_value + 1 + except TypeError: + pass + else: return start - return last_value + 1 + @classmethod def _missing_(cls, value): raise ValueError("%r is not a valid %s" % (value, cls.__name__)) @@ -616,8 +635,8 @@ class Flag(Enum): """Support for flags""" - @staticmethod - def _generate_next_value_(name, start, count, last_value): + + def _generate_next_value_(name, start, count, last_values): """ Generate the next value when not given. @@ -628,7 +647,12 @@ """ if not count: return start if start is not None else 1 - high_bit = _high_bit(last_value) + for last_value in reversed(last_values): + try: + high_bit = _high_bit(last_value) + break + except TypeError: + raise TypeError('Invalid Flag value: %r' % last_value) from None return 2 ** (high_bit+1) @classmethod diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -3,7 +3,7 @@ import pydoc import unittest from collections import OrderedDict -from enum import Enum, IntEnum, EnumMeta, Flag, IntFlag, unique +from enum import Enum, IntEnum, EnumMeta, Flag, IntFlag, unique, auto from io import StringIO from pickle import dumps, loads, PicklingError, HIGHEST_PROTOCOL from test import support @@ -113,6 +113,7 @@ '__', '___', '____', '_____',): self.assertFalse(enum._is_dunder(s)) +# tests class TestEnum(unittest.TestCase): @@ -1578,6 +1579,61 @@ self.assertEqual(LabelledList.unprocessed, 1) self.assertEqual(LabelledList(1), LabelledList.unprocessed) + def test_auto_number(self): + class Color(Enum): + red = auto() + blue = auto() + green = auto() + + self.assertEqual(list(Color), [Color.red, Color.blue, Color.green]) + self.assertEqual(Color.red.value, 1) + self.assertEqual(Color.blue.value, 2) + self.assertEqual(Color.green.value, 3) + + def test_auto_name(self): + class Color(Enum): + def _generate_next_value_(name, start, count, last): + return name + red = auto() + blue = auto() + green = auto() + + self.assertEqual(list(Color), [Color.red, Color.blue, Color.green]) + self.assertEqual(Color.red.value, 'red') + self.assertEqual(Color.blue.value, 'blue') + self.assertEqual(Color.green.value, 'green') + + def test_auto_name_inherit(self): + class AutoNameEnum(Enum): + def _generate_next_value_(name, start, count, last): + return name + class Color(AutoNameEnum): + red = auto() + blue = auto() + green = auto() + + self.assertEqual(list(Color), [Color.red, Color.blue, Color.green]) + self.assertEqual(Color.red.value, 'red') + self.assertEqual(Color.blue.value, 'blue') + self.assertEqual(Color.green.value, 'green') + + def test_auto_garbage(self): + class Color(Enum): + red = 'red' + blue = auto() + self.assertEqual(Color.blue.value, 1) + + def test_auto_garbage_corrected(self): + class Color(Enum): + red = 'red' + blue = 2 + green = auto() + + self.assertEqual(list(Color), [Color.red, Color.blue, Color.green]) + self.assertEqual(Color.red.value, 'red') + self.assertEqual(Color.blue.value, 2) + self.assertEqual(Color.green.value, 3) + class TestOrder(unittest.TestCase): @@ -1856,7 +1912,6 @@ test_pickle_dump_load(self.assertIs, FlagStooges.CURLY|FlagStooges.MOE) test_pickle_dump_load(self.assertIs, FlagStooges) - def test_containment(self): Perm = self.Perm R, W, X = Perm @@ -1877,6 +1932,24 @@ self.assertFalse(W in RX) self.assertFalse(X in RW) + def test_auto_number(self): + class Color(Flag): + red = auto() + blue = auto() + green = auto() + + self.assertEqual(list(Color), [Color.red, Color.blue, Color.green]) + self.assertEqual(Color.red.value, 1) + self.assertEqual(Color.blue.value, 2) + self.assertEqual(Color.green.value, 4) + + def test_auto_number_garbage(self): + with self.assertRaisesRegex(TypeError, 'Invalid Flag value: .not an int.'): + class Color(Flag): + red = 'not an int' + blue = auto() + + class TestIntFlag(unittest.TestCase): """Tests of the IntFlags.""" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 04:03:46 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 08:03:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326900=3A_Excluded?= =?utf-8?q?_underscored_names_and_other_private_API_from_limited_API=2E?= Message-ID: <20160911080342.36323.15650.5227532A@psf.io> https://hg.python.org/cpython/rev/d00f15af75ea changeset: 103610:d00f15af75ea user: Serhiy Storchaka date: Sun Sep 11 11:03:14 2016 +0300 summary: Issue #26900: Excluded underscored names and other private API from limited API. files: Doc/whatsnew/3.5.rst | 3 +-- Include/abstract.h | 10 +++++++++- Include/ceval.h | 2 ++ Include/descrobject.h | 2 ++ Include/dictobject.h | 6 +++++- Include/fileutils.h | 8 ++------ Include/import.h | 2 ++ Include/intrcheck.h | 3 +++ Include/longobject.h | 2 ++ Include/modsupport.h | 4 ++++ Include/namespaceobject.h | 2 ++ Include/object.h | 15 +++++++++++++-- Include/objimpl.h | 4 ++++ Include/pygetopt.h | 2 +- Include/pylifecycle.h | 2 ++ Include/pystate.h | 10 ++++++++++ Include/pystrhex.h | 2 ++ Include/sysmodule.h | 4 ++-- Include/traceback.h | 2 ++ Include/unicodeobject.h | 2 ++ Misc/NEWS | 2 ++ 21 files changed, 74 insertions(+), 15 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -2176,8 +2176,7 @@ * :c:func:`PyMem_RawCalloc`, * :c:func:`PyMem_Calloc`, -* :c:func:`PyObject_Calloc`, -* :c:func:`_PyObject_GC_Calloc`. +* :c:func:`PyObject_Calloc`. (Contributed by Victor Stinner in :issue:`21233`.) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -7,7 +7,9 @@ #ifdef PY_SSIZE_T_CLEAN #define PyObject_CallFunction _PyObject_CallFunction_SizeT #define PyObject_CallMethod _PyObject_CallMethod_SizeT +#ifndef Py_LIMITED_API #define _PyObject_CallMethodId _PyObject_CallMethodId_SizeT +#endif /* !Py_LIMITED_API */ #endif /* Abstract Object Interface (many thanks to Jim Fulton) */ @@ -385,6 +387,7 @@ Python expression: o.method(args). */ +#ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _PyObject_CallMethodId(PyObject *o, _Py_Identifier *method, const char *format, ...); @@ -393,6 +396,7 @@ Like PyObject_CallMethod, but expect a _Py_Identifier* as the method name. */ +#endif /* !Py_LIMITED_API */ PyAPI_FUNC(PyObject *) _PyObject_CallFunction_SizeT(PyObject *callable, const char *format, @@ -401,10 +405,12 @@ const char *name, const char *format, ...); +#ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _PyObject_CallMethodId_SizeT(PyObject *o, _Py_Identifier *name, const char *format, ...); +#endif /* !Py_LIMITED_API */ PyAPI_FUNC(PyObject *) PyObject_CallFunctionObjArgs(PyObject *callable, ...); @@ -420,9 +426,11 @@ PyAPI_FUNC(PyObject *) PyObject_CallMethodObjArgs(PyObject *o, PyObject *method, ...); +#ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _PyObject_CallMethodIdObjArgs(PyObject *o, struct _Py_Identifier *method, ...); +#endif /* !Py_LIMITED_API */ /* Call the method named m of object o with a variable number of @@ -1340,13 +1348,13 @@ PyAPI_FUNC(char *const *) _PySequence_BytesToCharpArray(PyObject* self); PyAPI_FUNC(void) _Py_FreeCharPArray(char *const array[]); -#endif /* For internal use by buffer API functions */ PyAPI_FUNC(void) _Py_add_one_to_index_F(int nd, Py_ssize_t *index, const Py_ssize_t *shape); PyAPI_FUNC(void) _Py_add_one_to_index_C(int nd, Py_ssize_t *index, const Py_ssize_t *shape); +#endif /* !Py_LIMITED_API */ #ifdef __cplusplus diff --git a/Include/ceval.h b/Include/ceval.h --- a/Include/ceval.h +++ b/Include/ceval.h @@ -179,7 +179,9 @@ PyAPI_FUNC(int) PyEval_ThreadsInitialized(void); PyAPI_FUNC(void) PyEval_InitThreads(void); +#ifndef Py_LIMITED_API PyAPI_FUNC(void) _PyEval_FiniThreads(void); +#endif /* !Py_LIMITED_API */ PyAPI_FUNC(void) PyEval_AcquireLock(void); PyAPI_FUNC(void) PyEval_ReleaseLock(void); PyAPI_FUNC(void) PyEval_AcquireThread(PyThreadState *tstate); diff --git a/Include/descrobject.h b/Include/descrobject.h --- a/Include/descrobject.h +++ b/Include/descrobject.h @@ -78,7 +78,9 @@ PyAPI_DATA(PyTypeObject) PyMethodDescr_Type; PyAPI_DATA(PyTypeObject) PyWrapperDescr_Type; PyAPI_DATA(PyTypeObject) PyDictProxy_Type; +#ifndef Py_LIMITED_API PyAPI_DATA(PyTypeObject) _PyMethodWrapper_Type; +#endif /* Py_LIMITED_API */ PyAPI_FUNC(PyObject *) PyDescr_NewMethod(PyTypeObject *, PyMethodDef *); PyAPI_FUNC(PyObject *) PyDescr_NewClassMethod(PyTypeObject *, PyMethodDef *); diff --git a/Include/dictobject.h b/Include/dictobject.h --- a/Include/dictobject.h +++ b/Include/dictobject.h @@ -73,9 +73,9 @@ Py_hash_t hash); #endif PyAPI_FUNC(PyObject *) PyDict_GetItemWithError(PyObject *mp, PyObject *key); +#ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _PyDict_GetItemIdWithError(PyObject *dp, struct _Py_Identifier *key); -#ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) PyDict_SetDefault( PyObject *mp, PyObject *key, PyObject *defaultobj); #endif @@ -145,9 +145,13 @@ int override); PyAPI_FUNC(PyObject *) PyDict_GetItemString(PyObject *dp, const char *key); +#ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _PyDict_GetItemId(PyObject *dp, struct _Py_Identifier *key); +#endif /* !Py_LIMITED_API */ PyAPI_FUNC(int) PyDict_SetItemString(PyObject *dp, const char *key, PyObject *item); +#ifndef Py_LIMITED_API PyAPI_FUNC(int) _PyDict_SetItemId(PyObject *dp, struct _Py_Identifier *key, PyObject *item); +#endif /* !Py_LIMITED_API */ PyAPI_FUNC(int) PyDict_DelItemString(PyObject *dp, const char *key); #ifndef Py_LIMITED_API diff --git a/Include/fileutils.h b/Include/fileutils.h --- a/Include/fileutils.h +++ b/Include/fileutils.h @@ -5,8 +5,6 @@ extern "C" { #endif -PyAPI_FUNC(PyObject *) _Py_device_encoding(int); - PyAPI_FUNC(wchar_t *) Py_DecodeLocale( const char *arg, size_t *size); @@ -17,6 +15,8 @@ #ifndef Py_LIMITED_API +PyAPI_FUNC(PyObject *) _Py_device_encoding(int); + #ifdef MS_WINDOWS struct _Py_stat_struct { unsigned long st_dev; @@ -46,13 +46,11 @@ PyAPI_FUNC(int) _Py_fstat_noraise( int fd, struct _Py_stat_struct *status); -#endif /* Py_LIMITED_API */ PyAPI_FUNC(int) _Py_stat( PyObject *path, struct stat *status); -#ifndef Py_LIMITED_API PyAPI_FUNC(int) _Py_open( const char *pathname, int flags); @@ -60,7 +58,6 @@ PyAPI_FUNC(int) _Py_open_noraise( const char *pathname, int flags); -#endif PyAPI_FUNC(FILE *) _Py_wfopen( const wchar_t *path, @@ -107,7 +104,6 @@ wchar_t *buf, size_t size); -#ifndef Py_LIMITED_API PyAPI_FUNC(int) _Py_get_inheritable(int fd); PyAPI_FUNC(int) _Py_set_inheritable(int fd, int inheritable, diff --git a/Include/import.h b/Include/import.h --- a/Include/import.h +++ b/Include/import.h @@ -7,7 +7,9 @@ extern "C" { #endif +#ifndef Py_LIMITED_API PyAPI_FUNC(void) _PyImportZip_Init(void); +#endif /* !Py_LIMITED_API */ PyMODINIT_FUNC PyInit_imp(void); PyAPI_FUNC(long) PyImport_GetMagicNumber(void); diff --git a/Include/intrcheck.h b/Include/intrcheck.h --- a/Include/intrcheck.h +++ b/Include/intrcheck.h @@ -8,12 +8,15 @@ PyAPI_FUNC(int) PyOS_InterruptOccurred(void); PyAPI_FUNC(void) PyOS_InitInterrupts(void); PyAPI_FUNC(void) PyOS_AfterFork(void); + +#ifndef Py_LIMITED_API PyAPI_FUNC(int) _PyOS_IsMainThread(void); #ifdef MS_WINDOWS /* windows.h is not included by Python.h so use void* instead of HANDLE */ PyAPI_FUNC(void*) _PyOS_SigintEvent(void); #endif +#endif /* !Py_LIMITED_API */ #ifdef __cplusplus } diff --git a/Include/longobject.h b/Include/longobject.h --- a/Include/longobject.h +++ b/Include/longobject.h @@ -204,8 +204,10 @@ PyAPI_FUNC(unsigned long) PyOS_strtoul(const char *, char **, int); PyAPI_FUNC(long) PyOS_strtol(const char *, char **, int); +#ifndef Py_LIMITED_API /* For use by the gcd function in mathmodule.c */ PyAPI_FUNC(PyObject *) _PyLong_GCD(PyObject *, PyObject *); +#endif /* !Py_LIMITED_API */ #ifdef __cplusplus } diff --git a/Include/modsupport.h b/Include/modsupport.h --- a/Include/modsupport.h +++ b/Include/modsupport.h @@ -15,12 +15,16 @@ #define PyArg_Parse _PyArg_Parse_SizeT #define PyArg_ParseTuple _PyArg_ParseTuple_SizeT #define PyArg_ParseTupleAndKeywords _PyArg_ParseTupleAndKeywords_SizeT +#ifndef Py_LIMITED_API #define PyArg_VaParse _PyArg_VaParse_SizeT #define PyArg_VaParseTupleAndKeywords _PyArg_VaParseTupleAndKeywords_SizeT +#endif /* !Py_LIMITED_API */ #define Py_BuildValue _Py_BuildValue_SizeT #define Py_VaBuildValue _Py_VaBuildValue_SizeT #else +#ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _Py_VaBuildValue_SizeT(const char *, va_list); +#endif /* !Py_LIMITED_API */ #endif /* Due to a glitch in 3.2, the _SizeT versions weren't exported from the DLL. */ diff --git a/Include/namespaceobject.h b/Include/namespaceobject.h --- a/Include/namespaceobject.h +++ b/Include/namespaceobject.h @@ -7,9 +7,11 @@ extern "C" { #endif +#ifndef Py_LIMITED_API PyAPI_DATA(PyTypeObject) _PyNamespace_Type; PyAPI_FUNC(PyObject *) _PyNamespace_New(PyObject *kwds); +#endif /* !Py_LIMITED_API */ #ifdef __cplusplus } diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -118,6 +118,7 @@ #define Py_TYPE(ob) (((PyObject*)(ob))->ob_type) #define Py_SIZE(ob) (((PyVarObject*)(ob))->ob_size) +#ifndef Py_LIMITED_API /********************* String Literals ****************************************/ /* This structure helps managing static strings. The basic usage goes like this: Instead of doing @@ -148,6 +149,8 @@ #define _Py_static_string(varname, value) static _Py_Identifier varname = _Py_static_string_init(value) #define _Py_IDENTIFIER(varname) _Py_static_string(PyId_##varname, #varname) +#endif /* !Py_LIMITED_API */ + /* Type objects contain a string containing the type name (to help somewhat in debugging), the allocation parameters (see PyObject_New() and @@ -512,8 +515,8 @@ #endif /* Generic operations on objects */ +#ifndef Py_LIMITED_API struct _Py_Identifier; -#ifndef Py_LIMITED_API PyAPI_FUNC(int) PyObject_Print(PyObject *, FILE *, int); PyAPI_FUNC(void) _Py_BreakPoint(void); PyAPI_FUNC(void) _PyObject_Dump(PyObject *); @@ -530,11 +533,11 @@ PyAPI_FUNC(PyObject *) PyObject_GetAttr(PyObject *, PyObject *); PyAPI_FUNC(int) PyObject_SetAttr(PyObject *, PyObject *, PyObject *); PyAPI_FUNC(int) PyObject_HasAttr(PyObject *, PyObject *); +#ifndef Py_LIMITED_API PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *); PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, struct _Py_Identifier *); PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, struct _Py_Identifier *, PyObject *); PyAPI_FUNC(int) _PyObject_HasAttrId(PyObject *, struct _Py_Identifier *); -#ifndef Py_LIMITED_API PyAPI_FUNC(PyObject **) _PyObject_GetDictPtr(PyObject *); #endif PyAPI_FUNC(PyObject *) PyObject_SelfIter(PyObject *); @@ -557,6 +560,7 @@ PyAPI_FUNC(int) PyObject_CallFinalizerFromDealloc(PyObject *); #endif +#ifndef Py_LIMITED_API /* Same as PyObject_Generic{Get,Set}Attr, but passing the attributes dict as the last parameter. */ PyAPI_FUNC(PyObject *) @@ -564,6 +568,7 @@ PyAPI_FUNC(int) _PyObject_GenericSetAttrWithDict(PyObject *, PyObject *, PyObject *, PyObject *); +#endif /* !Py_LIMITED_API */ /* Helper to look up a builtin object */ #ifndef Py_LIMITED_API @@ -888,8 +893,10 @@ PyAPI_FUNC(void) Py_IncRef(PyObject *); PyAPI_FUNC(void) Py_DecRef(PyObject *); +#ifndef Py_LIMITED_API PyAPI_DATA(PyTypeObject) _PyNone_Type; PyAPI_DATA(PyTypeObject) _PyNotImplemented_Type; +#endif /* !Py_LIMITED_API */ /* _Py_NoneStruct is an object of undefined type which can be used in contexts @@ -922,10 +929,12 @@ #define Py_GT 4 #define Py_GE 5 +#ifndef Py_LIMITED_API /* Maps Py_LT to Py_GT, ..., Py_GE to Py_LE. * Defined in object.c. */ PyAPI_DATA(int) _Py_SwappedOp[]; +#endif /* !Py_LIMITED_API */ /* @@ -1022,12 +1031,14 @@ with the call stack never exceeding a depth of PyTrash_UNWIND_LEVEL. */ +#ifndef Py_LIMITED_API /* This is the old private API, invoked by the macros before 3.2.4. Kept for binary compatibility of extensions using the stable ABI. */ PyAPI_FUNC(void) _PyTrash_deposit_object(PyObject*); PyAPI_FUNC(void) _PyTrash_destroy_chain(void); PyAPI_DATA(int) _PyTrash_delete_nesting; PyAPI_DATA(PyObject *) _PyTrash_delete_later; +#endif /* !Py_LIMITED_API */ /* The new thread-safe private API, invoked by the macros below. */ PyAPI_FUNC(void) _PyTrash_thread_deposit_object(PyObject*); diff --git a/Include/objimpl.h b/Include/objimpl.h --- a/Include/objimpl.h +++ b/Include/objimpl.h @@ -99,8 +99,10 @@ PyAPI_FUNC(void *) PyObject_Realloc(void *ptr, size_t new_size); PyAPI_FUNC(void) PyObject_Free(void *ptr); +#ifndef Py_LIMITED_API /* This function returns the number of allocated memory blocks, regardless of size */ PyAPI_FUNC(Py_ssize_t) _Py_GetAllocatedBlocks(void); +#endif /* !Py_LIMITED_API */ /* Macros */ #ifdef WITH_PYMALLOC @@ -323,8 +325,10 @@ (!PyTuple_CheckExact(obj) || _PyObject_GC_IS_TRACKED(obj))) #endif /* Py_LIMITED_API */ +#ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _PyObject_GC_Malloc(size_t size); PyAPI_FUNC(PyObject *) _PyObject_GC_Calloc(size_t size); +#endif /* !Py_LIMITED_API */ PyAPI_FUNC(PyObject *) _PyObject_GC_New(PyTypeObject *); PyAPI_FUNC(PyVarObject *) _PyObject_GC_NewVar(PyTypeObject *, Py_ssize_t); PyAPI_FUNC(void) PyObject_GC_Track(void *); diff --git a/Include/pygetopt.h b/Include/pygetopt.h --- a/Include/pygetopt.h +++ b/Include/pygetopt.h @@ -11,9 +11,9 @@ PyAPI_DATA(wchar_t *) _PyOS_optarg; PyAPI_FUNC(void) _PyOS_ResetGetOpt(void); -#endif PyAPI_FUNC(int) _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring); +#endif /* !Py_LIMITED_API */ #ifdef __cplusplus } diff --git a/Include/pylifecycle.h b/Include/pylifecycle.h --- a/Include/pylifecycle.h +++ b/Include/pylifecycle.h @@ -117,9 +117,11 @@ PyAPI_FUNC(PyOS_sighandler_t) PyOS_getsig(int); PyAPI_FUNC(PyOS_sighandler_t) PyOS_setsig(int, PyOS_sighandler_t); +#ifndef Py_LIMITED_API /* Random */ PyAPI_FUNC(int) _PyOS_URandom(void *buffer, Py_ssize_t size); PyAPI_FUNC(int) _PyOS_URandomNonblock(void *buffer, Py_ssize_t size); +#endif /* !Py_LIMITED_API */ #ifdef __cplusplus } diff --git a/Include/pystate.h b/Include/pystate.h --- a/Include/pystate.h +++ b/Include/pystate.h @@ -157,7 +157,9 @@ PyAPI_FUNC(PyInterpreterState *) PyInterpreterState_New(void); PyAPI_FUNC(void) PyInterpreterState_Clear(PyInterpreterState *); PyAPI_FUNC(void) PyInterpreterState_Delete(PyInterpreterState *); +#ifndef Py_LIMITED_API PyAPI_FUNC(int) _PyState_AddModule(PyObject*, struct PyModuleDef*); +#endif /* !Py_LIMITED_API */ #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03030000 /* New in 3.3 */ PyAPI_FUNC(int) PyState_AddModule(PyObject*, struct PyModuleDef*); @@ -169,14 +171,20 @@ #endif PyAPI_FUNC(PyThreadState *) PyThreadState_New(PyInterpreterState *); +#ifndef Py_LIMITED_API PyAPI_FUNC(PyThreadState *) _PyThreadState_Prealloc(PyInterpreterState *); PyAPI_FUNC(void) _PyThreadState_Init(PyThreadState *); +#endif /* !Py_LIMITED_API */ PyAPI_FUNC(void) PyThreadState_Clear(PyThreadState *); PyAPI_FUNC(void) PyThreadState_Delete(PyThreadState *); +#ifndef Py_LIMITED_API PyAPI_FUNC(void) _PyThreadState_DeleteExcept(PyThreadState *tstate); +#endif /* !Py_LIMITED_API */ #ifdef WITH_THREAD PyAPI_FUNC(void) PyThreadState_DeleteCurrent(void); +#ifndef Py_LIMITED_API PyAPI_FUNC(void) _PyGILState_Reinit(void); +#endif /* !Py_LIMITED_API */ #endif /* Return the current thread state. The global interpreter lock must be held. @@ -184,9 +192,11 @@ * the caller needn't check for NULL). */ PyAPI_FUNC(PyThreadState *) PyThreadState_Get(void); +#ifndef Py_LIMITED_API /* Similar to PyThreadState_Get(), but don't issue a fatal error * if it is NULL. */ PyAPI_FUNC(PyThreadState *) _PyThreadState_UncheckedGet(void); +#endif /* !Py_LIMITED_API */ PyAPI_FUNC(PyThreadState *) PyThreadState_Swap(PyThreadState *); PyAPI_FUNC(PyObject *) PyThreadState_GetDict(void); diff --git a/Include/pystrhex.h b/Include/pystrhex.h --- a/Include/pystrhex.h +++ b/Include/pystrhex.h @@ -5,10 +5,12 @@ extern "C" { #endif +#ifndef Py_LIMITED_API /* Returns a str() containing the hex representation of argbuf. */ PyAPI_FUNC(PyObject*) _Py_strhex(const char* argbuf, const Py_ssize_t arglen); /* Returns a bytes() containing the ASCII hex representation of argbuf. */ PyAPI_FUNC(PyObject*) _Py_strhex_bytes(const char* argbuf, const Py_ssize_t arglen); +#endif /* !Py_LIMITED_API */ #ifdef __cplusplus } diff --git a/Include/sysmodule.h b/Include/sysmodule.h --- a/Include/sysmodule.h +++ b/Include/sysmodule.h @@ -8,11 +8,11 @@ #endif PyAPI_FUNC(PyObject *) PySys_GetObject(const char *); +PyAPI_FUNC(int) PySys_SetObject(const char *, PyObject *); #ifndef Py_LIMITED_API PyAPI_FUNC(PyObject *) _PySys_GetObjectId(_Py_Identifier *key); +PyAPI_FUNC(int) _PySys_SetObjectId(_Py_Identifier *key, PyObject *); #endif -PyAPI_FUNC(int) PySys_SetObject(const char *, PyObject *); -PyAPI_FUNC(int) _PySys_SetObjectId(_Py_Identifier *key, PyObject *); PyAPI_FUNC(void) PySys_SetArgv(int, wchar_t **); PyAPI_FUNC(void) PySys_SetArgvEx(int, wchar_t **, int); diff --git a/Include/traceback.h b/Include/traceback.h --- a/Include/traceback.h +++ b/Include/traceback.h @@ -31,6 +31,7 @@ PyAPI_DATA(PyTypeObject) PyTraceBack_Type; #define PyTraceBack_Check(v) (Py_TYPE(v) == &PyTraceBack_Type) +#ifndef Py_LIMITED_API /* Write the Python traceback into the file 'fd'. For example: Traceback (most recent call first): @@ -79,6 +80,7 @@ int fd, PyInterpreterState *interp, PyThreadState *current_tstate); +#endif /* !Py_LIMITED_API */ #ifndef Py_LIMITED_API diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -2257,6 +2257,7 @@ int check_content); #endif +#ifndef Py_LIMITED_API /* Return an interned Unicode object for an Identifier; may fail if there is no memory.*/ PyAPI_FUNC(PyObject*) _PyUnicode_FromId(_Py_Identifier*); /* Clear all static strings. */ @@ -2265,6 +2266,7 @@ /* Fast equality check when the inputs are known to be exact unicode types and where the hash values are equal (i.e. a very probable match) */ PyAPI_FUNC(int) _PyUnicode_EQ(PyObject *, PyObject *); +#endif /* !Py_LIMITED_API */ #ifdef __cplusplus } diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -369,6 +369,8 @@ C API ----- +- Issue #26900: Excluded underscored names and other private API from limited API. + - Issue #26027: Add support for path-like objects in PyUnicode_FSConverter() & PyUnicode_FSDecoder(). -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 04:24:19 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 08:24:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2326885=3A_xmlrpc_n?= =?utf-8?q?ow_supports_unmarshalling_additional_data_types_used?= Message-ID: <20160911082403.36352.73146.D54E383A@psf.io> https://hg.python.org/cpython/rev/ef63e1c999bd changeset: 103611:ef63e1c999bd user: Serhiy Storchaka date: Sun Sep 11 11:23:38 2016 +0300 summary: Issue #26885: xmlrpc now supports unmarshalling additional data types used by Apache XML-RPC implementation for numerics and None. files: Doc/library/xmlrpc.client.rst | 17 +++++++- Doc/whatsnew/3.6.rst | 8 +++ Lib/test/test_xmlrpc.py | 49 +++++++++++++++++++++++ Lib/xmlrpc/client.py | 32 ++++++++++++-- Misc/NEWS | 3 + 5 files changed, 101 insertions(+), 8 deletions(-) diff --git a/Doc/library/xmlrpc.client.rst b/Doc/library/xmlrpc.client.rst --- a/Doc/library/xmlrpc.client.rst +++ b/Doc/library/xmlrpc.client.rst @@ -88,9 +88,13 @@ +======================+=======================================================+ | ``boolean`` | :class:`bool` | +----------------------+-------------------------------------------------------+ - | ``int`` or ``i4`` | :class:`int` in range from -2147483648 to 2147483647. | + | ``int``, ``i1``, | :class:`int` in range from -2147483648 to 2147483647. | + | ``i2``, ``i4``, | Values get the ```` tag. | + | ``i8`` or | | + | ``biginteger`` | | +----------------------+-------------------------------------------------------+ - | ``double`` | :class:`float` | + | ``double`` or | :class:`float`. Values get the ```` tag. | + | ``float`` | | +----------------------+-------------------------------------------------------+ | ``string`` | :class:`str` | +----------------------+-------------------------------------------------------+ @@ -114,6 +118,8 @@ | ``nil`` | The ``None`` constant. Passing is allowed only if | | | *allow_none* is true. | +----------------------+-------------------------------------------------------+ + | ``bigdecimal`` | :class:`decimal.Decimal`. Returned type only. | + +----------------------+-------------------------------------------------------+ This is the full set of data types supported by XML-RPC. Method calls may also raise a special :exc:`Fault` instance, used to signal XML-RPC server errors, or @@ -137,6 +143,13 @@ .. versionchanged:: 3.5 Added the *context* argument. + .. versionchanged:: 3.6 + Added support of type tags with prefixes (e.g.``ex:nil``). + Added support of unmarsalling additional types used by Apache XML-RPC + implementation for numerics: ``i1``, ``i2``, ``i8``, ``biginteger``, + ``float`` and ``bigdecimal``. + See http://ws.apache.org/xmlrpc/types.html for a description. + .. seealso:: diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -934,6 +934,14 @@ ` (:issue:`27982`). +xmlrpc.client +------------- + +The module now supports unmarshalling additional data types used by +Apache XML-RPC implementation for numerics and ``None``. +(Contributed by Serhiy Storchaka in :issue:`26885`.) + + zipfile ------- diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -1,5 +1,6 @@ import base64 import datetime +import decimal import sys import time import unittest @@ -237,6 +238,54 @@ '') self.assertRaises(ResponseError, xmlrpclib.loads, data) + def check_loads(self, s, value, **kwargs): + dump = '%s' % s + result, m = xmlrpclib.loads(dump, **kwargs) + (newvalue,) = result + self.assertEqual(newvalue, value) + self.assertIs(type(newvalue), type(value)) + self.assertIsNone(m) + + def test_load_standard_types(self): + check = self.check_loads + check('string', 'string') + check('string', 'string') + check('??????? string', '??????? string') + check('2056183947', 2056183947) + check('-2056183947', -2056183947) + check('2056183947', 2056183947) + check('46093.78125', 46093.78125) + check('0', False) + check('AGJ5dGUgc3RyaW5n/w==', + xmlrpclib.Binary(b'\x00byte string\xff')) + check('AGJ5dGUgc3RyaW5n/w==', + b'\x00byte string\xff', use_builtin_types=True) + check('20050210T11:41:23', + xmlrpclib.DateTime('20050210T11:41:23')) + check('20050210T11:41:23', + datetime.datetime(2005, 2, 10, 11, 41, 23), + use_builtin_types=True) + check('' + '12' + '', [1, 2]) + check('' + 'b2' + 'a1' + '', {'a': 1, 'b': 2}) + + def test_load_extension_types(self): + check = self.check_loads + check('', None) + check('', None) + check('205', 205) + check('20561', 20561) + check('9876543210', 9876543210) + check('98765432100123456789', + 98765432100123456789) + check('93.78125', 93.78125) + check('9876543210.0123456789', + decimal.Decimal('9876543210.0123456789')) + def test_get_host_info(self): # see bug #3613, this raised a TypeError transp = xmlrpc.client.Transport() diff --git a/Lib/xmlrpc/client.py b/Lib/xmlrpc/client.py --- a/Lib/xmlrpc/client.py +++ b/Lib/xmlrpc/client.py @@ -132,6 +132,7 @@ import sys import time from datetime import datetime +from decimal import Decimal import http.client import urllib.parse from xml.parsers import expat @@ -667,6 +668,8 @@ def start(self, tag, attrs): # prepare to handle this element + if ':' in tag: + tag = tag.split(':')[-1] if tag == "array" or tag == "struct": self._marks.append(len(self._stack)) self._data = [] @@ -682,9 +685,13 @@ try: f = self.dispatch[tag] except KeyError: - pass # unknown tag ? - else: - return f(self, "".join(self._data)) + if ':' not in tag: + return # unknown tag ? + try: + f = self.dispatch[tag.split(':')[-1]] + except KeyError: + return # unknown tag ? + return f(self, "".join(self._data)) # # accelerator support @@ -694,9 +701,13 @@ try: f = self.dispatch[tag] except KeyError: - pass # unknown tag ? - else: - return f(self, data) + if ':' not in tag: + return # unknown tag ? + try: + f = self.dispatch[tag.split(':')[-1]] + except KeyError: + return # unknown tag ? + return f(self, data) # # element decoders @@ -721,14 +732,23 @@ def end_int(self, data): self.append(int(data)) self._value = 0 + dispatch["i1"] = end_int + dispatch["i2"] = end_int dispatch["i4"] = end_int dispatch["i8"] = end_int dispatch["int"] = end_int + dispatch["biginteger"] = end_int def end_double(self, data): self.append(float(data)) self._value = 0 dispatch["double"] = end_double + dispatch["float"] = end_double + + def end_bigdecimal(self, data): + self.append(Decimal(data)) + self._value = 0 + dispatch["bigdecimal"] = end_bigdecimal def end_string(self, data): if self._encoding: diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -143,6 +143,9 @@ Library ------- +- Issue #26885: xmlrpc now supports unmarshalling additional data types used + by Apache XML-RPC implementation for numerics and None. + - Issue #28070: Fixed parsing inline verbose flag in regular expressions. - Issue #19500: Add client-side SSL session resumption to the ssl module. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 05:50:22 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 09:50:22 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2322493=3A_Inline_f?= =?utf-8?q?lags_now_should_be_used_only_at_the_start_of_the?= Message-ID: <20160911095020.20991.49896.44DB4E77@psf.io> https://hg.python.org/cpython/rev/31f8af1c3567 changeset: 103612:31f8af1c3567 user: Serhiy Storchaka date: Sun Sep 11 12:50:02 2016 +0300 summary: Issue #22493: Inline flags now should be used only at the start of the regular expression. Deprecation warning is emitted if uses them in the middle of the regular expression. files: Doc/library/re.rst | 8 ++------ Doc/whatsnew/3.6.rst | 9 +++++++++ Lib/distutils/filelist.py | 15 ++++++++++----- Lib/distutils/tests/test_filelist.py | 14 +++++++------- Lib/fnmatch.py | 2 +- Lib/http/cookies.py | 3 +-- Lib/sre_parse.py | 8 ++++++++ Lib/test/re_tests.py | 8 ++++---- Lib/test/test_fnmatch.py | 16 ++++++++-------- Lib/test/test_pyclbr.py | 2 +- Lib/test/test_re.py | 3 +++ Misc/NEWS | 4 ++++ 12 files changed, 58 insertions(+), 34 deletions(-) diff --git a/Doc/library/re.rst b/Doc/library/re.rst --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -224,12 +224,8 @@ flags are described in :ref:`contents-of-module-re`.) This is useful if you wish to include the flags as part of the regular expression, instead of passing a *flag* argument to the - :func:`re.compile` function. - - Note that the ``(?x)`` flag changes how the expression is parsed. It should be - used first in the expression string, or after one or more whitespace characters. - If there are non-whitespace characters before the flag, the results are - undefined. + :func:`re.compile` function. Flags should be used first in the + expression string. ``(?:...)`` A non-capturing version of regular parentheses. Matches whatever regular diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1124,6 +1124,15 @@ that will not be for several Python releases. (Contributed by Emanuel Barry in :issue:`27364`.) +* Inline flags ``(?letters)`` now should be used only at the start of the + regular expression. Inline flags in the middle of the regular expression + affects global flags in Python :mod:`re` module. This is an exception to + other regular expression engines that either apply flags to only part of + the regular expression or treat them as an error. To avoid distinguishing + inline flags in the middle of the regular expression now emit a deprecation + warning. It will be an error in future Python releases. + (Contributed by Serhiy Storchaka in :issue:`22493`.) + Deprecated Python behavior -------------------------- diff --git a/Lib/distutils/filelist.py b/Lib/distutils/filelist.py --- a/Lib/distutils/filelist.py +++ b/Lib/distutils/filelist.py @@ -302,21 +302,26 @@ else: return pattern + # ditch start and end characters + start, _, end = glob_to_re('_').partition('_') + if pattern: pattern_re = glob_to_re(pattern) + assert pattern_re.startswith(start) and pattern_re.endswith(end) else: pattern_re = '' if prefix is not None: - # ditch end of pattern character - empty_pattern = glob_to_re('') - prefix_re = glob_to_re(prefix)[:-len(empty_pattern)] + prefix_re = glob_to_re(prefix) + assert prefix_re.startswith(start) and prefix_re.endswith(end) + prefix_re = prefix_re[len(start): len(prefix_re) - len(end)] sep = os.sep if os.sep == '\\': sep = r'\\' - pattern_re = "^" + sep.join((prefix_re, ".*" + pattern_re)) + pattern_re = pattern_re[len(start): len(pattern_re) - len(end)] + pattern_re = r'%s\A%s%s.*%s%s' % (start, prefix_re, sep, pattern_re, end) else: # no prefix -- respect anchor flag if anchor: - pattern_re = "^" + pattern_re + pattern_re = r'%s\A%s' % (start, pattern_re[len(start):]) return re.compile(pattern_re) diff --git a/Lib/distutils/tests/test_filelist.py b/Lib/distutils/tests/test_filelist.py --- a/Lib/distutils/tests/test_filelist.py +++ b/Lib/distutils/tests/test_filelist.py @@ -51,14 +51,14 @@ for glob, regex in ( # simple cases - ('foo*', r'foo[^%(sep)s]*\Z(?ms)'), - ('foo?', r'foo[^%(sep)s]\Z(?ms)'), - ('foo??', r'foo[^%(sep)s][^%(sep)s]\Z(?ms)'), + ('foo*', r'(?s:foo[^%(sep)s]*)\Z'), + ('foo?', r'(?s:foo[^%(sep)s])\Z'), + ('foo??', r'(?s:foo[^%(sep)s][^%(sep)s])\Z'), # special cases - (r'foo\\*', r'foo\\\\[^%(sep)s]*\Z(?ms)'), - (r'foo\\\*', r'foo\\\\\\[^%(sep)s]*\Z(?ms)'), - ('foo????', r'foo[^%(sep)s][^%(sep)s][^%(sep)s][^%(sep)s]\Z(?ms)'), - (r'foo\\??', r'foo\\\\[^%(sep)s][^%(sep)s]\Z(?ms)')): + (r'foo\\*', r'(?s:foo\\\\[^%(sep)s]*)\Z'), + (r'foo\\\*', r'(?s:foo\\\\\\[^%(sep)s]*)\Z'), + ('foo????', r'(?s:foo[^%(sep)s][^%(sep)s][^%(sep)s][^%(sep)s])\Z'), + (r'foo\\??', r'(?s:foo\\\\[^%(sep)s][^%(sep)s])\Z')): regex = regex % {'sep': sep} self.assertEqual(glob_to_re(glob), regex) diff --git a/Lib/fnmatch.py b/Lib/fnmatch.py --- a/Lib/fnmatch.py +++ b/Lib/fnmatch.py @@ -106,4 +106,4 @@ res = '%s[%s]' % (res, stuff) else: res = res + re.escape(c) - return res + r'\Z(?ms)' + return r'(?s:%s)\Z' % res diff --git a/Lib/http/cookies.py b/Lib/http/cookies.py --- a/Lib/http/cookies.py +++ b/Lib/http/cookies.py @@ -458,7 +458,6 @@ _LegalKeyChars = r"\w\d!#%&'~_`><@,:/\$\*\+\-\.\^\|\)\(\?\}\{\=" _LegalValueChars = _LegalKeyChars + r'\[\]' _CookiePattern = re.compile(r""" - (?x) # This is a verbose pattern \s* # Optional whitespace at start of cookie (?P # Start of group 'key' [""" + _LegalKeyChars + r"""]+? # Any word of at least one letter @@ -475,7 +474,7 @@ )? # End of optional value group \s* # Any number of spaces. (\s+|;|$) # Ending either at space, semicolon, or EOS. - """, re.ASCII) # May be removed if safe. + """, re.ASCII | re.VERBOSE) # re.ASCII may be removed if safe. # At long last, here is the cookie class. Using this class is almost just like diff --git a/Lib/sre_parse.py b/Lib/sre_parse.py --- a/Lib/sre_parse.py +++ b/Lib/sre_parse.py @@ -279,6 +279,9 @@ break result += c return result + @property + def pos(self): + return self.index - len(self.next or '') def tell(self): return self.index - len(self.next or '') def seek(self, index): @@ -727,8 +730,13 @@ state.checklookbehindgroup(condgroup, source) elif char in FLAGS or char == "-": # flags + pos = source.pos flags = _parse_flags(source, state, char) if flags is None: # global flags + if pos != 3: # "(?x" + import warnings + warnings.warn('Flags not at the start of the expression', + DeprecationWarning, stacklevel=7) continue add_flags, del_flags = flags group = None diff --git a/Lib/test/re_tests.py b/Lib/test/re_tests.py --- a/Lib/test/re_tests.py +++ b/Lib/test/re_tests.py @@ -106,8 +106,8 @@ ('a.*b', 'acc\nccb', FAIL), ('a.{4,5}b', 'acc\nccb', FAIL), ('a.b', 'a\rb', SUCCEED, 'found', 'a\rb'), - ('a.b(?s)', 'a\nb', SUCCEED, 'found', 'a\nb'), - ('a.*(?s)b', 'acc\nccb', SUCCEED, 'found', 'acc\nccb'), + ('(?s)a.b', 'a\nb', SUCCEED, 'found', 'a\nb'), + ('(?s)a.*b', 'acc\nccb', SUCCEED, 'found', 'acc\nccb'), ('(?s)a.{4,5}b', 'acc\nccb', SUCCEED, 'found', 'acc\nccb'), ('(?s)a.b', 'a\nb', SUCCEED, 'found', 'a\nb'), @@ -563,7 +563,7 @@ # Check odd placement of embedded pattern modifiers # not an error under PCRE/PRE: - ('w(?i)', 'W', SUCCEED, 'found', 'W'), + ('(?i)w', 'W', SUCCEED, 'found', 'W'), # ('w(?i)', 'W', SYNTAX_ERROR), # Comments using the x embedded pattern modifier @@ -627,7 +627,7 @@ # bug 114033: nothing to repeat (r'(x?)?', 'x', SUCCEED, 'found', 'x'), # bug 115040: rescan if flags are modified inside pattern - (r' (?x)foo ', 'foo', SUCCEED, 'found', 'foo'), + (r'(?x) foo ', 'foo', SUCCEED, 'found', 'foo'), # bug 115618: negative lookahead (r'(? https://hg.python.org/cpython/rev/284676cf2ac8 changeset: 103613:284676cf2ac8 user: Berker Peksag date: Sun Sep 11 12:57:15 2016 +0300 summary: Issue #10740: sqlite3 no longer implicitly commit an open transaction before DDL statements This commit contains the following commits from ghaering/pysqlite: * https://github.com/ghaering/pysqlite/commit/f254c534948c41c0ceb8cbabf0d4a2f547754739 * https://github.com/ghaering/pysqlite/commit/796b3afe38cfdac5d7d5ec260826b0a596554631 * https://github.com/ghaering/pysqlite/commit/cae87ee68613697a5f4947b4a0941f59a28da1b6 * https://github.com/ghaering/pysqlite/commit/3567b31bb5e5b226ba006213a9c69dde3f155faf With the following additions: * Fixed a refcount error * Fixed a compiler warning * Made the string comparison a little more robust * Added a whatsnew entry files: Doc/library/sqlite3.rst | 7 +- Doc/whatsnew/3.6.rst | 3 + Lib/sqlite3/test/transactions.py | 36 ++++- Misc/NEWS | 3 + Modules/_sqlite/cursor.c | 125 +++--------------- Modules/_sqlite/cursor.h | 6 - Modules/_sqlite/statement.c | 18 ++ Modules/_sqlite/statement.h | 1 + 8 files changed, 83 insertions(+), 116 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -925,9 +925,7 @@ By default, the :mod:`sqlite3` module opens transactions implicitly before a Data Modification Language (DML) statement (i.e. -``INSERT``/``UPDATE``/``DELETE``/``REPLACE``), and commits transactions -implicitly before a non-DML, non-query statement (i. e. -anything other than ``SELECT`` or the aforementioned). +``INSERT``/``UPDATE``/``DELETE``/``REPLACE``). So if you are within a transaction and issue a command like ``CREATE TABLE ...``, ``VACUUM``, ``PRAGMA``, the :mod:`sqlite3` module will commit implicitly @@ -947,6 +945,9 @@ statement, or set it to one of SQLite's supported isolation levels: "DEFERRED", "IMMEDIATE" or "EXCLUSIVE". +.. versionchanged:: 3.6 + :mod:`sqlite3` used to implicitly commit an open transaction before DDL + statements. This is no longer the case. Using :mod:`sqlite3` efficiently diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -1195,6 +1195,9 @@ Changes in the Python API ------------------------- +* :mod:`sqlite3` no longer implicitly commit an open transaction before DDL + statements. + * On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool is initialized to increase the security. diff --git a/Lib/sqlite3/test/transactions.py b/Lib/sqlite3/test/transactions.py --- a/Lib/sqlite3/test/transactions.py +++ b/Lib/sqlite3/test/transactions.py @@ -52,13 +52,13 @@ except OSError: pass - def CheckDMLdoesAutoCommitBefore(self): + def CheckDMLDoesNotAutoCommitBefore(self): self.cur1.execute("create table test(i)") self.cur1.execute("insert into test(i) values (5)") self.cur1.execute("create table test2(j)") self.cur2.execute("select i from test") res = self.cur2.fetchall() - self.assertEqual(len(res), 1) + self.assertEqual(len(res), 0) def CheckInsertStartsTransaction(self): self.cur1.execute("create table test(i)") @@ -153,11 +153,6 @@ self.con = sqlite.connect(":memory:") self.cur = self.con.cursor() - def CheckVacuum(self): - self.cur.execute("create table test(i)") - self.cur.execute("insert into test(i) values (5)") - self.cur.execute("vacuum") - def CheckDropTable(self): self.cur.execute("create table test(i)") self.cur.execute("insert into test(i) values (5)") @@ -172,10 +167,35 @@ self.cur.close() self.con.close() +class TransactionalDDL(unittest.TestCase): + def setUp(self): + self.con = sqlite.connect(":memory:") + + def CheckDdlDoesNotAutostartTransaction(self): + # For backwards compatibility reasons, DDL statements should not + # implicitly start a transaction. + self.con.execute("create table test(i)") + self.con.rollback() + result = self.con.execute("select * from test").fetchall() + self.assertEqual(result, []) + + def CheckTransactionalDDL(self): + # You can achieve transactional DDL by issuing a BEGIN + # statement manually. + self.con.execute("begin") + self.con.execute("create table test(i)") + self.con.rollback() + with self.assertRaises(sqlite.OperationalError): + self.con.execute("select * from test") + + def tearDown(self): + self.con.close() + def suite(): default_suite = unittest.makeSuite(TransactionTests, "Check") special_command_suite = unittest.makeSuite(SpecialCommandTests, "Check") - return unittest.TestSuite((default_suite, special_command_suite)) + ddl_suite = unittest.makeSuite(TransactionalDDL, "Check") + return unittest.TestSuite((default_suite, special_command_suite, ddl_suite)) def test(): runner = unittest.TextTestRunner() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -143,6 +143,9 @@ Library ------- +- Issue #10740: sqlite3 no longer implicitly commit an open transaction + before DDL statements. + - Issue #22493: Inline flags now should be used only at the start of the regular expression. Deprecation warning is emitted if uses them in the middle of the regular expression. diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -29,44 +29,6 @@ static const char errmsg_fetch_across_rollback[] = "Cursor needed to be reset because of commit/rollback and can no longer be fetched from."; -static pysqlite_StatementKind detect_statement_type(const char* statement) -{ - char buf[20]; - const char* src; - char* dst; - - src = statement; - /* skip over whitepace */ - while (*src == '\r' || *src == '\n' || *src == ' ' || *src == '\t') { - src++; - } - - if (*src == 0) - return STATEMENT_INVALID; - - dst = buf; - *dst = 0; - while (Py_ISALPHA(*src) && (dst - buf) < ((Py_ssize_t)sizeof(buf) - 2)) { - *dst++ = Py_TOLOWER(*src++); - } - - *dst = 0; - - if (!strcmp(buf, "select")) { - return STATEMENT_SELECT; - } else if (!strcmp(buf, "insert")) { - return STATEMENT_INSERT; - } else if (!strcmp(buf, "update")) { - return STATEMENT_UPDATE; - } else if (!strcmp(buf, "delete")) { - return STATEMENT_DELETE; - } else if (!strcmp(buf, "replace")) { - return STATEMENT_REPLACE; - } else { - return STATEMENT_OTHER; - } -} - static int pysqlite_cursor_init(pysqlite_Cursor* self, PyObject* args, PyObject* kwargs) { pysqlite_Connection* connection; @@ -427,9 +389,9 @@ PyObject* func_args; PyObject* result; int numcols; - int statement_type; PyObject* descriptor; PyObject* second_argument = NULL; + sqlite_int64 lastrowid; if (!check_cursor(self)) { goto error; @@ -510,7 +472,7 @@ /* reset description and rowcount */ Py_INCREF(Py_None); Py_SETREF(self->description, Py_None); - self->rowcount = -1L; + self->rowcount = 0L; func_args = PyTuple_New(1); if (!func_args) { @@ -549,43 +511,19 @@ pysqlite_statement_reset(self->statement); pysqlite_statement_mark_dirty(self->statement); - statement_type = detect_statement_type(operation_cstr); - if (self->connection->begin_statement) { - switch (statement_type) { - case STATEMENT_UPDATE: - case STATEMENT_DELETE: - case STATEMENT_INSERT: - case STATEMENT_REPLACE: - if (!self->connection->inTransaction) { - result = _pysqlite_connection_begin(self->connection); - if (!result) { - goto error; - } - Py_DECREF(result); - } - break; - case STATEMENT_OTHER: - /* it's a DDL statement or something similar - - we better COMMIT first so it works for all cases */ - if (self->connection->inTransaction) { - result = pysqlite_connection_commit(self->connection, NULL); - if (!result) { - goto error; - } - Py_DECREF(result); - } - break; - case STATEMENT_SELECT: - if (multiple) { - PyErr_SetString(pysqlite_ProgrammingError, - "You cannot execute SELECT statements in executemany()."); - goto error; - } - break; + /* For backwards compatibility reasons, do not start a transaction if a + DDL statement is encountered. If anybody wants transactional DDL, + they can issue a BEGIN statement manually. */ + if (self->connection->begin_statement && !sqlite3_stmt_readonly(self->statement->st) && !self->statement->is_ddl) { + if (sqlite3_get_autocommit(self->connection->db)) { + result = _pysqlite_connection_begin(self->connection); + if (!result) { + goto error; + } + Py_DECREF(result); } } - while (1) { parameters = PyIter_Next(parameters_iter); if (!parameters) { @@ -671,6 +609,20 @@ } } + if (!sqlite3_stmt_readonly(self->statement->st)) { + self->rowcount += (long)sqlite3_changes(self->connection->db); + } else { + self->rowcount= -1L; + } + + if (!multiple) { + Py_DECREF(self->lastrowid); + Py_BEGIN_ALLOW_THREADS + lastrowid = sqlite3_last_insert_rowid(self->connection->db); + Py_END_ALLOW_THREADS + self->lastrowid = _pysqlite_long_from_int64(lastrowid); + } + if (rc == SQLITE_ROW) { if (multiple) { PyErr_SetString(pysqlite_ProgrammingError, "executemany() can only execute DML statements."); @@ -685,31 +637,6 @@ Py_CLEAR(self->statement); } - switch (statement_type) { - case STATEMENT_UPDATE: - case STATEMENT_DELETE: - case STATEMENT_INSERT: - case STATEMENT_REPLACE: - if (self->rowcount == -1L) { - self->rowcount = 0L; - } - self->rowcount += (long)sqlite3_changes(self->connection->db); - } - - Py_DECREF(self->lastrowid); - if (!multiple && - /* REPLACE is an alias for INSERT OR REPLACE */ - (statement_type == STATEMENT_INSERT || statement_type == STATEMENT_REPLACE)) { - sqlite_int64 lastrowid; - Py_BEGIN_ALLOW_THREADS - lastrowid = sqlite3_last_insert_rowid(self->connection->db); - Py_END_ALLOW_THREADS - self->lastrowid = _pysqlite_long_from_int64(lastrowid); - } else { - Py_INCREF(Py_None); - self->lastrowid = Py_None; - } - if (multiple) { pysqlite_statement_reset(self->statement); } diff --git a/Modules/_sqlite/cursor.h b/Modules/_sqlite/cursor.h --- a/Modules/_sqlite/cursor.h +++ b/Modules/_sqlite/cursor.h @@ -51,12 +51,6 @@ PyObject* in_weakreflist; /* List of weak references */ } pysqlite_Cursor; -typedef enum { - STATEMENT_INVALID, STATEMENT_INSERT, STATEMENT_DELETE, - STATEMENT_UPDATE, STATEMENT_REPLACE, STATEMENT_SELECT, - STATEMENT_OTHER -} pysqlite_StatementKind; - extern PyTypeObject pysqlite_CursorType; PyObject* pysqlite_cursor_execute(pysqlite_Cursor* self, PyObject* args); diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -54,6 +54,7 @@ int rc; const char* sql_cstr; Py_ssize_t sql_cstr_len; + const char* p; self->st = NULL; self->in_use = 0; @@ -72,6 +73,23 @@ Py_INCREF(sql); self->sql = sql; + /* determine if the statement is a DDL statement */ + self->is_ddl = 0; + for (p = sql_cstr; *p != 0; p++) { + switch (*p) { + case ' ': + case '\r': + case '\n': + case '\t': + continue; + } + + self->is_ddl = (PyOS_strnicmp(p, "create ", 7) == 0) + || (PyOS_strnicmp(p, "drop ", 5) == 0) + || (PyOS_strnicmp(p, "reindex ", 8) == 0); + break; + } + Py_BEGIN_ALLOW_THREADS rc = sqlite3_prepare(connection->db, sql_cstr, diff --git a/Modules/_sqlite/statement.h b/Modules/_sqlite/statement.h --- a/Modules/_sqlite/statement.h +++ b/Modules/_sqlite/statement.h @@ -38,6 +38,7 @@ sqlite3_stmt* st; PyObject* sql; int in_use; + int is_ddl; PyObject* in_weakreflist; /* List of weak references */ } pysqlite_Statement; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 06:02:23 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 11 Sep 2016 10:02:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327991=3A_Merge_from_3=2E5?= Message-ID: <20160911100222.59448.13329.326F13D0@psf.io> https://hg.python.org/cpython/rev/9a75fa28bd0a changeset: 103615:9a75fa28bd0a parent: 103613:284676cf2ac8 parent: 103614:0e45cd7859ec user: Berker Peksag date: Sun Sep 11 13:02:56 2016 +0300 summary: Issue #27991: Merge from 3.5 files: Doc/howto/argparse.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst --- a/Doc/howto/argparse.rst +++ b/Doc/howto/argparse.rst @@ -503,8 +503,8 @@ * Now here's a demonstration of what the "count" action gives. You've probably seen this sort of usage before. -* And, just like the "store_true" action, if you don't specify the ``-v`` flag, - that flag is considered to have ``None`` value. +* And if you don't specify the ``-v`` flag, that flag is considered to have + ``None`` value. * As should be expected, specifying the long form of the flag, we should get the same output. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 06:02:26 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 11 Sep 2016 10:02:26 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTkx?= =?utf-8?q?=3A_Remove_incorrect_sentence_about_store=5Ftrue?= Message-ID: <20160911100222.21311.27616.547BCF6A@psf.io> https://hg.python.org/cpython/rev/0e45cd7859ec changeset: 103614:0e45cd7859ec branch: 3.5 parent: 103606:c6e45f1f76b5 user: Berker Peksag date: Sun Sep 11 13:02:27 2016 +0300 summary: Issue #27991: Remove incorrect sentence about store_true Patch by py.user. files: Doc/howto/argparse.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst --- a/Doc/howto/argparse.rst +++ b/Doc/howto/argparse.rst @@ -503,8 +503,8 @@ * Now here's a demonstration of what the "count" action gives. You've probably seen this sort of usage before. -* And, just like the "store_true" action, if you don't specify the ``-v`` flag, - that flag is considered to have ``None`` value. +* And if you don't specify the ``-v`` flag, that flag is considered to have + ``None`` value. * As should be expected, specifying the long form of the flag, we should get the same output. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 06:07:29 2016 From: python-checkins at python.org (martin.panter) Date: Sun, 11 Sep 2016 10:07:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2327952=3A_Merge_fixcid=2Epy_from_3=2E5?= Message-ID: <20160911100729.59651.39318.B9212CFE@psf.io> https://hg.python.org/cpython/rev/8dde3dcab1e8 changeset: 103618:8dde3dcab1e8 parent: 103615:9a75fa28bd0a parent: 103617:ca68bb597df9 user: Martin Panter date: Sun Sep 11 10:06:38 2016 +0000 summary: Issue #27952: Merge fixcid.py from 3.5 files: Lib/test/test_tools/test_fixcid.py | 91 ++++++++++++++++++ Misc/NEWS | 7 + Tools/scripts/fixcid.py | 52 +++++---- 3 files changed, 125 insertions(+), 25 deletions(-) diff --git a/Lib/test/test_tools/test_fixcid.py b/Lib/test/test_tools/test_fixcid.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_tools/test_fixcid.py @@ -0,0 +1,91 @@ +'''Test Tools/scripts/fixcid.py.''' + +from io import StringIO +import os, os.path +import runpy +import sys +from test import support +from test.test_tools import skip_if_missing, scriptsdir +import unittest + +skip_if_missing() + +class Test(unittest.TestCase): + def test_parse_strings(self): + old1 = 'int xx = "xx\\"xx"[xx];\n' + old2 = "int xx = 'x\\'xx' + xx;\n" + output = self.run_script(old1 + old2) + new1 = 'int yy = "xx\\"xx"[yy];\n' + new2 = "int yy = 'x\\'xx' + yy;\n" + self.assertMultiLineEqual(output, + "1\n" + "< {old1}" + "> {new1}" + "{new1}" + "2\n" + "< {old2}" + "> {new2}" + "{new2}".format(old1=old1, old2=old2, new1=new1, new2=new2) + ) + + def test_alter_comments(self): + output = self.run_script( + substfile= + "xx yy\n" + "*aa bb\n", + args=("-c", "-",), + input= + "/* xx altered */\n" + "int xx;\n" + "/* aa unaltered */\n" + "int aa;\n", + ) + self.assertMultiLineEqual(output, + "1\n" + "< /* xx altered */\n" + "> /* yy altered */\n" + "/* yy altered */\n" + "2\n" + "< int xx;\n" + "> int yy;\n" + "int yy;\n" + "/* aa unaltered */\n" + "4\n" + "< int aa;\n" + "> int bb;\n" + "int bb;\n" + ) + + def test_directory(self): + os.mkdir(support.TESTFN) + self.addCleanup(support.rmtree, support.TESTFN) + c_filename = os.path.join(support.TESTFN, "file.c") + with open(c_filename, "w") as file: + file.write("int xx;\n") + with open(os.path.join(support.TESTFN, "file.py"), "w") as file: + file.write("xx = 'unaltered'\n") + script = os.path.join(scriptsdir, "fixcid.py") + output = self.run_script(args=(support.TESTFN,)) + self.assertMultiLineEqual(output, + "{}:\n" + "1\n" + '< int xx;\n' + '> int yy;\n'.format(c_filename) + ) + + def run_script(self, input="", *, args=("-",), substfile="xx yy\n"): + substfilename = support.TESTFN + ".subst" + with open(substfilename, "w") as file: + file.write(substfile) + self.addCleanup(support.unlink, substfilename) + + argv = ["fixcid.py", "-s", substfilename] + list(args) + script = os.path.join(scriptsdir, "fixcid.py") + with support.swap_attr(sys, "argv", argv), \ + support.swap_attr(sys, "stdin", StringIO(input)), \ + support.captured_stdout() as output: + try: + runpy.run_path(script, run_name="__main__") + except SystemExit as exit: + self.assertEqual(exit.code, 0) + return output.getvalue() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -429,6 +429,13 @@ - Issue #21122: Fix LTO builds on OS X. +Tools/Demos +----------- + +- Issue #27952: Get Tools/scripts/fixcid.py working with Python 3 and the + current "re" module, avoid invalid Python backslash escapes, and fix a bug + parsing escaped C quote signs. + Windows ------- diff --git a/Tools/scripts/fixcid.py b/Tools/scripts/fixcid.py --- a/Tools/scripts/fixcid.py +++ b/Tools/scripts/fixcid.py @@ -88,9 +88,9 @@ sys.exit(bad) # Change this regular expression to select a different set of files -Wanted = '^[a-zA-Z0-9_]+\.[ch]$' +Wanted = r'^[a-zA-Z0-9_]+\.[ch]$' def wanted(name): - return re.match(Wanted, name) >= 0 + return re.match(Wanted, name) def recursedown(dirname): dbg('recursedown(%r)\n' % (dirname,)) @@ -168,6 +168,7 @@ if filename == '-': return 0 # Done in filter mode f.close() if not g: return 0 # No changes + g.close() # Finishing touch -- move files @@ -193,21 +194,21 @@ # Tokenizing ANSI C (partly) -Identifier = '\(struct \)?[a-zA-Z_][a-zA-Z0-9_]+' -String = '"\([^\n\\"]\|\\\\.\)*"' -Char = '\'\([^\n\\\']\|\\\\.\)*\'' -CommentStart = '/\*' -CommentEnd = '\*/' +Identifier = '(struct )?[a-zA-Z_][a-zA-Z0-9_]+' +String = r'"([^\n\\"]|\\.)*"' +Char = r"'([^\n\\']|\\.)*'" +CommentStart = r'/\*' +CommentEnd = r'\*/' Hexnumber = '0[xX][0-9a-fA-F]*[uUlL]*' Octnumber = '0[0-7]*[uUlL]*' Decnumber = '[1-9][0-9]*[uUlL]*' -Intnumber = Hexnumber + '\|' + Octnumber + '\|' + Decnumber +Intnumber = Hexnumber + '|' + Octnumber + '|' + Decnumber Exponent = '[eE][-+]?[0-9]+' -Pointfloat = '\([0-9]+\.[0-9]*\|\.[0-9]+\)\(' + Exponent + '\)?' +Pointfloat = r'([0-9]+\.[0-9]*|\.[0-9]+)(' + Exponent + r')?' Expfloat = '[0-9]+' + Exponent -Floatnumber = Pointfloat + '\|' + Expfloat -Number = Floatnumber + '\|' + Intnumber +Floatnumber = Pointfloat + '|' + Expfloat +Number = Floatnumber + '|' + Intnumber # Anything else is an operator -- don't list this explicitly because of '/*' @@ -225,15 +226,16 @@ def fixline(line): global Program -## print '-->', repr(line) +## print('-->', repr(line)) i = 0 while i < len(line): - i = Program.search(line, i) - if i < 0: break - found = Program.group(0) -## if Program is InsideCommentProgram: print '...', -## else: print ' ', -## print found + match = Program.search(line, i) + if match is None: break + i = match.start() + found = match.group(0) +## if Program is InsideCommentProgram: print(end='... ') +## else: print(end=' ') +## print(found) if len(found) == 2: if found == '/*': Program = InsideCommentProgram @@ -247,15 +249,15 @@ print('Found in comment:', found) i = i + n continue - if NotInComment.has_key(found): -## print 'Ignored in comment:', -## print found, '-->', subst -## print 'Line:', line, + if found in NotInComment: +## print(end='Ignored in comment: ') +## print(found, '-->', subst) +## print('Line:', line, end='') subst = found ## else: -## print 'Substituting in comment:', -## print found, '-->', subst -## print 'Line:', line, +## print(end='Substituting in comment: ') +## print(found, '-->', subst) +## print('Line:', line, end='') line = line[:i] + subst + line[i+n:] n = len(subst) i = i + n -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 06:07:29 2016 From: python-checkins at python.org (martin.panter) Date: Sun, 11 Sep 2016 10:07:29 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3OTUy?= =?utf-8?q?=3A_Get_fixcid=2Epy_working_with_the_re_module?= Message-ID: <20160911100729.48439.85673.D74E792D@psf.io> https://hg.python.org/cpython/rev/ca68bb597df9 changeset: 103617:ca68bb597df9 branch: 3.5 parent: 103614:0e45cd7859ec user: Martin Panter date: Sun Sep 11 09:32:26 2016 +0000 summary: Issue #27952: Get fixcid.py working with the re module files: Lib/test/test_tools/test_fixcid.py | 91 ++++++++++++++++++ Misc/NEWS | 4 + Tools/scripts/fixcid.py | 52 +++++---- 3 files changed, 122 insertions(+), 25 deletions(-) diff --git a/Lib/test/test_tools/test_fixcid.py b/Lib/test/test_tools/test_fixcid.py new file mode 100644 --- /dev/null +++ b/Lib/test/test_tools/test_fixcid.py @@ -0,0 +1,91 @@ +'''Test Tools/scripts/fixcid.py.''' + +from io import StringIO +import os, os.path +import runpy +import sys +from test import support +from test.test_tools import skip_if_missing, scriptsdir +import unittest + +skip_if_missing() + +class Test(unittest.TestCase): + def test_parse_strings(self): + old1 = 'int xx = "xx\\"xx"[xx];\n' + old2 = "int xx = 'x\\'xx' + xx;\n" + output = self.run_script(old1 + old2) + new1 = 'int yy = "xx\\"xx"[yy];\n' + new2 = "int yy = 'x\\'xx' + yy;\n" + self.assertMultiLineEqual(output, + "1\n" + "< {old1}" + "> {new1}" + "{new1}" + "2\n" + "< {old2}" + "> {new2}" + "{new2}".format(old1=old1, old2=old2, new1=new1, new2=new2) + ) + + def test_alter_comments(self): + output = self.run_script( + substfile= + "xx yy\n" + "*aa bb\n", + args=("-c", "-",), + input= + "/* xx altered */\n" + "int xx;\n" + "/* aa unaltered */\n" + "int aa;\n", + ) + self.assertMultiLineEqual(output, + "1\n" + "< /* xx altered */\n" + "> /* yy altered */\n" + "/* yy altered */\n" + "2\n" + "< int xx;\n" + "> int yy;\n" + "int yy;\n" + "/* aa unaltered */\n" + "4\n" + "< int aa;\n" + "> int bb;\n" + "int bb;\n" + ) + + def test_directory(self): + os.mkdir(support.TESTFN) + self.addCleanup(support.rmtree, support.TESTFN) + c_filename = os.path.join(support.TESTFN, "file.c") + with open(c_filename, "w") as file: + file.write("int xx;\n") + with open(os.path.join(support.TESTFN, "file.py"), "w") as file: + file.write("xx = 'unaltered'\n") + script = os.path.join(scriptsdir, "fixcid.py") + output = self.run_script(args=(support.TESTFN,)) + self.assertMultiLineEqual(output, + "{}:\n" + "1\n" + '< int xx;\n' + '> int yy;\n'.format(c_filename) + ) + + def run_script(self, input="", *, args=("-",), substfile="xx yy\n"): + substfilename = support.TESTFN + ".subst" + with open(substfilename, "w") as file: + file.write(substfile) + self.addCleanup(support.unlink, substfilename) + + argv = ["fixcid.py", "-s", substfilename] + list(args) + script = os.path.join(scriptsdir, "fixcid.py") + with support.swap_attr(sys, "argv", argv), \ + support.swap_attr(sys, "stdin", StringIO(input)), \ + support.captured_stdout() as output: + try: + runpy.run_path(script, run_name="__main__") + except SystemExit as exit: + self.assertEqual(exit.code, 0) + return output.getvalue() diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -293,6 +293,10 @@ Tools/Demos ----------- +- Issue #27952: Get Tools/scripts/fixcid.py working with Python 3 and the + current "re" module, avoid invalid Python backslash escapes, and fix a bug + parsing escaped C quote signs. + - Issue #27332: Fixed the type of the first argument of module-level functions generated by Argument Clinic. Patch by Petr Viktorin. diff --git a/Tools/scripts/fixcid.py b/Tools/scripts/fixcid.py --- a/Tools/scripts/fixcid.py +++ b/Tools/scripts/fixcid.py @@ -88,9 +88,9 @@ sys.exit(bad) # Change this regular expression to select a different set of files -Wanted = '^[a-zA-Z0-9_]+\.[ch]$' +Wanted = r'^[a-zA-Z0-9_]+\.[ch]$' def wanted(name): - return re.match(Wanted, name) >= 0 + return re.match(Wanted, name) def recursedown(dirname): dbg('recursedown(%r)\n' % (dirname,)) @@ -168,6 +168,7 @@ if filename == '-': return 0 # Done in filter mode f.close() if not g: return 0 # No changes + g.close() # Finishing touch -- move files @@ -193,21 +194,21 @@ # Tokenizing ANSI C (partly) -Identifier = '\(struct \)?[a-zA-Z_][a-zA-Z0-9_]+' -String = '"\([^\n\\"]\|\\\\.\)*"' -Char = '\'\([^\n\\\']\|\\\\.\)*\'' -CommentStart = '/\*' -CommentEnd = '\*/' +Identifier = '(struct )?[a-zA-Z_][a-zA-Z0-9_]+' +String = r'"([^\n\\"]|\\.)*"' +Char = r"'([^\n\\']|\\.)*'" +CommentStart = r'/\*' +CommentEnd = r'\*/' Hexnumber = '0[xX][0-9a-fA-F]*[uUlL]*' Octnumber = '0[0-7]*[uUlL]*' Decnumber = '[1-9][0-9]*[uUlL]*' -Intnumber = Hexnumber + '\|' + Octnumber + '\|' + Decnumber +Intnumber = Hexnumber + '|' + Octnumber + '|' + Decnumber Exponent = '[eE][-+]?[0-9]+' -Pointfloat = '\([0-9]+\.[0-9]*\|\.[0-9]+\)\(' + Exponent + '\)?' +Pointfloat = r'([0-9]+\.[0-9]*|\.[0-9]+)(' + Exponent + r')?' Expfloat = '[0-9]+' + Exponent -Floatnumber = Pointfloat + '\|' + Expfloat -Number = Floatnumber + '\|' + Intnumber +Floatnumber = Pointfloat + '|' + Expfloat +Number = Floatnumber + '|' + Intnumber # Anything else is an operator -- don't list this explicitly because of '/*' @@ -225,15 +226,16 @@ def fixline(line): global Program -## print '-->', repr(line) +## print('-->', repr(line)) i = 0 while i < len(line): - i = Program.search(line, i) - if i < 0: break - found = Program.group(0) -## if Program is InsideCommentProgram: print '...', -## else: print ' ', -## print found + match = Program.search(line, i) + if match is None: break + i = match.start() + found = match.group(0) +## if Program is InsideCommentProgram: print(end='... ') +## else: print(end=' ') +## print(found) if len(found) == 2: if found == '/*': Program = InsideCommentProgram @@ -247,15 +249,15 @@ print('Found in comment:', found) i = i + n continue - if NotInComment.has_key(found): -## print 'Ignored in comment:', -## print found, '-->', subst -## print 'Line:', line, + if found in NotInComment: +## print(end='Ignored in comment: ') +## print(found, '-->', subst) +## print('Line:', line, end='') subst = found ## else: -## print 'Substituting in comment:', -## print found, '-->', subst -## print 'Line:', line, +## print(end='Substituting in comment: ') +## print(found, '-->', subst) +## print('Line:', line, end='') line = line[:i] + subst + line[i+n:] n = len(subst) i = i + n -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 06:07:30 2016 From: python-checkins at python.org (martin.panter) Date: Sun, 11 Sep 2016 10:07:30 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI3OTUy?= =?utf-8?q?=3A_Get_fixcid=2Epy_working_with_the_re_module?= Message-ID: <20160911100729.12514.13573.50F4144C@psf.io> https://hg.python.org/cpython/rev/740e43eb8138 changeset: 103616:740e43eb8138 branch: 2.7 parent: 103593:afc0d4478083 user: Martin Panter date: Sun Sep 11 09:48:57 2016 +0000 summary: Issue #27952: Get fixcid.py working with the re module files: Lib/test/test_tools.py | 83 +++++++++++++++++++++++++++++ Misc/NEWS | 7 ++ Tools/scripts/fixcid.py | 30 +++++---- 3 files changed, 106 insertions(+), 14 deletions(-) diff --git a/Lib/test/test_tools.py b/Lib/test/test_tools.py --- a/Lib/test/test_tools.py +++ b/Lib/test/test_tools.py @@ -5,9 +5,11 @@ """ import os +import runpy import sys import unittest import shutil +from cStringIO import StringIO import subprocess import sysconfig import tempfile @@ -359,6 +361,87 @@ self.pindent_test(clean, closed) +class FixcidTests(unittest.TestCase): + def test_parse_strings(self): + old1 = 'int xx = "xx\\"xx"[xx];\n' + old2 = "int xx = 'x\\'xx' + xx;\n" + output = self.run_script(old1 + old2) + new1 = 'int yy = "xx\\"xx"[yy];\n' + new2 = "int yy = 'x\\'xx' + yy;\n" + self.assertMultiLineEqual(output, + "1\n" + "< {old1}" + "> {new1}" + "{new1}" + "2\n" + "< {old2}" + "> {new2}" + "{new2}".format(old1=old1, old2=old2, new1=new1, new2=new2) + ) + + def test_alter_comments(self): + output = self.run_script( + substfile= + "xx yy\n" + "*aa bb\n", + args=("-c", "-",), + input= + "/* xx altered */\n" + "int xx;\n" + "/* aa unaltered */\n" + "int aa;\n", + ) + self.assertMultiLineEqual(output, + "1\n" + "< /* xx altered */\n" + "> /* yy altered */\n" + "/* yy altered */\n" + "2\n" + "< int xx;\n" + "> int yy;\n" + "int yy;\n" + "/* aa unaltered */\n" + "4\n" + "< int aa;\n" + "> int bb;\n" + "int bb;\n" + ) + + def test_directory(self): + os.mkdir(test_support.TESTFN) + self.addCleanup(test_support.rmtree, test_support.TESTFN) + c_filename = os.path.join(test_support.TESTFN, "file.c") + with open(c_filename, "w") as file: + file.write("int xx;\n") + with open(os.path.join(test_support.TESTFN, "file.py"), "w") as file: + file.write("xx = 'unaltered'\n") + script = os.path.join(scriptsdir, "fixcid.py") + output = self.run_script(args=(test_support.TESTFN,)) + self.assertMultiLineEqual(output, + "{}:\n" + "1\n" + '< int xx;\n' + '> int yy;\n'.format(c_filename) + ) + + def run_script(self, input="", args=("-",), substfile="xx yy\n"): + substfilename = test_support.TESTFN + ".subst" + with open(substfilename, "w") as file: + file.write(substfile) + self.addCleanup(test_support.unlink, substfilename) + + argv = ["fixcid.py", "-s", substfilename] + list(args) + script = os.path.join(scriptsdir, "fixcid.py") + with test_support.swap_attr(sys, "argv", argv), \ + test_support.swap_attr(sys, "stdin", StringIO(input)), \ + test_support.captured_stdout() as output: + try: + runpy.run_path(script, run_name="__main__") + except SystemExit as exit: + self.assertEqual(exit.code, 0) + return output.getvalue() + + def test_main(): test_support.run_unittest(*[obj for obj in globals().values() if isinstance(obj, type)]) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -191,6 +191,13 @@ - Issue #10910: Avoid C++ compilation errors on FreeBSD and OS X. Also update FreedBSD version checks for the original ctype UTF-8 workaround. +Tools/Demos +----------- + +- Issue #27952: Get Tools/scripts/fixcid.py working with the current "re" + module, avoid invalid Python backslash escapes, and fix a bug parsing + escaped C quote signs. + Windows ------- diff --git a/Tools/scripts/fixcid.py b/Tools/scripts/fixcid.py --- a/Tools/scripts/fixcid.py +++ b/Tools/scripts/fixcid.py @@ -88,9 +88,9 @@ sys.exit(bad) # Change this regular expression to select a different set of files -Wanted = '^[a-zA-Z0-9_]+\.[ch]$' +Wanted = r'^[a-zA-Z0-9_]+\.[ch]$' def wanted(name): - return re.match(Wanted, name) >= 0 + return re.match(Wanted, name) def recursedown(dirname): dbg('recursedown(%r)\n' % (dirname,)) @@ -168,6 +168,7 @@ if filename == '-': return 0 # Done in filter mode f.close() if not g: return 0 # No changes + g.close() # Finishing touch -- move files @@ -193,21 +194,21 @@ # Tokenizing ANSI C (partly) -Identifier = '\(struct \)?[a-zA-Z_][a-zA-Z0-9_]+' -String = '"\([^\n\\"]\|\\\\.\)*"' -Char = '\'\([^\n\\\']\|\\\\.\)*\'' -CommentStart = '/\*' -CommentEnd = '\*/' +Identifier = '(struct )?[a-zA-Z_][a-zA-Z0-9_]+' +String = r'"([^\n\\"]|\\.)*"' +Char = r"'([^\n\\']|\\.)*'" +CommentStart = r'/\*' +CommentEnd = r'\*/' Hexnumber = '0[xX][0-9a-fA-F]*[uUlL]*' Octnumber = '0[0-7]*[uUlL]*' Decnumber = '[1-9][0-9]*[uUlL]*' -Intnumber = Hexnumber + '\|' + Octnumber + '\|' + Decnumber +Intnumber = Hexnumber + '|' + Octnumber + '|' + Decnumber Exponent = '[eE][-+]?[0-9]+' -Pointfloat = '\([0-9]+\.[0-9]*\|\.[0-9]+\)\(' + Exponent + '\)?' +Pointfloat = r'([0-9]+\.[0-9]*|\.[0-9]+)(' + Exponent + r')?' Expfloat = '[0-9]+' + Exponent -Floatnumber = Pointfloat + '\|' + Expfloat -Number = Floatnumber + '\|' + Intnumber +Floatnumber = Pointfloat + '|' + Expfloat +Number = Floatnumber + '|' + Intnumber # Anything else is an operator -- don't list this explicitly because of '/*' @@ -228,9 +229,10 @@ ## print '-->', repr(line) i = 0 while i < len(line): - i = Program.search(line, i) - if i < 0: break - found = Program.group(0) + match = Program.search(line, i) + if match is None: break + i = match.start() + found = match.group(0) ## if Program is InsideCommentProgram: print '...', ## else: print ' ', ## print found -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 06:48:49 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 10:48:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327129=3A_Replaced?= =?utf-8?q?_wordcode_related_magic_constants_with_macros=2E?= Message-ID: <20160911104849.1723.54565.E1EFD55B@psf.io> https://hg.python.org/cpython/rev/dd046963bd42 changeset: 103619:dd046963bd42 user: Serhiy Storchaka date: Sun Sep 11 13:48:15 2016 +0300 summary: Issue #27129: Replaced wordcode related magic constants with macros. files: Include/code.h | 10 + Objects/frameobject.c | 4 +- Objects/genobject.c | 4 +- Python/ceval.c | 48 ++--- Python/compile.c | 13 +- Python/peephole.c | 217 +++++++++++++------------ Python/wordcode_helpers.h | 35 ++- 7 files changed, 173 insertions(+), 158 deletions(-) diff --git a/Include/code.h b/Include/code.h --- a/Include/code.h +++ b/Include/code.h @@ -7,6 +7,16 @@ extern "C" { #endif +typedef uint16_t _Py_CODEUNIT; + +#ifdef WORDS_BIGENDIAN +# define _Py_OPCODE(word) ((word) >> 8) +# define _Py_OPARG(word) ((word) & 255) +#else +# define _Py_OPCODE(word) ((word) & 255) +# define _Py_OPARG(word) ((word) >> 8) +#endif + /* Bytecode object */ typedef struct { PyObject_HEAD diff --git a/Objects/frameobject.c b/Objects/frameobject.c --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -189,7 +189,7 @@ memset(blockstack, '\0', sizeof(blockstack)); memset(in_finally, '\0', sizeof(in_finally)); blockstack_top = 0; - for (addr = 0; addr < code_len; addr += 2) { + for (addr = 0; addr < code_len; addr += sizeof(_Py_CODEUNIT)) { unsigned char op = code[addr]; switch (op) { case SETUP_LOOP: @@ -273,7 +273,7 @@ * can tell whether the jump goes into any blocks without coming out * again - in that case we raise an exception below. */ delta_iblock = 0; - for (addr = min_addr; addr < max_addr; addr += 2) { + for (addr = min_addr; addr < max_addr; addr += sizeof(_Py_CODEUNIT)) { unsigned char op = code[addr]; switch (op) { case SETUP_LOOP: diff --git a/Objects/genobject.c b/Objects/genobject.c --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -390,7 +390,7 @@ PyObject *bytecode = f->f_code->co_code; unsigned char *code = (unsigned char *)PyBytes_AS_STRING(bytecode); - if (code[f->f_lasti + 2] != YIELD_FROM) + if (code[f->f_lasti + sizeof(_Py_CODEUNIT)] != YIELD_FROM) return NULL; yf = f->f_stacktop[-1]; Py_INCREF(yf); @@ -498,7 +498,7 @@ assert(ret == yf); Py_DECREF(ret); /* Termination repetition of YIELD_FROM */ - gen->gi_frame->f_lasti += 2; + gen->gi_frame->f_lasti += sizeof(_Py_CODEUNIT); if (_PyGen_FetchStopIterationValue(&val) == 0) { ret = gen_send_ex(gen, val, 0, 0); Py_DECREF(val); diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -62,7 +62,7 @@ static void format_exc_check_arg(PyObject *, const char *, PyObject *); static void format_exc_unbound(PyCodeObject *co, int oparg); static PyObject * unicode_concatenate(PyObject *, PyObject *, - PyFrameObject *, const unsigned short *); + PyFrameObject *, const _Py_CODEUNIT *); static PyObject * special_lookup(PyObject *, _Py_Identifier *); #define NAME_ERROR_MSG \ @@ -725,7 +725,7 @@ int lastopcode = 0; #endif PyObject **stack_pointer; /* Next free slot in value stack */ - const unsigned short *next_instr; + const _Py_CODEUNIT *next_instr; int opcode; /* Current opcode */ int oparg; /* Current opcode argument, if any */ enum why_code why; /* Reason for block stack unwind */ @@ -743,7 +743,7 @@ time it is tested. */ int instr_ub = -1, instr_lb = 0, instr_prev = -1; - const unsigned short *first_instr; + const _Py_CODEUNIT *first_instr; PyObject *names; PyObject *consts; @@ -864,23 +864,16 @@ /* Code access macros */ -#ifdef WORDS_BIGENDIAN - #define OPCODE(word) ((word) >> 8) - #define OPARG(word) ((word) & 255) -#else - #define OPCODE(word) ((word) & 255) - #define OPARG(word) ((word) >> 8) -#endif /* The integer overflow is checked by an assertion below. */ -#define INSTR_OFFSET() (2*(int)(next_instr - first_instr)) +#define INSTR_OFFSET() (sizeof(_Py_CODEUNIT) * (int)(next_instr - first_instr)) #define NEXTOPARG() do { \ - unsigned short word = *next_instr; \ - opcode = OPCODE(word); \ - oparg = OPARG(word); \ + _Py_CODEUNIT word = *next_instr; \ + opcode = _Py_OPCODE(word); \ + oparg = _Py_OPARG(word); \ next_instr++; \ } while (0) -#define JUMPTO(x) (next_instr = first_instr + (x)/2) -#define JUMPBY(x) (next_instr += (x)/2) +#define JUMPTO(x) (next_instr = first_instr + (x) / sizeof(_Py_CODEUNIT)) +#define JUMPBY(x) (next_instr += (x) / sizeof(_Py_CODEUNIT)) /* OpCode prediction macros Some opcodes tend to come in pairs thus making it possible to @@ -913,10 +906,10 @@ #else #define PREDICT(op) \ do{ \ - unsigned short word = *next_instr; \ - opcode = OPCODE(word); \ + _Py_CODEUNIT word = *next_instr; \ + opcode = _Py_OPCODE(word); \ if (opcode == op){ \ - oparg = OPARG(word); \ + oparg = _Py_OPARG(word); \ next_instr++; \ goto PRED_##op; \ } \ @@ -1056,9 +1049,9 @@ freevars = f->f_localsplus + co->co_nlocals; assert(PyBytes_Check(co->co_code)); assert(PyBytes_GET_SIZE(co->co_code) <= INT_MAX); - assert(PyBytes_GET_SIZE(co->co_code) % 2 == 0); - assert(_Py_IS_ALIGNED(PyBytes_AS_STRING(co->co_code), 2)); - first_instr = (unsigned short*) PyBytes_AS_STRING(co->co_code); + assert(PyBytes_GET_SIZE(co->co_code) % sizeof(_Py_CODEUNIT) == 0); + assert(_Py_IS_ALIGNED(PyBytes_AS_STRING(co->co_code), sizeof(_Py_CODEUNIT))); + first_instr = (_Py_CODEUNIT *) PyBytes_AS_STRING(co->co_code); /* f->f_lasti refers to the index of the last instruction, unless it's -1 in which case next_instr should be first_instr. @@ -1074,10 +1067,11 @@ FOR_ITER is effectively a single opcode and f->f_lasti will point to the beginning of the combined pair.) */ + assert(f->f_lasti >= -1); next_instr = first_instr; if (f->f_lasti >= 0) { - assert(f->f_lasti % 2 == 0); - next_instr += f->f_lasti/2 + 1; + assert(f->f_lasti % sizeof(_Py_CODEUNIT) == 0); + next_instr += f->f_lasti / sizeof(_Py_CODEUNIT) + 1; } stack_pointer = f->f_stacktop; assert(stack_pointer != NULL); @@ -1125,7 +1119,7 @@ Py_MakePendingCalls() above. */ if (_Py_atomic_load_relaxed(&eval_breaker)) { - if (OPCODE(*next_instr) == SETUP_FINALLY) { + if (_Py_OPCODE(*next_instr) == SETUP_FINALLY) { /* Make the last opcode before a try: finally: block uninterruptible. */ goto fast_next_opcode; @@ -2049,7 +2043,7 @@ f->f_stacktop = stack_pointer; why = WHY_YIELD; /* and repeat... */ - f->f_lasti -= 2; + f->f_lasti -= sizeof(_Py_CODEUNIT); goto fast_yield; } @@ -5321,7 +5315,7 @@ static PyObject * unicode_concatenate(PyObject *v, PyObject *w, - PyFrameObject *f, const unsigned short *next_instr) + PyFrameObject *f, const _Py_CODEUNIT *next_instr) { PyObject *res; if (Py_REFCNT(v) == 2) { diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -4948,7 +4948,7 @@ Py_ssize_t len; unsigned char *lnotab; - d_bytecode = a->a_offset - a->a_lineno_off; + d_bytecode = (a->a_offset - a->a_lineno_off) * sizeof(_Py_CODEUNIT); d_lineno = i->i_lineno - a->a_lineno; assert(d_bytecode >= 0); @@ -5055,21 +5055,21 @@ { int size, arg = 0; Py_ssize_t len = PyBytes_GET_SIZE(a->a_bytecode); - char *code; + _Py_CODEUNIT *code; arg = i->i_oparg; size = instrsize(arg); if (i->i_lineno && !assemble_lnotab(a, i)) return 0; - if (a->a_offset + size >= len) { + if (a->a_offset + size >= len / (int)sizeof(_Py_CODEUNIT)) { if (len > PY_SSIZE_T_MAX / 2) return 0; if (_PyBytes_Resize(&a->a_bytecode, len * 2) < 0) return 0; } - code = PyBytes_AS_STRING(a->a_bytecode) + a->a_offset; + code = (_Py_CODEUNIT *)PyBytes_AS_STRING(a->a_bytecode) + a->a_offset; a->a_offset += size; - write_op_arg((unsigned char*)code, i->i_opcode, arg, size); + write_op_arg(code, i->i_opcode, arg, size); return 1; } @@ -5106,6 +5106,7 @@ if (instr->i_jrel) { instr->i_oparg -= bsize; } + instr->i_oparg *= sizeof(_Py_CODEUNIT); if (instrsize(instr->i_oparg) != isize) { extended_arg_recompile = 1; } @@ -5351,7 +5352,7 @@ if (_PyBytes_Resize(&a.a_lnotab, a.a_lnotab_off) < 0) goto error; - if (_PyBytes_Resize(&a.a_bytecode, a.a_offset) < 0) + if (_PyBytes_Resize(&a.a_bytecode, a.a_offset * sizeof(_Py_CODEUNIT)) < 0) goto error; co = makecode(c, &a); diff --git a/Python/peephole.c b/Python/peephole.c --- a/Python/peephole.c +++ b/Python/peephole.c @@ -17,7 +17,8 @@ || op==POP_JUMP_IF_FALSE || op==POP_JUMP_IF_TRUE \ || op==JUMP_IF_FALSE_OR_POP || op==JUMP_IF_TRUE_OR_POP) #define JUMPS_ON_TRUE(op) (op==POP_JUMP_IF_TRUE || op==JUMP_IF_TRUE_OR_POP) -#define GETJUMPTGT(arr, i) (get_arg(arr, i) + (ABSOLUTE_JUMP(arr[i]) ? 0 : i+2)) +#define GETJUMPTGT(arr, i) (get_arg(arr, i) / sizeof(_Py_CODEUNIT) + \ + (ABSOLUTE_JUMP(_Py_OPCODE(arr[i])) ? 0 : i+1)) #define ISBASICBLOCK(blocks, start, end) \ (blocks[start]==blocks[end]) @@ -40,7 +41,7 @@ #define CONST_STACK_PUSH_OP(i) do { \ PyObject *_x; \ - assert(codestr[i] == LOAD_CONST); \ + assert(_Py_OPCODE(codestr[i]) == LOAD_CONST); \ assert(PyList_GET_SIZE(consts) > (Py_ssize_t)get_arg(codestr, i)); \ _x = PyList_GET_ITEM(consts, get_arg(codestr, i)); \ if (++const_stack_top >= const_stack_size) { \ @@ -72,33 +73,33 @@ Callers are responsible to check CONST_STACK_LEN beforehand. */ static Py_ssize_t -lastn_const_start(unsigned char *codestr, Py_ssize_t i, Py_ssize_t n) +lastn_const_start(const _Py_CODEUNIT *codestr, Py_ssize_t i, Py_ssize_t n) { - assert(n > 0 && (i&1) == 0); + assert(n > 0); for (;;) { - i -= 2; + i--; assert(i >= 0); - if (codestr[i] == LOAD_CONST) { + if (_Py_OPCODE(codestr[i]) == LOAD_CONST) { if (!--n) { - while (i > 0 && codestr[i-2] == EXTENDED_ARG) { - i -= 2; + while (i > 0 && _Py_OPCODE(codestr[i-1]) == EXTENDED_ARG) { + i--; } return i; } } else { - assert(codestr[i] == NOP || codestr[i] == EXTENDED_ARG); + assert(_Py_OPCODE(codestr[i]) == NOP || + _Py_OPCODE(codestr[i]) == EXTENDED_ARG); } } } /* Scans through EXTENDED ARGs, seeking the index of the effective opcode */ static Py_ssize_t -find_op(unsigned char *codestr, Py_ssize_t i) +find_op(const _Py_CODEUNIT *codestr, Py_ssize_t i) { - assert((i&1) == 0); - while (codestr[i] == EXTENDED_ARG) { - i += 2; + while (_Py_OPCODE(codestr[i]) == EXTENDED_ARG) { + i++; } return i; } @@ -106,27 +107,34 @@ /* Given the index of the effective opcode, scan back to construct the oparg with EXTENDED_ARG */ static unsigned int -get_arg(unsigned char *codestr, Py_ssize_t i) +get_arg(const _Py_CODEUNIT *codestr, Py_ssize_t i) { - unsigned int oparg = codestr[i+1]; - assert((i&1) == 0); - if (i >= 2 && codestr[i-2] == EXTENDED_ARG) { - oparg |= codestr[i-1] << 8; - if (i >= 4 && codestr[i-4] == EXTENDED_ARG) { - oparg |= codestr[i-3] << 16; - if (i >= 6 && codestr[i-6] == EXTENDED_ARG) { - oparg |= codestr[i-5] << 24; + _Py_CODEUNIT word; + unsigned int oparg = _Py_OPARG(codestr[i]); + if (i >= 1 && _Py_OPCODE(word = codestr[i-1]) == EXTENDED_ARG) { + oparg |= _Py_OPARG(word) << 8; + if (i >= 2 && _Py_OPCODE(word = codestr[i-2]) == EXTENDED_ARG) { + oparg |= _Py_OPARG(word) << 16; + if (i >= 3 && _Py_OPCODE(word = codestr[i-3]) == EXTENDED_ARG) { + oparg |= _Py_OPARG(word) << 24; } } } return oparg; } +/* Fill the region with NOPs. */ +static void +fill_nops(_Py_CODEUNIT *codestr, Py_ssize_t start, Py_ssize_t end) +{ + memset(codestr + start, NOP, (end - start) * sizeof(_Py_CODEUNIT)); +} + /* Given the index of the effective opcode, attempt to replace the argument, taking into account EXTENDED_ARG. Returns -1 on failure, or the new op index on success */ static Py_ssize_t -set_arg(unsigned char *codestr, Py_ssize_t i, unsigned int oparg) +set_arg(_Py_CODEUNIT *codestr, Py_ssize_t i, unsigned int oparg) { unsigned int curarg = get_arg(codestr, i); int curilen, newilen; @@ -138,8 +146,8 @@ return -1; } - write_op_arg(codestr + i + 2 - curilen, codestr[i], oparg, newilen); - memset(codestr + i + 2 - curilen + newilen, NOP, curilen - newilen); + write_op_arg(codestr + i + 1 - curilen, _Py_OPCODE(codestr[i]), oparg, newilen); + fill_nops(codestr, i + 1 - curilen + newilen, i + 1); return i-curilen+newilen; } @@ -147,17 +155,16 @@ Preceding memory in the region is overwritten with NOPs. Returns -1 on failure, op index on success */ static Py_ssize_t -copy_op_arg(unsigned char *codestr, Py_ssize_t i, unsigned char op, +copy_op_arg(_Py_CODEUNIT *codestr, Py_ssize_t i, unsigned char op, unsigned int oparg, Py_ssize_t maxi) { int ilen = instrsize(oparg); - assert((i&1) == 0); if (i + ilen > maxi) { return -1; } write_op_arg(codestr + maxi - ilen, op, oparg, ilen); - memset(codestr + i, NOP, maxi - i - ilen); - return maxi - 2; + fill_nops(codestr, i, maxi - ilen); + return maxi - 1; } /* Replace LOAD_CONST c1, LOAD_CONST c2 ... LOAD_CONST cn, BUILD_TUPLE n @@ -170,7 +177,7 @@ test; for BUILD_SET it assembles a frozenset rather than a tuple. */ static Py_ssize_t -fold_tuple_on_constants(unsigned char *codestr, Py_ssize_t c_start, +fold_tuple_on_constants(_Py_CODEUNIT *codestr, Py_ssize_t c_start, Py_ssize_t opcode_end, unsigned char opcode, PyObject *consts, PyObject **objs, int n) { @@ -222,7 +229,7 @@ becoming large in the presence of code like: (None,)*1000. */ static Py_ssize_t -fold_binops_on_constants(unsigned char *codestr, Py_ssize_t c_start, +fold_binops_on_constants(_Py_CODEUNIT *codestr, Py_ssize_t c_start, Py_ssize_t opcode_end, unsigned char opcode, PyObject *consts, PyObject **objs) { @@ -311,7 +318,7 @@ } static Py_ssize_t -fold_unaryops_on_constants(unsigned char *codestr, Py_ssize_t c_start, +fold_unaryops_on_constants(_Py_CODEUNIT *codestr, Py_ssize_t c_start, Py_ssize_t opcode_end, unsigned char opcode, PyObject *consts, PyObject *v) { @@ -359,7 +366,7 @@ } static unsigned int * -markblocks(unsigned char *code, Py_ssize_t len) +markblocks(_Py_CODEUNIT *code, Py_ssize_t len) { unsigned int *blocks = PyMem_New(unsigned int, len); int i, j, opcode, blockcnt = 0; @@ -371,8 +378,8 @@ memset(blocks, 0, len*sizeof(int)); /* Mark labels in the first pass */ - for (i=0 ; i= 2 && codestr[op_start-2] == EXTENDED_ARG) { - op_start -= 2; + while (op_start >= 1 && _Py_OPCODE(codestr[op_start-1]) == EXTENDED_ARG) { + op_start--; } - nexti = i + 2; - while (nexti < codelen && codestr[nexti] == EXTENDED_ARG) - nexti += 2; - nextop = nexti < codelen ? codestr[nexti] : 0; + nexti = i + 1; + while (nexti < codelen && _Py_OPCODE(codestr[nexti]) == EXTENDED_ARG) + nexti++; + nextop = nexti < codelen ? _Py_OPCODE(codestr[nexti]) : 0; if (!in_consts) { CONST_STACK_RESET(); @@ -488,10 +496,10 @@ with POP_JUMP_IF_TRUE */ case UNARY_NOT: if (nextop != POP_JUMP_IF_FALSE - || !ISBASICBLOCK(blocks, op_start, i+2)) + || !ISBASICBLOCK(blocks, op_start, i + 1)) break; - memset(codestr + op_start, NOP, i - op_start + 2); - codestr[nexti] = POP_JUMP_IF_TRUE; + fill_nops(codestr, op_start, i + 1); + codestr[nexti] = PACKOPARG(POP_JUMP_IF_TRUE, _Py_OPARG(codestr[nexti])); break; /* not a is b --> a is not b @@ -503,10 +511,10 @@ j = get_arg(codestr, i); if (j < 6 || j > 9 || nextop != UNARY_NOT || - !ISBASICBLOCK(blocks, op_start, i + 2)) + !ISBASICBLOCK(blocks, op_start, i + 1)) break; - codestr[i+1] = (j^1); - memset(codestr + i + 2, NOP, nexti - i); + codestr[i] = PACKOPARG(opcode, j^1); + fill_nops(codestr, i + 1, nexti + 1); break; /* Skip over LOAD_CONST trueconst @@ -515,10 +523,10 @@ case LOAD_CONST: CONST_STACK_PUSH_OP(i); if (nextop != POP_JUMP_IF_FALSE || - !ISBASICBLOCK(blocks, op_start, i + 2) || + !ISBASICBLOCK(blocks, op_start, i + 1) || !PyObject_IsTrue(PyList_GET_ITEM(consts, get_arg(codestr, i)))) break; - memset(codestr + op_start, NOP, nexti - op_start + 2); + fill_nops(codestr, op_start, nexti + 1); CONST_STACK_POP(1); break; @@ -537,10 +545,10 @@ ISBASICBLOCK(blocks, h, op_start)) || ((opcode == BUILD_LIST || opcode == BUILD_SET) && ((nextop==COMPARE_OP && - (codestr[nexti+1]==6 || - codestr[nexti+1]==7)) || - nextop == GET_ITER) && ISBASICBLOCK(blocks, h, i + 2))) { - h = fold_tuple_on_constants(codestr, h, i+2, opcode, + (_Py_OPARG(codestr[nexti]) == PyCmp_IN || + _Py_OPARG(codestr[nexti]) == PyCmp_NOT_IN)) || + nextop == GET_ITER) && ISBASICBLOCK(blocks, h, i + 1))) { + h = fold_tuple_on_constants(codestr, h, i + 1, opcode, consts, CONST_STACK_LASTN(j), j); if (h >= 0) { CONST_STACK_POP(j); @@ -550,23 +558,20 @@ } } if (nextop != UNPACK_SEQUENCE || - !ISBASICBLOCK(blocks, op_start, i + 2) || + !ISBASICBLOCK(blocks, op_start, i + 1) || j != get_arg(codestr, nexti) || opcode == BUILD_SET) break; if (j < 2) { - memset(codestr+op_start, NOP, nexti - op_start + 2); + fill_nops(codestr, op_start, nexti + 1); } else if (j == 2) { - codestr[op_start] = ROT_TWO; - codestr[op_start + 1] = 0; - memset(codestr + op_start + 2, NOP, nexti - op_start); + codestr[op_start] = PACKOPARG(ROT_TWO, 0); + fill_nops(codestr, op_start + 1, nexti + 1); CONST_STACK_RESET(); } else if (j == 3) { - codestr[op_start] = ROT_THREE; - codestr[op_start + 1] = 0; - codestr[op_start + 2] = ROT_TWO; - codestr[op_start + 3] = 0; - memset(codestr + op_start + 4, NOP, nexti - op_start - 2); + codestr[op_start] = PACKOPARG(ROT_THREE, 0); + codestr[op_start + 1] = PACKOPARG(ROT_TWO, 0); + fill_nops(codestr, op_start + 2, nexti + 1); CONST_STACK_RESET(); } break; @@ -590,7 +595,7 @@ break; h = lastn_const_start(codestr, op_start, 2); if (ISBASICBLOCK(blocks, h, op_start)) { - h = fold_binops_on_constants(codestr, h, i+2, opcode, + h = fold_binops_on_constants(codestr, h, i + 1, opcode, consts, CONST_STACK_LASTN(2)); if (h >= 0) { CONST_STACK_POP(2); @@ -608,7 +613,7 @@ break; h = lastn_const_start(codestr, op_start, 1); if (ISBASICBLOCK(blocks, h, op_start)) { - h = fold_unaryops_on_constants(codestr, h, i+2, opcode, + h = fold_unaryops_on_constants(codestr, h, i + 1, opcode, consts, *CONST_STACK_LASTN(1)); if (h >= 0) { CONST_STACK_POP(1); @@ -628,15 +633,15 @@ x:JUMP_IF_FALSE_OR_POP y y:JUMP_IF_FALSE_OR_POP z --> x:JUMP_IF_FALSE_OR_POP z x:JUMP_IF_FALSE_OR_POP y y:JUMP_IF_TRUE_OR_POP z - --> x:POP_JUMP_IF_FALSE y+2 - where y+2 is the instruction following the second test. + --> x:POP_JUMP_IF_FALSE y+1 + where y+1 is the instruction following the second test. */ case JUMP_IF_FALSE_OR_POP: case JUMP_IF_TRUE_OR_POP: - h = get_arg(codestr, i); + h = get_arg(codestr, i) / sizeof(_Py_CODEUNIT); tgt = find_op(codestr, h); - j = codestr[tgt]; + j = _Py_OPCODE(codestr[tgt]); if (CONDITIONAL_JUMP(j)) { /* NOTE: all possible jumps here are absolute. */ if (JUMPS_ON_TRUE(j) == JUMPS_ON_TRUE(opcode)) { @@ -649,14 +654,14 @@ jump past it), and all conditional jumps pop their argument when they're not taken (so change the first jump to pop its argument when it's taken). */ - h = set_arg(codestr, i, tgt + 2); + h = set_arg(codestr, i, (tgt + 1) * sizeof(_Py_CODEUNIT)); j = opcode == JUMP_IF_TRUE_OR_POP ? POP_JUMP_IF_TRUE : POP_JUMP_IF_FALSE; } if (h >= 0) { nexti = h; - codestr[nexti] = j; + codestr[nexti] = PACKOPARG(j, _Py_OPARG(codestr[nexti])); break; } } @@ -678,32 +683,32 @@ tgt = find_op(codestr, h); /* Replace JUMP_* to a RETURN into just a RETURN */ if (UNCONDITIONAL_JUMP(opcode) && - codestr[tgt] == RETURN_VALUE) { - codestr[op_start] = RETURN_VALUE; - codestr[op_start + 1] = 0; - memset(codestr + op_start + 2, NOP, i - op_start); - } else if (UNCONDITIONAL_JUMP(codestr[tgt])) { + _Py_OPCODE(codestr[tgt]) == RETURN_VALUE) { + codestr[op_start] = PACKOPARG(RETURN_VALUE, 0); + fill_nops(codestr, op_start + 1, i + 1); + } else if (UNCONDITIONAL_JUMP(_Py_OPCODE(codestr[tgt]))) { j = GETJUMPTGT(codestr, tgt); if (opcode == JUMP_FORWARD) { /* JMP_ABS can go backwards */ opcode = JUMP_ABSOLUTE; } else if (!ABSOLUTE_JUMP(opcode)) { - if ((Py_ssize_t)j < i + 2) { + if ((Py_ssize_t)j < i + 1) { break; /* No backward relative jumps */ } - j -= i + 2; /* Calc relative jump addr */ + j -= i + 1; /* Calc relative jump addr */ } - copy_op_arg(codestr, op_start, opcode, j, i+2); + j *= sizeof(_Py_CODEUNIT); + copy_op_arg(codestr, op_start, opcode, j, i + 1); } break; /* Remove unreachable ops after RETURN */ case RETURN_VALUE: - h = i + 2; - while (h + 2 < codelen && ISBASICBLOCK(blocks, i, h + 2)) { - h += 2; + h = i + 1; + while (h + 1 < codelen && ISBASICBLOCK(blocks, i, h + 1)) { + h++; } - if (h > i + 2) { - memset(codestr + i + 2, NOP, h - i); + if (h > i + 1) { + fill_nops(codestr, i + 1, h + 1); nexti = find_op(codestr, h); } break; @@ -711,20 +716,21 @@ } /* Fixup lnotab */ - for (i=0, nops=0 ; i new code offset */ blocks[i] = i - nops; - if (codestr[i] == NOP) - nops += 2; + if (_Py_OPCODE(codestr[i]) == NOP) + nops++; } cum_orig_offset = 0; last_offset = 0; for (i=0 ; i < tabsiz ; i+=2) { unsigned int offset_delta, new_offset; cum_orig_offset += lnotab[i]; - assert((cum_orig_offset & 1) == 0); - new_offset = blocks[cum_orig_offset]; + assert(cum_orig_offset % sizeof(_Py_CODEUNIT) == 0); + new_offset = blocks[cum_orig_offset / sizeof(_Py_CODEUNIT)] * + sizeof(_Py_CODEUNIT); offset_delta = new_offset - last_offset; assert(offset_delta <= 255); lnotab[i] = (unsigned char)offset_delta; @@ -732,13 +738,13 @@ } /* Remove NOPs and fixup jump targets */ - for (op_start=0, i=0, h=0 ; i nexti) goto exitUnchanged; /* If instrsize(j) < nexti, we'll emit EXTENDED_ARG 0 */ @@ -772,7 +779,7 @@ CONST_STACK_DELETE(); PyMem_Free(blocks); - code = PyBytes_FromStringAndSize((char *)codestr, h); + code = PyBytes_FromStringAndSize((char *)codestr, h * sizeof(_Py_CODEUNIT)); PyMem_Free(codestr); return code; diff --git a/Python/wordcode_helpers.h b/Python/wordcode_helpers.h --- a/Python/wordcode_helpers.h +++ b/Python/wordcode_helpers.h @@ -2,35 +2,38 @@ optimizer. */ -/* Minimum number of bytes necessary to encode instruction with EXTENDED_ARGs */ +#ifdef WORDS_BIGENDIAN +# define PACKOPARG(opcode, oparg) ((_Py_CODEUNIT)(((opcode) << 8) | (oparg))) +#else +# define PACKOPARG(opcode, oparg) ((_Py_CODEUNIT)(((oparg) << 8) | (opcode))) +#endif + +/* Minimum number of code units necessary to encode instruction with + EXTENDED_ARGs */ static int instrsize(unsigned int oparg) { - return oparg <= 0xff ? 2 : - oparg <= 0xffff ? 4 : - oparg <= 0xffffff ? 6 : - 8; + return oparg <= 0xff ? 1 : + oparg <= 0xffff ? 2 : + oparg <= 0xffffff ? 3 : + 4; } /* Spits out op/oparg pair using ilen bytes. codestr should be pointed at the desired location of the first EXTENDED_ARG */ static void -write_op_arg(unsigned char *codestr, unsigned char opcode, +write_op_arg(_Py_CODEUNIT *codestr, unsigned char opcode, unsigned int oparg, int ilen) { switch (ilen) { - case 8: - *codestr++ = EXTENDED_ARG; - *codestr++ = (oparg >> 24) & 0xff; - case 6: - *codestr++ = EXTENDED_ARG; - *codestr++ = (oparg >> 16) & 0xff; case 4: - *codestr++ = EXTENDED_ARG; - *codestr++ = (oparg >> 8) & 0xff; + *codestr++ = PACKOPARG(EXTENDED_ARG, (oparg >> 24) & 0xff); + case 3: + *codestr++ = PACKOPARG(EXTENDED_ARG, (oparg >> 16) & 0xff); case 2: - *codestr++ = opcode; - *codestr++ = oparg & 0xff; + *codestr++ = PACKOPARG(EXTENDED_ARG, (oparg >> 8) & 0xff); + case 1: + *codestr++ = PACKOPARG(opcode, oparg & 0xff); break; default: assert(0); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 07:41:18 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 11:41:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Use_sequence_repetition_in?= =?utf-8?q?stead_of_bytes_constructor_with_integer_argument=2E?= Message-ID: <20160911114118.1897.22081.9BFFD73E@psf.io> https://hg.python.org/cpython/rev/aac69574afde changeset: 103620:aac69574afde user: Serhiy Storchaka date: Sun Sep 11 14:41:02 2016 +0300 summary: Use sequence repetition instead of bytes constructor with integer argument. files: Lib/base64.py | 2 +- Lib/gzip.py | 4 ++-- Lib/hmac.py | 2 +- Lib/pickletools.py | 2 +- Lib/test/test_bufio.py | 2 +- Lib/test/test_bz2.py | 4 ++-- Lib/test/test_gzip.py | 4 ++-- Lib/test/test_io.py | 2 +- Lib/test/test_ipaddress.py | 10 +++++----- Lib/test/test_lzma.py | 4 ++-- Lib/test/test_socketserver.py | 2 +- Lib/test/test_wsgiref.py | 2 +- 12 files changed, 20 insertions(+), 20 deletions(-) diff --git a/Lib/base64.py b/Lib/base64.py --- a/Lib/base64.py +++ b/Lib/base64.py @@ -155,7 +155,7 @@ leftover = len(s) % 5 # Pad the last quantum with zero bits if necessary if leftover: - s = s + bytes(5 - leftover) # Don't use += ! + s = s + b'\0' * (5 - leftover) # Don't use += ! encoded = bytearray() from_bytes = int.from_bytes b32tab2 = _b32tab2 diff --git a/Lib/gzip.py b/Lib/gzip.py --- a/Lib/gzip.py +++ b/Lib/gzip.py @@ -357,10 +357,10 @@ if offset < self.offset: raise OSError('Negative seek in write mode') count = offset - self.offset - chunk = bytes(1024) + chunk = b'\0' * 1024 for i in range(count // 1024): self.write(chunk) - self.write(bytes(count % 1024)) + self.write(b'\0' * (count % 1024)) elif self.mode == READ: self._check_not_closed() return self._buffer.seek(offset, whence) diff --git a/Lib/hmac.py b/Lib/hmac.py --- a/Lib/hmac.py +++ b/Lib/hmac.py @@ -77,7 +77,7 @@ if len(key) > blocksize: key = self.digest_cons(key).digest() - key = key + bytes(blocksize - len(key)) + key = key.ljust(blocksize, b'\0') self.outer.update(key.translate(trans_5C)) self.inner.update(key.translate(trans_36)) if msg is not None: diff --git a/Lib/pickletools.py b/Lib/pickletools.py --- a/Lib/pickletools.py +++ b/Lib/pickletools.py @@ -707,7 +707,7 @@ >>> enc = s.encode('utf-8') >>> enc b'abcd\xea\xaf\x8d' - >>> n = bytes([len(enc)]) + bytes(7) # little-endian 8-byte length + >>> n = bytes([len(enc)]) + b'\0' * 7 # little-endian 8-byte length >>> t = read_unicodestring8(io.BytesIO(n + enc + b'junk')) >>> s == t True diff --git a/Lib/test/test_bufio.py b/Lib/test/test_bufio.py --- a/Lib/test/test_bufio.py +++ b/Lib/test/test_bufio.py @@ -59,7 +59,7 @@ self.drive_one(b"1234567890\00\01\02\03\04\05\06") def test_nullpat(self): - self.drive_one(bytes(1000)) + self.drive_one(b'\0' * 1000) class CBufferSizeTest(BufferSizeTest, unittest.TestCase): diff --git a/Lib/test/test_bz2.py b/Lib/test/test_bz2.py --- a/Lib/test/test_bz2.py +++ b/Lib/test/test_bz2.py @@ -562,11 +562,11 @@ def testDecompressLimited(self): """Decompressed data buffering should be limited""" - bomb = bz2.compress(bytes(int(2e6)), compresslevel=9) + bomb = bz2.compress(b'\0' * int(2e6), compresslevel=9) self.assertLess(len(bomb), _compression.BUFFER_SIZE) decomp = BZ2File(BytesIO(bomb)) - self.assertEqual(bytes(1), decomp.read(1)) + self.assertEqual(decomp.read(1), b'\0') max_decomp = 1 + DEFAULT_BUFFER_SIZE self.assertLessEqual(decomp._buffer.raw.tell(), max_decomp, "Excessive amount of data was decompressed") diff --git a/Lib/test/test_gzip.py b/Lib/test/test_gzip.py --- a/Lib/test/test_gzip.py +++ b/Lib/test/test_gzip.py @@ -434,12 +434,12 @@ def test_decompress_limited(self): """Decompressed data buffering should be limited""" - bomb = gzip.compress(bytes(int(2e6)), compresslevel=9) + bomb = gzip.compress(b'\0' * int(2e6), compresslevel=9) self.assertLess(len(bomb), io.DEFAULT_BUFFER_SIZE) bomb = io.BytesIO(bomb) decomp = gzip.GzipFile(fileobj=bomb) - self.assertEqual(bytes(1), decomp.read(1)) + self.assertEqual(decomp.read(1), b'\0') max_decomp = 1 + io.DEFAULT_BUFFER_SIZE self.assertLessEqual(decomp._buffer.raw.tell(), max_decomp, "Excessive amount of data was decompressed") diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -1812,7 +1812,7 @@ with self.subTest(method): pair = self.tp(self.BytesIO(b"abcdef"), self.MockRawIO()) - data = byteslike(5) + data = byteslike(b'\0' * 5) self.assertEqual(getattr(pair, method)(data), 5) self.assertEqual(bytes(data), b"abcde") diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py --- a/Lib/test/test_ipaddress.py +++ b/Lib/test/test_ipaddress.py @@ -119,7 +119,7 @@ def test_bad_packed_length(self): def assertBadLength(length): - addr = bytes(length) + addr = b'\0' * length msg = "%r (len %d != 4) is not permitted as an IPv4 address" with self.assertAddressError(re.escape(msg % (addr, length))): self.factory(addr) @@ -139,11 +139,11 @@ self.assertInstancesEqual(3232235521, "::c0a8:1") def test_packed(self): - addr = bytes(12) + bytes.fromhex("00000000") + addr = b'\0'*12 + bytes.fromhex("00000000") self.assertInstancesEqual(addr, "::") - addr = bytes(12) + bytes.fromhex("c0a80001") + addr = b'\0'*12 + bytes.fromhex("c0a80001") self.assertInstancesEqual(addr, "::c0a8:1") - addr = bytes.fromhex("c0a80001") + bytes(12) + addr = bytes.fromhex("c0a80001") + b'\0'*12 self.assertInstancesEqual(addr, "c0a8:1::") def test_negative_ints_rejected(self): @@ -158,7 +158,7 @@ def test_bad_packed_length(self): def assertBadLength(length): - addr = bytes(length) + addr = b'\0' * length msg = "%r (len %d != 16) is not permitted as an IPv6 address" with self.assertAddressError(re.escape(msg % (addr, length))): self.factory(addr) diff --git a/Lib/test/test_lzma.py b/Lib/test/test_lzma.py --- a/Lib/test/test_lzma.py +++ b/Lib/test/test_lzma.py @@ -928,11 +928,11 @@ def test_decompress_limited(self): """Decompressed data buffering should be limited""" - bomb = lzma.compress(bytes(int(2e6)), preset=6) + bomb = lzma.compress(b'\0' * int(2e6), preset=6) self.assertLess(len(bomb), _compression.BUFFER_SIZE) decomp = LZMAFile(BytesIO(bomb)) - self.assertEqual(bytes(1), decomp.read(1)) + self.assertEqual(decomp.read(1), b'\0') max_decomp = 1 + DEFAULT_BUFFER_SIZE self.assertLessEqual(decomp._buffer.raw.tell(), max_decomp, "Excessive amount of data was decompressed") diff --git a/Lib/test/test_socketserver.py b/Lib/test/test_socketserver.py --- a/Lib/test/test_socketserver.py +++ b/Lib/test/test_socketserver.py @@ -406,7 +406,7 @@ self.server.sent1 = self.wfile.write(b'write data\n') # Should be sent immediately, without requiring flush() self.server.received = self.rfile.readline() - big_chunk = bytes(test.support.SOCK_MAX_SIZE) + big_chunk = b'\0' * test.support.SOCK_MAX_SIZE self.server.sent2 = self.wfile.write(big_chunk) server = socketserver.TCPServer((HOST, 0), Handler) diff --git a/Lib/test/test_wsgiref.py b/Lib/test/test_wsgiref.py --- a/Lib/test/test_wsgiref.py +++ b/Lib/test/test_wsgiref.py @@ -258,7 +258,7 @@ def app(environ, start_response): start_response("200 OK", []) - return [bytes(support.SOCK_MAX_SIZE)] + return [b'\0' * support.SOCK_MAX_SIZE] class WsgiHandler(NoLogRequestHandler, WSGIRequestHandler): pass -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 07:49:50 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 11:49:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Use_uint16=5Ft_instead_of_?= =?utf-8?q?short_in_audioop=2E?= Message-ID: <20160911114950.36352.17047.31E89CFE@psf.io> https://hg.python.org/cpython/rev/9687be64b2a5 changeset: 103621:9687be64b2a5 user: Serhiy Storchaka date: Sun Sep 11 14:48:16 2016 +0300 summary: Use uint16_t instead of short in audioop. files: Modules/audioop.c | 28 ++++++++++++++-------------- 1 files changed, 14 insertions(+), 14 deletions(-) diff --git a/Modules/audioop.c b/Modules/audioop.c --- a/Modules/audioop.c +++ b/Modules/audioop.c @@ -240,7 +240,7 @@ st_linear2alaw(int16_t pcm_val) /* 2's complement (13-bit range) */ { int16_t mask; - short seg; + int16_t seg; unsigned char aval; /* A-law using even bit inversion */ @@ -294,7 +294,7 @@ #define GETINT8(cp, i) GETINTX(signed char, (cp), (i)) -#define GETINT16(cp, i) GETINTX(short, (cp), (i)) +#define GETINT16(cp, i) GETINTX(int16_t, (cp), (i)) #define GETINT32(cp, i) GETINTX(int32_t, (cp), (i)) #if WORDS_BIGENDIAN @@ -311,7 +311,7 @@ #define SETINT8(cp, i, val) SETINTX(signed char, (cp), (i), (val)) -#define SETINT16(cp, i, val) SETINTX(short, (cp), (i), (val)) +#define SETINT16(cp, i, val) SETINTX(int16_t, (cp), (i), (val)) #define SETINT32(cp, i, val) SETINTX(int32_t, (cp), (i), (val)) #if WORDS_BIGENDIAN @@ -542,7 +542,7 @@ return PyLong_FromUnsignedLong(res); } -static double _sum2(const short *a, const short *b, Py_ssize_t len) +static double _sum2(const int16_t *a, const int16_t *b, Py_ssize_t len) { Py_ssize_t i; double sum = 0.0; @@ -600,7 +600,7 @@ Py_buffer *reference) /*[clinic end generated code: output=5752306d83cbbada input=62c305605e183c9a]*/ { - const short *cp1, *cp2; + const int16_t *cp1, *cp2; Py_ssize_t len1, len2; Py_ssize_t j, best_j; double aj_m1, aj_lm1; @@ -610,9 +610,9 @@ PyErr_SetString(AudioopError, "Strings should be even-sized"); return NULL; } - cp1 = (const short *)fragment->buf; + cp1 = (const int16_t *)fragment->buf; len1 = fragment->len >> 1; - cp2 = (const short *)reference->buf; + cp2 = (const int16_t *)reference->buf; len2 = reference->len >> 1; if (len1 < len2) { @@ -669,7 +669,7 @@ Py_buffer *reference) /*[clinic end generated code: output=14ea95652c1afcf8 input=816680301d012b21]*/ { - const short *cp1, *cp2; + const int16_t *cp1, *cp2; Py_ssize_t len; double sum_ri_2, sum_aij_ri, result; @@ -681,8 +681,8 @@ PyErr_SetString(AudioopError, "Samples should be same size"); return NULL; } - cp1 = (const short *)fragment->buf; - cp2 = (const short *)reference->buf; + cp1 = (const int16_t *)fragment->buf; + cp2 = (const int16_t *)reference->buf; len = fragment->len >> 1; sum_ri_2 = _sum2(cp2, cp2, len); sum_aij_ri = _sum2(cp1, cp2, len); @@ -711,7 +711,7 @@ Py_ssize_t length) /*[clinic end generated code: output=f008128233523040 input=2f304801ed42383c]*/ { - const short *cp1; + const int16_t *cp1; Py_ssize_t len1; Py_ssize_t j, best_j; double aj_m1, aj_lm1; @@ -721,7 +721,7 @@ PyErr_SetString(AudioopError, "Strings should be even-sized"); return NULL; } - cp1 = (const short *)fragment->buf; + cp1 = (const int16_t *)fragment->buf; len1 = fragment->len >> 1; if (length < 0 || len1 < length) { @@ -1122,7 +1122,7 @@ if (width == 1) val = GETINTX(unsigned char, fragment->buf, i); else if (width == 2) - val = GETINTX(unsigned short, fragment->buf, i); + val = GETINTX(uint16_t, fragment->buf, i); else if (width == 3) val = ((unsigned int)GETINT24(fragment->buf, i)) & 0xffffffu; else { @@ -1137,7 +1137,7 @@ if (width == 1) SETINTX(unsigned char, ncp, i, val); else if (width == 2) - SETINTX(unsigned short, ncp, i, val); + SETINTX(uint16_t, ncp, i, val); else if (width == 3) SETINT24(ncp, i, (int)val); else { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 07:52:41 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 11 Sep 2016 11:52:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2325497=3A_Rewrite_?= =?utf-8?q?test=5Frobotparser_to_use_a_class_based_design?= Message-ID: <20160911115241.48515.66181.4BCC94F6@psf.io> https://hg.python.org/cpython/rev/2193bd6f020c changeset: 103622:2193bd6f020c user: Berker Peksag date: Sun Sep 11 14:53:16 2016 +0300 summary: Issue #25497: Rewrite test_robotparser to use a class based design files: Lib/test/test_robotparser.py | 337 ++++++++-------------- 1 files changed, 128 insertions(+), 209 deletions(-) diff --git a/Lib/test/test_robotparser.py b/Lib/test/test_robotparser.py --- a/Lib/test/test_robotparser.py +++ b/Lib/test/test_robotparser.py @@ -10,84 +10,49 @@ threading = None -class RobotTestCase(unittest.TestCase): - def __init__(self, index=None, parser=None, url=None, good=None, - agent=None, request_rate=None, crawl_delay=None): - # workaround to make unittest discovery work (see #17066) - if not isinstance(index, int): - return - unittest.TestCase.__init__(self) - if good: - self.str = "RobotTest(%d, good, %s)" % (index, url) - else: - self.str = "RobotTest(%d, bad, %s)" % (index, url) - self.parser = parser - self.url = url - self.good = good - self.agent = agent - self.request_rate = request_rate - self.crawl_delay = crawl_delay +class BaseRobotTest: + robots_txt = '' + agent = 'test_robotparser' + good = [] + bad = [] - def runTest(self): - if isinstance(self.url, tuple): - agent, url = self.url - else: - url = self.url - agent = self.agent - if self.good: - self.assertTrue(self.parser.can_fetch(agent, url)) - self.assertEqual(self.parser.crawl_delay(agent), self.crawl_delay) - # if we have actual values for request rate - if self.request_rate and self.parser.request_rate(agent): - self.assertEqual( - self.parser.request_rate(agent).requests, - self.request_rate.requests - ) - self.assertEqual( - self.parser.request_rate(agent).seconds, - self.request_rate.seconds - ) - self.assertEqual(self.parser.request_rate(agent), self.request_rate) - else: - self.assertFalse(self.parser.can_fetch(agent, url)) + def setUp(self): + lines = io.StringIO(self.robots_txt).readlines() + self.parser = urllib.robotparser.RobotFileParser() + self.parser.parse(lines) - def __str__(self): - return self.str + def get_agent_and_url(self, url): + if isinstance(url, tuple): + agent, url = url + return agent, url + return self.agent, url -tests = unittest.TestSuite() + def test_good_urls(self): + for url in self.good: + agent, url = self.get_agent_and_url(url) + with self.subTest(url=url, agent=agent): + self.assertTrue(self.parser.can_fetch(agent, url)) -def RobotTest(index, robots_txt, good_urls, bad_urls, - request_rate, crawl_delay, agent="test_robotparser"): + def test_bad_urls(self): + for url in self.bad: + agent, url = self.get_agent_and_url(url) + with self.subTest(url=url, agent=agent): + self.assertFalse(self.parser.can_fetch(agent, url)) - lines = io.StringIO(robots_txt).readlines() - parser = urllib.robotparser.RobotFileParser() - parser.parse(lines) - for url in good_urls: - tests.addTest(RobotTestCase(index, parser, url, 1, agent, - request_rate, crawl_delay)) - for url in bad_urls: - tests.addTest(RobotTestCase(index, parser, url, 0, agent, - request_rate, crawl_delay)) -# Examples from http://www.robotstxt.org/wc/norobots.html (fetched 2002) - -# 1. -doc = """ +class UserAgentWildcardTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ User-agent: * Disallow: /cyberworld/map/ # This is an infinite virtual URL space Disallow: /tmp/ # these will soon disappear Disallow: /foo.html -""" + """ + good = ['/', '/test.html'] + bad = ['/cyberworld/map/index.html', '/tmp/xxx', '/foo.html'] -good = ['/','/test.html'] -bad = ['/cyberworld/map/index.html','/tmp/xxx','/foo.html'] -request_rate = None -crawl_delay = None -RobotTest(1, doc, good, bad, request_rate, crawl_delay) - -# 2. -doc = """ +class CrawlDelayAndCustomAgentTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ # robots.txt for http://www.example.com/ User-agent: * @@ -98,34 +63,23 @@ # Cybermapper knows where to go. User-agent: cybermapper Disallow: + """ + good = ['/', '/test.html', ('cybermapper', '/cyberworld/map/index.html')] + bad = ['/cyberworld/map/index.html'] -""" -good = ['/','/test.html',('cybermapper','/cyberworld/map/index.html')] -bad = ['/cyberworld/map/index.html'] -request_rate = None # The parameters should be equal to None since they -crawl_delay = None # don't apply to the cybermapper user agent - -RobotTest(2, doc, good, bad, request_rate, crawl_delay) - -# 3. -doc = """ +class RejectAllRobotsTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ # go away User-agent: * Disallow: / -""" + """ + good = [] + bad = ['/cyberworld/map/index.html', '/', '/tmp/'] -good = [] -bad = ['/cyberworld/map/index.html','/','/tmp/'] -request_rate = None -crawl_delay = None -RobotTest(3, doc, good, bad, request_rate, crawl_delay) - -# Examples from http://www.robotstxt.org/wc/norobots-rfc.html (fetched 2002) - -# 4. -doc = """ +class CrawlDelayAndRequestRateTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ User-agent: figtree Crawl-delay: 3 Request-rate: 9/30 @@ -133,28 +87,43 @@ Disallow: /a%3cd.html Disallow: /a%2fb.html Disallow: /%7ejoe/index.html -""" + """ + agent = 'figtree' + request_rate = namedtuple('req_rate', 'requests seconds')(9, 30) + crawl_delay = 3 + good = [('figtree', '/foo.html')] + bad = ['/tmp', '/tmp.html', '/tmp/a.html', '/a%3cd.html', '/a%3Cd.html', + '/a%2fb.html', '/~joe/index.html'] -good = [] # XFAIL '/a/b.html' -bad = ['/tmp','/tmp.html','/tmp/a.html', - '/a%3cd.html','/a%3Cd.html','/a%2fb.html', - '/~joe/index.html' - ] + def test_request_rate(self): + for url in self.good: + agent, url = self.get_agent_and_url(url) + with self.subTest(url=url, agent=agent): + if self.crawl_delay: + self.assertEqual( + self.parser.crawl_delay(agent), self.crawl_delay + ) + if self.request_rate and self.parser.request_rate(agent): + self.assertEqual( + self.parser.request_rate(agent).requests, + self.request_rate.requests + ) + self.assertEqual( + self.parser.request_rate(agent).seconds, + self.request_rate.seconds + ) -request_rate = namedtuple('req_rate', 'requests seconds') -request_rate.requests = 9 -request_rate.seconds = 30 -crawl_delay = 3 -request_rate_bad = None # not actually tested, but we still need to parse it -crawl_delay_bad = None # in order to accommodate the input parameters +class DifferentAgentTest(CrawlDelayAndRequestRateTest): + agent = 'FigTree Robot libwww-perl/5.04' + # these are not actually tested, but we still need to parse it + # in order to accommodate the input parameters + request_rate = None + crawl_delay = None -RobotTest(4, doc, good, bad, request_rate, crawl_delay, 'figtree' ) -RobotTest(5, doc, good, bad, request_rate_bad, crawl_delay_bad, - 'FigTree Robot libwww-perl/5.04') -# 6. -doc = """ +class InvalidRequestRateTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ User-agent: * Disallow: /tmp/ Disallow: /a%3Cd.html @@ -162,141 +131,102 @@ Disallow: /%7ejoe/index.html Crawl-delay: 3 Request-rate: 9/banana -""" + """ + good = ['/tmp'] + bad = ['/tmp/', '/tmp/a.html', '/a%3cd.html', '/a%3Cd.html', '/a/b.html', + '/%7Ejoe/index.html'] + crawl_delay = 3 -good = ['/tmp',] # XFAIL: '/a%2fb.html' -bad = ['/tmp/','/tmp/a.html', - '/a%3cd.html','/a%3Cd.html',"/a/b.html", - '/%7Ejoe/index.html'] -crawl_delay = 3 -request_rate = None # since request rate has invalid syntax, return None -RobotTest(6, doc, good, bad, None, None) - -# From bug report #523041 - -# 7. -doc = """ +class InvalidCrawlDelayTest(BaseRobotTest, unittest.TestCase): + # From bug report #523041 + robots_txt = """\ User-Agent: * Disallow: /. Crawl-delay: pears -""" + """ + good = ['/foo.html'] + # bug report says "/" should be denied, but that is not in the RFC + bad = [] -good = ['/foo.html'] -bad = [] # bug report says "/" should be denied, but that is not in the RFC -crawl_delay = None # since crawl delay has invalid syntax, return None -request_rate = None - -RobotTest(7, doc, good, bad, crawl_delay, request_rate) - -# From Google: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40364 - -# 8. -doc = """ +class AnotherInvalidRequestRateTest(BaseRobotTest, unittest.TestCase): + # also test that Allow and Diasallow works well with each other + robots_txt = """\ User-agent: Googlebot Allow: /folder1/myfile.html Disallow: /folder1/ Request-rate: whale/banana -""" + """ + agent = 'Googlebot' + good = ['/folder1/myfile.html'] + bad = ['/folder1/anotherfile.html'] -good = ['/folder1/myfile.html'] -bad = ['/folder1/anotherfile.html'] -crawl_delay = None -request_rate = None # invalid syntax, return none -RobotTest(8, doc, good, bad, crawl_delay, request_rate, agent="Googlebot") - -# 9. This file is incorrect because "Googlebot" is a substring of -# "Googlebot-Mobile", so test 10 works just like test 9. -doc = """ +class UserAgentOrderingTest(BaseRobotTest, unittest.TestCase): + # the order of User-agent should be correct. note + # that this file is incorrect because "Googlebot" is a + # substring of "Googlebot-Mobile" + robots_txt = """\ User-agent: Googlebot Disallow: / User-agent: Googlebot-Mobile Allow: / -""" + """ + agent = 'Googlebot' + bad = ['/something.jpg'] -good = [] -bad = ['/something.jpg'] -RobotTest(9, doc, good, bad, None, None, agent="Googlebot") +class UserAgentGoogleMobileTest(UserAgentOrderingTest): + agent = 'Googlebot-Mobile' -good = [] -bad = ['/something.jpg'] -RobotTest(10, doc, good, bad, None, None, agent="Googlebot-Mobile") - -# 11. Get the order correct. -doc = """ -User-agent: Googlebot-Mobile -Allow: / - -User-agent: Googlebot -Disallow: / -""" - -good = [] -bad = ['/something.jpg'] - -RobotTest(11, doc, good, bad, None, None, agent="Googlebot") - -good = ['/something.jpg'] -bad = [] - -RobotTest(12, doc, good, bad, None, None, agent="Googlebot-Mobile") - - -# 13. Google also got the order wrong in #8. You need to specify the -# URLs from more specific to more general. -doc = """ +class GoogleURLOrderingTest(BaseRobotTest, unittest.TestCase): + # Google also got the order wrong. You need + # to specify the URLs from more specific to more general + robots_txt = """\ User-agent: Googlebot Allow: /folder1/myfile.html Disallow: /folder1/ -""" + """ + agent = 'googlebot' + good = ['/folder1/myfile.html'] + bad = ['/folder1/anotherfile.html'] -good = ['/folder1/myfile.html'] -bad = ['/folder1/anotherfile.html'] -RobotTest(13, doc, good, bad, None, None, agent="googlebot") - - -# 14. For issue #6325 (query string support) -doc = """ +class DisallowQueryStringTest(BaseRobotTest, unittest.TestCase): + # see issue #6325 for details + robots_txt = """\ User-agent: * Disallow: /some/path?name=value -""" + """ + good = ['/some/path'] + bad = ['/some/path?name=value'] -good = ['/some/path'] -bad = ['/some/path?name=value'] -RobotTest(14, doc, good, bad, None, None) - -# 15. For issue #4108 (obey first * entry) -doc = """ +class UseFirstUserAgentWildcardTest(BaseRobotTest, unittest.TestCase): + # obey first * entry (#4108) + robots_txt = """\ User-agent: * Disallow: /some/path User-agent: * Disallow: /another/path -""" + """ + good = ['/another/path'] + bad = ['/some/path'] -good = ['/another/path'] -bad = ['/some/path'] -RobotTest(15, doc, good, bad, None, None) - -# 16. Empty query (issue #17403). Normalizing the url first. -doc = """ +class EmptyQueryStringTest(BaseRobotTest, unittest.TestCase): + # normalize the URL first (#17403) + robots_txt = """\ User-agent: * Allow: /some/path? Disallow: /another/path? -""" - -good = ['/some/path?'] -bad = ['/another/path?'] - -RobotTest(16, doc, good, bad, None, None) + """ + good = ['/some/path?'] + bad = ['/another/path?'] class RobotHandler(BaseHTTPRequestHandler): @@ -329,9 +259,6 @@ self.t.join() self.server.server_close() - def runTest(self): - self.testPasswordProtectedSite() - def testPasswordProtectedSite(self): addr = self.server.server_address url = 'http://' + support.HOST + ':' + str(addr[1]) @@ -341,8 +268,6 @@ parser.read() self.assertFalse(parser.can_fetch("*", robots_url)) - def __str__(self): - return '%s' % self.__class__.__name__ class NetworkTestCase(unittest.TestCase): @@ -356,11 +281,5 @@ self.assertTrue( parser.can_fetch("*", "http://www.python.org/robots.txt")) -def load_tests(loader, suite, pattern): - suite = unittest.makeSuite(NetworkTestCase) - suite.addTest(tests) - suite.addTest(PasswordProtectedSiteTestCase()) - return suite - if __name__=='__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 07:53:42 2016 From: python-checkins at python.org (vinay.sajip) Date: Sun, 11 Sep 2016 11:53:42 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogRml4ZXMgIzI1Njcx?= =?utf-8?q?=3A_Updated_prompt_handling_in_activate=2Efish=2E?= Message-ID: <20160911115341.48681.45475.212B8ADA@psf.io> https://hg.python.org/cpython/rev/f6125944ffb8 changeset: 103623:f6125944ffb8 branch: 3.5 parent: 103617:ca68bb597df9 user: Vinay Sajip date: Sun Sep 11 12:52:08 2016 +0100 summary: Fixes #25671: Updated prompt handling in activate.fish. files: Lib/venv/scripts/posix/activate.fish | 41 ++++++++------- 1 files changed, 21 insertions(+), 20 deletions(-) diff --git a/Lib/venv/scripts/posix/activate.fish b/Lib/venv/scripts/posix/activate.fish --- a/Lib/venv/scripts/posix/activate.fish +++ b/Lib/venv/scripts/posix/activate.fish @@ -15,10 +15,7 @@ if test -n "$_OLD_FISH_PROMPT_OVERRIDE" functions -e fish_prompt set -e _OLD_FISH_PROMPT_OVERRIDE - . ( begin - printf "function fish_prompt\n\t#" - functions _old_fish_prompt - end | psub ) + functions -c _old_fish_prompt fish_prompt functions -e _old_fish_prompt end @@ -47,27 +44,31 @@ # fish uses a function instead of an env var to generate the prompt. # save the current fish_prompt function as the function _old_fish_prompt - . ( begin - printf "function _old_fish_prompt\n\t#" - functions fish_prompt - end | psub ) + functions -c fish_prompt _old_fish_prompt # with the original prompt function renamed, we can override with our own. function fish_prompt + # Save the return status of the last command + set -l old_status $status + # Prompt override? - if test -n "__VENV_PROMPT__" - printf "%s%s%s" "__VENV_PROMPT__" (set_color normal) (_old_fish_prompt) - return + if test -n "__VENV_PROMPT__" + printf "%s%s" "__VENV_PROMPT__" (set_color normal) + else + # ...Otherwise, prepend env + set -l _checkbase (basename "$VIRTUAL_ENV") + if test $_checkbase = "__" + # special case for Aspen magic directories + # see http://www.zetadev.com/software/aspen/ + printf "%s[%s]%s " (set_color -b blue white) (basename (dirname "$VIRTUAL_ENV")) (set_color normal) + else + printf "%s(%s)%s" (set_color -b blue white) (basename "$VIRTUAL_ENV") (set_color normal) + end end - # ...Otherwise, prepend env - set -l _checkbase (basename "$VIRTUAL_ENV") - if test $_checkbase = "__" - # special case for Aspen magic directories - # see http://www.zetadev.com/software/aspen/ - printf "%s[%s]%s %s" (set_color -b blue white) (basename (dirname "$VIRTUAL_ENV")) (set_color normal) (_old_fish_prompt) - else - printf "%s(%s)%s%s" (set_color -b blue white) (basename "$VIRTUAL_ENV") (set_color normal) (_old_fish_prompt) - end + + # Restore the return status of the previous command. + echo "exit $old_status" | . + _old_fish_prompt end set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 07:53:42 2016 From: python-checkins at python.org (vinay.sajip) Date: Sun, 11 Sep 2016 11:53:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2325671=3A_Merged_fix_from_3=2E5=2E?= Message-ID: <20160911115342.69577.7235.88837285@psf.io> https://hg.python.org/cpython/rev/ed34b93519ba changeset: 103624:ed34b93519ba parent: 103621:9687be64b2a5 parent: 103623:f6125944ffb8 user: Vinay Sajip date: Sun Sep 11 12:52:53 2016 +0100 summary: Closes #25671: Merged fix from 3.5. files: Lib/venv/scripts/posix/activate.fish | 41 ++++++++------- 1 files changed, 21 insertions(+), 20 deletions(-) diff --git a/Lib/venv/scripts/posix/activate.fish b/Lib/venv/scripts/posix/activate.fish --- a/Lib/venv/scripts/posix/activate.fish +++ b/Lib/venv/scripts/posix/activate.fish @@ -15,10 +15,7 @@ if test -n "$_OLD_FISH_PROMPT_OVERRIDE" functions -e fish_prompt set -e _OLD_FISH_PROMPT_OVERRIDE - . ( begin - printf "function fish_prompt\n\t#" - functions _old_fish_prompt - end | psub ) + functions -c _old_fish_prompt fish_prompt functions -e _old_fish_prompt end @@ -47,27 +44,31 @@ # fish uses a function instead of an env var to generate the prompt. # save the current fish_prompt function as the function _old_fish_prompt - . ( begin - printf "function _old_fish_prompt\n\t#" - functions fish_prompt - end | psub ) + functions -c fish_prompt _old_fish_prompt # with the original prompt function renamed, we can override with our own. function fish_prompt + # Save the return status of the last command + set -l old_status $status + # Prompt override? - if test -n "__VENV_PROMPT__" - printf "%s%s%s" "__VENV_PROMPT__" (set_color normal) (_old_fish_prompt) - return + if test -n "__VENV_PROMPT__" + printf "%s%s" "__VENV_PROMPT__" (set_color normal) + else + # ...Otherwise, prepend env + set -l _checkbase (basename "$VIRTUAL_ENV") + if test $_checkbase = "__" + # special case for Aspen magic directories + # see http://www.zetadev.com/software/aspen/ + printf "%s[%s]%s " (set_color -b blue white) (basename (dirname "$VIRTUAL_ENV")) (set_color normal) + else + printf "%s(%s)%s" (set_color -b blue white) (basename "$VIRTUAL_ENV") (set_color normal) + end end - # ...Otherwise, prepend env - set -l _checkbase (basename "$VIRTUAL_ENV") - if test $_checkbase = "__" - # special case for Aspen magic directories - # see http://www.zetadev.com/software/aspen/ - printf "%s[%s]%s %s" (set_color -b blue white) (basename (dirname "$VIRTUAL_ENV")) (set_color normal) (_old_fish_prompt) - else - printf "%s(%s)%s%s" (set_color -b blue white) (basename "$VIRTUAL_ENV") (set_color normal) (_old_fish_prompt) - end + + # Restore the return status of the previous command. + echo "exit $old_status" | . + _old_fish_prompt end set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV" -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 07:53:42 2016 From: python-checkins at python.org (vinay.sajip) Date: Sun, 11 Sep 2016 11:53:42 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_default_-=3E_default?= =?utf-8?q?=29=3A_Merged_upstream_changes=2E?= Message-ID: <20160911115342.36379.36863.28741BB5@psf.io> https://hg.python.org/cpython/rev/0383ae836de9 changeset: 103625:0383ae836de9 parent: 103624:ed34b93519ba parent: 103622:2193bd6f020c user: Vinay Sajip date: Sun Sep 11 12:53:34 2016 +0100 summary: Merged upstream changes. files: Lib/test/test_robotparser.py | 337 ++++++++-------------- 1 files changed, 128 insertions(+), 209 deletions(-) diff --git a/Lib/test/test_robotparser.py b/Lib/test/test_robotparser.py --- a/Lib/test/test_robotparser.py +++ b/Lib/test/test_robotparser.py @@ -10,84 +10,49 @@ threading = None -class RobotTestCase(unittest.TestCase): - def __init__(self, index=None, parser=None, url=None, good=None, - agent=None, request_rate=None, crawl_delay=None): - # workaround to make unittest discovery work (see #17066) - if not isinstance(index, int): - return - unittest.TestCase.__init__(self) - if good: - self.str = "RobotTest(%d, good, %s)" % (index, url) - else: - self.str = "RobotTest(%d, bad, %s)" % (index, url) - self.parser = parser - self.url = url - self.good = good - self.agent = agent - self.request_rate = request_rate - self.crawl_delay = crawl_delay +class BaseRobotTest: + robots_txt = '' + agent = 'test_robotparser' + good = [] + bad = [] - def runTest(self): - if isinstance(self.url, tuple): - agent, url = self.url - else: - url = self.url - agent = self.agent - if self.good: - self.assertTrue(self.parser.can_fetch(agent, url)) - self.assertEqual(self.parser.crawl_delay(agent), self.crawl_delay) - # if we have actual values for request rate - if self.request_rate and self.parser.request_rate(agent): - self.assertEqual( - self.parser.request_rate(agent).requests, - self.request_rate.requests - ) - self.assertEqual( - self.parser.request_rate(agent).seconds, - self.request_rate.seconds - ) - self.assertEqual(self.parser.request_rate(agent), self.request_rate) - else: - self.assertFalse(self.parser.can_fetch(agent, url)) + def setUp(self): + lines = io.StringIO(self.robots_txt).readlines() + self.parser = urllib.robotparser.RobotFileParser() + self.parser.parse(lines) - def __str__(self): - return self.str + def get_agent_and_url(self, url): + if isinstance(url, tuple): + agent, url = url + return agent, url + return self.agent, url -tests = unittest.TestSuite() + def test_good_urls(self): + for url in self.good: + agent, url = self.get_agent_and_url(url) + with self.subTest(url=url, agent=agent): + self.assertTrue(self.parser.can_fetch(agent, url)) -def RobotTest(index, robots_txt, good_urls, bad_urls, - request_rate, crawl_delay, agent="test_robotparser"): + def test_bad_urls(self): + for url in self.bad: + agent, url = self.get_agent_and_url(url) + with self.subTest(url=url, agent=agent): + self.assertFalse(self.parser.can_fetch(agent, url)) - lines = io.StringIO(robots_txt).readlines() - parser = urllib.robotparser.RobotFileParser() - parser.parse(lines) - for url in good_urls: - tests.addTest(RobotTestCase(index, parser, url, 1, agent, - request_rate, crawl_delay)) - for url in bad_urls: - tests.addTest(RobotTestCase(index, parser, url, 0, agent, - request_rate, crawl_delay)) -# Examples from http://www.robotstxt.org/wc/norobots.html (fetched 2002) - -# 1. -doc = """ +class UserAgentWildcardTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ User-agent: * Disallow: /cyberworld/map/ # This is an infinite virtual URL space Disallow: /tmp/ # these will soon disappear Disallow: /foo.html -""" + """ + good = ['/', '/test.html'] + bad = ['/cyberworld/map/index.html', '/tmp/xxx', '/foo.html'] -good = ['/','/test.html'] -bad = ['/cyberworld/map/index.html','/tmp/xxx','/foo.html'] -request_rate = None -crawl_delay = None -RobotTest(1, doc, good, bad, request_rate, crawl_delay) - -# 2. -doc = """ +class CrawlDelayAndCustomAgentTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ # robots.txt for http://www.example.com/ User-agent: * @@ -98,34 +63,23 @@ # Cybermapper knows where to go. User-agent: cybermapper Disallow: + """ + good = ['/', '/test.html', ('cybermapper', '/cyberworld/map/index.html')] + bad = ['/cyberworld/map/index.html'] -""" -good = ['/','/test.html',('cybermapper','/cyberworld/map/index.html')] -bad = ['/cyberworld/map/index.html'] -request_rate = None # The parameters should be equal to None since they -crawl_delay = None # don't apply to the cybermapper user agent - -RobotTest(2, doc, good, bad, request_rate, crawl_delay) - -# 3. -doc = """ +class RejectAllRobotsTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ # go away User-agent: * Disallow: / -""" + """ + good = [] + bad = ['/cyberworld/map/index.html', '/', '/tmp/'] -good = [] -bad = ['/cyberworld/map/index.html','/','/tmp/'] -request_rate = None -crawl_delay = None -RobotTest(3, doc, good, bad, request_rate, crawl_delay) - -# Examples from http://www.robotstxt.org/wc/norobots-rfc.html (fetched 2002) - -# 4. -doc = """ +class CrawlDelayAndRequestRateTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ User-agent: figtree Crawl-delay: 3 Request-rate: 9/30 @@ -133,28 +87,43 @@ Disallow: /a%3cd.html Disallow: /a%2fb.html Disallow: /%7ejoe/index.html -""" + """ + agent = 'figtree' + request_rate = namedtuple('req_rate', 'requests seconds')(9, 30) + crawl_delay = 3 + good = [('figtree', '/foo.html')] + bad = ['/tmp', '/tmp.html', '/tmp/a.html', '/a%3cd.html', '/a%3Cd.html', + '/a%2fb.html', '/~joe/index.html'] -good = [] # XFAIL '/a/b.html' -bad = ['/tmp','/tmp.html','/tmp/a.html', - '/a%3cd.html','/a%3Cd.html','/a%2fb.html', - '/~joe/index.html' - ] + def test_request_rate(self): + for url in self.good: + agent, url = self.get_agent_and_url(url) + with self.subTest(url=url, agent=agent): + if self.crawl_delay: + self.assertEqual( + self.parser.crawl_delay(agent), self.crawl_delay + ) + if self.request_rate and self.parser.request_rate(agent): + self.assertEqual( + self.parser.request_rate(agent).requests, + self.request_rate.requests + ) + self.assertEqual( + self.parser.request_rate(agent).seconds, + self.request_rate.seconds + ) -request_rate = namedtuple('req_rate', 'requests seconds') -request_rate.requests = 9 -request_rate.seconds = 30 -crawl_delay = 3 -request_rate_bad = None # not actually tested, but we still need to parse it -crawl_delay_bad = None # in order to accommodate the input parameters +class DifferentAgentTest(CrawlDelayAndRequestRateTest): + agent = 'FigTree Robot libwww-perl/5.04' + # these are not actually tested, but we still need to parse it + # in order to accommodate the input parameters + request_rate = None + crawl_delay = None -RobotTest(4, doc, good, bad, request_rate, crawl_delay, 'figtree' ) -RobotTest(5, doc, good, bad, request_rate_bad, crawl_delay_bad, - 'FigTree Robot libwww-perl/5.04') -# 6. -doc = """ +class InvalidRequestRateTest(BaseRobotTest, unittest.TestCase): + robots_txt = """\ User-agent: * Disallow: /tmp/ Disallow: /a%3Cd.html @@ -162,141 +131,102 @@ Disallow: /%7ejoe/index.html Crawl-delay: 3 Request-rate: 9/banana -""" + """ + good = ['/tmp'] + bad = ['/tmp/', '/tmp/a.html', '/a%3cd.html', '/a%3Cd.html', '/a/b.html', + '/%7Ejoe/index.html'] + crawl_delay = 3 -good = ['/tmp',] # XFAIL: '/a%2fb.html' -bad = ['/tmp/','/tmp/a.html', - '/a%3cd.html','/a%3Cd.html',"/a/b.html", - '/%7Ejoe/index.html'] -crawl_delay = 3 -request_rate = None # since request rate has invalid syntax, return None -RobotTest(6, doc, good, bad, None, None) - -# From bug report #523041 - -# 7. -doc = """ +class InvalidCrawlDelayTest(BaseRobotTest, unittest.TestCase): + # From bug report #523041 + robots_txt = """\ User-Agent: * Disallow: /. Crawl-delay: pears -""" + """ + good = ['/foo.html'] + # bug report says "/" should be denied, but that is not in the RFC + bad = [] -good = ['/foo.html'] -bad = [] # bug report says "/" should be denied, but that is not in the RFC -crawl_delay = None # since crawl delay has invalid syntax, return None -request_rate = None - -RobotTest(7, doc, good, bad, crawl_delay, request_rate) - -# From Google: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40364 - -# 8. -doc = """ +class AnotherInvalidRequestRateTest(BaseRobotTest, unittest.TestCase): + # also test that Allow and Diasallow works well with each other + robots_txt = """\ User-agent: Googlebot Allow: /folder1/myfile.html Disallow: /folder1/ Request-rate: whale/banana -""" + """ + agent = 'Googlebot' + good = ['/folder1/myfile.html'] + bad = ['/folder1/anotherfile.html'] -good = ['/folder1/myfile.html'] -bad = ['/folder1/anotherfile.html'] -crawl_delay = None -request_rate = None # invalid syntax, return none -RobotTest(8, doc, good, bad, crawl_delay, request_rate, agent="Googlebot") - -# 9. This file is incorrect because "Googlebot" is a substring of -# "Googlebot-Mobile", so test 10 works just like test 9. -doc = """ +class UserAgentOrderingTest(BaseRobotTest, unittest.TestCase): + # the order of User-agent should be correct. note + # that this file is incorrect because "Googlebot" is a + # substring of "Googlebot-Mobile" + robots_txt = """\ User-agent: Googlebot Disallow: / User-agent: Googlebot-Mobile Allow: / -""" + """ + agent = 'Googlebot' + bad = ['/something.jpg'] -good = [] -bad = ['/something.jpg'] -RobotTest(9, doc, good, bad, None, None, agent="Googlebot") +class UserAgentGoogleMobileTest(UserAgentOrderingTest): + agent = 'Googlebot-Mobile' -good = [] -bad = ['/something.jpg'] -RobotTest(10, doc, good, bad, None, None, agent="Googlebot-Mobile") - -# 11. Get the order correct. -doc = """ -User-agent: Googlebot-Mobile -Allow: / - -User-agent: Googlebot -Disallow: / -""" - -good = [] -bad = ['/something.jpg'] - -RobotTest(11, doc, good, bad, None, None, agent="Googlebot") - -good = ['/something.jpg'] -bad = [] - -RobotTest(12, doc, good, bad, None, None, agent="Googlebot-Mobile") - - -# 13. Google also got the order wrong in #8. You need to specify the -# URLs from more specific to more general. -doc = """ +class GoogleURLOrderingTest(BaseRobotTest, unittest.TestCase): + # Google also got the order wrong. You need + # to specify the URLs from more specific to more general + robots_txt = """\ User-agent: Googlebot Allow: /folder1/myfile.html Disallow: /folder1/ -""" + """ + agent = 'googlebot' + good = ['/folder1/myfile.html'] + bad = ['/folder1/anotherfile.html'] -good = ['/folder1/myfile.html'] -bad = ['/folder1/anotherfile.html'] -RobotTest(13, doc, good, bad, None, None, agent="googlebot") - - -# 14. For issue #6325 (query string support) -doc = """ +class DisallowQueryStringTest(BaseRobotTest, unittest.TestCase): + # see issue #6325 for details + robots_txt = """\ User-agent: * Disallow: /some/path?name=value -""" + """ + good = ['/some/path'] + bad = ['/some/path?name=value'] -good = ['/some/path'] -bad = ['/some/path?name=value'] -RobotTest(14, doc, good, bad, None, None) - -# 15. For issue #4108 (obey first * entry) -doc = """ +class UseFirstUserAgentWildcardTest(BaseRobotTest, unittest.TestCase): + # obey first * entry (#4108) + robots_txt = """\ User-agent: * Disallow: /some/path User-agent: * Disallow: /another/path -""" + """ + good = ['/another/path'] + bad = ['/some/path'] -good = ['/another/path'] -bad = ['/some/path'] -RobotTest(15, doc, good, bad, None, None) - -# 16. Empty query (issue #17403). Normalizing the url first. -doc = """ +class EmptyQueryStringTest(BaseRobotTest, unittest.TestCase): + # normalize the URL first (#17403) + robots_txt = """\ User-agent: * Allow: /some/path? Disallow: /another/path? -""" - -good = ['/some/path?'] -bad = ['/another/path?'] - -RobotTest(16, doc, good, bad, None, None) + """ + good = ['/some/path?'] + bad = ['/another/path?'] class RobotHandler(BaseHTTPRequestHandler): @@ -329,9 +259,6 @@ self.t.join() self.server.server_close() - def runTest(self): - self.testPasswordProtectedSite() - def testPasswordProtectedSite(self): addr = self.server.server_address url = 'http://' + support.HOST + ':' + str(addr[1]) @@ -341,8 +268,6 @@ parser.read() self.assertFalse(parser.can_fetch("*", robots_url)) - def __str__(self): - return '%s' % self.__class__.__name__ class NetworkTestCase(unittest.TestCase): @@ -356,11 +281,5 @@ self.assertTrue( parser.can_fetch("*", "http://www.python.org/robots.txt")) -def load_tests(loader, suite, pattern): - suite = unittest.makeSuite(NetworkTestCase) - suite.addTest(tests) - suite.addTest(PasswordProtectedSiteTestCase()) - return suite - if __name__=='__main__': unittest.main() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:16:03 2016 From: python-checkins at python.org (vinay.sajip) Date: Sun, 11 Sep 2016 12:16:03 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closed_=2326533=3A_Merged_update_from_3=2E5=2E?= Message-ID: <20160911121603.21157.9888.6D9A99B5@psf.io> https://hg.python.org/cpython/rev/abd744bf62fb changeset: 103627:abd744bf62fb parent: 103625:0383ae836de9 parent: 103626:65a1abf4b432 user: Vinay Sajip date: Sun Sep 11 13:15:57 2016 +0100 summary: Closed #26533: Merged update from 3.5. files: Doc/library/logging.config.rst | 19 ++++++++++++++----- 1 files changed, 14 insertions(+), 5 deletions(-) diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -126,10 +126,10 @@ Starts up a socket server on the specified port, and listens for new configurations. If no port is specified, the module's default :const:`DEFAULT_LOGGING_CONFIG_PORT` is used. Logging configurations will be - sent as a file suitable for processing by :func:`fileConfig`. Returns a - :class:`~threading.Thread` instance on which you can call - :meth:`~threading.Thread.start` to start the server, and which you can - :meth:`~threading.Thread.join` when appropriate. To stop the server, + sent as a file suitable for processing by :func:`dictConfig` or + :func:`fileConfig`. Returns a :class:`~threading.Thread` instance on which + you can call :meth:`~threading.Thread.start` to start the server, and which + you can :meth:`~threading.Thread.join` when appropriate. To stop the server, call :func:`stopListening`. The ``verify`` argument, if specified, should be a callable which should @@ -165,9 +165,18 @@ ``verify`` argument to :func:`listen` to prevent unrecognised configurations from being applied. - .. versionchanged:: 3.4. + .. versionchanged:: 3.4 The ``verify`` argument was added. + .. note:: + + If you want to send configurations to the listener which don't + disable existing loggers, you will need to use a JSON format for + the configuration, which will use :func:`dictConfig` for configuration. + This method allows you to specify ``disable_existing_loggers`` as + ``False`` in the configuration you send. + + .. function:: stopListening() Stops the listening server which was created with a call to :func:`listen`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:16:03 2016 From: python-checkins at python.org (vinay.sajip) Date: Sun, 11 Sep 2016 12:16:03 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogQ2xvc2VkICMyNjUz?= =?utf-8?q?3=3A_expanded_on_behaviour_of_logging=2Econfig=2Elisten=28=29?= =?utf-8?q?=2E?= Message-ID: <20160911121602.1723.50633.CB52F5B6@psf.io> https://hg.python.org/cpython/rev/65a1abf4b432 changeset: 103626:65a1abf4b432 branch: 3.5 parent: 103623:f6125944ffb8 user: Vinay Sajip date: Sun Sep 11 13:15:06 2016 +0100 summary: Closed #26533: expanded on behaviour of logging.config.listen(). files: Doc/library/logging.config.rst | 19 ++++++++++++++----- 1 files changed, 14 insertions(+), 5 deletions(-) diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -126,10 +126,10 @@ Starts up a socket server on the specified port, and listens for new configurations. If no port is specified, the module's default :const:`DEFAULT_LOGGING_CONFIG_PORT` is used. Logging configurations will be - sent as a file suitable for processing by :func:`fileConfig`. Returns a - :class:`~threading.Thread` instance on which you can call - :meth:`~threading.Thread.start` to start the server, and which you can - :meth:`~threading.Thread.join` when appropriate. To stop the server, + sent as a file suitable for processing by :func:`dictConfig` or + :func:`fileConfig`. Returns a :class:`~threading.Thread` instance on which + you can call :meth:`~threading.Thread.start` to start the server, and which + you can :meth:`~threading.Thread.join` when appropriate. To stop the server, call :func:`stopListening`. The ``verify`` argument, if specified, should be a callable which should @@ -165,9 +165,18 @@ ``verify`` argument to :func:`listen` to prevent unrecognised configurations from being applied. - .. versionchanged:: 3.4. + .. versionchanged:: 3.4 The ``verify`` argument was added. + .. note:: + + If you want to send configurations to the listener which don't + disable existing loggers, you will need to use a JSON format for + the configuration, which will use :func:`dictConfig` for configuration. + This method allows you to specify ``disable_existing_loggers`` as + ``False`` in the configuration you send. + + .. function:: stopListening() Stops the listening server which was created with a call to :func:`listen`. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:17:18 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 11 Sep 2016 12:17:18 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Wrap_testPasswordProtected?= =?utf-8?q?Site_with_=40reap=5Fthreads?= Message-ID: <20160911121718.2793.75450.47B68172@psf.io> https://hg.python.org/cpython/rev/b31d2608990e changeset: 103628:b31d2608990e user: Berker Peksag date: Sun Sep 11 15:17:53 2016 +0300 summary: Wrap testPasswordProtectedSite with @reap_threads files: Lib/test/test_robotparser.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_robotparser.py b/Lib/test/test_robotparser.py --- a/Lib/test/test_robotparser.py +++ b/Lib/test/test_robotparser.py @@ -259,6 +259,7 @@ self.t.join() self.server.server_close() + @support.reap_threads def testPasswordProtectedSite(self): addr = self.server.server_address url = 'http://' + support.HOST + ':' + str(addr[1]) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:19:37 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 12:19:37 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixed_refactoring_bug_in_d?= =?utf-8?q?d046963bd42_=28issue27129=29=2E?= Message-ID: <20160911121936.21157.36479.1E11E980@psf.io> https://hg.python.org/cpython/rev/b49a938eaa31 changeset: 103629:b49a938eaa31 user: Serhiy Storchaka date: Sun Sep 11 15:19:12 2016 +0300 summary: Fixed refactoring bug in dd046963bd42 (issue27129). files: Python/peephole.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Python/peephole.c b/Python/peephole.c --- a/Python/peephole.c +++ b/Python/peephole.c @@ -475,7 +475,7 @@ CONST_STACK_CREATE(); for (i=find_op(codestr, 0) ; i= 1 && _Py_OPCODE(codestr[op_start-1]) == EXTENDED_ARG) { op_start--; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:26:32 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 11 Sep 2016 12:26:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Unskip_testPythonOrg_in_te?= =?utf-8?q?st=5Frobotparser?= Message-ID: <20160911122632.69549.56716.592A9CE1@psf.io> https://hg.python.org/cpython/rev/17ab2e41b753 changeset: 103630:17ab2e41b753 user: Berker Peksag date: Sun Sep 11 15:27:07 2016 +0300 summary: Unskip testPythonOrg in test_robotparser We should probably use pythontest.net for this. files: Lib/test/test_robotparser.py | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_robotparser.py b/Lib/test/test_robotparser.py --- a/Lib/test/test_robotparser.py +++ b/Lib/test/test_robotparser.py @@ -272,12 +272,11 @@ class NetworkTestCase(unittest.TestCase): - @unittest.skip('does not handle the gzip encoding delivered by pydotorg') def testPythonOrg(self): support.requires('network') with support.transient_internet('www.python.org'): parser = urllib.robotparser.RobotFileParser( - "http://www.python.org/robots.txt") + "https://www.python.org/robots.txt") parser.read() self.assertTrue( parser.can_fetch("*", "http://www.python.org/robots.txt")) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:36:55 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 11 Sep 2016 12:36:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328036=3A_Remove_u?= =?utf-8?q?nused_pysqlite=5Fflush=5Fstatement=5Fcache_function?= Message-ID: <20160911123655.59448.30509.19F8061D@psf.io> https://hg.python.org/cpython/rev/0bd7fe9b0bd7 changeset: 103631:0bd7fe9b0bd7 user: Berker Peksag date: Sun Sep 11 15:37:30 2016 +0300 summary: Issue #28036: Remove unused pysqlite_flush_statement_cache function files: Modules/_sqlite/connection.c | 20 -------------------- 1 files changed, 0 insertions(+), 20 deletions(-) diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -202,26 +202,6 @@ return 0; } -/* Empty the entire statement cache of this connection */ -void pysqlite_flush_statement_cache(pysqlite_Connection* self) -{ - pysqlite_Node* node; - pysqlite_Statement* statement; - - node = self->statement_cache->first; - - while (node) { - statement = (pysqlite_Statement*)(node->data); - (void)pysqlite_statement_finalize(statement); - node = node->next; - } - - Py_SETREF(self->statement_cache, - (pysqlite_Cache *)PyObject_CallFunction((PyObject *)&pysqlite_CacheType, "O", self)); - Py_DECREF(self); - self->statement_cache->decref_factory = 0; -} - /* action in (ACTION_RESET, ACTION_FINALIZE) */ void pysqlite_do_all_statements(pysqlite_Connection* self, int action, int reset_cursors) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:45:32 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 11 Sep 2016 12:45:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIwMTAw?= =?utf-8?q?=3A_Clarify_that_passing_flags_to_epoll=28=29_has_no_effect?= Message-ID: <20160911124532.13107.5814.B5727E57@psf.io> https://hg.python.org/cpython/rev/9023c4f5d467 changeset: 103632:9023c4f5d467 branch: 3.5 parent: 103626:65a1abf4b432 user: Berker Peksag date: Sun Sep 11 15:45:32 2016 +0300 summary: Issue #20100: Clarify that passing flags to epoll() has no effect files: Doc/library/select.rst | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Doc/library/select.rst b/Doc/library/select.rst --- a/Doc/library/select.rst +++ b/Doc/library/select.rst @@ -57,9 +57,7 @@ (Only supported on Linux 2.5.44 and newer.) Return an edge polling object, which can be used as Edge or Level Triggered interface for I/O - events. *sizehint* is deprecated and completely ignored. *flags* can be set - to :const:`EPOLL_CLOEXEC`, which causes the epoll descriptor to be closed - automatically when :func:`os.execve` is called. + events. *sizehint* and *flags* are deprecated and completely ignored. See the :ref:`epoll-objects` section below for the methods supported by epolling objects. @@ -77,6 +75,10 @@ Support for the :keyword:`with` statement was added. The new file descriptor is now non-inheritable. + .. deprecated:: 3.4 + The *flags* parameter. ``select.EPOLL_CLOEXEC`` is used by default now. + Use :func:`os.set_inheritable` to make the file descriptor inheritable. + .. function:: poll() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:45:33 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 11 Sep 2016 12:45:33 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2320100=3A_Merge_from_3=2E5?= Message-ID: <20160911124532.59404.57581.12243F73@psf.io> https://hg.python.org/cpython/rev/d2b5806fa770 changeset: 103633:d2b5806fa770 parent: 103631:0bd7fe9b0bd7 parent: 103632:9023c4f5d467 user: Berker Peksag date: Sun Sep 11 15:46:07 2016 +0300 summary: Issue #20100: Merge from 3.5 files: Doc/library/select.rst | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Doc/library/select.rst b/Doc/library/select.rst --- a/Doc/library/select.rst +++ b/Doc/library/select.rst @@ -57,9 +57,7 @@ (Only supported on Linux 2.5.44 and newer.) Return an edge polling object, which can be used as Edge or Level Triggered interface for I/O - events. *sizehint* is deprecated and completely ignored. *flags* can be set - to :const:`EPOLL_CLOEXEC`, which causes the epoll descriptor to be closed - automatically when :func:`os.execve` is called. + events. *sizehint* and *flags* are deprecated and completely ignored. See the :ref:`epoll-objects` section below for the methods supported by epolling objects. @@ -77,6 +75,10 @@ Support for the :keyword:`with` statement was added. The new file descriptor is now non-inheritable. + .. deprecated:: 3.4 + The *flags* parameter. ``select.EPOLL_CLOEXEC`` is used by default now. + Use :func:`os.set_inheritable` to make the file descriptor inheritable. + .. function:: poll() -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:46:11 2016 From: python-checkins at python.org (berker.peksag) Date: Sun, 11 Sep 2016 12:46:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Use_HTTP_in_testPythonOrg?= Message-ID: <20160911124611.48720.70550.8327B58D@psf.io> https://hg.python.org/cpython/rev/bc085b7e8fd8 changeset: 103634:bc085b7e8fd8 user: Berker Peksag date: Sun Sep 11 15:46:47 2016 +0300 summary: Use HTTP in testPythonOrg files: Lib/test/test_robotparser.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_robotparser.py b/Lib/test/test_robotparser.py --- a/Lib/test/test_robotparser.py +++ b/Lib/test/test_robotparser.py @@ -276,7 +276,7 @@ support.requires('network') with support.transient_internet('www.python.org'): parser = urllib.robotparser.RobotFileParser( - "https://www.python.org/robots.txt") + "http://www.python.org/robots.txt") parser.read() self.assertTrue( parser.can_fetch("*", "http://www.python.org/robots.txt")) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 08:55:46 2016 From: python-checkins at python.org (eric.smith) Date: Sun, 11 Sep 2016 12:55:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_24454=3A_Improve_the?= =?utf-8?q?_usability_of_the_re_match_object_named_group_API?= Message-ID: <20160911125546.13046.92171.4F718F4C@psf.io> https://hg.python.org/cpython/rev/ac0643314d12 changeset: 103635:ac0643314d12 user: Eric V. Smith date: Sun Sep 11 08:55:43 2016 -0400 summary: Issue 24454: Improve the usability of the re match object named group API files: Doc/library/re.rst | 16 ++++++++++++ Lib/test/test_re.py | 44 +++++++++++++++++++++++++++++++++ Misc/NEWS | 4 +++ Modules/_sre.c | 19 +++++++++++++- 4 files changed, 82 insertions(+), 1 deletions(-) diff --git a/Doc/library/re.rst b/Doc/library/re.rst --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -1015,6 +1015,22 @@ 'c3' +.. method:: match.__getitem__(g) + + This is identical to ``m.group(g)``. This allows easier access to + an individual group from a match: + + >>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist") + >>> m[0] # The entire match + 'Isaac Newton' + >>> m[1] # The first parenthesized subgroup. + 'Isaac' + >>> m[2] # The second parenthesized subgroup. + 'Newton' + + .. versionadded:: 3.6 + + .. method:: match.groups(default=None) Return a tuple containing all the subgroups of the match, from 1 up to however diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py --- a/Lib/test/test_re.py +++ b/Lib/test/test_re.py @@ -441,6 +441,50 @@ self.assertEqual(m.group(2, 1), ('b', 'a')) self.assertEqual(m.group(Index(2), Index(1)), ('b', 'a')) + def test_match_getitem(self): + pat = re.compile('(?:(?Pa)|(?Pb))(?Pc)?') + + m = pat.match('a') + self.assertEqual(m['a1'], 'a') + self.assertEqual(m['b2'], None) + self.assertEqual(m['c3'], None) + self.assertEqual('a1={a1} b2={b2} c3={c3}'.format_map(m), 'a1=a b2=None c3=None') + self.assertEqual(m[0], 'a') + self.assertEqual(m[1], 'a') + self.assertEqual(m[2], None) + self.assertEqual(m[3], None) + with self.assertRaisesRegex(IndexError, 'no such group'): + m['X'] + with self.assertRaisesRegex(IndexError, 'no such group'): + m[-1] + with self.assertRaisesRegex(IndexError, 'no such group'): + m[4] + with self.assertRaisesRegex(IndexError, 'no such group'): + m[0, 1] + with self.assertRaisesRegex(IndexError, 'no such group'): + m[(0,)] + with self.assertRaisesRegex(IndexError, 'no such group'): + m[(0, 1)] + with self.assertRaisesRegex(KeyError, 'a2'): + 'a1={a2}'.format_map(m) + + m = pat.match('ac') + self.assertEqual(m['a1'], 'a') + self.assertEqual(m['b2'], None) + self.assertEqual(m['c3'], 'c') + self.assertEqual('a1={a1} b2={b2} c3={c3}'.format_map(m), 'a1=a b2=None c3=c') + self.assertEqual(m[0], 'ac') + self.assertEqual(m[1], 'a') + self.assertEqual(m[2], None) + self.assertEqual(m[3], 'c') + + # Cannot assign. + with self.assertRaises(TypeError): + m[0] = 1 + + # No len(). + self.assertRaises(TypeError, len, m) + def test_re_fullmatch(self): # Issue 16203: Proposal: add re.fullmatch() method. self.assertEqual(re.fullmatch(r"a", "a").span(), (0, 1)) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -143,6 +143,10 @@ Library ------- +- Issue #24454: Regular expression match object groups are now + accessible using __getitem__. "mo[x]" is equivalent to + "mo.group(x)". + - Issue #10740: sqlite3 no longer implicitly commit an open transaction before DDL statements. diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -2121,6 +2121,12 @@ return result; } +static PyObject* +match_getitem(MatchObject* self, PyObject* name) +{ + return match_getslice(self, name, Py_None); +} + /*[clinic input] _sre.SRE_Match.groups @@ -2416,6 +2422,9 @@ Return subgroup(s) of the match by indices or names.\n\ For 0 returns the entire match."); +PyDoc_STRVAR(match_getitem_doc, +"__getitem__(name) <==> group(name).\n"); + static PyObject * match_lastindex_get(MatchObject *self) { @@ -2706,6 +2715,13 @@ pattern_getset, /* tp_getset */ }; +/* Match objects do not support length or assignment, but do support + __getitem__. */ +static PyMappingMethods match_as_mapping = { + NULL, + (binaryfunc)match_getitem, + NULL +}; static PyMethodDef match_methods[] = { {"group", (PyCFunction) match_group, METH_VARARGS, match_group_doc}, @@ -2717,6 +2733,7 @@ _SRE_SRE_MATCH_EXPAND_METHODDEF _SRE_SRE_MATCH___COPY___METHODDEF _SRE_SRE_MATCH___DEEPCOPY___METHODDEF + {"__getitem__", (PyCFunction)match_getitem, METH_O|METH_COEXIST, match_getitem_doc}, {NULL, NULL} }; @@ -2751,7 +2768,7 @@ (reprfunc)match_repr, /* tp_repr */ 0, /* tp_as_number */ 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ + &match_as_mapping, /* tp_as_mapping */ 0, /* tp_hash */ 0, /* tp_call */ 0, /* tp_str */ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 09:48:19 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 13:48:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixed_a_markup_in_docs=2E?= Message-ID: <20160911134815.69581.46813.5577DA49@psf.io> https://hg.python.org/cpython/rev/c45088b19342 changeset: 103636:c45088b19342 user: Serhiy Storchaka date: Sun Sep 11 16:47:59 2016 +0300 summary: Fixed a markup in docs. files: Doc/library/xmlrpc.client.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Doc/library/xmlrpc.client.rst b/Doc/library/xmlrpc.client.rst --- a/Doc/library/xmlrpc.client.rst +++ b/Doc/library/xmlrpc.client.rst @@ -144,7 +144,7 @@ Added the *context* argument. .. versionchanged:: 3.6 - Added support of type tags with prefixes (e.g.``ex:nil``). + Added support of type tags with prefixes (e.g. ``ex:nil``). Added support of unmarsalling additional types used by Apache XML-RPC implementation for numerics: ``i1``, ``i2``, ``i8``, ``biginteger``, ``float`` and ``bigdecimal``. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 09:50:53 2016 From: python-checkins at python.org (eric.smith) Date: Sun, 11 Sep 2016 13:50:53 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_24454=3A_Added_whats?= =?utf-8?q?new_entry=2C_removed_=5F=5Fgetitem=5F=5F_from_match=5Fmethods?= =?utf-8?q?=2E?= Message-ID: <20160911135053.21311.84867.9FE4133C@psf.io> https://hg.python.org/cpython/rev/3265247e08f0 changeset: 103637:3265247e08f0 user: Eric V. Smith date: Sun Sep 11 09:50:47 2016 -0400 summary: Issue 24454: Added whatsnew entry, removed __getitem__ from match_methods. Thanks Serhiy Storchaka. files: Doc/whatsnew/3.6.rst | 4 ++++ Modules/_sre.c | 1 - 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -724,6 +724,10 @@ ``'(?i)g(?-i:v)r'`` matches ``'GvR'`` and ``'gvr'``, but not ``'GVR'``. (Contributed by Serhiy Storchaka in :issue:`433028`.) +Match object groups can be accessed by ``__getitem__``, which is +equivalent to ``group()``. So ``mo['name']`` is now equivalent to +``mo.group('name')``. (Contributed by Eric Smith in :issue:`24454`.) + readline -------- diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -2733,7 +2733,6 @@ _SRE_SRE_MATCH_EXPAND_METHODDEF _SRE_SRE_MATCH___COPY___METHODDEF _SRE_SRE_MATCH___DEEPCOPY___METHODDEF - {"__getitem__", (PyCFunction)match_getitem, METH_O|METH_COEXIST, match_getitem_doc}, {NULL, NULL} }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 10:17:05 2016 From: python-checkins at python.org (vinay.sajip) Date: Sun, 11 Sep 2016 14:17:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Closes_=2323105=3A_Merged_update_from_3=2E5=2E?= Message-ID: <20160911141704.48560.39123.F727698F@psf.io> https://hg.python.org/cpython/rev/64610bcd326f changeset: 103640:64610bcd326f parent: 103637:3265247e08f0 parent: 103639:f1427491a8a5 user: Vinay Sajip date: Sun Sep 11 15:16:58 2016 +0100 summary: Closes #23105: Merged update from 3.5. files: Doc/library/os.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -948,7 +948,7 @@ O_EXCL O_TRUNC - These constants are available on Unix and Windows. + The above constants are available on Unix and Windows. .. data:: O_DSYNC @@ -957,11 +957,9 @@ O_NDELAY O_NONBLOCK O_NOCTTY - O_SHLOCK - O_EXLOCK O_CLOEXEC - These constants are only available on Unix. + The above constants are only available on Unix. .. versionchanged:: 3.3 Add :data:`O_CLOEXEC` constant. @@ -974,7 +972,7 @@ O_SEQUENTIAL O_TEXT - These constants are only available on Windows. + The above constants are only available on Windows. .. data:: O_ASYNC @@ -984,8 +982,10 @@ O_NOATIME O_PATH O_TMPFILE - - These constants are GNU extensions and not present if they are not defined by + O_SHLOCK + O_EXLOCK + + The above constants are extensions and not present if they are not defined by the C library. .. versionchanged:: 3.4 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 10:17:05 2016 From: python-checkins at python.org (vinay.sajip) Date: Sun, 11 Sep 2016 14:17:05 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzIzMTA1?= =?utf-8?q?=3A_Updated_documentation_on_open=28=29_flag_constants=2E?= Message-ID: <20160911141704.87363.6129.A28DC600@psf.io> https://hg.python.org/cpython/rev/5ca4c545dfe4 changeset: 103638:5ca4c545dfe4 branch: 2.7 parent: 103616:740e43eb8138 user: Vinay Sajip date: Sun Sep 11 15:11:50 2016 +0100 summary: Issue #23105: Updated documentation on open() flag constants. files: Doc/library/os.rst | 18 +++++++++--------- 1 files changed, 9 insertions(+), 9 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -883,7 +883,7 @@ O_EXCL O_TRUNC - These constants are available on Unix and Windows. + The above constants are available on Unix and Windows. .. data:: O_DSYNC @@ -892,10 +892,8 @@ O_NDELAY O_NONBLOCK O_NOCTTY - O_SHLOCK - O_EXLOCK - - These constants are only available on Unix. + + The above constants are only available on Unix. .. data:: O_BINARY @@ -906,7 +904,7 @@ O_SEQUENTIAL O_TEXT - These constants are only available on Windows. + The above constants are only available on Windows. .. data:: O_ASYNC @@ -914,9 +912,11 @@ O_DIRECTORY O_NOFOLLOW O_NOATIME - - These constants are GNU extensions and not present if they are not defined by - the C library. + O_SHLOCK + O_EXLOCK + + The above constants are extensions and not present if they are not + defined by the C library. .. _os-file-dir: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 10:17:06 2016 From: python-checkins at python.org (vinay.sajip) Date: Sun, 11 Sep 2016 14:17:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzIzMTA1?= =?utf-8?q?=3A_Updated_documentation_on_open=28=29_flag_constants=2E?= Message-ID: <20160911141704.21363.60114.D63AB21E@psf.io> https://hg.python.org/cpython/rev/f1427491a8a5 changeset: 103639:f1427491a8a5 branch: 3.5 parent: 103632:9023c4f5d467 user: Vinay Sajip date: Sun Sep 11 15:15:59 2016 +0100 summary: Issue #23105: Updated documentation on open() flag constants. files: Doc/library/os.rst | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -908,7 +908,7 @@ O_EXCL O_TRUNC - These constants are available on Unix and Windows. + The above constants are available on Unix and Windows. .. data:: O_DSYNC @@ -917,11 +917,9 @@ O_NDELAY O_NONBLOCK O_NOCTTY - O_SHLOCK - O_EXLOCK O_CLOEXEC - These constants are only available on Unix. + The above constants are only available on Unix. .. versionchanged:: 3.3 Add :data:`O_CLOEXEC` constant. @@ -934,7 +932,7 @@ O_SEQUENTIAL O_TEXT - These constants are only available on Windows. + The above constants are only available on Windows. .. data:: O_ASYNC @@ -944,8 +942,10 @@ O_NOATIME O_PATH O_TMPFILE - - These constants are GNU extensions and not present if they are not defined by + O_SHLOCK + O_EXLOCK + + The above constants are extensions and not present if they are not defined by the C library. .. versionchanged:: 3.4 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 10:20:32 2016 From: python-checkins at python.org (eric.smith) Date: Sun, 11 Sep 2016 14:20:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_24454=3A_Removed_unu?= =?utf-8?q?sed_match=5Fgetitem=5Fdoc=2E?= Message-ID: <20160911142031.19090.92008.48DFBBE1@psf.io> https://hg.python.org/cpython/rev/9eb38e0f1cad changeset: 103641:9eb38e0f1cad user: Eric V. Smith date: Sun Sep 11 10:20:27 2016 -0400 summary: Issue 24454: Removed unused match_getitem_doc. files: Modules/_sre.c | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -2422,9 +2422,6 @@ Return subgroup(s) of the match by indices or names.\n\ For 0 returns the entire match."); -PyDoc_STRVAR(match_getitem_doc, -"__getitem__(name) <==> group(name).\n"); - static PyObject * match_lastindex_get(MatchObject *self) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 12:45:32 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 16:45:32 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328076=3A_Variable?= =?utf-8?q?_annotations_should_be_mangled_for_private_names=2E?= Message-ID: <20160911164532.2824.54186.B339F224@psf.io> https://hg.python.org/cpython/rev/1c2a4f29e1ab changeset: 103642:1c2a4f29e1ab user: Guido van Rossum date: Sun Sep 11 09:45:24 2016 -0700 summary: Issue #28076: Variable annotations should be mangled for private names. By Ivan Levkivskyi. files: Doc/reference/simple_stmts.rst | 7 ++++--- Lib/test/test_grammar.py | 4 ++-- Python/compile.c | 8 +++++++- 3 files changed, 13 insertions(+), 6 deletions(-) diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -334,9 +334,10 @@ For simple names as assignment targets, if in class or module scope, the annotations are evaluated and stored in a special class or module attribute :attr:`__annotations__` -that is a dictionary mapping from variable names to evaluated annotations. -This attribute is writable and is automatically created at the start -of class or module body execution, if annotations are found statically. +that is a dictionary mapping from variable names (mangled if private) to +evaluated annotations. This attribute is writable and is automatically +created at the start of class or module body execution, if annotations +are found statically. For expressions as assignment targets, the annotations are evaluated if in class or module scope, but not stored. diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -328,12 +328,12 @@ # class semantics class C: - x: int + __foo: int s: str = "attr" z = 2 def __init__(self, x): self.x: int = x - self.assertEqual(C.__annotations__, {'x': int, 's': str}) + self.assertEqual(C.__annotations__, {'_C__foo': int, 's': str}) with self.assertRaises(NameError): class CBad: no_such_name_defined.attr: int = 0 diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -4562,6 +4562,7 @@ compiler_annassign(struct compiler *c, stmt_ty s) { expr_ty targ = s->v.AnnAssign.target; + PyObject* mangled; assert(s->kind == AnnAssign_kind); @@ -4576,8 +4577,13 @@ if (s->v.AnnAssign.simple && (c->u->u_scope_type == COMPILER_SCOPE_MODULE || c->u->u_scope_type == COMPILER_SCOPE_CLASS)) { + mangled = _Py_Mangle(c->u->u_private, targ->v.Name.id); + if (!mangled) { + return 0; + } VISIT(c, expr, s->v.AnnAssign.annotation); - ADDOP_O(c, STORE_ANNOTATION, targ->v.Name.id, names) + /* ADDOP_N decrefs its argument */ + ADDOP_N(c, STORE_ANNOTATION, mangled, names); } break; case Attribute_kind: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 13:50:01 2016 From: python-checkins at python.org (christian.heimes) Date: Sun, 11 Sep 2016 17:50:01 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328078=3A_Silence_?= =?utf-8?q?resource_warnings_in_test=5Fsocket=2E_Initial_patch_by_Xiang?= Message-ID: <20160911175001.2793.33642.B1443D4E@psf.io> https://hg.python.org/cpython/rev/c515bc3b29bf changeset: 103643:c515bc3b29bf user: Christian Heimes date: Sun Sep 11 19:49:56 2016 +0200 summary: Issue #28078: Silence resource warnings in test_socket. Initial patch by Xiang Zhang, thanks files: Lib/test/test_socket.py | 34 ++++++++++++++++------------ 1 files changed, 19 insertions(+), 15 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -5347,8 +5347,10 @@ sock.bind((typ, name)) except FileNotFoundError as e: # type / algorithm is not available + sock.close() raise unittest.SkipTest(str(e), typ, name) - return sock + else + return sock def test_sha256(self): expected = bytes.fromhex("ba7816bf8f01cfea414140de5dae2223b00361a396" @@ -5494,20 +5496,22 @@ def test_sendmsg_afalg_args(self): sock = socket.socket(socket.AF_ALG, socket.SOCK_SEQPACKET, 0) - with self.assertRaises(TypeError): - sock.sendmsg_afalg() - - with self.assertRaises(TypeError): - sock.sendmsg_afalg(op=None) - - with self.assertRaises(TypeError): - sock.sendmsg_afalg(1) - - with self.assertRaises(TypeError): - sock.sendmsg_afalg(op=socket.ALG_OP_ENCRYPT, assoclen=None) - - with self.assertRaises(TypeError): - sock.sendmsg_afalg(op=socket.ALG_OP_ENCRYPT, assoclen=-1) + with sock: + with self.assertRaises(TypeError): + sock.sendmsg_afalg() + + with self.assertRaises(TypeError): + sock.sendmsg_afalg(op=None) + + with self.assertRaises(TypeError): + sock.sendmsg_afalg(1) + + with self.assertRaises(TypeError): + sock.sendmsg_afalg(op=socket.ALG_OP_ENCRYPT, assoclen=None) + + with self.assertRaises(TypeError): + sock.sendmsg_afalg(op=socket.ALG_OP_ENCRYPT, assoclen=-1) + def test_main(): tests = [GeneralModuleTests, BasicTCPTest, TCPCloserTest, TCPTimeoutTest, -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 13:54:50 2016 From: python-checkins at python.org (christian.heimes) Date: Sun, 11 Sep 2016 17:54:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_28022=3A_Catch_depre?= =?utf-8?q?cation_warning_in_test=5Fhttplib=2C_reported_by_Martin?= Message-ID: <20160911175450.3133.44943.FDE1EC6B@psf.io> https://hg.python.org/cpython/rev/2e541e994927 changeset: 103644:2e541e994927 user: Christian Heimes date: Sun Sep 11 19:54:43 2016 +0200 summary: Issue 28022: Catch deprecation warning in test_httplib, reported by Martin Panter files: Lib/test/test_httplib.py | 15 +++++++++------ 1 files changed, 9 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -1642,14 +1642,16 @@ with self.assertRaises(ssl.CertificateError): h.request('GET', '/') # Same with explicit check_hostname=True - h = client.HTTPSConnection('localhost', server.port, context=context, - check_hostname=True) + with support.check_warnings(('', DeprecationWarning)): + h = client.HTTPSConnection('localhost', server.port, + context=context, check_hostname=True) with self.assertRaises(ssl.CertificateError): h.request('GET', '/') # With check_hostname=False, the mismatching is ignored context.check_hostname = False - h = client.HTTPSConnection('localhost', server.port, context=context, - check_hostname=False) + with support.check_warnings(('', DeprecationWarning)): + h = client.HTTPSConnection('localhost', server.port, + context=context, check_hostname=False) h.request('GET', '/nonexistent') resp = h.getresponse() resp.close() @@ -1666,8 +1668,9 @@ h.close() # Passing check_hostname to HTTPSConnection should override the # context's setting. - h = client.HTTPSConnection('localhost', server.port, context=context, - check_hostname=True) + with support.check_warnings(('', DeprecationWarning)): + h = client.HTTPSConnection('localhost', server.port, + context=context, check_hostname=True) with self.assertRaises(ssl.CertificateError): h.request('GET', '/') -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 14:03:52 2016 From: python-checkins at python.org (christian.heimes) Date: Sun, 11 Sep 2016 18:03:52 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328078=3A_Add_miss?= =?utf-8?q?ing_colon?= Message-ID: <20160911180352.8696.49025.49EE8D4D@psf.io> https://hg.python.org/cpython/rev/8b655668c04f changeset: 103645:8b655668c04f user: Christian Heimes date: Sun Sep 11 20:03:46 2016 +0200 summary: Issue #28078: Add missing colon files: Lib/test/test_socket.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -5349,7 +5349,7 @@ # type / algorithm is not available sock.close() raise unittest.SkipTest(str(e), typ, name) - else + else: return sock def test_sha256(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 14:11:45 2016 From: python-checkins at python.org (christian.heimes) Date: Sun, 11 Sep 2016 18:11:45 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_27744=3A_test=5Faes?= =?utf-8?q?=5Fcbc_is_blocking_x86-64_Ubuntu_15=2E10_Skylake_CPU_3=2Ex_for_?= =?utf-8?q?a?= Message-ID: <20160911181145.2532.49312.B38B59E5@psf.io> https://hg.python.org/cpython/rev/55d77f5a7cb3 changeset: 103646:55d77f5a7cb3 user: Christian Heimes date: Sun Sep 11 20:11:30 2016 +0200 summary: Issue 27744: test_aes_cbc is blocking x86-64 Ubuntu 15.10 Skylake CPU 3.x for a while. Require Kernel 4.3+ for now files: Lib/test/test_socket.py | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -5378,7 +5378,9 @@ op.sendall(b"what do ya want for nothing?") self.assertEqual(op.recv(512), expected) - @support.requires_linux_version(3, 19) + # Although it should work with 3.19 and newer the test blocks on + # Ubuntu 15.10 with Kernel 4.2.0-19. + @support.requires_linux_version(4, 3) def test_aes_cbc(self): key = bytes.fromhex('06a9214036b8a15b512e03d534120006') iv = bytes.fromhex('3dafba429d9eb430b422da802c9fac41') @@ -5419,7 +5421,7 @@ self.assertEqual(len(dec), msglen * multiplier) self.assertEqual(dec, msg * multiplier) - @support.requires_linux_version(3, 19) + @support.requires_linux_version(4, 3) # see test_aes_cbc def test_aead_aes_gcm(self): key = bytes.fromhex('c939cc13397c1d37de6ae0e1cb7c423c') iv = bytes.fromhex('b3d8cc017cbb89b39e0f67e2') @@ -5483,7 +5485,7 @@ res = op.recv(len(msg)) self.assertEqual(plain, res[assoclen:-taglen]) - @support.requires_linux_version(3, 19) + @support.requires_linux_version(4, 3) # see test_aes_cbc def test_drbg_pr_sha256(self): # deterministic random bit generator, prediction resistance, sha256 with self.create_alg('rng', 'drbg_pr_sha256') as algo: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 14:28:51 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 18:28:51 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327810=3A_Regenera?= =?utf-8?q?te_Argument_Clinic=2E?= Message-ID: <20160911182851.48474.54664.5AF0455E@psf.io> https://hg.python.org/cpython/rev/603bef7bb744 changeset: 103647:603bef7bb744 user: Serhiy Storchaka date: Sun Sep 11 21:25:45 2016 +0300 summary: Issue #27810: Regenerate Argument Clinic. files: Modules/_io/clinic/_iomodule.c.h | 8 +- Modules/_io/clinic/textio.c.h | 8 +- Modules/_sha3/clinic/sha3module.c.h | 14 ++-- Modules/cjkcodecs/clinic/multibytecodec.c.h | 26 +++++----- 4 files changed, 28 insertions(+), 28 deletions(-) diff --git a/Modules/_io/clinic/_iomodule.c.h b/Modules/_io/clinic/_iomodule.c.h --- a/Modules/_io/clinic/_iomodule.c.h +++ b/Modules/_io/clinic/_iomodule.c.h @@ -127,7 +127,7 @@ "opened in a binary mode."); #define _IO_OPEN_METHODDEF \ - {"open", (PyCFunction)_io_open, METH_VARARGS|METH_KEYWORDS, _io_open__doc__}, + {"open", (PyCFunction)_io_open, METH_FASTCALL, _io_open__doc__}, static PyObject * _io_open_impl(PyObject *module, PyObject *file, const char *mode, @@ -135,7 +135,7 @@ const char *newline, int closefd, PyObject *opener); static PyObject * -_io_open(PyObject *module, PyObject *args, PyObject *kwargs) +_io_open(PyObject *module, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"file", "mode", "buffering", "encoding", "errors", "newline", "closefd", "opener", NULL}; @@ -149,7 +149,7 @@ int closefd = 1; PyObject *opener = Py_None; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &file, &mode, &buffering, &encoding, &errors, &newline, &closefd, &opener)) { goto exit; } @@ -158,4 +158,4 @@ exit: return return_value; } -/*[clinic end generated code: output=14769629391a3130 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=c5b8fc8b83102bbf input=a9049054013a1b77]*/ diff --git a/Modules/_io/clinic/textio.c.h b/Modules/_io/clinic/textio.c.h --- a/Modules/_io/clinic/textio.c.h +++ b/Modules/_io/clinic/textio.c.h @@ -46,14 +46,14 @@ "\n"); #define _IO_INCREMENTALNEWLINEDECODER_DECODE_METHODDEF \ - {"decode", (PyCFunction)_io_IncrementalNewlineDecoder_decode, METH_VARARGS|METH_KEYWORDS, _io_IncrementalNewlineDecoder_decode__doc__}, + {"decode", (PyCFunction)_io_IncrementalNewlineDecoder_decode, METH_FASTCALL, _io_IncrementalNewlineDecoder_decode__doc__}, static PyObject * _io_IncrementalNewlineDecoder_decode_impl(nldecoder_object *self, PyObject *input, int final); static PyObject * -_io_IncrementalNewlineDecoder_decode(nldecoder_object *self, PyObject *args, PyObject *kwargs) +_io_IncrementalNewlineDecoder_decode(nldecoder_object *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"input", "final", NULL}; @@ -61,7 +61,7 @@ PyObject *input; int final = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &input, &final)) { goto exit; } @@ -464,4 +464,4 @@ { return _io_TextIOWrapper_close_impl(self); } -/*[clinic end generated code: output=7ec624a9bf6393f5 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=78ad14eba1667254 input=a9049054013a1b77]*/ diff --git a/Modules/_sha3/clinic/sha3module.c.h b/Modules/_sha3/clinic/sha3module.c.h --- a/Modules/_sha3/clinic/sha3module.c.h +++ b/Modules/_sha3/clinic/sha3module.c.h @@ -99,20 +99,20 @@ "Return the digest value as a string of binary data."); #define _SHA3_SHAKE_128_DIGEST_METHODDEF \ - {"digest", (PyCFunction)_sha3_shake_128_digest, METH_VARARGS|METH_KEYWORDS, _sha3_shake_128_digest__doc__}, + {"digest", (PyCFunction)_sha3_shake_128_digest, METH_FASTCALL, _sha3_shake_128_digest__doc__}, static PyObject * _sha3_shake_128_digest_impl(SHA3object *self, unsigned long length); static PyObject * -_sha3_shake_128_digest(SHA3object *self, PyObject *args, PyObject *kwargs) +_sha3_shake_128_digest(SHA3object *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"length", NULL}; static _PyArg_Parser _parser = {"k:digest", _keywords, 0}; unsigned long length; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &length)) { goto exit; } @@ -129,20 +129,20 @@ "Return the digest value as a string of hexadecimal digits."); #define _SHA3_SHAKE_128_HEXDIGEST_METHODDEF \ - {"hexdigest", (PyCFunction)_sha3_shake_128_hexdigest, METH_VARARGS|METH_KEYWORDS, _sha3_shake_128_hexdigest__doc__}, + {"hexdigest", (PyCFunction)_sha3_shake_128_hexdigest, METH_FASTCALL, _sha3_shake_128_hexdigest__doc__}, static PyObject * _sha3_shake_128_hexdigest_impl(SHA3object *self, unsigned long length); static PyObject * -_sha3_shake_128_hexdigest(SHA3object *self, PyObject *args, PyObject *kwargs) +_sha3_shake_128_hexdigest(SHA3object *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"length", NULL}; static _PyArg_Parser _parser = {"k:hexdigest", _keywords, 0}; unsigned long length; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &length)) { goto exit; } @@ -151,4 +151,4 @@ exit: return return_value; } -/*[clinic end generated code: output=50cff05f2c74d41e input=a9049054013a1b77]*/ +/*[clinic end generated code: output=9888beab45136a56 input=a9049054013a1b77]*/ diff --git a/Modules/cjkcodecs/clinic/multibytecodec.c.h b/Modules/cjkcodecs/clinic/multibytecodec.c.h --- a/Modules/cjkcodecs/clinic/multibytecodec.c.h +++ b/Modules/cjkcodecs/clinic/multibytecodec.c.h @@ -14,7 +14,7 @@ "registered with codecs.register_error that can handle UnicodeEncodeErrors."); #define _MULTIBYTECODEC_MULTIBYTECODEC_ENCODE_METHODDEF \ - {"encode", (PyCFunction)_multibytecodec_MultibyteCodec_encode, METH_VARARGS|METH_KEYWORDS, _multibytecodec_MultibyteCodec_encode__doc__}, + {"encode", (PyCFunction)_multibytecodec_MultibyteCodec_encode, METH_FASTCALL, _multibytecodec_MultibyteCodec_encode__doc__}, static PyObject * _multibytecodec_MultibyteCodec_encode_impl(MultibyteCodecObject *self, @@ -22,7 +22,7 @@ const char *errors); static PyObject * -_multibytecodec_MultibyteCodec_encode(MultibyteCodecObject *self, PyObject *args, PyObject *kwargs) +_multibytecodec_MultibyteCodec_encode(MultibyteCodecObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"input", "errors", NULL}; @@ -30,7 +30,7 @@ PyObject *input; const char *errors = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &input, &errors)) { goto exit; } @@ -52,7 +52,7 @@ "codecs.register_error that is able to handle UnicodeDecodeErrors.\""); #define _MULTIBYTECODEC_MULTIBYTECODEC_DECODE_METHODDEF \ - {"decode", (PyCFunction)_multibytecodec_MultibyteCodec_decode, METH_VARARGS|METH_KEYWORDS, _multibytecodec_MultibyteCodec_decode__doc__}, + {"decode", (PyCFunction)_multibytecodec_MultibyteCodec_decode, METH_FASTCALL, _multibytecodec_MultibyteCodec_decode__doc__}, static PyObject * _multibytecodec_MultibyteCodec_decode_impl(MultibyteCodecObject *self, @@ -60,7 +60,7 @@ const char *errors); static PyObject * -_multibytecodec_MultibyteCodec_decode(MultibyteCodecObject *self, PyObject *args, PyObject *kwargs) +_multibytecodec_MultibyteCodec_decode(MultibyteCodecObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"input", "errors", NULL}; @@ -68,7 +68,7 @@ Py_buffer input = {NULL, NULL}; const char *errors = NULL; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &input, &errors)) { goto exit; } @@ -89,7 +89,7 @@ "\n"); #define _MULTIBYTECODEC_MULTIBYTEINCREMENTALENCODER_ENCODE_METHODDEF \ - {"encode", (PyCFunction)_multibytecodec_MultibyteIncrementalEncoder_encode, METH_VARARGS|METH_KEYWORDS, _multibytecodec_MultibyteIncrementalEncoder_encode__doc__}, + {"encode", (PyCFunction)_multibytecodec_MultibyteIncrementalEncoder_encode, METH_FASTCALL, _multibytecodec_MultibyteIncrementalEncoder_encode__doc__}, static PyObject * _multibytecodec_MultibyteIncrementalEncoder_encode_impl(MultibyteIncrementalEncoderObject *self, @@ -97,7 +97,7 @@ int final); static PyObject * -_multibytecodec_MultibyteIncrementalEncoder_encode(MultibyteIncrementalEncoderObject *self, PyObject *args, PyObject *kwargs) +_multibytecodec_MultibyteIncrementalEncoder_encode(MultibyteIncrementalEncoderObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"input", "final", NULL}; @@ -105,7 +105,7 @@ PyObject *input; int final = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &input, &final)) { goto exit; } @@ -138,7 +138,7 @@ "\n"); #define _MULTIBYTECODEC_MULTIBYTEINCREMENTALDECODER_DECODE_METHODDEF \ - {"decode", (PyCFunction)_multibytecodec_MultibyteIncrementalDecoder_decode, METH_VARARGS|METH_KEYWORDS, _multibytecodec_MultibyteIncrementalDecoder_decode__doc__}, + {"decode", (PyCFunction)_multibytecodec_MultibyteIncrementalDecoder_decode, METH_FASTCALL, _multibytecodec_MultibyteIncrementalDecoder_decode__doc__}, static PyObject * _multibytecodec_MultibyteIncrementalDecoder_decode_impl(MultibyteIncrementalDecoderObject *self, @@ -146,7 +146,7 @@ int final); static PyObject * -_multibytecodec_MultibyteIncrementalDecoder_decode(MultibyteIncrementalDecoderObject *self, PyObject *args, PyObject *kwargs) +_multibytecodec_MultibyteIncrementalDecoder_decode(MultibyteIncrementalDecoderObject *self, PyObject **args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"input", "final", NULL}; @@ -154,7 +154,7 @@ Py_buffer input = {NULL, NULL}; int final = 0; - if (!_PyArg_ParseTupleAndKeywordsFast(args, kwargs, &_parser, + if (!_PyArg_ParseStack(args, nargs, kwnames, &_parser, &input, &final)) { goto exit; } @@ -330,4 +330,4 @@ #define _MULTIBYTECODEC___CREATE_CODEC_METHODDEF \ {"__create_codec", (PyCFunction)_multibytecodec___create_codec, METH_O, _multibytecodec___create_codec__doc__}, -/*[clinic end generated code: output=8e86fa162c85230b input=a9049054013a1b77]*/ +/*[clinic end generated code: output=134b9e36cb985939 input=a9049054013a1b77]*/ -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 14:54:26 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 18:54:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Regenerate_the_configure_s?= =?utf-8?q?cript=2E_Now_it_supports_--runstatedir_option?= Message-ID: <20160911185425.69446.66314.43364656@psf.io> https://hg.python.org/cpython/rev/816ae3abd928 changeset: 103648:816ae3abd928 user: Serhiy Storchaka date: Sun Sep 11 21:53:53 2016 +0300 summary: Regenerate the configure script. Now it supports --runstatedir option (directory for pid files etc, /run or /var/run). https://lists.gnu.org/archive/html/bug-autoconf/2013-09/msg00002.html files: configure | 14 +++++++++++++- 1 files changed, 13 insertions(+), 1 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -784,6 +784,7 @@ docdir oldincludedir includedir +runstatedir localstatedir sharedstatedir sysconfdir @@ -894,6 +895,7 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' +runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1146,6 +1148,15 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; + -runstatedir | --runstatedir | --runstatedi | --runstated \ + | --runstate | --runstat | --runsta | --runst | --runs \ + | --run | --ru | --r) + ac_prev=runstatedir ;; + -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ + | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ + | --run=* | --ru=* | --r=*) + runstatedir=$ac_optarg ;; + -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1283,7 +1294,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir + libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1436,6 +1447,7 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] + --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 15:02:58 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 19:02:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2323545=3A_Turn_on_?= =?utf-8?q?extra_warnings_on_GCC=2E?= Message-ID: <20160911190258.2591.63589.17082E0D@psf.io> https://hg.python.org/cpython/rev/99adb5df8cb6 changeset: 103649:99adb5df8cb6 user: Serhiy Storchaka date: Sun Sep 11 21:56:32 2016 +0300 summary: Issue #23545: Turn on extra warnings on GCC. files: configure | 124 +++++++++++++++++++++++++++++++++++++++ configure.ac | 61 +++++++++++++++++++ 2 files changed, 185 insertions(+), 0 deletions(-) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -6920,6 +6920,47 @@ yes) CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -Wextra" >&5 +$as_echo_n "checking for -Wextra... " >&6; } + ac_save_cc="$CC" + CC="$CC -Wextra -Werror" + if ${ac_cv_extra_warnings+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + + +int +main () +{ + + ; + return 0; +} + +_ACEOF +if ac_fn_c_try_compile "$LINENO"; then : + + ac_cv_extra_warnings=yes + +else + + ac_cv_extra_warnings=no + +fi +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext +fi + + CC="$ac_save_cc" + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_extra_warnings" >&5 +$as_echo "$ac_cv_extra_warnings" >&6; } + + if test $ac_cv_extra_warnings = yes + then + CFLAGS_NODIST="$CFLAGS_NODIST -Wextra" + fi + # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable # -fno-strict-aliasing on versions of GCC that support but produce @@ -7040,6 +7081,89 @@ if test $ac_cv_disable_unused_result_warning = yes then BASECFLAGS="$BASECFLAGS -Wno-unused-result" + CFLAGS_NODIST="$CFLAGS_NODIST -Wno-unused-result" + fi + + { $as_echo "$as_me:${as_lineno-$LINENO}: checking if we can turn off $CC unused parameter warning" >&5 +$as_echo_n "checking if we can turn off $CC unused parameter warning... " >&6; } + ac_save_cc="$CC" + CC="$CC -Wunused-parameter -Werror" + if ${ac_cv_disable_unused_parameter_warning+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + + +int +main () +{ + + ; + return 0; +} + +_ACEOF +if ac_fn_c_try_compile "$LINENO"; then : + + ac_cv_disable_unused_parameter_warning=yes + +else + + ac_cv_disable_unused_parameter_warning=no + +fi +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext +fi + + CC="$ac_save_cc" + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_disable_unused_parameter_warning" >&5 +$as_echo "$ac_cv_disable_unused_parameter_warning" >&6; } + + if test $ac_cv_disable_unused_parameter_warning = yes + then + CFLAGS_NODIST="$CFLAGS_NODIST -Wno-unused-parameter" + fi + + { $as_echo "$as_me:${as_lineno-$LINENO}: checking if we can turn off $CC missing field initializers warning" >&5 +$as_echo_n "checking if we can turn off $CC missing field initializers warning... " >&6; } + ac_save_cc="$CC" + CC="$CC -Wmissing-field-initializers -Werror" + if ${ac_cv_disable_missing_field_initializers+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + + +int +main () +{ + + ; + return 0; +} + +_ACEOF +if ac_fn_c_try_compile "$LINENO"; then : + + ac_cv_disable_missing_field_initializers=yes + +else + + ac_cv_disable_missing_field_initializers=no + +fi +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext +fi + + CC="$ac_save_cc" + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_disable_missing_field_initializers" >&5 +$as_echo "$ac_cv_disable_missing_field_initializers" >&6; } + + if test $ac_cv_disable_missing_field_initializers = yes + then + CFLAGS_NODIST="$CFLAGS_NODIST -Wno-missing-field-initializers" fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking if we can turn on $CC mixed sign comparison warning" >&5 diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -1517,6 +1517,26 @@ yes) CFLAGS_NODIST="$CFLAGS_NODIST -std=c99" + AC_MSG_CHECKING(for -Wextra) + ac_save_cc="$CC" + CC="$CC -Wextra -Werror" + AC_CACHE_VAL(ac_cv_extra_warnings, + AC_COMPILE_IFELSE( + [ + AC_LANG_PROGRAM([[]], [[]]) + ],[ + ac_cv_extra_warnings=yes + ],[ + ac_cv_extra_warnings=no + ])) + CC="$ac_save_cc" + AC_MSG_RESULT($ac_cv_extra_warnings) + + if test $ac_cv_extra_warnings = yes + then + CFLAGS_NODIST="$CFLAGS_NODIST -Wextra" + fi + # Python doesn't violate C99 aliasing rules, but older versions of # GCC produce warnings for legal Python code. Enable # -fno-strict-aliasing on versions of GCC that support but produce @@ -1581,6 +1601,47 @@ if test $ac_cv_disable_unused_result_warning = yes then BASECFLAGS="$BASECFLAGS -Wno-unused-result" + CFLAGS_NODIST="$CFLAGS_NODIST -Wno-unused-result" + fi + + AC_MSG_CHECKING(if we can turn off $CC unused parameter warning) + ac_save_cc="$CC" + CC="$CC -Wunused-parameter -Werror" + AC_CACHE_VAL(ac_cv_disable_unused_parameter_warning, + AC_COMPILE_IFELSE( + [ + AC_LANG_PROGRAM([[]], [[]]) + ],[ + ac_cv_disable_unused_parameter_warning=yes + ],[ + ac_cv_disable_unused_parameter_warning=no + ])) + CC="$ac_save_cc" + AC_MSG_RESULT($ac_cv_disable_unused_parameter_warning) + + if test $ac_cv_disable_unused_parameter_warning = yes + then + CFLAGS_NODIST="$CFLAGS_NODIST -Wno-unused-parameter" + fi + + AC_MSG_CHECKING(if we can turn off $CC missing field initializers warning) + ac_save_cc="$CC" + CC="$CC -Wmissing-field-initializers -Werror" + AC_CACHE_VAL(ac_cv_disable_missing_field_initializers, + AC_COMPILE_IFELSE( + [ + AC_LANG_PROGRAM([[]], [[]]) + ],[ + ac_cv_disable_missing_field_initializers=yes + ],[ + ac_cv_disable_missing_field_initializers=no + ])) + CC="$ac_save_cc" + AC_MSG_RESULT($ac_cv_disable_missing_field_initializers) + + if test $ac_cv_disable_missing_field_initializers = yes + then + CFLAGS_NODIST="$CFLAGS_NODIST -Wno-missing-field-initializers" fi AC_MSG_CHECKING(if we can turn on $CC mixed sign comparison warning) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 15:40:08 2016 From: python-checkins at python.org (xavier.degaye) Date: Sun, 11 Sep 2016 19:40:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327917=3A_Fix_test?= =?utf-8?q?=5Ftriplet=5Fin=5Fext=5Fsuffix_for_the_=27x86=27_Android_platfo?= =?utf-8?b?cm0u?= Message-ID: <20160911194008.13046.68305.9512BF14@psf.io> https://hg.python.org/cpython/rev/676e6c3d30db changeset: 103650:676e6c3d30db user: Xavier de Gaye date: Sun Sep 11 21:39:17 2016 +0200 summary: Issue #27917: Fix test_triplet_in_ext_suffix for the 'x86' Android platform. files: Lib/test/test_sysconfig.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -394,8 +394,9 @@ self.assertTrue('linux' in suffix, suffix) if re.match('(i[3-6]86|x86_64)$', machine): if ctypes.sizeof(ctypes.c_char_p()) == 4: - self.assertTrue(suffix.endswith('i386-linux-gnu.so') \ - or suffix.endswith('x86_64-linux-gnux32.so'), + self.assertTrue(suffix.endswith('i386-linux-gnu.so') or + suffix.endswith('i686-linux-android.so') or + suffix.endswith('x86_64-linux-gnux32.so'), suffix) else: # 8 byte pointer size self.assertTrue(suffix.endswith('x86_64-linux-gnu.so'), suffix) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 15:50:17 2016 From: python-checkins at python.org (ned.deily) Date: Sun, 11 Sep 2016 19:50:17 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_some_additional_suspic?= =?utf-8?q?ious_exemption_rules_for_recent_doc_changes=2E?= Message-ID: <20160911195017.13637.60013.D8322CC1@psf.io> https://hg.python.org/cpython/rev/e1f529263b19 changeset: 103651:e1f529263b19 user: Ned Deily date: Sun Sep 11 15:49:37 2016 -0400 summary: Add some additional suspicious exemption rules for recent doc changes. files: Doc/tools/susp-ignored.csv | 25 +++++++++++++++++++++++++ 1 files changed, 25 insertions(+), 0 deletions(-) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -25,6 +25,30 @@ howto/curses,,:magenta,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:cyan,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" howto/curses,,:white,"2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and 7:white. The" +howto/instrumentation,,::,python$target:::function-entry +howto/instrumentation,,:function,python$target:::function-entry +howto/instrumentation,,::,python$target:::function-return +howto/instrumentation,,:function,python$target:::function-return +howto/instrumentation,,:call,156641360502280 function-entry:call_stack.py:start:23 +howto/instrumentation,,:start,156641360502280 function-entry:call_stack.py:start:23 +howto/instrumentation,,:function,156641360518804 function-entry: call_stack.py:function_1:1 +howto/instrumentation,,:function,156641360532797 function-entry: call_stack.py:function_3:9 +howto/instrumentation,,:function,156641360546807 function-return: call_stack.py:function_3:10 +howto/instrumentation,,:function,156641360563367 function-return: call_stack.py:function_1:2 +howto/instrumentation,,:function,156641360578365 function-entry: call_stack.py:function_2:5 +howto/instrumentation,,:function,156641360591757 function-entry: call_stack.py:function_1:1 +howto/instrumentation,,:function,156641360605556 function-entry: call_stack.py:function_3:9 +howto/instrumentation,,:function,156641360617482 function-return: call_stack.py:function_3:10 +howto/instrumentation,,:function,156641360629814 function-return: call_stack.py:function_1:2 +howto/instrumentation,,:function,156641360642285 function-return: call_stack.py:function_2:6 +howto/instrumentation,,:function,156641360656770 function-entry: call_stack.py:function_3:9 +howto/instrumentation,,:function,156641360669707 function-return: call_stack.py:function_3:10 +howto/instrumentation,,:function,156641360687853 function-entry: call_stack.py:function_4:13 +howto/instrumentation,,:function,156641360700719 function-return: call_stack.py:function_4:14 +howto/instrumentation,,:function,156641360719640 function-entry: call_stack.py:function_5:18 +howto/instrumentation,,:function,156641360732567 function-return: call_stack.py:function_5:21 +howto/instrumentation,,:call,156641360747370 function-return:call_stack.py:start:28 +howto/instrumentation,,:start,156641360747370 function-return:call_stack.py:start:28 howto/ipaddress,,:DB8,>>> ipaddress.ip_address('2001:DB8::1') howto/ipaddress,,::,>>> ipaddress.ip_address('2001:DB8::1') howto/ipaddress,,:db8,IPv6Address('2001:db8::1') @@ -210,6 +234,7 @@ library/venv,,:param,":param progress: If setuptools or pip are installed, the progress of the" library/venv,,:param,":param nopip: If True, pip is not installed into the created" library/venv,,:param,:param context: The information for the virtual environment +library/xmlrpc.client,,:nil,ex:nil library/xmlrpc.client,,:pass,http://user:pass at host:port/path library/xmlrpc.client,,:pass,user:pass library/xmlrpc.client,,:port,http://user:pass at host:port/path -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 16:22:55 2016 From: python-checkins at python.org (xavier.degaye) Date: Sun, 11 Sep 2016 20:22:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328046=3A_get=5Fsy?= =?utf-8?q?sconfigdata=5Fname=28=29_uses_the_=5FPYTHON=5FSYSCONFIGDATA=5FN?= =?utf-8?q?AME?= Message-ID: <20160911202255.59651.36057.888BF670@psf.io> https://hg.python.org/cpython/rev/04b679dd11eb changeset: 103652:04b679dd11eb user: Xavier de Gaye date: Sun Sep 11 22:22:24 2016 +0200 summary: Issue #28046: get_sysconfigdata_name() uses the _PYTHON_SYSCONFIGDATA_NAME environment variable that is defined when cross-compiling. files: Lib/distutils/sysconfig.py | 5 +++-- Lib/sysconfig.py | 22 ++++++++-------------- configure | 16 ++-------------- configure.ac | 2 +- 4 files changed, 14 insertions(+), 31 deletions(-) diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py --- a/Lib/distutils/sysconfig.py +++ b/Lib/distutils/sysconfig.py @@ -418,11 +418,12 @@ def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" # _sysconfigdata is generated at build time, see the sysconfig module - name = '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( + name = os.environ.get('_PYTHON_SYSCONFIGDATA_NAME', + '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( abi=sys.abiflags, platform=sys.platform, multiarch=getattr(sys.implementation, '_multiarch', ''), - ) + )) _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) build_time_vars = _temp.build_time_vars global _config_vars diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -342,19 +342,13 @@ return os.path.join(get_path('stdlib'), config_dir_name, 'Makefile') -def _get_sysconfigdata_name(vars=None): - if vars is None: - return '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( - abi=sys.abiflags, - platform=sys.platform, - multiarch=getattr(sys.implementation, '_multiarch', ''), - ) - else: - return '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( - abi=vars['ABIFLAGS'], - platform=vars['MACHDEP'], - multiarch=vars.get('MULTIARCH', ''), - ) +def _get_sysconfigdata_name(): + return os.environ.get('_PYTHON_SYSCONFIGDATA_NAME', + '_sysconfigdata_{abi}_{platform}_{multiarch}'.format( + abi=sys.abiflags, + platform=sys.platform, + multiarch=getattr(sys.implementation, '_multiarch', ''), + )) def _generate_posix_vars(): @@ -397,7 +391,7 @@ # _sysconfigdata module manually and populate it with the build vars. # This is more than sufficient for ensuring the subsequent call to # get_platform() succeeds. - name = _get_sysconfigdata_name(vars) + name = _get_sysconfigdata_name() if 'darwin' in sys.platform: import types module = types.ModuleType(name) diff --git a/configure b/configure --- a/configure +++ b/configure @@ -784,7 +784,6 @@ docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir @@ -895,7 +894,6 @@ sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1148,15 +1146,6 @@ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1294,7 +1283,7 @@ for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1447,7 +1436,6 @@ --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -2940,7 +2928,7 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $interp" >&5 $as_echo "$interp" >&6; } - PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib '$interp + PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_$(ABIFLAGS)_$(MACHDEP)_$(MULTIARCH) '$interp fi # Used to comment out stuff for rebuilding generated files GENERATED_COMMENT='#' diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -78,7 +78,7 @@ AC_MSG_ERROR([python$PACKAGE_VERSION interpreter not found]) fi AC_MSG_RESULT($interp) - PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib '$interp + PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_$(ABIFLAGS)_$(MACHDEP)_$(MULTIARCH) '$interp fi # Used to comment out stuff for rebuilding generated files GENERATED_COMMENT='#' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 16:25:49 2016 From: python-checkins at python.org (ethan.furman) Date: Sun, 11 Sep 2016 20:25:49 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Enum=2E=5Fconvert=3A_sort_?= =?utf-8?q?by_value=2C_then_by_name?= Message-ID: <20160911202548.13088.86636.4E257503@psf.io> https://hg.python.org/cpython/rev/5637c9b4dd4c changeset: 103653:5637c9b4dd4c user: Ethan Furman date: Sun Sep 11 13:25:26 2016 -0700 summary: Enum._convert: sort by value, then by name files: Lib/enum.py | 13 ++++++++++--- 1 files changed, 10 insertions(+), 3 deletions(-) diff --git a/Lib/enum.py b/Lib/enum.py --- a/Lib/enum.py +++ b/Lib/enum.py @@ -616,9 +616,16 @@ # for a consistent reverse mapping of number to name when there # are multiple names for the same number rather than varying # between runs due to hash randomization of the module dictionary. - members = OrderedDict((name, source[name]) - for name in sorted(source.keys()) - if filter(name)) + members = [ + (name, source[name]) + for name in source.keys() + if filter(name)] + try: + # sort by value + members.sort(key=lambda t: (t[1], t[0])) + except TypeError: + # unless some values aren't comparable, in which case sort by name + members.sort(key=lambda t: t[0]) cls = cls(name, members, module=module) cls.__reduce_ex__ = _reduce_ex_by_name module_globals.update(cls.__members__) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 16:30:26 2016 From: python-checkins at python.org (ethan.furman) Date: Sun, 11 Sep 2016 20:30:26 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue28082=3A_use_IntFlag_?= =?utf-8?q?for_re_constants?= Message-ID: <20160911203026.69549.3161.E379B75A@psf.io> https://hg.python.org/cpython/rev/223731925d06 changeset: 103654:223731925d06 user: Ethan Furman date: Sun Sep 11 13:30:08 2016 -0700 summary: issue28082: use IntFlag for re constants files: Lib/re.py | 33 +++++++++++++++++++++------------ 1 files changed, 21 insertions(+), 12 deletions(-) diff --git a/Lib/re.py b/Lib/re.py --- a/Lib/re.py +++ b/Lib/re.py @@ -119,6 +119,7 @@ """ +import enum import sre_compile import sre_parse try: @@ -137,18 +138,26 @@ __version__ = "2.2.1" -# flags -A = ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale" -I = IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case -L = LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale -U = UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale" -M = MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline -S = DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline -X = VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments - -# sre extensions (experimental, don't rely on these) -T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking -DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation +class Flag(enum.IntFlag): + ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale" + IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case + LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale + UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale" + MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline + DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline + VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments + A = ASCII + I = IGNORECASE + L = LOCALE + U = UNICODE + M = MULTILINE + S = DOTALL + X = VERBOSE + # sre extensions (experimental, don't rely on these) + TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking + T = TEMPLATE + DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation +globals().update(Flag.__members__) # sre exception error = sre_compile.error -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 16:35:43 2016 From: python-checkins at python.org (ethan.furman) Date: Sun, 11 Sep 2016 20:35:43 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue28083=3A_add_IntFlag_?= =?utf-8?q?constants?= Message-ID: <20160911203542.2793.59124.E7119A43@psf.io> https://hg.python.org/cpython/rev/4a027e55dae3 changeset: 103655:4a027e55dae3 user: Ethan Furman date: Sun Sep 11 13:34:42 2016 -0700 summary: issue28083: add IntFlag constants files: Lib/socket.py | 12 +++++++++++- 1 files changed, 11 insertions(+), 1 deletions(-) diff --git a/Lib/socket.py b/Lib/socket.py --- a/Lib/socket.py +++ b/Lib/socket.py @@ -50,7 +50,7 @@ from _socket import * import os, sys, io, selectors -from enum import IntEnum +from enum import IntEnum, IntFlag try: import errno @@ -80,6 +80,16 @@ __name__, lambda C: C.isupper() and C.startswith('SOCK_')) +IntFlag._convert( + 'MsgFlag', + __name__, + lambda C: C.isupper() and C.startswith('MSG_')) + +IntFlag._convert( + 'AddressInfo', + __name__, + lambda C: C.isupper() and C.startswith('AI_')) + _LOCALHOST = '127.0.0.1' _LOCALHOST_V6 = '::1' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 16:47:09 2016 From: python-checkins at python.org (christian.heimes) Date: Sun, 11 Sep 2016 20:47:09 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328022=3A_Catch_an?= =?utf-8?q?other_deprecation_warning_in_imaplib?= Message-ID: <20160911204709.3168.50056.23CBC9C2@psf.io> https://hg.python.org/cpython/rev/57e88d1159fc changeset: 103656:57e88d1159fc user: Christian Heimes date: Sun Sep 11 22:47:02 2016 +0200 summary: Issue #28022: Catch another deprecation warning in imaplib files: Lib/test/test_imaplib.py | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_imaplib.py b/Lib/test/test_imaplib.py --- a/Lib/test/test_imaplib.py +++ b/Lib/test/test_imaplib.py @@ -638,8 +638,10 @@ def test_logincapa_with_client_certfile(self): with transient_internet(self.host): - _server = self.imap_class(self.host, self.port, certfile=CERTFILE) - self.check_logincapa(_server) + with support.check_warnings(('', DeprecationWarning)): + _server = self.imap_class(self.host, self.port, + certfile=CERTFILE) + self.check_logincapa(_server) def test_logincapa_with_client_ssl_context(self): with transient_internet(self.host): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 17:23:48 2016 From: python-checkins at python.org (r.david.murray) Date: Sun, 11 Sep 2016 21:23:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Merge=3A_=2319003=3A_Only_?= =?utf-8?q?replace_=5Cr_and/or_=5Cn_line_endings_in_email=2Egenerator=2E?= Message-ID: <20160911212345.87425.93620.3E79DA4E@psf.io> https://hg.python.org/cpython/rev/ccad4d142934 changeset: 103658:ccad4d142934 parent: 103656:57e88d1159fc user: R David Murray date: Sun Sep 11 17:23:33 2016 -0400 summary: Merge: #19003: Only replace \r and/or \n line endings in email.generator. files: Lib/email/generator.py | 16 ++++++++++------ Lib/test/test_email/test_email.py | 12 ++++++++++++ 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/Lib/email/generator.py b/Lib/email/generator.py --- a/Lib/email/generator.py +++ b/Lib/email/generator.py @@ -18,6 +18,7 @@ UNDERSCORE = '_' NL = '\n' # XXX: no longer used by the code below. +NLCRE = re.compile(r'\r\n|\r|\n') fcre = re.compile(r'^From ', re.MULTILINE) @@ -149,14 +150,17 @@ # We have to transform the line endings. if not lines: return - lines = lines.splitlines(True) + lines = NLCRE.split(lines) for line in lines[:-1]: - self.write(line.rstrip('\r\n')) + self.write(line) self.write(self._NL) - laststripped = lines[-1].rstrip('\r\n') - self.write(laststripped) - if len(lines[-1]) != len(laststripped): - self.write(self._NL) + if lines[-1]: + self.write(lines[-1]) + # XXX logic tells me this else should be needed, but the tests fail + # with it and pass without it. (NLCRE.split ends with a blank element + # if and only if there was a trailing newline.) + #else: + # self.write(self._NL) def _write(self, msg): # We can't write the headers yet because of the following scenario: diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -1599,6 +1599,18 @@ self.assertEqual(msg.get_payload(), '\uFFFD' * len(bytesdata)) self.assertEqual(msg2.get_payload(decode=True), bytesdata) + def test_binary_body_with_unicode_linend_encode_noop(self): + # Issue 19003: This is a variation on #16564. + bytesdata = b'\x0b\xfa\xfb\xfc\xfd\xfe\xff' + msg = MIMEApplication(bytesdata, _encoder=encoders.encode_noop) + self.assertEqual(msg.get_payload(decode=True), bytesdata) + s = BytesIO() + g = BytesGenerator(s) + g.flatten(msg) + wireform = s.getvalue() + msg2 = email.message_from_bytes(wireform) + self.assertEqual(msg2.get_payload(decode=True), bytesdata) + def test_binary_body_with_encode_quopri(self): # Issue 14360. bytesdata = b'\xfa\xfb\xfc\xfd\xfe\xff ' -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 17:23:48 2016 From: python-checkins at python.org (r.david.murray) Date: Sun, 11 Sep 2016 21:23:48 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogIzE5MDAzOiBPbmx5?= =?utf-8?q?_replace_=5Cr_and/or_=5Cn_line_endings_in_email=2Egenerator=2E?= Message-ID: <20160911212345.36352.25054.0B705934@psf.io> https://hg.python.org/cpython/rev/c0f5702e0f10 changeset: 103657:c0f5702e0f10 branch: 3.5 parent: 103639:f1427491a8a5 user: R David Murray date: Sun Sep 11 17:22:56 2016 -0400 summary: #19003: Only replace \r and/or \n line endings in email.generator. This is a further restoration of backward compatibility, as well as being correct per the RFCs. files: Lib/email/generator.py | 16 ++++++++++------ Lib/test/test_email/test_email.py | 12 ++++++++++++ Misc/NEWS | 4 ++++ 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/Lib/email/generator.py b/Lib/email/generator.py --- a/Lib/email/generator.py +++ b/Lib/email/generator.py @@ -18,6 +18,7 @@ UNDERSCORE = '_' NL = '\n' # XXX: no longer used by the code below. +NLCRE = re.compile(r'\r\n|\r|\n') fcre = re.compile(r'^From ', re.MULTILINE) @@ -149,14 +150,17 @@ # We have to transform the line endings. if not lines: return - lines = lines.splitlines(True) + lines = NLCRE.split(lines) for line in lines[:-1]: - self.write(line.rstrip('\r\n')) + self.write(line) self.write(self._NL) - laststripped = lines[-1].rstrip('\r\n') - self.write(laststripped) - if len(lines[-1]) != len(laststripped): - self.write(self._NL) + if lines[-1]: + self.write(lines[-1]) + # XXX logic tells me this else should be needed, but the tests fail + # with it and pass without it. (NLCRE.split ends with a blank element + # if and only if there was a trailing newline.) + #else: + # self.write(self._NL) def _write(self, msg): # We can't write the headers yet because of the following scenario: diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -1598,6 +1598,18 @@ self.assertEqual(msg.get_payload(), '\uFFFD' * len(bytesdata)) self.assertEqual(msg2.get_payload(decode=True), bytesdata) + def test_binary_body_with_unicode_linend_encode_noop(self): + # Issue 19003: This is a variation on #16564. + bytesdata = b'\x0b\xfa\xfb\xfc\xfd\xfe\xff' + msg = MIMEApplication(bytesdata, _encoder=encoders.encode_noop) + self.assertEqual(msg.get_payload(decode=True), bytesdata) + s = BytesIO() + g = BytesGenerator(s) + g.flatten(msg) + wireform = s.getvalue() + msg2 = email.message_from_bytes(wireform) + self.assertEqual(msg2.get_payload(decode=True), bytesdata) + def test_binary_body_with_encode_quopri(self): # Issue 14360. bytesdata = b'\xfa\xfb\xfc\xfd\xfe\xff ' diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -68,6 +68,10 @@ Library ------- + +- Issue #19003:m email.generator now replaces only \r and/or \n line + endings, per the RFC, instead of all unicode line endings. + - Issue #28019: itertools.count() no longer rounds non-integer step in range between 1.0 and 2.0 to 1. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 17:53:23 2016 From: python-checkins at python.org (serhiy.storchaka) Date: Sun, 11 Sep 2016 21:53:23 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2327213=3A_Fixed_di?= =?utf-8?q?fferent_issues_with_reworked_CALL=5FFUNCTION*_opcodes=2E?= Message-ID: <20160911215322.12514.90232.13FFA580@psf.io> https://hg.python.org/cpython/rev/51b635e81958 changeset: 103659:51b635e81958 user: Serhiy Storchaka date: Mon Sep 12 00:52:40 2016 +0300 summary: Issue #27213: Fixed different issues with reworked CALL_FUNCTION* opcodes. * BUILD_TUPLE_UNPACK and BUILD_MAP_UNPACK_WITH_CALL no longer generated with single tuple or dict. * Restored more informative error messages for incorrect var-positional and var-keyword arguments. * Removed code duplications in _PyEval_EvalCodeWithName(). * Removed redundant runtime checks and parameters in _PyStack_AsDict(). * Added a workaround and enabled previously disabled test in test_traceback. * Removed dead code from the dis module. files: Include/abstract.h | 4 +- Lib/dis.py | 2 - Lib/test/test_extcall.py | 18 +- Lib/test/test_traceback.py | 3 +- Objects/abstract.c | 26 +- Objects/methodobject.c | 2 +- Python/ceval.c | 141 +- Python/compile.c | 9 +- Python/importlib.h | 2894 ++++++++-------- Python/importlib_external.h | 3818 +++++++++++----------- 10 files changed, 3437 insertions(+), 3480 deletions(-) diff --git a/Include/abstract.h b/Include/abstract.h --- a/Include/abstract.h +++ b/Include/abstract.h @@ -275,9 +275,7 @@ PyAPI_FUNC(PyObject *) _PyStack_AsDict( PyObject **values, - Py_ssize_t nkwargs, - PyObject *kwnames, - PyObject *func); + PyObject *kwnames); /* Convert (args, nargs, kwargs) into a (stack, nargs, kwnames). diff --git a/Lib/dis.py b/Lib/dis.py --- a/Lib/dis.py +++ b/Lib/dis.py @@ -314,8 +314,6 @@ argrepr = argval elif op in hasfree: argval, argrepr = _get_name_info(arg, cells) - elif op in hasnargs: # unused - argrepr = "%d positional, %d keyword pair" % (arg%256, arg//256) yield Instruction(opname[op], op, arg, argval, argrepr, offset, starts_line, is_jump_target) diff --git a/Lib/test/test_extcall.py b/Lib/test/test_extcall.py --- a/Lib/test/test_extcall.py +++ b/Lib/test/test_extcall.py @@ -118,7 +118,7 @@ >>> g(*Nothing()) Traceback (most recent call last): ... - TypeError: 'Nothing' object is not iterable + TypeError: g() argument after * must be an iterable, not Nothing >>> class Nothing: ... def __len__(self): return 5 @@ -127,7 +127,7 @@ >>> g(*Nothing()) Traceback (most recent call last): ... - TypeError: 'Nothing' object is not iterable + TypeError: g() argument after * must be an iterable, not Nothing >>> class Nothing(): ... def __len__(self): return 5 @@ -231,32 +231,34 @@ >>> h(*h) Traceback (most recent call last): ... - TypeError: 'function' object is not iterable + TypeError: h() argument after * must be an iterable, not function >>> dir(*h) Traceback (most recent call last): ... - TypeError: 'function' object is not iterable + TypeError: dir() argument after * must be an iterable, not function >>> None(*h) Traceback (most recent call last): ... - TypeError: 'function' object is not iterable + TypeError: NoneType object argument after * must be an iterable, \ +not function >>> h(**h) Traceback (most recent call last): ... - TypeError: 'function' object is not a mapping + TypeError: h() argument after ** must be a mapping, not function >>> dir(**h) Traceback (most recent call last): ... - TypeError: 'function' object is not a mapping + TypeError: dir() argument after ** must be a mapping, not function >>> None(**h) Traceback (most recent call last): ... - TypeError: 'function' object is not a mapping + TypeError: NoneType object argument after ** must be a mapping, \ +not function >>> dir(b=1, **{'b': 1}) Traceback (most recent call last): diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -304,7 +304,6 @@ ]) # issue 26823 - Shrink recursive tracebacks - @unittest.skipIf(True, "FIXME: test broken, see issue #28050") def _check_recursive_traceback_display(self, render_exc): # Always show full diffs when this test fails # Note that rearranging things may require adjusting @@ -353,7 +352,7 @@ # Check the recursion count is roughly as expected rec_limit = sys.getrecursionlimit() - self.assertIn(int(re.search(r"\d+", actual[-2]).group()), range(rec_limit-50, rec_limit)) + self.assertIn(int(re.search(r"\d+", actual[-2]).group()), range(rec_limit-60, rec_limit)) # Check a known (limited) number of recursive invocations def g(count=10): diff --git a/Objects/abstract.c b/Objects/abstract.c --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2367,9 +2367,9 @@ } PyObject * -_PyStack_AsDict(PyObject **values, Py_ssize_t nkwargs, PyObject *kwnames, - PyObject *func) +_PyStack_AsDict(PyObject **values, PyObject *kwnames) { + Py_ssize_t nkwargs = PyTuple_GET_SIZE(kwnames); PyObject *kwdict; Py_ssize_t i; @@ -2378,24 +2378,12 @@ return NULL; } - for (i=0; i < nkwargs; i++) { - int err; + for (i = 0; i < nkwargs; i++) { PyObject *key = PyTuple_GET_ITEM(kwnames, i); PyObject *value = *values++; - - if (PyDict_GetItem(kwdict, key) != NULL) { - PyErr_Format(PyExc_TypeError, - "%.200s%s got multiple values " - "for keyword argument '%U'", - PyEval_GetFuncName(func), - PyEval_GetFuncDesc(func), - key); - Py_DECREF(kwdict); - return NULL; - } - - err = PyDict_SetItem(kwdict, key, value); - if (err) { + assert(PyUnicode_CheckExact(key)); + assert(PyDict_GetItem(kwdict, key) == NULL); + if (PyDict_SetItem(kwdict, key, value)) { Py_DECREF(kwdict); return NULL; } @@ -2479,7 +2467,7 @@ } if (nkwargs > 0) { - kwdict = _PyStack_AsDict(stack + nargs, nkwargs, kwnames, func); + kwdict = _PyStack_AsDict(stack + nargs, kwnames); if (kwdict == NULL) { return NULL; } diff --git a/Objects/methodobject.c b/Objects/methodobject.c --- a/Objects/methodobject.c +++ b/Objects/methodobject.c @@ -279,7 +279,7 @@ nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames); if (nkwargs > 0) { - kwdict = _PyStack_AsDict(stack + nargs, nkwargs, kwnames, func); + kwdict = _PyStack_AsDict(stack + nargs, kwnames); if (kwdict == NULL) { return NULL; } diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2513,14 +2513,9 @@ TARGET(BUILD_LIST_UNPACK) { int convert_to_tuple = opcode == BUILD_TUPLE_UNPACK; Py_ssize_t i; - PyObject *sum; + PyObject *sum = PyList_New(0); PyObject *return_value; - if (convert_to_tuple && oparg == 1 && PyTuple_CheckExact(TOP())) { - DISPATCH(); - } - - sum = PyList_New(0); if (sum == NULL) goto error; @@ -2708,13 +2703,7 @@ TARGET(BUILD_MAP_UNPACK) { int with_call = opcode == BUILD_MAP_UNPACK_WITH_CALL; Py_ssize_t i; - PyObject *sum; - - if (with_call && oparg == 1 && PyDict_CheckExact(TOP())) { - DISPATCH(); - } - - sum = PyDict_New(); + PyObject *sum = PyDict_New(); if (sum == NULL) goto error; @@ -3290,11 +3279,53 @@ PCALL(PCALL_ALL); if (oparg & 0x01) { kwargs = POP(); + if (!PyDict_CheckExact(kwargs)) { + PyObject *d = PyDict_New(); + if (d == NULL) + goto error; + if (PyDict_Update(d, kwargs) != 0) { + Py_DECREF(d); + /* PyDict_Update raises attribute + * error (percolated from an attempt + * to get 'keys' attribute) instead of + * a type error if its second argument + * is not a mapping. + */ + if (PyErr_ExceptionMatches(PyExc_AttributeError)) { + func = SECOND(); + PyErr_Format(PyExc_TypeError, + "%.200s%.200s argument after ** " + "must be a mapping, not %.200s", + PyEval_GetFuncName(func), + PyEval_GetFuncDesc(func), + kwargs->ob_type->tp_name); + } + goto error; + } + Py_DECREF(kwargs); + kwargs = d; + } assert(PyDict_CheckExact(kwargs)); } callargs = POP(); - assert(PyTuple_CheckExact(callargs)); func = TOP(); + if (!PyTuple_Check(callargs)) { + if (Py_TYPE(callargs)->tp_iter == NULL && + !PySequence_Check(callargs)) { + PyErr_Format(PyExc_TypeError, + "%.200s%.200s argument after * " + "must be an iterable, not %.200s", + PyEval_GetFuncName(func), + PyEval_GetFuncDesc(func), + callargs->ob_type->tp_name); + goto error; + } + Py_SETREF(callargs, PySequence_Tuple(callargs)); + if (callargs == NULL) { + goto error; + } + } + assert(PyTuple_Check(callargs)); result = do_call_core(func, callargs, kwargs); Py_DECREF(func); @@ -3796,8 +3827,8 @@ static PyObject * _PyEval_EvalCodeWithName(PyObject *_co, PyObject *globals, PyObject *locals, PyObject **args, Py_ssize_t argcount, - PyObject **kws, Py_ssize_t kwcount, - PyObject *kwnames, PyObject **kwstack, + PyObject **kwnames, PyObject **kwargs, + Py_ssize_t kwcount, int kwstep, PyObject **defs, Py_ssize_t defcount, PyObject *kwdefs, PyObject *closure, PyObject *name, PyObject *qualname) @@ -3811,9 +3842,6 @@ const Py_ssize_t total_args = co->co_argcount + co->co_kwonlyargcount; Py_ssize_t i, n; PyObject *kwdict; - Py_ssize_t kwcount2 = kwnames == NULL ? 0 : PyTuple_GET_SIZE(kwnames); - - assert((kwcount == 0) || (kws != NULL)); if (globals == NULL) { PyErr_SetString(PyExc_SystemError, @@ -3873,11 +3901,12 @@ } } - /* Handle keyword arguments passed as an array of (key, value) pairs */ - for (i = 0; i < kwcount; i++) { + /* Handle keyword arguments passed as two strided arrays */ + kwcount *= kwstep; + for (i = 0; i < kwcount; i += kwstep) { PyObject **co_varnames; - PyObject *keyword = kws[2*i]; - PyObject *value = kws[2*i + 1]; + PyObject *keyword = kwnames[i]; + PyObject *value = kwargs[i]; Py_ssize_t j; if (keyword == NULL || !PyUnicode_Check(keyword)) { @@ -3932,61 +3961,6 @@ SETLOCAL(j, value); } - /* Handle keyword arguments passed as keys tuple + values array */ - for (i = 0; i < kwcount2; i++) { - PyObject **co_varnames; - PyObject *keyword = PyTuple_GET_ITEM(kwnames, i); - PyObject *value = kwstack[i]; - int j; - if (keyword == NULL || !PyUnicode_Check(keyword)) { - PyErr_Format(PyExc_TypeError, - "%U() keywords must be strings", - co->co_name); - goto fail; - } - /* Speed hack: do raw pointer compares. As names are - normally interned this should almost always hit. */ - co_varnames = ((PyTupleObject *)(co->co_varnames))->ob_item; - for (j = 0; j < total_args; j++) { - PyObject *nm = co_varnames[j]; - if (nm == keyword) - goto kw_found2; - } - /* Slow fallback, just in case */ - for (j = 0; j < total_args; j++) { - PyObject *nm = co_varnames[j]; - int cmp = PyObject_RichCompareBool( - keyword, nm, Py_EQ); - if (cmp > 0) - goto kw_found2; - else if (cmp < 0) - goto fail; - } - if (j >= total_args && kwdict == NULL) { - PyErr_Format(PyExc_TypeError, - "%U() got an unexpected " - "keyword argument '%S'", - co->co_name, - keyword); - goto fail; - } - if (PyDict_SetItem(kwdict, keyword, value) == -1) { - goto fail; - } - continue; - kw_found2: - if (GETLOCAL(j) != NULL) { - PyErr_Format(PyExc_TypeError, - "%U() got multiple " - "values for argument '%S'", - co->co_name, - keyword); - goto fail; - } - Py_INCREF(value); - SETLOCAL(j, value); - } - /* Check the number of positional arguments */ if (argcount > co->co_argcount && !(co->co_flags & CO_VARARGS)) { too_many_positional(co, argcount, defcount, fastlocals); @@ -4138,8 +4112,7 @@ { return _PyEval_EvalCodeWithName(_co, globals, locals, args, argcount, - kws, kwcount, - NULL, NULL, + kws, kws + 1, kwcount, 2, defs, defcount, kwdefs, closure, NULL, NULL); @@ -4923,8 +4896,9 @@ } return _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, stack, nargs, - NULL, 0, - kwnames, stack + nargs, + nkwargs ? &PyTuple_GET_ITEM(kwnames, 0) : NULL, + stack + nargs, + nkwargs, 1, d, (int)nd, kwdefs, closure, name, qualname); } @@ -5014,8 +4988,7 @@ result = _PyEval_EvalCodeWithName((PyObject*)co, globals, (PyObject *)NULL, args, nargs, - k, nk, - NULL, NULL, + k, k + 1, nk, 2, d, nd, kwdefs, closure, name, qualname); Py_XDECREF(kwtuple); diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -3503,7 +3503,7 @@ asdl_seq *keywords) { Py_ssize_t i, nseen, nelts, nkwelts; - int musttupleunpack = 0, mustdictunpack = 0; + int mustdictunpack = 0; /* the number of tuples and dictionaries on the stack */ Py_ssize_t nsubargs = 0, nsubkwargs = 0; @@ -3532,7 +3532,6 @@ } VISIT(c, expr, elt->v.Starred.value); nsubargs++; - musttupleunpack = 1; } else { VISIT(c, expr, elt); @@ -3541,13 +3540,13 @@ } /* Same dance again for keyword arguments */ - if (musttupleunpack || mustdictunpack) { + if (nsubargs || mustdictunpack) { if (nseen) { /* Pack up any trailing positional arguments. */ ADDOP_I(c, BUILD_TUPLE, nseen); nsubargs++; } - if (musttupleunpack || nsubargs > 1) { + if (nsubargs > 1) { /* If we ended up with more than one stararg, we need to concatenate them into a single sequence. */ ADDOP_I(c, BUILD_TUPLE_UNPACK, nsubargs); @@ -3579,7 +3578,7 @@ return 0; nsubkwargs++; } - if (mustdictunpack || nsubkwargs > 1) { + if (nsubkwargs > 1) { /* Pack it all up */ ADDOP_I(c, BUILD_MAP_UNPACK_WITH_CALL, nsubkwargs); } diff --git a/Python/importlib.h b/Python/importlib.h --- a/Python/importlib.h +++ b/Python/importlib.h [stripped] diff --git a/Python/importlib_external.h b/Python/importlib_external.h --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -508,555 +508,578 @@ 97,105,108,115,32,116,104,101,110,32,73,109,112,111,114,116, 69,114,114,111,114,32,105,115,32,114,97,105,115,101,100,46, 10,10,32,32,32,32,78,99,2,0,0,0,0,0,0,0, - 4,0,0,0,4,0,0,0,31,0,0,0,115,68,0,0, + 4,0,0,0,4,0,0,0,31,0,0,0,115,66,0,0, 0,124,1,100,0,107,8,114,16,124,0,106,0,125,1,110, 32,124,0,106,0,124,1,107,3,114,48,116,1,100,1,124, 0,106,0,124,1,102,2,22,0,124,1,100,2,141,2,130, - 1,136,0,124,0,124,1,102,2,124,2,152,2,124,3,151, - 1,142,1,83,0,41,3,78,122,30,108,111,97,100,101,114, - 32,102,111,114,32,37,115,32,99,97,110,110,111,116,32,104, - 97,110,100,108,101,32,37,115,41,1,218,4,110,97,109,101, - 41,2,114,102,0,0,0,218,11,73,109,112,111,114,116,69, - 114,114,111,114,41,4,218,4,115,101,108,102,114,102,0,0, - 0,218,4,97,114,103,115,90,6,107,119,97,114,103,115,41, - 1,218,6,109,101,116,104,111,100,114,4,0,0,0,114,6, - 0,0,0,218,19,95,99,104,101,99,107,95,110,97,109,101, - 95,119,114,97,112,112,101,114,135,1,0,0,115,12,0,0, - 0,0,1,8,1,8,1,10,1,4,1,18,1,122,40,95, - 99,104,101,99,107,95,110,97,109,101,46,60,108,111,99,97, - 108,115,62,46,95,99,104,101,99,107,95,110,97,109,101,95, - 119,114,97,112,112,101,114,99,2,0,0,0,0,0,0,0, - 3,0,0,0,7,0,0,0,83,0,0,0,115,60,0,0, - 0,120,40,100,5,68,0,93,32,125,2,116,0,124,1,124, - 2,131,2,114,6,116,1,124,0,124,2,116,2,124,1,124, - 2,131,2,131,3,1,0,113,6,87,0,124,0,106,3,106, - 4,124,1,106,3,131,1,1,0,100,0,83,0,41,6,78, - 218,10,95,95,109,111,100,117,108,101,95,95,218,8,95,95, - 110,97,109,101,95,95,218,12,95,95,113,117,97,108,110,97, - 109,101,95,95,218,7,95,95,100,111,99,95,95,41,4,122, - 10,95,95,109,111,100,117,108,101,95,95,122,8,95,95,110, - 97,109,101,95,95,122,12,95,95,113,117,97,108,110,97,109, - 101,95,95,122,7,95,95,100,111,99,95,95,41,5,218,7, - 104,97,115,97,116,116,114,218,7,115,101,116,97,116,116,114, - 218,7,103,101,116,97,116,116,114,218,8,95,95,100,105,99, - 116,95,95,218,6,117,112,100,97,116,101,41,3,90,3,110, - 101,119,90,3,111,108,100,114,55,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,5,95,119,114, - 97,112,146,1,0,0,115,8,0,0,0,0,1,10,1,10, - 1,22,1,122,26,95,99,104,101,99,107,95,110,97,109,101, - 46,60,108,111,99,97,108,115,62,46,95,119,114,97,112,41, - 1,78,41,3,218,10,95,98,111,111,116,115,116,114,97,112, - 114,117,0,0,0,218,9,78,97,109,101,69,114,114,111,114, - 41,3,114,106,0,0,0,114,107,0,0,0,114,117,0,0, - 0,114,4,0,0,0,41,1,114,106,0,0,0,114,6,0, - 0,0,218,11,95,99,104,101,99,107,95,110,97,109,101,127, - 1,0,0,115,14,0,0,0,0,8,14,7,2,1,10,1, - 14,2,14,5,10,1,114,120,0,0,0,99,2,0,0,0, + 1,136,0,124,0,124,1,102,2,124,2,152,2,124,3,142, + 1,83,0,41,3,78,122,30,108,111,97,100,101,114,32,102, + 111,114,32,37,115,32,99,97,110,110,111,116,32,104,97,110, + 100,108,101,32,37,115,41,1,218,4,110,97,109,101,41,2, + 114,102,0,0,0,218,11,73,109,112,111,114,116,69,114,114, + 111,114,41,4,218,4,115,101,108,102,114,102,0,0,0,218, + 4,97,114,103,115,90,6,107,119,97,114,103,115,41,1,218, + 6,109,101,116,104,111,100,114,4,0,0,0,114,6,0,0, + 0,218,19,95,99,104,101,99,107,95,110,97,109,101,95,119, + 114,97,112,112,101,114,135,1,0,0,115,12,0,0,0,0, + 1,8,1,8,1,10,1,4,1,18,1,122,40,95,99,104, + 101,99,107,95,110,97,109,101,46,60,108,111,99,97,108,115, + 62,46,95,99,104,101,99,107,95,110,97,109,101,95,119,114, + 97,112,112,101,114,99,2,0,0,0,0,0,0,0,3,0, + 0,0,7,0,0,0,83,0,0,0,115,60,0,0,0,120, + 40,100,5,68,0,93,32,125,2,116,0,124,1,124,2,131, + 2,114,6,116,1,124,0,124,2,116,2,124,1,124,2,131, + 2,131,3,1,0,113,6,87,0,124,0,106,3,106,4,124, + 1,106,3,131,1,1,0,100,0,83,0,41,6,78,218,10, + 95,95,109,111,100,117,108,101,95,95,218,8,95,95,110,97, + 109,101,95,95,218,12,95,95,113,117,97,108,110,97,109,101, + 95,95,218,7,95,95,100,111,99,95,95,41,4,122,10,95, + 95,109,111,100,117,108,101,95,95,122,8,95,95,110,97,109, + 101,95,95,122,12,95,95,113,117,97,108,110,97,109,101,95, + 95,122,7,95,95,100,111,99,95,95,41,5,218,7,104,97, + 115,97,116,116,114,218,7,115,101,116,97,116,116,114,218,7, + 103,101,116,97,116,116,114,218,8,95,95,100,105,99,116,95, + 95,218,6,117,112,100,97,116,101,41,3,90,3,110,101,119, + 90,3,111,108,100,114,55,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,218,5,95,119,114,97,112, + 146,1,0,0,115,8,0,0,0,0,1,10,1,10,1,22, + 1,122,26,95,99,104,101,99,107,95,110,97,109,101,46,60, + 108,111,99,97,108,115,62,46,95,119,114,97,112,41,1,78, + 41,3,218,10,95,98,111,111,116,115,116,114,97,112,114,117, + 0,0,0,218,9,78,97,109,101,69,114,114,111,114,41,3, + 114,106,0,0,0,114,107,0,0,0,114,117,0,0,0,114, + 4,0,0,0,41,1,114,106,0,0,0,114,6,0,0,0, + 218,11,95,99,104,101,99,107,95,110,97,109,101,127,1,0, + 0,115,14,0,0,0,0,8,14,7,2,1,10,1,14,2, + 14,5,10,1,114,120,0,0,0,99,2,0,0,0,0,0, + 0,0,5,0,0,0,4,0,0,0,67,0,0,0,115,60, + 0,0,0,124,0,106,0,124,1,131,1,92,2,125,2,125, + 3,124,2,100,1,107,8,114,56,116,1,124,3,131,1,114, + 56,100,2,125,4,116,2,106,3,124,4,106,4,124,3,100, + 3,25,0,131,1,116,5,131,2,1,0,124,2,83,0,41, + 4,122,155,84,114,121,32,116,111,32,102,105,110,100,32,97, + 32,108,111,97,100,101,114,32,102,111,114,32,116,104,101,32, + 115,112,101,99,105,102,105,101,100,32,109,111,100,117,108,101, + 32,98,121,32,100,101,108,101,103,97,116,105,110,103,32,116, + 111,10,32,32,32,32,115,101,108,102,46,102,105,110,100,95, + 108,111,97,100,101,114,40,41,46,10,10,32,32,32,32,84, + 104,105,115,32,109,101,116,104,111,100,32,105,115,32,100,101, + 112,114,101,99,97,116,101,100,32,105,110,32,102,97,118,111, + 114,32,111,102,32,102,105,110,100,101,114,46,102,105,110,100, + 95,115,112,101,99,40,41,46,10,10,32,32,32,32,78,122, + 44,78,111,116,32,105,109,112,111,114,116,105,110,103,32,100, + 105,114,101,99,116,111,114,121,32,123,125,58,32,109,105,115, + 115,105,110,103,32,95,95,105,110,105,116,95,95,114,62,0, + 0,0,41,6,218,11,102,105,110,100,95,108,111,97,100,101, + 114,114,33,0,0,0,114,63,0,0,0,114,64,0,0,0, + 114,50,0,0,0,218,13,73,109,112,111,114,116,87,97,114, + 110,105,110,103,41,5,114,104,0,0,0,218,8,102,117,108, + 108,110,97,109,101,218,6,108,111,97,100,101,114,218,8,112, + 111,114,116,105,111,110,115,218,3,109,115,103,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,17,95,102,105, + 110,100,95,109,111,100,117,108,101,95,115,104,105,109,155,1, + 0,0,115,10,0,0,0,0,10,14,1,16,1,4,1,22, + 1,114,127,0,0,0,99,4,0,0,0,0,0,0,0,11, + 0,0,0,22,0,0,0,67,0,0,0,115,136,1,0,0, + 105,0,125,4,124,2,100,1,107,9,114,22,124,2,124,4, + 100,2,60,0,110,4,100,3,125,2,124,3,100,1,107,9, + 114,42,124,3,124,4,100,4,60,0,124,0,100,1,100,5, + 133,2,25,0,125,5,124,0,100,5,100,6,133,2,25,0, + 125,6,124,0,100,6,100,7,133,2,25,0,125,7,124,5, + 116,0,107,3,114,124,100,8,106,1,124,2,124,5,131,2, + 125,8,116,2,106,3,100,9,124,8,131,2,1,0,116,4, + 124,8,102,1,124,4,142,1,130,1,110,86,116,5,124,6, + 131,1,100,5,107,3,114,168,100,10,106,1,124,2,131,1, + 125,8,116,2,106,3,100,9,124,8,131,2,1,0,116,6, + 124,8,131,1,130,1,110,42,116,5,124,7,131,1,100,5, + 107,3,114,210,100,11,106,1,124,2,131,1,125,8,116,2, + 106,3,100,9,124,8,131,2,1,0,116,6,124,8,131,1, + 130,1,124,1,100,1,107,9,144,1,114,124,121,16,116,7, + 124,1,100,12,25,0,131,1,125,9,87,0,110,22,4,0, + 116,8,107,10,144,1,114,2,1,0,1,0,1,0,89,0, + 110,50,88,0,116,9,124,6,131,1,124,9,107,3,144,1, + 114,52,100,13,106,1,124,2,131,1,125,8,116,2,106,3, + 100,9,124,8,131,2,1,0,116,4,124,8,102,1,124,4, + 142,1,130,1,121,16,124,1,100,14,25,0,100,15,64,0, + 125,10,87,0,110,22,4,0,116,8,107,10,144,1,114,90, + 1,0,1,0,1,0,89,0,110,34,88,0,116,9,124,7, + 131,1,124,10,107,3,144,1,114,124,116,4,100,13,106,1, + 124,2,131,1,102,1,124,4,142,1,130,1,124,0,100,7, + 100,1,133,2,25,0,83,0,41,16,97,122,1,0,0,86, + 97,108,105,100,97,116,101,32,116,104,101,32,104,101,97,100, + 101,114,32,111,102,32,116,104,101,32,112,97,115,115,101,100, + 45,105,110,32,98,121,116,101,99,111,100,101,32,97,103,97, + 105,110,115,116,32,115,111,117,114,99,101,95,115,116,97,116, + 115,32,40,105,102,10,32,32,32,32,103,105,118,101,110,41, + 32,97,110,100,32,114,101,116,117,114,110,105,110,103,32,116, + 104,101,32,98,121,116,101,99,111,100,101,32,116,104,97,116, + 32,99,97,110,32,98,101,32,99,111,109,112,105,108,101,100, + 32,98,121,32,99,111,109,112,105,108,101,40,41,46,10,10, + 32,32,32,32,65,108,108,32,111,116,104,101,114,32,97,114, + 103,117,109,101,110,116,115,32,97,114,101,32,117,115,101,100, + 32,116,111,32,101,110,104,97,110,99,101,32,101,114,114,111, + 114,32,114,101,112,111,114,116,105,110,103,46,10,10,32,32, + 32,32,73,109,112,111,114,116,69,114,114,111,114,32,105,115, + 32,114,97,105,115,101,100,32,119,104,101,110,32,116,104,101, + 32,109,97,103,105,99,32,110,117,109,98,101,114,32,105,115, + 32,105,110,99,111,114,114,101,99,116,32,111,114,32,116,104, + 101,32,98,121,116,101,99,111,100,101,32,105,115,10,32,32, + 32,32,102,111,117,110,100,32,116,111,32,98,101,32,115,116, + 97,108,101,46,32,69,79,70,69,114,114,111,114,32,105,115, + 32,114,97,105,115,101,100,32,119,104,101,110,32,116,104,101, + 32,100,97,116,97,32,105,115,32,102,111,117,110,100,32,116, + 111,32,98,101,10,32,32,32,32,116,114,117,110,99,97,116, + 101,100,46,10,10,32,32,32,32,78,114,102,0,0,0,122, + 10,60,98,121,116,101,99,111,100,101,62,114,37,0,0,0, + 114,14,0,0,0,233,8,0,0,0,233,12,0,0,0,122, + 30,98,97,100,32,109,97,103,105,99,32,110,117,109,98,101, + 114,32,105,110,32,123,33,114,125,58,32,123,33,114,125,122, + 2,123,125,122,43,114,101,97,99,104,101,100,32,69,79,70, + 32,119,104,105,108,101,32,114,101,97,100,105,110,103,32,116, + 105,109,101,115,116,97,109,112,32,105,110,32,123,33,114,125, + 122,48,114,101,97,99,104,101,100,32,69,79,70,32,119,104, + 105,108,101,32,114,101,97,100,105,110,103,32,115,105,122,101, + 32,111,102,32,115,111,117,114,99,101,32,105,110,32,123,33, + 114,125,218,5,109,116,105,109,101,122,26,98,121,116,101,99, + 111,100,101,32,105,115,32,115,116,97,108,101,32,102,111,114, + 32,123,33,114,125,218,4,115,105,122,101,108,3,0,0,0, + 255,127,255,127,3,0,41,10,218,12,77,65,71,73,67,95, + 78,85,77,66,69,82,114,50,0,0,0,114,118,0,0,0, + 218,16,95,118,101,114,98,111,115,101,95,109,101,115,115,97, + 103,101,114,103,0,0,0,114,33,0,0,0,218,8,69,79, + 70,69,114,114,111,114,114,16,0,0,0,218,8,75,101,121, + 69,114,114,111,114,114,21,0,0,0,41,11,114,56,0,0, + 0,218,12,115,111,117,114,99,101,95,115,116,97,116,115,114, + 102,0,0,0,114,37,0,0,0,90,11,101,120,99,95,100, + 101,116,97,105,108,115,90,5,109,97,103,105,99,90,13,114, + 97,119,95,116,105,109,101,115,116,97,109,112,90,8,114,97, + 119,95,115,105,122,101,114,79,0,0,0,218,12,115,111,117, + 114,99,101,95,109,116,105,109,101,218,11,115,111,117,114,99, + 101,95,115,105,122,101,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,218,25,95,118,97,108,105,100,97,116,101, + 95,98,121,116,101,99,111,100,101,95,104,101,97,100,101,114, + 172,1,0,0,115,76,0,0,0,0,11,4,1,8,1,10, + 3,4,1,8,1,8,1,12,1,12,1,12,1,8,1,12, + 1,12,1,14,1,12,1,10,1,12,1,10,1,12,1,10, + 1,12,1,8,1,10,1,2,1,16,1,16,1,6,2,14, + 1,10,1,12,1,12,1,2,1,16,1,16,1,6,2,14, + 1,12,1,6,1,114,139,0,0,0,99,4,0,0,0,0, + 0,0,0,5,0,0,0,5,0,0,0,67,0,0,0,115, + 82,0,0,0,116,0,106,1,124,0,131,1,125,4,116,2, + 124,4,116,3,131,2,114,58,116,4,106,5,100,1,124,2, + 131,2,1,0,124,3,100,2,107,9,114,52,116,6,106,7, + 124,4,124,3,131,2,1,0,124,4,83,0,110,20,116,8, + 100,3,106,9,124,2,131,1,124,1,124,2,100,4,141,3, + 130,1,100,2,83,0,41,5,122,60,67,111,109,112,105,108, + 101,32,98,121,116,101,99,111,100,101,32,97,115,32,114,101, + 116,117,114,110,101,100,32,98,121,32,95,118,97,108,105,100, + 97,116,101,95,98,121,116,101,99,111,100,101,95,104,101,97, + 100,101,114,40,41,46,122,21,99,111,100,101,32,111,98,106, + 101,99,116,32,102,114,111,109,32,123,33,114,125,78,122,23, + 78,111,110,45,99,111,100,101,32,111,98,106,101,99,116,32, + 105,110,32,123,33,114,125,41,2,114,102,0,0,0,114,37, + 0,0,0,41,10,218,7,109,97,114,115,104,97,108,90,5, + 108,111,97,100,115,218,10,105,115,105,110,115,116,97,110,99, + 101,218,10,95,99,111,100,101,95,116,121,112,101,114,118,0, + 0,0,114,133,0,0,0,218,4,95,105,109,112,90,16,95, + 102,105,120,95,99,111,95,102,105,108,101,110,97,109,101,114, + 103,0,0,0,114,50,0,0,0,41,5,114,56,0,0,0, + 114,102,0,0,0,114,93,0,0,0,114,94,0,0,0,218, + 4,99,111,100,101,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,218,17,95,99,111,109,112,105,108,101,95,98, + 121,116,101,99,111,100,101,227,1,0,0,115,16,0,0,0, + 0,2,10,1,10,1,12,1,8,1,12,1,6,2,10,1, + 114,145,0,0,0,114,62,0,0,0,99,3,0,0,0,0, + 0,0,0,4,0,0,0,3,0,0,0,67,0,0,0,115, + 56,0,0,0,116,0,116,1,131,1,125,3,124,3,106,2, + 116,3,124,1,131,1,131,1,1,0,124,3,106,2,116,3, + 124,2,131,1,131,1,1,0,124,3,106,2,116,4,106,5, + 124,0,131,1,131,1,1,0,124,3,83,0,41,1,122,80, + 67,111,109,112,105,108,101,32,97,32,99,111,100,101,32,111, + 98,106,101,99,116,32,105,110,116,111,32,98,121,116,101,99, + 111,100,101,32,102,111,114,32,119,114,105,116,105,110,103,32, + 111,117,116,32,116,111,32,97,32,98,121,116,101,45,99,111, + 109,112,105,108,101,100,10,32,32,32,32,102,105,108,101,46, + 41,6,218,9,98,121,116,101,97,114,114,97,121,114,132,0, + 0,0,218,6,101,120,116,101,110,100,114,19,0,0,0,114, + 140,0,0,0,90,5,100,117,109,112,115,41,4,114,144,0, + 0,0,114,130,0,0,0,114,138,0,0,0,114,56,0,0, + 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, + 218,17,95,99,111,100,101,95,116,111,95,98,121,116,101,99, + 111,100,101,239,1,0,0,115,10,0,0,0,0,3,8,1, + 14,1,14,1,16,1,114,148,0,0,0,99,1,0,0,0, 0,0,0,0,5,0,0,0,4,0,0,0,67,0,0,0, - 115,60,0,0,0,124,0,106,0,124,1,131,1,92,2,125, - 2,125,3,124,2,100,1,107,8,114,56,116,1,124,3,131, - 1,114,56,100,2,125,4,116,2,106,3,124,4,106,4,124, - 3,100,3,25,0,131,1,116,5,131,2,1,0,124,2,83, - 0,41,4,122,155,84,114,121,32,116,111,32,102,105,110,100, - 32,97,32,108,111,97,100,101,114,32,102,111,114,32,116,104, - 101,32,115,112,101,99,105,102,105,101,100,32,109,111,100,117, - 108,101,32,98,121,32,100,101,108,101,103,97,116,105,110,103, - 32,116,111,10,32,32,32,32,115,101,108,102,46,102,105,110, - 100,95,108,111,97,100,101,114,40,41,46,10,10,32,32,32, - 32,84,104,105,115,32,109,101,116,104,111,100,32,105,115,32, - 100,101,112,114,101,99,97,116,101,100,32,105,110,32,102,97, - 118,111,114,32,111,102,32,102,105,110,100,101,114,46,102,105, - 110,100,95,115,112,101,99,40,41,46,10,10,32,32,32,32, - 78,122,44,78,111,116,32,105,109,112,111,114,116,105,110,103, - 32,100,105,114,101,99,116,111,114,121,32,123,125,58,32,109, - 105,115,115,105,110,103,32,95,95,105,110,105,116,95,95,114, - 62,0,0,0,41,6,218,11,102,105,110,100,95,108,111,97, - 100,101,114,114,33,0,0,0,114,63,0,0,0,114,64,0, - 0,0,114,50,0,0,0,218,13,73,109,112,111,114,116,87, - 97,114,110,105,110,103,41,5,114,104,0,0,0,218,8,102, - 117,108,108,110,97,109,101,218,6,108,111,97,100,101,114,218, - 8,112,111,114,116,105,111,110,115,218,3,109,115,103,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,17,95, - 102,105,110,100,95,109,111,100,117,108,101,95,115,104,105,109, - 155,1,0,0,115,10,0,0,0,0,10,14,1,16,1,4, - 1,22,1,114,127,0,0,0,99,4,0,0,0,0,0,0, - 0,11,0,0,0,22,0,0,0,67,0,0,0,115,142,1, - 0,0,105,0,125,4,124,2,100,1,107,9,114,22,124,2, - 124,4,100,2,60,0,110,4,100,3,125,2,124,3,100,1, - 107,9,114,42,124,3,124,4,100,4,60,0,124,0,100,1, - 100,5,133,2,25,0,125,5,124,0,100,5,100,6,133,2, - 25,0,125,6,124,0,100,6,100,7,133,2,25,0,125,7, - 124,5,116,0,107,3,114,126,100,8,106,1,124,2,124,5, - 131,2,125,8,116,2,106,3,100,9,124,8,131,2,1,0, - 116,4,124,8,102,1,124,4,151,1,142,1,130,1,110,86, - 116,5,124,6,131,1,100,5,107,3,114,170,100,10,106,1, - 124,2,131,1,125,8,116,2,106,3,100,9,124,8,131,2, - 1,0,116,6,124,8,131,1,130,1,110,42,116,5,124,7, - 131,1,100,5,107,3,114,212,100,11,106,1,124,2,131,1, - 125,8,116,2,106,3,100,9,124,8,131,2,1,0,116,6, - 124,8,131,1,130,1,124,1,100,1,107,9,144,1,114,130, - 121,16,116,7,124,1,100,12,25,0,131,1,125,9,87,0, - 110,22,4,0,116,8,107,10,144,1,114,4,1,0,1,0, - 1,0,89,0,110,52,88,0,116,9,124,6,131,1,124,9, - 107,3,144,1,114,56,100,13,106,1,124,2,131,1,125,8, - 116,2,106,3,100,9,124,8,131,2,1,0,116,4,124,8, - 102,1,124,4,151,1,142,1,130,1,121,16,124,1,100,14, - 25,0,100,15,64,0,125,10,87,0,110,22,4,0,116,8, - 107,10,144,1,114,94,1,0,1,0,1,0,89,0,110,36, - 88,0,116,9,124,7,131,1,124,10,107,3,144,1,114,130, - 116,4,100,13,106,1,124,2,131,1,102,1,124,4,151,1, - 142,1,130,1,124,0,100,7,100,1,133,2,25,0,83,0, - 41,16,97,122,1,0,0,86,97,108,105,100,97,116,101,32, - 116,104,101,32,104,101,97,100,101,114,32,111,102,32,116,104, - 101,32,112,97,115,115,101,100,45,105,110,32,98,121,116,101, - 99,111,100,101,32,97,103,97,105,110,115,116,32,115,111,117, - 114,99,101,95,115,116,97,116,115,32,40,105,102,10,32,32, - 32,32,103,105,118,101,110,41,32,97,110,100,32,114,101,116, - 117,114,110,105,110,103,32,116,104,101,32,98,121,116,101,99, - 111,100,101,32,116,104,97,116,32,99,97,110,32,98,101,32, - 99,111,109,112,105,108,101,100,32,98,121,32,99,111,109,112, - 105,108,101,40,41,46,10,10,32,32,32,32,65,108,108,32, - 111,116,104,101,114,32,97,114,103,117,109,101,110,116,115,32, - 97,114,101,32,117,115,101,100,32,116,111,32,101,110,104,97, - 110,99,101,32,101,114,114,111,114,32,114,101,112,111,114,116, - 105,110,103,46,10,10,32,32,32,32,73,109,112,111,114,116, - 69,114,114,111,114,32,105,115,32,114,97,105,115,101,100,32, - 119,104,101,110,32,116,104,101,32,109,97,103,105,99,32,110, - 117,109,98,101,114,32,105,115,32,105,110,99,111,114,114,101, - 99,116,32,111,114,32,116,104,101,32,98,121,116,101,99,111, - 100,101,32,105,115,10,32,32,32,32,102,111,117,110,100,32, - 116,111,32,98,101,32,115,116,97,108,101,46,32,69,79,70, - 69,114,114,111,114,32,105,115,32,114,97,105,115,101,100,32, - 119,104,101,110,32,116,104,101,32,100,97,116,97,32,105,115, - 32,102,111,117,110,100,32,116,111,32,98,101,10,32,32,32, - 32,116,114,117,110,99,97,116,101,100,46,10,10,32,32,32, - 32,78,114,102,0,0,0,122,10,60,98,121,116,101,99,111, - 100,101,62,114,37,0,0,0,114,14,0,0,0,233,8,0, - 0,0,233,12,0,0,0,122,30,98,97,100,32,109,97,103, - 105,99,32,110,117,109,98,101,114,32,105,110,32,123,33,114, - 125,58,32,123,33,114,125,122,2,123,125,122,43,114,101,97, - 99,104,101,100,32,69,79,70,32,119,104,105,108,101,32,114, - 101,97,100,105,110,103,32,116,105,109,101,115,116,97,109,112, - 32,105,110,32,123,33,114,125,122,48,114,101,97,99,104,101, - 100,32,69,79,70,32,119,104,105,108,101,32,114,101,97,100, - 105,110,103,32,115,105,122,101,32,111,102,32,115,111,117,114, - 99,101,32,105,110,32,123,33,114,125,218,5,109,116,105,109, - 101,122,26,98,121,116,101,99,111,100,101,32,105,115,32,115, - 116,97,108,101,32,102,111,114,32,123,33,114,125,218,4,115, - 105,122,101,108,3,0,0,0,255,127,255,127,3,0,41,10, - 218,12,77,65,71,73,67,95,78,85,77,66,69,82,114,50, - 0,0,0,114,118,0,0,0,218,16,95,118,101,114,98,111, - 115,101,95,109,101,115,115,97,103,101,114,103,0,0,0,114, - 33,0,0,0,218,8,69,79,70,69,114,114,111,114,114,16, - 0,0,0,218,8,75,101,121,69,114,114,111,114,114,21,0, - 0,0,41,11,114,56,0,0,0,218,12,115,111,117,114,99, - 101,95,115,116,97,116,115,114,102,0,0,0,114,37,0,0, - 0,90,11,101,120,99,95,100,101,116,97,105,108,115,90,5, - 109,97,103,105,99,90,13,114,97,119,95,116,105,109,101,115, - 116,97,109,112,90,8,114,97,119,95,115,105,122,101,114,79, - 0,0,0,218,12,115,111,117,114,99,101,95,109,116,105,109, - 101,218,11,115,111,117,114,99,101,95,115,105,122,101,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,25,95, - 118,97,108,105,100,97,116,101,95,98,121,116,101,99,111,100, - 101,95,104,101,97,100,101,114,172,1,0,0,115,76,0,0, - 0,0,11,4,1,8,1,10,3,4,1,8,1,8,1,12, - 1,12,1,12,1,8,1,12,1,12,1,16,1,12,1,10, - 1,12,1,10,1,12,1,10,1,12,1,8,1,10,1,2, - 1,16,1,16,1,6,2,14,1,10,1,12,1,14,1,2, - 1,16,1,16,1,6,2,14,1,12,1,8,1,114,139,0, - 0,0,99,4,0,0,0,0,0,0,0,5,0,0,0,5, - 0,0,0,67,0,0,0,115,82,0,0,0,116,0,106,1, - 124,0,131,1,125,4,116,2,124,4,116,3,131,2,114,58, - 116,4,106,5,100,1,124,2,131,2,1,0,124,3,100,2, - 107,9,114,52,116,6,106,7,124,4,124,3,131,2,1,0, - 124,4,83,0,110,20,116,8,100,3,106,9,124,2,131,1, - 124,1,124,2,100,4,141,3,130,1,100,2,83,0,41,5, - 122,60,67,111,109,112,105,108,101,32,98,121,116,101,99,111, - 100,101,32,97,115,32,114,101,116,117,114,110,101,100,32,98, - 121,32,95,118,97,108,105,100,97,116,101,95,98,121,116,101, - 99,111,100,101,95,104,101,97,100,101,114,40,41,46,122,21, - 99,111,100,101,32,111,98,106,101,99,116,32,102,114,111,109, - 32,123,33,114,125,78,122,23,78,111,110,45,99,111,100,101, - 32,111,98,106,101,99,116,32,105,110,32,123,33,114,125,41, - 2,114,102,0,0,0,114,37,0,0,0,41,10,218,7,109, - 97,114,115,104,97,108,90,5,108,111,97,100,115,218,10,105, - 115,105,110,115,116,97,110,99,101,218,10,95,99,111,100,101, - 95,116,121,112,101,114,118,0,0,0,114,133,0,0,0,218, - 4,95,105,109,112,90,16,95,102,105,120,95,99,111,95,102, - 105,108,101,110,97,109,101,114,103,0,0,0,114,50,0,0, - 0,41,5,114,56,0,0,0,114,102,0,0,0,114,93,0, - 0,0,114,94,0,0,0,218,4,99,111,100,101,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,17,95,99, - 111,109,112,105,108,101,95,98,121,116,101,99,111,100,101,227, - 1,0,0,115,16,0,0,0,0,2,10,1,10,1,12,1, - 8,1,12,1,6,2,10,1,114,145,0,0,0,114,62,0, - 0,0,99,3,0,0,0,0,0,0,0,4,0,0,0,3, - 0,0,0,67,0,0,0,115,56,0,0,0,116,0,116,1, - 131,1,125,3,124,3,106,2,116,3,124,1,131,1,131,1, - 1,0,124,3,106,2,116,3,124,2,131,1,131,1,1,0, - 124,3,106,2,116,4,106,5,124,0,131,1,131,1,1,0, - 124,3,83,0,41,1,122,80,67,111,109,112,105,108,101,32, - 97,32,99,111,100,101,32,111,98,106,101,99,116,32,105,110, - 116,111,32,98,121,116,101,99,111,100,101,32,102,111,114,32, - 119,114,105,116,105,110,103,32,111,117,116,32,116,111,32,97, - 32,98,121,116,101,45,99,111,109,112,105,108,101,100,10,32, - 32,32,32,102,105,108,101,46,41,6,218,9,98,121,116,101, - 97,114,114,97,121,114,132,0,0,0,218,6,101,120,116,101, - 110,100,114,19,0,0,0,114,140,0,0,0,90,5,100,117, - 109,112,115,41,4,114,144,0,0,0,114,130,0,0,0,114, - 138,0,0,0,114,56,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,17,95,99,111,100,101,95, - 116,111,95,98,121,116,101,99,111,100,101,239,1,0,0,115, - 10,0,0,0,0,3,8,1,14,1,14,1,16,1,114,148, - 0,0,0,99,1,0,0,0,0,0,0,0,5,0,0,0, - 4,0,0,0,67,0,0,0,115,62,0,0,0,100,1,100, - 2,108,0,125,1,116,1,106,2,124,0,131,1,106,3,125, - 2,124,1,106,4,124,2,131,1,125,3,116,1,106,5,100, - 2,100,3,131,2,125,4,124,4,106,6,124,0,106,6,124, - 3,100,1,25,0,131,1,131,1,83,0,41,4,122,121,68, - 101,99,111,100,101,32,98,121,116,101,115,32,114,101,112,114, - 101,115,101,110,116,105,110,103,32,115,111,117,114,99,101,32, - 99,111,100,101,32,97,110,100,32,114,101,116,117,114,110,32, - 116,104,101,32,115,116,114,105,110,103,46,10,10,32,32,32, - 32,85,110,105,118,101,114,115,97,108,32,110,101,119,108,105, - 110,101,32,115,117,112,112,111,114,116,32,105,115,32,117,115, - 101,100,32,105,110,32,116,104,101,32,100,101,99,111,100,105, - 110,103,46,10,32,32,32,32,114,62,0,0,0,78,84,41, - 7,218,8,116,111,107,101,110,105,122,101,114,52,0,0,0, - 90,7,66,121,116,101,115,73,79,90,8,114,101,97,100,108, - 105,110,101,90,15,100,101,116,101,99,116,95,101,110,99,111, - 100,105,110,103,90,25,73,110,99,114,101,109,101,110,116,97, - 108,78,101,119,108,105,110,101,68,101,99,111,100,101,114,218, - 6,100,101,99,111,100,101,41,5,218,12,115,111,117,114,99, - 101,95,98,121,116,101,115,114,149,0,0,0,90,21,115,111, - 117,114,99,101,95,98,121,116,101,115,95,114,101,97,100,108, - 105,110,101,218,8,101,110,99,111,100,105,110,103,90,15,110, - 101,119,108,105,110,101,95,100,101,99,111,100,101,114,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,13,100, - 101,99,111,100,101,95,115,111,117,114,99,101,249,1,0,0, - 115,10,0,0,0,0,5,8,1,12,1,10,1,12,1,114, - 153,0,0,0,41,2,114,124,0,0,0,218,26,115,117,98, - 109,111,100,117,108,101,95,115,101,97,114,99,104,95,108,111, - 99,97,116,105,111,110,115,99,2,0,0,0,2,0,0,0, - 9,0,0,0,19,0,0,0,67,0,0,0,115,18,1,0, - 0,124,1,100,1,107,8,114,60,100,2,125,1,116,0,124, - 2,100,3,131,2,114,70,121,14,124,2,106,1,124,0,131, - 1,125,1,87,0,113,70,4,0,116,2,107,10,114,56,1, - 0,1,0,1,0,89,0,113,70,88,0,110,10,116,3,106, - 4,124,1,131,1,125,1,116,5,106,6,124,0,124,2,124, - 1,100,4,141,3,125,4,100,5,124,4,95,7,124,2,100, - 1,107,8,114,156,120,54,116,8,131,0,68,0,93,40,92, - 2,125,5,125,6,124,1,106,9,116,10,124,6,131,1,131, - 1,114,108,124,5,124,0,124,1,131,2,125,2,124,2,124, - 4,95,11,80,0,113,108,87,0,100,1,83,0,124,3,116, - 12,107,8,114,222,116,0,124,2,100,6,131,2,114,228,121, - 14,124,2,106,13,124,0,131,1,125,7,87,0,110,20,4, - 0,116,2,107,10,114,208,1,0,1,0,1,0,89,0,113, - 228,88,0,124,7,114,228,103,0,124,4,95,14,110,6,124, - 3,124,4,95,14,124,4,106,14,103,0,107,2,144,1,114, - 14,124,1,144,1,114,14,116,15,124,1,131,1,100,7,25, - 0,125,8,124,4,106,14,106,16,124,8,131,1,1,0,124, - 4,83,0,41,8,97,61,1,0,0,82,101,116,117,114,110, - 32,97,32,109,111,100,117,108,101,32,115,112,101,99,32,98, - 97,115,101,100,32,111,110,32,97,32,102,105,108,101,32,108, - 111,99,97,116,105,111,110,46,10,10,32,32,32,32,84,111, - 32,105,110,100,105,99,97,116,101,32,116,104,97,116,32,116, - 104,101,32,109,111,100,117,108,101,32,105,115,32,97,32,112, - 97,99,107,97,103,101,44,32,115,101,116,10,32,32,32,32, - 115,117,98,109,111,100,117,108,101,95,115,101,97,114,99,104, - 95,108,111,99,97,116,105,111,110,115,32,116,111,32,97,32, - 108,105,115,116,32,111,102,32,100,105,114,101,99,116,111,114, - 121,32,112,97,116,104,115,46,32,32,65,110,10,32,32,32, - 32,101,109,112,116,121,32,108,105,115,116,32,105,115,32,115, - 117,102,102,105,99,105,101,110,116,44,32,116,104,111,117,103, - 104,32,105,116,115,32,110,111,116,32,111,116,104,101,114,119, - 105,115,101,32,117,115,101,102,117,108,32,116,111,32,116,104, - 101,10,32,32,32,32,105,109,112,111,114,116,32,115,121,115, - 116,101,109,46,10,10,32,32,32,32,84,104,101,32,108,111, - 97,100,101,114,32,109,117,115,116,32,116,97,107,101,32,97, - 32,115,112,101,99,32,97,115,32,105,116,115,32,111,110,108, - 121,32,95,95,105,110,105,116,95,95,40,41,32,97,114,103, - 46,10,10,32,32,32,32,78,122,9,60,117,110,107,110,111, - 119,110,62,218,12,103,101,116,95,102,105,108,101,110,97,109, - 101,41,1,218,6,111,114,105,103,105,110,84,218,10,105,115, - 95,112,97,99,107,97,103,101,114,62,0,0,0,41,17,114, - 112,0,0,0,114,155,0,0,0,114,103,0,0,0,114,3, - 0,0,0,114,67,0,0,0,114,118,0,0,0,218,10,77, - 111,100,117,108,101,83,112,101,99,90,13,95,115,101,116,95, - 102,105,108,101,97,116,116,114,218,27,95,103,101,116,95,115, - 117,112,112,111,114,116,101,100,95,102,105,108,101,95,108,111, - 97,100,101,114,115,114,96,0,0,0,114,97,0,0,0,114, - 124,0,0,0,218,9,95,80,79,80,85,76,65,84,69,114, - 157,0,0,0,114,154,0,0,0,114,40,0,0,0,218,6, - 97,112,112,101,110,100,41,9,114,102,0,0,0,90,8,108, - 111,99,97,116,105,111,110,114,124,0,0,0,114,154,0,0, - 0,218,4,115,112,101,99,218,12,108,111,97,100,101,114,95, - 99,108,97,115,115,218,8,115,117,102,102,105,120,101,115,114, - 157,0,0,0,90,7,100,105,114,110,97,109,101,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,23,115,112, - 101,99,95,102,114,111,109,95,102,105,108,101,95,108,111,99, - 97,116,105,111,110,10,2,0,0,115,62,0,0,0,0,12, - 8,4,4,1,10,2,2,1,14,1,14,1,8,2,10,8, - 16,1,6,3,8,1,16,1,14,1,10,1,6,1,6,2, - 4,3,8,2,10,1,2,1,14,1,14,1,6,2,4,1, - 8,2,6,1,12,1,6,1,12,1,12,2,114,165,0,0, - 0,99,0,0,0,0,0,0,0,0,0,0,0,0,4,0, - 0,0,64,0,0,0,115,80,0,0,0,101,0,90,1,100, - 0,90,2,100,1,90,3,100,2,90,4,100,3,90,5,100, - 4,90,6,101,7,100,5,100,6,132,0,131,1,90,8,101, - 7,100,7,100,8,132,0,131,1,90,9,101,7,100,14,100, - 10,100,11,132,1,131,1,90,10,101,7,100,15,100,12,100, - 13,132,1,131,1,90,11,100,9,83,0,41,16,218,21,87, - 105,110,100,111,119,115,82,101,103,105,115,116,114,121,70,105, - 110,100,101,114,122,62,77,101,116,97,32,112,97,116,104,32, - 102,105,110,100,101,114,32,102,111,114,32,109,111,100,117,108, - 101,115,32,100,101,99,108,97,114,101,100,32,105,110,32,116, - 104,101,32,87,105,110,100,111,119,115,32,114,101,103,105,115, - 116,114,121,46,122,59,83,111,102,116,119,97,114,101,92,80, - 121,116,104,111,110,92,80,121,116,104,111,110,67,111,114,101, - 92,123,115,121,115,95,118,101,114,115,105,111,110,125,92,77, - 111,100,117,108,101,115,92,123,102,117,108,108,110,97,109,101, - 125,122,65,83,111,102,116,119,97,114,101,92,80,121,116,104, - 111,110,92,80,121,116,104,111,110,67,111,114,101,92,123,115, - 121,115,95,118,101,114,115,105,111,110,125,92,77,111,100,117, - 108,101,115,92,123,102,117,108,108,110,97,109,101,125,92,68, - 101,98,117,103,70,99,2,0,0,0,0,0,0,0,2,0, - 0,0,11,0,0,0,67,0,0,0,115,50,0,0,0,121, - 14,116,0,106,1,116,0,106,2,124,1,131,2,83,0,4, - 0,116,3,107,10,114,44,1,0,1,0,1,0,116,0,106, - 1,116,0,106,4,124,1,131,2,83,0,88,0,100,0,83, - 0,41,1,78,41,5,218,7,95,119,105,110,114,101,103,90, - 7,79,112,101,110,75,101,121,90,17,72,75,69,89,95,67, - 85,82,82,69,78,84,95,85,83,69,82,114,42,0,0,0, - 90,18,72,75,69,89,95,76,79,67,65,76,95,77,65,67, - 72,73,78,69,41,2,218,3,99,108,115,114,5,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 14,95,111,112,101,110,95,114,101,103,105,115,116,114,121,90, - 2,0,0,115,8,0,0,0,0,2,2,1,14,1,14,1, - 122,36,87,105,110,100,111,119,115,82,101,103,105,115,116,114, - 121,70,105,110,100,101,114,46,95,111,112,101,110,95,114,101, - 103,105,115,116,114,121,99,2,0,0,0,0,0,0,0,6, - 0,0,0,16,0,0,0,67,0,0,0,115,112,0,0,0, - 124,0,106,0,114,14,124,0,106,1,125,2,110,6,124,0, - 106,2,125,2,124,2,106,3,124,1,100,1,116,4,106,5, - 100,0,100,2,133,2,25,0,22,0,100,3,141,2,125,3, - 121,38,124,0,106,6,124,3,131,1,143,18,125,4,116,7, - 106,8,124,4,100,4,131,2,125,5,87,0,100,0,81,0, - 82,0,88,0,87,0,110,20,4,0,116,9,107,10,114,106, - 1,0,1,0,1,0,100,0,83,0,88,0,124,5,83,0, - 41,5,78,122,5,37,100,46,37,100,114,59,0,0,0,41, - 2,114,123,0,0,0,90,11,115,121,115,95,118,101,114,115, - 105,111,110,114,32,0,0,0,41,10,218,11,68,69,66,85, - 71,95,66,85,73,76,68,218,18,82,69,71,73,83,84,82, - 89,95,75,69,89,95,68,69,66,85,71,218,12,82,69,71, - 73,83,84,82,89,95,75,69,89,114,50,0,0,0,114,8, - 0,0,0,218,12,118,101,114,115,105,111,110,95,105,110,102, - 111,114,169,0,0,0,114,167,0,0,0,90,10,81,117,101, - 114,121,86,97,108,117,101,114,42,0,0,0,41,6,114,168, - 0,0,0,114,123,0,0,0,90,12,114,101,103,105,115,116, - 114,121,95,107,101,121,114,5,0,0,0,90,4,104,107,101, - 121,218,8,102,105,108,101,112,97,116,104,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,16,95,115,101,97, - 114,99,104,95,114,101,103,105,115,116,114,121,97,2,0,0, - 115,22,0,0,0,0,2,6,1,8,2,6,1,6,1,22, - 1,2,1,12,1,26,1,14,1,6,1,122,38,87,105,110, - 100,111,119,115,82,101,103,105,115,116,114,121,70,105,110,100, - 101,114,46,95,115,101,97,114,99,104,95,114,101,103,105,115, - 116,114,121,78,99,4,0,0,0,0,0,0,0,8,0,0, - 0,14,0,0,0,67,0,0,0,115,120,0,0,0,124,0, - 106,0,124,1,131,1,125,4,124,4,100,0,107,8,114,22, - 100,0,83,0,121,12,116,1,124,4,131,1,1,0,87,0, - 110,20,4,0,116,2,107,10,114,54,1,0,1,0,1,0, - 100,0,83,0,88,0,120,58,116,3,131,0,68,0,93,48, - 92,2,125,5,125,6,124,4,106,4,116,5,124,6,131,1, - 131,1,114,64,116,6,106,7,124,1,124,5,124,1,124,4, - 131,2,124,4,100,1,141,3,125,7,124,7,83,0,113,64, - 87,0,100,0,83,0,41,2,78,41,1,114,156,0,0,0, - 41,8,114,175,0,0,0,114,41,0,0,0,114,42,0,0, - 0,114,159,0,0,0,114,96,0,0,0,114,97,0,0,0, - 114,118,0,0,0,218,16,115,112,101,99,95,102,114,111,109, - 95,108,111,97,100,101,114,41,8,114,168,0,0,0,114,123, - 0,0,0,114,37,0,0,0,218,6,116,97,114,103,101,116, - 114,174,0,0,0,114,124,0,0,0,114,164,0,0,0,114, - 162,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,9,102,105,110,100,95,115,112,101,99,112,2, - 0,0,115,26,0,0,0,0,2,10,1,8,1,4,1,2, - 1,12,1,14,1,6,1,16,1,14,1,6,1,8,1,8, - 1,122,31,87,105,110,100,111,119,115,82,101,103,105,115,116, - 114,121,70,105,110,100,101,114,46,102,105,110,100,95,115,112, - 101,99,99,3,0,0,0,0,0,0,0,4,0,0,0,3, - 0,0,0,67,0,0,0,115,36,0,0,0,124,0,106,0, - 124,1,124,2,131,2,125,3,124,3,100,1,107,9,114,28, - 124,3,106,1,83,0,110,4,100,1,83,0,100,1,83,0, - 41,2,122,108,70,105,110,100,32,109,111,100,117,108,101,32, - 110,97,109,101,100,32,105,110,32,116,104,101,32,114,101,103, - 105,115,116,114,121,46,10,10,32,32,32,32,32,32,32,32, - 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, - 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, - 101,120,101,99,95,109,111,100,117,108,101,40,41,32,105,110, - 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, - 78,41,2,114,178,0,0,0,114,124,0,0,0,41,4,114, - 168,0,0,0,114,123,0,0,0,114,37,0,0,0,114,162, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,102,105,110,100,95,109,111,100,117,108,101,128, - 2,0,0,115,8,0,0,0,0,7,12,1,8,1,8,2, - 122,33,87,105,110,100,111,119,115,82,101,103,105,115,116,114, - 121,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, - 117,108,101,41,2,78,78,41,1,78,41,12,114,109,0,0, - 0,114,108,0,0,0,114,110,0,0,0,114,111,0,0,0, - 114,172,0,0,0,114,171,0,0,0,114,170,0,0,0,218, - 11,99,108,97,115,115,109,101,116,104,111,100,114,169,0,0, - 0,114,175,0,0,0,114,178,0,0,0,114,179,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,166,0,0,0,78,2,0,0,115,20,0, - 0,0,8,2,4,3,4,3,4,2,4,2,12,7,12,15, - 2,1,12,15,2,1,114,166,0,0,0,99,0,0,0,0, - 0,0,0,0,0,0,0,0,2,0,0,0,64,0,0,0, - 115,48,0,0,0,101,0,90,1,100,0,90,2,100,1,90, - 3,100,2,100,3,132,0,90,4,100,4,100,5,132,0,90, - 5,100,6,100,7,132,0,90,6,100,8,100,9,132,0,90, - 7,100,10,83,0,41,11,218,13,95,76,111,97,100,101,114, - 66,97,115,105,99,115,122,83,66,97,115,101,32,99,108,97, - 115,115,32,111,102,32,99,111,109,109,111,110,32,99,111,100, - 101,32,110,101,101,100,101,100,32,98,121,32,98,111,116,104, - 32,83,111,117,114,99,101,76,111,97,100,101,114,32,97,110, - 100,10,32,32,32,32,83,111,117,114,99,101,108,101,115,115, - 70,105,108,101,76,111,97,100,101,114,46,99,2,0,0,0, - 0,0,0,0,5,0,0,0,3,0,0,0,67,0,0,0, - 115,64,0,0,0,116,0,124,0,106,1,124,1,131,1,131, - 1,100,1,25,0,125,2,124,2,106,2,100,2,100,1,131, - 2,100,3,25,0,125,3,124,1,106,3,100,2,131,1,100, - 4,25,0,125,4,124,3,100,5,107,2,111,62,124,4,100, - 5,107,3,83,0,41,6,122,141,67,111,110,99,114,101,116, - 101,32,105,109,112,108,101,109,101,110,116,97,116,105,111,110, - 32,111,102,32,73,110,115,112,101,99,116,76,111,97,100,101, - 114,46,105,115,95,112,97,99,107,97,103,101,32,98,121,32, - 99,104,101,99,107,105,110,103,32,105,102,10,32,32,32,32, - 32,32,32,32,116,104,101,32,112,97,116,104,32,114,101,116, - 117,114,110,101,100,32,98,121,32,103,101,116,95,102,105,108, - 101,110,97,109,101,32,104,97,115,32,97,32,102,105,108,101, - 110,97,109,101,32,111,102,32,39,95,95,105,110,105,116,95, - 95,46,112,121,39,46,114,31,0,0,0,114,61,0,0,0, - 114,62,0,0,0,114,59,0,0,0,218,8,95,95,105,110, - 105,116,95,95,41,4,114,40,0,0,0,114,155,0,0,0, - 114,36,0,0,0,114,34,0,0,0,41,5,114,104,0,0, - 0,114,123,0,0,0,114,98,0,0,0,90,13,102,105,108, - 101,110,97,109,101,95,98,97,115,101,90,9,116,97,105,108, - 95,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,157,0,0,0,147,2,0,0,115,8,0, - 0,0,0,3,18,1,16,1,14,1,122,24,95,76,111,97, - 100,101,114,66,97,115,105,99,115,46,105,115,95,112,97,99, - 107,97,103,101,99,2,0,0,0,0,0,0,0,2,0,0, - 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, - 83,0,41,2,122,42,85,115,101,32,100,101,102,97,117,108, - 116,32,115,101,109,97,110,116,105,99,115,32,102,111,114,32, - 109,111,100,117,108,101,32,99,114,101,97,116,105,111,110,46, - 78,114,4,0,0,0,41,2,114,104,0,0,0,114,162,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,218,13,99,114,101,97,116,101,95,109,111,100,117,108,101, - 155,2,0,0,115,0,0,0,0,122,27,95,76,111,97,100, - 101,114,66,97,115,105,99,115,46,99,114,101,97,116,101,95, - 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,3, - 0,0,0,4,0,0,0,67,0,0,0,115,56,0,0,0, - 124,0,106,0,124,1,106,1,131,1,125,2,124,2,100,1, - 107,8,114,36,116,2,100,2,106,3,124,1,106,1,131,1, - 131,1,130,1,116,4,106,5,116,6,124,2,124,1,106,7, - 131,3,1,0,100,1,83,0,41,3,122,19,69,120,101,99, - 117,116,101,32,116,104,101,32,109,111,100,117,108,101,46,78, - 122,52,99,97,110,110,111,116,32,108,111,97,100,32,109,111, - 100,117,108,101,32,123,33,114,125,32,119,104,101,110,32,103, - 101,116,95,99,111,100,101,40,41,32,114,101,116,117,114,110, - 115,32,78,111,110,101,41,8,218,8,103,101,116,95,99,111, - 100,101,114,109,0,0,0,114,103,0,0,0,114,50,0,0, - 0,114,118,0,0,0,218,25,95,99,97,108,108,95,119,105, - 116,104,95,102,114,97,109,101,115,95,114,101,109,111,118,101, - 100,218,4,101,120,101,99,114,115,0,0,0,41,3,114,104, - 0,0,0,218,6,109,111,100,117,108,101,114,144,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 11,101,120,101,99,95,109,111,100,117,108,101,158,2,0,0, - 115,10,0,0,0,0,2,12,1,8,1,6,1,10,1,122, - 25,95,76,111,97,100,101,114,66,97,115,105,99,115,46,101, - 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, - 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, - 12,0,0,0,116,0,106,1,124,0,124,1,131,2,83,0, - 41,1,122,26,84,104,105,115,32,109,111,100,117,108,101,32, - 105,115,32,100,101,112,114,101,99,97,116,101,100,46,41,2, - 114,118,0,0,0,218,17,95,108,111,97,100,95,109,111,100, - 117,108,101,95,115,104,105,109,41,2,114,104,0,0,0,114, - 123,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,11,108,111,97,100,95,109,111,100,117,108,101, - 166,2,0,0,115,2,0,0,0,0,2,122,25,95,76,111, - 97,100,101,114,66,97,115,105,99,115,46,108,111,97,100,95, - 109,111,100,117,108,101,78,41,8,114,109,0,0,0,114,108, - 0,0,0,114,110,0,0,0,114,111,0,0,0,114,157,0, - 0,0,114,183,0,0,0,114,188,0,0,0,114,190,0,0, - 0,114,4,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,181,0,0,0,142,2,0,0,115,10, - 0,0,0,8,3,4,2,8,8,8,3,8,8,114,181,0, - 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,3, - 0,0,0,64,0,0,0,115,74,0,0,0,101,0,90,1, - 100,0,90,2,100,1,100,2,132,0,90,3,100,3,100,4, - 132,0,90,4,100,5,100,6,132,0,90,5,100,7,100,8, - 132,0,90,6,100,9,100,10,132,0,90,7,100,18,100,12, - 156,1,100,13,100,14,132,2,90,8,100,15,100,16,132,0, - 90,9,100,17,83,0,41,19,218,12,83,111,117,114,99,101, - 76,111,97,100,101,114,99,2,0,0,0,0,0,0,0,2, - 0,0,0,1,0,0,0,67,0,0,0,115,8,0,0,0, - 116,0,130,1,100,1,83,0,41,2,122,178,79,112,116,105, - 111,110,97,108,32,109,101,116,104,111,100,32,116,104,97,116, - 32,114,101,116,117,114,110,115,32,116,104,101,32,109,111,100, - 105,102,105,99,97,116,105,111,110,32,116,105,109,101,32,40, - 97,110,32,105,110,116,41,32,102,111,114,32,116,104,101,10, - 32,32,32,32,32,32,32,32,115,112,101,99,105,102,105,101, - 100,32,112,97,116,104,44,32,119,104,101,114,101,32,112,97, - 116,104,32,105,115,32,97,32,115,116,114,46,10,10,32,32, - 32,32,32,32,32,32,82,97,105,115,101,115,32,73,79,69, - 114,114,111,114,32,119,104,101,110,32,116,104,101,32,112,97, - 116,104,32,99,97,110,110,111,116,32,98,101,32,104,97,110, - 100,108,101,100,46,10,32,32,32,32,32,32,32,32,78,41, - 1,218,7,73,79,69,114,114,111,114,41,2,114,104,0,0, - 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,218,10,112,97,116,104,95,109,116,105,109, - 101,173,2,0,0,115,2,0,0,0,0,6,122,23,83,111, - 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, - 109,116,105,109,101,99,2,0,0,0,0,0,0,0,2,0, - 0,0,3,0,0,0,67,0,0,0,115,14,0,0,0,100, - 1,124,0,106,0,124,1,131,1,105,1,83,0,41,2,97, - 170,1,0,0,79,112,116,105,111,110,97,108,32,109,101,116, - 104,111,100,32,114,101,116,117,114,110,105,110,103,32,97,32, - 109,101,116,97,100,97,116,97,32,100,105,99,116,32,102,111, - 114,32,116,104,101,32,115,112,101,99,105,102,105,101,100,32, - 112,97,116,104,10,32,32,32,32,32,32,32,32,116,111,32, - 98,121,32,116,104,101,32,112,97,116,104,32,40,115,116,114, - 41,46,10,32,32,32,32,32,32,32,32,80,111,115,115,105, - 98,108,101,32,107,101,121,115,58,10,32,32,32,32,32,32, - 32,32,45,32,39,109,116,105,109,101,39,32,40,109,97,110, - 100,97,116,111,114,121,41,32,105,115,32,116,104,101,32,110, - 117,109,101,114,105,99,32,116,105,109,101,115,116,97,109,112, - 32,111,102,32,108,97,115,116,32,115,111,117,114,99,101,10, - 32,32,32,32,32,32,32,32,32,32,99,111,100,101,32,109, - 111,100,105,102,105,99,97,116,105,111,110,59,10,32,32,32, - 32,32,32,32,32,45,32,39,115,105,122,101,39,32,40,111, - 112,116,105,111,110,97,108,41,32,105,115,32,116,104,101,32, - 115,105,122,101,32,105,110,32,98,121,116,101,115,32,111,102, - 32,116,104,101,32,115,111,117,114,99,101,32,99,111,100,101, - 46,10,10,32,32,32,32,32,32,32,32,73,109,112,108,101, - 109,101,110,116,105,110,103,32,116,104,105,115,32,109,101,116, - 104,111,100,32,97,108,108,111,119,115,32,116,104,101,32,108, - 111,97,100,101,114,32,116,111,32,114,101,97,100,32,98,121, - 116,101,99,111,100,101,32,102,105,108,101,115,46,10,32,32, - 32,32,32,32,32,32,82,97,105,115,101,115,32,73,79,69, - 114,114,111,114,32,119,104,101,110,32,116,104,101,32,112,97, - 116,104,32,99,97,110,110,111,116,32,98,101,32,104,97,110, - 100,108,101,100,46,10,32,32,32,32,32,32,32,32,114,130, - 0,0,0,41,1,114,193,0,0,0,41,2,114,104,0,0, - 0,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,218,10,112,97,116,104,95,115,116,97,116, - 115,181,2,0,0,115,2,0,0,0,0,11,122,23,83,111, - 117,114,99,101,76,111,97,100,101,114,46,112,97,116,104,95, - 115,116,97,116,115,99,4,0,0,0,0,0,0,0,4,0, - 0,0,3,0,0,0,67,0,0,0,115,12,0,0,0,124, - 0,106,0,124,2,124,3,131,2,83,0,41,1,122,228,79, + 115,62,0,0,0,100,1,100,2,108,0,125,1,116,1,106, + 2,124,0,131,1,106,3,125,2,124,1,106,4,124,2,131, + 1,125,3,116,1,106,5,100,2,100,3,131,2,125,4,124, + 4,106,6,124,0,106,6,124,3,100,1,25,0,131,1,131, + 1,83,0,41,4,122,121,68,101,99,111,100,101,32,98,121, + 116,101,115,32,114,101,112,114,101,115,101,110,116,105,110,103, + 32,115,111,117,114,99,101,32,99,111,100,101,32,97,110,100, + 32,114,101,116,117,114,110,32,116,104,101,32,115,116,114,105, + 110,103,46,10,10,32,32,32,32,85,110,105,118,101,114,115, + 97,108,32,110,101,119,108,105,110,101,32,115,117,112,112,111, + 114,116,32,105,115,32,117,115,101,100,32,105,110,32,116,104, + 101,32,100,101,99,111,100,105,110,103,46,10,32,32,32,32, + 114,62,0,0,0,78,84,41,7,218,8,116,111,107,101,110, + 105,122,101,114,52,0,0,0,90,7,66,121,116,101,115,73, + 79,90,8,114,101,97,100,108,105,110,101,90,15,100,101,116, + 101,99,116,95,101,110,99,111,100,105,110,103,90,25,73,110, + 99,114,101,109,101,110,116,97,108,78,101,119,108,105,110,101, + 68,101,99,111,100,101,114,218,6,100,101,99,111,100,101,41, + 5,218,12,115,111,117,114,99,101,95,98,121,116,101,115,114, + 149,0,0,0,90,21,115,111,117,114,99,101,95,98,121,116, + 101,115,95,114,101,97,100,108,105,110,101,218,8,101,110,99, + 111,100,105,110,103,90,15,110,101,119,108,105,110,101,95,100, + 101,99,111,100,101,114,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,218,13,100,101,99,111,100,101,95,115,111, + 117,114,99,101,249,1,0,0,115,10,0,0,0,0,5,8, + 1,12,1,10,1,12,1,114,153,0,0,0,41,2,114,124, + 0,0,0,218,26,115,117,98,109,111,100,117,108,101,95,115, + 101,97,114,99,104,95,108,111,99,97,116,105,111,110,115,99, + 2,0,0,0,2,0,0,0,9,0,0,0,19,0,0,0, + 67,0,0,0,115,18,1,0,0,124,1,100,1,107,8,114, + 60,100,2,125,1,116,0,124,2,100,3,131,2,114,70,121, + 14,124,2,106,1,124,0,131,1,125,1,87,0,113,70,4, + 0,116,2,107,10,114,56,1,0,1,0,1,0,89,0,113, + 70,88,0,110,10,116,3,106,4,124,1,131,1,125,1,116, + 5,106,6,124,0,124,2,124,1,100,4,141,3,125,4,100, + 5,124,4,95,7,124,2,100,1,107,8,114,156,120,54,116, + 8,131,0,68,0,93,40,92,2,125,5,125,6,124,1,106, + 9,116,10,124,6,131,1,131,1,114,108,124,5,124,0,124, + 1,131,2,125,2,124,2,124,4,95,11,80,0,113,108,87, + 0,100,1,83,0,124,3,116,12,107,8,114,222,116,0,124, + 2,100,6,131,2,114,228,121,14,124,2,106,13,124,0,131, + 1,125,7,87,0,110,20,4,0,116,2,107,10,114,208,1, + 0,1,0,1,0,89,0,113,228,88,0,124,7,114,228,103, + 0,124,4,95,14,110,6,124,3,124,4,95,14,124,4,106, + 14,103,0,107,2,144,1,114,14,124,1,144,1,114,14,116, + 15,124,1,131,1,100,7,25,0,125,8,124,4,106,14,106, + 16,124,8,131,1,1,0,124,4,83,0,41,8,97,61,1, + 0,0,82,101,116,117,114,110,32,97,32,109,111,100,117,108, + 101,32,115,112,101,99,32,98,97,115,101,100,32,111,110,32, + 97,32,102,105,108,101,32,108,111,99,97,116,105,111,110,46, + 10,10,32,32,32,32,84,111,32,105,110,100,105,99,97,116, + 101,32,116,104,97,116,32,116,104,101,32,109,111,100,117,108, + 101,32,105,115,32,97,32,112,97,99,107,97,103,101,44,32, + 115,101,116,10,32,32,32,32,115,117,98,109,111,100,117,108, + 101,95,115,101,97,114,99,104,95,108,111,99,97,116,105,111, + 110,115,32,116,111,32,97,32,108,105,115,116,32,111,102,32, + 100,105,114,101,99,116,111,114,121,32,112,97,116,104,115,46, + 32,32,65,110,10,32,32,32,32,101,109,112,116,121,32,108, + 105,115,116,32,105,115,32,115,117,102,102,105,99,105,101,110, + 116,44,32,116,104,111,117,103,104,32,105,116,115,32,110,111, + 116,32,111,116,104,101,114,119,105,115,101,32,117,115,101,102, + 117,108,32,116,111,32,116,104,101,10,32,32,32,32,105,109, + 112,111,114,116,32,115,121,115,116,101,109,46,10,10,32,32, + 32,32,84,104,101,32,108,111,97,100,101,114,32,109,117,115, + 116,32,116,97,107,101,32,97,32,115,112,101,99,32,97,115, + 32,105,116,115,32,111,110,108,121,32,95,95,105,110,105,116, + 95,95,40,41,32,97,114,103,46,10,10,32,32,32,32,78, + 122,9,60,117,110,107,110,111,119,110,62,218,12,103,101,116, + 95,102,105,108,101,110,97,109,101,41,1,218,6,111,114,105, + 103,105,110,84,218,10,105,115,95,112,97,99,107,97,103,101, + 114,62,0,0,0,41,17,114,112,0,0,0,114,155,0,0, + 0,114,103,0,0,0,114,3,0,0,0,114,67,0,0,0, + 114,118,0,0,0,218,10,77,111,100,117,108,101,83,112,101, + 99,90,13,95,115,101,116,95,102,105,108,101,97,116,116,114, + 218,27,95,103,101,116,95,115,117,112,112,111,114,116,101,100, + 95,102,105,108,101,95,108,111,97,100,101,114,115,114,96,0, + 0,0,114,97,0,0,0,114,124,0,0,0,218,9,95,80, + 79,80,85,76,65,84,69,114,157,0,0,0,114,154,0,0, + 0,114,40,0,0,0,218,6,97,112,112,101,110,100,41,9, + 114,102,0,0,0,90,8,108,111,99,97,116,105,111,110,114, + 124,0,0,0,114,154,0,0,0,218,4,115,112,101,99,218, + 12,108,111,97,100,101,114,95,99,108,97,115,115,218,8,115, + 117,102,102,105,120,101,115,114,157,0,0,0,90,7,100,105, + 114,110,97,109,101,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,218,23,115,112,101,99,95,102,114,111,109,95, + 102,105,108,101,95,108,111,99,97,116,105,111,110,10,2,0, + 0,115,62,0,0,0,0,12,8,4,4,1,10,2,2,1, + 14,1,14,1,8,2,10,8,16,1,6,3,8,1,16,1, + 14,1,10,1,6,1,6,2,4,3,8,2,10,1,2,1, + 14,1,14,1,6,2,4,1,8,2,6,1,12,1,6,1, + 12,1,12,2,114,165,0,0,0,99,0,0,0,0,0,0, + 0,0,0,0,0,0,4,0,0,0,64,0,0,0,115,80, + 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, + 2,90,4,100,3,90,5,100,4,90,6,101,7,100,5,100, + 6,132,0,131,1,90,8,101,7,100,7,100,8,132,0,131, + 1,90,9,101,7,100,14,100,10,100,11,132,1,131,1,90, + 10,101,7,100,15,100,12,100,13,132,1,131,1,90,11,100, + 9,83,0,41,16,218,21,87,105,110,100,111,119,115,82,101, + 103,105,115,116,114,121,70,105,110,100,101,114,122,62,77,101, + 116,97,32,112,97,116,104,32,102,105,110,100,101,114,32,102, + 111,114,32,109,111,100,117,108,101,115,32,100,101,99,108,97, + 114,101,100,32,105,110,32,116,104,101,32,87,105,110,100,111, + 119,115,32,114,101,103,105,115,116,114,121,46,122,59,83,111, + 102,116,119,97,114,101,92,80,121,116,104,111,110,92,80,121, + 116,104,111,110,67,111,114,101,92,123,115,121,115,95,118,101, + 114,115,105,111,110,125,92,77,111,100,117,108,101,115,92,123, + 102,117,108,108,110,97,109,101,125,122,65,83,111,102,116,119, + 97,114,101,92,80,121,116,104,111,110,92,80,121,116,104,111, + 110,67,111,114,101,92,123,115,121,115,95,118,101,114,115,105, + 111,110,125,92,77,111,100,117,108,101,115,92,123,102,117,108, + 108,110,97,109,101,125,92,68,101,98,117,103,70,99,2,0, + 0,0,0,0,0,0,2,0,0,0,11,0,0,0,67,0, + 0,0,115,50,0,0,0,121,14,116,0,106,1,116,0,106, + 2,124,1,131,2,83,0,4,0,116,3,107,10,114,44,1, + 0,1,0,1,0,116,0,106,1,116,0,106,4,124,1,131, + 2,83,0,88,0,100,0,83,0,41,1,78,41,5,218,7, + 95,119,105,110,114,101,103,90,7,79,112,101,110,75,101,121, + 90,17,72,75,69,89,95,67,85,82,82,69,78,84,95,85, + 83,69,82,114,42,0,0,0,90,18,72,75,69,89,95,76, + 79,67,65,76,95,77,65,67,72,73,78,69,41,2,218,3, + 99,108,115,114,5,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,218,14,95,111,112,101,110,95,114, + 101,103,105,115,116,114,121,90,2,0,0,115,8,0,0,0, + 0,2,2,1,14,1,14,1,122,36,87,105,110,100,111,119, + 115,82,101,103,105,115,116,114,121,70,105,110,100,101,114,46, + 95,111,112,101,110,95,114,101,103,105,115,116,114,121,99,2, + 0,0,0,0,0,0,0,6,0,0,0,16,0,0,0,67, + 0,0,0,115,112,0,0,0,124,0,106,0,114,14,124,0, + 106,1,125,2,110,6,124,0,106,2,125,2,124,2,106,3, + 124,1,100,1,116,4,106,5,100,0,100,2,133,2,25,0, + 22,0,100,3,141,2,125,3,121,38,124,0,106,6,124,3, + 131,1,143,18,125,4,116,7,106,8,124,4,100,4,131,2, + 125,5,87,0,100,0,81,0,82,0,88,0,87,0,110,20, + 4,0,116,9,107,10,114,106,1,0,1,0,1,0,100,0, + 83,0,88,0,124,5,83,0,41,5,78,122,5,37,100,46, + 37,100,114,59,0,0,0,41,2,114,123,0,0,0,90,11, + 115,121,115,95,118,101,114,115,105,111,110,114,32,0,0,0, + 41,10,218,11,68,69,66,85,71,95,66,85,73,76,68,218, + 18,82,69,71,73,83,84,82,89,95,75,69,89,95,68,69, + 66,85,71,218,12,82,69,71,73,83,84,82,89,95,75,69, + 89,114,50,0,0,0,114,8,0,0,0,218,12,118,101,114, + 115,105,111,110,95,105,110,102,111,114,169,0,0,0,114,167, + 0,0,0,90,10,81,117,101,114,121,86,97,108,117,101,114, + 42,0,0,0,41,6,114,168,0,0,0,114,123,0,0,0, + 90,12,114,101,103,105,115,116,114,121,95,107,101,121,114,5, + 0,0,0,90,4,104,107,101,121,218,8,102,105,108,101,112, + 97,116,104,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,16,95,115,101,97,114,99,104,95,114,101,103,105, + 115,116,114,121,97,2,0,0,115,22,0,0,0,0,2,6, + 1,8,2,6,1,6,1,22,1,2,1,12,1,26,1,14, + 1,6,1,122,38,87,105,110,100,111,119,115,82,101,103,105, + 115,116,114,121,70,105,110,100,101,114,46,95,115,101,97,114, + 99,104,95,114,101,103,105,115,116,114,121,78,99,4,0,0, + 0,0,0,0,0,8,0,0,0,14,0,0,0,67,0,0, + 0,115,120,0,0,0,124,0,106,0,124,1,131,1,125,4, + 124,4,100,0,107,8,114,22,100,0,83,0,121,12,116,1, + 124,4,131,1,1,0,87,0,110,20,4,0,116,2,107,10, + 114,54,1,0,1,0,1,0,100,0,83,0,88,0,120,58, + 116,3,131,0,68,0,93,48,92,2,125,5,125,6,124,4, + 106,4,116,5,124,6,131,1,131,1,114,64,116,6,106,7, + 124,1,124,5,124,1,124,4,131,2,124,4,100,1,141,3, + 125,7,124,7,83,0,113,64,87,0,100,0,83,0,41,2, + 78,41,1,114,156,0,0,0,41,8,114,175,0,0,0,114, + 41,0,0,0,114,42,0,0,0,114,159,0,0,0,114,96, + 0,0,0,114,97,0,0,0,114,118,0,0,0,218,16,115, + 112,101,99,95,102,114,111,109,95,108,111,97,100,101,114,41, + 8,114,168,0,0,0,114,123,0,0,0,114,37,0,0,0, + 218,6,116,97,114,103,101,116,114,174,0,0,0,114,124,0, + 0,0,114,164,0,0,0,114,162,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,9,102,105,110, + 100,95,115,112,101,99,112,2,0,0,115,26,0,0,0,0, + 2,10,1,8,1,4,1,2,1,12,1,14,1,6,1,16, + 1,14,1,6,1,8,1,8,1,122,31,87,105,110,100,111, + 119,115,82,101,103,105,115,116,114,121,70,105,110,100,101,114, + 46,102,105,110,100,95,115,112,101,99,99,3,0,0,0,0, + 0,0,0,4,0,0,0,3,0,0,0,67,0,0,0,115, + 36,0,0,0,124,0,106,0,124,1,124,2,131,2,125,3, + 124,3,100,1,107,9,114,28,124,3,106,1,83,0,110,4, + 100,1,83,0,100,1,83,0,41,2,122,108,70,105,110,100, + 32,109,111,100,117,108,101,32,110,97,109,101,100,32,105,110, + 32,116,104,101,32,114,101,103,105,115,116,114,121,46,10,10, + 32,32,32,32,32,32,32,32,84,104,105,115,32,109,101,116, + 104,111,100,32,105,115,32,100,101,112,114,101,99,97,116,101, + 100,46,32,32,85,115,101,32,101,120,101,99,95,109,111,100, + 117,108,101,40,41,32,105,110,115,116,101,97,100,46,10,10, + 32,32,32,32,32,32,32,32,78,41,2,114,178,0,0,0, + 114,124,0,0,0,41,4,114,168,0,0,0,114,123,0,0, + 0,114,37,0,0,0,114,162,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,11,102,105,110,100, + 95,109,111,100,117,108,101,128,2,0,0,115,8,0,0,0, + 0,7,12,1,8,1,8,2,122,33,87,105,110,100,111,119, + 115,82,101,103,105,115,116,114,121,70,105,110,100,101,114,46, + 102,105,110,100,95,109,111,100,117,108,101,41,2,78,78,41, + 1,78,41,12,114,109,0,0,0,114,108,0,0,0,114,110, + 0,0,0,114,111,0,0,0,114,172,0,0,0,114,171,0, + 0,0,114,170,0,0,0,218,11,99,108,97,115,115,109,101, + 116,104,111,100,114,169,0,0,0,114,175,0,0,0,114,178, + 0,0,0,114,179,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,166,0,0, + 0,78,2,0,0,115,20,0,0,0,8,2,4,3,4,3, + 4,2,4,2,12,7,12,15,2,1,12,15,2,1,114,166, + 0,0,0,99,0,0,0,0,0,0,0,0,0,0,0,0, + 2,0,0,0,64,0,0,0,115,48,0,0,0,101,0,90, + 1,100,0,90,2,100,1,90,3,100,2,100,3,132,0,90, + 4,100,4,100,5,132,0,90,5,100,6,100,7,132,0,90, + 6,100,8,100,9,132,0,90,7,100,10,83,0,41,11,218, + 13,95,76,111,97,100,101,114,66,97,115,105,99,115,122,83, + 66,97,115,101,32,99,108,97,115,115,32,111,102,32,99,111, + 109,109,111,110,32,99,111,100,101,32,110,101,101,100,101,100, + 32,98,121,32,98,111,116,104,32,83,111,117,114,99,101,76, + 111,97,100,101,114,32,97,110,100,10,32,32,32,32,83,111, + 117,114,99,101,108,101,115,115,70,105,108,101,76,111,97,100, + 101,114,46,99,2,0,0,0,0,0,0,0,5,0,0,0, + 3,0,0,0,67,0,0,0,115,64,0,0,0,116,0,124, + 0,106,1,124,1,131,1,131,1,100,1,25,0,125,2,124, + 2,106,2,100,2,100,1,131,2,100,3,25,0,125,3,124, + 1,106,3,100,2,131,1,100,4,25,0,125,4,124,3,100, + 5,107,2,111,62,124,4,100,5,107,3,83,0,41,6,122, + 141,67,111,110,99,114,101,116,101,32,105,109,112,108,101,109, + 101,110,116,97,116,105,111,110,32,111,102,32,73,110,115,112, + 101,99,116,76,111,97,100,101,114,46,105,115,95,112,97,99, + 107,97,103,101,32,98,121,32,99,104,101,99,107,105,110,103, + 32,105,102,10,32,32,32,32,32,32,32,32,116,104,101,32, + 112,97,116,104,32,114,101,116,117,114,110,101,100,32,98,121, + 32,103,101,116,95,102,105,108,101,110,97,109,101,32,104,97, + 115,32,97,32,102,105,108,101,110,97,109,101,32,111,102,32, + 39,95,95,105,110,105,116,95,95,46,112,121,39,46,114,31, + 0,0,0,114,61,0,0,0,114,62,0,0,0,114,59,0, + 0,0,218,8,95,95,105,110,105,116,95,95,41,4,114,40, + 0,0,0,114,155,0,0,0,114,36,0,0,0,114,34,0, + 0,0,41,5,114,104,0,0,0,114,123,0,0,0,114,98, + 0,0,0,90,13,102,105,108,101,110,97,109,101,95,98,97, + 115,101,90,9,116,97,105,108,95,110,97,109,101,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,157,0,0, + 0,147,2,0,0,115,8,0,0,0,0,3,18,1,16,1, + 14,1,122,24,95,76,111,97,100,101,114,66,97,115,105,99, + 115,46,105,115,95,112,97,99,107,97,103,101,99,2,0,0, + 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, + 0,115,4,0,0,0,100,1,83,0,41,2,122,42,85,115, + 101,32,100,101,102,97,117,108,116,32,115,101,109,97,110,116, + 105,99,115,32,102,111,114,32,109,111,100,117,108,101,32,99, + 114,101,97,116,105,111,110,46,78,114,4,0,0,0,41,2, + 114,104,0,0,0,114,162,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,218,13,99,114,101,97,116, + 101,95,109,111,100,117,108,101,155,2,0,0,115,0,0,0, + 0,122,27,95,76,111,97,100,101,114,66,97,115,105,99,115, + 46,99,114,101,97,116,101,95,109,111,100,117,108,101,99,2, + 0,0,0,0,0,0,0,3,0,0,0,4,0,0,0,67, + 0,0,0,115,56,0,0,0,124,0,106,0,124,1,106,1, + 131,1,125,2,124,2,100,1,107,8,114,36,116,2,100,2, + 106,3,124,1,106,1,131,1,131,1,130,1,116,4,106,5, + 116,6,124,2,124,1,106,7,131,3,1,0,100,1,83,0, + 41,3,122,19,69,120,101,99,117,116,101,32,116,104,101,32, + 109,111,100,117,108,101,46,78,122,52,99,97,110,110,111,116, + 32,108,111,97,100,32,109,111,100,117,108,101,32,123,33,114, + 125,32,119,104,101,110,32,103,101,116,95,99,111,100,101,40, + 41,32,114,101,116,117,114,110,115,32,78,111,110,101,41,8, + 218,8,103,101,116,95,99,111,100,101,114,109,0,0,0,114, + 103,0,0,0,114,50,0,0,0,114,118,0,0,0,218,25, + 95,99,97,108,108,95,119,105,116,104,95,102,114,97,109,101, + 115,95,114,101,109,111,118,101,100,218,4,101,120,101,99,114, + 115,0,0,0,41,3,114,104,0,0,0,218,6,109,111,100, + 117,108,101,114,144,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,218,11,101,120,101,99,95,109,111, + 100,117,108,101,158,2,0,0,115,10,0,0,0,0,2,12, + 1,8,1,6,1,10,1,122,25,95,76,111,97,100,101,114, + 66,97,115,105,99,115,46,101,120,101,99,95,109,111,100,117, + 108,101,99,2,0,0,0,0,0,0,0,2,0,0,0,3, + 0,0,0,67,0,0,0,115,12,0,0,0,116,0,106,1, + 124,0,124,1,131,2,83,0,41,1,122,26,84,104,105,115, + 32,109,111,100,117,108,101,32,105,115,32,100,101,112,114,101, + 99,97,116,101,100,46,41,2,114,118,0,0,0,218,17,95, + 108,111,97,100,95,109,111,100,117,108,101,95,115,104,105,109, + 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,11,108,111,97, + 100,95,109,111,100,117,108,101,166,2,0,0,115,2,0,0, + 0,0,2,122,25,95,76,111,97,100,101,114,66,97,115,105, + 99,115,46,108,111,97,100,95,109,111,100,117,108,101,78,41, + 8,114,109,0,0,0,114,108,0,0,0,114,110,0,0,0, + 114,111,0,0,0,114,157,0,0,0,114,183,0,0,0,114, + 188,0,0,0,114,190,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,181,0, + 0,0,142,2,0,0,115,10,0,0,0,8,3,4,2,8, + 8,8,3,8,8,114,181,0,0,0,99,0,0,0,0,0, + 0,0,0,0,0,0,0,3,0,0,0,64,0,0,0,115, + 74,0,0,0,101,0,90,1,100,0,90,2,100,1,100,2, + 132,0,90,3,100,3,100,4,132,0,90,4,100,5,100,6, + 132,0,90,5,100,7,100,8,132,0,90,6,100,9,100,10, + 132,0,90,7,100,18,100,12,156,1,100,13,100,14,132,2, + 90,8,100,15,100,16,132,0,90,9,100,17,83,0,41,19, + 218,12,83,111,117,114,99,101,76,111,97,100,101,114,99,2, + 0,0,0,0,0,0,0,2,0,0,0,1,0,0,0,67, + 0,0,0,115,8,0,0,0,116,0,130,1,100,1,83,0, + 41,2,122,178,79,112,116,105,111,110,97,108,32,109,101,116, + 104,111,100,32,116,104,97,116,32,114,101,116,117,114,110,115, + 32,116,104,101,32,109,111,100,105,102,105,99,97,116,105,111, + 110,32,116,105,109,101,32,40,97,110,32,105,110,116,41,32, + 102,111,114,32,116,104,101,10,32,32,32,32,32,32,32,32, + 115,112,101,99,105,102,105,101,100,32,112,97,116,104,44,32, + 119,104,101,114,101,32,112,97,116,104,32,105,115,32,97,32, + 115,116,114,46,10,10,32,32,32,32,32,32,32,32,82,97, + 105,115,101,115,32,73,79,69,114,114,111,114,32,119,104,101, + 110,32,116,104,101,32,112,97,116,104,32,99,97,110,110,111, + 116,32,98,101,32,104,97,110,100,108,101,100,46,10,32,32, + 32,32,32,32,32,32,78,41,1,218,7,73,79,69,114,114, + 111,114,41,2,114,104,0,0,0,114,37,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,10,112, + 97,116,104,95,109,116,105,109,101,173,2,0,0,115,2,0, + 0,0,0,6,122,23,83,111,117,114,99,101,76,111,97,100, + 101,114,46,112,97,116,104,95,109,116,105,109,101,99,2,0, + 0,0,0,0,0,0,2,0,0,0,3,0,0,0,67,0, + 0,0,115,14,0,0,0,100,1,124,0,106,0,124,1,131, + 1,105,1,83,0,41,2,97,170,1,0,0,79,112,116,105, + 111,110,97,108,32,109,101,116,104,111,100,32,114,101,116,117, + 114,110,105,110,103,32,97,32,109,101,116,97,100,97,116,97, + 32,100,105,99,116,32,102,111,114,32,116,104,101,32,115,112, + 101,99,105,102,105,101,100,32,112,97,116,104,10,32,32,32, + 32,32,32,32,32,116,111,32,98,121,32,116,104,101,32,112, + 97,116,104,32,40,115,116,114,41,46,10,32,32,32,32,32, + 32,32,32,80,111,115,115,105,98,108,101,32,107,101,121,115, + 58,10,32,32,32,32,32,32,32,32,45,32,39,109,116,105, + 109,101,39,32,40,109,97,110,100,97,116,111,114,121,41,32, + 105,115,32,116,104,101,32,110,117,109,101,114,105,99,32,116, + 105,109,101,115,116,97,109,112,32,111,102,32,108,97,115,116, + 32,115,111,117,114,99,101,10,32,32,32,32,32,32,32,32, + 32,32,99,111,100,101,32,109,111,100,105,102,105,99,97,116, + 105,111,110,59,10,32,32,32,32,32,32,32,32,45,32,39, + 115,105,122,101,39,32,40,111,112,116,105,111,110,97,108,41, + 32,105,115,32,116,104,101,32,115,105,122,101,32,105,110,32, + 98,121,116,101,115,32,111,102,32,116,104,101,32,115,111,117, + 114,99,101,32,99,111,100,101,46,10,10,32,32,32,32,32, + 32,32,32,73,109,112,108,101,109,101,110,116,105,110,103,32, + 116,104,105,115,32,109,101,116,104,111,100,32,97,108,108,111, + 119,115,32,116,104,101,32,108,111,97,100,101,114,32,116,111, + 32,114,101,97,100,32,98,121,116,101,99,111,100,101,32,102, + 105,108,101,115,46,10,32,32,32,32,32,32,32,32,82,97, + 105,115,101,115,32,73,79,69,114,114,111,114,32,119,104,101, + 110,32,116,104,101,32,112,97,116,104,32,99,97,110,110,111, + 116,32,98,101,32,104,97,110,100,108,101,100,46,10,32,32, + 32,32,32,32,32,32,114,130,0,0,0,41,1,114,193,0, + 0,0,41,2,114,104,0,0,0,114,37,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,218,10,112, + 97,116,104,95,115,116,97,116,115,181,2,0,0,115,2,0, + 0,0,0,11,122,23,83,111,117,114,99,101,76,111,97,100, + 101,114,46,112,97,116,104,95,115,116,97,116,115,99,4,0, + 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, + 0,0,115,12,0,0,0,124,0,106,0,124,2,124,3,131, + 2,83,0,41,1,122,228,79,112,116,105,111,110,97,108,32, + 109,101,116,104,111,100,32,119,104,105,99,104,32,119,114,105, + 116,101,115,32,100,97,116,97,32,40,98,121,116,101,115,41, + 32,116,111,32,97,32,102,105,108,101,32,112,97,116,104,32, + 40,97,32,115,116,114,41,46,10,10,32,32,32,32,32,32, + 32,32,73,109,112,108,101,109,101,110,116,105,110,103,32,116, + 104,105,115,32,109,101,116,104,111,100,32,97,108,108,111,119, + 115,32,102,111,114,32,116,104,101,32,119,114,105,116,105,110, + 103,32,111,102,32,98,121,116,101,99,111,100,101,32,102,105, + 108,101,115,46,10,10,32,32,32,32,32,32,32,32,84,104, + 101,32,115,111,117,114,99,101,32,112,97,116,104,32,105,115, + 32,110,101,101,100,101,100,32,105,110,32,111,114,100,101,114, + 32,116,111,32,99,111,114,114,101,99,116,108,121,32,116,114, + 97,110,115,102,101,114,32,112,101,114,109,105,115,115,105,111, + 110,115,10,32,32,32,32,32,32,32,32,41,1,218,8,115, + 101,116,95,100,97,116,97,41,4,114,104,0,0,0,114,94, + 0,0,0,90,10,99,97,99,104,101,95,112,97,116,104,114, + 56,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,218,15,95,99,97,99,104,101,95,98,121,116,101, + 99,111,100,101,194,2,0,0,115,2,0,0,0,0,8,122, + 28,83,111,117,114,99,101,76,111,97,100,101,114,46,95,99, + 97,99,104,101,95,98,121,116,101,99,111,100,101,99,3,0, + 0,0,0,0,0,0,3,0,0,0,1,0,0,0,67,0, + 0,0,115,4,0,0,0,100,1,83,0,41,2,122,150,79, 112,116,105,111,110,97,108,32,109,101,116,104,111,100,32,119, 104,105,99,104,32,119,114,105,116,101,115,32,100,97,116,97, 32,40,98,121,116,101,115,41,32,116,111,32,97,32,102,105, @@ -1065,1376 +1088,1353 @@ 101,110,116,105,110,103,32,116,104,105,115,32,109,101,116,104, 111,100,32,97,108,108,111,119,115,32,102,111,114,32,116,104, 101,32,119,114,105,116,105,110,103,32,111,102,32,98,121,116, - 101,99,111,100,101,32,102,105,108,101,115,46,10,10,32,32, - 32,32,32,32,32,32,84,104,101,32,115,111,117,114,99,101, - 32,112,97,116,104,32,105,115,32,110,101,101,100,101,100,32, - 105,110,32,111,114,100,101,114,32,116,111,32,99,111,114,114, - 101,99,116,108,121,32,116,114,97,110,115,102,101,114,32,112, - 101,114,109,105,115,115,105,111,110,115,10,32,32,32,32,32, - 32,32,32,41,1,218,8,115,101,116,95,100,97,116,97,41, - 4,114,104,0,0,0,114,94,0,0,0,90,10,99,97,99, - 104,101,95,112,97,116,104,114,56,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,15,95,99,97, - 99,104,101,95,98,121,116,101,99,111,100,101,194,2,0,0, - 115,2,0,0,0,0,8,122,28,83,111,117,114,99,101,76, - 111,97,100,101,114,46,95,99,97,99,104,101,95,98,121,116, - 101,99,111,100,101,99,3,0,0,0,0,0,0,0,3,0, - 0,0,1,0,0,0,67,0,0,0,115,4,0,0,0,100, - 1,83,0,41,2,122,150,79,112,116,105,111,110,97,108,32, - 109,101,116,104,111,100,32,119,104,105,99,104,32,119,114,105, - 116,101,115,32,100,97,116,97,32,40,98,121,116,101,115,41, - 32,116,111,32,97,32,102,105,108,101,32,112,97,116,104,32, - 40,97,32,115,116,114,41,46,10,10,32,32,32,32,32,32, - 32,32,73,109,112,108,101,109,101,110,116,105,110,103,32,116, - 104,105,115,32,109,101,116,104,111,100,32,97,108,108,111,119, - 115,32,102,111,114,32,116,104,101,32,119,114,105,116,105,110, - 103,32,111,102,32,98,121,116,101,99,111,100,101,32,102,105, - 108,101,115,46,10,32,32,32,32,32,32,32,32,78,114,4, - 0,0,0,41,3,114,104,0,0,0,114,37,0,0,0,114, - 56,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,195,0,0,0,204,2,0,0,115,0,0,0, - 0,122,21,83,111,117,114,99,101,76,111,97,100,101,114,46, - 115,101,116,95,100,97,116,97,99,2,0,0,0,0,0,0, - 0,5,0,0,0,16,0,0,0,67,0,0,0,115,82,0, - 0,0,124,0,106,0,124,1,131,1,125,2,121,14,124,0, - 106,1,124,2,131,1,125,3,87,0,110,48,4,0,116,2, - 107,10,114,72,1,0,125,4,1,0,122,20,116,3,100,1, - 124,1,100,2,141,2,124,4,130,2,87,0,89,0,100,3, - 100,3,125,4,126,4,88,0,110,2,88,0,116,4,124,3, - 131,1,83,0,41,4,122,52,67,111,110,99,114,101,116,101, - 32,105,109,112,108,101,109,101,110,116,97,116,105,111,110,32, - 111,102,32,73,110,115,112,101,99,116,76,111,97,100,101,114, - 46,103,101,116,95,115,111,117,114,99,101,46,122,39,115,111, - 117,114,99,101,32,110,111,116,32,97,118,97,105,108,97,98, - 108,101,32,116,104,114,111,117,103,104,32,103,101,116,95,100, - 97,116,97,40,41,41,1,114,102,0,0,0,78,41,5,114, - 155,0,0,0,218,8,103,101,116,95,100,97,116,97,114,42, - 0,0,0,114,103,0,0,0,114,153,0,0,0,41,5,114, - 104,0,0,0,114,123,0,0,0,114,37,0,0,0,114,151, - 0,0,0,218,3,101,120,99,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,218,10,103,101,116,95,115,111,117, - 114,99,101,211,2,0,0,115,14,0,0,0,0,2,10,1, - 2,1,14,1,16,1,4,1,28,1,122,23,83,111,117,114, - 99,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, - 114,99,101,114,31,0,0,0,41,1,218,9,95,111,112,116, - 105,109,105,122,101,99,3,0,0,0,1,0,0,0,4,0, - 0,0,8,0,0,0,67,0,0,0,115,22,0,0,0,116, - 0,106,1,116,2,124,1,124,2,100,1,100,2,124,3,100, - 3,141,6,83,0,41,4,122,130,82,101,116,117,114,110,32, - 116,104,101,32,99,111,100,101,32,111,98,106,101,99,116,32, - 99,111,109,112,105,108,101,100,32,102,114,111,109,32,115,111, - 117,114,99,101,46,10,10,32,32,32,32,32,32,32,32,84, - 104,101,32,39,100,97,116,97,39,32,97,114,103,117,109,101, - 110,116,32,99,97,110,32,98,101,32,97,110,121,32,111,98, - 106,101,99,116,32,116,121,112,101,32,116,104,97,116,32,99, - 111,109,112,105,108,101,40,41,32,115,117,112,112,111,114,116, - 115,46,10,32,32,32,32,32,32,32,32,114,186,0,0,0, - 84,41,2,218,12,100,111,110,116,95,105,110,104,101,114,105, - 116,114,72,0,0,0,41,3,114,118,0,0,0,114,185,0, - 0,0,218,7,99,111,109,112,105,108,101,41,4,114,104,0, - 0,0,114,56,0,0,0,114,37,0,0,0,114,200,0,0, + 101,99,111,100,101,32,102,105,108,101,115,46,10,32,32,32, + 32,32,32,32,32,78,114,4,0,0,0,41,3,114,104,0, + 0,0,114,37,0,0,0,114,56,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,195,0,0,0, + 204,2,0,0,115,0,0,0,0,122,21,83,111,117,114,99, + 101,76,111,97,100,101,114,46,115,101,116,95,100,97,116,97, + 99,2,0,0,0,0,0,0,0,5,0,0,0,16,0,0, + 0,67,0,0,0,115,82,0,0,0,124,0,106,0,124,1, + 131,1,125,2,121,14,124,0,106,1,124,2,131,1,125,3, + 87,0,110,48,4,0,116,2,107,10,114,72,1,0,125,4, + 1,0,122,20,116,3,100,1,124,1,100,2,141,2,124,4, + 130,2,87,0,89,0,100,3,100,3,125,4,126,4,88,0, + 110,2,88,0,116,4,124,3,131,1,83,0,41,4,122,52, + 67,111,110,99,114,101,116,101,32,105,109,112,108,101,109,101, + 110,116,97,116,105,111,110,32,111,102,32,73,110,115,112,101, + 99,116,76,111,97,100,101,114,46,103,101,116,95,115,111,117, + 114,99,101,46,122,39,115,111,117,114,99,101,32,110,111,116, + 32,97,118,97,105,108,97,98,108,101,32,116,104,114,111,117, + 103,104,32,103,101,116,95,100,97,116,97,40,41,41,1,114, + 102,0,0,0,78,41,5,114,155,0,0,0,218,8,103,101, + 116,95,100,97,116,97,114,42,0,0,0,114,103,0,0,0, + 114,153,0,0,0,41,5,114,104,0,0,0,114,123,0,0, + 0,114,37,0,0,0,114,151,0,0,0,218,3,101,120,99, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, + 10,103,101,116,95,115,111,117,114,99,101,211,2,0,0,115, + 14,0,0,0,0,2,10,1,2,1,14,1,16,1,4,1, + 28,1,122,23,83,111,117,114,99,101,76,111,97,100,101,114, + 46,103,101,116,95,115,111,117,114,99,101,114,31,0,0,0, + 41,1,218,9,95,111,112,116,105,109,105,122,101,99,3,0, + 0,0,1,0,0,0,4,0,0,0,8,0,0,0,67,0, + 0,0,115,22,0,0,0,116,0,106,1,116,2,124,1,124, + 2,100,1,100,2,124,3,100,3,141,6,83,0,41,4,122, + 130,82,101,116,117,114,110,32,116,104,101,32,99,111,100,101, + 32,111,98,106,101,99,116,32,99,111,109,112,105,108,101,100, + 32,102,114,111,109,32,115,111,117,114,99,101,46,10,10,32, + 32,32,32,32,32,32,32,84,104,101,32,39,100,97,116,97, + 39,32,97,114,103,117,109,101,110,116,32,99,97,110,32,98, + 101,32,97,110,121,32,111,98,106,101,99,116,32,116,121,112, + 101,32,116,104,97,116,32,99,111,109,112,105,108,101,40,41, + 32,115,117,112,112,111,114,116,115,46,10,32,32,32,32,32, + 32,32,32,114,186,0,0,0,84,41,2,218,12,100,111,110, + 116,95,105,110,104,101,114,105,116,114,72,0,0,0,41,3, + 114,118,0,0,0,114,185,0,0,0,218,7,99,111,109,112, + 105,108,101,41,4,114,104,0,0,0,114,56,0,0,0,114, + 37,0,0,0,114,200,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,218,14,115,111,117,114,99,101, + 95,116,111,95,99,111,100,101,221,2,0,0,115,4,0,0, + 0,0,5,12,1,122,27,83,111,117,114,99,101,76,111,97, + 100,101,114,46,115,111,117,114,99,101,95,116,111,95,99,111, + 100,101,99,2,0,0,0,0,0,0,0,10,0,0,0,43, + 0,0,0,67,0,0,0,115,94,1,0,0,124,0,106,0, + 124,1,131,1,125,2,100,1,125,3,121,12,116,1,124,2, + 131,1,125,4,87,0,110,24,4,0,116,2,107,10,114,50, + 1,0,1,0,1,0,100,1,125,4,89,0,110,162,88,0, + 121,14,124,0,106,3,124,2,131,1,125,5,87,0,110,20, + 4,0,116,4,107,10,114,86,1,0,1,0,1,0,89,0, + 110,126,88,0,116,5,124,5,100,2,25,0,131,1,125,3, + 121,14,124,0,106,6,124,4,131,1,125,6,87,0,110,20, + 4,0,116,7,107,10,114,134,1,0,1,0,1,0,89,0, + 110,78,88,0,121,20,116,8,124,6,124,5,124,1,124,4, + 100,3,141,4,125,7,87,0,110,24,4,0,116,9,116,10, + 102,2,107,10,114,180,1,0,1,0,1,0,89,0,110,32, + 88,0,116,11,106,12,100,4,124,4,124,2,131,3,1,0, + 116,13,124,7,124,1,124,4,124,2,100,5,141,4,83,0, + 124,0,106,6,124,2,131,1,125,8,124,0,106,14,124,8, + 124,2,131,2,125,9,116,11,106,12,100,6,124,2,131,2, + 1,0,116,15,106,16,12,0,144,1,114,90,124,4,100,1, + 107,9,144,1,114,90,124,3,100,1,107,9,144,1,114,90, + 116,17,124,9,124,3,116,18,124,8,131,1,131,3,125,6, + 121,30,124,0,106,19,124,2,124,4,124,6,131,3,1,0, + 116,11,106,12,100,7,124,4,131,2,1,0,87,0,110,22, + 4,0,116,2,107,10,144,1,114,88,1,0,1,0,1,0, + 89,0,110,2,88,0,124,9,83,0,41,8,122,190,67,111, + 110,99,114,101,116,101,32,105,109,112,108,101,109,101,110,116, + 97,116,105,111,110,32,111,102,32,73,110,115,112,101,99,116, + 76,111,97,100,101,114,46,103,101,116,95,99,111,100,101,46, + 10,10,32,32,32,32,32,32,32,32,82,101,97,100,105,110, + 103,32,111,102,32,98,121,116,101,99,111,100,101,32,114,101, + 113,117,105,114,101,115,32,112,97,116,104,95,115,116,97,116, + 115,32,116,111,32,98,101,32,105,109,112,108,101,109,101,110, + 116,101,100,46,32,84,111,32,119,114,105,116,101,10,32,32, + 32,32,32,32,32,32,98,121,116,101,99,111,100,101,44,32, + 115,101,116,95,100,97,116,97,32,109,117,115,116,32,97,108, + 115,111,32,98,101,32,105,109,112,108,101,109,101,110,116,101, + 100,46,10,10,32,32,32,32,32,32,32,32,78,114,130,0, + 0,0,41,3,114,136,0,0,0,114,102,0,0,0,114,37, + 0,0,0,122,13,123,125,32,109,97,116,99,104,101,115,32, + 123,125,41,3,114,102,0,0,0,114,93,0,0,0,114,94, + 0,0,0,122,19,99,111,100,101,32,111,98,106,101,99,116, + 32,102,114,111,109,32,123,125,122,10,119,114,111,116,101,32, + 123,33,114,125,41,20,114,155,0,0,0,114,83,0,0,0, + 114,70,0,0,0,114,194,0,0,0,114,192,0,0,0,114, + 16,0,0,0,114,197,0,0,0,114,42,0,0,0,114,139, + 0,0,0,114,103,0,0,0,114,134,0,0,0,114,118,0, + 0,0,114,133,0,0,0,114,145,0,0,0,114,203,0,0, + 0,114,8,0,0,0,218,19,100,111,110,116,95,119,114,105, + 116,101,95,98,121,116,101,99,111,100,101,114,148,0,0,0, + 114,33,0,0,0,114,196,0,0,0,41,10,114,104,0,0, + 0,114,123,0,0,0,114,94,0,0,0,114,137,0,0,0, + 114,93,0,0,0,218,2,115,116,114,56,0,0,0,218,10, + 98,121,116,101,115,95,100,97,116,97,114,151,0,0,0,90, + 11,99,111,100,101,95,111,98,106,101,99,116,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,184,0,0,0, + 229,2,0,0,115,78,0,0,0,0,7,10,1,4,1,2, + 1,12,1,14,1,10,2,2,1,14,1,14,1,6,2,12, + 1,2,1,14,1,14,1,6,2,2,1,4,1,4,1,12, + 1,18,1,6,2,8,1,6,1,6,1,2,1,8,1,10, + 1,12,1,12,1,20,1,10,1,6,1,10,1,2,1,14, + 1,16,1,16,1,6,1,122,21,83,111,117,114,99,101,76, + 111,97,100,101,114,46,103,101,116,95,99,111,100,101,78,114, + 91,0,0,0,41,10,114,109,0,0,0,114,108,0,0,0, + 114,110,0,0,0,114,193,0,0,0,114,194,0,0,0,114, + 196,0,0,0,114,195,0,0,0,114,199,0,0,0,114,203, + 0,0,0,114,184,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,191,0,0, + 0,171,2,0,0,115,14,0,0,0,8,2,8,8,8,13, + 8,10,8,7,8,10,14,8,114,191,0,0,0,99,0,0, + 0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0, + 0,0,115,80,0,0,0,101,0,90,1,100,0,90,2,100, + 1,90,3,100,2,100,3,132,0,90,4,100,4,100,5,132, + 0,90,5,100,6,100,7,132,0,90,6,101,7,135,0,102, + 1,100,8,100,9,132,8,131,1,90,8,101,7,100,10,100, + 11,132,0,131,1,90,9,100,12,100,13,132,0,90,10,135, + 0,90,11,100,14,83,0,41,15,218,10,70,105,108,101,76, + 111,97,100,101,114,122,103,66,97,115,101,32,102,105,108,101, + 32,108,111,97,100,101,114,32,99,108,97,115,115,32,119,104, + 105,99,104,32,105,109,112,108,101,109,101,110,116,115,32,116, + 104,101,32,108,111,97,100,101,114,32,112,114,111,116,111,99, + 111,108,32,109,101,116,104,111,100,115,32,116,104,97,116,10, + 32,32,32,32,114,101,113,117,105,114,101,32,102,105,108,101, + 32,115,121,115,116,101,109,32,117,115,97,103,101,46,99,3, + 0,0,0,0,0,0,0,3,0,0,0,2,0,0,0,67, + 0,0,0,115,16,0,0,0,124,1,124,0,95,0,124,2, + 124,0,95,1,100,1,83,0,41,2,122,75,67,97,99,104, + 101,32,116,104,101,32,109,111,100,117,108,101,32,110,97,109, + 101,32,97,110,100,32,116,104,101,32,112,97,116,104,32,116, + 111,32,116,104,101,32,102,105,108,101,32,102,111,117,110,100, + 32,98,121,32,116,104,101,10,32,32,32,32,32,32,32,32, + 102,105,110,100,101,114,46,78,41,2,114,102,0,0,0,114, + 37,0,0,0,41,3,114,104,0,0,0,114,123,0,0,0, + 114,37,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,182,0,0,0,30,3,0,0,115,4,0, + 0,0,0,3,6,1,122,19,70,105,108,101,76,111,97,100, + 101,114,46,95,95,105,110,105,116,95,95,99,2,0,0,0, + 0,0,0,0,2,0,0,0,2,0,0,0,67,0,0,0, + 115,24,0,0,0,124,0,106,0,124,1,106,0,107,2,111, + 22,124,0,106,1,124,1,106,1,107,2,83,0,41,1,78, + 41,2,218,9,95,95,99,108,97,115,115,95,95,114,115,0, + 0,0,41,2,114,104,0,0,0,218,5,111,116,104,101,114, + 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, + 6,95,95,101,113,95,95,36,3,0,0,115,4,0,0,0, + 0,1,12,1,122,17,70,105,108,101,76,111,97,100,101,114, + 46,95,95,101,113,95,95,99,1,0,0,0,0,0,0,0, + 1,0,0,0,3,0,0,0,67,0,0,0,115,20,0,0, + 0,116,0,124,0,106,1,131,1,116,0,124,0,106,2,131, + 1,65,0,83,0,41,1,78,41,3,218,4,104,97,115,104, + 114,102,0,0,0,114,37,0,0,0,41,1,114,104,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 218,14,115,111,117,114,99,101,95,116,111,95,99,111,100,101, - 221,2,0,0,115,4,0,0,0,0,5,12,1,122,27,83, - 111,117,114,99,101,76,111,97,100,101,114,46,115,111,117,114, - 99,101,95,116,111,95,99,111,100,101,99,2,0,0,0,0, - 0,0,0,10,0,0,0,43,0,0,0,67,0,0,0,115, - 94,1,0,0,124,0,106,0,124,1,131,1,125,2,100,1, - 125,3,121,12,116,1,124,2,131,1,125,4,87,0,110,24, - 4,0,116,2,107,10,114,50,1,0,1,0,1,0,100,1, - 125,4,89,0,110,162,88,0,121,14,124,0,106,3,124,2, - 131,1,125,5,87,0,110,20,4,0,116,4,107,10,114,86, - 1,0,1,0,1,0,89,0,110,126,88,0,116,5,124,5, - 100,2,25,0,131,1,125,3,121,14,124,0,106,6,124,4, - 131,1,125,6,87,0,110,20,4,0,116,7,107,10,114,134, - 1,0,1,0,1,0,89,0,110,78,88,0,121,20,116,8, - 124,6,124,5,124,1,124,4,100,3,141,4,125,7,87,0, - 110,24,4,0,116,9,116,10,102,2,107,10,114,180,1,0, - 1,0,1,0,89,0,110,32,88,0,116,11,106,12,100,4, - 124,4,124,2,131,3,1,0,116,13,124,7,124,1,124,4, - 124,2,100,5,141,4,83,0,124,0,106,6,124,2,131,1, - 125,8,124,0,106,14,124,8,124,2,131,2,125,9,116,11, - 106,12,100,6,124,2,131,2,1,0,116,15,106,16,12,0, - 144,1,114,90,124,4,100,1,107,9,144,1,114,90,124,3, - 100,1,107,9,144,1,114,90,116,17,124,9,124,3,116,18, - 124,8,131,1,131,3,125,6,121,30,124,0,106,19,124,2, - 124,4,124,6,131,3,1,0,116,11,106,12,100,7,124,4, - 131,2,1,0,87,0,110,22,4,0,116,2,107,10,144,1, - 114,88,1,0,1,0,1,0,89,0,110,2,88,0,124,9, - 83,0,41,8,122,190,67,111,110,99,114,101,116,101,32,105, - 109,112,108,101,109,101,110,116,97,116,105,111,110,32,111,102, - 32,73,110,115,112,101,99,116,76,111,97,100,101,114,46,103, - 101,116,95,99,111,100,101,46,10,10,32,32,32,32,32,32, - 32,32,82,101,97,100,105,110,103,32,111,102,32,98,121,116, - 101,99,111,100,101,32,114,101,113,117,105,114,101,115,32,112, - 97,116,104,95,115,116,97,116,115,32,116,111,32,98,101,32, - 105,109,112,108,101,109,101,110,116,101,100,46,32,84,111,32, - 119,114,105,116,101,10,32,32,32,32,32,32,32,32,98,121, - 116,101,99,111,100,101,44,32,115,101,116,95,100,97,116,97, - 32,109,117,115,116,32,97,108,115,111,32,98,101,32,105,109, - 112,108,101,109,101,110,116,101,100,46,10,10,32,32,32,32, - 32,32,32,32,78,114,130,0,0,0,41,3,114,136,0,0, - 0,114,102,0,0,0,114,37,0,0,0,122,13,123,125,32, - 109,97,116,99,104,101,115,32,123,125,41,3,114,102,0,0, - 0,114,93,0,0,0,114,94,0,0,0,122,19,99,111,100, - 101,32,111,98,106,101,99,116,32,102,114,111,109,32,123,125, - 122,10,119,114,111,116,101,32,123,33,114,125,41,20,114,155, - 0,0,0,114,83,0,0,0,114,70,0,0,0,114,194,0, - 0,0,114,192,0,0,0,114,16,0,0,0,114,197,0,0, - 0,114,42,0,0,0,114,139,0,0,0,114,103,0,0,0, - 114,134,0,0,0,114,118,0,0,0,114,133,0,0,0,114, - 145,0,0,0,114,203,0,0,0,114,8,0,0,0,218,19, - 100,111,110,116,95,119,114,105,116,101,95,98,121,116,101,99, - 111,100,101,114,148,0,0,0,114,33,0,0,0,114,196,0, - 0,0,41,10,114,104,0,0,0,114,123,0,0,0,114,94, - 0,0,0,114,137,0,0,0,114,93,0,0,0,218,2,115, - 116,114,56,0,0,0,218,10,98,121,116,101,115,95,100,97, - 116,97,114,151,0,0,0,90,11,99,111,100,101,95,111,98, - 106,101,99,116,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,184,0,0,0,229,2,0,0,115,78,0,0, - 0,0,7,10,1,4,1,2,1,12,1,14,1,10,2,2, - 1,14,1,14,1,6,2,12,1,2,1,14,1,14,1,6, - 2,2,1,4,1,4,1,12,1,18,1,6,2,8,1,6, - 1,6,1,2,1,8,1,10,1,12,1,12,1,20,1,10, - 1,6,1,10,1,2,1,14,1,16,1,16,1,6,1,122, - 21,83,111,117,114,99,101,76,111,97,100,101,114,46,103,101, - 116,95,99,111,100,101,78,114,91,0,0,0,41,10,114,109, - 0,0,0,114,108,0,0,0,114,110,0,0,0,114,193,0, + 218,8,95,95,104,97,115,104,95,95,40,3,0,0,115,2, + 0,0,0,0,1,122,19,70,105,108,101,76,111,97,100,101, + 114,46,95,95,104,97,115,104,95,95,99,2,0,0,0,0, + 0,0,0,2,0,0,0,3,0,0,0,3,0,0,0,115, + 16,0,0,0,116,0,116,1,124,0,131,2,106,2,124,1, + 131,1,83,0,41,1,122,100,76,111,97,100,32,97,32,109, + 111,100,117,108,101,32,102,114,111,109,32,97,32,102,105,108, + 101,46,10,10,32,32,32,32,32,32,32,32,84,104,105,115, + 32,109,101,116,104,111,100,32,105,115,32,100,101,112,114,101, + 99,97,116,101,100,46,32,32,85,115,101,32,101,120,101,99, + 95,109,111,100,117,108,101,40,41,32,105,110,115,116,101,97, + 100,46,10,10,32,32,32,32,32,32,32,32,41,3,218,5, + 115,117,112,101,114,114,207,0,0,0,114,190,0,0,0,41, + 2,114,104,0,0,0,114,123,0,0,0,41,1,114,208,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,190,0,0, + 0,43,3,0,0,115,2,0,0,0,0,10,122,22,70,105, + 108,101,76,111,97,100,101,114,46,108,111,97,100,95,109,111, + 100,117,108,101,99,2,0,0,0,0,0,0,0,2,0,0, + 0,1,0,0,0,67,0,0,0,115,6,0,0,0,124,0, + 106,0,83,0,41,1,122,58,82,101,116,117,114,110,32,116, + 104,101,32,112,97,116,104,32,116,111,32,116,104,101,32,115, + 111,117,114,99,101,32,102,105,108,101,32,97,115,32,102,111, + 117,110,100,32,98,121,32,116,104,101,32,102,105,110,100,101, + 114,46,41,1,114,37,0,0,0,41,2,114,104,0,0,0, + 114,123,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,155,0,0,0,55,3,0,0,115,2,0, + 0,0,0,3,122,23,70,105,108,101,76,111,97,100,101,114, + 46,103,101,116,95,102,105,108,101,110,97,109,101,99,2,0, + 0,0,0,0,0,0,3,0,0,0,9,0,0,0,67,0, + 0,0,115,32,0,0,0,116,0,106,1,124,1,100,1,131, + 2,143,10,125,2,124,2,106,2,131,0,83,0,81,0,82, + 0,88,0,100,2,83,0,41,3,122,39,82,101,116,117,114, + 110,32,116,104,101,32,100,97,116,97,32,102,114,111,109,32, + 112,97,116,104,32,97,115,32,114,97,119,32,98,121,116,101, + 115,46,218,1,114,78,41,3,114,52,0,0,0,114,53,0, + 0,0,90,4,114,101,97,100,41,3,114,104,0,0,0,114, + 37,0,0,0,114,57,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,197,0,0,0,60,3,0, + 0,115,4,0,0,0,0,2,14,1,122,19,70,105,108,101, + 76,111,97,100,101,114,46,103,101,116,95,100,97,116,97,78, + 41,12,114,109,0,0,0,114,108,0,0,0,114,110,0,0, + 0,114,111,0,0,0,114,182,0,0,0,114,210,0,0,0, + 114,212,0,0,0,114,120,0,0,0,114,190,0,0,0,114, + 155,0,0,0,114,197,0,0,0,90,13,95,95,99,108,97, + 115,115,99,101,108,108,95,95,114,4,0,0,0,114,4,0, + 0,0,41,1,114,208,0,0,0,114,6,0,0,0,114,207, + 0,0,0,25,3,0,0,115,14,0,0,0,8,3,4,2, + 8,6,8,4,8,3,16,12,12,5,114,207,0,0,0,99, + 0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0, + 64,0,0,0,115,46,0,0,0,101,0,90,1,100,0,90, + 2,100,1,90,3,100,2,100,3,132,0,90,4,100,4,100, + 5,132,0,90,5,100,6,100,7,156,1,100,8,100,9,132, + 2,90,6,100,10,83,0,41,11,218,16,83,111,117,114,99, + 101,70,105,108,101,76,111,97,100,101,114,122,62,67,111,110, + 99,114,101,116,101,32,105,109,112,108,101,109,101,110,116,97, + 116,105,111,110,32,111,102,32,83,111,117,114,99,101,76,111, + 97,100,101,114,32,117,115,105,110,103,32,116,104,101,32,102, + 105,108,101,32,115,121,115,116,101,109,46,99,2,0,0,0, + 0,0,0,0,3,0,0,0,3,0,0,0,67,0,0,0, + 115,22,0,0,0,116,0,124,1,131,1,125,2,124,2,106, + 1,124,2,106,2,100,1,156,2,83,0,41,2,122,33,82, + 101,116,117,114,110,32,116,104,101,32,109,101,116,97,100,97, + 116,97,32,102,111,114,32,116,104,101,32,112,97,116,104,46, + 41,2,122,5,109,116,105,109,101,122,4,115,105,122,101,41, + 3,114,41,0,0,0,218,8,115,116,95,109,116,105,109,101, + 90,7,115,116,95,115,105,122,101,41,3,114,104,0,0,0, + 114,37,0,0,0,114,205,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,194,0,0,0,70,3, + 0,0,115,4,0,0,0,0,2,8,1,122,27,83,111,117, + 114,99,101,70,105,108,101,76,111,97,100,101,114,46,112,97, + 116,104,95,115,116,97,116,115,99,4,0,0,0,0,0,0, + 0,5,0,0,0,5,0,0,0,67,0,0,0,115,24,0, + 0,0,116,0,124,1,131,1,125,4,124,0,106,1,124,2, + 124,3,124,4,100,1,141,3,83,0,41,2,78,41,1,218, + 5,95,109,111,100,101,41,2,114,101,0,0,0,114,195,0, + 0,0,41,5,114,104,0,0,0,114,94,0,0,0,114,93, + 0,0,0,114,56,0,0,0,114,44,0,0,0,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,196,0,0, + 0,75,3,0,0,115,4,0,0,0,0,2,8,1,122,32, + 83,111,117,114,99,101,70,105,108,101,76,111,97,100,101,114, + 46,95,99,97,99,104,101,95,98,121,116,101,99,111,100,101, + 105,182,1,0,0,41,1,114,217,0,0,0,99,3,0,0, + 0,1,0,0,0,9,0,0,0,17,0,0,0,67,0,0, + 0,115,250,0,0,0,116,0,124,1,131,1,92,2,125,4, + 125,5,103,0,125,6,120,40,124,4,114,56,116,1,124,4, + 131,1,12,0,114,56,116,0,124,4,131,1,92,2,125,4, + 125,7,124,6,106,2,124,7,131,1,1,0,113,18,87,0, + 120,108,116,3,124,6,131,1,68,0,93,96,125,7,116,4, + 124,4,124,7,131,2,125,4,121,14,116,5,106,6,124,4, + 131,1,1,0,87,0,113,68,4,0,116,7,107,10,114,118, + 1,0,1,0,1,0,119,68,89,0,113,68,4,0,116,8, + 107,10,114,162,1,0,125,8,1,0,122,18,116,9,106,10, + 100,1,124,4,124,8,131,3,1,0,100,2,83,0,100,2, + 125,8,126,8,88,0,113,68,88,0,113,68,87,0,121,28, + 116,11,124,1,124,2,124,3,131,3,1,0,116,9,106,10, + 100,3,124,1,131,2,1,0,87,0,110,48,4,0,116,8, + 107,10,114,244,1,0,125,8,1,0,122,20,116,9,106,10, + 100,1,124,1,124,8,131,3,1,0,87,0,89,0,100,2, + 100,2,125,8,126,8,88,0,110,2,88,0,100,2,83,0, + 41,4,122,27,87,114,105,116,101,32,98,121,116,101,115,32, + 100,97,116,97,32,116,111,32,97,32,102,105,108,101,46,122, + 27,99,111,117,108,100,32,110,111,116,32,99,114,101,97,116, + 101,32,123,33,114,125,58,32,123,33,114,125,78,122,12,99, + 114,101,97,116,101,100,32,123,33,114,125,41,12,114,40,0, + 0,0,114,48,0,0,0,114,161,0,0,0,114,35,0,0, + 0,114,30,0,0,0,114,3,0,0,0,90,5,109,107,100, + 105,114,218,15,70,105,108,101,69,120,105,115,116,115,69,114, + 114,111,114,114,42,0,0,0,114,118,0,0,0,114,133,0, + 0,0,114,58,0,0,0,41,9,114,104,0,0,0,114,37, + 0,0,0,114,56,0,0,0,114,217,0,0,0,218,6,112, + 97,114,101,110,116,114,98,0,0,0,114,29,0,0,0,114, + 25,0,0,0,114,198,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,195,0,0,0,80,3,0, + 0,115,42,0,0,0,0,2,12,1,4,2,16,1,12,1, + 14,2,14,1,10,1,2,1,14,1,14,2,6,1,16,3, + 6,1,8,1,20,1,2,1,12,1,16,1,16,2,8,1, + 122,25,83,111,117,114,99,101,70,105,108,101,76,111,97,100, + 101,114,46,115,101,116,95,100,97,116,97,78,41,7,114,109, + 0,0,0,114,108,0,0,0,114,110,0,0,0,114,111,0, 0,0,114,194,0,0,0,114,196,0,0,0,114,195,0,0, - 0,114,199,0,0,0,114,203,0,0,0,114,184,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,191,0,0,0,171,2,0,0,115,14,0, - 0,0,8,2,8,8,8,13,8,10,8,7,8,10,14,8, - 114,191,0,0,0,99,0,0,0,0,0,0,0,0,0,0, - 0,0,4,0,0,0,0,0,0,0,115,80,0,0,0,101, - 0,90,1,100,0,90,2,100,1,90,3,100,2,100,3,132, - 0,90,4,100,4,100,5,132,0,90,5,100,6,100,7,132, - 0,90,6,101,7,135,0,102,1,100,8,100,9,132,8,131, - 1,90,8,101,7,100,10,100,11,132,0,131,1,90,9,100, - 12,100,13,132,0,90,10,135,0,90,11,100,14,83,0,41, - 15,218,10,70,105,108,101,76,111,97,100,101,114,122,103,66, - 97,115,101,32,102,105,108,101,32,108,111,97,100,101,114,32, - 99,108,97,115,115,32,119,104,105,99,104,32,105,109,112,108, - 101,109,101,110,116,115,32,116,104,101,32,108,111,97,100,101, - 114,32,112,114,111,116,111,99,111,108,32,109,101,116,104,111, - 100,115,32,116,104,97,116,10,32,32,32,32,114,101,113,117, - 105,114,101,32,102,105,108,101,32,115,121,115,116,101,109,32, - 117,115,97,103,101,46,99,3,0,0,0,0,0,0,0,3, - 0,0,0,2,0,0,0,67,0,0,0,115,16,0,0,0, - 124,1,124,0,95,0,124,2,124,0,95,1,100,1,83,0, - 41,2,122,75,67,97,99,104,101,32,116,104,101,32,109,111, - 100,117,108,101,32,110,97,109,101,32,97,110,100,32,116,104, - 101,32,112,97,116,104,32,116,111,32,116,104,101,32,102,105, - 108,101,32,102,111,117,110,100,32,98,121,32,116,104,101,10, - 32,32,32,32,32,32,32,32,102,105,110,100,101,114,46,78, - 41,2,114,102,0,0,0,114,37,0,0,0,41,3,114,104, - 0,0,0,114,123,0,0,0,114,37,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,182,0,0, - 0,30,3,0,0,115,4,0,0,0,0,3,6,1,122,19, - 70,105,108,101,76,111,97,100,101,114,46,95,95,105,110,105, - 116,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, - 2,0,0,0,67,0,0,0,115,24,0,0,0,124,0,106, - 0,124,1,106,0,107,2,111,22,124,0,106,1,124,1,106, - 1,107,2,83,0,41,1,78,41,2,218,9,95,95,99,108, - 97,115,115,95,95,114,115,0,0,0,41,2,114,104,0,0, - 0,218,5,111,116,104,101,114,114,4,0,0,0,114,4,0, - 0,0,114,6,0,0,0,218,6,95,95,101,113,95,95,36, - 3,0,0,115,4,0,0,0,0,1,12,1,122,17,70,105, - 108,101,76,111,97,100,101,114,46,95,95,101,113,95,95,99, - 1,0,0,0,0,0,0,0,1,0,0,0,3,0,0,0, - 67,0,0,0,115,20,0,0,0,116,0,124,0,106,1,131, - 1,116,0,124,0,106,2,131,1,65,0,83,0,41,1,78, - 41,3,218,4,104,97,115,104,114,102,0,0,0,114,37,0, - 0,0,41,1,114,104,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,218,8,95,95,104,97,115,104, - 95,95,40,3,0,0,115,2,0,0,0,0,1,122,19,70, - 105,108,101,76,111,97,100,101,114,46,95,95,104,97,115,104, - 95,95,99,2,0,0,0,0,0,0,0,2,0,0,0,3, - 0,0,0,3,0,0,0,115,16,0,0,0,116,0,116,1, - 124,0,131,2,106,2,124,1,131,1,83,0,41,1,122,100, - 76,111,97,100,32,97,32,109,111,100,117,108,101,32,102,114, - 111,109,32,97,32,102,105,108,101,46,10,10,32,32,32,32, - 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, - 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, - 85,115,101,32,101,120,101,99,95,109,111,100,117,108,101,40, - 41,32,105,110,115,116,101,97,100,46,10,10,32,32,32,32, - 32,32,32,32,41,3,218,5,115,117,112,101,114,114,207,0, - 0,0,114,190,0,0,0,41,2,114,104,0,0,0,114,123, - 0,0,0,41,1,114,208,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,190,0,0,0,43,3,0,0,115,2,0, - 0,0,0,10,122,22,70,105,108,101,76,111,97,100,101,114, - 46,108,111,97,100,95,109,111,100,117,108,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,1,0,0,0,67,0,0, - 0,115,6,0,0,0,124,0,106,0,83,0,41,1,122,58, - 82,101,116,117,114,110,32,116,104,101,32,112,97,116,104,32, - 116,111,32,116,104,101,32,115,111,117,114,99,101,32,102,105, - 108,101,32,97,115,32,102,111,117,110,100,32,98,121,32,116, - 104,101,32,102,105,110,100,101,114,46,41,1,114,37,0,0, - 0,41,2,114,104,0,0,0,114,123,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,155,0,0, - 0,55,3,0,0,115,2,0,0,0,0,3,122,23,70,105, + 0,114,4,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,215,0,0,0,66,3,0,0,115,8, + 0,0,0,8,2,4,2,8,5,8,5,114,215,0,0,0, + 99,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0, + 0,64,0,0,0,115,32,0,0,0,101,0,90,1,100,0, + 90,2,100,1,90,3,100,2,100,3,132,0,90,4,100,4, + 100,5,132,0,90,5,100,6,83,0,41,7,218,20,83,111, + 117,114,99,101,108,101,115,115,70,105,108,101,76,111,97,100, + 101,114,122,45,76,111,97,100,101,114,32,119,104,105,99,104, + 32,104,97,110,100,108,101,115,32,115,111,117,114,99,101,108, + 101,115,115,32,102,105,108,101,32,105,109,112,111,114,116,115, + 46,99,2,0,0,0,0,0,0,0,5,0,0,0,5,0, + 0,0,67,0,0,0,115,48,0,0,0,124,0,106,0,124, + 1,131,1,125,2,124,0,106,1,124,2,131,1,125,3,116, + 2,124,3,124,1,124,2,100,1,141,3,125,4,116,3,124, + 4,124,1,124,2,100,2,141,3,83,0,41,3,78,41,2, + 114,102,0,0,0,114,37,0,0,0,41,2,114,102,0,0, + 0,114,93,0,0,0,41,4,114,155,0,0,0,114,197,0, + 0,0,114,139,0,0,0,114,145,0,0,0,41,5,114,104, + 0,0,0,114,123,0,0,0,114,37,0,0,0,114,56,0, + 0,0,114,206,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,114,184,0,0,0,115,3,0,0,115, + 8,0,0,0,0,1,10,1,10,1,14,1,122,29,83,111, + 117,114,99,101,108,101,115,115,70,105,108,101,76,111,97,100, + 101,114,46,103,101,116,95,99,111,100,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, + 115,4,0,0,0,100,1,83,0,41,2,122,39,82,101,116, + 117,114,110,32,78,111,110,101,32,97,115,32,116,104,101,114, + 101,32,105,115,32,110,111,32,115,111,117,114,99,101,32,99, + 111,100,101,46,78,114,4,0,0,0,41,2,114,104,0,0, + 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,199,0,0,0,121,3,0,0,115,2, + 0,0,0,0,2,122,31,83,111,117,114,99,101,108,101,115, + 115,70,105,108,101,76,111,97,100,101,114,46,103,101,116,95, + 115,111,117,114,99,101,78,41,6,114,109,0,0,0,114,108, + 0,0,0,114,110,0,0,0,114,111,0,0,0,114,184,0, + 0,0,114,199,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,220,0,0,0, + 111,3,0,0,115,6,0,0,0,8,2,4,2,8,6,114, + 220,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, + 0,3,0,0,0,64,0,0,0,115,92,0,0,0,101,0, + 90,1,100,0,90,2,100,1,90,3,100,2,100,3,132,0, + 90,4,100,4,100,5,132,0,90,5,100,6,100,7,132,0, + 90,6,100,8,100,9,132,0,90,7,100,10,100,11,132,0, + 90,8,100,12,100,13,132,0,90,9,100,14,100,15,132,0, + 90,10,100,16,100,17,132,0,90,11,101,12,100,18,100,19, + 132,0,131,1,90,13,100,20,83,0,41,21,218,19,69,120, + 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, + 114,122,93,76,111,97,100,101,114,32,102,111,114,32,101,120, + 116,101,110,115,105,111,110,32,109,111,100,117,108,101,115,46, + 10,10,32,32,32,32,84,104,101,32,99,111,110,115,116,114, + 117,99,116,111,114,32,105,115,32,100,101,115,105,103,110,101, + 100,32,116,111,32,119,111,114,107,32,119,105,116,104,32,70, + 105,108,101,70,105,110,100,101,114,46,10,10,32,32,32,32, + 99,3,0,0,0,0,0,0,0,3,0,0,0,2,0,0, + 0,67,0,0,0,115,16,0,0,0,124,1,124,0,95,0, + 124,2,124,0,95,1,100,0,83,0,41,1,78,41,2,114, + 102,0,0,0,114,37,0,0,0,41,3,114,104,0,0,0, + 114,102,0,0,0,114,37,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,182,0,0,0,138,3, + 0,0,115,4,0,0,0,0,1,6,1,122,28,69,120,116, + 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, + 46,95,95,105,110,105,116,95,95,99,2,0,0,0,0,0, + 0,0,2,0,0,0,2,0,0,0,67,0,0,0,115,24, + 0,0,0,124,0,106,0,124,1,106,0,107,2,111,22,124, + 0,106,1,124,1,106,1,107,2,83,0,41,1,78,41,2, + 114,208,0,0,0,114,115,0,0,0,41,2,114,104,0,0, + 0,114,209,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,210,0,0,0,142,3,0,0,115,4, + 0,0,0,0,1,12,1,122,26,69,120,116,101,110,115,105, + 111,110,70,105,108,101,76,111,97,100,101,114,46,95,95,101, + 113,95,95,99,1,0,0,0,0,0,0,0,1,0,0,0, + 3,0,0,0,67,0,0,0,115,20,0,0,0,116,0,124, + 0,106,1,131,1,116,0,124,0,106,2,131,1,65,0,83, + 0,41,1,78,41,3,114,211,0,0,0,114,102,0,0,0, + 114,37,0,0,0,41,1,114,104,0,0,0,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,114,212,0,0,0, + 146,3,0,0,115,2,0,0,0,0,1,122,28,69,120,116, + 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, + 46,95,95,104,97,115,104,95,95,99,2,0,0,0,0,0, + 0,0,3,0,0,0,4,0,0,0,67,0,0,0,115,36, + 0,0,0,116,0,106,1,116,2,106,3,124,1,131,2,125, + 2,116,0,106,4,100,1,124,1,106,5,124,0,106,6,131, + 3,1,0,124,2,83,0,41,2,122,38,67,114,101,97,116, + 101,32,97,110,32,117,110,105,116,105,97,108,105,122,101,100, + 32,101,120,116,101,110,115,105,111,110,32,109,111,100,117,108, + 101,122,38,101,120,116,101,110,115,105,111,110,32,109,111,100, + 117,108,101,32,123,33,114,125,32,108,111,97,100,101,100,32, + 102,114,111,109,32,123,33,114,125,41,7,114,118,0,0,0, + 114,185,0,0,0,114,143,0,0,0,90,14,99,114,101,97, + 116,101,95,100,121,110,97,109,105,99,114,133,0,0,0,114, + 102,0,0,0,114,37,0,0,0,41,3,114,104,0,0,0, + 114,162,0,0,0,114,187,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,183,0,0,0,149,3, + 0,0,115,10,0,0,0,0,2,4,1,10,1,6,1,12, + 1,122,33,69,120,116,101,110,115,105,111,110,70,105,108,101, + 76,111,97,100,101,114,46,99,114,101,97,116,101,95,109,111, + 100,117,108,101,99,2,0,0,0,0,0,0,0,2,0,0, + 0,4,0,0,0,67,0,0,0,115,36,0,0,0,116,0, + 106,1,116,2,106,3,124,1,131,2,1,0,116,0,106,4, + 100,1,124,0,106,5,124,0,106,6,131,3,1,0,100,2, + 83,0,41,3,122,30,73,110,105,116,105,97,108,105,122,101, + 32,97,110,32,101,120,116,101,110,115,105,111,110,32,109,111, + 100,117,108,101,122,40,101,120,116,101,110,115,105,111,110,32, + 109,111,100,117,108,101,32,123,33,114,125,32,101,120,101,99, + 117,116,101,100,32,102,114,111,109,32,123,33,114,125,78,41, + 7,114,118,0,0,0,114,185,0,0,0,114,143,0,0,0, + 90,12,101,120,101,99,95,100,121,110,97,109,105,99,114,133, + 0,0,0,114,102,0,0,0,114,37,0,0,0,41,2,114, + 104,0,0,0,114,187,0,0,0,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,114,188,0,0,0,157,3,0, + 0,115,6,0,0,0,0,2,14,1,6,1,122,31,69,120, + 116,101,110,115,105,111,110,70,105,108,101,76,111,97,100,101, + 114,46,101,120,101,99,95,109,111,100,117,108,101,99,2,0, + 0,0,0,0,0,0,2,0,0,0,4,0,0,0,3,0, + 0,0,115,36,0,0,0,116,0,124,0,106,1,131,1,100, + 1,25,0,137,0,116,2,135,0,102,1,100,2,100,3,132, + 8,116,3,68,0,131,1,131,1,83,0,41,4,122,49,82, + 101,116,117,114,110,32,84,114,117,101,32,105,102,32,116,104, + 101,32,101,120,116,101,110,115,105,111,110,32,109,111,100,117, + 108,101,32,105,115,32,97,32,112,97,99,107,97,103,101,46, + 114,31,0,0,0,99,1,0,0,0,0,0,0,0,2,0, + 0,0,4,0,0,0,51,0,0,0,115,26,0,0,0,124, + 0,93,18,125,1,136,0,100,0,124,1,23,0,107,2,86, + 0,1,0,113,2,100,1,83,0,41,2,114,182,0,0,0, + 78,114,4,0,0,0,41,2,114,24,0,0,0,218,6,115, + 117,102,102,105,120,41,1,218,9,102,105,108,101,95,110,97, + 109,101,114,4,0,0,0,114,6,0,0,0,250,9,60,103, + 101,110,101,120,112,114,62,166,3,0,0,115,2,0,0,0, + 4,1,122,49,69,120,116,101,110,115,105,111,110,70,105,108, + 101,76,111,97,100,101,114,46,105,115,95,112,97,99,107,97, + 103,101,46,60,108,111,99,97,108,115,62,46,60,103,101,110, + 101,120,112,114,62,41,4,114,40,0,0,0,114,37,0,0, + 0,218,3,97,110,121,218,18,69,88,84,69,78,83,73,79, + 78,95,83,85,70,70,73,88,69,83,41,2,114,104,0,0, + 0,114,123,0,0,0,114,4,0,0,0,41,1,114,223,0, + 0,0,114,6,0,0,0,114,157,0,0,0,163,3,0,0, + 115,6,0,0,0,0,2,14,1,12,1,122,30,69,120,116, + 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, + 46,105,115,95,112,97,99,107,97,103,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, + 115,4,0,0,0,100,1,83,0,41,2,122,63,82,101,116, + 117,114,110,32,78,111,110,101,32,97,115,32,97,110,32,101, + 120,116,101,110,115,105,111,110,32,109,111,100,117,108,101,32, + 99,97,110,110,111,116,32,99,114,101,97,116,101,32,97,32, + 99,111,100,101,32,111,98,106,101,99,116,46,78,114,4,0, + 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,184,0, + 0,0,169,3,0,0,115,2,0,0,0,0,2,122,28,69, + 120,116,101,110,115,105,111,110,70,105,108,101,76,111,97,100, + 101,114,46,103,101,116,95,99,111,100,101,99,2,0,0,0, + 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, + 115,4,0,0,0,100,1,83,0,41,2,122,53,82,101,116, + 117,114,110,32,78,111,110,101,32,97,115,32,101,120,116,101, + 110,115,105,111,110,32,109,111,100,117,108,101,115,32,104,97, + 118,101,32,110,111,32,115,111,117,114,99,101,32,99,111,100, + 101,46,78,114,4,0,0,0,41,2,114,104,0,0,0,114, + 123,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,114,199,0,0,0,173,3,0,0,115,2,0,0, + 0,0,2,122,30,69,120,116,101,110,115,105,111,110,70,105, + 108,101,76,111,97,100,101,114,46,103,101,116,95,115,111,117, + 114,99,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,6,0,0,0,124,0,106, + 0,83,0,41,1,122,58,82,101,116,117,114,110,32,116,104, + 101,32,112,97,116,104,32,116,111,32,116,104,101,32,115,111, + 117,114,99,101,32,102,105,108,101,32,97,115,32,102,111,117, + 110,100,32,98,121,32,116,104,101,32,102,105,110,100,101,114, + 46,41,1,114,37,0,0,0,41,2,114,104,0,0,0,114, + 123,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,114,155,0,0,0,177,3,0,0,115,2,0,0, + 0,0,3,122,32,69,120,116,101,110,115,105,111,110,70,105, 108,101,76,111,97,100,101,114,46,103,101,116,95,102,105,108, - 101,110,97,109,101,99,2,0,0,0,0,0,0,0,3,0, - 0,0,9,0,0,0,67,0,0,0,115,32,0,0,0,116, - 0,106,1,124,1,100,1,131,2,143,10,125,2,124,2,106, - 2,131,0,83,0,81,0,82,0,88,0,100,2,83,0,41, - 3,122,39,82,101,116,117,114,110,32,116,104,101,32,100,97, - 116,97,32,102,114,111,109,32,112,97,116,104,32,97,115,32, - 114,97,119,32,98,121,116,101,115,46,218,1,114,78,41,3, - 114,52,0,0,0,114,53,0,0,0,90,4,114,101,97,100, - 41,3,114,104,0,0,0,114,37,0,0,0,114,57,0,0, + 101,110,97,109,101,78,41,14,114,109,0,0,0,114,108,0, + 0,0,114,110,0,0,0,114,111,0,0,0,114,182,0,0, + 0,114,210,0,0,0,114,212,0,0,0,114,183,0,0,0, + 114,188,0,0,0,114,157,0,0,0,114,184,0,0,0,114, + 199,0,0,0,114,120,0,0,0,114,155,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,114,221,0,0,0,130,3,0,0,115,20,0,0,0, + 8,6,4,2,8,4,8,4,8,3,8,8,8,6,8,6, + 8,4,8,4,114,221,0,0,0,99,0,0,0,0,0,0, + 0,0,0,0,0,0,2,0,0,0,64,0,0,0,115,96, + 0,0,0,101,0,90,1,100,0,90,2,100,1,90,3,100, + 2,100,3,132,0,90,4,100,4,100,5,132,0,90,5,100, + 6,100,7,132,0,90,6,100,8,100,9,132,0,90,7,100, + 10,100,11,132,0,90,8,100,12,100,13,132,0,90,9,100, + 14,100,15,132,0,90,10,100,16,100,17,132,0,90,11,100, + 18,100,19,132,0,90,12,100,20,100,21,132,0,90,13,100, + 22,83,0,41,23,218,14,95,78,97,109,101,115,112,97,99, + 101,80,97,116,104,97,38,1,0,0,82,101,112,114,101,115, + 101,110,116,115,32,97,32,110,97,109,101,115,112,97,99,101, + 32,112,97,99,107,97,103,101,39,115,32,112,97,116,104,46, + 32,32,73,116,32,117,115,101,115,32,116,104,101,32,109,111, + 100,117,108,101,32,110,97,109,101,10,32,32,32,32,116,111, + 32,102,105,110,100,32,105,116,115,32,112,97,114,101,110,116, + 32,109,111,100,117,108,101,44,32,97,110,100,32,102,114,111, + 109,32,116,104,101,114,101,32,105,116,32,108,111,111,107,115, + 32,117,112,32,116,104,101,32,112,97,114,101,110,116,39,115, + 10,32,32,32,32,95,95,112,97,116,104,95,95,46,32,32, + 87,104,101,110,32,116,104,105,115,32,99,104,97,110,103,101, + 115,44,32,116,104,101,32,109,111,100,117,108,101,39,115,32, + 111,119,110,32,112,97,116,104,32,105,115,32,114,101,99,111, + 109,112,117,116,101,100,44,10,32,32,32,32,117,115,105,110, + 103,32,112,97,116,104,95,102,105,110,100,101,114,46,32,32, + 70,111,114,32,116,111,112,45,108,101,118,101,108,32,109,111, + 100,117,108,101,115,44,32,116,104,101,32,112,97,114,101,110, + 116,32,109,111,100,117,108,101,39,115,32,112,97,116,104,10, + 32,32,32,32,105,115,32,115,121,115,46,112,97,116,104,46, + 99,4,0,0,0,0,0,0,0,4,0,0,0,2,0,0, + 0,67,0,0,0,115,36,0,0,0,124,1,124,0,95,0, + 124,2,124,0,95,1,116,2,124,0,106,3,131,0,131,1, + 124,0,95,4,124,3,124,0,95,5,100,0,83,0,41,1, + 78,41,6,218,5,95,110,97,109,101,218,5,95,112,97,116, + 104,114,97,0,0,0,218,16,95,103,101,116,95,112,97,114, + 101,110,116,95,112,97,116,104,218,17,95,108,97,115,116,95, + 112,97,114,101,110,116,95,112,97,116,104,218,12,95,112,97, + 116,104,95,102,105,110,100,101,114,41,4,114,104,0,0,0, + 114,102,0,0,0,114,37,0,0,0,218,11,112,97,116,104, + 95,102,105,110,100,101,114,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,114,182,0,0,0,190,3,0,0,115, + 8,0,0,0,0,1,6,1,6,1,14,1,122,23,95,78, + 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,105, + 110,105,116,95,95,99,1,0,0,0,0,0,0,0,4,0, + 0,0,3,0,0,0,67,0,0,0,115,38,0,0,0,124, + 0,106,0,106,1,100,1,131,1,92,3,125,1,125,2,125, + 3,124,2,100,2,107,2,114,30,100,6,83,0,124,1,100, + 5,102,2,83,0,41,7,122,62,82,101,116,117,114,110,115, + 32,97,32,116,117,112,108,101,32,111,102,32,40,112,97,114, + 101,110,116,45,109,111,100,117,108,101,45,110,97,109,101,44, + 32,112,97,114,101,110,116,45,112,97,116,104,45,97,116,116, + 114,45,110,97,109,101,41,114,61,0,0,0,114,32,0,0, + 0,114,8,0,0,0,114,37,0,0,0,90,8,95,95,112, + 97,116,104,95,95,41,2,122,3,115,121,115,122,4,112,97, + 116,104,41,2,114,228,0,0,0,114,34,0,0,0,41,4, + 114,104,0,0,0,114,219,0,0,0,218,3,100,111,116,90, + 2,109,101,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,23,95,102,105,110,100,95,112,97,114,101,110,116, + 95,112,97,116,104,95,110,97,109,101,115,196,3,0,0,115, + 8,0,0,0,0,2,18,1,8,2,4,3,122,38,95,78, + 97,109,101,115,112,97,99,101,80,97,116,104,46,95,102,105, + 110,100,95,112,97,114,101,110,116,95,112,97,116,104,95,110, + 97,109,101,115,99,1,0,0,0,0,0,0,0,3,0,0, + 0,3,0,0,0,67,0,0,0,115,28,0,0,0,124,0, + 106,0,131,0,92,2,125,1,125,2,116,1,116,2,106,3, + 124,1,25,0,124,2,131,2,83,0,41,1,78,41,4,114, + 235,0,0,0,114,114,0,0,0,114,8,0,0,0,218,7, + 109,111,100,117,108,101,115,41,3,114,104,0,0,0,90,18, + 112,97,114,101,110,116,95,109,111,100,117,108,101,95,110,97, + 109,101,90,14,112,97,116,104,95,97,116,116,114,95,110,97, + 109,101,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,230,0,0,0,206,3,0,0,115,4,0,0,0,0, + 1,12,1,122,31,95,78,97,109,101,115,112,97,99,101,80, + 97,116,104,46,95,103,101,116,95,112,97,114,101,110,116,95, + 112,97,116,104,99,1,0,0,0,0,0,0,0,3,0,0, + 0,3,0,0,0,67,0,0,0,115,80,0,0,0,116,0, + 124,0,106,1,131,0,131,1,125,1,124,1,124,0,106,2, + 107,3,114,74,124,0,106,3,124,0,106,4,124,1,131,2, + 125,2,124,2,100,0,107,9,114,68,124,2,106,5,100,0, + 107,8,114,68,124,2,106,6,114,68,124,2,106,6,124,0, + 95,7,124,1,124,0,95,2,124,0,106,7,83,0,41,1, + 78,41,8,114,97,0,0,0,114,230,0,0,0,114,231,0, + 0,0,114,232,0,0,0,114,228,0,0,0,114,124,0,0, + 0,114,154,0,0,0,114,229,0,0,0,41,3,114,104,0, + 0,0,90,11,112,97,114,101,110,116,95,112,97,116,104,114, + 162,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,218,12,95,114,101,99,97,108,99,117,108,97,116, + 101,210,3,0,0,115,16,0,0,0,0,2,12,1,10,1, + 14,3,18,1,6,1,8,1,6,1,122,27,95,78,97,109, + 101,115,112,97,99,101,80,97,116,104,46,95,114,101,99,97, + 108,99,117,108,97,116,101,99,1,0,0,0,0,0,0,0, + 1,0,0,0,2,0,0,0,67,0,0,0,115,12,0,0, + 0,116,0,124,0,106,1,131,0,131,1,83,0,41,1,78, + 41,2,218,4,105,116,101,114,114,237,0,0,0,41,1,114, + 104,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, + 0,0,0,218,8,95,95,105,116,101,114,95,95,223,3,0, + 0,115,2,0,0,0,0,1,122,23,95,78,97,109,101,115, + 112,97,99,101,80,97,116,104,46,95,95,105,116,101,114,95, + 95,99,3,0,0,0,0,0,0,0,3,0,0,0,3,0, + 0,0,67,0,0,0,115,14,0,0,0,124,2,124,0,106, + 0,124,1,60,0,100,0,83,0,41,1,78,41,1,114,229, + 0,0,0,41,3,114,104,0,0,0,218,5,105,110,100,101, + 120,114,37,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,218,11,95,95,115,101,116,105,116,101,109, + 95,95,226,3,0,0,115,2,0,0,0,0,1,122,26,95, + 78,97,109,101,115,112,97,99,101,80,97,116,104,46,95,95, + 115,101,116,105,116,101,109,95,95,99,1,0,0,0,0,0, + 0,0,1,0,0,0,2,0,0,0,67,0,0,0,115,12, + 0,0,0,116,0,124,0,106,1,131,0,131,1,83,0,41, + 1,78,41,2,114,33,0,0,0,114,237,0,0,0,41,1, + 114,104,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,218,7,95,95,108,101,110,95,95,229,3,0, + 0,115,2,0,0,0,0,1,122,22,95,78,97,109,101,115, + 112,97,99,101,80,97,116,104,46,95,95,108,101,110,95,95, + 99,1,0,0,0,0,0,0,0,1,0,0,0,2,0,0, + 0,67,0,0,0,115,12,0,0,0,100,1,106,0,124,0, + 106,1,131,1,83,0,41,2,78,122,20,95,78,97,109,101, + 115,112,97,99,101,80,97,116,104,40,123,33,114,125,41,41, + 2,114,50,0,0,0,114,229,0,0,0,41,1,114,104,0, + 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,218,8,95,95,114,101,112,114,95,95,232,3,0,0,115, + 2,0,0,0,0,1,122,23,95,78,97,109,101,115,112,97, + 99,101,80,97,116,104,46,95,95,114,101,112,114,95,95,99, + 2,0,0,0,0,0,0,0,2,0,0,0,2,0,0,0, + 67,0,0,0,115,12,0,0,0,124,1,124,0,106,0,131, + 0,107,6,83,0,41,1,78,41,1,114,237,0,0,0,41, + 2,114,104,0,0,0,218,4,105,116,101,109,114,4,0,0, + 0,114,4,0,0,0,114,6,0,0,0,218,12,95,95,99, + 111,110,116,97,105,110,115,95,95,235,3,0,0,115,2,0, + 0,0,0,1,122,27,95,78,97,109,101,115,112,97,99,101, + 80,97,116,104,46,95,95,99,111,110,116,97,105,110,115,95, + 95,99,2,0,0,0,0,0,0,0,2,0,0,0,2,0, + 0,0,67,0,0,0,115,16,0,0,0,124,0,106,0,106, + 1,124,1,131,1,1,0,100,0,83,0,41,1,78,41,2, + 114,229,0,0,0,114,161,0,0,0,41,2,114,104,0,0, + 0,114,244,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,161,0,0,0,238,3,0,0,115,2, + 0,0,0,0,1,122,21,95,78,97,109,101,115,112,97,99, + 101,80,97,116,104,46,97,112,112,101,110,100,78,41,14,114, + 109,0,0,0,114,108,0,0,0,114,110,0,0,0,114,111, + 0,0,0,114,182,0,0,0,114,235,0,0,0,114,230,0, + 0,0,114,237,0,0,0,114,239,0,0,0,114,241,0,0, + 0,114,242,0,0,0,114,243,0,0,0,114,245,0,0,0, + 114,161,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,114,227,0,0,0,183,3, + 0,0,115,22,0,0,0,8,5,4,2,8,6,8,10,8, + 4,8,13,8,3,8,3,8,3,8,3,8,3,114,227,0, + 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,3, + 0,0,0,64,0,0,0,115,80,0,0,0,101,0,90,1, + 100,0,90,2,100,1,100,2,132,0,90,3,101,4,100,3, + 100,4,132,0,131,1,90,5,100,5,100,6,132,0,90,6, + 100,7,100,8,132,0,90,7,100,9,100,10,132,0,90,8, + 100,11,100,12,132,0,90,9,100,13,100,14,132,0,90,10, + 100,15,100,16,132,0,90,11,100,17,83,0,41,18,218,16, + 95,78,97,109,101,115,112,97,99,101,76,111,97,100,101,114, + 99,4,0,0,0,0,0,0,0,4,0,0,0,4,0,0, + 0,67,0,0,0,115,18,0,0,0,116,0,124,1,124,2, + 124,3,131,3,124,0,95,1,100,0,83,0,41,1,78,41, + 2,114,227,0,0,0,114,229,0,0,0,41,4,114,104,0, + 0,0,114,102,0,0,0,114,37,0,0,0,114,233,0,0, 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 114,197,0,0,0,60,3,0,0,115,4,0,0,0,0,2, - 14,1,122,19,70,105,108,101,76,111,97,100,101,114,46,103, - 101,116,95,100,97,116,97,78,41,12,114,109,0,0,0,114, - 108,0,0,0,114,110,0,0,0,114,111,0,0,0,114,182, - 0,0,0,114,210,0,0,0,114,212,0,0,0,114,120,0, - 0,0,114,190,0,0,0,114,155,0,0,0,114,197,0,0, - 0,90,13,95,95,99,108,97,115,115,99,101,108,108,95,95, - 114,4,0,0,0,114,4,0,0,0,41,1,114,208,0,0, - 0,114,6,0,0,0,114,207,0,0,0,25,3,0,0,115, - 14,0,0,0,8,3,4,2,8,6,8,4,8,3,16,12, - 12,5,114,207,0,0,0,99,0,0,0,0,0,0,0,0, - 0,0,0,0,3,0,0,0,64,0,0,0,115,46,0,0, - 0,101,0,90,1,100,0,90,2,100,1,90,3,100,2,100, - 3,132,0,90,4,100,4,100,5,132,0,90,5,100,6,100, - 7,156,1,100,8,100,9,132,2,90,6,100,10,83,0,41, - 11,218,16,83,111,117,114,99,101,70,105,108,101,76,111,97, - 100,101,114,122,62,67,111,110,99,114,101,116,101,32,105,109, - 112,108,101,109,101,110,116,97,116,105,111,110,32,111,102,32, - 83,111,117,114,99,101,76,111,97,100,101,114,32,117,115,105, - 110,103,32,116,104,101,32,102,105,108,101,32,115,121,115,116, - 101,109,46,99,2,0,0,0,0,0,0,0,3,0,0,0, - 3,0,0,0,67,0,0,0,115,22,0,0,0,116,0,124, - 1,131,1,125,2,124,2,106,1,124,2,106,2,100,1,156, - 2,83,0,41,2,122,33,82,101,116,117,114,110,32,116,104, - 101,32,109,101,116,97,100,97,116,97,32,102,111,114,32,116, - 104,101,32,112,97,116,104,46,41,2,122,5,109,116,105,109, - 101,122,4,115,105,122,101,41,3,114,41,0,0,0,218,8, - 115,116,95,109,116,105,109,101,90,7,115,116,95,115,105,122, - 101,41,3,114,104,0,0,0,114,37,0,0,0,114,205,0, + 114,182,0,0,0,244,3,0,0,115,2,0,0,0,0,1, + 122,25,95,78,97,109,101,115,112,97,99,101,76,111,97,100, + 101,114,46,95,95,105,110,105,116,95,95,99,2,0,0,0, + 0,0,0,0,2,0,0,0,2,0,0,0,67,0,0,0, + 115,12,0,0,0,100,1,106,0,124,1,106,1,131,1,83, + 0,41,2,122,115,82,101,116,117,114,110,32,114,101,112,114, + 32,102,111,114,32,116,104,101,32,109,111,100,117,108,101,46, + 10,10,32,32,32,32,32,32,32,32,84,104,101,32,109,101, + 116,104,111,100,32,105,115,32,100,101,112,114,101,99,97,116, + 101,100,46,32,32,84,104,101,32,105,109,112,111,114,116,32, + 109,97,99,104,105,110,101,114,121,32,100,111,101,115,32,116, + 104,101,32,106,111,98,32,105,116,115,101,108,102,46,10,10, + 32,32,32,32,32,32,32,32,122,25,60,109,111,100,117,108, + 101,32,123,33,114,125,32,40,110,97,109,101,115,112,97,99, + 101,41,62,41,2,114,50,0,0,0,114,109,0,0,0,41, + 2,114,168,0,0,0,114,187,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,11,109,111,100,117, + 108,101,95,114,101,112,114,247,3,0,0,115,2,0,0,0, + 0,7,122,28,95,78,97,109,101,115,112,97,99,101,76,111, + 97,100,101,114,46,109,111,100,117,108,101,95,114,101,112,114, + 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, + 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, + 78,84,114,4,0,0,0,41,2,114,104,0,0,0,114,123, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,114,157,0,0,0,0,4,0,0,115,2,0,0,0, + 0,1,122,27,95,78,97,109,101,115,112,97,99,101,76,111, + 97,100,101,114,46,105,115,95,112,97,99,107,97,103,101,99, + 2,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, + 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,78, + 114,32,0,0,0,114,4,0,0,0,41,2,114,104,0,0, + 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,199,0,0,0,3,4,0,0,115,2, + 0,0,0,0,1,122,27,95,78,97,109,101,115,112,97,99, + 101,76,111,97,100,101,114,46,103,101,116,95,115,111,117,114, + 99,101,99,2,0,0,0,0,0,0,0,2,0,0,0,6, + 0,0,0,67,0,0,0,115,16,0,0,0,116,0,100,1, + 100,2,100,3,100,4,100,5,141,4,83,0,41,6,78,114, + 32,0,0,0,122,8,60,115,116,114,105,110,103,62,114,186, + 0,0,0,84,41,1,114,201,0,0,0,41,1,114,202,0, + 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,184,0, + 0,0,6,4,0,0,115,2,0,0,0,0,1,122,25,95, + 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, + 103,101,116,95,99,111,100,101,99,2,0,0,0,0,0,0, + 0,2,0,0,0,1,0,0,0,67,0,0,0,115,4,0, + 0,0,100,1,83,0,41,2,122,42,85,115,101,32,100,101, + 102,97,117,108,116,32,115,101,109,97,110,116,105,99,115,32, + 102,111,114,32,109,111,100,117,108,101,32,99,114,101,97,116, + 105,111,110,46,78,114,4,0,0,0,41,2,114,104,0,0, + 0,114,162,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,183,0,0,0,9,4,0,0,115,0, + 0,0,0,122,30,95,78,97,109,101,115,112,97,99,101,76, + 111,97,100,101,114,46,99,114,101,97,116,101,95,109,111,100, + 117,108,101,99,2,0,0,0,0,0,0,0,2,0,0,0, + 1,0,0,0,67,0,0,0,115,4,0,0,0,100,0,83, + 0,41,1,78,114,4,0,0,0,41,2,114,104,0,0,0, + 114,187,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,188,0,0,0,12,4,0,0,115,2,0, + 0,0,0,1,122,28,95,78,97,109,101,115,112,97,99,101, + 76,111,97,100,101,114,46,101,120,101,99,95,109,111,100,117, + 108,101,99,2,0,0,0,0,0,0,0,2,0,0,0,3, + 0,0,0,67,0,0,0,115,26,0,0,0,116,0,106,1, + 100,1,124,0,106,2,131,2,1,0,116,0,106,3,124,0, + 124,1,131,2,83,0,41,2,122,98,76,111,97,100,32,97, + 32,110,97,109,101,115,112,97,99,101,32,109,111,100,117,108, + 101,46,10,10,32,32,32,32,32,32,32,32,84,104,105,115, + 32,109,101,116,104,111,100,32,105,115,32,100,101,112,114,101, + 99,97,116,101,100,46,32,32,85,115,101,32,101,120,101,99, + 95,109,111,100,117,108,101,40,41,32,105,110,115,116,101,97, + 100,46,10,10,32,32,32,32,32,32,32,32,122,38,110,97, + 109,101,115,112,97,99,101,32,109,111,100,117,108,101,32,108, + 111,97,100,101,100,32,119,105,116,104,32,112,97,116,104,32, + 123,33,114,125,41,4,114,118,0,0,0,114,133,0,0,0, + 114,229,0,0,0,114,189,0,0,0,41,2,114,104,0,0, + 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,190,0,0,0,15,4,0,0,115,6, + 0,0,0,0,7,6,1,8,1,122,28,95,78,97,109,101, + 115,112,97,99,101,76,111,97,100,101,114,46,108,111,97,100, + 95,109,111,100,117,108,101,78,41,12,114,109,0,0,0,114, + 108,0,0,0,114,110,0,0,0,114,182,0,0,0,114,180, + 0,0,0,114,247,0,0,0,114,157,0,0,0,114,199,0, + 0,0,114,184,0,0,0,114,183,0,0,0,114,188,0,0, + 0,114,190,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,114,246,0,0,0,243, + 3,0,0,115,16,0,0,0,8,1,8,3,12,9,8,3, + 8,3,8,3,8,3,8,3,114,246,0,0,0,99,0,0, + 0,0,0,0,0,0,0,0,0,0,4,0,0,0,64,0, + 0,0,115,106,0,0,0,101,0,90,1,100,0,90,2,100, + 1,90,3,101,4,100,2,100,3,132,0,131,1,90,5,101, + 4,100,4,100,5,132,0,131,1,90,6,101,4,100,6,100, + 7,132,0,131,1,90,7,101,4,100,8,100,9,132,0,131, + 1,90,8,101,4,100,17,100,11,100,12,132,1,131,1,90, + 9,101,4,100,18,100,13,100,14,132,1,131,1,90,10,101, + 4,100,19,100,15,100,16,132,1,131,1,90,11,100,10,83, + 0,41,20,218,10,80,97,116,104,70,105,110,100,101,114,122, + 62,77,101,116,97,32,112,97,116,104,32,102,105,110,100,101, + 114,32,102,111,114,32,115,121,115,46,112,97,116,104,32,97, + 110,100,32,112,97,99,107,97,103,101,32,95,95,112,97,116, + 104,95,95,32,97,116,116,114,105,98,117,116,101,115,46,99, + 1,0,0,0,0,0,0,0,2,0,0,0,4,0,0,0, + 67,0,0,0,115,42,0,0,0,120,36,116,0,106,1,106, + 2,131,0,68,0,93,22,125,1,116,3,124,1,100,1,131, + 2,114,12,124,1,106,4,131,0,1,0,113,12,87,0,100, + 2,83,0,41,3,122,125,67,97,108,108,32,116,104,101,32, + 105,110,118,97,108,105,100,97,116,101,95,99,97,99,104,101, + 115,40,41,32,109,101,116,104,111,100,32,111,110,32,97,108, + 108,32,112,97,116,104,32,101,110,116,114,121,32,102,105,110, + 100,101,114,115,10,32,32,32,32,32,32,32,32,115,116,111, + 114,101,100,32,105,110,32,115,121,115,46,112,97,116,104,95, + 105,109,112,111,114,116,101,114,95,99,97,99,104,101,115,32, + 40,119,104,101,114,101,32,105,109,112,108,101,109,101,110,116, + 101,100,41,46,218,17,105,110,118,97,108,105,100,97,116,101, + 95,99,97,99,104,101,115,78,41,5,114,8,0,0,0,218, + 19,112,97,116,104,95,105,109,112,111,114,116,101,114,95,99, + 97,99,104,101,218,6,118,97,108,117,101,115,114,112,0,0, + 0,114,249,0,0,0,41,2,114,168,0,0,0,218,6,102, + 105,110,100,101,114,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,249,0,0,0,33,4,0,0,115,6,0, + 0,0,0,4,16,1,10,1,122,28,80,97,116,104,70,105, + 110,100,101,114,46,105,110,118,97,108,105,100,97,116,101,95, + 99,97,99,104,101,115,99,2,0,0,0,0,0,0,0,3, + 0,0,0,12,0,0,0,67,0,0,0,115,86,0,0,0, + 116,0,106,1,100,1,107,9,114,30,116,0,106,1,12,0, + 114,30,116,2,106,3,100,2,116,4,131,2,1,0,120,50, + 116,0,106,1,68,0,93,36,125,2,121,8,124,2,124,1, + 131,1,83,0,4,0,116,5,107,10,114,72,1,0,1,0, + 1,0,119,38,89,0,113,38,88,0,113,38,87,0,100,1, + 83,0,100,1,83,0,41,3,122,46,83,101,97,114,99,104, + 32,115,121,115,46,112,97,116,104,95,104,111,111,107,115,32, + 102,111,114,32,97,32,102,105,110,100,101,114,32,102,111,114, + 32,39,112,97,116,104,39,46,78,122,23,115,121,115,46,112, + 97,116,104,95,104,111,111,107,115,32,105,115,32,101,109,112, + 116,121,41,6,114,8,0,0,0,218,10,112,97,116,104,95, + 104,111,111,107,115,114,63,0,0,0,114,64,0,0,0,114, + 122,0,0,0,114,103,0,0,0,41,3,114,168,0,0,0, + 114,37,0,0,0,90,4,104,111,111,107,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,218,11,95,112,97,116, + 104,95,104,111,111,107,115,41,4,0,0,115,16,0,0,0, + 0,3,18,1,12,1,12,1,2,1,8,1,14,1,12,2, + 122,22,80,97,116,104,70,105,110,100,101,114,46,95,112,97, + 116,104,95,104,111,111,107,115,99,2,0,0,0,0,0,0, + 0,3,0,0,0,19,0,0,0,67,0,0,0,115,102,0, + 0,0,124,1,100,1,107,2,114,42,121,12,116,0,106,1, + 131,0,125,1,87,0,110,20,4,0,116,2,107,10,114,40, + 1,0,1,0,1,0,100,2,83,0,88,0,121,14,116,3, + 106,4,124,1,25,0,125,2,87,0,110,40,4,0,116,5, + 107,10,114,96,1,0,1,0,1,0,124,0,106,6,124,1, + 131,1,125,2,124,2,116,3,106,4,124,1,60,0,89,0, + 110,2,88,0,124,2,83,0,41,3,122,210,71,101,116,32, + 116,104,101,32,102,105,110,100,101,114,32,102,111,114,32,116, + 104,101,32,112,97,116,104,32,101,110,116,114,121,32,102,114, + 111,109,32,115,121,115,46,112,97,116,104,95,105,109,112,111, + 114,116,101,114,95,99,97,99,104,101,46,10,10,32,32,32, + 32,32,32,32,32,73,102,32,116,104,101,32,112,97,116,104, + 32,101,110,116,114,121,32,105,115,32,110,111,116,32,105,110, + 32,116,104,101,32,99,97,99,104,101,44,32,102,105,110,100, + 32,116,104,101,32,97,112,112,114,111,112,114,105,97,116,101, + 32,102,105,110,100,101,114,10,32,32,32,32,32,32,32,32, + 97,110,100,32,99,97,99,104,101,32,105,116,46,32,73,102, + 32,110,111,32,102,105,110,100,101,114,32,105,115,32,97,118, + 97,105,108,97,98,108,101,44,32,115,116,111,114,101,32,78, + 111,110,101,46,10,10,32,32,32,32,32,32,32,32,114,32, + 0,0,0,78,41,7,114,3,0,0,0,114,47,0,0,0, + 218,17,70,105,108,101,78,111,116,70,111,117,110,100,69,114, + 114,111,114,114,8,0,0,0,114,250,0,0,0,114,135,0, + 0,0,114,254,0,0,0,41,3,114,168,0,0,0,114,37, + 0,0,0,114,252,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,218,20,95,112,97,116,104,95,105, + 109,112,111,114,116,101,114,95,99,97,99,104,101,54,4,0, + 0,115,22,0,0,0,0,8,8,1,2,1,12,1,14,3, + 6,1,2,1,14,1,14,1,10,1,16,1,122,31,80,97, + 116,104,70,105,110,100,101,114,46,95,112,97,116,104,95,105, + 109,112,111,114,116,101,114,95,99,97,99,104,101,99,3,0, + 0,0,0,0,0,0,6,0,0,0,3,0,0,0,67,0, + 0,0,115,82,0,0,0,116,0,124,2,100,1,131,2,114, + 26,124,2,106,1,124,1,131,1,92,2,125,3,125,4,110, + 14,124,2,106,2,124,1,131,1,125,3,103,0,125,4,124, + 3,100,0,107,9,114,60,116,3,106,4,124,1,124,3,131, + 2,83,0,116,3,106,5,124,1,100,0,131,2,125,5,124, + 4,124,5,95,6,124,5,83,0,41,2,78,114,121,0,0, + 0,41,7,114,112,0,0,0,114,121,0,0,0,114,179,0, + 0,0,114,118,0,0,0,114,176,0,0,0,114,158,0,0, + 0,114,154,0,0,0,41,6,114,168,0,0,0,114,123,0, + 0,0,114,252,0,0,0,114,124,0,0,0,114,125,0,0, + 0,114,162,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,218,16,95,108,101,103,97,99,121,95,103, + 101,116,95,115,112,101,99,76,4,0,0,115,18,0,0,0, + 0,4,10,1,16,2,10,1,4,1,8,1,12,1,12,1, + 6,1,122,27,80,97,116,104,70,105,110,100,101,114,46,95, + 108,101,103,97,99,121,95,103,101,116,95,115,112,101,99,78, + 99,4,0,0,0,0,0,0,0,9,0,0,0,5,0,0, + 0,67,0,0,0,115,170,0,0,0,103,0,125,4,120,160, + 124,2,68,0,93,130,125,5,116,0,124,5,116,1,116,2, + 102,2,131,2,115,30,113,10,124,0,106,3,124,5,131,1, + 125,6,124,6,100,1,107,9,114,10,116,4,124,6,100,2, + 131,2,114,72,124,6,106,5,124,1,124,3,131,2,125,7, + 110,12,124,0,106,6,124,1,124,6,131,2,125,7,124,7, + 100,1,107,8,114,94,113,10,124,7,106,7,100,1,107,9, + 114,108,124,7,83,0,124,7,106,8,125,8,124,8,100,1, + 107,8,114,130,116,9,100,3,131,1,130,1,124,4,106,10, + 124,8,131,1,1,0,113,10,87,0,116,11,106,12,124,1, + 100,1,131,2,125,7,124,4,124,7,95,8,124,7,83,0, + 100,1,83,0,41,4,122,63,70,105,110,100,32,116,104,101, + 32,108,111,97,100,101,114,32,111,114,32,110,97,109,101,115, + 112,97,99,101,95,112,97,116,104,32,102,111,114,32,116,104, + 105,115,32,109,111,100,117,108,101,47,112,97,99,107,97,103, + 101,32,110,97,109,101,46,78,114,178,0,0,0,122,19,115, + 112,101,99,32,109,105,115,115,105,110,103,32,108,111,97,100, + 101,114,41,13,114,141,0,0,0,114,73,0,0,0,218,5, + 98,121,116,101,115,114,0,1,0,0,114,112,0,0,0,114, + 178,0,0,0,114,1,1,0,0,114,124,0,0,0,114,154, + 0,0,0,114,103,0,0,0,114,147,0,0,0,114,118,0, + 0,0,114,158,0,0,0,41,9,114,168,0,0,0,114,123, + 0,0,0,114,37,0,0,0,114,177,0,0,0,218,14,110, + 97,109,101,115,112,97,99,101,95,112,97,116,104,90,5,101, + 110,116,114,121,114,252,0,0,0,114,162,0,0,0,114,125, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, + 0,0,218,9,95,103,101,116,95,115,112,101,99,91,4,0, + 0,115,40,0,0,0,0,5,4,1,10,1,14,1,2,1, + 10,1,8,1,10,1,14,2,12,1,8,1,2,1,10,1, + 4,1,6,1,8,1,8,5,14,2,12,1,6,1,122,20, + 80,97,116,104,70,105,110,100,101,114,46,95,103,101,116,95, + 115,112,101,99,99,4,0,0,0,0,0,0,0,6,0,0, + 0,4,0,0,0,67,0,0,0,115,104,0,0,0,124,2, + 100,1,107,8,114,14,116,0,106,1,125,2,124,0,106,2, + 124,1,124,2,124,3,131,3,125,4,124,4,100,1,107,8, + 114,42,100,1,83,0,110,58,124,4,106,3,100,1,107,8, + 114,96,124,4,106,4,125,5,124,5,114,90,100,2,124,4, + 95,5,116,6,124,1,124,5,124,0,106,2,131,3,124,4, + 95,4,124,4,83,0,113,100,100,1,83,0,110,4,124,4, + 83,0,100,1,83,0,41,3,122,141,84,114,121,32,116,111, + 32,102,105,110,100,32,97,32,115,112,101,99,32,102,111,114, + 32,39,102,117,108,108,110,97,109,101,39,32,111,110,32,115, + 121,115,46,112,97,116,104,32,111,114,32,39,112,97,116,104, + 39,46,10,10,32,32,32,32,32,32,32,32,84,104,101,32, + 115,101,97,114,99,104,32,105,115,32,98,97,115,101,100,32, + 111,110,32,115,121,115,46,112,97,116,104,95,104,111,111,107, + 115,32,97,110,100,32,115,121,115,46,112,97,116,104,95,105, + 109,112,111,114,116,101,114,95,99,97,99,104,101,46,10,32, + 32,32,32,32,32,32,32,78,90,9,110,97,109,101,115,112, + 97,99,101,41,7,114,8,0,0,0,114,37,0,0,0,114, + 4,1,0,0,114,124,0,0,0,114,154,0,0,0,114,156, + 0,0,0,114,227,0,0,0,41,6,114,168,0,0,0,114, + 123,0,0,0,114,37,0,0,0,114,177,0,0,0,114,162, + 0,0,0,114,3,1,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,114,178,0,0,0,123,4,0,0, + 115,26,0,0,0,0,6,8,1,6,1,14,1,8,1,6, + 1,10,1,6,1,4,3,6,1,16,1,6,2,6,2,122, + 20,80,97,116,104,70,105,110,100,101,114,46,102,105,110,100, + 95,115,112,101,99,99,3,0,0,0,0,0,0,0,4,0, + 0,0,3,0,0,0,67,0,0,0,115,30,0,0,0,124, + 0,106,0,124,1,124,2,131,2,125,3,124,3,100,1,107, + 8,114,24,100,1,83,0,124,3,106,1,83,0,41,2,122, + 170,102,105,110,100,32,116,104,101,32,109,111,100,117,108,101, + 32,111,110,32,115,121,115,46,112,97,116,104,32,111,114,32, + 39,112,97,116,104,39,32,98,97,115,101,100,32,111,110,32, + 115,121,115,46,112,97,116,104,95,104,111,111,107,115,32,97, + 110,100,10,32,32,32,32,32,32,32,32,115,121,115,46,112, + 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, + 104,101,46,10,10,32,32,32,32,32,32,32,32,84,104,105, + 115,32,109,101,116,104,111,100,32,105,115,32,100,101,112,114, + 101,99,97,116,101,100,46,32,32,85,115,101,32,102,105,110, + 100,95,115,112,101,99,40,41,32,105,110,115,116,101,97,100, + 46,10,10,32,32,32,32,32,32,32,32,78,41,2,114,178, + 0,0,0,114,124,0,0,0,41,4,114,168,0,0,0,114, + 123,0,0,0,114,37,0,0,0,114,162,0,0,0,114,4, + 0,0,0,114,4,0,0,0,114,6,0,0,0,114,179,0, + 0,0,147,4,0,0,115,8,0,0,0,0,8,12,1,8, + 1,4,1,122,22,80,97,116,104,70,105,110,100,101,114,46, + 102,105,110,100,95,109,111,100,117,108,101,41,1,78,41,2, + 78,78,41,1,78,41,12,114,109,0,0,0,114,108,0,0, + 0,114,110,0,0,0,114,111,0,0,0,114,180,0,0,0, + 114,249,0,0,0,114,254,0,0,0,114,0,1,0,0,114, + 1,1,0,0,114,4,1,0,0,114,178,0,0,0,114,179, + 0,0,0,114,4,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,114,248,0,0,0,29,4,0,0, + 115,22,0,0,0,8,2,4,2,12,8,12,13,12,22,12, + 15,2,1,12,31,2,1,12,23,2,1,114,248,0,0,0, + 99,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0, + 0,64,0,0,0,115,90,0,0,0,101,0,90,1,100,0, + 90,2,100,1,90,3,100,2,100,3,132,0,90,4,100,4, + 100,5,132,0,90,5,101,6,90,7,100,6,100,7,132,0, + 90,8,100,8,100,9,132,0,90,9,100,19,100,11,100,12, + 132,1,90,10,100,13,100,14,132,0,90,11,101,12,100,15, + 100,16,132,0,131,1,90,13,100,17,100,18,132,0,90,14, + 100,10,83,0,41,20,218,10,70,105,108,101,70,105,110,100, + 101,114,122,172,70,105,108,101,45,98,97,115,101,100,32,102, + 105,110,100,101,114,46,10,10,32,32,32,32,73,110,116,101, + 114,97,99,116,105,111,110,115,32,119,105,116,104,32,116,104, + 101,32,102,105,108,101,32,115,121,115,116,101,109,32,97,114, + 101,32,99,97,99,104,101,100,32,102,111,114,32,112,101,114, + 102,111,114,109,97,110,99,101,44,32,98,101,105,110,103,10, + 32,32,32,32,114,101,102,114,101,115,104,101,100,32,119,104, + 101,110,32,116,104,101,32,100,105,114,101,99,116,111,114,121, + 32,116,104,101,32,102,105,110,100,101,114,32,105,115,32,104, + 97,110,100,108,105,110,103,32,104,97,115,32,98,101,101,110, + 32,109,111,100,105,102,105,101,100,46,10,10,32,32,32,32, + 99,2,0,0,0,0,0,0,0,5,0,0,0,5,0,0, + 0,7,0,0,0,115,88,0,0,0,103,0,125,3,120,40, + 124,2,68,0,93,32,92,2,137,0,125,4,124,3,106,0, + 135,0,102,1,100,1,100,2,132,8,124,4,68,0,131,1, + 131,1,1,0,113,10,87,0,124,3,124,0,95,1,124,1, + 112,58,100,3,124,0,95,2,100,6,124,0,95,3,116,4, + 131,0,124,0,95,5,116,4,131,0,124,0,95,6,100,5, + 83,0,41,7,122,154,73,110,105,116,105,97,108,105,122,101, + 32,119,105,116,104,32,116,104,101,32,112,97,116,104,32,116, + 111,32,115,101,97,114,99,104,32,111,110,32,97,110,100,32, + 97,32,118,97,114,105,97,98,108,101,32,110,117,109,98,101, + 114,32,111,102,10,32,32,32,32,32,32,32,32,50,45,116, + 117,112,108,101,115,32,99,111,110,116,97,105,110,105,110,103, + 32,116,104,101,32,108,111,97,100,101,114,32,97,110,100,32, + 116,104,101,32,102,105,108,101,32,115,117,102,102,105,120,101, + 115,32,116,104,101,32,108,111,97,100,101,114,10,32,32,32, + 32,32,32,32,32,114,101,99,111,103,110,105,122,101,115,46, + 99,1,0,0,0,0,0,0,0,2,0,0,0,3,0,0, + 0,51,0,0,0,115,22,0,0,0,124,0,93,14,125,1, + 124,1,136,0,102,2,86,0,1,0,113,2,100,0,83,0, + 41,1,78,114,4,0,0,0,41,2,114,24,0,0,0,114, + 222,0,0,0,41,1,114,124,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,224,0,0,0,176,4,0,0,115,2, + 0,0,0,4,0,122,38,70,105,108,101,70,105,110,100,101, + 114,46,95,95,105,110,105,116,95,95,46,60,108,111,99,97, + 108,115,62,46,60,103,101,110,101,120,112,114,62,114,61,0, + 0,0,114,31,0,0,0,78,114,91,0,0,0,41,7,114, + 147,0,0,0,218,8,95,108,111,97,100,101,114,115,114,37, + 0,0,0,218,11,95,112,97,116,104,95,109,116,105,109,101, + 218,3,115,101,116,218,11,95,112,97,116,104,95,99,97,99, + 104,101,218,19,95,114,101,108,97,120,101,100,95,112,97,116, + 104,95,99,97,99,104,101,41,5,114,104,0,0,0,114,37, + 0,0,0,218,14,108,111,97,100,101,114,95,100,101,116,97, + 105,108,115,90,7,108,111,97,100,101,114,115,114,164,0,0, + 0,114,4,0,0,0,41,1,114,124,0,0,0,114,6,0, + 0,0,114,182,0,0,0,170,4,0,0,115,16,0,0,0, + 0,4,4,1,14,1,28,1,6,2,10,1,6,1,8,1, + 122,19,70,105,108,101,70,105,110,100,101,114,46,95,95,105, + 110,105,116,95,95,99,1,0,0,0,0,0,0,0,1,0, + 0,0,2,0,0,0,67,0,0,0,115,10,0,0,0,100, + 3,124,0,95,0,100,2,83,0,41,4,122,31,73,110,118, + 97,108,105,100,97,116,101,32,116,104,101,32,100,105,114,101, + 99,116,111,114,121,32,109,116,105,109,101,46,114,31,0,0, + 0,78,114,91,0,0,0,41,1,114,7,1,0,0,41,1, + 114,104,0,0,0,114,4,0,0,0,114,4,0,0,0,114, + 6,0,0,0,114,249,0,0,0,184,4,0,0,115,2,0, + 0,0,0,2,122,28,70,105,108,101,70,105,110,100,101,114, + 46,105,110,118,97,108,105,100,97,116,101,95,99,97,99,104, + 101,115,99,2,0,0,0,0,0,0,0,3,0,0,0,2, + 0,0,0,67,0,0,0,115,42,0,0,0,124,0,106,0, + 124,1,131,1,125,2,124,2,100,1,107,8,114,26,100,1, + 103,0,102,2,83,0,124,2,106,1,124,2,106,2,112,38, + 103,0,102,2,83,0,41,2,122,197,84,114,121,32,116,111, + 32,102,105,110,100,32,97,32,108,111,97,100,101,114,32,102, + 111,114,32,116,104,101,32,115,112,101,99,105,102,105,101,100, + 32,109,111,100,117,108,101,44,32,111,114,32,116,104,101,32, + 110,97,109,101,115,112,97,99,101,10,32,32,32,32,32,32, + 32,32,112,97,99,107,97,103,101,32,112,111,114,116,105,111, + 110,115,46,32,82,101,116,117,114,110,115,32,40,108,111,97, + 100,101,114,44,32,108,105,115,116,45,111,102,45,112,111,114, + 116,105,111,110,115,41,46,10,10,32,32,32,32,32,32,32, + 32,84,104,105,115,32,109,101,116,104,111,100,32,105,115,32, + 100,101,112,114,101,99,97,116,101,100,46,32,32,85,115,101, + 32,102,105,110,100,95,115,112,101,99,40,41,32,105,110,115, + 116,101,97,100,46,10,10,32,32,32,32,32,32,32,32,78, + 41,3,114,178,0,0,0,114,124,0,0,0,114,154,0,0, + 0,41,3,114,104,0,0,0,114,123,0,0,0,114,162,0, 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,194,0,0,0,70,3,0,0,115,4,0,0,0,0, - 2,8,1,122,27,83,111,117,114,99,101,70,105,108,101,76, - 111,97,100,101,114,46,112,97,116,104,95,115,116,97,116,115, - 99,4,0,0,0,0,0,0,0,5,0,0,0,5,0,0, - 0,67,0,0,0,115,24,0,0,0,116,0,124,1,131,1, - 125,4,124,0,106,1,124,2,124,3,124,4,100,1,141,3, - 83,0,41,2,78,41,1,218,5,95,109,111,100,101,41,2, - 114,101,0,0,0,114,195,0,0,0,41,5,114,104,0,0, - 0,114,94,0,0,0,114,93,0,0,0,114,56,0,0,0, - 114,44,0,0,0,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,196,0,0,0,75,3,0,0,115,4,0, - 0,0,0,2,8,1,122,32,83,111,117,114,99,101,70,105, - 108,101,76,111,97,100,101,114,46,95,99,97,99,104,101,95, - 98,121,116,101,99,111,100,101,105,182,1,0,0,41,1,114, - 217,0,0,0,99,3,0,0,0,1,0,0,0,9,0,0, - 0,17,0,0,0,67,0,0,0,115,250,0,0,0,116,0, - 124,1,131,1,92,2,125,4,125,5,103,0,125,6,120,40, - 124,4,114,56,116,1,124,4,131,1,12,0,114,56,116,0, - 124,4,131,1,92,2,125,4,125,7,124,6,106,2,124,7, - 131,1,1,0,113,18,87,0,120,108,116,3,124,6,131,1, - 68,0,93,96,125,7,116,4,124,4,124,7,131,2,125,4, - 121,14,116,5,106,6,124,4,131,1,1,0,87,0,113,68, - 4,0,116,7,107,10,114,118,1,0,1,0,1,0,119,68, - 89,0,113,68,4,0,116,8,107,10,114,162,1,0,125,8, - 1,0,122,18,116,9,106,10,100,1,124,4,124,8,131,3, - 1,0,100,2,83,0,100,2,125,8,126,8,88,0,113,68, - 88,0,113,68,87,0,121,28,116,11,124,1,124,2,124,3, - 131,3,1,0,116,9,106,10,100,3,124,1,131,2,1,0, - 87,0,110,48,4,0,116,8,107,10,114,244,1,0,125,8, - 1,0,122,20,116,9,106,10,100,1,124,1,124,8,131,3, - 1,0,87,0,89,0,100,2,100,2,125,8,126,8,88,0, - 110,2,88,0,100,2,83,0,41,4,122,27,87,114,105,116, - 101,32,98,121,116,101,115,32,100,97,116,97,32,116,111,32, - 97,32,102,105,108,101,46,122,27,99,111,117,108,100,32,110, - 111,116,32,99,114,101,97,116,101,32,123,33,114,125,58,32, - 123,33,114,125,78,122,12,99,114,101,97,116,101,100,32,123, - 33,114,125,41,12,114,40,0,0,0,114,48,0,0,0,114, - 161,0,0,0,114,35,0,0,0,114,30,0,0,0,114,3, - 0,0,0,90,5,109,107,100,105,114,218,15,70,105,108,101, - 69,120,105,115,116,115,69,114,114,111,114,114,42,0,0,0, - 114,118,0,0,0,114,133,0,0,0,114,58,0,0,0,41, - 9,114,104,0,0,0,114,37,0,0,0,114,56,0,0,0, - 114,217,0,0,0,218,6,112,97,114,101,110,116,114,98,0, - 0,0,114,29,0,0,0,114,25,0,0,0,114,198,0,0, - 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 114,195,0,0,0,80,3,0,0,115,42,0,0,0,0,2, - 12,1,4,2,16,1,12,1,14,2,14,1,10,1,2,1, - 14,1,14,2,6,1,16,3,6,1,8,1,20,1,2,1, - 12,1,16,1,16,2,8,1,122,25,83,111,117,114,99,101, - 70,105,108,101,76,111,97,100,101,114,46,115,101,116,95,100, - 97,116,97,78,41,7,114,109,0,0,0,114,108,0,0,0, - 114,110,0,0,0,114,111,0,0,0,114,194,0,0,0,114, - 196,0,0,0,114,195,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,215,0, - 0,0,66,3,0,0,115,8,0,0,0,8,2,4,2,8, - 5,8,5,114,215,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,2,0,0,0,64,0,0,0,115,32,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 100,3,132,0,90,4,100,4,100,5,132,0,90,5,100,6, - 83,0,41,7,218,20,83,111,117,114,99,101,108,101,115,115, - 70,105,108,101,76,111,97,100,101,114,122,45,76,111,97,100, - 101,114,32,119,104,105,99,104,32,104,97,110,100,108,101,115, - 32,115,111,117,114,99,101,108,101,115,115,32,102,105,108,101, - 32,105,109,112,111,114,116,115,46,99,2,0,0,0,0,0, - 0,0,5,0,0,0,5,0,0,0,67,0,0,0,115,48, - 0,0,0,124,0,106,0,124,1,131,1,125,2,124,0,106, - 1,124,2,131,1,125,3,116,2,124,3,124,1,124,2,100, - 1,141,3,125,4,116,3,124,4,124,1,124,2,100,2,141, - 3,83,0,41,3,78,41,2,114,102,0,0,0,114,37,0, - 0,0,41,2,114,102,0,0,0,114,93,0,0,0,41,4, - 114,155,0,0,0,114,197,0,0,0,114,139,0,0,0,114, - 145,0,0,0,41,5,114,104,0,0,0,114,123,0,0,0, - 114,37,0,0,0,114,56,0,0,0,114,206,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,184, - 0,0,0,115,3,0,0,115,8,0,0,0,0,1,10,1, - 10,1,14,1,122,29,83,111,117,114,99,101,108,101,115,115, - 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, - 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, - 0,41,2,122,39,82,101,116,117,114,110,32,78,111,110,101, - 32,97,115,32,116,104,101,114,101,32,105,115,32,110,111,32, - 115,111,117,114,99,101,32,99,111,100,101,46,78,114,4,0, - 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, - 0,0,121,3,0,0,115,2,0,0,0,0,2,122,31,83, - 111,117,114,99,101,108,101,115,115,70,105,108,101,76,111,97, - 100,101,114,46,103,101,116,95,115,111,117,114,99,101,78,41, - 6,114,109,0,0,0,114,108,0,0,0,114,110,0,0,0, - 114,111,0,0,0,114,184,0,0,0,114,199,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,220,0,0,0,111,3,0,0,115,6,0,0, - 0,8,2,4,2,8,6,114,220,0,0,0,99,0,0,0, - 0,0,0,0,0,0,0,0,0,3,0,0,0,64,0,0, - 0,115,92,0,0,0,101,0,90,1,100,0,90,2,100,1, - 90,3,100,2,100,3,132,0,90,4,100,4,100,5,132,0, - 90,5,100,6,100,7,132,0,90,6,100,8,100,9,132,0, - 90,7,100,10,100,11,132,0,90,8,100,12,100,13,132,0, - 90,9,100,14,100,15,132,0,90,10,100,16,100,17,132,0, - 90,11,101,12,100,18,100,19,132,0,131,1,90,13,100,20, - 83,0,41,21,218,19,69,120,116,101,110,115,105,111,110,70, - 105,108,101,76,111,97,100,101,114,122,93,76,111,97,100,101, - 114,32,102,111,114,32,101,120,116,101,110,115,105,111,110,32, - 109,111,100,117,108,101,115,46,10,10,32,32,32,32,84,104, - 101,32,99,111,110,115,116,114,117,99,116,111,114,32,105,115, - 32,100,101,115,105,103,110,101,100,32,116,111,32,119,111,114, - 107,32,119,105,116,104,32,70,105,108,101,70,105,110,100,101, - 114,46,10,10,32,32,32,32,99,3,0,0,0,0,0,0, - 0,3,0,0,0,2,0,0,0,67,0,0,0,115,16,0, - 0,0,124,1,124,0,95,0,124,2,124,0,95,1,100,0, - 83,0,41,1,78,41,2,114,102,0,0,0,114,37,0,0, - 0,41,3,114,104,0,0,0,114,102,0,0,0,114,37,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,182,0,0,0,138,3,0,0,115,4,0,0,0,0, - 1,6,1,122,28,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,95,95,105,110,105,116,95, - 95,99,2,0,0,0,0,0,0,0,2,0,0,0,2,0, - 0,0,67,0,0,0,115,24,0,0,0,124,0,106,0,124, - 1,106,0,107,2,111,22,124,0,106,1,124,1,106,1,107, - 2,83,0,41,1,78,41,2,114,208,0,0,0,114,115,0, - 0,0,41,2,114,104,0,0,0,114,209,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,210,0, - 0,0,142,3,0,0,115,4,0,0,0,0,1,12,1,122, - 26,69,120,116,101,110,115,105,111,110,70,105,108,101,76,111, - 97,100,101,114,46,95,95,101,113,95,95,99,1,0,0,0, - 0,0,0,0,1,0,0,0,3,0,0,0,67,0,0,0, - 115,20,0,0,0,116,0,124,0,106,1,131,1,116,0,124, - 0,106,2,131,1,65,0,83,0,41,1,78,41,3,114,211, - 0,0,0,114,102,0,0,0,114,37,0,0,0,41,1,114, - 104,0,0,0,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,114,212,0,0,0,146,3,0,0,115,2,0,0, - 0,0,1,122,28,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,95,95,104,97,115,104,95, - 95,99,2,0,0,0,0,0,0,0,3,0,0,0,4,0, - 0,0,67,0,0,0,115,36,0,0,0,116,0,106,1,116, - 2,106,3,124,1,131,2,125,2,116,0,106,4,100,1,124, - 1,106,5,124,0,106,6,131,3,1,0,124,2,83,0,41, - 2,122,38,67,114,101,97,116,101,32,97,110,32,117,110,105, - 116,105,97,108,105,122,101,100,32,101,120,116,101,110,115,105, - 111,110,32,109,111,100,117,108,101,122,38,101,120,116,101,110, - 115,105,111,110,32,109,111,100,117,108,101,32,123,33,114,125, - 32,108,111,97,100,101,100,32,102,114,111,109,32,123,33,114, - 125,41,7,114,118,0,0,0,114,185,0,0,0,114,143,0, - 0,0,90,14,99,114,101,97,116,101,95,100,121,110,97,109, - 105,99,114,133,0,0,0,114,102,0,0,0,114,37,0,0, - 0,41,3,114,104,0,0,0,114,162,0,0,0,114,187,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,183,0,0,0,149,3,0,0,115,10,0,0,0,0, - 2,4,1,10,1,6,1,12,1,122,33,69,120,116,101,110, - 115,105,111,110,70,105,108,101,76,111,97,100,101,114,46,99, - 114,101,97,116,101,95,109,111,100,117,108,101,99,2,0,0, - 0,0,0,0,0,2,0,0,0,4,0,0,0,67,0,0, - 0,115,36,0,0,0,116,0,106,1,116,2,106,3,124,1, - 131,2,1,0,116,0,106,4,100,1,124,0,106,5,124,0, - 106,6,131,3,1,0,100,2,83,0,41,3,122,30,73,110, - 105,116,105,97,108,105,122,101,32,97,110,32,101,120,116,101, - 110,115,105,111,110,32,109,111,100,117,108,101,122,40,101,120, - 116,101,110,115,105,111,110,32,109,111,100,117,108,101,32,123, - 33,114,125,32,101,120,101,99,117,116,101,100,32,102,114,111, - 109,32,123,33,114,125,78,41,7,114,118,0,0,0,114,185, - 0,0,0,114,143,0,0,0,90,12,101,120,101,99,95,100, - 121,110,97,109,105,99,114,133,0,0,0,114,102,0,0,0, - 114,37,0,0,0,41,2,114,104,0,0,0,114,187,0,0, - 0,114,4,0,0,0,114,4,0,0,0,114,6,0,0,0, - 114,188,0,0,0,157,3,0,0,115,6,0,0,0,0,2, - 14,1,6,1,122,31,69,120,116,101,110,115,105,111,110,70, - 105,108,101,76,111,97,100,101,114,46,101,120,101,99,95,109, - 111,100,117,108,101,99,2,0,0,0,0,0,0,0,2,0, - 0,0,4,0,0,0,3,0,0,0,115,36,0,0,0,116, - 0,124,0,106,1,131,1,100,1,25,0,137,0,116,2,135, - 0,102,1,100,2,100,3,132,8,116,3,68,0,131,1,131, - 1,83,0,41,4,122,49,82,101,116,117,114,110,32,84,114, - 117,101,32,105,102,32,116,104,101,32,101,120,116,101,110,115, - 105,111,110,32,109,111,100,117,108,101,32,105,115,32,97,32, - 112,97,99,107,97,103,101,46,114,31,0,0,0,99,1,0, - 0,0,0,0,0,0,2,0,0,0,4,0,0,0,51,0, - 0,0,115,26,0,0,0,124,0,93,18,125,1,136,0,100, - 0,124,1,23,0,107,2,86,0,1,0,113,2,100,1,83, - 0,41,2,114,182,0,0,0,78,114,4,0,0,0,41,2, - 114,24,0,0,0,218,6,115,117,102,102,105,120,41,1,218, - 9,102,105,108,101,95,110,97,109,101,114,4,0,0,0,114, - 6,0,0,0,250,9,60,103,101,110,101,120,112,114,62,166, - 3,0,0,115,2,0,0,0,4,1,122,49,69,120,116,101, - 110,115,105,111,110,70,105,108,101,76,111,97,100,101,114,46, - 105,115,95,112,97,99,107,97,103,101,46,60,108,111,99,97, - 108,115,62,46,60,103,101,110,101,120,112,114,62,41,4,114, - 40,0,0,0,114,37,0,0,0,218,3,97,110,121,218,18, - 69,88,84,69,78,83,73,79,78,95,83,85,70,70,73,88, - 69,83,41,2,114,104,0,0,0,114,123,0,0,0,114,4, - 0,0,0,41,1,114,223,0,0,0,114,6,0,0,0,114, - 157,0,0,0,163,3,0,0,115,6,0,0,0,0,2,14, - 1,12,1,122,30,69,120,116,101,110,115,105,111,110,70,105, - 108,101,76,111,97,100,101,114,46,105,115,95,112,97,99,107, - 97,103,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, - 0,41,2,122,63,82,101,116,117,114,110,32,78,111,110,101, - 32,97,115,32,97,110,32,101,120,116,101,110,115,105,111,110, - 32,109,111,100,117,108,101,32,99,97,110,110,111,116,32,99, - 114,101,97,116,101,32,97,32,99,111,100,101,32,111,98,106, - 101,99,116,46,78,114,4,0,0,0,41,2,114,104,0,0, - 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,184,0,0,0,169,3,0,0,115,2, - 0,0,0,0,2,122,28,69,120,116,101,110,115,105,111,110, - 70,105,108,101,76,111,97,100,101,114,46,103,101,116,95,99, - 111,100,101,99,2,0,0,0,0,0,0,0,2,0,0,0, - 1,0,0,0,67,0,0,0,115,4,0,0,0,100,1,83, - 0,41,2,122,53,82,101,116,117,114,110,32,78,111,110,101, - 32,97,115,32,101,120,116,101,110,115,105,111,110,32,109,111, - 100,117,108,101,115,32,104,97,118,101,32,110,111,32,115,111, - 117,114,99,101,32,99,111,100,101,46,78,114,4,0,0,0, - 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,199,0,0,0, - 173,3,0,0,115,2,0,0,0,0,2,122,30,69,120,116, - 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, - 46,103,101,116,95,115,111,117,114,99,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,6,0,0,0,124,0,106,0,83,0,41,1,122,58,82, - 101,116,117,114,110,32,116,104,101,32,112,97,116,104,32,116, - 111,32,116,104,101,32,115,111,117,114,99,101,32,102,105,108, - 101,32,97,115,32,102,111,117,110,100,32,98,121,32,116,104, - 101,32,102,105,110,100,101,114,46,41,1,114,37,0,0,0, - 41,2,114,104,0,0,0,114,123,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,114,155,0,0,0, - 177,3,0,0,115,2,0,0,0,0,3,122,32,69,120,116, - 101,110,115,105,111,110,70,105,108,101,76,111,97,100,101,114, - 46,103,101,116,95,102,105,108,101,110,97,109,101,78,41,14, - 114,109,0,0,0,114,108,0,0,0,114,110,0,0,0,114, - 111,0,0,0,114,182,0,0,0,114,210,0,0,0,114,212, - 0,0,0,114,183,0,0,0,114,188,0,0,0,114,157,0, - 0,0,114,184,0,0,0,114,199,0,0,0,114,120,0,0, - 0,114,155,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,221,0,0,0,130, - 3,0,0,115,20,0,0,0,8,6,4,2,8,4,8,4, - 8,3,8,8,8,6,8,6,8,4,8,4,114,221,0,0, - 0,99,0,0,0,0,0,0,0,0,0,0,0,0,2,0, - 0,0,64,0,0,0,115,96,0,0,0,101,0,90,1,100, - 0,90,2,100,1,90,3,100,2,100,3,132,0,90,4,100, - 4,100,5,132,0,90,5,100,6,100,7,132,0,90,6,100, - 8,100,9,132,0,90,7,100,10,100,11,132,0,90,8,100, - 12,100,13,132,0,90,9,100,14,100,15,132,0,90,10,100, - 16,100,17,132,0,90,11,100,18,100,19,132,0,90,12,100, - 20,100,21,132,0,90,13,100,22,83,0,41,23,218,14,95, - 78,97,109,101,115,112,97,99,101,80,97,116,104,97,38,1, - 0,0,82,101,112,114,101,115,101,110,116,115,32,97,32,110, - 97,109,101,115,112,97,99,101,32,112,97,99,107,97,103,101, - 39,115,32,112,97,116,104,46,32,32,73,116,32,117,115,101, - 115,32,116,104,101,32,109,111,100,117,108,101,32,110,97,109, - 101,10,32,32,32,32,116,111,32,102,105,110,100,32,105,116, - 115,32,112,97,114,101,110,116,32,109,111,100,117,108,101,44, - 32,97,110,100,32,102,114,111,109,32,116,104,101,114,101,32, - 105,116,32,108,111,111,107,115,32,117,112,32,116,104,101,32, - 112,97,114,101,110,116,39,115,10,32,32,32,32,95,95,112, - 97,116,104,95,95,46,32,32,87,104,101,110,32,116,104,105, - 115,32,99,104,97,110,103,101,115,44,32,116,104,101,32,109, - 111,100,117,108,101,39,115,32,111,119,110,32,112,97,116,104, - 32,105,115,32,114,101,99,111,109,112,117,116,101,100,44,10, - 32,32,32,32,117,115,105,110,103,32,112,97,116,104,95,102, - 105,110,100,101,114,46,32,32,70,111,114,32,116,111,112,45, - 108,101,118,101,108,32,109,111,100,117,108,101,115,44,32,116, - 104,101,32,112,97,114,101,110,116,32,109,111,100,117,108,101, - 39,115,32,112,97,116,104,10,32,32,32,32,105,115,32,115, - 121,115,46,112,97,116,104,46,99,4,0,0,0,0,0,0, - 0,4,0,0,0,2,0,0,0,67,0,0,0,115,36,0, - 0,0,124,1,124,0,95,0,124,2,124,0,95,1,116,2, - 124,0,106,3,131,0,131,1,124,0,95,4,124,3,124,0, - 95,5,100,0,83,0,41,1,78,41,6,218,5,95,110,97, - 109,101,218,5,95,112,97,116,104,114,97,0,0,0,218,16, - 95,103,101,116,95,112,97,114,101,110,116,95,112,97,116,104, - 218,17,95,108,97,115,116,95,112,97,114,101,110,116,95,112, - 97,116,104,218,12,95,112,97,116,104,95,102,105,110,100,101, - 114,41,4,114,104,0,0,0,114,102,0,0,0,114,37,0, - 0,0,218,11,112,97,116,104,95,102,105,110,100,101,114,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,182, - 0,0,0,190,3,0,0,115,8,0,0,0,0,1,6,1, - 6,1,14,1,122,23,95,78,97,109,101,115,112,97,99,101, - 80,97,116,104,46,95,95,105,110,105,116,95,95,99,1,0, - 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, - 0,0,115,38,0,0,0,124,0,106,0,106,1,100,1,131, - 1,92,3,125,1,125,2,125,3,124,2,100,2,107,2,114, - 30,100,6,83,0,124,1,100,5,102,2,83,0,41,7,122, - 62,82,101,116,117,114,110,115,32,97,32,116,117,112,108,101, - 32,111,102,32,40,112,97,114,101,110,116,45,109,111,100,117, - 108,101,45,110,97,109,101,44,32,112,97,114,101,110,116,45, - 112,97,116,104,45,97,116,116,114,45,110,97,109,101,41,114, - 61,0,0,0,114,32,0,0,0,114,8,0,0,0,114,37, - 0,0,0,90,8,95,95,112,97,116,104,95,95,41,2,122, - 3,115,121,115,122,4,112,97,116,104,41,2,114,228,0,0, - 0,114,34,0,0,0,41,4,114,104,0,0,0,114,219,0, - 0,0,218,3,100,111,116,90,2,109,101,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,23,95,102,105,110, - 100,95,112,97,114,101,110,116,95,112,97,116,104,95,110,97, - 109,101,115,196,3,0,0,115,8,0,0,0,0,2,18,1, - 8,2,4,3,122,38,95,78,97,109,101,115,112,97,99,101, - 80,97,116,104,46,95,102,105,110,100,95,112,97,114,101,110, - 116,95,112,97,116,104,95,110,97,109,101,115,99,1,0,0, - 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, - 0,115,28,0,0,0,124,0,106,0,131,0,92,2,125,1, - 125,2,116,1,116,2,106,3,124,1,25,0,124,2,131,2, - 83,0,41,1,78,41,4,114,235,0,0,0,114,114,0,0, - 0,114,8,0,0,0,218,7,109,111,100,117,108,101,115,41, - 3,114,104,0,0,0,90,18,112,97,114,101,110,116,95,109, - 111,100,117,108,101,95,110,97,109,101,90,14,112,97,116,104, - 95,97,116,116,114,95,110,97,109,101,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,230,0,0,0,206,3, - 0,0,115,4,0,0,0,0,1,12,1,122,31,95,78,97, - 109,101,115,112,97,99,101,80,97,116,104,46,95,103,101,116, - 95,112,97,114,101,110,116,95,112,97,116,104,99,1,0,0, - 0,0,0,0,0,3,0,0,0,3,0,0,0,67,0,0, - 0,115,80,0,0,0,116,0,124,0,106,1,131,0,131,1, - 125,1,124,1,124,0,106,2,107,3,114,74,124,0,106,3, - 124,0,106,4,124,1,131,2,125,2,124,2,100,0,107,9, - 114,68,124,2,106,5,100,0,107,8,114,68,124,2,106,6, - 114,68,124,2,106,6,124,0,95,7,124,1,124,0,95,2, - 124,0,106,7,83,0,41,1,78,41,8,114,97,0,0,0, - 114,230,0,0,0,114,231,0,0,0,114,232,0,0,0,114, - 228,0,0,0,114,124,0,0,0,114,154,0,0,0,114,229, - 0,0,0,41,3,114,104,0,0,0,90,11,112,97,114,101, - 110,116,95,112,97,116,104,114,162,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,12,95,114,101, - 99,97,108,99,117,108,97,116,101,210,3,0,0,115,16,0, - 0,0,0,2,12,1,10,1,14,3,18,1,6,1,8,1, - 6,1,122,27,95,78,97,109,101,115,112,97,99,101,80,97, - 116,104,46,95,114,101,99,97,108,99,117,108,97,116,101,99, - 1,0,0,0,0,0,0,0,1,0,0,0,2,0,0,0, - 67,0,0,0,115,12,0,0,0,116,0,124,0,106,1,131, - 0,131,1,83,0,41,1,78,41,2,218,4,105,116,101,114, - 114,237,0,0,0,41,1,114,104,0,0,0,114,4,0,0, - 0,114,4,0,0,0,114,6,0,0,0,218,8,95,95,105, - 116,101,114,95,95,223,3,0,0,115,2,0,0,0,0,1, - 122,23,95,78,97,109,101,115,112,97,99,101,80,97,116,104, - 46,95,95,105,116,101,114,95,95,99,3,0,0,0,0,0, - 0,0,3,0,0,0,3,0,0,0,67,0,0,0,115,14, - 0,0,0,124,2,124,0,106,0,124,1,60,0,100,0,83, - 0,41,1,78,41,1,114,229,0,0,0,41,3,114,104,0, - 0,0,218,5,105,110,100,101,120,114,37,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,11,95, - 95,115,101,116,105,116,101,109,95,95,226,3,0,0,115,2, - 0,0,0,0,1,122,26,95,78,97,109,101,115,112,97,99, - 101,80,97,116,104,46,95,95,115,101,116,105,116,101,109,95, - 95,99,1,0,0,0,0,0,0,0,1,0,0,0,2,0, - 0,0,67,0,0,0,115,12,0,0,0,116,0,124,0,106, - 1,131,0,131,1,83,0,41,1,78,41,2,114,33,0,0, - 0,114,237,0,0,0,41,1,114,104,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,7,95,95, - 108,101,110,95,95,229,3,0,0,115,2,0,0,0,0,1, - 122,22,95,78,97,109,101,115,112,97,99,101,80,97,116,104, - 46,95,95,108,101,110,95,95,99,1,0,0,0,0,0,0, + 0,114,121,0,0,0,190,4,0,0,115,8,0,0,0,0, + 7,10,1,8,1,8,1,122,22,70,105,108,101,70,105,110, + 100,101,114,46,102,105,110,100,95,108,111,97,100,101,114,99, + 6,0,0,0,0,0,0,0,7,0,0,0,6,0,0,0, + 67,0,0,0,115,26,0,0,0,124,1,124,2,124,3,131, + 2,125,6,116,0,124,2,124,3,124,6,124,4,100,1,141, + 4,83,0,41,2,78,41,2,114,124,0,0,0,114,154,0, + 0,0,41,1,114,165,0,0,0,41,7,114,104,0,0,0, + 114,163,0,0,0,114,123,0,0,0,114,37,0,0,0,90, + 4,115,109,115,108,114,177,0,0,0,114,124,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,4, + 1,0,0,202,4,0,0,115,6,0,0,0,0,1,10,1, + 8,1,122,20,70,105,108,101,70,105,110,100,101,114,46,95, + 103,101,116,95,115,112,101,99,78,99,3,0,0,0,0,0, + 0,0,14,0,0,0,15,0,0,0,67,0,0,0,115,98, + 1,0,0,100,1,125,3,124,1,106,0,100,2,131,1,100, + 3,25,0,125,4,121,24,116,1,124,0,106,2,112,34,116, + 3,106,4,131,0,131,1,106,5,125,5,87,0,110,24,4, + 0,116,6,107,10,114,66,1,0,1,0,1,0,100,10,125, + 5,89,0,110,2,88,0,124,5,124,0,106,7,107,3,114, + 92,124,0,106,8,131,0,1,0,124,5,124,0,95,7,116, + 9,131,0,114,114,124,0,106,10,125,6,124,4,106,11,131, + 0,125,7,110,10,124,0,106,12,125,6,124,4,125,7,124, + 7,124,6,107,6,114,218,116,13,124,0,106,2,124,4,131, + 2,125,8,120,72,124,0,106,14,68,0,93,54,92,2,125, + 9,125,10,100,5,124,9,23,0,125,11,116,13,124,8,124, + 11,131,2,125,12,116,15,124,12,131,1,114,152,124,0,106, + 16,124,10,124,1,124,12,124,8,103,1,124,2,131,5,83, + 0,113,152,87,0,116,17,124,8,131,1,125,3,120,88,124, + 0,106,14,68,0,93,78,92,2,125,9,125,10,116,13,124, + 0,106,2,124,4,124,9,23,0,131,2,125,12,116,18,106, + 19,100,6,124,12,100,3,100,7,141,3,1,0,124,7,124, + 9,23,0,124,6,107,6,114,226,116,15,124,12,131,1,114, + 226,124,0,106,16,124,10,124,1,124,12,100,8,124,2,131, + 5,83,0,113,226,87,0,124,3,144,1,114,94,116,18,106, + 19,100,9,124,8,131,2,1,0,116,18,106,20,124,1,100, + 8,131,2,125,13,124,8,103,1,124,13,95,21,124,13,83, + 0,100,8,83,0,41,11,122,111,84,114,121,32,116,111,32, + 102,105,110,100,32,97,32,115,112,101,99,32,102,111,114,32, + 116,104,101,32,115,112,101,99,105,102,105,101,100,32,109,111, + 100,117,108,101,46,10,10,32,32,32,32,32,32,32,32,82, + 101,116,117,114,110,115,32,116,104,101,32,109,97,116,99,104, + 105,110,103,32,115,112,101,99,44,32,111,114,32,78,111,110, + 101,32,105,102,32,110,111,116,32,102,111,117,110,100,46,10, + 32,32,32,32,32,32,32,32,70,114,61,0,0,0,114,59, + 0,0,0,114,31,0,0,0,114,182,0,0,0,122,9,116, + 114,121,105,110,103,32,123,125,41,1,90,9,118,101,114,98, + 111,115,105,116,121,78,122,25,112,111,115,115,105,98,108,101, + 32,110,97,109,101,115,112,97,99,101,32,102,111,114,32,123, + 125,114,91,0,0,0,41,22,114,34,0,0,0,114,41,0, + 0,0,114,37,0,0,0,114,3,0,0,0,114,47,0,0, + 0,114,216,0,0,0,114,42,0,0,0,114,7,1,0,0, + 218,11,95,102,105,108,108,95,99,97,99,104,101,114,7,0, + 0,0,114,10,1,0,0,114,92,0,0,0,114,9,1,0, + 0,114,30,0,0,0,114,6,1,0,0,114,46,0,0,0, + 114,4,1,0,0,114,48,0,0,0,114,118,0,0,0,114, + 133,0,0,0,114,158,0,0,0,114,154,0,0,0,41,14, + 114,104,0,0,0,114,123,0,0,0,114,177,0,0,0,90, + 12,105,115,95,110,97,109,101,115,112,97,99,101,90,11,116, + 97,105,108,95,109,111,100,117,108,101,114,130,0,0,0,90, + 5,99,97,99,104,101,90,12,99,97,99,104,101,95,109,111, + 100,117,108,101,90,9,98,97,115,101,95,112,97,116,104,114, + 222,0,0,0,114,163,0,0,0,90,13,105,110,105,116,95, + 102,105,108,101,110,97,109,101,90,9,102,117,108,108,95,112, + 97,116,104,114,162,0,0,0,114,4,0,0,0,114,4,0, + 0,0,114,6,0,0,0,114,178,0,0,0,207,4,0,0, + 115,70,0,0,0,0,5,4,1,14,1,2,1,24,1,14, + 1,10,1,10,1,8,1,6,2,6,1,6,1,10,2,6, + 1,4,2,8,1,12,1,16,1,8,1,10,1,8,1,24, + 4,8,2,16,1,16,1,16,1,12,1,8,1,10,1,12, + 1,6,1,12,1,12,1,8,1,4,1,122,20,70,105,108, + 101,70,105,110,100,101,114,46,102,105,110,100,95,115,112,101, + 99,99,1,0,0,0,0,0,0,0,9,0,0,0,13,0, + 0,0,67,0,0,0,115,194,0,0,0,124,0,106,0,125, + 1,121,22,116,1,106,2,124,1,112,22,116,1,106,3,131, + 0,131,1,125,2,87,0,110,30,4,0,116,4,116,5,116, + 6,102,3,107,10,114,58,1,0,1,0,1,0,103,0,125, + 2,89,0,110,2,88,0,116,7,106,8,106,9,100,1,131, + 1,115,84,116,10,124,2,131,1,124,0,95,11,110,78,116, + 10,131,0,125,3,120,64,124,2,68,0,93,56,125,4,124, + 4,106,12,100,2,131,1,92,3,125,5,125,6,125,7,124, + 6,114,138,100,3,106,13,124,5,124,7,106,14,131,0,131, + 2,125,8,110,4,124,5,125,8,124,3,106,15,124,8,131, + 1,1,0,113,96,87,0,124,3,124,0,95,11,116,7,106, + 8,106,9,116,16,131,1,114,190,100,4,100,5,132,0,124, + 2,68,0,131,1,124,0,95,17,100,6,83,0,41,7,122, + 68,70,105,108,108,32,116,104,101,32,99,97,99,104,101,32, + 111,102,32,112,111,116,101,110,116,105,97,108,32,109,111,100, + 117,108,101,115,32,97,110,100,32,112,97,99,107,97,103,101, + 115,32,102,111,114,32,116,104,105,115,32,100,105,114,101,99, + 116,111,114,121,46,114,0,0,0,0,114,61,0,0,0,122, + 5,123,125,46,123,125,99,1,0,0,0,0,0,0,0,2, + 0,0,0,3,0,0,0,83,0,0,0,115,20,0,0,0, + 104,0,124,0,93,12,125,1,124,1,106,0,131,0,146,2, + 113,4,83,0,114,4,0,0,0,41,1,114,92,0,0,0, + 41,2,114,24,0,0,0,90,2,102,110,114,4,0,0,0, + 114,4,0,0,0,114,6,0,0,0,250,9,60,115,101,116, + 99,111,109,112,62,28,5,0,0,115,2,0,0,0,6,0, + 122,41,70,105,108,101,70,105,110,100,101,114,46,95,102,105, + 108,108,95,99,97,99,104,101,46,60,108,111,99,97,108,115, + 62,46,60,115,101,116,99,111,109,112,62,78,41,18,114,37, + 0,0,0,114,3,0,0,0,90,7,108,105,115,116,100,105, + 114,114,47,0,0,0,114,255,0,0,0,218,15,80,101,114, + 109,105,115,115,105,111,110,69,114,114,111,114,218,18,78,111, + 116,65,68,105,114,101,99,116,111,114,121,69,114,114,111,114, + 114,8,0,0,0,114,9,0,0,0,114,10,0,0,0,114, + 8,1,0,0,114,9,1,0,0,114,87,0,0,0,114,50, + 0,0,0,114,92,0,0,0,218,3,97,100,100,114,11,0, + 0,0,114,10,1,0,0,41,9,114,104,0,0,0,114,37, + 0,0,0,90,8,99,111,110,116,101,110,116,115,90,21,108, + 111,119,101,114,95,115,117,102,102,105,120,95,99,111,110,116, + 101,110,116,115,114,244,0,0,0,114,102,0,0,0,114,234, + 0,0,0,114,222,0,0,0,90,8,110,101,119,95,110,97, + 109,101,114,4,0,0,0,114,4,0,0,0,114,6,0,0, + 0,114,12,1,0,0,255,4,0,0,115,34,0,0,0,0, + 2,6,1,2,1,22,1,20,3,10,3,12,1,12,7,6, + 1,10,1,16,1,4,1,18,2,4,1,14,1,6,1,12, + 1,122,22,70,105,108,101,70,105,110,100,101,114,46,95,102, + 105,108,108,95,99,97,99,104,101,99,1,0,0,0,0,0, + 0,0,3,0,0,0,3,0,0,0,7,0,0,0,115,18, + 0,0,0,135,0,135,1,102,2,100,1,100,2,132,8,125, + 2,124,2,83,0,41,3,97,20,1,0,0,65,32,99,108, + 97,115,115,32,109,101,116,104,111,100,32,119,104,105,99,104, + 32,114,101,116,117,114,110,115,32,97,32,99,108,111,115,117, + 114,101,32,116,111,32,117,115,101,32,111,110,32,115,121,115, + 46,112,97,116,104,95,104,111,111,107,10,32,32,32,32,32, + 32,32,32,119,104,105,99,104,32,119,105,108,108,32,114,101, + 116,117,114,110,32,97,110,32,105,110,115,116,97,110,99,101, + 32,117,115,105,110,103,32,116,104,101,32,115,112,101,99,105, + 102,105,101,100,32,108,111,97,100,101,114,115,32,97,110,100, + 32,116,104,101,32,112,97,116,104,10,32,32,32,32,32,32, + 32,32,99,97,108,108,101,100,32,111,110,32,116,104,101,32, + 99,108,111,115,117,114,101,46,10,10,32,32,32,32,32,32, + 32,32,73,102,32,116,104,101,32,112,97,116,104,32,99,97, + 108,108,101,100,32,111,110,32,116,104,101,32,99,108,111,115, + 117,114,101,32,105,115,32,110,111,116,32,97,32,100,105,114, + 101,99,116,111,114,121,44,32,73,109,112,111,114,116,69,114, + 114,111,114,32,105,115,10,32,32,32,32,32,32,32,32,114, + 97,105,115,101,100,46,10,10,32,32,32,32,32,32,32,32, + 99,1,0,0,0,0,0,0,0,1,0,0,0,4,0,0, + 0,19,0,0,0,115,34,0,0,0,116,0,124,0,131,1, + 115,20,116,1,100,1,124,0,100,2,141,2,130,1,136,0, + 124,0,102,1,136,1,152,2,142,0,83,0,41,3,122,45, + 80,97,116,104,32,104,111,111,107,32,102,111,114,32,105,109, + 112,111,114,116,108,105,98,46,109,97,99,104,105,110,101,114, + 121,46,70,105,108,101,70,105,110,100,101,114,46,122,30,111, + 110,108,121,32,100,105,114,101,99,116,111,114,105,101,115,32, + 97,114,101,32,115,117,112,112,111,114,116,101,100,41,1,114, + 37,0,0,0,41,2,114,48,0,0,0,114,103,0,0,0, + 41,1,114,37,0,0,0,41,2,114,168,0,0,0,114,11, + 1,0,0,114,4,0,0,0,114,6,0,0,0,218,24,112, + 97,116,104,95,104,111,111,107,95,102,111,114,95,70,105,108, + 101,70,105,110,100,101,114,40,5,0,0,115,6,0,0,0, + 0,2,8,1,12,1,122,54,70,105,108,101,70,105,110,100, + 101,114,46,112,97,116,104,95,104,111,111,107,46,60,108,111, + 99,97,108,115,62,46,112,97,116,104,95,104,111,111,107,95, + 102,111,114,95,70,105,108,101,70,105,110,100,101,114,114,4, + 0,0,0,41,3,114,168,0,0,0,114,11,1,0,0,114, + 17,1,0,0,114,4,0,0,0,41,2,114,168,0,0,0, + 114,11,1,0,0,114,6,0,0,0,218,9,112,97,116,104, + 95,104,111,111,107,30,5,0,0,115,4,0,0,0,0,10, + 14,6,122,20,70,105,108,101,70,105,110,100,101,114,46,112, + 97,116,104,95,104,111,111,107,99,1,0,0,0,0,0,0, 0,1,0,0,0,2,0,0,0,67,0,0,0,115,12,0, 0,0,100,1,106,0,124,0,106,1,131,1,83,0,41,2, - 78,122,20,95,78,97,109,101,115,112,97,99,101,80,97,116, - 104,40,123,33,114,125,41,41,2,114,50,0,0,0,114,229, - 0,0,0,41,1,114,104,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,218,8,95,95,114,101,112, - 114,95,95,232,3,0,0,115,2,0,0,0,0,1,122,23, - 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, - 95,114,101,112,114,95,95,99,2,0,0,0,0,0,0,0, - 2,0,0,0,2,0,0,0,67,0,0,0,115,12,0,0, - 0,124,1,124,0,106,0,131,0,107,6,83,0,41,1,78, - 41,1,114,237,0,0,0,41,2,114,104,0,0,0,218,4, - 105,116,101,109,114,4,0,0,0,114,4,0,0,0,114,6, - 0,0,0,218,12,95,95,99,111,110,116,97,105,110,115,95, - 95,235,3,0,0,115,2,0,0,0,0,1,122,27,95,78, - 97,109,101,115,112,97,99,101,80,97,116,104,46,95,95,99, - 111,110,116,97,105,110,115,95,95,99,2,0,0,0,0,0, - 0,0,2,0,0,0,2,0,0,0,67,0,0,0,115,16, - 0,0,0,124,0,106,0,106,1,124,1,131,1,1,0,100, - 0,83,0,41,1,78,41,2,114,229,0,0,0,114,161,0, - 0,0,41,2,114,104,0,0,0,114,244,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,161,0, - 0,0,238,3,0,0,115,2,0,0,0,0,1,122,21,95, - 78,97,109,101,115,112,97,99,101,80,97,116,104,46,97,112, - 112,101,110,100,78,41,14,114,109,0,0,0,114,108,0,0, - 0,114,110,0,0,0,114,111,0,0,0,114,182,0,0,0, - 114,235,0,0,0,114,230,0,0,0,114,237,0,0,0,114, - 239,0,0,0,114,241,0,0,0,114,242,0,0,0,114,243, - 0,0,0,114,245,0,0,0,114,161,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,114,227,0,0,0,183,3,0,0,115,22,0,0,0,8, - 5,4,2,8,6,8,10,8,4,8,13,8,3,8,3,8, - 3,8,3,8,3,114,227,0,0,0,99,0,0,0,0,0, - 0,0,0,0,0,0,0,3,0,0,0,64,0,0,0,115, - 80,0,0,0,101,0,90,1,100,0,90,2,100,1,100,2, - 132,0,90,3,101,4,100,3,100,4,132,0,131,1,90,5, - 100,5,100,6,132,0,90,6,100,7,100,8,132,0,90,7, - 100,9,100,10,132,0,90,8,100,11,100,12,132,0,90,9, - 100,13,100,14,132,0,90,10,100,15,100,16,132,0,90,11, - 100,17,83,0,41,18,218,16,95,78,97,109,101,115,112,97, - 99,101,76,111,97,100,101,114,99,4,0,0,0,0,0,0, - 0,4,0,0,0,4,0,0,0,67,0,0,0,115,18,0, - 0,0,116,0,124,1,124,2,124,3,131,3,124,0,95,1, - 100,0,83,0,41,1,78,41,2,114,227,0,0,0,114,229, - 0,0,0,41,4,114,104,0,0,0,114,102,0,0,0,114, - 37,0,0,0,114,233,0,0,0,114,4,0,0,0,114,4, - 0,0,0,114,6,0,0,0,114,182,0,0,0,244,3,0, - 0,115,2,0,0,0,0,1,122,25,95,78,97,109,101,115, - 112,97,99,101,76,111,97,100,101,114,46,95,95,105,110,105, - 116,95,95,99,2,0,0,0,0,0,0,0,2,0,0,0, - 2,0,0,0,67,0,0,0,115,12,0,0,0,100,1,106, - 0,124,1,106,1,131,1,83,0,41,2,122,115,82,101,116, - 117,114,110,32,114,101,112,114,32,102,111,114,32,116,104,101, - 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, - 32,32,84,104,101,32,109,101,116,104,111,100,32,105,115,32, - 100,101,112,114,101,99,97,116,101,100,46,32,32,84,104,101, - 32,105,109,112,111,114,116,32,109,97,99,104,105,110,101,114, - 121,32,100,111,101,115,32,116,104,101,32,106,111,98,32,105, - 116,115,101,108,102,46,10,10,32,32,32,32,32,32,32,32, - 122,25,60,109,111,100,117,108,101,32,123,33,114,125,32,40, - 110,97,109,101,115,112,97,99,101,41,62,41,2,114,50,0, - 0,0,114,109,0,0,0,41,2,114,168,0,0,0,114,187, + 78,122,16,70,105,108,101,70,105,110,100,101,114,40,123,33, + 114,125,41,41,2,114,50,0,0,0,114,37,0,0,0,41, + 1,114,104,0,0,0,114,4,0,0,0,114,4,0,0,0, + 114,6,0,0,0,114,243,0,0,0,48,5,0,0,115,2, + 0,0,0,0,1,122,19,70,105,108,101,70,105,110,100,101, + 114,46,95,95,114,101,112,114,95,95,41,1,78,41,15,114, + 109,0,0,0,114,108,0,0,0,114,110,0,0,0,114,111, + 0,0,0,114,182,0,0,0,114,249,0,0,0,114,127,0, + 0,0,114,179,0,0,0,114,121,0,0,0,114,4,1,0, + 0,114,178,0,0,0,114,12,1,0,0,114,180,0,0,0, + 114,18,1,0,0,114,243,0,0,0,114,4,0,0,0,114, + 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,5, + 1,0,0,161,4,0,0,115,20,0,0,0,8,7,4,2, + 8,14,8,4,4,2,8,12,8,5,10,48,8,31,12,18, + 114,5,1,0,0,99,4,0,0,0,0,0,0,0,6,0, + 0,0,11,0,0,0,67,0,0,0,115,146,0,0,0,124, + 0,106,0,100,1,131,1,125,4,124,0,106,0,100,2,131, + 1,125,5,124,4,115,66,124,5,114,36,124,5,106,1,125, + 4,110,30,124,2,124,3,107,2,114,56,116,2,124,1,124, + 2,131,2,125,4,110,10,116,3,124,1,124,2,131,2,125, + 4,124,5,115,84,116,4,124,1,124,2,124,4,100,3,141, + 3,125,5,121,36,124,5,124,0,100,2,60,0,124,4,124, + 0,100,1,60,0,124,2,124,0,100,4,60,0,124,3,124, + 0,100,5,60,0,87,0,110,20,4,0,116,5,107,10,114, + 140,1,0,1,0,1,0,89,0,110,2,88,0,100,0,83, + 0,41,6,78,218,10,95,95,108,111,97,100,101,114,95,95, + 218,8,95,95,115,112,101,99,95,95,41,1,114,124,0,0, + 0,90,8,95,95,102,105,108,101,95,95,90,10,95,95,99, + 97,99,104,101,100,95,95,41,6,218,3,103,101,116,114,124, + 0,0,0,114,220,0,0,0,114,215,0,0,0,114,165,0, + 0,0,218,9,69,120,99,101,112,116,105,111,110,41,6,90, + 2,110,115,114,102,0,0,0,90,8,112,97,116,104,110,97, + 109,101,90,9,99,112,97,116,104,110,97,109,101,114,124,0, + 0,0,114,162,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,218,14,95,102,105,120,95,117,112,95, + 109,111,100,117,108,101,54,5,0,0,115,34,0,0,0,0, + 2,10,1,10,1,4,1,4,1,8,1,8,1,12,2,10, + 1,4,1,14,1,2,1,8,1,8,1,8,1,12,1,14, + 2,114,23,1,0,0,99,0,0,0,0,0,0,0,0,3, + 0,0,0,3,0,0,0,67,0,0,0,115,38,0,0,0, + 116,0,116,1,106,2,131,0,102,2,125,0,116,3,116,4, + 102,2,125,1,116,5,116,6,102,2,125,2,124,0,124,1, + 124,2,103,3,83,0,41,1,122,95,82,101,116,117,114,110, + 115,32,97,32,108,105,115,116,32,111,102,32,102,105,108,101, + 45,98,97,115,101,100,32,109,111,100,117,108,101,32,108,111, + 97,100,101,114,115,46,10,10,32,32,32,32,69,97,99,104, + 32,105,116,101,109,32,105,115,32,97,32,116,117,112,108,101, + 32,40,108,111,97,100,101,114,44,32,115,117,102,102,105,120, + 101,115,41,46,10,32,32,32,32,41,7,114,221,0,0,0, + 114,143,0,0,0,218,18,101,120,116,101,110,115,105,111,110, + 95,115,117,102,102,105,120,101,115,114,215,0,0,0,114,88, + 0,0,0,114,220,0,0,0,114,78,0,0,0,41,3,90, + 10,101,120,116,101,110,115,105,111,110,115,90,6,115,111,117, + 114,99,101,90,8,98,121,116,101,99,111,100,101,114,4,0, + 0,0,114,4,0,0,0,114,6,0,0,0,114,159,0,0, + 0,77,5,0,0,115,8,0,0,0,0,5,12,1,8,1, + 8,1,114,159,0,0,0,99,1,0,0,0,0,0,0,0, + 12,0,0,0,12,0,0,0,67,0,0,0,115,188,1,0, + 0,124,0,97,0,116,0,106,1,97,1,116,0,106,2,97, + 2,116,1,106,3,116,4,25,0,125,1,120,56,100,26,68, + 0,93,48,125,2,124,2,116,1,106,3,107,7,114,58,116, + 0,106,5,124,2,131,1,125,3,110,10,116,1,106,3,124, + 2,25,0,125,3,116,6,124,1,124,2,124,3,131,3,1, + 0,113,32,87,0,100,5,100,6,103,1,102,2,100,7,100, + 8,100,6,103,2,102,2,102,2,125,4,120,118,124,4,68, + 0,93,102,92,2,125,5,125,6,116,7,100,9,100,10,132, + 0,124,6,68,0,131,1,131,1,115,142,116,8,130,1,124, + 6,100,11,25,0,125,7,124,5,116,1,106,3,107,6,114, + 174,116,1,106,3,124,5,25,0,125,8,80,0,113,112,121, + 16,116,0,106,5,124,5,131,1,125,8,80,0,87,0,113, + 112,4,0,116,9,107,10,114,212,1,0,1,0,1,0,119, + 112,89,0,113,112,88,0,113,112,87,0,116,9,100,12,131, + 1,130,1,116,6,124,1,100,13,124,8,131,3,1,0,116, + 6,124,1,100,14,124,7,131,3,1,0,116,6,124,1,100, + 15,100,16,106,10,124,6,131,1,131,3,1,0,121,14,116, + 0,106,5,100,17,131,1,125,9,87,0,110,26,4,0,116, + 9,107,10,144,1,114,52,1,0,1,0,1,0,100,18,125, + 9,89,0,110,2,88,0,116,6,124,1,100,17,124,9,131, + 3,1,0,116,0,106,5,100,19,131,1,125,10,116,6,124, + 1,100,19,124,10,131,3,1,0,124,5,100,7,107,2,144, + 1,114,120,116,0,106,5,100,20,131,1,125,11,116,6,124, + 1,100,21,124,11,131,3,1,0,116,6,124,1,100,22,116, + 11,131,0,131,3,1,0,116,12,106,13,116,2,106,14,131, + 0,131,1,1,0,124,5,100,7,107,2,144,1,114,184,116, + 15,106,16,100,23,131,1,1,0,100,24,116,12,107,6,144, + 1,114,184,100,25,116,17,95,18,100,18,83,0,41,27,122, + 205,83,101,116,117,112,32,116,104,101,32,112,97,116,104,45, + 98,97,115,101,100,32,105,109,112,111,114,116,101,114,115,32, + 102,111,114,32,105,109,112,111,114,116,108,105,98,32,98,121, + 32,105,109,112,111,114,116,105,110,103,32,110,101,101,100,101, + 100,10,32,32,32,32,98,117,105,108,116,45,105,110,32,109, + 111,100,117,108,101,115,32,97,110,100,32,105,110,106,101,99, + 116,105,110,103,32,116,104,101,109,32,105,110,116,111,32,116, + 104,101,32,103,108,111,98,97,108,32,110,97,109,101,115,112, + 97,99,101,46,10,10,32,32,32,32,79,116,104,101,114,32, + 99,111,109,112,111,110,101,110,116,115,32,97,114,101,32,101, + 120,116,114,97,99,116,101,100,32,102,114,111,109,32,116,104, + 101,32,99,111,114,101,32,98,111,111,116,115,116,114,97,112, + 32,109,111,100,117,108,101,46,10,10,32,32,32,32,114,52, + 0,0,0,114,63,0,0,0,218,8,98,117,105,108,116,105, + 110,115,114,140,0,0,0,90,5,112,111,115,105,120,250,1, + 47,218,2,110,116,250,1,92,99,1,0,0,0,0,0,0, + 0,2,0,0,0,3,0,0,0,115,0,0,0,115,26,0, + 0,0,124,0,93,18,125,1,116,0,124,1,131,1,100,0, + 107,2,86,0,1,0,113,2,100,1,83,0,41,2,114,31, + 0,0,0,78,41,1,114,33,0,0,0,41,2,114,24,0, + 0,0,114,81,0,0,0,114,4,0,0,0,114,4,0,0, + 0,114,6,0,0,0,114,224,0,0,0,113,5,0,0,115, + 2,0,0,0,4,0,122,25,95,115,101,116,117,112,46,60, + 108,111,99,97,108,115,62,46,60,103,101,110,101,120,112,114, + 62,114,62,0,0,0,122,30,105,109,112,111,114,116,108,105, + 98,32,114,101,113,117,105,114,101,115,32,112,111,115,105,120, + 32,111,114,32,110,116,114,3,0,0,0,114,27,0,0,0, + 114,23,0,0,0,114,32,0,0,0,90,7,95,116,104,114, + 101,97,100,78,90,8,95,119,101,97,107,114,101,102,90,6, + 119,105,110,114,101,103,114,167,0,0,0,114,7,0,0,0, + 122,4,46,112,121,119,122,6,95,100,46,112,121,100,84,41, + 4,122,3,95,105,111,122,9,95,119,97,114,110,105,110,103, + 115,122,8,98,117,105,108,116,105,110,115,122,7,109,97,114, + 115,104,97,108,41,19,114,118,0,0,0,114,8,0,0,0, + 114,143,0,0,0,114,236,0,0,0,114,109,0,0,0,90, + 18,95,98,117,105,108,116,105,110,95,102,114,111,109,95,110, + 97,109,101,114,113,0,0,0,218,3,97,108,108,218,14,65, + 115,115,101,114,116,105,111,110,69,114,114,111,114,114,103,0, + 0,0,114,28,0,0,0,114,13,0,0,0,114,226,0,0, + 0,114,147,0,0,0,114,24,1,0,0,114,88,0,0,0, + 114,161,0,0,0,114,166,0,0,0,114,170,0,0,0,41, + 12,218,17,95,98,111,111,116,115,116,114,97,112,95,109,111, + 100,117,108,101,90,11,115,101,108,102,95,109,111,100,117,108, + 101,90,12,98,117,105,108,116,105,110,95,110,97,109,101,90, + 14,98,117,105,108,116,105,110,95,109,111,100,117,108,101,90, + 10,111,115,95,100,101,116,97,105,108,115,90,10,98,117,105, + 108,116,105,110,95,111,115,114,23,0,0,0,114,27,0,0, + 0,90,9,111,115,95,109,111,100,117,108,101,90,13,116,104, + 114,101,97,100,95,109,111,100,117,108,101,90,14,119,101,97, + 107,114,101,102,95,109,111,100,117,108,101,90,13,119,105,110, + 114,101,103,95,109,111,100,117,108,101,114,4,0,0,0,114, + 4,0,0,0,114,6,0,0,0,218,6,95,115,101,116,117, + 112,88,5,0,0,115,82,0,0,0,0,8,4,1,6,1, + 6,3,10,1,10,1,10,1,12,2,10,1,16,3,22,1, + 14,2,22,1,8,1,10,1,10,1,4,2,2,1,10,1, + 6,1,14,1,12,2,8,1,12,1,12,1,18,3,2,1, + 14,1,16,2,10,1,12,3,10,1,12,3,10,1,10,1, + 12,3,14,1,14,1,10,1,10,1,10,1,114,32,1,0, + 0,99,1,0,0,0,0,0,0,0,2,0,0,0,3,0, + 0,0,67,0,0,0,115,72,0,0,0,116,0,124,0,131, + 1,1,0,116,1,131,0,125,1,116,2,106,3,106,4,116, + 5,106,6,124,1,142,0,103,1,131,1,1,0,116,7,106, + 8,100,1,107,2,114,56,116,2,106,9,106,10,116,11,131, + 1,1,0,116,2,106,9,106,10,116,12,131,1,1,0,100, + 2,83,0,41,3,122,41,73,110,115,116,97,108,108,32,116, + 104,101,32,112,97,116,104,45,98,97,115,101,100,32,105,109, + 112,111,114,116,32,99,111,109,112,111,110,101,110,116,115,46, + 114,27,1,0,0,78,41,13,114,32,1,0,0,114,159,0, + 0,0,114,8,0,0,0,114,253,0,0,0,114,147,0,0, + 0,114,5,1,0,0,114,18,1,0,0,114,3,0,0,0, + 114,109,0,0,0,218,9,109,101,116,97,95,112,97,116,104, + 114,161,0,0,0,114,166,0,0,0,114,248,0,0,0,41, + 2,114,31,1,0,0,90,17,115,117,112,112,111,114,116,101, + 100,95,108,111,97,100,101,114,115,114,4,0,0,0,114,4, + 0,0,0,114,6,0,0,0,218,8,95,105,110,115,116,97, + 108,108,156,5,0,0,115,12,0,0,0,0,2,8,1,6, + 1,20,1,10,1,12,1,114,34,1,0,0,41,1,122,3, + 119,105,110,41,2,114,1,0,0,0,114,2,0,0,0,41, + 1,114,49,0,0,0,41,1,78,41,3,78,78,78,41,3, + 78,78,78,41,2,114,62,0,0,0,114,62,0,0,0,41, + 1,78,41,1,78,41,58,114,111,0,0,0,114,12,0,0, + 0,90,37,95,67,65,83,69,95,73,78,83,69,78,83,73, + 84,73,86,69,95,80,76,65,84,70,79,82,77,83,95,66, + 89,84,69,83,95,75,69,89,114,11,0,0,0,114,13,0, + 0,0,114,19,0,0,0,114,21,0,0,0,114,30,0,0, + 0,114,40,0,0,0,114,41,0,0,0,114,45,0,0,0, + 114,46,0,0,0,114,48,0,0,0,114,58,0,0,0,218, + 4,116,121,112,101,218,8,95,95,99,111,100,101,95,95,114, + 142,0,0,0,114,17,0,0,0,114,132,0,0,0,114,16, + 0,0,0,114,20,0,0,0,90,17,95,82,65,87,95,77, + 65,71,73,67,95,78,85,77,66,69,82,114,77,0,0,0, + 114,76,0,0,0,114,88,0,0,0,114,78,0,0,0,90, + 23,68,69,66,85,71,95,66,89,84,69,67,79,68,69,95, + 83,85,70,70,73,88,69,83,90,27,79,80,84,73,77,73, + 90,69,68,95,66,89,84,69,67,79,68,69,95,83,85,70, + 70,73,88,69,83,114,83,0,0,0,114,89,0,0,0,114, + 95,0,0,0,114,99,0,0,0,114,101,0,0,0,114,120, + 0,0,0,114,127,0,0,0,114,139,0,0,0,114,145,0, + 0,0,114,148,0,0,0,114,153,0,0,0,218,6,111,98, + 106,101,99,116,114,160,0,0,0,114,165,0,0,0,114,166, + 0,0,0,114,181,0,0,0,114,191,0,0,0,114,207,0, + 0,0,114,215,0,0,0,114,220,0,0,0,114,226,0,0, + 0,114,221,0,0,0,114,227,0,0,0,114,246,0,0,0, + 114,248,0,0,0,114,5,1,0,0,114,23,1,0,0,114, + 159,0,0,0,114,32,1,0,0,114,34,1,0,0,114,4, 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,109,111,100,117,108,101,95,114,101,112,114,247, - 3,0,0,115,2,0,0,0,0,7,122,28,95,78,97,109, - 101,115,112,97,99,101,76,111,97,100,101,114,46,109,111,100, - 117,108,101,95,114,101,112,114,99,2,0,0,0,0,0,0, - 0,2,0,0,0,1,0,0,0,67,0,0,0,115,4,0, - 0,0,100,1,83,0,41,2,78,84,114,4,0,0,0,41, - 2,114,104,0,0,0,114,123,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,114,157,0,0,0,0, - 4,0,0,115,2,0,0,0,0,1,122,27,95,78,97,109, - 101,115,112,97,99,101,76,111,97,100,101,114,46,105,115,95, - 112,97,99,107,97,103,101,99,2,0,0,0,0,0,0,0, - 2,0,0,0,1,0,0,0,67,0,0,0,115,4,0,0, - 0,100,1,83,0,41,2,78,114,32,0,0,0,114,4,0, - 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,199,0, - 0,0,3,4,0,0,115,2,0,0,0,0,1,122,27,95, - 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, - 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, - 0,0,0,2,0,0,0,6,0,0,0,67,0,0,0,115, - 16,0,0,0,116,0,100,1,100,2,100,3,100,4,100,5, - 141,4,83,0,41,6,78,114,32,0,0,0,122,8,60,115, - 116,114,105,110,103,62,114,186,0,0,0,84,41,1,114,201, - 0,0,0,41,1,114,202,0,0,0,41,2,114,104,0,0, - 0,114,123,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,184,0,0,0,6,4,0,0,115,2, - 0,0,0,0,1,122,25,95,78,97,109,101,115,112,97,99, - 101,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, - 99,2,0,0,0,0,0,0,0,2,0,0,0,1,0,0, - 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, - 122,42,85,115,101,32,100,101,102,97,117,108,116,32,115,101, - 109,97,110,116,105,99,115,32,102,111,114,32,109,111,100,117, - 108,101,32,99,114,101,97,116,105,111,110,46,78,114,4,0, - 0,0,41,2,114,104,0,0,0,114,162,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,183,0, - 0,0,9,4,0,0,115,0,0,0,0,122,30,95,78,97, - 109,101,115,112,97,99,101,76,111,97,100,101,114,46,99,114, - 101,97,116,101,95,109,111,100,117,108,101,99,2,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,0,83,0,41,1,78,114,4,0,0, - 0,41,2,114,104,0,0,0,114,187,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,188,0,0, - 0,12,4,0,0,115,2,0,0,0,0,1,122,28,95,78, - 97,109,101,115,112,97,99,101,76,111,97,100,101,114,46,101, - 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, - 0,0,0,2,0,0,0,3,0,0,0,67,0,0,0,115, - 26,0,0,0,116,0,106,1,100,1,124,0,106,2,131,2, - 1,0,116,0,106,3,124,0,124,1,131,2,83,0,41,2, - 122,98,76,111,97,100,32,97,32,110,97,109,101,115,112,97, - 99,101,32,109,111,100,117,108,101,46,10,10,32,32,32,32, - 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, - 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, - 85,115,101,32,101,120,101,99,95,109,111,100,117,108,101,40, - 41,32,105,110,115,116,101,97,100,46,10,10,32,32,32,32, - 32,32,32,32,122,38,110,97,109,101,115,112,97,99,101,32, - 109,111,100,117,108,101,32,108,111,97,100,101,100,32,119,105, - 116,104,32,112,97,116,104,32,123,33,114,125,41,4,114,118, - 0,0,0,114,133,0,0,0,114,229,0,0,0,114,189,0, - 0,0,41,2,114,104,0,0,0,114,123,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,190,0, - 0,0,15,4,0,0,115,6,0,0,0,0,7,6,1,8, - 1,122,28,95,78,97,109,101,115,112,97,99,101,76,111,97, - 100,101,114,46,108,111,97,100,95,109,111,100,117,108,101,78, - 41,12,114,109,0,0,0,114,108,0,0,0,114,110,0,0, - 0,114,182,0,0,0,114,180,0,0,0,114,247,0,0,0, - 114,157,0,0,0,114,199,0,0,0,114,184,0,0,0,114, - 183,0,0,0,114,188,0,0,0,114,190,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,114,246,0,0,0,243,3,0,0,115,16,0,0,0, - 8,1,8,3,12,9,8,3,8,3,8,3,8,3,8,3, - 114,246,0,0,0,99,0,0,0,0,0,0,0,0,0,0, - 0,0,4,0,0,0,64,0,0,0,115,106,0,0,0,101, - 0,90,1,100,0,90,2,100,1,90,3,101,4,100,2,100, - 3,132,0,131,1,90,5,101,4,100,4,100,5,132,0,131, - 1,90,6,101,4,100,6,100,7,132,0,131,1,90,7,101, - 4,100,8,100,9,132,0,131,1,90,8,101,4,100,17,100, - 11,100,12,132,1,131,1,90,9,101,4,100,18,100,13,100, - 14,132,1,131,1,90,10,101,4,100,19,100,15,100,16,132, - 1,131,1,90,11,100,10,83,0,41,20,218,10,80,97,116, - 104,70,105,110,100,101,114,122,62,77,101,116,97,32,112,97, - 116,104,32,102,105,110,100,101,114,32,102,111,114,32,115,121, - 115,46,112,97,116,104,32,97,110,100,32,112,97,99,107,97, - 103,101,32,95,95,112,97,116,104,95,95,32,97,116,116,114, - 105,98,117,116,101,115,46,99,1,0,0,0,0,0,0,0, - 2,0,0,0,4,0,0,0,67,0,0,0,115,42,0,0, - 0,120,36,116,0,106,1,106,2,131,0,68,0,93,22,125, - 1,116,3,124,1,100,1,131,2,114,12,124,1,106,4,131, - 0,1,0,113,12,87,0,100,2,83,0,41,3,122,125,67, - 97,108,108,32,116,104,101,32,105,110,118,97,108,105,100,97, - 116,101,95,99,97,99,104,101,115,40,41,32,109,101,116,104, - 111,100,32,111,110,32,97,108,108,32,112,97,116,104,32,101, - 110,116,114,121,32,102,105,110,100,101,114,115,10,32,32,32, - 32,32,32,32,32,115,116,111,114,101,100,32,105,110,32,115, - 121,115,46,112,97,116,104,95,105,109,112,111,114,116,101,114, - 95,99,97,99,104,101,115,32,40,119,104,101,114,101,32,105, - 109,112,108,101,109,101,110,116,101,100,41,46,218,17,105,110, - 118,97,108,105,100,97,116,101,95,99,97,99,104,101,115,78, - 41,5,114,8,0,0,0,218,19,112,97,116,104,95,105,109, - 112,111,114,116,101,114,95,99,97,99,104,101,218,6,118,97, - 108,117,101,115,114,112,0,0,0,114,249,0,0,0,41,2, - 114,168,0,0,0,218,6,102,105,110,100,101,114,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, - 0,33,4,0,0,115,6,0,0,0,0,4,16,1,10,1, - 122,28,80,97,116,104,70,105,110,100,101,114,46,105,110,118, - 97,108,105,100,97,116,101,95,99,97,99,104,101,115,99,2, - 0,0,0,0,0,0,0,3,0,0,0,12,0,0,0,67, - 0,0,0,115,86,0,0,0,116,0,106,1,100,1,107,9, - 114,30,116,0,106,1,12,0,114,30,116,2,106,3,100,2, - 116,4,131,2,1,0,120,50,116,0,106,1,68,0,93,36, - 125,2,121,8,124,2,124,1,131,1,83,0,4,0,116,5, - 107,10,114,72,1,0,1,0,1,0,119,38,89,0,113,38, - 88,0,113,38,87,0,100,1,83,0,100,1,83,0,41,3, - 122,46,83,101,97,114,99,104,32,115,121,115,46,112,97,116, - 104,95,104,111,111,107,115,32,102,111,114,32,97,32,102,105, - 110,100,101,114,32,102,111,114,32,39,112,97,116,104,39,46, - 78,122,23,115,121,115,46,112,97,116,104,95,104,111,111,107, - 115,32,105,115,32,101,109,112,116,121,41,6,114,8,0,0, - 0,218,10,112,97,116,104,95,104,111,111,107,115,114,63,0, - 0,0,114,64,0,0,0,114,122,0,0,0,114,103,0,0, - 0,41,3,114,168,0,0,0,114,37,0,0,0,90,4,104, - 111,111,107,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,11,95,112,97,116,104,95,104,111,111,107,115,41, - 4,0,0,115,16,0,0,0,0,3,18,1,12,1,12,1, - 2,1,8,1,14,1,12,2,122,22,80,97,116,104,70,105, - 110,100,101,114,46,95,112,97,116,104,95,104,111,111,107,115, - 99,2,0,0,0,0,0,0,0,3,0,0,0,19,0,0, - 0,67,0,0,0,115,102,0,0,0,124,1,100,1,107,2, - 114,42,121,12,116,0,106,1,131,0,125,1,87,0,110,20, - 4,0,116,2,107,10,114,40,1,0,1,0,1,0,100,2, - 83,0,88,0,121,14,116,3,106,4,124,1,25,0,125,2, - 87,0,110,40,4,0,116,5,107,10,114,96,1,0,1,0, - 1,0,124,0,106,6,124,1,131,1,125,2,124,2,116,3, - 106,4,124,1,60,0,89,0,110,2,88,0,124,2,83,0, - 41,3,122,210,71,101,116,32,116,104,101,32,102,105,110,100, - 101,114,32,102,111,114,32,116,104,101,32,112,97,116,104,32, - 101,110,116,114,121,32,102,114,111,109,32,115,121,115,46,112, - 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, - 104,101,46,10,10,32,32,32,32,32,32,32,32,73,102,32, - 116,104,101,32,112,97,116,104,32,101,110,116,114,121,32,105, - 115,32,110,111,116,32,105,110,32,116,104,101,32,99,97,99, - 104,101,44,32,102,105,110,100,32,116,104,101,32,97,112,112, - 114,111,112,114,105,97,116,101,32,102,105,110,100,101,114,10, - 32,32,32,32,32,32,32,32,97,110,100,32,99,97,99,104, - 101,32,105,116,46,32,73,102,32,110,111,32,102,105,110,100, - 101,114,32,105,115,32,97,118,97,105,108,97,98,108,101,44, - 32,115,116,111,114,101,32,78,111,110,101,46,10,10,32,32, - 32,32,32,32,32,32,114,32,0,0,0,78,41,7,114,3, - 0,0,0,114,47,0,0,0,218,17,70,105,108,101,78,111, - 116,70,111,117,110,100,69,114,114,111,114,114,8,0,0,0, - 114,250,0,0,0,114,135,0,0,0,114,254,0,0,0,41, - 3,114,168,0,0,0,114,37,0,0,0,114,252,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,218, - 20,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, - 99,97,99,104,101,54,4,0,0,115,22,0,0,0,0,8, - 8,1,2,1,12,1,14,3,6,1,2,1,14,1,14,1, - 10,1,16,1,122,31,80,97,116,104,70,105,110,100,101,114, - 46,95,112,97,116,104,95,105,109,112,111,114,116,101,114,95, - 99,97,99,104,101,99,3,0,0,0,0,0,0,0,6,0, - 0,0,3,0,0,0,67,0,0,0,115,82,0,0,0,116, - 0,124,2,100,1,131,2,114,26,124,2,106,1,124,1,131, - 1,92,2,125,3,125,4,110,14,124,2,106,2,124,1,131, - 1,125,3,103,0,125,4,124,3,100,0,107,9,114,60,116, - 3,106,4,124,1,124,3,131,2,83,0,116,3,106,5,124, - 1,100,0,131,2,125,5,124,4,124,5,95,6,124,5,83, - 0,41,2,78,114,121,0,0,0,41,7,114,112,0,0,0, - 114,121,0,0,0,114,179,0,0,0,114,118,0,0,0,114, - 176,0,0,0,114,158,0,0,0,114,154,0,0,0,41,6, - 114,168,0,0,0,114,123,0,0,0,114,252,0,0,0,114, - 124,0,0,0,114,125,0,0,0,114,162,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,218,16,95, - 108,101,103,97,99,121,95,103,101,116,95,115,112,101,99,76, - 4,0,0,115,18,0,0,0,0,4,10,1,16,2,10,1, - 4,1,8,1,12,1,12,1,6,1,122,27,80,97,116,104, - 70,105,110,100,101,114,46,95,108,101,103,97,99,121,95,103, - 101,116,95,115,112,101,99,78,99,4,0,0,0,0,0,0, - 0,9,0,0,0,5,0,0,0,67,0,0,0,115,170,0, - 0,0,103,0,125,4,120,160,124,2,68,0,93,130,125,5, - 116,0,124,5,116,1,116,2,102,2,131,2,115,30,113,10, - 124,0,106,3,124,5,131,1,125,6,124,6,100,1,107,9, - 114,10,116,4,124,6,100,2,131,2,114,72,124,6,106,5, - 124,1,124,3,131,2,125,7,110,12,124,0,106,6,124,1, - 124,6,131,2,125,7,124,7,100,1,107,8,114,94,113,10, - 124,7,106,7,100,1,107,9,114,108,124,7,83,0,124,7, - 106,8,125,8,124,8,100,1,107,8,114,130,116,9,100,3, - 131,1,130,1,124,4,106,10,124,8,131,1,1,0,113,10, - 87,0,116,11,106,12,124,1,100,1,131,2,125,7,124,4, - 124,7,95,8,124,7,83,0,100,1,83,0,41,4,122,63, - 70,105,110,100,32,116,104,101,32,108,111,97,100,101,114,32, - 111,114,32,110,97,109,101,115,112,97,99,101,95,112,97,116, - 104,32,102,111,114,32,116,104,105,115,32,109,111,100,117,108, - 101,47,112,97,99,107,97,103,101,32,110,97,109,101,46,78, - 114,178,0,0,0,122,19,115,112,101,99,32,109,105,115,115, - 105,110,103,32,108,111,97,100,101,114,41,13,114,141,0,0, - 0,114,73,0,0,0,218,5,98,121,116,101,115,114,0,1, - 0,0,114,112,0,0,0,114,178,0,0,0,114,1,1,0, - 0,114,124,0,0,0,114,154,0,0,0,114,103,0,0,0, - 114,147,0,0,0,114,118,0,0,0,114,158,0,0,0,41, - 9,114,168,0,0,0,114,123,0,0,0,114,37,0,0,0, - 114,177,0,0,0,218,14,110,97,109,101,115,112,97,99,101, - 95,112,97,116,104,90,5,101,110,116,114,121,114,252,0,0, - 0,114,162,0,0,0,114,125,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,6,0,0,0,218,9,95,103,101,116, - 95,115,112,101,99,91,4,0,0,115,40,0,0,0,0,5, - 4,1,10,1,14,1,2,1,10,1,8,1,10,1,14,2, - 12,1,8,1,2,1,10,1,4,1,6,1,8,1,8,5, - 14,2,12,1,6,1,122,20,80,97,116,104,70,105,110,100, - 101,114,46,95,103,101,116,95,115,112,101,99,99,4,0,0, - 0,0,0,0,0,6,0,0,0,4,0,0,0,67,0,0, - 0,115,104,0,0,0,124,2,100,1,107,8,114,14,116,0, - 106,1,125,2,124,0,106,2,124,1,124,2,124,3,131,3, - 125,4,124,4,100,1,107,8,114,42,100,1,83,0,110,58, - 124,4,106,3,100,1,107,8,114,96,124,4,106,4,125,5, - 124,5,114,90,100,2,124,4,95,5,116,6,124,1,124,5, - 124,0,106,2,131,3,124,4,95,4,124,4,83,0,113,100, - 100,1,83,0,110,4,124,4,83,0,100,1,83,0,41,3, - 122,141,84,114,121,32,116,111,32,102,105,110,100,32,97,32, - 115,112,101,99,32,102,111,114,32,39,102,117,108,108,110,97, - 109,101,39,32,111,110,32,115,121,115,46,112,97,116,104,32, - 111,114,32,39,112,97,116,104,39,46,10,10,32,32,32,32, - 32,32,32,32,84,104,101,32,115,101,97,114,99,104,32,105, - 115,32,98,97,115,101,100,32,111,110,32,115,121,115,46,112, - 97,116,104,95,104,111,111,107,115,32,97,110,100,32,115,121, - 115,46,112,97,116,104,95,105,109,112,111,114,116,101,114,95, - 99,97,99,104,101,46,10,32,32,32,32,32,32,32,32,78, - 90,9,110,97,109,101,115,112,97,99,101,41,7,114,8,0, - 0,0,114,37,0,0,0,114,4,1,0,0,114,124,0,0, - 0,114,154,0,0,0,114,156,0,0,0,114,227,0,0,0, - 41,6,114,168,0,0,0,114,123,0,0,0,114,37,0,0, - 0,114,177,0,0,0,114,162,0,0,0,114,3,1,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, - 178,0,0,0,123,4,0,0,115,26,0,0,0,0,6,8, - 1,6,1,14,1,8,1,6,1,10,1,6,1,4,3,6, - 1,16,1,6,2,6,2,122,20,80,97,116,104,70,105,110, - 100,101,114,46,102,105,110,100,95,115,112,101,99,99,3,0, - 0,0,0,0,0,0,4,0,0,0,3,0,0,0,67,0, - 0,0,115,30,0,0,0,124,0,106,0,124,1,124,2,131, - 2,125,3,124,3,100,1,107,8,114,24,100,1,83,0,124, - 3,106,1,83,0,41,2,122,170,102,105,110,100,32,116,104, - 101,32,109,111,100,117,108,101,32,111,110,32,115,121,115,46, - 112,97,116,104,32,111,114,32,39,112,97,116,104,39,32,98, - 97,115,101,100,32,111,110,32,115,121,115,46,112,97,116,104, - 95,104,111,111,107,115,32,97,110,100,10,32,32,32,32,32, - 32,32,32,115,121,115,46,112,97,116,104,95,105,109,112,111, - 114,116,101,114,95,99,97,99,104,101,46,10,10,32,32,32, - 32,32,32,32,32,84,104,105,115,32,109,101,116,104,111,100, - 32,105,115,32,100,101,112,114,101,99,97,116,101,100,46,32, - 32,85,115,101,32,102,105,110,100,95,115,112,101,99,40,41, - 32,105,110,115,116,101,97,100,46,10,10,32,32,32,32,32, - 32,32,32,78,41,2,114,178,0,0,0,114,124,0,0,0, - 41,4,114,168,0,0,0,114,123,0,0,0,114,37,0,0, - 0,114,162,0,0,0,114,4,0,0,0,114,4,0,0,0, - 114,6,0,0,0,114,179,0,0,0,147,4,0,0,115,8, - 0,0,0,0,8,12,1,8,1,4,1,122,22,80,97,116, - 104,70,105,110,100,101,114,46,102,105,110,100,95,109,111,100, - 117,108,101,41,1,78,41,2,78,78,41,1,78,41,12,114, - 109,0,0,0,114,108,0,0,0,114,110,0,0,0,114,111, - 0,0,0,114,180,0,0,0,114,249,0,0,0,114,254,0, - 0,0,114,0,1,0,0,114,1,1,0,0,114,4,1,0, - 0,114,178,0,0,0,114,179,0,0,0,114,4,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, - 248,0,0,0,29,4,0,0,115,22,0,0,0,8,2,4, - 2,12,8,12,13,12,22,12,15,2,1,12,31,2,1,12, - 23,2,1,114,248,0,0,0,99,0,0,0,0,0,0,0, - 0,0,0,0,0,3,0,0,0,64,0,0,0,115,90,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 100,3,132,0,90,4,100,4,100,5,132,0,90,5,101,6, - 90,7,100,6,100,7,132,0,90,8,100,8,100,9,132,0, - 90,9,100,19,100,11,100,12,132,1,90,10,100,13,100,14, - 132,0,90,11,101,12,100,15,100,16,132,0,131,1,90,13, - 100,17,100,18,132,0,90,14,100,10,83,0,41,20,218,10, - 70,105,108,101,70,105,110,100,101,114,122,172,70,105,108,101, - 45,98,97,115,101,100,32,102,105,110,100,101,114,46,10,10, - 32,32,32,32,73,110,116,101,114,97,99,116,105,111,110,115, - 32,119,105,116,104,32,116,104,101,32,102,105,108,101,32,115, - 121,115,116,101,109,32,97,114,101,32,99,97,99,104,101,100, - 32,102,111,114,32,112,101,114,102,111,114,109,97,110,99,101, - 44,32,98,101,105,110,103,10,32,32,32,32,114,101,102,114, - 101,115,104,101,100,32,119,104,101,110,32,116,104,101,32,100, - 105,114,101,99,116,111,114,121,32,116,104,101,32,102,105,110, - 100,101,114,32,105,115,32,104,97,110,100,108,105,110,103,32, - 104,97,115,32,98,101,101,110,32,109,111,100,105,102,105,101, - 100,46,10,10,32,32,32,32,99,2,0,0,0,0,0,0, - 0,5,0,0,0,5,0,0,0,7,0,0,0,115,88,0, - 0,0,103,0,125,3,120,40,124,2,68,0,93,32,92,2, - 137,0,125,4,124,3,106,0,135,0,102,1,100,1,100,2, - 132,8,124,4,68,0,131,1,131,1,1,0,113,10,87,0, - 124,3,124,0,95,1,124,1,112,58,100,3,124,0,95,2, - 100,6,124,0,95,3,116,4,131,0,124,0,95,5,116,4, - 131,0,124,0,95,6,100,5,83,0,41,7,122,154,73,110, - 105,116,105,97,108,105,122,101,32,119,105,116,104,32,116,104, - 101,32,112,97,116,104,32,116,111,32,115,101,97,114,99,104, - 32,111,110,32,97,110,100,32,97,32,118,97,114,105,97,98, - 108,101,32,110,117,109,98,101,114,32,111,102,10,32,32,32, - 32,32,32,32,32,50,45,116,117,112,108,101,115,32,99,111, - 110,116,97,105,110,105,110,103,32,116,104,101,32,108,111,97, - 100,101,114,32,97,110,100,32,116,104,101,32,102,105,108,101, - 32,115,117,102,102,105,120,101,115,32,116,104,101,32,108,111, - 97,100,101,114,10,32,32,32,32,32,32,32,32,114,101,99, - 111,103,110,105,122,101,115,46,99,1,0,0,0,0,0,0, - 0,2,0,0,0,3,0,0,0,51,0,0,0,115,22,0, - 0,0,124,0,93,14,125,1,124,1,136,0,102,2,86,0, - 1,0,113,2,100,0,83,0,41,1,78,114,4,0,0,0, - 41,2,114,24,0,0,0,114,222,0,0,0,41,1,114,124, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,224,0, - 0,0,176,4,0,0,115,2,0,0,0,4,0,122,38,70, - 105,108,101,70,105,110,100,101,114,46,95,95,105,110,105,116, - 95,95,46,60,108,111,99,97,108,115,62,46,60,103,101,110, - 101,120,112,114,62,114,61,0,0,0,114,31,0,0,0,78, - 114,91,0,0,0,41,7,114,147,0,0,0,218,8,95,108, - 111,97,100,101,114,115,114,37,0,0,0,218,11,95,112,97, - 116,104,95,109,116,105,109,101,218,3,115,101,116,218,11,95, - 112,97,116,104,95,99,97,99,104,101,218,19,95,114,101,108, - 97,120,101,100,95,112,97,116,104,95,99,97,99,104,101,41, - 5,114,104,0,0,0,114,37,0,0,0,218,14,108,111,97, - 100,101,114,95,100,101,116,97,105,108,115,90,7,108,111,97, - 100,101,114,115,114,164,0,0,0,114,4,0,0,0,41,1, - 114,124,0,0,0,114,6,0,0,0,114,182,0,0,0,170, - 4,0,0,115,16,0,0,0,0,4,4,1,14,1,28,1, - 6,2,10,1,6,1,8,1,122,19,70,105,108,101,70,105, - 110,100,101,114,46,95,95,105,110,105,116,95,95,99,1,0, - 0,0,0,0,0,0,1,0,0,0,2,0,0,0,67,0, - 0,0,115,10,0,0,0,100,3,124,0,95,0,100,2,83, - 0,41,4,122,31,73,110,118,97,108,105,100,97,116,101,32, - 116,104,101,32,100,105,114,101,99,116,111,114,121,32,109,116, - 105,109,101,46,114,31,0,0,0,78,114,91,0,0,0,41, - 1,114,7,1,0,0,41,1,114,104,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,114,249,0,0, - 0,184,4,0,0,115,2,0,0,0,0,2,122,28,70,105, - 108,101,70,105,110,100,101,114,46,105,110,118,97,108,105,100, - 97,116,101,95,99,97,99,104,101,115,99,2,0,0,0,0, - 0,0,0,3,0,0,0,2,0,0,0,67,0,0,0,115, - 42,0,0,0,124,0,106,0,124,1,131,1,125,2,124,2, - 100,1,107,8,114,26,100,1,103,0,102,2,83,0,124,2, - 106,1,124,2,106,2,112,38,103,0,102,2,83,0,41,2, - 122,197,84,114,121,32,116,111,32,102,105,110,100,32,97,32, - 108,111,97,100,101,114,32,102,111,114,32,116,104,101,32,115, - 112,101,99,105,102,105,101,100,32,109,111,100,117,108,101,44, - 32,111,114,32,116,104,101,32,110,97,109,101,115,112,97,99, - 101,10,32,32,32,32,32,32,32,32,112,97,99,107,97,103, - 101,32,112,111,114,116,105,111,110,115,46,32,82,101,116,117, - 114,110,115,32,40,108,111,97,100,101,114,44,32,108,105,115, - 116,45,111,102,45,112,111,114,116,105,111,110,115,41,46,10, - 10,32,32,32,32,32,32,32,32,84,104,105,115,32,109,101, - 116,104,111,100,32,105,115,32,100,101,112,114,101,99,97,116, - 101,100,46,32,32,85,115,101,32,102,105,110,100,95,115,112, - 101,99,40,41,32,105,110,115,116,101,97,100,46,10,10,32, - 32,32,32,32,32,32,32,78,41,3,114,178,0,0,0,114, - 124,0,0,0,114,154,0,0,0,41,3,114,104,0,0,0, - 114,123,0,0,0,114,162,0,0,0,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,121,0,0,0,190,4, - 0,0,115,8,0,0,0,0,7,10,1,8,1,8,1,122, - 22,70,105,108,101,70,105,110,100,101,114,46,102,105,110,100, - 95,108,111,97,100,101,114,99,6,0,0,0,0,0,0,0, - 7,0,0,0,6,0,0,0,67,0,0,0,115,26,0,0, - 0,124,1,124,2,124,3,131,2,125,6,116,0,124,2,124, - 3,124,6,124,4,100,1,141,4,83,0,41,2,78,41,2, - 114,124,0,0,0,114,154,0,0,0,41,1,114,165,0,0, - 0,41,7,114,104,0,0,0,114,163,0,0,0,114,123,0, - 0,0,114,37,0,0,0,90,4,115,109,115,108,114,177,0, - 0,0,114,124,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,4,1,0,0,202,4,0,0,115, - 6,0,0,0,0,1,10,1,8,1,122,20,70,105,108,101, - 70,105,110,100,101,114,46,95,103,101,116,95,115,112,101,99, - 78,99,3,0,0,0,0,0,0,0,14,0,0,0,15,0, - 0,0,67,0,0,0,115,98,1,0,0,100,1,125,3,124, - 1,106,0,100,2,131,1,100,3,25,0,125,4,121,24,116, - 1,124,0,106,2,112,34,116,3,106,4,131,0,131,1,106, - 5,125,5,87,0,110,24,4,0,116,6,107,10,114,66,1, - 0,1,0,1,0,100,10,125,5,89,0,110,2,88,0,124, - 5,124,0,106,7,107,3,114,92,124,0,106,8,131,0,1, - 0,124,5,124,0,95,7,116,9,131,0,114,114,124,0,106, - 10,125,6,124,4,106,11,131,0,125,7,110,10,124,0,106, - 12,125,6,124,4,125,7,124,7,124,6,107,6,114,218,116, - 13,124,0,106,2,124,4,131,2,125,8,120,72,124,0,106, - 14,68,0,93,54,92,2,125,9,125,10,100,5,124,9,23, - 0,125,11,116,13,124,8,124,11,131,2,125,12,116,15,124, - 12,131,1,114,152,124,0,106,16,124,10,124,1,124,12,124, - 8,103,1,124,2,131,5,83,0,113,152,87,0,116,17,124, - 8,131,1,125,3,120,88,124,0,106,14,68,0,93,78,92, - 2,125,9,125,10,116,13,124,0,106,2,124,4,124,9,23, - 0,131,2,125,12,116,18,106,19,100,6,124,12,100,3,100, - 7,141,3,1,0,124,7,124,9,23,0,124,6,107,6,114, - 226,116,15,124,12,131,1,114,226,124,0,106,16,124,10,124, - 1,124,12,100,8,124,2,131,5,83,0,113,226,87,0,124, - 3,144,1,114,94,116,18,106,19,100,9,124,8,131,2,1, - 0,116,18,106,20,124,1,100,8,131,2,125,13,124,8,103, - 1,124,13,95,21,124,13,83,0,100,8,83,0,41,11,122, - 111,84,114,121,32,116,111,32,102,105,110,100,32,97,32,115, - 112,101,99,32,102,111,114,32,116,104,101,32,115,112,101,99, - 105,102,105,101,100,32,109,111,100,117,108,101,46,10,10,32, - 32,32,32,32,32,32,32,82,101,116,117,114,110,115,32,116, - 104,101,32,109,97,116,99,104,105,110,103,32,115,112,101,99, - 44,32,111,114,32,78,111,110,101,32,105,102,32,110,111,116, - 32,102,111,117,110,100,46,10,32,32,32,32,32,32,32,32, - 70,114,61,0,0,0,114,59,0,0,0,114,31,0,0,0, - 114,182,0,0,0,122,9,116,114,121,105,110,103,32,123,125, - 41,1,90,9,118,101,114,98,111,115,105,116,121,78,122,25, - 112,111,115,115,105,98,108,101,32,110,97,109,101,115,112,97, - 99,101,32,102,111,114,32,123,125,114,91,0,0,0,41,22, - 114,34,0,0,0,114,41,0,0,0,114,37,0,0,0,114, - 3,0,0,0,114,47,0,0,0,114,216,0,0,0,114,42, - 0,0,0,114,7,1,0,0,218,11,95,102,105,108,108,95, - 99,97,99,104,101,114,7,0,0,0,114,10,1,0,0,114, - 92,0,0,0,114,9,1,0,0,114,30,0,0,0,114,6, - 1,0,0,114,46,0,0,0,114,4,1,0,0,114,48,0, - 0,0,114,118,0,0,0,114,133,0,0,0,114,158,0,0, - 0,114,154,0,0,0,41,14,114,104,0,0,0,114,123,0, - 0,0,114,177,0,0,0,90,12,105,115,95,110,97,109,101, - 115,112,97,99,101,90,11,116,97,105,108,95,109,111,100,117, - 108,101,114,130,0,0,0,90,5,99,97,99,104,101,90,12, - 99,97,99,104,101,95,109,111,100,117,108,101,90,9,98,97, - 115,101,95,112,97,116,104,114,222,0,0,0,114,163,0,0, - 0,90,13,105,110,105,116,95,102,105,108,101,110,97,109,101, - 90,9,102,117,108,108,95,112,97,116,104,114,162,0,0,0, - 114,4,0,0,0,114,4,0,0,0,114,6,0,0,0,114, - 178,0,0,0,207,4,0,0,115,70,0,0,0,0,5,4, - 1,14,1,2,1,24,1,14,1,10,1,10,1,8,1,6, - 2,6,1,6,1,10,2,6,1,4,2,8,1,12,1,16, - 1,8,1,10,1,8,1,24,4,8,2,16,1,16,1,16, - 1,12,1,8,1,10,1,12,1,6,1,12,1,12,1,8, - 1,4,1,122,20,70,105,108,101,70,105,110,100,101,114,46, - 102,105,110,100,95,115,112,101,99,99,1,0,0,0,0,0, - 0,0,9,0,0,0,13,0,0,0,67,0,0,0,115,194, - 0,0,0,124,0,106,0,125,1,121,22,116,1,106,2,124, - 1,112,22,116,1,106,3,131,0,131,1,125,2,87,0,110, - 30,4,0,116,4,116,5,116,6,102,3,107,10,114,58,1, - 0,1,0,1,0,103,0,125,2,89,0,110,2,88,0,116, - 7,106,8,106,9,100,1,131,1,115,84,116,10,124,2,131, - 1,124,0,95,11,110,78,116,10,131,0,125,3,120,64,124, - 2,68,0,93,56,125,4,124,4,106,12,100,2,131,1,92, - 3,125,5,125,6,125,7,124,6,114,138,100,3,106,13,124, - 5,124,7,106,14,131,0,131,2,125,8,110,4,124,5,125, - 8,124,3,106,15,124,8,131,1,1,0,113,96,87,0,124, - 3,124,0,95,11,116,7,106,8,106,9,116,16,131,1,114, - 190,100,4,100,5,132,0,124,2,68,0,131,1,124,0,95, - 17,100,6,83,0,41,7,122,68,70,105,108,108,32,116,104, - 101,32,99,97,99,104,101,32,111,102,32,112,111,116,101,110, - 116,105,97,108,32,109,111,100,117,108,101,115,32,97,110,100, - 32,112,97,99,107,97,103,101,115,32,102,111,114,32,116,104, - 105,115,32,100,105,114,101,99,116,111,114,121,46,114,0,0, - 0,0,114,61,0,0,0,122,5,123,125,46,123,125,99,1, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,83, - 0,0,0,115,20,0,0,0,104,0,124,0,93,12,125,1, - 124,1,106,0,131,0,146,2,113,4,83,0,114,4,0,0, - 0,41,1,114,92,0,0,0,41,2,114,24,0,0,0,90, - 2,102,110,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,250,9,60,115,101,116,99,111,109,112,62,28,5,0, - 0,115,2,0,0,0,6,0,122,41,70,105,108,101,70,105, - 110,100,101,114,46,95,102,105,108,108,95,99,97,99,104,101, - 46,60,108,111,99,97,108,115,62,46,60,115,101,116,99,111, - 109,112,62,78,41,18,114,37,0,0,0,114,3,0,0,0, - 90,7,108,105,115,116,100,105,114,114,47,0,0,0,114,255, - 0,0,0,218,15,80,101,114,109,105,115,115,105,111,110,69, - 114,114,111,114,218,18,78,111,116,65,68,105,114,101,99,116, - 111,114,121,69,114,114,111,114,114,8,0,0,0,114,9,0, - 0,0,114,10,0,0,0,114,8,1,0,0,114,9,1,0, - 0,114,87,0,0,0,114,50,0,0,0,114,92,0,0,0, - 218,3,97,100,100,114,11,0,0,0,114,10,1,0,0,41, - 9,114,104,0,0,0,114,37,0,0,0,90,8,99,111,110, - 116,101,110,116,115,90,21,108,111,119,101,114,95,115,117,102, - 102,105,120,95,99,111,110,116,101,110,116,115,114,244,0,0, - 0,114,102,0,0,0,114,234,0,0,0,114,222,0,0,0, - 90,8,110,101,119,95,110,97,109,101,114,4,0,0,0,114, - 4,0,0,0,114,6,0,0,0,114,12,1,0,0,255,4, - 0,0,115,34,0,0,0,0,2,6,1,2,1,22,1,20, - 3,10,3,12,1,12,7,6,1,10,1,16,1,4,1,18, - 2,4,1,14,1,6,1,12,1,122,22,70,105,108,101,70, - 105,110,100,101,114,46,95,102,105,108,108,95,99,97,99,104, - 101,99,1,0,0,0,0,0,0,0,3,0,0,0,3,0, - 0,0,7,0,0,0,115,18,0,0,0,135,0,135,1,102, - 2,100,1,100,2,132,8,125,2,124,2,83,0,41,3,97, - 20,1,0,0,65,32,99,108,97,115,115,32,109,101,116,104, - 111,100,32,119,104,105,99,104,32,114,101,116,117,114,110,115, - 32,97,32,99,108,111,115,117,114,101,32,116,111,32,117,115, - 101,32,111,110,32,115,121,115,46,112,97,116,104,95,104,111, - 111,107,10,32,32,32,32,32,32,32,32,119,104,105,99,104, - 32,119,105,108,108,32,114,101,116,117,114,110,32,97,110,32, - 105,110,115,116,97,110,99,101,32,117,115,105,110,103,32,116, - 104,101,32,115,112,101,99,105,102,105,101,100,32,108,111,97, - 100,101,114,115,32,97,110,100,32,116,104,101,32,112,97,116, - 104,10,32,32,32,32,32,32,32,32,99,97,108,108,101,100, - 32,111,110,32,116,104,101,32,99,108,111,115,117,114,101,46, - 10,10,32,32,32,32,32,32,32,32,73,102,32,116,104,101, - 32,112,97,116,104,32,99,97,108,108,101,100,32,111,110,32, - 116,104,101,32,99,108,111,115,117,114,101,32,105,115,32,110, - 111,116,32,97,32,100,105,114,101,99,116,111,114,121,44,32, - 73,109,112,111,114,116,69,114,114,111,114,32,105,115,10,32, - 32,32,32,32,32,32,32,114,97,105,115,101,100,46,10,10, - 32,32,32,32,32,32,32,32,99,1,0,0,0,0,0,0, - 0,1,0,0,0,4,0,0,0,19,0,0,0,115,34,0, - 0,0,116,0,124,0,131,1,115,20,116,1,100,1,124,0, - 100,2,141,2,130,1,136,0,124,0,102,1,136,1,152,2, - 142,0,83,0,41,3,122,45,80,97,116,104,32,104,111,111, - 107,32,102,111,114,32,105,109,112,111,114,116,108,105,98,46, - 109,97,99,104,105,110,101,114,121,46,70,105,108,101,70,105, - 110,100,101,114,46,122,30,111,110,108,121,32,100,105,114,101, - 99,116,111,114,105,101,115,32,97,114,101,32,115,117,112,112, - 111,114,116,101,100,41,1,114,37,0,0,0,41,2,114,48, - 0,0,0,114,103,0,0,0,41,1,114,37,0,0,0,41, - 2,114,168,0,0,0,114,11,1,0,0,114,4,0,0,0, - 114,6,0,0,0,218,24,112,97,116,104,95,104,111,111,107, - 95,102,111,114,95,70,105,108,101,70,105,110,100,101,114,40, - 5,0,0,115,6,0,0,0,0,2,8,1,12,1,122,54, - 70,105,108,101,70,105,110,100,101,114,46,112,97,116,104,95, - 104,111,111,107,46,60,108,111,99,97,108,115,62,46,112,97, - 116,104,95,104,111,111,107,95,102,111,114,95,70,105,108,101, - 70,105,110,100,101,114,114,4,0,0,0,41,3,114,168,0, - 0,0,114,11,1,0,0,114,17,1,0,0,114,4,0,0, - 0,41,2,114,168,0,0,0,114,11,1,0,0,114,6,0, - 0,0,218,9,112,97,116,104,95,104,111,111,107,30,5,0, - 0,115,4,0,0,0,0,10,14,6,122,20,70,105,108,101, - 70,105,110,100,101,114,46,112,97,116,104,95,104,111,111,107, - 99,1,0,0,0,0,0,0,0,1,0,0,0,2,0,0, - 0,67,0,0,0,115,12,0,0,0,100,1,106,0,124,0, - 106,1,131,1,83,0,41,2,78,122,16,70,105,108,101,70, - 105,110,100,101,114,40,123,33,114,125,41,41,2,114,50,0, - 0,0,114,37,0,0,0,41,1,114,104,0,0,0,114,4, - 0,0,0,114,4,0,0,0,114,6,0,0,0,114,243,0, - 0,0,48,5,0,0,115,2,0,0,0,0,1,122,19,70, - 105,108,101,70,105,110,100,101,114,46,95,95,114,101,112,114, - 95,95,41,1,78,41,15,114,109,0,0,0,114,108,0,0, - 0,114,110,0,0,0,114,111,0,0,0,114,182,0,0,0, - 114,249,0,0,0,114,127,0,0,0,114,179,0,0,0,114, - 121,0,0,0,114,4,1,0,0,114,178,0,0,0,114,12, - 1,0,0,114,180,0,0,0,114,18,1,0,0,114,243,0, - 0,0,114,4,0,0,0,114,4,0,0,0,114,4,0,0, - 0,114,6,0,0,0,114,5,1,0,0,161,4,0,0,115, - 20,0,0,0,8,7,4,2,8,14,8,4,4,2,8,12, - 8,5,10,48,8,31,12,18,114,5,1,0,0,99,4,0, - 0,0,0,0,0,0,6,0,0,0,11,0,0,0,67,0, - 0,0,115,146,0,0,0,124,0,106,0,100,1,131,1,125, - 4,124,0,106,0,100,2,131,1,125,5,124,4,115,66,124, - 5,114,36,124,5,106,1,125,4,110,30,124,2,124,3,107, - 2,114,56,116,2,124,1,124,2,131,2,125,4,110,10,116, - 3,124,1,124,2,131,2,125,4,124,5,115,84,116,4,124, - 1,124,2,124,4,100,3,141,3,125,5,121,36,124,5,124, - 0,100,2,60,0,124,4,124,0,100,1,60,0,124,2,124, - 0,100,4,60,0,124,3,124,0,100,5,60,0,87,0,110, - 20,4,0,116,5,107,10,114,140,1,0,1,0,1,0,89, - 0,110,2,88,0,100,0,83,0,41,6,78,218,10,95,95, - 108,111,97,100,101,114,95,95,218,8,95,95,115,112,101,99, - 95,95,41,1,114,124,0,0,0,90,8,95,95,102,105,108, - 101,95,95,90,10,95,95,99,97,99,104,101,100,95,95,41, - 6,218,3,103,101,116,114,124,0,0,0,114,220,0,0,0, - 114,215,0,0,0,114,165,0,0,0,218,9,69,120,99,101, - 112,116,105,111,110,41,6,90,2,110,115,114,102,0,0,0, - 90,8,112,97,116,104,110,97,109,101,90,9,99,112,97,116, - 104,110,97,109,101,114,124,0,0,0,114,162,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,218,14, - 95,102,105,120,95,117,112,95,109,111,100,117,108,101,54,5, - 0,0,115,34,0,0,0,0,2,10,1,10,1,4,1,4, - 1,8,1,8,1,12,2,10,1,4,1,14,1,2,1,8, - 1,8,1,8,1,12,1,14,2,114,23,1,0,0,99,0, - 0,0,0,0,0,0,0,3,0,0,0,3,0,0,0,67, - 0,0,0,115,38,0,0,0,116,0,116,1,106,2,131,0, - 102,2,125,0,116,3,116,4,102,2,125,1,116,5,116,6, - 102,2,125,2,124,0,124,1,124,2,103,3,83,0,41,1, - 122,95,82,101,116,117,114,110,115,32,97,32,108,105,115,116, - 32,111,102,32,102,105,108,101,45,98,97,115,101,100,32,109, - 111,100,117,108,101,32,108,111,97,100,101,114,115,46,10,10, - 32,32,32,32,69,97,99,104,32,105,116,101,109,32,105,115, - 32,97,32,116,117,112,108,101,32,40,108,111,97,100,101,114, - 44,32,115,117,102,102,105,120,101,115,41,46,10,32,32,32, - 32,41,7,114,221,0,0,0,114,143,0,0,0,218,18,101, - 120,116,101,110,115,105,111,110,95,115,117,102,102,105,120,101, - 115,114,215,0,0,0,114,88,0,0,0,114,220,0,0,0, - 114,78,0,0,0,41,3,90,10,101,120,116,101,110,115,105, - 111,110,115,90,6,115,111,117,114,99,101,90,8,98,121,116, - 101,99,111,100,101,114,4,0,0,0,114,4,0,0,0,114, - 6,0,0,0,114,159,0,0,0,77,5,0,0,115,8,0, - 0,0,0,5,12,1,8,1,8,1,114,159,0,0,0,99, - 1,0,0,0,0,0,0,0,12,0,0,0,12,0,0,0, - 67,0,0,0,115,188,1,0,0,124,0,97,0,116,0,106, - 1,97,1,116,0,106,2,97,2,116,1,106,3,116,4,25, - 0,125,1,120,56,100,26,68,0,93,48,125,2,124,2,116, - 1,106,3,107,7,114,58,116,0,106,5,124,2,131,1,125, - 3,110,10,116,1,106,3,124,2,25,0,125,3,116,6,124, - 1,124,2,124,3,131,3,1,0,113,32,87,0,100,5,100, - 6,103,1,102,2,100,7,100,8,100,6,103,2,102,2,102, - 2,125,4,120,118,124,4,68,0,93,102,92,2,125,5,125, - 6,116,7,100,9,100,10,132,0,124,6,68,0,131,1,131, - 1,115,142,116,8,130,1,124,6,100,11,25,0,125,7,124, - 5,116,1,106,3,107,6,114,174,116,1,106,3,124,5,25, - 0,125,8,80,0,113,112,121,16,116,0,106,5,124,5,131, - 1,125,8,80,0,87,0,113,112,4,0,116,9,107,10,114, - 212,1,0,1,0,1,0,119,112,89,0,113,112,88,0,113, - 112,87,0,116,9,100,12,131,1,130,1,116,6,124,1,100, - 13,124,8,131,3,1,0,116,6,124,1,100,14,124,7,131, - 3,1,0,116,6,124,1,100,15,100,16,106,10,124,6,131, - 1,131,3,1,0,121,14,116,0,106,5,100,17,131,1,125, - 9,87,0,110,26,4,0,116,9,107,10,144,1,114,52,1, - 0,1,0,1,0,100,18,125,9,89,0,110,2,88,0,116, - 6,124,1,100,17,124,9,131,3,1,0,116,0,106,5,100, - 19,131,1,125,10,116,6,124,1,100,19,124,10,131,3,1, - 0,124,5,100,7,107,2,144,1,114,120,116,0,106,5,100, - 20,131,1,125,11,116,6,124,1,100,21,124,11,131,3,1, - 0,116,6,124,1,100,22,116,11,131,0,131,3,1,0,116, - 12,106,13,116,2,106,14,131,0,131,1,1,0,124,5,100, - 7,107,2,144,1,114,184,116,15,106,16,100,23,131,1,1, - 0,100,24,116,12,107,6,144,1,114,184,100,25,116,17,95, - 18,100,18,83,0,41,27,122,205,83,101,116,117,112,32,116, - 104,101,32,112,97,116,104,45,98,97,115,101,100,32,105,109, - 112,111,114,116,101,114,115,32,102,111,114,32,105,109,112,111, - 114,116,108,105,98,32,98,121,32,105,109,112,111,114,116,105, - 110,103,32,110,101,101,100,101,100,10,32,32,32,32,98,117, - 105,108,116,45,105,110,32,109,111,100,117,108,101,115,32,97, - 110,100,32,105,110,106,101,99,116,105,110,103,32,116,104,101, - 109,32,105,110,116,111,32,116,104,101,32,103,108,111,98,97, - 108,32,110,97,109,101,115,112,97,99,101,46,10,10,32,32, - 32,32,79,116,104,101,114,32,99,111,109,112,111,110,101,110, - 116,115,32,97,114,101,32,101,120,116,114,97,99,116,101,100, - 32,102,114,111,109,32,116,104,101,32,99,111,114,101,32,98, - 111,111,116,115,116,114,97,112,32,109,111,100,117,108,101,46, - 10,10,32,32,32,32,114,52,0,0,0,114,63,0,0,0, - 218,8,98,117,105,108,116,105,110,115,114,140,0,0,0,90, - 5,112,111,115,105,120,250,1,47,218,2,110,116,250,1,92, - 99,1,0,0,0,0,0,0,0,2,0,0,0,3,0,0, - 0,115,0,0,0,115,26,0,0,0,124,0,93,18,125,1, - 116,0,124,1,131,1,100,0,107,2,86,0,1,0,113,2, - 100,1,83,0,41,2,114,31,0,0,0,78,41,1,114,33, - 0,0,0,41,2,114,24,0,0,0,114,81,0,0,0,114, - 4,0,0,0,114,4,0,0,0,114,6,0,0,0,114,224, - 0,0,0,113,5,0,0,115,2,0,0,0,4,0,122,25, - 95,115,101,116,117,112,46,60,108,111,99,97,108,115,62,46, - 60,103,101,110,101,120,112,114,62,114,62,0,0,0,122,30, - 105,109,112,111,114,116,108,105,98,32,114,101,113,117,105,114, - 101,115,32,112,111,115,105,120,32,111,114,32,110,116,114,3, - 0,0,0,114,27,0,0,0,114,23,0,0,0,114,32,0, - 0,0,90,7,95,116,104,114,101,97,100,78,90,8,95,119, - 101,97,107,114,101,102,90,6,119,105,110,114,101,103,114,167, - 0,0,0,114,7,0,0,0,122,4,46,112,121,119,122,6, - 95,100,46,112,121,100,84,41,4,122,3,95,105,111,122,9, - 95,119,97,114,110,105,110,103,115,122,8,98,117,105,108,116, - 105,110,115,122,7,109,97,114,115,104,97,108,41,19,114,118, - 0,0,0,114,8,0,0,0,114,143,0,0,0,114,236,0, - 0,0,114,109,0,0,0,90,18,95,98,117,105,108,116,105, - 110,95,102,114,111,109,95,110,97,109,101,114,113,0,0,0, - 218,3,97,108,108,218,14,65,115,115,101,114,116,105,111,110, - 69,114,114,111,114,114,103,0,0,0,114,28,0,0,0,114, - 13,0,0,0,114,226,0,0,0,114,147,0,0,0,114,24, - 1,0,0,114,88,0,0,0,114,161,0,0,0,114,166,0, - 0,0,114,170,0,0,0,41,12,218,17,95,98,111,111,116, - 115,116,114,97,112,95,109,111,100,117,108,101,90,11,115,101, - 108,102,95,109,111,100,117,108,101,90,12,98,117,105,108,116, - 105,110,95,110,97,109,101,90,14,98,117,105,108,116,105,110, - 95,109,111,100,117,108,101,90,10,111,115,95,100,101,116,97, - 105,108,115,90,10,98,117,105,108,116,105,110,95,111,115,114, - 23,0,0,0,114,27,0,0,0,90,9,111,115,95,109,111, - 100,117,108,101,90,13,116,104,114,101,97,100,95,109,111,100, - 117,108,101,90,14,119,101,97,107,114,101,102,95,109,111,100, - 117,108,101,90,13,119,105,110,114,101,103,95,109,111,100,117, - 108,101,114,4,0,0,0,114,4,0,0,0,114,6,0,0, - 0,218,6,95,115,101,116,117,112,88,5,0,0,115,82,0, - 0,0,0,8,4,1,6,1,6,3,10,1,10,1,10,1, - 12,2,10,1,16,3,22,1,14,2,22,1,8,1,10,1, - 10,1,4,2,2,1,10,1,6,1,14,1,12,2,8,1, - 12,1,12,1,18,3,2,1,14,1,16,2,10,1,12,3, - 10,1,12,3,10,1,10,1,12,3,14,1,14,1,10,1, - 10,1,10,1,114,32,1,0,0,99,1,0,0,0,0,0, - 0,0,2,0,0,0,3,0,0,0,67,0,0,0,115,74, - 0,0,0,116,0,124,0,131,1,1,0,116,1,131,0,125, - 1,116,2,106,3,106,4,116,5,106,6,124,1,152,1,142, - 0,103,1,131,1,1,0,116,7,106,8,100,1,107,2,114, - 58,116,2,106,9,106,10,116,11,131,1,1,0,116,2,106, - 9,106,10,116,12,131,1,1,0,100,2,83,0,41,3,122, - 41,73,110,115,116,97,108,108,32,116,104,101,32,112,97,116, - 104,45,98,97,115,101,100,32,105,109,112,111,114,116,32,99, - 111,109,112,111,110,101,110,116,115,46,114,27,1,0,0,78, - 41,13,114,32,1,0,0,114,159,0,0,0,114,8,0,0, - 0,114,253,0,0,0,114,147,0,0,0,114,5,1,0,0, - 114,18,1,0,0,114,3,0,0,0,114,109,0,0,0,218, - 9,109,101,116,97,95,112,97,116,104,114,161,0,0,0,114, - 166,0,0,0,114,248,0,0,0,41,2,114,31,1,0,0, - 90,17,115,117,112,112,111,114,116,101,100,95,108,111,97,100, - 101,114,115,114,4,0,0,0,114,4,0,0,0,114,6,0, - 0,0,218,8,95,105,110,115,116,97,108,108,156,5,0,0, - 115,12,0,0,0,0,2,8,1,6,1,22,1,10,1,12, - 1,114,34,1,0,0,41,1,122,3,119,105,110,41,2,114, - 1,0,0,0,114,2,0,0,0,41,1,114,49,0,0,0, - 41,1,78,41,3,78,78,78,41,3,78,78,78,41,2,114, - 62,0,0,0,114,62,0,0,0,41,1,78,41,1,78,41, - 58,114,111,0,0,0,114,12,0,0,0,90,37,95,67,65, - 83,69,95,73,78,83,69,78,83,73,84,73,86,69,95,80, - 76,65,84,70,79,82,77,83,95,66,89,84,69,83,95,75, - 69,89,114,11,0,0,0,114,13,0,0,0,114,19,0,0, - 0,114,21,0,0,0,114,30,0,0,0,114,40,0,0,0, - 114,41,0,0,0,114,45,0,0,0,114,46,0,0,0,114, - 48,0,0,0,114,58,0,0,0,218,4,116,121,112,101,218, - 8,95,95,99,111,100,101,95,95,114,142,0,0,0,114,17, - 0,0,0,114,132,0,0,0,114,16,0,0,0,114,20,0, - 0,0,90,17,95,82,65,87,95,77,65,71,73,67,95,78, - 85,77,66,69,82,114,77,0,0,0,114,76,0,0,0,114, - 88,0,0,0,114,78,0,0,0,90,23,68,69,66,85,71, - 95,66,89,84,69,67,79,68,69,95,83,85,70,70,73,88, - 69,83,90,27,79,80,84,73,77,73,90,69,68,95,66,89, - 84,69,67,79,68,69,95,83,85,70,70,73,88,69,83,114, - 83,0,0,0,114,89,0,0,0,114,95,0,0,0,114,99, - 0,0,0,114,101,0,0,0,114,120,0,0,0,114,127,0, - 0,0,114,139,0,0,0,114,145,0,0,0,114,148,0,0, - 0,114,153,0,0,0,218,6,111,98,106,101,99,116,114,160, - 0,0,0,114,165,0,0,0,114,166,0,0,0,114,181,0, - 0,0,114,191,0,0,0,114,207,0,0,0,114,215,0,0, - 0,114,220,0,0,0,114,226,0,0,0,114,221,0,0,0, - 114,227,0,0,0,114,246,0,0,0,114,248,0,0,0,114, - 5,1,0,0,114,23,1,0,0,114,159,0,0,0,114,32, - 1,0,0,114,34,1,0,0,114,4,0,0,0,114,4,0, - 0,0,114,4,0,0,0,114,6,0,0,0,218,8,60,109, - 111,100,117,108,101,62,8,0,0,0,115,108,0,0,0,4, - 16,4,1,4,1,2,1,6,3,8,17,8,5,8,5,8, - 6,8,12,8,10,8,9,8,5,8,7,10,22,10,121,16, - 1,12,2,4,1,4,2,6,2,6,2,8,2,16,45,8, - 34,8,19,8,12,8,12,8,28,8,17,10,55,10,12,10, - 10,8,14,6,3,4,1,14,67,14,64,14,29,16,110,14, - 41,18,45,18,16,4,3,18,53,14,60,14,42,14,127,0, - 5,14,127,0,22,10,23,8,11,8,68, + 0,0,218,8,60,109,111,100,117,108,101,62,8,0,0,0, + 115,108,0,0,0,4,16,4,1,4,1,2,1,6,3,8, + 17,8,5,8,5,8,6,8,12,8,10,8,9,8,5,8, + 7,10,22,10,121,16,1,12,2,4,1,4,2,6,2,6, + 2,8,2,16,45,8,34,8,19,8,12,8,12,8,28,8, + 17,10,55,10,12,10,10,8,14,6,3,4,1,14,67,14, + 64,14,29,16,110,14,41,18,45,18,16,4,3,18,53,14, + 60,14,42,14,127,0,5,14,127,0,22,10,23,8,11,8, + 68, }; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 17:54:46 2016 From: python-checkins at python.org (ethan.furman) Date: Sun, 11 Sep 2016 21:54:46 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_issue28082=3A_better_name_?= =?utf-8?q?for_Flag?= Message-ID: <20160911215445.13637.18017.16C567C4@psf.io> https://hg.python.org/cpython/rev/7369ec91d0f7 changeset: 103660:7369ec91d0f7 user: Ethan Furman date: Sun Sep 11 14:54:27 2016 -0700 summary: issue28082: better name for Flag files: Lib/re.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Lib/re.py b/Lib/re.py --- a/Lib/re.py +++ b/Lib/re.py @@ -138,7 +138,7 @@ __version__ = "2.2.1" -class Flag(enum.IntFlag): +class RegexFlag(enum.IntFlag): ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale" IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale @@ -157,7 +157,7 @@ TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking T = TEMPLATE DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation -globals().update(Flag.__members__) +globals().update(RegexFlag.__members__) # sre exception error = sre_compile.error -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 18:01:58 2016 From: python-checkins at python.org (christian.heimes) Date: Sun, 11 Sep 2016 22:01:58 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328085=3A_Add_PROT?= =?utf-8?q?OCOL=5FTLS=5FCLIENT_and_PROTOCOL=5FTLS=5FSERVER_for_SSLContext?= Message-ID: <20160911220158.21124.48254.CE5D008D@psf.io> https://hg.python.org/cpython/rev/3ea641343244 changeset: 103661:3ea641343244 user: Christian Heimes date: Mon Sep 12 00:01:11 2016 +0200 summary: Issue #28085: Add PROTOCOL_TLS_CLIENT and PROTOCOL_TLS_SERVER for SSLContext files: Doc/library/ssl.rst | 43 ++++++++++++---- Lib/ssl.py | 2 + Lib/test/test_ssl.py | 32 ++++++++++++ Modules/_ssl.c | 80 +++++++++++++++++++++++-------- 4 files changed, 124 insertions(+), 33 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -610,6 +610,22 @@ .. versionadded:: 3.6 +.. data:: PROTOCOL_TLS_CLIENT + + Auto-negotiate the the highest protocol version like :data:`PROTOCOL_SSLv23`, + but only support client-side :class:`SSLSocket` connections. The protocol + enables :data:`CERT_REQUIRED` and :attr:`~SSLContext.check_hostname` by + default. + + .. versionadded:: 3.6 + +.. data:: PROTOCOL_TLS_SERVER + + Auto-negotiate the the highest protocol version like :data:`PROTOCOL_SSLv23`, + but only support server-side :class:`SSLSocket` connections. + + .. versionadded:: 3.6 + .. data:: PROTOCOL_SSLv23 Alias for data:`PROTOCOL_TLS`. @@ -2235,18 +2251,20 @@ SSL versions 2 and 3 are considered insecure and are therefore dangerous to use. If you want maximum compatibility between clients and servers, it is -recommended to use :const:`PROTOCOL_TLS` as the protocol version and then -disable SSLv2 and SSLv3 explicitly using the :data:`SSLContext.options` -attribute:: - - context = ssl.SSLContext(ssl.PROTOCOL_TLS) - context.options |= ssl.OP_NO_SSLv2 - context.options |= ssl.OP_NO_SSLv3 - context.options |= ssl.OP_NO_TLSv1 - context.options |= ssl.OP_NO_TLSv1_1 +recommended to use :const:`PROTOCOL_TLS_CLIENT` or +:const:`PROTOCOL_TLS_SERVER` as the protocol version. SSLv2 and SSLv3 are +disabled by default. + + client_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + client_context.options |= ssl.OP_NO_TLSv1 + client_context.options |= ssl.OP_NO_TLSv1_1 + The SSL context created above will only allow TLSv1.2 and later (if -supported by your system) connections. +supported by your system) connections to a server. :const:`PROTOCOL_TLS_CLIENT` +implies certificate validation and hostname checks by default. You have to +load certificates into the context. + Cipher selection '''''''''''''''' @@ -2257,8 +2275,9 @@ ssl module disables certain weak ciphers by default, but you may want to further restrict the cipher choice. Be sure to read OpenSSL's documentation about the `cipher list format `_. -If you want to check which ciphers are enabled by a given cipher list, use the -``openssl ciphers`` command on your system. +If you want to check which ciphers are enabled by a given cipher list, use +:meth:`SSLContext.get_ciphers` or the ``openssl ciphers`` command on your +system. Multi-processing ^^^^^^^^^^^^^^^^ diff --git a/Lib/ssl.py b/Lib/ssl.py --- a/Lib/ssl.py +++ b/Lib/ssl.py @@ -52,6 +52,8 @@ PROTOCOL_SSLv3 PROTOCOL_SSLv23 PROTOCOL_TLS +PROTOCOL_TLS_CLIENT +PROTOCOL_TLS_SERVER PROTOCOL_TLSv1 PROTOCOL_TLSv1_1 PROTOCOL_TLSv1_2 diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -1342,6 +1342,17 @@ ctx.check_hostname = False self.assertFalse(ctx.check_hostname) + def test_context_client_server(self): + # PROTOCOL_TLS_CLIENT has sane defaults + ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + self.assertTrue(ctx.check_hostname) + self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED) + + # PROTOCOL_TLS_SERVER has different but also sane defaults + ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + self.assertFalse(ctx.check_hostname) + self.assertEqual(ctx.verify_mode, ssl.CERT_NONE) + class SSLErrorTests(unittest.TestCase): @@ -2280,12 +2291,33 @@ if support.verbose: sys.stdout.write("\n") for protocol in PROTOCOLS: + if protocol in {ssl.PROTOCOL_TLS_CLIENT, ssl.PROTOCOL_TLS_SERVER}: + continue with self.subTest(protocol=ssl._PROTOCOL_NAMES[protocol]): context = ssl.SSLContext(protocol) context.load_cert_chain(CERTFILE) server_params_test(context, context, chatty=True, connectionchatty=True) + client_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + client_context.load_verify_locations(SIGNING_CA) + server_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + # server_context.load_verify_locations(SIGNING_CA) + server_context.load_cert_chain(SIGNED_CERTFILE2) + + with self.subTest(client='PROTOCOL_TLS_CLIENT', server='PROTOCOL_TLS_SERVER'): + server_params_test(client_context=client_context, + server_context=server_context, + chatty=True, connectionchatty=True, + sni_name='fakehostname') + + with self.subTest(client='PROTOCOL_TLS_SERVER', server='PROTOCOL_TLS_CLIENT'): + with self.assertRaises(ssl.SSLError): + server_params_test(client_context=server_context, + server_context=client_context, + chatty=True, connectionchatty=True, + sni_name='fakehostname') + def test_getpeercert(self): if support.verbose: sys.stdout.write("\n") diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -140,6 +140,8 @@ #endif #define TLS_method SSLv23_method +#define TLS_client_method SSLv23_client_method +#define TLS_server_method SSLv23_server_method static int X509_NAME_ENTRY_set(const X509_NAME_ENTRY *ne) { @@ -233,14 +235,16 @@ enum py_ssl_version { PY_SSL_VERSION_SSL2, PY_SSL_VERSION_SSL3=1, - PY_SSL_VERSION_TLS, + PY_SSL_VERSION_TLS, /* SSLv23 */ #if HAVE_TLSv1_2 PY_SSL_VERSION_TLS1, PY_SSL_VERSION_TLS1_1, - PY_SSL_VERSION_TLS1_2 + PY_SSL_VERSION_TLS1_2, #else - PY_SSL_VERSION_TLS1 + PY_SSL_VERSION_TLS1, #endif + PY_SSL_VERSION_TLS_CLIENT=0x10, + PY_SSL_VERSION_TLS_SERVER, }; #ifdef WITH_THREAD @@ -2566,6 +2570,33 @@ * _SSLContext objects */ +static int +_set_verify_mode(SSL_CTX *ctx, enum py_ssl_cert_requirements n) +{ + int mode; + int (*verify_cb)(int, X509_STORE_CTX *) = NULL; + + switch(n) { + case PY_SSL_CERT_NONE: + mode = SSL_VERIFY_NONE; + break; + case PY_SSL_CERT_OPTIONAL: + mode = SSL_VERIFY_PEER; + break; + case PY_SSL_CERT_REQUIRED: + mode = SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT; + break; + default: + PyErr_SetString(PyExc_ValueError, + "invalid value for verify_mode"); + return -1; + } + /* keep current verify cb */ + verify_cb = SSL_CTX_get_verify_callback(ctx); + SSL_CTX_set_verify(ctx, mode, verify_cb); + return 0; +} + /*[clinic input] @classmethod _ssl._SSLContext.__new__ @@ -2602,8 +2633,12 @@ else if (proto_version == PY_SSL_VERSION_SSL2) ctx = SSL_CTX_new(SSLv2_method()); #endif - else if (proto_version == PY_SSL_VERSION_TLS) + else if (proto_version == PY_SSL_VERSION_TLS) /* SSLv23 */ ctx = SSL_CTX_new(TLS_method()); + else if (proto_version == PY_SSL_VERSION_TLS_CLIENT) + ctx = SSL_CTX_new(TLS_client_method()); + else if (proto_version == PY_SSL_VERSION_TLS_SERVER) + ctx = SSL_CTX_new(TLS_server_method()); else proto_version = -1; PySSL_END_ALLOW_THREADS @@ -2636,9 +2671,20 @@ self->set_hostname = NULL; #endif /* Don't check host name by default */ - self->check_hostname = 0; + if (proto_version == PY_SSL_VERSION_TLS_CLIENT) { + self->check_hostname = 1; + if (_set_verify_mode(self->ctx, PY_SSL_CERT_REQUIRED) == -1) { + Py_DECREF(self); + return NULL; + } + } else { + self->check_hostname = 0; + if (_set_verify_mode(self->ctx, PY_SSL_CERT_NONE) == -1) { + Py_DECREF(self); + return NULL; + } + } /* Defaults */ - SSL_CTX_set_verify(self->ctx, SSL_VERIFY_NONE, NULL); options = SSL_OP_ALL & ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS; if (proto_version != PY_SSL_VERSION_SSL2) options |= SSL_OP_NO_SSLv2; @@ -2982,28 +3028,16 @@ static int set_verify_mode(PySSLContext *self, PyObject *arg, void *c) { - int n, mode; + int n; if (!PyArg_Parse(arg, "i", &n)) return -1; - if (n == PY_SSL_CERT_NONE) - mode = SSL_VERIFY_NONE; - else if (n == PY_SSL_CERT_OPTIONAL) - mode = SSL_VERIFY_PEER; - else if (n == PY_SSL_CERT_REQUIRED) - mode = SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT; - else { - PyErr_SetString(PyExc_ValueError, - "invalid value for verify_mode"); - return -1; - } - if (mode == SSL_VERIFY_NONE && self->check_hostname) { + if (n == PY_SSL_CERT_NONE && self->check_hostname) { PyErr_SetString(PyExc_ValueError, "Cannot set verify_mode to CERT_NONE when " "check_hostname is enabled."); return -1; } - SSL_CTX_set_verify(self->ctx, mode, NULL); - return 0; + return _set_verify_mode(self->ctx, n); } static PyObject * @@ -5313,6 +5347,10 @@ PY_SSL_VERSION_TLS); PyModule_AddIntConstant(m, "PROTOCOL_TLS", PY_SSL_VERSION_TLS); + PyModule_AddIntConstant(m, "PROTOCOL_TLS_CLIENT", + PY_SSL_VERSION_TLS_CLIENT); + PyModule_AddIntConstant(m, "PROTOCOL_TLS_SERVER", + PY_SSL_VERSION_TLS_SERVER); PyModule_AddIntConstant(m, "PROTOCOL_TLSv1", PY_SSL_VERSION_TLS1); #if HAVE_TLSv1_2 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 18:31:41 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 22:31:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328079=3A_Update_t?= =?utf-8?q?yping_and_test_typing_from_python/typing_repo=2E?= Message-ID: <20160911223141.8602.5368.ECD28FCB@psf.io> https://hg.python.org/cpython/rev/b66e0856ab24 changeset: 103662:b66e0856ab24 user: Guido van Rossum date: Sun Sep 11 15:31:27 2016 -0700 summary: Issue #28079: Update typing and test typing from python/typing repo. Ivan Levkivskyi (3.6 version) files: Lib/test/test_typing.py | 143 +++++++++---- Lib/typing.py | 294 ++++++++++++++++++--------- 2 files changed, 291 insertions(+), 146 deletions(-) diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4,8 +4,6 @@ import re import sys from unittest import TestCase, main, skipUnless, SkipTest -from collections import ChainMap -from test import ann_module, ann_module2, ann_module3 from typing import Any from typing import TypeVar, AnyStr @@ -969,46 +967,6 @@ right_hints = get_type_hints(t.add_right, globals(), locals()) self.assertEqual(right_hints['node'], Optional[Node[T]]) - def test_get_type_hints(self): - gth = get_type_hints - self.assertEqual(gth(ann_module), {'x': int, 'y': str}) - self.assertEqual(gth(ann_module.C, ann_module.__dict__), - ChainMap({'y': Optional[ann_module.C]}, {})) - self.assertEqual(gth(ann_module2), {}) - self.assertEqual(gth(ann_module3), {}) - self.assertEqual(repr(gth(ann_module.j_class)), 'ChainMap({}, {})') - self.assertEqual(gth(ann_module.M), ChainMap({'123': 123, 'o': type}, - {}, {})) - self.assertEqual(gth(ann_module.D), - ChainMap({'j': str, 'k': str, - 'y': Optional[ann_module.C]}, {})) - self.assertEqual(gth(ann_module.Y), ChainMap({'z': int}, {})) - self.assertEqual(gth(ann_module.h_class), - ChainMap({}, {'y': Optional[ann_module.C]}, {})) - self.assertEqual(gth(ann_module.S), ChainMap({'x': str, 'y': str}, - {})) - self.assertEqual(gth(ann_module.foo), {'x': int}) - - def testf(x, y): ... - testf.__annotations__['x'] = 'int' - self.assertEqual(gth(testf), {'x': int}) - self.assertEqual(gth(ann_module2.NTC.meth), {}) - - # interactions with ClassVar - class B: - x: ClassVar[Optional['B']] = None - y: int - class C(B): - z: ClassVar['C'] = B() - class G(Generic[T]): - lst: ClassVar[List[T]] = [] - self.assertEqual(gth(B, locals()), - ChainMap({'y': int, 'x': ClassVar[Optional[B]]}, {})) - self.assertEqual(gth(C, locals()), - ChainMap({'z': ClassVar[C]}, - {'y': int, 'x': ClassVar[Optional[B]]}, {})) - self.assertEqual(gth(G), ChainMap({'lst': ClassVar[List[T]]},{},{})) - def test_forwardref_instance_type_error(self): fr = typing._ForwardRef('int') with self.assertRaises(TypeError): @@ -1198,6 +1156,84 @@ if PY35: exec(PY35_TESTS) +PY36 = sys.version_info[:2] >= (3, 6) + +PY36_TESTS = """ +from test import ann_module, ann_module2, ann_module3 +from collections import ChainMap + +class B: + x: ClassVar[Optional['B']] = None + y: int +class CSub(B): + z: ClassVar['CSub'] = B() +class G(Generic[T]): + lst: ClassVar[List[T]] = [] + +class CoolEmployee(NamedTuple): + name: str + cool: int +""" + +if PY36: + exec(PY36_TESTS) + +gth = get_type_hints + +class GetTypeHintTests(BaseTestCase): + @skipUnless(PY36, 'Python 3.6 required') + def test_get_type_hints_modules(self): + self.assertEqual(gth(ann_module), {'x': int, 'y': str}) + self.assertEqual(gth(ann_module2), {}) + self.assertEqual(gth(ann_module3), {}) + + @skipUnless(PY36, 'Python 3.6 required') + def test_get_type_hints_classes(self): + self.assertEqual(gth(ann_module.C, ann_module.__dict__), + ChainMap({'y': Optional[ann_module.C]}, {})) + self.assertEqual(repr(gth(ann_module.j_class)), 'ChainMap({}, {})') + self.assertEqual(gth(ann_module.M), ChainMap({'123': 123, 'o': type}, + {}, {})) + self.assertEqual(gth(ann_module.D), + ChainMap({'j': str, 'k': str, + 'y': Optional[ann_module.C]}, {})) + self.assertEqual(gth(ann_module.Y), ChainMap({'z': int}, {})) + self.assertEqual(gth(ann_module.h_class), + ChainMap({}, {'y': Optional[ann_module.C]}, {})) + self.assertEqual(gth(ann_module.S), ChainMap({'x': str, 'y': str}, + {})) + self.assertEqual(gth(ann_module.foo), {'x': int}) + + @skipUnless(PY36, 'Python 3.6 required') + def test_respect_no_type_check(self): + @no_type_check + class NoTpCheck: + class Inn: + def __init__(self, x: 'not a type'): ... + self.assertTrue(NoTpCheck.__no_type_check__) + self.assertTrue(NoTpCheck.Inn.__init__.__no_type_check__) + self.assertEqual(gth(ann_module2.NTC.meth), {}) + class ABase(Generic[T]): + def meth(x: int): ... + @no_type_check + class Der(ABase): ... + self.assertEqual(gth(ABase.meth), {'x': int}) + + + def test_previous_behavior(self): + def testf(x, y): ... + testf.__annotations__['x'] = 'int' + self.assertEqual(gth(testf), {'x': int}) + + @skipUnless(PY36, 'Python 3.6 required') + def test_get_type_hints_ClassVar(self): + self.assertEqual(gth(B, globals()), + ChainMap({'y': int, 'x': ClassVar[Optional[B]]}, {})) + self.assertEqual(gth(CSub, globals()), + ChainMap({'z': ClassVar[CSub]}, + {'y': int, 'x': ClassVar[Optional[B]]}, {})) + self.assertEqual(gth(G), ChainMap({'lst': ClassVar[List[T]]},{},{})) + class CollectionsAbcTests(BaseTestCase): @@ -1505,6 +1541,18 @@ joe = new_user(BasicUser) + def test_type_optional(self): + A = Optional[Type[BaseException]] + + def foo(a: A) -> Optional[BaseException]: + if a is None: + return None + else: + return a() + + assert isinstance(foo(KeyboardInterrupt), KeyboardInterrupt) + assert foo(None) is None + class NewTypeTests(BaseTestCase): @@ -1542,6 +1590,17 @@ self.assertEqual(Emp._fields, ('name', 'id')) self.assertEqual(Emp._field_types, dict(name=str, id=int)) + @skipUnless(PY36, 'Python 3.6 required') + def test_annotation_usage(self): + tim = CoolEmployee('Tim', 9000) + self.assertIsInstance(tim, CoolEmployee) + self.assertIsInstance(tim, tuple) + self.assertEqual(tim.name, 'Tim') + self.assertEqual(tim.cool, 9000) + self.assertEqual(CoolEmployee.__name__, 'CoolEmployee') + self.assertEqual(CoolEmployee._fields, ('name', 'cool')) + self.assertEqual(CoolEmployee._field_types, dict(name=str, cool=int)) + def test_pickle(self): global Emp # pickle wants to reference the class by name Emp = NamedTuple('Emp', [('name', str), ('id', int)]) diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -6,11 +6,12 @@ import re as stdlib_re # Avoid confusion with the re we export. import sys import types - try: import collections.abc as collections_abc except ImportError: import collections as collections_abc # Fallback for PY3.2. +if sys.version_info[:2] >= (3, 3): + from collections import ChainMap # Please keep __all__ alphabetized within each category. @@ -1204,53 +1205,135 @@ return res -def get_type_hints(obj, globalns=None, localns=None): - """Return type hints for an object. +if sys.version_info[:2] >= (3, 3): + def get_type_hints(obj, globalns=None, localns=None): + """Return type hints for an object. - This is often the same as obj.__annotations__, but it handles - forward references encoded as string literals, and if necessary - adds Optional[t] if a default value equal to None is set. + This is often the same as obj.__annotations__, but it handles + forward references encoded as string literals, and if necessary + adds Optional[t] if a default value equal to None is set. - The argument may be a module, class, method, or function. The annotations - are returned as a dictionary, or in the case of a class, a ChainMap of - dictionaries. + The argument may be a module, class, method, or function. The annotations + are returned as a dictionary, or in the case of a class, a ChainMap of + dictionaries. - TypeError is raised if the argument is not of a type that can contain - annotations, and an empty dictionary is returned if no annotations are - present. + TypeError is raised if the argument is not of a type that can contain + annotations, and an empty dictionary is returned if no annotations are + present. - BEWARE -- the behavior of globalns and localns is counterintuitive - (unless you are familiar with how eval() and exec() work). The - search order is locals first, then globals. + BEWARE -- the behavior of globalns and localns is counterintuitive + (unless you are familiar with how eval() and exec() work). The + search order is locals first, then globals. - - If no dict arguments are passed, an attempt is made to use the - globals from obj, and these are also used as the locals. If the - object does not appear to have globals, an exception is raised. + - If no dict arguments are passed, an attempt is made to use the + globals from obj, and these are also used as the locals. If the + object does not appear to have globals, an exception is raised. - - If one dict argument is passed, it is used for both globals and - locals. + - If one dict argument is passed, it is used for both globals and + locals. - - If two dict arguments are passed, they specify globals and - locals, respectively. - """ + - If two dict arguments are passed, they specify globals and + locals, respectively. + """ - if getattr(obj, '__no_type_check__', None): - return {} - if globalns is None: - globalns = getattr(obj, '__globals__', {}) - if localns is None: + if getattr(obj, '__no_type_check__', None): + return {} + if globalns is None: + globalns = getattr(obj, '__globals__', {}) + if localns is None: + localns = globalns + elif localns is None: localns = globalns - elif localns is None: - localns = globalns - if (isinstance(obj, types.FunctionType) or - isinstance(obj, types.BuiltinFunctionType) or - isinstance(obj, types.MethodType)): + if (isinstance(obj, types.FunctionType) or + isinstance(obj, types.BuiltinFunctionType) or + isinstance(obj, types.MethodType)): + defaults = _get_defaults(obj) + hints = obj.__annotations__ + for name, value in hints.items(): + if value is None: + value = type(None) + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + if name in defaults and defaults[name] is None: + value = Optional[value] + hints[name] = value + return hints + + if isinstance(obj, types.ModuleType): + try: + hints = obj.__annotations__ + except AttributeError: + return {} + # we keep only those annotations that can be accessed on module + members = obj.__dict__ + hints = {name: value for name, value in hints.items() + if name in members} + for name, value in hints.items(): + if value is None: + value = type(None) + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + hints[name] = value + return hints + + if isinstance(object, type): + cmap = None + for base in reversed(obj.__mro__): + new_map = collections.ChainMap if cmap is None else cmap.new_child + try: + hints = base.__dict__['__annotations__'] + except KeyError: + cmap = new_map() + else: + for name, value in hints.items(): + if value is None: + value = type(None) + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + hints[name] = value + cmap = new_map(hints) + return cmap + + raise TypeError('{!r} is not a module, class, method, ' + 'or function.'.format(obj)) + +else: + def get_type_hints(obj, globalns=None, localns=None): + """Return type hints for a function or method object. + + This is often the same as obj.__annotations__, but it handles + forward references encoded as string literals, and if necessary + adds Optional[t] if a default value equal to None is set. + + BEWARE -- the behavior of globalns and localns is counterintuitive + (unless you are familiar with how eval() and exec() work). The + search order is locals first, then globals. + + - If no dict arguments are passed, an attempt is made to use the + globals from obj, and these are also used as the locals. If the + object does not appear to have globals, an exception is raised. + + - If one dict argument is passed, it is used for both globals and + locals. + + - If two dict arguments are passed, they specify globals and + locals, respectively. + """ + if getattr(obj, '__no_type_check__', None): + return {} + if globalns is None: + globalns = getattr(obj, '__globals__', {}) + if localns is None: + localns = globalns + elif localns is None: + localns = globalns defaults = _get_defaults(obj) - hints = obj.__annotations__ + hints = dict(obj.__annotations__) for name, value in hints.items(): - if value is None: - value = type(None) if isinstance(value, str): value = _ForwardRef(value) value = _eval_type(value, globalns, localns) @@ -1259,62 +1342,30 @@ hints[name] = value return hints - if isinstance(obj, types.ModuleType): - try: - hints = obj.__annotations__ - except AttributeError: - return {} - # we keep only those annotations that can be accessed on module - members = obj.__dict__ - hints = {name: value for name, value in hints.items() - if name in members} - for name, value in hints.items(): - if value is None: - value = type(None) - if isinstance(value, str): - value = _ForwardRef(value) - value = _eval_type(value, globalns, localns) - hints[name] = value - return hints - - if isinstance(object, type): - cmap = None - for base in reversed(obj.__mro__): - new_map = collections.ChainMap if cmap is None else cmap.new_child - try: - hints = base.__dict__['__annotations__'] - except KeyError: - cmap = new_map() - else: - for name, value in hints.items(): - if value is None: - value = type(None) - if isinstance(value, str): - value = _ForwardRef(value) - value = _eval_type(value, globalns, localns) - hints[name] = value - cmap = new_map(hints) - return cmap - - raise TypeError('{!r} is not a module, class, method, ' - 'or function.'.format(obj)) - def no_type_check(arg): """Decorator to indicate that annotations are not type hints. The argument must be a class or function; if it is a class, it - applies recursively to all methods defined in that class (but not - to methods defined in its superclasses or subclasses). + applies recursively to all methods and classes defined in that class + (but not to methods defined in its superclasses or subclasses). - This mutates the function(s) in place. + This mutates the function(s) or class(es) in place. """ if isinstance(arg, type): - for obj in arg.__dict__.values(): + arg_attrs = arg.__dict__.copy() + for attr, val in arg.__dict__.items(): + if val in arg.__bases__: + arg_attrs.pop(attr) + for obj in arg_attrs.values(): if isinstance(obj, types.FunctionType): obj.__no_type_check__ = True - else: + if isinstance(obj, type): + no_type_check(obj) + try: arg.__no_type_check__ = True + except TypeError: # built-in classes + pass return arg @@ -1725,7 +1776,7 @@ # This is not a real generic class. Don't use outside annotations. -class Type(type, Generic[CT_co], extra=type): +class Type(Generic[CT_co], extra=type): """A special construct usable to annotate class objects. For example, suppose we have the following classes:: @@ -1750,31 +1801,66 @@ """ -def NamedTuple(typename, fields): - """Typed version of namedtuple. - - Usage:: - - Employee = typing.NamedTuple('Employee', [('name', str), 'id', int)]) - - This is equivalent to:: - - Employee = collections.namedtuple('Employee', ['name', 'id']) - - The resulting class has one extra attribute: _field_types, - giving a dict mapping field names to types. (The field names - are in the _fields attribute, which is part of the namedtuple - API.) - """ - fields = [(n, t) for n, t in fields] - cls = collections.namedtuple(typename, [n for n, t in fields]) - cls._field_types = dict(fields) - # Set the module to the caller's module (otherwise it'd be 'typing'). +def _make_nmtuple(name, types): + nm_tpl = collections.namedtuple(name, [n for n, t in types]) + nm_tpl._field_types = dict(types) try: - cls.__module__ = sys._getframe(1).f_globals.get('__name__', '__main__') + nm_tpl.__module__ = sys._getframe(2).f_globals.get('__name__', '__main__') except (AttributeError, ValueError): pass - return cls + return nm_tpl + + +if sys.version_info[:2] >= (3, 6): + class NamedTupleMeta(type): + + def __new__(cls, typename, bases, ns, *, _root=False): + if _root: + return super().__new__(cls, typename, bases, ns) + types = ns.get('__annotations__', {}) + return _make_nmtuple(typename, types.items()) + + class NamedTuple(metaclass=NamedTupleMeta, _root=True): + """Typed version of namedtuple. + + Usage:: + + class Employee(NamedTuple): + name: str + id: int + + This is equivalent to:: + + Employee = collections.namedtuple('Employee', ['name', 'id']) + + The resulting class has one extra attribute: _field_types, + giving a dict mapping field names to types. (The field names + are in the _fields attribute, which is part of the namedtuple + API.) Backward-compatible usage:: + + Employee = NamedTuple('Employee', [('name', str), ('id', int)]) + """ + + def __new__(self, typename, fields): + return _make_nmtuple(typename, fields) +else: + def NamedTuple(typename, fields): + """Typed version of namedtuple. + + Usage:: + + Employee = typing.NamedTuple('Employee', [('name', str), 'id', int)]) + + This is equivalent to:: + + Employee = collections.namedtuple('Employee', ['name', 'id']) + + The resulting class has one extra attribute: _field_types, + giving a dict mapping field names to types. (The field names + are in the _fields attribute, which is part of the namedtuple + API.) + """ + return _make_nmtuple(typename, fields) def NewType(name, tp): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 18:34:06 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 22:34:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Null_merge_3=2E5-=3E3=2E6?= Message-ID: <20160911223406.12852.19933.AC606BEA@psf.io> https://hg.python.org/cpython/rev/48a05cb8d7b4 changeset: 103663:48a05cb8d7b4 parent: 103662:b66e0856ab24 parent: 103657:c0f5702e0f10 user: Guido van Rossum date: Sun Sep 11 15:33:31 2016 -0700 summary: Null merge 3.5->3.6 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 18:35:57 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 22:35:57 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Null_merge_3=2E5-=3E3=2E6?= Message-ID: <20160911223557.19131.58021.298CC0CA@psf.io> https://hg.python.org/cpython/rev/9408e3e797d1 changeset: 103665:9408e3e797d1 parent: 103663:48a05cb8d7b4 parent: 103664:cf7da3fd6450 user: Guido van Rossum date: Sun Sep 11 15:35:30 2016 -0700 summary: Null merge 3.5->3.6 files: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 18:35:57 2016 From: python-checkins at python.org (guido.van.rossum) Date: Sun, 11 Sep 2016 22:35:57 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4MDc5?= =?utf-8?q?=3A_Update_typing_and_test_typing_from_python/typing_repo=2E?= Message-ID: <20160911223557.59404.75643.0E2D104F@psf.io> https://hg.python.org/cpython/rev/cf7da3fd6450 changeset: 103664:cf7da3fd6450 branch: 3.5 parent: 103657:c0f5702e0f10 user: Guido van Rossum date: Sun Sep 11 15:34:56 2016 -0700 summary: Issue #28079: Update typing and test typing from python/typing repo. Ivan Levkivskyi (3.5 version) files: Lib/test/test_typing.py | 142 +++++++++++- Lib/typing.py | 332 ++++++++++++++++++++++----- 2 files changed, 409 insertions(+), 65 deletions(-) diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -9,9 +9,9 @@ from typing import TypeVar, AnyStr from typing import T, KT, VT # Not in __all__. from typing import Union, Optional -from typing import Tuple +from typing import Tuple, List from typing import Callable -from typing import Generic +from typing import Generic, ClassVar from typing import cast from typing import get_type_hints from typing import no_type_check, no_type_check_decorator @@ -827,6 +827,43 @@ with self.assertRaises(Exception): D[T] +class ClassVarTests(BaseTestCase): + + def test_basics(self): + with self.assertRaises(TypeError): + ClassVar[1] + with self.assertRaises(TypeError): + ClassVar[int, str] + with self.assertRaises(TypeError): + ClassVar[int][str] + + def test_repr(self): + self.assertEqual(repr(ClassVar), 'typing.ClassVar') + cv = ClassVar[int] + self.assertEqual(repr(cv), 'typing.ClassVar[int]') + cv = ClassVar[Employee] + self.assertEqual(repr(cv), 'typing.ClassVar[%s.Employee]' % __name__) + + def test_cannot_subclass(self): + with self.assertRaises(TypeError): + class C(type(ClassVar)): + pass + with self.assertRaises(TypeError): + class C(type(ClassVar[int])): + pass + + def test_cannot_init(self): + with self.assertRaises(TypeError): + type(ClassVar)() + with self.assertRaises(TypeError): + type(ClassVar[Optional[int]])() + + def test_no_isinstance(self): + with self.assertRaises(TypeError): + isinstance(1, ClassVar[int]) + with self.assertRaises(TypeError): + issubclass(int, ClassVar) + class VarianceTests(BaseTestCase): @@ -1119,6 +1156,84 @@ if PY35: exec(PY35_TESTS) +PY36 = sys.version_info[:2] >= (3, 6) + +PY36_TESTS = """ +from test import ann_module, ann_module2, ann_module3 +from collections import ChainMap + +class B: + x: ClassVar[Optional['B']] = None + y: int +class CSub(B): + z: ClassVar['CSub'] = B() +class G(Generic[T]): + lst: ClassVar[List[T]] = [] + +class CoolEmployee(NamedTuple): + name: str + cool: int +""" + +if PY36: + exec(PY36_TESTS) + +gth = get_type_hints + +class GetTypeHintTests(BaseTestCase): + @skipUnless(PY36, 'Python 3.6 required') + def test_get_type_hints_modules(self): + self.assertEqual(gth(ann_module), {'x': int, 'y': str}) + self.assertEqual(gth(ann_module2), {}) + self.assertEqual(gth(ann_module3), {}) + + @skipUnless(PY36, 'Python 3.6 required') + def test_get_type_hints_classes(self): + self.assertEqual(gth(ann_module.C, ann_module.__dict__), + ChainMap({'y': Optional[ann_module.C]}, {})) + self.assertEqual(repr(gth(ann_module.j_class)), 'ChainMap({}, {})') + self.assertEqual(gth(ann_module.M), ChainMap({'123': 123, 'o': type}, + {}, {})) + self.assertEqual(gth(ann_module.D), + ChainMap({'j': str, 'k': str, + 'y': Optional[ann_module.C]}, {})) + self.assertEqual(gth(ann_module.Y), ChainMap({'z': int}, {})) + self.assertEqual(gth(ann_module.h_class), + ChainMap({}, {'y': Optional[ann_module.C]}, {})) + self.assertEqual(gth(ann_module.S), ChainMap({'x': str, 'y': str}, + {})) + self.assertEqual(gth(ann_module.foo), {'x': int}) + + @skipUnless(PY36, 'Python 3.6 required') + def test_respect_no_type_check(self): + @no_type_check + class NoTpCheck: + class Inn: + def __init__(self, x: 'not a type'): ... + self.assertTrue(NoTpCheck.__no_type_check__) + self.assertTrue(NoTpCheck.Inn.__init__.__no_type_check__) + self.assertEqual(gth(ann_module2.NTC.meth), {}) + class ABase(Generic[T]): + def meth(x: int): ... + @no_type_check + class Der(ABase): ... + self.assertEqual(gth(ABase.meth), {'x': int}) + + + def test_previous_behavior(self): + def testf(x, y): ... + testf.__annotations__['x'] = 'int' + self.assertEqual(gth(testf), {'x': int}) + + @skipUnless(PY36, 'Python 3.6 required') + def test_get_type_hints_ClassVar(self): + self.assertEqual(gth(B, globals()), + ChainMap({'y': int, 'x': ClassVar[Optional[B]]}, {})) + self.assertEqual(gth(CSub, globals()), + ChainMap({'z': ClassVar[CSub]}, + {'y': int, 'x': ClassVar[Optional[B]]}, {})) + self.assertEqual(gth(G), ChainMap({'lst': ClassVar[List[T]]},{},{})) + class CollectionsAbcTests(BaseTestCase): @@ -1426,6 +1541,18 @@ joe = new_user(BasicUser) + def test_type_optional(self): + A = Optional[Type[BaseException]] + + def foo(a: A) -> Optional[BaseException]: + if a is None: + return None + else: + return a() + + assert isinstance(foo(KeyboardInterrupt), KeyboardInterrupt) + assert foo(None) is None + class NewTypeTests(BaseTestCase): @@ -1463,6 +1590,17 @@ self.assertEqual(Emp._fields, ('name', 'id')) self.assertEqual(Emp._field_types, dict(name=str, id=int)) + @skipUnless(PY36, 'Python 3.6 required') + def test_annotation_usage(self): + tim = CoolEmployee('Tim', 9000) + self.assertIsInstance(tim, CoolEmployee) + self.assertIsInstance(tim, tuple) + self.assertEqual(tim.name, 'Tim') + self.assertEqual(tim.cool, 9000) + self.assertEqual(CoolEmployee.__name__, 'CoolEmployee') + self.assertEqual(CoolEmployee._fields, ('name', 'cool')) + self.assertEqual(CoolEmployee._field_types, dict(name=str, cool=int)) + def test_pickle(self): global Emp # pickle wants to reference the class by name Emp = NamedTuple('Emp', [('name', str), ('id', int)]) diff --git a/Lib/typing.py b/Lib/typing.py --- a/Lib/typing.py +++ b/Lib/typing.py @@ -10,6 +10,8 @@ import collections.abc as collections_abc except ImportError: import collections as collections_abc # Fallback for PY3.2. +if sys.version_info[:2] >= (3, 3): + from collections import ChainMap # Please keep __all__ alphabetized within each category. @@ -17,6 +19,7 @@ # Super-special typing primitives. 'Any', 'Callable', + 'ClassVar', 'Generic', 'Optional', 'Tuple', @@ -270,7 +273,7 @@ def _get_type_vars(types, tvars): for t in types: - if isinstance(t, TypingMeta): + if isinstance(t, TypingMeta) or isinstance(t, _ClassVar): t._get_type_vars(tvars) @@ -281,7 +284,7 @@ def _eval_type(t, globalns, localns): - if isinstance(t, TypingMeta): + if isinstance(t, TypingMeta) or isinstance(t, _ClassVar): return t._eval_type(globalns, localns) else: return t @@ -1114,6 +1117,67 @@ return obj +class _ClassVar(metaclass=TypingMeta, _root=True): + """Special type construct to mark class variables. + + An annotation wrapped in ClassVar indicates that a given + attribute is intended to be used as a class variable and + should not be set on instances of that class. Usage:: + + class Starship: + stats: ClassVar[Dict[str, int]] = {} # class variable + damage: int = 10 # instance variable + + ClassVar accepts only types and cannot be further subscribed. + + Note that ClassVar is not a class itself, and should not + be used with isinstance() or issubclass(). + """ + + def __init__(self, tp=None, _root=False): + cls = type(self) + if _root: + self.__type__ = tp + else: + raise TypeError('Cannot initialize {}'.format(cls.__name__[1:])) + + def __getitem__(self, item): + cls = type(self) + if self.__type__ is None: + return cls(_type_check(item, + '{} accepts only types.'.format(cls.__name__[1:])), + _root=True) + raise TypeError('{} cannot be further subscripted' + .format(cls.__name__[1:])) + + def _eval_type(self, globalns, localns): + return type(self)(_eval_type(self.__type__, globalns, localns), + _root=True) + + def _get_type_vars(self, tvars): + if self.__type__: + _get_type_vars(self.__type__, tvars) + + def __repr__(self): + cls = type(self) + if not self.__type__: + return '{}.{}'.format(cls.__module__, cls.__name__[1:]) + return '{}.{}[{}]'.format(cls.__module__, cls.__name__[1:], + _type_repr(self.__type__)) + + def __hash__(self): + return hash((type(self).__name__, self.__type__)) + + def __eq__(self, other): + if not isinstance(other, _ClassVar): + return NotImplemented + if self.__type__ is not None: + return self.__type__ == other.__type__ + return self is other + +ClassVar = _ClassVar(_root=True) + + def cast(typ, val): """Cast a value to a type. @@ -1141,62 +1205,167 @@ return res -def get_type_hints(obj, globalns=None, localns=None): - """Return type hints for a function or method object. +if sys.version_info[:2] >= (3, 3): + def get_type_hints(obj, globalns=None, localns=None): + """Return type hints for an object. - This is often the same as obj.__annotations__, but it handles - forward references encoded as string literals, and if necessary - adds Optional[t] if a default value equal to None is set. + This is often the same as obj.__annotations__, but it handles + forward references encoded as string literals, and if necessary + adds Optional[t] if a default value equal to None is set. - BEWARE -- the behavior of globalns and localns is counterintuitive - (unless you are familiar with how eval() and exec() work). The - search order is locals first, then globals. + The argument may be a module, class, method, or function. The annotations + are returned as a dictionary, or in the case of a class, a ChainMap of + dictionaries. - - If no dict arguments are passed, an attempt is made to use the - globals from obj, and these are also used as the locals. If the - object does not appear to have globals, an exception is raised. + TypeError is raised if the argument is not of a type that can contain + annotations, and an empty dictionary is returned if no annotations are + present. - - If one dict argument is passed, it is used for both globals and - locals. + BEWARE -- the behavior of globalns and localns is counterintuitive + (unless you are familiar with how eval() and exec() work). The + search order is locals first, then globals. - - If two dict arguments are passed, they specify globals and - locals, respectively. - """ - if getattr(obj, '__no_type_check__', None): - return {} - if globalns is None: - globalns = getattr(obj, '__globals__', {}) - if localns is None: + - If no dict arguments are passed, an attempt is made to use the + globals from obj, and these are also used as the locals. If the + object does not appear to have globals, an exception is raised. + + - If one dict argument is passed, it is used for both globals and + locals. + + - If two dict arguments are passed, they specify globals and + locals, respectively. + """ + + if getattr(obj, '__no_type_check__', None): + return {} + if globalns is None: + globalns = getattr(obj, '__globals__', {}) + if localns is None: + localns = globalns + elif localns is None: localns = globalns - elif localns is None: - localns = globalns - defaults = _get_defaults(obj) - hints = dict(obj.__annotations__) - for name, value in hints.items(): - if isinstance(value, str): - value = _ForwardRef(value) - value = _eval_type(value, globalns, localns) - if name in defaults and defaults[name] is None: - value = Optional[value] - hints[name] = value - return hints + + if (isinstance(obj, types.FunctionType) or + isinstance(obj, types.BuiltinFunctionType) or + isinstance(obj, types.MethodType)): + defaults = _get_defaults(obj) + hints = obj.__annotations__ + for name, value in hints.items(): + if value is None: + value = type(None) + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + if name in defaults and defaults[name] is None: + value = Optional[value] + hints[name] = value + return hints + + if isinstance(obj, types.ModuleType): + try: + hints = obj.__annotations__ + except AttributeError: + return {} + # we keep only those annotations that can be accessed on module + members = obj.__dict__ + hints = {name: value for name, value in hints.items() + if name in members} + for name, value in hints.items(): + if value is None: + value = type(None) + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + hints[name] = value + return hints + + if isinstance(object, type): + cmap = None + for base in reversed(obj.__mro__): + new_map = collections.ChainMap if cmap is None else cmap.new_child + try: + hints = base.__dict__['__annotations__'] + except KeyError: + cmap = new_map() + else: + for name, value in hints.items(): + if value is None: + value = type(None) + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + hints[name] = value + cmap = new_map(hints) + return cmap + + raise TypeError('{!r} is not a module, class, method, ' + 'or function.'.format(obj)) + +else: + def get_type_hints(obj, globalns=None, localns=None): + """Return type hints for a function or method object. + + This is often the same as obj.__annotations__, but it handles + forward references encoded as string literals, and if necessary + adds Optional[t] if a default value equal to None is set. + + BEWARE -- the behavior of globalns and localns is counterintuitive + (unless you are familiar with how eval() and exec() work). The + search order is locals first, then globals. + + - If no dict arguments are passed, an attempt is made to use the + globals from obj, and these are also used as the locals. If the + object does not appear to have globals, an exception is raised. + + - If one dict argument is passed, it is used for both globals and + locals. + + - If two dict arguments are passed, they specify globals and + locals, respectively. + """ + if getattr(obj, '__no_type_check__', None): + return {} + if globalns is None: + globalns = getattr(obj, '__globals__', {}) + if localns is None: + localns = globalns + elif localns is None: + localns = globalns + defaults = _get_defaults(obj) + hints = dict(obj.__annotations__) + for name, value in hints.items(): + if isinstance(value, str): + value = _ForwardRef(value) + value = _eval_type(value, globalns, localns) + if name in defaults and defaults[name] is None: + value = Optional[value] + hints[name] = value + return hints def no_type_check(arg): """Decorator to indicate that annotations are not type hints. The argument must be a class or function; if it is a class, it - applies recursively to all methods defined in that class (but not - to methods defined in its superclasses or subclasses). + applies recursively to all methods and classes defined in that class + (but not to methods defined in its superclasses or subclasses). - This mutates the function(s) in place. + This mutates the function(s) or class(es) in place. """ if isinstance(arg, type): - for obj in arg.__dict__.values(): + arg_attrs = arg.__dict__.copy() + for attr, val in arg.__dict__.items(): + if val in arg.__bases__: + arg_attrs.pop(attr) + for obj in arg_attrs.values(): if isinstance(obj, types.FunctionType): obj.__no_type_check__ = True - else: + if isinstance(obj, type): + no_type_check(obj) + try: arg.__no_type_check__ = True + except TypeError: # built-in classes + pass return arg @@ -1300,6 +1469,8 @@ else: if (not attr.startswith('_abc_') and attr != '__abstractmethods__' and + attr != '__annotations__' and + attr != '__weakref__' and attr != '_is_protocol' and attr != '__dict__' and attr != '__args__' and @@ -1605,7 +1776,7 @@ # This is not a real generic class. Don't use outside annotations. -class Type(type, Generic[CT_co], extra=type): +class Type(Generic[CT_co], extra=type): """A special construct usable to annotate class objects. For example, suppose we have the following classes:: @@ -1630,31 +1801,66 @@ """ -def NamedTuple(typename, fields): - """Typed version of namedtuple. - - Usage:: - - Employee = typing.NamedTuple('Employee', [('name', str), 'id', int)]) - - This is equivalent to:: - - Employee = collections.namedtuple('Employee', ['name', 'id']) - - The resulting class has one extra attribute: _field_types, - giving a dict mapping field names to types. (The field names - are in the _fields attribute, which is part of the namedtuple - API.) - """ - fields = [(n, t) for n, t in fields] - cls = collections.namedtuple(typename, [n for n, t in fields]) - cls._field_types = dict(fields) - # Set the module to the caller's module (otherwise it'd be 'typing'). +def _make_nmtuple(name, types): + nm_tpl = collections.namedtuple(name, [n for n, t in types]) + nm_tpl._field_types = dict(types) try: - cls.__module__ = sys._getframe(1).f_globals.get('__name__', '__main__') + nm_tpl.__module__ = sys._getframe(2).f_globals.get('__name__', '__main__') except (AttributeError, ValueError): pass - return cls + return nm_tpl + + +if sys.version_info[:2] >= (3, 6): + class NamedTupleMeta(type): + + def __new__(cls, typename, bases, ns, *, _root=False): + if _root: + return super().__new__(cls, typename, bases, ns) + types = ns.get('__annotations__', {}) + return _make_nmtuple(typename, types.items()) + + class NamedTuple(metaclass=NamedTupleMeta, _root=True): + """Typed version of namedtuple. + + Usage:: + + class Employee(NamedTuple): + name: str + id: int + + This is equivalent to:: + + Employee = collections.namedtuple('Employee', ['name', 'id']) + + The resulting class has one extra attribute: _field_types, + giving a dict mapping field names to types. (The field names + are in the _fields attribute, which is part of the namedtuple + API.) Backward-compatible usage:: + + Employee = NamedTuple('Employee', [('name', str), ('id', int)]) + """ + + def __new__(self, typename, fields): + return _make_nmtuple(typename, fields) +else: + def NamedTuple(typename, fields): + """Typed version of namedtuple. + + Usage:: + + Employee = typing.NamedTuple('Employee', [('name', str), 'id', int)]) + + This is equivalent to:: + + Employee = collections.namedtuple('Employee', ['name', 'id']) + + The resulting class has one extra attribute: _field_types, + giving a dict mapping field names to types. (The field names + are in the _fields attribute, which is part of the namedtuple + API.) + """ + return _make_nmtuple(typename, fields) def NewType(name, tp): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 18:58:24 2016 From: python-checkins at python.org (eric.smith) Date: Sun, 11 Sep 2016 22:58:24 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Make_an_f-string_error_mes?= =?utf-8?q?sage_more_exact_and_consistent=2E?= Message-ID: <20160911225824.69549.52994.5D94C7E0@psf.io> https://hg.python.org/cpython/rev/3992c7f690e1 changeset: 103666:3992c7f690e1 user: Eric V. Smith date: Sun Sep 11 18:58:20 2016 -0400 summary: Make an f-string error message more exact and consistent. files: Lib/test/test_fstring.py | 3 ++- Python/ast.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -182,9 +182,10 @@ self.assertEqual(f'{"#"}', '#') self.assertEqual(f'{d["#"]}', 'hash') - self.assertAllRaise(SyntaxError, "f-string cannot include '#'", + self.assertAllRaise(SyntaxError, "f-string expression part cannot include '#'", ["f'{1#}'", # error because the expression becomes "(1#)" "f'{3(#)}'", + "f'{#}'", ]) def test_many_expressions(self): diff --git a/Python/ast.c b/Python/ast.c --- a/Python/ast.c +++ b/Python/ast.c @@ -4419,7 +4419,7 @@ } else if (ch == '#') { /* Error: can't include a comment character, inside parens or not. */ - ast_error(c, n, "f-string cannot include '#'"); + ast_error(c, n, "f-string expression part cannot include '#'"); return -1; } else if (nested_depth == 0 && (ch == '!' || ch == ':' || ch == '}')) { -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 19:01:29 2016 From: python-checkins at python.org (eric.smith) Date: Sun, 11 Sep 2016 23:01:29 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_another_f-string_comme?= =?utf-8?q?nt_test=2C_to_make_sure_=23_are_being_caught_in_the_right?= Message-ID: <20160911230126.59674.69921.F39BEC8D@psf.io> https://hg.python.org/cpython/rev/ceb9f08e8155 changeset: 103667:ceb9f08e8155 user: Eric V. Smith date: Sun Sep 11 19:01:22 2016 -0400 summary: Add another f-string comment test, to make sure # are being caught in the right place. files: Lib/test/test_fstring.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -186,6 +186,8 @@ ["f'{1#}'", # error because the expression becomes "(1#)" "f'{3(#)}'", "f'{#}'", + "f'{)#}'", # When wrapped in parens, this becomes + # '()#)'. Make sure that doesn't compile. ]) def test_many_expressions(self): -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 19:14:54 2016 From: python-checkins at python.org (christian.heimes) Date: Sun, 11 Sep 2016 23:14:54 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Update_whatsnew_with_my_co?= =?utf-8?q?ntributions?= Message-ID: <20160911231453.1800.93605.94D8A176@psf.io> https://hg.python.org/cpython/rev/301a847890a3 changeset: 103668:301a847890a3 user: Christian Heimes date: Mon Sep 12 01:14:35 2016 +0200 summary: Update whatsnew with my contributions files: Doc/library/ssl.rst | 6 +- Doc/whatsnew/3.6.rst | 80 ++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+), 3 deletions(-) diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -2255,9 +2255,9 @@ :const:`PROTOCOL_TLS_SERVER` as the protocol version. SSLv2 and SSLv3 are disabled by default. - client_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) - client_context.options |= ssl.OP_NO_TLSv1 - client_context.options |= ssl.OP_NO_TLSv1_1 + >>> client_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + >>> client_context.options |= ssl.OP_NO_TLSv1 + >>> client_context.options |= ssl.OP_NO_TLSv1_1 The SSL context created above will only allow TLSv1.2 and later (if diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -86,6 +86,13 @@ is initialized to increase the security. See the :pep:`524` for the rationale. +* :mod:`hashlib` and :mod:`ssl` now support OpenSSL 1.1.0. + +* The default settings and feature set of the :mod:`ssl` have been improved. + +* The :mod:`hashlib` module has got support for BLAKE2, SHA-3 and SHAKE hash + algorithms and :func:`~hashlib.scrypt` key derivation function. + Windows improvements: * PEP 529: :ref:`Change Windows filesystem encoding to UTF-8 ` @@ -646,6 +653,31 @@ :issue:`23848`.) +hashlib +------- + +:mod:`hashlib` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. +It has been tested with 0.9.8zc, 0.9.8zh and 1.0.1t as well as LibreSSL 2.3 +and 2.4. +(Contributed by Christian Heimes in :issue:`26470`.) + +BLAKE2 hash functions were added to the module. :func:`~hashlib.blake2b` +and :func:`~hashlib.blake2s` are always available and support the full +feature set of BLAKE2. +(Contributed by Christian Heimes in :issue:`26798` based on code by +Dmitry Chestnykh and Samuel Neves. Documentation written by Dmitry Chestnykh.) + +The SHA-3 hash functions :func:`~hashlib.sha3_224`, :func:`~hashlib.sha3_256`, +:func:`~hashlib.sha3_384`, :func:`~hashlib.sha3_512`, and SHAKE hash functions +:func:`~hashlib.shake_128` and :func:`~hashlib.shake_256` were added. +(Contributed by Christian Heimes in :issue:`16113`. Keccak Code Package +by Guido Bertoni, Joan Daemen, Micha?l Peeters, Gilles Van Assche, and +Ronny Van Keer.) + +The password-based key derivation function :func:`~hashlib.scrypt` is now +available with OpenSSL 1.1.0 and newer. +(Contributed by Christian Heimes in :issue:`27928`.) + http.client ----------- @@ -775,6 +807,11 @@ ``SO_PROTOCOL``, ``SO_PEERSEC``, and ``SO_PASSSEC`` are now supported. (Contributed by Christian Heimes in :issue:`26907`.) +The socket module now supports the address family +:data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, +``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added. +(Contributed by Christian Heimes in :issue:`27744` with support from +Victor Stinner.) socketserver ------------ @@ -791,6 +828,39 @@ calling :meth:`~io.BufferedIOBase.write` is now guaranteed to send the data in full. (Contributed by Martin Panter in :issue:`26721`.) +ssl +--- + +:mod:`ssl` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. +It has been tested with 0.9.8zc, 0.9.8zh and 1.0.1t as well as LibreSSL 2.3 +and 2.4. +(Contributed by Christian Heimes in :issue:`26470`.) + +3DES has been removed from the default cipher suites and ChaCha20 Poly1305 +cipher suites are now in the right position. +(Contributed by Christian Heimes in :issue:`27850` and :issue:`27766`.) + +:class:`~ssl.SSLContext` has better default configuration for options +and ciphers. +(Contributed by Christian Heimes in :issue:`28043`.) + +SSL session can be copied from one client-side connection to another +with :class:`~ssl.SSLSession`. TLS session resumption can speed up +the initial handshake, reduce latency and improve performance +(Contributed by Christian Heimes in :issue:`19500` based on a draft by +Alex Warhawk.) + +All constants and flags have been converted to :class:`~enum.IntEnum` and +:class:`~enum.IntFlags`. +(Contributed by Christian Heimes in :issue:`28025`.) + +Server and client-side specific TLS protocols for :class:`~ssl.SSLContext` +were added. +(Contributed by Christian Heimes in :issue:`28085`.) + +General resource ids (``GEN_RID``) in subject alternative name extensions +no longer case a SystemError. +(Contributed by Christian Heimes in :issue:`27691`.) subprocess ---------- @@ -1137,6 +1207,16 @@ warning. It will be an error in future Python releases. (Contributed by Serhiy Storchaka in :issue:`22493`.) +* SSL-related arguments like ``certfile``, ``keyfile`` and ``check_hostname`` + in :mod:`ftplib`, :mod:`http.client`, :mod:`imaplib`, :mod:`poplib`, + and :mod:`smtplib` have been deprecated in favor of ``context``. + (Contributed by Christian Heimes in :issue:`28022`.) + +* A couple of protocols and functions of the :mod:`ssl` module are now + deprecated. Some features will no longer be available in future versions + of OpenSSL. Other features are deprecated in favor of a different API. + (Contributed by Christian Heimes in :issue:`28022` and :issue:`26470`.) + Deprecated Python behavior -------------------------- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 20:24:10 2016 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 12 Sep 2016 00:24:10 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_merge?= Message-ID: <20160912002410.19090.74048.10265D17@psf.io> https://hg.python.org/cpython/rev/130df8dd5984 changeset: 103670:130df8dd5984 parent: 103668:301a847890a3 parent: 103669:f38b831cb6b9 user: Raymond Hettinger date: Sun Sep 11 17:24:05 2016 -0700 summary: merge files: Doc/reference/expressions.rst | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1315,8 +1315,9 @@ -------------------- The operators :keyword:`is` and :keyword:`is not` test for object identity: ``x -is y`` is true if and only if *x* and *y* are the same object. ``x is not y`` -yields the inverse truth value. [#]_ +is y`` is true if and only if *x* and *y* are the same object. Object identity +is determined using the :meth:`id` function. ``x is not y`` yields the inverse +truth value. [#]_ .. _booleans: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 20:24:10 2016 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 12 Sep 2016 00:24:10 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI2NTEx?= =?utf-8?q?=3A__Reference_the_id=28=29_function_in_the_=27is=27_and_=27is_?= =?utf-8?q?not=27_docs?= Message-ID: <20160912002410.19131.72286.23C84869@psf.io> https://hg.python.org/cpython/rev/f38b831cb6b9 changeset: 103669:f38b831cb6b9 branch: 3.5 parent: 103664:cf7da3fd6450 user: Raymond Hettinger date: Sun Sep 11 17:23:49 2016 -0700 summary: Issue #26511: Reference the id() function in the 'is' and 'is not' docs files: Doc/reference/expressions.rst | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1315,8 +1315,9 @@ -------------------- The operators :keyword:`is` and :keyword:`is not` test for object identity: ``x -is y`` is true if and only if *x* and *y* are the same object. ``x is not y`` -yields the inverse truth value. [#]_ +is y`` is true if and only if *x* and *y* are the same object. Object identity +is determined using the :meth:`id` function. ``x is not y`` yields the inverse +truth value. [#]_ .. _booleans: -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 21:11:11 2016 From: python-checkins at python.org (yury.selivanov) Date: Mon, 12 Sep 2016 01:11:11 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogYXN5bmNpbzogQWRk?= =?utf-8?q?_set=5Fprotocol_/_get=5Fprotocol_methods_to_Transports?= Message-ID: <20160912011111.1897.25387.32C51471@psf.io> https://hg.python.org/cpython/rev/f12a59311885 changeset: 103671:f12a59311885 branch: 3.5 parent: 103669:f38b831cb6b9 user: Yury Selivanov date: Sun Sep 11 21:11:02 2016 -0400 summary: asyncio: Add set_protocol / get_protocol methods to Transports files: Lib/asyncio/base_subprocess.py | 6 ++++++ Lib/asyncio/proactor_events.py | 6 ++++++ Lib/asyncio/selector_events.py | 6 ++++++ Lib/asyncio/sslproto.py | 6 ++++++ Lib/asyncio/transports.py | 8 ++++++++ Lib/asyncio/unix_events.py | 12 ++++++++++++ Lib/test/test_asyncio/test_sslproto.py | 1 + 7 files changed, 45 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/base_subprocess.py b/Lib/asyncio/base_subprocess.py --- a/Lib/asyncio/base_subprocess.py +++ b/Lib/asyncio/base_subprocess.py @@ -87,6 +87,12 @@ def _start(self, args, shell, stdin, stdout, stderr, bufsize, **kwargs): raise NotImplementedError + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closed diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -66,6 +66,12 @@ def _set_extra(self, sock): self._extra['pipe'] = sock + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closing diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -560,6 +560,12 @@ def abort(self): self._force_close(None) + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closing diff --git a/Lib/asyncio/sslproto.py b/Lib/asyncio/sslproto.py --- a/Lib/asyncio/sslproto.py +++ b/Lib/asyncio/sslproto.py @@ -305,6 +305,12 @@ """Get optional transport information.""" return self._ssl_protocol._get_extra_info(name, default) + def set_protocol(self, protocol): + self._app_protocol = protocol + + def get_protocol(self): + return self._app_protocol + def is_closing(self): return self._closed diff --git a/Lib/asyncio/transports.py b/Lib/asyncio/transports.py --- a/Lib/asyncio/transports.py +++ b/Lib/asyncio/transports.py @@ -33,6 +33,14 @@ """ raise NotImplementedError + def set_protocol(self, protocol): + """Set a new protocol.""" + raise NotImplementedError + + def get_protocol(self): + """Return the current protocol.""" + raise NotImplementedError + class ReadTransport(BaseTransport): """Interface for read-only transports.""" diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -374,6 +374,12 @@ def resume_reading(self): self._loop.add_reader(self._fileno, self._read_ready) + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closing @@ -570,6 +576,12 @@ self._loop.remove_reader(self._fileno) self._loop.call_soon(self._call_connection_lost, None) + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closing diff --git a/Lib/test/test_asyncio/test_sslproto.py b/Lib/test/test_asyncio/test_sslproto.py --- a/Lib/test/test_asyncio/test_sslproto.py +++ b/Lib/test/test_asyncio/test_sslproto.py @@ -25,6 +25,7 @@ sslcontext = test_utils.dummy_ssl_context() app_proto = asyncio.Protocol() proto = sslproto.SSLProtocol(self.loop, app_proto, sslcontext, waiter) + self.assertIs(proto._app_transport.get_protocol(), app_proto) self.addCleanup(proto._app_transport.close) return proto -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 21:11:11 2016 From: python-checkins at python.org (yury.selivanov) Date: Mon, 12 Sep 2016 01:11:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_=28asyncio=29?= Message-ID: <20160912011111.59651.99620.F0FCCE03@psf.io> https://hg.python.org/cpython/rev/f44a54f80fd4 changeset: 103672:f44a54f80fd4 parent: 103670:130df8dd5984 parent: 103671:f12a59311885 user: Yury Selivanov date: Sun Sep 11 21:11:19 2016 -0400 summary: Merge 3.5 (asyncio) files: Lib/asyncio/base_subprocess.py | 6 ++++++ Lib/asyncio/proactor_events.py | 6 ++++++ Lib/asyncio/selector_events.py | 6 ++++++ Lib/asyncio/sslproto.py | 6 ++++++ Lib/asyncio/transports.py | 8 ++++++++ Lib/asyncio/unix_events.py | 12 ++++++++++++ Lib/test/test_asyncio/test_sslproto.py | 1 + 7 files changed, 45 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/base_subprocess.py b/Lib/asyncio/base_subprocess.py --- a/Lib/asyncio/base_subprocess.py +++ b/Lib/asyncio/base_subprocess.py @@ -87,6 +87,12 @@ def _start(self, args, shell, stdin, stdout, stderr, bufsize, **kwargs): raise NotImplementedError + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closed diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -66,6 +66,12 @@ def _set_extra(self, sock): self._extra['pipe'] = sock + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closing diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -560,6 +560,12 @@ def abort(self): self._force_close(None) + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closing diff --git a/Lib/asyncio/sslproto.py b/Lib/asyncio/sslproto.py --- a/Lib/asyncio/sslproto.py +++ b/Lib/asyncio/sslproto.py @@ -305,6 +305,12 @@ """Get optional transport information.""" return self._ssl_protocol._get_extra_info(name, default) + def set_protocol(self, protocol): + self._app_protocol = protocol + + def get_protocol(self): + return self._app_protocol + def is_closing(self): return self._closed diff --git a/Lib/asyncio/transports.py b/Lib/asyncio/transports.py --- a/Lib/asyncio/transports.py +++ b/Lib/asyncio/transports.py @@ -33,6 +33,14 @@ """ raise NotImplementedError + def set_protocol(self, protocol): + """Set a new protocol.""" + raise NotImplementedError + + def get_protocol(self): + """Return the current protocol.""" + raise NotImplementedError + class ReadTransport(BaseTransport): """Interface for read-only transports.""" diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -374,6 +374,12 @@ def resume_reading(self): self._loop.add_reader(self._fileno, self._read_ready) + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closing @@ -571,6 +577,12 @@ self._loop.remove_reader(self._fileno) self._loop.call_soon(self._call_connection_lost, None) + def set_protocol(self, protocol): + self._protocol = protocol + + def get_protocol(self): + return self._protocol + def is_closing(self): return self._closing diff --git a/Lib/test/test_asyncio/test_sslproto.py b/Lib/test/test_asyncio/test_sslproto.py --- a/Lib/test/test_asyncio/test_sslproto.py +++ b/Lib/test/test_asyncio/test_sslproto.py @@ -25,6 +25,7 @@ sslcontext = test_utils.dummy_ssl_context() app_proto = asyncio.Protocol() proto = sslproto.SSLProtocol(self.loop, app_proto, sslcontext, waiter) + self.assertIs(proto._app_transport.get_protocol(), app_proto) self.addCleanup(proto._app_transport.close) return proto -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 21:24:48 2016 From: python-checkins at python.org (yury.selivanov) Date: Mon, 12 Sep 2016 01:24:48 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Merge_3=2E5_=28asyncio/NEW?= =?utf-8?q?S=29?= Message-ID: <20160912012448.18971.59944.C896EFE0@psf.io> https://hg.python.org/cpython/rev/681096fb4aac changeset: 103673:681096fb4aac user: Yury Selivanov date: Sun Sep 11 21:25:01 2016 -0400 summary: Merge 3.5 (asyncio/NEWS) files: Misc/NEWS | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -363,6 +363,8 @@ - Issue #21201: Improves readability of multiprocessing error message. Thanks to Wojciech Walczak for patch. +- asyncio: Add set_protocol / get_protocol to Transports. + IDLE ---- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 21:44:06 2016 From: python-checkins at python.org (yury.selivanov) Date: Mon, 12 Sep 2016 01:44:06 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI3NDU2?= =?utf-8?q?=3A_asyncio=3A_Set_TCP=5FNODELAY_by_default=2E?= Message-ID: <20160912014406.59815.66869.522A8E93@psf.io> https://hg.python.org/cpython/rev/3f7e4ae9eba3 changeset: 103674:3f7e4ae9eba3 branch: 3.5 parent: 103671:f12a59311885 user: Yury Selivanov date: Sun Sep 11 21:39:31 2016 -0400 summary: Issue #27456: asyncio: Set TCP_NODELAY by default. files: Lib/asyncio/selector_events.py | 16 ++++++++++++++++ Misc/NEWS | 2 ++ 2 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -39,6 +39,17 @@ return bool(key.events & event) +if hasattr(socket, 'TCP_NODELAY'): + def _set_nodelay(sock): + if (sock.family in {socket.AF_INET, socket.AF_INET6} and + sock.type == socket.SOCK_STREAM and + sock.proto == socket.IPPROTO_TCP): + sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) +else: + def _set_nodelay(sock): + pass + + class BaseSelectorEventLoop(base_events.BaseEventLoop): """Selector event loop. @@ -640,6 +651,11 @@ self._eof = False self._paused = False + # Disable the Nagle algorithm -- small writes will be + # sent without waiting for the TCP ACK. This generally + # decreases the latency (in some cases significantly.) + _set_nodelay(self._sock) + self._loop.call_soon(self._protocol.connection_made, self) # only start reading when connection_made() has been called self._loop.call_soon(self._loop.add_reader, diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -254,6 +254,8 @@ - Issue #21201: Improves readability of multiprocessing error message. Thanks to Wojciech Walczak for patch. +- Issue #27456: asyncio: Set TCP_NODELAY by default. + IDLE ---- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 21:44:06 2016 From: python-checkins at python.org (yury.selivanov) Date: Mon, 12 Sep 2016 01:44:06 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?b?KTogTWVyZ2UgMy41IChpc3N1ZSAjMjc0NTYp?= Message-ID: <20160912014406.19052.54650.76CE67FE@psf.io> https://hg.python.org/cpython/rev/10384c5c18f5 changeset: 103675:10384c5c18f5 parent: 103673:681096fb4aac parent: 103674:3f7e4ae9eba3 user: Yury Selivanov date: Sun Sep 11 21:44:17 2016 -0400 summary: Merge 3.5 (issue #27456) files: Lib/asyncio/selector_events.py | 16 ++++++++++++++++ Misc/NEWS | 2 ++ 2 files changed, 18 insertions(+), 0 deletions(-) diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -39,6 +39,17 @@ return bool(key.events & event) +if hasattr(socket, 'TCP_NODELAY'): + def _set_nodelay(sock): + if (sock.family in {socket.AF_INET, socket.AF_INET6} and + sock.type == socket.SOCK_STREAM and + sock.proto == socket.IPPROTO_TCP): + sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) +else: + def _set_nodelay(sock): + pass + + class BaseSelectorEventLoop(base_events.BaseEventLoop): """Selector event loop. @@ -641,6 +652,11 @@ self._eof = False self._paused = False + # Disable the Nagle algorithm -- small writes will be + # sent without waiting for the TCP ACK. This generally + # decreases the latency (in some cases significantly.) + _set_nodelay(self._sock) + self._loop.call_soon(self._protocol.connection_made, self) # only start reading when connection_made() has been called self._loop.call_soon(self._loop.add_reader, diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -365,6 +365,8 @@ - asyncio: Add set_protocol / get_protocol to Transports. +- Issue #27456: asyncio: Set TCP_NODELAY by default. + IDLE ---- -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 21:56:27 2016 From: python-checkins at python.org (martin.panter) Date: Mon, 12 Sep 2016 01:56:27 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4MDY2?= =?utf-8?q?=3A_Fix_include_search_directory_logic_for_out-of-tree_builds?= Message-ID: <20160912015627.19574.50427.B3D696D9@psf.io> https://hg.python.org/cpython/rev/c26dce72a4da changeset: 103676:c26dce72a4da branch: 3.5 parent: 103674:3f7e4ae9eba3 user: Martin Panter date: Mon Sep 12 01:32:03 2016 +0000 summary: Issue #28066: Fix include search directory logic for out-of-tree builds files: Misc/NEWS | 3 +++ configure | 2 +- configure.ac | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -322,6 +322,9 @@ Build ----- +- Issue #28066: Fix the logic that searches build directories for generated + include files when building outside the source tree. + - Issue #27566: Fix clean target in freeze makefile (patch by Lisa Roach) - Issue #27705: Update message in validate_ucrtbase.py diff --git a/configure b/configure --- a/configure +++ b/configure @@ -2820,7 +2820,7 @@ -if test "$abs_srcdir" != "$abs_builddir"; then +if test "$srcdir" != . -a "$srcdir" != "$(pwd)"; then # If we're building out-of-tree, we need to make sure the following # resources get picked up before their $srcdir counterparts. # Objects/ -> typeslots.inc diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -10,7 +10,7 @@ AC_INIT(python, PYTHON_VERSION, https://bugs.python.org/) AC_SUBST(BASECPPFLAGS) -if test "$abs_srcdir" != "$abs_builddir"; then +if test "$srcdir" != . -a "$srcdir" != "$(pwd)"; then # If we're building out-of-tree, we need to make sure the following # resources get picked up before their $srcdir counterparts. # Objects/ -> typeslots.inc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 21:56:27 2016 From: python-checkins at python.org (martin.panter) Date: Mon, 12 Sep 2016 01:56:27 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2328066=3A_Merge_srcdir_fix_from_3=2E5?= Message-ID: <20160912015627.8602.62840.25E1D665@psf.io> https://hg.python.org/cpython/rev/3ef078f96494 changeset: 103677:3ef078f96494 parent: 103675:10384c5c18f5 parent: 103676:c26dce72a4da user: Martin Panter date: Mon Sep 12 01:51:44 2016 +0000 summary: Issue #28066: Merge srcdir fix from 3.5 files: Misc/NEWS | 3 +++ configure | 2 +- configure.ac | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -872,6 +872,9 @@ Build ----- +- Issue #28066: Fix the logic that searches build directories for generated + include files when building outside the source tree. + - Issue #27442: Expose the Android API level that python was built against, in sysconfig.get_config_vars() as 'ANDROID_API_LEVEL'. diff --git a/configure b/configure --- a/configure +++ b/configure @@ -2679,7 +2679,7 @@ -if test "$abs_srcdir" != "$abs_builddir"; then +if test "$srcdir" != . -a "$srcdir" != "$(pwd)"; then # If we're building out-of-tree, we need to make sure the following # resources get picked up before their $srcdir counterparts. # Objects/ -> typeslots.inc diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -10,7 +10,7 @@ AC_INIT(python, PYTHON_VERSION, https://bugs.python.org/) AC_SUBST(BASECPPFLAGS) -if test "$abs_srcdir" != "$abs_builddir"; then +if test "$srcdir" != . -a "$srcdir" != "$(pwd)"; then # If we're building out-of-tree, we need to make sure the following # resources get picked up before their $srcdir counterparts. # Objects/ -> typeslots.inc -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 22:18:20 2016 From: python-checkins at python.org (zach.ware) Date: Mon, 12 Sep 2016 02:18:20 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328065=3A_Update_x?= =?utf-8?q?z_to_5=2E2=2E2_on_Windows=2C_and_build_it_from_source?= Message-ID: <20160912021819.9081.38077.57FEE823@psf.io> https://hg.python.org/cpython/rev/f06e7501e530 changeset: 103678:f06e7501e530 user: Zachary Ware date: Sun Sep 11 21:18:07 2016 -0500 summary: Issue #28065: Update xz to 5.2.2 on Windows, and build it from source files: Misc/NEWS | 2 + PCbuild/_lzma.vcxproj | 12 +- PCbuild/get_externals.bat | 2 +- PCbuild/liblzma.vcxproj | 216 ++++++++++++++++++++++++++ PCbuild/python.props | 2 +- 5 files changed, 227 insertions(+), 7 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -447,6 +447,8 @@ Windows ------- +- Issue #28065: Update xz dependency to 5.2.2 and build it from source. + - Issue #25144: Ensures TargetDir is set before continuing with custom install. diff --git a/PCbuild/_lzma.vcxproj b/PCbuild/_lzma.vcxproj --- a/PCbuild/_lzma.vcxproj +++ b/PCbuild/_lzma.vcxproj @@ -61,13 +61,11 @@
    - $(lzmaDir)include;%(AdditionalIncludeDirectories) + $(lzmaDir)src/liblzma/api;%(AdditionalIncludeDirectories) WIN32;_FILE_OFFSET_BITS=64;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;LZMA_API_STATIC;%(PreprocessorDefinitions) - $(lzmaDir)\bin_i486\liblzma.a;%(AdditionalDependencies) - $(lzmaDir)\bin_x86-64\liblzma.a;%(AdditionalDependencies) - false + $(OutDir)/liblzma$(PyDebugExt).lib @@ -81,8 +79,12 @@ {cf7ac3d1-e2df-41d2-bea6-1e2556cdea26} false + + {12728250-16eC-4dc6-94d7-e21dd88947f8} + false + - \ No newline at end of file + diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -59,7 +59,7 @@ if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tcl-core-8.6.6.0 if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tk-8.6.6.0 if NOT "%IncludeTkinter%"=="false" set libraries=%libraries% tix-8.4.3.6 -set libraries=%libraries% xz-5.0.5 +set libraries=%libraries% xz-5.2.2 for %%e in (%libraries%) do ( if exist %%e ( diff --git a/PCbuild/liblzma.vcxproj b/PCbuild/liblzma.vcxproj new file mode 100644 --- /dev/null +++ b/PCbuild/liblzma.vcxproj @@ -0,0 +1,216 @@ +? + + + + Debug + Win32 + + + Release + Win32 + + + PGInstrument + Win32 + + + PGInstrument + x64 + + + PGUpdate + Win32 + + + PGUpdate + x64 + + + Debug + x64 + + + Release + x64 + + + + {12728250-16EC-4DC6-94D7-E21DD88947F8} + liblzma + true + + + + + + + StaticLibrary + + + + + + + + + + + <_ProjectFileVersion>10.0.30319.1 + + + + WIN32;HAVE_CONFIG_H;_DEBUG;_LIB;%(PreprocessorDefinitions) + Level3 + ProgramDatabase + Disabled + $(lzmaDir)windows;$(lzmaDir)src/liblzma/common;$(lzmaDir)src/common;$(lzmaDir)src/liblzma/api;$(lzmaDir)src/liblzma/check;$(lzmaDir)src/liblzma/delta;$(lzmaDir)src/liblzma/lz;$(lzmaDir)src/liblzma/lzma;$(lzmaDir)src/liblzma/rangecoder;$(lzmaDir)src/liblzma/simple + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/PCbuild/python.props b/PCbuild/python.props --- a/PCbuild/python.props +++ b/PCbuild/python.props @@ -45,7 +45,7 @@ $([System.IO.Path]::GetFullPath(`$(PySourcePath)externals\`)) $(ExternalsDir)sqlite-3.14.1.0\ $(ExternalsDir)bzip2-1.0.6\ - $(ExternalsDir)xz-5.0.5\ + $(ExternalsDir)xz-5.2.2\ $(ExternalsDir)openssl-1.0.2h\ $(opensslDir)include32 $(opensslDir)include64 -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 22:44:11 2016 From: python-checkins at python.org (steve.dower) Date: Mon, 12 Sep 2016 02:44:11 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Fixes_test=5Fgetargs2_to_g?= =?utf-8?q?et_the_buildbots_working_again=2E?= Message-ID: <20160912024411.21095.89458.35BA1187@psf.io> https://hg.python.org/cpython/rev/7793d34609cb changeset: 103679:7793d34609cb user: Steve Dower date: Sun Sep 11 19:43:51 2016 -0700 summary: Fixes test_getargs2 to get the buildbots working again. files: Lib/test/test_getargs2.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_getargs2.py b/Lib/test/test_getargs2.py --- a/Lib/test/test_getargs2.py +++ b/Lib/test/test_getargs2.py @@ -471,7 +471,7 @@ ret = get_args(*TupleSubclass([1, 2])) self.assertEqual(ret, (1, 2)) - self.assertIs(type(ret), tuple) + self.assertIsInstance(ret, tuple) ret = get_args() self.assertIn(ret, ((), None)) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 22:55:19 2016 From: python-checkins at python.org (alexander.belopolsky) Date: Mon, 12 Sep 2016 02:55:19 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Closes_=2325283=3A_Make_tm?= =?utf-8?q?=5Fgmtoff_and_tm=5Fzone_available_on_all_platforms=2E?= Message-ID: <20160912025519.8517.3451.528DDC59@psf.io> https://hg.python.org/cpython/rev/a96101dd105c changeset: 103680:a96101dd105c user: Alexander Belopolsky date: Sun Sep 11 22:55:16 2016 -0400 summary: Closes #25283: Make tm_gmtoff and tm_zone available on all platforms. files: Doc/library/time.rst | 8 +- Misc/NEWS | 4 + Modules/timemodule.c | 114 ++++++++++++++++++++++-------- 3 files changed, 89 insertions(+), 37 deletions(-) diff --git a/Doc/library/time.rst b/Doc/library/time.rst --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -83,6 +83,10 @@ and :attr:`tm_zone` attributes when platform supports corresponding ``struct tm`` members. + .. versionchanged:: 3.6 + The :class:`struct_time` attributes :attr:`tm_gmtoff` and :attr:`tm_zone` + are now available on all platforms. + * Use the following functions to convert between time representations: +-------------------------+-------------------------+-------------------------+ @@ -566,10 +570,6 @@ :class:`struct_time`, or having elements of the wrong type, a :exc:`TypeError` is raised. - .. versionchanged:: 3.3 - :attr:`tm_gmtoff` and :attr:`tm_zone` attributes are available on platforms - with C library supporting the corresponding fields in ``struct tm``. - .. function:: time() Return the time in seconds since the epoch as a floating point number. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -143,6 +143,10 @@ Library ------- +- Issue #25283: Attributes tm_gmtoff and tm_zone are now available on + all platforms in the return values of time.localtime() and + time.gmtime(). + - Issue #24454: Regular expression match object groups are now accessible using __getitem__. "mo[x]" is equivalent to "mo.group(x)". diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -250,10 +250,8 @@ {"tm_wday", "day of week, range [0, 6], Monday is 0"}, {"tm_yday", "day of year, range [1, 366]"}, {"tm_isdst", "1 if summer time is in effect, 0 if not, and -1 if unknown"}, -#ifdef HAVE_STRUCT_TM_TM_ZONE {"tm_zone", "abbreviation of timezone name"}, {"tm_gmtoff", "offset from UTC in seconds"}, -#endif /* HAVE_STRUCT_TM_TM_ZONE */ {0} }; @@ -275,7 +273,11 @@ static PyObject * -tmtotuple(struct tm *p) +tmtotuple(struct tm *p +#ifndef HAVE_STRUCT_TM_TM_ZONE + , const char *zone, int gmtoff +#endif +) { PyObject *v = PyStructSequence_New(&StructTimeType); if (v == NULL) @@ -296,6 +298,10 @@ PyStructSequence_SET_ITEM(v, 9, PyUnicode_DecodeLocale(p->tm_zone, "surrogateescape")); SET(10, p->tm_gmtoff); +#else + PyStructSequence_SET_ITEM(v, 9, + PyUnicode_DecodeLocale(zone, "surrogateescape")); + SET(10, gmtoff); #endif /* HAVE_STRUCT_TM_TM_ZONE */ #undef SET if (PyErr_Occurred()) { @@ -348,9 +354,26 @@ return PyErr_SetFromErrno(PyExc_OSError); } buf = *local; +#ifdef HAVE_STRUCT_TM_TM_ZONE return tmtotuple(&buf); +#else + return tmtotuple(&buf, "UTC", 0); +#endif } +#ifndef HAVE_TIMEGM +static time_t +timegm(struct tm *p) +{ + /* XXX: the following implementation will not work for tm_year < 1970. + but it is likely that platforms that don't have timegm do not support + negative timestamps anyways. */ + return p->tm_sec + p->tm_min*60 + p->tm_hour*3600 + p->tm_yday*86400 + + (p->tm_year-70)*31536000 + ((p->tm_year-69)/4)*86400 - + ((p->tm_year-1)/100)*86400 + ((p->tm_year+299)/400)*86400; +} +#endif + PyDoc_STRVAR(gmtime_doc, "gmtime([seconds]) -> (tm_year, tm_mon, tm_mday, tm_hour, tm_min,\n\ tm_sec, tm_wday, tm_yday, tm_isdst)\n\ @@ -391,7 +414,18 @@ return NULL; if (pylocaltime(&when, &buf) == -1) return NULL; +#ifdef HAVE_STRUCT_TM_TM_ZONE return tmtotuple(&buf); +#else + { + struct tm local = buf; + char zone[100]; + int gmtoff; + strftime(zone, sizeof(buf), "%Z", &buf); + gmtoff = timegm(&buf) - when; + return tmtotuple(&local, zone, gmtoff); + } +#endif } PyDoc_STRVAR(localtime_doc, @@ -1146,6 +1180,27 @@ Get information of the specified clock."); static void +get_zone(char *zone, int n, struct tm *p) +{ +#ifdef HAVE_STRUCT_TM_TM_ZONE + strncpy(zone, p->tm_zone ? p->tm_zone : " ", n); +#else + tzset(); + strftime(zone, n, "%Z", p); +#endif +} + +static int +get_gmtoff(time_t t, struct tm *p) +{ +#ifdef HAVE_STRUCT_TM_TM_ZONE + return p->tm_gmtoff; +#else + return timegm(p) - t; +#endif +} + +static void PyInit_timezone(PyObject *m) { /* This code moved from PyInit_time wholesale to allow calling it from time_tzset. In the future, some parts of it can be moved back @@ -1177,7 +1232,6 @@ otz1 = PyUnicode_DecodeLocale(tzname[1], "surrogateescape"); PyModule_AddObject(m, "tzname", Py_BuildValue("(NN)", otz0, otz1)); #else /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/ -#ifdef HAVE_STRUCT_TM_TM_ZONE { #define YEAR ((time_t)((365 * 24 + 6) * 3600)) time_t t; @@ -1186,13 +1240,13 @@ char janname[10], julyname[10]; t = (time((time_t *)0) / YEAR) * YEAR; p = localtime(&t); - janzone = -p->tm_gmtoff; - strncpy(janname, p->tm_zone ? p->tm_zone : " ", 9); + get_zone(janname, 9, p); + janzone = -get_gmtoff(t, p); janname[9] = '\0'; t += YEAR/2; p = localtime(&t); - julyzone = -p->tm_gmtoff; - strncpy(julyname, p->tm_zone ? p->tm_zone : " ", 9); + get_zone(julyname, 9, p); + julyzone = -get_gmtoff(t, p); julyname[9] = '\0'; if( janzone < julyzone ) { @@ -1214,8 +1268,6 @@ janname, julyname)); } } -#else -#endif /* HAVE_STRUCT_TM_TM_ZONE */ #ifdef __CYGWIN__ tzset(); PyModule_AddIntConstant(m, "timezone", _timezone); @@ -1225,25 +1277,6 @@ Py_BuildValue("(zz)", _tzname[0], _tzname[1])); #endif /* __CYGWIN__ */ #endif /* !HAVE_TZNAME || __GLIBC__ || __CYGWIN__*/ - -#if defined(HAVE_CLOCK_GETTIME) - PyModule_AddIntMacro(m, CLOCK_REALTIME); -#ifdef CLOCK_MONOTONIC - PyModule_AddIntMacro(m, CLOCK_MONOTONIC); -#endif -#ifdef CLOCK_MONOTONIC_RAW - PyModule_AddIntMacro(m, CLOCK_MONOTONIC_RAW); -#endif -#ifdef CLOCK_HIGHRES - PyModule_AddIntMacro(m, CLOCK_HIGHRES); -#endif -#ifdef CLOCK_PROCESS_CPUTIME_ID - PyModule_AddIntMacro(m, CLOCK_PROCESS_CPUTIME_ID); -#endif -#ifdef CLOCK_THREAD_CPUTIME_ID - PyModule_AddIntMacro(m, CLOCK_THREAD_CPUTIME_ID); -#endif -#endif /* HAVE_CLOCK_GETTIME */ } @@ -1350,17 +1383,32 @@ /* Set, or reset, module variables like time.timezone */ PyInit_timezone(m); +#if defined(HAVE_CLOCK_GETTIME) + PyModule_AddIntMacro(m, CLOCK_REALTIME); +#ifdef CLOCK_MONOTONIC + PyModule_AddIntMacro(m, CLOCK_MONOTONIC); +#endif +#ifdef CLOCK_MONOTONIC_RAW + PyModule_AddIntMacro(m, CLOCK_MONOTONIC_RAW); +#endif +#ifdef CLOCK_HIGHRES + PyModule_AddIntMacro(m, CLOCK_HIGHRES); +#endif +#ifdef CLOCK_PROCESS_CPUTIME_ID + PyModule_AddIntMacro(m, CLOCK_PROCESS_CPUTIME_ID); +#endif +#ifdef CLOCK_THREAD_CPUTIME_ID + PyModule_AddIntMacro(m, CLOCK_THREAD_CPUTIME_ID); +#endif +#endif /* HAVE_CLOCK_GETTIME */ + if (!initialized) { if (PyStructSequence_InitType2(&StructTimeType, &struct_time_type_desc) < 0) return NULL; } Py_INCREF(&StructTimeType); -#ifdef HAVE_STRUCT_TM_TM_ZONE PyModule_AddIntConstant(m, "_STRUCT_TM_ITEMS", 11); -#else - PyModule_AddIntConstant(m, "_STRUCT_TM_ITEMS", 9); -#endif PyModule_AddObject(m, "struct_time", (PyObject*) &StructTimeType); initialized = 1; return m; -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 23:19:50 2016 From: python-checkins at python.org (steve.dower) Date: Mon, 12 Sep 2016 03:19:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Make_PGO_use_usual_build_d?= =?utf-8?q?irectory_on_Windows=2E?= Message-ID: <20160912031950.19131.57300.8CE52220@psf.io> https://hg.python.org/cpython/rev/3308f06726e9 changeset: 103682:3308f06726e9 user: Steve Dower date: Sun Sep 11 20:19:35 2016 -0700 summary: Make PGO use usual build directory on Windows. files: PCbuild/pyproject.props | 1 - PCbuild/python.props | 1 - Tools/msi/buildrelease.bat | 9 +-------- 3 files changed, 1 insertions(+), 10 deletions(-) diff --git a/PCbuild/pyproject.props b/PCbuild/pyproject.props --- a/PCbuild/pyproject.props +++ b/PCbuild/pyproject.props @@ -7,7 +7,6 @@ $(OutDir)\ $(MSBuildThisFileDirectory)obj\ $(Py_IntDir)\$(ArchName)_$(Configuration)\$(ProjectName)\ - $(Py_IntDir)\$(ArchName)_PGO\$(ProjectName)\ $(ProjectName) $(TargetName)$(PyDebugExt) false diff --git a/PCbuild/python.props b/PCbuild/python.props --- a/PCbuild/python.props +++ b/PCbuild/python.props @@ -25,7 +25,6 @@ --> amd64 win32 - $(ArchName)-pgo $([System.IO.Path]::GetFullPath($(MSBuildThisFileDirectory)\..\)) diff --git a/Tools/msi/buildrelease.bat b/Tools/msi/buildrelease.bat --- a/Tools/msi/buildrelease.bat +++ b/Tools/msi/buildrelease.bat @@ -111,16 +111,10 @@ set BUILD_PLAT=Win32 set OUTDIR_PLAT=win32 set OBJDIR_PLAT=x86 -) else if "%~2" NEQ "" ( - call "%PCBUILD%env.bat" amd64 - set PGO=%~2 - set BUILD=%PCBUILD%amd64-pgo\ - set BUILD_PLAT=x64 - set OUTDIR_PLAT=amd64 - set OBJDIR_PLAT=x64 ) else ( call "%PCBUILD%env.bat" amd64 set BUILD=%PCBUILD%amd64\ + set PGO=%~2 set BUILD_PLAT=x64 set OUTDIR_PLAT=amd64 set OBJDIR_PLAT=x64 @@ -177,7 +171,6 @@ ) set BUILDOPTS=/p:Platform=%1 /p:BuildForRelease=true /p:DownloadUrl=%DOWNLOAD_URL% /p:DownloadUrlBase=%DOWNLOAD_URL_BASE% /p:ReleaseUri=%RELEASE_URI% -if "%PGO%" NEQ "" set BUILDOPTS=%BUILDOPTS% /p:PGOBuildPath=%BUILD% msbuild "%D%bundle\releaselocal.wixproj" /t:Rebuild %BUILDOPTS% %CERTOPTS% /p:RebuildAll=true if errorlevel 1 exit /B msbuild "%D%bundle\releaseweb.wixproj" /t:Rebuild %BUILDOPTS% %CERTOPTS% /p:RebuildAll=false -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 23:19:50 2016 From: python-checkins at python.org (steve.dower) Date: Mon, 12 Sep 2016 03:19:50 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Adds_missing_assert_suppre?= =?utf-8?q?ssion=2E?= Message-ID: <20160912031950.8568.62620.B6AE8DAF@psf.io> https://hg.python.org/cpython/rev/5c479bb0c31e changeset: 103681:5c479bb0c31e user: Steve Dower date: Sun Sep 11 20:19:32 2016 -0700 summary: Adds missing assert suppression. files: Modules/posixmodule.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -5030,11 +5030,13 @@ mode = _P_OVERLAY; Py_BEGIN_ALLOW_THREADS + _Py_BEGIN_SUPPRESS_IPH #ifdef HAVE_WSPAWNV spawnval = _wspawnv(mode, path->wide, argvlist); #else spawnval = _spawnv(mode, path->narrow, argvlist); #endif + _Py_END_SUPPRESS_IPH Py_END_ALLOW_THREADS free_string_array(argvlist, argc); @@ -5122,11 +5124,13 @@ mode = _P_OVERLAY; Py_BEGIN_ALLOW_THREADS + _Py_BEGIN_SUPPRESS_IPH #ifdef HAVE_WSPAWNV spawnval = _wspawnve(mode, path->wide, argvlist, envlist); #else spawnval = _spawnve(mode, path->narrow, argvlist, envlist); #endif + _Py_END_SUPPRESS_IPH Py_END_ALLOW_THREADS if (spawnval == -1) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Sun Sep 11 23:38:32 2016 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 12 Sep 2016 03:38:32 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzI2NTU3?= =?utf-8?q?=3A__Note_that_mapping_view_methods_are_not_present_in_UserDict?= =?utf-8?q?_or?= Message-ID: <20160912033832.8732.20881.29033EF8@psf.io> https://hg.python.org/cpython/rev/2b4bfef0d54a changeset: 103683:2b4bfef0d54a branch: 2.7 parent: 103638:5ca4c545dfe4 user: Raymond Hettinger date: Sun Sep 11 20:38:27 2016 -0700 summary: Issue #26557: Note that mapping view methods are not present in UserDict or shelves. files: Doc/library/shelve.rst | 8 ++++++-- Doc/library/userdict.rst | 3 +++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Doc/library/shelve.rst b/Doc/library/shelve.rst --- a/Doc/library/shelve.rst +++ b/Doc/library/shelve.rst @@ -53,8 +53,12 @@ to load a shelf from an untrusted source. Like with pickle, loading a shelf can execute arbitrary code. -Shelf objects support all methods supported by dictionaries. This eases the -transition from dictionary based scripts to those requiring persistent storage. +Shelf objects support most of the methods supported by dictionaries. This +eases the transition from dictionary based scripts to those requiring +persistent storage. + +Note, the Python 3 transition methods (:meth:`~dict.viewkeys`, +:meth:`~dict.viewvalues`, and :meth:`~dict.viewitems`) are not supported. Two additional methods are supported: diff --git a/Doc/library/userdict.rst b/Doc/library/userdict.rst --- a/Doc/library/userdict.rst +++ b/Doc/library/userdict.rst @@ -74,6 +74,9 @@ Starting with Python version 2.6, it is recommended to use :class:`collections.MutableMapping` instead of :class:`DictMixin`. + Note that DictMixin does not implement the :meth:`~dict.viewkeys`, + :meth:`~dict.viewvalues`, or :meth:`~dict.viewitems` methods. + :mod:`UserList` --- Class wrapper for list objects ================================================== -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 12 00:16:05 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 12 Sep 2016 04:16:05 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328037=3A_Use_sqli?= =?utf-8?q?te3=5Fget=5Fautocommit=28=29_instead_of_setting?= Message-ID: <20160912041604.8909.44444.D170BE2B@psf.io> https://hg.python.org/cpython/rev/80946f95e88a changeset: 103684:80946f95e88a parent: 103682:3308f06726e9 user: Berker Peksag date: Mon Sep 12 07:16:43 2016 +0300 summary: Issue #28037: Use sqlite3_get_autocommit() instead of setting Connection->inTransaction manually Patch adapted from https://github.com/ghaering/pysqlite/commit/9b79188edbc50faa24dc178afe24a10454f3fcad files: Misc/NEWS | 3 ++ Modules/_sqlite/connection.c | 31 +++++++++++++---------- Modules/_sqlite/connection.h | 4 --- Modules/_sqlite/cursor.c | 9 ------ 4 files changed, 20 insertions(+), 27 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -143,6 +143,9 @@ Library ------- +- Issue #28037: Use sqlite3_get_autocommit() instead of setting + Connection->inTransaction manually. + - Issue #25283: Attributes tm_gmtoff and tm_zone are now available on all platforms in the return values of time.localtime() and time.gmtime(). diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -165,7 +165,6 @@ self->statement_cache->decref_factory = 0; Py_DECREF(self); - self->inTransaction = 0; self->detect_types = detect_types; self->timeout = timeout; (void)sqlite3_busy_timeout(self->db, (int)(timeout*1000)); @@ -385,9 +384,7 @@ } rc = pysqlite_step(statement, self); - if (rc == SQLITE_DONE) { - self->inTransaction = 1; - } else { + if (rc != SQLITE_DONE) { _pysqlite_seterror(self->db, statement); } @@ -418,7 +415,7 @@ return NULL; } - if (self->inTransaction) { + if (!sqlite3_get_autocommit(self->db)) { Py_BEGIN_ALLOW_THREADS rc = sqlite3_prepare(self->db, "COMMIT", -1, &statement, &tail); @@ -429,9 +426,7 @@ } rc = pysqlite_step(statement, self); - if (rc == SQLITE_DONE) { - self->inTransaction = 0; - } else { + if (rc != SQLITE_DONE) { _pysqlite_seterror(self->db, statement); } @@ -463,7 +458,7 @@ return NULL; } - if (self->inTransaction) { + if (!sqlite3_get_autocommit(self->db)) { pysqlite_do_all_statements(self, ACTION_RESET, 1); Py_BEGIN_ALLOW_THREADS @@ -475,9 +470,7 @@ } rc = pysqlite_step(statement, self); - if (rc == SQLITE_DONE) { - self->inTransaction = 0; - } else { + if (rc != SQLITE_DONE) { _pysqlite_seterror(self->db, statement); } @@ -1158,6 +1151,17 @@ } } +static PyObject* pysqlite_connection_get_in_transaction(pysqlite_Connection* self, void* unused) +{ + if (!pysqlite_check_connection(self)) { + return NULL; + } + if (!sqlite3_get_autocommit(self->db)) { + Py_RETURN_TRUE; + } + Py_RETURN_FALSE; +} + static int pysqlite_connection_set_isolation_level(pysqlite_Connection* self, PyObject* isolation_level) { if (isolation_level == Py_None) { @@ -1168,7 +1172,6 @@ Py_DECREF(res); self->begin_statement = NULL; - self->inTransaction = 0; } else { const char * const *candidate; PyObject *uppercase_level; @@ -1606,6 +1609,7 @@ static PyGetSetDef connection_getset[] = { {"isolation_level", (getter)pysqlite_connection_get_isolation_level, (setter)pysqlite_connection_set_isolation_level}, {"total_changes", (getter)pysqlite_connection_get_total_changes, (setter)0}, + {"in_transaction", (getter)pysqlite_connection_get_in_transaction, (setter)0}, {NULL} }; @@ -1667,7 +1671,6 @@ {"NotSupportedError", T_OBJECT, offsetof(pysqlite_Connection, NotSupportedError), READONLY}, {"row_factory", T_OBJECT, offsetof(pysqlite_Connection, row_factory)}, {"text_factory", T_OBJECT, offsetof(pysqlite_Connection, text_factory)}, - {"in_transaction", T_BOOL, offsetof(pysqlite_Connection, inTransaction), READONLY}, {NULL} }; diff --git a/Modules/_sqlite/connection.h b/Modules/_sqlite/connection.h --- a/Modules/_sqlite/connection.h +++ b/Modules/_sqlite/connection.h @@ -37,10 +37,6 @@ PyObject_HEAD sqlite3* db; - /* 1 if we are currently within a transaction, i. e. if a BEGIN has been - * issued */ - char inTransaction; - /* the type detection mode. Only 0, PARSE_DECLTYPES, PARSE_COLNAMES or a * bitwise combination thereof makes sense */ int detect_types; diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -644,15 +644,6 @@ } error: - /* just to be sure (implicit ROLLBACKs with ON CONFLICT ROLLBACK/OR - * ROLLBACK could have happened */ - #ifdef SQLITE_VERSION_NUMBER - #if SQLITE_VERSION_NUMBER >= 3002002 - if (self->connection && self->connection->db) - self->connection->inTransaction = !sqlite3_get_autocommit(self->connection->db); - #endif - #endif - Py_XDECREF(parameters); Py_XDECREF(parameters_iter); Py_XDECREF(parameters_list); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 12 00:26:56 2016 From: python-checkins at python.org (ned.deily) Date: Mon, 12 Sep 2016 04:26:56 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328095=3A_Temporar?= =?utf-8?q?ily_disable_part_of_test=5Fstartup=5Fimports_on_OS_X=2E?= Message-ID: <20160912042656.2686.87341.D7FD5F79@psf.io> https://hg.python.org/cpython/rev/9f0016055820 changeset: 103685:9f0016055820 user: Ned Deily date: Mon Sep 12 00:26:20 2016 -0400 summary: Issue #28095: Temporarily disable part of test_startup_imports on OS X. files: Lib/test/test_site.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -470,7 +470,9 @@ 'heapq', 'itertools', 'keyword', 'operator', 'reprlib', 'types', 'weakref' }.difference(sys.builtin_module_names) - self.assertFalse(modules.intersection(collection_mods), stderr) + # http://bugs.python.org/issue28095 + if sys.platform != 'darwin': + self.assertFalse(modules.intersection(collection_mods), stderr) if __name__ == "__main__": -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 12 00:46:55 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 12 Sep 2016 04:46:55 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Issue_=2328045=3A_Merge_from_3=2E5?= Message-ID: <20160912044655.8705.10908.F06B8235@psf.io> https://hg.python.org/cpython/rev/924a64745168 changeset: 103687:924a64745168 parent: 103685:9f0016055820 parent: 103686:957084395042 user: Berker Peksag date: Mon Sep 12 07:47:33 2016 +0300 summary: Issue #28045: Merge from 3.5 files: Objects/rangeobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -399,7 +399,7 @@ tmp2 = PyNumber_Remainder(tmp1, r->step); if (tmp2 == NULL) goto end; - /* result = (int(ob) - start % step) == 0 */ + /* result = ((int(ob) - start) % step) == 0 */ result = PyObject_RichCompareBool(tmp2, zero, Py_EQ); end: Py_XDECREF(tmp1); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 12 00:46:55 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 12 Sep 2016 04:46:55 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMy41KTogSXNzdWUgIzI4MDQ1?= =?utf-8?q?=3A_Fix_comment_in_range=5Fcontains=5Flong=28=29?= Message-ID: <20160912044655.59601.67622.1C258B93@psf.io> https://hg.python.org/cpython/rev/957084395042 changeset: 103686:957084395042 branch: 3.5 parent: 103676:c26dce72a4da user: Berker Peksag date: Mon Sep 12 07:47:04 2016 +0300 summary: Issue #28045: Fix comment in range_contains_long() Patch by wim glenn. files: Objects/rangeobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -406,7 +406,7 @@ tmp2 = PyNumber_Remainder(tmp1, r->step); if (tmp2 == NULL) goto end; - /* result = (int(ob) - start % step) == 0 */ + /* result = ((int(ob) - start) % step) == 0 */ result = PyObject_RichCompareBool(tmp2, zero, Py_EQ); end: Py_XDECREF(tmp1); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 12 00:59:21 2016 From: python-checkins at python.org (berker.peksag) Date: Mon, 12 Sep 2016 04:59:21 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Add_missing_versionadded_d?= =?utf-8?q?irectives?= Message-ID: <20160912045921.2824.76404.511F4756@psf.io> https://hg.python.org/cpython/rev/45cd0b7b78c6 changeset: 103688:45cd0b7b78c6 user: Berker Peksag date: Mon Sep 12 08:00:01 2016 +0300 summary: Add missing versionadded directives files: Doc/library/dis.rst | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -614,6 +614,8 @@ or module body contains :term:`variable annotations ` statically. + .. versionadded:: 3.6 + .. opcode:: IMPORT_STAR Loads all symbols not starting with ``'_'`` directly from the module TOS to @@ -900,6 +902,8 @@ Stores TOS as ``locals()['__annotations__'][co_names[namei]] = TOS``. + .. versionadded:: 3.6 + .. opcode:: LOAD_CLOSURE (i) -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 12 01:02:35 2016 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 12 Sep 2016 05:02:35 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Issue_=2328071=3A_Add_earl?= =?utf-8?q?y-out_for_differencing_from_an_empty_set=2E?= Message-ID: <20160912050235.1874.59573.E2C84529@psf.io> https://hg.python.org/cpython/rev/fa0af1a6344d changeset: 103689:fa0af1a6344d user: Raymond Hettinger date: Sun Sep 11 22:02:28 2016 -0700 summary: Issue #28071: Add early-out for differencing from an empty set. files: Misc/NEWS | 2 ++ Objects/setobject.c | 8 ++++++++ 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -33,6 +33,8 @@ - Issue #28046: Remove platform-specific directories from sys.path. +- Issue #28071: Add early-out for differencing from an empty set. + - Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch by Eryk Sun) diff --git a/Objects/setobject.c b/Objects/setobject.c --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -1476,6 +1476,10 @@ static int set_difference_update_internal(PySetObject *so, PyObject *other) { + if (PySet_GET_SIZE(so) == 0) { + return 0; + } + if ((PyObject *)so == other) return set_clear_internal(so); @@ -1550,6 +1554,10 @@ Py_ssize_t pos = 0; int rv; + if (PySet_GET_SIZE(so) == 0) { + return set_copy(so); + } + if (!PyAnySet_Check(other) && !PyDict_CheckExact(other)) { return set_copy_and_difference(so, other); } -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 12 01:46:08 2016 From: python-checkins at python.org (raymond.hettinger) Date: Mon, 12 Sep 2016 05:46:08 +0000 Subject: [Python-checkins] =?utf-8?q?cpython=3A_Revert_part_of_3471a351582?= =?utf-8?q?7_that_caused_a_performance_regression?= Message-ID: <20160912054608.8502.43453.98B7AEFF@psf.io> https://hg.python.org/cpython/rev/fdb5ae2f25c6 changeset: 103690:fdb5ae2f25c6 user: Raymond Hettinger date: Sun Sep 11 22:45:53 2016 -0700 summary: Revert part of 3471a3515827 that caused a performance regression files: Modules/_collectionsmodule.c | 52 ++++++++++++++++++++--- 1 files changed, 44 insertions(+), 8 deletions(-) diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -403,10 +403,28 @@ iternext = *Py_TYPE(it)->tp_iternext; while ((item = iternext(it)) != NULL) { - if (deque_append_internal(deque, item, maxlen) < 0) { - Py_DECREF(item); - Py_DECREF(it); - return NULL; + if (deque->rightindex == BLOCKLEN - 1) { + block *b = newblock(); + if (b == NULL) { + Py_DECREF(item); + Py_DECREF(it); + return NULL; + } + b->leftlink = deque->rightblock; + CHECK_END(deque->rightblock->rightlink); + deque->rightblock->rightlink = b; + deque->rightblock = b; + MARK_END(b->rightlink); + deque->rightindex = -1; + } + Py_SIZE(deque)++; + deque->rightindex++; + deque->rightblock->data[deque->rightindex] = item; + if (NEEDS_TRIM(deque, maxlen)) { + PyObject *olditem = deque_popleft(deque, NULL); + Py_DECREF(olditem); + } else { + deque->state++; } } return finalize_iterator(it); @@ -450,10 +468,28 @@ iternext = *Py_TYPE(it)->tp_iternext; while ((item = iternext(it)) != NULL) { - if (deque_appendleft_internal(deque, item, maxlen) < 0) { - Py_DECREF(item); - Py_DECREF(it); - return NULL; + if (deque->leftindex == 0) { + block *b = newblock(); + if (b == NULL) { + Py_DECREF(item); + Py_DECREF(it); + return NULL; + } + b->rightlink = deque->leftblock; + CHECK_END(deque->leftblock->leftlink); + deque->leftblock->leftlink = b; + deque->leftblock = b; + MARK_END(b->leftlink); + deque->leftindex = BLOCKLEN; + } + Py_SIZE(deque)++; + deque->leftindex--; + deque->leftblock->data[deque->leftindex] = item; + if (NEEDS_TRIM(deque, maxlen)) { + PyObject *olditem = deque_pop(deque, NULL); + Py_DECREF(olditem); + } else { + deque->state++; } } return finalize_iterator(it); -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 12 01:57:41 2016 From: python-checkins at python.org (terry.reedy) Date: Mon, 12 Sep 2016 05:57:41 +0000 Subject: [Python-checkins] =?utf-8?b?Y3B5dGhvbiAoMi43KTogSXNzdWUgIzE1MzA4?= =?utf-8?q?=3A_Add_=27interrupt_execution=27_=28=5EC=29_to_Shell_menu=2E?= Message-ID: <20160912055741.20970.5475.3B698495@psf.io> https://hg.python.org/cpython/rev/9148a2213631 changeset: 103691:9148a2213631 branch: 2.7 parent: 103683:2b4bfef0d54a user: Terry Jan Reedy date: Mon Sep 12 01:49:55 2016 -0400 summary: Issue #15308: Add 'interrupt execution' (^C) to Shell menu. Patch by Roger Serwy, updated by Bayard Randel. files: Doc/library/idle.rst | 3 +++ Lib/idlelib/Bindings.py | 2 ++ Lib/idlelib/README.txt | 11 ++++++----- Lib/idlelib/help.html | 4 +++- 4 files changed, 14 insertions(+), 6 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -222,6 +222,9 @@ Restart Shell Restart the shell to clean the environment. +Interrupt Execution + Stop a running program. + Debug menu (Shell window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/Bindings.py b/Lib/idlelib/Bindings.py --- a/Lib/idlelib/Bindings.py +++ b/Lib/idlelib/Bindings.py @@ -67,6 +67,8 @@ ('shell', [ ('_View Last Restart', '<>'), ('_Restart Shell', '<>'), + None, + ('_Interrupt Execution', '<>'), ]), ('debug', [ ('_Go to File/Line', '<>'), diff --git a/Lib/idlelib/README.txt b/Lib/idlelib/README.txt --- a/Lib/idlelib/README.txt +++ b/Lib/idlelib/README.txt @@ -161,14 +161,15 @@ Show surrounding parens # ParenMatch (& Hyperparser) Shell # PyShell - View Last Restart # PyShell.? - Restart Shell # PyShell.? + View Last Restart # PyShell.PyShell.view_restart_mark + Restart Shell # PyShell.PyShell.restart_shell + Interrupt Execution # pyshell.PyShell.cancel_callback Debug (Shell only) Go to File/Line - Debugger # Debugger, RemoteDebugger - Stack Viewer # StackViewer - Auto-open Stack Viewer # StackViewer + Debugger # Debugger, RemoteDebugger, PyShell.toggle_debuger + Stack Viewer # StackViewer, PyShell.open_stack_viewer + Auto-open Stack Viewer # StackViewer Format (Editor only) Indent Region diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -238,6 +238,8 @@
    Scroll the shell window to the last Shell restart.
    Restart Shell
    Restart the shell to clean the environment.
    +
    Interrupt Execution
    +
    Stop a running program.
    @@ -703,7 +705,7 @@ The Python Software Foundation is a non-profit corporation. Please donate.
    - Last updated on Aug 30, 2016. + Last updated on Sep 12, 2016. Found a bug?
    Created using Sphinx 1.3.6. -- Repository URL: https://hg.python.org/cpython From python-checkins at python.org Mon Sep 12 01:57:41 2016 From: python-checkins at python.org (terry.reedy) Date: Mon, 12 Sep 2016 05:57:41 +0000 Subject: [Python-checkins] =?utf-8?q?cpython_=28merge_3=2E5_-=3E_default?= =?utf-8?q?=29=3A_Merge_3=2E5_-_Issue_=2315308=3A_Add_=27interrupt_executi?= =?utf-8?b?b24nICheQykgdG8gU2hlbGwgbWVudS4=?= Message-ID: <20160912055741.21237.29935.BDFEEF9B@psf.io> https://hg.python.org/cpython/rev/63f6ac38d18d changeset: 103693:63f6ac38d18d parent: 103690:fdb5ae2f25c6 parent: 103692:74b84014bc27 user: Terry Jan Reedy date: Mon Sep 12 01:57:25 2016 -0400 summary: Merge 3.5 - Issue #15308: Add 'interrupt execution' (^C) to Shell menu. Patch by Roger Serwy, updated by Bayard Randel. files: Doc/library/idle.rst | 3 + Lib/idlelib/NEWS.txt | 3 + Lib/idlelib/README.txt | 9 ++- Lib/idlelib/help.html | 62 +++++++++++++++++++--------- Lib/idlelib/mainmenu.py | 2 + Misc/ACKS | 1 + Misc/NEWS | 3 + 7 files changed, 58 insertions(+), 25 deletions(-) diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -226,6 +226,9 @@ Restart Shell Restart the shell to clean the environment. +Interrupt Execution + Stop a running program. + Debug menu (Shell window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -2,6 +2,9 @@ =========================== *Release date: 2016-12-16?* +- Issue #15308: Add 'interrupt execution' (^C) to Shell menu. + Patch by Roger Serwy, updated by Bayard Randel. + - Issue #27922: Stop IDLE tests from 'flashing' gui widgets on the screen. - Issue #27891: Consistently group and sort imports within idlelib modules. diff --git a/Lib/idlelib/README.txt b/Lib/idlelib/README.txt --- a/Lib/idlelib/README.txt +++ b/Lib/idlelib/README.txt @@ -160,13 +160,14 @@ Show surrounding parens # parenmatch (& Hyperparser) Shell # pyshell - View Last Restart# pyshell.? - Restart Shell # pyshell.? + View Last Restart # pyshell.PyShell.view_restart_mark + Restart Shell # pyshell.PyShell.restart_shell + Interrupt Execution # pyshell.PyShell.cancel_callback Debug (Shell only) Go to File/Line - debugger # debugger, debugger_r - Stack Viewer # stackviewer + debugger # debugger, debugger_r, PyShell.toggle_debuger + Stack Viewer # stackviewer, PyShell.open_stack_viewer Auto-open Stack Viewer # stackviewer Format (Editor only) diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -65,6 +65,21 @@ +
  • + + + + + | +
  • +
    @@ -240,6 +255,8 @@
    Scroll the shell window to the last Shell restart.
    Restart Shell
    Restart the shell to clean the environment.
    +
    Interrupt Execution
    +
    Stop a running program.
    @@ -649,26 +666,14 @@

    Next topic

    25.6. Other Graphical User Interface Packages

    -

    This Page

    - - - - +
    +

    This Page

    + +
    @@ -697,6 +702,21 @@ +
  • + + + + + | +
  • + @@ -240,6 +255,8 @@
    Scroll the shell window to the last Shell restart.
    Restart Shell
    Restart the shell to clean the environment.
    +
    Interrupt Execution
    +
    Stop a running program.
    @@ -649,26 +666,14 @@

    Next topic

    25.6. Other Graphical User Interface Packages

    -

    This Page

    - - - - +
    +

    This Page

    + +
    @@ -697,6 +702,21 @@ +
  • + + + + + | +
  • + @@ -240,6 +255,8 @@
    Scroll the shell window to the last Shell restart.
    Restart Shell
    Restart the shell to clean the environment.
    +
    Interrupt Execution
    +
    Stop a running program.
    @@ -649,26 +666,14 @@

    Next topic

    25.6. Other Graphical User Interface Packages

    -

    This Page

    - - - - +
    +

    This Page

    + +
    @@ -697,6 +702,21 @@ +
  • + + + + + | +
  • +