Mailman 3 October 2021 - Python-checkins

bpo-45600: Enhanced / clarified the docs for os.environ and os.environb (GH-29204) (GH-29322)
by ambv Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/0538351fcaa320e62cb4ef92ec629b7e24… commit: 0538351fcaa320e62cb4ef92ec629b7e24a73e9c branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ambv <lukasz(a)langa.pl> date: 2021-10-29T22:45:33+02:00 summary: bpo-45600: Enhanced / clarified the docs for os.environ and os.environb (GH-29204) (GH-29322) Co-authored-by: Łukasz Langa <lukasz(a)langa.pl> (cherry picked from commit b17cfd150f4dc2816975d304a71110a2d445eaf0) Co-authored-by: andrei kulakov <andrei.avk(a)gmail.com> files: M Doc/library/os.rst diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 237ca4b4b58ef..39d7e40dd915c 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -170,9 +170,10 @@ process and user. .. data:: environ - A :term:`mapping` object representing the string environment. For example, - ``environ['HOME']`` is the pathname of your home directory (on some platforms), - and is equivalent to ``getenv("HOME")`` in C. + A :term:`mapping` object where keys and values are strings that represent + the process environment. For example, ``environ['HOME']`` is the pathname + of your home directory (on some platforms), and is equivalent to + ``getenv("HOME")`` in C. This mapping is captured the first time the :mod:`os` module is imported, typically during Python startup as part of processing :file:`site.py`. Changes @@ -209,10 +210,10 @@ process and user. .. data:: environb - Bytes version of :data:`environ`: a :term:`mapping` object representing the - environment as byte strings. :data:`environ` and :data:`environb` are - synchronized (modify :data:`environb` updates :data:`environ`, and vice - versa). + Bytes version of :data:`environ`: a :term:`mapping` object where both keys + and values are :class:`bytes` objects representing the process environment. + :data:`environ` and :data:`environb` are synchronized (modifying + :data:`environb` updates :data:`environ`, and vice versa). :data:`environb` is only available if :data:`supports_bytes_environ` is ``True``.

1 0

bpo-45335: Add note to `sqlite3` docs about "timestamp" converter (GH-29200) (GH-29320)
by ambv Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/e1560313d4d9bff8eba0f851ef325f7ee1… commit: e1560313d4d9bff8eba0f851ef325f7ee19f7ba9 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ambv <lukasz(a)langa.pl> date: 2021-10-29T22:42:38+02:00 summary: bpo-45335: Add note to `sqlite3` docs about "timestamp" converter (GH-29200) (GH-29320) (cherry picked from commit 3877fc02f7a8801ba5ce0e94b6075b3fdd9778d0) Co-authored-by: Ian Fisher <ian(a)iafisher.com> files: M Doc/library/sqlite3.rst diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index b9436daaebb4b..b24a2f0985489 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1062,6 +1062,12 @@ If a timestamp stored in SQLite has a fractional part longer than 6 numbers, its value will be truncated to microsecond precision by the timestamp converter. +.. note:: + + The default "timestamp" converter ignores UTC offsets in the database and + always returns a naive :class:`datetime.datetime` object. To preserve UTC + offsets in timestamps, either leave converters disabled, or register an + offset-aware converter with :func:`register_converter`. .. _sqlite3-controlling-transactions:

1 0

bpo-45335: Add note to `sqlite3` docs about "timestamp" converter (GH-29200) (GH-29319)
by ambv Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/8ea665c730cd86a321c558c012bef84f45… commit: 8ea665c730cd86a321c558c012bef84f454efa4f branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ambv <lukasz(a)langa.pl> date: 2021-10-29T22:41:45+02:00 summary: bpo-45335: Add note to `sqlite3` docs about "timestamp" converter (GH-29200) (GH-29319) (cherry picked from commit 3877fc02f7a8801ba5ce0e94b6075b3fdd9778d0) Co-authored-by: Ian Fisher <ian(a)iafisher.com> files: M Doc/library/sqlite3.rst diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index dc9aeb2ba3f59..16d2c2e85f42b 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1072,6 +1072,12 @@ If a timestamp stored in SQLite has a fractional part longer than 6 numbers, its value will be truncated to microsecond precision by the timestamp converter. +.. note:: + + The default "timestamp" converter ignores UTC offsets in the database and + always returns a naive :class:`datetime.datetime` object. To preserve UTC + offsets in timestamps, either leave converters disabled, or register an + offset-aware converter with :func:`register_converter`. .. _sqlite3-controlling-transactions:

1 0

bpo-45581: Raise `MemoryError` in `sqlite3.connect` if SQLite signals memory error (GH-29171)
by ambv Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/e2e62b3808691e15fa44b883270023e42d… commit: e2e62b3808691e15fa44b883270023e42dcad958 branch: main author: Erlend Egeberg Aasland <erlend.aasland(a)innova.no> committer: ambv <lukasz(a)langa.pl> date: 2021-10-29T22:21:58+02:00 summary: bpo-45581: Raise `MemoryError` in `sqlite3.connect` if SQLite signals memory error (GH-29171) files: A Misc/NEWS.d/next/Library/2021-10-22-21-57-02.bpo-45581.rlH6ay.rst M Modules/_sqlite/connection.c diff --git a/Misc/NEWS.d/next/Library/2021-10-22-21-57-02.bpo-45581.rlH6ay.rst b/Misc/NEWS.d/next/Library/2021-10-22-21-57-02.bpo-45581.rlH6ay.rst new file mode 100644 index 0000000000000..13a3b237434eb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-10-22-21-57-02.bpo-45581.rlH6ay.rst @@ -0,0 +1,2 @@ +:meth:`sqlite3.connect` now correctly raises :exc:`MemoryError` if the +underlying SQLite API signals memory error. Patch by Erlend E. Aasland. diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index da2f12e8f99c2..94c38ad395440 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -165,6 +165,10 @@ pysqlite_connection_init_impl(pysqlite_Connection *self, (uri ? SQLITE_OPEN_URI : 0), NULL); Py_END_ALLOW_THREADS + if (self->db == NULL && rc == SQLITE_NOMEM) { + PyErr_NoMemory(); + return -1; + } if (rc != SQLITE_OK) { _pysqlite_seterror(state, self->db); return -1;

1 0

bpo-45600: Enhanced / clarified the docs for os.environ and os.environb (GH-29204)
by ambv Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/b17cfd150f4dc2816975d304a71110a2d4… commit: b17cfd150f4dc2816975d304a71110a2d445eaf0 branch: main author: andrei kulakov <andrei.avk(a)gmail.com> committer: ambv <lukasz(a)langa.pl> date: 2021-10-29T22:18:45+02:00 summary: bpo-45600: Enhanced / clarified the docs for os.environ and os.environb (GH-29204) Co-authored-by: Łukasz Langa <lukasz(a)langa.pl> files: M Doc/library/os.rst diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 3bfefc6bcd57d..8092397be650f 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -170,9 +170,10 @@ process and user. .. data:: environ - A :term:`mapping` object representing the string environment. For example, - ``environ['HOME']`` is the pathname of your home directory (on some platforms), - and is equivalent to ``getenv("HOME")`` in C. + A :term:`mapping` object where keys and values are strings that represent + the process environment. For example, ``environ['HOME']`` is the pathname + of your home directory (on some platforms), and is equivalent to + ``getenv("HOME")`` in C. This mapping is captured the first time the :mod:`os` module is imported, typically during Python startup as part of processing :file:`site.py`. Changes @@ -209,10 +210,10 @@ process and user. .. data:: environb - Bytes version of :data:`environ`: a :term:`mapping` object representing the - environment as byte strings. :data:`environ` and :data:`environb` are - synchronized (modify :data:`environb` updates :data:`environ`, and vice - versa). + Bytes version of :data:`environ`: a :term:`mapping` object where both keys + and values are :class:`bytes` objects representing the process environment. + :data:`environ` and :data:`environb` are synchronized (modifying + :data:`environb` updates :data:`environ`, and vice versa). :data:`environb` is only available if :data:`supports_bytes_environ` is ``True``.

1 0

bpo-45335: Add note to `sqlite3` docs about "timestamp" converter (GH-29200)
by ambv Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/3877fc02f7a8801ba5ce0e94b6075b3fdd… commit: 3877fc02f7a8801ba5ce0e94b6075b3fdd9778d0 branch: main author: Ian Fisher <ian(a)iafisher.com> committer: ambv <lukasz(a)langa.pl> date: 2021-10-29T22:17:22+02:00 summary: bpo-45335: Add note to `sqlite3` docs about "timestamp" converter (GH-29200) files: M Doc/library/sqlite3.rst diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index fe1b64ade9561..19a4155542382 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1095,6 +1095,12 @@ If a timestamp stored in SQLite has a fractional part longer than 6 numbers, its value will be truncated to microsecond precision by the timestamp converter. +.. note:: + + The default "timestamp" converter ignores UTC offsets in the database and + always returns a naive :class:`datetime.datetime` object. To preserve UTC + offsets in timestamps, either leave converters disabled, or register an + offset-aware converter with :func:`register_converter`. .. _sqlite3-controlling-transactions:

1 0

bpo-45562: Ensure all tokenizer debug messages are printed to stderr (GH-29270)
by miss-islington Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/d8ca47c943c4ad396a9c70aff35bbbf4b1… commit: d8ca47c943c4ad396a9c70aff35bbbf4b1868876 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2021-10-29T10:21:15-07:00 summary: bpo-45562: Ensure all tokenizer debug messages are printed to stderr (GH-29270) (cherry picked from commit cdc7a5827754bec83970bb052d410d55f85b3fff) Co-authored-by: Pablo Galindo Salgado <Pablogsal(a)gmail.com> files: M Parser/tokenizer.c diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 6563cdfd7cc60..519300f3731f0 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1047,7 +1047,7 @@ tok_nextc(struct tok_state *tok) #if defined(Py_DEBUG) if (Py_DebugFlag) { fprintf(stderr, "line[%d] = ", tok->lineno); - print_escape(stdout, tok->cur, tok->inp - tok->cur); + print_escape(stderr, tok->cur, tok->inp - tok->cur); fprintf(stderr, " tok->done = %d\n", tok->done); } #endif

1 0

bpo-45548: Some test modules must be built as shared libs (GH-29268)
by tiran Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/f0150ac94a85c863ec1dcb58b9e33ed7ce… commit: f0150ac94a85c863ec1dcb58b9e33ed7ce465ec8 branch: main author: Christian Heimes <christian(a)python.org> committer: tiran <christian(a)python.org> date: 2021-10-29T17:49:57+02:00 summary: bpo-45548: Some test modules must be built as shared libs (GH-29268) Some test cases don't work when test modules are static extensions. Add dependency on Modules/config.c to trigger a rebuild whenever a module build type is changed. ``makesetup`` puts shared extensions into ``Modules/`` directory. Create symlinks from pybuilddir so the extensions can be imported. Note: It is not possible to use the content of pybuilddir.txt as a build target. Makefile evaluates target variables in the first pass. The pybuilddir.txt file does not exist at that point. files: A Misc/NEWS.d/next/Build/2021-10-28-14-47-22.bpo-45548.mdCBxB.rst M Doc/whatsnew/3.11.rst M Makefile.pre.in M Modules/Setup M Modules/makesetup M setup.py diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 21ad4669476ef..156bfbd9109ee 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -507,6 +507,13 @@ Build Changes except empty tuple singleton. (Contributed by Christian Heimes in :issue:`45522`) +* ``Modules/Setup`` and ``Modules/makesetup`` have been improved and tied up. + Extension modules can now be built through ``makesetup``. All except some + test modules can be linked statically into main binary or library. + (Contributed by Brett Cannon and Christian Heimes in :issue:`45548`, + :issue:`45570`, :issue:`45571`, and :issue:`43974`.) + + C API Changes ============= diff --git a/Makefile.pre.in b/Makefile.pre.in index 60acc16beeee3..322800ae26e4c 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -705,8 +705,17 @@ $(DLLLIBRARY) libpython$(LDVERSION).dll.a: $(LIBRARY_OBJS) fi -oldsharedmods: $(SHAREDMODS) - +# create relative links from build/lib.platform/egg.so to Modules/egg.so +# pybuilddir.txt is created too late. We cannot use it in Makefile +# targets. ln --relative is not portable. +oldsharedmods: $(SHAREDMODS) pybuilddir.txt + @target=`cat pybuilddir.txt`; \ + $(MKDIR_P) $$target; \ + for mod in X $(SHAREDMODS); do \ + if test $$mod != X; then \ + $(LN) -sf ../../$$mod $$target/`basename $$mod`; \ + fi; \ + done Makefile Modules/config.c: Makefile.pre \ $(srcdir)/Modules/config.c.in \ diff --git a/Misc/NEWS.d/next/Build/2021-10-28-14-47-22.bpo-45548.mdCBxB.rst b/Misc/NEWS.d/next/Build/2021-10-28-14-47-22.bpo-45548.mdCBxB.rst new file mode 100644 index 0000000000000..e6ccd52274de5 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2021-10-28-14-47-22.bpo-45548.mdCBxB.rst @@ -0,0 +1,4 @@ +``Modules/Setup`` and ``Modules/makesetup`` have been improved. The +``Setup`` file now contains working rules for all extensions. Outdated +comments have been removed. Rules defined by ``makesetup`` track +dependencies correctly. diff --git a/Modules/Setup b/Modules/Setup index b4eae38fdff60..57584f38fd8e7 100644 --- a/Modules/Setup +++ b/Modules/Setup @@ -297,14 +297,16 @@ xxsubtype xxsubtype.c # Required for the test suite to pass! #_xxsubinterpreters _xxsubinterpretersmodule.c #_xxtestfuzz _xxtestfuzz/_xxtestfuzz.c _xxtestfuzz/fuzzer.c -#_ctypes_test _ctypes/_ctypes_test.c #_testbuffer _testbuffer.c -#_testimportmultiple _testimportmultiple.c #_testinternalcapi _testinternalcapi.c -#_testmultiphase _testmultiphase.c + +# Some testing modules MUST be built as shared libraries. #*shared* -#_testcapi _testcapimodule.c # CANNOT be statically compiled! +#_ctypes_test _ctypes/_ctypes_test.c +#_testcapi _testcapimodule.c +#_testimportmultiple _testimportmultiple.c +#_testmultiphase _testmultiphase.c # --- # Uncommenting the following line tells makesetup that all following modules diff --git a/Modules/makesetup b/Modules/makesetup index 7547315281643..543992c4a0294 100755 --- a/Modules/makesetup +++ b/Modules/makesetup @@ -241,7 +241,8 @@ sed -e 's/[ ]*#.*//' -e '/^[ ]*$/d' | cc="$cc \$(PY_BUILTIN_MODULE_CFLAGS)";; esac mods_upper=$(echo $mods | tr '[a-z]' '[A-Z]') - rule="$obj: $src \$(MODULE_${mods_upper}_DEPS) \$(PYTHON_HEADERS); $cc $cpps -c $src -o $obj" + # force rebuild when header file or module build flavor (static/shared) is changed + rule="$obj: $src \$(MODULE_${mods_upper}_DEPS) \$(PYTHON_HEADERS) Modules/config.c; $cc $cpps -c $src -o $obj" echo "$rule" >>$rulesf done case $doconfig in diff --git a/setup.py b/setup.py index f32dd4c6e5e14..6fa8a674eda31 100644 --- a/setup.py +++ b/setup.py @@ -426,12 +426,13 @@ def update_sources_depends(self): # re-compile extensions if a header file has been changed ext.depends.extend(headers) - def remove_configured_extensions(self): + def handle_configured_extensions(self): # The sysconfig variables built by makesetup that list the already # built modules and the disabled modules as configured by the Setup # files. - sysconf_built = sysconfig.get_config_var('MODBUILT_NAMES').split() - sysconf_dis = sysconfig.get_config_var('MODDISABLED_NAMES').split() + sysconf_built = set(sysconfig.get_config_var('MODBUILT_NAMES').split()) + sysconf_shared = set(sysconfig.get_config_var('MODSHARED_NAMES').split()) + sysconf_dis = set(sysconfig.get_config_var('MODDISABLED_NAMES').split()) mods_built = [] mods_disabled = [] @@ -449,11 +450,15 @@ def remove_configured_extensions(self): mods_configured] # Remove the shared libraries built by a previous build. for ext in mods_configured: + # Don't remove shared extensions which have been built + # by Modules/Setup + if ext.name in sysconf_shared: + continue fullpath = self.get_ext_fullpath(ext.name) - if os.path.exists(fullpath): + if os.path.lexists(fullpath): os.unlink(fullpath) - return (mods_built, mods_disabled) + return mods_built, mods_disabled def set_compiler_executables(self): # When you run "make CC=altcc" or something similar, you really want @@ -478,7 +483,7 @@ def build_extensions(self): self.remove_disabled() self.update_sources_depends() - mods_built, mods_disabled = self.remove_configured_extensions() + mods_built, mods_disabled = self.handle_configured_extensions() self.set_compiler_executables() if LIST_MODULE_NAMES:

1 0

[3.10] bpo-45628: Check all parts of the suffix for an extension match. (GH-29310) (GH-29314)
by vsajip Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/191a93905a84f272b2232701dc5dcc6998… commit: 191a93905a84f272b2232701dc5dcc69987330f5 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: vsajip <vinay_sajip(a)yahoo.co.uk> date: 2021-10-29T16:25:31+01:00 summary: [3.10] bpo-45628: Check all parts of the suffix for an extension match. (GH-29310) (GH-29314) files: M Lib/logging/handlers.py diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py index 4dcbe4530fcfc..4e8f0a8cc2f46 100644 --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -371,8 +371,13 @@ def getFilesToDelete(self): for fileName in fileNames: if fileName[:plen] == prefix: suffix = fileName[plen:] - if self.extMatch.match(suffix): - result.append(os.path.join(dirName, fileName)) + # See bpo-45628: The date/time suffix could be anywhere in the + # filename + parts = suffix.split('.') + for part in parts: + if self.extMatch.match(part): + result.append(os.path.join(dirName, fileName)) + break if len(result) < self.backupCount: result = [] else:

1 0

[3.9] bpo-45628: Check all parts of the suffix for an extension match. (GH-29310) (GH-29313)
by vsajip Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/317e0c99e3804310f4bee23e497d9d84b7… commit: 317e0c99e3804310f4bee23e497d9d84b717d7f7 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: vsajip <vinay_sajip(a)yahoo.co.uk> date: 2021-10-29T16:24:41+01:00 summary: [3.9] bpo-45628: Check all parts of the suffix for an extension match. (GH-29310) (GH-29313) files: M Lib/logging/handlers.py diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py index 7f1f10551c304..f8040c498c10d 100644 --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -368,8 +368,13 @@ def getFilesToDelete(self): for fileName in fileNames: if fileName[:plen] == prefix: suffix = fileName[plen:] - if self.extMatch.match(suffix): - result.append(os.path.join(dirName, fileName)) + # See bpo-45628: The date/time suffix could be anywhere in the + # filename + parts = suffix.split('.') + for part in parts: + if self.extMatch.match(part): + result.append(os.path.join(dirName, fileName)) + break if len(result) < self.backupCount: result = [] else:

1 0