Mailman 3 October 2022 - Python-checkins

gh-98576: Fix types in dataclass.InitVar example (gh-98577)
by miss-islington 31 Oct '22

31 Oct '22

https://github.com/python/cpython/commit/1907e5a7cca9b7348f76c3f1128058ddc4… commit: 1907e5a7cca9b7348f76c3f1128058ddc4623f22 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-10-31T08:08:54-07:00 summary: gh-98576: Fix types in dataclass.InitVar example (gh-98577) (cherry picked from commit 880bafc574bcd811dd7244f9a82056430b489996) Co-authored-by: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 02de64288d0d..f66010082ad1 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -561,8 +561,8 @@ value is not provided when creating the class:: @dataclass class C: i: int - j: int = None - database: InitVar[DatabaseType] = None + j: int | None = None + database: InitVar[DatabaseType | None] = None def __post_init__(self, database): if self.j is None and database is not None:

1 0

gh-98576: Fix types in dataclass.InitVar example (gh-98577)
by ericvsmith 31 Oct '22

31 Oct '22

https://github.com/python/cpython/commit/880bafc574bcd811dd7244f9a82056430b… commit: 880bafc574bcd811dd7244f9a82056430b489996 branch: main author: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> committer: ericvsmith <ericvsmith(a)users.noreply.github.com> date: 2022-10-31T11:02:02-04:00 summary: gh-98576: Fix types in dataclass.InitVar example (gh-98577) files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index c4895aa822bf..847299649d1e 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -578,8 +578,8 @@ value is not provided when creating the class:: @dataclass class C: i: int - j: int = None - database: InitVar[DatabaseType] = None + j: int | None = None + database: InitVar[DatabaseType | None] = None def __post_init__(self, database): if self.j is None and database is not None:

1 0

[3.11] gh-96151: Use a private name for passing builtins to dataclass. This now allows for a field named BUILTIN (gh-98143) (gh-98900)
by ericvsmith 31 Oct '22

31 Oct '22

https://github.com/python/cpython/commit/ca24e496bad2fcae0994b0de583f9cedc8… commit: ca24e496bad2fcae0994b0de583f9cedc85bbd80 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ericvsmith <ericvsmith(a)users.noreply.github.com> date: 2022-10-31T10:59:00-04:00 summary: [3.11] gh-96151: Use a private name for passing builtins to dataclass. This now allows for a field named BUILTIN (gh-98143) (gh-98900) gh-96151: Use a private name for passing builtins to dataclass. This now allows for a field named BUILTIN (gh-98143) (cherry picked from commit 29f98b46b77ee528477b9a7b335974b9682f7f14) Co-authored-by: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst M Lib/dataclasses.py M Lib/test/test_dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index a567a33d646f..37e4ff702d61 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -412,13 +412,11 @@ def wrapper(self): def _create_fn(name, args, body, *, globals=None, locals=None, return_type=MISSING): - # Note that we mutate locals when exec() is called. Caller - # beware! The only callers are internal to this module, so no + # Note that we may mutate locals. Callers beware! + # The only callers are internal to this module, so no # worries about external callers. if locals is None: locals = {} - if 'BUILTINS' not in locals: - locals['BUILTINS'] = builtins return_annotation = '' if return_type is not MISSING: locals['_return_type'] = return_type @@ -444,7 +442,7 @@ def _field_assign(frozen, name, value, self_name): # self_name is what "self" is called in this function: don't # hard-code "self", since that might be a field name. if frozen: - return f'BUILTINS.object.__setattr__({self_name},{name!r},{value})' + return f'__dataclass_builtins_object__.__setattr__({self_name},{name!r},{value})' return f'{self_name}.{name}={value}' @@ -551,6 +549,7 @@ def _init_fn(fields, std_fields, kw_only_fields, frozen, has_post_init, locals.update({ 'MISSING': MISSING, '_HAS_DEFAULT_FACTORY': _HAS_DEFAULT_FACTORY, + '__dataclass_builtins_object__': object, }) body_lines = [] diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index 63380ea0b680..0d809bd2eb9a 100644 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -231,6 +231,14 @@ class C: c = C('foo') self.assertEqual(c.object, 'foo') + def test_field_named_BUILTINS_frozen(self): + # gh-96151 + @dataclass(frozen=True) + class C: + BUILTINS: int + c = C(5) + self.assertEqual(c.BUILTINS, 5) + def test_field_named_like_builtin(self): # Attribute names can shadow built-in names # since code generation is used. diff --git a/Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst b/Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst new file mode 100644 index 000000000000..700c9748735f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst @@ -0,0 +1 @@ +Allow ``BUILTINS`` to be a valid field name for frozen dataclasses.

1 0

[3.10] gh-96151: Use a private name for passing builtins to dataclass. This now allows for a field named BUILTIN (gh-98143) (gh-98899)
by ericvsmith 31 Oct '22

31 Oct '22

https://github.com/python/cpython/commit/789d1322f626e9a91ca1c186ee163197d4… commit: 789d1322f626e9a91ca1c186ee163197d4ffb842 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ericvsmith <ericvsmith(a)users.noreply.github.com> date: 2022-10-31T10:59:18-04:00 summary: [3.10] gh-96151: Use a private name for passing builtins to dataclass. This now allows for a field named BUILTIN (gh-98143) (gh-98899) gh-96151: Use a private name for passing builtins to dataclass. This now allows for a field named BUILTIN (gh-98143) (cherry picked from commit 29f98b46b77ee528477b9a7b335974b9682f7f14) Co-authored-by: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst M Lib/dataclasses.py M Lib/test/test_dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index 105a95b95540..e1687a117d6f 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -411,13 +411,11 @@ def wrapper(self): def _create_fn(name, args, body, *, globals=None, locals=None, return_type=MISSING): - # Note that we mutate locals when exec() is called. Caller - # beware! The only callers are internal to this module, so no + # Note that we may mutate locals. Callers beware! + # The only callers are internal to this module, so no # worries about external callers. if locals is None: locals = {} - if 'BUILTINS' not in locals: - locals['BUILTINS'] = builtins return_annotation = '' if return_type is not MISSING: locals['_return_type'] = return_type @@ -443,7 +441,7 @@ def _field_assign(frozen, name, value, self_name): # self_name is what "self" is called in this function: don't # hard-code "self", since that might be a field name. if frozen: - return f'BUILTINS.object.__setattr__({self_name},{name!r},{value})' + return f'__dataclass_builtins_object__.__setattr__({self_name},{name!r},{value})' return f'{self_name}.{name}={value}' @@ -550,6 +548,7 @@ def _init_fn(fields, std_fields, kw_only_fields, frozen, has_post_init, locals.update({ 'MISSING': MISSING, '_HAS_DEFAULT_FACTORY': _HAS_DEFAULT_FACTORY, + '__dataclass_builtins_object__': object, }) body_lines = [] diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index f72e81c31e61..e805f0cf4cc7 100644 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -230,6 +230,14 @@ class C: c = C('foo') self.assertEqual(c.object, 'foo') + def test_field_named_BUILTINS_frozen(self): + # gh-96151 + @dataclass(frozen=True) + class C: + BUILTINS: int + c = C(5) + self.assertEqual(c.BUILTINS, 5) + def test_field_named_like_builtin(self): # Attribute names can shadow built-in names # since code generation is used. diff --git a/Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst b/Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst new file mode 100644 index 000000000000..700c9748735f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst @@ -0,0 +1 @@ +Allow ``BUILTINS`` to be a valid field name for frozen dataclasses.

1 0

gh-98410: move getbufferproc and releasebufferproc to buffer.h (#31158)
by encukou 31 Oct '22

31 Oct '22

https://github.com/python/cpython/commit/e98923c0be644c474ce0e318d40e0595e4… commit: e98923c0be644c474ce0e318d40e0595e4484106 branch: main author: David Hewitt <1939362+davidhewitt(a)users.noreply.github.com> committer: encukou <encukou(a)gmail.com> date: 2022-10-31T15:01:32+01:00 summary: gh-98410: move getbufferproc and releasebufferproc to buffer.h (#31158) This adds them to the Limited API. files: A Misc/NEWS.d/next/C API/2022-10-25-17-50-43.gh-issue-98410.NSXYfm.rst M Doc/data/stable_abi.dat M Include/cpython/object.h M Include/pybuffer.h M Misc/stable_abi.toml diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index 133658491c5a..db8fc15d93d1 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -865,6 +865,7 @@ type,descrsetfunc,3.2,, type,destructor,3.2,, type,getattrfunc,3.2,, type,getattrofunc,3.2,, +type,getbufferproc,3.12,, type,getiterfunc,3.2,, type,getter,3.2,, type,hashfunc,3.2,, @@ -875,6 +876,7 @@ type,lenfunc,3.2,, type,newfunc,3.2,, type,objobjargproc,3.2,, type,objobjproc,3.2,, +type,releasebufferproc,3.12,, type,reprfunc,3.2,, type,richcmpfunc,3.2,, type,setattrfunc,3.2,, diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 900b52321dff..fa0cfb244849 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -51,10 +51,6 @@ typedef struct _Py_Identifier { #endif /* NEEDS_PY_IDENTIFIER */ -typedef int (*getbufferproc)(PyObject *, Py_buffer *, int); -typedef void (*releasebufferproc)(PyObject *, Py_buffer *); - - typedef struct { /* Number implementations must check *both* arguments for proper type and implement the necessary conversions diff --git a/Include/pybuffer.h b/Include/pybuffer.h index 6893505e66e3..bbac60972f51 100644 --- a/Include/pybuffer.h +++ b/Include/pybuffer.h @@ -32,6 +32,9 @@ typedef struct { void *internal; } Py_buffer; +typedef int (*getbufferproc)(PyObject *, Py_buffer *, int); +typedef void (*releasebufferproc)(PyObject *, Py_buffer *); + /* Return 1 if the getbuffer function is available, otherwise return 0. */ PyAPI_FUNC(int) PyObject_CheckBuffer(PyObject *obj); diff --git a/Misc/NEWS.d/next/C API/2022-10-25-17-50-43.gh-issue-98410.NSXYfm.rst b/Misc/NEWS.d/next/C API/2022-10-25-17-50-43.gh-issue-98410.NSXYfm.rst new file mode 100644 index 000000000000..d98bc4e81031 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-10-25-17-50-43.gh-issue-98410.NSXYfm.rst @@ -0,0 +1 @@ +Add ``getbufferproc`` and ``releasebufferproc`` to the stable API. diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml index e18a6e8f6a9c..0ba0f51b2de4 100644 --- a/Misc/stable_abi.toml +++ b/Misc/stable_abi.toml @@ -2299,3 +2299,7 @@ added = '3.12' [macro.PY_VECTORCALL_ARGUMENTS_OFFSET] added = '3.12' +[typedef.getbufferproc] + added = '3.12' +[typedef.releasebufferproc] + added = '3.12'

1 0

gh-98878: Use builtins from the bound frame when offering a suggestion (#98880)
by pablogsal 31 Oct '22

31 Oct '22

https://github.com/python/cpython/commit/a41de32942b7ac14524587a08d219f3bfe… commit: a41de32942b7ac14524587a08d219f3bfe346415 branch: main author: Batuhan Taskaya <isidentical(a)gmail.com> committer: pablogsal <Pablogsal(a)gmail.com> date: 2022-10-31T13:27:13Z summary: gh-98878: Use builtins from the bound frame when offering a suggestion (#98880) files: A Misc/NEWS.d/next/Library/2022-10-30-22-42-48.gh-issue-98878.fgrykp.rst M Lib/test/test_traceback.py M Lib/traceback.py diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index 56b168735d15..149d0234fe8a 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -7,6 +7,7 @@ import types import inspect import importlib +import builtins import unittest import re import tempfile @@ -3209,6 +3210,14 @@ def func(): actual = self.get_suggestion(func) self.assertIn("'ZeroDivisionError'?", actual) + def test_name_error_suggestions_from_builtins_when_builtins_is_module(self): + def func(): + custom_globals = globals().copy() + custom_globals["__builtins__"] = builtins + print(eval("ZeroDivisionErrrrr", custom_globals)) + actual = self.get_suggestion(func) + self.assertIn("'ZeroDivisionError'?", actual) + def test_name_error_suggestions_do_not_trigger_for_long_names(self): def func(): somethingverywronghehehehehehe = None diff --git a/Lib/traceback.py b/Lib/traceback.py index 0f0f2b317de2..cf5f355ff04c 100644 --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -1035,7 +1035,7 @@ def _compute_suggestion_error(exc_value, tb, wrong_name): d = ( list(frame.f_locals) + list(frame.f_globals) - + list(frame.f_globals['__builtins__']) + + list(frame.f_builtins) ) if len(d) > _MAX_CANDIDATE_ITEMS: return None diff --git a/Misc/NEWS.d/next/Library/2022-10-30-22-42-48.gh-issue-98878.fgrykp.rst b/Misc/NEWS.d/next/Library/2022-10-30-22-42-48.gh-issue-98878.fgrykp.rst new file mode 100644 index 000000000000..e83422a77a22 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-10-30-22-42-48.gh-issue-98878.fgrykp.rst @@ -0,0 +1,2 @@ +Use the frame bound builtins when offering a name suggestion in +:mod:`traceback` to prevent crashing when ``__builtins__`` is not a dict.

1 0

gh-98811: use full source location to simplify __future__ imports error checking. This also fixes an incorrect error offset. (GH-98812)
by iritkatriel 31 Oct '22

31 Oct '22

https://github.com/python/cpython/commit/39448adc9db0c0444443a42f13a9d7e144… commit: 39448adc9db0c0444443a42f13a9d7e1441b88e8 branch: main author: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2022-10-31T13:08:03Z summary: gh-98811: use full source location to simplify __future__ imports error checking. This also fixes an incorrect error offset. (GH-98812) files: A Misc/NEWS.d/next/Core and Builtins/2022-10-28-13-59-51.gh-issue-98811.XQypJa.rst M Include/cpython/compile.h M Lib/test/test_future.py M Python/compile.c M Python/future.c diff --git a/Include/cpython/compile.h b/Include/cpython/compile.h index 518a37649929..f5a62a8ec6dd 100644 --- a/Include/cpython/compile.h +++ b/Include/cpython/compile.h @@ -31,11 +31,26 @@ typedef struct { #define _PyCompilerFlags_INIT \ (PyCompilerFlags){.cf_flags = 0, .cf_feature_version = PY_MINOR_VERSION} +/* source location information */ +typedef struct { + int lineno; + int end_lineno; + int col_offset; + int end_col_offset; +} _PyCompilerSrcLocation; + +#define SRC_LOCATION_FROM_AST(n) \ + (_PyCompilerSrcLocation){ \ + .lineno = (n)->lineno, \ + .end_lineno = (n)->end_lineno, \ + .col_offset = (n)->col_offset, \ + .end_col_offset = (n)->end_col_offset } + /* Future feature support */ typedef struct { - int ff_features; /* flags set by future statements */ - int ff_lineno; /* line number of last future statement */ + int ff_features; /* flags set by future statements */ + _PyCompilerSrcLocation ff_location; /* location of last future statement */ } PyFutureFeatures; #define FUTURE_NESTED_SCOPES "nested_scopes" diff --git a/Lib/test/test_future.py b/Lib/test/test_future.py index 189cbdc4365b..b8b591a1bcf2 100644 --- a/Lib/test/test_future.py +++ b/Lib/test/test_future.py @@ -60,7 +60,7 @@ def test_badfuture6(self): def test_badfuture7(self): with self.assertRaises(SyntaxError) as cm: from test import badsyntax_future7 - self.check_syntax_error(cm.exception, "badsyntax_future7", 3, 53) + self.check_syntax_error(cm.exception, "badsyntax_future7", 3, 54) def test_badfuture8(self): with self.assertRaises(SyntaxError) as cm: diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-10-28-13-59-51.gh-issue-98811.XQypJa.rst b/Misc/NEWS.d/next/Core and Builtins/2022-10-28-13-59-51.gh-issue-98811.XQypJa.rst new file mode 100644 index 000000000000..ce90a516cf0d --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-10-28-13-59-51.gh-issue-98811.XQypJa.rst @@ -0,0 +1,4 @@ +Use complete source locations to simplify detection of ``__future__`` +imports which are not at the beginning of the file. Also corrects the offset +in the exception raised in one case, which was off by one and impeded +highlighting. diff --git a/Python/compile.c b/Python/compile.c index 779729119e2e..f8924789f4e9 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -125,18 +125,23 @@ (c->c_flags->cf_flags & PyCF_ALLOW_TOP_LEVEL_AWAIT) \ && (c->u->u_ste->ste_type == ModuleBlock)) -typedef struct location_ { - int lineno; - int end_lineno; - int col_offset; - int end_col_offset; -} location; +typedef _PyCompilerSrcLocation location; #define LOCATION(LNO, END_LNO, COL, END_COL) \ ((const location){(LNO), (END_LNO), (COL), (END_COL)}) static location NO_LOCATION = {-1, -1, -1, -1}; +/* Return true if loc1 starts after loc2 ends. */ +static inline bool +location_is_after(location loc1, location loc2) { + return (loc1.lineno > loc2.end_lineno) || + ((loc1.lineno == loc2.end_lineno) && + (loc1.col_offset > loc2.end_col_offset)); +} + +#define LOC(x) SRC_LOCATION_FROM_AST(x) + typedef struct jump_target_label_ { int id; } jump_target_label; @@ -1012,11 +1017,6 @@ basicblock_next_instr(basicblock *b) // Artificial instructions #define UNSET_LOC(c) -#define LOC(x) LOCATION((x)->lineno, \ - (x)->end_lineno, \ - (x)->col_offset, \ - (x)->end_col_offset) - /* Return the stack effect of opcode with argument oparg. @@ -3911,59 +3911,61 @@ compiler_import(struct compiler *c, stmt_ty s) static int compiler_from_import(struct compiler *c, stmt_ty s) { - location loc = LOC(s); - Py_ssize_t i, n = asdl_seq_LEN(s->v.ImportFrom.names); - PyObject *names; + Py_ssize_t n = asdl_seq_LEN(s->v.ImportFrom.names); - ADDOP_LOAD_CONST_NEW(c, loc, PyLong_FromLong(s->v.ImportFrom.level)); + ADDOP_LOAD_CONST_NEW(c, LOC(s), PyLong_FromLong(s->v.ImportFrom.level)); - names = PyTuple_New(n); - if (!names) + PyObject *names = PyTuple_New(n); + if (!names) { return 0; + } /* build up the names */ - for (i = 0; i < n; i++) { + for (Py_ssize_t i = 0; i < n; i++) { alias_ty alias = (alias_ty)asdl_seq_GET(s->v.ImportFrom.names, i); Py_INCREF(alias->name); PyTuple_SET_ITEM(names, i, alias->name); } - if (s->lineno > c->c_future->ff_lineno && s->v.ImportFrom.module && - _PyUnicode_EqualToASCIIString(s->v.ImportFrom.module, "__future__")) { + if (location_is_after(LOC(s), c->c_future->ff_location) && + s->v.ImportFrom.module && + _PyUnicode_EqualToASCIIString(s->v.ImportFrom.module, "__future__")) + { Py_DECREF(names); - return compiler_error(c, loc, "from __future__ imports must occur " + return compiler_error(c, LOC(s), "from __future__ imports must occur " "at the beginning of the file"); } - ADDOP_LOAD_CONST_NEW(c, loc, names); + ADDOP_LOAD_CONST_NEW(c, LOC(s), names); if (s->v.ImportFrom.module) { - ADDOP_NAME(c, loc, IMPORT_NAME, s->v.ImportFrom.module, names); + ADDOP_NAME(c, LOC(s), IMPORT_NAME, s->v.ImportFrom.module, names); } else { _Py_DECLARE_STR(empty, ""); - ADDOP_NAME(c, loc, IMPORT_NAME, &_Py_STR(empty), names); + ADDOP_NAME(c, LOC(s), IMPORT_NAME, &_Py_STR(empty), names); } - for (i = 0; i < n; i++) { + for (Py_ssize_t i = 0; i < n; i++) { alias_ty alias = (alias_ty)asdl_seq_GET(s->v.ImportFrom.names, i); identifier store_name; if (i == 0 && PyUnicode_READ_CHAR(alias->name, 0) == '*') { assert(n == 1); - ADDOP(c, loc, IMPORT_STAR); + ADDOP(c, LOC(s), IMPORT_STAR); return 1; } - ADDOP_NAME(c, loc, IMPORT_FROM, alias->name, names); + ADDOP_NAME(c, LOC(s), IMPORT_FROM, alias->name, names); store_name = alias->name; - if (alias->asname) + if (alias->asname) { store_name = alias->asname; + } - if (!compiler_nameop(c, loc, store_name, Store)) { + if (!compiler_nameop(c, LOC(s), store_name, Store)) { return 0; } } /* remove imported module */ - ADDOP(c, loc, POP_TOP); + ADDOP(c, LOC(s), POP_TOP); return 1; } diff --git a/Python/future.c b/Python/future.c index d465608ca454..2a45d2ebeab9 100644 --- a/Python/future.c +++ b/Python/future.c @@ -2,8 +2,6 @@ #include "pycore_ast.h" // _PyAST_GetDocString() #define UNDEFINED_FUTURE_FEATURE "future feature %.100s is not defined" -#define ERR_LATE_FUTURE \ -"from __future__ imports must occur at the beginning of the file" static int future_check_features(PyFutureFeatures *ff, stmt_ty s, PyObject *filename) @@ -56,59 +54,42 @@ future_check_features(PyFutureFeatures *ff, stmt_ty s, PyObject *filename) static int future_parse(PyFutureFeatures *ff, mod_ty mod, PyObject *filename) { - int i, done = 0, prev_line = 0; - - if (!(mod->kind == Module_kind || mod->kind == Interactive_kind)) + if (!(mod->kind == Module_kind || mod->kind == Interactive_kind)) { return 1; + } - if (asdl_seq_LEN(mod->v.Module.body) == 0) + Py_ssize_t n = asdl_seq_LEN(mod->v.Module.body); + if (n == 0) { return 1; + } - /* A subsequent pass will detect future imports that don't - appear at the beginning of the file. There's one case, - however, that is easier to handle here: A series of imports - joined by semi-colons, where the first import is a future - statement but some subsequent import has the future form - but is preceded by a regular import. - */ - - i = 0; - if (_PyAST_GetDocString(mod->v.Module.body) != NULL) + Py_ssize_t i = 0; + if (_PyAST_GetDocString(mod->v.Module.body) != NULL) { i++; + } - for (; i < asdl_seq_LEN(mod->v.Module.body); i++) { + for (; i < n; i++) { stmt_ty s = (stmt_ty)asdl_seq_GET(mod->v.Module.body, i); - if (done && s->lineno > prev_line) - return 1; - prev_line = s->lineno; - - /* The tests below will return from this function unless it is - still possible to find a future statement. The only things - that can precede a future statement are another future - statement and a doc string. - */ + /* The only things that can precede a future statement + * are another future statement and a doc string. + */ if (s->kind == ImportFrom_kind) { identifier modname = s->v.ImportFrom.module; if (modname && _PyUnicode_EqualToASCIIString(modname, "__future__")) { - if (done) { - PyErr_SetString(PyExc_SyntaxError, - ERR_LATE_FUTURE); - PyErr_SyntaxLocationObject(filename, s->lineno, s->col_offset); + if (!future_check_features(ff, s, filename)) { return 0; } - if (!future_check_features(ff, s, filename)) - return 0; - ff->ff_lineno = s->lineno; + ff->ff_location = SRC_LOCATION_FROM_AST(s); } else { - done = 1; + return 1; } } else { - done = 1; + return 1; } } return 1; @@ -126,7 +107,7 @@ _PyFuture_FromAST(mod_ty mod, PyObject *filename) return NULL; } ff->ff_features = 0; - ff->ff_lineno = -1; + ff->ff_location = (_PyCompilerSrcLocation){-1, -1, -1, -1}; if (!future_parse(ff, mod, filename)) { PyObject_Free(ff);

1 0

gh-96151: Use a private name for passing builtins to dataclass. This now allows for a field named BUILTIN (gh-98143)
by ericvsmith 31 Oct '22

31 Oct '22

https://github.com/python/cpython/commit/29f98b46b77ee528477b9a7b335974b968… commit: 29f98b46b77ee528477b9a7b335974b9682f7f14 branch: main author: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> committer: ericvsmith <ericvsmith(a)users.noreply.github.com> date: 2022-10-31T08:31:01-04:00 summary: gh-96151: Use a private name for passing builtins to dataclass. This now allows for a field named BUILTIN (gh-98143) files: A Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst M Lib/dataclasses.py M Lib/test/test_dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index bf7f290af162..b54e16984cb1 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -426,13 +426,11 @@ def wrapper(self): def _create_fn(name, args, body, *, globals=None, locals=None, return_type=MISSING): - # Note that we mutate locals when exec() is called. Caller - # beware! The only callers are internal to this module, so no + # Note that we may mutate locals. Callers beware! + # The only callers are internal to this module, so no # worries about external callers. if locals is None: locals = {} - if 'BUILTINS' not in locals: - locals['BUILTINS'] = builtins return_annotation = '' if return_type is not MISSING: locals['_return_type'] = return_type @@ -462,7 +460,7 @@ def _field_assign(frozen, name, value, self_name): # self_name is what "self" is called in this function: don't # hard-code "self", since that might be a field name. if frozen: - return f'BUILTINS.object.__setattr__({self_name},{name!r},{value})' + return f'__dataclass_builtins_object__.__setattr__({self_name},{name!r},{value})' return f'{self_name}.{name}={value}' @@ -569,6 +567,7 @@ def _init_fn(fields, std_fields, kw_only_fields, frozen, has_post_init, locals.update({ 'MISSING': MISSING, '_HAS_DEFAULT_FACTORY': _HAS_DEFAULT_FACTORY, + '__dataclass_builtins_object__': object, }) body_lines = [] diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index 637c456dd49e..01e66840085b 100644 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -257,6 +257,14 @@ class C: c = C('foo') self.assertEqual(c.object, 'foo') + def test_field_named_BUILTINS_frozen(self): + # gh-96151 + @dataclass(frozen=True) + class C: + BUILTINS: int + c = C(5) + self.assertEqual(c.BUILTINS, 5) + def test_field_named_like_builtin(self): # Attribute names can shadow built-in names # since code generation is used. diff --git a/Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst b/Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst new file mode 100644 index 000000000000..700c9748735f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-10-10-07-07-31.gh-issue-96151.K9fwoq.rst @@ -0,0 +1 @@ +Allow ``BUILTINS`` to be a valid field name for frozen dataclasses.

1 0

[3.11] gh-96853: Restore test coverage for Py_Initialize(Ex) (GH-98874)
by ncoghlan 30 Oct '22

30 Oct '22

https://github.com/python/cpython/commit/57dd11038f22379770d9f16e527f787fde… commit: 57dd11038f22379770d9f16e527f787fdea978fd branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ncoghlan <ncoghlan(a)gmail.com> date: 2022-10-31T09:15:12+10:00 summary: [3.11] gh-96853: Restore test coverage for Py_Initialize(Ex) (GH-98874) * As most of `test_embed` now uses `Py_InitializeFromConfig`, add a specific test case to cover `Py_Initialize` (and `Py_InitializeEx`) * Rename `_testembed` init helper to clarify the API used * Add a `PyConfig_Clear` call in `Py_InitializeEx` to make the code more obviously correct (it already didn't leak as none of the dynamically allocated config fields were being populated, but it's clearer if the wrappers follow the documented API usage guidelines) (cherry picked from commit 05e48865be69e1e5824f6915b588ff054717bb42) Co-authored-by: Nick Coghlan <ncoghlan(a)gmail.com> files: A Misc/NEWS.d/next/C API/2022-10-16-15-00-25.gh-issue-96853.V0wiXP.rst A Misc/NEWS.d/next/Tests/2022-10-12-14-57-06.gh-issue-96853.ANe-bw.rst M Lib/test/test_embed.py M Programs/_testembed.c M Python/pylifecycle.c diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index b8e3c37bbda3..f4b371d917c8 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -341,6 +341,12 @@ def test_finalize_structseq(self): out, err = self.run_embedded_interpreter("test_repeated_init_exec", code) self.assertEqual(out, 'Tests passed\n' * INIT_LOOPS) + def test_simple_initialization_api(self): + # _testembed now uses Py_InitializeFromConfig by default + # This case specifically checks Py_Initialize(Ex) still works + out, err = self.run_embedded_interpreter("test_repeated_simple_init") + self.assertEqual(out, 'Finalized\n' * INIT_LOOPS) + def test_quickened_static_code_gets_unquickened_at_Py_FINALIZE(self): # https://github.com/python/cpython/issues/92031 diff --git a/Misc/NEWS.d/next/C API/2022-10-16-15-00-25.gh-issue-96853.V0wiXP.rst b/Misc/NEWS.d/next/C API/2022-10-16-15-00-25.gh-issue-96853.V0wiXP.rst new file mode 100644 index 000000000000..d7e3cc423ac8 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-10-16-15-00-25.gh-issue-96853.V0wiXP.rst @@ -0,0 +1,4 @@ +``Py_InitializeEx`` now correctly calls ``PyConfig_Clear`` after initializing +the interpreter (the omission didn't cause a memory leak only because none +of the dynamically allocated config fields are populated by the wrapper +function) diff --git a/Misc/NEWS.d/next/Tests/2022-10-12-14-57-06.gh-issue-96853.ANe-bw.rst b/Misc/NEWS.d/next/Tests/2022-10-12-14-57-06.gh-issue-96853.ANe-bw.rst new file mode 100644 index 000000000000..89958c577219 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-10-12-14-57-06.gh-issue-96853.ANe-bw.rst @@ -0,0 +1,3 @@ +Added explicit coverage of ``Py_Initialize`` (and hence ``Py_InitializeEx``) +back to the embedding tests (all other embedding tests migrated to +``Py_InitializeFromConfig`` in Python 3.11) diff --git a/Programs/_testembed.c b/Programs/_testembed.c index e5b138ce84bb..13eae178f91d 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -22,7 +22,7 @@ char **main_argv; /********************************************************* * Embedded interpreter tests that need a custom exe * - * Executed via 'EmbeddingTests' in Lib/test/test_capi.py + * Executed via Lib/test/test_embed.py *********************************************************/ // Use to display the usage @@ -73,7 +73,7 @@ static void init_from_config_clear(PyConfig *config) } -static void _testembed_Py_Initialize(void) +static void _testembed_Py_InitializeFromConfig(void) { PyConfig config; _PyConfig_InitCompatConfig(&config); @@ -81,6 +81,12 @@ static void _testembed_Py_Initialize(void) init_from_config_clear(&config); } +static void _testembed_Py_Initialize(void) +{ + Py_SetProgramName(PROGRAM_NAME); + Py_Initialize(); +} + /***************************************************** * Test repeated initialisation and subinterpreters @@ -110,7 +116,7 @@ static int test_repeated_init_and_subinterpreters(void) for (int i=1; i <= INIT_LOOPS; i++) { printf("--- Pass %d ---\n", i); - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); mainstate = PyThreadState_Get(); PyEval_ReleaseThread(mainstate); @@ -168,7 +174,7 @@ static int test_repeated_init_exec(void) fprintf(stderr, "--- Loop #%d ---\n", i); fflush(stderr); - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); int err = PyRun_SimpleString(code); Py_Finalize(); if (err) { @@ -178,6 +184,23 @@ static int test_repeated_init_exec(void) return 0; } +/**************************************************************************** + * Test the Py_Initialize(Ex) convenience/compatibility wrappers + ***************************************************************************/ +// This is here to help ensure there are no wrapper resource leaks (gh-96853) +static int test_repeated_simple_init(void) +{ + for (int i=1; i <= INIT_LOOPS; i++) { + fprintf(stderr, "--- Loop #%d ---\n", i); + fflush(stderr); + + _testembed_Py_Initialize(); + Py_Finalize(); + printf("Finalized\n"); // Give test_embed some output to check + } + return 0; +} + /***************************************************** * Test forcing a particular IO encoding @@ -199,7 +222,7 @@ static void check_stdio_details(const char *encoding, const char * errors) fflush(stdout); /* Force the given IO encoding */ Py_SetStandardStreamEncoding(encoding, errors); - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); PyRun_SimpleString( "import sys;" "print('stdin: {0.encoding}:{0.errors}'.format(sys.stdin));" @@ -308,7 +331,7 @@ static int test_pre_initialization_sys_options(void) dynamic_xoption = NULL; _Py_EMBED_PREINIT_CHECK("Initializing interpreter\n"); - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); _Py_EMBED_PREINIT_CHECK("Check sys module contents\n"); PyRun_SimpleString("import sys; " "print('sys.warnoptions:', sys.warnoptions); " @@ -352,7 +375,7 @@ static int test_bpo20891(void) return 1; } - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); unsigned long thrd = PyThread_start_new_thread(bpo20891_thread, &lock); if (thrd == PYTHREAD_INVALID_THREAD_ID) { @@ -375,7 +398,7 @@ static int test_bpo20891(void) static int test_initialize_twice(void) { - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); /* bpo-33932: Calling Py_Initialize() twice should do nothing * (and not crash!). */ @@ -393,7 +416,7 @@ static int test_initialize_pymain(void) L"print(f'Py_Main() after Py_Initialize: " L"sys.argv={sys.argv}')"), L"arg2"}; - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); /* bpo-34008: Calling Py_Main() after Py_Initialize() must not crash */ Py_Main(Py_ARRAY_LENGTH(argv), argv); @@ -416,7 +439,7 @@ dump_config(void) static int test_init_initialize_config(void) { - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); dump_config(); Py_Finalize(); return 0; @@ -765,7 +788,7 @@ static int test_init_compat_env(void) /* Test initialization from environment variables */ Py_IgnoreEnvironmentFlag = 0; set_all_env_vars(); - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); dump_config(); Py_Finalize(); return 0; @@ -801,7 +824,7 @@ static int test_init_env_dev_mode(void) /* Test initialization from environment variables */ Py_IgnoreEnvironmentFlag = 0; set_all_env_vars_dev_mode(); - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); dump_config(); Py_Finalize(); return 0; @@ -814,7 +837,7 @@ static int test_init_env_dev_mode_alloc(void) Py_IgnoreEnvironmentFlag = 0; set_all_env_vars_dev_mode(); putenv("PYTHONMALLOC=malloc"); - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); dump_config(); Py_Finalize(); return 0; @@ -1154,7 +1177,7 @@ static int test_open_code_hook(void) } Py_IgnoreEnvironmentFlag = 0; - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); result = 0; PyObject *r = PyFile_OpenCode("$$test-filename"); @@ -1218,7 +1241,7 @@ static int _test_audit(Py_ssize_t setValue) Py_IgnoreEnvironmentFlag = 0; PySys_AddAuditHook(_audit_hook, &sawSet); - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); if (PySys_Audit("_testembed.raise", NULL) == 0) { printf("No error raised"); @@ -1274,7 +1297,7 @@ static int test_audit_subinterpreter(void) { Py_IgnoreEnvironmentFlag = 0; PySys_AddAuditHook(_audit_subinterpreter_hook, NULL); - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); Py_NewInterpreter(); Py_NewInterpreter(); @@ -1869,13 +1892,13 @@ static int test_unicode_id_init(void) _Py_IDENTIFIER(test_unicode_id_init); // Initialize Python once without using the identifier - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); Py_Finalize(); // Now initialize Python multiple times and use the identifier. // The first _PyUnicode_FromId() call initializes the identifier index. for (int i=0; i<3; i++) { - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); PyObject *str1, *str2; @@ -2007,7 +2030,7 @@ unwrap_allocator(PyMemAllocatorEx *allocator) static int test_get_incomplete_frame(void) { - _testembed_Py_Initialize(); + _testembed_Py_InitializeFromConfig(); PyMemAllocatorEx allocator; wrap_allocator(&allocator); // Force an allocation with an incomplete (generator) frame: @@ -2039,6 +2062,7 @@ struct TestCase static struct TestCase TestCases[] = { // Python initialization {"test_repeated_init_exec", test_repeated_init_exec}, + {"test_repeated_simple_init", test_repeated_simple_init}, {"test_forced_io_encoding", test_forced_io_encoding}, {"test_repeated_init_and_subinterpreters", test_repeated_init_and_subinterpreters}, {"test_repeated_init_and_inittab", test_repeated_init_and_inittab}, diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 960a38aebef8..0363e2e1dcae 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1289,6 +1289,7 @@ Py_InitializeEx(int install_sigs) config.install_signal_handlers = install_sigs; status = Py_InitializeFromConfig(&config); + PyConfig_Clear(&config); if (_PyStatus_EXCEPTION(status)) { Py_ExitStatusException(status); }

1 0

gh-98879: Remove unreachable error case from COMPARE_OP_STR_JUMP (GH-98882)
by sweeneyde 30 Oct '22

30 Oct '22

https://github.com/python/cpython/commit/87b5fd9d9de11c68b9940d3a1dba902138… commit: 87b5fd9d9de11c68b9940d3a1dba902138129109 branch: main author: Dennis Sweeney <36520290+sweeneyde(a)users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde(a)users.noreply.github.com> date: 2022-10-30T19:07:11-04:00 summary: gh-98879: Remove unreachable error case from COMPARE_OP_STR_JUMP (GH-98882) Thanks to PEP 623 changes, the comparison cannot fail. files: M Python/ceval.c diff --git a/Python/ceval.c b/Python/ceval.c index 24af419d29c5..f2250ffbc71f 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -3420,9 +3420,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int DEOPT_IF(!PyUnicode_CheckExact(right), COMPARE_OP); STAT_INC(COMPARE_OP, hit); int res = _PyUnicode_Equal(left, right); - if (res < 0) { - goto error; - } assert(oparg == Py_EQ || oparg == Py_NE); JUMPBY(INLINE_CACHE_ENTRIES_COMPARE_OP); NEXTOPARG();

1 0