Mailman 3 May 2021 - Python-checkins

Noted my recent contributions in "What's New In Python 3.10". (#25771)
by larryhastings May 1, 2021

May 1, 2021

https://github.com/python/cpython/commit/99f71aea5c5ddfdd13dc99e4253a7e3a6d… commit: 99f71aea5c5ddfdd13dc99e4253a7e3a6d10aead branch: master author: larryhastings <larry(a)hastings.org> committer: larryhastings <larry(a)hastings.org> date: 2021-04-30T22:55:21-07:00 summary: Noted my recent contributions in "What's New In Python 3.10". (#25771) Noted my recent contributions in "What's New In Python 3.10". Also made some edits clarifying "annotations" vs "type hints", and some other edits for correctness. files: M Doc/whatsnew/3.10.rst diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 9c8e296993879..797e1e3b21bf2 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -659,10 +659,10 @@ are added to enable the warning. See :ref:`io-text-encoding` for more information. -New Features Related to Type Annotations -======================================== +New Features Related to Type Hints +================================== -This section covers major changes affecting :pep:`484` type annotations and +This section covers major changes affecting :pep:`484` type hints and the :mod:`typing` module. @@ -671,7 +671,7 @@ PEP 604: New Type Union Operator A new type union operator was introduced which enables the syntax ``X | Y``. This provides a cleaner way of expressing 'either type X or type Y' instead of -using :data:`typing.Union`, especially in type hints (annotations). +using :data:`typing.Union`, especially in type hints. In previous versions of Python, to apply a type hint for functions accepting arguments of multiple types, :data:`typing.Union` was used:: @@ -722,8 +722,8 @@ See :class:`typing.Callable`, :class:`typing.ParamSpec`, Zijlstra in :issue:`43783`. PEP written by Mark Mendoza.) -PEP 613: TypeAlias Annotation ------------------------------ +PEP 613: TypeAlias +------------------ :pep:`484` introduced the concept of type aliases, only requiring them to be top-level unannotated assignments. This simplicity sometimes made it difficult @@ -733,8 +733,8 @@ especially when forward references or invalid types were involved. Compare:: StrCache = 'Cache[str]' # a type alias LOG_PREFIX = 'LOG[DEBUG]' # a module constant -Now the :mod:`typing` module has a special annotation :data:`TypeAlias` to -declare type aliases more explicitly:: +Now the :mod:`typing` module has a special value :data:`TypeAlias` +which lets you declare type aliases more explicitly:: StrCache: TypeAlias = 'Cache[str]' # a type alias LOG_PREFIX = 'LOG[DEBUG]' # a module constant @@ -805,6 +805,10 @@ Other Language Changes defined by :pep:`526`) no longer cause any runtime effects with ``from __future__ import annotations``. (Contributed by Batuhan Taskaya in :issue:`42737`.) +* Class and module objects now lazy-create empty annotations dicts on demand. + The annotations dicts are stored in the object’s ``__dict__`` for + backwards compatibility. + (Contributed by Larry Hastings in :issue:`43901`.) New Modules =========== @@ -987,10 +991,18 @@ inspect When a module does not define ``__loader__``, fall back to ``__spec__.loader``. (Contributed by Brett Cannon in :issue:`42133`.) -Added *globalns* and *localns* parameters in :func:`~inspect.signature` and -:meth:`inspect.Signature.from_callable` to retrieve the annotations in given -local and global namespaces. -(Contributed by Batuhan Taskaya in :issue:`41960`.) +Added :func:`inspect.get_annotations`, which safely computes the annotations +defined on an object. It works around the quirks of accessing the annotations +on various types of objects, and makes very few assumptions about the object +it examines. :func:`inspect.get_annotations` can also correctly un-stringize +stringized annotations. :func:`inspect.get_annotations` is now considered +best practice for accessing the annotations dict defined on any Python object. +Relatedly, :func:`inspect.signature`, +:func:`inspect.Signature.from_callable`, and ``inspect.Signature.from_function`` +now call :func:`inspect.get_annotations` to retrieve annotations. This means +:func:`inspect.signature` and :func:`inspect.Signature.from_callable` can +also now un-stringize stringized annotations. +(Contributed by Larry Hastings in :issue:`43817`.) linecache --------- @@ -1155,7 +1167,7 @@ of types readily interpretable by type checkers. typing ------ -For major changes, see `New Features Related to Type Annotations`_. +For major changes, see `New Features Related to Type Hints`_. The behavior of :class:`typing.Literal` was changed to conform with :pep:`586` and to match the behavior of static type checkers specified in the PEP. @@ -1255,11 +1267,12 @@ Optimizations faster, lzma decompression 1.20x ~ 1.32x faster, ``GzipFile.read(-1)`` 1.11x ~ 1.18x faster. (Contributed by Ma Lin, reviewed by Gregory P. Smith, in :issue:`41486`) -* Function parameters and their annotations are no longer computed at runtime, - but rather at compilation time. They are stored as a tuple of strings at the - bytecode level. It is now around 2 times faster to create a function with - parameter annotations. (Contributed by Yurii Karabas and Inada Naoki - in :issue:`42202`) +* When using stringized annotations, annotations dicts for functions are no longer + created when the function is created. Instead, they're stored as a tuple of + strings, and the function object lazily converts this into the annotations dict + on demand. This optimization cuts the CPU time needed to define an annotated + function by half. + (Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`) * Substring search functions such as ``str1 in str2`` and ``str2.find(str1)`` now sometimes use Crochemore & Perrin's "Two-Way" string searching @@ -1569,8 +1582,8 @@ Changes in the Python API CPython bytecode changes ======================== -* The ``MAKE_FUNCTION`` instruction accepts tuple of strings as annotations - instead of dictionary. +* The ``MAKE_FUNCTION`` instruction now accepts either a dict or a tuple of + strings as the function's annotations. (Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`) Build Changes

1 0

bpo-42269: Add slots parameter to dataclass decorator (GH-24171)
by ericvsmith May 1, 2021

May 1, 2021

https://github.com/python/cpython/commit/c24199184bea0c851c1a7296ae54aaf18e… commit: c24199184bea0c851c1a7296ae54aaf18ee56752 branch: master author: Yurii Karabas <1998uriyyo(a)gmail.com> committer: ericvsmith <ericvsmith(a)users.noreply.github.com> date: 2021-04-30T22:14:30-04:00 summary: bpo-42269: Add slots parameter to dataclass decorator (GH-24171) Add slots parameter to dataclass decorator and make_dataclass function. files: A Misc/NEWS.d/next/Library/2021-01-08-22-32-13.bpo-42269.W5v8z4.rst M Doc/library/dataclasses.rst M Doc/whatsnew/3.10.rst M Lib/dataclasses.py M Lib/test/test_dataclasses.py diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 0e8db5003f5fe4..dbbc5d66c410be 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -46,7 +46,7 @@ directly specified in the ``InventoryItem`` definition shown above. Module-level decorators, classes, and functions ----------------------------------------------- -.. decorator:: dataclass(*, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False) +.. decorator:: dataclass(*, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False) This function is a :term:`decorator` that is used to add generated :term:`special method`\s to classes, as described below. @@ -79,7 +79,7 @@ Module-level decorators, classes, and functions class C: ... - @dataclass(init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False) + @dataclass(init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False) class C: ... @@ -173,6 +173,11 @@ Module-level decorators, classes, and functions glossary entry for details. Also see the ``dataclasses.KW_ONLY`` section. + - ``slots``: If true (the default is ``False``), :attr:`__slots__` attribute + will be generated and new class will be returned instead of the original one. + If :attr:`__slots__` is already defined in the class, then :exc:`TypeError` + is raised. + ``field``\s may optionally specify a default value, using normal Python syntax:: @@ -337,7 +342,7 @@ Module-level decorators, classes, and functions Raises :exc:`TypeError` if ``instance`` is not a dataclass instance. -.. function:: make_dataclass(cls_name, fields, *, bases=(), namespace=None, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False) +.. function:: make_dataclass(cls_name, fields, *, bases=(), namespace=None, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False) Creates a new dataclass with name ``cls_name``, fields as defined in ``fields``, base classes as given in ``bases``, and initialized @@ -346,8 +351,8 @@ Module-level decorators, classes, and functions or ``(name, type, Field)``. If just ``name`` is supplied, ``typing.Any`` is used for ``type``. The values of ``init``, ``repr``, ``eq``, ``order``, ``unsafe_hash``, ``frozen``, - ``match_args``, and ``kw_only`` have the same meaning as they do - in :func:`dataclass`. + ``match_args``, ``kw_only``, and ``slots`` have the same meaning as + they do in :func:`dataclass`. This function is not strictly required, because any Python mechanism for creating a new class with ``__annotations__`` can diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 1a390663c8a404..9c8e2969938798 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -895,6 +895,12 @@ The ``BUTTON5_*`` constants are now exposed in the :mod:`curses` module if they are provided by the underlying curses library. (Contributed by Zackery Spytz in :issue:`39273`.) +dataclasses +----------- + +Added ``slots`` parameter in :func:`dataclasses.dataclass` decorator. +(Contributed by Yurii Karabas in :issue:`42269`) + .. _distutils-deprecated: distutils diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index 3de50cf390cc37..5e5716316f0954 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -874,7 +874,7 @@ def _hash_exception(cls, fields, globals): def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen, - match_args, kw_only): + match_args, kw_only, slots): # Now that dicts retain insertion order, there's no reason to use # an ordered dict. I am leveraging that ordering here, because # derived class fields overwrite base class fields, but the order @@ -1086,14 +1086,46 @@ def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen, _set_new_attribute(cls, '__match_args__', tuple(f.name for f in std_init_fields)) + if slots: + cls = _add_slots(cls) + abc.update_abstractmethods(cls) return cls +def _add_slots(cls): + # Need to create a new class, since we can't set __slots__ + # after a class has been created. + + # Make sure __slots__ isn't already set. + if '__slots__' in cls.__dict__: + raise TypeError(f'{cls.__name__} already specifies __slots__') + + # Create a new dict for our new class. + cls_dict = dict(cls.__dict__) + field_names = tuple(f.name for f in fields(cls)) + cls_dict['__slots__'] = field_names + for field_name in field_names: + # Remove our attributes, if present. They'll still be + # available in _MARKER. + cls_dict.pop(field_name, None) + + # Remove __dict__ itself. + cls_dict.pop('__dict__', None) + + # And finally create the class. + qualname = getattr(cls, '__qualname__', None) + cls = type(cls)(cls.__name__, cls.__bases__, cls_dict) + if qualname is not None: + cls.__qualname__ = qualname + + return cls + + def dataclass(cls=None, /, *, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, - kw_only=False): + kw_only=False, slots=False): """Returns the same class as was passed in, with dunder methods added based on the fields defined in the class. @@ -1105,12 +1137,13 @@ def dataclass(cls=None, /, *, init=True, repr=True, eq=True, order=False, __hash__() method function is added. If frozen is true, fields may not be assigned to after instance creation. If match_args is true, the __match_args__ tuple is added. If kw_only is true, then by - default all fields are keyword-only. + default all fields are keyword-only. If slots is true, an + __slots__ attribute is added. """ def wrap(cls): return _process_class(cls, init, repr, eq, order, unsafe_hash, - frozen, match_args, kw_only) + frozen, match_args, kw_only, slots) # See if we're being called as @dataclass or @dataclass(). if cls is None: @@ -1269,7 +1302,7 @@ def _astuple_inner(obj, tuple_factory): def make_dataclass(cls_name, fields, *, bases=(), namespace=None, init=True, repr=True, eq=True, order=False, unsafe_hash=False, - frozen=False, match_args=True): + frozen=False, match_args=True, slots=False): """Return a new dynamically created dataclass. The dataclass name will be 'cls_name'. 'fields' is an iterable @@ -1336,7 +1369,7 @@ def exec_body_callback(ns): # Apply the normal decorator. return dataclass(cls, init=init, repr=repr, eq=eq, order=order, unsafe_hash=unsafe_hash, frozen=frozen, - match_args=match_args) + match_args=match_args, slots=slots) def replace(obj, /, **changes): diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index 670648a1b112b0..2fa0ae0126bf88 100644 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -2781,6 +2781,59 @@ class Derived(Base): # We can add a new field to the derived instance. d.z = 10 + def test_generated_slots(self): + @dataclass(slots=True) + class C: + x: int + y: int + + c = C(1, 2) + self.assertEqual((c.x, c.y), (1, 2)) + + c.x = 3 + c.y = 4 + self.assertEqual((c.x, c.y), (3, 4)) + + with self.assertRaisesRegex(AttributeError, "'C' object has no attribute 'z'"): + c.z = 5 + + def test_add_slots_when_slots_exists(self): + with self.assertRaisesRegex(TypeError, '^C already specifies __slots__$'): + @dataclass(slots=True) + class C: + __slots__ = ('x',) + x: int + + def test_generated_slots_value(self): + @dataclass(slots=True) + class Base: + x: int + + self.assertEqual(Base.__slots__, ('x',)) + + @dataclass(slots=True) + class Delivered(Base): + y: int + + self.assertEqual(Delivered.__slots__, ('x', 'y')) + + @dataclass + class AnotherDelivered(Base): + z: int + + self.assertTrue('__slots__' not in AnotherDelivered.__dict__) + + def test_returns_new_class(self): + class A: + x: int + + B = dataclass(A, slots=True) + self.assertIsNot(A, B) + + self.assertFalse(hasattr(A, "__slots__")) + self.assertTrue(hasattr(B, "__slots__")) + + class TestDescriptors(unittest.TestCase): def test_set_name(self): # See bpo-33141. diff --git a/Misc/NEWS.d/next/Library/2021-01-08-22-32-13.bpo-42269.W5v8z4.rst b/Misc/NEWS.d/next/Library/2021-01-08-22-32-13.bpo-42269.W5v8z4.rst new file mode 100644 index 00000000000000..595f8733d0eab9 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-01-08-22-32-13.bpo-42269.W5v8z4.rst @@ -0,0 +1,3 @@ +Add ``slots`` parameter to ``dataclasses.dataclass`` decorator to +automatically generate ``__slots__`` for class. Patch provided by Yurii +Karabas.

1 0

bpo-43916: Export the _PyStructSequence_InitType to fix build errors in the curses module (GH-25768)
by pablogsal May 1, 2021

May 1, 2021

https://github.com/python/cpython/commit/558df9010915c8fe94f4d7f842e7c5aabb… commit: 558df9010915c8fe94f4d7f842e7c5aabbb06b14 branch: master author: Pablo Galindo <Pablogsal(a)gmail.com> committer: pablogsal <Pablogsal(a)gmail.com> date: 2021-05-01T02:21:19+01:00 summary: bpo-43916: Export the _PyStructSequence_InitType to fix build errors in the curses module (GH-25768) files: M Doc/data/stable_abi.dat M Include/structseq.h M Misc/stable_abi.txt M PC/python3dll.c diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index 833228f7fd755d..b21374a05cb6a5 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -918,6 +918,7 @@ Py_tp_setattro Py_tp_str Py_tp_traverse Py_uintptr_t +_PyStructSequence_InitType _frame _node allocfunc diff --git a/Include/structseq.h b/Include/structseq.h index af3af415b013a7..890e03130b4edb 100644 --- a/Include/structseq.h +++ b/Include/structseq.h @@ -27,12 +27,10 @@ PyAPI_FUNC(void) PyStructSequence_InitType(PyTypeObject *type, PyAPI_FUNC(int) PyStructSequence_InitType2(PyTypeObject *type, PyStructSequence_Desc *desc); #endif -#ifdef Py_BUILD_CORE -extern int _PyStructSequence_InitType( +PyAPI_FUNC(int) _PyStructSequence_InitType( PyTypeObject *type, PyStructSequence_Desc *desc, unsigned long tp_flags); -#endif PyAPI_FUNC(PyTypeObject*) PyStructSequence_NewType(PyStructSequence_Desc *desc); PyAPI_FUNC(PyObject *) PyStructSequence_New(PyTypeObject* type); diff --git a/Misc/stable_abi.txt b/Misc/stable_abi.txt index 2a802bd5f4b831..d79746e0a4ec04 100644 --- a/Misc/stable_abi.txt +++ b/Misc/stable_abi.txt @@ -2121,6 +2121,8 @@ function PyObject_GetAiter added 3.10 data PyExc_EncodingWarning added 3.10 +function _PyStructSequence_InitType + added 3.10 # Support for Stable ABI in debug builds diff --git a/PC/python3dll.c b/PC/python3dll.c index 574d4dcad9f75b..30868c1864622e 100755 --- a/PC/python3dll.c +++ b/PC/python3dll.c @@ -35,6 +35,7 @@ EXPORT_FUNC(_PyObject_GC_Resize) EXPORT_FUNC(_PyObject_New) EXPORT_FUNC(_PyObject_NewVar) EXPORT_FUNC(_PyState_AddModule) +EXPORT_FUNC(_PyStructSequence_InitType) EXPORT_FUNC(_PyThreadState_Init) EXPORT_FUNC(_PyThreadState_Prealloc) EXPORT_FUNC(_PyTrash_deposit_object)

1 0

bpo-43971: Add spaces around annotated arg default '=' (GH-25702)
by terryjreedy May 1, 2021

May 1, 2021

https://github.com/python/cpython/commit/606bb1c77c66eca081d03374ad4b53d9e6… commit: 606bb1c77c66eca081d03374ad4b53d9e603dd9d branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: terryjreedy <tjreedy(a)udel.edu> date: 2021-04-30T20:31:00-04:00 summary: bpo-43971: Add spaces around annotated arg default '=' (GH-25702) Result: "quantity_on_hand: int = 0". (cherry picked from commit e726a902b7c73a7056b7421d801e47ffff255873) Co-authored-by: Mohamed Moselhy <look4d(a)gmail.com> Co-authored-by: Mohamed Moselhy <look4d(a)gmail.com> files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 9b53372b3d3ab..c47ee0a932ae2 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -33,7 +33,7 @@ using :pep:`526` type annotations. For example this code:: Will add, among other things, a :meth:`__init__` that looks like:: - def __init__(self, name: str, unit_price: float, quantity_on_hand: int=0): + def __init__(self, name: str, unit_price: float, quantity_on_hand: int = 0): self.name = name self.unit_price = unit_price self.quantity_on_hand = quantity_on_hand

1 0

bpo-43971: Add spaces around annotated arg default '=' (GH-25702)
by terryjreedy May 1, 2021

May 1, 2021

https://github.com/python/cpython/commit/e48405a9be067d50d6d0d83dc22013fc68… commit: e48405a9be067d50d6d0d83dc22013fc68f15657 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: terryjreedy <tjreedy(a)udel.edu> date: 2021-04-30T20:30:38-04:00 summary: bpo-43971: Add spaces around annotated arg default '=' (GH-25702) Result: "quantity_on_hand: int = 0". (cherry picked from commit e726a902b7c73a7056b7421d801e47ffff255873) Co-authored-by: Mohamed Moselhy <look4d(a)gmail.com> Co-authored-by: Mohamed Moselhy <look4d(a)gmail.com> files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 6cd8d8ed0c264..8711191500c60 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -33,7 +33,7 @@ using :pep:`526` type annotations. For example this code:: Will add, among other things, a :meth:`__init__` that looks like:: - def __init__(self, name: str, unit_price: float, quantity_on_hand: int=0): + def __init__(self, name: str, unit_price: float, quantity_on_hand: int = 0): self.name = name self.unit_price = unit_price self.quantity_on_hand = quantity_on_hand

1 0