On Fri, 18 Aug 2023 at 08:35, Ran Benita via Typing-sig < typing-sig@python.org> wrote:

The traditional way plugins did this was to use an attribute on the `item`:

def pytest_runtest_setup(item: pytest.Item) -> None: item.my_plugin_attr = MyPluginAttrType(...)

def pytest_runtest_teardown(item: pytest.Item) -> None: use_my_attr(item.my_plugin_attr) # Maybe del item.my_plugin_attr

The problem with this is that type checkers are not aware of `my_plugin_attr` and they complain. To make things clean for type checkers, and type safe (no casts/asserts) an alternative solution is needed.

This sounds very much like `types.SimpleNamespace`. So maybe you could look at how typeshed annotates that type? Paul

I haven't tested it in-depth, but what about using a Protocol `ItemLike` that describes the Item interface as the argument annotation in `pytest_runtest_setup`? Then plugins could just create a protocol that includes `ItemLike` and the attributes they are setting themselves.     from typing import Protocol     from pytest import ItemLike     class MyPluginItemLike(ItemLike, Protocol):       plugin_attr: int       # etc.     def pytest_runtest_setup(item: MyPluginItemLike) -> None:       item.plugin_attr = 1     def pytest_runtest_teardown(item: MyPluginItemLike) -> None:       print(item.plugin_attr) ---- ORIGINAL MESSAGE ---- Date: 2023-08-18 09:34:11 UTC+0200 From:typing-sig@python.org To:typing-sig@python.org Subject: [Typing-sig] Heterogeneous, mutable, type-safe mapping

This post doesn't have any particular aim, just a design report. I'm curious if anyone else had done this and has experience to share, or maybe there's a nicer solution for this in Python.

### Motivation

In pytest we have plugin system where external plugins may implement various hooks. For the sake of example let's say a plugin wants to do something on each test ("item") setup and teardown:

    def pytest_runtest_setup(item: pytest.Item) -> None: ...     def pytest_runtest_teardown(item: pytest.Item) -> None: ...

One thing plugins often want to do is to store/associate some data with the item on setup, then use it on teardown.

The traditional way plugins did this was to use an attribute on the `item`:

    def pytest_runtest_setup(item: pytest.Item) -> None:         item.my_plugin_attr = MyPluginAttrType(...)

    def pytest_runtest_teardown(item: pytest.Item) -> None:         use_my_attr(item.my_plugin_attr)         # Maybe         del item.my_plugin_attr

The problem with this is that type checkers are not aware of `my_plugin_attr` and they complain. To make things clean for type checkers, and type safe (no casts/asserts) an alternative solution is needed.

### Rejected solution 1

We can add a dict to each item on the pytest side:

    class Item:         def __init__(self, ...) -> None:             ...             self.store: Final[Dict[str, ???]] = {}

which plugins will use as

    def pytest_runtest_setup(item: pytest.Item) -> None:         item.store["my_plugin_attr"] = MyPluginAttrType(...)

This fixes the "undefined attribute" issue, but as the `???` indicates, it's still not good. We can either use `object` and force casts/asserts, or `Any` and lose type safety.

### Rejected solution 2

We can imagine a typing feature which allows a 3rd-party to "overlay" a type it did not define with extra fields/methods. I think some languages support this (TypeScript?). But it does not seem like a desirable feature to me.

### Solution we ended up with

We define a `Stash` type (because plugins "stash" their data there), which is used as follows:

    # In pytest

    class Item:         def __init__(self, ...) -> None:             ...             self.stash: Final[Stash] = Stash()

    # In plugin

    my_plugin_attr: Final = pytest.StashKey[MyPluginAttrType]()

    def pytest_runtest_setup(item: pytest.Item) -> None:         item.stash[my_plugin_attr] = MyPluginAttrType(...)

    def pytest_runtest_teardown(item: pytest.Item) -> None:         use_my_attr(item.stash[my_plugin_attr])         # Maybe         del item.stash[my_plugin_attr]

Each plugin defines `StashKey`s for its attributes, and the stash key carries the attribute type which makes things type safe.

The `Stash` is defined like this (see [0] for full implementation in pytest):

    T = TypeVar("T")     D = TypeVar("D")

    class StashKey(Generic[T]):         __slots__ = ()

    class Stash:         __slots__ = ("_storage",)

        def __init__(self) -> None:             self._storage: Dict[StashKey[Any], object] = {}

        def __setitem__(self, key: StashKey[T], value: T) -> None:             self._storage[key] = value

        def __getitem__(self, key: StashKey[T]) -> T:             return cast(T, self._storage[key])

        def __delitem__(self, key: StashKey[T]) -> None:             del self._storage[key]

        # .. other mutable mapping methods -- same idea.

[0] https://github.com/pytest-dev/pytest/blob/7.4.0/src/_pytest/stash.py

### Advantages

No undefined attributes. Type safe. The `StashKey`s are namespaced objects, which makes conflicts between plugins impossible, unlike with string keys.

### Disadvantages

Not standard, users are not familiar with idea and are confused, try to use it as a dict.

Less ergonomic, need to define the `StashKey`s at the module level.

_______________________________________________ Typing-sig mailing list --typing-sig@python.org To unsubscribe send an email totyping-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address:dev@reggx.eu

Why not something like: from typing import Any, Generic, Mapping, TypeVar, TypedDict, cast T = TypeVar("T", bound=Mapping[str, Any]) class Item(Generic[T]): def __init__(self) -> None: self.stash: T = cast(T, {}) class MyStash(TypedDict): my_plugin_attr: int def pytest_runtest_setup(item: Item[MyStash]) -> None: item.stash["my_plugin_attr"] = 8 def pytest_runtest_teardown(item: Item[MyStash]) -> None: print(item.stash["my_plugin_attr"]) # Maybe del item.stash["my_plugin_attr"] # type error here because the key is required This works in both pyright and mypy.