Mailman 3 Matching TypedDicts and other values in JSON - Python-ideas

Nov. 22, 2020

      I am excited about the potential of the new PEP 634-636 "match" statement to
match JSON data received by Python web applications. Often this JSON data is
in the form of structured dictionaries (TypedDicts) containing Lists and
other primitives (str, float, bool, None).

PEP 634-636 already contain the ability to match all of those underlying 
data
types except for TypedDicts, so I'd like to explore what it might look 
like to
match a TypedDict...

Consider an example web application that wants to provide a service to draw
shapes, perhaps on a connected physical billboard.

The service has a '/draw_shape' endpoint which takes a JSON object
(a Shape TypedDict) describing a shape to draw:

     from bottle import HTTPResponse, request, route
     from typing import Literal, TypedDict, Union

     class Point2D(TypedDict):
         x: float
         y: float

     class Circle(TypedDict):
         type: Literal['circle']
         center: Point2D  # has a nested TypedDict!
         radius: float

     class Rect(TypedDict):
         type: Literal['rect']
         x: float
         y: float
         width: float
         height: float

     Shape = Union[Circle, Rect]  # a Tagged Union / Discriminated Union

     @route('/draw_shape')
     def draw_shape() -> None:
         match request.json:  # a Shape?
             ...
             case _:
                 return HTTPResponse(status=400)  # Bad Request

Now, what syntax could we have at the ... inside the "match" statement to
effectively pull apart a Shape?

The current version of PEP 634-636 would require duplicating all the keys
and value types that are defined in Shape's underlying Circle and Rect 
types:

         match request.json:  # a Shape?
             case {'type': 'circle', 'center': {'x': float(), 'y': 
float()}, \
                     radius: float()} as circle:
                 draw_circle(circle)  # type is inferred as Circle
             case {'type': 'rect', 'x': float(), 'y': float(), \
                     'width': float(), 'height': float()} as rect:
                 draw_rect(rect)  # type is inferred as Rect
             case _:
                 return HTTPResponse(status=400)  # Bad Request

Wouldn't it be nicer if we could use class patterns instead?

         match request.json:  # a Shape?
             case Circle() as circle:
                 draw_circle(circle)
             case Rect() as rect:
                 draw_rect(rect)
             case _:
                 return HTTPResponse(status=400)  # Bad Request

Now that syntax almost works except that Circle and Rect, being TypedDicts,
do not support isinstance() checks. PEP 589 ("TypedDict") did not define how
such isinstance() checks should work initially because it's somewhat complex
to specify. From the PEP:
...
In particular, TypedDict type objects cannot be used in isinstance() 
tests
such as isinstance(d, Movie). The reason is that there is no existing
support for checking types of dictionary item values, since isinstance()
does not work with many PEP 484 types, including common ones like 
List[str].
[...]
This is consistent with how isinstance() is not supported for List[str].
Well, what if we (or I) took the time to specify how isinstance() worked 
with
TypedDict? Then the match syntax above with TypedDict as a class pattern
would work!

Refining the example above even further, it would be nice if we didn't 
have to
enumerate all the different types of Shapes directly in the match-statement.
What if we could match on a Shape directly?

         match request.json:  # a Shape?
             case Shape() as shape:
                 draw_shape(shape)
             case _:
                 return HTTPResponse(status=400)  # Bad Request

Now for that syntax to work it must be possible for an isinstance() check to
work on a Shape, which is defined to be a Union[Circle, Rect], and 
isinstance()
checks also aren't currently defined for Union types. So it would be useful
to define isinstance() for Union types as well.

Of course that match-statement is now simple enough to just be rewriten 
as an
if-statement:

         if isinstance(shape := request.json, Shape):
             draw_shape(shape)
         else:
             return HTTPResponse(status=400)  # Bad Request

Now *that* is a wonderfully short bit of parsing code that results in
well-typed objects as output. 🎉

So to summarize, I believe it's possible to support really powerful matching
on JSON objects if we just define how isinstance() should work with a 
handful
of new types.

In particular the above example would work if isinstance() was defined for:
     * Union[T1, T2], Optional[T]
     * T extends TypedDict
     * Literal['foo']

For arbitrary JSON beyond the example, we'd also want to support
isinstance() for:
     * List[T]

We already support isinstance() for the other JSON primitive types:
     * str
     * float
     * bool
     * type(None)

So what do folks think? If I were to start writing a PEP to extend 
isinstance()
to cover at least the above cases, would that be welcome?

-- 
David Foster | Seattle, WA, USA
Contributor to TypedDict support for mypy

Matching TypedDicts and other values in JSON

David Foster

Guido van Rossum

Samuel Colvin

Guido van Rossum

David Foster

Guido van Rossum

tags

participants (3)