[Python-ideas] typing.modifiers

[אלעזר]

On Fri, Sep 16, 2016 at 1:16 PM Steven D'Aprano <steve at pearwood.info> wrote:

> On Fri, Sep 16, 2016 at 12:10:22AM +0000, אלעזר wrote:
>
> [...]
> > Benefits of putting such a collection in stdlib (instead of as an
> external
> > package) include:
>
> Slow down! Before getting all excited about adding these typing hints(?)
> into typing.modifiers, you first have to convince people that they are
> useful and deserve a place in the std library.
>
> What problem are these hints/classes supposed to solve? What
> solutions already exist? Why aren't those solutions good enough?
>


> It is partially described below: mainly, reasoning and readability. I will
try to elaborate, and please forgive me if what I'm writing is obvious to
everyone here. Note that by "readability" I mean "conventional,
non-repetitive syntax, the allow human reasoning", not the subjective issue
(which is also important).

Q. Why standard?

A. Python is not very good at allowing strict reasoning. The confidence in
Python programs, insofar it exists, comes from its reliance on very strong
conventions ("One Way To Do It") and mostly-explicit control flow and name
binding. Data flow, however, is very much implicit since the language
encourages mutation.

    Tools that employ conventions are generally linters; these are very
useful in finding trivial bugs, but have zero guarantees about their
absence - unless the inference is trivial. linters and IDEs can also use
basic form of type information, but will not analyze it deeply; hence the
need for convention for modifiers that themselves constrain mutability,
subclassing, etc.As a gradually-typed language, constraints are opt-in at
best, but there should be a way to opt-them-in in a conventional way. Type
hints are very good, and the suggested modifiers complement them.

---
Q. What solutions already exist? Why aren't those solutions good enough?

A:
* Named: There are no conventions and no standard way to prohibit
monkey-patching, since this flexibility is often useful. That's fine. But
what if I want to tell the other programmer that this class should not be
monkey patched, but its fields are mutable? Currently the general solution
is something like

    class Person:
        name = None
        id = None
        def __init__(self, name, id):
            self.name = name
            self.od = id

Say we ignore the repetition. Will tools catch the last typo? Some might
warn, but how should a tool know that it is not intentional? There's no
standard way to express this intent. mypy will say that's an error (if
there are type annotations), but we get the other problem of *allowing*
monkey patching.
StackOverflow suggests
<http://stackoverflow.com/questions/35988/c-like-structures-in-python> either
NamedTuple (which is not what I want in this case, since it is immutable
and indexable), another answer without __init__ (which requires useful
default values, and no sign of the "non-mankey-patch" intention). another
answer with arbitrary **kwargs assignment (very flexible, but zero
reasoning). Yet another suggests `dict` which is useful but has nothing to
do with reasoning.
There are many solutions in pypi, most only address the boilerplate issue;
but the "attrs" package seems nice; it might fit in the alternative
decorator-oriented framework I mentioned.
There isn't even a standard name for that feature. "Struct" means something
else.

* Immutable: The importance to reasoning is obvious. There are many
alternative suggestions; again, most upvoted is NamedTuple, which is great
but also indexable. This means one kind of type error that will not be
caught. (actually two, since the __init__ method takes positional arguments
that can be mixed). Additionally, inheriting classes can be monkey-patched,
and there's no standard way to disallow inheritance. The above problems can
be solved in an ad-hoc manner, but it won't give tool support. Other
solutions involve messing up with __setattr__ etc. It Again, "attrs" seems
has a solution in the form of "freeze".

* Sealed: Useful for typechecker (prevent erroneus cast) and optimizers
that need to know the actual implementation of method dispatch (I believe
it can be used in CPython too, but that's a different topic).
The only solution I've found is this answer
http://stackoverflow.com/questions/16564198/pythons-equivalent-of-nets-sealed-class
which gives a hand-crafeted metaclass. But if it's not conventional, it
won't be used by public tools.

* Array: If I want a tuple with mutable cells, I currently simply don't
have any option. There's list, which is extendable. There's array.array and
numpy's matrices, which are intended for numeric processing. There might be
some package for that, but I can't find it - and of course none that allow
type annotations. Besides, there's no reason to assume it will work
together with some other "Named" modifier implementation.

* Namespace (as a shorthand for Final+Abstract): "not instantiable" will
help catch both erroneous instantiations (which is implementable by hand)
and erroneous "isinstance" (which is not). It might be used as an
equivalent to "object" in Scala.

----
My suggestion is (again) two steps mixed:
1. Add support for modifiers, by convention and library
2. Some concrete syntax, inspired by Enum and the new NamedTuple syntax.

Alternative syntax, using "attrs"-like decorators, might be:

       @modifiers(Immutable, Named)
    class Command:
        cmd : str = "NOP"
        value : int = None

    load = Command(cmd="load", value=1000)

But then `enum` and `NamedTuple` should be updated to match this convention.

Again, even if this suggestion is not accepted right now, there should be
some decision to avoid accumulating parts of it in a way that is not
consistent and will not allow combinations. For example, If it the addition
of modifiers was forced on the language by God, would it be in the form of
decorators? if your answer is yes, then I believe the new NamedTuple form
should become a decorator too, before it is too late (as in Enum).

--
P.S. how do I change the name in my quotes? I believe אלעזר is not very
easy to address...

~Elazar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160916/720bd94e/attachment-0001.html>