With the new variable annotation syntax, it is possible to implement a useful "modifiers" library, all used as superclasses. Possible modifiers: * Named / Struct: annotation-defined fields. No monkey patching. * Immutable / Const * Sealed / Final: unsubclassable class * Array: a mutable tuple. (No "+" or "*" operators) There of course may be others. Subclasses of the modifiers get default methods and behavior. The modifiers can used in combinations. For example: class Command(Immutable, Named): cmd : str = "NOP" value : int = None load = Command(cmd="load", value=1000) # no positional arguments The syntax for Array is less pretty: class Command(Array): _0 : str = "NOP" _1 : int = None load = Command("load", 1000) # no positional arguments Many other combinations are useful too. "Sealed" is almost orthogonal to the others, Obviously, some combinations already exists: * (Immutable, Array) is similar to Tuple * NamedTuple is another name for (Immutable, Named, Array). * Enum should be on this list too Some less promising modifiers: * Namespace / Static: uninstantiable class. A module. * Volatile, for externally facing classes. Hints the static checkers / jitters that they should not assume they know the values. * Abstract Alternatives and cons : I suggest base classes instead of decorators since NamedTuple and tuple go this way, and since it is static information. I am not sure which is better though. Metaclass parameters can be used (and actually used in my implementation, and in NamedTuple's), but the syntax is uglier. "array" is only for numeric values in Python, so this name is problematic.So is struct. I have a rough implementation for most of the above; much of it is not hard in general, though some details are hard to get right. --- Benefits of putting such a collection in stdlib (instead of as an external package) include: 1. This information can be used by typecheckers, and also by users, to reason about programs. If isinstance(x, ImmutableArray), then x is an instantiation of ImmutableArray. 2. A conventional syntax and a single answer for "How do I make my class immutable", "How do I make my class unsubclassable" 3. The syntax, especially for Struct as above, is pretty and clean. The Array syntax is less so. 4. I think that the array implementation can use internal CPython details to be implemented efficiently. I am not sure that typing.modifiers is the right place, since these are not exactly type hints; they generate methods, and are intended to be enforced at runtime. I think that even if this idea is not accepted, the general theme is something that might be useful to keep in mind, stdlib might accumulate such modifiers, and it will be nice to keep things uniform. Opinions? ~Elazar
On Fri, Sep 16, 2016 at 12:10:22AM +0000, אלעזר wrote: [...]
Benefits of putting such a collection in stdlib (instead of as an external package) include:
Slow down! Before getting all excited about adding these typing hints(?) into typing.modifiers, you first have to convince people that they are useful and deserve a place in the std library. What problem are these hints/classes supposed to solve? What solutions already exist? Why aren't those solutions good enough?
That's how type-checkers work. The class doesn't need to be in the std lib for a type-checker to reason about it.
2. A conventional syntax and a single answer for "How do I make my class immutable", "How do I make my class unsubclassable"
Do we need syntax for those?
3. The syntax, especially for Struct as above, is pretty and clean. The Array syntax is less so.
I don't even understand what the Array syntax is supposed to mean.
4. I think that the array implementation can use internal CPython details to be implemented efficiently.
What happens to Jython, PyPy, IronPython etc?
Then I think your answer is: no, typing.modifiers is NOT the right place.
The stdlib might accumulate many things. Why should it accumulate these? -- Steve
Thanks for the reply בתאריך יום ו׳, 16 בספט' 2016, 13:16, מאת Steven D'Aprano < steve@pearwood.info>:
I was thinking this is the place to do this? What problem are these hints/classes supposed to solve? What solutions
already exist? Why aren't those solutions good enough?
I have addressed that only very briefly; I will try to elaborate later (I'm writing from the phone. Sorry)
No, it's not how they work, since it's not true. I meant the actual type, not a subtype.
That's already a bad sign... Each underscore gives type to its index. The last index is the maximal.
Similarly or even more so.
Speculating it will make its way, where should it land then?
In this part I wasn't talking about what should happen, but rather what might happen gradually, in which case it'll be nice to fit in a uniform structure.
On Fri, Sep 16, 2016 at 1:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
It is partially described below: mainly, reasoning and readability. I will
try to elaborate, and please forgive me if what I'm writing is obvious to everyone here. Note that by "readability" I mean "conventional, non-repetitive syntax, the allow human reasoning", not the subjective issue (which is also important). Q. Why standard? A. Python is not very good at allowing strict reasoning. The confidence in Python programs, insofar it exists, comes from its reliance on very strong conventions ("One Way To Do It") and mostly-explicit control flow and name binding. Data flow, however, is very much implicit since the language encourages mutation. Tools that employ conventions are generally linters; these are very useful in finding trivial bugs, but have zero guarantees about their absence - unless the inference is trivial. linters and IDEs can also use basic form of type information, but will not analyze it deeply; hence the need for convention for modifiers that themselves constrain mutability, subclassing, etc.As a gradually-typed language, constraints are opt-in at best, but there should be a way to opt-them-in in a conventional way. Type hints are very good, and the suggested modifiers complement them. --- Q. What solutions already exist? Why aren't those solutions good enough? A: * Named: There are no conventions and no standard way to prohibit monkey-patching, since this flexibility is often useful. That's fine. But what if I want to tell the other programmer that this class should not be monkey patched, but its fields are mutable? Currently the general solution is something like class Person: name = None id = None def __init__(self, name, id): self.name = name self.od = id Say we ignore the repetition. Will tools catch the last typo? Some might warn, but how should a tool know that it is not intentional? There's no standard way to express this intent. mypy will say that's an error (if there are type annotations), but we get the other problem of *allowing* monkey patching. StackOverflow suggests <http://stackoverflow.com/questions/35988/c-like-structures-in-python> either NamedTuple (which is not what I want in this case, since it is immutable and indexable), another answer without __init__ (which requires useful default values, and no sign of the "non-mankey-patch" intention). another answer with arbitrary **kwargs assignment (very flexible, but zero reasoning). Yet another suggests `dict` which is useful but has nothing to do with reasoning. There are many solutions in pypi, most only address the boilerplate issue; but the "attrs" package seems nice; it might fit in the alternative decorator-oriented framework I mentioned. There isn't even a standard name for that feature. "Struct" means something else. * Immutable: The importance to reasoning is obvious. There are many alternative suggestions; again, most upvoted is NamedTuple, which is great but also indexable. This means one kind of type error that will not be caught. (actually two, since the __init__ method takes positional arguments that can be mixed). Additionally, inheriting classes can be monkey-patched, and there's no standard way to disallow inheritance. The above problems can be solved in an ad-hoc manner, but it won't give tool support. Other solutions involve messing up with __setattr__ etc. It Again, "attrs" seems has a solution in the form of "freeze". * Sealed: Useful for typechecker (prevent erroneus cast) and optimizers that need to know the actual implementation of method dispatch (I believe it can be used in CPython too, but that's a different topic). The only solution I've found is this answer http://stackoverflow.com/questions/16564198/pythons-equivalent-of-nets-seale... which gives a hand-crafeted metaclass. But if it's not conventional, it won't be used by public tools. * Array: If I want a tuple with mutable cells, I currently simply don't have any option. There's list, which is extendable. There's array.array and numpy's matrices, which are intended for numeric processing. There might be some package for that, but I can't find it - and of course none that allow type annotations. Besides, there's no reason to assume it will work together with some other "Named" modifier implementation. * Namespace (as a shorthand for Final+Abstract): "not instantiable" will help catch both erroneous instantiations (which is implementable by hand) and erroneous "isinstance" (which is not). It might be used as an equivalent to "object" in Scala. ---- My suggestion is (again) two steps mixed: 1. Add support for modifiers, by convention and library 2. Some concrete syntax, inspired by Enum and the new NamedTuple syntax. Alternative syntax, using "attrs"-like decorators, might be: @modifiers(Immutable, Named) class Command: cmd : str = "NOP" value : int = None load = Command(cmd="load", value=1000) But then `enum` and `NamedTuple` should be updated to match this convention. Again, even if this suggestion is not accepted right now, there should be some decision to avoid accumulating parts of it in a way that is not consistent and will not allow combinations. For example, If it the addition of modifiers was forced on the language by God, would it be in the form of decorators? if your answer is yes, then I believe the new NamedTuple form should become a decorator too, before it is too late (as in Enum). -- P.S. how do I change the name in my quotes? I believe אלעזר is not very easy to address... ~Elazar
On 16 September 2016 at 14:22, אלעזר <elazarg@gmail.com> wrote:
P.S. how do I change the name in my quotes? I believe אלעזר is not very easy to address...
As you're posting from a gmail account, it's likely your name from personal info in your Google Accounts settings. Your mail client (if it's not the gmail web app) may have a way of changing it, too. Paul
On Fri, Sep 16, 2016 at 1:39 PM, Chris Angelico <rosuav@gmail.com> wrote:
If people can't cite names using Hebrew script, that's their problem, not yours. :)
On Sat, Sep 17, 2016 at 03:39:08AM +1000, Chris Angelico wrote:
[in-joke, that Chris will get] But how do we know that אלעזר is his real name? It looks made up to me. [/in-joke] Still, thank you Elazar for also providing a Latin-1 compatible name which is readable and pronounceable by English speakers. -- Steve
On Fri, Sep 16, 2016 at 12:10:22AM +0000, אלעזר wrote: [...]
Benefits of putting such a collection in stdlib (instead of as an external package) include:
Slow down! Before getting all excited about adding these typing hints(?) into typing.modifiers, you first have to convince people that they are useful and deserve a place in the std library. What problem are these hints/classes supposed to solve? What solutions already exist? Why aren't those solutions good enough?
That's how type-checkers work. The class doesn't need to be in the std lib for a type-checker to reason about it.
2. A conventional syntax and a single answer for "How do I make my class immutable", "How do I make my class unsubclassable"
Do we need syntax for those?
3. The syntax, especially for Struct as above, is pretty and clean. The Array syntax is less so.
I don't even understand what the Array syntax is supposed to mean.
4. I think that the array implementation can use internal CPython details to be implemented efficiently.
What happens to Jython, PyPy, IronPython etc?
Then I think your answer is: no, typing.modifiers is NOT the right place.
The stdlib might accumulate many things. Why should it accumulate these? -- Steve
Thanks for the reply בתאריך יום ו׳, 16 בספט' 2016, 13:16, מאת Steven D'Aprano < steve@pearwood.info>:
I was thinking this is the place to do this? What problem are these hints/classes supposed to solve? What solutions
already exist? Why aren't those solutions good enough?
I have addressed that only very briefly; I will try to elaborate later (I'm writing from the phone. Sorry)
No, it's not how they work, since it's not true. I meant the actual type, not a subtype.
That's already a bad sign... Each underscore gives type to its index. The last index is the maximal.
Similarly or even more so.
Speculating it will make its way, where should it land then?
In this part I wasn't talking about what should happen, but rather what might happen gradually, in which case it'll be nice to fit in a uniform structure.
On Fri, Sep 16, 2016 at 1:16 PM Steven D'Aprano <steve@pearwood.info> wrote:
It is partially described below: mainly, reasoning and readability. I will
try to elaborate, and please forgive me if what I'm writing is obvious to everyone here. Note that by "readability" I mean "conventional, non-repetitive syntax, the allow human reasoning", not the subjective issue (which is also important). Q. Why standard? A. Python is not very good at allowing strict reasoning. The confidence in Python programs, insofar it exists, comes from its reliance on very strong conventions ("One Way To Do It") and mostly-explicit control flow and name binding. Data flow, however, is very much implicit since the language encourages mutation. Tools that employ conventions are generally linters; these are very useful in finding trivial bugs, but have zero guarantees about their absence - unless the inference is trivial. linters and IDEs can also use basic form of type information, but will not analyze it deeply; hence the need for convention for modifiers that themselves constrain mutability, subclassing, etc.As a gradually-typed language, constraints are opt-in at best, but there should be a way to opt-them-in in a conventional way. Type hints are very good, and the suggested modifiers complement them. --- Q. What solutions already exist? Why aren't those solutions good enough? A: * Named: There are no conventions and no standard way to prohibit monkey-patching, since this flexibility is often useful. That's fine. But what if I want to tell the other programmer that this class should not be monkey patched, but its fields are mutable? Currently the general solution is something like class Person: name = None id = None def __init__(self, name, id): self.name = name self.od = id Say we ignore the repetition. Will tools catch the last typo? Some might warn, but how should a tool know that it is not intentional? There's no standard way to express this intent. mypy will say that's an error (if there are type annotations), but we get the other problem of *allowing* monkey patching. StackOverflow suggests <http://stackoverflow.com/questions/35988/c-like-structures-in-python> either NamedTuple (which is not what I want in this case, since it is immutable and indexable), another answer without __init__ (which requires useful default values, and no sign of the "non-mankey-patch" intention). another answer with arbitrary **kwargs assignment (very flexible, but zero reasoning). Yet another suggests `dict` which is useful but has nothing to do with reasoning. There are many solutions in pypi, most only address the boilerplate issue; but the "attrs" package seems nice; it might fit in the alternative decorator-oriented framework I mentioned. There isn't even a standard name for that feature. "Struct" means something else. * Immutable: The importance to reasoning is obvious. There are many alternative suggestions; again, most upvoted is NamedTuple, which is great but also indexable. This means one kind of type error that will not be caught. (actually two, since the __init__ method takes positional arguments that can be mixed). Additionally, inheriting classes can be monkey-patched, and there's no standard way to disallow inheritance. The above problems can be solved in an ad-hoc manner, but it won't give tool support. Other solutions involve messing up with __setattr__ etc. It Again, "attrs" seems has a solution in the form of "freeze". * Sealed: Useful for typechecker (prevent erroneus cast) and optimizers that need to know the actual implementation of method dispatch (I believe it can be used in CPython too, but that's a different topic). The only solution I've found is this answer http://stackoverflow.com/questions/16564198/pythons-equivalent-of-nets-seale... which gives a hand-crafeted metaclass. But if it's not conventional, it won't be used by public tools. * Array: If I want a tuple with mutable cells, I currently simply don't have any option. There's list, which is extendable. There's array.array and numpy's matrices, which are intended for numeric processing. There might be some package for that, but I can't find it - and of course none that allow type annotations. Besides, there's no reason to assume it will work together with some other "Named" modifier implementation. * Namespace (as a shorthand for Final+Abstract): "not instantiable" will help catch both erroneous instantiations (which is implementable by hand) and erroneous "isinstance" (which is not). It might be used as an equivalent to "object" in Scala. ---- My suggestion is (again) two steps mixed: 1. Add support for modifiers, by convention and library 2. Some concrete syntax, inspired by Enum and the new NamedTuple syntax. Alternative syntax, using "attrs"-like decorators, might be: @modifiers(Immutable, Named) class Command: cmd : str = "NOP" value : int = None load = Command(cmd="load", value=1000) But then `enum` and `NamedTuple` should be updated to match this convention. Again, even if this suggestion is not accepted right now, there should be some decision to avoid accumulating parts of it in a way that is not consistent and will not allow combinations. For example, If it the addition of modifiers was forced on the language by God, would it be in the form of decorators? if your answer is yes, then I believe the new NamedTuple form should become a decorator too, before it is too late (as in Enum). -- P.S. how do I change the name in my quotes? I believe אלעזר is not very easy to address... ~Elazar
On 16 September 2016 at 14:22, אלעזר <elazarg@gmail.com> wrote:
P.S. how do I change the name in my quotes? I believe אלעזר is not very easy to address...
As you're posting from a gmail account, it's likely your name from personal info in your Google Accounts settings. Your mail client (if it's not the gmail web app) may have a way of changing it, too. Paul
On Fri, Sep 16, 2016 at 1:39 PM, Chris Angelico <rosuav@gmail.com> wrote:
If people can't cite names using Hebrew script, that's their problem, not yours. :)
On Sat, Sep 17, 2016 at 03:39:08AM +1000, Chris Angelico wrote:
[in-joke, that Chris will get] But how do we know that אלעזר is his real name? It looks made up to me. [/in-joke] Still, thank you Elazar for also providing a Latin-1 compatible name which is readable and pronounceable by English speakers. -- Steve
participants (5)
-
Alexander Belopolsky -
Chris Angelico -
Paul Moore -
Steven D'Aprano -
אלעזר