What is the intended behavior if `Tree` is already a name in scope? On Mon, Aug 24, 2015 at 12:19 AM, Prof. Dr. L. Humbert < humbert@uni-wuppertal.de> wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Hi, when looking at pep-0484 I find the *non-orthogonal* construction, which may lead to a misconcetion:

What students should be able to code:

1. varinat #-------------wishful----------------------------------\ class Tree: def __init__(self, left: Tree, right: Tree): self.left = left self.right = right

what students have to write instead:

#-------------bad workaround----------------------------\ class Tree: def __init__(self, left: 'Tree', right: 'Tree'): self.left = left self.right = right

/ Please enable: from __future__ import annotations

so the *first* variant should be possible \ At this very moment (python 3.5rc1), it is not possible, but we need it, so the construction will be orthogonal from the point of view for students(!) - _one_ concept should work in different circumstances.

TNX Ludger Humbert - -- https://twitter.com/n770 http://ddi.uni-wuppertal.de/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2

iEYEARECAAYFAlXam2MACgkQJQsN9FQ+jJ+RHgCfdcTgjVmZ3ULLwjerpJ3NdN7d NH8AoIvdTqWbkcfi7o8e7JuAYXbgZk0V =OMhU -----END PGP SIGNATURE----- _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 24.08.2015 06:19, Prof. Dr. L. Humbert wrote:

At this very moment (python 3.5rc1), it is not possible, but we need it, so the construction will be orthogonal from the point of view for students(!) - _one_ concept should work in different circumstances.

How about not using type hints when teaching the basics of trees? Type hints do not replace good variables names. What about using left_tree instead of left and right_tree instead of right? That should simplify the example for the students, remove advanced concepts and teach them something about right naming (conventions) which is desirable these days when looking at production code (readability, maintainability and so forth). Regards, Sven R. Kunze

Sven R. Kunze writes:

On 24.08.2015 06:19, Prof. Dr. L. Humbert wrote:

...

At this very moment (python 3.5rc1), it is not possible, but we need it, so the construction will be orthogonal from the point of view for students(!) - _one_ concept should work in different circumstances.

How about not using type hints when teaching the basics of trees?

That's somewhat unfair. I'll let Prof. Humbert explain his own thinking, but I can imagine a number of pedagogical contexts where I would use Python because it doesn't get in the programmer's way very often, but require students to provide type hints as a compact (and machine-checkable!) way of documenting that aspect of their design. I found his pedagogical approoach perfectly plausible when he posted originally.

On 8/24/2015 12:19 AM, Prof. Dr. L. Humbert wrote:

What students should be able to code:

1. varinat #-------------wishful----------------------------------\ class Tree: def __init__(self, left: Tree, right: Tree): self.left = left self.right = right

As you should know, at least after reading previous responses, making this work would require one of two major changes to Python class statements. 1. The class name has special (context sensitive) meaning in enclosed def statements. The compiler would have to compile def statements differently than it would the same def statements not in a Tree class. It would then have to patch all methods after the class is created. See the annoclass function below. A proposal to make the definition name of a function special within its definition has already been rejected. 2. Class statements would initially create an empty class bound to the class name. This could break back compatibility, and would require cleanup in case of a syntax error in the body. This would be similar to import statements initially putting a empty module in sys.modules to support circular imports. This is messy and still bug prone is use.

what students have to write instead:

#-------------bad workaround----------------------------\ class Tree: def __init__(self, left: 'Tree', right: 'Tree'): self.left = left self.right = right

You did not say why you think this is bad. Is it a) students have to type "'"s?, or b) the resulting annotations are strings instead of the class? The latter can easily be fixed. --- from types import FunctionType def annofix(klass): classname = klass.__name__ for ob in klass.__dict__.values(): if type(ob) is FunctionType: annotations = ob.__annotations__ for arg, anno in annotations.items(): if anno == classname: annotations[arg] = klass return klass @annofix class Tree: def __init__(self, left: 'Tree', right: 'Tree'): self.left = left self.right = right print(Tree.__init__.__annotations__) # {'left': , 'right': } --- An alternative is to use a placeholder object instead of the class name. This is less direct, not repeating the name of the class throughout the definition makes it easier to rename the class or copy methods to another class. --- class Klass: pass # An annotation object meaning 'the class this method is defined in' def annofix2(klass): for ob in klass.__dict__.values(): if type(ob) is FunctionType: annotations = ob.__annotations__ for arg, anno in annotations.items(): if anno == Klass: annotations[arg] = klass return klass @annofix2 class Tree2: def __init__(self, left: Klass, right: Klass): self.left = left self.right = right print(Tree2.__init__.__annotations__) {'right': , 'left': } -- Terry Jan Reedy

On Tue, Aug 25, 2015 at 11:15 AM, Terry Reedy wrote:

2. Class statements would initially create an empty class bound to the class name. This could break back compatibility, and would require cleanup in case of a syntax error in the body. This would be similar to import statements initially putting a empty module in sys.modules to support circular imports. This is messy and still bug prone is use.

I have been thinking about this lately in a different context, and I would very much favor this approach. I think in large part because it works this way for modules it would make sense for it to work for classes as well. The fact that ClassName is bound to an object that will *eventually* become the class as soon as the parser has read in: class ClassName: represents, to me (and I would suspect to many students as well), the least astonishment. I realize it would be a very non-trivial change, however.

...

what students have to write instead:

#-------------bad workaround----------------------------\ class Tree: def __init__(self, left: 'Tree', right: 'Tree'): self.left = left self.right = right

What about:

...

...

class Tree: ... """Forward declaration of Tree type.""" ... class Tree(Tree): ... """Tree implementation.""" ... def __init__(self, left: Tree, right: Tree): ... self.left = left ... self.right = right

A little ugly, and potentially error-prone (but only, I think, in exceptional cases). It's also a decent opportunity to teach something about forward-declaration, which I think is worth knowing about. And I think this makes what's going on clearer than the string-based workaround. I didn't follow every single thread about PEP-484 though and I don't know if, or why this approach to forward-declaration was rejected. Erik

On 25.08.2015 18:01, Prof. Dr. L. Humbert wrote:

1st class pedagogical/didactical thinking … Consider: there are recursive defined ADTs and we want to enable students to understand concepts and produce python-code to realize, what they understood.

The main point: if the students already understood, that it is possible to place type hints to place type hints for arguments and results of functions/methods they should be able to reuse the notation in an orthogonal manner.

I am sorry about going back to this but why not teaching this in a different lesson?

When it comes to recursive ADTs they should be able to write class Tree: def __init__(self, left: Tree, right: Tree): self.left = left self.right = right

I still think, only looking at recursive ADTs, it is enough to for them to write class Tree: def __init__(self, left_tree, right_tree): self.left_tree = left_tree self.right_tree = right_tree This way, you can teach them about code style, proper variable names and so forth. Regards, Sven R. Kunze

On 8/25/2015 12:19 PM, Erik Bray wrote:

On Tue, Aug 25, 2015 at 11:15 AM, Terry Reedy wrote:

...

2. Class statements would initially create an empty class bound to the class name. This could break back compatibility, and would require cleanup in case of a syntax error in the body. This would be similar to import statements initially putting a empty module in sys.modules to support circular imports. This is messy and still bug prone is use.

'in use'.

I have been thinking about this lately in a different context, and I would very much favor this approach. I think in large part because it works this way for modules it would make sense for it to work for classes as well. The fact that ClassName is bound to an object that will *eventually* become the class as soon as the parser has read in:

class ClassName:

represents, to me (and I would suspect to many students as well), the least astonishment.

I realize it would be a very non-trivial change, however.

It might be more useful to have def statements work that way (bind name to blank function object). Then def fac(n, _fac=fac): # less confusing than 'fac=fac' return _fac(n-1)*n if n > 1 else 1 would actually be recursive regardless of external name bindings. But as is the case with modules, exposing incomplete objects easily leads to buggy code, such as def f(n, code=f.__code__): pass

...

...

what students have to write instead:

#-------------bad workaround----------------------------\ class Tree: def __init__(self, left: 'Tree', right: 'Tree'): self.left = left self.right = right

What about:

...

...

...

class Tree: ... """Forward declaration of Tree type.""" ... class Tree(Tree): ... """Tree implementation.""" ... def __init__(self, left: Tree, right: Tree): ... self.left = left ... self.right = right

A little ugly, and potentially error-prone (but only, I think, in exceptional cases). It's also a decent opportunity to teach something about forward-declaration, which I think is worth knowing about. And I think this makes what's going on clearer than the string-based workaround.

I like this better than my decorator version. Notice that if Python were changed so that 'Tree' were bound to a blank class first thing, then Tree(Tree) would be subclassing itself, breaking code like the above unless a special rule was added to remove a class from its list of subclasses. -- Terry Jan Reedy

On Aug 25, 2015, at 09:19, Erik Bray wrote:

...

On Tue, Aug 25, 2015 at 11:15 AM, Terry Reedy wrote: 2. Class statements would initially create an empty class bound to the class name. This could break back compatibility, and would require cleanup in case of a syntax error in the body. This would be similar to import statements initially putting a empty module in sys.modules to support circular imports. This is messy and still bug prone is use.

I have been thinking about this lately in a different context, and I would very much favor this approach. I think in large part because it works this way for modules it would make sense for it to work for classes as well. The fact that ClassName is bound to an object that will *eventually* become the class as soon as the parser has read in:

class ClassName:

represents, to me (and I would suspect to many students as well), the least astonishment.

The problem here is, what if someone writes this: def __init__(self, left: Tree, right: Tree): # something with left.left Or: @classmethod def maketree(cls): return Tree(None, None) Here, Tree is "defined", but the type checker can't actually infer the type of left.left or the arguments of Tree's constructor (even if __init__ was defined before maketree). There are various ways you could special-case things to deal with this problem. The simplest would be that a forward-declared class just has no methods or other attributes, or maybe that it has only the ones inherited from superclasses or metaclasses, until the definition is completed, but my naive intuition says that it's obvious what both of the above mean, and the only reason I'd expect it to be an error is by understanding how it has to work under the covers.

Prof. Dr. L. Humbert writes:

1st class pedagogical/didactical thinking … Consider: there are recursive defined ADTs and we want to enable students to understand concepts and produce python-code to realize, what they understood.

I think we all understand your point. The problem is that Python as designed is simply not capable of that. You have a choice: python-code with its "class statements bind names to (fully-constructed) class objects" semantics, or pseudo-python with "class statements are declarations" semantics. Python simply isn't a declarative language in that sense. I don't have any objection to a language with different semantics, but I find Python's semantics very consistent (except for module import -- which is improved a lot thanks to Brett -- and the occasional class which creates attributes in __getattr__ -- which I dislike for this reason). I doubt Python-Dev will want to give up that consistency; I know I don't want to. Although the PEP doesn't explicitly say it's a good practice, AFAICS using the name of a type (ie, a string) is supported everywhere a type identifier is. (Explicitly permitted are *forward* references to *undefined* types. However, in the "Django" example where "A.a" is the name of a type defined in module A, "B.b" is defined in B, and each uses the other, in import A import B # refers to A.a using the string "A.a" A.a is an existing type. I conclude that unless the implementation is excessively complicated, it's permitted to refer to already defined types using the string name.) In other words, although class Leaf(): def __init__(self, value: int): self.value = value class Tree: # with leaves def __init__(self, left: Union['Tree', Leaf], right: Union['Tree', Leaf]): self.left = left self.right = right is indeed non-orthogonal and ugly, you could declare the constructors def __init__(self, value: 'int'): def __init__(self, left: 'Union[Tree, Leaf]', right: 'Union[Tree, Leaf]'): using actual names of types (strings like "'Tree'") instead of bound names (identifiers like "Tree") everywhere. That may not be quite as clean as you'd like, but it seems orthogonal enough to me: you have a consistent syntax for all type annotations. I'd also point out that your own notation isn't quite orthogonal: self isn't annotated in your method definitions. If students can handle the special syntax for "self", I suppose that they can handle a special syntax for recursively defined types (or you could use Steven d'A's approach of a placeholder class "RecursivelyDefined", which would require augmenting the typechecker to recognize it). Steve

On 8/25/2015 1:48 PM, Sven R. Kunze wrote:

I think the main issue here is the gab between intuition and what the compiler actually does. The following line:

class MyClass: # first appearance of MyClass

basically creates MyClass in the mind of the developer reading this piece of code.

I think the gap is less than you think ;-). Or maybe we think differently when reading code. Both human and compiler create the concept 'MyClass' (properly quoted) as an instance of the concept 'class'. In a static language like C, types are only concepts in the minds of programmers and compilers. There are no runtime char, int, float, or struct xyz objects, only the names or concepts. When the compiler is done, there are only bytes in a sense not true of Python.

Thus, he expects to be able to use it after this line.

One can use the string 'MyClass' in an annotation, for instance, and eventually dereference it to the object after the object is created. A smart type checker could understand that 'MyClass' in annotations within the class MyClass statement means instances of the future MyClass object. A developer should not expect to use not-yet-existent attributes and methods of the object. -- Terry Jan Reedy

On Aug 26, 2015, at 13:22, Sven R. Kunze wrote:

...

On 26.08.2015 07:50, Terry Reedy wrote: A developer should not expect to use not-yet-existent attributes and methods of the object.

Unfortunately, that is where I disagree. The definition of "not-yet-existent attribute" can vary from developer to developer.

But it has to mean something, and what it means dramatically affects what the code does. That's why Python has a simple rule: a straightforward imperative execution order that means you can easily tell whether the attribute was created before use. Also note that in Python, attributes can be added, replaced, and removed later, and their values can be mutated later. So the notion of "later" has to be simple to understand. Just saying "evaluate these statements in some order that's legal" doesn't work when some of those statements can be mutating state. In Python, a statement can create, replace, or destroys attributes of the module or any other object, and even an expression can mutate the values of those attributes. And in fact that's what everything in Python is doing, even declarative-looking statements like class, so you can't just block mutation, you have to deal with it as a fundamental thing. Besides fully compiler-driven evaluation order, there are two obvious alternatives that let you to keep linear order, but make sense of using values before they're created: lazy evaluation, as in Haskell, and dataflow evaluation, as in Oz. Maybe one of those is what you want here. But trying to fit either of those together with mutable objects sensibly is not trivial, nor is it trivial to fit them together with the kind of dynamic OO that Python provides, much less both in one. I'd love to see what someone could come up with by pursuing either of those, but I suspect it wouldn't feel much like Python.

On 8/26/2015 10:43 PM, Steven D'Aprano wrote:

On Tue, Aug 25, 2015 at 07:48:39PM +0200, Sven R. Kunze wrote:

...

I think the main issue here is the gab between intuition and what the compiler actually does. The following line:

class MyClass: # first appearance of MyClass

basically creates MyClass in the mind of the developer reading this piece of code. Thus, he expects to be able to use it after this line.

Intuition according to whom? Some people expect that. Others do not.

People hold all sorts of miscomprehensions and misunderstandings about the languages they use, and Python is no different.

...

However, Python first assigns the class to the name MyClass at the end of the class definition. Thus, it is usable only after that.

People get around this (especially since one doesn't need it thus often), but it still feels... different.

To me, it feels intuitive and natural. Of course you can't use the class until after you have finished creating it. To me, alternatives like Javascript's function hoisting feel weird. This looks like time travel:

// print is provided by the Rhino JS interpreter var x = f(); print(x); // multiple pages later function f() {return "Hello World!";};

How can you call a function that doesn't exist yet? There are even stranger examples, but for the sake of brevity let's just say that what seems "intuitive" to one person may be "weird" to another.

With one or two minor exceptions, the Python interactive interpreter behaves identically to the non-interactive interpreter. If you have valid Python code, you can run it interactively. The same can't be said for Javascript. You can't run the above example interactively without *actual* time travel, if you try, it fails:

[steve@ando ~]$ rhino Rhino 1.7 release 0.7.r2.3.el5_6 2011 05 04 js> var x = f(); js: "<stdin>", line 2: uncaught JavaScript runtime exception: ReferenceError: "f" is not defined. at <stdin>:2

A nice, clean, easy to understand execution model is easy to reason about. Predictability is much more important than convenience: I much prefer code which does what I expect over code that saves me a few characters, or lines, of typing, but surprises me by acting in a way I didn't expect. The fewer special cases I have to learn, the more predictable the language and the less often I am surprised.

Python treats functions and classes as ordinary values bound to ordinary names in the ordinary way: the binding doesn't occur until the statement is executed. I like it that way.

So do I. The same is true of import statements -- the binding of the name to the module does not happen until the module is built. It happens that the import machinery has a cache where is sticks an initially empty module in case of circular imports. But that is normally invisible to the code with the import statement. -- Terry Jan Reedy

Prof. Dr. L. Humbert writes:

It is indeed possible to run the following code / from typing import List class Tree: def __init__(self, left: 'Tree', right: 'Tree'): self.left = left self.right = right def leaves(self) -> List['Tree']: return [] def greeting(name: 'str') -> 'str': return 'Hello ' + name \

but not … def leaves(self) -> 'List'['Tree']: … which would be orthogonal, when deciding to put all used types in '…'

Perhaps there will be a chance to make this a valid construction?

The issue is that Python does not have separate type/value universes. 'Tree' is just a type hint, not a type in the conventional sense. I would very much like if Python *did* have separate types/values, so that one could write (OCaml): class tree (left : tree) (right : tree) = object val left = left val right = right end ;; Which is an uninhabited type, since you need a tree to construct a tree! :) Thus: class tree (left : tree option) (right : tree option) = object val left = left val right = right end ;; Stefan Krah

...

It is indeed possible to run the following code / from typing import List class Tree: def __init__(self, left: 'Tree', right: 'Tree'): self.left = left self.right = right def leaves(self) -> List['Tree']: return [] def greeting(name: 'str') -> 'str': return 'Hello ' + name \

…

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 28.08.2015 15:58, Petr Viktorin wrote: … pv> You can put the entire hint in a string: pv> def leaves(self) -> 'List[Tree]': TNX … solves the problem in this example and makes it orthogonal. Next showstopper will come, when working on/with datastructures, which contains entangled class-structures, perhaps the instantiation of a class, when we have to use self.node = Node(…) but not self.node= 'Node'(…) So I think, we as educators have to live with this pedagogical »suboptimal« solution(s) and have to communicate those non-orthogonal notation and make clear, what the reason is all about. TNX Ludger - -- https://twitter.com/n770 http://ddi.uni-wuppertal.de/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlXgtLMACgkQJQsN9FQ+jJ+4cgCfTaer3dFG4CscmNu/yo4AWxti 0AQAoI9nyTaA2hgDfyCEFk5WdkW1/28L =Wp9U -----END PGP SIGNATURE-----