Improving the expressivity of function annotations
While the PEP 3107 annotations are pretty nice, and an interesting improvements re third-party static analysis of Python code, I think they are insufficient in three areas which makes them… insufficient. And even if third-parties solved these issues independently, it likely would not reach criticality. type parameters (generics) ========================== Python simply has no syntax (at the moment) to express the idea of generics and generic containers. The simplest way (and one I find syntactically clean, YMMV) would be to define ``__getitem__`` on ``type`` instances (with e.g. ``list[int]`` yielding a list with some kind of ``generics`` attribute), but that is of course unavailable to third-party libraries as they should not override builtins. Plus it would have to be added on ``type`` itself to ensure all types have this property (and I am not sure how C types are defined in relation to ``type``). The possibility to perform this call would probably need to be opt-in or opt-out, as generics don't make sense for all types (or maybe all types could accept an empty generic spec e.g. ``int[]``? though I'm not sure that is even valid python syntax) Type parameters greatly increase the expressivity of a type specification and make deep type checks possible, creating much static safety by avoiding implicit or explicit (unsafe) type casts. sum types ========= It is not uncommon for Python functions and methods to take or return a value from a set of types rather than from a type itself. That is also (I would guess) one of the reasons why ``isinstance`` can take a tuple of types rather than being limited to a single type (as with Javascript's or Java's ``instanceof``). As previously, it would probably be easy to alter ``type`` to syntactically support this construct: defining ``__or__`` on ``type`` as yielding some kind of TypeSet (or Types) instance, which would be a set of all OR'ed types (with the right subclass hooks set up so ``isinstance`` works correctly). It would result in something like: Sum = (A | B | C) assert issubclass(A, Sum) assert issubclass(B, Sum) assert issubclass(C, Sum) Structural types ================ The third issue is also the biggest by far. A small portion of it is resolved by abcs (especially if tools include special support for ABCs), but not all of it by a long shot: statically defining and checking "duck types". abcs check for structure, so they are a convenient label for *some* "duck" types but their meaning isn't even well defined (e.g. does the ``item: Mapping`` signature mean ``item`` implements the abstract methods of Mapping, or does it have to *also* implement the mixin methods, even if Mapping was not actually mixed in it?) Anyway this is where "structural types" come in: defining a type not by its name but by its shape (a set of methods and properties, and their signatures). Again, this can be handled by defining (enough) abcs, but it soon becomes tiresome as ABCs are still pretty verbose to write and I don't doubt there would soon be about 5 million versions of ``Readable`` (requirement of a single no-args method ``read`` producing bytes). As a result, the ability to define a type as a set of methods (and their signatures) without having to give it an explicit name would — I think — be great. On the other hand, this is *the* case for which I have no syntactical idea, definitely not a simple one, by simply augmenting existing types with new properties (without introducing actual new syntax). The OCaml syntax ``< method: (*input) -> output; method2: (*input) -> output; property: output >`` definitely does not fit. Your thoughts?
On Sun, Apr 3, 2011 at 10:31 PM, Masklinn <masklinn@masklinn.net> wrote:
It wouldn't be that hard to make a class decorator that would work like: @interface class Duck: def quack(): pass def walk(): pass class Mallard: def quack(): print("quack") def walk(): print("waddle") assert issubclass(Mallard, Duck) #Passes. Where "interface" is a loose term like Go, and anything with the relevant methods counts as a subclass. It would probably make sense to add something like that to the abc module.
On 2011-04-04, at 10:44 , Carl M. Johnson wrote:
The "issue" is roughly the same as for abcs, in that you still have to give a name (and pre-define a type) for something which may not warrant it, and this naming may even serve to muddy the water some cases when used in type signatures (see Mapping abc example) Otherwise, sure.
On Sun, Apr 3, 2011 at 10:49 PM, Masklinn <masklinn@masklinn.net> wrote:
The "issue" is roughly the same as for abcs, in that you still have to give a name (and pre-define a type) for something which may not warrant it, and this naming may even serve to muddy the water some cases when used in type signatures (see Mapping abc example)
Otherwise, sure.
I guess I'm not getting it then. What's the alternative? We can either a) make up an ABC ourselves b) have a prefilled library of ABCs and pick one off the shelf or c) not use ABCs at all. In the case that something doesn't "warrant" an ABC, then just pick c. In fact, some people believe that we should always pick c to be more Pythonic and duck type-ish. Or am I missing an option? I suppose there could be a factory of some sort to make new ABCs based on existing ABCs already in some sort of library… Is that what you're proposing?
Again, this can be handled by defining (enough) abcs, but it soon becomes
OK, reading this again, maybe you mean something more like a Java interface? @interface class Writer: def write(output: bytes) -> None: "Takes in bytes, returns None. Raises Foo on error." pass class DuckWriter(Writer): def write(output): #do something return 1 d = DuckWriter() d.write("hello") #Raises TypeError, output must be bytes d.write(b"hello") #Raises TypeError, return type must be NoneType
On Sun, Apr 3, 2011 at 11:09 PM, Carl M. Johnson < cmjohnson.mailinglist@gmail.com> wrote:
OK, reading this again, maybe you mean something more like a Java interface?
Thinking about it a little more (It's bedtime in my timezone! That's my excuse!), I think there are two separate things that might be nice to have. One is an easy way to declare Go-style duck typing interfaces, the other is an easy way to declare Java-style type checking interfaces. I suppose there could be two class decorators. Call the first one "interface" and the second one "typechecking". Interface makes things reply to issubclass with True so long as they have the same methods and parameters. Typechecking creates a new class that will raise an error if anything trying to subclass it fails to use the proper types for input and output. Do those sounds like useful decorators/metaclasses to have?
On 2011-04-04, at 11:02 , Carl M. Johnson wrote:
The problem of option C and not using ABCs (which is indeed what I'm interested in) is that you then lose the possibility to type the object. If you're not using ABCs and you're not using concrete nominative types (e.g. ``dict``, ``list`` or a custom type) what are you left with? As far as I can tell, nothing, and that's the hole I think needs plugging in the third part of the initial mail. On 2011-04-04, at 11:09 , Carl M. Johnson wrote:
Yes in that the signature of the methods would be part of the type, no in that the "link" should be implicit (structural) not explicit (nominative). Your example does not show anything as it would be a function annotation (not a type annotation), but making it write to a file-like object would work: what I'd like is something along those lines (ignore the notation for now) class Foo: def write_to(stream: < write(bytes) -> None >): # stuff class StringWriter: def write(output: str) -> None # stuff class BytesWriter: def write(output: bytes) -> None # stuff d = DuckWriter() d.write_to(StringWriter()) -> TypeError, not a structural subtype of what write_to needs d.write_to(BytesWriter()) -> OK On 2011-04-04, at 11:17 , Carl M. Johnson wrote:
Maybe, but that's not the scope I'm considering at all, I'm only talking about function annotations.
On 4 April 2011 10:56, Masklinn <masklinn@masklinn.net> wrote:
I'm not at all sure I follow your intent or terminology (structural vs nominative) here - perhaps you could provide a full (but short!) example of what you're trying to do. The above is close, but names like "Foo" and hiding actual implementations behind "# stuff" is confusing me. If I follow what you're after, surely the #stuff in StringWriter would naturally raise a TypeError when you tried to use a bytes object as if it were a string? That's essentially what duck typing is about. If you're trying to enforce stricter typing (closer to the call site, the nearest Python can get to "compile time" type checking) then why are you trying to avoid ABCs? Isn't that what they are for? If what you are after is not (in some sense) a Pythonic version of static type checking, then I apologise for misunderstanding, but could you clarify? Paul.
On 2011-04-04, at 12:34 , Paul Moore wrote:
Well thing is I'm hiding the implementation because it's not relevant to my proposal, which is about the function annotations and what they express (actual static checking being supposed to be implemented by third-party tools). As to structural v nominative: * Nominative typing is typing based on names, as in Java for instance: a type and a name are the same thing, subtypes explicitly inherit(/extend) their super types by name. In Python, this is what you do when you use isinstance (without using ABCs anyway): you check that an object has a type of a given name. * Structural typing is basically static duck typing: you ignore names, and instead you check for "shapes". Types are seen as sets of methods (a method being a triplet of a name, an input type and an output type). So instead of checking that an object has the right type-name, you check that it has the right methods. And you can do that statically.
And as I mentioned there's the issue of the ABC's scope: are mixin methods included in the requirements to be a subclass/instance of ABCs or not? If not, why not? If yes, doesn't that make ABCs much less useful for a huge bunch of objects in the wild? And third, there's the issue that encoding all subsets of existing concrete types in abcs yields an exponential explosion in cases. While not all cases are used by a long short consider the mere concept of "file-like object": some will iterate it, some will read() it, some will readline() or readlines() it, should they interact with bytes or with strings, does the file-like object need to support seek() or not, etc… These are all questions which would not be solved easily by a single ABC.
On 4 April 2011 11:58, Masklinn <masklinn@masklinn.net> wrote:
OK. I understand now, thanks for the clarification. I'm -1 on this. Static typing is not appropriate for Python, IMO. Just as one example, note that even ABCs are not statically deterministic. Consider ABCMeta.register and ABCMeta.__subclasshook__. If a tool implements any form of static type checking, it will by definition fail to handle a certain proportion of valid Python programs. Whether this is acceptable is, of course, a personal decision. Paul.
On 2011-04-04, at 14:48 , Paul Moore wrote:
Just as one example, note that even ABCs are not statically deterministic. Consider ABCMeta.register and ABCMeta.__subclasshook__.
ABCs can be interpreted as structural type tags, making ABCMeta.register and ABCMeta.__subclasshook__ ignorable at the cost of potential false positives (a type with the right structure would pass the static type check even if it didn't pass an issubclass check). I'm more than willing to live with that.
And the vast majority of static type systems have loopholes, Python just has far more than others. On 2011-04-04, at 15:06 , Nick Coghlan wrote:
On Mon, Apr 4, 2011 at 11:33 PM, Masklinn <masklinn@masklinn.net> wrote:
They're not useful for comprehensive static typing without defining additional conventions. But that's OK, since that's not their purpose - at the language level, they're just a place to associate arbitrary data with parameters and the function as a whole, typically for use by specific decorators that are also applied at function definition time. Which annotations you use and which decorators you use will accordingly be task specific. (For example, here's a simple one that uses function annotations to define relatively type safe calls out to C functions via ctypes: http://code.activestate.com/recipes/576731-c-function-decorator/) As it stands, there's absolutely nothing stopping you implementing the suggestions in your original post as an external library (albeit without being able to use infix syntax on the builtin types, but using tuples instead should work just fine for the sum case, and you can define your own Generic ABCs for the generics case). As far as the structural types go, just use function annotations and appropriate decorators on the methods in an ABC definition (although you would want your own new metaclass to enforce the additional rules). As far as the language itself goes, however, we made a deliberate decision not to ascribe any specific semantics to function annotations, and nothing has happened in the intervening time to convince us to go back on that decision. It would require an annotations-based approach to a task to prove incredibly popular for us to bless for it inclusion in the standard library. If anything was going to happen on that front, my money would actually be on some utilities in ctypes, such as the cookbook recipe I linked above. This is *not* an area where theoretical arguments are going to hold any water: only practical solutions that help make code dealing with real-world problems significantly more readable. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
If what you want is static typing, perhaps using a statically typed language would be a good place to start? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan wrote:
If what you want is static typing, perhaps using a statically typed language would be a good place to start?
You might be interested in Cobra, which is a Python-influenced language with optional static typing. http://cobra-language.com/ Comparison to Python: http://cobra-language.com/docs/python/ -- Steven
I am not sure, what, if anything, we should do to make annotations more useful. I think it is too soon to do anything *official* yet. On 4/4/2011 4:31 AM, Masklinn wrote:
type parameters (generics)
I do not know what 'generics' means (and neither, that I could find, does Wikipedia) but type-parameter is not obvious as a synonym ;-).
``type`` instances (with e.g. ``list[int]`` yielding a list with some kind of ``generics`` attribute),
That sounds a lot like array with a typecode. You can use a dict subclass or UserDict to get a mapping with restricted keys and values, with bad inputs ignored, coerced, or error raising as you please. Ditto for lists.
Sum = (A | B | C) assert issubclass(A, Sum)
That appears to be exactly the same as Sum = (A,B,C) assert issubclass(A, Sum) Sum2=Sum|D # same as Sum2=Sum+(D,) so the purpose of the duplication is not obvious.
When it comes to tracebacks, multiple unbound functions with duplicate f.__name__ == '<lambda>' attributes are inferior to functions with, in a particular context, unique names. I would feel the same about multiple stypes with .__name__ == '<stype>'. -- Terry Jan Reedy
On Mon, Apr 4, 2011 at 11:22 AM, Terry Reedy <tjreedy@udel.edu> wrote:
I am not sure, what, if anything, we should do to make annotations more useful. I think it is too soon to do anything *official* yet.
For sure it is too soon -- the intention was that the new syntax would enable 3rd party experiments. We're still waiting for reports on such experiments. -- --Guido van Rossum (python.org/~guido)
On 4 April 2011 19:22, Terry Reedy <tjreedy@udel.edu> wrote:
http://en.wikipedia.org/wiki/Generic_programming It's a standard term for languages like C# and Java, but if you don't use these languages there is no reason you should know it. Generics is a solution (hack - but still an elegant hack) that allows you to write "generic" containers and functions that can work with any types whilst still being type safe. For example, in C# you might have a list type that can contain elements of a specific type. The list can be written once using generics, then can be used with any type - with the specific type being specified (the type parameter) at compile time where you use it. E.g. var list_of_ints = new List[int](); HTH, Michael Foord
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On 4 April 2011 20:04, Masklinn <masklinn@masklinn.net> wrote:
Because with a good dynamic type system they are completely unnecessary. Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On 2011-04-04, at 21:05 , Michael Foord wrote:
If you go that way, types themselves are unnecessary "and therefore hacks", static or not. I don't think that makes much sense, though I can see you were probably replying in jest I was interested in the answer.
On 4 April 2011 20:30, Masklinn <masklinn@masklinn.net> wrote:
I wasn't entirely joking, and no a dynamic type system doesn't make types themselves redundant - just the declaration of them all over the place (and often multiple times for the same use in languages like Java and C#). Generics are a hack within the language syntax to tell the compiler that different types *might* be used (and allow you to refer to these types in your implementation without knowing what they will be), whereas a smarter compiler could deduce this for itself anyway. All the best, Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On Mon, 4 Apr 2011 20:47:45 +0100 Michael Foord <fuzzyman@gmail.com> wrote:
A good implementation of static types will do that for you as well.
Generics (in the named languages) feel like a hack because, well, they were hacked onto a type system (inherited from C) that didn't allow for such things. If you design your type system ab initio expecting that you're going to allow different types to appear in some places - so long as they satisfy the appropriate conditions - then what you have may look like generics, but they aren't such hacks. It doesn't matter whether the type system is dynamic (conditions checked at run time) or static (conditions checked at compile time). <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
Michael Foord wrote:
You seem to be conflating the idea of generic types, aka parametric types, with the particular syntax used to express them in certain languages. Some of the syntaxes do seem rather hackish, particularly in C++. But I don't think the concept itself is hackish at all. -- Greg
On 2011-04-04, at 21:47 , Michael Foord wrote:
If you mean that Java and C#'s suck I have no problem with that, but I still don't see how that ends up yielding "generics are unnecessary". Even if types are inferred, they're still there and still parametric.
whereas a smarter compiler could deduce this for itself anyway. I don't think that isn't possible (for all cases) for a non-total language unless the compiler can solve the halting problem. But I might be mistaken.
On 2011-04-04, at 20:22 , Terry Reedy wrote:
It's similar, but extended to any arbitrary type, including non-collection types.
You can use a dict subclass or UserDict to get a mapping with restricted keys and values, with bad inputs ignored, coerced, or error raising as you please. Ditto for lists. That is not very useful for function annotations, especially when using built-in collections. Users would be restricted to non-built-in types and static systems still would not have any information on what could and could not go into e.g. collections.
On Mon, Apr 4, 2011 at 9:03 AM, Masklinn <masklinn@masklinn.net> wrote:
Again, the issue is function annotations, which was my context. The example was just to give an idea of the operation performed. In function annotations, unless you went with type parameters and ``tuple[A, B, C]``, ``(A, B, C)`` would likely be interpreted as "a triple of instances of types A, B and C".
Yes, but since function annotation don't do anything at present (and maybe never will by default), your function will have to use a decorator to turn on the type checking. At that point, it's just a matter of API design how best to do it. For example, let's say I have a sum function, and I want it to take an iterator containing some type and then return something of the same type. In semi-C-ish style, we'd call this list[int] -> int or list[Generic number] -> Generic number. But since we're making up our own API for type checking, in this case we can make it work however we want. We don't need to have a particular grammar baked into the language. Try something like: from type_checking_annotations import * g = Generic() @typechecked def sum(iterable: Container(g)) -> g: iterator = iter(iterable) total = next(iterator) for item in iterator: total += item return total sum([0, 0.0]) #Raise TypeError: First item in container was type int, subsequent item was type float
On 2011-04-05, at 02:44 , Carl M. Johnson wrote:
Indeed, function annotations do currently provide a common (very bare) starting point which was found good enough (at least as an experiment) to replace/supersede/complement the thousand of possible docstring styles of function annotations (from annotations-like to Sphinx info-fields through doxygen, epydoc and dozens of ad-hoc schemes). The root of my proposal is that, after using both statically typed and dynamically typed languages and writing a bunch of API docs (mostly for Python and Javascript), I don't think the current function annotations provide enough ground for language users to stand on, be it as documentation or as meta-information for static checking tools.
On Tue, Apr 5, 2011 at 4:35 PM, Masklinn <masklinn@masklinn.net> wrote:
I would still think that, for the usage of function annotation to take off (and be used by multiple tools), there needs to be some kind of commonality between all tools. At least for the features closest to core.
You're free to think that. We disagree, as is explicitly documented in PEP 3107: "Function annotations are nothing more than a way of associating arbitrary Python expressions with various parts of a function at compile-time." Function annotations, on their own, mean absolutely nothing. They only acquire meaning when associated with a specific consumer of those annotations. If a decorator takes a lot of arguments about how to handle particular parameters (or a function's return value), then it is a prime candidate for refactoring to be annotation based instead. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, Apr 3, 2011 at 10:31 PM, Masklinn <masklinn@masklinn.net> wrote:
It wouldn't be that hard to make a class decorator that would work like: @interface class Duck: def quack(): pass def walk(): pass class Mallard: def quack(): print("quack") def walk(): print("waddle") assert issubclass(Mallard, Duck) #Passes. Where "interface" is a loose term like Go, and anything with the relevant methods counts as a subclass. It would probably make sense to add something like that to the abc module.
On 2011-04-04, at 10:44 , Carl M. Johnson wrote:
The "issue" is roughly the same as for abcs, in that you still have to give a name (and pre-define a type) for something which may not warrant it, and this naming may even serve to muddy the water some cases when used in type signatures (see Mapping abc example) Otherwise, sure.
On Sun, Apr 3, 2011 at 10:49 PM, Masklinn <masklinn@masklinn.net> wrote:
The "issue" is roughly the same as for abcs, in that you still have to give a name (and pre-define a type) for something which may not warrant it, and this naming may even serve to muddy the water some cases when used in type signatures (see Mapping abc example)
Otherwise, sure.
I guess I'm not getting it then. What's the alternative? We can either a) make up an ABC ourselves b) have a prefilled library of ABCs and pick one off the shelf or c) not use ABCs at all. In the case that something doesn't "warrant" an ABC, then just pick c. In fact, some people believe that we should always pick c to be more Pythonic and duck type-ish. Or am I missing an option? I suppose there could be a factory of some sort to make new ABCs based on existing ABCs already in some sort of library… Is that what you're proposing?
Again, this can be handled by defining (enough) abcs, but it soon becomes
OK, reading this again, maybe you mean something more like a Java interface? @interface class Writer: def write(output: bytes) -> None: "Takes in bytes, returns None. Raises Foo on error." pass class DuckWriter(Writer): def write(output): #do something return 1 d = DuckWriter() d.write("hello") #Raises TypeError, output must be bytes d.write(b"hello") #Raises TypeError, return type must be NoneType
On Sun, Apr 3, 2011 at 11:09 PM, Carl M. Johnson < cmjohnson.mailinglist@gmail.com> wrote:
OK, reading this again, maybe you mean something more like a Java interface?
Thinking about it a little more (It's bedtime in my timezone! That's my excuse!), I think there are two separate things that might be nice to have. One is an easy way to declare Go-style duck typing interfaces, the other is an easy way to declare Java-style type checking interfaces. I suppose there could be two class decorators. Call the first one "interface" and the second one "typechecking". Interface makes things reply to issubclass with True so long as they have the same methods and parameters. Typechecking creates a new class that will raise an error if anything trying to subclass it fails to use the proper types for input and output. Do those sounds like useful decorators/metaclasses to have?
On 2011-04-04, at 11:02 , Carl M. Johnson wrote:
The problem of option C and not using ABCs (which is indeed what I'm interested in) is that you then lose the possibility to type the object. If you're not using ABCs and you're not using concrete nominative types (e.g. ``dict``, ``list`` or a custom type) what are you left with? As far as I can tell, nothing, and that's the hole I think needs plugging in the third part of the initial mail. On 2011-04-04, at 11:09 , Carl M. Johnson wrote:
Yes in that the signature of the methods would be part of the type, no in that the "link" should be implicit (structural) not explicit (nominative). Your example does not show anything as it would be a function annotation (not a type annotation), but making it write to a file-like object would work: what I'd like is something along those lines (ignore the notation for now) class Foo: def write_to(stream: < write(bytes) -> None >): # stuff class StringWriter: def write(output: str) -> None # stuff class BytesWriter: def write(output: bytes) -> None # stuff d = DuckWriter() d.write_to(StringWriter()) -> TypeError, not a structural subtype of what write_to needs d.write_to(BytesWriter()) -> OK On 2011-04-04, at 11:17 , Carl M. Johnson wrote:
Maybe, but that's not the scope I'm considering at all, I'm only talking about function annotations.
On 4 April 2011 10:56, Masklinn <masklinn@masklinn.net> wrote:
I'm not at all sure I follow your intent or terminology (structural vs nominative) here - perhaps you could provide a full (but short!) example of what you're trying to do. The above is close, but names like "Foo" and hiding actual implementations behind "# stuff" is confusing me. If I follow what you're after, surely the #stuff in StringWriter would naturally raise a TypeError when you tried to use a bytes object as if it were a string? That's essentially what duck typing is about. If you're trying to enforce stricter typing (closer to the call site, the nearest Python can get to "compile time" type checking) then why are you trying to avoid ABCs? Isn't that what they are for? If what you are after is not (in some sense) a Pythonic version of static type checking, then I apologise for misunderstanding, but could you clarify? Paul.
On 2011-04-04, at 12:34 , Paul Moore wrote:
Well thing is I'm hiding the implementation because it's not relevant to my proposal, which is about the function annotations and what they express (actual static checking being supposed to be implemented by third-party tools). As to structural v nominative: * Nominative typing is typing based on names, as in Java for instance: a type and a name are the same thing, subtypes explicitly inherit(/extend) their super types by name. In Python, this is what you do when you use isinstance (without using ABCs anyway): you check that an object has a type of a given name. * Structural typing is basically static duck typing: you ignore names, and instead you check for "shapes". Types are seen as sets of methods (a method being a triplet of a name, an input type and an output type). So instead of checking that an object has the right type-name, you check that it has the right methods. And you can do that statically.
And as I mentioned there's the issue of the ABC's scope: are mixin methods included in the requirements to be a subclass/instance of ABCs or not? If not, why not? If yes, doesn't that make ABCs much less useful for a huge bunch of objects in the wild? And third, there's the issue that encoding all subsets of existing concrete types in abcs yields an exponential explosion in cases. While not all cases are used by a long short consider the mere concept of "file-like object": some will iterate it, some will read() it, some will readline() or readlines() it, should they interact with bytes or with strings, does the file-like object need to support seek() or not, etc… These are all questions which would not be solved easily by a single ABC.
On 4 April 2011 11:58, Masklinn <masklinn@masklinn.net> wrote:
OK. I understand now, thanks for the clarification. I'm -1 on this. Static typing is not appropriate for Python, IMO. Just as one example, note that even ABCs are not statically deterministic. Consider ABCMeta.register and ABCMeta.__subclasshook__. If a tool implements any form of static type checking, it will by definition fail to handle a certain proportion of valid Python programs. Whether this is acceptable is, of course, a personal decision. Paul.
On 2011-04-04, at 14:48 , Paul Moore wrote:
Just as one example, note that even ABCs are not statically deterministic. Consider ABCMeta.register and ABCMeta.__subclasshook__.
ABCs can be interpreted as structural type tags, making ABCMeta.register and ABCMeta.__subclasshook__ ignorable at the cost of potential false positives (a type with the right structure would pass the static type check even if it didn't pass an issubclass check). I'm more than willing to live with that.
And the vast majority of static type systems have loopholes, Python just has far more than others. On 2011-04-04, at 15:06 , Nick Coghlan wrote:
On Mon, Apr 4, 2011 at 11:33 PM, Masklinn <masklinn@masklinn.net> wrote:
They're not useful for comprehensive static typing without defining additional conventions. But that's OK, since that's not their purpose - at the language level, they're just a place to associate arbitrary data with parameters and the function as a whole, typically for use by specific decorators that are also applied at function definition time. Which annotations you use and which decorators you use will accordingly be task specific. (For example, here's a simple one that uses function annotations to define relatively type safe calls out to C functions via ctypes: http://code.activestate.com/recipes/576731-c-function-decorator/) As it stands, there's absolutely nothing stopping you implementing the suggestions in your original post as an external library (albeit without being able to use infix syntax on the builtin types, but using tuples instead should work just fine for the sum case, and you can define your own Generic ABCs for the generics case). As far as the structural types go, just use function annotations and appropriate decorators on the methods in an ABC definition (although you would want your own new metaclass to enforce the additional rules). As far as the language itself goes, however, we made a deliberate decision not to ascribe any specific semantics to function annotations, and nothing has happened in the intervening time to convince us to go back on that decision. It would require an annotations-based approach to a task to prove incredibly popular for us to bless for it inclusion in the standard library. If anything was going to happen on that front, my money would actually be on some utilities in ctypes, such as the cookbook recipe I linked above. This is *not* an area where theoretical arguments are going to hold any water: only practical solutions that help make code dealing with real-world problems significantly more readable. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
If what you want is static typing, perhaps using a statically typed language would be a good place to start? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan wrote:
If what you want is static typing, perhaps using a statically typed language would be a good place to start?
You might be interested in Cobra, which is a Python-influenced language with optional static typing. http://cobra-language.com/ Comparison to Python: http://cobra-language.com/docs/python/ -- Steven
I am not sure, what, if anything, we should do to make annotations more useful. I think it is too soon to do anything *official* yet. On 4/4/2011 4:31 AM, Masklinn wrote:
type parameters (generics)
I do not know what 'generics' means (and neither, that I could find, does Wikipedia) but type-parameter is not obvious as a synonym ;-).
``type`` instances (with e.g. ``list[int]`` yielding a list with some kind of ``generics`` attribute),
That sounds a lot like array with a typecode. You can use a dict subclass or UserDict to get a mapping with restricted keys and values, with bad inputs ignored, coerced, or error raising as you please. Ditto for lists.
Sum = (A | B | C) assert issubclass(A, Sum)
That appears to be exactly the same as Sum = (A,B,C) assert issubclass(A, Sum) Sum2=Sum|D # same as Sum2=Sum+(D,) so the purpose of the duplication is not obvious.
When it comes to tracebacks, multiple unbound functions with duplicate f.__name__ == '<lambda>' attributes are inferior to functions with, in a particular context, unique names. I would feel the same about multiple stypes with .__name__ == '<stype>'. -- Terry Jan Reedy
On Mon, Apr 4, 2011 at 11:22 AM, Terry Reedy <tjreedy@udel.edu> wrote:
I am not sure, what, if anything, we should do to make annotations more useful. I think it is too soon to do anything *official* yet.
For sure it is too soon -- the intention was that the new syntax would enable 3rd party experiments. We're still waiting for reports on such experiments. -- --Guido van Rossum (python.org/~guido)
On 4 April 2011 19:22, Terry Reedy <tjreedy@udel.edu> wrote:
http://en.wikipedia.org/wiki/Generic_programming It's a standard term for languages like C# and Java, but if you don't use these languages there is no reason you should know it. Generics is a solution (hack - but still an elegant hack) that allows you to write "generic" containers and functions that can work with any types whilst still being type safe. For example, in C# you might have a list type that can contain elements of a specific type. The list can be written once using generics, then can be used with any type - with the specific type being specified (the type parameter) at compile time where you use it. E.g. var list_of_ints = new List[int](); HTH, Michael Foord
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On 4 April 2011 20:04, Masklinn <masklinn@masklinn.net> wrote:
Because with a good dynamic type system they are completely unnecessary. Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On 2011-04-04, at 21:05 , Michael Foord wrote:
If you go that way, types themselves are unnecessary "and therefore hacks", static or not. I don't think that makes much sense, though I can see you were probably replying in jest I was interested in the answer.
On 4 April 2011 20:30, Masklinn <masklinn@masklinn.net> wrote:
I wasn't entirely joking, and no a dynamic type system doesn't make types themselves redundant - just the declaration of them all over the place (and often multiple times for the same use in languages like Java and C#). Generics are a hack within the language syntax to tell the compiler that different types *might* be used (and allow you to refer to these types in your implementation without knowing what they will be), whereas a smarter compiler could deduce this for itself anyway. All the best, Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On Mon, 4 Apr 2011 20:47:45 +0100 Michael Foord <fuzzyman@gmail.com> wrote:
A good implementation of static types will do that for you as well.
Generics (in the named languages) feel like a hack because, well, they were hacked onto a type system (inherited from C) that didn't allow for such things. If you design your type system ab initio expecting that you're going to allow different types to appear in some places - so long as they satisfy the appropriate conditions - then what you have may look like generics, but they aren't such hacks. It doesn't matter whether the type system is dynamic (conditions checked at run time) or static (conditions checked at compile time). <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
Michael Foord wrote:
You seem to be conflating the idea of generic types, aka parametric types, with the particular syntax used to express them in certain languages. Some of the syntaxes do seem rather hackish, particularly in C++. But I don't think the concept itself is hackish at all. -- Greg
On 2011-04-04, at 21:47 , Michael Foord wrote:
If you mean that Java and C#'s suck I have no problem with that, but I still don't see how that ends up yielding "generics are unnecessary". Even if types are inferred, they're still there and still parametric.
whereas a smarter compiler could deduce this for itself anyway. I don't think that isn't possible (for all cases) for a non-total language unless the compiler can solve the halting problem. But I might be mistaken.
On 2011-04-04, at 20:22 , Terry Reedy wrote:
It's similar, but extended to any arbitrary type, including non-collection types.
You can use a dict subclass or UserDict to get a mapping with restricted keys and values, with bad inputs ignored, coerced, or error raising as you please. Ditto for lists. That is not very useful for function annotations, especially when using built-in collections. Users would be restricted to non-built-in types and static systems still would not have any information on what could and could not go into e.g. collections.
On Mon, Apr 4, 2011 at 9:03 AM, Masklinn <masklinn@masklinn.net> wrote:
Again, the issue is function annotations, which was my context. The example was just to give an idea of the operation performed. In function annotations, unless you went with type parameters and ``tuple[A, B, C]``, ``(A, B, C)`` would likely be interpreted as "a triple of instances of types A, B and C".
Yes, but since function annotation don't do anything at present (and maybe never will by default), your function will have to use a decorator to turn on the type checking. At that point, it's just a matter of API design how best to do it. For example, let's say I have a sum function, and I want it to take an iterator containing some type and then return something of the same type. In semi-C-ish style, we'd call this list[int] -> int or list[Generic number] -> Generic number. But since we're making up our own API for type checking, in this case we can make it work however we want. We don't need to have a particular grammar baked into the language. Try something like: from type_checking_annotations import * g = Generic() @typechecked def sum(iterable: Container(g)) -> g: iterator = iter(iterable) total = next(iterator) for item in iterator: total += item return total sum([0, 0.0]) #Raise TypeError: First item in container was type int, subsequent item was type float
On 2011-04-05, at 02:44 , Carl M. Johnson wrote:
Indeed, function annotations do currently provide a common (very bare) starting point which was found good enough (at least as an experiment) to replace/supersede/complement the thousand of possible docstring styles of function annotations (from annotations-like to Sphinx info-fields through doxygen, epydoc and dozens of ad-hoc schemes). The root of my proposal is that, after using both statically typed and dynamically typed languages and writing a bunch of API docs (mostly for Python and Javascript), I don't think the current function annotations provide enough ground for language users to stand on, be it as documentation or as meta-information for static checking tools.
On Tue, Apr 5, 2011 at 4:35 PM, Masklinn <masklinn@masklinn.net> wrote:
I would still think that, for the usage of function annotation to take off (and be used by multiple tools), there needs to be some kind of commonality between all tools. At least for the features closest to core.
You're free to think that. We disagree, as is explicitly documented in PEP 3107: "Function annotations are nothing more than a way of associating arbitrary Python expressions with various parts of a function at compile-time." Function annotations, on their own, mean absolutely nothing. They only acquire meaning when associated with a specific consumer of those annotations. If a decorator takes a lot of arguments about how to handle particular parameters (or a function's return value), then it is a prime candidate for refactoring to be annotation based instead. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (10)
-
Carl M. Johnson -
Greg Ewing -
Guido van Rossum -
Masklinn -
Michael Foord -
Mike Meyer -
Nick Coghlan -
Paul Moore -
Steven D'Aprano -
Terry Reedy