[Python-ideas] PEP 484 (Type Hints) -- first draft round

[Cem Karan]

On Jan 18, 2015, at 2:34 PM, Guido van Rossum <guido at python.org> wrote:

> On Sun, Jan 18, 2015 at 5:37 AM, Cem Karan <cfkaran2 at gmail.com> wrote:
> 
>> On Jan 17, 2015, at 9:27 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>> 
>> > Cem Karan <cfkaran2 at gmail.com> writes:
>> >
>> >> […] I feel like annotations have the capability to be MUCH more useful
>> >> than just as a place to hold type hints.
>> >
>> > They have that capability. How long do you propose we wait for this
>> > capability to be realised?
>> >
>> > Function annotations have been part of Python since version 3.0, and
>> > Guido (IIUC) is deciding that the wait is over: we already have the best
>> > use of function annotations we're going to get.
>> 
>> I understand what you're saying, and you're right that they are not used much.  I only learned about them after I started playing with Python 3.3.  But then I'm the bleeding edge for my group at work; everyone else is down in the python 2 world somewhere (some of them are WAY down there somewhere).
>> 
>> Changing a language takes time.  Do you program in C?  Do you still use pthreads? Why? C11 specifies the thread.h header, you should switch to C11 threads right now!  Except... many compilers still don't fully support C11, so we're coding to an earlier standard.  There is a lot of code that already works as it is. Etc., etc. etc...
>> 
>> When a language changes, there is a LOT of catching up to do.  First, tools need to support the new facilities and then programmers need to become aware of new facilities, and they have to realize there is a legitimate use for those facilities.  Most people I know are unaware that annotations exist; that is an argument for making the change to supporting typing only, simply because there will be fewer people that are affected.  At the same time, if someone has a really good idea that could be layered into annotations WELL, they will be hampered by a proposal that locks them out.  If typing is done via a Typing class like I mentioned in my last email, then there is a very simple way of determining if we're looking at a type or at something else.  There may be other, better ways of layering it in that don't lock other uses out, and which are better suited for static analysis.
>> 
>> I'm just asking for a very simple, very clear method of distinguishing uses when parsing code.  Types can get complicated quickly, and ad-hoc rules to determine if we're looking at a type, or at something else, can get us in trouble.  Simply turning it on or off for a chunk of code also sucks; I LIKE types, so I'd like to put them into my code, but if I have a way of adding documentation to my code as well, then I'd like to do that in addition to the types.  There may be other uses of annotations that programmers haven't yet thought of, simply because they don't know that annotations exist.  Locking out those uses would suck.
>> 
>> Does all this make sense?
> 
> I hear you, and I'm not buying it. I've thought about this a lot (more than you, I'm sure) and I'm putting a line in the sand: annotations are for types, and all other uses will eventually have to find some other way (most likely decorators). (The "type: OFF" comments are a transitional measure to avoid breaking other uses without warning in 3.5.)

As the BDFL, that is your right.  But I would personally rather see this:

@functools.lru_cache()
def foo(a: {Type(int), Doc("some doc"), Something_else(4,5,6)}):
    """
    Some block of documentation for the function as a whole that I'm too lazy 
    to write out,  so I'm just going to copy in a paragraph of 
    `Lorem ipsum <http://en.wikipedia.org/wiki/Ipse_lorum>`_ to add as filler
    to break up the mental flow a bit.

    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
    tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim
    veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea
    commodo consequat. Duis aute irure dolor in reprehenderit in voluptate
    velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat
    cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id
    est laborum. 
    """ 
    pass

Then something like this:

@functools.lru_cache()
@Something_else(a, 4, 5, 6)
def foo(a: int):
    """
    Some block of documentation for the function as a whole that I'm too lazy 
    to write out,  so I'm just going to copy in a paragraph of 
    `Lorem ipsum <http://en.wikipedia.org/wiki/Ipse_lorum>`_ to add as filler
    to break up the mental flow a bit.

    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
    tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim
    veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea
    commodo consequat. Duis aute irure dolor in reprehenderit in voluptate
    velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat
    cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id
    est laborum.

    :param a: some doc
    """
    pass

Finally, consider what decorators are NOT required to do; they are not required to return the original object.  That means that ordering can potentially matter:

@Doc(a, "some doc")  # Documenting the argument of the wrapped function
@functools.lru_cache()  # Creating a new function that wraps the original function
@Something_else(a, 4, 5, 6)  # Doing something to the argument of the original function
def foo(a: int):
    pass

versus

@functools.lru_cache()  # Wrapping the function
@Something_else(a, 4, 5, 6)  # Doing something to the argument of the original function
@Doc(a, "some doc")  # Documenting the argument of the function itself.
def foo(a: int):
    pass

So what do automatic documentation generators do in this case?  More than likely, we want the second case, which means that someone needs to tell programmers what order to use the decorators in.  What if Something_else returns a wrapped version of the function?  You need to do this:

@functools.lru_cache()  # Wrapping the modified function
@Something_else(a, 4, 5, 6)  # Doing something to the argument of the modified function
@Something_else(b, 4, 5, 6)  # Doing something to the argument of the original function
@Doc(a, "some doc")  # Documenting the argument of the function itself.
@Doc(b, "some other doc")  # Documenting the argument of the function itself.
def foo(a: int, b: str):
    pass

instead of this:

@functools.lru_cache()  # Wrapping the modified, modified function
@Doc(a, "some doc")  # Documenting the argument of the modified version of the modified function.
@Something_else(a, 4, 5, 6)  # Doing something to the argument of the modified function
@Doc(b, "some other doc")  # Documenting the argument of the modified function.
@Something_else(b, 4, 5, 6)  # Doing something to the argument of the original function
def foo(a: int, b: str):
    pass

This starts to feel unnatural, and is definitely confusing.  We can then put in place a whole bunch of rules regarding decorators, but then we've got two problems; forcing annotations to be for types only, and forcing new rules on decorators that weren't there in the first place.

I know that something like "def foo(a: {Type(int), Doc("some doc"), Something_else(4,5,6)})" isn't pretty.  But it is order-independent, it operates on the original function only, and it is pretty easy to parse, even for a static analyzer, AND there is no question that Type() means you want a type check done.  Finally, annotations will still be open for other uses.

Thanks,
Cem Karan