TypeHinting: From variable name to type
Imaging you are developing in the django context.
Everytime you use a variable named "request" or "response" your human brains knows that this is a subclass of django.http.HttpRequest and django.http.HttpResponse.
How to give the IDE this knowledge?
One solution is the do typehinting everywhere the veriable gets used.
But why does the human brain not need this?
Because it is intelligent?
I would not call this intelligence. There is a simple dictionary in the brain of the developer, which maps:
variable-name --> Type
And this mapping dict exists once per library.
If you are developing in the requests http lib, then there is a different mapping. Then "response" means type requests.Response.
Now my idea: Per module and/or per file type hinting from variable name.
Maybe a magic docstring in the __init__.py file:
""" variable-name-mapping: { request: django.http.HttpRequest, ... } """
This makes the type mapping available for all files in this directory (or sub-directories).
What do you think?
Disclaimer: For reasons I don't want to explain in detail, I am not allowed to do implement things like this. This is just an idea. I would feel proud and thankfull if someone likes it that much, that he implements it.
On 19 Oct 2018, at 09:18, Thomas Güttler guettliml@thomas-guettler.de wrote:
Imaging you are developing in the django context.
Everytime you use a variable named "request" or "response" your human brains knows that this is a subclass of django.http.HttpRequest and django.http.HttpResponse.
How to give the IDE this knowledge?
One solution is the do typehinting everywhere the veriable gets used.
But why does the human brain not need this?
Because it is intelligent?
I would not call this intelligence. There is a simple dictionary in the brain of the developer, which maps:
variable-name --> Type
And this mapping dict exists once per library.
If you are developing in the requests http lib, then there is a different mapping. Then "response" means type requests.Response.
Now my idea: Per module and/or per file type hinting from variable name.
Maybe a magic docstring in the __init__.py file:
""" variable-name-mapping: { request: django.http.HttpRequest, ... } """
This makes the type mapping available for all files in this directory (or sub-directories).
What do you think?
I tried to implement this in mypy quite recently actually, but gave up for various reasons.
I am very much +1 on this. This would be a huge boon to preexisting code bases.
/ Anders
That would be a feature in my "python dialect" future library. Taking one valid python program and producing another python program :
def f(request): return HttpResponse('ok')
→
def f(request: HttpRequest): return HttpResponse('ok')
The dialect options would include "all top functions in this module" for example, and parameters about the naming convention (explicit list of conversion or rules to create the mapping).
Le ven. 19 oct. 2018 à 09:18, Thomas Güttler guettliml@thomas-guettler.de a écrit :
Imaging you are developing in the django context.
Everytime you use a variable named "request" or "response" your human brains knows that this is a subclass of django.http.HttpRequest and django.http.HttpResponse.
How to give the IDE this knowledge?
One solution is the do typehinting everywhere the veriable gets used.
But why does the human brain not need this?
Because it is intelligent?
I would not call this intelligence. There is a simple dictionary in the brain of the developer, which maps:
variable-name --> Type
And this mapping dict exists once per library.
If you are developing in the requests http lib, then there is a different mapping. Then "response" means type requests.Response.
Now my idea: Per module and/or per file type hinting from variable name.
Maybe a magic docstring in the __init__.py file:
""" variable-name-mapping: { request: django.http.HttpRequest, ... } """
This makes the type mapping available for all files in this directory (or sub-directories).
What do you think?
Disclaimer: For reasons I don't want to explain in detail, I am not allowed to do implement things like this. This is just an idea. I would feel proud and thankfull if someone likes it that much, that he implements it.
-- Thomas Guettler http://www.thomas-guettler.de/ I am looking for feedback: https://github.com/guettli/programming-guidelines _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Am 19.10.18 um 10:22 schrieb Robert Vanden Eynde:
That would be a feature in my "python dialect" future library. Taking one valid python program and producing another python program :
def f(request): return HttpResponse('ok')
→
def f(request: HttpRequest): return HttpResponse('ok')
The dialect options would include "all top functions in this module" for example, and parameters about the naming convention (explicit list of conversion or rules to create the mapping).
I think creating source code from source code is most of the time not the best solution. Except you call the result not "source code" but "binary executable code". [1]
I think it would be great if this could work for local variables (which are not in the arg/kwargs of the method), too:
Example:
def foo(): response = get_response_from_paradise()
If `get_response_from_paradise()` has not spec for the return type, then it would be great to guess the return type from the general "wisdom".
Regards, Thomas
[1]: https://github.com/guettli/programming-guidelines/blob/master/README.rst#sou...
On Fri, 19 Oct 2018 at 08:19, Thomas Güttler guettliml@thomas-guettler.de wrote:
[...] This makes the type mapping available for all files in this directory (or sub-directories). What do you think?
I don't think this should be a language feature. _Maybe_ it can be some kind of an agreed format between IDEs. But in any case I would first discuss this on PyCharm and VScode (IntelliSense?) trackers.
Also I don't think it is a good fit for mypy (and probably other type checkers, but better ask pytype and pyre devs). I would rather expect that IDEs auto-add annotations using some guided heuristics, and then type checkers can verify them.
-- Ivan
Am 19.10.18 um 11:42 schrieb Ivan Levkivskyi:
On Fri, 19 Oct 2018 at 08:19, Thomas Güttler <guettliml@thomas-guettler.de mailto:guettliml@thomas-guettler.de> wrote:
[...] This makes the type mapping available for all files in this directory (or sub-directories). What do you think?I don't think this should be a language feature. _Maybe_ it can be some kind of an agreed format between IDEs. But in any case I would first discuss this on PyCharm and VScode (IntelliSense?) trackers.
Also I don't think it is a good fit for mypy (and probably other type checkers, but better ask pytype and pyre devs). I would rather expect that IDEs auto-add annotations using some guided heuristics, and then type checkers can verify them.
I agree. For me IDE support is all I need.
Regards, Thomas Güttler
On Fri, Oct 19, 2018 at 09:18:02AM +0200, Thomas Güttler wrote:
Imaging you are developing in the django context.
Everytime you use a variable named "request" or "response" your human brains knows that this is a subclass of django.http.HttpRequest and django.http.HttpResponse.
Not *my* human brain.
I've seen far too many variables called (let's say) "mylist" which actually hold a dict or a tuple (or in one memorable case, a string!) to unconditionally believe the name.
But I'll accept that given the context, there's a very strong likelihood that a variable called "request" might be a HttpRequest object, and one called "response" might be a HttpResponse object.
How to give the IDE this knowledge?
One solution is the do typehinting everywhere the veriable gets used.
You shouldn't need to. Your IDE or type-checker should be able to do type-inference and infer the type. You only need to add a hint if it cannot infer the type.
If your IDE doesn't do type inference, get a better IDE *wink*
But why does the human brain not need this?
Because it is intelligent?
Yes.
I would not call this intelligence. There is a simple dictionary in the brain of the developer, which maps:
variable-name --> Type
Is that a fact? How do you know?
How does the information in this dict get filled in?
I think it is FAR more likely that it is intelligence: the human programmer understands the *meaning of the code* and would equally recognise that
myresponse response2 reply answer rspns
are all HttpResponse objects too, from the context in which they appear. We do that because we know the meaning of the words, and start from the default assumption that the coder isn't lying to us by using a name like "mylist" to represent a floating point number or "page_size" to represent the name of a file.
In other words, we *read* and *understand* the code, not just mechanically map names to types. That's why we have no problem with dynamically typed languages like Python were the one name can refer to objects of many different types.
When the context is different, we interpret the name differently:
response = input("Continue? Y/n ")
response = chatbot.lookup(question)
response = mapping[challenge]
would all be interpreted differently, even if the module used Django. I doubt any reasonable programmer would imagine that they were HttpResponse objects.
And this mapping dict exists once per library.
Or more likely, doesn't exist at all.
If you are developing in the requests http lib, then there is a different mapping. Then "response" means type requests.Response.
What if you are using *both* django and requests in the same module? You could have both of these:
response = something_returning_requests_Response()
response = something_returning_django_HttpResponse()
in the one module.
On 19 Oct 2018, at 12:15, Steven D'Aprano steve@pearwood.info wrote:
On Fri, Oct 19, 2018 at 09:18:02AM +0200, Thomas Güttler wrote:
Imaging you are developing in the django context.
Everytime you use a variable named "request" or "response" your human brains knows that this is a subclass of django.http.HttpRequest and django.http.HttpResponse.
Not *my* human brain.
I've seen far too many variables called (let's say) "mylist" which actually hold a dict or a tuple (or in one memorable case, a string!) to unconditionally believe the name.
This is an even stronger argument for the proposal I think. If IDEs and static analysis tools looked at stuff like this and assumed a type _and then found that this assumption is violated_, it would be a huge win!
How to give the IDE this knowledge?
One solution is the do typehinting everywhere the veriable gets used.
You shouldn't need to. Your IDE or type-checker should be able to do type-inference and infer the type. You only need to add a hint if it cannot infer the type.
If your IDE doesn't do type inference, get a better IDE *wink*
Which IDE would this be? PyCharm doesn't do this in the general case. Not even close in the code base I work on.
And this mapping dict exists once per library.
Or more likely, doesn't exist at all.
You seem to argue here, and generally, that the normal case for code bases is that you have no consistent naming. This seems strange to me. Can I ask what type of code and how big code bases you work on? To me it seems we have radically different perspectives, and we'll continue talking past each other if we don't explain those perspectives to each other.
I'll go first: I work on a code base that: - is ~240k dry lines - is ~15 years old - it a web app with just one production install - has very good consistent naming, for example: - the "request" example above exists over 6000 times in the code base and all of them are HttpRequestBase-derived (or in a few cases a mock that pretends to be that). - the word "party" exists over 11 thousand times, all of them are an instance of the Party django model
Being able to tell PyCharm to assume the type of these and flag obvious violations would be great.
If you are developing in the requests http lib, then there is a different mapping. Then "response" means type requests.Response.
What if you are using *both* django and requests in the same module? You could have both of these:
response = something_returning_requests_Response()
response = something_returning_django_HttpResponse()
in the one module.
Then you'd use the normal typing annotations to override the default.
/ Anders
Related: https://github.com/python/mypy/issues/2103
- Elazar
On Fri, Oct 19, 2018 at 2:15 PM Anders Hovmöller boxed@killingar.net wrote:
On 19 Oct 2018, at 12:15, Steven D'Aprano steve@pearwood.info wrote:
On Fri, Oct 19, 2018 at 09:18:02AM +0200, Thomas Güttler wrote:
Imaging you are developing in the django context.
Everytime you use a variable named "request" or "response" your human
brains
knows that this is a subclass of django.http.HttpRequest and django.http.HttpResponse.
Not *my* human brain.
I've seen far too many variables called (let's say) "mylist" which actually hold a dict or a tuple (or in one memorable case, a string!) to unconditionally believe the name.
This is an even stronger argument for the proposal I think. If IDEs and static analysis tools looked at stuff like this and assumed a type _and then found that this assumption is violated_, it would be a huge win!
How to give the IDE this knowledge?
One solution is the do typehinting everywhere the veriable gets used.
You shouldn't need to. Your IDE or type-checker should be able to do type-inference and infer the type. You only need to add a hint if it cannot infer the type.
If your IDE doesn't do type inference, get a better IDE *wink*
Which IDE would this be? PyCharm doesn't do this in the general case. Not even close in the code base I work on.
And this mapping dict exists once per library.
Or more likely, doesn't exist at all.
You seem to argue here, and generally, that the normal case for code bases is that you have no consistent naming. This seems strange to me. Can I ask what type of code and how big code bases you work on? To me it seems we have radically different perspectives, and we'll continue talking past each other if we don't explain those perspectives to each other.
I'll go first: I work on a code base that:
- is ~240k dry lines
- is ~15 years old
- it a web app with just one production install
- has very good consistent naming, for example:
- the "request" example above exists over 6000 times in the code base
and all of them are HttpRequestBase-derived (or in a few cases a mock that pretends to be that). - the word "party" exists over 11 thousand times, all of them are an instance of the Party django model
Being able to tell PyCharm to assume the type of these and flag obvious violations would be great.
If you are developing in the requests http lib, then there is a different mapping. Then "response" means type requests.Response.
What if you are using *both* django and requests in the same module? You could have both of these:
response = something_returning_requests_Response()
response = something_returning_django_HttpResponse()
in the one module.
Then you'd use the normal typing annotations to override the default.
/ Anders _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Am 19.10.18 um 12:15 schrieb Steven D'Aprano:
On Fri, Oct 19, 2018 at 09:18:02AM +0200, Thomas Güttler wrote:
Imaging you are developing in the django context.
Everytime you use a variable named "request" or "response" your human brains knows that this is a subclass of django.http.HttpRequest and django.http.HttpResponse.
Not *my* human brain.
I know that I know nothing, or at least not much. My laptop does compute much faster than I do
I've seen far too many variables called (let's say) "mylist" which actually hold a dict or a tuple (or in one memorable case, a string!) to unconditionally believe the name.
But I'll accept that given the context, there's a very strong likelihood that a variable called "request" might be a HttpRequest object, and one called "response" might be a HttpResponse object.
Great, we agree here.
How to give the IDE this knowledge?
One solution is the do typehinting everywhere the veriable gets used.
You shouldn't need to. Your IDE or type-checker should be able to do type-inference and infer the type. You only need to add a hint if it cannot infer the type.
If your IDE doesn't do type inference, get a better IDE *wink*
I use the free community version of pyCharm, and it often does not know
which type a variable has. A simple name2type mapping would improve
the situation a lot.
But why does the human brain not need this?
Because it is intelligent?
Yes.
I would not call this intelligence. There is a simple dictionary in the brain of the developer, which maps:
variable-name --> Type
Is that a fact? How do you know?
How does the information in this dict get filled in?
as I already said before a docstring in the __init__.py file would
be a pragmatic solution.
I think it is FAR more likely that it is intelligence: the human programmer understands the *meaning of the code* and would equally recognise that
myresponse response2 reply answer rspns
I try to use the same variable name again and again. I guess in my
code base in 95% an instance of HttpRequest is stored in a variable called "request".
You say: the human programmer understands the *meaning of the code*
Great if that is the case for you. I often look at the code and don't get it at all.
I wrote the lines two years ago, and today I don't understand why they exist.
This does not happen daily, but at least once a month.
In other words, we *read* and *understand* the code, not just mechanically map names to types. That's why we have no problem with dynamically typed languages like Python were the one name can refer to objects of many different types.
When the context is different, we interpret the name differently:
response = input("Continue? Y/n ") response = chatbot.lookup(question) response = mapping[challenge]would all be interpreted differently, even if the module used Django. I doubt any reasonable programmer would imagine that they were HttpResponse objects.
relax. I mean this like I said. Please relax.
Imagine you have this: reposense = obj.get_response()
First: if the variable is the return value of a method, and the method
has type-annotated the return value, then this type information gets used.
End of type detect.
If the type information can't be found from the method, then use the name2type
mapping which I suggest in this mail.
And this mapping dict exists once per library.
Or more likely, doesn't exist at all.
If you are developing in the requests http lib, then there is a different mapping. Then "response" means type requests.Response.
What if you are using *both* django and requests in the same module? You could have both of these:
response = something_returning_requests_Response() response = something_returning_django_HttpResponse()in the one module.
Do you really care for this edge-case? I don't.
The way to get the type info is the same: Check the method,
if it has it use this. Do not use name2type mapping. End of type detection.
Then check the current file or the above __init__.py of the code. Take
name2type mapping from there.
Regards,
Thomas Güttler
On Fri, Oct 19, 2018 at 3:18 AM Thomas Güttler guettliml@thomas-guettler.de wrote:
Now my idea: Per module and/or per file type hinting from variable name. Maybe a magic docstring in the __init__.py file: variable-name-mapping: { request: django.http.HttpRequest, }
In general, I like this idea; but really only on a per-file basis. A short header at the top would be easy enough for an IDE or linter to scan. But imposing the conventions project-wide feels too broad.
There might, of course, be cases where the same name is used for different purposes in the same file. But the tool can alert the developer of that... and in that one file, she could either remove the header of refactor the names used, as made sense for that particular code.
This is certainly not something that requires language support. It can easily be purely a convention, as long as different IDEs, linters, type checkers, etc. agree on what the convention is. Maybe at some point in the future, if the convention becomes adopted, there might be some help in having a standard library module, or even minimal language recognition, of the convention. But let's work on adopting a convention first.
On 2018-10-21 10:44, David Mertz wrote:
On Fri, Oct 19, 2018 at 3:18 AM Thomas Güttler <guettliml@thomas-guettler.de mailto:guettliml@thomas-guettler.de> wrote:
Now my idea: Per module and/or per file type hinting from variable name. Maybe a magic docstring in the __init__.py file: variable-name-mapping: { request: django.http.HttpRequest, }In general, I like this idea; but really only on a per-file basis. A short header at the top would be easy enough for an IDE or linter to scan. But imposing the conventions project-wide feels too broad.
Maybe in a .editorconfig file? This would allow type hinting at a number of directory levels.
On Sat, Oct 20, 2018 at 07:53:18PM +0200, Thomas Güttler Lists wrote:
Am 19.10.18 um 12:15 schrieb Steven D'Aprano:
One solution is the do typehinting everywhere the veriable gets used.
You shouldn't need to. Your IDE or type-checker should be able to do type-inference and infer the type. You only need to add a hint if it cannot infer the type.
If your IDE doesn't do type inference, get a better IDE *wink*
I use the free community version of pyCharm, and it often does not know which type a variable has. A simple name2type mapping would improve the situation a lot.
If the free community version of PyCharm doesn't support type-inference, what makes you think it would support this "name2type" mapping?
I don't use PyCharm, and I don't know if this feature applies to the free version, but according to their blog, PyCharm not only supports type-inference but also uses dynamic runtime information to enhance static type checking:
https://blog.jetbrains.com/pycharm/2013/02/dynamic-runtime-type-inference-in...
If you find yourself needing to annotate lots of variables, I suspect that you're fighting the IDE rather than using it correctly. Without seeing your code, and knowing a lot more about PyCharm, I cannot say further.
But the critical point here is that we should not add a language feature to make up for the limitations of a single IDE. If the free version of PyCharm is underpowered, perhaps you ought to try the paid version, or another IDE, or submit a feature request to PyCharm, *before* turning to the Python language.
But why does the human brain not need this?
Because it is intelligent?
Yes.
I would not call this intelligence. There is a simple dictionary in the brain of the developer, which maps:
variable-name --> Type
Is that a fact? How do you know?
How does the information in this dict get filled in?
as I already said before a docstring in the __init__.py file would be a pragmatic solution.
You misunderstand me. You claim that there is a dictionary in the brain of the developer, right now. How does that dictionary *in the brain* get there, if it isn't through the use of human intelligence?
I believe you are too focused on a single specific implementation:
"use a dict to map variable name to type -- I know how to implement this, therefore that is what the brain must do"
whereas I think that the reality is more complex: what you see as a mapping, I expect is just a special case of more general, and far more powerful, general reasoning intelligence.
We can infer types from variable names, because we know what the name means, not because "there is a simple dictionary in the brain of the developer" mapping name to type.
exchange_rate # probably a float num_rows # number of rows, therefore an int name # a string phone_number # not a float! should be a string response # a response from something (depends on context)
If we read Greek, we'd probably even recognise that a variable called "όνομα" was likely to be a string.
[...]
When the context is different, we interpret the name differently:
response = input("Continue? Y/n ") response = chatbot.lookup(question) response = mapping[challenge]would all be interpreted differently, even if the module used Django. I doubt any reasonable programmer would imagine that they were HttpResponse objects.
relax. I mean this like I said. Please relax.
Please don't patronise me. Just because I disagree with your suggestion doesn't mean I am freaking out and need to be told to relax.
We should not implement new features after ONLY considering when the feature would be useful. We should also consider when the feature is not needed, and when it would be actively harmful, and weigh it on the balance.
Your suggestion is useful when:
- you use the same variable name over and over again;
- always (or almost always) to mean the same type;
- and type inference won't work;
- and you don't want to annotate the variable each time;
- and the IDE supports this new feature;
- and there is little or no chance that you might want to use the same name for a different type in the same module.
It is not useful, maybe even harmful, if any of those conditions are not met. I'm especially concerned about the last one: type-checking false alarms due to a clash between the name2type mapping and the programmer's intention.
But on the other hand, if this is a convention purely implemented by IDEs, then I don't give a damn about this one bit. (Except to say, take the discussion to the "code-quality" mailing list, not here.) It doesn't sound like something that needs language support.
What if you are using *both* django and requests in the same module? You could have both of these:
response = something_returning_requests_Response() response = something_returning_django_HttpResponse()in the one module.
Do you really care for this edge-case? I don't.
Of course I do. It isn't an edge-case, it is representative of the vast majority of variable names:
- "A single variable name is always the same type" is the edge-case.
- "A single variable name can represent different types in different contexts" is the general case in dynamically-typed languages like Python.
So we should care about the case where variables are reused in different contexts.
The way to get the type info is the same: Check the method, if it has it use this. Do not use name2type mapping. End of type detection.
So why doesn't that already work for you? Why do you need name2type mapping *at all* if this problem is so easy to solve?
The bottom line here is that I don't think this needs language support, I would be against adding language support, but if IDEs want to support this, so long as they offer a switch to turn it off I don't care what they do :-)
Am 21.10.18 um 16:44 schrieb David Mertz:
On Fri, Oct 19, 2018 at 3:18 AM Thomas Güttler <guettliml@thomas-guettler.de mailto:guettliml@thomas-guettler.de> wrote:
Now my idea: Per module and/or per file type hinting from variable name. Maybe a magic docstring in the __init__.py file: variable-name-mapping: { request: django.http.HttpRequest, }In general, I like this idea; but really only on a per-file basis. A short header at the top would be easy enough for an IDE or linter to scan. But imposing the conventions project-wide feels too broad.
There might, of course, be cases where the same name is used for different purposes in the same file. But the tool can alert the developer of that... and in that one file, she could either remove the header of refactor the names used, as made sense for that particular code.
This is certainly not something that requires language support. It can easily be purely a convention, as long as different IDEs, linters, type checkers, etc. agree on what the convention is. Maybe at some point in the future, if the convention becomes adopted, there might be some help in having a standard library module, or even minimal language recognition, of the convention. But let's work on adopting a convention first.
Yes, this sounds good. There is no need to change the python language, it is should be a convetion.
Regards, Thomas Güttler
But the critical point here is that we should not add a language feature to make up for the limitations of a single IDE. If the free version of PyCharm is underpowered, perhaps you ought to try the paid version, or another IDE, or submit a feature request to PyCharm, *before* turning to the Python language.
Which IDE do you use that is so much greater than PyCharm? I would love to try it!
Of course I do. It isn't an edge-case, it is representative of the vast majority of variable names:
- "A single variable name is always the same type" is the edge-case.
I strongly disagree. I also wrote a mail with my motivation and context where I asked you to supply your basis for believing this. You have not replied. I think we need to try to understand each others perspective here, but it's really hard to understand yours if you won't give us any information on it.
/ Anders
This is certainly not something that requires language support. It can easily be purely a convention, as long as different IDEs, linters, type checkers, etc. agree on what the convention is. Maybe at some point in the future, if the convention becomes adopted, there might be some help in having a standard library module, or even minimal language recognition, of the convention. But let's work on adopting a convention first.
Yes, this sounds good. There is no need to change the python language, it is should be a convetion.
Might not even be a convention. I have now tried using https://github.com/edreamleo/make-stub-files https://github.com/edreamleo/make-stub-files on the code base at work. I had to fix some fatal bugs, and it's still pretty buggy even after those fixes, AND PyCharm has some nasty bugs with stub files that are blockers, but it's clear that generating stub files to do this is feasible.
/ Anders
On Mon, Oct 22, 2018 at 02:35:40PM +0200, Anders Hovmöller wrote:
But the critical point here is that we should not add a language feature to make up for the limitations of a single IDE. If the free version of PyCharm is underpowered, perhaps you ought to try the paid version, or another IDE, or submit a feature request to PyCharm, *before* turning to the Python language.
Which IDE do you use that is so much greater than PyCharm? I would love to try it!
My IDE is Unix. (Technically, Linux.)
https://sanctum.geek.nz/arabesque/series/unix-as-ide/
Or just google https://duckduckgo.com/?q=unix+as+an+ide
Although I'm not as hard-core as some (I use an actual GUI editor, not Vim or Emacs). I'm not opposed to the concept of IDEs as such, my first two development environments (THINK Pascal, and Hypercard) were IDEs and they were great, especially considing the limited resources they had available back in 1988 or so.
But... modern IDEs... I dunno... I don't begrudge you if you like them, but I don't think they're for me.
Of course I do. It isn't an edge-case, it is representative of the vast majority of variable names:
- "A single variable name is always the same type" is the edge-case.
I strongly disagree.
Okay.
On 10/22/18 9:04 AM, Steven D'Aprano wrote:
My IDE is Unix. (Technically, Linux.)
+1
Or just google https://duckduckgo.com/?q=unix+as+an+ide
Thank you for not verbing DuckDuckGo! :-)
... (I use an actual GUI editor, not Vim or Emacs) ...
[ ... must ... resist ... holy war ... ]
Dan
On Fri, Oct 19, 2018 at 01:14:39PM +0200, Anders Hovmöller wrote:
[I wrote this]
I've seen far too many variables called (let's say) "mylist" which actually hold a dict or a tuple (or in one memorable case, a string!) to unconditionally believe the name.
This is an even stronger argument for the proposal I think. If IDEs and static analysis tools looked at stuff like this and assumed a type _and then found that this assumption is violated_, it would be a huge win!
That's an argument for *static type checking*, not necessarily an argument for *this* specific proposal. Any method of checking types, be it static typing, mandatory type declarations, type inference, stub files, annotations, or anything else, would help in that situation.
This specific proposal assumes that when you have a single name, it always (or at least, nearly always) means the same type, across an entire package. Therefore it is safe to map the name *alone* to the type, rather than annotate the variable (name in a specific context) with the type.
Of course I get it that you only choose *selected* names to label with this name2type mapping. I just don't think this happens often enough to make it a language feature, or would be valuable enough when it does happen, to make up for the disadvantages.
You're using a *project-wide global* declaration, distant from the variable's context, possibly even in a completely different file from where the variable is used. That's just about the worst possible way to annotate a variable with a type.
But if some, or all, IDEs want to support this, I have no objections. Just don't use it in any code I have to read :-)
[...]
If your IDE doesn't do type inference, get a better IDE *wink*
Which IDE would this be? PyCharm doesn't do this in the general case. Not even close in the code base I work on.
I don't know whether PyCharm does type-inference, but it *claims* to, and it does have a plugin which runs MyPy, and MyPy certainly does.
If PyCharm lacks this feature, have you checked out the competition (e.g. Wing IDE, PyDev, Spyder)? Have you submitted a feature request? It could integrate better with MyPy, or possibly even the new type-checking tool from Facebook:
https://pypi.org/project/pyre-check/
or Google's:
https://github.com/google/pytype
Jedi is another project which claims to implement type-inference and only require hints as a fallback:
https://jedi.readthedocs.io/en/latest/docs/features.html#type-hinting
And this mapping dict exists once per library.
Or more likely, doesn't exist at all.
You seem to argue here, and generally, that the normal case for code bases is that you have no consistent naming. This seems strange to me.
The point of modules, and functions, is *encapsulation*. If I name a variable "spam" in function 1, why *must* a similar variable use the same name in function 2 let alone function 202, distant in another module? That implies a level of coupling that makes me uncomfortable.
And probably an excess of generic names like "response" and too few *specific* names like "client_response" and "server_response".
I am impressed by you and your team's attention to detail at requiring consistent names across such a large code base. I don't know if it is a good thing or a bad thing or just a thing, but I can admire the discipline it requires. I am certainly not that meticulous.
But I'm also faintly horrified at how much pointless pretend- productivity this book-keeping may (or not!) have involved. You know the sort of time-wasting busy-work coders can get up to when they want to look busy without actually thinking too hard:
- prettifying code layout and data structures; - pointless PEP-8-ifying code, whether it needs it or not; - obsessing about consistent names everywhere;
etc. I know this because I've wasted many hours doing all of these.
(I shouldn't *need* to say this, but I will anyway: I am making no comment on *you and your team* specifically, as I don't know you. I'm making a general observation about the tendency of many programmers, myself included, to waste time in unproductive "refactoring" which doesn't actually help code quality.)
I don't see "inconsistent" names in different functions or modules to be a problem that needs to be avoided. Hence, I don't avoid it. I don't go out of my way to use unique names, but nor do I try to impose a single-name policy.
In any case, we're still up against the fact that in 2018, the state of the art in type checkers is that they ought to be able to do what a human reader does and infer the type of your variables. It shouldn't matter whether you call it "response" or "spam" or "zpaqxerag", if your type-checker can't work out that
foo = HttpResponse(*args)
is a HttpResponse object, something has gone wrong somewhere. Adding yet another way to annotate variables instead of improving type-inference seems like a sub-optimal choice to me.
https://mypy.readthedocs.io/en/latest/type_inference_and_annotations.html
I'll just simply reply here that Mypy on our entire code base just silently outputs nothing. Nothing at all. I tried to look at the code to find out why and maybe implement the name checking feature discussed here and was terrified by the Byzantine complexity of the code base. I gave up.
From what I later understood of Mypy it is supposed to be used to check all types or none on a file level, not just check what can be checked so you can gradually add typing in reasonable places. This seems the totally wrong approach to me. I think we agree on this point.
As for why we have consistent names, it was before my time but keeping this consistency isn't much work, we certainly don't go back and change code to enforce conventions.
On 23 Oct 2018, at 01:02, Steven D'Aprano steve@pearwood.info wrote:
On Fri, Oct 19, 2018 at 01:14:39PM +0200, Anders Hovmöller wrote:
[I wrote this]
I've seen far too many variables called (let's say) "mylist" which actually hold a dict or a tuple (or in one memorable case, a string!) to unconditionally believe the name.
This is an even stronger argument for the proposal I think. If IDEs and static analysis tools looked at stuff like this and assumed a type _and then found that this assumption is violated_, it would be a huge win!
That's an argument for *static type checking*, not necessarily an argument for *this* specific proposal. Any method of checking types, be it static typing, mandatory type declarations, type inference, stub files, annotations, or anything else, would help in that situation.
This specific proposal assumes that when you have a single name, it always (or at least, nearly always) means the same type, across an entire package. Therefore it is safe to map the name *alone* to the type, rather than annotate the variable (name in a specific context) with the type.
Of course I get it that you only choose *selected* names to label with this name2type mapping. I just don't think this happens often enough to make it a language feature, or would be valuable enough when it does happen, to make up for the disadvantages.
You're using a *project-wide global* declaration, distant from the variable's context, possibly even in a completely different file from where the variable is used. That's just about the worst possible way to annotate a variable with a type.
But if some, or all, IDEs want to support this, I have no objections. Just don't use it in any code I have to read :-)
[...]
If your IDE doesn't do type inference, get a better IDE *wink*
Which IDE would this be? PyCharm doesn't do this in the general case. Not even close in the code base I work on.
I don't know whether PyCharm does type-inference, but it *claims* to, and it does have a plugin which runs MyPy, and MyPy certainly does.
If PyCharm lacks this feature, have you checked out the competition (e.g. Wing IDE, PyDev, Spyder)? Have you submitted a feature request? It could integrate better with MyPy, or possibly even the new type-checking tool from Facebook:
https://pypi.org/project/pyre-check/
or Google's:
https://github.com/google/pytype
Jedi is another project which claims to implement type-inference and only require hints as a fallback:
https://jedi.readthedocs.io/en/latest/docs/features.html#type-hinting
And this mapping dict exists once per library.
Or more likely, doesn't exist at all.
You seem to argue here, and generally, that the normal case for code bases is that you have no consistent naming. This seems strange to me.
The point of modules, and functions, is *encapsulation*. If I name a variable "spam" in function 1, why *must* a similar variable use the same name in function 2 let alone function 202, distant in another module? That implies a level of coupling that makes me uncomfortable.
And probably an excess of generic names like "response" and too few *specific* names like "client_response" and "server_response".
I am impressed by you and your team's attention to detail at requiring consistent names across such a large code base. I don't know if it is a good thing or a bad thing or just a thing, but I can admire the discipline it requires. I am certainly not that meticulous.
But I'm also faintly horrified at how much pointless pretend- productivity this book-keeping may (or not!) have involved. You know the sort of time-wasting busy-work coders can get up to when they want to look busy without actually thinking too hard:
- prettifying code layout and data structures;
- pointless PEP-8-ifying code, whether it needs it or not;
- obsessing about consistent names everywhere;
etc. I know this because I've wasted many hours doing all of these.
(I shouldn't *need* to say this, but I will anyway: I am making no comment on *you and your team* specifically, as I don't know you. I'm making a general observation about the tendency of many programmers, myself included, to waste time in unproductive "refactoring" which doesn't actually help code quality.)
I don't see "inconsistent" names in different functions or modules to be a problem that needs to be avoided. Hence, I don't avoid it. I don't go out of my way to use unique names, but nor do I try to impose a single-name policy.
In any case, we're still up against the fact that in 2018, the state of the art in type checkers is that they ought to be able to do what a human reader does and infer the type of your variables. It shouldn't matter whether you call it "response" or "spam" or "zpaqxerag", if your type-checker can't work out that
foo = HttpResponse(*args)
is a HttpResponse object, something has gone wrong somewhere. Adding yet another way to annotate variables instead of improving type-inference seems like a sub-optimal choice to me.
https://mypy.readthedocs.io/en/latest/type_inference_and_annotations.html
-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
participants (10)
-
Anders Hovmöller -
Dan Sommers -
David Mertz -
Elazar -
Ivan Levkivskyi -
Kyle Lahnakoski -
Robert Vanden Eynde -
Steven D'Aprano -
Thomas Güttler -
Thomas Güttler Lists