Note: this suggestion should echo https://mail.python.org/archives/list/python-ideas@python.org/thread/UUFFAI3... but has a bigger scope and completely differs in the way to proceed. Currently, Python lacks a way to integrate code parts into string; when I say code, I don't talk only about identifiers (on which is focusing `nameof` suggestion) but also about expression. Actually, this feature exists in Python but is hidden in a 3.8 [f-string feature](https://docs.python.org/3.8/whatsnew/3.8.html#f-strings-support-for-self-doc...). Expression string should be made accessible for itself. It could be done by adding a leading token `:` to the expression. Thus `f"{:a + b}" == "a + b"`. Like "normal" formatted expression, it would be handled by IDEs, especially for refactoring. The difference with "normal" formatted expression is that the expression would not be evaluated. A use case example: ```python def foo(bar: int): if bar != 42: # If `bar` is renamed, error message will be renamed too by the IDE raise ValueError(f"{:bar} is not 42") ``` By the way, there will be the following equivalence: `f"{a + b=}" == f"{:a + b}={a + b}"`. The idea is to reuse existing mechanisms. (Of course the syntax suggestion is arbitrary and can be discussed. I have considered a leading token because there are already a lot of modifiers at the end of the expression; I have thought about `:` and `!` and my preferences has gone to `:`.) Joseph
On Thu, Sep 17, 2020 at 11:47:03PM -0000, Joseph Perez wrote:
Thus `f"{:a + b}" == "a + b"`.
That can already be done by just *not* using a f-string. This looks like a case of "when the only tool you have is a hammer, everything looks like a nail". If you don't want a string literal to be evaluated, don't put it inside a f-string. There is no need to have a extra token inside f-strings to turn off evaluation. We already have a way to not evaluate strings.
I'm not aware of many refactoring tools for Python *at all*, the wiki still lists BicycleRepairman which hasn't been updated for seven years. https://wiki.python.org/moin/AutomatedRefactoringTools I don't know how well IDEs like VisualStudio and PyCharm do refactoring, but I would think that if you wanted to refactor strings, a more promising approach would be a tagged comment rather than f-strings:
def foo(bar: int): if bar != 42: # IDE:refactor raise ValueError("bar is not 42") tells the IDE that it's okay to refactor inside the string literal in the following line. I doubt this feature exists today, but if you expect linters to learn to refactor f-strings they could learn to refactor regular strings too. It seems to me that rather than modifying Python to create special syntax for no-ops to be used as directives for the benefit of IDEs, it would be better for IDEs to get together and work on a common syntax for directives which can be included in regular comments. -- Steve
There are a number of refactoring tools for Python. From https://github.com/beat-no/roper#alternatives : ```rst Alternatives ============ IDEs ---- * PyCharm - https://www.jetbrains.com/help/pycharm/refactoring-source-code.html Editor plugins -------------- * vim - https://github.com/python-rope/ropevim * emacs - https://github.com/python-rope/ropemacs Libraries --------- * bowler - https://pybowler.io/ * libcst - https://libcst.readthedocs.io/en/latest/ * undebt - https://github.com/Yelp/undebt * redbaron - http://redbaron.pycqa.org/en/latest/ Resources ========= * https://github.com/python-rope/rope * https://realpython.com/python-refactoring/ ``` "Support f"literal {string} interpolation" (PEP-498, Python 3.6)" (fixed) https://github.com/python-rope/rope/issues/226 "Rename symbol misses symbols in formatted strings f"{foo}" (fixed because rope fixed it) https://github.com/microsoft/vscode-python/issues/187 "Can Bowler rename the variable in the format string literals?" https://github.com/facebookincubator/Bowler/issues/87 On Thu, Sep 17, 2020 at 8:27 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Sep 17, 2020 at 9:32 PM Wes Turner <wes.turner@gmail.com> wrote:
Another: https://sourcery.ai/ On Thu, Sep 17, 2020 at 8:27 PM Steven D'Aprano <steve@pearwood.info> wrote:
Using the same example from the Bowler issue <https://github.com/facebookincubator/Bowler/issues/87>: - print(f'bar is {bar}, not {zar}') - print('bar is {bar}, not {zar}'.format(bar=bar, zar=zar)) ..both rope and Pycharm are currently smart enough to refactor the part in the curly braces: - f'bar is {bar}, not {zar}' → f'bar is {bar_new}, not {zar}' - .format(bar=bar, zar=zar) → .format(bar=bar_new, zar=zar) As far as I know, none of these tools know how to do the renaming of the FIRST bar to bar_new: - f'bar is {bar}, not {zar}' → f'*bar_new* is {bar_new}, not {zar}' - .format(bar=bar, zar=zar) → .format(*bar_new*=bar_new, zar=zar) It seems to me that rather than modifying Python to create special
The effort for this just doesn't seem worth it. All IDEs have find and replace (pycharm's is very nice). And even python developers who eschew the IDE use SOMETHING to write their code... whether it's vim or notepad++ or vscode (people tell me it's not an IDE) or emacs or whatever, and that something invariably provides find and replace functionality (vim has the :substitute command. notepad++ has ctr+r. Microsoft Word has ctl+h ;) ). But if you really want it or need it in a situation where find and replace isn't a great option, you can hack a utility function using the 3.8 syntax: def nameof(x): return x.split("=", 1)[0] ...and use it in a nested fstring:
f"{nameof('{foo=}')}" foo
Now refactoring can work like a charm for single variables: - f'{nameof(f"{bar=}")} is {bar}, not {zar}' → f'{nameof(f"{bar_new=}")} is {bar_new}, not {zar}' The OP example doesn't look fantastic, but it works:
f"{nameof(f'{a=}')} + {nameof(f'{b=}')}" == "a + b" True
--- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
Ricky Teachey wrote:
But if you really want it or need it in a situation where find and replace isn't a great option
The example i've given is a good example where find and replace isn't a great option. In fact, you don't want to use find and replace to change a function parameter, because it will be tedious to change all function calls with named parameter accross your call. That's why refactoring tools exist. I agree, that Pycharm find and replace is very nice (especially using regex), but I use it only for modify copy-pasted things with regex, never for refactoring. By the way, I like your "hack", but as you say it's not very fantastic; maybe it deserve an official simplification.
Steven D'Aprano wrote:
I'm not aware of many refactoring tools for Python at all, [...] I don't know how well IDEs like VisualStudio and PyCharm do refactoring.
This help me to understand your point of view. But if I consider your proposition, it's evident that a simple comment `# IDE:refactor` would not be enough for IDE. Because a string is full of words and several of them could matchs the symbols of your code; by the way, the IDE would not be able to delimitate expression. So you would have to put redondant information into your comment, and redundancy is not cool for the user; if the shortcut `f"{foo=}"` has been introduced in 3.8, that's to avoid "to repeat yourself" (word taken from feature author). By the way, there is (as far as I know) no comment standardisation in Python, they are specific to the tools which use them. See how many docstring standard we have; Both Pycharm and Black have their own comment directive to toggle formatting on/off; when mypy and flake8 find an issue in your code, you have to put # type: ignore # noqa… How could an approach using comment be promising? And as you can see in other responses to your post, when the Python language introduce a feature, IDEs adapt to fully support it. That's the case for f-string where refactoring, linting, type-checking, reformatting, etc. are handled inside. This is the difference between a literal string and a "not-evaluated" string.
On Fri, Sep 18, 2020 at 08:05:27AM -0000, Joseph Perez wrote:
I see your point, but I don't think it's a large problem in practice. As Ricky and Wes have pointed out, IDE refactoring tools are already able to look inside not just f-strings but regular strings too. If your point is a practical problem, it is *already* a practical problem for refactoring tools. How do they deal with this? The point is, whatever techniques they use, whatever they need, we shouldn't jump into adding new language features to the language because they *could* be useful for IDEs and refactoring tools. Find out from the writers of the software what they actually need, not just what you guess they might use, and then we can have a better discussion as to whether that should go into the language or not.
That is why I am suggesting that tool users should start cooperating on a standard for directives. -- Steve
As Ricky and Wes have pointed out, IDE refactoring tools are already able to look inside not just f-strings but regular strings too.
Ricky has written the opposite : "As far as I know, none of these tools know how to do the renaming of the FIRST bar to bar_new:", and i can confirm
If your point is a practical problem, it is already a practical problem for refactoring tools. How do they deal with this?
They don't do it automatically, so they give the burden to the user which has to review all the potential changes in strings literal (with a lot of false-positive, and some missing occurrences); and that's when they do, because mostly they don't.
That is why I am suggesting that tool users should start cooperating on a standard for directives.
Even if I agree with you on this point (for example, I'm happy that new versions of Pycharm support `# noqa` directive to disable some checks), this features makes it irrelevant IMO as I already stated that it would imply user repeating himself.
Is there a list of supported preprocessor directives or compiler directives supported by CPython and other tools? - from __future__ import ... - __debug__ https://docs.python.org/3/library/constants.html#__debug__ - encoding https://www.python.org/dev/peps/pep-0263/#defining-the-encoding # coding=<encoding name> # -*- coding: <encoding name> -*- # vim: set fileencoding=<encoding name> : ^[ \t\f]*#.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+) - # noqa: - https://flake8.pycqa.org/en/latest/user/violations.html - # flake8: noqa https://docs.python.org/3/distutils/apiref.html#distutils.ccompiler.CCompile... https://docs.python.org/3/distutils/setupscript.html#preprocessor-options https://github.com/interpreters/pypreprocessor#syntax https://en.wikipedia.org/wiki/Directive_(programming) : programming language, and may vary from compiler to compiler. They can be processed by a preprocessor to specify compiler behavior, or function as a form of in-band parameterization.
In some cases directives specify global behavior, while in other cases
they only affect a local section, such as a block of programming code. In some cases, such as some C programs, directives are optional compiler hints, and may be ignored, but normally they are prescriptive, and must be followed. However, a directive does not perform any action in the language itself, but rather only a change in the behavior of the compiler.
This term could be used to refer to proprietary third party tags and
commands (or markup) embedded in code that result in additional executable processing that extend the existing compiler, assembler and language constructs present in the development environment. The term "directive" is also applied in a variety of ways that are similar to the term command. On Fri, Sep 18, 2020 at 6:06 AM Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Sep 18, 2020 at 12:32:32PM -0400, Wes Turner wrote:
Is there a list of supported preprocessor directives or compiler directives supported by CPython and other tools?
I expect that most tools will have their own list. Python doesn't really have directives other than `from __future__ import ...`. You can view the available future imports by treating it as a regular module and inspecting that: import __future__ dir(__future__) There's also the command line options you can give when invoking the interpreter. -- Steve
On Thu, Sep 17, 2020 at 11:47:03PM -0000, Joseph Perez wrote:
Thus `f"{:a + b}" == "a + b"`.
That can already be done by just *not* using a f-string. This looks like a case of "when the only tool you have is a hammer, everything looks like a nail". If you don't want a string literal to be evaluated, don't put it inside a f-string. There is no need to have a extra token inside f-strings to turn off evaluation. We already have a way to not evaluate strings.
I'm not aware of many refactoring tools for Python *at all*, the wiki still lists BicycleRepairman which hasn't been updated for seven years. https://wiki.python.org/moin/AutomatedRefactoringTools I don't know how well IDEs like VisualStudio and PyCharm do refactoring, but I would think that if you wanted to refactor strings, a more promising approach would be a tagged comment rather than f-strings:
def foo(bar: int): if bar != 42: # IDE:refactor raise ValueError("bar is not 42") tells the IDE that it's okay to refactor inside the string literal in the following line. I doubt this feature exists today, but if you expect linters to learn to refactor f-strings they could learn to refactor regular strings too. It seems to me that rather than modifying Python to create special syntax for no-ops to be used as directives for the benefit of IDEs, it would be better for IDEs to get together and work on a common syntax for directives which can be included in regular comments. -- Steve
There are a number of refactoring tools for Python. From https://github.com/beat-no/roper#alternatives : ```rst Alternatives ============ IDEs ---- * PyCharm - https://www.jetbrains.com/help/pycharm/refactoring-source-code.html Editor plugins -------------- * vim - https://github.com/python-rope/ropevim * emacs - https://github.com/python-rope/ropemacs Libraries --------- * bowler - https://pybowler.io/ * libcst - https://libcst.readthedocs.io/en/latest/ * undebt - https://github.com/Yelp/undebt * redbaron - http://redbaron.pycqa.org/en/latest/ Resources ========= * https://github.com/python-rope/rope * https://realpython.com/python-refactoring/ ``` "Support f"literal {string} interpolation" (PEP-498, Python 3.6)" (fixed) https://github.com/python-rope/rope/issues/226 "Rename symbol misses symbols in formatted strings f"{foo}" (fixed because rope fixed it) https://github.com/microsoft/vscode-python/issues/187 "Can Bowler rename the variable in the format string literals?" https://github.com/facebookincubator/Bowler/issues/87 On Thu, Sep 17, 2020 at 8:27 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Sep 17, 2020 at 9:32 PM Wes Turner <wes.turner@gmail.com> wrote:
Another: https://sourcery.ai/ On Thu, Sep 17, 2020 at 8:27 PM Steven D'Aprano <steve@pearwood.info> wrote:
Using the same example from the Bowler issue <https://github.com/facebookincubator/Bowler/issues/87>: - print(f'bar is {bar}, not {zar}') - print('bar is {bar}, not {zar}'.format(bar=bar, zar=zar)) ..both rope and Pycharm are currently smart enough to refactor the part in the curly braces: - f'bar is {bar}, not {zar}' → f'bar is {bar_new}, not {zar}' - .format(bar=bar, zar=zar) → .format(bar=bar_new, zar=zar) As far as I know, none of these tools know how to do the renaming of the FIRST bar to bar_new: - f'bar is {bar}, not {zar}' → f'*bar_new* is {bar_new}, not {zar}' - .format(bar=bar, zar=zar) → .format(*bar_new*=bar_new, zar=zar) It seems to me that rather than modifying Python to create special
The effort for this just doesn't seem worth it. All IDEs have find and replace (pycharm's is very nice). And even python developers who eschew the IDE use SOMETHING to write their code... whether it's vim or notepad++ or vscode (people tell me it's not an IDE) or emacs or whatever, and that something invariably provides find and replace functionality (vim has the :substitute command. notepad++ has ctr+r. Microsoft Word has ctl+h ;) ). But if you really want it or need it in a situation where find and replace isn't a great option, you can hack a utility function using the 3.8 syntax: def nameof(x): return x.split("=", 1)[0] ...and use it in a nested fstring:
f"{nameof('{foo=}')}" foo
Now refactoring can work like a charm for single variables: - f'{nameof(f"{bar=}")} is {bar}, not {zar}' → f'{nameof(f"{bar_new=}")} is {bar_new}, not {zar}' The OP example doesn't look fantastic, but it works:
f"{nameof(f'{a=}')} + {nameof(f'{b=}')}" == "a + b" True
--- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
Ricky Teachey wrote:
But if you really want it or need it in a situation where find and replace isn't a great option
The example i've given is a good example where find and replace isn't a great option. In fact, you don't want to use find and replace to change a function parameter, because it will be tedious to change all function calls with named parameter accross your call. That's why refactoring tools exist. I agree, that Pycharm find and replace is very nice (especially using regex), but I use it only for modify copy-pasted things with regex, never for refactoring. By the way, I like your "hack", but as you say it's not very fantastic; maybe it deserve an official simplification.
Steven D'Aprano wrote:
I'm not aware of many refactoring tools for Python at all, [...] I don't know how well IDEs like VisualStudio and PyCharm do refactoring.
This help me to understand your point of view. But if I consider your proposition, it's evident that a simple comment `# IDE:refactor` would not be enough for IDE. Because a string is full of words and several of them could matchs the symbols of your code; by the way, the IDE would not be able to delimitate expression. So you would have to put redondant information into your comment, and redundancy is not cool for the user; if the shortcut `f"{foo=}"` has been introduced in 3.8, that's to avoid "to repeat yourself" (word taken from feature author). By the way, there is (as far as I know) no comment standardisation in Python, they are specific to the tools which use them. See how many docstring standard we have; Both Pycharm and Black have their own comment directive to toggle formatting on/off; when mypy and flake8 find an issue in your code, you have to put # type: ignore # noqa… How could an approach using comment be promising? And as you can see in other responses to your post, when the Python language introduce a feature, IDEs adapt to fully support it. That's the case for f-string where refactoring, linting, type-checking, reformatting, etc. are handled inside. This is the difference between a literal string and a "not-evaluated" string.
On Fri, Sep 18, 2020 at 08:05:27AM -0000, Joseph Perez wrote:
I see your point, but I don't think it's a large problem in practice. As Ricky and Wes have pointed out, IDE refactoring tools are already able to look inside not just f-strings but regular strings too. If your point is a practical problem, it is *already* a practical problem for refactoring tools. How do they deal with this? The point is, whatever techniques they use, whatever they need, we shouldn't jump into adding new language features to the language because they *could* be useful for IDEs and refactoring tools. Find out from the writers of the software what they actually need, not just what you guess they might use, and then we can have a better discussion as to whether that should go into the language or not.
That is why I am suggesting that tool users should start cooperating on a standard for directives. -- Steve
As Ricky and Wes have pointed out, IDE refactoring tools are already able to look inside not just f-strings but regular strings too.
Ricky has written the opposite : "As far as I know, none of these tools know how to do the renaming of the FIRST bar to bar_new:", and i can confirm
If your point is a practical problem, it is already a practical problem for refactoring tools. How do they deal with this?
They don't do it automatically, so they give the burden to the user which has to review all the potential changes in strings literal (with a lot of false-positive, and some missing occurrences); and that's when they do, because mostly they don't.
That is why I am suggesting that tool users should start cooperating on a standard for directives.
Even if I agree with you on this point (for example, I'm happy that new versions of Pycharm support `# noqa` directive to disable some checks), this features makes it irrelevant IMO as I already stated that it would imply user repeating himself.
Is there a list of supported preprocessor directives or compiler directives supported by CPython and other tools? - from __future__ import ... - __debug__ https://docs.python.org/3/library/constants.html#__debug__ - encoding https://www.python.org/dev/peps/pep-0263/#defining-the-encoding # coding=<encoding name> # -*- coding: <encoding name> -*- # vim: set fileencoding=<encoding name> : ^[ \t\f]*#.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+) - # noqa: - https://flake8.pycqa.org/en/latest/user/violations.html - # flake8: noqa https://docs.python.org/3/distutils/apiref.html#distutils.ccompiler.CCompile... https://docs.python.org/3/distutils/setupscript.html#preprocessor-options https://github.com/interpreters/pypreprocessor#syntax https://en.wikipedia.org/wiki/Directive_(programming) : programming language, and may vary from compiler to compiler. They can be processed by a preprocessor to specify compiler behavior, or function as a form of in-band parameterization.
In some cases directives specify global behavior, while in other cases
they only affect a local section, such as a block of programming code. In some cases, such as some C programs, directives are optional compiler hints, and may be ignored, but normally they are prescriptive, and must be followed. However, a directive does not perform any action in the language itself, but rather only a change in the behavior of the compiler.
This term could be used to refer to proprietary third party tags and
commands (or markup) embedded in code that result in additional executable processing that extend the existing compiler, assembler and language constructs present in the development environment. The term "directive" is also applied in a variety of ways that are similar to the term command. On Fri, Sep 18, 2020 at 6:06 AM Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Sep 18, 2020 at 12:32:32PM -0400, Wes Turner wrote:
Is there a list of supported preprocessor directives or compiler directives supported by CPython and other tools?
I expect that most tools will have their own list. Python doesn't really have directives other than `from __future__ import ...`. You can view the available future imports by treating it as a regular module and inspecting that: import __future__ dir(__future__) There's also the command line options you can give when invoking the interpreter. -- Steve
participants (4)
-
Joseph Perez -
Ricky Teachey -
Steven D'Aprano -
Wes Turner