gofmt for Python: standardized styling as a language feature
The Zen of Python <https://www.python.org/dev/peps/pep-0020/> says: There should be one— and preferably only one —obvious way to do it. Python already has a style guide in PEP 8 <https://www.python.org/dev/peps/pep-0008/>. More importantly, from a practical standpoint, Python has automated third-party tools for checking and enforcing PEP 8 guidelines: - pep8 <https://pypi.python.org/pypi/pep8/>: Python style guide checker - autopep8 <https://pypi.python.org/pypi/autopep8/>: A tool that automatically formats Python code to conform to the PEP 8 style guide I say “more importantly” because these tools are what make the style guide alive. If a Python project of any non-trivial size is PEP 8-compliant, you can bet it’s because the project contributors automated the process of enforcing compliance using these or similar tools. (Certainly, some parts of PEP 8 are hard to automate. But those parts that *can* be automated have been, to the benefit of Python programmers everywhere.) Having a style guide provides benefits that I think are well-understood. Additionally having a standardized style that can be *applied automatically* makes whatever benefits of the style guide that much more prevalent in the greater community and actually effective. As many of you may already know, the Go-lang folks have such a feature for their language (though they call it formatting as opposed to styling). They note: go fmt your code <http://blog.golang.org/go-fmt-your-code>: Gofmt <http://golang.org/cmd/gofmt/> is a tool that automatically formats Go source code. Gofmt’d code is: - easier to *write*: never worry about minor formatting concerns while hacking away, - easier to *read*: when all code looks the same you need not mentally convert others’ formatting style into something you can understand. - easier to *maintain*: mechanical changes to the source don’t cause unrelated changes to the file’s formatting; diffs show only the real changes. - *uncontroversial*: never have a debate about spacing or brace position ever again! Of course, we don’t worry about brace positions, but these benefits of a standardized style carry over into any language. :) So, in the spirit of The Zen of Python, and taking a cue from Go-lang’s gofmt <http://blog.golang.org/go-fmt-your-code>, I propose we promote standardized auto-styling to a first-class language feature. UI-wise, I’m not sure how exactly we’d present this to the user. Perhaps something like: python -m style script.py Don’t read too much into this UI suggestion, as I’m really not sure how it would be implemented. The point is to expose some auto-styler as a standard language tool—i.e. something like autopep8, but as a built-in utility. On running a command like that, script.py would automatically be updated to conform to the standard style we want to promote. This is what gofmt does for Go code. Promoting an auto-styler to be a first-class language feature builds on the foundation laid in PEP 8 and in tools like pep8 and autopep8. It helps reduce code style to more of a non-issue for Python projects than it already is. ------------------------------ *Side Note 1:* There was a previous python-ideas discussion about introducing standardized formatting here <https://groups.google.com/d/msg/python-ideas/ic-aPSvamS4/2lj53Em0VrsJ>, but it was totally derailed by the OP’s suggestion that the formatter introduce block delimiters. I am proposing no such thing here. *Side Note 2:* I found these additional comments <http://blog.golang.org/go-fmt-your-code> about gofmt interesting: Gofmt also enables gofix <http://golang.org/cmd/fix/>, which can make arbitrarily complex source transformations. Gofix was an invaluable tool during the early days when we regularly made breaking changes to the language and libraries. Introducing Gofix <http://blog.golang.org/introducing-gofix>: Gofix gives us the ability to fix mistakes or completely rethink package APIs without worrying about the cost of converting existing code. Python has its own analog to gofix in 2to3 <https://docs.python.org/3/library/2to3.html>, though that was built for a specific migration rather than as a regular language feature. I wonder if having a gofmt analog in Python would similarly facilitate future migrations, though I doubt we’ll ever have a big as migration as the 2-to-3 one.
On Mar 17, 2015, at 11:31 AM, Nicholas Chammas <nicholas.chammas@gmail.com> wrote:
The Zen of Python says:
There should be one— and preferably only one —obvious way to do it.
As someone who uses autopep8 regularly but has at least once had resistance to getting into a team's workflow, +1 on the idea. The analogy with gofmt isn't perfect because gofmt only deals with some simple issues where there's an unambiguous 100%-clear standard, while PEP 8 includes a wide range of guidelines that are usually but not always consistent and appropriate, but that (obviously) doesn't mean autopep8 is an impossible or pointless non-starter. So, is your proposal that autopep8 (and therefore pep8, on which it depends) be added to the stdlib with only minor changes? If so: * You'd probably want to ask whether Hideo Hattori and Johann C. Rocholl are willing to relicense and contribute their code, and maintain it for years to come, before asking whether Python will accept the result. * Will it be easy to transform a new version of the PyPI libraries into a new version of the stdlib module in some mostly-automated way for each new Python release? * Given that Python updates less often than pep8 and autopep8, will it generally be "good enough" to use the autopep8 1.1.3 that came with your Python 3.5 instead of installing the 1.1.7 that's on PyPI? What if you're still using Python 3.5 in another two years, and Python 3.6 came with 1.2.1, and 1.2.5 is on PyPI; is using 1.1.3 still a good idea? (Compare with pysqlite; I've used sqlite3 many times, and only wanted to get the newer version twice--and once just because it was the easiest way to build with a local variant of the C library on Windows.) * Are all of the current features and bugs (see issues on GitHub) acceptable as-is for the stdlib today? If the answers to those questions is "yes", this seems like the way to go. Otherwise, you're probably suggesting someone either fork the existing projects or write a brand new one (and as the one with the itch and the ideas, who better than you?).
Python already has a style guide in PEP 8. More importantly, from a practical standpoint, Python has automated third-party tools for checking and enforcing PEP 8 guidelines:
pep8: Python style guide checker autopep8: A tool that automatically formats Python code to conform to the PEP 8 style guide I say “more importantly” because these tools are what make the style guide alive. If a Python project of any non-trivial size is PEP 8-compliant, you can bet it’s because the project contributors automated the process of enforcing compliance using these or similar tools. (Certainly, some parts of PEP 8 are hard to automate. But those parts that can be automated have been, to the benefit of Python programmers everywhere.)
Having a style guide provides benefits that I think are well-understood. Additionally having a standardized style that can be applied automatically makes whatever benefits of the style guide that much more prevalent in the greater community and actually effective.
As many of you may already know, the Go-lang folks have such a feature for their language (though they call it formatting as opposed to styling). They note:
go fmt your code:
Gofmt is a tool that automatically formats Go source code.
Gofmt’d code is:
easier to write: never worry about minor formatting concerns while hacking away, easier to read: when all code looks the same you need not mentally convert others’ formatting style into something you can understand. easier to maintain: mechanical changes to the source don’t cause unrelated changes to the file’s formatting; diffs show only the real changes. uncontroversial: never have a debate about spacing or brace position ever again! Of course, we don’t worry about brace positions, but these benefits of a standardized style carry over into any language. :)
So, in the spirit of The Zen of Python, and taking a cue from Go-lang’s gofmt, I propose we promote standardized auto-styling to a first-class language feature.
UI-wise, I’m not sure how exactly we’d present this to the user. Perhaps something like:
python -m style script.py Don’t read too much into this UI suggestion, as I’m really not sure how it would be implemented. The point is to expose some auto-styler as a standard language tool—i.e. something like autopep8, but as a built-in utility.
On running a command like that, script.py would automatically be updated to conform to the standard style we want to promote. This is what gofmt does for Go code.
Promoting an auto-styler to be a first-class language feature builds on the foundation laid in PEP 8 and in tools like pep8 and autopep8. It helps reduce code style to more of a non-issue for Python projects than it already is.
Side Note 1: There was a previous python-ideas discussion about introducing standardized formatting here, but it was totally derailed by the OP’s suggestion that the formatter introduce block delimiters. I am proposing no such thing here.
Side Note 2: I found these additional comments about gofmt interesting:
Gofmt also enables gofix, which can make arbitrarily complex source transformations. Gofix was an invaluable tool during the early days when we regularly made breaking changes to the language and libraries.
Introducing Gofix:
Gofix gives us the ability to fix mistakes or completely rethink package APIs without worrying about the cost of converting existing code.
Python has its own analog to gofix in 2to3, though that was built for a specific migration rather than as a regular language feature. I wonder if having a gofmt analog in Python would similarly facilitate future migrations, though I doubt we’ll ever have a big as migration as the 2-to-3 one.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Mar 17, 2015 at 6:16 PM, Andrew Barnert <abarnert@yahoo.com> wrote:
So, is your proposal that autopep8 (and therefore pep8, on which it depends) be added to the stdlib with only minor changes?
Not necessarily, though that is certainly a possible approach. (In fact, it's probably the approach most likely to succeed, since it builds on existing and tested tools.) I was more interested in first testing the _idea_ of an included auto-styler before proposing a specific path to implementing it. On Tue, Mar 17, 2015 at 6:28 PM, Steven D'Aprano <steve@pearwood.info> wrote: Unlike Go-lang, which has the resources of Google behind it and can just
pay some person or persons to work on this with as high a priority as Google wishes to give it, Python is reliant on volunteers. For major proposals like this, you're likely to get much more traction if you are volunteering yourself, rather than just hoping to convince somebody else to volunteer.
Totally understood and agreed. To be upfront, I am not volunteering myself for this effort just yet. This is my first time posting to python-ideas. (I subscribed today.) I gather from Andrew and Steven's responses that ideas discussed on this list are supposed to be more "fully baked" than my proposal here. My apologies. For what it's worth, I certainly wouldn't expect myself or anyone else to invest any effort in an initiative like this if there is a chance that it will be rejected for reasons that can be explained upfront. On the other hand, if there is some consensus that this would be good thing to have at least in theory, then perhaps that would better set the stage for it to happen in practice. Hence the motivation for my original post. Nick
On Wed, Mar 18, 2015 at 12:55:17AM +0000, Nicholas Chammas wrote:
This is my first time posting to python-ideas. (I subscribed today.) I gather from Andrew and Steven's responses that ideas discussed on this list are supposed to be more "fully baked" than my proposal here. My apologies.
Not necessarily. This list frequently includes half-baked ideas, but the "bigger" the proposal, the more baked it should be to be taken seriously. I can proposal that Python should include a fully-functioning self-aware computer AI, but unless I actually have a fully-functioning self-aware AI written in Python to include, I might as well just wish for a flying pony. On the other hand, "Add math.zero through math.ten constants" doesn't need anything but a proposal to be discussed. Still, occassionally its useful to make a really big proposal. Even if it doesn't end up being accepted, it might lead somewhere interesting. -- Steve
On Tue, Mar 17, 2015 at 06:31:10PM +0000, Nicholas Chammas wrote:
Additionally having a standardized style that can be *applied automatically* makes whatever benefits of the style guide that much more prevalent in the greater community and actually effective.
If you're talking about Gofmt and Gofix and enforcing PEP-8 rules for variable names, you're basically talking about refactoring tools, not merely source code reformatters. Do you have such tools you would like to propose for the standard library? If you do have such tools, or would like to volunteer to write them, then they should spend some time as a third party project to knock the sharp edges off them. Once their usefulness is well-established and proven, and their API and featureset stable, then they can be proposed for the standard library. Unlike Go-lang, which has the resources of Google behind it and can just pay some person or persons to work on this with as high a priority as Google wishes to give it, Python is reliant on volunteers. For major proposals like this, you're likely to get much more traction if you are volunteering yourself, rather than just hoping to convince somebody else to volunteer. I think good refactoring tools for Python would be useful. I'm not volunteering to write them. -- Steve
On 3/17/2015 2:31 PM, Nicholas Chammas wrote:
The Zen of Python <https://www.python.org/dev/peps/pep-0020/> says:
There should be one— and preferably only one —obvious way to do it.
Python already has a style guide in PEP 8 <https://www.python.org/dev/peps/pep-0008/>. More importantly, from a practical standpoint, Python has automated third-party tools for checking and enforcing PEP 8 guidelines:
* |pep8| <https://pypi.python.org/pypi/pep8/>: Python style guide checker * |autopep8| <https://pypi.python.org/pypi/autopep8/>: A tool that automatically formats Python code to conform to the PEP 8 style guide
Except for idlelib and more recently turtledemo, the stdlib consists of *library* modules that define constants, mostly functions and classes, for import into and use by other libraries and applications. Notice that the stdlib contains lib2to3, while the 2to3 application that uses lib2to3 is in /Tools/Scripts, along with other application scripts, mostly short, written by core developers. pep8, autopep8, and similar applications do not belong in the stdlib. They are easily installed with, for instance, 'pip install autopep8'. The install process puts a startup file in /Scripts, which (for Windows at least) is on PATH (unlike /Tools/Scripts). After installing, I entered 'autopep8 file.py' and it runs. There is an issue on the tracker to add to the Idle IDE a gui frontend to pip, so beginners who use Idle will be able to manage (install, update, delete) 3rd-party apps like these without knowing about 'command lines' or having to remember pip options. (There is also an issue to apply such apps to the contents of an Editor window.) -- Terry Jan Reedy
On Tue, Mar 17, 2015 at 8:04 PM, Terry Reedy <tjreedy@udel.edu> wrote: pep8, autopep8, and similar applications do not belong in the stdlib. They
are easily installed with, for instance, 'pip install autopep8'.
I am not proposing putting autopep8 in the standard library. I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on *included* and *first-class*. That auto-styler may well be based somehow on autopep8, but that is a separate discussion about *implementation*. I am first checking that the *idea* is sound. I can’t tell if you had any objections to that idea. Yes, I know that today I can install and use autopep8 as a third-party utility using pip, but autopep8 today is neither included with Python nor a first-class utility. (FWIW, couldn’t autopep8 somehow be adapted to be an included module that can be invoked via python -m <https://www.python.org/dev/peps/pep-0338/> — e.g. python -m compileall, python -m SimpleHTTPServer — just as far as the mechanics are concerned? As I noted in my original post, I’m not in a position yet to specifically propose how an included auto-styler would work, but let’s say for argument’s sake that 1) we decided we liked the idea and 2) autopep8 fit the bill for promotion to an included utility, and 3) we addressed Andrew’s points.) Nick
On 03/17/2015 06:54 PM, Nicholas Chammas wrote:
I am not proposing putting |autopep8| in the standard library. I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on /included/ and /first-class/.
Perhaps you could explain what you mean by a /first-class/ utility? -- ~Ethan~
First class means it's like a language feature. Actually, in my case I am using "first-class" and "included with the language" synonymously. Not sure if that's appropriate terminology, but that's what I mean. To explain by rough analogy, Twisted is to asyncio what autopep8 would be to a first-class auto-styler. (I say "rough" because I understand that Twisted is not supposed to be replaced entirely by asyncio, but I hope my point is clear.) Nick On Tue, Mar 17, 2015 at 9:58 PM Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/17/2015 06:54 PM, Nicholas Chammas wrote:
I am not proposing putting |autopep8| in the standard library. I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on /included/ and /first-class/.
Perhaps you could explain what you mean by a /first-class/ utility?
-- ~Ethan~
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Interesting coincidence :-) We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code. It should be posted to Github within the next few days or weeks, under a permissive license (Apache 2.0 or something similar). The tool works like gofmt and clang-format (the C++/JS version provided as part of the Clang toolchain) - it only acts on whitespace and performs re-formatting of code (PEP-8 oriented for Python but it's configurable). It does not rename variables/classes/functions to conform to any particular style. The algorithm is based on clang-format, and lib2to3 is used for the parsing side - so it's very solid. We use it on Python 2 code internally, but Python 3 support is also planned and should be easy. We intend to keep actively developing the tool in the open and will be happy to accept test cases, fixes and patches in general. So stay tuned! P.S. I'm also against such a tool being a part of the Python distribution, at least in the near term. A tool/library has to first prove itself outside of core for a while, and only when/if it proves to be widely used and stable, an incorporation into core should be considered. The stdlib is big as it is, and any additional code is a burden on all core developers. Incorporation into the stdlib also imposes a very slow pace of development/release on said library/tool, and hence should only be considered when it's been extremely stable for a while. Eli On Tue, Mar 17, 2015 at 7:17 PM, Nicholas Chammas < nicholas.chammas@gmail.com> wrote:
First class means it's like a language feature. Actually, in my case I am using "first-class" and "included with the language" synonymously. Not sure if that's appropriate terminology, but that's what I mean.
To explain by rough analogy, Twisted is to asyncio what autopep8 would be to a first-class auto-styler. (I say "rough" because I understand that Twisted is not supposed to be replaced entirely by asyncio, but I hope my point is clear.)
Nick
On Tue, Mar 17, 2015 at 9:58 PM Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/17/2015 06:54 PM, Nicholas Chammas wrote:
I am not proposing putting |autopep8| in the standard library. I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on /included/ and /first-class/.
Perhaps you could explain what you mean by a /first-class/ utility?
-- ~Ethan~
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong? What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this? And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind). Good coding style really should not be something we ship in the library, it should be something we teach. 2¢ On Tue, Mar 17, 2015 at 7:17 PM, Nicholas Chammas <nicholas.chammas@gmail.com <mailto:nicholas.chammas@gmail.com>> wrote:
First class means it's like a language feature. Actually, in my case I am using "first-class" and "included with the language" synonymously. Not sure if that's appropriate terminology, but that's what I mean.
To explain by rough analogy, Twisted is to asyncio what autopep8 would be to a first-class auto-styler. (I say "rough" because I understand that Twisted is not supposed to be replaced entirely by asyncio, but I hope my point is clear.)
Nick
On Tue, Mar 17, 2015 at 9:58 PM Ethan Furman <ethan@stoneleaf.us <mailto:ethan@stoneleaf.us>> wrote:
On 03/17/2015 06:54 PM, Nicholas Chammas wrote:
> I am not proposing putting |autopep8| in the standard library. I am proposing that an auto-styler be included with > Python as a first-class utility. The emphasis here is on /included/ and /first-class/.
Perhaps you could explain what you mean by a /first-class/ utility?
-- ~Ethan~
_______________________________________________ Python-ideas mailing list Python-ideas@python.org <mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org <mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Mar 18, 2015 at 6:42 AM, Alexander Walters <tritium-list@sdamon.com> wrote:
What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong?
+1 to make it optional. I use 2 space indent for most of my code. I'd also add that policy to enforce rigid rules is harmful. The recent normalization of Python package versions made some of my apps that use semantic version invalid for PyPI.
On Tue, Mar 17, 2015 at 11:29 PM, Eli Bendersky <eliben@gmail.com> wrote:
We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code.
Interesting. Are y'all familiar with `autopep8`? What are the differences between the tool y'all are releasing and `autopep8`? On Tue, Mar 17, 2015 at 11:29 PM, Eli Bendersky <eliben@gmail.com> wrote:
P.S. I'm also against such a tool being a part of the Python distribution, at least in the near term. A tool/library has to first prove itself outside of core for a while, and only when/if it proves to be widely used and stable, an incorporation into core should be considered. The stdlib is big as it is, and any additional code is a burden on all core developers. Incorporation into the stdlib also imposes a very slow pace of development/release on said library/tool, and hence should only be considered when it's been extremely stable for a while.
This makes sense. In fact, I recently watched a keynote that Guido gave and he said something similar <https://youtu.be/EBRMq2Ioxsc?t=49m23s>. On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters <tritium-list@sdamon.com
wrote:
What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong?
What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this?
I might be wrong, but I see PEP 8 as a living document. If the community believes parts of it are out of date, I would think it's better to update PEP 8 (or issue a replacement PEP) to reflect current thinking. Any auto-styling tool included in the standard library would work best if it was based on a written "spec" like PEP 8 (granted, I don't think PEP 8 was meant to be a spec for programmatic implementation, but you get the idea). If stuff is not part of PEP 8 that should be, or if parts of PEP 8 are agreed to be "wrong", then PEP 8 should be updated accordingly. Any auto-styling tool, I would hope, would just follow from PEP 8 (or a similar document). The tool should not be the specification. On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters <tritium-list@sdamon.com
wrote:
And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind).
I see what you mean. Several people have made comments to this effect, and this is a good point. I guess this is a separate discussion, but is there some way to offer users the advantages of having a library be included with Python without imposing the 2-year release cycle and additional burden on the library's developers? This seems to be the root of the concern about including, for example, something like autopep8 with Python. On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters <tritium-list@sdamon.com
wrote:
Good coding style really should not be something we ship in the library, it should be something we teach.
This point is orthogonal to the discussion of whether or not to package an auto-styler, but there are many style points that are so mechanical I see little benefit in expending energy on teaching them. Best would be to automate as much of it away so that people can focus on bigger problems like, what is the right abstraction, did I design this API well, etc. Put differently, consider the following straw man: "Good memory management should not be something we ship in the language, it should be something we teach." There is always something to teach. But when we can automate part of the problem away, I think we generally should. On Wed, Mar 18, 2015 at 2:38 AM, Ian Lee <ianlee1521@gmail.com> wrote:
And perhaps more to the point, should PEP-8 (and if so how), evolve over time to incorporate newer ideas that the community has now moved towards accepting as the "more Pythonic way to do things" now with the benefit of additional hindsight.
I consider PEP 8 to be a living document. That is, it is already changing with time. For example, I see that though PEP 8 was created in 2001, it was updated in 2013 <https://www.python.org/dev/peps/pep-0008/#id8>. I assume it will continue to be updated as accepted standards change. On Wed, Mar 18, 2015 at 4:52 AM, Steven D'Aprano <steve@pearwood.info> wrote:
Not necessarily. This list frequently includes half-baked ideas, but the "bigger" the proposal, the more baked it should be to be taken seriously.
Duly noted! Nick
On Mar 18, 2015, at 9:52 AM, Nicholas Chammas <nicholas.chammas@gmail.com> wrote:
I might be wrong, but I see PEP 8 as a living document. If the community believes parts of it are out of date, I would think it's better to update PEP 8 (or issue a replacement PEP) to reflect current thinking.
Any auto-styling tool included in the standard library would work best if it was based on a written "spec" like PEP 8 (granted, I don't think PEP 8 was meant to be a spec for programmatic implementation, but you get the idea).
If stuff is not part of PEP 8 that should be, or if parts of PEP 8 are agreed to be "wrong", then PEP 8 should be updated accordingly. Any auto-styling tool, I would hope, would just follow from PEP 8 (or a similar document). The tool should not be the specification.
I agree with all of this. But that implies that it's worth someone making a spec out of PEP 8. That implies going through PEP 8, getting a sense of what parts are (unambiguously and uncontroversially) mechanizable, and reorganizing and minimally rewriting it to get that sense across. Having numbered rules, and named anchors for each one in the document (or at least to narrower sections, so you can link to something more specific than "Programming Recommendations") would also be helpful. I'm not sure such a thing would be accepted if you did it. But it seems like if it were rejected, the standard styler would be no more "standard" than what already exists--and likely to be rejected for the same reasons.
On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters <tritium-list@sdamon.com> wrote:
And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind).
I see what you mean. Several people have made comments to this effect, and this is a good point.
I guess this is a separate discussion, but is there some way to offer users the advantages of having a library be included with Python without imposing the 2-year release cycle and additional burden on the library's developers?
It depends what you mean. First, if pip comes with Python, and the library doesn't require C compilation, users already pretty much have that advantage. (And the really nice thing is that 2.7, 3.4, etc. users also have that advantage, which isn't true for something added to the stdlib in 3.5.) It might even be possible to have language in the official docs and/or the Getting Started or other parts of the site that suggest (as strongly as seems appropriate) that users "pip install autopep8" and use it. (I'm saying "autopep8" as a shorthand for "autopep8, or whatever alternative you'd end up suggesting if you did".) If the suggestion that IDLE add a graphical package manager wrapping pip goes forward, it could presumably add some kind of "recommended packages" feature, maybe even suggesting it at first run, but it seems like you'd have to jump-start the discussion among IDLE developers and users and then bring it back here, or wait and see if they add such a thing so you can suggest adding autopep8 to the list. Beyond that, there's really no way to be "included without being included". If it's in the standard source and binary distributions, it's part of the release cycle. I suppose you could have something that came with Python but could be updated with pip, with users strongly suggested to update it--but that doesn't seem like a good idea; if you want to get everyone to pip install -U it, the best way to do that is not to give them an obsolete version, but to just not give it to them. And bootstrapping it the same way as pip doesn't seem reasonable (it's reasonable for pip because "just pip install pip" obviously doesn't work--sometimes, special cases _are_ special enough...). One thing that might be reasonable is to include an autopep8 script that runs autopep8 if present, and otherwise just tells you it's not present and suggests pip installing it (similar to what Apple does with Java and X11 binaries). Of course nothing else in Python works that way today, and I suspect a lot of people would resist it, but technically it's certainly doable, if you want to make a case for it. Finally, remember that not everyone gets Python through an official Python.org installer. Almost every non-Windows, non-mobile platform comes with a "system python", and also has a package manager (possibly third-party, like Homebrew on Mac) that many people who want a different Python turn to. And some people build from source. And others (including on Windows) use third-party Python distributions like Enthought or PythonWin. To these users, what "comes with" Python is not the same thing that Python.org crams into the Windows and Mac installers--and can be much more up-to-date (but isn't automatically, of course). Apple's system Python comes with PyObjC. Some linux distros split off parts of the stdlib into packages named "python-spam" and then have non-stdlib packages in the same repo named "python-eggs". Enthought comes with the SciPy stack, and PythonWin with PyWin32. Anaconda has its own package management system, and a set of recommended extra packages. And so on.
This seems to be the root of the concern about including, for example, something like autopep8 with Python.
On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters <tritium-list@sdamon.com> wrote: Good coding style really should not be something we ship in the library, it should be something we teach.
This point is orthogonal to the discussion of whether or not to package an auto-styler, but there are many style points that are so mechanical I see little benefit in expending energy on teaching them. Best would be to automate as much of it away so that people can focus on bigger problems like, what is the right abstraction, did I design this API well, etc.
Put differently, consider the following straw man: "Good memory management should not be something we ship in the language, it should be something we teach."
That's a great example, because it's a little of both. Python automates memory management as far as is reasonable, and makes it easy for people who need to do more (clear recommendations on when to del/reassign variables, tools like weakrefdict, etc.), so some users don't need to be taught anything, and the more advanced users can be taught more easily than in, say, Objective C.
There is always something to teach. But when we can automate part of the problem away, I think we generally should.
On Wed, Mar 18, 2015 at 4:52 AM, Steven D'Aprano <steve@pearwood.info> wrote: Not necessarily. This list frequently includes half-baked ideas, but the "bigger" the proposal, the more baked it should be to be taken seriously.
Duly noted!
Also keep in mind that, to whatever extent it isn't fully-baked, it should be clear that it is at least bakeable, and that you look willing and able to do the baking. That's a bit harder for a brand new list member than for, say, Guido (his initial proposal for asyncio, for example, wouldn't have gotten the same interest of it was your initial proposal), but it's doable, and necessary.
On 3/18/2015 12:52 PM, Nicholas Chammas wrote:
I might be wrong, but I see PEP 8 as a living document.
PEP 8 is Guido's view on how new CPython stdlib code should be written, with input from other core developers. "This document gives coding conventions for the Python code comprising the standard library in the main Python distribution." It changes as he and we change our views. It is against Guido's policy to blindly change old stdlib code to follow PEP 8. (Note that autoPEP8 only changes whitespace, which is a small part of PEP 8 recommendations, and the safest aspect of code to change.)
If the community believes parts of it are out of date, I would think it's better to update PEP 8 (or issue a replacement PEP) to reflect current thinking.
The 'community' is free to follow it or not, but not to change it. Other people and organizations have written their own style guides. Good for them.
If stuff is not part of PEP 8 that should be, or if parts of PEP 8 are agreed to be "wrong", then PEP 8 should be updated accordingly.
PEP 8 is by definition 'right' until Guido decides otherwise, with input from other core devs.
I guess this is a separate discussion, but is there some way to offer users the advantages of having a library be included with Python without imposing the 2-year release cycle and additional burden on the library's developers?
Use pip. It was added to make pypi (and other repositories) an easy-to-use extension of the stdlib, without imposing stdlib requirements and burdens. We are working on making pip even easier to use. -- Terry Jan Reedy
On Wed, Mar 18, 2015 at 9:52 AM, Nicholas Chammas < nicholas.chammas@gmail.com> wrote:
On Tue, Mar 17, 2015 at 11:29 PM, Eli Bendersky <eliben@gmail.com> wrote:
We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code.
Interesting. Are y'all familiar with `autopep8`?
Yes, very familiar.
What are the differences between the tool y'all are releasing and `autopep8`?
autopep8's scope is both smaller and larger. It only detects PEP8 violations and attempts to fix them, not restricting itself to whitespace. Our tool works more like the gofmt tool mentioned in the beginning of the thread - it pretty much only cares about formatting, and it auto-reformats the code to its own liking, covering more whitespace-related stuff than PEP8. Basically the idea is "forget about formatting your code, just run it through the tool", which is a similar philosophy as gofmt (and clang-format) and has been hugely successful for those tools/languages. Eli
On Tue, Mar 17, 2015 at 11:29 PM, Eli Bendersky <eliben@gmail.com> wrote:
P.S. I'm also against such a tool being a part of the Python distribution, at least in the near term. A tool/library has to first prove itself outside of core for a while, and only when/if it proves to be widely used and stable, an incorporation into core should be considered. The stdlib is big as it is, and any additional code is a burden on all core developers. Incorporation into the stdlib also imposes a very slow pace of development/release on said library/tool, and hence should only be considered when it's been extremely stable for a while.
This makes sense. In fact, I recently watched a keynote that Guido gave and he said something similar <https://youtu.be/EBRMq2Ioxsc?t=49m23s>.
On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters < tritium-list@sdamon.com> wrote:
What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong?
What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this?
I might be wrong, but I see PEP 8 as a living document. If the community believes parts of it are out of date, I would think it's better to update PEP 8 (or issue a replacement PEP) to reflect current thinking.
Any auto-styling tool included in the standard library would work best if it was based on a written "spec" like PEP 8 (granted, I don't think PEP 8 was meant to be a spec for programmatic implementation, but you get the idea).
If stuff is not part of PEP 8 that should be, or if parts of PEP 8 are agreed to be "wrong", then PEP 8 should be updated accordingly. Any auto-styling tool, I would hope, would just follow from PEP 8 (or a similar document). The tool should not be the specification.
On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters < tritium-list@sdamon.com> wrote:
And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind).
I see what you mean. Several people have made comments to this effect, and this is a good point.
I guess this is a separate discussion, but is there some way to offer users the advantages of having a library be included with Python without imposing the 2-year release cycle and additional burden on the library's developers?
This seems to be the root of the concern about including, for example, something like autopep8 with Python.
On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters < tritium-list@sdamon.com> wrote:
Good coding style really should not be something we ship in the library, it should be something we teach.
This point is orthogonal to the discussion of whether or not to package an auto-styler, but there are many style points that are so mechanical I see little benefit in expending energy on teaching them. Best would be to automate as much of it away so that people can focus on bigger problems like, what is the right abstraction, did I design this API well, etc.
Put differently, consider the following straw man: "Good memory management should not be something we ship in the language, it should be something we teach."
There is always something to teach. But when we can automate part of the problem away, I think we generally should.
On Wed, Mar 18, 2015 at 2:38 AM, Ian Lee <ianlee1521@gmail.com> wrote:
And perhaps more to the point, should PEP-8 (and if so how), evolve over time to incorporate newer ideas that the community has now moved towards accepting as the "more Pythonic way to do things" now with the benefit of additional hindsight.
I consider PEP 8 to be a living document. That is, it is already changing with time. For example, I see that though PEP 8 was created in 2001, it was updated in 2013 <https://www.python.org/dev/peps/pep-0008/#id8>. I assume it will continue to be updated as accepted standards change.
On Wed, Mar 18, 2015 at 4:52 AM, Steven D'Aprano <steve@pearwood.info> wrote:
Not necessarily. This list frequently includes half-baked ideas, but the "bigger" the proposal, the more baked it should be to be taken seriously.
Duly noted!
Nick
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
OK, I hereby abandon this proposal. :) To summarize the conclusion I believe this thread has arrived at: - An auto-styler needs to have wide community adoption before it can be considered for inclusion into the standard library. - Even then, adding stuff to the standard library at this point in Python’s life is generally a bad thing. Putting an auto-styler in the standard library would be a big burden on its maintainers and slow down improvements that could be made to it. - Ignoring that difficulty, there is also the problem of what “spec” to use for any such auto-styler. PEP 8 is an obvious choice, but it was written as a guide for humans, not a spec for programmatic implementation. Furthermore, there are doubts about PEP 8’s utility as the basis for a universal auto-styler, as it was intended for code specifically in the Python standard library. - Let a thousand auto-stylers bloom outside the standard library. Some final responses to various people’s comments: On Wed, Mar 18, 2015 at 4:38 PM, Andrew Barnert <abarnert@yahoo.com> wrote: But that implies that it's worth someone making a spec out of PEP 8. That
implies going through PEP 8, getting a sense of what parts are (unambiguously and uncontroversially) mechanizable, and reorganizing and minimally rewriting it to get that sense across. Having numbered rules, and named anchors for each one in the document (or at least to narrower sections, so you can link to something more specific than "Programming Recommendations") would also be helpful.
This is an excellent point and one I did not consider. On Wed, Mar 18, 2015 at 6:17 PM, Terry Reedy <tjreedy@udel.edu> wrote: PEP 8 is Guido's view on how new CPython stdlib code should be written,
with input from other core developers. "This document gives coding conventions for the Python code comprising the standard library in the main Python distribution."
Good point. For my all my talk about PEP 8, I missed this key point. Of course, it should be noted that PEP 8 is being used by all kinds of people for stuff outside of stdlib, but then again as you pointed out people are also putting together their own style guides. On Wed, Mar 18, 2015 at 6:17 PM, Terry Reedy <tjreedy@udel.edu> wrote: (Note that autoPEP8 only changes whitespace, which is a small part of PEP 8
recommendations, and the safest aspect of code to change.)
Minor point: autopep8 can do more than just whitespace changes with the --aggressive flag <https://pypi.python.org/pypi/autopep8/#more-advanced-usage>. On Wed, Mar 18, 2015 at 7:21 PM, Terry Reedy <tjreedy@udel.edu> wrote: We should be careful about adding anything. When we do, we should add
libraries, not applications. We should especially not officially endorse one of many opinion-based competitors.
I have come around to this view. On Wed, Mar 18, 2015 at 7:30 PM, Eli Bendersky <eliben@gmail.com> wrote: Basically the idea is "forget about formatting your code, just run it
through the tool", which is a similar philosophy as gofmt (and clang-format) and has been hugely successful for those tools/languages.
Sounds good. I’m looking forward to seeing it released. As I said in an earlier email, I think any part of the development process we automate away is a good thing (stdlib inclusion or not notwithstanding). Nick
On 3/19/2015 10:32 AM, Nicholas Chammas wrote:
OK, I hereby abandon this proposal. :)
Thank you for being so sensible ;-) Let me first quote (out of order) your conclusion:
I think any part of the development process we automate away is a good thing (stdlib inclusion or not notwithstanding).>
I agree, which is why I expect to do some work making it easier to download 3rd party modules and apply them to python code, both in a gui IDE framework (Idle). I think automation should respect and augment, not replace. the human programmer.
To summarize the conclusion I believe this thread has arrived at:
...
Ignoring that difficulty, there is also the problem of what “spec” to use for any such auto-styler. PEP 8 is an obvious choice, but it was written as a guide for humans, not a spec for programmatic implementation.
A program that treats PEP 8 guidelines as rigid rules, that ignores the first guideline "A Foolish Consistency is the Hobgoblin of Little Minds", that removes the human choices intentionally included in PEP 8, is not PEP 8 compliant. I believe one of the motivations for some recent PEP 8 amendments was some unhappiness with pep8.py or other checkers.
Furthermore, there are doubts about PEP 8’s utility as the basis for a universal auto-styler, as it was intended for code specifically in the Python standard library.
Pydev is a relatively small group of volunteers who otherwise work either for themselves or for various organizations scattered around the world. Having the freedom to put actual readability ahead of rigid consistency makes volunteering more attractive. I regard code as a form of poetry, but prefer a flexible but defined structure to free-form mishmash. Google, as an example, is a very large corporation that pays people well to adhere to its guidelines and rules. This is a different situation and its best choice for Python styling may well be different from pydev.
On Wed, Mar 18, 2015 at 4:38 PM, Andrew Barnert <abarnert@yahoo.com <mailto:abarnert@yahoo.com>> wrote:
But that implies that it's worth someone making a spec out of PEP 8. That implies going through PEP 8, getting a sense of what parts are (unambiguously and uncontroversially) mechanizable, and reorganizing and minimally rewriting it to get that sense across. Having numbered rules, and named anchors for each one in the document (or at least to narrower sections, so you can link to something more specific than "Programming Recommendations") would also be helpful.
This is an excellent point and one I did not consider.
Having the guidelines numbered (id'ed), even though still regarded as guidelines, could help communication. Checkers could then easily refer to specific guidelines.
On Wed, Mar 18, 2015 at 6:17 PM, Terry Reedy <tjreedy@udel.edu <mailto:tjreedy@udel.edu>> wrote:
(Note that autoPEP8 only changes whitespace, which is a small part of PEP 8 recommendations, and the safest aspect of code to change.)
Minor point: autopep8 can do more than just whitespace changes with the |--aggressive| flag <https://pypi.python.org/pypi/autopep8/#more-advanced-usage>.
Thanks for the link. I was specifically thinking of global renamings to satisfy PEP 8's Naming Conventions. Idlelib has a mishmash of module and function/method name styles. (I might look and see what can be done with the undocumented (except as 'unstable') libe2to3.)
On Wed, Mar 18, 2015 at 7:21 PM, Terry Reedy <tjreedy@udel.edu <mailto:tjreedy@udel.edu>> wrote:
We should be careful about adding anything. When we do, we should add libraries, not applications. We should especially not officially endorse one of many opinion-based competitors.
I have come around to this view.
Accepting this view, we can still ask if the stdlib should have more (policy-neutral) string or list of strings reforming components than it does now. Textwrap has a few functions for strings in general. Idle has about 10 formatting functions exposed on its editor Format menu, but they are not exposed for general use. Tools/Scripts/reindent.py contain some of the same functions as does autopep8. Should there be a code format module that all three, and other programs, could import? Idle and Tools/Scrips/highlight.py both have code to analyze Python code tokens into categories and colorize by category. (Idle uses tkinter tags, highlight.py uses ANSI or HTML tags). Can and should both use a common code analysis function that could be imported? This might actually ease maintainance. Should a generalized 2to3 package be renamed 'codefix' and documented? As 2to3, it is already used by both Tools/Scripts/2to2 and optionally by autopep8, and perhaps by others. I believe it could also be used for some non-2to3 pep8 fixes. -- Terry Jan Reedy
On Thu, Mar 19, 2015 at 1:40 PM, Terry Reedy <tjreedy@udel.edu> wrote:
A program that treats PEP 8 guidelines as rigid rules, that ignores the first guideline "A Foolish Consistency is the Hobgoblin of Little Minds", that removes the human choices intentionally included in PEP 8, is not PEP 8 compliant. I believe one of the motivations for some recent PEP 8 amendments was some unhappiness with pep8.py or other checkers.
I have always been skeptical of automatic style checkers. They encourage a nit-pickish attitude that makes the development process less effective, by encouraging critique of superficial style elements rather than of the code's structure and meaning. In contrast, the intention of PEP 8 was to make code more readable so that its structure and meaning are more easily understood.
On Wed, Mar 18, 2015 at 4:38 PM, Andrew Barnert
<abarnert@yahoo.com <mailto:abarnert@yahoo.com>> wrote:
But that implies that it's worth someone making a spec out of PEP 8. That implies going through PEP 8, getting a sense of what parts are (unambiguously and uncontroversially) mechanizable, and reorganizing and minimally rewriting it to get that sense across. Having numbered rules, and named anchors for each one in the document (or at least to narrower sections, so you can link to something more specific than "Programming Recommendations") would also be helpful.
This is an excellent point and one I did not consider.
Having the guidelines numbered (id'ed), even though still regarded as guidelines, could help communication. Checkers could then easily refer to specific guidelines.
I worry this will just encourage the nit-picky attitude more.
On Wed, Mar 18, 2015 at 6:17 PM, Terry Reedy <tjreedy@udel.edu
<mailto:tjreedy@udel.edu>> wrote:
(Note that autoPEP8 only changes whitespace, which is a small part of PEP 8 recommendations, and the safest aspect of code to change.)
Minor point: autopep8 can do more than just whitespace changes with the |--aggressive| flag <https://pypi.python.org/pypi/autopep8/#more-advanced-usage>.
Thanks for the link. I was specifically thinking of global renamings to satisfy PEP 8's Naming Conventions. Idlelib has a mishmash of module and function/method name styles. (I might look and see what can be done with the undocumented (except as 'unstable') libe2to3.)
This feels hard to automate, because you don't know what names are part of an external specification.
On Wed, Mar 18, 2015 at 7:21 PM, Terry Reedy <tjreedy@udel.edu
<mailto:tjreedy@udel.edu>> wrote:
We should be careful about adding anything. When we do, we should add libraries, not applications. We should especially not officially endorse one of many opinion-based competitors.
I have come around to this view.
Accepting this view, we can still ask if the stdlib should have more (policy-neutral) string or list of strings reforming components than it does now. Textwrap has a few functions for strings in general. Idle has about 10 formatting functions exposed on its editor Format menu, but they are not exposed for general use. Tools/Scripts/reindent.py contain some of the same functions as does autopep8. Should there be a code format module that all three, and other programs, could import?
Idle and Tools/Scrips/highlight.py both have code to analyze Python code tokens into categories and colorize by category. (Idle uses tkinter tags, highlight.py uses ANSI or HTML tags). Can and should both use a common code analysis function that could be imported? This might actually ease maintainance.
Should a generalized 2to3 package be renamed 'codefix' and documented? As 2to3, it is already used by both Tools/Scripts/2to2 and optionally by autopep8, and perhaps by others. I believe it could also be used for some non-2to3 pep8 fixes.
I think that a quick "clean up whitespace" feature would be a nice addition to IDLE, provided it's a user-selectable menu items (similar to the existing indent/dedent operations). Refactoring, however, is fiendishly hard for Python -- give it a try in e.g. PyCharm and you will agree. -- --Guido van Rossum (python.org/~guido)
On Mar 19, 2015, at 5:01 PM, Guido van Rossum <guido@python.org> wrote:
On Thu, Mar 19, 2015 at 1:40 PM, Terry Reedy <tjreedy@udel.edu <mailto:tjreedy@udel.edu>> wrote: A program that treats PEP 8 guidelines as rigid rules, that ignores the first guideline "A Foolish Consistency is the Hobgoblin of Little Minds", that removes the human choices intentionally included in PEP 8, is not PEP 8 compliant. I believe one of the motivations for some recent PEP 8 amendments was some unhappiness with pep8.py or other checkers.
I have always been skeptical of automatic style checkers. They encourage a nit-pickish attitude that makes the development process less effective, by encouraging critique of superficial style elements rather than of the code's structure and meaning. In contrast, the intention of PEP 8 was to make code more readable so that its structure and meaning are more easily understood.
My experience is actually the exact opposite. When a machine is responsible for style enforcement people generally don’t feel as bad about being asked to adjust the formatting because it’s just a machine, it can’t make value judgements. When a person asks someone to adjust something to make it easier to read it often times ends up causing a distraction where people tend to feel like since it’s a person asking them to do it they are passing judgement on their code, or they don’t agree with it and since it’s a human saying it it feels more like there is room to budge so you end up with people more likely to be willing to ask for an exemption. In my experience projects without mechnical enforcement ends up with massively inconsistent style all throughout the code base. The Python stdlib is a good example of this. Contrast this to projects like github.com/pyca/cryptography <http://github.com/pyca/cryptography> where there is an automated mechanical enforcement of style and it’s got some bare minimum of consistency throughout the entire code base.
On Wed, Mar 18, 2015 at 4:38 PM, Andrew Barnert <abarnert@yahoo.com <mailto:abarnert@yahoo.com> <mailto:abarnert@yahoo.com <mailto:abarnert@yahoo.com>>> wrote:
But that implies that it's worth someone making a spec out of PEP 8. That implies going through PEP 8, getting a sense of what parts are (unambiguously and uncontroversially) mechanizable, and reorganizing and minimally rewriting it to get that sense across. Having numbered rules, and named anchors for each one in the document (or at least to narrower sections, so you can link to something more specific than "Programming Recommendations") would also be helpful.
This is an excellent point and one I did not consider.
Having the guidelines numbered (id'ed), even though still regarded as guidelines, could help communication. Checkers could then easily refer to specific guidelines.
I worry this will just encourage the nit-picky attitude more.
On Wed, Mar 18, 2015 at 6:17 PM, Terry Reedy <tjreedy@udel.edu <mailto:tjreedy@udel.edu> <mailto:tjreedy@udel.edu <mailto:tjreedy@udel.edu>>> wrote:
(Note that autoPEP8 only changes whitespace, which is a small part of PEP 8 recommendations, and the safest aspect of code to change.)
Minor point: autopep8 can do more than just whitespace changes with the |--aggressive| flag <https://pypi.python.org/pypi/autopep8/#more-advanced-usage <https://pypi.python.org/pypi/autopep8/#more-advanced-usage>>.
Thanks for the link. I was specifically thinking of global renamings to satisfy PEP 8's Naming Conventions. Idlelib has a mishmash of module and function/method name styles. (I might look and see what can be done with the undocumented (except as 'unstable') libe2to3.)
This feels hard to automate, because you don't know what names are part of an external specification.
On Wed, Mar 18, 2015 at 7:21 PM, Terry Reedy <tjreedy@udel.edu <mailto:tjreedy@udel.edu> <mailto:tjreedy@udel.edu <mailto:tjreedy@udel.edu>>> wrote:
We should be careful about adding anything. When we do, we should add libraries, not applications. We should especially not officially endorse one of many opinion-based competitors.
I have come around to this view.
Accepting this view, we can still ask if the stdlib should have more (policy-neutral) string or list of strings reforming components than it does now. Textwrap has a few functions for strings in general. Idle has about 10 formatting functions exposed on its editor Format menu, but they are not exposed for general use. Tools/Scripts/reindent.py contain some of the same functions as does autopep8. Should there be a code format module that all three, and other programs, could import?
Idle and Tools/Scrips/highlight.py both have code to analyze Python code tokens into categories and colorize by category. (Idle uses tkinter tags, highlight.py uses ANSI or HTML tags). Can and should both use a common code analysis function that could be imported? This might actually ease maintainance.
Should a generalized 2to3 package be renamed 'codefix' and documented? As 2to3, it is already used by both Tools/Scripts/2to2 and optionally by autopep8, and perhaps by others. I believe it could also be used for some non-2to3 pep8 fixes.
I think that a quick "clean up whitespace" feature would be a nice addition to IDLE, provided it's a user-selectable menu items (similar to the existing indent/dedent operations). Refactoring, however, is fiendishly hard for Python -- give it a try in e.g. PyCharm and you will agree.
-- --Guido van Rossum (python.org/~guido <http://python.org/~guido>) _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
From what I’ve seen so far, the process of a patch failing style checks
I second Donald’s remarks regarding the impact that automatic style checkers have on the dev process. The Apache Spark <https://github.com/apache/spark> project, for example, makes a number of automated checks on all proposed patches. One of those checks is a style check. When a proposed patch fails a style check, an informative message explaining why is posted in the CI build log for the author to review, and the author typically just updates their patch accordingly. Here’s an example <https://github.com/apache/spark/pull/4916#issuecomment-77555917> from an Apache Spark PR that touched some Python code. The author’s patch had some blank lines that contained whitespace, so the patch failed the automated style check. The author just updated their patch to pass the style check and the discussion quickly returned to the meat of what the patch was about. plays out almost exactly like it would if a contributor had submitted a patch that failed unit tests. The contributor just updates their patch to pass those checks and everyone moves on. There is no arguing or friction. And the result in the case of the Spark project is that all the Python code in the repo passes the pep8 checks we’ve enabled. Granted, this doesn’t mean the code is readable or beautiful or correct, but it simply means that there is a small but non-trivial set of style problems and inconsistencies that we just don’t have to worry about. For a large, open source project with hundreds of contributors from many different organizations and backgrounds, that’s a solid win. Nick On Thu, Mar 19, 2015 at 5:42 PM Donald Stufft <donald@stufft.io> wrote:
On Mar 19, 2015, at 5:01 PM, Guido van Rossum <guido@python.org> wrote:
On Thu, Mar 19, 2015 at 1:40 PM, Terry Reedy <tjreedy@udel.edu> wrote:
A program that treats PEP 8 guidelines as rigid rules, that ignores the first guideline "A Foolish Consistency is the Hobgoblin of Little Minds", that removes the human choices intentionally included in PEP 8, is not PEP 8 compliant. I believe one of the motivations for some recent PEP 8 amendments was some unhappiness with pep8.py or other checkers.
I have always been skeptical of automatic style checkers. They encourage a nit-pickish attitude that makes the development process less effective, by encouraging critique of superficial style elements rather than of the code's structure and meaning. In contrast, the intention of PEP 8 was to make code more readable so that its structure and meaning are more easily understood.
My experience is actually the exact opposite. When a machine is responsible for style enforcement people generally don’t feel as bad about being asked to adjust the formatting because it’s just a machine, it can’t make value judgements. When a person asks someone to adjust something to make it easier to read it often times ends up causing a distraction where people tend to feel like since it’s a person asking them to do it they are passing judgement on their code, or they don’t agree with it and since it’s a human saying it it feels more like there is room to budge so you end up with people more likely to be willing to ask for an exemption.
In my experience projects without mechnical enforcement ends up with massively inconsistent style all throughout the code base. The Python stdlib is a good example of this. Contrast this to projects like github.com/pyca/cryptography where there is an automated mechanical enforcement of style and it’s got some bare minimum of consistency throughout the entire code base.
On Wed, Mar 18, 2015 at 4:38 PM, Andrew Barnert
<abarnert@yahoo.com <mailto:abarnert@yahoo.com>> wrote:
But that implies that it's worth someone making a spec out of PEP 8. That implies going through PEP 8, getting a sense of what parts are (unambiguously and uncontroversially) mechanizable, and reorganizing and minimally rewriting it to get that sense across. Having numbered rules, and named anchors for each one in the document (or at least to narrower sections, so you can link to something more specific than "Programming Recommendations") would also be helpful.
This is an excellent point and one I did not consider.
Having the guidelines numbered (id'ed), even though still regarded as guidelines, could help communication. Checkers could then easily refer to specific guidelines.
I worry this will just encourage the nit-picky attitude more.
On Wed, Mar 18, 2015 at 6:17 PM, Terry Reedy <tjreedy@udel.edu
<mailto:tjreedy@udel.edu>> wrote:
(Note that autoPEP8 only changes whitespace, which is a small part of PEP 8 recommendations, and the safest aspect of code to change.)
Minor point: autopep8 can do more than just whitespace changes with the |--aggressive| flag <https://pypi.python.org/pypi/autopep8/#more-advanced-usage>.
Thanks for the link. I was specifically thinking of global renamings to satisfy PEP 8's Naming Conventions. Idlelib has a mishmash of module and function/method name styles. (I might look and see what can be done with the undocumented (except as 'unstable') libe2to3.)
This feels hard to automate, because you don't know what names are part of an external specification.
On Wed, Mar 18, 2015 at 7:21 PM, Terry Reedy <tjreedy@udel.edu
<mailto:tjreedy@udel.edu>> wrote:
We should be careful about adding anything. When we do, we should add libraries, not applications. We should especially not officially endorse one of many opinion-based competitors.
I have come around to this view.
Accepting this view, we can still ask if the stdlib should have more (policy-neutral) string or list of strings reforming components than it does now. Textwrap has a few functions for strings in general. Idle has about 10 formatting functions exposed on its editor Format menu, but they are not exposed for general use. Tools/Scripts/reindent.py contain some of the same functions as does autopep8. Should there be a code format module that all three, and other programs, could import?
Idle and Tools/Scrips/highlight.py both have code to analyze Python code tokens into categories and colorize by category. (Idle uses tkinter tags, highlight.py uses ANSI or HTML tags). Can and should both use a common code analysis function that could be imported? This might actually ease maintainance.
Should a generalized 2to3 package be renamed 'codefix' and documented? As 2to3, it is already used by both Tools/Scripts/2to2 and optionally by autopep8, and perhaps by others. I believe it could also be used for some non-2to3 pep8 fixes.
I think that a quick "clean up whitespace" feature would be a nice addition to IDLE, provided it's a user-selectable menu items (similar to the existing indent/dedent operations). Refactoring, however, is fiendishly hard for Python -- give it a try in e.g. PyCharm and you will agree.
-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Donald Stufft writes:
Guido van Rossum writes:
I have always been skeptical of automatic style checkers. They encourage a nit-pickish attitude that makes the development process less effective, by encouraging critique of superficial style elements rather than of the code's structure and meaning. In contrast, the intention of PEP 8 was to make code more readable so that its structure and meaning are more easily understood.
My experience is actually the exact opposite.
This is one of those things where people differ. I'm happy to follow Guido on this, although personally I've been able to adapt to mechanical rules where required.
In my experience projects without mechnical enforcement ends up with massively inconsistent style all throughout the code base. The Python stdlib is a good example of this. Contrast this to projects like github.com/pyca/cryptography <http://github.com/pyca/cryptography> where there is an automated mechanical enforcement of style and it’s got some bare minimum of consistency throughout the entire code base.
Which proves what? That enforcing minimal style rules (and I think we all agree that there's plenty of good and bad style that machines can't yet reliably distinguish) results in a code base that follows minimal style rules?
On Mar 20, 2015, at 1:02 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Donald Stufft writes:
Guido van Rossum writes:
I have always been skeptical of automatic style checkers. They encourage a nit-pickish attitude that makes the development process less effective, by encouraging critique of superficial style elements rather than of the code's structure and meaning. In contrast, the intention of PEP 8 was to make code more readable so that its structure and meaning are more easily understood.
My experience is actually the exact opposite.
This is one of those things where people differ. I'm happy to follow Guido on this, although personally I've been able to adapt to mechanical rules where required.
In my experience projects without mechnical enforcement ends up with massively inconsistent style all throughout the code base. The Python stdlib is a good example of this. Contrast this to projects like github.com/pyca/cryptography <http://github.com/pyca/cryptography> where there is an automated mechanical enforcement of style and it’s got some bare minimum of consistency throughout the entire code base.
Which proves what? That enforcing minimal style rules (and I think we all agree that there's plenty of good and bad style that machines can't yet reliably distinguish) results in a code base that follows minimal style rules?
Sure, just like tests can’t guarantee that a code base works in every situation but that it works in some minimal set of tests (where minimal is whatever tests were written). Humans are inconsistent and emotional, people are more likely to nitpick a patch that they don’t particularly like in an unconscious attempt to discourage the contribution. They are also far more likely to let things slip when it’s a contribution from someone they like or even from themselves. Working on the Python stdlib is somewhat frustrating to me in this regard because the code in the stdlib is often times wildly inconsistent even within the same module. Maybe having inconsistent style which is inconsistently applied doesn’t bother you, that’s fine. I don’t particularly mind if the stdlib gains an automated style checker or formatter because I’m happy using third party modules. I was merely offering my experience that *anything* which relies on a human to verify it is, without exception, going to be verified unevenly and that using a human to verify it invites people to attempt to argue against it more often than when a machine does it. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Mar 19, 2015, at 10:12 PM, Donald Stufft <donald@stufft.io> wrote:
Working on the Python stdlib is somewhat frustrating to me in this regard because the code in the stdlib is often times wildly inconsistent even within the same module.
The stdlib is full of milkmen^H^H^H^H^H^H^Hmodules. Some of them are very old. Often older than PEP 8--and presumably the inspiration for PEP 8 to be written in the first place. At the time PEP 8 was written, obviously nobody went through and rewrote the whole stdlib to be compliant. Why not? Maybe no one volunteered--or maybe someone did, but it was rejected because the costs (risk of introducing new bugs, headaches every time you need to backport a fix across the rewrite, or even just look at the diffs to see what changed when, etc.) outweighed the benefits. Over time, some parts that touched on the visible interface, that needed to be rewritten anyway, or that are intended to serve as sample code (as in, the source is linked from the docs) have been restyled, but most of the code hasn't. Since most of those updates were in 3.0, and it seems an obvious place for a flag-day restyling if ever there were to be one, I'll bet if you search the py3k archives you'll find some relevant discussions. (Unless Guido or someone else knows off the top of his head.)
Donald Stufft writes:
Working on the Python stdlib is somewhat frustrating to me in this regard because the code in the stdlib is often times wildly inconsistent even within the same module.
Maybe you haven't paid attention to past discussions, but the Python stdlib is a terrible example because fixing it up is a FAQ on this list, on core-mentoriship, and (in the past) on python-dev. It has been repeatedly vetoed on the grounds that changes, even in whitespace, are likely to introduce more bugs than consistent style is worth. If you're actually working *on* the stdlib, then improve the style as you fix bugs or add features. If you are working on something that requires studying the stdlib, then you're out of luck. None of the above means that improving the stdlib's coding style is off the table as far as I'm concerned. I personally wouldn't mind seeing it happen, but I don't care that much. What it means is simply that the stdlib is the worst possible example to make your case -- everybody agrees with you about the inconsistent style, but Those Who Make Decisions Around Here think there are more important considerations than style.
I was merely offering my experience that *anything* which relies on a human to verify it is, without exception, going to be verified unevenly and that using a human to verify it invites people to attempt to argue against it more often than when a machine does it.
But the stdlib is a terrible example for that purpose for the same reason. But I take your point that using a machine can smooth correction of "mechanical" errors. Mine is simply that I don't see that it makes such a big difference in my experience; most people who have horrible coding styles respond to human advice quite well, and the "minor" differences are -- minor. YMMV.
On Mar 20, 2015, at 2:06 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Donald Stufft writes:
Working on the Python stdlib is somewhat frustrating to me in this regard because the code in the stdlib is often times wildly inconsistent even within the same module.
Maybe you haven't paid attention to past discussions, but the Python stdlib is a terrible example because fixing it up is a FAQ on this list, on core-mentoriship, and (in the past) on python-dev. It has been repeatedly vetoed on the grounds that changes, even in whitespace, are likely to introduce more bugs than consistent style is worth.
If you're actually working *on* the stdlib, then improve the style as you fix bugs or add features. If you are working on something that requires studying the stdlib, then you're out of luck.
None of the above means that improving the stdlib's coding style is off the table as far as I'm concerned. I personally wouldn't mind seeing it happen, but I don't care that much.
What it means is simply that the stdlib is the worst possible example to make your case -- everybody agrees with you about the inconsistent style, but Those Who Make Decisions Around Here think there are more important considerations than style.
I was merely offering my experience that *anything* which relies on a human to verify it is, without exception, going to be verified unevenly and that using a human to verify it invites people to attempt to argue against it more often than when a machine does it.
But the stdlib is a terrible example for that purpose for the same reason.
But I take your point that using a machine can smooth correction of "mechanical" errors. Mine is simply that I don't see that it makes such a big difference in my experience; most people who have horrible coding styles respond to human advice quite well, and the "minor" differences are -- minor.
YMMV.
I used the stdlib primarily because it was an agreed upon fact that the style there was inconsistent. However it’s not just in old modules, it’s in any module that has had more than a handful of contributors to it. I was picking on the stdlib to avoid calling out other people’s software where it wasn’t an agreed upon thing that it was bad. If you want non stdlib examples, pip itself and virtualenv are both great (terrible?) examples of wildly inconsistent style. Pip is better now because we went through and fixed it all and are now running automated style checks as part of our CI. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 3/19/2015 5:01 PM, Guido van Rossum wrote:
On Thu, Mar 19, 2015 at 1:40 PM, Terry Reedy <tjreedy@udel.edu
Having the guidelines numbered (id'ed), even though still regarded as guidelines, could help communication. Checkers could then easily refer to specific guidelines.
I worry this will just encourage the nit-picky attitude more.
I agree that this is a possible negative.
Thanks for the link. I was specifically thinking of global renamings to satisfy PEP 8's Naming Conventions. Idlelib has a mishmash of module and function/method name styles. (I might look and see what can be done with the undocumented (except as 'unstable') libe2to3.)
This feels hard to automate, because you don't know what names are part of an external specification.
Modernizing idlelib names, in relation to PEP 434, is a separate discussion.
I think that a quick "clean up whitespace" feature would be a nice addition to IDLE, provided it's a user-selectable menu items (similar to the existing indent/dedent operations).
The Format menu now has 'Strip trailing whitespace', which may be new since you last looked. It needs to strip trailing blank lines, as reindent.py does, in addition to trailing whitespace on each line, to make files ready to commit to the cpython repository. Did you have in mind also doing within-line cleanups (as a separate menu entry)? -- Terry Jan Reedy
~ Ian Lee On Tue, Mar 17, 2015 at 8:42 PM, Alexander Walters <tritium-list@sdamon.com> wrote:
What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong?
What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this?
And perhaps more to the point, should PEP-8 (and if so how), evolve over time to incorporate newer ideas that the community has now moved towards accepting as the "more Pythonic way to do things" now with the benefit of additional hindsight. I didn't want to derail this conversation, so I spun off a separate thread to discuss: https://mail.python.org/pipermail/python-ideas/2015-March/032557.html
And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind).
Good coding style really should not be something we ship in the library, it should be something we teach.
2¢
On Tue, Mar 17, 2015 at 7:17 PM, Nicholas Chammas < nicholas.chammas@gmail.com> wrote:
First class means it's like a language feature. Actually, in my case
I am using "first-class" and "included with the language" synonymously. Not sure if that's appropriate terminology, but that's what I mean.
To explain by rough analogy, Twisted is to asyncio what autopep8 would be to a first-class auto-styler. (I say "rough" because I understand that Twisted is not supposed to be replaced entirely by asyncio, but I hope my point is clear.)
Nick
On Tue, Mar 17, 2015 at 9:58 PM Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/17/2015 06:54 PM, Nicholas Chammas wrote:
I am not proposing putting |autopep8| in the standard library. I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on /included/ and /first-class/.
Perhaps you could explain what you mean by a /first-class/ utility?
-- ~Ethan~
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing listPython-ideas@python.orghttps://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 17Mar2015 23:42, Alexander Walters <tritium-list@sdamon.com> wrote:
What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong?
autopep8 at least lets you pick and choose specific issues to apply or ignore. As someone who now aims for a "mostly pep8" style, that is very handy. Aside: I always wish "fix it" tools to have this kind of granularity. In a former life I got to use SGI's C compiler. Like GCC, it had a "lots of warnings" mode, somewhat like a builtin lint. _Unlike_ GCC, the SGI compiler let one suppress specific warnings, which was very very nice.
What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this?
Such things get progressively harder to formalise into stuff a tool can implement. I imagine one could implement quite a lot of knobs, but I also imagine that some knobs would require huge code support, and perhaps be intrusive enough to make the default knobs (very?) hard to debug/maintain. Using PEP8 as a guide at least gives a concrete baseline for the feature set. I'm not arguing that that is all one should ever want, just that it is well defined and very broadly appplicable.
And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind).
Sure. But that isn't always a bad thing; life is too short. In an ideal world the included battery is amenable to extension and then the supplied battery enables faster third party work, at least in the form of adding to the battery.
Good coding style really should not be something we ship in the library, it should be something we teach.
Yes and no. I wholely agree that good style comes from within, and that it should be taught "what and why"; blindly applied mechanical style does not itself improve code quality. Though having consistent style does at least usually improve code readability and maintainability. Not to mention diff noise. That said, having a tool that will _apply_ a desirable style is very useful. Taking myself as an example: my personal library has gads of old code with bad style (and bad design choices!) I tend to mostly-PEP8 the old code as I come across it during development (committing separately from semantic changes unless I've rewritten something outright). That work is slow and incremental. Having just downloaded autopep8 and given it a tiny trial run against a single module I found that a bare run produced a diff of more or less the entire file, but adding "--indent-size 2" (my personal major divergence from PEP8) got the diff down to a reviewable bunch of small changes, all of which I approve. Performing that task by hand was an ongoing indefinite inconsistent minor improvement activity. Now I can branch and restyle vigorously, because I have a tool to do the picky editing. Yay! Aside: why the bad style from the outset? When coming to a new language or system I find it useful to write batteries myself, sometimes because the supplied battery does not match my needs but usually because the battery is tractable and I learn quite a lot about the problem domain during this process. And the same applies to style; my old code has poor style, some leaked across from other languages. Having written plenty of code according to my whim, I can then look at a suggested style like PEP8 and evaluate it with context. Now I mostly aim for PEP8 because I've come to informed decisions about its choices, and mostly agree with them. Metaaside: the above is also something of an excuse for the state of some of the code I've put out there:-) Cheers, Cameron Simpson <cs@zip.com.au> Mac OS X. Because making Unix user-friendly is easier than debugging Windows. - Mike Dawson, Macintosh Systems Administrator and Consultation. mdawson@mac.com http://herowars.onestop.net
Cameron Simpson <cs@zip.com.au> writes:
In a former life I got to use SGI's C compiler. Like GCC, it had a "lots of warnings" mode, somewhat like a builtin lint. _Unlike_ GCC, the SGI compiler let one suppress specific warnings, which was very very nice.
GCC now has “diagnostic pragmas” to turn on/off specific diagnostics <URL:https://gcc.gnu.org/onlinedocs/gcc-4.3.3/gcc/Diagnostic-Pragmas.html#Diagnostic-Pragmas>, is that what you mean? -- \ “I cannot be angry at God, in whom I do not believe.” —Simone | `\ De Beauvoir | _o__) | Ben Finney
On 20Mar2015 14:02, Ben Finney <ben+python@benfinney.id.au> wrote:
Cameron Simpson <cs@zip.com.au> writes:
In a former life I got to use SGI's C compiler. Like GCC, it had a "lots of warnings" mode, somewhat like a builtin lint. _Unlike_ GCC, the SGI compiler let one suppress specific warnings, which was very very nice.
GCC now has “diagnostic pragmas” to turn on/off specific diagnostics <URL:https://gcc.gnu.org/onlinedocs/gcc-4.3.3/gcc/Diagnostic-Pragmas.html#Diagnostic-Pragmas>, is that what you mean?
In terms of function, yes, though it says "only warnings [...] can be controlled, and not all of them". In terms of usability, no. I'd rather put compiler options to control noise in Makefiles (or whatever built system) rather than litter code with compiler specific junk which has no effect (one hopes) elsewhere (other platforms, other compilers). Also, I don't think these #pragmas were around at the time:-) Cheers, Cameron Simpson <cs@zip.com.au> Tim's Top 5: #5. Know when to give cages the bird. #4. I'm not hanging off, so you don't need to hang off either. (especially on the "up" side) #3. Always let me know when you're gonna get on and off. And by God, don't let your boot scratch my polished exhaust canister. #2. Remain seated until the aircraft has safely stopped at the passenger terminal and the seat belt sign has been extinguished. #1. Shut the hell up, it'll all be over in a minute. - Timothy R. Redden <reddent@sequoia.csus.edu>
Can't you do that witout pragmas? gcc tst.c -Wall -Wno-parenthesis -Wno-unused-variable ... On Fri, Mar 20, 2015 at 5:00 PM, Cameron Simpson <cs@zip.com.au> wrote:
On 20Mar2015 14:02, Ben Finney <ben+python@benfinney.id.au> wrote:
Cameron Simpson <cs@zip.com.au> writes:
In a former life I got to use SGI's C compiler. Like GCC, it had a "lots of warnings" mode, somewhat like a builtin lint. _Unlike_ GCC, the SGI compiler let one suppress specific warnings, which was very very nice.
GCC now has “diagnostic pragmas” to turn on/off specific diagnostics <URL:https://gcc.gnu.org/onlinedocs/gcc-4.3.3/gcc/ Diagnostic-Pragmas.html#Diagnostic-Pragmas>, is that what you mean?
In terms of function, yes, though it says "only warnings [...] can be controlled, and not all of them". In terms of usability, no.
I'd rather put compiler options to control noise in Makefiles (or whatever built system) rather than litter code with compiler specific junk which has no effect (one hopes) elsewhere (other platforms, other compilers).
Also, I don't think these #pragmas were around at the time:-)
Cheers, Cameron Simpson <cs@zip.com.au>
Tim's Top 5: #5. Know when to give cages the bird. #4. I'm not hanging off, so you don't need to hang off either. (especially on the "up" side) #3. Always let me know when you're gonna get on and off. And by God, don't let your boot scratch my polished exhaust canister. #2. Remain seated until the aircraft has safely stopped at the passenger terminal and the seat belt sign has been extinguished. #1. Shut the hell up, it'll all be over in a minute. - Timothy R. Redden <reddent@sequoia.csus.edu>
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/
On 20Mar2015 17:06, Ryan Gonzalez <rymg19@gmail.com> wrote:
Can't you do that witout pragmas?
gcc tst.c -Wall -Wno-parenthesis -Wno-unused-variable ...
Now, presumably yes. Then, definitely no. So I presume the situation is improved a lot. Cheers, Cameron Simpson <cs@zip.com.au> If you 'aint falling off, you ar'nt going hard enough. - Fred Gassit
On Tue, Mar 17, 2015 at 8:29 PM, Eli Bendersky <eliben@gmail.com> wrote:
Interesting coincidence :-)
We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code. It should be posted to Github within the next few days or weeks, under a permissive license (Apache 2.0 or something similar).
The tool works like gofmt and clang-format (the C++/JS version provided as part of the Clang toolchain) - it only acts on whitespace and performs re-formatting of code (PEP-8 oriented for Python but it's configurable). It does not rename variables/classes/functions to conform to any particular style. The algorithm is based on clang-format, and lib2to3 is used for the parsing side - so it's very solid. We use it on Python 2 code internally, but Python 3 support is also planned and should be easy.
We intend to keep actively developing the tool in the open and will be happy to accept test cases, fixes and patches in general.
So stay tuned!
https://github.com/google/yapf Eli
P.S. I'm also against such a tool being a part of the Python distribution, at least in the near term. A tool/library has to first prove itself outside of core for a while, and only when/if it proves to be widely used and stable, an incorporation into core should be considered. The stdlib is big as it is, and any additional code is a burden on all core developers. Incorporation into the stdlib also imposes a very slow pace of development/release on said library/tool, and hence should only be considered when it's been extremely stable for a while.
Eli
On Tue, Mar 17, 2015 at 7:17 PM, Nicholas Chammas < nicholas.chammas@gmail.com> wrote:
First class means it's like a language feature. Actually, in my case I am using "first-class" and "included with the language" synonymously. Not sure if that's appropriate terminology, but that's what I mean.
To explain by rough analogy, Twisted is to asyncio what autopep8 would be to a first-class auto-styler. (I say "rough" because I understand that Twisted is not supposed to be replaced entirely by asyncio, but I hope my point is clear.)
Nick
On Tue, Mar 17, 2015 at 9:58 PM Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/17/2015 06:54 PM, Nicholas Chammas wrote:
I am not proposing putting |autopep8| in the standard library. I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on /included/ and /first-class/.
Perhaps you could explain what you mean by a /first-class/ utility?
-- ~Ethan~
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 3/25/2015 8:42 AM, Eli Bendersky wrote:
On Tue, Mar 17, 2015 at 8:29 PM, Eli Bendersky <eliben@gmail.com <mailto:eliben@gmail.com>> wrote:
Interesting coincidence :-)
We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code. It should be posted to Github within the next few days or weeks, under a permissive license (Apache 2.0 or something similar).
The tool works like gofmt and clang-format (the C++/JS version provided as part of the Clang toolchain) - it only acts on whitespace and performs re-formatting of code (PEP-8 oriented for Python but it's configurable). It does not rename variables/classes/functions to conform to any particular style. The algorithm is based on clang-format, and lib2to3 is used for the parsing side - so it's very solid. We use it on Python 2 code internally, but Python 3 support is also planned and should be easy.
We intend to keep actively developing the tool in the open and will be happy to accept test cases, fixes and patches in general.
So stay tuned!
yet-another-python-formatter The so-called pep8 style in style.py has # The number of spaces required before a trailing comment. SPACES_BEFORE_COMMENT=1, PEP8 actually says "Inline comments should be separated by at least two spaces from the statement.", which is what you have in the google style. -- Terry Jan Reedy
I'm not sure everyone that works on yapf actually reads this ML, so it'd be a good idea to use github's bug tracker when reporting problems in the future. I filed a ticket for you: https://github.com/google/yapf/issues/14 -- Devin On Wed, Mar 25, 2015 at 9:38 AM, Terry Reedy <tjreedy@udel.edu> wrote:
On 3/25/2015 8:42 AM, Eli Bendersky wrote:
On Tue, Mar 17, 2015 at 8:29 PM, Eli Bendersky <eliben@gmail.com <mailto:eliben@gmail.com>> wrote:
Interesting coincidence :-)
We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code. It should be posted to Github within the next few days or weeks, under a permissive license (Apache 2.0 or something similar).
The tool works like gofmt and clang-format (the C++/JS version provided as part of the Clang toolchain) - it only acts on whitespace and performs re-formatting of code (PEP-8 oriented for Python but it's configurable). It does not rename variables/classes/functions to conform to any particular style. The algorithm is based on clang-format, and lib2to3 is used for the parsing side - so it's very solid. We use it on Python 2 code internally, but Python 3 support is also planned and should be easy.
We intend to keep actively developing the tool in the open and will be happy to accept test cases, fixes and patches in general.
So stay tuned!
yet-another-python-formatter
The so-called pep8 style in style.py has
# The number of spaces required before a trailing comment. SPACES_BEFORE_COMMENT=1,
PEP8 actually says "Inline comments should be separated by at least two spaces from the statement.", which is what you have in the google style.
-- Terry Jan Reedy
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Mar 25, 2015 at 9:38 AM, Terry Reedy <tjreedy@udel.edu> wrote:
On 3/25/2015 8:42 AM, Eli Bendersky wrote:
On Tue, Mar 17, 2015 at 8:29 PM, Eli Bendersky <eliben@gmail.com <mailto:eliben@gmail.com>> wrote:
Interesting coincidence :-)
We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code. It should be posted to Github within the next few days or weeks, under a permissive license (Apache 2.0 or something similar).
The tool works like gofmt and clang-format (the C++/JS version provided as part of the Clang toolchain) - it only acts on whitespace and performs re-formatting of code (PEP-8 oriented for Python but it's configurable). It does not rename variables/classes/functions to conform to any particular style. The algorithm is based on clang-format, and lib2to3 is used for the parsing side - so it's very solid. We use it on Python 2 code internally, but Python 3 support is also planned and should be easy.
We intend to keep actively developing the tool in the open and will be happy to accept test cases, fixes and patches in general.
So stay tuned!
yet-another-python-formatter
The so-called pep8 style in style.py has
# The number of spaces required before a trailing comment. SPACES_BEFORE_COMMENT=1,
PEP8 actually says "Inline comments should be separated by at least two spaces from the statement.", which is what you have in the google style.
Thanks Terry. Devin opened an issue on Github and we'll fix this. Eli
-- Terry Jan Reedy
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 3/17/2015 9:54 PM, Nicholas Chammas wrote:
I am not proposing putting |autopep8| in the standard library.
You gave that as an example and I went with it the absence of there being anything else concrete, even a name. Substitute a mythical 'autostyle' if you wish.
I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on /included/ and /first-class/.
We are wary about including anything more with the CPython distribution. It is already too large to be managed by the core developer group as well as we would like. I disagree than not being included in 'cpython' necessarily makes something less than first-class, as you seemed to say in another response.
That auto-styler may well be based somehow on |autopep8|, but that is a separate discussion about /implementation/. I am first checking that the /idea/ is sound. I can’t tell if you had any objections to that idea.
I do. We should be careful about adding anything. When we do, we should add libraries, not applications. We should especially not officially endorse one of many opinion-based competitors. Pip is the intended way to include 3rd party modules, as one chooses, as virtual members of the stdlib or distribution. FWIW, CPython already comes with a minimal autostyle script, Tools/Scripts/reindent.py, which is used to enforce minimal requirements for committing new or altered stdlib .py files to the cpython hg repository.
couldn’t |autopep8| somehow be adapted to be an included module that can be invoked via |python -m|
Today it is invoked with 'autopep8'. Why would you want to have to add 'python -m'? For Idle, I would like to move in the other direction and allow Idle to be started on a command line with 'idle' rather than 'python -m idlelib'. (This is possible today, but only if one adds '.../python/lib/idlelib' to PATH, or, on windows, moves an edited version of idle.bat to /Scripts.) -- Terry Jan Reedy
participants (16)
-
Alexander Walters -
anatoly techtonik -
Andrew Barnert -
Ben Finney -
Cameron Simpson -
Devin Jeanpierre -
Donald Stufft -
Eli Bendersky -
Ethan Furman -
Guido van Rossum -
Ian Lee -
Nicholas Chammas -
Ryan Gonzalez -
Stephen J. Turnbull -
Steven D'Aprano -
Terry Reedy