On Mar 17, 2015, at 11:31 AM, Nicholas Chammas wrote:

The Zen of Python says:

There should be one— and preferably only one —obvious way to do it.

As someone who uses autopep8 regularly but has at least once had resistance to getting into a team's workflow, +1 on the idea. The analogy with gofmt isn't perfect because gofmt only deals with some simple issues where there's an unambiguous 100%-clear standard, while PEP 8 includes a wide range of guidelines that are usually but not always consistent and appropriate, but that (obviously) doesn't mean autopep8 is an impossible or pointless non-starter. So, is your proposal that autopep8 (and therefore pep8, on which it depends) be added to the stdlib with only minor changes? If so: * You'd probably want to ask whether Hideo Hattori and Johann C. Rocholl are willing to relicense and contribute their code, and maintain it for years to come, before asking whether Python will accept the result. * Will it be easy to transform a new version of the PyPI libraries into a new version of the stdlib module in some mostly-automated way for each new Python release? * Given that Python updates less often than pep8 and autopep8, will it generally be "good enough" to use the autopep8 1.1.3 that came with your Python 3.5 instead of installing the 1.1.7 that's on PyPI? What if you're still using Python 3.5 in another two years, and Python 3.6 came with 1.2.1, and 1.2.5 is on PyPI; is using 1.1.3 still a good idea? (Compare with pysqlite; I've used sqlite3 many times, and only wanted to get the newer version twice--and once just because it was the easiest way to build with a local variant of the C library on Windows.) * Are all of the current features and bugs (see issues on GitHub) acceptable as-is for the stdlib today? If the answers to those questions is "yes", this seems like the way to go. Otherwise, you're probably suggesting someone either fork the existing projects or write a brand new one (and as the one with the itch and the ideas, who better than you?).

Python already has a style guide in PEP 8. More importantly, from a practical standpoint, Python has automated third-party tools for checking and enforcing PEP 8 guidelines:

pep8: Python style guide checker autopep8: A tool that automatically formats Python code to conform to the PEP 8 style guide I say “more importantly” because these tools are what make the style guide alive. If a Python project of any non-trivial size is PEP 8-compliant, you can bet it’s because the project contributors automated the process of enforcing compliance using these or similar tools. (Certainly, some parts of PEP 8 are hard to automate. But those parts that can be automated have been, to the benefit of Python programmers everywhere.)

Having a style guide provides benefits that I think are well-understood. Additionally having a standardized style that can be applied automatically makes whatever benefits of the style guide that much more prevalent in the greater community and actually effective.

As many of you may already know, the Go-lang folks have such a feature for their language (though they call it formatting as opposed to styling). They note:

go fmt your code:

Gofmt is a tool that automatically formats Go source code.

Gofmt’d code is:

easier to write: never worry about minor formatting concerns while hacking away, easier to read: when all code looks the same you need not mentally convert others’ formatting style into something you can understand. easier to maintain: mechanical changes to the source don’t cause unrelated changes to the file’s formatting; diffs show only the real changes. uncontroversial: never have a debate about spacing or brace position ever again! Of course, we don’t worry about brace positions, but these benefits of a standardized style carry over into any language. :)

So, in the spirit of The Zen of Python, and taking a cue from Go-lang’s gofmt, I propose we promote standardized auto-styling to a first-class language feature.

UI-wise, I’m not sure how exactly we’d present this to the user. Perhaps something like:

python -m style script.py Don’t read too much into this UI suggestion, as I’m really not sure how it would be implemented. The point is to expose some auto-styler as a standard language tool—i.e. something like autopep8, but as a built-in utility.

On running a command like that, script.py would automatically be updated to conform to the standard style we want to promote. This is what gofmt does for Go code.

Promoting an auto-styler to be a first-class language feature builds on the foundation laid in PEP 8 and in tools like pep8 and autopep8. It helps reduce code style to more of a non-issue for Python projects than it already is.

Side Note 1: There was a previous python-ideas discussion about introducing standardized formatting here, but it was totally derailed by the OP’s suggestion that the formatter introduce block delimiters. I am proposing no such thing here.

Side Note 2: I found these additional comments about gofmt interesting:

Gofmt also enables gofix, which can make arbitrarily complex source transformations. Gofix was an invaluable tool during the early days when we regularly made breaking changes to the language and libraries.

Introducing Gofix:

Gofix gives us the ability to fix mistakes or completely rethink package APIs without worrying about the cost of converting existing code.

Python has its own analog to gofix in 2to3, though that was built for a specific migration rather than as a regular language feature. I wonder if having a gofmt analog in Python would similarly facilitate future migrations, though I doubt we’ll ever have a big as migration as the 2-to-3 one.

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 3/17/2015 2:31 PM, Nicholas Chammas wrote:

The Zen of Python https://www.python.org/dev/peps/pep-0020/ says:

There should be one— and preferably only one —obvious way to do it.

Python already has a style guide in PEP 8 https://www.python.org/dev/peps/pep-0008/. More importantly, from a practical standpoint, Python has automated third-party tools for checking and enforcing PEP 8 guidelines:

* |pep8| https://pypi.python.org/pypi/pep8/: Python style guide checker * |autopep8| https://pypi.python.org/pypi/autopep8/: A tool that automatically formats Python code to conform to the PEP 8 style guide

Except for idlelib and more recently turtledemo, the stdlib consists of *library* modules that define constants, mostly functions and classes, for import into and use by other libraries and applications. Notice that the stdlib contains lib2to3, while the 2to3 application that uses lib2to3 is in /Tools/Scripts, along with other application scripts, mostly short, written by core developers. pep8, autopep8, and similar applications do not belong in the stdlib. They are easily installed with, for instance, 'pip install autopep8'. The install process puts a startup file in /Scripts, which (for Windows at least) is on PATH (unlike /Tools/Scripts). After installing, I entered 'autopep8 file.py' and it runs. There is an issue on the tracker to add to the Idle IDE a gui frontend to pip, so beginners who use Idle will be able to manage (install, update, delete) 3rd-party apps like these without knowing about 'command lines' or having to remember pip options. (There is also an issue to apply such apps to the contents of an Editor window.) -- Terry Jan Reedy

On Tue, Mar 17, 2015 at 8:04 PM, Terry Reedy wrote: pep8, autopep8, and similar applications do not belong in the stdlib. They

are easily installed with, for instance, 'pip install autopep8'.

I am not proposing putting autopep8 in the standard library. I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on *included* and *first-class*. That auto-styler may well be based somehow on autopep8, but that is a separate discussion about *implementation*. I am first checking that the *idea* is sound. I can’t tell if you had any objections to that idea. Yes, I know that today I can install and use autopep8 as a third-party utility using pip, but autopep8 today is neither included with Python nor a first-class utility. (FWIW, couldn’t autopep8 somehow be adapted to be an included module that can be invoked via python -m https://www.python.org/dev/peps/pep-0338/ — e.g. python -m compileall, python -m SimpleHTTPServer — just as far as the mechanics are concerned? As I noted in my original post, I’m not in a position yet to specifically propose how an included auto-styler would work, but let’s say for argument’s sake that 1) we decided we liked the idea and 2) autopep8 fit the bill for promotion to an included utility, and 3) we addressed Andrew’s points.) Nick ​

First class means it's like a language feature. Actually, in my case I am using "first-class" and "included with the language" synonymously. Not sure if that's appropriate terminology, but that's what I mean. To explain by rough analogy, Twisted is to asyncio what autopep8 would be to a first-class auto-styler. (I say "rough" because I understand that Twisted is not supposed to be replaced entirely by asyncio, but I hope my point is clear.) Nick On Tue, Mar 17, 2015 at 9:58 PM Ethan Furman wrote:

On 03/17/2015 06:54 PM, Nicholas Chammas wrote:

...

I am not proposing putting |autopep8| in the standard library. I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on /included/ and /first-class/.

Perhaps you could explain what you mean by a /first-class/ utility?

-- ~Ethan~

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong? What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this? And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind). Good coding style really should not be something we ship in the library, it should be something we teach. 2¢ On Tue, Mar 17, 2015 at 7:17 PM, Nicholas Chammas mailto:nicholas.chammas@gmail.com> wrote:

First class means it's like a language feature. Actually, in my case I am using "first-class" and "included with the language" synonymously. Not sure if that's appropriate terminology, but that's what I mean.

To explain by rough analogy, Twisted is to asyncio what autopep8 would be to a first-class auto-styler. (I say "rough" because I understand that Twisted is not supposed to be replaced entirely by asyncio, but I hope my point is clear.)

Nick

On Tue, Mar 17, 2015 at 9:58 PM Ethan Furman mailto:ethan@stoneleaf.us> wrote:

On 03/17/2015 06:54 PM, Nicholas Chammas wrote:

> I am not proposing putting |autopep8| in the standard library. I am proposing that an auto-styler be included with > Python as a first-class utility. The emphasis here is on /included/ and /first-class/.

Perhaps you could explain what you mean by a /first-class/ utility?

-- ~Ethan~

_______________________________________________ Python-ideas mailing list Python-ideas@python.org mailto:Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org mailto:Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

~ Ian Lee On Tue, Mar 17, 2015 at 8:42 PM, Alexander Walters wrote:

What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong?

What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this?

And perhaps more to the point, should PEP-8 (and if so how), evolve over time to incorporate newer ideas that the community has now moved towards accepting as the "more Pythonic way to do things" now with the benefit of additional hindsight. I didn't want to derail this conversation, so I spun off a separate thread to discuss: https://mail.python.org/pipermail/python-ideas/2015-March/032557.html

And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind).

Good coding style really should not be something we ship in the library, it should be something we teach.

2¢

On Tue, Mar 17, 2015 at 7:17 PM, Nicholas Chammas < nicholas.chammas@gmail.com> wrote:

First class means it's like a language feature. Actually, in my case

...

I am using "first-class" and "included with the language" synonymously. Not sure if that's appropriate terminology, but that's what I mean.

To explain by rough analogy, Twisted is to asyncio what autopep8 would be to a first-class auto-styler. (I say "rough" because I understand that Twisted is not supposed to be replaced entirely by asyncio, but I hope my point is clear.)

Nick

On Tue, Mar 17, 2015 at 9:58 PM Ethan Furman wrote:

...

On 03/17/2015 06:54 PM, Nicholas Chammas wrote:

...

I am not proposing putting |autopep8| in the standard library. I am proposing that an auto-styler be included with Python as a first-class utility. The emphasis here is on /included/ and /first-class/.

Perhaps you could explain what you mean by a /first-class/ utility?

-- ~Ethan~

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing listPython-ideas@python.orghttps://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Mar 17, 2015 at 11:29 PM, Eli Bendersky wrote:

We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code.

Interesting. Are y'all familiar with `autopep8`? What are the differences between the tool y'all are releasing and `autopep8`? On Tue, Mar 17, 2015 at 11:29 PM, Eli Bendersky wrote:

P.S. I'm also against such a tool being a part of the Python distribution, at least in the near term. A tool/library has to first prove itself outside of core for a while, and only when/if it proves to be widely used and stable, an incorporation into core should be considered. The stdlib is big as it is, and any additional code is a burden on all core developers. Incorporation into the stdlib also imposes a very slow pace of development/release on said library/tool, and hence should only be considered when it's been extremely stable for a while.

This makes sense. In fact, I recently watched a keynote that Guido gave and he said something similar https://youtu.be/EBRMq2Ioxsc?t=49m23s. On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters

wrote:

What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong?

What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this?

I might be wrong, but I see PEP 8 as a living document. If the community believes parts of it are out of date, I would think it's better to update PEP 8 (or issue a replacement PEP) to reflect current thinking. Any auto-styling tool included in the standard library would work best if it was based on a written "spec" like PEP 8 (granted, I don't think PEP 8 was meant to be a spec for programmatic implementation, but you get the idea). If stuff is not part of PEP 8 that should be, or if parts of PEP 8 are agreed to be "wrong", then PEP 8 should be updated accordingly. Any auto-styling tool, I would hope, would just follow from PEP 8 (or a similar document). The tool should not be the specification. On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters

wrote:

And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind).

I see what you mean. Several people have made comments to this effect, and this is a good point. I guess this is a separate discussion, but is there some way to offer users the advantages of having a library be included with Python without imposing the 2-year release cycle and additional burden on the library's developers? This seems to be the root of the concern about including, for example, something like autopep8 with Python. On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters

wrote:

Good coding style really should not be something we ship in the library, it should be something we teach.

This point is orthogonal to the discussion of whether or not to package an auto-styler, but there are many style points that are so mechanical I see little benefit in expending energy on teaching them. Best would be to automate as much of it away so that people can focus on bigger problems like, what is the right abstraction, did I design this API well, etc. Put differently, consider the following straw man: "Good memory management should not be something we ship in the language, it should be something we teach." There is always something to teach. But when we can automate part of the problem away, I think we generally should. On Wed, Mar 18, 2015 at 2:38 AM, Ian Lee wrote:

And perhaps more to the point, should PEP-8 (and if so how), evolve over time to incorporate newer ideas that the community has now moved towards accepting as the "more Pythonic way to do things" now with the benefit of additional hindsight.

I consider PEP 8 to be a living document. That is, it is already changing with time. For example, I see that though PEP 8 was created in 2001, it was updated in 2013 https://www.python.org/dev/peps/pep-0008/#id8. I assume it will continue to be updated as accepted standards change. On Wed, Mar 18, 2015 at 4:52 AM, Steven D'Aprano wrote:

Not necessarily. This list frequently includes half-baked ideas, but the "bigger" the proposal, the more baked it should be to be taken seriously.

Duly noted! Nick

On 3/18/2015 12:52 PM, Nicholas Chammas wrote:

I might be wrong, but I see PEP 8 as a living document.

PEP 8 is Guido's view on how new CPython stdlib code should be written, with input from other core developers. "This document gives coding conventions for the Python code comprising the standard library in the main Python distribution." It changes as he and we change our views. It is against Guido's policy to blindly change old stdlib code to follow PEP 8. (Note that autoPEP8 only changes whitespace, which is a small part of PEP 8 recommendations, and the safest aspect of code to change.)

If the community believes parts of it are out of date, I would think it's better to update PEP 8 (or issue a replacement PEP) to reflect current thinking.

The 'community' is free to follow it or not, but not to change it. Other people and organizations have written their own style guides. Good for them.

If stuff is not part of PEP 8 that should be, or if parts of PEP 8 are agreed to be "wrong", then PEP 8 should be updated accordingly.

PEP 8 is by definition 'right' until Guido decides otherwise, with input from other core devs.

I guess this is a separate discussion, but is there some way to offer users the advantages of having a library be included with Python without imposing the 2-year release cycle and additional burden on the library's developers?

Use pip. It was added to make pypi (and other repositories) an easy-to-use extension of the stdlib, without imposing stdlib requirements and burdens. We are working on making pip even easier to use. -- Terry Jan Reedy

On Wed, Mar 18, 2015 at 9:52 AM, Nicholas Chammas < nicholas.chammas@gmail.com> wrote:

On Tue, Mar 17, 2015 at 11:29 PM, Eli Bendersky wrote:

...

We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code.

Interesting. Are y'all familiar with `autopep8`?

Yes, very familiar.

What are the differences between the tool y'all are releasing and `autopep8`?

autopep8's scope is both smaller and larger. It only detects PEP8 violations and attempts to fix them, not restricting itself to whitespace. Our tool works more like the gofmt tool mentioned in the beginning of the thread - it pretty much only cares about formatting, and it auto-reformats the code to its own liking, covering more whitespace-related stuff than PEP8. Basically the idea is "forget about formatting your code, just run it through the tool", which is a similar philosophy as gofmt (and clang-format) and has been hugely successful for those tools/languages. Eli

On Tue, Mar 17, 2015 at 11:29 PM, Eli Bendersky wrote:

...

P.S. I'm also against such a tool being a part of the Python distribution, at least in the near term. A tool/library has to first prove itself outside of core for a while, and only when/if it proves to be widely used and stable, an incorporation into core should be considered. The stdlib is big as it is, and any additional code is a burden on all core developers. Incorporation into the stdlib also imposes a very slow pace of development/release on said library/tool, and hence should only be considered when it's been extremely stable for a while.

This makes sense. In fact, I recently watched a keynote that Guido gave and he said something similar https://youtu.be/EBRMq2Ioxsc?t=49m23s.

On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters < tritium-list@sdamon.com> wrote:

...

What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong?

What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this?

I might be wrong, but I see PEP 8 as a living document. If the community believes parts of it are out of date, I would think it's better to update PEP 8 (or issue a replacement PEP) to reflect current thinking.

Any auto-styling tool included in the standard library would work best if it was based on a written "spec" like PEP 8 (granted, I don't think PEP 8 was meant to be a spec for programmatic implementation, but you get the idea).

If stuff is not part of PEP 8 that should be, or if parts of PEP 8 are agreed to be "wrong", then PEP 8 should be updated accordingly. Any auto-styling tool, I would hope, would just follow from PEP 8 (or a similar document). The tool should not be the specification.

On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters < tritium-list@sdamon.com> wrote:

...

And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind).

I see what you mean. Several people have made comments to this effect, and this is a good point.

I guess this is a separate discussion, but is there some way to offer users the advantages of having a library be included with Python without imposing the 2-year release cycle and additional burden on the library's developers?

This seems to be the root of the concern about including, for example, something like autopep8 with Python.

On Tue, Mar 17, 2015 at 11:42 PM, Alexander Walters < tritium-list@sdamon.com> wrote:

...

Good coding style really should not be something we ship in the library, it should be something we teach.

This point is orthogonal to the discussion of whether or not to package an auto-styler, but there are many style points that are so mechanical I see little benefit in expending energy on teaching them. Best would be to automate as much of it away so that people can focus on bigger problems like, what is the right abstraction, did I design this API well, etc.

Put differently, consider the following straw man: "Good memory management should not be something we ship in the language, it should be something we teach."

There is always something to teach. But when we can automate part of the problem away, I think we generally should.

On Wed, Mar 18, 2015 at 2:38 AM, Ian Lee wrote:

...

And perhaps more to the point, should PEP-8 (and if so how), evolve over time to incorporate newer ideas that the community has now moved towards accepting as the "more Pythonic way to do things" now with the benefit of additional hindsight.

I consider PEP 8 to be a living document. That is, it is already changing with time. For example, I see that though PEP 8 was created in 2001, it was updated in 2013 https://www.python.org/dev/peps/pep-0008/#id8. I assume it will continue to be updated as accepted standards change.

On Wed, Mar 18, 2015 at 4:52 AM, Steven D'Aprano wrote:

...

Not necessarily. This list frequently includes half-baked ideas, but the "bigger" the proposal, the more baked it should be to be taken seriously.

Duly noted!

Nick

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 3/19/2015 10:32 AM, Nicholas Chammas wrote:

OK, I hereby abandon this proposal. :)

Thank you for being so sensible ;-) Let me first quote (out of order) your conclusion:

I think any part of the development process we automate away is a good thing (stdlib inclusion or not notwithstanding).>

I agree, which is why I expect to do some work making it easier to download 3rd party modules and apply them to python code, both in a gui IDE framework (Idle). I think automation should respect and augment, not replace. the human programmer.

To summarize the conclusion I believe this thread has arrived at:

...

Ignoring that difficulty, there is also the problem of what “spec” to use for any such auto-styler. PEP 8 is an obvious choice, but it was written as a guide for humans, not a spec for programmatic implementation.

A program that treats PEP 8 guidelines as rigid rules, that ignores the first guideline "A Foolish Consistency is the Hobgoblin of Little Minds", that removes the human choices intentionally included in PEP 8, is not PEP 8 compliant. I believe one of the motivations for some recent PEP 8 amendments was some unhappiness with pep8.py or other checkers.

Furthermore, there are doubts about PEP 8’s utility as the basis for a universal auto-styler, as it was intended for code specifically in the Python standard library.

Pydev is a relatively small group of volunteers who otherwise work either for themselves or for various organizations scattered around the world. Having the freedom to put actual readability ahead of rigid consistency makes volunteering more attractive. I regard code as a form of poetry, but prefer a flexible but defined structure to free-form mishmash. Google, as an example, is a very large corporation that pays people well to adhere to its guidelines and rules. This is a different situation and its best choice for Python styling may well be different from pydev.

On Wed, Mar 18, 2015 at 4:38 PM, Andrew Barnert mailto:abarnert@yahoo.com> wrote:

But that implies that it's worth someone making a spec out of PEP 8. That implies going through PEP 8, getting a sense of what parts are (unambiguously and uncontroversially) mechanizable, and reorganizing and minimally rewriting it to get that sense across. Having numbered rules, and named anchors for each one in the document (or at least to narrower sections, so you can link to something more specific than "Programming Recommendations") would also be helpful.

This is an excellent point and one I did not consider.

Having the guidelines numbered (id'ed), even though still regarded as guidelines, could help communication. Checkers could then easily refer to specific guidelines.

On Wed, Mar 18, 2015 at 6:17 PM, Terry Reedy mailto:tjreedy@udel.edu> wrote:

(Note that autoPEP8 only changes whitespace, which is a small part of PEP 8 recommendations, and the safest aspect of code to change.)

Minor point: autopep8 can do more than just whitespace changes with the |--aggressive| flag https://pypi.python.org/pypi/autopep8/#more-advanced-usage.

Thanks for the link. I was specifically thinking of global renamings to satisfy PEP 8's Naming Conventions. Idlelib has a mishmash of module and function/method name styles. (I might look and see what can be done with the undocumented (except as 'unstable') libe2to3.)

On Wed, Mar 18, 2015 at 7:21 PM, Terry Reedy mailto:tjreedy@udel.edu> wrote:

We should be careful about adding anything. When we do, we should add libraries, not applications. We should especially not officially endorse one of many opinion-based competitors.

I have come around to this view.

Accepting this view, we can still ask if the stdlib should have more (policy-neutral) string or list of strings reforming components than it does now. Textwrap has a few functions for strings in general. Idle has about 10 formatting functions exposed on its editor Format menu, but they are not exposed for general use. Tools/Scripts/reindent.py contain some of the same functions as does autopep8. Should there be a code format module that all three, and other programs, could import? Idle and Tools/Scrips/highlight.py both have code to analyze Python code tokens into categories and colorize by category. (Idle uses tkinter tags, highlight.py uses ANSI or HTML tags). Can and should both use a common code analysis function that could be imported? This might actually ease maintainance. Should a generalized 2to3 package be renamed 'codefix' and documented? As 2to3, it is already used by both Tools/Scripts/2to2 and optionally by autopep8, and perhaps by others. I believe it could also be used for some non-2to3 pep8 fixes. -- Terry Jan Reedy

On Mar 19, 2015, at 5:01 PM, Guido van Rossum wrote:

On Thu, Mar 19, 2015 at 1:40 PM, Terry Reedy mailto:tjreedy@udel.edu> wrote: A program that treats PEP 8 guidelines as rigid rules, that ignores the first guideline "A Foolish Consistency is the Hobgoblin of Little Minds", that removes the human choices intentionally included in PEP 8, is not PEP 8 compliant. I believe one of the motivations for some recent PEP 8 amendments was some unhappiness with pep8.py or other checkers.

I have always been skeptical of automatic style checkers. They encourage a nit-pickish attitude that makes the development process less effective, by encouraging critique of superficial style elements rather than of the code's structure and meaning. In contrast, the intention of PEP 8 was to make code more readable so that its structure and meaning are more easily understood.

My experience is actually the exact opposite. When a machine is responsible for style enforcement people generally don’t feel as bad about being asked to adjust the formatting because it’s just a machine, it can’t make value judgements. When a person asks someone to adjust something to make it easier to read it often times ends up causing a distraction where people tend to feel like since it’s a person asking them to do it they are passing judgement on their code, or they don’t agree with it and since it’s a human saying it it feels more like there is room to budge so you end up with people more likely to be willing to ask for an exemption. In my experience projects without mechnical enforcement ends up with massively inconsistent style all throughout the code base. The Python stdlib is a good example of this. Contrast this to projects like github.com/pyca/cryptography http://github.com/pyca/cryptography where there is an automated mechanical enforcement of style and it’s got some bare minimum of consistency throughout the entire code base.

On Wed, Mar 18, 2015 at 4:38 PM, Andrew Barnert mailto:abarnert@yahoo.com mailto:abarnert@yahoo.com>> wrote:

But that implies that it's worth someone making a spec out of PEP 8. That implies going through PEP 8, getting a sense of what parts are (unambiguously and uncontroversially) mechanizable, and reorganizing and minimally rewriting it to get that sense across. Having numbered rules, and named anchors for each one in the document (or at least to narrower sections, so you can link to something more specific than "Programming Recommendations") would also be helpful.

This is an excellent point and one I did not consider.

Having the guidelines numbered (id'ed), even though still regarded as guidelines, could help communication. Checkers could then easily refer to specific guidelines.

I worry this will just encourage the nit-picky attitude more.

On Wed, Mar 18, 2015 at 6:17 PM, Terry Reedy mailto:tjreedy@udel.edu mailto:tjreedy@udel.edu>> wrote:

(Note that autoPEP8 only changes whitespace, which is a small part of PEP 8 recommendations, and the safest aspect of code to change.)

Minor point: autopep8 can do more than just whitespace changes with the |--aggressive| flag <https://pypi.python.org/pypi/autopep8/#more-advanced-usage https://pypi.python.org/pypi/autopep8/#more-advanced-usage>.

Thanks for the link. I was specifically thinking of global renamings to satisfy PEP 8's Naming Conventions. Idlelib has a mishmash of module and function/method name styles. (I might look and see what can be done with the undocumented (except as 'unstable') libe2to3.)

This feels hard to automate, because you don't know what names are part of an external specification.

On Wed, Mar 18, 2015 at 7:21 PM, Terry Reedy mailto:tjreedy@udel.edu mailto:tjreedy@udel.edu>> wrote:

We should be careful about adding anything. When we do, we should add libraries, not applications. We should especially not officially endorse one of many opinion-based competitors.

I have come around to this view.

Accepting this view, we can still ask if the stdlib should have more (policy-neutral) string or list of strings reforming components than it does now. Textwrap has a few functions for strings in general. Idle has about 10 formatting functions exposed on its editor Format menu, but they are not exposed for general use. Tools/Scripts/reindent.py contain some of the same functions as does autopep8. Should there be a code format module that all three, and other programs, could import?

Idle and Tools/Scrips/highlight.py both have code to analyze Python code tokens into categories and colorize by category. (Idle uses tkinter tags, highlight.py uses ANSI or HTML tags). Can and should both use a common code analysis function that could be imported? This might actually ease maintainance.

Should a generalized 2to3 package be renamed 'codefix' and documented? As 2to3, it is already used by both Tools/Scripts/2to2 and optionally by autopep8, and perhaps by others. I believe it could also be used for some non-2to3 pep8 fixes.

I think that a quick "clean up whitespace" feature would be a nice addition to IDLE, provided it's a user-selectable menu items (similar to the existing indent/dedent operations). Refactoring, however, is fiendishly hard for Python -- give it a try in e.g. PyCharm and you will agree.

-- --Guido van Rossum (python.org/~guido http://python.org/~guido) _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

On 17Mar2015 23:42, Alexander Walters wrote:

What coding style would be used for such a tool? PEP-8? What about a tool for those of us who believe that PEP-8 got a few things wrong?

autopep8 at least lets you pick and choose specific issues to apply or ignore. As someone who now aims for a "mostly pep8" style, that is very handy. Aside: I always wish "fix it" tools to have this kind of granularity. In a former life I got to use SGI's C compiler. Like GCC, it had a "lots of warnings" mode, somewhat like a builtin lint. _Unlike_ GCC, the SGI compiler let one suppress specific warnings, which was very very nice.

What about for the tribal knowledge that pythonistas develop over time that are not part of the standard? Would such a tool include this?

Such things get progressively harder to formalise into stuff a tool can implement. I imagine one could implement quite a lot of knobs, but I also imagine that some knobs would require huge code support, and perhaps be intrusive enough to make the default knobs (very?) hard to debug/maintain. Using PEP8 as a guide at least gives a concrete baseline for the feature set. I'm not arguing that that is all one should ever want, just that it is well defined and very broadly appplicable.

And what of third party tools that do the same thing? I have no evidence to support this, but I have the feeling that when the standard library includes something in a new space, third party work in that space slows (json and sqlite come to mind).

Sure. But that isn't always a bad thing; life is too short. In an ideal world the included battery is amenable to extension and then the supplied battery enables faster third party work, at least in the form of adding to the battery.

Good coding style really should not be something we ship in the library, it should be something we teach.

Yes and no. I wholely agree that good style comes from within, and that it should be taught "what and why"; blindly applied mechanical style does not itself improve code quality. Though having consistent style does at least usually improve code readability and maintainability. Not to mention diff noise. That said, having a tool that will _apply_ a desirable style is very useful. Taking myself as an example: my personal library has gads of old code with bad style (and bad design choices!) I tend to mostly-PEP8 the old code as I come across it during development (committing separately from semantic changes unless I've rewritten something outright). That work is slow and incremental. Having just downloaded autopep8 and given it a tiny trial run against a single module I found that a bare run produced a diff of more or less the entire file, but adding "--indent-size 2" (my personal major divergence from PEP8) got the diff down to a reviewable bunch of small changes, all of which I approve. Performing that task by hand was an ongoing indefinite inconsistent minor improvement activity. Now I can branch and restyle vigorously, because I have a tool to do the picky editing. Yay! Aside: why the bad style from the outset? When coming to a new language or system I find it useful to write batteries myself, sometimes because the supplied battery does not match my needs but usually because the battery is tractable and I learn quite a lot about the problem domain during this process. And the same applies to style; my old code has poor style, some leaked across from other languages. Having written plenty of code according to my whim, I can then look at a suggested style like PEP8 and evaluate it with context. Now I mostly aim for PEP8 because I've come to informed decisions about its choices, and mostly agree with them. Metaaside: the above is also something of an excuse for the state of some of the code I've put out there:-) Cheers, Cameron Simpson Mac OS X. Because making Unix user-friendly is easier than debugging Windows. - Mike Dawson, Macintosh Systems Administrator and Consultation. mdawson@mac.com http://herowars.onestop.net

Donald Stufft writes:

...

...

...

...

Guido van Rossum writes:

...

I have always been skeptical of automatic style checkers. They encourage a nit-pickish attitude that makes the development process less effective, by encouraging critique of superficial style elements rather than of the code's structure and meaning. In contrast, the intention of PEP 8 was to make code more readable so that its structure and meaning are more easily understood.

My experience is actually the exact opposite.

This is one of those things where people differ. I'm happy to follow Guido on this, although personally I've been able to adapt to mechanical rules where required.

In my experience projects without mechnical enforcement ends up with massively inconsistent style all throughout the code base. The Python stdlib is a good example of this. Contrast this to projects like github.com/pyca/cryptography http://github.com/pyca/cryptography where there is an automated mechanical enforcement of style and it’s got some bare minimum of consistency throughout the entire code base.

Which proves what? That enforcing minimal style rules (and I think we all agree that there's plenty of good and bad style that machines can't yet reliably distinguish) results in a code base that follows minimal style rules?

On Mar 20, 2015, at 1:02 AM, Stephen J. Turnbull wrote:

Donald Stufft writes:

...

...

...

...

...

Guido van Rossum writes:

...

...

I have always been skeptical of automatic style checkers. They encourage a nit-pickish attitude that makes the development process less effective, by encouraging critique of superficial style elements rather than of the code's structure and meaning. In contrast, the intention of PEP 8 was to make code more readable so that its structure and meaning are more easily understood.

My experience is actually the exact opposite.

This is one of those things where people differ. I'm happy to follow Guido on this, although personally I've been able to adapt to mechanical rules where required.

...

In my experience projects without mechnical enforcement ends up with massively inconsistent style all throughout the code base. The Python stdlib is a good example of this. Contrast this to projects like github.com/pyca/cryptography http://github.com/pyca/cryptography where there is an automated mechanical enforcement of style and it’s got some bare minimum of consistency throughout the entire code base.

Which proves what? That enforcing minimal style rules (and I think we all agree that there's plenty of good and bad style that machines can't yet reliably distinguish) results in a code base that follows minimal style rules?

Sure, just like tests can’t guarantee that a code base works in every situation but that it works in some minimal set of tests (where minimal is whatever tests were written). Humans are inconsistent and emotional, people are more likely to nitpick a patch that they don’t particularly like in an unconscious attempt to discourage the contribution. They are also far more likely to let things slip when it’s a contribution from someone they like or even from themselves. Working on the Python stdlib is somewhat frustrating to me in this regard because the code in the stdlib is often times wildly inconsistent even within the same module. Maybe having inconsistent style which is inconsistently applied doesn’t bother you, that’s fine. I don’t particularly mind if the stdlib gains an automated style checker or formatter because I’m happy using third party modules. I was merely offering my experience that *anything* which relies on a human to verify it is, without exception, going to be verified unevenly and that using a human to verify it invites people to attempt to argue against it more often than when a machine does it. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

Donald Stufft writes:

Working on the Python stdlib is somewhat frustrating to me in this regard because the code in the stdlib is often times wildly inconsistent even within the same module.

Maybe you haven't paid attention to past discussions, but the Python stdlib is a terrible example because fixing it up is a FAQ on this list, on core-mentoriship, and (in the past) on python-dev. It has been repeatedly vetoed on the grounds that changes, even in whitespace, are likely to introduce more bugs than consistent style is worth. If you're actually working *on* the stdlib, then improve the style as you fix bugs or add features. If you are working on something that requires studying the stdlib, then you're out of luck. None of the above means that improving the stdlib's coding style is off the table as far as I'm concerned. I personally wouldn't mind seeing it happen, but I don't care that much. What it means is simply that the stdlib is the worst possible example to make your case -- everybody agrees with you about the inconsistent style, but Those Who Make Decisions Around Here think there are more important considerations than style.

I was merely offering my experience that *anything* which relies on a human to verify it is, without exception, going to be verified unevenly and that using a human to verify it invites people to attempt to argue against it more often than when a machine does it.

But the stdlib is a terrible example for that purpose for the same reason. But I take your point that using a machine can smooth correction of "mechanical" errors. Mine is simply that I don't see that it makes such a big difference in my experience; most people who have horrible coding styles respond to human advice quite well, and the "minor" differences are -- minor. YMMV.

On 20Mar2015 14:02, Ben Finney wrote:

Cameron Simpson writes:

...

In a former life I got to use SGI's C compiler. Like GCC, it had a "lots of warnings" mode, somewhat like a builtin lint. _Unlike_ GCC, the SGI compiler let one suppress specific warnings, which was very very nice.

GCC now has “diagnostic pragmas” to turn on/off specific diagnostics URL:https://gcc.gnu.org/onlinedocs/gcc-4.3.3/gcc/Diagnostic-Pragmas.html#Dia..., is that what you mean?

In terms of function, yes, though it says "only warnings [...] can be controlled, and not all of them". In terms of usability, no. I'd rather put compiler options to control noise in Makefiles (or whatever built system) rather than litter code with compiler specific junk which has no effect (one hopes) elsewhere (other platforms, other compilers). Also, I don't think these #pragmas were around at the time:-) Cheers, Cameron Simpson Tim's Top 5: #5. Know when to give cages the bird. #4. I'm not hanging off, so you don't need to hang off either. (especially on the "up" side) #3. Always let me know when you're gonna get on and off. And by God, don't let your boot scratch my polished exhaust canister. #2. Remain seated until the aircraft has safely stopped at the passenger terminal and the seat belt sign has been extinguished. #1. Shut the hell up, it'll all be over in a minute. - Timothy R. Redden

On 20Mar2015 17:06, Ryan Gonzalez wrote:

Can't you do that witout pragmas?

gcc tst.c -Wall -Wno-parenthesis -Wno-unused-variable ...

Now, presumably yes. Then, definitely no. So I presume the situation is improved a lot. Cheers, Cameron Simpson If you 'aint falling off, you ar'nt going hard enough. - Fred Gassit

On 3/25/2015 8:42 AM, Eli Bendersky wrote:

On Tue, Mar 17, 2015 at 8:29 PM, Eli Bendersky mailto:eliben@gmail.com> wrote:

Interesting coincidence :-)

We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code. It should be posted to Github within the next few days or weeks, under a permissive license (Apache 2.0 or something similar).

The tool works like gofmt and clang-format (the C++/JS version provided as part of the Clang toolchain) - it only acts on whitespace and performs re-formatting of code (PEP-8 oriented for Python but it's configurable). It does not rename variables/classes/functions to conform to any particular style. The algorithm is based on clang-format, and lib2to3 is used for the parsing side - so it's very solid. We use it on Python 2 code internally, but Python 3 support is also planned and should be easy.

We intend to keep actively developing the tool in the open and will be happy to accept test cases, fixes and patches in general.

So stay tuned!

https://github.com/google/yapf

yet-another-python-formatter The so-called pep8 style in style.py has # The number of spaces required before a trailing comment. SPACES_BEFORE_COMMENT=1, PEP8 actually says "Inline comments should be separated by at least two spaces from the statement.", which is what you have in the google style. -- Terry Jan Reedy

On Wed, Mar 25, 2015 at 9:38 AM, Terry Reedy wrote:

On 3/25/2015 8:42 AM, Eli Bendersky wrote:

...

On Tue, Mar 17, 2015 at 8:29 PM, Eli Bendersky mailto:eliben@gmail.com> wrote:

Interesting coincidence :-)

We (at Google) are just on the verge of releasing an open-source tool for auto-formatting of Python code. It should be posted to Github within the next few days or weeks, under a permissive license (Apache 2.0 or something similar).

The tool works like gofmt and clang-format (the C++/JS version provided as part of the Clang toolchain) - it only acts on whitespace and performs re-formatting of code (PEP-8 oriented for Python but it's configurable). It does not rename variables/classes/functions to conform to any particular style. The algorithm is based on clang-format, and lib2to3 is used for the parsing side - so it's very solid. We use it on Python 2 code internally, but Python 3 support is also planned and should be easy.

We intend to keep actively developing the tool in the open and will be happy to accept test cases, fixes and patches in general.

So stay tuned!

https://github.com/google/yapf

yet-another-python-formatter

The so-called pep8 style in style.py has

# The number of spaces required before a trailing comment. SPACES_BEFORE_COMMENT=1,

PEP8 actually says "Inline comments should be separated by at least two spaces from the statement.", which is what you have in the google style.

Thanks Terry. Devin opened an issue on Github and we'll fix this. Eli

-- Terry Jan Reedy

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/