[Python-ideas] gofmt for Python: standardized styling as a language feature

Thu Mar 19 23:13:53 CET 2015

I second Donald’s remarks regarding the impact that automatic style
checkers have on the dev process.

The Apache Spark <https://github.com/apache/spark> project, for example,
makes a number of automated checks on all proposed patches. One of those
checks is a style check.

When a proposed patch fails a style check, an informative message
explaining why is posted in the CI build log for the author to review, and
the author typically just updates their patch accordingly.

Here’s an example
<https://github.com/apache/spark/pull/4916#issuecomment-77555917> from an
Apache Spark PR that touched some Python code. The author’s patch had some
blank lines that contained whitespace, so the patch failed the automated
style check. The author just updated their patch to pass the style check
and the discussion quickly returned to the meat of what the patch was about.

>From what I’ve seen so far, the process of a patch failing style checks
plays out almost exactly like it would if a contributor had submitted a
patch that failed unit tests. The contributor just updates their patch to
pass those checks and everyone moves on. There is no arguing or friction.

And the result in the case of the Spark project is that all the Python code
in the repo passes the pep8 checks we’ve enabled. Granted, this doesn’t
mean the code is readable or beautiful or correct, but it simply means that
there is a small but non-trivial set of style problems and inconsistencies
that we just don’t have to worry about.

For a large, open source project with hundreds of contributors from many
different organizations and backgrounds, that’s a solid win.

Nick

On Thu, Mar 19, 2015 at 5:42 PM Donald Stufft <donald at stufft.io> wrote:

> On Mar 19, 2015, at 5:01 PM, Guido van Rossum <guido at python.org> wrote:
>
> On Thu, Mar 19, 2015 at 1:40 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>
>> A program that treats PEP 8 guidelines as rigid rules, that ignores the
>> first guideline "A Foolish Consistency is the Hobgoblin of Little Minds",
>> that removes the human choices intentionally included in PEP 8, is not PEP
>> 8 compliant.  I believe one of the motivations for some recent PEP 8
>> amendments was some unhappiness with pep8.py or other checkers.
>>
>
> I have always been skeptical of automatic style checkers. They encourage a
> nit-pickish attitude that makes the development process less effective, by
> encouraging critique of superficial style elements rather than of the
> code's structure and meaning. In contrast, the intention of PEP 8 was to
> make code more readable so that its structure and meaning are more easily
> understood.
>
>
> My experience is actually the exact opposite. When a machine is
> responsible for style enforcement people generally don’t feel as bad about
> being asked to adjust the formatting because it’s just a machine, it can’t
> make value judgements. When a person asks someone to adjust something to
> make it easier to read it often times ends up causing a distraction where
> people tend to feel like since it’s a person asking them to do it they are
> passing judgement on their code, or they don’t agree with it and since it’s
> a human saying it it feels more like there is room to budge so you end up
> with people more likely to be willing to ask for an exemption.
>
> In my experience projects without mechnical enforcement ends up with
> massively inconsistent style all throughout the code base. The Python
> stdlib is a good example of this. Contrast this to projects like
> github.com/pyca/cryptography where there is an automated mechanical
> enforcement of style and it’s got some bare minimum of consistency
> throughout the entire code base.
>
>
>
>>  On Wed, Mar 18, 2015 at 4:38 PM, Andrew Barnert
>>> <abarnert at yahoo.com
>>> <mailto:abarnert at yahoo.com>> wrote:
>>>
>>>     But that implies that it's worth someone making a spec out of PEP 8.
>>>     That implies going through PEP 8, getting a sense of what parts are
>>>     (unambiguously and uncontroversially) mechanizable, and reorganizing
>>>     and minimally rewriting it to get that sense across. Having numbered
>>>     rules, and named anchors for each one in the document (or at least
>>>     to narrower sections, so you can link to something more specific
>>>     than "Programming Recommendations") would also be helpful.
>>>
>>> This is an excellent point and one I did not consider.
>>>
>>
>> Having the guidelines numbered (id'ed), even though still regarded as
>> guidelines, could help communication. Checkers could then easily refer to
>> specific guidelines.
>>
>
> I worry this will just encourage the nit-picky attitude more.
>
>>
>>  On Wed, Mar 18, 2015 at 6:17 PM, Terry Reedy <tjreedy at udel.edu
>>> <mailto:tjreedy at udel.edu>> wrote:
>>>
>>>     (Note that autoPEP8 only changes whitespace, which is a small part
>>>     of PEP 8 recommendations, and the safest aspect of code to change.)
>>>
>>> Minor point: autopep8 can do more than just whitespace changes with the
>>> |--aggressive| flag
>>> <https://pypi.python.org/pypi/autopep8/#more-advanced-usage>.
>>>
>>
>> Thanks for the link.  I was specifically thinking of global renamings to
>> satisfy PEP 8's Naming Conventions.  Idlelib has a mishmash of module and
>> function/method name styles.  (I might look and see what can be done with
>> the undocumented (except as 'unstable') libe2to3.)
>>
>
> This feels hard to automate, because you don't know what names are part of
> an external specification.
>
>
>>  On Wed, Mar 18, 2015 at 7:21 PM, Terry Reedy <tjreedy at udel.edu
>>> <mailto:tjreedy at udel.edu>> wrote:
>>>
>>>     We should be careful about adding anything.  When we do, we should
>>>     add libraries, not applications.  We should especially not
>>>     officially endorse one of many opinion-based competitors.
>>>
>>> I have come around to this view.
>>>
>>
>> Accepting this view, we can still ask if the stdlib should have more
>> (policy-neutral) string or list of strings reforming components than it
>> does now.  Textwrap has a few functions for strings in general.  Idle has
>> about 10 formatting functions exposed on its editor Format menu, but they
>> are not exposed for general use.  Tools/Scripts/reindent.py contain some of
>> the same functions as does autopep8.  Should there be a code format module
>> that all three, and other programs, could import?
>>
>> Idle and Tools/Scrips/highlight.py both have code to analyze Python code
>> tokens into categories and colorize by category.  (Idle uses tkinter tags,
>> highlight.py uses ANSI or HTML tags).  Can and should both use a common
>> code analysis function that could be imported?  This might actually ease
>> maintainance.
>>
>> Should a generalized 2to3 package be renamed 'codefix' and documented? As
>> 2to3, it is already used by both Tools/Scripts/2to2 and optionally by
>> autopep8, and perhaps by others.  I believe it could also be used for some
>> non-2to3 pep8 fixes.
>
>
> I think that a quick "clean up whitespace" feature would be a nice
> addition to IDLE, provided it's a user-selectable menu items (similar to
> the existing indent/dedent operations). Refactoring, however, is fiendishly
> hard for Python -- give it a try in e.g. PyCharm and you will agree.
>
> --
> --Guido van Rossum (python.org/~guido)
>  _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150319/2db40745/attachment.html>