Hi Stephen. Not sure about all of the pep8 suggestions, but I think this is important. Some are purely style/consistency, like whitespace before ')'. Some would hold yt back from Python3 in the future, like .has_key() is deprecated, use 'in'. Do you think it will be enough to autopep and run the full tests? I have no idea about coverage, but it would be great to fix some of these. Best, Casey On Tue, Jan 31, 2012 at 7:00 PM, Stephen Skory wrote:

Hi all,

while vegging out in front of the TV, I've been playing with a pep8 checker (http://pypi.python.org/pypi/pep8), and I get the statistics below for all the *.py files in yt. How much do we care about this stuff? There are automatic tools (http://pypi.python.org/pypi/autopep8/0.3) but I don't know if we can trust them. Thoughts?

(the first column is the number of incidents)

70 E101 indentation contains mixed spaces and tabs 76 E111 indentation is not a multiple of four 605 E201 whitespace after '[' 68 E202 whitespace before ')' 94 E203 whitespace before ',' 105 E211 whitespace before '[' 191 E221 multiple spaces before operator 57 E222 multiple spaces after operator 3769 E225 missing whitespace around operator 4271 E231 missing whitespace after ',' 1080 E251 no spaces around keyword / parameter equals 986 E261 at least two spaces before inline comment 6 E262 inline comment should start with '# ' 414 E301 expected 1 blank line, found 0 2061 E302 expected 2 blank lines, found 1 369 E303 too many blank lines (2) 168 E401 multiple imports on one line 4644 E501 line too long (80 characters) 1760 E701 multiple statements on one line (colon) 276 E702 multiple statements on one line (semicolon) 70 W191 indentation contains tabs 1585 W291 trailing whitespace 3 W292 no newline at end of file 3163 W293 blank line contains whitespace 221 W391 blank line at end of file 197 W601 .has_key() is deprecated, use 'in' 17 W602 deprecated form of raising exception

-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

I'm in favor of this, too. There's a good amount of code written by myself that does not comply but has not gotten changed because I didn't want to break old scripts. I wouldn't mind fixing it all as a part of one big effort. On Wed, Feb 1, 2012 at 7:42 AM, Matthew Turk wrote:

I am broadly in favor of using PEP-8 standards. (As long as we're on the topic, Kacper also had a great suggestion that we run static analysis like pyflakes.) Our oldest open ticket, from pre-BB migration, is about PEP-8.

https://bitbucket.org/yt_analysis/yt/issue/39/follow-pep8-style-conventions

Fixing line length and so on I am fine with being done automatically with one of those tools. For changing method names, which I am also in favor of, we should either identify them and fix them manually or use a refactoring tool like rope or bicycle repair man.

For all of this I would like to do it in one go, with a single bandaid to rip off, and beforehand consolidate all extant branches. RIght now we have a couple things under active development; we could do this relatively soon, but I think we need some kind of coordinated effort to merge relatively quickly back into those branches to make sure we don't just cause a ton of merge conflicts. Here's my proposal:

* Make the change to main branch * Merge those into geometry, rockstar, volume rendering * Add pyflakes and/or pep8 checks to Jenkins

Thoughts?

-Matt

On Tue, Jan 31, 2012 at 10:56 PM, Casey W. Stark wrote:

...

Hi Stephen.

Not sure about all of the pep8 suggestions, but I think this is important. Some are purely style/consistency, like whitespace before ')'. Some would hold yt back from Python3 in the future, like .has_key() is deprecated, use 'in'.

Do you think it will be enough to autopep and run the full tests? I have no idea about coverage, but it would be great to fix some of these.

Best, Casey

On Tue, Jan 31, 2012 at 7:00 PM, Stephen Skory wrote:

...

Hi all,

while vegging out in front of the TV, I've been playing with a pep8 checker (http://pypi.python.org/pypi/pep8), and I get the statistics below for all the *.py files in yt. How much do we care about this stuff? There are automatic tools (http://pypi.python.org/pypi/autopep8/0.3) but I don't know if we can trust them. Thoughts?

(the first column is the number of incidents)

70 E101 indentation contains mixed spaces and tabs 76 E111 indentation is not a multiple of four 605 E201 whitespace after '[' 68 E202 whitespace before ')' 94 E203 whitespace before ',' 105 E211 whitespace before '[' 191 E221 multiple spaces before operator 57 E222 multiple spaces after operator 3769 E225 missing whitespace around operator 4271 E231 missing whitespace after ',' 1080 E251 no spaces around keyword / parameter equals 986 E261 at least two spaces before inline comment 6 E262 inline comment should start with '# ' 414 E301 expected 1 blank line, found 0 2061 E302 expected 2 blank lines, found 1 369 E303 too many blank lines (2) 168 E401 multiple imports on one line 4644 E501 line too long (80 characters) 1760 E701 multiple statements on one line (colon) 276 E702 multiple statements on one line (semicolon) 70 W191 indentation contains tabs 1585 W291 trailing whitespace 3 W292 no newline at end of file 3163 W293 blank line contains whitespace 221 W391 blank line at end of file 197 W601 .has_key() is deprecated, use 'in' 17 W602 deprecated form of raising exception

-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

On Wed, Feb 1, 2012 at 10:49 AM, Sam Skillman wrote:

I'm on the fence.  I think I'd be okay with this if it was done at something like a 3.0 release, which is what it is slated for in the issues.  Doing so before that seems like it will cause headaches for everyone with an active fork.  What I'm more worried about is how it is handled moving forward.  Is it reasonable to require all contributors to run with pyflakes or flake8 (http://pypi.python.org/pypi/flake8) with an option that all failures stop the commit?  Or do we just ask that they do so, let Jenkins tell us if they screwed up, and ask them to fix it?

I'm also fine with waiting for a 3.0 release, which I would like to have out by the end of this calendar year. An overall eyes-on-code initiative would fit with that. Unfortunately this is the kind of activity that fits much better with coordinated development, either remotely through some kind of audio video chat, or in person around a table. As for mandating people do it, I think having Jenkins complain and then either encouraging compliance or manually fixing minor things would be fine. Right now we are still setting the standards for how to develop; we can strive to set a good example and encourage those practices by others. Mentoring during early development is a good idea, as well. We actually have style guides which aren't commonly read, but if we are serious about improving the code smell we can try to make this a bigger priority when evaluating pull requests, suggesting projects, going over diffs, etc. -Matt

Anyways, I guess I'm a +0.

Sam

On Wed, Feb 1, 2012 at 5:45 AM, Britton Smith wrote:

...

I'm in favor of this, too.  There's a good amount of code written by myself that does not comply but has not gotten changed because I didn't want to break old scripts.  I wouldn't mind fixing it all as a part of one big effort.

On Wed, Feb 1, 2012 at 7:42 AM, Matthew Turk wrote:

...

I am broadly in favor of using PEP-8 standards.  (As long as we're on the topic, Kacper also had a great suggestion that we run static analysis like pyflakes.)  Our oldest open ticket, from pre-BB migration, is about PEP-8.

https://bitbucket.org/yt_analysis/yt/issue/39/follow-pep8-style-conventions

Fixing line length and so on I am fine with being done automatically with one of those tools.  For changing method names, which I am also in favor of, we should either identify them and fix them manually or use a refactoring tool like rope or bicycle repair man.

For all of this I would like to do it in one go, with a single bandaid to rip off, and beforehand consolidate all extant branches.  RIght now we have a couple things under active development; we could do this relatively soon, but I think we need some kind of coordinated effort to merge relatively quickly back into those branches to make sure we don't just cause a ton of merge conflicts.  Here's my proposal:

 * Make the change to main branch  * Merge those into geometry, rockstar, volume rendering  * Add pyflakes and/or pep8 checks to Jenkins

Thoughts?

-Matt

On Tue, Jan 31, 2012 at 10:56 PM, Casey W. Stark wrote:

...

Hi Stephen.

Not sure about all of the pep8 suggestions, but I think this is important. Some are purely style/consistency, like whitespace before ')'. Some would hold yt back from Python3 in the future, like .has_key() is deprecated, use 'in'.

Do you think it will be enough to autopep and run the full tests? I have no idea about coverage, but it would be great to fix some of these.

Best, Casey

On Tue, Jan 31, 2012 at 7:00 PM, Stephen Skory wrote:

...

Hi all,

while vegging out in front of the TV, I've been playing with a pep8 checker (http://pypi.python.org/pypi/pep8), and I get the statistics below for all the *.py files in yt. How much do we care about this stuff? There are automatic tools (http://pypi.python.org/pypi/autopep8/0.3) but I don't know if we can trust them. Thoughts?

(the first column is the number of incidents)

70      E101 indentation contains mixed spaces and tabs 76      E111 indentation is not a multiple of four 605     E201 whitespace after '[' 68      E202 whitespace before ')' 94      E203 whitespace before ',' 105     E211 whitespace before '[' 191     E221 multiple spaces before operator 57      E222 multiple spaces after operator 3769    E225 missing whitespace around operator 4271    E231 missing whitespace after ',' 1080    E251 no spaces around keyword / parameter equals 986     E261 at least two spaces before inline comment 6       E262 inline comment should start with '# ' 414     E301 expected 1 blank line, found 0 2061    E302 expected 2 blank lines, found 1 369     E303 too many blank lines (2) 168     E401 multiple imports on one line 4644    E501 line too long (80 characters) 1760    E701 multiple statements on one line (colon) 276     E702 multiple statements on one line (semicolon) 70      W191 indentation contains tabs 1585    W291 trailing whitespace 3       W292 no newline at end of file 3163    W293 blank line contains whitespace 221     W391 blank line at end of file 197     W601 .has_key() is deprecated, use 'in' 17      W602 deprecated form of raising exception

-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Hi Stephen, On Wed, Feb 1, 2012 at 12:04 PM, Stephen Skory wrote:

For what it's worth, I played around with the autopep8 thing I linked to before, and after applying it "python setup.py install" works (modulo a few source files autopep8 couldn't handle for some reason). Looking through the output, it fixed a few things like comment and operator spacing, but it didn't fix things like too-long lines, even when it is pretty obvious how to do that (like in a list or function prototype with many items separated by commas).

Looking at the source, looks like you have to define transformation functions: https://github.com/hhatto/autopep8/blob/master/autopep8.py#L156 Error 501 is the line-too-long, and it's not defined. (-v supplies this info.) Also, looks like autopep8 is written by the same person as pyprof2html.

I haven't run any tests to see if any logic was broken.

I agree with Matt that we'd probably have to do this by hand. I propose that we at least try to run the pep8 checker on any file we are about to commit a change to before committing, and fix the errors. Doing it this way would take a long time to do the whole code base, clearly, but there's no reason to make the problem worse before we decide to do the whole thing.

This is definitely possible. It's almost always a bad idea to propose a technical "solution" to a sociological problem, but we could supply optional pep8 hg hooks for adding new files. -Matt

-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

As a followup, Stephen submitted a pull request to update a bunch of PEP8 stuff he did by hand. He and I chatted in IRC before I accepted it, and he has told me he's going to refrain from any further pull requests until we bring in both the geometry_handling and volume_refactor branches. Changing intra-line whitespace is a nice easy way of causing merge conflicts that are super annoying, so having fewer branches when we do this is a good idea. On Wed, Feb 1, 2012 at 12:20 PM, Matthew Turk wrote:

Here's a diff of the entire code base run through autopep8:

http://yt-project.org/files/pep8.diff

On Wed, Feb 1, 2012 at 12:15 PM, Matthew Turk wrote:

...

Hi Stephen,

On Wed, Feb 1, 2012 at 12:04 PM, Stephen Skory wrote:

...

For what it's worth, I played around with the autopep8 thing I linked to before, and after applying it "python setup.py install" works (modulo a few source files autopep8 couldn't handle for some reason). Looking through the output, it fixed a few things like comment and operator spacing, but it didn't fix things like too-long lines, even when it is pretty obvious how to do that (like in a list or function prototype with many items separated by commas).

Looking at the source, looks like you have to define transformation functions:

https://github.com/hhatto/autopep8/blob/master/autopep8.py#L156

Error 501 is the line-too-long, and it's not defined.  (-v supplies this info.)

Also, looks like autopep8 is written by the same person as pyprof2html.

...

I haven't run any tests to see if any logic was broken.

I agree with Matt that we'd probably have to do this by hand. I propose that we at least try to run the pep8 checker on any file we are about to commit a change to before committing, and fix the errors. Doing it this way would take a long time to do the whole code base, clearly, but there's no reason to make the problem worse before we decide to do the whole thing.

This is definitely possible.  It's almost always a bad idea to propose a technical "solution" to a sociological problem, but we could supply optional pep8 hg hooks for adding new files.

-Matt

...

-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org