I just checked in a whitespace normalization change that was way too big. Should this task be automated? Assuming the answer is yes, these are the questions should be answered: 1) Which branches should this occur on: trunk, 2.5 (last release), 3k 2) Should there be a special user for these checkins 3) How often should this be done 4) Are there any other tasks that can be automated My preferences are: 1) all active branches (ie, trunk and currently 2.5 and p3yk) 2) Yes, there should be a special user 3) Once per week 4) Not that I know of The way to fix the files is to run: python ./Tools/scripts/reindent.py -r Lib At least that's what I did. Hopefully I didn't screw anything up. :-) n
Neal Norwitz schrieb:
I just checked in a whitespace normalization change that was way too big. Should this task be automated?
Assuming the answer is yes, these are the questions should be answered: 1) Which branches should this occur on: trunk, 2.5 (last release), 3k 2) Should there be a special user for these checkins
Absolutely, since that will be very helpful when doing "svn praise".
3) How often should this be done
Weekly makes sense. Of course, we could also setup a svn pre-commit hook that rejects trailing whitespace >:->
4) Are there any other tasks that can be automated
My preferences are: 1) all active branches (ie, trunk and currently 2.5 and p3yk) 2) Yes, there should be a special user 3) Once per week 4) Not that I know of
The way to fix the files is to run: python ./Tools/scripts/reindent.py -r Lib At least that's what I did. Hopefully I didn't screw anything up. :-)
I looked over it, and everything seems alright ;) Though I was impressed that we had some tab- and 2 space-indents lying around. Georg
Georg Brandl wrote:
Of course, we could also setup a svn pre-commit hook that rejects trailing whitespace >:->
I was about to suggest a mail to python-checkins when a bad whitespace is detected, but had no idea that a pre-commit check existed in SVN. It'd be great if the system won't let you commit if whitespace is bad. Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/
Facundo Batista schrieb:
Georg Brandl wrote:
Of course, we could also setup a svn pre-commit hook that rejects trailing whitespace >:->
I was about to suggest a mail to python-checkins when a bad whitespace is detected, but had no idea that a pre-commit check existed in SVN.
It'd be great if the system won't let you commit if whitespace is bad.
Well, there are editors that don't intelligently strip whitespace, so that people using them would be constantly pained by such a hook. Things like two-space indents or tabs, however, should really be rejected IMO. Stripping the trailing whitespace automatically on checkin would also be fine, but I don't think that kind of thing is possible with SVN (also, it would mean that your WC is out of date after checkin). Geog -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
On 4/25/07, Georg Brandl
Well, there are editors that don't intelligently strip whitespace, so that people using them would be constantly pained by such a hook.
And they should. There really is no excuse for letting one developer's poor choice of tools cause later grief for all other developers. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
>> Well, there are editors that don't intelligently strip whitespace, so >> that people using them would be constantly pained by such a hook. Guido> And they should. There really is no excuse for letting one Guido> developer's poor choice of tools cause later grief for all other Guido> developers. Just a little FYI, python-mode (the one Barry and I manage - dunno about the one distributed w/ GNU Emacs these days) is one of those tools that leaves trailing whitespace behind when advancing to the next line.. Skip
On 4/25/07, skip@pobox.com
>> Well, there are editors that don't intelligently strip whitespace, so >> that people using them would be constantly pained by such a hook.
Guido> And they should. There really is no excuse for letting one Guido> developer's poor choice of tools cause later grief for all other Guido> developers.
Just a little FYI, python-mode (the one Barry and I manage - dunno about the one distributed w/ GNU Emacs these days) is one of those tools that leaves trailing whitespace behind when advancing to the next line..
Funny -- that's not my experience. Perhaps this depends on a separate Emacs setting? That's what I always assumed. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
skip> Just a little FYI, python-mode (the one Barry and I manage - dunno skip> about the one distributed w/ GNU Emacs these days) is one of those skip> tools that leaves trailing whitespace behind when advancing to the skip> next line.. At least so I thiought. I know I've seen these turds before, but a quick check wouldn't reproduce it... Skip
skip@pobox.com schrieb:
skip> Just a little FYI, python-mode (the one Barry and I manage - dunno skip> about the one distributed w/ GNU Emacs these days) is one of those skip> tools that leaves trailing whitespace behind when advancing to the skip> next line..
At least so I thiought. I know I've seen these turds before, but a quick check wouldn't reproduce it...
This isn't my experience either. Also, there's always show-trailing-whitespace and trailing-whitespace-face... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Apr 25, 2007, at 2:37 PM, skip@pobox.com wrote:
Well, there are editors that don't intelligently strip whitespace, so that people using them would be constantly pained by such a hook.
Guido> And they should. There really is no excuse for letting one Guido> developer's poor choice of tools cause later grief for all other Guido> developers.
Just a little FYI, python-mode (the one Barry and I manage - dunno about the one distributed w/ GNU Emacs these days) is one of those tools that leaves trailing whitespace behind when advancing to the next line..
Maybe we should include this function in the mode: ;; untabify and clean up lines with just whitespace (defun baw-whitespace-normalization () "Like untabify, but also cleans up lines with trailing whitespace." (interactive) (save-excursion (save-restriction (untabify (point-min) (point-max)) (goto-char (point-min)) (while (re-search-forward "[ \t]+$" nil t) (let ((bol (save-excursion (beginning-of-line) (point))) (eol (save-excursion (end-of-line) (point)))) (goto-char (match-beginning 0)) (if (and (bolp) (eq (char-after) ?\)) (forward-char 1)) (skip-chars-backward " \t" bol) (delete-region (point) eol) )) ;; Now clean up any trailing blank lines (goto-char (point-max)) (skip-chars-backward " \t\n") (if (not (bolp)) (forward-char 1)) (delete-region (point) (point-max)) ;; But make sure the files ends in a newline (if (not (bolp)) (newline)) ))) (defalias 'baw-normalize-whitespace 'baw-whitespace-normalization) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRi+2y3EjvBPtnXfVAQKyCAP5AUwHxTgOCRkkgCRfAP/oWCxf1WT/Y1lk PUrb4eRaR2s5B8XP417V3O/uMPHfiQr9Ci1txoIuq22PRbPvOFT4bfSYXFxPrrGS FOlvivpE8rnXSxv4bJC/3B11GNDoyQJds8D4/13XIZXnFE4W2WtyA0fG1kxQXBEc gvjk/FXuRS8= =3Wll -----END PGP SIGNATURE-----
[Skip]
Just a little FYI, python-mode (the one Barry and I manage - dunno about the one distributed w/ GNU Emacs these days) is one of those tools that leaves trailing whitespace behind when advancing to the next line..
Shouldn't be -- unless the behavior of the Emacs newline-and-indent has changed. The pymode version (py-newline-and-indent) relies in part on newline-and-indent; the intent is documented in the py-newline-and-indent docstring: (defun py-newline-and-indent () "... In general, deletes the whitespace before point, inserts a newline, and takes an educated guess as to how you want the new line indented." IIRC, pymode leaves C-j bound to plain old newline, so if you're in the habit of starting a new line via C-j no changes of any kind are made to whitespace. But that's not the intended way to use pymode.
skip@pobox.com wrote:
Just a little FYI, python-mode (the one Barry and I manage - dunno about the one distributed w/ GNU Emacs these days) is one of those tools that leaves trailing whitespace behind when advancing to the next line..
I get extremely annoyed with editors that *don't* leave whitespace at the end of lines alone, at least while you're working on the file. You type part of a line, leave a space, go to copy something from somewhere else to put on the end, and when you get back, the space you deliberately left there is gone. Stripping trailing whitespace on saving wouldn't be so bad, though. Is there an option for that in python-mode? -- Greg
>> Just a little FYI, python-mode (the one Barry and I manage - dunno >> about the one distributed w/ GNU Emacs these days) is one of those >> tools that leaves trailing whitespace behind when advancing to the >> next line.. (See my earlier retraction...) Greg> I get extremely annoyed with editors that *don't* leave whitespace Greg> at the end of lines alone, at least while you're working on the Greg> file. You type part of a line, leave a space, go to copy something Greg> from somewhere else to put on the end, and when you get back, the Greg> space you deliberately left there is gone. That's not the way python-mode works. If I enter this: def f(a): b = a * a @ % then hit LF or RET after the statement, it inserts a newline and inserts enough spaces to leave the cursor where I placed the @ sign. If I hit another LF or RET it removes the "trailing" whitespace on the current line, enters a newline and inserts enough spaces to leave the cursor where I placed the % sign. Greg> Stripping trailing whitespace on saving wouldn't be Greg> so bad, though. Is there an option for that in python-mode? Not by default. You could add a hook to write-file-hooks though. Skip
Greg Ewing
Stripping trailing whitespace on saving wouldn't be so bad, though. Is there an option for that in python-mode?
I've got thje following in my ~/.emacs: (if (fboundp 'delete-trailing-whitespace) (add-hook 'write-file-hooks 'delete-trailing-whitespace) ) In modern Emacsen, the `if` should be unnecessary. -- Christian Tanzer http://www.c-tanzer.at/
Guido van Rossum schrieb:
On 4/25/07, Georg Brandl
wrote: Well, there are editors that don't intelligently strip whitespace, so that people using them would be constantly pained by such a hook.
And they should. There really is no excuse for letting one developer's poor choice of tools cause later grief for all other developers.
Okay, attached is a pre-commit hook script for that purpose. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. #!/usr/bin/env python SVNLOOK = '/usr/bin/svnlook' SCRIPTS_PATH = '/home/gbr/devel/python/Tools/scripts' import sys sys.path.insert(0, SCRIPTS_PATH) _, repos, txn = sys.argv from subprocess import Popen, PIPE from reindent import Reindenter from StringIO import StringIO def get_output(*command): popen = Popen(command, stdout=PIPE) stdout = popen.communicate()[0] if popen.returncode: sys.exit(1) else: return stdout bad = 0 for line in get_output(SVNLOOK, 'changed', '-t', txn, repos).splitlines(): filename = line[4:].strip() if not filename[-3:] == '.py': continue content = StringIO(get_output(SVNLOOK, 'cat', '-t', txn, repos, filename)) reindenter = Reindenter(content) if reindenter.run(): print >>sys.stderr, "file %s is not whitespace-normalized" % filename bad += 1 if bad: sys.exit(1)
Martin v. Löwis schrieb:
Okay, attached is a pre-commit hook script for that purpose.
How does that deal with deletions?
Ah, you'd have to look at the first two chars of every lines that I cut away.
What do you think about the attached alternative?
I'd say it's better since it uses the Python SVN bindings instead of calling external programs... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
"Guido van Rossum"
> Just a little FYI, python-mode (the one Barry and I manage - dunno > about the one distributed w/ GNU Emacs these days) is one of those > tools that leaves trailing whitespace behind when advancing to the > next line.. Okay, I figured this out. The dangling whitespace turds are caused by LF or RET followed by a regular cursor motion keypress (e.g. C-n or the arrow keys) which move the cursor away from that line. Those commands operate outside python-mode's control there so it doesn't get the opportunity to remove the whitespace before moving the cursor. XEmacs (at least) doesn't have a cursor-motion-hook so there's no clean way to do this. I've been using Emacs for so many years that my central control program long ago pushed out most of the basic cursor motion control code to ganglia which reside in my wrists. The channel is full duplex but highly assymetric. The upload speed is very slow. It thus takes a couple days to realize in my frontal cortex how this low level stuff works... Skip
"Neal Norwitz"
I just checked in a whitespace normalization change that was way too big. Should this task be automated?
IMHO, changing whitespace retrospectively in a version control system is a bad idea. How much overhead would it be to have a checkin hook which compares each modified file against the output of running reindent.py over the same file and rejects the checkin if it changes anything? (With of course an appropriate message suggesting the use of Reindent.py before reatttempting the checkin). That way the whitespace ought to stay normalized so you shouldn't need a separate cleanup step and you won't be breaking diff and blame for the sources (and if the reindent does ever break anything it should be more tracable).
Duncan Booth wrote:
"Neal Norwitz"
wrote in news:ee2a432c0704242352w5a489a98sd1520427737dda6c@mail.gmail.com: I just checked in a whitespace normalization change that was way too big. Should this task be automated?
IMHO, changing whitespace retrospectively in a version control system is a bad idea.
How much overhead would it be to have a checkin hook which compares each modified file against the output of running reindent.py over the same file and rejects the checkin if it changes anything? (With of course an appropriate message suggesting the use of Reindent.py before reatttempting the checkin).
That way the whitespace ought to stay normalized so you shouldn't need a separate cleanup step and you won't be breaking diff and blame for the sources (and if the reindent does ever break anything it should be more tracable).
+1 Enforcing whitespace correctness on checkin has the added advantage that we will be able to screw another 1% out of uncle Timmy, who will no longer have to make his repeated whitespace correction checkins and will therefore have time for more productive tasks. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Recent Ramblings http://holdenweb.blogspot.com
On Wednesday 25 April 2007, Steve Holden wrote:
Duncan Booth wrote:
That way the whitespace ought to stay normalized so you shouldn't need a separate cleanup step and you won't be breaking diff and blame for the sources (and if the reindent does ever break anything it should be more tracable).
+1
+1 here as well; there's no need to let things like this get out-of-sync from what we want. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
Steve Holden schrieb:
Duncan Booth wrote:
"Neal Norwitz"
wrote in news:ee2a432c0704242352w5a489a98sd1520427737dda6c@mail.gmail.com: I just checked in a whitespace normalization change that was way too big. Should this task be automated?
IMHO, changing whitespace retrospectively in a version control system is a bad idea.
How much overhead would it be to have a checkin hook which compares each modified file against the output of running reindent.py over the same file and rejects the checkin if it changes anything? (With of course an appropriate message suggesting the use of Reindent.py before reatttempting the checkin).
That way the whitespace ought to stay normalized so you shouldn't need a separate cleanup step and you won't be breaking diff and blame for the sources (and if the reindent does ever break anything it should be more tracable).
+1
Enforcing whitespace correctness on checkin has the added advantage that we will be able to screw another 1% out of uncle Timmy, who will no longer have to make his repeated whitespace correction checkins and will therefore have time for more productive tasks.
As I said, this might be a pain for users of editors which don't strip indentation whitespace intelligently; e.g. if you write def foo(): print 1 print 2 there might be 4 spaces left on line 3. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
On Wed, Apr 25, 2007 at 08:40:22AM +0100, Duncan Booth wrote:
IMHO, changing whitespace retrospectively in a version control system is a bad idea.
In my experience Duncan's assertion is absolutely true, and all the more so in a project that maintains a large body of pending patches, as Python does. Spurious whitespaces changes in the repo will generate patch rejections that will drive both maintainers (as they check in a patch) and submitters (as they try to refresh patches against head) mad. I'm very much interested in VC systems *not* driving developers mad! Dustin
[Neal Norwitz]
... The way to fix the files is to run: python ./Tools/scripts/reindent.py -r Lib
I apply it to everything in the checkout. That is, I run reindent.py from the root of my sandbox, and use "." instead of "Lib". The goal is that every .py file (not just under Lib/) that eventually shows up in a release use whitespace consistently.
At least that's what I did. Hopefully I didn't screw anything up. :-)
reindent.py has never been blamed for a "legitimate" screwup. On rare occasions it has broken tests, but only when the code mysteriously relied on significant trailing whitespace in the .py file. Such invisible dependence is considered (by me :-)) to be a bug in the code, not in reindent.py. The other no-brainer is to run Tools/scripts/svneol.py regularly. That finds text files that don't have the svn:eol-style property set, and sets it.
Tim> ... but only when the code mysteriously Tim> relied on significant trailing whitespace in the .py file. Maybe \s should expand to a single space by the lexer so people who want to rely on trailing spaces can be explicit about it. There already exists non-whitespace escape sequences for tabs and newlines. Skip
[Skip]
Maybe \s should expand to a single space by the lexer so people who want to rely on trailing spaces can be explicit about it. There already exists non-whitespace escape sequences for tabs and newlines.
Trailing whitspace is never significant on a code line, only inside a multiline string literal. In the extremely rare cases where one of those requires trailing spaces, \x20 works fine.
Neal Norwitz schrieb:
I just checked in a whitespace normalization change that was way too big. Should this task be automated?
Assuming the answer is yes, these are the questions should be answered: 1) Which branches should this occur on: trunk, 2.5 (last release), 3k 2) Should there be a special user for these checkins
What *really* should be there is a pre-commit hook that rejects a checkin if the violates the formatting style in force. Regards, Martin
participants (16)
-
"Martin v. Löwis"
-
Barry Warsaw
-
Duncan Booth
-
dustin@v.igoro.us
-
Facundo Batista
-
Fred L. Drake, Jr.
-
Georg Brandl
-
glyph@divmod.com
-
Greg Ewing
-
Guido van Rossum
-
Neal Norwitz
-
skip@pobox.com
-
Steve Holden
-
tanzer@swing.co.at
-
Terry Reedy
-
Tim Peters