[Python-3000] problem with checking whitespace in svn pre-commit hook

[Neal Norwitz]

When I originally tried to check in rev 55797, I got this exception:

Traceback (most recent call last):
  File "/data/repos/projects/hooks/checkwhitespace.py", line 50, in ?
    run_app(main)
  File "/usr/lib/python2.3/site-packages/svn/core.py", line 33, in run_app
    return apply(func, (pool,) + args, kw)
  File "/data/repos/projects/hooks/checkwhitespace.py", line 43, in main
    if reindenter.run():
  File "/data/repos/projects/hooks/reindent.py", line 166, in run
    tokenize.tokenize(self.getline, self.tokeneater)
  File "/usr/lib/python2.3/tokenize.py", line 153, in tokenize
    tokenize_loop(readline, tokeneater)
  File "/usr/lib/python2.3/tokenize.py", line 159, in tokenize_loop
    for token_info in generate_tokens(readline):
  File "/usr/lib/python2.3/tokenize.py", line 233, in generate_tokens
    raise TokenError, ("EOF in multi-line statement", (lnum, 0))
tokenize.TokenError: ('EOF in multi-line statement', (315, 0))

I'm guessing this is because tokenization has changed between 2.3 and
3.0.  I didn't have 2.3 on my system to test with.  I ran reindent
prior to committing, but that had no effect (ie, still go the error).

I disabled the hook so I could check in the change which I'm pretty
sure is normalized.  However, we are likely to have this problem in
the future.  I fixed the script so it shouldn't raise an exception any
longer.  But people will still be prevented from checking in if this
happens again.

I wish I had modified the commit hook *before* checking in so I would
at least know which file caused it.  Oh well, I shouldn't do these
things at the end of the day or beginning depending on how you look at
it. :-)

n