[issue2384] [Py3k] line number is wrong after encoding declaration

[Hirokazu Yamamoto]

New submission from Hirokazu Yamamoto <ocean-city at users.sourceforge.net>:

# This issue inherits from issue2301.

If there is "# coding: ????" is in source code and
coding is neigher utf-8 nor iso-8859-1, line number (tok->lineno)
becomes wrong.

Please look into Parser/tokenizer.c. In this case, 
tok->decoding_state becomes STATE_NORMAL, so fp_setreadl
newly opens file but *doesn't* seek to current position.
(Or maybe can we reuse already opened file?)

So

# coding: ascii
# 1
# 2
# 3
raise RuntimeError("a")
# 4
# 5
# 6

outputs 

C:\Documents and Settings\WhiteRabbit>py3k ascii.py

Traceback (most recent call last):
  File "ascii.py", line 6, in <module>
    # 4
RuntimeError: a
[22821 refs]

One line shifted because line number wrongly +1

And

# dummy
# coding: ascii
# 1
# 2
# 3
raise RuntimeError("a")
# 4
# 5
# 6

outputs

C:\Documents and Settings\WhiteRabbit>py3k ascii.py

Traceback (most recent call last):
  File "ascii.py", line 8, in <module>
    # 5
RuntimeError: a
[22821 refs]

Two lines shifted because line number wrongly +2

----------
components: None
messages: 63905
nosy: ocean-city
severity: normal
status: open
title: [Py3k] line number is wrong after encoding declaration
versions: Python 3.0

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue2384>
__________________________________