tokenize.untokenize adding line continuation characters

[sg552 at hotmail.co.uk]

On Tuesday, January 17, 2017 at 2:47:03 AM UTC, Steven D'Aprano wrote:
> On Tuesday 17 January 2017 09:42, Rotwang wrote:
> 
> > Here's something odd I've found with the tokenize module: 
> [...]
> > Copypasted from iPython:
> 
> It's not impossible that iPython is doing something funny with the tokenize 
> module.

It happens outside iPython:

Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import io, tokenize
>>> tokenize.untokenize(tokenize.tokenize(io.BytesIO('if x:\n    y'.encode()).readline)).decode()
'if x:\n    y\\\n'


and also in Python 2:

Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import io, tokenize
>>> tokenize.untokenize(tokenize.generate_tokens(io.BytesIO('if x:\n    y').readline))
'if x:\n    y\\\n'


> Before reporting it as a bug, I recommend that you confirm that it also 
> occurs in the standard Python interpreter and isn't iPython specific.

Is this behaviour actually a bug, as opposed to a feature I don't understand?