are final newlines appended if output lacks it?

[Bengt Richter]

On Sun, 4 May 2003 13:09:29 -0400, "Tim Peters" <tim_one at email.msn.com> wrote:

>[Bengt Richter]
>> So is the goal is to guarantee a final EOL if the last output to
>> sys.stdout was from a print?
>
>I don't know, but that appears to be the intent of the code.
>
>> (Apparently sys.stdout.write() resets sys.stdout.softspace
>> unconditionally,
>
>Yes.
>
>> so it's allowed to have the last word with no tack-ons,
>> but a final print with a trailing comma will get its softspace
>> turned to an EOL?
>
>Looks that way, yes.
>
>> I guess this is just policy, and not related to the apparent
>> (IMO) bug that confuses interactive echo with program output
>> to stdout, so that interactive echo of a typed line end
>> erroneously (IMO) resets sys.stdout.softspace.
>
>Heh -- I couldn't parse that.  If you think you have a bug, and you care
>about it, please open a bug report on SourceForge.
>
Well, I think there is a bug, but my original diagnosis was wrong.
What I mean is that during an interactive session there are effectively
several output streams that intertwingle unless there is redirection.
One stream is the echoing of the source you type into the console,
(handled by the console device, as you point out), and others are programmed
outputs to the console via stdout or stderr or sneakily.

Print goes to stdout unless redirected, so normally it is desirable to
flush the output from each chunk of interactive source processed, to
keep things nicely sequenced on the screen. Ok so far.

It is also desirable to have a new line at the end, so the prompt
can be on a fresh line, yet avoiding extra newlines. This probably
led to the temptation of processing softspace in flush, since it
seems similar to what successive prints have to do.

But why should flushing do anything with softspace? Flushing is not logically
for creating new output, it is just making sure all buffered output has gone out.
Softspace is not logically part of that data IMO. To generate a '\n' conditionally
for interactive screen formatting purposes is logically a different thing from
concatenating print statement outputs. Using flush to do the combined function
is mistaken IMO, and it shows up especially when stdout is not going to the screen.

I suspect the flush at the end of a block of input should only flush the buffers,
and not generate any output to stdout, and not change softspace. Instead of using
up softspace to generate a conditional '\n' through stdout, it should output a
conditional '\n' to the prompt stream (stderr), where the condition should be
sys.stdout is sys.__stdout__ and sys.stdout.softspace. IOW, if stdout is not
going to the console, the last interactive input will have echoed a newline, so
there would be no need, and if stdout _is_ going to the console, then a newline
can go to stderr if softspace is set -- without resetting it, however.

Not resetting softspace will mean a possible leading space on a line from
a subsequent print statement. That is good! It shows what was actually pending
in the output sequence.

>> Note:
>>
>>  >>> import sys
>>  >>> class SO:
>>  ...     def write(self,s): sys.stderr.write(`s`+'\n')
>>  ...
>>  >>> sys.stdout = SO()
>>  >>> print sys.stdout.softspace,; print sys.stdout.softspace,
>>  '0'
>>  ' '
>>  '1'
>>  '\n'
>
>Python calls Py_FlushLine() (I showed the source before) after interpreting
>each block it reads from the shell.  This was one block of input, so
As mentioned, ISTM Py_Flushline should not do anything with softspace, nor
create a new output character.

>Py_FlushLine was called once after the entire block completed, and because
One could argue that if stdout is redirected, there is no need to flush it,
unless you think there might be side effects from the output that should be
observed chunk by chunk. This seems like a debugging aid, rather than how
the program would act being run non-interactively. So maybe non-console
stdout flushing should be a command line option?

>the block left sys.stdout.softspace true, Py_FlushLine() wrote a newline to
>sys.stdout and cleared sys.stdout.softspace.

>
>>  >>> print sys.stdout.softspace,; print sys.stdout.softspace
>>  '0'
>>  ' '
>>  '1'
>>  '\n'
>
>Ditto.
>
>>  >>> def foo():
>>  ...     print sys.stdout.softspace,; print sys.stdout.softspace,
>>  ...     print sys.stdout.softspace,; print sys.stdout.softspace,
>>  ...
>
>This block didn't set sys.stdout.softspace, so Py_FlushLine didn't do
>anything when the interpretation of the block completed.
>
>>  >>> foo()
>>  '0'
>>  ' '
>>  '1'
>>  ' '
>>  '1'
>>  ' '
>>  '1'
>>  '\n'
>
>This block left sys.stdout.softspace set when the block (the call to foo())
>completed, so Py_FlushLine did write a newline to sys.stdount (and cleared
>sys.stdout.softspace).
IMO if you want to flush stdout, softspace should be left pending, and
only the characters _up to_ the as-yet-undetermined space-or-EOL should be
flushed.

>
>> Note that the interactive call to foo() only echoed one EOL, so the
>> softspace from the first print line survived to influence the second line
>> inside foo, unlike for the separate invocations preceding.
>
>Yes, there's one call to Py_FlushLine per interactive block, after the block
>completes executing.  The number of Python statements executed per
>interactive block makes no difference.
Ok & ditto.
>
>> If the interactive _echo_ is actually going to sys.__stdout__
>
>I'm not sure what "interactive echo" means, but under the only plausible
>meanings I can think of, it would be out of Python's hands, controlled by
>however your platform C's (and possible GNU readline, if you're using that)
>reading from stdin interacts with the console device.  Python isn't trying
>to echo, or not to echo, anything you type.  Python prints interactive
>*prompts* ('>>> ' and '... ') to C's idea of what stderr means.
>
>> when normal sys.stdout is redirected, perhaps sys.__stdout__.softspace
>> should be reset for the interactive EOL echo instead of
>> sys.stdout.softspace?
>
>There is no "interactive EOL echo" in a sense I can grok.  Python is
>apparently trying to make sure that the output produced by print statements,
>to sys.stdout, produced by each interactive block, ends with a newline, and
ISTM the only reason to force an ending newline is for interactive screen purposes.
That's different from the softspace issue. Faking the current flush softspace processing
with a conditional newline to stderr after flushing stdout _up to_ the softspace would be
ok, for better screen interaction, but only ok if softspace processing were left pending, IMO,
so that a redirected stdout output stream is independent of whether you typed
your print statements separately or e.g. in a block under an "if 1:".

>that's all.  Whether the EOLs (or anything else) you type get echoed is a
>function of your console device, and don't affect sys.stdout.softspace
>either way.
Right. I sort of jumped to the right (IMO) conclusion for the wrong reason ;-/
But from the above, do you now see the bug I see?

I don't even think converting softspace to a final EOL would be desirable at
the end of a program (on close) executed as an ordinary script. Would it
break code to change the flush logic? Maybe such code should break ;-)

Isn't softspace logically just about concatenating successive print statement
outputs (with convenience resetting by stdout.write)? And shouldn't the
result be independent of whether (and how chunked) those statements were
executed interactively, if stdout is redirected?

Regards,
Bengt Richter