[Tutor] Terminating "\M" characters with regard to csv.DictWriter

[Peter Otten]

On 09/08/2021 04:05, Alex Kleider wrote:
> Using the csv module's DictReader and DictWriter to update data in a file
> one of my functions is as follows:
> 
> def dict_write(f, fieldnames, iterable):
>      """
>      Writes all records received from <iterable> into a new csv
>      file named <f>.  <fieldnames> defines the record keys.
>      Code writen in such a way that <iterable> could be
>      a generator function.
>      """
>      with open(f, 'w', newline="") as outfile_obj:
>          print("Opening {} for output...".format(outfile_obj.name))
>          dict_writer = csv.DictWriter(outfile_obj, fieldnames)
>          dict_writer.writeheader()
>          for record in iterable:
>              dict_writer.writerow(record)
> 
> All works as I would like except that the newly generated csv file has a
> '\M' at the end of each line.

As explained by others the ^M or chr(13) or "\r" is part of a windows 
line ending "\r\n".
I'd like to clarify that it is *not* inserted by the file object but by 
the csv.writer(). Its default dialect is "excel" which writes "\r\n" as 
the line separator regardless of the operating system's line terminator 
(os.linesep). To avoid it you can either override the line terminator with

# excel dialect, but with newlines only
writer = csv.writer(outstream, lineterminator="\n")

or pick a different dialect

writer = csv.writer(outstream, dialect="unix")


 >>> f = open("tmp.csv", "w", newline="")
 >>> csv.writer(f).dialect.lineterminator
'\r\n'
 >>> csv.writer(f, lineterminator="\n").dialect.lineterminator
'\n'
 >>> csv.writer(f, dialect="unix").dialect.lineterminator
'\n'

   I only became aware of this when doing a
> diff between the original file and the one produced by my function. When
> examining the files using vim (my text editor of choice) they look the same
> (except for the changes I meant to make) so things seem to be working fine
> and the system doesn't seem to mind.
> Should I be setting the named 'newline' parameter to something other than
> an empty string?  From where do the '\M' characters come???  I'd very much
> like to be able to use the 'diff' command to check if the correct changes
> are being made but at present, diff marks every line as being different!
> TIA
> Alex Kleider
> 
> (Using python 7 on Debian GNU/Linux)
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>