<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Jan 12, 2017 at 7:50 AM, Stephen J. Turnbull <span dir="ltr"><<a href="mailto:turnbull.stephen.fw@u.tsukuba.ac.jp" target="_blank">turnbull.stephen.fw@u.<wbr>tsukuba.ac.jp</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="m_4509115223586986053gmail-"> > So I see no downside to using utf-8 when the C locale is defined.<br>
<br>
</span>You don't have much incentive to look for one, and I doubt you have<br>
the experience of the edge cases (if you do, please correct me), so<br>
that does not surprise me.<br></blockquote><div><br></div><div>that's correct. I left out a sentence:</div><div><br></div><div>This is a good time for others' with experience with the ugly edge cases to speak up!</div><div><br></div><div>The real challenge is that "output" has three (at least :-) ) use cases:</div><div><br></div><div>1) Passing on data the came from input from the same system: Victors' "Unix pipe style". In that case, if a supposedly ASCII-based system has non ascii data, most users would want it to get passed through unchanged. They not likely to expect their python program to enforce their locale (unless it was a program designed to do that - but then it could be explicit about things).</div><div><br></div><div>2) The program generating data itself: the mentioned "outputting boxes to the console" example. I think that folks writing these programs should consider whether they really need non-ascii output -- but if they do do this -- I"d image most folks would rather see weird characters in the console than have the program crash.</div><div><br></div><div>So these both point to utf-8 (with surrogateescape)</div><div><br></div><div>3) A program getting input from a user, or a data file, or...... (like a filename, etc). This may be a program intended to be able to handle unicode filenames, etc. (this is my use-case :-) ) -- what should it do when run on an ASCII-only system?</div><div><br></div><div>This is the tough one -- if the C-locale indicated "non configured", then users would likely want the _something_ written to the FS, rather than a program crash: so utf-8.</div><div><br></div><div>However, if the system really is properly configured to be ASCII only, then they may want a program to never write non-ascii to the filesystem. However, ultimately, I think it's up to the application developer, rather than to Python itself (Or the sysadmin for the OS that it's running on) to know whether the app is supposed to support non-ascii filenames, etc. i.e. one should expect that running a unicode-aware app on an ascii-only filesystem is going to lead to problems anyway.</div><div><br></div><div>So I think the "never crash" option is the better one in this imperfect trade-off.</div><div><br></div><div>-CHB</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div><br></div>-- <br><div class="m_4509115223586986053gmail_signature"><br>Christopher Barker, Ph.D.<br>Oceanographer<br><br>Emergency Response Division<br>NOAA/NOS/OR&R <a href="tel:(206)%20526-6959" value="+12065266959" target="_blank">(206) 526-6959</a> voice<br>7600 Sand Point Way NE <a href="tel:(206)%20526-6329" value="+12065266329" target="_blank">(206) 526-6329</a> fax<br>Seattle, WA 98115 <a href="tel:(206)%20526-6317" value="+12065266317" target="_blank">(206) 526-6317</a> main reception<br><br><a href="mailto:Chris.Barker@noaa.gov" target="_blank">Chris.Barker@noaa.gov</a></div>
</div></div>