<div dir="ltr">On Wed, Jan 15, 2014 at 9:57 AM, Charles R Harris <span dir="ltr"><<a href="mailto:charlesr.harris@gmail.com" target="_blank">charlesr.harris@gmail.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">There was a discussion of this long ago and UCS-4 was chosen as the numpy standard. There are just too many complications that arise in supporting both.</div>
</blockquote><div><br></div><div style>fair enough -- but loadtxt appears to be broken just the same. Any proposals for that?</div><div style><br></div><div style>My proposal:</div><div style><br></div><div style>loadtxt accepts an encoding argument.</div>
<div style><br></div><div style>default is ascii -- that's what it's doing now, anyway, yes?</div><div style><br></div><div style>If the file is encoded ascii, then a one-byte-per character dtype is used for text data, unless the user specifies otherwise (do they need to specify anyway?)</div>
<div style><br></div><div style>If the file has another encoding, the the default dtype for text is unicode.</div><div style><br></div><div style>Not sure about other one-byte per character encodings (e.g. latin-1)</div>
<div style>
<br></div><div style>The defaults may be moot, if the loadtxt doesn't have auto-detection of text in a filie anyway.</div><div style><br></div><div style>This all required that there be an obvious way for the user to spell the one-byte-per character dtype -- I think 'S' will do it.</div>
<div style><br></div><div style>Note to OP: what happens if you specify 'S' for your dtype, rather than str - it works for me on py2:</div><div style><br></div><div style><div>In [16]: np.loadtxt('pathlist.txt', dtype='S')</div>
<div>Out[16]: </div><div>array(['C:\\Users\\Documents\\Project\\mytextfile1.txt',</div><div> 'C:\\Users\\Documents\\Project\\mytextfile2.txt',</div><div> 'C:\\Users\\Documents\\Project\\mytextfile3.txt'], </div>
<div> dtype='|S42')</div><div><br></div><div style>Note: this leaves us with what to pass back to the user when they index into an array of type 'S*' -- a bytes object or a unicode object (decoded as ascii). I think a unicode object, in keeping with proper py3 behavior. This would be like we currently do with, say floating point numbers:</div>
<div style><br></div><div style>We can store/operate with 32 bit floats, but when you pass it back as a python type, you get the native python float -- 64bit.</div><div style><br></div><div style>NOTE: another option is to use latin-1 all around, rather than ascii -- you may get garbage from the higher value bytes, but it won't barf on you.</div>
<div style><br></div><div style>-Chris</div><div style><br></div><div style><br></div><div style><br></div><div style><br></div><div><br></div></div><div style><br></div><div style><br></div><div style><br></div><div style>
<br></div><div style><br></div><div style><br></div><div style><br></div><div style><br></div><div style> </div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>Chuck <br>
</div><br></div></div></div>
<br>_______________________________________________<br>
NumPy-Discussion mailing list<br>
<a href="mailto:NumPy-Discussion@scipy.org">NumPy-Discussion@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">http://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><br>Christopher Barker, Ph.D.<br>Oceanographer<br><br>Emergency Response Division<br>NOAA/NOS/OR&R (206) 526-6959 voice<br>7600 Sand Point Way NE (206) 526-6329 fax<br>
Seattle, WA 98115 (206) 526-6317 main reception<br><br><a href="mailto:Chris.Barker@noaa.gov" target="_blank">Chris.Barker@noaa.gov</a>
</div></div>