<div dir="ltr">On Wed, Aug 14, 2013 at 9:00 AM, Chris Barker - NOAA Federal <span dir="ltr"><<a href="mailto:chris.barker@noaa.gov" target="_blank">chris.barker@noaa.gov</a>></span> wrote:<br><div class="gmail_extra">
<div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="im">On Tue, Aug 13, 2013 at 5:54 PM, Charles R Harris<br>
<<a href="mailto:charlesr.harris@gmail.com">charlesr.harris@gmail.com</a>> wrote:<br>
> I wish it were. It seems unreasonably difficult to get constructive<br>
> feedback. Chris is pretty much the only one making the attempt and that<br>
> discussion petered out.<br>
<br>
</div>well, Nathaniel Smith chimed in, and Mark Weibe commented a bit (as he<br>
wrote the code in the first place). Wes McKinney also chimed in,<br>
essentially saying that Pandas mostly has to do its own thing, and I'm<br>
pretty sure would prefer that the current TZ converting was ripped out<br>
as I suggest.<br>
<br>
In the thread on the mialing list I started:<br>
<br>
<a href="http://numpy-discussion.10968.n7.nabble.com/timezones-and-datetime64-tt33407.html#none" target="_blank">http://numpy-discussion.10968.n7.nabble.com/timezones-and-datetime64-tt33407.html#none</a><br>
<br>
A bunch of core people chimed in. I just re-read that thread and<br>
provide a summary below:<br>
<br>
- No one said they liked it as it is, and there were a number of +1<br>
(and even a +10) comment to cleaning up the current conversion issues.<br>
<br>
Some folks would like "real" improvements:<br>
- proper, full time zone handling<br>
- proper (or maybe just better?) handling of leap-seconds, and all that.<br>
<br>
- Marc Weibe had a suggestion or two that were "not a trivial<br>
addition" -- so off the table for this release, and would need some<br>
real discussion, a NEP, and developer time...<br>
<br>
- Marc also pointed out that handling the Locale time zone is helpful<br>
for the interactive use case, which is a major numpy use-case.<br>
However, doing it half-way isn't really helpful anyway.<br>
<br>
- Marc also commented that making datetime64 time-zone naive would be<br>
"the easy way"<br></blockquote><div><br></div><div>I've experimented a little bit with one possibility in this direction within dynd. Basically, adding a time zone to the metadata which initially only can be UTC or abstract. (I'm calling it "abstract" because I don't think "naive" communicates the right idea about it.) I'm -1 on simply removing a distinction between UTC and abstract ISO 8601 strings, I think equivocating the two would be as bad as the current situation. Adding a limited timezone with just these two options might be a way to improve the situation without too much development effort, while maintaining a clear path to proper timezone support.</div>
<div><br></div><div>In this formulation, strings without a timezone are parsed as abstract datetimes, and strings with a timezone as UTC datetimes, without allowing implicit conversions:</div><div><br></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div class="gmail_extra"><div class="gmail_quote"><div><div>>>> from dynd import nd, ndt</div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div>>>> nd.array('2011-03-22T12:00', dtype='datetime("min")')</div>
</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div>nd.array(2011-03-22T12:00, datetime<unit=minute,tz=abstract>)</div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div>
<div>>>> nd.array('2011-03-22T12:00Z', dtype='datetime("min", "UTC")')</div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div>nd.array(2011-03-22T12:00Z, datetime<unit=minute,tz=utc>)</div>
</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div>>>> nd.array('2011-03-22T12:00Z', dtype='datetime("min")')</div></div></div></div><div class="gmail_extra"><div class="gmail_quote">
<div><div>Traceback (most recent call last):</div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div> File "<stdin>", line 1, in <module></div></div></div></div><div class="gmail_extra">
<div class="gmail_quote"><div><div> File "_pydynd.pyx", line 892, in _pydynd.w_array.__cinit__ (_pydynd.cxx:5774)</div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div>RuntimeError: cannot parse "2011-03-22T12:00Z" as an abstract datetime using rule "strict", because a timezone was present in the string</div>
</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div>>>> nd.array('2011-03-22T12:00', dtype='datetime("min", "UTC")')</div></div></div></div><div class="gmail_extra">
<div class="gmail_quote"><div><div>Traceback (most recent call last):</div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div> File "<stdin>", line 1, in <module></div></div>
</div></div><div class="gmail_extra"><div class="gmail_quote"><div><div> File "_pydynd.pyx", line 892, in _pydynd.w_array.__cinit__ (_pydynd.cxx:5774)</div></div></div></div><div class="gmail_extra"><div class="gmail_quote">
<div><div>RuntimeError: cannot parse "2011-03-22T12:00" as a datetime with timezone using</div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div>rule "strict", because no timezone was present in the string</div>
</div></div></div></blockquote><div><br></div>I've also implemented one way of getting a more relaxed parsing mode, with the parameter that controls casting strictness. Note that in this case it is throwing away the timezone information, so the "-0400" doesn't get incorporated in the third value.<div>
<br></div><div>>>> nd.array(['2011-03-22T12:00', '2012-04-23T01:00Z', '2012-05-01T13:00-0400'])</div><div>nd.array([2011-03-22T12:00, 2012-04-23T01:00, 2012-05-01T13:00], strided_dim<convert<to=datetime<unit=minute,tz=abstract>, from=string, errmode=none>>)<br>
<div><br></div><div>This stuff is all experimental/work in progress, but if you want to try these examples you can install Anaconda and use its conda package manager to update the package dynd-python. (<a href="https://github.com/ContinuumIO/dynd-python/blob/master/README.md#trying-out-dynd">https://github.com/ContinuumIO/dynd-python/blob/master/README.md#trying-out-dynd</a>).</div>
<div><br></div><div>Thanks,</div><div>Mark<br><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div> </div></div></div><div class="gmail_extra"><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
- There was also discussion about having a settable epoch -- I think</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
that would be really useful, but not for this release, and would</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
require a NEP, etc...</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
- From Travis O.:"It seems to me that the biggest issue is just the</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
automatic conversion that is occurring on string or date-time input.</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
We should stop using the local time-zone (explicit is better than</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
implicit strikes again) and not use any time-zone unless time-zone</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
information is provided in the string. I am definitely +1 on that."</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
My conclusions:</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
The current code is not really usable for ANY use without careful</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
checking of inputs and/or work-arounds.</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
No one involved in the numpy list would mind removing the time zone</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
translation on I/O. A number of people would find it a real</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
improvement. (granted most users don't get involved with these</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
discussion on the list....)</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
There are some great ideas for how to improve it that would require</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
NEPs, discussion and real developer time -- that's not happening any</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
time soon.</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Ripping out the translation on I/O would be fairly easy (at least for</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
someone familiar with the code)</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
So I say let's do it!</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
But who? Travis offered up himself and Mark as possibilities, but who</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
knows what their schedules allow at this point -- are either of you</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
following this thread?</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
It's probably not too big a deal for anyone familiar with the numpy</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
code base -- if no one else steps up, I can give it a try, though I'm</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
sure I wouldn't have even found what code to edit in the time it would</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
take Mark to fix it.</blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><br></div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im">
-Chris</div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><br></div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im">
--</div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><br></div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im">
Christopher Barker, Ph.D.</div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im">
Oceanographer</div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><br></div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im">
Emergency Response Division</div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im">
NOAA/NOS/OR&R <a href="tel:%28206%29%20526-6959" value="+12065266959">(206) 526-6959</a> voice</div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im">
7600 Sand Point Way NE <a href="tel:%28206%29%20526-6329" value="+12065266329">(206) 526-6329</a> fax</div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im">
Seattle, WA 98115 <a href="tel:%28206%29%20526-6317" value="+12065266317">(206) 526-6317</a> main reception</div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><br></div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><a href="mailto:Chris.Barker@noaa.gov">Chris.Barker@noaa.gov</a></div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class=""><div class="h5">_______________________________________________</div></div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class=""><div class="h5">
NumPy-Discussion mailing list</div></div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class=""><div class="h5"><a href="mailto:NumPy-Discussion@scipy.org">NumPy-Discussion@scipy.org</a></div></div></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class=""><div class="h5"><a href="http://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">http://mail.scipy.org/mailman/listinfo/numpy-discussion</a></div></div></blockquote></div></div></blockquote>
<div class="gmail_extra"><br></div></div></div></div>