[Numpy-discussion] About ready to start 1.8 release process.

Wed Aug 14 13:35:47 EDT 2013

On Wed, Aug 14, 2013 at 9:00 AM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> On Tue, Aug 13, 2013 at 5:54 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> > I wish it were. It seems unreasonably difficult to get constructive
> > feedback. Chris is pretty much the only one making the attempt and that
> > discussion petered out.
>
> well, Nathaniel Smith chimed in, and Mark Weibe commented a bit (as he
> wrote the code in the first place). Wes McKinney also chimed in,
> essentially saying that Pandas mostly has to do its own thing, and I'm
> pretty sure would prefer that the current TZ converting was ripped out
> as I suggest.
>
> In the thread on the mialing list I started:
>
>
> http://numpy-discussion.10968.n7.nabble.com/timezones-and-datetime64-tt33407.html#none
>
> A bunch of core people chimed in. I just re-read that thread and
> provide a summary below:
>
>  - No one said they liked it as it is, and there were a number of +1
> (and even a +10) comment to cleaning up the current conversion issues.
>
> Some folks would like "real" improvements:
>   - proper, full time zone handling
>   - proper (or maybe just better?) handling of leap-seconds, and all that.
>
>  - Marc Weibe had a suggestion or two that were "not a trivial
> addition" -- so off the table for this release, and would need some
> real discussion, a NEP, and developer time...
>
> - Marc also pointed out that handling the Locale time zone is helpful
> for the interactive use case, which is a major numpy use-case.
> However, doing it half-way isn't really helpful anyway.
>
> - Marc also commented that making datetime64 time-zone naive would be
> "the easy way"
>

I've experimented a little bit with one possibility in this direction
within dynd. Basically, adding a time zone to the metadata which initially
only can be UTC or abstract. (I'm calling it "abstract" because I don't
think "naive" communicates the right idea about it.) I'm -1 on simply
removing a distinction between UTC and abstract ISO 8601 strings, I think
equivocating the two would be as bad as the current situation. Adding a
limited timezone with just these two options might be a way to improve the
situation without too much development effort, while maintaining a clear
path to proper timezone support.

In this formulation, strings without a timezone are parsed as abstract
datetimes, and strings with a timezone as UTC datetimes, without allowing
implicit conversions:

>>> from dynd import nd, ndt
>>> nd.array('2011-03-22T12:00', dtype='datetime("min")')
nd.array(2011-03-22T12:00, datetime<unit=minute,tz=abstract>)
>>> nd.array('2011-03-22T12:00Z', dtype='datetime("min", "UTC")')
nd.array(2011-03-22T12:00Z, datetime<unit=minute,tz=utc>)
>>> nd.array('2011-03-22T12:00Z', dtype='datetime("min")')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "_pydynd.pyx", line 892, in _pydynd.w_array.__cinit__
(_pydynd.cxx:5774)
RuntimeError: cannot parse "2011-03-22T12:00Z" as an abstract datetime
using rule "strict", because a timezone was present in the string
>>> nd.array('2011-03-22T12:00', dtype='datetime("min", "UTC")')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "_pydynd.pyx", line 892, in _pydynd.w_array.__cinit__
(_pydynd.cxx:5774)
RuntimeError: cannot parse "2011-03-22T12:00" as a datetime with timezone
using
rule "strict", because no timezone was present in the string

I've also implemented one way of getting a more relaxed parsing mode, with
the parameter that controls casting strictness. Note that in this case it
is throwing away the timezone information, so the "-0400" doesn't get
incorporated in the third value.

>>> nd.array(['2011-03-22T12:00', '2012-04-23T01:00Z',
'2012-05-01T13:00-0400'])
nd.array([2011-03-22T12:00, 2012-04-23T01:00, 2012-05-01T13:00],
strided_dim<convert<to=datetime<unit=minute,tz=abstract>, from=string,
errmode=none>>)

This stuff is all experimental/work in progress, but if you want to try
these examples you can install Anaconda and use its conda package manager
to update the package dynd-python. (
https://github.com/ContinuumIO/dynd-python/blob/master/README.md#trying-out-dynd
).

Thanks,
Mark

> - There was also discussion about having a settable epoch -- I think

that would be really useful, but not for this release, and would

require a NEP, etc...

> - From Travis O.:"It seems to me that the biggest issue is just the

automatic conversion that is occurring on string or date-time input.

We should stop using the local time-zone (explicit is better than

implicit strikes again) and not use any time-zone unless time-zone

information is provided in the string. I am definitely +1 on that."

> My conclusions:

> The current code is not really usable for ANY use without careful

checking of inputs and/or work-arounds.

> No one involved in the numpy list would mind removing the time zone

translation on I/O. A number of people would find it a real

improvement. (granted most users don't get involved with these

discussion on the list....)

> There are some great ideas for how to improve it that would require

NEPs, discussion and real developer time -- that's not happening any

time soon.

> Ripping out the translation on I/O would be fairly easy (at least for

someone familiar with the code)

> So I say let's do it!

> But who? Travis offered up himself and Mark as possibilities, but who

knows what their schedules allow at this point -- are either of you

following this thread?

> It's probably not too big a deal for anyone familiar with the numpy

code base -- if no one else steps up, I can give it a try, though I'm

sure I wouldn't have even found what code to edit in the time it would

take Mark to fix it.

> -Chris
>

> --
>

> Christopher Barker, Ph.D.
>
Oceanographer
>

> Emergency Response Division
>
NOAA/NOS/OR&R            (206) 526-6959   voice
>
7600 Sand Point Way NE   (206) 526-6329   fax
>
Seattle, WA  98115       (206) 526-6317   main reception
>

> Chris.Barker at noaa.gov
>
_______________________________________________
>
NumPy-Discussion mailing list
>
NumPy-Discussion at scipy.org
>
http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130814/f95d55e4/attachment.html>