py3 datetime woes (was Re: Code Freeze for NumPy 1.7)
On Sun, Jul 15, 2012 at 5:32 PM, Ralf Gommers
On Sun, Jul 15, 2012 at 5:57 PM, Nathaniel Smith
wrote: On Sun, Jul 15, 2012 at 1:08 PM, Ralf Gommers
wrote: On Sun, Jul 15, 2012 at 12:45 AM, Travis Oliphant
wrote: Hey all,
We are nearing a code-freeze for NumPy 1.7. Are there any last-minute changes people are wanting to push into NumPy 1.7? We should discuss them as soon as possible.
I'm proposing a code-freeze at midnight UTC on July 18th (7:00pm CDT on July 17th). This will allow the creation of beta releases of NumPy on the 18th of July. This is a few days later than originally hoped for --- largely due to unexpected travel schedules of Ondrej and I, but it does give people a few more days to get patches in. Of course, we will be able to apply bug-fixes to the 1.7.x branch once the tag is made.
What about the tickets still open for 1.7.0 (http://projects.scipy.org/numpy/report/3)? There are a few important ones left.
These I would consider blockers: - #2108 Datetime failures with MinGW
Is there a description anywhere of what the problem actually is here? I looked at the ticket, which referred to a PR, and it's hard to work out from the PR discussion what the actual remaining test failures are -- and there definitely doesn't seem to be any description of the underlying problem. (Something about working 64-bit time_t on windows being difficult depending on the compiler used?)
There's a lot more discussion on http://projects.scipy.org/numpy/ticket/1909 https://github.com/numpy/numpy/pull/156 https://github.com/numpy/numpy/pull/161.
The issue is that for MinGW 3.x some _s / _t functions seem to be missing. And we don't yet support MinGW 4.x.
Current issues can be seen from the last test log on our Windows XP buildbot (June 29, http://buildbot.scipy.org/builders/Windows_XP_x86/builds/1124/steps/shell_1/...):
====================================================================== ERROR: test_datetime_arange (test_datetime.TestDateTime) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\numpy\b11\numpy-install\Lib\site-packages\numpy\core\tests\test_datetime.py", line 1351, in test_datetime_arange assert_raises(ValueError, np.arange, np.datetime64('today'), OSError: Failed to use '_localtime64_s' to convert to a local time
...I don't understand how this is even building if the functions are missing. Well, anyway. MSVC provides both _s and regular versions of localtime and friends. Mingw only provides the regular version. It looks like there are two difference between the regular and _s versions of these functions: 1) The _s version reports errors by calling the "invalid parameter handler" instead of just returning an error code. (I.e., by default, if you try to work with a date that's out of range, then your program will just abort(). Very friendly.)[1] 2) MSVC whines if you use the regular version Basically AFAICT this is an interface designed to make it easier for lousy programmers to write broken code that will still limp along more successfully than it otherwise would. There's a legitimate role for such interfaces, but I'm not sure numpy is it -- I think we can write actual error handling code? The classic localtime() interface AFAICT works perfectly -- it's even threadsafe, unlike most unix versions [2]. I think the best solution here is to just switch to using localtime() everywhere, and tell MSVC to shut up by using a #pragma. Here's an example of how to do this taken from the Boost developer guidelines [3]: #if defined(_MSC_VER) #pragma warning(push) // Save warning settings. #pragma warning(disable : 4996) // Disable deprecated localtime/gmtime warning. #endif ... #if defined(_MSC_VER) #pragma warning(pop) // Restore warnings to previous state. #endif (Yes, Boost's canonical example of how you control warnings on MSVC is specifically for letting you use localtime() in piece -- I didn't add that. Of course this could be hidden in a macro or helper function... npy_localtime() or something.) Or alternatively one can just #define _CRT_SECURE_NO_DEPRECATE to kill this warning in general. The Boost developer's general advice on this particular warning is "Unless you strongly believe that the 'secure' versions are useful, suppress."[3] [1] http://msdn.microsoft.com/en-us/library/a442x3ye%28v=vs.80%29.aspx , and note the "community content" at the bottom. [2] http://old.nabble.com/Re%3A-Need-localtime_s-p13088250.html [3] https://svn.boost.org/trac/boost/wiki/Guidelines/WarningsGuidelines -n
On Mon, Jul 16, 2012 at 3:27 PM, Nathaniel Smith
On Sun, Jul 15, 2012 at 5:32 PM, Ralf Gommers
wrote: Current issues can be seen from the last test log on our Windows XP buildbot (June 29,
http://buildbot.scipy.org/builders/Windows_XP_x86/builds/1124/steps/shell_1/... ):
====================================================================== ERROR: test_datetime_arange (test_datetime.TestDateTime) ---------------------------------------------------------------------- Traceback (most recent call last): File
"C:\buildbot\numpy\b11\numpy-install\Lib\site-packages\numpy\core\tests\test_datetime.py",
line 1351, in test_datetime_arange assert_raises(ValueError, np.arange, np.datetime64('today'), OSError: Failed to use '_localtime64_s' to convert to a local time
...I don't understand how this is even building if the functions are missing.
Because of the build magic added in https://github.com/numpy/numpy/pull/156 Ralf
participants (2)
-
Nathaniel Smith
-
Ralf Gommers