<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Apr 26, 2014 at 10:20 AM, <span dir="ltr"><<a href="mailto:josef.pktd@gmail.com" target="_blank">josef.pktd@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">
<div><div class="h5">On Sat, Apr 26, 2014 at 10:10 AM, <span dir="ltr"><<a href="mailto:josef.pktd@gmail.com" target="_blank">josef.pktd@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">
<div><div>On Fri, Apr 25, 2014 at 1:21 AM, Matthew Brett <span dir="ltr"><<a href="mailto:matthew.brett@gmail.com" target="_blank">matthew.brett@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hi,<br>
<div><div><br>
On Thu, Apr 24, 2014 at 5:26 PM, <<a href="mailto:josef.pktd@gmail.com" target="_blank">josef.pktd@gmail.com</a>> wrote:<br>
><br>
><br>
><br>
> On Thu, Apr 24, 2014 at 7:29 PM, <<a href="mailto:josef.pktd@gmail.com" target="_blank">josef.pktd@gmail.com</a>> wrote:<br>
>><br>
>><br>
>><br>
>><br>
>> On Thu, Apr 24, 2014 at 7:20 PM, Charles R Harris<br>
>> <<a href="mailto:charlesr.harris@gmail.com" target="_blank">charlesr.harris@gmail.com</a>> wrote:<br>
>>><br>
>>><br>
>>><br>
>>><br>
>>> On Thu, Apr 24, 2014 at 5:08 PM, <<a href="mailto:josef.pktd@gmail.com" target="_blank">josef.pktd@gmail.com</a>> wrote:<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> On Thu, Apr 24, 2014 at 7:00 PM, Charles R Harris<br>
>>>> <<a href="mailto:charlesr.harris@gmail.com" target="_blank">charlesr.harris@gmail.com</a>> wrote:<br>
>>>>><br>
>>>>><br>
>>>>> Hi Matthew,<br>
>>>>><br>
>>>>> On Thu, Apr 24, 2014 at 3:56 PM, Matthew Brett<br>
>>>>> <<a href="mailto:matthew.brett@gmail.com" target="_blank">matthew.brett@gmail.com</a>> wrote:<br>
>>>>>><br>
>>>>>> Hi,<br>
>>>>>><br>
>>>>>> Thanks to Cark Kleffner's toolchain and some help from Clint Whaley<br>
>>>>>> (main author of ATLAS), I've built 64-bit windows numpy and scipy<br>
>>>>>> wheels for testing.<br>
>>>>>><br>
>>>>>> The build uses Carl's custom mingw-w64 build with static linking.<br>
>>>>>><br>
>>>>>> There are two harmless test failures on scipy (being discussed on the<br>
>>>>>> list at the moment) - tests otherwise clean.<br>
>>>>>><br>
>>>>>> Wheels are here:<br>
>>>>>><br>
>>>>>><br>
>>>>>> <a href="https://nipy.bic.berkeley.edu/scipy_installers/numpy-1.8.1-cp27-none-win_amd64.whl" target="_blank">https://nipy.bic.berkeley.edu/scipy_installers/numpy-1.8.1-cp27-none-win_amd64.whl</a><br>
>>>>>><br>
>>>>>> <a href="https://nipy.bic.berkeley.edu/scipy_installers/scipy-0.13.3-cp27-none-win_amd64.whl" target="_blank">https://nipy.bic.berkeley.edu/scipy_installers/scipy-0.13.3-cp27-none-win_amd64.whl</a><br>
>>>>>><br>
>>>>>> You can test with:<br>
>>>>>><br>
>>>>>> pip install -U pip # to upgrade pip to latest<br>
>>>>>> pip install -f <a href="https://nipy.bic.berkeley.edu/scipy_installers" target="_blank">https://nipy.bic.berkeley.edu/scipy_installers</a> numpy<br>
>>>>>> scipy<br>
>>>>>><br>
>>>>>> Please do send feedback.<br>
>>>>>><br>
>>>>>> ATLAS binary here:<br>
>>>>>><br>
>>>>>><br>
>>>>>> <a href="https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds/atlas-64-full-sse2.tar.bz2" target="_blank">https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds/atlas-64-full-sse2.tar.bz2</a><br>
>>>>>><br>
>>>>>> Many thanks for Carl in particular for doing all the hard work,<br>
>>>>>><br>
>>>>><br>
>>>>> Cool. After all these long years... Now all we need is a box running<br>
>>>>> tests for CI.<br>
>>>>><br>
>>>>> Chuck<br>
>>>>><br>
>>>>> _______________________________________________<br>
>>>>> NumPy-Discussion mailing list<br>
>>>>> <a href="mailto:NumPy-Discussion@scipy.org" target="_blank">NumPy-Discussion@scipy.org</a><br>
>>>>> <a href="http://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">http://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
>>>>><br>
>>>><br>
>>>> I get two test failures with numpy<br>
>>>><br>
>>>> Josef<br>
>>>><br>
>>>> >>> np.test()<br>
>>>> Running unit tests for numpy<br>
>>>> NumPy version 1.8.1<br>
>>>> NumPy is installed in C:\Python27\lib\site-packages\numpy<br>
>>>> Python version 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit<br>
>>>> (AMD64)]<br>
>>>> nose version 1.1.2<br>
>>>><br>
>>>> ======================================================================<br>
>>>> FAIL: test_iterator.test_iter_broadcasting_errors<br>
>>>> ----------------------------------------------------------------------<br>
>>>> Traceback (most recent call last):<br>
>>>> File "C:\Python27\lib\site-packages\nose\case.py", line 197, in<br>
>>>> runTest<br>
>>>> self.test(*self.arg)<br>
>>>> File<br>
>>>> "C:\Python27\lib\site-packages\numpy\core\tests\test_iterator.py", line 657,<br>
>>>> in test_iter_broadcasting_errors<br>
>>>> '(2)->(2,newaxis)') % msg)<br>
>>>> File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 44,<br>
>>>> in assert_<br>
>>>> raise AssertionError(msg)<br>
>>>> AssertionError: Message "operands could not be broadcast together with<br>
>>>> remapped shapes [original->remapped]: (2,3)->(2,3) (2,)->(2,newaxis) and<br>
>>>> requested shape (4,3)" doesn't contain remapped operand<br>
>>>> shape(2)->(2,newaxis)<br>
>>>><br>
>>>> ======================================================================<br>
>>>> FAIL: test_iterator.test_iter_array_cast<br>
>>>> ----------------------------------------------------------------------<br>
>>>> Traceback (most recent call last):<br>
>>>> File "C:\Python27\lib\site-packages\nose\case.py", line 197, in<br>
>>>> runTest<br>
>>>> self.test(*self.arg)<br>
>>>> File<br>
>>>> "C:\Python27\lib\site-packages\numpy\core\tests\test_iterator.py", line 836,<br>
>>>> in test_iter_array_cast<br>
>>>> assert_equal(i.operands[0].strides, (-96,8,-32))<br>
>>>> File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 255,<br>
>>>> in assert_equal<br>
>>>> assert_equal(actual[k], desired[k], 'item=%r\n%s' % (k, err_msg),<br>
>>>> verbose)<br>
>>>> File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 317,<br>
>>>> in assert_equal<br>
>>>> raise AssertionError(msg)<br>
>>>> AssertionError:<br>
>>>> Items are not equal:<br>
>>>> item=0<br>
>>>><br>
>>>> ACTUAL: 96L<br>
>>>> DESIRED: -96<br>
>>>><br>
>>>> ----------------------------------------------------------------------<br>
>>>> Ran 4828 tests in 46.306s<br>
>>>><br>
>>>> FAILED (KNOWNFAIL=10, SKIP=8, failures=2)<br>
>>>> <nose.result.TextTestResult run=4828 errors=0 failures=2><br>
>>>><br>
>>><br>
>>> Strange. That second one looks familiar, at least the "-96" part. Wonder<br>
>>> why this doesn't show up with the MKL builds.<br>
>><br>
>><br>
>> ok tried again, this time deleting the old numpy directories before<br>
>> installing<br>
>><br>
>> Ran 4760 tests in 42.124s<br>
>><br>
>> OK (KNOWNFAIL=10, SKIP=8)<br>
>> <nose.result.TextTestResult run=4760 errors=0 failures=0><br>
>><br>
>><br>
>> so pip also seems to be reusing leftover files.<br>
>><br>
>> all clear.<br>
><br>
><br>
> Running the statsmodels test suite, I get a failure in<br>
> test_discrete.TestProbitCG where fmin_cg converges to something that differs<br>
> in the 3rd decimal.<br>
><br>
> I usually only test the 32-bit version, so I don't know if this is specific<br>
> to this scipy version, but we haven't seen this in a long time.<br>
> I used our nightly binaries <a href="http://statsmodels.sourceforge.net/binaries/" target="_blank">http://statsmodels.sourceforge.net/binaries/</a><br>
<br>
</div></div>That's interesting, you saw also we're getting failures on the tests<br>
for powell optimization because of small unit-at-last-place<br>
differences in the exp function in mingw-w64. Is there any chance you<br>
can track down where the optimization path is diverging and why?<br>
It's just that - if this is also the exp function maybe we can see if<br>
the error is exceeding reasonable bounds and then feed back to<br>
mingw-w64 and fall back to the numpy default implementation in the<br>
meantime.<br></blockquote><div><br></div></div></div><div>I'm a bit doubtful it's exp, the probit model is based on the normal distribution and has an exp only in the gradient via norm._pdf, the objective function uses norm._cdf.</div>
<div><br></div><div>I can look into it.</div></div></div></div></blockquote></div></div></div></div></div></blockquote><div><br></div><div>with 32 bit official binaries MingW 32</div><div><br></div><div><div>Warning: Desired error not necessarily achieved due to precision loss.</div>
<div> Current function value: 0.400588</div><div> Iterations: 75</div><div> Function evaluations: 213</div><div> Gradient evaluations: 201</div></div><div><br></div><div>relative and absolute deviation from "desired"</div>
<div><div>[ -1.26257296e-05 -4.77535711e-05 -9.93794940e-06 -1.78815725e-05]</div><div>[ -2.05270407e-05 -2.47024202e-06 -1.41748189e-05 1.33259208e-04]</div></div><div><br></div><div><br></div><div>with your wheels, after increasing maxiter in the test case</div>
<div><br></div><div><div>Optimization terminated successfully.</div><div> Current function value: 0.400588</div><div> Iterations: 766</div><div> Function evaluations: 1591</div><div> Gradient evaluations: 1591</div>
</div><div><br></div><div>relative and absolute deviation from "desired"<br></div><div><div>[ -1.57311713e-07 -4.25324806e-08 -3.01557919e-08 -1.19794357e-07]</div><div>[ -2.55758996e-07 -2.20016050e-09 -4.30121820e-08 8.92745931e-07]</div>
</div><div><br></div><div>So actually the 64 bit wheel has the better final result, and just needs more iterations to get close enough to what we had required in the unit tests.</div><div><br></div><div>The trace of the 64bit version seems to slow down in the movement, but then doesn't run into the "precision loss"</div>
<div>From visual comparison, after the 20 iteration the parameters start to slowly diverge in the 5th decimal.</div><div><br></div><div>attached is a script that replicates the testcase </div><div><br></div><div>Thanks</div>
<div><br></div><div>Josef</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>However: </div><div>We don't use fmin_cg for anything by default, it's part of "testing all supported scipy optimizers" and we had problems with it before on various machines <a href="https://github.com/statsmodels/statsmodels/issues/109" target="_blank">https://github.com/statsmodels/statsmodels/issues/109</a></div>
<div>The test was completely disabled on Windows for a while, and I might have to turn some screws again.</div><div><br></div><div>I'm fighting with more serious problems with fmin_slsqp and fmin_bfgs, which we really need to use.</div>
<div><br></div><div>If minor precision issues matter, then the code is not "robust" and should be fixed.</div><div><br></div><div>compared to precision issues. I'm fighting more with the large scale properties of exp.</div>
<div><a href="https://github.com/scipy/scipy/issues/3581" target="_blank">https://github.com/scipy/scipy/issues/3581</a><br></div><div><br></div><div><br></div><div>Neverthless,</div><div>I would really like to know why I'm running into many platform differences and problems with scipy.optimize.</div>
</div></div></div></blockquote><div><br></div></div></div><div>To avoid giving a wrong impression:</div><div><br></div><div>Scipy.optimize works in general very well for statsmodels, we use it heavily and we have a large set of test cases for it.</div>
<div>It's just the last 5% or so of cases where I spend a considerable amount of time figuring out how to get around convergence problems, which are sometimes platform specific and sometimes not.</div><span class=""><font color="#888888"><div>
<br></div><div>
Josef</div></font></span><div class=""><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div><br></div><div>Cheers,</div><div><br></div><div>Josef</div><div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Cheers,<br>
<br>
Matthew<br>
<div><div>_______________________________________________<br>
NumPy-Discussion mailing list<br>
<a href="mailto:NumPy-Discussion@scipy.org" target="_blank">NumPy-Discussion@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">http://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
</div></div></blockquote></div></div><br></div></div>
</blockquote></div></div><br></div></div>
</blockquote></div><br></div></div>