[Numpy-discussion] Scipy 0.15.0 beta 1 release

Charles R Harris charlesr.harris at gmail.com
Wed Nov 26 09:14:04 EST 2014


On Wed, Nov 26, 2014 at 6:59 AM, Julian Taylor <
jtaylor.debian at googlemail.com> wrote:

> On 11/26/2014 02:19 PM, Charles R Harris wrote:
> >
> >
> > On Wed, Nov 26, 2014 at 2:06 AM, Julian Taylor
> > <jtaylor.debian at googlemail.com <mailto:jtaylor.debian at googlemail.com>>
> > wrote:
> >
> >     On 11/26/2014 12:50 AM, Andrea Gavana wrote:
> >     >
> >     > On 25 November 2014 at 19:33, David Cournapeau <cournape at gmail.com
> <mailto:cournape at gmail.com>
> >     > <mailto:cournape at gmail.com <mailto:cournape at gmail.com>>> wrote:
> >     >
> >     >
> >     >
> >     >     On Tue, Nov 25, 2014 at 6:10 PM, Sturla Molden
> >     >     <sturla.molden at gmail.com <mailto:sturla.molden at gmail.com>
> >     <mailto:sturla.molden at gmail.com <mailto:sturla.molden at gmail.com>>>
> >     wrote:
> >     >
> >     >         David Cournapeau <cournape at gmail.com <mailto:
> cournape at gmail.com>
> >     >         <mailto:cournape at gmail.com <mailto:cournape at gmail.com>>>
> wrote:
> >     >         > Shall we consider <a
> >     >         > href="https://github.com/scipy/scipy/issues/4168">
> https://github.com/scipy/scipy/issues/4168</a>
> >     >         > to be a
> >     >         > blocker (the issue arises on scipy master as well as
> 0.14.1) ?
> >     >         >
> >     >
> >     >         It is really bad, but does anyone know what is really
> going on?
> >     >
> >     >
> >     >     Yes, it is in the bug report.
> >     >
> >     >
> >     >
> >     > Nice move.
> >     >
> >     > I've now recommended to hold back any
> upgrade/update/pip-crap/enpkg-fun
> >     > thing on NumPy/SciPy across the whole user base of Python in the
> >     > company. We will probably move to 64bit-in-any-sense soon enough, I
> >     > guess before this issue is solved. Tell me, NumPy, was the array
> aligned
> >     > enough in 1.8? Is NumPy stricter in its checking because of SPARC?
> SPARC?!?
> >
> >     yes, before the change numpy accepted a factor 10 performance
> regression
> >     in complex indexing to satisfy sparc.
> >
> >     >
> >     > Dozens of f2py compiled extensions are going to fail soon here -
> which
> >     > I'll reluctantly check tomorrow. I don't want to think about other
> >     > people on Win32 facing the same issue elsewhere... :-)
> >     >
> >
> >     There haven't been any real complaints from applications yet, only
> >     testsuite failure of scipy.
> >     Eithe the one thing that is broken in scipy isn't much used or
> windows
> >     32 users aren't using 1.9 yet.
> >     The majority of f2py should still be working, numpys own f2py
> testsuite
> >     passes on win32. I still don't know what exactly arpack is doing
> >     different but I also did not have time yet to look at the testcase
> david
> >     created.
> >
> >     It should of course be fixed, I have an old branch with aligned
> >     allocators lying around somewhere which should fix the issue or at
> least
> >     give users a way to work around it.
> >
> >
> > The lesson I take from this is that we need a test ;) Also, that it
> > would be nice if we got more testing by users before releases. But
> > things are as they are, this will get fixed, and we will move on with
> > one more lesson learned.
> >
> > If anyone is to blame, it is the reviewer, namely, myself.
> >
>
>
> concerning actually fixing it I guess we have 3 options:
>
> 1. reduce maximum copy alignment from currently 16 to 8 on platforms
> that need it.
> That will automatically reduce the needed alignment of complex on win32
> to 8 bytes. Do other compilers provide something similar to gccs
> __BIGGEST_ALIGNMENT__?
> 2. remove bloating of complex alignment and let sparc crash.
> 3. add an aligned allocator
>
> I somewhat favor 1, it has the risk that a vectorizing compiler might
> wreak havoc but to my knowledge numpy does not actually have real 16
> byte copies. There are some occurrences of npy_int128, but those likely
> are not used on windows.
>
> fwiw I could reproduce the issue on linux i386 with -malign-double, (I
> could have sworn I tested that configuration too...)
>

I would also go for 1) on the general principal that alignment needs to be
platform dependent. I suppose the problem here is that is might also be
compiler dependent. Where do the aligned copies  take place? What is the
downside to that solution?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20141126/f2465938/attachment.html>


More information about the NumPy-Discussion mailing list