![](https://secure.gravatar.com/avatar/e9ba173af9420bda3010f9c2405847f8.jpg?s=120&d=mm&r=g)
This conversation gets discussed often with Numpy developers but since the requirement for optimized Blas is pretty common these days, how about distributing Numpy with OpenBlas by default? People who don't want optimized BLAS or OpenBLAS can then edit the site.cfg file to add/remove. I can never remember if Numpy comes with Atlas by default but either way, if using MKL is not feasible because of its licensing issues then Numpy has to be re-compiled with OpenBLAS (for example). Why not make it easier for developers to use Numpy with an in-built optimized Blas. Btw, just in case some folks from Intel are listening: how about releasing MKL binaries for all platforms for developers to do with it what they want ie. free. You know it makes sense!
![](https://secure.gravatar.com/avatar/da3a0a1942fbdc5ee9a9b8115ac5dae7.jpg?s=120&d=mm&r=g)
26.01.2014 14:44, Dinesh Vadhia kirjoitti:
The Numpy Windows binaries distributed in the numpy project at sourceforge.net are compiled with ATLAS, which should count as an optimized BLAS. I don't recall what's the situation with OSX binaries, but I'd believe they're with Atlas too. If you are suggesting bundling OpenBLAS with Numpy source releases --- arguments against: OpenBLAS is big, and still rapidly moving. Moreover, bundling it with Numpy does not really make it any easier to build. -- Pauli Virtanen
![](https://secure.gravatar.com/avatar/af6c39d6943bd4b0e1fde23161e7bb8c.jpg?s=120&d=mm&r=g)
On Sun, 26 Jan 2014 16:40:44 +0200, Pauli Virtanen wrote:
Was a switch made away from Accelerate after this? http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html Stéfan
![](https://secure.gravatar.com/avatar/c0da24f75f763b6bac90b519064f30b3.jpg?s=120&d=mm&r=g)
On 26.01.2014 18:06, Stéfan van der Walt wrote:
if this issue disqualifies accelerate, it also disqualifies openblas as a default. openblas has the same issue, we stuck a big fat warning into the docs (site.cfg) for this now as people keep running into it. openblas is also a little dodgy concerning stability, in the past it crashed constantly on pretty standard problems, like dgemm on data > 64 mb etc. While the stability has improved with latest releases (>= 0.2.9) I think its still too early to consider openblas for a default. multithreaded ATLAS on the other hand seems works fine, at least I have not seen any similar issues with ATLAS in a very long time. Building optimized ATLAS is also a breeze on Debian based systems (see the README.Debian file) but I admit it is hard on any other platform.
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Julian Taylor <jtaylor.debian@googlemail.com> wrote:
What? Last time I checked, OpenBLAS (and GotoBLAS2) used OpenMP, not the GCD on Mac. Since OpenMP compiles to pthreads, it should not do this (pure POSIX). Accelerate uses the GCD yes, but it's hardly any better than ATLAS. If OpenBLAS now uses the GCD on Mac someone in China should be flogged. It is sad to hear about stability issues with OpenBLAS, it's predecessor GotoBLAS2 was rock solid. Sturla
![](https://secure.gravatar.com/avatar/c0da24f75f763b6bac90b519064f30b3.jpg?s=120&d=mm&r=g)
On 26.01.2014 22:33, Sturla Molden wrote:
the use of gnu openmp is probably be the problem, forking and gomp is only possible in very limited circumstances. see e.g. https://github.com/xianyi/OpenBLAS/issues/294 maybe it will work with clangs intel based openmp which should be coming soon. the current workaround is single threaded openblas, python3.4 forkserver or use atlas.
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Julian Taylor <jtaylor.debian@googlemail.com> wrote:
Yes, it seems to be a GNU problem: http://bisqwit.iki.fi/story/howto/openmp/#OpenmpAndFork This Howto also claims Intel compilers is not affected. :) Sturla
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Sturla Molden <sturla.molden@gmail.com> wrote:
It seems another patch has been proposed to the libgomp team today: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60035 Unfortunaltely I don't think the libgomp team really appreciates the absurdity that a call to BLAS function like DGEMM can interfere with a subsequent call to fork. Mixing multithreading and forks is one thing, but the raison d'etre for OpenMP is to simplify vectorization of serial code. Sturla
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Sun, Jan 26, 2014 at 6:06 PM, Stéfan van der Walt <stefan@sun.ac.za>wrote:
http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html
No, nothing changed. Still using Accelerate for all official binaries. Ralf
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
Did you consider to check the experimental binaries on https://code.google.com/p/mingw-w64-static/ for Python-2.7? These binaries has been build with with a customized mingw-w64 toolchain. These builds are fully statically build and are link against the MSVC90 runtime libraries (gcc runtime is linked statically) and OpenBLAS. Carl 2014-01-26 Pauli Virtanen <pav@iki.fi>
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
On 27/01/14 12:01, Carl Kleffner wrote:
Building OpenBLAS and LAPACK is very easy. I used TDM-GCC for Win64. It's just two makefile (not even a configure script). OpenBLAS and LAPACK are probably the easiest libraries to build there is. The main problem for using OpenBLAS with NumPy and SciPy on Windows is that Python 2.7 from www.python.org does not ship with libpython27.a for 64-bit Python, so we need to maintain our own. Also, GNU compilers are required to build OpenBLAS. This means we have to build our own libgfortran as well. The binary is incompatible with the MSVC runtime we use. I.e. not impossible, but painful. http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063740.html Sturla
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
I agree, building OpenBLAS with mingw-w64 is a snap. The problem is choosing and adapting a mingw based gcc-toolchain and patching the numpy sources according to this toolchain. For the last years I was a happy user of the mingw.org based toolchain. After searching for a 64-bit alternative I stumbled upon mingw-w64 and its derivatives. I tried out several mingw-w64 based toolchains, i.e. TDM, equation.com and more. All mingw-w64 derivatives have there pros and cons. You may know, that you have to choose not only for bitness (32 vs 64 bit) and gcc version, but also for exception handling (sjlj, dwarf, seh) and the way threading is supported (win32 vs. posix threads). Not all of these derivatives describe what they use in a clearly manner. And the TDM toolchain i.e. has introduced some API incompatibilities to standard gcc-toolchains due to its own patches. A serious problem is gcc linking to runtimes other than msvcrt.dll. Mingw-w64 HAS import libraries fort msvcr80, msvcr90, msvcr100, msvcr110. However correct linkage to say to msvcr90 is more than just adding -lmsvcr90 to the linker command. You have to create a spec file for gcc and adapt it to your need. It is also very important (especially for msvcr90) to link manifest files to the binaries you create. This has to do with the way Microsoft searches for DLLs. "Akruis" (Anselm Kruis, science + computing AG) did the job to iron out these problems concerning mingw-w64 and python. Unfortunately his blog disappears for some time now. The maintainers of the mingw-w64 toolchains DO NOT focus on the problem with alternative runtime linking. A related problem is that symbols are used by OpenMP and winpthreads you can resolve in msvcrt.dll, but not in msvcr90.dll, so "_ftime"has to be exchanged with "ftime64" if you want to use OpenMP or winpthreads. In the end my solution was to build my own toolchain. This is time consuming but simple with the help of the set of scripts you can find here: https://github.com/niXman/mingw-builds/tree/develop With this set of scripts and msys2 http://sourceforge.net/projects/msys2/and my own "_ftime" patch I build a 'statically' mingw-w64 toolchain. Let me say a word about statically build: GCC can be build statically. This means, that all of the C, C++, Gfortran runtime is statically linked to every binary. There is not much bloat as you might expect when the binaries are stripped. And yes, it is necessary to build an import lib for python. This import lib is specific to the toolchain you are going to use. My idea is to create and add all import libs (py2.6 up to py3.4) to the toolchain and do not use any of the importlibs that might exist in the python/libs/ folder. My conclusion is: mixing different compiler architectures for building Python extensions on Windows is possible but makes it necessary to build a 'vendor' gcc toolchain. I did not find the time to put my latest binaries on the web or make numpy pull requests the github way due to my workload. Hopefully I find some time next weekend. with best regards Carl 2014-01-30 Sturla Molden <sturla.molden@gmail.com>:
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
On 30/01/14 12:01, Carl Kleffner wrote:
Right. This makes a nice twist on the infamous XML and Regex story: - There once was a man who had a problem building NumPy. Then he thought, "I'll just use a custom compiler toolchain." Now he had two problems. Setting up a custom GNU toolchain for NumPy on Windows would not be robust enough. And when there be bugs, we have two places to look for them instead of one. By using a tested and verified compiler toolchain, there is one place less things can go wrong. I would rather consider distributing NumPy binaries linked with MKL, if Intel's license allows it. Sturla
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
I fully agree with you. But you have to consider the following: - the officially mingw-w64 toolchains are build almost the same way. The only difference is, that they have non-static builds (that would be preferable for C++ development BTW) - you won't get the necessary addons like spec-files, manifest resource files for msvcr90,100 from there. - there is a urgent need for a free and portable C,C++, Fortran compiler for Windows with full blas, lapack support. You won't get that with numpy-MKL, but with a GNU toolchain and OpenBLAS. Not everyone can buy the Intel Fortran compiler or is allowed to install it. - you can build 3rd party extensions which use blas,lapack directly or with cython with such a toolchain regardless if you use numpy/scipy-MKL or mingw-based numpy/scipy - The licence question of numpy-MKL is unclear. I know that MKL is linked in statically. But can I redistribite it myself or use it in commercial context without buying a Intel licence? Carl 2014-01-30 Sturla Molden <sturla.molden@gmail.com>:
![](https://secure.gravatar.com/avatar/b4929294417e9ac44c17967baae75a36.jpg?s=120&d=mm&r=g)
Hi, On Thu, Jan 30, 2014 at 4:29 AM, Carl Kleffner <cmkleffner@gmail.com> wrote:
Thanks for doing this - I'd love to see the toolchain. If there's anything I can do to help, please let me know. The only obvious thing I can think of is using our buildbots or just the spare machines we have: http://nipy.bic.berkeley.edu/ but if you can think of anything else, please let me know. Cheers, Matthew
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
By the way, it seems OpenBLAS builds with clang on MacOSX, so presumably it works on Windows as well. Unlike GNU toolchains, there is a cl-clang frontend which is supposed to be MSVC compatible. BTW, clang is a fantastic compiler, but little known among Windows users where MSVC and MinGW dominate. Sturla On 30/01/14 13:29, Carl Kleffner wrote:
![](https://secure.gravatar.com/avatar/1afc6ca15c6efe13893e5ddde6b33bd6.jpg?s=120&d=mm&r=g)
Hi all- The following snippet appears to leak memory badly (about 10 MB per execution): P = randint(0,2,(30,13)) for i in range(50): print "\r", i, "/", 50 for ai in ndindex((2,)*13): j = np.sum(P.dot(ai)) If instead you execute (no np.sum call): P = randint(0,2,(30,13)) for i in range(50): print "\r", i, "/", 50 for ai in ndindex((2,)*13): j = P.dot(ai) There is no leak. Any thoughts? I’m stumped. Best, Chris -- Chris Laumann Sent with Airmail
![](https://secure.gravatar.com/avatar/09939f25b639512a537ce2c90f77f958.jpg?s=120&d=mm&r=g)
Just to chime in here about the SciPy Superpack... this distribution tracks the master branch of many projects, and then puts out releases, on the assumption that master contains pristine code, I guess. I have gone down strange rabbit holes thinking that a particular bug was fixed already and the user telling me a version number that would confirm that, only to discover that the superpack actually packaged matplotlib about a month prior to releasing a version. I will not comment on how good or bad of an idea it is for the Superpack to do that, but I just wanted to make other developers aware of this to keep them from falling down the same rabbit hole. Cheers! Ben Root On Fri, Jan 31, 2014 at 10:37 AM, Nathaniel Smith <njs@pobox.com> wrote:
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On Fri, Jan 31, 2014 at 4:29 PM, Benjamin Root <ben.root@ou.edu> wrote:
Wow, that is good to know. Esp. since the web page: http://fonnesbeck.github.io/ScipySuperpack/ simply advertises that it gives you things like numpy 1.9 and scipy 0.14, which don't exist. (With some note about dev versions buried in prose a few sentences later.) Empirically, development versions of numpy have always contained bugs, regressions, and compatibility breaks that were fixed in the released version; and we make absolutely no guarantees about compatibility between dev versions and any release versions. And it sort of has to be that way for us to be able to make progress. But if too many people start using dev versions for daily use, then we and downstream dependencies will have to start adding compatibility hacks and stuff to support those dev versions. Which would be a nightmare for developers and users both. Recommending this build for daily use by non-developers strikes me as dangerous for both users and the wider ecosystem. -n
![](https://secure.gravatar.com/avatar/c0da24f75f763b6bac90b519064f30b3.jpg?s=120&d=mm&r=g)
On 31.01.2014 18:12, Nathaniel Smith wrote:
while probably not good for the user I think its very good for us. This is the second bug I introduced found by superpack users. This one might have gone unnoticed into the next release as it is pretty much impossible to find via tests. Even in valgrind reports its hard to find as its lumped in with all of pythons hundreds of memory arena still-reachable leaks. Concerning the fix, it seems if python sees tp_free == PYObject_Del/Free it replaces it with the tp_free of the base type which is int_free in this case. int_free uses a special allocator for even lower overhead so we start leaking. We either need to find the right flag to set for our scalars so it stops doing that, add an indirection so the function pointers don't match or stop using the object allocator as we are apparently digging to deep into pythons internal implementation details by doing so.
![](https://secure.gravatar.com/avatar/1afc6ca15c6efe13893e5ddde6b33bd6.jpg?s=120&d=mm&r=g)
Hi all- Thanks for the info re: memory leak. In trying to work around it, I think I’ve discovered another (still using SuperPack). This leaks ~30MB / run: hists = zeros((50,64), dtype=int) for i in range(50): for j in range(2**13): hists[i,j%64] += 1 The code leaks using hists[i,j] = hists[i,j] + 1 as well. Is this the same leak or different? Doesn’t seem to have much in common. Incidentally, using a = ones(v.shape[0]) a.dot(v) Instead of np.sum (in the previous example that i sent) does not leak. Re: superpack.. As a fairly technically proficient user, I’m aware that the super pack installs dev builds and that they may therefore by somewhat less reliable. I’m okay with that tradeoff and I don’t expect you guys to actually treat the super pack as a stable release — I also try to report that I’m using the superpack when I report bugs. I sometimes run git versions of ipython, numpy, etc in order to fiddle with the code and make tiny bug fixes/contributions myself. I don’t know the statistics re: superpack users but there is no link from scipy.org’s main install page so most new users won’t find it easily. Fonnesbeck’s webpage does say they are dev builds only two sentences into the paragraph. Best, Chris -- Chris Laumann Sent with Airmail On January 31, 2014 at 9:31:40 AM, Julian Taylor (jtaylor.debian@googlemail.com) wrote: On 31.01.2014 18:12, Nathaniel Smith wrote:
while probably not good for the user I think its very good for us. This is the second bug I introduced found by superpack users. This one might have gone unnoticed into the next release as it is pretty much impossible to find via tests. Even in valgrind reports its hard to find as its lumped in with all of pythons hundreds of memory arena still-reachable leaks. Concerning the fix, it seems if python sees tp_free == PYObject_Del/Free it replaces it with the tp_free of the base type which is int_free in this case. int_free uses a special allocator for even lower overhead so we start leaking. We either need to find the right flag to set for our scalars so it stops doing that, add an indirection so the function pointers don't match or stop using the object allocator as we are apparently digging to deep into pythons internal implementation details by doing so. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
On 26/01/14 13:44, Dinesh Vadhia wrote:> This conversation gets discussed often with Numpy developers but since
There is an active discussion on this here: https://github.com/xianyi/OpenBLAS/issues/294 Sturla
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
Concerning numpy-MKL licence I refer to this question on the Intel Forum: http://software.intel.com/en-us/forums/topic/328344 "Question on Redistribution related to numpy/scipy" In the case of numpy-MKL the MKL binaries are statically linked to the pyd-files. Given the usefulness, performance and robustness of the MKL-based binaries a definite answer to this question would be desirable. Say: "Can I use and re-redistribute a product with a precompiled numpy-MKL in a commercial enviroment without the need to by a Intel licence? Carl 2014-01-26 Dinesh Vadhia <dineshbvadhia@hotmail.com>:
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Carl Kleffner <cmkleffner@gmail.com> wrote:
I don't see why this is relevant. If you make commercial software chances are you can afford a commercial license for Intel's C++ or Fortran compiler. If you don't, you don't charge your customers enough. Also consider this: Can software packed and linked with MKL be sold in a store? That is also redistribution, and the store is likely not to own an MKL license. There is thus only one reasonable answer. And besides, if you make commercial software, chances are your solicitor verifies the license rights. If you don't consult a solicitor, that would be at your own risk. Sturla
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
If you work in an academia world it can be relevant once third parties are involved in a bigger project. A situation may be reached, where you just have to prove the license situation of all of your software components. Numpy and scipy is 'selled' as BSD or MIT based foundation for scientific software without components with copyleft licences. For the MKL part a clear statement would be welcome. Otherwise the usage of MKL based binaries has to be avoided in such situations, even if you don't sell something. 2014-02-02 Sturla Molden <sturla.molden@gmail.com>:
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Carl Kleffner <cmkleffner@gmail.com> wrote:
If you involve third parties outside academia, you need a commercial license. Binaries with academic license is for academic use only. Personally I pay Enthought Inc. to provide me with NumPy, and then it's their responsibility to work out the license details in their software stack. I am licensing my Python software from Enthought Inc., and I cannot go through and verify every single one of their licenses. If asked I will just refer to the license that comes with my Canopy subscription, and that will be the end of it.
That is utter nonsence. MKL is not different from any other commercial software. With this way of backwards thinking, no commercial software could ever be used. You could e.g. never use Windows, because you might be asked to prove Microsoft's license for third-party libraries used by their operating system. That is just bullshit. I might be asked to prove my license with Microsoft, but it's Microsoft's responsibility to work out the internal license details for the software they sell. Sturla
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
I just thought i'd mention this: Why not use Eigen? It has a full BLAS implementation and passes the BLAS test suite, and is generelly faster than MKL except on Core i7. Also, Eigen requires no build process, just include the header files. Yes, Eigen is based on C++, but OpenBLAS is parially coded in assembly. Whatever language is used internally in the BLAS implementation should be of no concern to NumPy. BTW: The performance of OpenBLAS is far behind Eigen, MKL and ACML, but better than ATLAS and Accelerate. Sturla "Dinesh Vadhia" <dineshbvadhia@hotmail.com> wrote:
![](https://secure.gravatar.com/avatar/6a42c582fd9b4b60aae189e6321731b0.jpg?s=120&d=mm&r=g)
On 2014-02-06 11:10, Sturla Molden wrote:
Sorry for going a bit off-topic, but: do you have any links to the benchmarks? I googled around, but I haven't found anything. FWIW, on my own machines OpenBLAS is on par with MKL (on an i5 laptop and an older Xeon server) and actually slightly faster than ACML (on an FX8150) for my use cases (I mainly tested DGEMM/SGEMM, and a few LAPACK calls). So your claim is very surprising for me. Also, I'd be highly surprised if OpenBLAS would be slower than Eigen, given than the developers themselves say that Eigen is "nearly as fast as GotoBLAS"[1], and that OpenBLAS was originally forked from GotoBLAS. Cheers Thomas [1] http://eigen.tuxfamily.org/index.php?title=3.0
![](https://secure.gravatar.com/avatar/c0da24f75f763b6bac90b519064f30b3.jpg?s=120&d=mm&r=g)
On Thu, Feb 6, 2014 at 1:11 PM, Thomas Unterthiner < thomas_unterthiner@web.de> wrote:
I'm also a little sceptical about the benchmarks, e.g. according to the FAQ eigen does not seem to support AVX which is relatively important for blas level 3 performance. The lazy evaluation is probably eigens main selling point, which is something we cannot make use of in numpy currently. But nevertheless eigen could be an interesting alternative for our binary releases on windows. Having the stuff as headers makes it probably easier to build than ATLAS we are currently using.
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Thu, Feb 6, 2014 at 5:27 AM, Julian Taylor <jtaylor.debian@googlemail.com
wrote:
The Eigen license is MPL-2. That doesn't look to be incompatible with BSD, but it may complicate things. Q8: I want to distribute (outside my organization) executable programs or libraries that I have compiled from someone else's unchanged MPL-licensed source code, either standalone or part of a larger work. What do I have to do? <http://www.mozilla.org/MPL/2.0/FAQ.html#distribute-my-binaries> You must inform the recipients where they can get the source for the executable program you are distributing (i.e., you must comply with Section 3.2). You may also distribute any executables you create under a license of your choosing, as long as that license does not interfere with the recipients' rights to the source under the terms of the MPL. Chuck
![](https://secure.gravatar.com/avatar/60e03bd1fd9f2dbc750e0899b9e7e71d.jpg?s=120&d=mm&r=g)
According to the discussions on the ML, they switched from GPL to MPL to enable the kind of distribution numpy/scipy is looking for. They had some hesitations between BSD and MPL, but IIRC their official stand is to allow inclusion inside BSD-licensed code. Cheers, Matthieu 2014-02-06 20:09 GMT+00:00 Charles R Harris <charlesr.harris@gmail.com>:
-- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/
![](https://secure.gravatar.com/avatar/764323a14e554c97ab74177e0bce51d4.jpg?s=120&d=mm&r=g)
On Thu, Feb 6, 2014 at 9:45 PM, Matthieu Brucher <matthieu.brucher@gmail.com> wrote:
If they want BSD-licensed projects to incorporate their code, they need to license it under the BSD license (or similar). They are not in a position to "allow" their MPL-licensed code to be included in a BSD-licensed project. That just doesn't mean anything. We never needed their permission. We could be "BSD-licensed except for this one bit which is MPLed", but we don't want to be. -- Robert Kern
![](https://secure.gravatar.com/avatar/082c34b6c1544c177cb7a66609360089.jpg?s=120&d=mm&r=g)
On Fri, Feb 7, 2014 at 4:31 AM, Robert Kern <robert.kern@gmail.com> wrote:
I agree that we shouldn't include Eigen code in NumPy. But what distributing windows binaries that include Eigen headers? They wrote this on their web site: """ Virtually any software may use Eigen. For example, closed-source software may use Eigen without having to disclose its own source code. Many proprietary and closed-source software projects are using Eigen right now, as well as many BSD-licensed projects. """ Fred
![](https://secure.gravatar.com/avatar/764323a14e554c97ab74177e0bce51d4.jpg?s=120&d=mm&r=g)
On Fri, Feb 7, 2014 at 2:46 PM, Frédéric Bastien <nouiz@nouiz.org> wrote:
I don't mind anyone distributing such binaries. I think it might even be reasonable for the project itself to distribute such binaries through a page on numpy.org. I do *not* think it would be wise to do so through our PyPI page or the download section of Sourceforge. Those bare lists of files provide insufficient context for us to be able to say "this one particular build includes MPL-licensed code in addition to the usual BSD-licensed code". -- Robert Kern
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Charles R Harris <charlesr.harris@gmail.com> wrote:
MPL-2 is not viral like the GPL. The obligation to inform about the source code only applies to the MPL code. That is, if we use Eigen we must inform where the Eigen source code can be obtained. MPL is a compromize between BSD/MIT and GPL. Using MPL code in binaries should be fine. Sturla
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Thomas Unterthiner <thomas_unterthiner@web.de> wrote:
I was thinking about the benchmarks on Eigen's website, but it might be a bit old now and possibly biased: http://eigen.tuxfamily.org/index.php?title=Benchmark It uses a single thread only, but for smaller matrix sizes Eigen tends to be the better. Carl Kleffner alerted me to this benchmark today: http://gcdart.blogspot.de/2013/06/fast-matrix-multiply-and-ml.html It shows superb performance and unparallelled scalability for OpenBLAS on Opteron. MKL might be better on Intel CPUs though. ATLAS is doing quite well too, better than I would expect, and generally better than Eigen. It is also interesting that ACML is crap, except with a single-threaded BLAS. Sturla
![](https://secure.gravatar.com/avatar/da3a0a1942fbdc5ee9a9b8115ac5dae7.jpg?s=120&d=mm&r=g)
26.01.2014 14:44, Dinesh Vadhia kirjoitti:
The Numpy Windows binaries distributed in the numpy project at sourceforge.net are compiled with ATLAS, which should count as an optimized BLAS. I don't recall what's the situation with OSX binaries, but I'd believe they're with Atlas too. If you are suggesting bundling OpenBLAS with Numpy source releases --- arguments against: OpenBLAS is big, and still rapidly moving. Moreover, bundling it with Numpy does not really make it any easier to build. -- Pauli Virtanen
![](https://secure.gravatar.com/avatar/af6c39d6943bd4b0e1fde23161e7bb8c.jpg?s=120&d=mm&r=g)
On Sun, 26 Jan 2014 16:40:44 +0200, Pauli Virtanen wrote:
Was a switch made away from Accelerate after this? http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html Stéfan
![](https://secure.gravatar.com/avatar/c0da24f75f763b6bac90b519064f30b3.jpg?s=120&d=mm&r=g)
On 26.01.2014 18:06, Stéfan van der Walt wrote:
if this issue disqualifies accelerate, it also disqualifies openblas as a default. openblas has the same issue, we stuck a big fat warning into the docs (site.cfg) for this now as people keep running into it. openblas is also a little dodgy concerning stability, in the past it crashed constantly on pretty standard problems, like dgemm on data > 64 mb etc. While the stability has improved with latest releases (>= 0.2.9) I think its still too early to consider openblas for a default. multithreaded ATLAS on the other hand seems works fine, at least I have not seen any similar issues with ATLAS in a very long time. Building optimized ATLAS is also a breeze on Debian based systems (see the README.Debian file) but I admit it is hard on any other platform.
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Julian Taylor <jtaylor.debian@googlemail.com> wrote:
What? Last time I checked, OpenBLAS (and GotoBLAS2) used OpenMP, not the GCD on Mac. Since OpenMP compiles to pthreads, it should not do this (pure POSIX). Accelerate uses the GCD yes, but it's hardly any better than ATLAS. If OpenBLAS now uses the GCD on Mac someone in China should be flogged. It is sad to hear about stability issues with OpenBLAS, it's predecessor GotoBLAS2 was rock solid. Sturla
![](https://secure.gravatar.com/avatar/c0da24f75f763b6bac90b519064f30b3.jpg?s=120&d=mm&r=g)
On 26.01.2014 22:33, Sturla Molden wrote:
the use of gnu openmp is probably be the problem, forking and gomp is only possible in very limited circumstances. see e.g. https://github.com/xianyi/OpenBLAS/issues/294 maybe it will work with clangs intel based openmp which should be coming soon. the current workaround is single threaded openblas, python3.4 forkserver or use atlas.
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Julian Taylor <jtaylor.debian@googlemail.com> wrote:
Yes, it seems to be a GNU problem: http://bisqwit.iki.fi/story/howto/openmp/#OpenmpAndFork This Howto also claims Intel compilers is not affected. :) Sturla
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Sturla Molden <sturla.molden@gmail.com> wrote:
It seems another patch has been proposed to the libgomp team today: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60035 Unfortunaltely I don't think the libgomp team really appreciates the absurdity that a call to BLAS function like DGEMM can interfere with a subsequent call to fork. Mixing multithreading and forks is one thing, but the raison d'etre for OpenMP is to simplify vectorization of serial code. Sturla
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Sun, Jan 26, 2014 at 6:06 PM, Stéfan van der Walt <stefan@sun.ac.za>wrote:
http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html
No, nothing changed. Still using Accelerate for all official binaries. Ralf
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
Did you consider to check the experimental binaries on https://code.google.com/p/mingw-w64-static/ for Python-2.7? These binaries has been build with with a customized mingw-w64 toolchain. These builds are fully statically build and are link against the MSVC90 runtime libraries (gcc runtime is linked statically) and OpenBLAS. Carl 2014-01-26 Pauli Virtanen <pav@iki.fi>
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
On 27/01/14 12:01, Carl Kleffner wrote:
Building OpenBLAS and LAPACK is very easy. I used TDM-GCC for Win64. It's just two makefile (not even a configure script). OpenBLAS and LAPACK are probably the easiest libraries to build there is. The main problem for using OpenBLAS with NumPy and SciPy on Windows is that Python 2.7 from www.python.org does not ship with libpython27.a for 64-bit Python, so we need to maintain our own. Also, GNU compilers are required to build OpenBLAS. This means we have to build our own libgfortran as well. The binary is incompatible with the MSVC runtime we use. I.e. not impossible, but painful. http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063740.html Sturla
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
I agree, building OpenBLAS with mingw-w64 is a snap. The problem is choosing and adapting a mingw based gcc-toolchain and patching the numpy sources according to this toolchain. For the last years I was a happy user of the mingw.org based toolchain. After searching for a 64-bit alternative I stumbled upon mingw-w64 and its derivatives. I tried out several mingw-w64 based toolchains, i.e. TDM, equation.com and more. All mingw-w64 derivatives have there pros and cons. You may know, that you have to choose not only for bitness (32 vs 64 bit) and gcc version, but also for exception handling (sjlj, dwarf, seh) and the way threading is supported (win32 vs. posix threads). Not all of these derivatives describe what they use in a clearly manner. And the TDM toolchain i.e. has introduced some API incompatibilities to standard gcc-toolchains due to its own patches. A serious problem is gcc linking to runtimes other than msvcrt.dll. Mingw-w64 HAS import libraries fort msvcr80, msvcr90, msvcr100, msvcr110. However correct linkage to say to msvcr90 is more than just adding -lmsvcr90 to the linker command. You have to create a spec file for gcc and adapt it to your need. It is also very important (especially for msvcr90) to link manifest files to the binaries you create. This has to do with the way Microsoft searches for DLLs. "Akruis" (Anselm Kruis, science + computing AG) did the job to iron out these problems concerning mingw-w64 and python. Unfortunately his blog disappears for some time now. The maintainers of the mingw-w64 toolchains DO NOT focus on the problem with alternative runtime linking. A related problem is that symbols are used by OpenMP and winpthreads you can resolve in msvcrt.dll, but not in msvcr90.dll, so "_ftime"has to be exchanged with "ftime64" if you want to use OpenMP or winpthreads. In the end my solution was to build my own toolchain. This is time consuming but simple with the help of the set of scripts you can find here: https://github.com/niXman/mingw-builds/tree/develop With this set of scripts and msys2 http://sourceforge.net/projects/msys2/and my own "_ftime" patch I build a 'statically' mingw-w64 toolchain. Let me say a word about statically build: GCC can be build statically. This means, that all of the C, C++, Gfortran runtime is statically linked to every binary. There is not much bloat as you might expect when the binaries are stripped. And yes, it is necessary to build an import lib for python. This import lib is specific to the toolchain you are going to use. My idea is to create and add all import libs (py2.6 up to py3.4) to the toolchain and do not use any of the importlibs that might exist in the python/libs/ folder. My conclusion is: mixing different compiler architectures for building Python extensions on Windows is possible but makes it necessary to build a 'vendor' gcc toolchain. I did not find the time to put my latest binaries on the web or make numpy pull requests the github way due to my workload. Hopefully I find some time next weekend. with best regards Carl 2014-01-30 Sturla Molden <sturla.molden@gmail.com>:
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
On 30/01/14 12:01, Carl Kleffner wrote:
Right. This makes a nice twist on the infamous XML and Regex story: - There once was a man who had a problem building NumPy. Then he thought, "I'll just use a custom compiler toolchain." Now he had two problems. Setting up a custom GNU toolchain for NumPy on Windows would not be robust enough. And when there be bugs, we have two places to look for them instead of one. By using a tested and verified compiler toolchain, there is one place less things can go wrong. I would rather consider distributing NumPy binaries linked with MKL, if Intel's license allows it. Sturla
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
I fully agree with you. But you have to consider the following: - the officially mingw-w64 toolchains are build almost the same way. The only difference is, that they have non-static builds (that would be preferable for C++ development BTW) - you won't get the necessary addons like spec-files, manifest resource files for msvcr90,100 from there. - there is a urgent need for a free and portable C,C++, Fortran compiler for Windows with full blas, lapack support. You won't get that with numpy-MKL, but with a GNU toolchain and OpenBLAS. Not everyone can buy the Intel Fortran compiler or is allowed to install it. - you can build 3rd party extensions which use blas,lapack directly or with cython with such a toolchain regardless if you use numpy/scipy-MKL or mingw-based numpy/scipy - The licence question of numpy-MKL is unclear. I know that MKL is linked in statically. But can I redistribite it myself or use it in commercial context without buying a Intel licence? Carl 2014-01-30 Sturla Molden <sturla.molden@gmail.com>:
![](https://secure.gravatar.com/avatar/b4929294417e9ac44c17967baae75a36.jpg?s=120&d=mm&r=g)
Hi, On Thu, Jan 30, 2014 at 4:29 AM, Carl Kleffner <cmkleffner@gmail.com> wrote:
Thanks for doing this - I'd love to see the toolchain. If there's anything I can do to help, please let me know. The only obvious thing I can think of is using our buildbots or just the spare machines we have: http://nipy.bic.berkeley.edu/ but if you can think of anything else, please let me know. Cheers, Matthew
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
By the way, it seems OpenBLAS builds with clang on MacOSX, so presumably it works on Windows as well. Unlike GNU toolchains, there is a cl-clang frontend which is supposed to be MSVC compatible. BTW, clang is a fantastic compiler, but little known among Windows users where MSVC and MinGW dominate. Sturla On 30/01/14 13:29, Carl Kleffner wrote:
![](https://secure.gravatar.com/avatar/1afc6ca15c6efe13893e5ddde6b33bd6.jpg?s=120&d=mm&r=g)
Hi all- The following snippet appears to leak memory badly (about 10 MB per execution): P = randint(0,2,(30,13)) for i in range(50): print "\r", i, "/", 50 for ai in ndindex((2,)*13): j = np.sum(P.dot(ai)) If instead you execute (no np.sum call): P = randint(0,2,(30,13)) for i in range(50): print "\r", i, "/", 50 for ai in ndindex((2,)*13): j = P.dot(ai) There is no leak. Any thoughts? I’m stumped. Best, Chris -- Chris Laumann Sent with Airmail
![](https://secure.gravatar.com/avatar/09939f25b639512a537ce2c90f77f958.jpg?s=120&d=mm&r=g)
Just to chime in here about the SciPy Superpack... this distribution tracks the master branch of many projects, and then puts out releases, on the assumption that master contains pristine code, I guess. I have gone down strange rabbit holes thinking that a particular bug was fixed already and the user telling me a version number that would confirm that, only to discover that the superpack actually packaged matplotlib about a month prior to releasing a version. I will not comment on how good or bad of an idea it is for the Superpack to do that, but I just wanted to make other developers aware of this to keep them from falling down the same rabbit hole. Cheers! Ben Root On Fri, Jan 31, 2014 at 10:37 AM, Nathaniel Smith <njs@pobox.com> wrote:
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On Fri, Jan 31, 2014 at 4:29 PM, Benjamin Root <ben.root@ou.edu> wrote:
Wow, that is good to know. Esp. since the web page: http://fonnesbeck.github.io/ScipySuperpack/ simply advertises that it gives you things like numpy 1.9 and scipy 0.14, which don't exist. (With some note about dev versions buried in prose a few sentences later.) Empirically, development versions of numpy have always contained bugs, regressions, and compatibility breaks that were fixed in the released version; and we make absolutely no guarantees about compatibility between dev versions and any release versions. And it sort of has to be that way for us to be able to make progress. But if too many people start using dev versions for daily use, then we and downstream dependencies will have to start adding compatibility hacks and stuff to support those dev versions. Which would be a nightmare for developers and users both. Recommending this build for daily use by non-developers strikes me as dangerous for both users and the wider ecosystem. -n
![](https://secure.gravatar.com/avatar/c0da24f75f763b6bac90b519064f30b3.jpg?s=120&d=mm&r=g)
On 31.01.2014 18:12, Nathaniel Smith wrote:
while probably not good for the user I think its very good for us. This is the second bug I introduced found by superpack users. This one might have gone unnoticed into the next release as it is pretty much impossible to find via tests. Even in valgrind reports its hard to find as its lumped in with all of pythons hundreds of memory arena still-reachable leaks. Concerning the fix, it seems if python sees tp_free == PYObject_Del/Free it replaces it with the tp_free of the base type which is int_free in this case. int_free uses a special allocator for even lower overhead so we start leaking. We either need to find the right flag to set for our scalars so it stops doing that, add an indirection so the function pointers don't match or stop using the object allocator as we are apparently digging to deep into pythons internal implementation details by doing so.
![](https://secure.gravatar.com/avatar/1afc6ca15c6efe13893e5ddde6b33bd6.jpg?s=120&d=mm&r=g)
Hi all- Thanks for the info re: memory leak. In trying to work around it, I think I’ve discovered another (still using SuperPack). This leaks ~30MB / run: hists = zeros((50,64), dtype=int) for i in range(50): for j in range(2**13): hists[i,j%64] += 1 The code leaks using hists[i,j] = hists[i,j] + 1 as well. Is this the same leak or different? Doesn’t seem to have much in common. Incidentally, using a = ones(v.shape[0]) a.dot(v) Instead of np.sum (in the previous example that i sent) does not leak. Re: superpack.. As a fairly technically proficient user, I’m aware that the super pack installs dev builds and that they may therefore by somewhat less reliable. I’m okay with that tradeoff and I don’t expect you guys to actually treat the super pack as a stable release — I also try to report that I’m using the superpack when I report bugs. I sometimes run git versions of ipython, numpy, etc in order to fiddle with the code and make tiny bug fixes/contributions myself. I don’t know the statistics re: superpack users but there is no link from scipy.org’s main install page so most new users won’t find it easily. Fonnesbeck’s webpage does say they are dev builds only two sentences into the paragraph. Best, Chris -- Chris Laumann Sent with Airmail On January 31, 2014 at 9:31:40 AM, Julian Taylor (jtaylor.debian@googlemail.com) wrote: On 31.01.2014 18:12, Nathaniel Smith wrote:
while probably not good for the user I think its very good for us. This is the second bug I introduced found by superpack users. This one might have gone unnoticed into the next release as it is pretty much impossible to find via tests. Even in valgrind reports its hard to find as its lumped in with all of pythons hundreds of memory arena still-reachable leaks. Concerning the fix, it seems if python sees tp_free == PYObject_Del/Free it replaces it with the tp_free of the base type which is int_free in this case. int_free uses a special allocator for even lower overhead so we start leaking. We either need to find the right flag to set for our scalars so it stops doing that, add an indirection so the function pointers don't match or stop using the object allocator as we are apparently digging to deep into pythons internal implementation details by doing so. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
On 26/01/14 13:44, Dinesh Vadhia wrote:> This conversation gets discussed often with Numpy developers but since
There is an active discussion on this here: https://github.com/xianyi/OpenBLAS/issues/294 Sturla
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
Concerning numpy-MKL licence I refer to this question on the Intel Forum: http://software.intel.com/en-us/forums/topic/328344 "Question on Redistribution related to numpy/scipy" In the case of numpy-MKL the MKL binaries are statically linked to the pyd-files. Given the usefulness, performance and robustness of the MKL-based binaries a definite answer to this question would be desirable. Say: "Can I use and re-redistribute a product with a precompiled numpy-MKL in a commercial enviroment without the need to by a Intel licence? Carl 2014-01-26 Dinesh Vadhia <dineshbvadhia@hotmail.com>:
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Carl Kleffner <cmkleffner@gmail.com> wrote:
I don't see why this is relevant. If you make commercial software chances are you can afford a commercial license for Intel's C++ or Fortran compiler. If you don't, you don't charge your customers enough. Also consider this: Can software packed and linked with MKL be sold in a store? That is also redistribution, and the store is likely not to own an MKL license. There is thus only one reasonable answer. And besides, if you make commercial software, chances are your solicitor verifies the license rights. If you don't consult a solicitor, that would be at your own risk. Sturla
![](https://secure.gravatar.com/avatar/29a8424a5c1ddc5e0e79104965a85011.jpg?s=120&d=mm&r=g)
If you work in an academia world it can be relevant once third parties are involved in a bigger project. A situation may be reached, where you just have to prove the license situation of all of your software components. Numpy and scipy is 'selled' as BSD or MIT based foundation for scientific software without components with copyleft licences. For the MKL part a clear statement would be welcome. Otherwise the usage of MKL based binaries has to be avoided in such situations, even if you don't sell something. 2014-02-02 Sturla Molden <sturla.molden@gmail.com>:
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Carl Kleffner <cmkleffner@gmail.com> wrote:
If you involve third parties outside academia, you need a commercial license. Binaries with academic license is for academic use only. Personally I pay Enthought Inc. to provide me with NumPy, and then it's their responsibility to work out the license details in their software stack. I am licensing my Python software from Enthought Inc., and I cannot go through and verify every single one of their licenses. If asked I will just refer to the license that comes with my Canopy subscription, and that will be the end of it.
That is utter nonsence. MKL is not different from any other commercial software. With this way of backwards thinking, no commercial software could ever be used. You could e.g. never use Windows, because you might be asked to prove Microsoft's license for third-party libraries used by their operating system. That is just bullshit. I might be asked to prove my license with Microsoft, but it's Microsoft's responsibility to work out the internal license details for the software they sell. Sturla
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
I just thought i'd mention this: Why not use Eigen? It has a full BLAS implementation and passes the BLAS test suite, and is generelly faster than MKL except on Core i7. Also, Eigen requires no build process, just include the header files. Yes, Eigen is based on C++, but OpenBLAS is parially coded in assembly. Whatever language is used internally in the BLAS implementation should be of no concern to NumPy. BTW: The performance of OpenBLAS is far behind Eigen, MKL and ACML, but better than ATLAS and Accelerate. Sturla "Dinesh Vadhia" <dineshbvadhia@hotmail.com> wrote:
![](https://secure.gravatar.com/avatar/6a42c582fd9b4b60aae189e6321731b0.jpg?s=120&d=mm&r=g)
On 2014-02-06 11:10, Sturla Molden wrote:
Sorry for going a bit off-topic, but: do you have any links to the benchmarks? I googled around, but I haven't found anything. FWIW, on my own machines OpenBLAS is on par with MKL (on an i5 laptop and an older Xeon server) and actually slightly faster than ACML (on an FX8150) for my use cases (I mainly tested DGEMM/SGEMM, and a few LAPACK calls). So your claim is very surprising for me. Also, I'd be highly surprised if OpenBLAS would be slower than Eigen, given than the developers themselves say that Eigen is "nearly as fast as GotoBLAS"[1], and that OpenBLAS was originally forked from GotoBLAS. Cheers Thomas [1] http://eigen.tuxfamily.org/index.php?title=3.0
![](https://secure.gravatar.com/avatar/c0da24f75f763b6bac90b519064f30b3.jpg?s=120&d=mm&r=g)
On Thu, Feb 6, 2014 at 1:11 PM, Thomas Unterthiner < thomas_unterthiner@web.de> wrote:
I'm also a little sceptical about the benchmarks, e.g. according to the FAQ eigen does not seem to support AVX which is relatively important for blas level 3 performance. The lazy evaluation is probably eigens main selling point, which is something we cannot make use of in numpy currently. But nevertheless eigen could be an interesting alternative for our binary releases on windows. Having the stuff as headers makes it probably easier to build than ATLAS we are currently using.
![](https://secure.gravatar.com/avatar/96dd777e397ab128fedab46af97a3a4a.jpg?s=120&d=mm&r=g)
On Thu, Feb 6, 2014 at 5:27 AM, Julian Taylor <jtaylor.debian@googlemail.com
wrote:
The Eigen license is MPL-2. That doesn't look to be incompatible with BSD, but it may complicate things. Q8: I want to distribute (outside my organization) executable programs or libraries that I have compiled from someone else's unchanged MPL-licensed source code, either standalone or part of a larger work. What do I have to do? <http://www.mozilla.org/MPL/2.0/FAQ.html#distribute-my-binaries> You must inform the recipients where they can get the source for the executable program you are distributing (i.e., you must comply with Section 3.2). You may also distribute any executables you create under a license of your choosing, as long as that license does not interfere with the recipients' rights to the source under the terms of the MPL. Chuck
![](https://secure.gravatar.com/avatar/60e03bd1fd9f2dbc750e0899b9e7e71d.jpg?s=120&d=mm&r=g)
According to the discussions on the ML, they switched from GPL to MPL to enable the kind of distribution numpy/scipy is looking for. They had some hesitations between BSD and MPL, but IIRC their official stand is to allow inclusion inside BSD-licensed code. Cheers, Matthieu 2014-02-06 20:09 GMT+00:00 Charles R Harris <charlesr.harris@gmail.com>:
-- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/
![](https://secure.gravatar.com/avatar/764323a14e554c97ab74177e0bce51d4.jpg?s=120&d=mm&r=g)
On Thu, Feb 6, 2014 at 9:45 PM, Matthieu Brucher <matthieu.brucher@gmail.com> wrote:
If they want BSD-licensed projects to incorporate their code, they need to license it under the BSD license (or similar). They are not in a position to "allow" their MPL-licensed code to be included in a BSD-licensed project. That just doesn't mean anything. We never needed their permission. We could be "BSD-licensed except for this one bit which is MPLed", but we don't want to be. -- Robert Kern
![](https://secure.gravatar.com/avatar/082c34b6c1544c177cb7a66609360089.jpg?s=120&d=mm&r=g)
On Fri, Feb 7, 2014 at 4:31 AM, Robert Kern <robert.kern@gmail.com> wrote:
I agree that we shouldn't include Eigen code in NumPy. But what distributing windows binaries that include Eigen headers? They wrote this on their web site: """ Virtually any software may use Eigen. For example, closed-source software may use Eigen without having to disclose its own source code. Many proprietary and closed-source software projects are using Eigen right now, as well as many BSD-licensed projects. """ Fred
![](https://secure.gravatar.com/avatar/764323a14e554c97ab74177e0bce51d4.jpg?s=120&d=mm&r=g)
On Fri, Feb 7, 2014 at 2:46 PM, Frédéric Bastien <nouiz@nouiz.org> wrote:
I don't mind anyone distributing such binaries. I think it might even be reasonable for the project itself to distribute such binaries through a page on numpy.org. I do *not* think it would be wise to do so through our PyPI page or the download section of Sourceforge. Those bare lists of files provide insufficient context for us to be able to say "this one particular build includes MPL-licensed code in addition to the usual BSD-licensed code". -- Robert Kern
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Charles R Harris <charlesr.harris@gmail.com> wrote:
MPL-2 is not viral like the GPL. The obligation to inform about the source code only applies to the MPL code. That is, if we use Eigen we must inform where the Eigen source code can be obtained. MPL is a compromize between BSD/MIT and GPL. Using MPL code in binaries should be fine. Sturla
![](https://secure.gravatar.com/avatar/2a9d09b311f11f92cdc6a91b3c6519b1.jpg?s=120&d=mm&r=g)
Thomas Unterthiner <thomas_unterthiner@web.de> wrote:
I was thinking about the benchmarks on Eigen's website, but it might be a bit old now and possibly biased: http://eigen.tuxfamily.org/index.php?title=Benchmark It uses a single thread only, but for smaller matrix sizes Eigen tends to be the better. Carl Kleffner alerted me to this benchmark today: http://gcdart.blogspot.de/2013/06/fast-matrix-multiply-and-ml.html It shows superb performance and unparallelled scalability for OpenBLAS on Opteron. MKL might be better on Intel CPUs though. ATLAS is doing quite well too, better than I would expect, and generally better than Eigen. It is also interesting that ACML is crap, except with a single-threaded BLAS. Sturla
participants (16)
-
Benjamin Root
-
Carl Kleffner
-
Charles R Harris
-
Chris Laumann
-
Dinesh Vadhia
-
Frédéric Bastien
-
Julian Taylor
-
Matthew Brett
-
Matthieu Brucher
-
Nathaniel Smith
-
Pauli Virtanen
-
Ralf Gommers
-
Robert Kern
-
Sturla Molden
-
Stéfan van der Walt
-
Thomas Unterthiner