Hello, I am currently working on building scipy for WebAssembly as part of pyodide project (https://github.com/iodide-project/pyodide) and I was hoping for some feedback on that process. There is a preliminary build in https://github.com/iodide-project/pyodide/pull/211 Currently, this build uses scipy 0.17.1 as from what I understood, that was one of the last versions that only included f77 without any f90 (https://github.com/scipy/scipy/issues/2829#issuecomment-223764054). In the WebAssembly environment there is currently no reliably working Fortran compiler (https://github.com/iodide-project/pyodide/issues/184), and f90 cannot be converted to C with f2c unlike f77. If one wanted to (experimentally) compile the latest version of scipy without a Fortran compiler what would be your suggestions? i.e. - are there any alternatives to f2c that you think might work for f90 - or any automatic converter f90 to f77 (so that f2c could be used) I did search but maybe I missed something. Alternatively, if that's really not realistic, could someone please comment on the rate of adoption for f90/f95 in the scipy code base? The decision to support f90 was taken before the 0.18 release in 2016 but I'm not sure what impact it had on the code base. In other words maybe there is a later version than 0.17.1 that might (mostly) work? Another point is linking BLAS/LAPACK. Currently reference BLAS and CLAPACK are linked statically as I haven't managed to do this dynamically yet. The issue is the package size. As LAPACK, which is quite large, gets repeatedly included in around ~10 different .so modules, resulting in a 170MB large package (after compression), as opposed to ~30MB compressed package without BLAS/LAPACK. That is quite problematic when one is expected to download the dependencies at each page load (excluding caching). I'm not sure if there are other distributions of scipy that use static linking of LAPACK? or other things worth trying to reduce the package size, short of trying to make dynamic linking work or try to detect and strip unused symbols? Also in scipy/linalg/setup.py I was wondering why/how the ATLAS_INFO macro defined the existence of `scipy.linalg._clapack`? For instance, when using CLAPACK (with libf2c), would the following be correct? lapack_opt = {'libraries': ['f2c', 'blas', 'lapack'], 'include_dirs': [], 'library_dirs': ['<path_to_blas_lapack_lib>'], 'language': 'f77', 'define_macros': [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]} which does not appear to build `scipy.linalg._clapack`. Finally, in later scipy versions, https://scipy.github.io/devdocs/building/linux.html suggests that with “export LAPACK=” one can build scipy without LAPACK, but then _flapack is still a mandatory import in scipy.linalg.__init___ (via .lapack) as far as I can tell. Is scipy.linalg (and anything that depends on it) expected to produce import errors in the latter case then? In general, any comments or suggestions would be appreciated, Thank you, Roman
On Tue, Oct 30, 2018 at 4:56 PM Roman Yurchak <rth.yurchak@pm.me> wrote:
Hello,
I am currently working on building scipy for WebAssembly as part of pyodide project (https://github.com/iodide-project/pyodide) and I was hoping for some feedback on that process. There is a preliminary build in https://github.com/iodide-project/pyodide/pull/211
Currently, this build uses scipy 0.17.1 as from what I understood, that was one of the last versions that only included f77 without any f90 (https://github.com/scipy/scipy/issues/2829#issuecomment-223764054). In the WebAssembly environment there is currently no reliably working Fortran compiler (https://github.com/iodide-project/pyodide/issues/184), and f90 cannot be converted to C with f2c unlike f77.
If one wanted to (experimentally) compile the latest version of scipy without a Fortran compiler what would be your suggestions? i.e. - are there any alternatives to f2c that you think might work for f90 - or any automatic converter f90 to f77 (so that f2c could be used) I did search but maybe I missed something. Alternatively, if that's really not realistic, could someone please comment on the rate of adoption for f90/f95 in the scipy code base? The decision to support f90 was taken before the 0.18 release in 2016 but I'm not sure what impact it had on the code base. In other words maybe there is a later version than 0.17.1 that might (mostly) work?
There has been very little new Fortran code added since then. The PR that gh-2829 linked to (odeint rewrite) was not merged in the end. I wouldn't be surprised if current master turns out to be f77 compliant.
Another point is linking BLAS/LAPACK. Currently reference BLAS and CLAPACK are linked statically as I haven't managed to do this dynamically yet. The issue is the package size. As LAPACK, which is quite large, gets repeatedly included in around ~10 different .so modules, resulting in a 170MB large package (after compression), as opposed to ~30MB compressed package without BLAS/LAPACK. That is quite problematic when one is expected to download the dependencies at each page load (excluding caching). I'm not sure if there are other distributions of scipy that use static linking of LAPACK? or other things worth trying to reduce the package size, short of trying to make dynamic linking work or try to detect and strip unused symbols?
Nothing I can think of worth trying really.
Also in scipy/linalg/setup.py I was wondering why/how the ATLAS_INFO macro defined the existence of `scipy.linalg._clapack`? For instance, when using CLAPACK (with libf2c), would the following be correct?
lapack_opt = {'libraries': ['f2c', 'blas', 'lapack'], 'include_dirs': [], 'library_dirs': ['<path_to_blas_lapack_lib>'], 'language': 'f77', 'define_macros': [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]}
which does not appear to build `scipy.linalg._clapack`.
You're asking about pretty arcane details:) Don't know the answer, sorry.
Finally, in later scipy versions, https://scipy.github.io/devdocs/building/linux.html suggests that with “export LAPACK=” one can build scipy without LAPACK, but then _flapack is still a mandatory import in scipy.linalg.__init___ (via .lapack) as far as I can tell. Is scipy.linalg (and anything that depends on it) expected to produce import errors in the latter case then?
Yes. I'm not sure when that sentence was written, but you definitely don't want a SciPy without LAPACK - lots of things will not work. Cheers, Ralf
On 02/11/2018 00:11, Ralf Gommers wrote:
The decision to support f90 was taken before the 0.18 release in 2016 but I'm not sure what impact it had on the code base.
There has been very little new Fortran code added since then. The PR that gh-2829 linked to (odeint rewrite) was not merged in the end. I wouldn't be surprised if current master turns out to be f77 compliant.
That is very useful to know, thanks. Also thank you for confirming the other points, -- Roman
participants (2)
-
Ralf Gommers -
Roman Yurchak