[scikit-learn] scikit-learn 0.24 installation fails with ModuleNotFoundError: No module named 'scipy'
Guillaume Lemaître
g.lemaitre58 at gmail.com
Fri Jan 22 09:04:32 EST 2021
OK, so the normal install is working. Now, to fix your issue we need to
understand how `sc.install_pypi_package` is working and mainly how does it
call `pip`. We need to make sure that it call the right pip (the system
`pip3` in your case).
On Fri, 22 Jan 2021 at 14:39, Bertrand B. <bertrand25mtl at gmail.com> wrote:
> Thank you Guillaume for your help,
>
> I am using : (running on AWS EMR-6.2)
> pip3 --version
> pip 9.0.3 from /usr/lib/python3.7/site-packages (python 3.7)
>
>
> pip3 install scikit-learn
>
> Collecting scikit-learn
> Using cached
> https://files.pythonhosted.org/packages/f4/7b/d415b0c89babf23dcd8ee631015f043e2d76795edd9c7359d6e63257464b/scikit-learn-0.24.1.tar.gz
> Requirement already satisfied: numpy>=1.13.3 in
> /usr/local/lib64/python3.7/site-packages (from scikit-learn)
> Collecting scipy>=0.19.1 (from scikit-learn)
> Using cached
> https://files.pythonhosted.org/packages/58/9d/8296d8211318d690119eba6d293b7a149c1c51c945342dd4c3816f79e1ba/scipy-1.6.0-cp37-cp37m-manylinux1_x86_64.whl
> Requirement already satisfied: joblib>=0.11 in
> /usr/local/lib64/python3.7/site-packages (from scikit-learn)
> Requirement already satisfied: threadpoolctl>=2.0.0 in
> /usr/local/lib/python3.7/site-packages (from scikit-learn)
> Installing collected packages: scipy, scikit-learn
> Running setup.py install for scikit-learn ... error
> Complete output from command /usr/bin/python3 -u -c "import
> setuptools,
> tokenize;__file__='/mnt/tmp/pip-build-93pagltp/scikit-learn/setup.py';f=getattr(tokenize,
> 'open', open)(__file__);code=f.read().replace('\r\n',
> '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
> /tmp/pip-0ulalx36-record/install-record.txt
> --single-version-externally-managed --compile:
> Partial import of sklearn during the build process.
> Traceback (most recent call last):
> File
> "/mnt/tmp/pip-build-93pagltp/scikit-learn/sklearn/_build_utils/__init__.py",
> line 27, in _check_cython_version
> import Cython
> ModuleNotFoundError: No module named 'Cython'
>
>
> Upgrading pip to 20.3.3 :
>
> sudo pip3 install --upgrade pip
> sudo ln -s /usr/local/bin/pip3 /usr/bin/pip3
>
> pip3 --version
> pip 20.3.3 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
>
> let me install from the whl file :
> pip3 install scikit-learn
> Collecting scikit-learn
> Downloading scikit_learn-0.24.1-cp37-cp37m-manylinux2010_x86_64.whl
> (22.3 MB)
>
> However, using the API sc.install_pypi_package("scikit-learn") still uses
> the tar file instead of the whl file (even after the pip upgrade).
>
> Collecting scikit-learn
> Using cached https://files.pythonhosted.org/packages/f4/7b/d415b0c89babf23dcd8ee631015f043e2d76795edd9c7359d6e63257464b/scikit-learn-0.24.1.tar.gz
>
>
> Thanks for your help,
>
> Cheers,
>
> Bertrand
>
> Le ven. 22 janv. 2021 à 04:13, Guillaume Lemaître <g.lemaitre58 at gmail.com>
> a écrit :
>
>> @Bertrand Could you tell us which version of `pip` to you use (you need
>> pip >= 19.0 for manylinux2010 and pip >= 19.3 for manylinux2014)
>>
>> On Fri, 22 Jan 2021 at 09:49, Guillaume Lemaître <g.lemaitre58 at gmail.com>
>> wrote:
>>
>>> We might experience an issue with PyPI not selecting the manylinux2010
>>> wheel: https://github.com/scikit-learn/scikit-learn/issues/19233
>>> We have to check but we will probably shortly upload manylinux1 wheels
>>> that should resolve the issue.
>>>
>>> I am curious if fetching the wheel by hand and installing via `pip`
>>> would be a workaround (not practical for automated usage thought).
>>>
>>> On Thu, 21 Jan 2021 at 00:34, The Helmbolds via scikit-learn <
>>> scikit-learn at python.org> wrote:
>>>
>>>> Use the Anaconda Python installation.
>>>>
>>>> "You won't find the right answers if you don't ask the right
>>>> questions!" (Robert Helmbold, 2013)
>>>>
>>>>
>>>> On Wednesday, January 20, 2021, 04:16:15 PM MST, Guillaume Lemaître <
>>>> g.lemaitre58 at gmail.com> wrote:
>>>>
>>>>
>>>> Basically it get the tar with the source and recompile instead of using
>>>> the wheel. Could you force an install from PyPI without using the cached
>>>> file.
>>>>
>>>> We pushed wheels yesterday for 0.24.1 as well so it should not get the
>>>> 0.24.0 version.
>>>>
>>>> For 0.23.2, you can see that it used the wheel (.whl).
>>>>
>>>> Sent from my phone - sorry to be brief and potential misspell.
>>>> *From:* bertrand25mtl at gmail.com
>>>> *Sent:* 20 January 2021 23:21
>>>> *To:* scikit-learn at python.org
>>>> *Reply to:* scikit-learn at python.org
>>>> *Subject:* [scikit-learn] scikit-learn 0.24 installation fails with
>>>> ModuleNotFoundError: No module named 'scipy'
>>>>
>>>> To whom it may concern,
>>>>
>>>> I am trying to install scikit-learn in a PySpark job using the
>>>> install_pypi_package PySpark API but the install fails with :
>>>>
>>>> sc.install_pypi_package("scikit-learn")
>>>>
>>>> Collecting scikit-learn
>>>> Using cached https://files.pythonhosted.org/packages/db/e2/9c0bde5f81394b627f623557690536b12017b84988a4a1f98ec826edab9e/scikit-learn-0.24.0.tar.gz
>>>> Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib64/python3.7/site-packages (from scikit-learn)
>>>> Collecting scipy>=0.19.1 (from scikit-learn)
>>>> Using cached https://files.pythonhosted.org/packages/58/9d/8296d8211318d690119eba6d293b7a149c1c51c945342dd4c3816f79e1ba/scipy-1.6.0-cp37-cp37m-manylinux1_x86_64.whl
>>>> Requirement already satisfied: joblib>=0.11 in /usr/local/lib64/python3.7/site-packages (from scikit-learn)
>>>> Collecting threadpoolctl>=2.0.0 (from scikit-learn)
>>>> Using cached https://files.pythonhosted.org/packages/f7/12/ec3f2e203afa394a149911729357aa48affc59c20e2c1c8297a60f33f133/threadpoolctl-2.1.0-py3-none-any.whl
>>>> Building wheels for collected packages: scikit-learn
>>>> Running setup.py bdist_wheelfor scikit-learn: started
>>>> Running setup.py bdist_wheelfor scikit-learn: finished with status 'error'
>>>> Complete output from command /tmp/1611000009300-0/bin/python -u -c "import setuptools, tokenize;__file__='/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py ';f=getattr(tokenize, 'open', open)(__file__);code=f.read ().replace('\r\n', '\n');f.close ();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpry3gf9r0pip-wheel- --python-tag cp37:
>>>> Partial import of sklearn during the build process.
>>>> Traceback (most recent call last):
>>>> File "/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py ", line 201, in check_package_status
>>>> module = importlib.import_module(package)
>>>> File "/tmp/1611000009300-0/lib64/python3.7/importlib/__init__.py", line 127, in import_module
>>>> return _bootstrap._gcd_import(name[level:], package, level)
>>>> File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
>>>> File "<frozen importlib._bootstrap>", line 983, in _find_and_load
>>>> File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
>>>> ModuleNotFoundError: No module named 'scipy'
>>>> Traceback (most recent call last):
>>>> File "<string>", line 1, in <module>
>>>> File "/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py ", line 306, in <module>
>>>> setup_package()
>>>> File "/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py ", line 294, in setup_package
>>>> check_package_status('scipy', min_deps.SCIPY_MIN_VERSION)
>>>> File "/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py ", line 227, in check_package_status
>>>> .format(package, req_str, instructions))
>>>> ImportError: scipy is not installed.
>>>> scikit-learn requires scipy >= 0.19.1.
>>>>
>>>> I do not encounter this error with scikit-learn 0.23.2 :
>>>>
>>>> sc.install_pypi_package("scikit-learn==0.23.2")
>>>>
>>>> Collecting scikit-learn==0.23.2
>>>> Using cached https://files.pythonhosted.org/packages/f4/cb/64623369f348e9bfb29ff898a57ac7c91ed4921f228e9726546614d63ccb/scikit_learn-0.23.2-cp37-cp37m-manylinux1_x86_64.whl
>>>> Requirement already satisfied: scipy>=0.19.1 in /mnt/tmp/1611000009300-0/lib/python3.7/site-packages (from scikit-learn==0.23.2)
>>>> Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib64/python3.7/site-packages (from scikit-learn==0.23.2)
>>>> Requirement already satisfied: joblib>=0.11 in /usr/local/lib64/python3.7/site-packages (from scikit-learn==0.23.2)
>>>> Requirement already satisfied: threadpoolctl>=2.0.0 in /mnt/tmp/1611000009300-0/lib/python3.7/site-packages (from scikit-learn==0.23.2)
>>>> Installing collected packages: scikit-learn
>>>> Successfully installed scikit-learn-0.23.2
>>>>
>>>>
>>>> Could you please help me understand why the scikit-learn 0.24
>>>> installation fails ?
>>>>
>>>> Thank you for your help,
>>>>
>>>> Bertrand
>>>> _______________________________________________
>>>> scikit-learn mailing list
>>>> scikit-learn at python.org
>>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>>> _______________________________________________
>>>> scikit-learn mailing list
>>>> scikit-learn at python.org
>>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>>>
>>>
>>>
>>> --
>>> Guillaume Lemaitre
>>> Scikit-learn @ Inria Foundation
>>> https://glemaitre.github.io/
>>>
>>
>>
>> --
>> Guillaume Lemaitre
>> Scikit-learn @ Inria Foundation
>> https://glemaitre.github.io/
>> _______________________________________________
>> scikit-learn mailing list
>> scikit-learn at python.org
>> https://mail.python.org/mailman/listinfo/scikit-learn
>>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
--
Guillaume Lemaitre
Scikit-learn @ Inria Foundation
https://glemaitre.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scikit-learn/attachments/20210122/83d26c99/attachment-0001.html>
More information about the scikit-learn
mailing list