[scikit-learn] Scikit Learn in a Cray computer

[Mauricio Reis]

Finally I was able to access the Cray computer and run the <cpu_count> 
routine.

I am sending below the files and commands I used and the result found, 
where you can see "ncpus = 1" (I still do not know why 4 lines were 
printed - I only know that this amount depends on the value of the 
"aprun" command used in the file "ncpus.pbs"). But I do not know if you 
know the Cray computer environment and you'll understand what I did!

I use Cray XK7 computer which has 10 blades, each blade has 4 nodes 
(total of 40 nodes) and each node has 1 CPU and 1 GPU!

---
Ats.,
Mauricio Reis


----------------------------------------------------------------------------------------------
=== p.sh ===
#!/bin/bash
/usr/local/python_3.7/bin/python3.7 $1

=== ncpus.py ===
from sklearn.externals import joblib
import sklearn
print('The scikit-learn version is {}.'.format(sklearn.__version__))
ncpus = joblib.cpu_count()
print("--- ncpus =", ncpus)

=== ncpus.pbs ===
#!/bin/bash
#PBS -l select=1:ncpus=8:mpiprocs=8
#PBS -j oe
#PBS -l walltime=00:00:10

date

echo "[$PBS_O_WORKDIR]"
cd $PBS_O_WORKDIR

aprun -n 4 p.sh ./ncpus.py

=== command ===
qsub ncpus.pbs

=== output ===
Thu Jun 27 05:22:35 BRT 2019
[/home/reismc]
The scikit-learn version is 0.20.3.
The scikit-learn version is 0.20.3.
The scikit-learn version is 0.20.3.
The scikit-learn version is 0.20.3.
--- ncpus = 1
--- ncpus = 1
--- ncpus = 1
--- ncpus = 1
Application 32826 resources: utime ~8s, stime ~1s, Rss ~43168, inblocks 
~102981, outblocks ~0
----------------------------------------------------------------------------------------------


Em 19/06/2019 17:44, Olivier Grisel escreveu:
> How many cores du you have on this machine?
> 
> joblib.cpu_count()
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn