Hi sklearn developers ans users, The cohen_kappa_score calculates overall score for all classes. y_true = [0, 1, 2, 2, 2] y_pred = [0, 0, 2, 2, 1] target_names = ['class 0', 'class 1', 'class 2'] from sklearn.metrics import cohen_kappa_score print (cohen_kappa_score(y_true, y_pred)) 0.375 How can I calculate cohen_kappa_score class wise, like the classification_report does. I hope someone can help me. Thank you. Pavel G.