Validation

stanscofi.validation.AP(y_true, y_pred, u, u1)

stanscofi.validation.AUC(y_true, y_pred, k, u1)

stanscofi.validation.DCGk(y_true, y_pred, k, u1)

stanscofi.validation.ERR(y_true, y_pred, max=10, max_grade=2)

source: https://raw.githubusercontent.com/skondo/evaluation_measures/master/evaluations_measures.py

stanscofi.validation.F1K(y_true, y_pred, k, u1)

stanscofi.validation.Fscore(y_true, y_pred, u, beta)

stanscofi.validation.HRk(y_true, y_pred, k, u1)

stanscofi.validation.MAP(y_true, y_pred, u, u1)

stanscofi.validation.MRR(y_true, y_pred, u, u1)

stanscofi.validation.MeanRank(y_true, y_pred, k, u1)

stanscofi.validation.NDCGk(y_true, y_pred, k, u1)

stanscofi.validation.PrecisionK(y_true, y_pred, k, u1)

stanscofi.validation.RP(y_true, y_pred, u, u1)

stanscofi.validation.RecallK(y_true, y_pred, k, u1)

stanscofi.validation.Rscore(y_true, y_pred, u, u1)

stanscofi.validation.TAU(y_true, y_pred, u, u1)

stanscofi.validation.compute_metrics(scores, predictions, dataset, metrics, k=1, beta=1, verbose=False)

Computes user-wise validation metrics for a given set of scores and predictions w.r.t. a dataset

…

Parameters

scoresCOO-array of shape (n_items, n_users)

sparse matrix in COOrdinate format

predictionsCOO-array of shape (n_items, n_users)

sparse matrix in COOrdinate format with values in {-1,1}

datasetstanscofi.Dataset

dataset on which the metrics should be computed

metricslst of str

list of metrics which should be computed

kint (default: 1)

Argument of the metric to optimize. Implemented metrics are in validation.py

betafloat (default: 1)

Argument of the metric to optimize. Implemented metrics are in validation.py

verbosebool

prints out information about ignored users for the computation of validation metrics, that is, users which pairs are only associated to a single class (i.e., all pairs with this users are either assigned 0, -1 or 1)

Returns

metricspandas.DataFrame of shape (len(metrics), 2)

table of metrics: metrics in rows, average and standard deviation across users in columns

plots_argsdict

dictionary of arguments to feed to the plot_metrics function to plot the Precision-Recall and the Receiver Operating Chracteristic (ROC) curves

stanscofi.validation.plot_metrics(y_true=None, y_pred=None, scores=None, ground_truth=None, predictions=None, aucs=None, fscores=None, tprs=None, recs=None, figsize=(16, 5), model_name='Model')

Plots the ROC curve, the Precision-Recall curve, the boxplot of predicted scores and the piechart of classes associated to the predictions y_pred in input w.r.t. ground truth y_true

…

Parameters

y_truearray-like of shape (n_ratings,)

an array which contains the binary ground truth labels in {0,1}

y_predarray-like of shape (n_ratings,)

an array which contains the binary predicted labels in {0,1}

scoresarray-like of shape (n_ratings,)

an array which contains the predicted scores

ground_trutharray-like of shape (n_ratings,)

an array which contains the ground truth labels in {-1,0,1}

predictionsarray-like of shape (n_ratings,)

an array which contains the predicted labels in {-1,0,1}

aucslist

list of AUCs per user

fscoreslist

list of F-scores per user

tprsarray-like of shape (n_thresholds,)

Increasing true positive rates such that element i is the true positive rate of predictions with score >= thresholds[i], where thresholds was given as input to sklearn.metrics.roc_curve

recsarray-like of shape (n_thresholds,)

Decreasing recall values such that element i is the recall of predictions with score >= thresholds[i] and the last element is 0, where thresholds was given as input to sklearn.metrics.precision_recall_curve

figsizetuple of size 2

width and height of the figure

model_namestr

model which predicted the ratings

Returns

metricspandas.DataFrame of shape (2, 2)

table of metrics: AUC, F_beta score in rows, average and standard deviation across users in columns

plots_argsdict

dictionary of arguments to feed to the plot_metrics function