eval_stats_regression

Source code: sensai/evaluation/eval_stats/eval_stats_regression.py

---

class RegressionMetric(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)

Bases: Metric[RegressionEvalStats], ABC

Parameters:

• name – the name of the metric; if None use the class’ name attribute

• bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

compute_value_for_eval_stats(eval_stats: RegressionEvalStats)

abstract compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

classmethod compute_errors(y_true: ndarray, y_predicted: ndarray)

classmethod compute_abs_errors(y_true: ndarray, y_predicted: ndarray)

name: str

class RegressionMetricMAE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)

Bases: RegressionMetric

Parameters:

• name – the name of the metric; if None use the class’ name attribute

• bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'MAE'

compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

class RegressionMetricMSE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)

Bases: RegressionMetric

Parameters:

• name – the name of the metric; if None use the class’ name attribute

• bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'MSE'

compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

class RegressionMetricRMSE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)

Bases: RegressionMetric

Parameters:

• name – the name of the metric; if None use the class’ name attribute

• bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'RMSE'

compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

class RegressionMetricRRSE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)

Bases: RegressionMetric

Parameters:

• name – the name of the metric; if None use the class’ name attribute

• bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'RRSE'

compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

class RegressionMetricR2(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)

Bases: RegressionMetric

Parameters:

• name – the name of the metric; if None use the class’ name attribute

• bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'R2'

compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

class RegressionMetricPCC(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)

Bases: RegressionMetric

Pearson’s correlation coefficient, aka Pearson’s R. This metric does not consider sample weights.

Parameters:

• name – the name of the metric; if None use the class’ name attribute

• bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'PCC'

compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

class RegressionMetricStdDevAE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)

Bases: RegressionMetric

The standard deviation of the absolute error. This metric does not consider sample weights.

Parameters:

• name – the name of the metric; if None use the class’ name attribute

• bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'StdDevAE'

compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

class RegressionMetricMedianAE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)

Bases: RegressionMetric

The median absolute error. This metric does not consider sample weights.

Parameters:

• name – the name of the metric; if None use the class’ name attribute

• bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'MedianAE'

compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

class RegressionMetricFromBinaryClassificationMetric(classification_metric: BinaryClassificationMetric, class_generator: ClassGenerator)

Bases: RegressionMetric

Supports the computation of binary classification metrics by converting predicted/target values to class labels. This metric does not consider sample weights.

Parameters:

• classification_metric – the classification metric (which shall consider True as the positive label)

• class_generator – the class generator, which generates True and False labels from regression values

class ClassGenerator

Bases: ABC

abstract compute_class(predicted_value: float) → bool

Computes the class from the given value

Parameters:

predicted_value – the value predicted by the regressor or regressor target value

Returns:

the class

abstract get_metric_qualifier() → str

Returns:

A (short) string which will be added to the original classification metric’s name to represent the class conversion logic

class ClassGeneratorPositiveBeyond(min_value_for_positive: float)

Bases: ClassGenerator

compute_class(predicted_value: float) → bool

Computes the class from the given value

Parameters:

predicted_value – the value predicted by the regressor or regressor target value

Returns:

the class

get_metric_qualifier() → str

Returns:

A (short) string which will be added to the original classification metric’s name to represent the class conversion logic

compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[ndarray] = None)

name: str

class HeatMapColorMapFactory

Bases: ABC

abstract create_color_map(min_sample_weight: float, total_weight: float, num_quantization_levels: int)

class HeatMapColorMapFactoryWhiteToRed

Bases: HeatMapColorMapFactory

create_color_map(min_sample_weight: float, total_weight: float, num_quantization_levels: int)

class RegressionEvalStats(y_predicted: Optional[Union[ndarray, Series, DataFrame, list]] = None, y_true: Optional[Union[ndarray, Series, DataFrame, list]] = None, metrics: Optional[Sequence[RegressionMetric]] = None, additional_metrics: Optional[Sequence[RegressionMetric]] = None, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None, weights: Optional[Union[ndarray, Series, list]] = None)

Bases: PredictionEvalStats[RegressionMetric]

Collects data for the evaluation of predicted continuous values and computes corresponding metrics

Parameters:

• y_predicted – the predicted values

• y_true – the true values

• metrics – the metrics to compute for evaluation; if None, will use DEFAULT_REGRESSION_METRICS

• additional_metrics – the metrics to additionally compute

• weights – optional data point weights

HEATMAP_COLORMAP_FACTORY = <sensai.evaluation.eval_stats.eval_stats_regression.HeatMapColorMapFactoryWhiteToRed object>

HEATMAP_DIAGONAL_COLOR = 'green'

HEATMAP_ERROR_BOUNDARY_VALUE = None

HEATMAP_ERROR_BOUNDARY_COLOR = (0.8, 0.8, 0.8)

SCATTER_PLOT_POINT_COLOR = (0, 0, 1, 0.05)

compute_metric_value(metric: RegressionMetric) → float

compute_mse()

Computes the mean squared error (MSE)

compute_rrse()

Computes the root relative squared error

compute_pcc()

Gets the Pearson correlation coefficient (PCC)

compute_r2()

Gets the R^2 score

compute_mae()

Gets the mean absolute error

compute_rmse()

Gets the root mean squared error

compute_std_dev_ae()

Gets the standard deviation of the absolute error

create_eval_stats_collection() → RegressionEvalStatsCollection

For the case where we collected data on multiple dimensions, obtain a stats collection where each object in the collection holds stats on just one dimension

plot_error_distribution(bins='auto', title_add=None) → Optional[Figure]

Parameters:

• bins – bin specification (see HistogramPlot)

• title_add – a string to add to the title (on a second line)

Returns:

the resulting figure object or None

plot_scatter_ground_truth_predictions(figure=True, title_add=None, **kwargs) → Optional[Figure]

Parameters:

• figure – whether to plot in a separate figure and return that figure

• title_add – a string to be added to the title in a second line

• kwargs – parameters to be passed on to plt.scatter()

Returns:

the resulting figure object or None

plot_heatmap_ground_truth_predictions(figure=True, cmap=None, bins=60, title_add=None, error_boundary: Optional[float] = None, weighted: bool = False, ax: Optional[Axes] = None, **kwargs) → Optional[Figure]

Parameters:

• figure – whether to create a new figure and return that figure (only applies if ax is None)

• cmap – the colour map to use (see corresponding parameter of plt.imshow for further information); if None, use factory defined in HEATMAP_COLORMAP_FACTORY (which can be centrally set to achieve custom behaviour throughout an application)

• bins – how many bins to use for constructing the heatmap

• title_add – a string to add to the title (on a second line)

• error_boundary – if not None, add two lines (above and below the diagonal) indicating this absolute regression error boundary; if None (default), use static member HEATMAP_ERROR_BOUNDARY_VALUE (which is also None by default, but can be centrally set to achieve custom behaviour throughout an application)

• weighted – whether to consider data point weights

• ax – the axis to plot in. If None, use the current axes (which will be the axis of the newly created figure if figure=True).

• kwargs – will be passed to plt.imshow()

Returns:

the newly created figure object (if figure=True) or None

class RegressionEvalStatsCollection(eval_stats_list: List[RegressionEvalStats])

Bases: EvalStatsCollection[RegressionEvalStats, RegressionMetric]

get_combined_eval_stats() → RegressionEvalStats

Returns:

an EvalStats object that combines the data from all contained EvalStats objects

class RegressionEvalStatsPlot(*args, **kwds)

Bases: EvalStatsPlot[RegressionEvalStats], ABC

class RegressionEvalStatsPlotErrorDistribution(*args, **kwds)

Bases: RegressionEvalStatsPlot

create_figure(eval_stats: RegressionEvalStats, subtitle: str) → Figure

Parameters:

• eval_stats – the evaluation stats from which to generate the plot

• subtitle – the plot’s subtitle

Returns:

the figure or None if this plot is not applicable/cannot be created

class RegressionEvalStatsPlotHeatmapGroundTruthPredictions(weighted: bool = False)

Bases: RegressionEvalStatsPlot

is_applicable(eval_stats: RegressionEvalStats) → bool

create_figure(eval_stats: RegressionEvalStats, subtitle: str) → Figure

Parameters:

• eval_stats – the evaluation stats from which to generate the plot

• subtitle – the plot’s subtitle

Returns:

the figure or None if this plot is not applicable/cannot be created

class RegressionEvalStatsPlotScatterGroundTruthPredictions(*args, **kwds)

Bases: RegressionEvalStatsPlot

create_figure(eval_stats: RegressionEvalStats, subtitle: str) → Figure

Parameters:

• eval_stats – the evaluation stats from which to generate the plot

• subtitle – the plot’s subtitle

Returns:

the figure or None if this plot is not applicable/cannot be created