eval_stats_regression#

Source code: sensai/evaluation/eval_stats/eval_stats_regression.py

class RegressionMetric(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)[source]#

Bases: Metric[RegressionEvalStats], ABC

Parameters:

  • name – the name of the metric; if None use the class’ name attribute

  • bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

compute_value_for_eval_stats(eval_stats: RegressionEvalStats, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
abstract classmethod compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
classmethod compute_errors(y_true: ndarray, y_predicted: ndarray)[source]#
classmethod compute_abs_errors(y_true: ndarray, y_predicted: ndarray)[source]#
name: str#
class RegressionMetricMAE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)[source]#

Bases: RegressionMetric

Parameters:

  • name – the name of the metric; if None use the class’ name attribute

  • bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'MAE'#
classmethod compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
class RegressionMetricMSE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)[source]#

Bases: RegressionMetric

Parameters:

  • name – the name of the metric; if None use the class’ name attribute

  • bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'MSE'#
classmethod compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
class RegressionMetricRMSE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)[source]#

Bases: RegressionMetric

Parameters:

  • name – the name of the metric; if None use the class’ name attribute

  • bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'RMSE'#
classmethod compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
class RegressionMetricRRSE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)[source]#

Bases: RegressionMetric

Parameters:

  • name – the name of the metric; if None use the class’ name attribute

  • bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'RRSE'#
classmethod compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
class RegressionMetricR2(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)[source]#

Bases: RegressionMetric

Parameters:

  • name – the name of the metric; if None use the class’ name attribute

  • bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'R2'#
classmethod compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
class RegressionMetricPCC(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)[source]#

Bases: RegressionMetric

Parameters:

  • name – the name of the metric; if None use the class’ name attribute

  • bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'PCC'#
classmethod compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
class RegressionMetricStdDevAE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)[source]#

Bases: RegressionMetric

Parameters:

  • name – the name of the metric; if None use the class’ name attribute

  • bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'StdDevAE'#
classmethod compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
class RegressionMetricMedianAE(name: Optional[str] = None, bounds: Optional[Tuple[float, float]] = None)[source]#

Bases: RegressionMetric

Parameters:

  • name – the name of the metric; if None use the class’ name attribute

  • bounds – the minimum and maximum values the metric can take on (or None if the bounds are not specified)

name: str = 'MedianAE'#
classmethod compute_value(y_true: ndarray, y_predicted: ndarray, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#
class RegressionEvalStats(y_predicted: Optional[Union[ndarray, Series, DataFrame, list]] = None, y_true: Optional[Union[ndarray, Series, DataFrame, list]] = None, metrics: Optional[Sequence[RegressionMetric]] = None, additional_metrics: Optional[Sequence[RegressionMetric]] = None, model: Optional[VectorRegressionModel] = None, io_data: Optional[InputOutputData] = None)[source]#

Bases: PredictionEvalStats[RegressionMetric]

Collects data for the evaluation of predicted continuous values and computes corresponding metrics

Parameters:

  • y_predicted – the predicted values

  • y_true – the true values

  • metrics – the metrics to compute for evaluation; if None, will use DEFAULT_REGRESSION_METRICS

  • additional_metrics – the metrics to additionally compute

HEATMAP_COLORMAP_FACTORY()#
HEATMAP_DIAGONAL_COLOR = 'green'#
HEATMAP_ERROR_BOUNDARY_VALUE = None#
HEATMAP_ERROR_BOUNDARY_COLOR = (0.8, 0.8, 0.8)#
SCATTER_PLOT_POINT_COLOR = (0, 0, 1, 0.05)#
compute_metric_value(metric: RegressionMetric) → float[source]#
compute_mse()[source]#

Computes the mean squared error (MSE)

compute_rrse()[source]#

Computes the root relative squared error

compute_pcc()[source]#

Gets the Pearson correlation coefficient (PCC)

compute_r2()[source]#

Gets the R^2 score

compute_mae()[source]#

Gets the mean absolute error

compute_rmse()[source]#

Gets the root mean squared error

compute_std_dev_ae()[source]#

Gets the standard deviation of the absolute error

create_eval_stats_collection() → RegressionEvalStatsCollection[source]#

For the case where we collected data on multiple dimensions, obtain a stats collection where each object in the collection holds stats on just one dimension

plot_error_distribution(bins='auto', title_add=None) → Optional[Figure][source]#
Parameters:

  • bins – bin specification (see HistogramPlot)

  • title_add – a string to add to the title (on a second line)

Returns:

the resulting figure object or None

plot_scatter_ground_truth_predictions(figure=True, title_add=None, **kwargs) → Optional[Figure][source]#
Parameters:

  • figure – whether to plot in a separate figure and return that figure

  • title_add – a string to be added to the title in a second line

  • kwargs – parameters to be passed on to plt.scatter()

Returns:

the resulting figure object or None

plot_heatmap_ground_truth_predictions(figure=True, cmap=None, bins=60, title_add=None, error_boundary: Optional[float] = None, **kwargs) → Optional[Figure][source]#
Parameters:

  • figure – whether to plot in a separate figure and return that figure

  • cmap – the colour map to use (see corresponding parameter of plt.imshow for further information); if None, use factory defined in HEATMAP_COLORMAP_FACTORY (which can be centrally set to achieve custom behaviour throughout an application)

  • bins – how many bins to use for constructing the heatmap

  • title_add – a string to add to the title (on a second line)

  • error_boundary – if not None, add two lines (above and below the diagonal) indicating this absolute regression error boundary; if None (default), use static member HEATMAP_ERROR_BOUNDARY_VALUE (which is also None by default, but can be centrally set to achieve custom behaviour throughout an application)

  • kwargs – will be passed to plt.imshow()

Returns:

the resulting figure object or None

class RegressionEvalStatsCollection(eval_stats_list: List[RegressionEvalStats])[source]#

Bases: EvalStatsCollection[RegressionEvalStats, RegressionMetric]

get_combined_eval_stats() → RegressionEvalStats[source]#
Returns:

an EvalStats object that combines the data from all contained EvalStats objects

class RegressionEvalStatsPlot(*args, **kwds)[source]#

Bases: EvalStatsPlot[RegressionEvalStats], ABC

class RegressionEvalStatsPlotErrorDistribution(*args, **kwds)[source]#

Bases: RegressionEvalStatsPlot

create_figure(eval_stats: RegressionEvalStats, subtitle: str) → Figure[source]#
Parameters:

  • eval_stats – the evaluation stats from which to generate the plot

  • subtitle – the plot’s subtitle

Returns:

the figure or None if this plot is not applicable/cannot be created

class RegressionEvalStatsPlotHeatmapGroundTruthPredictions(*args, **kwds)[source]#

Bases: RegressionEvalStatsPlot

create_figure(eval_stats: RegressionEvalStats, subtitle: str) → Figure[source]#
Parameters:

  • eval_stats – the evaluation stats from which to generate the plot

  • subtitle – the plot’s subtitle

Returns:

the figure or None if this plot is not applicable/cannot be created

class RegressionEvalStatsPlotScatterGroundTruthPredictions(*args, **kwds)[source]#

Bases: RegressionEvalStatsPlot

create_figure(eval_stats: RegressionEvalStats, subtitle: str) → Figure[source]#
Parameters:

  • eval_stats – the evaluation stats from which to generate the plot

  • subtitle – the plot’s subtitle

Returns:

the figure or None if this plot is not applicable/cannot be created