vector_model

vector_model#

This module defines base classes for models that use pandas.DataFrames for inputs and outputs, where each data frame row represents a single model input or output. Since every row contains a vector of data (one-dimensional array), we refer to them as vector-based models. Hence the name of the module and of the central base class VectorModel.

class VectorModelBase[source]#

Bases: ABC, ToStringMixin

Base class for vector models, which defines the fundamental prediction interface. A vector model takes data frames as input, where each row represents a vector of information.

abstract predict(x: DataFrame) → DataFrame[source]#

abstract is_regression_model() → bool[source]#

abstract get_predicted_variable_names() → list[source]#

with_name(name: str) → TVectorModelBase[source]#

Sets the model’s name.

Parameters:: name – the name
Returns:: self

set_name(name)[source]#

get_name()[source]#

class VectorModelFittableBase[source]#

Bases: VectorModelBase, ABC

Base class for vector models, which encompasses the fundamental prediction and fitting interfaces. A vector model takes data frames as input, where each row represents a vector of information.

abstract fit(x: DataFrame, y: DataFrame)[source]#

abstract is_fitted() → bool[source]#

class TrainingContext(original_input: DataFrame, original_output: DataFrame)[source]#

Bases: object

Contains context information for an ongoing training process

class VectorModel(check_input_columns=True)[source]#

Bases: VectorModelFittableBase, PickleLoadSaveMixin, ABC

Represents a model which uses data frames as inputs and outputs whose rows define individual data points. Every data frame row represents a vector of information (one-dimensional array), hence the name of the model. Note that the vectors in question are not necessarily vectors in the mathematical sense, as the information in each cell is not required to be numeric or uniform but can be arbitrarily complex.

Parameters:: check_input_columns – whether to check if the input column list (that is fed to the underlying model, i.e. after feature generation) during inference coincides with the input column list that was observed during training. This should be disabled if feature generation is not performed by the model itself, e.g. in meta-models such as ensemble models.

TOSTRING_INCLUDE_PREPROCESSORS = True#

with_raw_input_transformers(*transformers: Union[DataFrameTransformer, List[DataFrameTransformer]]) → TVectorModel[source]#

Makes the model use the given transformers (removing previously set raw input transformers, if any), which are to be applied to the raw input data frame (prior to feature generation).

Parameters:: transformers – DataFrameTransformer instances to use (in sequence) for the transformation of inputs
Returns:: self

with_feature_transformers(*transformers: Union[DataFrameTransformer, List[DataFrameTransformer]], add=False) → TVectorModel[source]#

Makes the model use the given transformers which are to be applied to the data frames generated by feature generators. (If the model does not use feature generators, the transformers will be applied to whatever is produced by the raw input transformers or, if there are none, the original raw input data frame).

Parameters:

transformers – DataFrameTransformer instances to use (in sequence) for the transformation of features
add – whether to add the transformers to the existing transformers rather than replacing them

Returns:

self

with_input_transformers(*input_transformers: Union[DataFrameTransformer, List[DataFrameTransformer]]) → TVectorModel[source]#

Makes the model use the given feature transformers (removing previously set transformers, if any), i.e. it transforms the data frame that is generated by the feature generators (if any).

Parameters:: input_transformers – DataFrameTransformer instances to use (in sequence) for the transformation of inputs
Returns:: self

with_feature_generator(feature_generator: Optional[FeatureGenerator]) → TVectorModel[source]#

Makes the model use the given feature generator in order to obtain the model inputs. If the model shall use more than one feature generator, pass a MultiFeatureGenerator which combines them or use the perhaps more convenient FeatureCollector in conjunction with withFeatureCollector().

Note: Feature computation takes place before input transformation.

Parameters:: feature_generator – the feature generator to use for input computation
Returns:: self

with_feature_collector(feature_collector: FeatureCollector, shared: bool = False) → TVectorModel[source]#

Makes the model use a multi-feature generator obtained from the given collector in order compute the underlying model’s input from the data frame that is given. Overrides any feature generator previously passed to withFeatureGenerator() (if any).

Note: Feature generation takes place before feature transformation.

Parameters:

feature_collector – the feature collector from which to obtain the multi-feature generator
shared – whether the given feature collector is shared between models (i.e. whether the same instance is passed to multiple models). Passing shared=True ensures that models using the same collector do not end up using the same multi-feature collector instance and instead receive an independent instance.

Returns:

self

is_fitted()[source]#

Returns:: True if the model has been fitted, False otherwise

compute_model_inputs(x: DataFrame)[source]#

Applies feature generators and input transformers (if any) to generate from an input data frame the input for the underlying model

Parameters:: x – the input data frame, to which input preprocessing is to be applied
Returns:: the input data frame that serves as input for the underlying model

compute_model_outputs(y: DataFrame) → DataFrame[source]#

predict(x: DataFrame) → DataFrame[source]#

Applies the model to the given input data frame

Parameters:: x – the input data frame
Returns:: the model outputs in the form of a data frame whose index corresponds to the index of x

fit_input_output_data(io_data: InputOutputData, fit_preprocessors=True, fit_model=True)[source]#

Fits the model using the given data

Parameters:

io_data – the input/output data
fit_preprocessors – whether the model’s preprocessors (feature generators and data frame transformers) shall be fitted
fit_model – whether the model itself shall be fitted

fit(x: DataFrame, y: Optional[DataFrame], fit_preprocessors=True, fit_model=True)[source]#

Fits the model using the given data

Parameters:

x – a data frame containing input data
y – a data frame containing output data; may be None if the underlying model does not actually require fitting, e.g. in the case of a rule-based models, but fitting is still necessary for preprocessors
fit_preprocessors – whether the model’s preprocessors (feature generators and data frame transformers) shall be fitted
fit_model – whether the model itself shall be fitted

is_being_fitted() → bool[source]#

Returns:: True if the model is currently in the process of being fitted, False otherwise

get_predicted_variable_names()[source]#

Returns:: the list of variable names that are ultimately output by this model (i.e. the columns of the data frame output by predict())

get_model_input_variable_names() → Optional[List[str]][source]#

Returns:: the list of variable names required by the underlying model as input (after feature generation and data frame transformation) or None if the model has not been fitted (or is a rule-based model which does not determine the variable names).

get_input_transformer(cls: Type[DataFrameTransformer])[source]#

Gets the (first) feature transformer of the given type (if any) within this models feature transformer chain

Parameters:: cls – the type of transformer to look for
Returns:: the first matching feature transformer or None

get_feature_transformer(cls: Type[DataFrameTransformer])[source]#

Gets the (first) feature transformer of the given type (if any) within this models feature transformer chain

Parameters:: cls – the type of transformer to look for
Returns:: the first matching feature transformer or None

get_raw_input_transformer(cls: Type[DataFrameTransformer])[source]#

Gets the (first) raw input transformer of the given type (if any) within this models raw input transformer chain

Parameters:: cls – the type of transformer to look for
Returns:: the first matching raw input transformer or None

get_input_transformer_chain() → DataFrameTransformerChain[source]#

Returns:: the model’s feature transformer chain (which may be empty and contain no actual transformers), i.e. the transformers that are applied after feature generation

get_raw_input_transformer_chain() → DataFrameTransformerChain[source]#

Returns:: the model’s raw input transformer chain (which may be empty and contain no actual transformers), i.e. the transformers that are applied before feature generation

get_feature_transformer_chain() → DataFrameTransformerChain[source]#

Returns:: the model’s feature transformer chain (which may be empty and contain no actual transformers), i.e. the transformers that are applied after feature generation

set_feature_generator(feature_generator: Optional[FeatureGenerator])[source]#

get_feature_generator() → Optional[FeatureGenerator][source]#

Returns:: the model’s feature generator (if any)

remove_input_preprocessors()[source]#: Removes all input preprocessors (i.e. raw input transformers, feature generators and feature transformers) from the model

class VectorRegressionModel(check_input_columns=True)[source]#

Bases: VectorModel, ABC

Parameters:: check_input_columns – Whether to check if the input column list (after feature generation) during inference coincides with the input column list during fit. This should be disabled if feature generation is not performed by the model itself, e.g. in ensemble models.

is_regression_model() → bool[source]#

with_output_transformers(*output_transformers: Union[DataFrameTransformer, List[DataFrameTransformer]]) → TVectorRegressionModel[source]#

Makes the model use the given output transformers. Call with empty input to remove existing output transformers. The transformers are ignored during the fit phase. Not supported for rule-based models.

Important: The output columns names of the last output transformer should be the same as the first one’s input column names. If this fails to hold, an exception will be raised when predict() is called.

Note: Output transformers perform post-processing after the actual predictions have been made. Contrary to invertible target transformers, they are not invoked during the fit phase. Therefore, any losses computed there, including the losses on validation sets (e.g. for early stopping), will be computed on the non-post-processed data. A possible use case for such post-processing is if you know how improve the predictions of your fittable model by some heuristics or by hand-crafted rules.

How not to use: Output transformers are not meant to transform the predictions into something with a different semantic meaning (e.g. normalized into non-normalized or something like that) - you should consider using a targetTransformer for this purpose. Instead, they give the possibility to improve predictions through post processing, when this is desired.

Parameters:: output_transformers – DataFrameTransformers for the transformation of outputs (after the model has been applied)
Returns:: self

with_target_transformer(target_transformer: Optional[InvertibleDataFrameTransformer]) → TVectorRegressionModel[source]#

Makes the model use the given target transformers such that the underlying low-level model is trained on the transformed targets, but this high-level model still outputs the original (untransformed) values, i.e. the transformation is applied to targets during training and the inverse transformation is applied to the underlying model’s predictions during inference. Hence the requirement of the transformer being invertible.

This method is not supported for rule-based models, because they are not trained and therefore the transformation would serve no purpose.

NOTE: All feature generators and data frame transformers - should they make use of outputs - will be fit on the untransformed target. The targetTransformer only affects the fitting of the underlying model.

Parameters:: target_transformer – a transformer which transforms the targets (training data outputs) prior to learning the model, such that the model learns to predict the transformed outputs
Returns:: self

get_target_transformer()[source]#

get_output_transformer_chain()[source]#

predict(x: DataFrame) → DataFrame[source]#

Applies the model to the given input data frame

Parameters:: x – the input data frame
Returns:: the model outputs in the form of a data frame whose index corresponds to the index of x

is_fitted()[source]#

Returns:: True if the model has been fitted, False otherwise

get_model_output_variable_names()[source]#: Gets the list of variable names predicted by the underlying model. For the case where at training time the ground truth is transformed by a target transformer which changes column names, the names of the variables prior to the transformation will be returned. Thus this method always returns the variable names that are actually predicted by the underlying model alone. For the variable names that are ultimately output by the entire VectorModel instance when calling predict, use getPredictedVariableNames.

class VectorClassificationModel(check_input_columns=True)[source]#

Bases: VectorModel, ABC

Parameters:: check_input_columns – Whether to check if the input column list (after feature generation) during inference coincides with the input column list during fit. This should be disabled if feature generation is not performed by the model itself, e.g. in ensemble models.

is_regression_model() → bool[source]#

get_class_labels() → List[Any][source]#

convert_class_probabilities_to_predictions(df: DataFrame)[source]#

Converts from a data frame as returned by predictClassProbabilities to a result as return by predict.

Parameters:: df – the output data frame from predictClassProbabilities
Returns:: an output data frame as it would be returned by predict

predict_class_probabilities(x: DataFrame) → DataFrame[source]#

Parameters:: x – the input data
Returns:: a data frame where the list of columns is the list of class labels and the values are probabilities, with the same index as the input data frame. Raises an exception if the classifier cannot predict probabilities.

class RuleBasedVectorRegressionModel(predicted_variable_names: list)[source]#

Bases: VectorRegressionModel, ABC

Parameters:: predicted_variable_names – These are typically known at init time for rule-based models

class RuleBasedVectorClassificationModel(labels: list, predicted_variable_name='predictedLabel')[source]#

Bases: VectorClassificationModel, ABC

Parameters:

labels –
predicted_variable_name –