io_data#

Source code: sensai/data/io_data.py

class BaseInputOutputData(inputs: T, outputs: T)[source]#

Bases: Generic[T], ABC

Parameters:

  • inputs – expected to have shape and __len__

  • outputs – expected to have shape and __len__

abstract filter_indices(indices: Sequence[int]) → BaseInputOutputData[source]#
class InputOutputArrays(inputs: ndarray, outputs: ndarray)[source]#

Bases: BaseInputOutputData[ndarray]

Parameters:

  • inputs – expected to have shape and __len__

  • outputs – expected to have shape and __len__

filter_indices(indices: Sequence[int]) → InputOutputArrays[source]#
to_torch_data_loader(batch_size=64, shuffle=True)[source]#
class InputOutputData(inputs: DataFrame, outputs: DataFrame)[source]#

Bases: BaseInputOutputData[DataFrame], ToStringMixin

Holds input and output data for learning problems

Parameters:

  • inputs – expected to have shape and __len__

  • outputs – expected to have shape and __len__

classmethod from_data_frame(df: DataFrame, *output_columns: str) → InputOutputData[source]#
Parameters:

  • df – a data frame containing both input and output columns

  • output_columns – the output column name(s)

Returns:

an InputOutputData instance with inputs and outputs separated

filter_indices(indices: Sequence[int]) → InputOutputData[source]#
filter_index(index_elements: Sequence[any]) → InputOutputData[source]#
property input_dim#
property output_dim#
compute_input_output_correlation()[source]#
to_df() → DataFrame[source]#
class DataSplitter(*args, **kwds)[source]#

Bases: ABC, Generic[TInputOutputData]

abstract split(data: TInputOutputData) → Tuple[TInputOutputData, TInputOutputData][source]#
class DataSplitterFractional(fractional_size_of_first_set: float, shuffle=True, random_seed=42)[source]#

Bases: DataSplitter

split_with_indices(data) → Tuple[Tuple[Sequence[int], Sequence[int]], Tuple[TInputOutputData, TInputOutputData]][source]#
split(data: TInputOutputData) → Tuple[TInputOutputData, TInputOutputData][source]#
class DataSplitterFromDataFrameSplitter(data_frame_splitter: DataFrameSplitter, fractional_size_of_first_set: float, apply_to_input=True)[source]#

Bases: DataSplitter[InputOutputData]

Creates a DataSplitter from a DataFrameSplitter, which can be applied either to the input or the output data. It supports only InputOutputData, not other subclasses of BaseInputOutputData.

Parameters:

  • data_frame_splitter – the splitter to apply

  • fractional_size_of_first_set – the desired fractional size of the first set when applying the splitter

  • apply_to_input – if True, apply the splitter to the input data frame; if False, apply it to the output data frame

split(data: InputOutputData) → Tuple[InputOutputData, InputOutputData][source]#
class DataSplitterFromSkLearnSplitter(sklearn_splitter)[source]#

Bases: DataSplitter

Parameters:

sklearn_splitter – an instance of one of the splitter classes from sklearn.model_selection, see https://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection

split(data: TInputOutputData) → Tuple[TInputOutputData, TInputOutputData][source]#
class DataSplitterStratifiedShuffleSplit(fractional_size_of_first_set: float, random_seed=42)[source]#

Bases: DataSplitterFromSkLearnSplitter

Parameters:

sklearn_splitter – an instance of one of the splitter classes from sklearn.model_selection, see https://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection

static is_applicable(io_data: InputOutputData)[source]#
class DataFrameSplitter[source]#

Bases: ABC

abstract compute_split_indices(df: DataFrame, fractional_size_of_first_set: float) → Tuple[Sequence[int], Sequence[int]][source]#
static split_with_indices(df: DataFrame, indices_pair: Tuple[Sequence[int], Sequence[int]]) → Tuple[DataFrame, DataFrame][source]#
split(df: DataFrame, fractional_size_of_first_set: float) → Tuple[DataFrame, DataFrame][source]#
class DataFrameSplitterFractional(shuffle=False, random_seed=42)[source]#

Bases: DataFrameSplitter

compute_split_indices(df: DataFrame, fractional_size_of_first_set: float) → Tuple[Sequence[int], Sequence[int]][source]#
class DataFrameSplitterColumnEquivalenceClass(column: str, shuffle=True, random_seed=42)[source]#

Bases: DataFrameSplitter

Performs a split that keeps together data points/rows that have the same value in a given column, i.e. with respect to that column, the items having the same values are viewed as a unit; they form an equivalence class, and all data points belonging to the same class are either in the first set or the second set.

The split is performed at the level of unique items in the column, i.e. the given fraction of equivalence classes will end up in the first set and the rest in the second set.

The list if unique items in the column can be shuffled before applying the split. If no shuffling is applied, the original order in the data frame is maintained, and if the items were grouped by equivalence class in the original data frame, the split will correspond to a fractional split without shuffling where the split boundary is adjusted to not separate an equivalence class.

Parameters:

  • column – the column which defines the equivalence classes (groups of data points/rows that must not be separated)

  • shuffle – whether to shuffle the list of unique values in the given column before applying the split

  • random_seed –

compute_split_indices(df: DataFrame, fractional_size_of_first_set: float) → Tuple[Sequence[int], Sequence[int]][source]#