pickle#

Source code: sensai/util/pickle.py

load_pickle(path: Union[str, Path], backend='pickle')[source]#
dump_pickle(obj, pickle_path: Union[str, Path], backend='pickle', protocol=5)[source]#
class PickleFailureDebugger[source]#

Bases: object

A collection of methods for testing whether objects can be pickled and logging useful infos in case they cannot

enabled = False#
classmethod debug_failure(obj) → List[str][source]#

Recursively tries to pickle the given object and returns a list of failed paths

Parameters:

obj – the object for which to recursively test pickling

Returns:

a list of object paths that failed to pickle

classmethod log_failure_if_enabled(obj, context_info: Optional[str] = None)[source]#

If the class flag ‘enabled’ is set to true, the pickling of the given object is recursively tested and the results are logged at error level if there are problems and info level otherwise. If the flag is disabled, no action is taken.

Parameters:

  • obj – the object for which to recursively test pickling

  • context_info – optional additional string to be included in the log message

class PersistableObject[source]#

Bases: object

Base class which can be used for objects that shall support being persisted via pickle.

IMPORTANT: The implementations correspond to the default behaviour of pickle for the case where an object has a non-empty set of attributes. However, for the case where the set of attributes can be empty adding the explicit implementation of __getstate__ is crucial in ensuring that __setstate__ will be called upon unpickling. So if an object initially has no attributes and is persisted in that state, then any future refactorings cannot be handled via __setstate__ by default, but they can when using this class.

setstate(cls, obj, state: Dict[str, Any], renamed_properties: Optional[Dict[str, Union[str, Tuple[str, Callable[[Dict[str, Any]], Any]]]]] = None, new_optional_properties: Optional[List[str]] = None, new_default_properties: Optional[Dict[str, Any]] = None, removed_properties: Optional[List[str]] = None) → None[source]#

Helper function for safe implementations of __setstate__ in classes, which appropriately handles the cases where a parent class already implements __setstate__ and where it does not. Call this function whenever you would actually like to call the super-class’ implementation. Unfortunately, __setstate__ is not implemented in object, rendering super().__setstate__(state) invalid in the general case.

Parameters:

  • cls – the class in which you are implementing __setstate__

  • obj – the instance of cls

  • state – the state dictionary

  • renamed_properties – can be used for renaming as well as for assigning new values. If passed must map an old property name to either a new property name or to tuple of a new property name and a function that computes the new value from the state dictionary.

  • new_optional_properties – a list of names of new property names, which, if not present, shall be initialised with None

  • new_default_properties – a dictionary mapping property names to their default values, which shall be added if they are not present

  • removed_properties – a list of names of properties that are no longer being used

getstate(cls, obj, transient_properties: Optional[Iterable[str]] = None, excluded_properties: Optional[Iterable[str]] = None, override_properties: Optional[Dict[str, Any]] = None, excluded_default_properties: Optional[Dict[str, Any]] = None) → Dict[str, Any][source]#

Helper function for safe implementations of __getstate__ in classes, which appropriately handles the cases where a parent class already implements __getstate__ and where it does not. Call this function whenever you would actually like to call the super-class’ implementation. Unfortunately, __getstate__ is not implemented in object, rendering super().__getstate__() invalid in the general case.

Parameters:

  • cls – the class in which you are implementing __getstate__

  • obj – the instance of cls

  • transient_properties – transient properties which be set to None in serialisations

  • excluded_properties – properties which shall be completely removed from serialisations

  • override_properties – a mapping from property names to values specifying (new or existing) properties which are to be set; use this to set a fixed value for an existing property or to add a completely new property

  • excluded_default_properties – properties which shall be completely removed from serialisations, if they are set to the given default value

Returns:

the state dictionary, which may be modified by the receiver