ModelResultSet
Model objects from a pipeline come back from the server organized into ModelResultSets and Configurations. A ModelResultSet represents everything returned by the pipeline and a Configuration is a subset pertaining to a single train-validate-optimize (TVO) configuration or combination. It is very common for a ModelResultSet to have a configuration containing many models. See the tutorials for more information about models, their metrics, and recognition capabilities.
- class sensiml.datamanager.modelresults.ModelMetrics(configuration, sandbox, index, model_result)
Base class for a model metrics object.
- confusion_matrix_stats
comprehensive metrics returned for the model
- Type
list[ConfusionMatrix]
- train_set
indices of the input data that the model was trained with
- Type
list
- validation_set
indices of the input data that the model was validated with
- Type
list
- test_set
indices of the input data that the model was tested with
- Type
list
- debug
structure containing debug information for some models
- Type
dict
- neurons
model neuron array
- Type
list[dict]
- parameters
model parameters
- Type
dict
- knowledgepack
knowledgepack associated with the model
- Type
- property knowledgepack: KnowledgePack
The model’s KnowledgePack object
- recognize_signal(capturefile: Optional[str] = None, stop_step: Optional[int] = False, datafile: Optional[str] = None, segmenter: Optional[Union[Segmenter, bool]] = True, lock: bool = True, silent: bool = True, platform: str = 'emulator', compare_labels: bool = False, **kwargs) DataFrame
Sends a DataFrame of raw signals to be run through the feature generation pipeline and recognized.
- Parameters
capturefile (str) – The name of a file uploaded through the data capture lab
datafile (str) – The name of an uploading datafile
platform (str) – “emulator” or “cloud”. The “emulator” will run compiled c code giving device exact results, the cloud runs similarly to training providing more flexibility in returning early results by setting the stop step.
stop_step (int) – for debugging, if you want to stop the pipeline at a particular step, set stop_step to its index
compare_labels (bool) – If there are labels for the input dataframe, use them to create a confusion matrix
segmenter (bool or FunctionCall) – to suppress or override the segmentation algorithm in the original pipeline, set this to False or a function call of type ‘segmenter’ (defaults to True)
lock (bool, True) – If True, waits for the result to return before releasing the ipython cell.
- Returns
a dataframe containing the results of recognition and a dictionary containing the execution summary and the confusion_matrix when labels are provided.
execution_summary: a summary of which steps ran in the execution engine
confusion_matrix: a confusion matrix, only if the input data has a ‘Label’ column
- Return type
(DataFrame, dict)
- recognize_vectors(vectors: DataFrame) DataFrame
Sends a DataFrame of feature vectors to the model’s KnowledgePack for recognition.
- Parameters
vectors (DataFrame) – where each row is a feature vector with column headings named the same as
pipeline (the features generated by the) –
included (and label columns may be) –
- Returns
contains the results of recognition, including predicted class, neuron ID, and distance
- Return type
(DataFrame)
- summarize(metrics_set: str = 'validation')
Prints a formatted summary of the model metrics
- Parameters
metrics_set (optional[str]) – options are ‘validation’ (the default), ‘test’, and ‘train’
- class sensiml.datamanager.modelresults.ModelResultSet(project: Project, sandbox: Sandbox)
Base class for a model result set object.
- get_knowledgepack_by_index(config_index: int, model_index: int)
Returns the KnowledgePack of the given configuration index and model index.
- sort_models(metric: str = 'accuracy', metrics_set: str = 'validation')
Sorts the models within all configurations by a given metric or property.
- Parameters
metric (optional[str]) – options are ‘accuracy’ (the default), ‘index’, and ‘number_of_neurons’
metrics_set (optional[str]) – options are ‘validation’ (the default), ‘test’, and ‘train’
- summarize(metrics_set: str = 'validation', report_number: int = 5)
Prints a basic summary of each configuration and its models.
- Parameters
metrics_set (optional[str]) – options are ‘validation’ (the default), ‘test’, and ‘train’
- class sensiml.datamanager.confusion_matrix.ConfusionMatrix(metrics_dict)
This object is a representation of a confusion matrix that contains properties for statistics that can be generated from a confusion matrix as well as a DataFrame representation for easy viewing
- pp_by_class_name(class_name)
Returns the positive predictivity of a given class in the confusion matrix
- Parameters
class_name (str) – name of the class to return positive predictivity of
- Returns
if class name exists in matrix return PP, else NaN
- Return type
pp_by_class
- se_by_class_name(class_name)
Returns the sensitivity of a given class in the confusion matrix
- Parameters
class_name (str) – name of the class to return sensitivity of
- Returns
if class name exists in matrix return SE, else NaN
- Return type
se_by_class