# ModelResultSet¶

Model objects from a pipeline come back from the server organized into ModelResultSets and Configurations. A ModelResultSet represents everything returned by the pipeline and a Configuration is a subset pertaining to a single train-validate-optimize (TVO) configuration or combination. It is very common for a ModelResultSet to have a configuration containing many models. See the tutorials for more information about models, their metrics, and recognition capabilities.

class sensiml.datamanager.modelresults.ModelMetrics(configuration, sandbox, index, model_result)

Base class for a model metrics object.

confusion_matrix_stats: list[ConfusionMatrix]

comprehensive metrics returned for the model

train_set: list

indices of the input data that the model was trained with

validation_set: list

indices of the input data that the model was validated with

test_set: list

indices of the input data that the model was tested with

debug: dict

structure containing debug information for some models

neurons: list[dict]

model neuron array

parameters: dict

model parameters

knowledgepack: KnowledgePack

knowledgepack associated with the model

property knowledgepack

The model’s KnowledgePack object

recognize_signal(capturefile=None, stop_step=False, datafile=None, segmenter=True, lock=True, silent=True, platform='emulator', compare_labels=False, **kwargs)

Sends a DataFrame of raw signals to be run through the feature generation pipeline and recognized.

Parameters
• capturefile (str) – The name of a captured file uploaded throught the data capture lab

• datafile (str) – The name of an uploading datafile

• platform (str) – “emulator” or “cloud”. The “emulator” will run compiled c code giving device exact results, the cloud runs similary to training providing more flexibility in returning early results by setting the stop step.

• stop_step (int) – for debugging, if you want to stop the pipeline at a particular step, set stop_step to its index

• compare_labels (bool) – If there are labels for the input dataframe, use them to create a confusion matrix

• segmenter (bool or FunctionCall) – to suppress or override the segmentation algorithm in the original pipeline, set this to False or a function call of type ‘segmenter’ (defaults to True)

• lock (bool, True) – If True, waits for the result to return before releasing the ipython cell.

Returns

a dataframe containing the results of recognition and a dictionary containing the execution summary and the confusion_matrix when labels are provided.

• execution_summary: a summary of which steps ran in the execution engine

• confusion_matrix: a confusion matrix, only if the input data has a ‘Label’ column

Return type

(DataFrame, dict)

recognize_vectors(vectors)

Sends a DataFrame of feature vectors to the model’s KnowledgePack for recognition.

Parameters
• vectors (DataFrame) – where each row is a feature vector with column headings named the same as

• features generated by the pipeline (the) –

• label columns may be included (and) –

Returns

contains the results of recognition, including predicted class, neuron ID, and distance

Return type

(DataFrame)

summarize(metrics_set='validation')

Prints a formatted summary of the model metrics

Parameters

metrics_set (optional[str]) – options are ‘validation’ (the default), ‘test’, and ‘train’

class sensiml.datamanager.modelresults.ModelResultSet(project, sandbox)

Base class for a model result set object.

get_knowledgepack_by_index(config_index, model_index)

Returns the KnowledgePack of the given configuration index and model index.

sort_models(metric='accuracy', metrics_set='validation')

Sorts the models within all configurations by a given metric or property.

Parameters
• metric (optional[str]) – options are ‘accuracy’ (the default), ‘index’, and ‘number_of_neurons’

• metrics_set (optional[str]) – options are ‘validation’ (the default), ‘test’, and ‘train’

summarize(metrics_set='validation', report_number=5)

Prints a basic summary of each configuration and its models.

Parameters

metrics_set (optional[str]) – options are ‘validation’ (the default), ‘test’, and ‘train’

class sensiml.datamanager.confusion_matrix.ConfusionMatrix(metrics_dict)

This object is a representation of a confusion matrix that contains properties for statistics that can be generated from a confusion matrix as well as a DataFrame representation for easy viewing

pp_by_class_name(class_name)

Returns the positive predictivity of a given class in the confusion matrix

Parameters

class_name (str) – name of the class to return positive predictivity of

Returns

if class name exists in matrix return PP, else NaN

Return type

pp_by_class

se_by_class_name(class_name)

Returns the sensitivity of a given class in the confusion matrix

Parameters

class_name (str) – name of the class to return sensitivity of

Returns

if class name exists in matrix return SE, else NaN

Return type

se_by_class