CSV Upload

In some cases, you may already have a CSV file that you want to use. This can be raw sensor data with labels or features that you have cached. We separate this into DataFiles and Featurefiles which we explain below.

DataFiles

A DataFile allows you to upload sensor data into a pipeline for testing, rather than using Project Capture data. Files must be in CSV format. Using datafile is convenient when:

  1. You have test data that you want to test against a model without adding the file to your Project Capture list.

Examples:

client.list_datafiles()

# if you want to upload directly from a csv file, force=True overwrites the file on the server if it exists.
client.upload_data_file(name, path, force=True)

# if you already have a dataframe
client.upload_dataframe(name, dataframe)

FeatureFiles

A featurefile can be used to directly load data into a pipeline, rather than querying Project Capture data. Files must be in CSV format. Using featurefiles is convenient when:

  1. You want to cache features locally and then use those as input into the training algorithm so you can avoid running previous steps in the pipeline.

Examples:

client.list_featurefiles()

# if you want to upload directly from a csv file
client.upload_feature_file(name, path)

# if you already have a dataframe
client.upload_dataframe(name, dataframe)
class sensiml.datamanager.featurefiles.FeatureFiles(connection, project)

Base class for a collection of featurefiles.

build_datafile_list()

Populates the function_list property from the server.

build_featurefile_list()

Populates the function_list property from the server.

build_full_list()

Populates the function_list property from the server.

create_featurefile(filename, path, is_features=False)

Creates a featurefile object from the filename and path.

Parameters
  • filename (str) – desired name of the featurefile on the server, must have a .csv or .arff extension

  • path (str) – full local path to the file, including the file’s local name and extension

Returns

featurefile object

Raises

FeatureFileExistsError, if the featurefile already exists on the server

get_by_name(filename)

Gets a featurefile or datafile from the server referenced by name.

Parameters

filename – name of the featurefile as stored on the server

Returns

featurefile object or None if it does not exist

get_featurefile(uuid)

Gets a list of all featurefiles in the project.

Returns

list (featurefiles)

get_featurefiles()

Gets a list of all featurefiles in the project.

Returns

list (featurefiles)

new_featurefile()

Initializes a new featurefile object, but does not insert it.

class sensiml.datamanager.featurefile.FeatureFile(connection, project, name='', path='', is_features=True, uuid=None)

Base class for a featurefile object.

property created_at

Date of the Pipeline creation

delete()

Calls the REST API and deletes the featurefile from the server.

download()

Calls the REST API and retrieves the featurefile’s binary data.

Returns

featurefile contents

download_json()

Calls the REST API and retrieves the featurefile’s json data.

Returns

featurefile contents as json

property filename

The name of the file as stored on the server

Note

Filename must contain a .csv or .arff extension

insert()

Calls the REST API to insert a new featurefile.

property is_features

If this is a DataFile or FeatureFile

refresh()

Calls the REST API and populate the featurefile’s properties from the server.

update()

Calls the REST API to update the featurefile’s properties on the server.