Adding Custom Device Firmware

The Data Capture Lab supports data collection from any hardware device that implements the SensiML MQTT-SN Interface Specification or the Simple Streaming Interface. There is a pre-defined JSON format for how the Data Capture Lab will collect the data from your device. We call this format the Device Plugin format.

Note

The MQTT Interface Specification grants full command and control capabilities within the Data Capture Lab. It allows precise configuration to be done on-the-fly with a device.

We also provide the Simple Streaming Interface for quicker prototyping with projects.

The Simple Streaming Interface does not require Integrating MQTT and it’s topics, but also offers no run-time configuration options.

Device Plugins

Device Plugins are a list of properties that describe how the DCL will collect data from your device. For example, the device plugin may contain a list of sample rates that your device supports. This allows the DCL to collect data from any device that has been built to accept the supported parameters below.

How to Import a Device Plugin

The Data Capture Lab allows you to import Device Plugins via .SSF files. We will define the SSF file format in the next section below. For now, just note that you can import your SSF file through the menu item Edit → Import Device Plugin…

Next, you will be able to select your plugin protocol

SSF Files are used for both Simple Streaming and MQTT-SN. The only difference in the files are the device connection types used in the file.

Example SSF Files

You can download example SSF files for both the MQTT-SN and Simple Streaming protocols below. These files were built using the QuickFeather Development Kit.

• QuickFeather - MQTT-SN Example

• QuickFeather - Simple Streaming Example

SSF File Format

The Data Capture Lab allows you to import Device Plugins via .SSF files. Let’s go over the SSF file format and how this file will be used in the DCL. The SSF file format is a JSON based format with JSON properties that the DCL will use when configuring your Device Plugin.

When you import your Device Plugin the DCL will use the plugin to dynamically build out a screen for a user to configure sensor properties during data collection. See a screenshot below for a better understanding of how these parameters will be displayed in the DCL:

Notice that there are options for selecting your Device Plugin sample rate and sensors in the screenshot.

Now, let’s look at how this screen will map to the SSF file format. See below for a code snippet for the SensorTile.Box device. We will give more detailed definitions about the JSON properties within this snippet in the next section.

{
    "name": "SensorTile.Box",
    "uuid": "579e81fb-8347-48e9-8939-fcf50bcc0718",
    "collection_methods": [
        {
            "name": "live",
            "display_name": "Live Stream Capture",
            "storage_path": null,
            "is_default": true
        },
        {
            "name": "sd_card",
            "display_name": "SD Card",
            "storage_path": null,
            "is_default": false
        }
    ],
    "device_connections": [
        {
            "display_name": "Serial Port",
            "value": 1,
            "is_default": true,
            "serial_port_configuration": {
                "com_port": null,
                "baud": 921600,
                "stop_bits": 1,
                "parity": 0,
                "handshake": 0,
                "max_live_sample_rate": 209
            }
        }
    ],
    "capture_sources": [
        {
            "max_throughput": 0,
            "name": "Motion",
            "part": "Default",
            "sample_rates": [
                6667,
                3333,
                1666,
                833,
                416,
                208,
                104,
                52,
                26
            ],
            "is_default": true,
            "sensors": [
                {
                    "column_count": 3,
                    "is_default": true,
                    "column_suffixes": [
                        "X",
                        "Y",
                        "Z"
                    ],
                    "type": "Accelerometer",
                    "parameters": [
                        {
                            "name": "Sensor Range",
                            "values": [
                                {
                                    "actual_value": 20,
                                    "num_bytes": 1,
                                    "display_value": "+/- 2G"
                                },
                                {
                                    "actual_value": 40,
                                    "num_bytes": 1,
                                    "display_value": "+/- 4G"
                                },
                                {
                                    "actual_value": 80,
                                    "num_bytes": 1,
                                    "display_value": "+/- 8G"
                                },
                                {
                                    "actual_value": 160,
                                    "num_bytes": 1,
                                    "display_value": "+/- 16G"
                                }
                            ],
                            "units": null
                        }
                    ],
                    "sensor_id": 1229804865,
                    "can_live_stream": true
                },
                {
                    "column_count": 3,
                    "is_default": true,
                    "column_suffixes": [
                        "X",
                        "Y",
                        "Z"
                    ],
                    "type": "Gyroscope",
                    "parameters": [
                        {
                            "name": "Sensor Range",
                            "values": [
                                {
                                    "actual_value": 0,
                                    "num_bytes": 1,
                                    "display_value": "+/- 2000 DPS"
                                },
                                {
                                    "actual_value": 1,
                                    "num_bytes": 1,
                                    "display_value": "+/- 1000 DPS"
                                },
                                {
                                    "actual_value": 2,
                                    "num_bytes": 1,
                                    "display_value": "+/- 500 DPS"
                                },
                                {
                                    "actual_value": 3,
                                    "num_bytes": 1,
                                    "display_value": "+/- 250 DPS"
                                },
                                {
                                    "actual_value": 4,
                                    "num_bytes": 1,
                                    "display_value": "+/- 125 DPS"
                                }
                            ],
                            "units": null
                        }
                    ],
                    "sensor_id": 1229804871,
                    "can_live_stream": true
                }
            ],
            "sensor_combinations": [
                {
                    "combined_id": 1229804803,
                    "sensors_in_combo": [
                        1229804865,
                        1229804871
                    ]
                }
            ]
        },
        {
            "max_throughput": 0,
            "name": "Audio",
            "part": "Default",
            "sample_rates": [
                16000
            ],
            "is_default": false,
            "sensors": [
                {
                    "column_count": 1,
                    "column_suffixes": [
                        ""
                    ],
                    "type": "Microphone",
                    "is_default": true,
                    "parameters": [
                        {
                            "name": "Bits/sample",
                            "values": [
                                {
                                    "actual_value": 16,
                                    "num_bytes": 1,
                                    "display_value": "16"
                                }
                            ],
                            "units": null
                        },
                        {
                            "name": "Channels",
                            "values": [
                                {
                                    "actual_value": 1,
                                    "num_bytes": 8,
                                    "display_value": "One"
                                }
                            ],
                            "units": null
                        }
                    ],
                    "sensor_id": 1096107087,
                    "can_live_stream": false
                }
            ],
            "sensor_combinations": null
        }
    ],
    "is_little_endian": true
}

JSON Object Definitions

Device Plugin

name	(String) Name of your device
uuid	(GUID) A unique ID to identify your plugin
capture_sources	(List<Object>) Defines each of the sensors in your device. See how to define the capture_source JSON property in the Capture Source section
collection_methods	(List<Object>) Defines the collection methods your board supports. See how to define the collection_method JSON property in the Collection Method section
device_connections	(List<Object>) Defines the connection protocols your board supports. See how to how to define the device_connection JSON property in the Device Connection section
is_little_endian	(Boolean) Defines if the data is big endian (false) or little endian (true). Sensor data is transmitted in the byte order of the system. You will see incorrect data being captured if this is specified incorrectly

Capture Source

name	(String) Display name for your source. In the example above, Motion is used to describe a source that can supply both an Accelerometer and a Gyroscope
part	(String) Name of the device hardware part. Set to “Default” if a device only has one.
sample_rates	(List<Integer>) A list of all available sample rates your device can support
is_default	(Boolean) Defines if this capture source is the default option in the Data Capture Lab user interface
sensors	(List<Object>) Defines the sensors your board supports. See how to how to define the sensor JSON property in the Sensor section
sensor_combinations	(List<Object>) Defines a list of definitions for making a virtual/combined sensor. This is especially useful in the case of 6-axis and 9-axis motion sensors, where the data coming from all axes may wish to be treated as a single sample. See how to how to define the sensor_combination JSON property in the Sensor Combination section

Sensor

sensor_id	(Integer) Identification number of the sensor. This will be sent as an unsigned integer value to the device. It must match the definition used on your device. This is used for both data collection and for a Knowledge Pack to configure your sensors
type	(String) Class of the sensor. Example: Accelerometer, Gyroscope, Microphone
is_default	(Boolean) Defines if this sensor is selected as a default option in the Data Capture Lab user interface
column_count	(Integer) Defines how many columns of data this sensor sends Example: Accelerometer X, Y, Z has 3 columns
column_suffixes	(List<String>) A list of strings used to describe sensors Axes or channels. During data collection the Data Capture Lab will append the column_suffix to the type, in the same order that the list is defined. Example: {“X”, “Y”, “Z”} for Gyroscope type will be saved as GyroscopeX, GyroscopeY, GyroscopeZ
parameters	(List<Object>) Defines a list of sensor specific parameters, such as Accel/Gyroscope range, Microphone gain, etc. See how to how to define the sensor_parameter JSON property in the Sensor Parameter section
can_live_stream	(Boolean) Defines if this sensor can also be live-streamed while recording (using down-sampling for high data rate)

Sensor Parameter

name	(String) Name of the parameter sent as an unsigned integer value to the device. It must match the definition used on your device. This is used for both data collection and for a Knowledge Pack to configure your sensors
values	(List<Object>) Defines a list of available parameter values for the user to select. See how to how to define the value JSON property in the Sensor Parameter Value section
units	(String) Defines the unit of the parameter. This property is currently only used internally and can be set to null

Sensor Parameter Value

display_value	(String) Value to be displayed to the user
actual_value	(Integer) Value to be used when configuring the device (saved as unsigned, 64-bit value). This can be turned into a byte array with the num_bytes property
num_bytes	(Integer) The number of bytes that the actual_value property needs to be converted. Typically, this is one byte. This is used to create the configuration array in TOPIC_SENSOR_ADD of the SensiML Interface Specification

Sensor Combination

Sensor combinations are used to create a virtual/combined sensor to be treated as a single sample. When all sensors in a defined combination are selected in the Data Capture Lab configuration UI, the Data Capture Lab will automatically use the combined_id of a given combination when configuring the device.

combined_id	(Integer) ID for the combination
sensors_in_combo	(List<Integer>) Defines a list of unsigned integers that match the sensors being combined. In the example JSON file above, see the combination for Accelerometer and Gyroscope

Collection Method

name	(String) Internal name for the collection method. There are three available options: live sd_card onboard_flash
display_name	(String) Name to be displayed to the user
is_default	(Boolean) Defines if this collection method is the default option in the Data Capture Lab user interface
storage_path	(String) Location where files being saved to the device should be stored. See TOPIC_STORAGE section of the MQTT-SN Interface Specification for more information

Device Connection

Device connections define the protocol for how you will connect to your device (Bluetooth-Low Energy UART, Serial/Wired UART Port, or Wi-Fi). Note: Bluetooth-Low Energy UART is not supported through imported Device Plugins

display_name	(String) Name to be displayed to the user
value	(Integer) Value to define the connection type. There are three available options: 0 : Bluetooth-Low Energy UART 1 : Serial/Wired UART Port 2 : Wi-Fi
is_default	(Boolean) Defines if this connection is the default option in the Data Capture Lab user interface
bluetooth_le_configuration	(Object) Defines Bluetooth-Low Energy UART specific configuration options. See how to how to define the bluetooth_le_configuration JSON property in the Bluetooth UART Configuration section
serial_port_configuration	(Object) Defines Serial/Wired UART Port specific configuration options. See how to how to define the serial_port_configuration JSON property in the Serial Port Configuration section
wifi_configuration	(Object) Defines Wi-Fi specific configuration options. See how to how to define the wifi_configuration JSON property in the Wi-Fi Configuration section

Bluetooth UART Configuration

For Bluetooth-Low Energy UART connections, the service and characteristic UUIDs are needed. It is assumed that the BLE UART will be following similar protocol to the Nordic UART Service. This allows the device to act as a virtual COM port, enabling serial communication much the same as a wired connection.

service_uuid	(GUID) UART Service ID
char_tx_uuid	(GUID) UART Transmit Characteristic ID
char_rx_uuid	(GUID) UART Receive Characteristic ID
max_live_sample_rate	(Integer) The maximum frequency (in Hertz) your device is able to reliably stream sensor data without dropping packets or overflows. This is used to determine the SENSOR_COUNTDOWN field reference in TOPIC_LIVE_SET_RATE_REQ in the MQTT-SN Interface Specification. Default: 100

Serial Port Configuration

com_port	(String) COM port should be left as null. This will be set internally by the Data Capture Lab based on the user’s computer, USB input, etc
baud_rate	(Integer) Speed at which you communicate. Default: 115200 Refer to Microsoft BaudRate Documentation for more details
stop_bits	(Integer) Number of stop bits. Default: 1 Refer to Microsoft StopBits Documentation for more details
parity	(Integer) Parity scheme. Default: 0 Refer to Microsoft Parity Documentation for more details
handshake	(Integer) Handshake scheme. Default: 0 Refer to Microsoft Handshake Documentation for more details
max_live_sample_rate	(Integer) The maximum frequency (in Hertz) your device is able to reliably stream sensor data without dropping packets or overflows. This is used to determine the SENSOR_COUNTDOWN field reference in TOPIC_LIVE_SET_RATE_REQ in the SensiML Interface Specification. Default: 100

Device Connection Types

MQTT-SN Device Connections

Currently, MQTT-SN is done only on a serial port. Below is an example of that configuration.

{
    "display_name": "Serial Port",
    "value": 1,
    "is_default": true,
    "serial_port_configuration": {
        "com_port": null,
        "baud": 921600,
        "stop_bits": 1,
        "parity": 0,
        "handshake": 0,
        "max_live_sample_rate": 209
    }
}

Simple Streaming Device Connections

Simple streaming interface currently supports serial ports and a simplified Wi-Fi interface.

For the serial port

{
    "display_name": "Serial Port",
    "value": 1,
    "is_default": true,
    "serial_port_configuration": {
        "com_port": null,
        "baud": 460800,
        "stop_bits": 1,
        "parity": 0,
        "handshake": 0,
        "max_live_sample_rate": 3301
    }
}

And for Wi-Fi

{
    "display_name": "Wi-Fi",
    "value": 2,
    "is_default": true,
    "wifi_configuration": {
        "use_external_broker": false,
        "external_broker_address":"",
        "broker_port":1885,
        "device_ip_address": null,
        "device_port": 0,
        "max_live_sample_rate": 1000000
    }
}