SensiML FAQ's

SensiML offers cutting-edge AutoML software tools enabling developers to quickly create AI-powered sensing applications that run on ultra-low-power microcontrollers that serve as the brains for nearly all IoT devices. Known categorically as TinyML, AI/ML models that can run autonomously on the extreme endpoint nodes (ex. smart doorbells, wearable fitness devices, battery-powered remote sensors) present a unique challenge. While cloud AI has the benefit of huge data centers and powerful servers to perform AI tasks, TinyML models must provide similarly accurate results relying on the memory, compute, and power available on the IoT device’s processor.

SensiML is an industry leader in providing such tools to developers of intelligent IoT devices spanning consumer devices, commercial smart building, factory automation and predictive maintenance, and remote sensing applications. The company’s flagship solution, SensiML Analytics Toolkit, provides an end-to-end development platform addressing ALL aspects of ML model development:

• Data collection
• Labeling
• Algorithm Optimization
• Firmware Auto-generation
• Device testing

The SensiML Toolkit supports a broad array of Arm^® Cortex^®-M class and higher microcontroller cores, Intel^® x86-based CPUs, and heterogeneous multi-core SoCs. SensiML’s IoT edge ML/AI development software enables developers to build intelligent endpoints quickly and easily (500% faster or more than hand-coded solutions) with minimal data science expertise.

Automated machine learning (AutoML), as the name suggests, is the automation of the process for constructing machine learning models. Its use is best illustrated by comparing it to the traditional means for constructing ML models. Without AutoML, the following tasks are left to the modeler to determine based on their understanding of the problem, desired model performance, and most critically their expertise in the proper application of signal processing and machine learning classifiers:

• Deciding how to segment for regions of interest in the input data
• Determining which pre-processing and feature transforms are needed to convert raw input data into a suitable input vector for the classifier
• Selecting which type of machine learning classifier to use to deliver best results
• Optimizing model parameters and tuning of hyperparameters
• Assessing the need for post-processing to further enhance model performance

In contrast, AutoML employs high performance computing and use of search optimization algorithms to augment human know-how in performing the task of constructing an ideal ML model for a given application and train/test dataset. The advantages of AutoML include the ability to evaluate hundreds of thousands to millions of model permutations in the same timeframe as a human data science expert may be able to evaluate only a handful. And with directed search constraints, the combination of AutoML in the hands of a skilled user can focus searches on the most promising permutations rather than just execute brute-force grid searches. The upshot is that AutoML can be a powerful ally in algorithm development whether empowering an AI novice or extending the capabilities of a seasoned data science expert.

SensiML’s AutoML process uses the power of cloud computing to aid in the model creation process for ML classifiers intended to run on the most resource-constrained computing platforms at the IoT edge. Ordinarily, creating ML models for this class of devices requires a unique blend of skillsets that include domain knowledge for the application use case, data science and numerical methods expertise in signal transformation and classification algorithms, and firmware programming and optimization expertise to condense theoretical ML pipelines to practical and efficient code that fits the device.

Regardless of whether your development team possesses one or all of these unique skills, the effort and cost of developing such algorithms by hand or with general-purpose AI frameworks is not scalable and often adds unnecessary cost and risk to fixed product schedules. Alternatively, the use of AutoML can greatly accelerate the model development and optimization process by augmenting or replacing manual programming with automation. Search optimization algorithms can traverse many thousands of model permutations delivering results in minutes to hours of machine time. The search space can also be much expanded as SensiML’s AutoML tool considers not just a single classification technique such as neural networks, but rather a library of classifiers that range from classic decision trees to distance-based classifiers to neural networks. The best model is the one the provides the most accurate results with as little compute latency, power consumption, and memory use as possible.

Depending on your perspective, the answer is both yes and no. By definition, machine learning is the principle of providing systems the ability to learn and improve from experience without being explicitly programmed. In place of explicit code or instructions, such devices are ‘taught’ through the use of labeled training data applied to classifier models whereby the data itself influences the parameters of the model to tune its behavior. So at this level, we can state that such algorithms aren’t actually programmed. However, the tools or AI frameworks popularly used for constructing such underlying classifier models are often built on programming languages and have programming interfaces and APIs. Python, Tensorflow, Keras are a few such examples. So the first programming hurdle developers must overcome with most ML tools is the learning curve of gaining familiarity with these domain-specific programming environments for AI and machine learning.

With SensiML, Python has always been a supported programmatic interface for those who have already overcome this learning curve and prefer to work in this manner. However, SensiML also provides a parallel simple UI interface that can be used by ML novices and seasoned practitioners alike to rapidly construct models using AutoML methods without the need to program. Additionally, in nearly all present hardware architectures, machine learning models once created still ultimately need to be translated into instruction set architectures that execute in a traditional explicit programming manner on processors.

Given most AI frameworks were built from data center and server origins, the degree to which this translation from abstract ML classification to efficient ISA execution varies widely and is typically sub-optimal or not achievable for the smallest of processors found in IoT edge devices. Thus comes the next programming task often encountered, which is to modify general purpose C code output from AI frameworks for practical application on microcontrollers with smaller memory footprints, no GPUs (and sometimes not even FPUs!),limited clock frequencies, and one or a few cores. Rather than starting from cloud execution environments, SensiML Analytics Toolkit was built from the ground up as an edge AI tool optimized for efficient execution of ML algorithms on these smallest and most resource limited devices. So developers using SensiML can be confident the models they build can be efficiently run on edge devices without further code customization needed to fit on their embedded platforms.

The accuracy of machine learning models heavily relies on the quality and quantity of training data. While it might be tempting to assume that more data is always better, the relationship between training data size and model performance is more nuanced. Collecting and labeling sensor data is resource-intensive, so striking the right balance is crucial.

1. Data Diversity: The diversity of your training data plays a significant role. A diverse dataset that covers a wide range of scenarios, environmental conditions, and user behaviors helps the model generalize better to real-world situations. Aim for a dataset that represents the variability you expect the model to encounter during deployment.
2. Complexity of the Task: The complexity of the task you’re trying to accomplish with your sensor data also impacts the amount of training data needed. Simple binary classifications might require less data compared to intricate multi-class or regression tasks.
3. Feature Space: The dimensionality of your feature space is another factor. If you’re extracting numerous features from your sensor data, you might need more samples to ensure effective coverage of the feature space.

Determining the Optimal Training Data Size

SensiML empowers you to experiment and fine-tune your model to achieve the best possible results. Here’s a step-by-step approach to determining the optimal training data size:

1. Preliminary Experiments: Begin with a relatively small labeled dataset and train your initial models. This will serve as a baseline to assess the model’s performance. Use metrics like accuracy, precision, recall, and F1-score to gauge effectiveness.
2. Data Augmentation: Before collecting more data, consider data augmentation techniques. These techniques involve creating new training samples from your existing data through transformations like rotation, noise addition, and scaling. Augmentation can artificially expand your dataset and improve model generalization.
3. Incremental Data Addition: Gradually increase the size of your training dataset while monitoring performance improvements. Observe how the metrics change as you add more data. At some point, the returns might start diminishing, indicating that you’ve reached a suitable training data size.
4. Validation and Testing: Always keep a separate validation and testing dataset to evaluate your model’s performance objectively. This helps prevent overfitting and ensures your model generalizes well to unseen data.
5. Cross-Validation: Perform cross-validation to validate the stability of your model’s performance across different subsets of your data. This provides a more robust understanding of how your model is likely to perform in the wild.

Conclusion

Accurate model results are at the heart of successful sensor-based applications, and determining the right amount of labeled training data is a crucial step in achieving this goal. SensiML offers a robust platform that empowers developers to experiment with different dataset sizes and fine-tune their models for optimal performance. By considering factors like data diversity, task complexity, and feature space, and following a systematic approach of preliminary experiments, data augmentation, incremental data addition, and validation/testing, developers can strike the right balance and create models that excel in real-world scenarios.

General purpose AI frameworks were made to simplify the application of machine learning algorithms to datasets across a broad array of applications. Problems ranging from credit card fraud detection to image classification or even predicting survivors of the Titanic disaster from passenger metadata have been tackled using such tools. Moreover, these tools are typically cloud-centric with adaptations to simplify their application for edge environments in some cases.

Alternatively, SensiML is a purpose-built AutoML development tool for edge devices intended for schedule-driven IoT sensor product development projects where time, investment, and efficient code are mission-critical. SensiML Analytics Toolkit has evolved over ten years originating as an Intel SW tool into an independent industry-leading multi-platform edge AI tool for building optimal models on resource-constrained IoT edge devices. To do so, our toolkit provides means for users to rapidly traverse countless combinations of segmenters, features, classifiers, and associated hyperparameters. For those not versed in data science, this AutoML search process can be done with just simple constraints letting the tool do what it does best. For those accomplished in the practice of machine learning, the tool will often surprise with results not previously considered including solutions using classifiers thought too simple to perform against more complex deep learning approaches. Nonetheless, for power users, granular control using a Python interface is included to adjust as desired the output from the AutoML results for the AI framework style experience. That’s the power and the difference of SensiML’s take on AutoML and the workflow optimization of a toolkit intended to do purposeful and scalable algorithm development for real products versus science experiments.

Unlike most general purpose AI frameworks, SensiML Analytics Toolkit includes a client application to make the process of collecting, labeling, and cleansing supervised ML datasets vastly easier, manageable, and scalable from most methods currently in use by developers. In fact, most developers faced with dataset creation and management resort to their own means as the state of production quality tools for this task are pretty limited for AI algorithm development. SensiML Data Capture Lab (DCL) provides a means to collect data directly from IoT sensor evaluation boards and prototype devices either through real-time streaming of raw sensor data or using offline storage to flash with subsequent batch mode retrieval. Data Capture Lab also supports bulk importing of existing sensor data using CSV text files. Once collected within a SensiML Data Capture Lab project file, the DCL application provides version managed labeling of datasets by one or more users for a truly scalable and managed process of treating valuable sensor data intellectual property in a methodical workflow. Given labeled ML datasets can be as important a source of valuable IP as code which developers are accustomed to retaining in VCS tools, it makes sense to apply similar methodology to labeled ML data as SensiML Data Capture Lab uniquely enables.

Analytics Studio is the core AutoML application that runs in the cloud to build models based on your provided labeled datasets and return executable code for predictive models that can run on your target edge hardware platform. Designed as a UI based tool, SensiML Analytics Studio provides a standardized workflow for selecting labeled data and metadata of interest from a given input dataset, establishing model constraints and target performance objectives, configuring AutoML seed, segmentation, and feature parameters, and then executing the AutoML search optimizer. Once results are returned, bit-exact code emulation can be run for a chosen platform to assess performance and resource consumption prior to ever flashing the model for physical test. Test vectors can be provided to test model generalization beyond the train/test data used during cross-validation. Visualization of results in tabular, summary, or graphical form are supported and details of the resulting model can be explored and refined. With a desired candidate model selected, the final steps are to generate embedded code in either fully compiled executable binary form, object code, or with Standard or Enterprise editions full C source code can be generated.

SensiML Open Gateway is an easy-to-adapt connectivity tool that can be extended and modified quickly. Written in Python, SensiML Open Gateway can run on a variety of platforms and overcome communication limitations of small-form-factor IoT devices. Out of the box, the application can accommodate many different connection types and sensor configurations and with user customization, SensiML Open Gateway can be tailored to meet user-specific needs for getting data out of the IoT / embedded sensor node and into AI software tools, cloud analytics, and application code running on PCs and smartphones.

The SensiML TestApp is an application that shows real-time event classifications output from the SensiML AI model (Knowledge Pack) running on your edge device. The SensiML TestApp can show custom class names and pictures with your live classification results. TestApp is available as both a Windows10 PC application, or for field use, as an Android smartphone or tablet application.

SensiML welcomes those wish to publish datasets, subject to review, on our Data Depot (https://datadepot.https://sensiml.com). Such datasets can be submitted and made available for distribution to others under one of several licenses. See datadepot.https://sensiml.com for more information.

If you are developing a smart IoT device or embedded application that involves sensors and sensor data processing, it is very likely that you can benefit from the SensiML Analytics Toolkit. The graphic below provides just a few examples of use cases where SensiML can be utilized to build models that transform raw sensor signals into useful application insight:

As an AutoML tool, SensiML can be utilized by a broad range of IoT device developers interacting at different levels with the toolkit. Data scientists and those with strong ML/AI skills can use the tool as a productivity aid to rapidly search and optimize for efficient model implementation for resource-constrained edge devices using a familiar Python client front-end. Domain experts, embedded developers, and hardware engineers who may be less skilled in machine learning and AI can utilize the UI driven constraints approach to define parameters and input datasets for the AutoML processing of potential algorithms to try. Academic researchers, startups, makers, or established product development teams can effectively use SensiML to quickly generate working sensor code that can learn from new projects or existing labeled datasets.

For those familiar with Python ML programming, we make the full capabilities of SensiML’s client API available in our Python SDK for greatest flexibility. More details can be found at https://pypi.org/project/SensiML/.

SensiML Analytics Toolkit is offered as a quarterly SaaS service plan with service levels that fit the needs and project phases of developers, engineers, and data scientists building intelligence into their products. Community Edition – A zero-cost entry to our software enabling the rapid creation of proof-of-concepts and software evaluation. Unlike other competing tools, our free tier is NOT trialware but rather is fully functional and available as long as required for your evaluation and prototyping purposes. Community Edition comes with the same functionality as our paid Proto Edition license, except it is geared to evaluation stage projects (single-user access, 1GB of data storage, 3 hours/month of AutoML compute time, 2500 labeled segments, 1000 inferences without device reset), and does not provide upgrade add-ons as with any of the paid plans including Proto Edition. Proto Edition – SensiML Proto Edition is intended to best suit those undertaking their first SensiML application. With full modeling capability, unrestricted device firmware functionality, and access to online, email, and available direct premier support options, Proto plan provides all the features you need to build your first real-world commercial-ready smart IoT device using AutoML technology. Advanced Edition – A more flexible version of our most popular tier of service, Advanced Edition provides an extremely affordable solution for extracting more intelligence and insight from your IoT devices for multi-project developers with ongoing needs for AutoML modeling and embedded smart sensing code generation. Advanced Edition is tailored to provide maximum value and productivity for experienced small and mid-sized company development teams. With the flexibility to features, users, capabilities, and premier support, Advanced Edition can be customized to meet your team’s precise needs. Premium Edition – For large teams and those seeking our highest level of support, SensiML Premium Edition includes the full capabilities of SensiML Analytics Toolkit and comes with cost-effective bundled inclusion of add-on features and scalability suited for enterprise development projects. Premium Edition comes with support for 10 users (and additional users can be added as well), up to 100GB of datasets, full C source code output, and the ability to generate models for sensors up to 1MHz sampling rate. Premier support packages for direct technical and application support are available and can be customized to suit your team’s needs and budget.

SensiML makes sample datasets available for evaluating the toolkit even prior to having your own data. We publish a growing collection of additional datasets for evaluation in our application examples. All tiers of SensiML from Community Edition through Enterprise edition can utilize these sample datasets. The SensiML Data Capture Lab tool also greatly simplifies the collection and labeling of new datasets from sensor devices than would otherwise be possible using DIY data acquisition and custom data translation and cleansing scripts.

Community Edition is best suited for those users who are still exploring what edge AI can do for them and how they might apply it to their IoT application(s). By working through provided example applications, it is possible to familiarize yourself with what can be done and start formulating a defined project. We also have published a whitepaper that describes in more depth the benefits and potential for edge AI sensor algorithms.

SensiML Basic, Standard, and Enterprise Editions were replaced with new, more flexible plans as of August 1, 2021.

Basic Edition is replaced with Advanced Edition which is functionally equivalent to Basic but now offers the flexibility of customized plan add-ons such as Premier technical support options, the ability to add users, and upgrades in output and application support levels. Existing Basic Edition customers need not do anything to enjoy these additional account benefits. Current Basic Edition plans qualify automatically for Advanced Edition add-ons and subscription pricing will not change.

Standard Edition is similarly replaced with Premium Edition which is functionally equivalent to Standard but now offers the flexibility of customized plan add-ons such as Premier technical support options, the ability to add users, and upgrades in output and application support levels. Existing Standard Edition customers need not do anything to enjoy these additional account benefits. Current Standard Edition plans qualify automatically for Premium Edition add-ons and subscription pricing will not change.

Enterprise Edition is no longer offered for new service or as an upgrade for existing accounts. Instead, customers seeking our highest level of functionality and support should choose Premium Edition along with the Premier technical support add-on package that best suits their needs. Existing Enterprise Edition users need not change their plan as SensiML will continue supporting this service tier for existing accounts. For more information, please consult the plan comparison table.