For a long time, artificial intelligence has been traditionally developed and employed for data analysis and recognition tasks. These models analyze massive datasets, identify patterns, and make predictions with remarkable accuracy.
But what if AI could not only interpret the world around us, but also generate new realities that closely resemble existing scenarios. This is the essence of Generative Artificial Intelligence (Generative AI) a field pushing the boundaries of machine creativity. Generative AI algorithms are trained on massive datasets of text, code, images, or audio. By uncovering the underlying relations and distributions of the historical data, they learn to generate new pieces that mimic the inner patterns, styles, fine structures, and correlations inherent in the data. These algorithms usually produce not just generic outputs, but often create new contents with impeccable quality.
SensiML provides a comprehensive solution designed to streamline sensor data collection and embraces the potential of AutoML to build edge AI applications. To date, the main focus of the SensiML toolkit has been adopting well-established machine learning models for tasks like classification and regression. However, with its advancement, the toolkit is now ready for a significant leap forward with the integration of generative models. In this blog article, we delve into the potential of generative AI and its capacity for the SensiML toolkit to become more powerful, allowing developers to build more reliable AI/ML models. Generative models have the potential to reduce data collection costs and often enhance performance of the end product by covering unforeseen edge cases.
Beyond generative AI, other types of generative algorithms might establish mathematical foundations for numerically modeling the physical behavior of systems. These simulations can address data gaps or improve the quality of the generated data to better reflect real-world scenarios. In this article, we use the term “generative models” to encompass any model capable of creating data and/or codes.
Types of Generative AI
Here are some important types of generative AI algorithms capable of generating new content. Each of these approaches can benefit the SensiML toolkit in a unique way.
- Generative Adversarial Networks (GANs): GANs employ a competitive training paradigm. A generative model continuously strives to produce novel data instances, while a discriminative model attempts to determine differences between the generated and genuine data. Albeit this is an iterative process, both models refine their capabilities, enabling GANs to generate highly realistic outputs such as time-series, images, audio, video, and even music.
- Variational Autoencoders (VAEs): VAEs can be used for their ability to reduce dimensionality. They compress complex data (e.g., time-series, audio, images, text) into a latent space, a lower-dimensional representation that captures the essence of the data. VAEs can then reconstruct the original data or generate variations by manipulating points within this latent space. Simplified VAEs can be deployed at the edge to minimize data transmission and storage requirements, thereby addressing security risks and optimizing resource consumption. Additionally, the reduced number of features can help recognition algorithms focus only on the most relevant information hidden in the data.
- Autoregressive Models: Autoregressive models excel in predicting sequential data. By analyzing a sequence, these models learn underlying patterns and relationships between elements. They can predict the most likely continuation of the sequence, enabling them to generate entirely new sequences that adhere to learned patterns or forecast upcoming elements of the sequence. These models can be adopted for data augmentation or anomaly detection when the collected data significantly deviates from the forecasted version.
- Recurrent Neural Networks (RNNs): RNNs take advantage of a form of internal memory that allows them to process sequential data effectively. They can retain information from earlier parts of a sequence and utilize it to understand the context of the current element. This capability makes RNNs well-suited for tasks like text generation, language translation, and acoustic recognition, where comprehending the order and relationships within the data is crucial.
- Long Short-Term Memory (LSTM): LSTM is a type of RNN designed to address some issues such as the vanishing gradient problem. LSTMs excel at handling sequential data like sensor readings, time series data, or short audio snippets collected at the edge. Their inherent computational complexity however makes them more challenging for direct deployment on edge devices. Despite this, their power can be harnessed in the model development phase, particularly during data generation and training.
- Transformer-based models: GPT series of transformers are the most successful and popular in this group. The use of attention mechanisms makes them excellent at understanding relationships between words across long distances in a sentence, leading to more coherent and grammatically correct text compared to older models. They are particularly effective in generating optimized code when combined with well-crafted prompts, useful for creating data processing modules or optimizing data transformation units.
- Gaussian Processes (GPs): These are powerful probabilistic models that can be applied effectively in the context of exploring, modeling and generating time series data. GPs are flexible in handling various time series patterns, including trends, seasonalities, and irregularities. They can capture complex dependencies and non-linear relationships between time points. GPs can generate new time series data points by sampling from the learned distribution. This makes them useful for simulating data for scenarios where historical data is limited or unavailable. GPs can be used in prediction, forecasting, and anomaly detection. GPs also offer a powerful approach to hyperparameter optimization, with great potential to significantly enhance the performance of the SensiML AutoML engine.
How Generative Models Will Strengthen SensiML
Generative models open up a multitude of exciting possibilities, such as:
- Data Augmentation: Synthetic data generation addresses data limitations by replicating original real-world data, particularly valuable when collecting actual data is challenging, expensive, and time-consuming. Artificial data facilitates testing edge cases and mitigating bias in imbalanced datasets. Additionally, it enables the simulation of adversarial scenarios to assess model resilience under harsh conditions. Example generators include GANs, autoregressive models, LSTMs, and GPs.
- Data Annotation: This is the process of identifying and labeling sections of data that share similar patterns. This helps improve the accuracy and efficiency of data preparation for machine learning. Techniques like autoregression and autocorrelation can be used to analyze data patterns, while tools like GPT-based models can assist in building functions that automatically identify relevant segments.
- Preprocessing: Some generative AI algorithms are capable of developing sophisticated methods to identify and eliminate noise from sensor data, as well as creating efficient DSP filters to capture the most prominent properties of the signal. With the help of generative AI, code can be optimized for the most efficient deployment on embedded devices.
- Data Cleansing: Generative models can assist in identifying anomalous data points in a time series for removal before training an AI/ML algorithm. Models like GPs, LSTMs, and other autoregressive algorithms make predictions, where significant deviations from their predictions can be flagged as anomalies. TimeGPT is one of such generative tools that is capable of detecting outliers in time series. TadGAN is another example of anomaly detection that leverages an LSTM architecture as its base generator.
GAN algorithms have demonstrated promising results in denoising time series data, such as audio data. When integrated into the SensiML toolkit, users can effectively denoise collected audio data and prepare it for augmenting with various background noises. - Postprocessing: In a similar fashion, generative AI can assist in automating complex postprocessing tasks, sometimes in the form of a code block. Specifying the output of the primary AI/ML predictor and the target prediction, a GPT-based model can easily structure a code that transforms the result to better align with the project objectives.
- Feature Engineering: SensiML aims to enhance feature engineering by generating new features that capture more nuanced relationships within the data, leading to stronger models. While the platform currently utilizes standard libraries for feature generation, incorporating generative AI could open doors for more advanced and customized feature engineering codes, pushing the boundaries of possibilities in the modeling process.
Addressing the Data Bottleneck
The field of generative AI is currently developing rapidly. It holds significant promise for addressing data scarcity and thus improving the overall performance of data-hungry ML models such as deep neural networks.
Here are several ways that generative models can assist:
- Overcoming Data Scarcity: Generative models can create synthetic data that mimics the characteristics of the existing data. This helps to artificially expand datasets, leading to more robust and generalizable machine learning models.
- Enhancing Data Diversity: Generative AI can be used to create variations of the available data, introducing slight modifications or distortions. This helps models learn from a wider spectrum of scenarios and improve their performance on unseen data.
- Data Simulation: Sensor data can be simulated with different seasonal trends, outliers, or anomalies representing equipment malfunctions. In particular, this is valuable for training models for anomaly detection or predictive maintenance tasks. Here are two example scenarios that illustrate how generative AI can be useful:
- Scenario 1: Anomaly Detection in Factory Production Line: Imagine you have sensor data from a factory production line that tracks temperature, pressure, and vibration levels. Synthetic data can be generated to simulate sensor spikes or deviations from normal operating ranges, which help the model learn to identify these anomalies and flag potential equipment failures before they occur.
- Scenario 2: Predictive Maintenance for Wind Turbines: Sensor data from wind turbines can include wind speed, direction, and generator output. Generative models could create synthetic data with simulated variations in wind patterns or introduce noise that mimics sensor degradation. This allows a model to learn how these factors affect turbine performance and predict maintenance needs before breakdowns occur.
Actively Developing Generative Models for Piccolo AI™
Recently, SensiML launched a fully open-source variant of its powerful AutoML tool for use on private servers and PCs. Called Piccolo AI, this was a strategic initiative for SensiML which is committed to the open-source development model and to building a vibrant community of contributing experts to rapidly improve the current state of the art in tinyML™ software modeling tools. From the list of generative AI development opportunities listed earlier, we can readily see why such a collaborative model is necessary to fully realize the available potential on the horizon.
There are several approaches on how to surface the power of the generative models. One approach is to enable users to directly interact with the generative models hosted and/or served by SensiML to fulfill their specific demands. Here are a few examples of the user interaction flow:
- Prompt-Based Generation: Users provide detailed prompts outlining the desired data/code characteristics, which guide the model in generating data/code that meets the specified criteria. With the guidance of prompt engineering tools, users can craft more efficient queries.
- Provide Sample Data and Generate: In this method, users provide sample input data. The model analyzes this data to infer underlying patterns and generates new data reflecting those patterns. This approach can also be employed similarly to optimize code snippets for embedded devices.
In another approach, generative models are seamlessly integrated into the Piccolo AI development tool. Through user interactions, these models evolve and become more adept at handling various tasks. Here are a few examples:
- Human Gesture IMU Data: Simulating realistic human motion patterns for training gesture recognition algorithms. While the generative model is fed with existing data for specific gestures (e.g., waving, pointing) it is also prompted to create variations on similar gestures.
- Synthetic IMU Data for Robotics:
- Enhanced Training: By integrating IMU sensor data or virtually tracing desired points on the robot arm, synthetic data can be generated for diverse movement scenarios. Eliminating the need for a physical robot opens many paths for quick feasibility studies, developing models, tuning and testing.
- Expanded Anomalies: Synthetic data can include a wider range of anomalies and movement variations that are time-consuming or impractical to collect with a real robot arm. For example, sensor data can be simulated for rare events such as joint malfunctions or unexpected collisions.
- Cost-Effective: Generating synthetic data avoids the cost and risk of damaging a physical robot during training and testing, especially for anomaly detection applications where exploring extreme scenarios is crucial.
- Time-Series Data Simulation: One of SensiML’s objectives is to integrate generative tools to create more realistic time-series data. Researchers and industrial designers often develop simulation software to study and optimize the behavior of instruments and/or environments. When such simulation toolkits are equipped with appropriate interfaces, the synthetically generated time-series from various sensors positioned on machines and/or in their environments can be accessed and integrated into the model generation pipeline.
Lastly, generative AI can be leveraged in the form of external and third-party AI services. In this approach, generative AI services are employed to generate data for specific use cases. Here, pre-trained models served by third-party vendors are accessed via their APIs, rather than directly integrating a specific generative model into the toolkit.
- Keyword Spotting Use Case Example: A “text-to-speech” generative model is a prime example of how this approach might extend the Piccolo AI tool. Collecting vast amounts of training data to build a keyword-spotting model can be resource-intensive. “Text-to-speech” services enable Piccolo AI to effortlessly generate large volumes of high-quality audio data, eliminating the need for extensive manual data collection. This allows the SensiML pipeline to focus on efficiently integrating the generated data, while the burden of data generation falls on external generative AI providers.
- AI-Powered Conversational Agents: Piccolo AI can benefit from an intelligent chat agent that assists users with their data needs, helps them set up experiments, guides them in collecting sensor data, and aids in data augmentation and model generation. While the SensiML toolkit offers a wide variety of pipeline templates and preprocessing transformations for different use cases, there is potential for improvement in creating custom pipelines for specific needs.
When equipped with the right data and commands, GPT-based services can provide insights into the components of the AutoML pipeline and resolve user confusion by helping them identify the appropriate feature generators and transformation functions. The ultimate goal would be for users to provide their data and describe their needs, allowing multiple generative agents to handle all aspects of MLOps—from data annotation to orchestrating and optimizing the model-building pipeline for their device of choice.
Challenges
All that said, it is worth mentioning that generative algorithms are still in the early stages of development and are not free of issues. One should be aware of their potential and limitations before fully trusting their results. These systems are often prone to “hallucination”, where they generate outputs that seem plausible but can not be rationally or physically mapped to real scenarios. Similar to any other AI/ML algorithm, the performance of these models heavily depends on the quality of the data they are built upon. Moreover, generative models often require substantial computational resources to train which makes them expensive to train. However, once trained they can be recycled and served many times in different ways. There are also concerns about their potential to generate deceptive content far from reality. Blind usage of such data may bias products or cause diversion. Understanding and addressing these issues is crucial for the effective and responsible deployment of generative AI technologies.
Summary
In conclusion, generative algorithms appear very promising and are rapidly evolving, if still far from perfection. Integrating generative AI into SensiML’s open-source Piccolo AI toolkit is a significant opportunity ripe for prototyping, testing, and ongoing refinement. Our vision at SensiML is to equip developers with powerful tools to build more efficient AI models for tiny devices. We see collaborative development from the broader community of experts and the open exchange of ideas as the best means to do so and have contributed our core IP to help bootstrap this notion. Generative models can address data shortage and enhance data diversity. When used to generate codes and transformations, they can create optimized solutions for data exploration, data processing, and feature extraction.
If you share our enthusiasm for the prospects of generative AI in the tinyML realm, we hope you will take the effort to explore the Piccolo AI GitHub repository and get involved to actively help contribute ideas, concerns, and code to advancing this exciting area.
