Part 1: The plan to create an acoustic-aware smart door application
One need not look very long to find examples of smart connected devices increasingly making their way into every aspect of our homes and daily life. Smart lightbulbs, smart hubs, thermostats, appliances, and even smart dog bowls. Nearly all these devices are connected, and some are even what you might call “smart” — though the catch is most get their wits via the cloud or a companion smartphone app that provides the intelligence. Few such devices are actually smart in their own right by utilizing edge inferencing to provide autonomous local insight they can process and act upon without a network-dependent assist. That limitation is quickly changing with the latest embedded silicon and microcontrollers targeting IoT edge devices possessing more powerful processors and even purpose-built AI acceleration.
Recent hardware advances are enabling this trend toward new truly smart IoT edge applications and devices. Continuous hardware innovation is bringing AI out of the cloud and down to the ultra-low power silicon powering IoT edge devices themselves. In addition to the hardware, edge AI-capable chips also need powerful companion software tools that allow developers to easily tap the full potential of these new processors. Doing so without incurring a steep AI learning curve is a known challenge. After all, the process of constructing a product-worthy, data-driven AI model is a multi-disciplinary one involving AI, digital signal processing, domain expertise, and firmware optimization. These have typically been specialist areas with specialist tools that have evolved independently to suit expert users in each of these specific aspects of embedded sensor data processing. Fortunately, the latest hardware advances are now accompanied by new AutoML software tools like SensiML’s Analytics Toolkit that unify and streamline the workflow for building truly smart IoT edge devices as we shall demonstrate over the course of this blog series.
Introducing Silicon Labs ‘ MG24 and BG24 AI-Accelerated SoCs
One noteworthy example of this latest generation of AI-capable edge processors comes from SensiML partner Silicon Labs. The arrival of Silicon Labs’ MG24 and BG24 SoCs with AI acceleration combined with the latest release of SensiML Analytics Toolkit, supporting the full capabilities of these devices, provides developers a major step forward in building truly smart IoT edge devices. Specifically, for advanced use cases like audio classification that require complex neural network models, the combination of the Silicon Labs MG24 / BG24 SoC family and SensiML allows power-efficient, low-latency AI to be easily created by project teams with existing IoT development skillsets.
To illustrate what is now possible using this combined hardware/software solution, in this multi-part blog series we will embark on building a truly smart door lock that utilizes only a single microphone and powerful AI inferencing at the edge to determine a variety of acoustic events of interest for productizing a better, safer home entry lock.
The Concept – An Acoustic Aware Smart Door
We’ve seen a variety of connected devices appearing in the market to improve our experience with the first aspect one encounters in a smarthome… the front entry door. Smart doorbells now integrate connected cameras, door locks have connectivity and interfaces to allow for flexible and programmable access alternatives to a conventional lock and key. To this mix, what we will explore with this blog series is a concept for adding acoustic awareness and intelligence to such devices. Utilizing the Silicon Labs xG24 Dev Kit and SensiML Analytics Toolkit, we will integrate a microphone and AI processing to bring additional autonomous insight to a connected smart door.
Our acoustic-aware, truly smart door lock will monitor ambient sounds with a mechanical layout that helps accentuate the signal of structural sounds within the door itself. Using data-driven AutoML model building and the SensiML Analytics Toolkit, we will capture a variety of interesting acoustic events with the intent of building a recognition model that can identify such events locally without the need for networked AI, the cloud, or companion smartphone processing.
Some of the interesting events we will capture include:
- Door Knocking
- Keylock Insertion
- Deadbolt Engagement
- Extraneous Sounds (to avoid false-positive events)
Expanding on the concept, there are many additional events that could be added to our list:
- Lock tampering
- Door pounding / banging
- Attempted break-ins
- Glass breakage
- User voice identification
In each installment of this series we will progress through the development effort of our smart acoustic-aware door lock covering the following:
Part 2: Acoustic raw sensor data collection and labeling using SensiML Toolkit
Part 3: Edge AI model generation for the AI-accelerated SiLabs xG24 Dev Kit
Part 4: Profiling the performance of the AI-accelerated EFR32MG24 model
Part 5: Using data augmentation to enhance model accuracy <Coming Soon>
Follow us in this multi-part blog as we showcase how the Silicon Labs EFR32xG24 Dev Kit and SensiML Analytics Studio can transform the notion of smart in smart connected home IoT devices.
Learn more about SensiML’s Accelerated AI Solution with Silicon Labs
