How is Zephyr used for edge AI and sensors?

Zephyr is a scalable open-source real-time operating system (RTOS) hosted by the Internet of Things (IoT) and embedded technology section of the Linux Foundation. Its modular architecture makes it highly flexible and suitable for resource-constrained edge devices and sensors.

This article reviews Zephyr’s structure and features, examines how Zephyr supports neural networks, and discusses how Tiny Machine Learning (TinyML) and TensorFlow Lite Micro (TFLite Micro) combine to support ML on Zephyr. It concludes with an example of how Zephyr supports flexible edge connectivity.

Zephyr’s modularity enables scalability and performance optimization. Some key features include the following as shown in Figure 1:

Comprehensive, lightweight kernel
Integrated device drivers
Supports preemptive and cooperative threading
Memory protection includes stack overflow protection, kernel object and device driver permission tracking, and thread isolation.
Numerous connectivity options, including:
- Bluetooth 5.0 and Bluetooth Low Energy (with both controller and host, BLE Mesh
- Constrained application protocol (CoAP), light weight machine-to-machine (LwM2M), and message queuing telemetry transport (MQTT) IoT protocols
- USB & USB-C
- Wi-Fi, Ethernet, and CAN bus

Zephyr and neural nets

Neural networks can be highly useful in edge sensor applications, especially when handling noisy and unstructured data from accelerometers, biometric sensors, and cameras. Open-source frameworks specifically for supporting this class of applications have been developed to run Zephyr on computationally limited devices with minimal available memory. These frameworks can support optimization algorithms and neural compilers.

A unified application programming interface (API) has been developed for neural network inference frameworks on Zephyr. It includes methods for loading the models, entering inputs, running the inference engine, and capturing the results. It enables designers to select the inference engine that best matches the application requirements without significant changes to the application code.

For example, a hearing aid including an integrated deep neural network (DNN) has been implemented using Zephyr. The DNN was trained using 12 million sounds to enable the hearing aid to process audio inputs and provide an output optimized to support how the brain processes sound.

Hearing aids are highly resource-constrained devices that demand minimal power consumption, small code size, and, in this case, provide a significant amount of processing power. A custom chipset based on Zephyr was developed that can run always-on digital signal processing (DSP) algorithms. Several high-efficiency processing cores are included in the system. In addition to audio processing, they manage real-time and continuous wireless connectivity between the left and right hearing aids. The devices can run on a single button cell battery for an entire day.

TinyML and Zephyr

TinyML and TFLite Micro are powerful tools that enable ML algorithms to be run on resource-constrained devices like IoT sensor nodes. This can be especially important for real-time control applications that demand rapid processing and quick decision-making, where latencies associated with cloud connectivity cannot be tolerated.

TFLite Micro is an interpreter specifically designed to run algorithms like TinyML on resource-constrained platforms. A TFLite Micro module has been developed on the Zephyr RTOS, enabling designers to incorporate ML into low-power edge sensors and other embedded applications.

Zephyr and edge connectivity

Connectivity is an important consideration for many edge AI devices like IoT sensors. The Zephyr open-source BLE stack can support a variety of RF architectures. The controller link layer has been split into a common Upper Link generic for all implementations and a Lower Link Layer enabling developers to customize drivers for proprietary RF baseband devices, as outlined below in Figure 2.

In addition, the Zephyr BLE stack can support complex use cases like stereo hearing aids or wireless headsets connected to mobile phones, televisions, and other devices. Continuous audio streaming has strict timing requirements and needs to support relatively high data rates for long periods while maintaining synchronization between wirelessly connected stereo speakers.

Summary

Zephyr is a highly capable, modular, and scalable open-source RTOS. As shown, it can support DNNs for handling noisy and unstructured data on IoT nodes and sensors, run TinyML on those same devices and embedded systems, and provide customized BLE edge connectivity.