How is Zephyr used for edge AI and sensors?

Zephyr is a scalable open-source real-time operating system (RTOS) hosted by the Internet of Things (IoT) and embedded technology section of the Linux Foundation. Its modular architecture makes it highly flexible and suitable for use on resource-constrained edge devices and sensors.

This article reviews Zephyr’s structure and features, examines how Zephyr supports neural networks and discusses how Tiny Machine Learning (TinyML) and TensorFlow Lite Micro (TFLite Micro) combine to support ML on Zephyr. It concludes with an example of how Zephyr supports flexible edge connectivity.

Zephyr’s modularity enables scalability and performance optimization. Some key features include (Figure 1):

Comprehensive, lightweight kernel.
Integrated device drivers.
Supports preemptive and cooperative threading.
Memory protection includes stack overflow protection, kernel object and device driver permission tracking, and thread isolation.
Numerous connectivity options including
- Bluetooth 5.0 and Bluetooth Low Energy (with both controller and host, BLE Mesh.
- Constrained application protocol (CoAP), light weight machine-to-machine (LwM2M), and message queuing telemetry transport (MQTT) IoT protocols
- USB & USB-C.
- Wi-Fi, Ethernet, and CANbus.

Zephyr and neural nets

Neural networks can be highly useful in edge sensor applications, especially when handling noisy and unstructured data from devices like accelerometers, biometric sensors, and cameras. Open-source frameworks specifically for supporting this class of applications have been developed for running Zephyr on computationally limited devices with minimal available memory. These frameworks can support optimization algorithms and neural compliers.

A unified application programming interface (API) has been developed for neural network inference frameworks on Zephyr. It includes methods for loading the models, entering inputs, running the inference engine, and capturing the results. It enables designers to select the inference engine that best matches the application requirements without significant changes to the application code.

For example, a hearing aid including an integrated deep neural network (DNN) has been implemented using Zephyr. The DNN was trained using 12 million sounds to enable the hearing aid to process audio inputs and provide an output optimized to support how the brain processes sound.

Hearing aids are highly resource-constrained devices that demand minimal power consumption, small code size, and, in this case, provide a significant amount of processing power. A custom chipset based on Zephyr was developed that can run always-on digital signal processing (DSP) algorithms. Several high-efficiency processing cores are included in the system. In addition to audio processing, they manage real-time and continuous wireless connectivity between the left and right hearing aids. The devices can run for an entire day on a single-button cell battery.

TinyML and Zephyr

TinyML and TFLite Micro are powerful tools that enable ML algorithms to be run on resource-constrained devices like IoT sensor nodes. This can be especially important for real-time control applications that demand rapid processing and quick decision-making, where latencies associated with cloud connectivity cannot be tolerated.

TFLite Micro is an interpreter specifically designed for running ML algorithms like TinyML on resources constrained platforms. A TFLite Micro module has been developed for use on the Zephyr RTOS. That enables designers to incorporate ML into low-power edge sensors and other embedded applications.

Zephyr and edge connectivity

Connectivity is an important consideration for many edge AI devices like IoT sensors. The Zephyr open-source BLE stack can support a variety of RF architectures. The controller link layer has been split into a common Upper Link that is generic for all implementations and a Lower Link Layer that enables developers to customize drivers for proprietary RF baseband devices (Figure 2).

In addition, the Zephyr BLE stack can support complex use cases like stereo hearing aids or wireless headsets connected to mobile phones, televisions, and other devices. Continuous audio streaming has strict timing requirements and needs to support relatively high data rates for long periods while maintaining synchronization between wirelessly connected stereo speakers.

Summary

Zephyr is a highly capable modular and scalable open-source RTOS. As shown, it can support DNNs for handling noisy and unstructured data on IoT nodes and sensors, run TinyML on those same devices and embedded systems, and provide customized BLE edge connectivity.