Renesas Electronics Corporation announced the joint development of a voice-controlled multimodal AI solution that enables low-power contactless operation for image processing in vision AI-based IoT and edge systems, such as self-checkout machines, security cameras, and video conference systems, and smart appliances such as robotic cleaning devices.
The new solution combines the Renesas RZ/V Series vision AI microprocessor unit (MPU) and the low-power multimodal, multi-feature Syntiant NDP120 Neural Decision Processor to deliver advanced voice and image processing capabilities. The joint solution features always-on functionality with quick voice-triggered activation from standby mode to perform object recognition, facial recognition, and other vision-based tasks that are critical functions in security cameras and other systems. For example, while user-defined voice cues drive activation and system operation, vision AI recognition tracks operator behavior and controls operation or issues a warning when suspicious actions are detected.
The multimodal architecture makes it easier to create contactless user experiences for vision AI-based systems. Using a dedicated, power-efficient chip for voice recognition reduces standby power consumption while speeding up system development because it is possible to develop software independently of the vision AI functionality.
The Renesas RZ/V Series MPU for vision AI incorporates Renesas’ exclusive DRP-AI (Dynamically Reconfigurable Processor-AI) accelerator and combines high-precision AI inference with a power efficiency that is among the best in the industry. This superior power performance eliminates the need for heat dispersion measures such as heat sinks or cooling fans, which reduces the bill of materials (BOM) cost and makes it possible to integrate vision AI into a wide range of embedded applications.
The Syntiant NDP120 chip incorporates sophisticated AI capabilities that can be used to implement many high-precision, hands-free voice functions, including speech recognition, keyword detection, multiple wake words, and local command recognition. Packaged with the Syntiant Core 2 neural network inference engine, the NDP120 can also run multiple applications simultaneously while minimizing power consumption to 1mW battery power.
The new voice-controlled multimodal AI solution uses multiple mutually compatible devices from the broader Renesas portfolio to provide customers an elevated prototyping platform for faster time to market and reduced risk. The new solution is part of Renesas’ Winning Combinations, which feature compelling analog, power, and embedded processing product combinations that help customers accelerate their designs and get to market faster.
The reference design for the new multimodal AI solution is available now, including circuit diagrams and BOM lists. For more information, please visit the multimodal AI solution