A bio-inspired camera paired with AI can identify obstacles around a vehicle more swiftly and efficiently than existing systems, consuming less computational power. This innovation comes from Daniel Gehrig and Davide Scaramuzza at the University of Zurich’s Department of Informatics, as reported in Nature.
Traditional cameras used in driver assistance systems capture images at regular intervals, typically between 30 to 50 frames per second. These frame-based cameras, combined with artificial neural networks, help recognize objects such as pedestrians, bicycles, and other vehicles.
However, the time gap of 20 to 30 milliseconds between frames can result in delayed detection of fast-moving objects. Increasing the frame rate could address this issue but would also demand more real-time data processing and computational power, according to Gehrig.
READ MORE: Marelli and Hesai Group Unveil Lidar-Integrated Headlamps for Enhanced Vehicle Safety
Event cameras, a recent advancement, feature smart pixels that record information upon detecting rapid movements, eliminating blind spots between frames. “These neuromorphic cameras mimic human vision and can detect obstacles more quickly,” explained Scaramuzza, head of UZH’s Robotics and Perception Group.
Despite their advantages, event cameras struggle with slow-moving objects, and their data isn’t easily compatible with AI training algorithms.
Gehrig and Scaramuzza’s solution is a hybrid system that combines a standard camera, capturing 20 frames per second, with a convolutional neural network trained to identify vehicles and pedestrians. This is integrated with data from an event camera processed by an asynchronous graph neural network, an AI model adept at analyzing dynamic 3D data.
The innovative system allows for rapid detection akin to a standard camera operating at 5,000 frames per second but with the data bandwidth of a 50-frame-per-second camera. “Our visual detector matches the speed of high-frame-rate cameras while using significantly less bandwidth,” Gehrig noted.
Tests showed that their hybrid system outperformed top automotive cameras and visual algorithms, providing detections a hundred times faster while reducing data transmission and computational demands without sacrificing accuracy. The system excels at identifying vehicles and pedestrians that appear between frames of the standard camera.
The researchers believe that integrating their hybrid system with LiDAR sensors could further enhance its capabilities. “Such hybrid systems are essential for advancing autonomous driving, ensuring safety without a significant increase in data and computational requirements,” Scaramuzza concluded.
READ MORE: James Bond-Style Amphibious Car for Sale at £70,000: Road and Water Adventure Awaits
Subscribe today for the freshest car news delivered to your inbox
