A vision chip with complementary pathways for open-world sen

A vision chip with complementary pathways for open-world sensing

Image sensors face substantial challenges when dealing with dynamic, diverse and unpredictable scenes in open-world applications. However, the development of image sensors towards high speed, high resolution, large dynamic range and high precision is limited by power and bandwidth. Here we present a complementary sensing paradigm inspired by the human visual system that involves parsing visual information into primitive-based representations and assembling these primitives to form two complementary vision pathways: a cognition-oriented pathway for accurate cognition and an action-oriented pathway for rapid response. To realize this paradigm, a vision chip called Tianmouc is developed, incorporating a hybrid pixel array and a parallel-and-heterogeneous readout architecture. Leveraging the characteristics of the complementary vision pathway, Tianmouc achieves high-speed sensing of up to 10,000 fps, a dynamic range of 130 dB and an advanced figure of merit in terms of spatial resolution, speed and dynamic range. Furthermore, it adaptively reduces bandwidth by 90%. We demonstrate the integration of a Tianmouc chip into an autonomous driving system, showcasing its abilities to enable accurate, fast and robust perception, even in challenging corner cases on open roads. The primitive-based complementary sensing paradigm helps in overcoming fundamental limitations in developing vision systems for diverse open-world applications. Inspired by the human visual system, a vision chip with primitive-based complementary pathways is developed to overcome the power and bandwidth wall of vision systems, achieving fast, precise, robust and high-dynamic-range sensing efficiently in the open world.

Related Keywords

Optik Photonik , , International Conference On Computer Visionand Pattern Recognition , International Conference Proc , Elsevier , International Image Sensor Workshop , International Solid , International Conference On Computer Vision , Electron Devices , High Performance Silicon Imaging , Integrated Imaging , Vision Techniques , Industrial Inspection , Signal Processing , Digital Still Cameras , Computer Visionand Pattern Recognition , Computer Vision , Pattern Recognition , Vehicles Symposium , Def Con , Dataset Shift , Machine Learning , Mit Press , Stereoscopic Vision , Solid State Circuits Conference , Integrated Circuits Conference , State Circuits , Solid State Circuits , Very Large Scale , Video Compression , Multimedia Engineering , Neural Science , Frame Interpolation , Convolutional Block Attention , European Conference , Medical Image Computing , Computer Assisted Intervention , International Conference , Only Look Once , Panoptic Driving ,

© 2024 Vimarsana