(Utilizing Finger Sensors Combined with an "AI Brain" to Understand and Operate Physical Objects)

Meta states that the core of developing corresponding AI robots lies in enabling the robot's sensors to perceive and understand the physical world, while the "AI brain" precisely controls the robot's responses to that world. The tactile perception capabilities currently being developed by the team aim to allow robots to detect the material and texture of the objects they interact with, enabling the AI to determine how the robot should operate these items (for example, picking up an egg).

Meta has announced several research achievements, including Meta Sparsh, Digit 360, and Meta Digit Plexus. Among them, Meta Sparsh is an AI-based tactile encoder that primarily uses the self-supervised learning capabilities of AI to achieve cross-scene tactile perception. This allows the robot's AI brain to flexibly "perceive" the characteristics of relevant objects after learning the "tactile feel" of a particular item.

Digit 360, on the other hand, is a high-precision sensor that is mainly deployed on the robot's fingers, claiming to possess multimodal sensing capabilities that can capture subtle tactile changes. Meta asserts that this sensor can simulate human touch and supports multiple sensing abilities, including vibration and temperature.

Meta Digit Plexus is an open platform that aims to integrate a variety of sensors, enabling robots to more comprehensively perceive their surroundings through sensors and interface with the AI brain in real-time through a unified standard.

In addition, Meta has developed the PARTNR benchmark, a testing framework designed to evaluate human-robot collaboration capabilities. This framework aims to test the planning and reasoning abilities of robotic AI systems in real-life scenarios and includes 100,000 natural language tasks that simulate various situations in a home environment, helping developers assess the robots' understanding and execution capabilities of natural language.