It is difficult to train a general-purpose robot. Sapna is a robot like Jetson’s livelihood Perform a series of Family Task, such as washing or folding clothes. But for this to happen, the robot needs to learn from one Large amount of data It matches the real-world conditions-this can be difficult to collect data. Currently, most training data are collected from several stable cameras that are to be installed carefully to collect useful information. But what if bots can learn from everyday conversations that we already have with the physical world?
This is a question that General-purpose robotics and AI Lab In Nyu, led by Assistant Professor Lerael pintoExpected to answer with EgazeroA smart-glass system that helps in learning robots by collecting data with a soup-up version of meta glasses.
One in Recently pre-printWhich serves as a proof of the concept to the approach, researchers trained a robot to complete seven manipulations, such as lifting a piece of bread and putting it on a nearby plate. For each task, he collected 20 minutes of data from humans doing these tasks while recording his tasks with Meta glasses. Project Aria(These sensor-laden glasses are used exclusively for research purposes.) Then when these functions are deployed to complete these functions with a robot, the system achieved 70 percent success rate.
Benefits of Egoistic Data
The “ego” part of Egazero refers to the “arrogant” nature of the data, which means that it is collected from the point of view of a person doing a working person. “The camera walks with you,” as how our eyes go with us, say Rawk bhirangiA postdorel researcher in Nyu Lab.
It has two main benefits: First, the setup is more portable than external cameras. Second, the glasses are more likely to capture the required information because the wear will ensure that they – and thus the camera – can see what is necessary to do a task. “For example, I say that I had something bent under a table and I want to make it unique. I will lean down, I will look at that hook and then make it unique, as opposite to a third person’s camera, who is not active,” says Bhirarangi. “With this egoistic perspective, you are cooked that information in your data for free.”
The second half of the Egosero name refers to the fact that the system is trained without any robot data, which can be expensive and difficult to collect; Human data alone is enough for robots to learn a new task. It is capable of a framework developed by the lab of Pinto that tracks points into space rather than full images. “Training of robots on image-based data,” what a human hand looks like and what a robot weapon looks like, “says Bhirangi. “This structure tracks the points on the hand, which is mapped on the points on the robot.
The Egosero system takes data from humans wearing smart glasses and converts it into 3D navigation data used for robots to perform normal manipulation tasks.Vincent Liu, Edmi Edenji, Hotian Zan et al.
Reducing the points in 3D space means the image means that the model can track the movement in the same way regardless of specific robot appendages. Bhirangi says, “As long as the robots run in the same way relative to the point object, the way humans run, we are good,” says Bhirangi.
It all leads to a common model that requires a lot of diverse robot data to otherwise train. If the robot was trained on the data lifting a piece of bread, a roll – it can normalize that information to take a piece of ciabatta in a new environment.
A scalable solution
In addition to Egosero, the research group is working on several projects to help using general-purified robots a reality, including open-source robot design, flexible, flexible Touch sensorAnd in addition to collecting real -world training data.
For example, as an alternative to Egosero, researchers have also designed a setup with a 3D-printed handheld gripper that looks more closely as most robots “hands”. A smartphone attached to the gripper captures the video with the same point-intercourse method that is used in Egazero. But by collecting data without bringing a robot to their homes, both approach can provide more scalable solutions to collect training data.
That scalability is eventually the researcher’s goal. Large language models can use the entire internet, but there is no internet equivalent for the physical world. Tapping in everyday conversations with smart glass can help in filling that difference.
From your site articles
Related articles around web
