The robots watch human-interaction videos to master household chores

Scarlett Evans, Assistant Editor, IoT World Today

July 25, 2022

2 Min Read

Researchers from Carnegie Mellon University have developed a new learning method for robots based on imitating human behavior. 

The In-the-Wild Human Imitating Robot Learning, or WHIRL, method involves robots watching videos of humans undertaking generic tasks and mimicking their behavior to ultimately complete the same actions themselves.

“Imitation is a great way to learn,” said Shikhar Bahl, a Ph.D. student at the Robotics Institute at Carnegie Mellon University. “Having robots actually learn from directly watching humans remains an unsolved problem in the field, but this work takes a significant step in enabling that ability.”

Adding a camera and newly-developed software to a robot, the team taught the unit to perform more than 20 household tasks, ranging from opening and closing appliances to putting a lid on a pan and taking out the garbage. For each task, the robot watched a human completing the task once, and practiced mimicking the action until it succeeded.

The team presented its research this month at the Robotics: Science and Systems conference in New York.

“This work presents a way to bring robots into the home,” said Deepak Pathak, a member of the research team. “Instead of waiting for robots to be programmed or trained … this technology allows us to deploy the robots and have them learn how to complete tasks, all the while adapting to their environments and improving solely by watching.”

Typically, imitation and reinforcement learning methods are the most popular for training robots, however, these approaches bring challenges in requiring high levels of data to train the algorithm – known as “sample inefficiency” – as well as being constrained to lab settings. 

The Carnegie method looks to tackle this by turning to third-person perspective learning. As WHIRL also allows robots to learn from any video, the algorithm is scalable and applicable to all home environments, with particular use cases in situations with elderly or impaired homeowners. 

About the Author(s)

Scarlett Evans

Assistant Editor, IoT World Today

Scarlett Evans is the assistant editor for IoT World Today, with a particular focus on robotics and smart city technologies. Scarlett has previous experience in minerals and resources with Mine Australia, Mine Technology and Power Technology. She joined Informa in April 2022.

Sign Up for the Newsletter
The most up-to-date news and insights into the latest emerging technologies ... delivered right to your inbox!

You May Also Like