A robot in Cornell's Personal Robotics Lab has learned to
foresee human action in order to step in and offer a helping hand, or more
accurately, roll in and offer a helping claw.
Understanding when and where to pour a beer or knowing when
to offer assistance opening a refrigerator door can be difficult for a robot
because of the many variables it encounters while assessing the situation. A
team from Cornell has created a solution.
How it works
Gazing intently with a Microsoft Kinect 3-D camera and using
a database of 3D videos, the Cornell robot identifies the activities it sees,
considers what uses are possible with the objects in the scene and determines
how those uses fit with the activities. It then generates a set of possible
continuations into the future – such as eating, drinking, cleaning, putting
away – and finally chooses the most probable. As the action continues, the
robot constantly updates and refines its predictions.
"We extract the general principles of how people
behave," said Ashutosh Saxena, Cornell professor of computer science and
co-author of a new study tied to the research. "Drinking coffee is a big
activity, but there are several parts to it." The robot builds a
"vocabulary" of such small parts that it can put together in various
ways to recognize a variety of big activities, he explained.
In tests, the robot made correct predictions 82% of the time
when looking one second into the future, 71% correct for three seconds and 57%
correct for 10 seconds.
"Even though humans are predictable, they are only
predictable part of the time," Saxena said. "The future would be to
figure out how the robot plans its action. Right now we are almost hard-coding
the responses, but there should be a way for the robot to learn how to