3 votes Robot learns to see in thirty minutes (2022) Posted March 27 by skybrian Tags: robotics, vision, machine learning, author.antonio loquercio, author.ashish kumar, author.jitendra malik https://antonilo.github.io/vision_locomotion/ Link information This data is scraped automatically and may be incorrect. Title Learning Visual Locomotion with Cross-Modal Supervision Word count 497 words 1 comment Collapse replies Expand all Comments sorted by most votes newest first order posted relevance OK skybrian (OP) March 27 Link Here's the abstract: Interesting videos on the page. Here's the abstract: In this work, we show how to learn a visual walking policy that only uses a monocular RGB camera and proprioception to walk. Since simulating RGB is hard, we necessarily have to learn vision in the real world. We start with a blind walking policy trained in simulation. This policy can traverse some terrains in the real world but often struggles since it lacks knowledge of the upcoming geometry. This can be resolved with the use of vision. We train a visual module in the real world to predict the upcoming terrain with our proposed algorithm Cross-Modal Supervision (CMS). CMS uses time-shifted proprioception to supervise vision and allows the policy to continually improve with more real-world experience. We evaluate our vision-based walking policy over a diverse set of terrains including stairs (up to 19cm high), slippery slopes (inclination of 35 degrees), curbs and tall steps (up to 20cm), and complex discrete terrains. We achieve this performance with less than 30 minutes of real-world data. Finally, we show that our policy can adapt to shifts in the visual field with a limited amount of real-world experience. Interesting videos on the page. 1 vote
Here's the abstract:
Interesting videos on the page.