New Benchmark Measures Memory Capabilities of VLM-Powered Robots

In-home assistive robots might be able to clean and organize your home, but if they can’t tell you where they put your keys or wallet, they can become a major inconvenience.

To improve robot performance and advance memory capabilities needed for reliable home use, Georgia Tech researchers have developed a new benchmark to evaluate the memory capabilities of vision-language models (VLMs).

That includes Gemini 2.0-flash and GPT-4o.

In-home assistive robots could clean and organize a home while the owner is away, but without enhanced memory capabilities, they could cause more problems than they solve.

They could, for example, cause a major inconvenience if they move important objects like wallets and keys but are unable to tell the owner where they put them.

A new benchmark created by Georgia Tech researchers tests the memory capability of vision-language models (VLMs), including Gemini 2.0-flash and GPT-4o, to determine whether they could improve robot performance.