MindTopo is a controlled diagnostic benchmark developed by researchers at Northwestern University and Stanford to evaluate vision-language models' spatial and topological reasoning abilities. Testing across a broad set of proprietary and open-weight models revealed that performance was consistently stronger on static reasoning than on interactive planning, though both remained well below human performance.
The error patterns differed between task types. Static reasoning mistakes usually began with perception failures, such as missing a wall, opening, or crossing. Planning mistakes occurred after scenes were understood, when models followed locally plausible moves without tracking later consequences, lost the task over multiple turns, or proposed actions that violated environment dynamics.
Researchers also tested whether image and video generation could help models maintain topological understanding. Image generation sometimes helped when the relevant relation was visible in a single frame but remained unreliable across sequences of crossings or moves. Video rollouts frequently altered topology or violated task dynamics.
The findings highlight a gap for applications like robots, accessibility tools, and interactive assistants, which must understand not only object locations but also what remains connected, enclosed, ordered, or knotted as actions unfold. The authors suggest closing this gap may require models that carry explicit topological state or world models whose predictions preserve topology by construction.