Think Twice, Act Once: Verifier-Guided Action Selection For Embodied Agents

Short summary

VegAS improves embodied AI agents by sampling multiple action candidates and using verification to pick the most robust choice, rather than committing to a single action. The approach uses curriculum learning to train verifiers on diverse failure modes. It achieves up to 36% performance improvement on complex multi-object, long-horizon tasks in Habitat and ALFRED benchmarks.

• •Verifier-guided ensemble selection improves MLLM-based embodied agent robustness
• •Curriculum learning exposes verifiers to diverse failure modes at training time
• •36% relative performance gain on complex multi-object, long-horizon tasks

Generated with AI, which can make mistakes.