TL;DR
- Problem: End-to-end VLAs often lose reliable language–vision grounding under visual distraction, including competing objects, absent referents, background shifts, and unseen objects.
- Idea: DRAGON-VLA integrates frozen, two-stage object-centric grounding and masked relative-depth grounding with a VLA fine-tuned only on clean, single-object demonstrations.
- Results: DRAGON-VLA retains around 80% task success with seven distractors while the evaluated baselines fall below 10%, and achieves approximately 95% absent-target rejection. Ablations identify object-centric filtering as the main source of improvement and depth grounding as most beneficial on unseen objects.
Abstract
Vision–Language–Action (VLA) models adapt pretrained Vision–Language Models (VLMs) for robotic control, yet often fail to bind instructions to the correct visual evidence. Our single-object sanity check shows that state-of-the-art policies frequently grasp the visible object when the requested referent is absent. We ask whether explicit grounding can improve instruction–scene alignment under visual distraction and distribution shift without distractor-rich demonstrations or auxiliary perceptual objectives.
We propose Distraction-Robust Action via Geometric and Object-ceNtric Grounding (DRAGON-VLA), an integrated perception–control system. Its two-stage object-centric module uses an off-the-shelf VLM to select task-relevant segmentation proposals in a base view and semantically match the same objects in a wrist view. A geometric module converts the selected regions into masked monocular relative-depth observations that preserve shape and spatial layout while suppressing appearance, background, and distractors. These grounded multi-view inputs condition a pretrained VLA fine-tuned only on clean, single-object demonstrations. DRAGON-VLA thus provides a task-aware, calibration-free interface combining base-view anchoring, semantic cross-view matching, and explicit depth grounding without additional high-level grounding supervision.
On a real-world UR10e tabletop setup, DRAGON-VLA outperforms strong VLA baselines under competing distractors, absent referents requiring abstention, background shifts, and held-out object categories. With seven distractors, the grounded policies retain around 80% task success while the baselines fall below 10%; they also achieve approximately 95% absent-target rejection. Ablations show that object-centric filtering accounts for the largest gains, while masked relative-depth inputs provide their clearest additional benefit on unseen objects. These results support explicit perception grounding as an effective systems intervention for referentially reliable VLA manipulation. Physical clutter requiring obstacle-aware motion and interaction remains outside scope.
Approach
DRAGON-VLA takes synchronized base- and wrist-view RGB observations and first uses a real-time segmentation model to generate object-mask proposals. Given the instruction, an off-the-shelf VLM selects task-relevant proposals in the base view and semantically matches the same objects in the wrist view using base-view reference crops. The selected masks suppress irrelevant regions in both views. A geometric grounding module then applies these masks to monocular relative-depth estimates, producing task-conditioned observations that preserve object shape and spatial layout while reducing appearance, background, and distractor cues. These grounded observations, together with the instruction and robot proprioception, condition a pretrained VLA fine-tuned only on clean, single-object demonstrations. The perception grounding module remains frozen, and the interface does not require camera calibration.
Experiments
Visually Distracting Scenes with Distractor Objects
DRAGON-VLA consistently maintains attention on the instructed target even with 1 to 7 distractors, whereas end-to-end VLA baselines are often diverted to task-irrelevant objects and degrade sharply as visual distraction increases. With object- and geometry-grounded inputs, DRAGON-VLA sustains roughly 80 to 90 percent success with multiple distractors, indicating that explicit perceptual grounding is the key driver of reliable language following rather than additional action-centric tuning.
Spatial Reasoning
”place object on the left in the bin”
”place object in the middle in the bin”
”place object on the right in the bin”
With purely relational prompts (“left/middle/right”), DRAGON-VLA’s cross-view grounding disambiguates partially visible objects, driving large gains over Pi-0/FAST. Base-view anchors keep wrist-view grounding from drifting to the most salient blob, so spatial picks stay reliably correct.
Absent-Target Rejection
”pick up the mustard bottle and place it in the bin”
When the instruction is infeasible (target missing), DRAGON-VLA almost always abstains. The heatmaps for DRAGON Pi-0 and DRAGON Pi-0 FAST show near-zero pick-up rates on off-diagonal requested-versus-shown pairs, while end-to-end baselines retain large off-diagonal mass and still grasp more than 75 percent of the time.
Background Appearance Shift
”place the instructed item despite background shift”
Swapping tablecloths, backdrops, or both barely dents DRAGON-VLA, while baselines lose 10 to 30 points. The grounded inputs suppress background textures, so the policy does not overfit to new colors or patterns. Appearance shifts are handled by filtering out background variation rather than relying on the policy to generalize from action tuning alone.
Unseen Objects under Visual Distraction
Even when every item is novel, DRAGON-VLA keeps success high while baselines crater. Geometry-aware grounding decouples grasp decisions from texture and color priors learned on seen objects, so novel bottles and bags still map cleanly to instructions, enabling zero-shot object generalization without synthetic distractor data.
BibTeX citation
@misc{dragonvla, author = "{Khoa Vo and Taisei Hanyu and Yuki Ikebe and Trong Thang Pham and Nhat Chung and Minh Nhat Vu and Duy Nguyen Ho Minh and Anh Nguyen and Anthony Gunderman and Chase Rainwater and Ngan Le}", title = "Distraction-Robust Vision-Language-Action Models through Object-Centric and Geometry Grounding", year = "2025", eprint = "2512.22519", archivePrefix = "arXiv", primaryClass = "cs.RO",}