VR-Loop: Dual-Arm Rope Untangling

Manuscript Under Review · Second-Author Paper

Problem Formulation

Deformable linear objects (such as tangled ropes and cables) exhibit high-dimensional configuration spaces and complex self-occlusions. When vision-language models (VLMs) are applied in an open-loop manner to propose manipulation primitives, spatial grounding errors can place grasp points off the rope mask or pull along directions that tighten rather than loosen the tangle.

Technical Approach & Architecture

VR-Loop reformulates VLM-based manipulation from one-shot open-loop prediction into an iterative pre-execution Verify-and-Refine loop:

1. Action Proposal & Image-Overlay Rendering: Candidate dual-arm grasp points and pulling vectors ${(\mathbf{p}_1, \mathbf{v}_1), (\mathbf{p}_2, \mathbf{v}_2)}$ are projected and visually rendered onto the RGB observation.
2. Closed-Loop VLM Critique: The verifier inspects the overlaid candidate action against knot geometry and crossing topology, rejecting actions that risk tightening the tangle and generating structured directional feedback.
3. Mask-Based & Depth-Informed Geometric Execution: Accepted interaction primitives are constrained to segmented rope masks $\mathcal{M}_{\text{rope}}$ and lifted via RGB-D depth geometry into collision-free dual-arm trajectories.

Manuscript

Iterative Verify-and-Refine Vision-Language Agent for Reliable Dual-Arm Manipulation
Yuqi Cheng, Zhilun Hu, Cheng Jiang, Qingyu Peng, Weiming Shen.
Manuscript under review, 2026.

Publication Entry