Active Observation and Visual Servoing
IEEE CASE 2026 · Accepted (Paper ThCT6.3) · First-Author Paper
Problem Formulation
In vision-guided robotic manipulation, fixed external cameras and single-arm eye-in-hand configurations suffer from a fundamental geometric limitation: as the gripper approaches the target object for terminal alignment, the manipulator structure or gripper fingers inevitably cast an occlusion cone ($\Omega_{\text{occ}}$) over the target. Losing visual feedback during the final centimeters of grasping degrades closed-loop accuracy under calibration error or target perturbation.
Technical Approach & Architecture
This work introduces a dual-arm active observation and relative visual servoing framework that decouples viewpoint control from manipulation execution while coupling them in image feedback space:
- Two-Stage Viewpoint Strategy: Coordinates global approach alignment (Stage I) with terminal co-visibility optimization (Stage II), positioning an active RGB-D camera arm to maintain simultaneous observation of both the target object and the grasping gripper.
- Target–Gripper Relative Visual Servoing: Formulates closed-loop control directly on the relative image-space error $\mathbf{e}{\text{rel}}(t) = \mathbf{s}{\text{target}}(t) - \mathbf{s}_{\text{gripper}}(t)$, reducing sensitivity to extrinsic camera-to-base calibration drift.
- Occlusion-Triggered Viewpoint Exploration: Continuously monitors visual feedback quality and actively repositions the camera arm along collision-free viewpoint candidates when self-occlusion or environmental obstruction arises.
Publication
Active Observation for Robust Visual Servoing in Robotic Manipulation
Zhilun Hu, Yuqi Cheng, Yuxiang Tan, Wen-pan Li, Weiming Shen.
IEEE International Conference on Automation Science and Engineering (CASE), 2026.