Vision-language grounding for cluttered bin picking

A lab at FU Berlin combined a vision-language model with a classical grasp planner and reports 31% fewer grasp failures on the standard cluttered-bin benchmark.

The approach

Instead of end-to-end learning, the system uses the VLM only for object disambiguation — deciding what to pick when instructions are vague — and hands the actual grasp synthesis to a geometry-based planner.

• Natural-language picking ("the smaller blue connector") without task-specific training
• Runs on a single consumer GPU next to the robot
• Failure cases concentrate in transparent and reflective objects, consistent with prior work

The paper is on arXiv; an interactive demo is promised for the next open lab night.