CoRA-VTLA: Contact-Guided Attention and Queryable Response Memory for Tactile Post-Training
Abstract
In precision manipulation, small errors in alignment or force can cause jamming or deformation, making contact feedback essential for adapting pretrained vision–language–action (VLA) models to these tasks. A single tactile observation leaves the response to recent actions ambiguous, and directly introducing tactile inputs during post-training can disrupt general-purpose capabilities acquired during pretraining and draw visual attention away from contact regions. To address these challenges, CoRA-VTLA augments pretrained VLAs with lightweight tactile modules. A queryable action–response memory retrieves recent contact responses so that the action expert can combine them with immediate tactile feedback. Contact-guided gaze supervision concentrates the visual attention of the tactile semantic token on image regions corresponding to fingertip contact. CoRA-VTLA achieves 73.33% mean success across three MuJoCo tasks, compared with 50% for direct tactile fusion with the same sensor inputs. It also achieves 67% success on an optical-tactile task from the public UniVTAC benchmark and 66.7% mean success across three real-robot tasks.
CoRA-VTLA at a glance
(a) Blocked insertions with fingertip pressure maps and load traces around withdrawal. (b) A tactile semantic token enters the VLM, while current contact and queried action–response memory condition the action expert. (c) Success rates in MuJoCo, on UniVTAC Insert Tube, and on three real-robot tasks. (d) Rollout keyframes.
Method
CoRA-VTLA adds three lightweight tactile pathways to a pretrained π0.5 policy: a current contact token carrying the instantaneous pressure and wrist force/torque, a queryable action–response memory token that encodes the last 0.8 s of measured contact responses and is read with a query built from the current contact and robot state, and a tactile semantic token appended to the VLM prefix, grounded by contact-guided gaze supervision.
The two action-side encoders are first initialized by response pretraining: from task demonstrations alone, they are trained to predict the next-step change in pressure, wrist force/torque, and robot motion, after which the temporary prediction head is discarded. All modules are then trained jointly with the full policy under the standard flow-matching objective, with attribute and gaze supervision on the semantic branch.
Simulated rollouts
Plug insertion






A plug insertion completed by CoRA-VTLA.
Knob operation






CoRA-VTLA grasps the knob, presses to unlock, and turns it to the target angle.
Nail insertion






A nail insertion completed by CoRA-VTLA.
The signals behind a recovery



A successful plug insertion: fingertip load, wrist wrench, and realized displacement — the responses the memory token reads.
Rollouts of CoRA-VTLA
Recovery after blocked contact on plug, knob, and nail insertion.
Real robot
On the real robot, the vision-only baseline keeps pressing after contact is blocked, tilts or drops grasped objects, and trips force limits — while CoRA-VTLA registers the load response, adjusts, and completes the task.
Vision-only baseline (failures)
Pressing through the blockage until failure · force-limit trip · tilting the cup during stacking.
CoRA-VTLA (successes)
Withdrawing and realigning after a blockage · gentle pressing on contact · a complete task episode.
Platform





Real-robot tasks and rollouts. CoRA-VTLA achieves the highest success rate on all three tasks — cup stacking, stamping, and tube insertion — averaging 66.7%, 22.0 points above the strongest baseline.
Results
| Method | MuJoCo (%) | UniVTAC (%) | Real robot (%) | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Nail | Plug | Knob | Avg. | Insert Tube | Cup | Stamp | Tube | Avg. | |
| Base (π0.5) | 74.0 | 0.0 | 20.0 | 31.33 | 46 | 38 | 30 | 30 | 32.7 |
| Direct | 68.0 | 32.0 | 50.0 | 50.00 | 51 | 50 | 48 | 34 | 44.0 |
| FTP-π0.5 | 56.0 | 4.0 | 72.0 | 44.00 | 57 | 46 | 44 | 44 | 44.7 |
| CoRA-VTLA | 78.0 | 76.0 | 66.0 | 73.33 | 67 | 68 | 76 | 56 | 66.7 |
Success rates under identical demonstrations, evaluation scenes, and tactile inputs within each benchmark; optical tactile on UniVTAC.
Where the policy looks
Contact-region attention on matched observations. With gaze supervision, the tactile semantic token places 52.38% of its main-camera attention inside the projected fingertip contact region at the supervised layer; without gaze, 2.77%.
CoRA-VTLA on UniVTAC Insert Tube with optical tactile sensing.
Citation
@misc{song2026coravtla, author = "{Chengye Song and Rui Song and Shengqi Wang and Jiawei Zhang and Hongzhou Wang and Xueyang Guo and Yang Liu}", title = "CoRA-VTLA: Contact-Guided Attention and Queryable Response Memory for Tactile Post-Training", year = "2026", howpublished = "\url{https://arxiv.org/abs/2611.23963}",}