Nvidia releases Cosmos3-H-Dreams real-time generative surgical simulator

Cosmos3-H-Dreams builds on the 16‑billion‑parameter Cosmos3‑H‑Embodied‑Nano model and specializes a bidirectional teacher on curated cholecystectomy video at 832 × 480 resolution sampled at 10 Hz. Training first extends the horizon to 24 future frames and then to 72 frames (2.4 s and 7.2 s). Distillation creates a causal student that generates video in short blocks as new actions arrive, using four‑step and two‑step checkpoints. The model accepts 28 action dimensions – pose and gripper data for two instruments (a left Cadiere grasper and a right monopolar hook) plus energy and thumb‑stick controls. Developers can explore the trade‑off between visual quality and inference cost, and the repository includes a step‑by‑step guide for adapting the pipeline to new datasets. The authors stress that Cosmos3-H-Dreams is a research simulator, not a validated system for patient care, and that visual realism is only a starting point for further biomechanical validation.