Ubbo VisserComputer Science · University of Miami

ARA-URS: Autonomous Robot-Assisted Ureteroscopy

Surgical robotics · Digital twins · Autonomous laser lithotripsy

Our ARA-URS project combines a robotic ureteroscope, a digital twin, and computer vision to investigate autonomy for kidney stone treatment. Working across computer science and urology at the University of Miami, we develop methods that connect instrument positioning and stone targeting with automated laser firing.

Annotated ARA-URS benchtop setup showing the Thulio laser, control electronics, robotic ureteroscope, endoscopic display, and kidney model with irrigation
Figure 1 from our ICRA 2027 submission: the integrated ARA-URS benchtop setup. The submission is under review.

Autonomous laser lithotripsy: ICRA 2027 submission

Under review. Our submission, “Towards Fully Autonomous Laser Lithotripsy: A Benchtop Study of Vision-Guided Stone Targeting and Ablation in Robot-Assisted Ureteroscopy,” extends the platform from navigation to autonomous stone ablation.

The system segments a stone in endoscopic video, estimates its three-dimensional position, and guides the ureteroscope toward a firing position. A second vision model assesses readiness to fire; laser commands are issued when both the vision criterion and the hardware ready state are satisfied. Training combines annotated surgical footage with synthetic images generated using the digital twin.

In benchtop experiments on five calcium oxalate stones, the submission reports a mean stone mass ablation rate of 2.83 mg/min and a mean successful-firing rate of 88.04%. Successful firing is defined by stone ablation. The surgeon-operated teleoperation reference achieved 11.03 mg/min and 93.24%, respectively. These preliminary results demonstrate the integrated perception, navigation, and laser-control pipeline while showing the remaining gap in ablation efficiency.

Platform and digital twin

Physical ARA-URS prototype with its ureteroscope, motorized linear stage, and rotation mechanism
The physical ARA-URS research platform.
Digital twin of the ARA-URS platform modeled in NVIDIA Isaac Sim
Its digital twin in NVIDIA Isaac Sim.

The platform actuates a Wolf RIWO Digital Ureteroscope with three degrees of freedom: insertion and retraction, shaft rotation, and bending of the flexible tip. Custom mounts, motors, and a kinematic model translate a desired tip position into instrument motion.

The digital twin reproduces the robot and its motion in NVIDIA Isaac Sim. Virtual kidney and stone models provide an environment for generating synthetic endoscopic scenes, developing vision and control methods, and comparing simulated behavior with the physical system.

Full digital twin showing the robotic ureteroscope extending from its motorized platform into a virtual kidney model
The simulated setup connects the robotic instrument to a virtual kidney model.

Foundation: vision-guided stone targeting at ICRA 2026

Our ICRA 2026 paper evaluates two tasks: tip positioning in an in-air setup and autonomous navigation toward a kidney stone in simulation and a physical kidney model. A computer vision pipeline and inverse kinematics guide the ureteroscope toward the target.

For stone targeting, the paper reports the following average fraction of the endoscopic image occupied by the stone after navigation:

Environment Average stone image coverage
Digital twin 88.77%
Physical benchtop model 59.04%

Image coverage measures how much of the camera view the stone occupies after targeting. The experiments establish a proof of concept for targeting and show a performance gap between simulation and the physical platform. The study identifies mechanical vibration, optical tracking error, and simplified experimental conditions as areas for further work.

Next steps

The ICRA 2027 submission identifies tip-position sensing and learning-based control as directions for improving spatial localization and ablation efficiency. Further research must address flexible-instrument deformation, anatomical resistance, and the broader variability encountered during a procedure.

Papers and team

ICRA 2027 submission — under review. “Towards Fully Autonomous Laser Lithotripsy: A Benchtop Study of Vision-Guided Stone Targeting and Ablation in Robot-Assisted Ureteroscopy.”

Sarvesh Saini, Julio Ojalvo, Jonathan E. Katz, and Ubbo Visser. “Autonomous Robot-Assisted Ureteroscopy (ARA-URS) for the Treatment of Kidney Stones.” IEEE International Conference on Robotics and Automation (ICRA), 2026.

Read the paper on IEEE Xplore · ICRA 2026 abstract

The project brings together the Department of Computer Science and the Desai Sethi Urology Institute at the University of Miami Miller School of Medicine. Images on this page are from our ICRA 2026 paper and ICRA 2027 submission materials.