A new AI technique called xvr is designed to make that alignment fast and precise. Developed by scientists and clinicians at MIT and collaborating institutions, and published in Nature, it matches a patient's X-rays with their 3D scan in seconds, with sub-millimeter precision. This article explains what xvr is, how it works, and what it could mean for surgical navigation.
1. What Is xvr?
xvr, short for X-ray volume registration, is an AI technique that automatically aligns X-rays captured during surgery with a patient's preoperative 3D scan, such as a CT or MRI. It adapts to each patient in about five minutes, then completes the alignment in seconds with sub-millimeter precision.
Sub-millimeter simply means the alignment error is smaller than one millimeter. Across a wide range of patients, body parts, and medical procedures, xvr outperformed existing AI methods by an order of magnitude, a term that usually points to a difference of about ten times. The researchers describe the goal as making it easier for clinicians to precisely pilot surgical tools, leading to faster and safer procedures.
2. Why Is X-ray Guidance Difficult in Minimally Invasive Surgery?
Many minimally invasive procedures, such as angioplasty to open blocked arteries, follow a similar pattern. A clinician inserts instruments through a tiny incision and uses a high-speed mobile X-ray scanner to generate images that show the procedure from any angle.
The challenge is the format of those images. Because X-rays are flat, it can be hard to determine exactly where a tool is located and how it is oriented inside the body. That uncertainty increases the risk of complications, since clinicians need to guide instruments without accidentally damaging other tissue.
2.1 What Is Registration?
To solve this, clinicians align the real-time X-rays with the patient's preoperative MRI or CT scan. This process is called registration. In simple terms, it means lining up two different views of the same patient so they match, much like fitting a street map over a photo of the same city. Once the views are aligned, the clinician can see where a tool is in relation to the anatomical structures around it.
2.2 Why Is Manual Registration Slow?
Manual registration is slow and burdensome. Clinicians may need to estimate the position of an instrument by typing numbers into a computer or clicking anatomical landmarks on a screen. Reading grainy 2D images and understanding how everything is oriented also takes decades of training.
The aim of the xvr team is to make 2D X-rays more informative, so these life-saving procedures become safer and easier to perform.
3. Why Have Earlier AI Tools Struggled?
Researchers have been developing AI models that predict 2D/3D registration automatically. These tools work in principle, but human anatomy is highly diverse. A model that performs well for some patients may fail for others.
Training data is part of the problem. Building a deep-learning model that adapts to many patients requires a large amount of high-quality annotated medical image data, meaning images that experts have labeled. That kind of data is hard to come by. As a result, existing AI tools have struggled to align images robustly for all patients, which has kept them from being practical in real procedures.
4. What Makes xvr Different? One Model for One Patient
Instead of building a single model meant to work for everyone, the team built a model designed to adapt extremely well to one specific patient. Each patient gets a tailored model. If it does not perform well on other people, that does not matter, because those people will have models of their own.
This shifts the problem in a useful way. The model no longer has to handle the full range of human anatomy. It only has to understand one person's body, and it does so using that person's own scan.
5. How Does xvr Work?
The process can be understood in three parts.
5.1 Start With the Patient's Own Scan
xvr takes one patient's preoperative 3D scan, such as an MRI or CT. Everything that follows is built from this single source, which keeps the system tied to the anatomy of the person on the table.
5.2 Generate Synthetic X-rays Using Physics
From that scan, xvr creates thousands of synthetic X-rays from many different angles, at a rate of about 1,000 images each second. Synthetic here means computer-made rather than captured by an X-ray machine.
To keep these images realistic, xvr uses a physics-based simulation of the X-ray process. Put simply, it calculates how X-rays would pass through the patient's body and what image they would produce.
This is different from some types of generative AI, which create data from nothing. Here, the simulation is based entirely on the patient's own CT or MRI. Because the data is produced in a purely physics-based way, the researchers note there is no room for hallucinations. A hallucination is when an AI system produces details that are not real, which would be a serious concern in a medical setting.
5.3 Train a Model to Align the Images
xvr then uses this simulated data to train an AI model that can accurately align the patient's 2D X-rays with their 3D scan in a matter of seconds. Because the training examples come from the patient's own scan, this approach also sidesteps the shortage of labeled medical images described earlier.
6. How Can It Be Fast Enough for Emergencies?
A registration model trained from scratch for each patient is highly accurate, but it would take about 12 hours to train. That makes it impossible to use in an emergency, when time is limited.
To solve this, the researchers used xvr to pretrain a more versatile system called a foundation model. A foundation model is an AI system first trained on broad, varied data so it can be adjusted quickly for a specific task. It works a bit like an experienced professional who needs only a short briefing before starting a new case.
The team collected whole-body 3D medical scans from more than 2,000 patients. These covered a wide range of ages, image modalities (the different types of scans, such as CT and MRI), and body regions. xvr used these scans to generate synthetic X-rays and train the foundation model to perform 2D/3D registration.
The pretrained model can adapt to a new patient in about five minutes, and it performs registration with the same accuracy as a model trained from scratch. The outcome is patient-specific accuracy on a much faster timeline.
7. How Well Did xvr Perform?
The team tested the model on the largest available dataset of real 2D/3D registrations. It included data from five hospitals and covered dozens of bones and organ systems in both adult and pediatric patients. The model learns from synthetic images, but this test used real cases.
xvr significantly outperformed other AI-based methods in accuracy and robustness. Robustness refers to how reliably a model performs across different patients and conditions, not only in ideal ones. The researchers also report that it operates fast enough for emergency surgeries, and that the model could be used to improve the performance of robotic surgery technologies.
8. Which Procedures Could Benefit?
Any procedure where clinicians rely on X-rays for navigation is a potential use case, with orthopedics and neurosurgery named as example fields. Angioplasty and emergency stroke interventions are among the procedures discussed in the research. Because the foundation model was built from whole-body scans and the testing covered dozens of bones and organ systems, the approach is not limited to a single body region.
9. Why Does This Matter for Access to Care?
Speed and ease of use matter beyond the operating room. Lead author Vivek Gopalakrishnan, a postdoc at MIT CSAIL, points out that a majority of Americans live more than an hour away from a center that can perform noninvasive procedures such as emergency stroke interventions. He adds that an hour is a very significant amount of time in stroke care.
His view is that making these procedures easier by combining 2D and 3D information could make highly specialized, life-saving procedures accessible to a much broader part of the population. This is the researchers' perspective on the potential, and it depends on the technique moving from research into everyday clinical tools.
10. Who Developed xvr?
xvr was developed by scientists and clinicians at MIT and collaborating institutions. The paper was led by Gopalakrishnan, with Polina Golland of MIT and Neel Dey of Harvard Medical School and Massachusetts General Hospital as co-senior authors. Additional co-authors include a neurosurgeon, a pediatric clinician, and radiology faculty from institutions such as Harvard, Brigham and Women's Hospital, Boston Children's Hospital, and Shriners Children's Hospital.
The mix of technical and clinical backgrounds is worth noting for a tool meant for surgical settings. Funding came in part from the National Institutes of Health and several MIT programs.
11. What Comes Next for xvr?
The team has outlined three directions:
- Making xvr faster for real-time deployment
- Running further studies to verify its reliability in additional situations
- Extending the system to handle more complex scenarios, such as moving body parts
The researchers have spent the past two years developing and validating the algorithm. They are now collaborating closely with surgical robotics companies and clinical groups to turn this research into useful tools for navigation or deployment. For now, xvr is a research result, and these next steps show where the work is headed.
12. Conclusion
xvr addresses a long-standing challenge in minimally invasive surgery: knowing where tools are inside the body when the guiding image is flat. By building a model for each patient from that patient's own scan, and using physics-based synthetic X-rays, it aims for patient-specific accuracy without invented image details. A pretrained foundation model shortens preparation from about 12 hours to about five minutes, and testing across five hospitals showed it outperformed other AI methods in accuracy and robustness.
Key takeaways:
- xvr aligns X-rays with a patient's 3D scan in seconds, with sub-millimeter precision.
- Each patient gets a tailored model built from their own scan.
- A foundation model trained on scans from more than 2,000 patients allows adaptation in about five minutes.
- The technique is still at the research stage, with work underway on real-time speed, further reliability studies, and moving body parts.
As the team works with clinical groups and surgical robotics companies, the next stage will show how far this approach can go in making X-ray guided procedures safer and more accessible.