Picking the right treatment for a patient is rarely a one-step decision. A doctor has to think about the disease, the patient’s other health conditions, the drugs they’re already taking, possible interactions, and warnings that keep changing as new research comes out. It’s a slow, careful process of gathering facts before concluding.
A team of researchers at Harvard Medical School, working with collaborators from the University of Oxford, the Icahn School of Medicine at Mount Sinai, and the Clalit Research Institute in Israel, decided to see if an AI system could be trained to do something similar. Their result is a new AI agent called ATHENA-R1, described in a recent paper on treatment reasoning. Instead of just answering questions from memory, ATHENA-R1 is built to look things up, reason over what it finds, and revise its thinking as it goes, much like a clinician working through a complicated case.
Why This Is a Hard Problem for AI
Most AI language models answer medical questions using information baked into their training. That approach works reasonably well for simple facts, but it runs into trouble with real treatment decisions. A drug that’s perfectly safe for one patient might be dangerous for another because of an interaction, a kidney condition, or a pregnancy. The correct answer depends on details that the model has to actively seek out, not something it can simply recall.
Giving an AI access to biomedical databases and tools doesn’t automatically fix this. The system still needs to figure out what information matters, which source to check, how to interpret what comes back, and what to do if the evidence is incomplete or conflicting. That decision-making process is what the researchers set out to teach.
How ATHENA-R1 Works
ATHENA-R1 has access to a library of 212 biomedical tools that can pull real-time information from sources like FDA drug labels and biomedical knowledge bases. When given a case, it works through the problem in steps: it identifies what it doesn’t yet know, picks a tool to find that information, reads the result, and decides what to check next. It keeps going until it has enough evidence to make a recommendation, and it produces a written trace showing exactly what it looked up and why.
For complicated cases, the system can even split a problem into smaller pieces, exploring several treatment options at once before combining the results into one final answer.
Training a system to do this wasn’t straightforward, since no one could realistically write out hundreds of thousands of step-by-step reasoning examples by hand. So the team used a two-stage approach. First, a set of AI agents automatically generated a large training dataset, over 85,000 treatment reasoning tasks built from FDA-approved drug information going back to 1939. Second, ATHENA-R1 was refined through reinforcement learning, where it practiced using the tools directly and received feedback on things like whether it gathered enough evidence, used tools correctly, and avoided repeating itself.
How Well Did It Perform?
The results were tested in several ways. On a benchmark of over 3,000 drug-related questions built from FDA prescribing information, ATHENA-R1 reached 94.7% accuracy, well ahead of GPT-5, DeepSeek-R1, and Qwen3. On a tougher benchmark involving patient-specific treatment decisions, where the right answer depends on things like a patient’s other conditions, ATHENA-R1 scored 82.9%, again ahead of the other systems tested.
Interestingly, the researchers also found that simply giving a large model access to the same tools didn’t help much. GPT-5, for instance, barely used the tools available to it even when allowed to, and its accuracy didn’t improve. This suggests the real bottleneck isn’t access to information, but knowing how and when to use it.
Beyond benchmark scores, the system was also tested with real people. Experts from 28 rare disease organizations reviewed blinded responses to difficult treatment cases and preferred ATHENA-R1 over other models across every criterion they were asked about, especially for how clearly it explained its reasoning. Separately, physicians reviewed ATHENA-R1’s recommendations on real, complicated hospital cases involving heart surgery patients and infections, and rated its reasoning favorably.
The team also used ATHENA-R1 to generate hypotheses about possible drug side effects and then checked them against health records from 5.4 million patients. Several of its predictions held up, showing meaningfully increased risks in specific patient groups, while unrelated “negative control” checks showed no such effect, a sign the system wasn’t just generating noise.
What This Means
The researchers are careful to note that ATHENA-R1 isn’t meant to replace clinical judgment or serve as a bedside tool. It’s a research system built to test whether the evidence-gathering process behind treatment decisions can be learned rather than simply memorized. The results suggest that training an AI to reason step by step with real tools can outperform simply making language models bigger, at least for this kind of task.
There’s still a lot left to work on. The system doesn’t yet handle images, lab results, or genetic data, and it doesn’t express how confident it is in its own conclusions. But the core idea, that reasoning about treatment can be treated as a skill to train rather than knowledge to store, is a genuinely interesting direction for medical AI research going forward.
Article Source: Reference Paper | Availability: Web | Code availability: GitHub
Disclaimer:
The research discussed in this article was conducted and published by the authors of the referenced paper. CBIRT has no involvement in the research itself. This article is intended solely to raise awareness about recent developments and does not claim authorship or endorsement of the research.
Important Note: arXiv releases preprints that have not yet undergone peer review. As a result, it is important to note that these papers should not be considered conclusive evidence, nor should they be used to direct clinical practice or influence health-related behavior. It is also important to understand that the information presented in these papers is not yet considered established or confirmed.
Follow Us!
Learn More:
Anchal is a consulting scientific writing intern at CBIRT with a passion for bioinformatics and its miracles. She is pursuing an MTech in Bioinformatics from Delhi Technological University, Delhi. Through engaging prose, she invites readers to explore the captivating world of bioinformatics, showcasing its groundbreaking contributions to understanding the mysteries of life. Besides science, she enjoys reading and painting.
