Scaling Complex Clinical Reviews Without Sacrificing Rigor in Amphitheater

Complex clinical case reviews are the hardest to accelerate, precisely because they require advanced critical thinking and human judgment. A clinical pharmacy services company that serves employers, third-party administrators, and pharmacy benefit managers, introduced Amphitheater to their workflow to help reviewers reach well-supported decisions more efficiently while maintaining complex clinical standards. 

With Amphitheater, the client company’s pharmacists reduced average clinical review times from a baseline of 23.1 minutes to 4.4 minutes per case; further, alignment between pharmacists and Amphitheater’s AI climbed from roughly 60% in late June to 89% by late August across a sample of 562 cases from this two-month period.

Unlike utilization management solutions that automate preconfigured rule sets, Amphitheater’s core clinical AI workflow automatically retrieves documented client criteria, but also performs additional research for each individual case. The AI then shows its work to a human pharmacist. Amphitheater registers every decision outcome from its workflow, including when a human reviewer challenges the AI’s research or reasoning, or reaches a conclusion outside underlying client-approved criteria. Disagreements become structured evidence for examining the AI, available context, and even the currency of criteria, all with the final aim of reducing clinical risk.

Unlocking Capacity for Higher Case Volumes

The company’s clinical pharmacy services team routinely interprets patient histories against client-specific clinical criteria. While the typical length of case documentation runs 5-20 pages, longer cases processed in Amphitheater between June and August 2026 ran from 40 to over 160 pages. Unsurprisingly, pharmacist reviewers indicated evaluations were especially time-consuming when relevant information was buried in longer documentation.  

From June 23, 2026 through August 31, 2026, the company’s average review time fell from 23.1 minutes per case as a baseline to 4.4 minutes per case: a reduction exceeding 80%, unlocking greater capacity for clinical staff to absorb fluctuating case volumes. 

For this particular client, however, speed was not an isolated goal. Company leadership and the clinical pharmacy team emphasized from the outset that each AI recommendation had to be transparent to be clinically defensible, as well as aligned with their pharmacists’ judgment in ~85% of cases to be trustworthy. To get there, Epidaurus initially limited Amphitheater’s AI to trusted evidence sources and discrete, structured clinical data from the patient record. Early pharmacist feedback, however, indicated that the AI was missing deeper narrative context from physician notes, including details that explain patient history and disease progression. Epidaurus responded by expanding the clinical context available to the AI while further strengthening the quality and currency of the evidence used to guide its analysis. 

Gaining the Trust of Clinicians 

The targeted 85% human-AI agreement threshold was reached at the end of August, though the full-period average landed at 78.5%. Human-AI agreement fluctuated daily (especially on low-volume days where a few disagreements would swing the daily average substantially), yet the broader trend demonstrated upward momentum, progressing from roughly 60% in late June to the mid-to-high 80s by late August, with a seven-day trailing average of 89% agreement on August 31st. During the full evaluation period, agreement remained strong across complex disease states, including mental and behavioral disorders (87.2%), nervous system conditions (84.2%), and endocrine/metabolic diseases (83.6%). With regard to times for these complex disease states, reviewers averaged 6.97 minutes for cases treating mental and behavioral disorders, 2.58 minutes for nervous system conditions, and 4.07 minutes for endocrine and metabolic diseases.

To maintain an upward trend in human-AI agreement going forward, Epidaurus intends to continue streamlining the human feedback loop for complex disease states, and ideally curating more precise clinical criteria in alignment with higher clinical quality standards. Amphitheater’s built-in analytics support this work by showing where recommendations align consistently with pharmacists’ decisions and isolate the cases most valuable for further examination.

“Being able to see the AI's reasoning and push back when something didn't fit was key to building trust for our pharmacists when utilizing AI. Epidaurus was committed to hitting our 85% human-AI agreement target and worked closely with us on our feedback to continually improve the AI. That partnership, along with review times dropping from over 20 minutes to under 5, has freed our team to focus on the cases with real clinical complexity.”
— Assistant Director of UM Services

Turning Disagreement into Evidence

Amphitheater is Software 3.0 for pharmacy benefits, with science in the loop. Each disagreement is treated as more than an exception to an alignment rate. A disagreement can expose a weakness in AI reasoning, an incorrect application of criteria, or a gap in official criteria when compared against evolving research and clinical standards. Capturing the reasons behind the disagreements steadily accumulates into a body of evidence that can drive measurable clinical quality improvements. 

For the clinical pharmacy services team, this means increasing their capacity for case reviews, without having to model and maintain complex clinical decision trees. Instead, Amphitheater’s AI-native workflow simply reduces the work required to parse documentation and evaluate each case, while leaving room for nuanced insights that no rule set could accommodate.