The Report That Can't Answer the Question
A launch program by the numbers
Six months after a new obesity therapy is approved, a Medical Affairs team and its medical communications agency sit down to review the HCP education program they built together. The email series reached 12,000 clinicians with a 31% open rate. About 2,100 finished the on-demand module, average time on page was just over nine minutes, satisfaction came in at 4.6 out of 5, and 72% said they planned to change their practice. By the standards most teams report against, it's a good quarter.
The question nobody on the call can answer
Then the brand medical lead asks what the program was actually for. A patient with type 2 diabetes, stage 3 kidney disease, and a history of pancreatitis asks about the new therapy. Do the clinicians who finished the module screen her properly, set realistic expectations, and explain the safety data accurately? Nobody knows. Every number in the report describes what happened to the content, and none of them describes what clinicians can now do.
The Gap Medical Affairs Already Named
Everyone wants behavior change, few measure it
In a 2023 industry survey, EPG Health, an IQVIA business, asked pharma executives, HCPs, and service providers about HCP engagement for its report The Future of HCP Engagement Impact [1]. On digital HCP engagement, 63% of pharma respondents called demonstrating behavioral impact and outcomes a major challenge, just behind demonstrating financial return at 69% [1]. The numbers on what companies actually track were much lower. Fewer than 20% measure knowledge gain, behavior change, or impact in practice for most of their HCP education, and reach is still the main way impact gets judged [1]. Nearly two-thirds see the problem, and fewer than one in five are measuring it.
Agencies are closer to the answer
The same report found that service providers were on average twice as likely as pharma to measure knowledge and behavioral metrics for most educational activities [1]. Medical communications agencies already feel the pressure to prove impact to their clients. What they usually lack is a practical way to watch behavior at scale, because clinical practice happens in thousands of exam rooms nobody can see into.
Deliberate Practice Is How Behavior Changes
Content builds knowledge, practice builds skill
Anders Ericsson's 1993 research on expert performance described deliberate practice as effortful activity designed to improve performance. He found that differences in performance, even among elite performers, were closely tied to how much of it people had done [2]. Medicine has tested the idea directly. A 2011 meta-analysis led by William McGaghie screened 3,742 articles, kept 14 studies, and found an overall effect size of 0.71 in favor of simulation-based education with deliberate practice over traditional clinical education [3]. Most HCP education delivers the content and stops. The clinician's first real attempt to use it happens with a patient, and no feedback ever comes back.
Where virtual humans fit
AI virtual humans add the missing practice step, and they can be set up in several ways. The most familiar is a virtual patient. After the module, the clinician has a spoken conversation with the woman from that case, who worries about side effects, may or may not mention her kidney function, and responds to what the clinician actually says. A virtual expert clinician may be even more useful for Medical Affairs and agency teams. A virtual obesity specialist can walk a learner through a case step by step, stop to ask what they'd do next and why, push back on shaky reasoning, or just answer questions about the evidence. The same specialist can run a series of cases that get harder, or debrief the learner right after a virtual patient encounter. In each version, the conversation is scored against a rubric built from the approved content, the clinician gets specific feedback, and they can try again.
A Different Set of KPIs
From engagement proxies to performance measures
A practice layer produces numbers that engagement reporting can't. Teams can track how often each clinician practiced, how much they improved from first attempt to last, and how they did on the specific behaviors the program exists to change, like screening for contraindications or describing safety data accurately. They can also check whether that performance holds on a case the clinician hasn't seen before. These sit next to reach and completion rather than replacing them, and they come much closer to the behavioral impact that 63% of pharma respondents say they struggle to show.
Cohort transcripts show where the message breaks
Every conversation leaves a transcript, and across a cohort those transcripts show how the education is actually being applied. Picture 800 clinicians working the same case. The post-test might say 88% know the renal considerations, while the transcripts show a third never ask about kidney function unless the patient brings it up, and many overstate how much weight patients can expect to lose. With the expert-clinician format, transcripts also capture the questions clinicians ask when nobody's grading them, which is a direct read on unmet information needs. Medical Affairs can use that to fix the next wave of content and point MSLs toward specific misconceptions. Agencies get findings their clients can act on, in clinicians' own words.
What the Evidence Shows So Far
A small but encouraging body of research
Research on this kind of practice is still young. A 2026 systematic review in the Journal of Medical Internet Research found only 15 studies of generative AI virtual patients that met its criteria, with 645 participants in total across nursing, medicine, pharmacy, radiography, and first-responder training [4]. In the studies that compared virtual patients against a control condition, the virtual patients consistently came out ahead [4]. Cost looks favorable too. In a UK trial of 396 medical students, AI practice cost £33.48 per student compared with £61.75 for actor-based training, although students reported somewhat larger communication-skill gains and greater satisfaction after the actor-based sessions [5]. At Karolinska Institutet, 115 students who received AI-generated feedback after virtual patient encounters scored higher on an OSCE-based assessment given right after the training than students who completed the same cases without that feedback, a medium-to-large effect [6].
Practice beat reading in a head-to-head trial
In a 2026 randomized trial, researchers split 51 health professions students and practitioners into two groups. The practitioners included residents, nurses, and physician assistants [7]. One group spent up to 30 minutes practicing serious illness conversations with an AI virtual patient that gave them feedback. The other spent the same time reading about the same communication framework [7]. Afterward, both groups talked with a human standardized patient, and raters who didn't know which group anyone was in scored the conversations. The practice group improved significantly more in all three skills measured: empowering the patient, communicating clearly and directly, and responding with empathy [7]. Most HCP education programs face the same choice: give clinicians more to read, or give them somewhere to practice.
What it doesn't show yet
The review is clear about the limits. Samples were small, sessions were often a single sitting under 10 minutes, and almost none followed learners over time. Its authors found stronger support for feasibility and acceptability than for learning gains or transfer to clinical practice [4]. More than half the participants in the serious illness trial were students [7]. No study yet links simulation performance to prescribing or to real patient conversations. Sponsored education programs that track both are well placed to produce that evidence.
From Reach to Readiness
The program becomes the measurement
Most HCP education today runs like a study that never collects its results. Teams build the content, send it out, count who opened it, and then plan the next program on the same guesses they started with. Adding practice changes where the evidence comes from. Each clinician's attempts become the outcome measure, and each cohort shows which parts of the message held up under a patient's questions and which fell apart. Medical Affairs and its agencies can then fix the next program based on what clinicians actually did, long before any of them sit down with the next real patient.
References
- EPG Health (an IQVIA business). The Future of HCP Engagement Impact. 2023. https://epghealth.com/reports/the-future-of-hcp-engagement-impact/. Summary published by PMLiVE, "HCP Engagement Study: Pharma shifts top priority to HCP insights and impact metrics," December 8, 2023. https://pmlive.com/pmhub/epg_health/hcp_engagement_study_pharma_shifts_priority_to_hcp_insights_and_impact_metrics
- Ericsson KA, Krampe RT, Tesch-Römer C. The role of deliberate practice in the acquisition of expert performance. Psychological Review. 1993;100(3):363–406. doi:10.1037/0033-295X.100.3.363
- McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Academic Medicine. 2011;86(6):706–711. doi:10.1097/ACM.0b013e318217e119
- Jiang J, Ye MZ, Kwok TTO, Wong JYH. GenAI-supported virtual patients in health care education: systematic review. Journal of Medical Internet Research. 2026;28:e82756. doi:10.2196/82756
- Tyrrell EG, Sandhu SK, Berry K, Ghannam SF, Lewis SA, Crowfoot D, Sahota GS, Carson J, Wilson EE, Taggar J. Web-based AI-driven virtual patient simulator versus actor-based simulation for teaching consultation skills: multicenter randomized crossover study. JMIR Formative Research. 2025;9:e71667. doi:10.2196/71667
- Borg A, Schiött J, Ivegren W, Gentline C, Huss V, Hugelius AM, Jobs B, Ruiz M, Edelbring S, Georg C, Skantze G, Parodis I. AI-generated feedback following social robotic virtual patient interactions and medical student performance: nonrandomized quasi-experimental study. JMIR Medical Education. 2026;12:e90368. doi:10.2196/90368
- Haut KG, Hasan M, Carroll T, Epstein RM, Sen T, Hoque E. The effects of generative AI virtual patient in serious illness communication skills: randomized controlled trial. JMIR Medical Education. 2026. Forthcoming/in press. doi:10.2196/93034
