An AI-Assisted CME Score Leaves a Retention Question Open

A successful task with AI available shows performance under those conditions. If a CME activity aims to develop a skill clinicians can retain and use without that particular assistance, the assessment needs to examine that ability separately. A better assisted score cannot answer both questions by itself.

The August 31 Annals On Call episode asking whether AI erodes clinical competence puts the issue on the education agenda. Its discussion distinguishes reference use from decision support and describes ways clinicians might use AI to develop their thinking. Those are practitioner perspectives, not proof of improved or diminished competence. For an outcomes lead, they suggest specifying which assistance is present when performance is assessed.

The clinical evidence does not give a single deskilling verdict

A 2025 Polish colonoscopy study compared standard, non-AI-assisted examinations before and after AI was introduced at four centers. Nineteen experienced endoscopists performed the procedures. The reported adenoma detection rate was lower after exposure to AI. Although the surrounding ACCEPT project assigned examinations to AI or standard conditions, this before-and-after deskilling analysis was retrospective and observational. Changing patient cohorts and other temporal factors limit a causal claim that AI exposure reduced skill.

A Norwegian study published online in June 2026 followed 13 endoscopists through three consecutive phases: before computer-aided detection, during its use, and after its removal. Among inexperienced endoscopists, detection of polyps at least 5 mm increased while assistance was available. Comparing the unaided phases before and after exposure, the study found no significant change. That result did not establish lasting learning or an absence of all possible deskilling effects; the authors noted limited power and case-mix differences.

The studies also measured different endpoints. Adenoma detection requires histological confirmation; the Norwegian measure counted polyps of a specified size and was not dependent on histology. Their operator groups, settings, and time courses differed too. Their results should not be pooled into a general estimate of what AI does to clinician competence.

The useful common feature is the question they asked: what happened when assistance was absent? Neither the immediate usefulness of a tool nor a headline about deskilling resolves that question for another specialty, another AI system, or a particular CME course.

Decide what the assessment is meant to show

The distinction has older educational roots. A 2015 review of learning and performance explains why performance during instruction can be an unreliable indicator of durable learning. It draws on motor and verbal learning research, not clinical AI trials. It supplies a reason to examine retention and transfer, without prescribing a clinical test or follow-up interval.

Our earlier discussion of designing CME for retention and transfer asked providers to build time and changed context into the activity. AI adds another condition to specify: which parts of the task were supported by the tool?

For a course teaching AI-supported work, an assisted assessment may be entirely appropriate. It can examine whether the learner uses the tool responsibly and completes the task. If the course also claims to preserve or build an independent skill, plan a suitable unaided task and a delayed assessment of that skill. Clinical faculty should choose an educational scenario or simulation where this comparison makes sense; this is not a recommendation to withdraw an approved tool during patient care.

Define “unaided” carefully. Removing a particular AI assistant does not necessarily mean prohibiting every reference resource. State what remains available, use comparable task demands, and distinguish immediate performance from later retention. These are proposed assessment-design choices, not a validated universal protocol or a claim that every AI tool causes skill loss.

When the outcomes report is written, name the condition alongside the result. Performance with assistance, performance without it, and retained performance after a delay are different findings. A program may want all three. It should not let one score stand in for the others.