Clinicians Insist on Co-Designing the AI Tools They Will Actually Use
Earlier coverage of ai oversight and its implications for CME providers.
AI-assisted practice is exposing a harder CME problem: protecting unaided clinical skill while designing education that produces measurable change.
An educator discussion this week pointed to a concrete AI risk: experienced endoscopists using AI-assisted colonoscopy for months later detected fewer polyps when the AI was removed. The example is GI-specific, but the education problem is broader for any specialty where AI may become part of diagnostic or procedural work.
The sharpest signal came from a medical education discussion of AI de-skilling. The concern was not that clinicians should avoid AI. It was that clinicians can become less capable when the assistive layer disappears. In the polyp-detection example, the educator described experienced physicians becoming so reliant on AI-assisted colonoscopy that their unaided detection performance fell after three months of use (source).
That changes the CME design question. A course on AI in practice cannot stop at how the tool works, when to use it, or how to interpret its output. It also has to ask whether clinicians can still perform the underlying task without the tool. The same discussion framed the everyday verification habit plainly: “Sometimes I believe it, sometimes I don't, sometimes I go double check it.” That habit needs to become assessable, not aspirational.
A urology-focused AI conversation added the governance side: clinicians need to understand transparency, validation in relevant populations, model drift, human oversight, and escalation pathways when tools underperform (source). We saw a related pattern in an earlier brief on clinicians needing evaluation frameworks for LLM tools; this week’s difference is that the risk is no longer only poor evaluation of AI output. It is possible decay of the clinician’s own unaided performance.
For CME teams, the implication is simple: AI curricula should alternate AI-assisted cases with no-AI cases, measure skill retention, and require learners to document when they override or verify an AI suggestion.
The second signal came from provider-owned educational content, so it should be treated as expert framing rather than independent clinician consensus. Still, it named a design problem many CME teams face: a 15-, 30-, or 60-minute activity is rarely enough to produce durable behavior change by itself.
The discussion argued for using stages-of-change thinking, root-cause analysis, cognitive load principles, and transfer-focused evaluation before choosing format or assessment strategy (source). The useful point for providers is not the terminology. It is the discipline of deciding whether the activity is trying to shift awareness, attitude, competency, performance, or sustained practice — and then measuring the right thing.
That matters especially for AI-related education. If the gap is over-reliance, a post-test on AI limitations will not be enough. If the gap is lack of confidence overriding an AI output, the activity needs practice cases, feedback, and a transfer plan. If the gap is skill decay, the outcome is not learner satisfaction or intent to change; it is retained unaided performance.
The concrete question for CME teams: before building the next activity, can you name the learner’s stage, the behavior you expect to move, and the evidence you will accept that transfer occurred?
The next CME problem will not be whether clinicians have heard of AI or can describe its risks. It will be whether education can preserve the human capability that AI is meant to augment. Providers that can show both safe AI use and retained unaided skill will have a stronger outcomes story than those that only teach the tool.
Demonstrates documented loss of polyp detection skill in experienced endoscopists after months of AI assistance.
Open sourceDiscusses trainee skill acquisition risks and the requirement for low-stakes AI uses versus high-stakes oversight.
Open sourceEarlier coverage of ai oversight and its implications for CME providers.
Earlier coverage of ai oversight and its implications for CME providers.
Earlier coverage of ai oversight and its implications for CME providers.
ChatCME surfaces the questions clinicians actually ask — so you can build activities that close real knowledge gaps.
Request a demoArgues that learning-science tools (stages of change, cognitive load, transfer) are underused and that short activities require root-cause attitude/competency focus to achieve behavior change.