Clinicians Are Already Supervising Multi-Agent AI—CME Still Teaches Tool Basics

Clinicians are describing AI less as a tool to try and more as a set of agents they must direct, check, and overrule. The signal is narrow and oncology-led, with physician/social and society sources in the mix, but the provider implication is broader: CME design has to prove that learners can manage the handoff between content, tool, and decision.

AI education needs observable supervision behaviors

The clearest AI signal this week was not another call for general literacy. It was the move from “use the model” to “supervise the model.” In an oncology AI thread, Stephen V Liu, MD summarized the role change this way: “physicians move from users to supervisors.”

A separate clinician-facing prompt example showed what that supervision can look like at the task level: asking a model to explain CTCAE toxicity grades, include when treatment should be held, dose reduced, or discontinued, and then taking the same prompt to another LLM for comparison (source). That is not autonomous decision-making. It is role assignment plus verification.

For CME providers, the design gap is straightforward. Many AI modules still stop at tool orientation: what the model can do, what the risks are, and how to write a better prompt. The stronger learning objective is behavioral: can the clinician define the model’s role, test the answer against a second source or standard, recognize ambiguity, and decide when escalation is needed?

That extends an earlier brief on testing what clinicians do when AI output is wrong, but this week’s conversation makes the handoff more concrete. The question for CME teams is whether an AI activity can be scored on the learner’s verification steps, not only on whether the learner knows the tool exists.

Format choice should follow the learning task

The second signal came mainly from a society-sponsored medical education source, so it should not be treated as broad clinician consensus. Still, the point is useful for CME operations: multimedia is not a virtue by itself.

In an ASH medical educators segment, speakers framed format choice around goal and learner need, not novelty. Visual learners may need something other than a podcast; complex pathways may be better served by an infographic than by audio alone (source).

That matters because many CME catalogs still treat format as a production decision made after the content plan is set. The better sequence is reversed: first define what the learner must recognize, compare, rehearse, or recall; then choose the medium that makes that task easier. A webinar, infographic, video, podcast, or interactive summary should each have a reason to exist.

The implication is not “make everything multimodal.” It is to stop using one format as the default answer. For each activity, CME teams should be able to explain why the chosen medium fits the cognitive work the learner is being asked to do.

What CME Teams Should Do Now

• Replace “describe AI tool capabilities” objectives with objectives that require learners to assign a role, verify output, and decide when to escalate.
• Score at least one AI exercise on the quality of the learner’s verification steps, not just completion of a prompt or case.
• Add a format-selection check to planning: what must the learner do after the activity, and which medium best supports that action?

What to reconsider

Tool familiarity and format variety are weak proxies for learning quality. This week’s signal points to a stricter test: can the activity show that clinicians can supervise a tool, verify a recommendation, and receive information in a medium that fits the task? If not, the activity may be teaching awareness when the market conversation has already moved toward supervised use.