Read CME Confidence Gains Alongside Correct Answers

When a CME report says learners became more confident, the next question is: confident about which answers? An outcomes team can make that result more useful by pairing confidence with correctness on the same task, then examining how the pair changes for returning learners.

An October 15 JCEHP podcast offers a specific reason to do this. Its discussion of spaced repetition focuses on confidence in answers that physicians first got wrong and later got right. That is a narrower, more informative question than whether the average confidence score rose after education.

What the study actually tested

The underlying randomized study used the American Board of Family Medicine's quarterly Continuous Knowledge Self-Assessment. It included 16,751 physicians who completed assessments at baseline, quarter six, and quarter ten. Participants rated confidence on a six-point scale. Physicians were randomized to control or repetition groups; assessments used repeated questions and questions testing the same point in another scenario.

The confidence analysis concerned initially incorrect answers that became correct. Confidence gains were larger with two repetitions than with one; gains on the changed-scenario questions were smaller. The article first appeared online in August 2025 and is in the journal's 2026 volume. The October podcast is renewed discussion of that work, not a second trial.

In the podcast's discussion of limitations, the authors distinguish internet-delivered question performance from actual practice and leave the real-world meaning of the confidence change open. This evidence supports examining confidence alongside learning; it does not establish fewer diagnostic errors or better patient outcomes.

There is also a useful counterpoint in the earlier history. A retrospective study of family physicians taking both higher- and lower-stakes assessments, published online in March 2023, found more correct answers but lower confidence in the higher-stakes setting. Assessment conditions differed, and this was not a randomized experiment showing what raising the stakes would do. It does show why confidence and accuracy should remain separate measures: they can move in different directions.

Give the report a next decision

For a CME team adding follow-up questions to an activity, a practical starting point is to retain each answer together with the learner's confidence rating, collected before revealing feedback. Clinical faculty should review the answer key, explanation, and any changed-scenario item; the outcomes lead should define the comparison and reporting rules before launch.

The following is a proposed reporting approach, not a validated assessment instrument. Use the same confidence scale at each time point and define any high/low grouping in advance. Keep the original ratings available rather than letting a convenient threshold become a competence standard.

Consider a hypothetical learner who initially selects an incorrect answer with high confidence. At follow-up, several different stories are possible:

• Correct and confident: examine whether the learner can also handle a different scenario testing the same point.
• Correct but unsure: retain the accuracy gain and use feedback to explore the remaining uncertainty.
• Incorrect but less confident: the learner may now recognize uncertainty, but the knowledge gap still needs attention.
• Incorrect and still confident: review the explanation and the learner's reasoning before describing the response as a successful learning result.

These are prompts for educational follow-up. A response to one question cannot establish readiness for independent clinical practice, and a confident wrong answer can also expose an ambiguous item or faulty key. Faculty review belongs in the loop.

Our earlier article on assessing retained skill under clearly stated conditions separates assisted performance from what remains after a delay. Here the additional distinction is between getting an answer right and knowing how certain to be about it. Both distinctions help an outcomes team state exactly what its assessment shows.

Keep the missing learners visible

Report how many learners answered initially, how many returned, and how many provided usable paired responses. A change among returners is a result for that group. It cannot silently become a claim about everyone who enrolled.

Likewise, keep a focused analysis of wrong-to-right answers beside the overall pattern of correct and incorrect responses. Selecting only successful transitions leaves unanswered what happened to learners who remained incorrect or moved from correct to incorrect.

A useful closing sentence for a report would name the task, interval, paired-response denominator, accuracy result, and confidence result. If the program needs to make a claim about practice, plan a separate, appropriate practice measure with the clinical partner. The confidence question has earned a place in the report when it helps explain what learners need next.