Physicians May Be Inaccurate In Confidence Levels of Their Diagnoses, Pitt Study Finds

Physicians often lack correct perceptions of the accuracy of their diagnoses at the time they make them, and in significant numbers of cases they may be overconfident (wrong when they believe they are right) or underconfident (right when they believe they are wrong) about their diagnostic assessments, according to a Pitt study.

“An overconfident physician will not seek help or may ignore help that is offered; someone underconfident may, in extreme cases, be talked out of what was an accurate assessment,” wrote Charles P. Friedman, professor of medicine at Pitt, in an article published in the April issue of the Journal of General Internal Medicine.

Friedman reported that confidence plays a major role in clinicians’ use of outside resources in determining diagnoses and making other medical decisions. These resources include colleagues, medical books, journals, and computer-based decision support systems.

“Decision support occurs in two modes: either clinicians must seek medical information to supplement what they already know, or if it is ‘pushed’ to them in the form of a computer-generated alert or reminder, they must be open to the advice,” Friedman said. “The whole decision support process presupposes that clinicians know when to seek help or pay attention to help that is offered.”

Friedman, who is on leave from Pitt to work as a senior scholar and program officer in the National Library of Medicine at the National Institutes of Health in Bethesda, Md., and his colleagues developed detailed written synopses from 36 detailed diagnostically challenging cases from patient records at the University of Illinois at Chicago, the University of Michigan, and the University of North Carolina. They recruited 216 volunteer subjects from those three institutions: 72 fourth-year medical students, 72 second- and third-year internal medicine residents, and 72 general internists with faculty appointments and at least two years of postresidency experience.

Each subject worked on nine cases. For each case, subjects were asked to provide a diagnosis and a measure of their confidence in the diagnosis.

“Focusing on the residents and faculty physicians, what we found was that correctness and confidence were not aligned in about one-third of the cases,” Friedman said. “For those cases where confidence and correctness were not aligned, physicians were more likely to lack confidence in a correct diagnosis than the reverse.”

Nonetheless, he added, residents and more experienced internists were “overconfident,” believing they are correct when in fact they were not, in 12–15 percent of the cases overall.

“Overconfidence is the zone of greatest concern when one thinks about how to improve practice using information technology,” Friedman said. “While this was a study performed in the laboratory and not in the clinic, the apparent prevalence of overconfidence represents a major challenge to designers of decision support systems—if the advice of these systems is to be sought and attended to when it is really needed.” • Alan Aldinger