Using information technology to reduce diagnostic error: still a bridge too far?

Abstract

Diagnostic errors are common, occurring in 5–15% of all clinical encounters and causing serious patient harm in up to 1.0% of hospital admissions and general practitioner visits.1 Diagnostic error is the most frequent cause of malpractice claims, and between 70% and 80% errors are deemed potentially preventable.1 Most errors arise from cognitive missteps in the diagnostic reasoning process of individual clinicians, the most common being failure to consider the correct diagnosis within the differential diagnosis (DDx). With the advent of the digital age, computerised diagnostic decision support systems (CDDSS) have emerged, aimed at improving diagnostic performance of clinicians. The first CDDSS in the 1970s were stand-alone 'expert' rule-based or logic-based systems that generated a list of possible diagnoses in response to manual entry of data on symptoms and signs. These systems were cumbersome to use and addressed only specific domains such as infectious diseases and gastrointestinal disorders.2, 3 In the 1980s/1990s, more sophisticated and broadly applicable DDx generators emerged such as QMR, ILIAD and INTERNIST, although individual studies and meta-analyses revealed only small improvements in diagnostic accuracy.4-6 These systems still required user training and time-intensive manual data entry, generated long lists of possible but non-prioritised diagnoses and were not seen as useful by clinicians. Consequently, they all passed into obscurity. Then in the 2000s/2010s, with the rise of electronic medical records (EMR), CDDSS such as ISABEL, DXplain, SimulCast and DiagnosisPro emerged that generated prioritised DDx based on a smaller number of variables that could be input through the EMR, gave fast turnaround times (down to 90 s or less), and could be used at various stages in the reasoning process. In a meta-analysis of 36 studies of 11 DDx generators published in 2016, the generators retrieved the correct diagnosis in about 70% (95% confidence interval (CI) 63–77) of pooled cases, with ISABEL the best performer at 89% (95% CI 83–94).7 However, this performance was no better than clinicians, apart from a small improvement when clinicians consulted the generator after, not before, formulating their own DDx. Only six studies were rated as high quality, there was considerable between-study heterogeneity and only five of the generators remain in existence. Since then, further refinements in these generators have been undertaken and recent clinical trials involving simulated cases in both general practice and hospital settings show a reasonably consistent 7–8% absolute improvement in diagnostic accuracy.8-10 However, despite their potential, DDx generators have had limited use, with integration into busy clinical workflows being a major challenge. Other CDDSS scour EMR for data that suggest a specific diagnosis might have been missed, and issuing an alert to the treating clinician. One system was able to detect missed cases of pneumonia or heart failure (HF) in hospitalised patients with a sensitivity and specificity of 92% and 90%, and 94% and 90% respectively.11 Another system could diagnose appendicitis in children presenting with abdominal pain with a sensitivity of 87% and positive predictive value of 86%.12 A third system that used a standardised abdominal pain order set, web-based risk stratification tool and ordering alert decreased use of abdominal computed tomography scans in children by 54% with no change in rates of missed appendicitis or appendectomies.13 Systems have also been devised to identify patients who may be at risk of diagnostic error and whose records should be reviewed to ensure errors have not occurred. They operate by applying pre-specified e-triggers to EMR data to identify 'red flag' cases suggesting delays in diagnostic evaluation. In one hospital-based controlled study, patients with possible colorectal cancer (a positive faecal occult blood test being the trigger), lung cancer (a suspicious lesion on chest X-ray) or prostate cancer (an elevated prostatic serum antigen level) were targeted, with 11.8% (1256/10 673) of admissions flagged as being high risk.14 Of these, 749 (59.6%) were found on chart review to be true positives, and time to diagnostic evaluation was significantly lower among intervention than control patients for the colon and prostate triggers, but not the lung trigger. In another study involving acute paediatric admissions, e-triggers of various types identified 23.7% (453/1915) as potential diagnostic omissions, of which 92 (20.3%) were classified on review as likely diagnostic errors, with only six (6.5%) having been detected using other methods.15 More recently, there has been a surge in interest in using artificial intelligence, and more specifically machine learning (ML), to assist clinicians in both diagnostic reasoning using EMR data16 and, more notably, correct interpretation of diagnostic investigations that involve imaged data, such as electrocardiograms, radiological films and histopathological slides.17 While many in silico studies in research settings suggest equivalent or better performance than clinicians, especially less experienced ones, relatively few studies have been conducted in clearly reported, representative clinical work environments, with most showing no difference in performance. Many studies are rated as being at high risk of bias, lack external validation and have not assessed implementation and effectiveness in real-world clinical practice.18 Rather than directly assist diagnostic reasoning, a more practical function of CDDSS might be to ensure the results of diagnostic investigations are notified to treating clinicians. Failure to relay clinically significant or critical test results to the ordering clinician is a source of diagnostic error, especially for patients recently discharged from hospital or seen in ambulatory care settings.19 Such errors may be reduced by automated results notification systems (RNS) embedded within EMR that ensure timely communication of such results, either synchronously (i.e. in real time using pager alerts or phone calls) or asynchronously (i.e. as an email). However, even this has proven difficult to achieve, with studies of RNS showing some positive but often mixed results in the timeliness and reliability of receipt, acknowledgement and action on test results.20, 21 Significant barriers to implementation comprise inconsistent policies and procedures (e.g. weekday vs weekend), inadequate staff and resources, poor system design, limited integration into workflows, lack of digital connectivity between hospitals and external laboratories, uncertainty as to who is the current treating clinician because of unpredictable clinician work schedules, handoffs and availability and variations in clinician response to alerted results. Undoubtedly, CDDSS will continue to evolve and mature over time, and may move from providing advice for single diagnostic steps (such as generating a DDx or interpreting a chest X-ray) to helping clinicians select the initial reasoning paths most likely to arrive at the correct diagnosis.22 Automated retrieval and synthesis of longitudinal EMR data related to past medical history, risk factors and previous tests and treatments may generate suggestions that enable clinicians to refine their line of thinking more quickly and accurately. Is all this sounding like a bridge too far? While the aforementioned functions might become technically feasible over time, there will be challenges in getting clinicians to use CDDSS, for several reasons. First, clinicians may not think diagnostic error is that big a problem and see little need for CDDSS, despite the prevalence of diagnostic error and evidence showing self-assessment of diagnostic accuracy is often poor.23 Second, clinicians might not trust CDDSS advice, particularly 'black-box' ML-based systems giving insufficient explanation as to how advice was generated.24 Third, clinicians might fear loss of professional autonomy or medicolegal consequences if they wrongly reject CDDSS advice.25 Fourth, if CDDSS do not align well with how clinicians think and cannot quickly and consistently provide trustworthy advice when and where it is required, or even worse, disrupt clinical workflows, they will be viewed as a waste of time.26 Last, other widely consulted decision-support resources, such as UpToDate or BMJ Best Practice, have been shown to improve diagnosis27 and may be more palatable to clinicians. In conclusion, using information technology to reduce diagnostic error currently has limited evidence of effectiveness, with DDx generators and e-triggers embedded in EMR showing the most promise. However, while clinicians may view CDDSS as nowhere near ready for prime-time use, many patients seem eager to access websites or download smart-phone apps that provide triage and diagnostic advice,28 despite their uncertain accuracy in many cases. Just to be able to advise patients on the pitfalls of such technologies means clinicians must acquire an understanding of where, when, how and in whom CDDSS might be capable of expediting the journey to a correct diagnosis and reducing diagnostic error. The 'Clinical Decision Support Five Rights' framework has been suggested to guide CDDSS development and implementation.29 In order to improve care, CDDSS must communicate: (i) the right information: accurate, suitable to guide action, pertinent to the circumstance; (ii) to the right person: clinicians, patients and other care providers; (iii) in the right format, such as an alert or suggestion or reference information; (iv) through the right channel: such as EMR or mobile devices; and (v) at the right time in workflow. Achieving all five rights will not be easy, but putting men on the moon was not easy either, but it was eventually achieved.

Ähnliche Arbeiten