Challenges with diagnoses: sketchy reference standards

Abstract

There are many areas of contention in the realm of diagnostic accuracy research and during the validation of selected tests and measures. The majority of the contentious elements have to do with design biases (fixable problems) that have occurred during the improper creation of a particular study. For example, failure to blind researchers often inflates the diagnostic accuracy of a test because the testing clinician is either aware of the diagnosis or is aware of the test result when performing the reference assessment.1 Other design biases include inconsistent methods in which a test is performed, comparing healthy subjects to unhealthy subjects, overly weighting a sample toward the disease process investigated, or failing to identify a positive finding for a given test.2,3 These problems are not easily fixable, but can be repaired, albeit with great effort. Occasionally, diagnostic accuracy studies stumble because of intrinsic problems associated with the reference standard that was used by the clinician researchers to identify the actual diagnosis of a patient. A reference standard refers to the best available method for establishing the presence or absence of a condition of interest, and is also known as the correct representation of the targeted illness.4 Typically, reference standards include dedicated clinical findings and/or imaging or surgical identification/confirmation. Occasionally, a usable surrogate is considered acceptable (for example, a Computed Tomography scan used in lieu of magnetic resonance imaging (MRI), or diagnostic ultrasound used in place of an MRI). In rare occasions, the reference standard is woefully vague, is lacking in agreement across the clinical research community, or, oddly enough, there isn’t one. In situations where more than one reference standard exits, there may be a hierarchy among the standards; with the ‘gold standard’ representing the highest level of reference. An example of a ‘gold standard’ would be the evidence of a traverse femoral fracture on a radiograph or the present of a full thickness tear during investigation with arthroscopic surgery. A criterion standard is a lower level of corollary in that the reference standard used is designed to epitomize the diagnosis at hand and is imperfect in its representation. In essence, the use of a criterion standard is necessary for a number of diagnoses that have no outwardly tangible pathology that is seen, felt, imaged, tested using lab findings, or identified during surgical exploration. An example of a criterion standard is the use of an electromyography (EMG) finding to represent cervical radiculopathy. Another example of a criterion standard is the use of a combined clinical diagnosis/imaging confirmation in the diagnosis of cervical myelopathy.5 As stated, in some cases the reference standard is vague or is lacking in agreement across the clinical research community. This is a problem that is well recognized and in past meta-analyses statistical adjustments have been made for a fuzzy reference standard, as the diagnostic accuracy of a test is often understated in its presence.6,7 It has been my experience that nothing promotes greater ire in a manuscript reviewer than if they do not agree with the reference standard used within a diagnostic accuracy study. It’s something I’ve personally experienced in four published diagnostic accuracy papers,5,8–10 and in nearly all cases it can’t be helped. As an orthopedic clinical community we have grown to recognize conditions such as thoracic outlet syndrome, scapular dyskinesia, and lumbar spine instability, despite our inability to quantify each of the conditions with a solid reference standard and despite our inability to tie these conditions with direct biological causation. These conditions, sometimes referred to as symptom-based diagnoses, place innumerable challenges upon the clinician researcher. For example, many symptom-based diagnoses look markedly different from person to person and are based solely on a collection of symptoms (e.g., fibromyalgia). Indeed, if we elect to recognize the conditions then we need accurate tests and measures that are distinctive to each condition. We need to agree upon a reference standard that best represents each condition, with full knowledge that the reference standard is and will likely always be, imperfect. And, we need to adopt a culture of understanding that selected diagnoses are very difficult to quantify and because of sketchy references standards, are nearly undiagnosable.

Ähnliche Arbeiten