A Roland Morris Disability Questionnaire Target Value to Distinguish between Functional and Dysfunctional States in People with Low Back Pain

* School of Rehabilitation Science, McMaster University, Hamilton, Ont.

Daniel L. Riddle

† Departments of Physical Therapy and Orthopaedic Surgery, Virginia Commonwealth University, Richmond, Va.

† Departments of Physical Therapy and Orthopaedic Surgery, Virginia Commonwealth University, Richmond, Va.

Contributors: All authors designed the study; collected, analyzed, and interpreted the data; drafted or critically revised the article; and approved the final draft.

Correspondence to: Paul W. Stratford, School of Rehabilitation Science, Institute for Applied Health Sciences, McMaster University, 1400 Main St. W., Hamilton, ON L8S 1C7; ac.retsamcm@roftarts

Abstract

Purpose: To estimate a threshold Roland Morris Disability Questionnaire (RMQ) value that could be used to classify patients with low back pain (LBP) as functional or dysfunctional. Methods: In this secondary analysis of data from a study that estimated clinically important RMQ change scores, participants were adults with LBP attending one of three physical therapy clinics. Diagnostic test methodology and a reference standard of goals met were applied to estimate a threshold RMQ value that best distinguished between participants with a functional status and those whose status was dysfunctional. Results: Of 143 participants, 104 (73%) met their goals. An RMQ threshold value of 4/24 best distinguished between those who met their goals and those who did not. Sensitivity and specificity for a threshold score of 4 were 94% (95% CI, 88–98) and 69% (95% CI, 52–83), respectively. Conclusions: A threshold value of 4 RMQ points provided a reasonably accurate classification of patients. Further research is necessary to cross-validate this estimate and to examine the stability of the estimated value in people with diverse functional demands.

Résumé

Objectif : Estimer une valeur seuil du questionnaire Roland-Morris (QRM) qui pourrait servir à classer les patients qui souffrent de lombalgie dans les catégories de patients « fonctionnels » ou « dysfonctionnels ». Méthodes : Dans la présente analyse secondaire de données provenant d'une étude qui faisait l'estimation des cotations de changements importants sur le plan clinique selon le QRM, les participants étaient des adultes atteints de lombalgie, qui fréquentaient l'une de trois cliniques de physiothérapie. On a appliqué la méthodologie de test de diagnostic et une norme de référence des objectifs atteints pour estimer une valeur seuil du QRM qui permettait de distinguer le plus clairement possible les participants qui étaient fonctionnels de ceux qui étaient dysfonctionnels. Résultats : Parmi les 143 participants, 104 (73%) ont atteint leurs objectifs. Une valeur seuil du QRM de 4/24 permettait de distinguer le plus clairement possible ceux qui avaient atteint leurs objectifs de ceux qui ne les avaient pas atteints. La sensibilité et la spécificité pour une cotation seuil de 4 étaient de 94% (IC de 95%, 88–98) et de 69% (IC de 95%, 52–83) respectivement. Conclusions : Une valeur seuil de 4 points selon le QRM permettait de classer les patients de façon raisonnablement exacte. Il faudra effectuer une recherche approfondie pour faire la contre-validation de cette estimation et pour examiner la stabilité de la valeur estimée chez les personnes ayant diverses demandes fonctionnelles.

Establishing and evaluating measurable goals is a key component of physical therapy practice. Two aspects of a measurable goal are the outcome value and the expected interval for achieving the goal. Measurable goals can be written in terms of change scores or target values. For example, “increase knee flexion by 10° in 2 weeks” is a measurable goal written as a change score, whereas “increase knee flexion to 143° in 4 weeks” is a measurable goal written as a target value. To date, research and application have focused on estimating, reporting, and defining successful outcomes in terms of change scores. If the threshold change score is met or exceeded, the outcome is considered clinically significant or successful. A limitation of this method is that it is possible for a patient to meet the threshold change score yet still have significant functional limitations. In this article, we estimate the target value for the Roland Morris Disability Questionnaire (RMQ), a measure of functional limitation, in people with low back pain (LBP). 1

In both clinical practice and clinical research, continuous outcomes are often compressed into dichotomous decisions. 2 Evaluating whether a patient has achieved a target goal value is a popular approach for doing so. In clinical practice, the decision frequently involves determining whether the target value has been achieved to aid in the decision to discharge a patient from active treatment. In clinical research, outcomes are often labelled successes or failures to avoid a situation in which, despite a statistically significant between-groups difference in mean scores, few or none of the patients in the intervention group achieve a clinically important improvement. The potential for this disconnect stems from the fact that an important between-groups difference is less than an important within-patient improvement. 3 For the RMQ, a between-groups difference of 2 points is considered clinically important, whereas a within-patient change of 4 or 5 points is recognized as the threshold for a clinically important improvement. 4

Recognizing that a successful outcome is multifaceted, Jacobson and colleagues 5 proposed that a patient must move from a dysfunctional to a functional state and that the change must be statistically reliable. To address the functional–dysfunctional state criterion, Jacobson and colleagues applied diagnostic test methodology to answer the question “Does the level of functioning at posttest suggest that the participant is statistically more likely to be in the functional population than in the dysfunctional population; that is, is the posttest score statistically more likely to be drawn from the functional than the dysfunctional distribution?” 5(p.340)

More recently, Tubach and colleagues 6 have introduced a variation on this theme referred to as the Patient Acceptable Symptom State, defined as “the value beyond which patients can consider themselves well.” 6(p.34) In addition to meeting the threshold functional score, Jacobson and colleagues 5 proposed a second standard—the reliability change index (RCI)—to minimize the chance that a patient with a pretest score close to the target value will move from a dysfunctional score to a functional score as a result of measurement error. The RCI is calculated as the difference between pretest and posttest scores divided by the standard error of measurement (SEM) times the square root of 2. 7 It is interpreted as a standard normal deviate (Z score). Accordingly, SEM×√2 multiplied by an RCI of 1.65 (where 1.65 is the Z value for the 90% CI) would be analogous to a minimal detectable change at the 90% confidence level.

The RMQ is a commonly used patient-reported outcome measure 1 that assesses pain-related functional status; its measurement properties are consistent with or better than those of competing measures. 8 A body of work has estimated threshold change scores for the RMQ using both reliability-based and diagnostic test methodologies. 9–13 Typical estimates of true and important change for the RMQ are around 5 points. 9,14,15

A challenge of assessing health concepts such as functional status is the lack of a gold standard or error-free reference standard against which the health concept or outcome of interest can be directly compared. In such situations, a construct validation process is applied, which involves forming theories about the condition or outcome of interest and then testing the extent to which the measure provides results consistent with the theories. 16 We are aware of only one study that has attempted to estimate a recovery score for the RMQ. Using an 11-point global perceived effect reference standard (−5=vastly worse, 5=completely recovered) completed by patients, Kamper and colleagues 17 estimated that an RMQ score of ≤2 best distinguished between patients who considered themselves completely recovered and all other patients. However, they found little difference in diagnostic accuracy between this cut-score and a cut-score of ≤4. Given the limited information available concerning a target RMQ value, our study's purpose was to estimate a threshold target RMQ value that could be used to classify patients with LBP as functional or dysfunctional. In the context of a construct validation design, our theory was that patients who were meeting their treatment goals would be more likely to be in a functional state than those who were not meeting their goals.

Methods

This study was a secondary analysis of data gathered during a previously reported investigation. 14 The institutional review board exempted this study from the requirement to obtain informed consent because the data were obtained as part of routine care at these sites. All data were de-identified before analysis. The purpose of the original study was to estimate the minimal clinically important change for RMQ scores and to determine the extent to which the change estimates were dependent on baseline scores. The present study, however, focused on identifying the final discharge score that best differentiates functional from dysfunctional patients, as categorized by the goals-met reference standard, with the goal of fully exploring a threshold for interpreting discharge RMQ scores.

Participants

Patients were eligible for inclusion if they were referred by physicians between November 1993 and December 1995 to any of three physical therapy clinics for treatment of LBP. Patients were excluded if they were referred with other conditions in addition to LBP or if, in the therapist's judgment, they had been diagnosed with other problems that might adversely affect their disability. The study sample was one of convenience and included all patients who fulfilled the eligibility criteria and who provided both admission and discharge RMQ scores.

Protocol

Patients completed the RMQ immediately before their initial examination and immediately after their final visit; they were blinded to their initial RMQ responses when completing their discharge RMQ. On the basis of information obtained at the initial assessment, therapists in collaboration with patients developed a treatment plan and set goals. At the final visit, the therapist specified the reason for discharge: (1) patient achieved all treatment goals, (2) loss of insurance coverage, (3) referral to another practitioner, (4) patient did not appear for appointment, and (5) other. Therapists were unaware of the RMQ discharge score when specifying the reason for discharge.

Goal setting and evaluation

All study sites were part of the same private practice corporation and applied a similar approach to history taking and assessment. Using a collaborative process, patients and therapists jointly set treatment goals. Therapists were required to set at least one goal, and if a patient's employment status was affected by LBP, achievement of the goal was conditional on his or her return to the pre-injury work level. Patients were asked to identify functional activities affected by their LBP; goals were patient specific and included a spectrum of activity levels. Patients indicated to their therapists whether or not they had achieved their goals. All goals had to be met for a patient to be classified as “goals met.”

Roland Morris Disability Questionnaire

The RMQ is a 24-item patient-reported outcome measure that inquires about pain-related disability resulting from LBP. 1 Items are scored 0 if left blank or 1 if endorsed, for a total RMQ score ranging from 0 to 24; higher scores represent higher levels of pain-related disability. Typical RMQ test–retest estimates are in the range of 0.79 to 0.88 points for relative reliability (intra-class correlation) and 1.7 to 2.0 points for absolute reliability (SEM). 18,19 The threshold for important change has been estimated to be approximately 5 RMQ points. 15,19,20

Data analysis

We analyzed the data using STATA version 13.1 (StataCorp LP, College Station, TX). Patient characteristics and RMQ scores were summarized as quartiles for continuous variables and as proportions or counts for categorical variables.

We applied receiver operating characteristic (ROC) curve analysis to identify the RMQ score that most accurately classified patients as having met or not met their treatment goals. 21 An ROC curve plots sensitivity (the number of patients correctly identified by the RMQ as having met their goals divided by the number of patients who truly met their goals) against 1−specificity (number of patients correctly identified by the RMQ as not having met their goals divided by number of patients who truly did not meet their goals). The area under the ROC curve, which can take any value from 0 to 1, quantifies the measure's accuracy: The closer the area is to 1, the greater the accuracy. An area of 0.50 indicates that the measure does no better than chance at classifying patients as having met their goals or not. We estimated the cut-point that best classified patients as having met or not met their goals as the RMQ score that jointly maximized sensitivity and specificity, then applied sensitivity and specificity values obtained for the cut-point score to estimate the chance that a patient's goals were met by combining the pre-RMQ chance of a patient's achieving his or her goals with sensitivity and specificity information (see Appendix for an example). We use the term information gain to denote the difference between the pre-RMQ chance of labeling a patient's outcome as goals met and the post-RMQ chance of doing so.

We provide a vignette illustrating a 50% pre-RMQ chance for patients meeting their goals in the Discussion with elaboration in the Appendix. Previous research has shown that misclassification errors (i.e., labeling patients as having met their goals when in fact they have not, or vice versa) are minimized when the pretest chance of an outcome is 50%. 22 In the context of writing a measurable goal for discharge, the best timeframe for an individual patient to achieve his or her goal would be when 50% of patients with similar characteristics would be expected to achieve their goals. Moreover, typical recovery curves, such as the one provided by Gurcay and colleagues 23 that we refer to in the Discussion section, provide the average change trajectory for patients. Taken literally, this would be the 50th percentile value, which translates into a 50% chance that any patient sharing the characteristics of the sample would achieve the target value within the specified timeframe.

Results

The original study by Riddle and colleagues 14 described the characteristics of our participant sample in detail. In brief, 143 patients provided admission and discharge RMQ scores. The median duration of treatment was 30 (1st, 3rd quartiles: 19, 150) days, and the sample's median age was 39 (1st, 3rd quartiles: 31, 48) years; 60 patients (42%) were male, and 7 (5%) were receiving workers' compensation. Table 1 and Figure 1 report summary RMQ scores.

Table 1

Summary Roland Morris Scores