 |
| About Bioline | All Journals | Testimonials | Membership | News |
|
||||||
|
||||||
African Health Sciences, Vol. 5, No. 2, June, 2005, pp.145-151 The Response Inventory for Stressful Life Events (RISLE) II: Valid tion of the 36-item Version Emilio Ovuga1 Jed Boardman2 Danuta Wasserman3
Code Number: hs05027 ABSTRACT Background: A 36-item version of the Response Inventory for
Stressful Life Events (RISLE) was derived from the longer 100item version.
The 36-item version may be more appropriate for use in larger population sample. Depression1-4 and suicide ideation 5-10 are prevalent in the general population. However many cases of depression reain undiagnosed and do not recieve appropriate medicatation or general psychosocial intervention. The present study investigaed the feasibility of using a screen instrument to enhance the recognition and diagnosis of depression and significan suicidal thoughts in the general popultion. The accompanying paper 11 presented the RISLE and teh mean by which the longer, 100-item version, was reduced to a more practical 36-item version. Since the instrument has to be utilized clinically it is essential to validate the shorter 36-item version as a screening instrument. This paper presents the reslult of further vlaidation of the 36-item RISLE by comparing it to the results of clincial reserach diagnotic interviews conducted on a sub-sample of the original population and student samples. METHOD Study Populations For the further validation of the 36-item RISLE, subsamples of the population sample from the Adjumani District and from teh fresher student sample described in Ovuag et al 12-14 were used. General population sample At the beginning of questionnaire surveys in Adjumani district, every alternate participant out of a sub-sample of 117 respondents was informed that the author or his assistant would interview him or her further. Each of the participants selected for clinical interviews was notified that the outcome of his or her interview would be explained to him or her in confidence, and that appropriate advice for further action would be provided if required. At the end of questionnaire sur veys inter viewers selected 57 alter nate respondents for research clinical interviews. Student Sample At the start of questionnaire surveys, the first author notified the participating students that all students who consented to participate in the study would be requested to report for clinical interviews within 2 weeks of the surveys. The first author informed students that he or his assistant would provide confidential feedback and appropriate advice to every student on the outcome of clinical interviews immediately after the interviews. Research clinical interviews were conducted for 58 students within three weeks of questionnaire surveys. A Master of Science clinical psychology student carried out ten interviews and the first author conducted the rest. Research Clinical Interviews Interviews were conducted using the Mini Interna15 tional Neuropsychiatric Interview (MINI). Interviewerswere not aware of the RISLE scores of the subjects that were interviewed as data entry and analysis had not yet been carried out. It can be reliably administered by lay interviewers using appropriate training. The depression, manic, anxiety, alcohol dependence, post-traumatic stress disorder, and past month suicidality modules of the schedule were used in the study. DSM-IV diagnoses are made according to diagnostic algorithms, which require a fixed number of symptoms, a minimum duration of symptoms as distressing, and a definite impairment of social functioning as a result of symptoms. The MINI was translated by independent translators; translated back and forth into the local languages of the participating communities by the interviewers under the supervision of EO, and pretested during a one-week intensive training workshop for the five interviewers and two research assistants. The training workshops were conducted in the district hospital in each district. Comments received from participants during the pre-testing exercise were used to modify the instrument where necessary. Establishing the gold standard Using the MINI, three standards were used to compare to the RISLE: current DSM-IV depressive disorder; any current DSM-IV psychiatric disorder (depressive disorder, anxiety, manic, alcohol dependence, post-traumatic stress disorder); and suicide ideation at clinical interview. The depression module of the MINI was used to make clinical diagnosis of any current depressive disorder, and the manic, anxiety, alcohol dependence or post-traumatic stress disorder modules were used to make additional diagnosis of any current psychiatric disorder. The proportion of respondents with suicide ideation, and risk of suicide potential among respondents over the preceding month was estimated using the suicidality module on the MINI. Data management and analysisData was entered using Epi Info version 6.04 and analyzed with SPSS version 11.0. Receiver operating characteristic curves (ROC) 18, 19 were constructed for the shortened RISLE instrument using current depressive disorder, any current psychiatric disorder that considered all respondents with a psychiatric disorder, and past month suicidality rates derived from clinical interviews with the MINI. The ROCs were used to determine specificities, sensitivities for the new short form of the RISLE. Cohen's kappa values for the degree of agreement between the RISLE and clinical interview results were calculated to determine optimal cut-off points on the revised RISLE for students and members of the general population. Sensitivities, specificities, predictive values and likelihood ratios were calculated for varying cut-off scores of the 36-item RISLE. RESULTSGeneral population Clinical interviews identified 49 out of 57 respondents (86.0%) who met DSM-IV diagnostic criteria for any current psychiatric disorder. Of the 57 respondents 35 (61.4%) met DSM-IV diagnostic criteria for any current depressive disorder. Student population Of the 101 students who completed study questionnaires, 58 (57.4%) agreed to be interviewed. Nineteen out the 58 respondents (32.8%) met diagnostic criteria for any current psychiatric disorder according to the MINI; nine students out of the 58 (15.5%) met clinical criteria for current depressive disorder. In 15 of 19 (78.9%) cases with any current psychiatric disorder, respondents had two or more diagnoses. The demographic characteristics of the students who participated in clinical interviews were not significantly different from those who did not take part in clinical interviews. ROC analysis The student and general population samples were combined for this analysis in order to increase the number of subjects available for analysis. ROC curves were constructed separately for current depressive disorder, any current psychiatric disorder, and past month suicidality (figures 1, 2, 3). The area under the curve represents the probability that an individual with a positive screen on a survey instrument has depressive disorder, any current psychiatric disorder or high risk suicide ideation. The probability of a correct detection of an individual with current depressive disorder was fair at 79%; with any current psychiatric disorder the probability of a correct detection was good at 83%, and with past month suicidality probability was 83%. Agreement between the RISLE and Clinical Interview Method Cohen's kappa values indicated that the best agreement between the revised 36-item RISLE and clinical interview with the MINI among students for any current psychiatric disorder was achieved at a cut-off point of 6 (kappa = 0.501), and among the general population, the cut-off point was 10 (kappa = 0.508). Cut-off Points and RISLE Sensitivity, Specificity Table 1 depicts the sensitivities, specificities, positive predictive values and positive likelihood ratios of the 36item RISLE. At a cut-off point of 6 for combined general population and student samples for any current psychiatric disorder, sensitivity was 88.1%, specificity was 60.4%, positive predictive value was 75.6%, negative predictive value was 78.4%, false positive rate was 11.9%, false negative rate was 39.6%, and positive likelihood ratio was 1.9. At a cut-off point of 10 for the combined samples for any current psychiatric disorder, sensitivity was 74.6%, specificity was 77.1%, positive predictive rate was 82.0%, negative predictive rate was 68.5%, false positive rate was 25.4%, false negative rate was 22.9% and positive likelihood ratio was 3.6. Using a cut off of 10 the prevalence of current depressive disorder according to the 36-item RISLE among the general population was 61.1% and from the clinical interview technique was 61.4%. The prevalence rates among students at a cut-off point of 6 were 19.8% for the RISLE and 13.8% for clinical interview method. The prevalence of any current psychiatric disorder among members of the general population (cut point 10) and among students (cut point 6) according to the RISLE and clinical inter view method were 89.6% and 86.0% respectively. The rates among students (cut point 6) for any current psychiatric disorder were 41.6% and 32.7% respectively. Relationship between RISLE and suicidal behavior and ideation People with scores of 10 or more on the 36 item RISLE were more than twice as likely to be associated with a past history of suicide attempt than those with scores of 9 or less (OR=2.25, 95% CI = 1.34-3.96, X2 = 9.61, df = 1, P = 0.002); almost four times as likely to be associated with lifetime suicide urge (OR=3.86, 95% CI =2.60-5.67, X2 = 53.01, df = 1, P = 0.0000) and almost three times as likely to be associated with lifetime death wish (OR=2.87, 95% CI = 2.00-4.18, X2 = 35.43, df = 1, P = 0.0000). Scores of 10 or above were also associated with presence of suicide ideation within the past week defined as suicide urge or death wish (OR=2.33, 95% CI = 1.32-4.40, X2 = 8.64, df = 1, P = 0.003). Demographic characteristics, medical histories and RISLE performance The individual demographic characteristics, histories of suicidal attempts and ideation, family history of completed suicides and attempts, and psychiatric disorders of the 57 interviewees in Adjumani district were compared to their 36-item RISLE scores. Of 50 interviewees with any current psychiatric disorder, the 36-item RISLE positively detected 46 (92.0%) and missed 4 respondents who met diagnostic criteria for any current psychiatric disorder, but mislabeled 7 (12.3%) other respondents out of the 57 interviewees as false positives at a cutoff point of 10. Though numbers were small for purposes ofstatistical analysis, the individuals who were false positives were more likely to have a positive personal lifetime history of suicide ideation defined as lifetime suicide wish and or lifetime death wish, and positive personal history of suicide attempt. In addition, the individuals concerned were more likely to report a positive family history of suicide behavior defined as family history of suicide and or suicide attempt. A similar exercise conducted among the students showed that the 36-item RISLE did not do as well as among the general population. At a cut point of 10 the RISLE correctly detected 5 of 19 respondents (26.3%) who met diagnostic criteria for any current psychiatric disorder. Lowering the cut point to 6 for the student population improved the detection ability of the instrument from 5 to 14 out of 19 respondents (73.7%) with any current psychiatric disorder. The false negatives among students at a cut point of 6 were less likely to report a personal history of suicide ideation than students with true positives. DISCUSSIONThis study has taken the 100-item RISLE and attempted to refine and validate the instrument using general population and student samples. The accompanying paper reduced the length of the RISLE to 36 items and found moderate correlations between the 36-item RISLE and two well known questionnaires, the BDI20-22 and BSS23. Since the instrument has to be clinically utilized it was essential to validate the refined 36-item RISLE as a screening tool for use in the Ugandan general population. This paper examined the clinical validity of the 36-item RISLE by comparing it to standards obtained through a clinical diagnostic research interview. Performance of the 36-item RISLE The results from the ROC analysis suggests that overall the 36-item RISLE is better at detecting people with any current psychiatric disorders and people who had experienced recent suicidal thoughts than those with depressive disorders. This suggests that the RISLE may not be a specific measure of depression, possibly because of its composite nature, and may better reflect general psychological distress which correlates better with suicidal thoughts and acts and overall psychological morbidity. This is common to other questionnaires in this field such as the Beck Depression Inventory22 and General Health Question naire24. The cut-off point for optimal detection for the entire sample appeared to be optimal at 10, giving a sensitivity of 74.6% and specificity of 77.1%. At this cutoff point the 36-item RISLE correctly detected 46 of 50 respondents (92.0%) from the general population with any form of current psychiatric illness and was associated with individual and family experiences of suicidal acts or thoughts. However it did less well at this cut-off point in the student population detecting only 5 of 19 students who met criteria for any current psychiatric disorder. The performance of the 36-item RISLE improved when the cut point for students was lowered to 6 at which the kappa value was 0.501, and the instrument correctly detected 14 out of 19 (73.7%) with any current psychiatric disorder. This difference between RISLE performance for the general and student populations may reflect the differences in prevalence of disorder in the two populations, a parameter that determines the cut-off point for screening instruments. For example the GHQ optimal cut-off score is sensitive to the prevalence of psychiatric disorder and it has been suggested that the median of the GHQ score gives a reliable estimate of the cut-off score25. This may be the case for the RISLE. Comparison with other studies In a review of studies involving 18 case-finding instruments, Mulrow and colleagues26 reported that sensitivities and specificities for detecting major depression ranged from 67% to 99% and 40% to 95% respectively; overall sensitivity was 84% and specificity, 72%. In another review of the accuracy and precision of depression questionnaires and clinical examination, Williams and colleagues reported that the median positive likelihood ratio for major depression was 3.3 with a range of 2.3 to 12.2 27. These results are in overall agreement with the RISLE sensitivity, specificity and positive likelihood ratio for any current psychiatric disorder of 74.6%, 77.1% and 3.6 respectively at cut point of 10. Eaton and colleagues28 have suggested that agreement between questionnaire29 surveys and clinical interview results was poor with agreement between the two methods being fair at a kappa value of 0.20. Similarly Regier and colleagues have reported that clinical interviews produced consistently higher prevalence rates for major depression than population-based surveys involving the use of questionnaires. These findings have not been replicated in this study, which has produced a kappa value of 0.508 for any current psychiatric disorder and overall prevalence rates being higher on the RISLE than for clinical interview method. Methodological issues There were two main methodological drawbacks to the study. First the low numbers of individuals interviewed to provide the gold standard. This was the case for both the general population and student samples and has resulted in the need to combine samples for parts of the analysis. Further work on larger samples in prospective follow-up studies is required. The sample size however did not affect the first part of the study; the refinement of the 100-item version and the comparison to the BDI and BSS which, particularly for the general population sample, used sufficient numbers. A second problem was the time lapse between the administration of questionnaires and the conducting of the interviews during which time the clinical state of the subjects may have changed. However, some measures such as lifetime experience of suicidal thoughts which were unlikely to have changed did correlate with the RISLE. These limitations reinforce the preliminary nature of this validation exercise. Implications Reducing the number of items making it more adaptable to general population research has refined the RISLE. The resulting 36-item RISLE gives scores that are highly correlated with the 100-item RISLE and also with previously known and accepted questionnaires measuring depressive symptoms and suicidal thoughts. The validation of the 36-item RISLE suggests that, in common with other clinical questionnaires measuring psychological symptoms, it is a better measure of overall psychological distress than specific to depressive symptomatology. Its value as a screening instrument is perhaps less well established, but nonetheless the current findings suggest it is an adequate screening instrument in the general population at a cut-off score of 10. The results suggest that the RISLE is able to screen for both depressed mood and high-risk suicide behavior at recommended cut-off points. Further work is however required to establish its worth as a screening device and its performance in different populations. The present study suggests that it can be used in African populations and may be of value in establishing estimated levels of prevalence of psychological symptoms in such populations. We suggest, however, that the RISLE is a dimensional scale and should not be used as a categorical diagnostic instrument. As is the usual practice anyone who screens positive on the RISLE at the recommended cut-off points should undergo clinical diagnostic interview to ascertain the presence and nature of psychiatric disorder. ACKNOWLEDGEMENTSThanks to The Swedish National Center for Suicide Research and Prevention of Mental Ill-Health (NASP) and the Department of Public Health Sciences at the Karolinska Institute, Stockholm, Sweden for developing in collaboration with the Makerere University, Kampala, Uganda of a Ugandan National Plan for Prevention of Mental Ill-Health and Suicide. We are grateful to Dr Seggane Musisi for valuable comments and guidance. DECLARATION OF INTERESTThis study was supported by SIDA/SAREC funds to build research capacity at Makerere University. REFERENCE:
Copyright 2005 - Makerere Medical School, Uganda
|
