Malaysian Journal of Medical Sciences School of Medical Sciences, Universiti Sains Malaysia ISSN: 1394-195X Vol. 13, Num. 1, 2006, pp. 58-62

ORIGINAL ARTICLE

THE CLINICAL INTERVIEW SCHEDULE- REVISED (CIS-R)– MALAY VERSION, CLINICAL VALIDATION

Kavitha Subramaniam, Saroja Krishnaswamy*, Abdul Aziz Jemain, Abdul Hamid*, Vikram Patel**

Department of Statistics, National University of Malaysia, Bangi, Malaysia *Department of Psychiatry, National University of Malaysia Hospital, Kuala Lumpur, Malaysia. **London School of Hygiene and Tropical Medicine, London, England.
Correspondence : Professor Saroja Krishnaswamy MBBS (Malaya), FRCP (UK), MSc Clinical (EPIB Aus) Department of Psychiatry, National University of Malaysia hospital, Jalan Yaacob Latif, Bandar Tun Razak, 56000 Cheras, Kuala Lumpur Malaysia Tel : 603-91702665 (office) 6019-2393413 (mobile) Facs : 603-91702665 Email : saroja@mail.hukm.ukm.my, saroja54 @hotmail.com

Code Number: mj06010

Use of instruments or questionnaires in different cultural settings without proper validation can result in inaccurate results. Issues like reliability, validity, feasibility and acceptability should be considered in the use of an instrument. The study aims to determine the usefulness of the CIS-R Malay version in detecting common mental health problems specifically to establish the validity. The CIS-R instrument (PROQSY* format) was translated through the back translation process into Malay. Inter rater reliability was established for raters who were medical students. Cases and controls for the study were psychiatric in patients, out patient and relatives or friends accompanying the patients to the clinic or visiting the inpatients. The Malay version of CIS-R was administered to all cases and controls. All cases and controls involved in the study were rated by psychiatrists for psychiatric morbidity using the SCID as a guideline. Specificity and sensitivity of the CIS-R to the assessment by the psychiatrist were determined. The Malay version of CIS-R showed 100% sensitivity and 96.15% specificity at a cut off score of 9. The CIS-R can be a useful instrument for clinical and research use in the Malaysian population for diagnosing common mental disorders like depression and anxiety.

Key words : Mental health, psychiatrist, clinical interview schedule

Introduction

The clinical Interview Schedule- Revised is a fully structured diagnostic instrument that was developed from an existing instrument, the Clinical Interview Schedule (CIS), which was designed for the use of clinically experienced interviewers such as psychiatrists (1). The CIS was revised and developed into a fully structured interview in order to increase standardisation and to make it suitable to be used by trained lay interviewers in assessing minor psychiatric morbidity in the community, general hospital, occupational and primary care research (2). The ICD-10 diagnoses could be derived for the CIS-R data from an algorithm, based on the Published Diagnostic Criteria for Research (3). A computer algorithm was also developed to enable generation of ICD-10 diagnosis using the Programmable Questionnaire System (PROQSY) (4). PROQSY enables the automatic generation of diagnoses without psychiatric consultation.

The validation of the CIS-R instrument by comparing it with various other assessments have been carried out before; the results of these studies have been published widely. In Britain, the CIS-R instrument was validated in a community setting by comparing with a semi structured instrument, Schedule for Clinical Assessment in Neuropsychiatry, SCAN. The subjects for the study were adults aged 16 to 64 years. Results of the study showed that the overall specificity of the CIS-R instrument was good, but sensitivity was poor. The range of the overall percentage agreement was from 07-0.9. The percentage agreement calculated after the exclusion of those who scored negative in both studies ranged from 0-0.3. Concordance findings (concordance between diagnoses by the 2 instruments) according to kappa were in the poor to very poor range (1). The author concluded that there is a poor agreement between the SCAN and the CISR on the identification of a range of specific ICD- 10 neurotic disorders(1). A validation study was carried out in Australia by comparing the self administered computerised form of the CIS-R with Composite International Diagnostic Interview (CIDI) instrument. It was a two phase study involving Australian High School students (adolescents with a mean age of 15.7 years). The diagnostic algorithm of the CIS-R was revised using the data from a calibration study, in order to provide maximum agreement between the CIS-R and CIDI instruments. The revised algorithm was used for the survey. The original CIS-R algorithm had high specificity value of 0.97 and a low sensitivity value of 0.49. The revised algorithm had a higher specificity and PPV rates (0.99 and 0.75) and lower sensitivity rate (0.30) in comparison to the original algorithm(5).

The CIS-R instrument has also been translated into many other languages and validated. One such study was carried out in Sri Lanka. The Sinhala version of the CIS-R instrument was validated in a community setting by comparing with assessment by psychiatrists using local (emic) diagnostic procedures. The subjects for the study were adolescents aged 15-19 attending the psychiatric clinic for the first time (6). It was reported that all the filtering questions had a high level of sensitivity (80%-96%). The Spanish version of the instrument has been used extensively. The Spanish translated instrument has showed a good reliability and validity (2).

The CIS-R instrument has proved to be a viable and valid instrument for detection of common mental disorders in other countries such as Zimbabwe (7), India (8) and Taiwan (9). In this study, we have two main objectives. One is to determine whether the Malay translated CIS-R is a valid instrument for assessment of minor psychiatric morbidity in the Malaysian population and second to obtain a suitable cut off score or the optimal operating point (OOP) for the instrument in the Malaysian context.

Methodology

The CIS-R questionnaire was translated into the Malay language and back translated into English. The original questionnaire and back-translated questionnaire were compared for meanings; necessary changes were made to the Malay translation to produce similar understandings with the original English questionnaire. The corrections were made with the help of a language teacher.

The interviewers or raters for this study were medical students. The medical students were trained to use the computerised version of the CIS-R instrument and given six cases each to note. Interrater reliability was assessed with the main authors for the diagnosis. Only raters who were able to rate all the six cases correctly according to diagnosis generated by psychiatrists (i.e. 100% reliability) were allowed to continue with the study

A total of 59 subjects were recruited for this survey over a period of 2 months. The recruitments were carried on in the National University of Malaysia Hospital, (HUKM). The subjects were psychiatric inpatients, outpatients the relatives and other visitors who were there at that time. Only adults 16 and above who consented were eligible for the study. Those with dementia or psychotic disorders and not able to answer questions well were excluded from the study.

The recruited subjects were given reference numbers randomly. A list of reference numbers was given to the each rater. Each subject was assessed using the CIS-R instrument. All the subjects were assessed again for psychiatric morbidity by psychiatrists (author) using the SCID as a guideline. Psychiatrists were blinded for cases and controls. The PROQSY diagnoses (as generated) and diagnoses by the psychiatrists were noted for each reference number for further analysis.

The frequency distribution curves for the case and control category were plotted. Overlapping points were taken as the potential cut off points. Diagnosis made by the psychiatrists and PROQSY were compared. The outcomes of the PROQSY diagnoses for each potential cut off scores were cross tabulated against the outcomes of the SCID diagnoses. Based on the classification, the specificity, sensitivity and 100-specificity, Positive predictive Value (PPV) and Negative Predictive value (NPV) were calculated for all the cut-off scores. The Relative Operating Character (ROC) curve or the sensitivity versus 100-specificity curve was plotted for the cut off points. The suitable cut off score was chosen based on the calculated values and the visual display of the scores in the ROC curve.

Out of the 59 subjects , 33 were noted to be cases and 26 were controls when the unblinding was done. The sample included 51% Malays, 29% Chinese and 20% Indians. Majority of the respondents, (71%) were females and 29% were males. The respondents were aged between 16-64 years old. The mean age was 34.75 ± 13.08.

The control group showed a score range from 0-13. The mean score was 2.08 ± 3.11 . The mode score was 0. The case group showed a score range from 9-36.The mean score was 20.00 ± 6.80 . The mode score was 15. All the cases from a range of anxiety and depressive disorders, but this diagnostics were not sampled separately.

The overall sensitivity, specificity, Positive predictive value and negaive predictive value for the instrument were calculated. The results are explained in (Table 1). The frequency versus score curves were plotted for both case and control groups. The overlapping region of both the curves was from the score 8 to 14. Seven potential cut off points, 8,9,10,11,12,13 and 14 were identified. The specificity and sensitivity values for each cut off score were calculated. The ROC curve was plotted (Fig. 1). The results are given below (Table 2)

The cut off points 9 and 10 has the best sensitivity and specificity rates, 100% and 96.15% each. The visual display of the potential cut off points in the ROC curve show that the scores 9 and 10 fall on the same point in the graph; the two points at the upper left corner of the curve, being the highest point that is the nearest to the y-axis. The best Area Under Curve (AUC) value, was obtained when the score 9 or 10 was chosen as the up most point of the curve. Thus the best cut off score for the Malaysian population is either 9 or 10. The scores 9 and 10 exhibit the same sensitivity, specificity and PPV value. The two scores differ in the NPV value. The NPV value for score 9 is 1.00 and 10 is 0.93. This finding indicate that a person classified as not having any common psychiatric disorders with 9 as the cut off point has a 100% likelihood of not being neurotic, whereas a person classified as not having a psychiatric disorder with 10 as the cut off score has 93% of not having any disorders. There is a 7% chance for misclassification. In epidemiological studies, where the sample sizes are expected to be large, 7% misclassification would lead to a misclassification of a large number of samples. The accuracy of the study would decrease. Therefore, 9 is the best cut off score, with the best discrimination for the Malaysian population

The aim of this study was to validate the Malay version of the Revised Clinical Interview Schedule instrument and obtain a suitable cut-off score for the Malaysian population. Specifically it compares the computerised version of CIS-R with the assessment by psychiatrists using SCID as a guideline.

Our study shows that the Clinical Interview Schedule revised (CIS-R) instrument is suitable for assessing minor psychiatric morbidity in the population. The instrument exhibited 100% sensitivity, 96.15% sensitivity, 3.85% chances of false positive results, a positive predictive value of

0.97 and a negative predictive value of 1.00.The best cut off score for the Malaysian population is 9.00.

The sensitivity value of the CIS-R instrument obtained in this study was very high in comparison to the other studies quoted. It could be explained by the design of our study; the focus was on the ability of the instrument to discriminate between cases and control and produce a diagnosis as diagnosed by the doctors. We did not compare the severity of each individual symptom for both assessments. Our study was carried out in clinical settings and most of our subjects (56%) were a range of patients from ward and clinics exhibiting a range of anxiety and depressive symptoms. There were not many borderline cases. The population participated in the survey is therefore not a proper representative of the Malaysian population. Unlike in the clinical settings, it would be more difficult to identify the positive subjects in the population. In the British study by Brugha et al, comparing the CIS-R with SCAN, the author noted that less frequent conditions exhibited a poorer level of concordance. The level of concordance showed a slight improvement when specific disorders were grouped together into overall ICD-10 categories (1). Lower agreements were obtained when less frequent cases are classified specifically. In the Australian study by Patton et al., on the adolescents, it was highlighted that the low sensitivity rate of CIS-R to CIDI could possibly be due to the different time frame for symptom definition used in the two phases of the study; the CIS-R used a current definition whereas the CIDI generated a lifetime diagnosis (5).

The cut off score obtained in this study was 9. The suitable cut of score obtained for the British population was between 11-12 (2). The findings from the studies in Goa India showed that the suitable cut off score for the Indian population is 15 (10). A study in Tanzania on the prevalence of common mental disorders among those attending primary health care clinics and traditional healers was carried on using 12 as the cut off score (11). A Study in Australia on the cannabis use among the youths took a total score of 12 or greater for CIS-R to define the mixed state of depression and anxiety (12). The cut off score obtained in our study is relatively lower in comparison to other studies.

The cut off point obtained for the Malaysian population was rather small in comparison to other populations. The difference could have occurred due to the fact that our study was conducted in a clinical setting with majority of the respondents being patients and females. This sample was not a proper representation of the Malaysian community. The sample size used for this study is small and thus could be counted as a limitation.

The cut off score obtained in this study would be useful for surveys that intend to trace those who have potential metal health problems. A validation study in the community setting with samples drawn via a proper sampling method should be carried on to obtain a more precise validation coefficients and cut off score for the society. Patton et al suggested that the computerised CIS-R instrument has a practical utility as the first phase screen for adolescence depression (5). Findings from our studies almost suggest the same that the computerised CIS-R instrument is appropriate to be used as the first phase screening for common mental health problems in the Malaysian community.

1. Brugha T.S, Bebbington P.E., Jenkins R., Meltzer H., Taub N.A, Janas M. and Vernon J. Cross validation of a general population survey diagnostic interview: a comparison of CIS-R with SCAN ICD-10 diagnostic categories. Psychological Medicine: 1999; 24: 16-27.
2. Lewis G., Pelosi A.J., Araya R. et al. Measuring psychiatric disorders in the community: a standardised assessment for use by lay interviewers. Psychological Medicine. 1992; 22: 465-486.
3. World Health Organisation 1993. The ICD-10 Classification of Mental and Behavioural Disorders:Diagnostic Criteria for Research. WHO: Geneva.
4. Lewis G. Computerised assessment of psychiatric disorders using PROQSY. Journal of Royal Society of Medicine. 1992; 85: 403-406
5. Patton G.C., Coffey C., Posterino M., Carlin J.B., Wolfe R. and Bowes G. A computerised screening instrument for adolescent depression: population-based validation and application to a two-phase case-control study. Social Psychiatry Psychaiatric Epidemiol. 1999; 34: 166-172
6. Wickrmasighe S.C., Abeysinghe R. and Prince M. The Clinical Interview Schedule-Sinhala Version: Validation in a community setting in Sri Lanka. International Journal of methods in psychiatric research 2002; 11: 169-177.
7. Patel V. and Mann A. Etic and emic criteria for nonpsychotic mental disorders: a study of the CIS-R and case provider assessment in Harare. Social Psychiatry and Psychaitric epidemiology. 1997; 32: 84-89.
8. Patel V., Pereira J., Couthinho L., et al. Poverty, psychological disorder and disability in primary care attenders in Goa India. British Journal of Psychiatry 1998; 171: 60-64.
9. Liu S.I., Prince M., Blizard R. et al. The Prevalence of psychiatric morbidity and its associated factors in general health care in Taiwan. Psychological Medicine. 2002; 32: 629-637.
10. Patel V., Chisholm D., Rabe-Hesketh S., et al. Efficacy and cost-effectiveness of drug and psychological treatments for common mental disorders in general health care in Goa, India : A randomised, controlled trial. The Lancet 2003; 361: 33-39
11. Ngoma M.C., Prince M. and Mann A. Common mental disorders among those attending primary health clinics and traditional healers in Urban Tanzania. British Journal of Psychiatry. 2003; 183: 349-355.
12. Patton G.C., Coffey C., Carlin J.B., Degenhart L., Lynskey M. and Hall W..Cannabis use and mental health in young people: Cohort study. British Medical Journa l. 2002; 325: 7374.

© Copyright 2006 - Malaysian Journal of Medical Science

The following images related to this document are available:

Photo images