|
Journal of Health, Population and Nutrition, Vol. 23, No. 3, Sept, 2005, pp. 282-291 Impact of Training on Assessment of Diarrhoea and Acute Respiratory Infection at Government Health Facilities in Egypt Ray L. Langsten1 , Mahmoud el-Mougi2 , and Robert E. Black3 1 Social Research Center, American University in Cairo, Egypt, 2 Department of Pediatrics,
Faculty of Medicine, Al-Azhar University, Cairo, and 3 Department of International Health,
Johns Hopkins University, Baltimore, MD 21205, USA Code Number: hn05036 ABSTRACT Egypt began training of physicians in case management of diarrhoea and acute respiratory infection (ARI) in the 1980s and 1990s respectively. Data from an independent assessment demonstrate the impact of training in the use of clinical practice guidelines on the quality of clinical examinations of 579 children presenting with diarrhoea or ARI. These examinations were conducted by 115 government physicians in 80 government health facilities and in two Egyptian governorates. The quality of care, although better than that in the past, remains poor, and training had little effect. Poor examinations were related to misclassification and inappropriate treatment of serious illnesses. Such errors have potentially serious consequences. Egypt is now implementing the Integrated Management of Childhood Illness strategy. This effort should be independently assessed to measure the improvement in the quality of care provided by government health facilities in Egypt. Key words: Acute respiratory infections; Diarrhoea; Integrated Management of Childhood Illness;Training; Quality of care; Guidelines; Health facilities; Impact studies; Egypt Introduction Since the 1980s, health projects in Egypt have sought to improve the quality of care provided by physicians at government health facilities to young children suffering from diarrhoea and acute respiratory infection (ARI). The Egypt National Control of Diarrhoeal Diseases Project (NCDDP), begun in 1982 (1), focused on reducing diarrhoea-related mortality by expanding access to, and use of, oral rehydration therapy. Over the course of the 1980s, thousands of government physicians were trained to diagnose and treat children suffering from diarrhoea (2-4). This training continued into the 1990s. The ARI component of the Egypt Child Survival Project (ECSP), begun in 1989, was not fully implemented until the early 1990s. Between 1990 and 1995, almost 13,000 physicians were trained in standard case management of ARI (5). An early assessment of the Project precipitated changes in the training, which were finalized by 1994. Following these changes, a survey in 1994 showed a significant difference in performance between trained and untrained physicians (6). Both diarrhoea and ARI programmes involved training in the use of clinical practice guidelines that laid out the process of care. Such guidelines are meant to improve diagnosis and treatment and decrease child mortality (7). The 3-6-day diarrhoea case-management courses included instruction on: pidemiology, aetiology, and pathophysiology of diarrhoea; case management of diarrhoea; and oral and intravenous rehydration. Practical work included: demonstration of cases, role-play, and supervised clinical practice (8). The 3-4-day ARI case-management training included instruction on: definition and epidemiology of respiratory infection; assessment, classification, and treatment of children with cough or difficult reathing; communication and health education; and control and prevention of respiratory infection. This course also included supervised hands-on clinical practice in the management of children with ARI (6,9). This paper assesses these training efforts by measuring compliance with the guidelines for examining children aged less than five years presenting with diarrhoea or ARI. To the best of our knowledge, it is the first large-scale, independent study of such a national physician-training programme. The results are relevant for the Integrated Management of Childhood Illness (IMCI) programme now being implemented. Materials and Methods Data were obtained from the Two Governorate Linkage Survey, a large community-based, longitudinal health survey of a representative, multi-stage, clustered, systematic random sample of about 3,000 households in each of two Egyptian governorates (for a total sample size of about 6,000 households). Sample size was estimated based on judgements about expected differentials on a number of health indicators. The two governorates - Minya and Qaliubia - were selected arbitrarily by the Ministry of Health to represent 'traditional-Upper' and 'modern-Lower' Egypt respectively. Minya, in Upper Egypt, 200-250 km south of Cairo, is a relatively poor, traditional area. Fertility and mortality remain relatively high, while education is low, particularly among females. Qaliubia, on the other hand, is located immediately to the north of Cairo, in the Nile delta (Lower Egypt). Fertility and mortality are substantially lower in Qaliubia than in Minya. Educational levels are relatively high, with greater equality between males and females. The northernmost, periurban, industrialized suburbs of Cairo are located in this governorate and were included in the sample. During routine data collection in sample households, all mothers of children aged less than five years were asked if their children had had diarrhoea, cough, fever, or rash in the two weeks prior to the date of interview. Mothers of children who suffered from any of these illnesses were asked a number of questions about the treatment they received, including the exact source of care. Mothers of sick children were asked explicitly if their children had been taken to a government health facility for treatment. At least some children in all sample clusters were treated for diarrhoea or cough at the local government health facility. Another part of the research involved a community-level questionnaire. In this interview, key informants in sample areas were asked about the location of the government health facility used by mothers of sick children of that neighbourhood. These health units, along with the outpatient clinics of the first-level referral hospitals that serve these communities, constitute the sample of government health facilities for the quality-of-care component of this study. This sample was determined by the use of health facilities among those living in the sample of households. In 1997, the quality-of-care research collected information about these government health facilities, about the doctors who worked there, and about the quality of care that the doctors and other medical staff delivered. Several sets of questionnaire were used. They recorded: (i) basic characteristics of the government health facility; (ii) supplies and medicines in stock at the time of the interviewer's visit; (iii) observations of the assessment and treatment of children aged less than five years presenting at the government health facility with ARI or diarrhoea; (iv) interviews with the physicians who provided this care; and (v) observations of paediatric vaccinations at the government health facility. This report used data from observations of physicians as they examined and treated children, aged less than five years, who came to a government health facility, suffering from either diarrhoea or ARI (henceforth, referred to as target children) and from the subsequent physician-interview. Well-trained physician-observers (henceforth, referred to as observers) carried out the observations and physician-interview. All but one of the observers had completed their basic medical training at least five years prior to the survey. They all had advanced medical degrees and previous special training on case management of diarrhoea and ARI. At the time of the survey, all were working either in government or university clinics and in private practice. Two groups of observers - one in each governorate - were used. All observers were local residents and were affiliated with the local university. For the purposes of this research, the observers received two weeks of additional training using the tools of the Control of Diarrhoeal Diseases and ARI programmes. The training focused on proper assessment, classification, and management of diarrhoea and ARI and on use of the questionnaire and included hands-on clinical practice. Each team of observers was supervised by a senior paediatrician from the Faculty of Medicine of a local university and was assisted by a trained interviewing supervisor who reviewed the completed questionnaire for completeness and accuracy. The observations were carried out in the following manner: The observer introduced himself (all the observers were male) to the staff of the government health facility and specifically to its physician (henceforth, referred to as physician) responsible for care of children aged less than five years. The observer explained that he would watch and record information about the examination and treatment of children and would conduct his own examination of each child. Following observations and re-examinations of five cases, he would inter-view the physician. While the physician assessed, classified, and prescribed treatment for target children and conducted whatever counselling s/he felt appropriate, the observer took notes on the back of the questionnaire and, when necessary, asked questions about what the physician was doing to or noticing about the child. When the physician had completed her/his work with the child, the observer conducted his own assessment, classification, and treatment in a separate room and separately counselled the mother, all removed from the observation of the physician. All notes were transcribed into the structured questionnaire as soon as possible after completing the observation of a case. The Egyptian Ministry of Health and the ethical review committees at American University in Cairo and Johns Hopkins University approved the research. Both physicians and mothers of target children gave free and informed verbal consent to participate in the study. It is also evident from the above description that the physician of the government health facility was fully aware of the presence and work of the observer. Thus, the results reported here include any 'observer effect' resulting from this research (10-13). In the Egyptian health facilities, a trained physician is the 'first-level health worker' (11). Nurses and other healthcare workers play a relatively minor supporting role in curative care in the government health facilities. In most government health facilities, just one physician handles all responsibilities. In such cases, this physician was observed. In some large, urban government health facilities and especially in the first-level referral hospitals, there may be several physicians and, in some cases, a number of them deal with children aged less than five years. In these cases, up to two physicians were observed. The observers were instructed to collect data on the two physicians who had primary responsibility for treatment of children aged less than five years. If there were more than two such physicians, an arbitrary selection was made of the two to be observed. In general, each physician was observed assessing five children. (In some cases, adjustments in the number of cases per physician were made in response to scheduling concerns). After completing the observations, the physician-interview collected information on the physician's background (age, when completed her/his medical education, etc.), whether and when s/he had received special training from the ECSP or the NCDDP on management of ARI and diarrhoea respectively, and on a number of areas of the physician's knowledge about classification and treatment of diarrhoea, and particularly ARI. As in other countries (14,15), about two-thirds (62% diarrhoea; 70% ARI) of sick children in Egypt who receive professional healthcare are treated by private physicians. Nevertheless, this study was conducted in 80 government health facilities because the physicians working in these facilities were the target of the diarrhoea and ARI training programmes. In the government health facilities, assessment and treatment of 579 target children aged less than five years were observed, and 115 physicians were interviewed. The unit of observation for this study is the physician. To take full advantage of all the data available, we combined information from all the observations made by each physician. For example, consider the question: "Does the physician ask about convulsions?" If the particular physician examined five children, we summed the number of 'yes' responses to this question for these five children: Say, s/he asked about convulsions for four of the five children. The proportion of 'yes' answers was computed: in this case, the proportion is 0.80. If the physician examined just four children and asked the mothers of only two of them about convulsions, the proportion of 'yes' responses would be 0.50. We, therefore, created a new dataset with one record for each of the 115 physicians in the study. The nature of the variables has changed from "Did the physician do X for this child?" to "For what proportion of the children examined did the physician do X?" The principal goal of this study was to assess the impact of in-service training programmes on compliance with the case-management guidelines. When considering general danger signs and issues relating to cough, ARI case-management training was assessed. When looking at the examination and treatment of children with diarrhoea, we controlled for training by the NCDDP. The training variable was based on the response of physician to questions about whether s/he has been trained or not, and, if trained, the timing of the training. In both the cases, the training status variable we used had three categories: (i) no training, (ii) trained in 1993 or earlier, (iii) trained in 1994 or later. The period 1993/1994 was used for splitting the trained physicians because by this period the ECSP ARI case-management training programme had been subjected to testing and revision, and it has been said that training from this period on was more effective than training given earlier (6). We used this same period for dividing the physicians trained by the diarrhoeal disease-control programme because this turned out to be convenient for splitting the group of physicians who treated children suffering from diarrhoea more or less equally. Results were also shown for year of completion of medical education - 1991 or before versus 1992 and later. Data were analyzed using SPSS. Most results presented are mean proportions of children assessed for a given sign or symptom, controlling for training status or timing of completion of medical education. One-way ANOVA tests were used for determining the significance of differences for each sign or symptom individually. We have also described the potential consequences of failure to comply with the case-management guide-lines. We have done this by examining classification and treatment of severe disease. Since severe disease was relatively rare, this analysis was based on individual children and the characteristics of their examining physicians. Comparisons were made between the physicians' and the observers' diagnoses of very severe illness, severe pneumonia, severe dehydration, and severe malnutrition. (In a number of cases, these diagnoses overlapped). Whether an antibiotic was prescribed and the severely-ill child was referred to hospital were also considered. ResultsIn total, 115 physicians were observed as they treated 579 target children. Of these children, only 3.3% were aged less than two months, 42.8% were aged 2-11 months, 23.0% were aged 12-23 months, and 30.9% were aged 2-5 years. Of the 115 physicians, 113 had complete data for analysis of general danger signs. For analysis of cough and diarrhoea, we included only those physicians who had treated two or more children suffering from the respective illnesses. Thus, 110 physicians had information for analysis of cough and background indicators, while 79 had complete data for analysis of diarrhoea. The impact of the training on physician compliance with the guideline procedures is shown in Table 1. In this table, training status is the independent variable. One-way ANOVA was used for testing the significance of the effects of the training on each of the 13 signs and symptoms included as the dependent variables. Physicians trained in 1994 or later were more likely to ask/ assess 12 of these items. But the difference was statistically significant for only one sign (checking for sunken eyes). In the case of eight signs and symptoms, the increment in the mean proportion of children examined by recently-trained physicians, compared to those who are untrained, was small (less than 0.10). On the other hand, physicians trained in 1993 or before were generally the least likely to assess signs and symptoms: for 11 of the 13 indicators studied, these physicians assessed, on average, the lowest proportion of children they examined, fewer even than the untrained physicians. Beyond the question of training, however, these data demonstrate the general poor performance of the large majority of the physicians. This can be seen in columns 4 and 5 of Table 1. Column 4 shows the overall mean proportion of children examined. For five of the 13 items assessed, including all the general danger signs, the physicians checked, on average, fewer than one in five children. For only four of the 13 items did the physicians check, on average, half of the children or more. Column 5 shows the percentage of physicians who assessed each sign for no children they examined. For two of the indicators, 90% or more of the physicians checked no children, while for nine indicator 50% or more of the physicians checked no children. The results using timing of graduation as the independent variable were similar. In general, recent graduates (medical education completed in 1992 or later) are less likely to be trained: 28% had received training on ARI and 26% on diarrhoea compared to 83% and 70% of those who graduated in 1991 or before. These recent graduates may have had up-to-date case-management instruction in medical school. Whatever the reason is, recent graduates are more likely to assess each sign and symptom than those who completed their medical education before 1992 (Table 2). The differences were generally small and were statistically significant for just two indicators. Poor assessment of sick children has potentially serious consequences. An examination of cases of seriously-ill children illustrates the dangers. Twenty-eight cases of very severe disease were identified - four by the physicians and 24 by the observers. The two groups did not agree on any cases. For the four cases identified by the physicians, the observers diagnosed less serious respiratory diseases. Three of the four physicians who examined these children were trained. Yet, they asked just one mother whether her child had difficulty in drinking or eating, and no mother was asked if her child was unusually sleepy/difficult to wake. Chest indrawing was checked for one child, and the respiration rate counted for three children. Neither the physicians nor the observers referred any of these four children to hospital. The physicians prescribed antibiotics for three children and the observers for two.Twenty-two of the 24 children, diagnosed with very severe disease by the observers, received an immediate dose of an antibiotic and were referred to hospital - the recommended treatment for this illness. The physicians diagnosed less serious respiratory illnesses, mainly pneumonia, among the children. They referred only three children to hospital and prescribed no antibiotics for nine. Trained physicians examined 18 of these 24 children. The picture is similar for 19 cases of severe pneumonia diagnosed - seven by the physicians and 14 by the observers, two of which overlapped. The key indicators for pneumonia are the respiration rate and chest indrawing. In the two cases where the diagnoses agreed, the physicians checked both the indicators. In 17 cases of disagreement on diagnosis, the physicians checked neither of the key indicators for 10 children. Of 12 children diagnosed with severe pneumonia by the observers, but with less severe respiratory infections by the physicians, eight were referred to hospital and were prescribed antibiotics by the observers. In contrast, the physicians referred just one child to hospital, although all but two were prescribed antibiotics. Eight of 12 physicians who examined and treated the children had received training in case management of ARI from the ECSP. Eight of these 12 physicians completed their medical education in 1991 or before and six of them before 1985. The observers identified 29 children as being severely malnourished. The training of the physicians did not include instructions to weigh children, although scales were available in all the government health facilities. In the event, the physicians weighed just three children. The physicians referred five children to hospital, including two who were referred for nutrition-related causes, although the physician did not weigh these two children. The observers, on the other hand, referred 15 children to hospital. Fourteen children, diagnosed with very severe disease by the observers, were also severely malnourished. Twelve of them were referred to hospital. The physicians, on the other hand, referred just three children to hospital. Trained physicians examined 10 of these 14 children. Finally, the observers and physicians agreed on the three severely-dehydrated children identified. Both ob-servers and physicians referred them to hospital. Two physicians involved were recent graduates, and the other had recently received training in case management of diarrhoea. Sixty-nine percent of all the examinations were estimated to take less than five minutes, and only 4% took 10 minutes or more. DiscussionThe study assessed the work of 115 physicians in 80 government health facilities in two Egyptian governorates, focusing on the process of examining sick children. Process measures, which have a long history (10,16,17), are good indicators of the quality of care when there is an accepted link between process and outcome (18) and when the goal is to influence the behaviour of the health-care system (19). For example, many children present at health facilities with cough and other symptoms of respiratory infection. Only a few of these children have serious, life-threatening, illness. The World Health Organization believes that the ARI case-management procedures can help correctly identify and treat those few seriously-ill children. This, in turn, "can greatly reduce deaths in children" (20). The specific steps laid out in the well-tested diarrhoea and ARI case-management guidelines provide a reliable standard against which process measures can be interpreted (21). The results in Tables 1 and 2 show poor compliance with the examination guidelines. Some say it is not necessary to follow guidelines precisely. For example, some believe that an experienced physician can assess rapid breathing without counting breaths per minute for each child. In our data, non-compliance with the guide-lines was strongly associated with incorrect diagnoses and inadequate treatment for seriously-ill children. Description of inconsistencies between the diagnoses of physicians and observers in the Results section shows that the physicians missed many cases of serious illness. Perhaps, the results for severe pneumonia best illustrate the problem. Key signs that allow physicians to identify the few very sick children from the many with less severe respiratory infections - rapid breathing and chest indrawing - have been carefully tested and are well-known (22-27). The observers in our study could readily see whether a physician counted the respiration rate and checked chest indrawing or not. Experienced, trained physicians of the government health facilities frequently failed to check these signs and classified seriously-ill children as having simple pneumonia, or even less severe respiratory infection. Although the percentage of all cases of respiratory infection found by the observers to have been under-classified as milder disease by the physicians is small, these cases are particularly important. Under-classification often results in inappropriate treatment. We have shown that many children with severe pneumonia, who required hospitalization, were not referred to hospital by the physicians, and some were not even prescribed antibiotics. In one case, a 23-month old girl, determined on examination by the observer to have a respiration rate of 56 breaths per minute and chest indrawing, would have been sent home with no antibiotics by a trained physician with more than 15 years of medical practice, who failed to check either of the key signs. Results of other studies also showed that severe and very severe illnesses were disproportionately under-diagnosed and inappropriately treated by trained health workers (12,13,28,29). Training, if well-done and followed up, can change practices (30). Why then did the diarrhoea and ARI training programmes in Egypt fail to improve the examination procedures? Perhaps, in part, because the training was not followed up. Both the programmes involved brief (3-6 days) training programmes, and the physicians were then sent back to their clinics without follow-on training, or supervision. Other studies in developing coun-tries observed that similar training programmes have had little impact on behaviour (28,31-33). Training primarily increases knowledge, but some argue that greater knowledge is not necessarily reflected in improved practice (31,34,35). Two major reviews of clinical practice guidelines (36,37) and other studies (38) have further emphasized that external barriers, such as administrative or economic problems, can impede compliance. And, in the end, guidelines may be ignored simply because there are neither rewards for compliance nor penalties for failure to comply (39). Another factor may contribute to poor diagnosis and treatment. The physicians in our study spent very little time with each child. An examination that complies with the IMCI guidelines requires at least 10-15 minutes for an experienced physician (30). No careful examination could be completed in lesser time. Yet, 69% of observed examinations took less than five minutes, and most (96%) took less than 10 minutes. Despite the lack of compliance with guidelines, these results do reflect the improved quality of clinical examinations since the early 1990s. At that time, a small study in a single district of Upper Egypt (11) observed that "no child was questioned on ability to drink or experience of seizures" and "no physician checked for stridor, wheeze or chest indrawing, or counted the respiratory rate." By comparison, in the present study, even the untrained physicians, on average, asked about ability to drink of 9% of the children examined, checked chest indrawing for 23%, counted the respiration rate for 31%, and listened to respiration sounds for 99%. This improvement may result from other positive contributions of the ECSP and NCDDP programmes: 69 (86%) of the 80 health facilities had wall-posters and 75 (94%) had instruction manuals about treatment of diarrhoea. Although only 28 (35%) had wall-posters showing correct case management of ARI, 68 (85%) had ARI instruction manuals available. Thus, information to guide the physicians through proper assessment and case management was available in most clinics. It is also possible that some untrained physicians have learnt from their trained colleagues (40). There are some weaknesses in our data. The training status of the physician was based on questions asking the physician if s/he had received special training in case management of ARI, or in the treatment of children with diarrhoea, and about the date of her/his training. The need to rely on a self-report to simple questions about training is a potential problem. We have no independent source to verify the accuracy of the reports. If there is an error in reported training status, the effect of the training variable will be reduced and, thus, the widespread lack of a training effect may result from error. At the same time, the information has been provided by presumably well-educated physicians, and the training courses are well-known. Thus, we assume that most reports are accurate. It is also sometimes difficult for an observer to know what a physician is checking for. For example, in looking at the child, did the physician note whether her/his eyes were sunken, or not. In this situation, however, the observer was instructed to ask the physician to describe her/ his actions: what was s/he noting when s/he looked at the child? In general, we should know if the physician checked for sunken eyes, and other similar signs, or not. Finally, our observations did not note some circumstances that may have eliminated the need for the physicians to ask certain questions, or check certain signs. For example, if the sick child was obviously awake and alert, it would not be necessary to ask if the child was "always sleeping and difficult to wake". In this situation, and for one or two other sign(s) where it would have been helpful, we failed to include an indicator to show that the needed information was evident to any observer. This problem does not pertain, however, in the diagnosis and treatment of severe pneumonia. We are confident that our results correctly reflect that the physicians checked neither the respiration rate nor chest indrawing for seven of the 12 cases of severe pneumonia they under-diagnosed. The results of the study showed that most physicians, whether trained or not, whether experienced or new graduates, failed to comply with the case-management guidelines of the diarrhoea and ARI treatment programmes. We believe that this failure played a major role in the incorrect diagnoses and inappropriate treatment of many seriously-ill children. These errors, for these children, put lives at risk. Egypt adopted the IMCI strategy in 1997 and developed a national plan of action (41). As of late 2004, the programme had spread to 16 of 26 Egyptian governorates, although even in these programme governorates many districts and many more health facilities remain un-affected. The programme is slowly expanding into the remainder of the country (42). The IMCI strategy promotes appropriate management of sick children by health workers at first-level outpatient health facilities (43). Health workers are trained to assess children using only specific "simple clinical signs, without laboratory tests" (43) and can fully treat a child by following "detailed clinical guidelines that specify a defined process for how the provider should deal with common health problems" (30,44). In Egypt, the IMCI strategy integrates standard case-management guidelines from the vertical diarrhoea and ARI programmes (42). The IMCI strategy, however, also includes changes to the health system, follow-up of physician-training, and improved community and carer practices (30,42,45,46) that may make the training more effective. A health facility survey in 2002 measured the quality of care pro-vided by trained IMCI physicians working in 50 health facilities designated as part of the IMCI programme (13). The IMCI health facilities were provided with equipment, supplies, and essential drugs and were organized for "smooth patient flow, a functional referral system, a supportive supervisory system, and a trusted reliable information system" (42). Even considering the possibility of observer-bias, this survey found much better compliance with guidelines for assessment tasks than we found in our earlier research. All signs and symptoms were assessed for between 75% and 100% of presenting children. Despite improved assessment, three of the six cases of severe illness requiring urgent referral or admission to hospital were under-diagnosed by the physicians who appropriately managed only one of the three correctly-diagnosed cases (13). Similarly, early assessment of the ARI case-management training found very positive results (6) that were not confirmed by the independent work reported here. We believe that an independent assessment of the IMCI programme is needed to more fully examine the improvements in the government health services in Egypt and the quality of care now provided. Future work should in-clude a large number of cases of severe illness, and diagnosis and management of these cases should receive special attention. These are the cases that are most often mishandled and are also the cases it is most important to identify and manage correctly. Under-diagnosis and inappropriate treatment of these cases put children at risk of death. AcknowledgementsStudy design and data collection were supported by the U.S. Agency for International Development (USAID) through the Johns Hopkins Family Health and Child Survival Cooperative Agreement with USAID and through a sub-contract from the Clark Atlanta Child Survival Project to the Department of International Health of the Johns Hopkins School of Public Health. The authors thank Dr. Magdi Moustafa Kamel for helping train and coordinate the research team in Minya. There are no conflicts of interest. References
© 2005 ICDDR,B: Centre for Health and Population Research The following images related to this document are available:Photo images[hn05036t1.jpg] [hn05036t2.jpg] |
|