|
African Journal of Reproductive Health, Vol. 6, No. 2, August, 2002 pp. 13-22 WHO Indicators for Evaluation of Maternal Health Care Services, Applicability in Least Developed Countries: A Case Study from Eritrea Philip Gottlieb1 and Gunilla Lindmark1 1Department of Women's
and Children's Health, Section for International Maternal and Child Health,
Uppsala University, Uppsala, Sweden. Code Number: rh02017 ABSTRACT The World Health Organization has recommended a number of process indicators to monitor the effect of health care programmes on maternal mortality. This study was therefore conducted to know if the recommended process indicators are useful also in the least developed countries. In 1994, all 17 health facilities offering maternal health care in a rural province in Eritrea were visited. An assessment was made of the obstetric services provided, obstetric complications, and accessibility of health facilities. The study revealed that necessary data were available for most indicators. The indicators were helpful to follow the coverage of obstetric care and to identify problems within the health care system. However, in countries where the coverage of assisted deliveries is low with few obstetric complications seen within the health care system, the indicators cannot be used as a tool to monitor the effect of maternal health care programmes on maternal mortality. (Afr J Reprod Health 2002; 6[2]: 1322) RÉSUMÉ Les indicateurs de l'OMS pour l'évaluation des services de soin de la santé maternelle, l'applicabilité dans les pays les moins développés: une etude de l'Erythrée. L'Organisation mondiale de la santé a préconisé un certain nombre d'indicateurs du processus pour surveiller l'effet des programmes de service de santé sur la mortalité maternelle. Cette etude a été donc menée pour savoir si les indicateurs de processus préconisés sont également utiles dans les pays les moins développés. En 1994, nous avons visité les 17 services de santé qui assuraient le soin maternel dans une province rurale en Erythrée. Nous avons évalué les services obstétriques disponibles, les complications obstétriques et l'accessibilité des services de santé. L'étude a montré que la plupart des indicateurs disposaient des donnés nécessaires. Les indicateurs ont été utiles quand il s'agissait de suivre la couverture de soin obstétrique et d'identifier les problèmes qui se posent au sein du système des services médicaux. Toutefois, dans les pays où le niveau de la couverture des accouchements aidés est bas, et où on remarque peu de complications obstétriques dans le système des services médicaux, les indicateurs ne peuvent pas servir d'outil pour surveiller l'effet des programmes de soins de santé maternelle sur la mortalité maternelle. (Rev Afr Santé Reprod 2002; 6[2]: 1322) KEY WORDS: Maternal health, developing countries, Eritrea, WHO INTRODUCTION The high maternal mortality in the least developed countries is a most disturbing reality that countries and health planners have to face. The high death rate amongst pregnant women in the developing world is caused by a number of socio-economic and cultural factors together with inefficient health facilities. However, studies also show that up to 90% of all maternal deaths are related to the structure and performance of health care services and could be avoided by improving accessibility and quality of services offered.1,2 One way of addressing the problem of maternal deaths has been the development of health care programmes like the Safe Motherhood Initiative (SMI) and the Prevent Maternal Mortality (PMM).3,4 In order to monitor these programmes a group of indicators have been developed related to maternal morbidity and mortality. In 1993, a WHO technical working group assessed and recommended eight of these indicators for implementation.5 Two types of indicators were differentiated impact indicators and process indicators. Impact indicators are related directly to the impact or the ultimate outcome of a given programme, and the two impact indicators recommended were maternal mortality ratio and annual number of maternal deaths. The problem of the impact indicators are that, in the setting found in many developing countries, it may not be possible to retrieve the necessary data, or that it requires costly and tedious surveys. Therefore, the process indicators have been emphasised and can, to some extent, replace the impact indicators as a tool to monitor the effect of health care programmes.6 Process indicators, such as proportion of women delivered by trained staff and proportion of women delivered by caesarean section describe intermediate outputs in the chain of events leading to the ultimate goal. However, in order for a process indicator to be used instead of an impact indicator the causal linkage to the outcome must be clear and scientifically proven. In the mother-baby package in 1994, WHO further developed the use of process indicators, and in 1997 UNICEF, in co-operation with WHO and UNFPA, published detailed recommendations for the collection of data and interpretation of process indicators.6,7 The process indicators recommended are listed in Table 1. The objectives of this study are (1) to assess if the data required for computing the WHO maternal health care indicators are easily available and reliable even in the least developed countries such as Eritrea; (2) to assess if the indicators are a useful tool to monitor the health care services and as such a valuable tool for health management; and (3) to know if the process indicators can be used, under the prevailing conditions, to monitor the effect of health care programmes on the maternal mortality. The indicators assessed in this study are presented in Table 1. The Study Setting Eritrea is one of the youngest countries in Africa. In 1991 she gained her independence after a long liberation war, and the government is now shouldering the immense task of rebuilding the country including the health institutions. The health status of Eritrea population is poor, as indicated by an infant mortality rate (IMR) estimated at 114 per 1000 live births and an under five mortality rate (U5MR) of 195 per 1000 live births.8 In 1995 the Eritrea Demographic Health Survey was carried out by the government, which indicated that the maternal mortality rate (MMR) is very high, 998 per 100,000 live births. In order to address this high MMR a national policy and a national action plan for safe motherhood have been formulated, and the Ministry of Health has produced a National Protocol on Safe Motherhood.9 Theoretically, the structure of the health care services is formed as a pyramid with community-based village health workers and trained birth attendants at the base. The second level is represented by health stations, where a nurse or a health assistant takes care of normal deliveries and provides antenatal care. In the Eritrea setting most of the health stations have from five to fifteen beds. Health centres form the third level. Their staff, nurses and sometimes trained midwifes are expected to do assisted vaginal deliveries, manual exploration and curettage of the uterus. They carry out antenatal care and immunisations at the health facility and as outreach activity in the communities. The top of the pyramid is the provincial hospital. It serves as the first referral level where life-saving operations including caesarean section are performed and blood transfusions are given. The health structure lacked trained staff at all levels when the data were collected in 1994. In the province of Akele Guzai there were 57 health assistants with little formal training in midwifery but often several years of practical experience. Out of 19 midwives only five were registered with full training and 14 of the 20 nurses were registered. Of the four physicians dealing with maternity care, none was an obstetrician, two were stationed in the provincial hospital and two at a NGO health station. Altogether, 77% of the staff in maternity care were trained during the armed struggle with emphasis on practical experience and less emphasis on theoretical training. Auxiliary midwives, therefore, often assisted women who delivered within the health care system. Only 12 of 400 trained TBAs needed were working at the periphery and community health workers were also very few. Women who delivered at home were, therefore, rarely assisted by staff from health facilities or by TBAs who were recognised by the health authorities. The government now emphasises upgrading of staff to registered nurses and midwifes and a nationwide training of TBAs has also started. The province of Akele Guzai, which is located in the centre of Eritrea, was chosen because it was the most densely populated. The population of 440,000 is 12.5% of Eritrea population and the area of 10,500km2 was 8.5% of the area of Eritrea. The province contains highlands with settled farmers and lowlands with pastoral people. Administratively the province is divided into five sub-provinces and 45 districts. METHODS Permission to carry out the study was obtained from the national and provincial health authorities. In 1994, over a period of eight weeks, all 17 existing health facilities where delivery care was offered were visited. All visits were done during working hours without prior notice. Each visit took three to five hours. The language used during interviews was English when possible, otherwise Tigrinya. The study retrospectively used information during a nine-month period, from January 1st to September 30th 1994. The number of people living in the province and in the catchments area of each health facility was obtained from the provincial government using demographic data collected in 1993. Indicators Assessed and Methods used for Data Collection Proportion of women attended at least once during pregnancy by trained personnel for reasons related to the pregnancy This indicator gives information about the level of utilisation of antenatal care by pregnant women. Trained personnel is the staff giving antenatal care, and in the Eritrea setting consists of health assistants, nurses and midwifes. The nominator is the number of first visits seen at antenatal care sessions in the health facilities and during the outreach activities. The denominator is the total number of expected live births calculated from the crude birth rate and the total population. The antenatal care services were uniform over the province. A card, the maternal health card, was filled, a risk screening was done and health education was given in groups. The nominator of the indicator was found from the monthly records, which both governmental and non-governmental clinics reported to the provincial health office. The figures were controlled at the office of statistics in the provincial health office. Proportion of births attended by trained health personnel The indicator gives information about the utilisation of health facilities by women. The nominator is the number of deliveries attended by trained staff during the study period. Trained staff was defined as the staff working at the health facilities, the majority of which were not trained midwives but health assistants and nurses. TBAs were excluded in the definition but deliveries attended in the homes by staff from the clinics were included. The denominator is the total number of expected live births calculated from the crude birth rate and total population. The data were retrieved using the logbooks of the clinics and also from the statistics reported monthly to the provincial health office. Number of facilities providing essential obstetric care per 500,000 population The indicator measures the availability of health facilities offering essential obstetric care (EOC). The WHO working group emphasises that the indicator should focus on functional facilities rather than facilities with the theoretical capacity to offer essential obstetric care.5 The minimum level of services that a health facility should offer in order to qualify as a basic EOC facility is shown in Table 2. A comprehensive EOC facility is also called the facility of 1st referral level and, in addition to the basic EOC services, it should be able to perform life-saving surgery (caesarean section) and give blood transfusion.7 The nominator of the indicator is the number of health facilities that qualify as EOC facilities in a population of 500,000. In each facility, using a checklist, an inventory was done of the availability of antibiotics for parenteral use, anticonvulsants and oxytocin. The availability of equipment for obstetrics was assessed. The knowledge of staff to use the equipment and capacity to perform assisted vaginal delivery was assessed through interviews with the staff and by studying the logbooks. Proportion of complicated obstetric cases managed at EOC health facilities The indicator gives information about the met need to treat obstetric complications and also information about the utilisation and accessibility of health facilities. The denominator is the expected number of complications per year, estimated as 15% of live births. The number of live births is calculated from the crude birth rate and total population. Complicated obstetric cases are potentially life-threatening conditions such as complicated abortion, haemorrhage ante and postpartum, prolonged obstructed labour and rupture of uterus, sepsis, pre-eclampsia and ectopic pregnancy.5 The data on obstetric complications were retrieved in all health facilities from the logbooks and by interviewing the staff using a checklist. Thereafter, the proportion of complications encountered at the EOC facilities could be assessed. Caesarean sections as a percentage of all births in the population The indicator gives information about the availability and utilisation of services. The nominator is the number of caesarean sections performed during the study period. The denominator is the total number of expected live births in the same geographical area during the same period, estimated from the crude birth rate. The number of caesarean sections performed at the provincial hospital during the study period was recorded from the logbook in the operating theatre. Percentage of population within one hour travel time to a health facility offering emergency obstetric care including caesarean section The indicator is defined as the proportion of people in a specific area living within one hour travel time (by whatever means of transport) from the nearest facility giving EOC, including caesarean section and blood transfusion. The indicator gives information about the availability and accessibility of health facilities. The number of villages served by each of the 17 health facilities and the number of people living in the catchments area were obtained from the provincial administration using demographic data collected in 1993. The walking distance expressed in hours from the villages to the health facility was known by the staff. The mode of transport from the village to the nearest health facility was explored by asking the staff. (Out of 100 mothers in labour, how many would come to the clinic on foot, riding a donkey, mule or camel, by vehicle or other means of transport?) The distance in hours from each health facility to the provincial hospital was assessed taking into consideration the availability of transport. For each of the 16 health facilities, the average time to reach the provincial hospital was estimated including distance from village to health facility and from health facility to provincial hospital. RESULTS Proportion of Women Attended at least Once during Pregnancy by Trained Personnel for Reasons related to the Pregnancy The total population in the province in 1994 was 441,909 and the crude birth rate was 46.9 per 1000 according to the Ministry of Health, which gives an expected number of deliveries per year of 20,725 or 15,545 in the nine-month study period. The coverage of antenatal care in the five sub-provinces is indicated in Table 3. For the whole province the coverage was 41%. The data for the indicator were easily available based on monthly reports on first visits. The data found were considered to be reliable because they were based on records filled in at each antenatal care session and reported to the provincial health office every month. The indicator is useful for health care management to evaluate the utilisation of antenatal care, and for this purpose it can be used for national comparison. Proportion of Births Assisted by Trained Health Personnel As indicated in Table 3, 9% of trained staff assisted all deliveries. Table 3 shows the great variation in utilisation of facilities within the province. The necessary data were easily available and considered reliable, as the information was recorded and regularly sent to the provincial health care office. However, for three health facilities a minor discrepancy was found in the number of deliveries reported to the provincial health office and the number found in the study by looking through the logbooks. The indicator can be used by health managers to evaluate the utilisation of health facilities and for national comparison. Availability of EOC Facilities in a Population of 500,000 The number of health facilities that offered maternal and obstetric care were 11 health stations, five health centres and one provincial hospital, totalling 17 facilities. Considering the six obstetric services, which an EOC facility should offer, it was found that two clinics qualified as basic EOC facilities, one health station run by an NGO and one health centre run by the government. The provincial hospital gave blood transfusion and performed caesarean sections, and thus qualified as a comprehensive EOC facility. The equipment for vacuum extraction did not work in the provincial hospital but the staff used obstetric forceps instead. The obstetric services required to qualify as an EOC facility and the number of clinics offering the services are shown in Table 2. The necessary information to evaluate if a health facility would qualify as an EOC facility required a detailed study of each health facility including the equipment and drugs available for obstetrics, the knowledge and routines of the staff in particular concerning management of obstetric complications. Therefore, data for the indicator were not easily available, but since each facility was visited the data were considered reliable. The indicator can be used by health managers to evaluate the performance of the health facilities and the availability of obstetric services; in that way the indicator can be used for national comparison. Proportion of Complicated Obstetric Cases Managed at EOC Health Facilities More than three of the 17 health facilities qualified as EOC facilities. The number of obstetric complications managed at these three health facilities are shown in Table 4, and it is related to the number of complications seen at all 17 health facilities. The number of deliveries that need qualified help is considered to be at least 15% of the total expected number of deliveries, in this case 2,332.5 At the three EOC facilities, 209 cases of obstetric compli- cations were managed (or 9% of the total expected). Obstetric complications were in 1994 not supposed to be registered and reported centrally. Health stations only registered the number of assisted deliveries at home and at the clinic, while health centres and hospitals in addition registered if the deliveries were normal or abnormal. The necessary information for the indicator was found from the logbooks and interviews with staff, relying on their memory and using a checklist. The data were therefore not easily available and not reliable, as there was also doubt about the diagnostic accuracy of the complications. The indicator is, therefore, not easy to use as a routine under the prevailing conditions. Number of Caesarean Sections as a Percentage of All Births in the Population During the nine-month study period, 20 women (or 0.13% of all expected deliveries) were delivered by caesarean section. The study did not seek to find out indications for the operations and no major complications were noted. Necessary information was easily available from the logbook in the operating theatre of the provincial hospital. A source for incorrect data would be the self-referral of mothers to the capital for caesarean section. However, that number was not considered great, as the reputation of the district hospital was good compared to the crowded facilities in the capital. For health managers the indicator is a useful tool to assess the availability and utilisation of health care services and also valuable for national comparison. Percentage of Population Within One Hour Travel Time to a Health Facility Offering Essential Obstetric Care Including Caesarean Section The great majority of women who came to deliver at the nearest health facility came after having tried to deliver at home and they were carried sitting on an armchair. More rarely, they would be riding a donkey or a camel, or transported in a vehicle. The roads leading from the health facilities were passable all year round for 13 of the 16 health facilities. When a mother in labour was transferred she could rely on a car of the facility or from the community in nine of the 16 health facilities. In seven facilities she would have to rely on public transport or be carried to a road and get a lift from there. It was found that 5% of the population had one hour or less to reach the comprehensive EOC facility for the whole province. Sixty per cent of the population had between one and five hours journey and 35% had more than five hours to travel. The necessary information for the indicator required a visit to each of the 17 health facilities and calculations based on demographic data and the knowledge of the staff concerning number of villages and travelling distance from villages to the health facility. Some inaccuracy may exist in this process, but in the present setting it was considered minor. The population estimates in the catchments area were based on data prepared for a consensus in 1993. The indicator was, therefore, considered reliable but not easily available and is a valuable tool for health managers to evaluate the accessibility to health facilities and can be used for national comparison. DISCUSSION In this study, six of the nine process indicators recommended in the mother-baby package are applied to maternal health services in one of the least developed countries in Africa in order to see if the indicators are useful even in the poorest countries. The data used in the present paper were collected eight years ago. However, the question of the applicability of the indicators under those conditions is still valid and the paper does not aim at a comprehensive picture of the present health care facilities in Eritrea. A summary of the findings concerning the usefulness of the six indicators is given in Table 5. When we evaluate the result of the indicators they should be considered together. Two indicators, the third and sixth, give information about the availability and accessibility of health services. The number of EOC facilities per 500,000 was found to be three instead of the recommended five. Only five per cent of the population live within one hour travel time from a facility where blood transfusion or emergency operation can be performed. This was partly because no emergency referral system existed at the village level and the great majority of women had to be carried at least part of the way to reach the appropriate health facility. The other four indicators the first, second, fourth and fifth give information about the utilisation of health services. The coverage of antenatal care was 41% for the province and the coverage of deliveries by trained staff was 9% (Table 3). The very low coverage of antenatal care and assisted deliveries found in the sub-province of Irafaile, where the health facilities served a nomadic population, was explained partly by the lack of infrastructure in the sub-province and lack of outreach activities from the health facilities. Other factors like male staff taking care of pregnant women and lack of staff who could speak the predominant language in the sub-province may be contributing factors. The proportion of obstetric complications seen at EOC facilities was 9%. WHO recommends that all major obstetric complications (or 100%) be treated in EOC facilities, but other authors argue that local targets should be set for the indicator.10 The proportion of women delivered by caesarean section was 0.13%. WHO recommends the level of the indicator to be around 5%, but low levels around 0.5% are also found in other African countries.10 The low coverage of antenatal and delivery care, the low proportion of obstetric complications treated at EOC facilities, and the very low proportion of women delivered by caesarean section were explained partly by the poor availability and accessibility, as shown by the third and sixth indicators. Further studies are needed to explain the poor compliance to utilising the existing health facilities. Three of the indicators, the third, fourth and sixth, involved the concept of EOC facilities. It was thus necessary to evaluate if the health facilities qualified as such. The results are summarised in Tables 2 and 4. The services provided by few facilities were, assisted vaginal delivery, manual exploration of uterus, and curettage of uterus. The complications that were not treated at the appropriate level were, prolonged obstructed labour, haemorrhage and complicated abortions. The two tables show the same picture; the staff needs upgrading to perform assisted vaginal delivery, manual exploration of uterus and curettage of uterus. The indicators are thus helpful to identify specific problems and show which skills need to be upgraded and equipment to be supplied. The common denominator for the first, second, fourth and fifth indicators is based on the crude birth rate and total population figures, which are often not known. The numerical values of the indicators should, therefore, be considered with caution especially when comparing countries. However, providing that no major changes in the population and birth rate take place, the numerical values of the indicators can be used to see if there is a change over time in the same geographical area. Obstetric complications were not registered routinely and problems existed in acknowledging obstetric complications and defining which complications to register. Similar difficulties were also found in studies from India and West Africa.11,12 However, obstetric complications are key events leading to maternal morbidity and death. Health care programmes aimed at reducing maternal mortality should emphasise the registration of maternal complications in order to identify the specific problems and their treatment. Registration of complications would require upgrading of staff to uniformly identify and diagnose the complications. For the provincial health planners, necessary information to calculate the process indicators can to a great extent be obtained through monthly reports. Indicators involving EOC facilities would require a survey to register which facilities qualify as such. The same would be the case for the indicator showing the proportion of people within one hour travel time to an EOC facility. Under the prevailing conditions, with relatively few health facilities in each province, the collection of data for the recommended process indicators would not impose an unrealistic workload on the provincial health authorities. In order to improve the low rates of utilisation, accessibility and availability of resources the government will have to upgrade staff to registered nurses and midwifes, build new health facilities and upgrade some of the existing ones. This is in fact what the Eritrean government has been emphasising since the study took place in 1994. The process indicators studied are a recommended tool to monitor the possible effect of maternal health care programmes on maternal morbidity and mortality. The indicators monitor activities within the health care facilities, and this study shows that they are useful for this purpose even in the least developed countries. But how useful are the process indicators to monitor the effect on maternal mortality when the majority of women deliver outside the health facilities? One can also ask: will a change in any of the indicators depict a change/decrease in maternal mortality? A doubling of caesarean sections performed will increase the rate to 0.26%, which is still very far from the recommended 515% of all deliveries. Similarly, an increase from 9% to 18% of the proportion of obstetric complications seen is still very low compared to the 100% of complications that should be treated at EOC facilities. A change in the numerical value will not necessarily indicate a reduction of the overall maternal mortality rate, as the majority of women still deliver outside the control of the system and the complications with the highest case fatality risk are the least likely to reach the clinics. The proportion of deliveries assisted by trained staff is considered to be 6% in Eritrea13, and for Africa as a whole the proportion of assisted deliveries is in the range of 33%.14 The recommended WHO process indicators are therefore of limited value in many African countries as a tool to describe the effect of maternal health care programmes on maternal mortality. Maternal health care programmes should also focus on ways to influence and monitor health behaviour at the community level where the majority of women in Africa deliver. This is the major challenge of maternal health care programmes in rural Africa today. The PMM network in West Africa developed interventions to facilitate emergency transport at the community level. A loan fund was established from which communities could fund emergency transport. Indicators to monitor this were, for instance, number of women transported for emergency care and number of loan funds established.12 In the Eritrean setting government emphasises the training of TBAs. This may have some impact on identifying obstetric complications but will not address the problem of early transport and treatment of obstetric complications. Research is needed on compliance with advice to deliver at health facilities. Decision-makers at the village level must be sensitised to take responsibility for the pregnant women. A strategy should be worked out on how to recognise warning signs and secure early transport of mothers with obstetric complications. Process indicators suitable to monitor the proposed interventions can then be developed; for instance, proportion of villages where elders have been sensitised, proportion of villages where an emergency transport system has been established and number of women transported for emergency care. The Eritrean government has a clear policy to emphasise preventive health care and the civil administration is well organised with a tradition of community participation. The women's association is organised with representatives at all levels. Since the country is small and distances relatively short, it is feasible to conduct campaigns to sensitise villages concerning the welfare of pregnant women. On the other hand, in several parts of the country women have a low status and interventions to prevent maternal morbidity and mortality may not be easily accepted. ACKNOWLEDGEMENTS The present study was a SIDA (Swedish International Development Co-operation Agency) funded minor field study through the Section for International Child Health, University of Uppsala. The author is grateful to Dr Woldeab Yisak, president of the University of Asmara, for his assistance, and to the Ministry of Health in Eritrea. I am grateful to Dr Berhane Deberu, province health director in Akele Guzai and Mr. Saleh Ahmed Ijaj, governor of Akele Guzai, for their interest and assistance. Mr. Ghebreab Tesfamariam was a very valuable companion when the data was being collected. My warmest thanks go to the staff of all the health facilities who willingly gave me their time. REFERENCES
Copyright 2002 - Women's Health and Action Research Centre The following images related to this document are available:Photo images[rh02017t4.jpg] [rh02017t1.jpg] [rh02017t3.jpg] [rh02017t5.jpg] [rh02017t2.jpg] |
|