Can Community Health Workers Report Accurately on Births and Deaths? Results of Field Assessments in Ethiopia, Malawi and Mali
Can Community Health Workers Report Accurately on Births and Deaths? Results of Field Assessments in Ethiopia, Malawi and Mali
Romesh Silva 0 1
Agbessi Amouzou 0 1
Melinda Munos 0 1
Andrew Marsh 0 1
Elizabeth Hazel 0 1
Cesar Victora 0 1
Robert Black 0 1
Jennifer Bryce 0 1
RMM Working Group 0 1
Membership of the RMM Working group can be found in the Acknowledgments. 0 1
0 1 Institute for International Programs, Johns Hopkins Bloomberg School of Public Health , Baltimore , Maryland, United States of America, 2 Economic and Social Commission for Western Asia, United Nations, Beirut, Lebanon, 3 Division of Data, Research and Policy, UNICEF, New York, New York United States of America, 4 Universidade Federal de Pelotas , Pelotas , Brazil
1 Editor: David Joseph Diemert, The George Washington University School of Medicine and Health Sciences , UNITED STATES
Most low-income countries lack complete and accurate vital registration systems. As a result, measures of under-five mortality rates rely mostly on household surveys. In collaboration with partners in Ethiopia, Ghana, Malawi, and Mali, we assessed the completeness and accuracy of reporting of births and deaths by community-based health workers, and the accuracy of annualized under-five mortality rate estimates derived from these data. Here we report on results from Ethiopia, Malawi and Mali.
Funding: This study was funded by a grant from the
Canadian Department of Foreign Affairs, Trade and
Development under the Catalytic Initiative to Save a
Million Lives to the Institute of International Programs,
Johns Hopkins Bloomberg School of Public Health. In
Malawi and Ethiopia, the RMM work benefitted from
the joint presence of broader evaluations supported
by the Bill & Melinda Gates Foundation (BMGF).
In all three countries, community health workers (CHWs) were trained, equipped and
supported to report pregnancies, births and deaths within defined geographic areas over a
period of at least fifteen months. In-country institutions collected these data every month. At
each study site, we administered a full birth history (FBH) or full pregnancy history (FPH), to
women of reproductive age via a census of households in Mali and via household surveys
in Ethiopia and Malawi. Using these FBHs/FPHs as a validation data source, we assessed
the completeness of the counts of births and deaths and the accuracy of under-five, infant,
and neonatal mortality rates from the community-based method against the retrospective
FBH/FPH for rolling twelve-month periods. For each method we calculated total cost, average annual cost per 1,000 population, and average cost per vital event reported.
On average, CHWs submitted monthly vital event reports for over 95 percent of catchment areas in Ethiopia and Malawi, and for 100 percent of catchment areas in Mali. The
UNICEF Ethiopia also funded evaluation activities
that were complementary to the RMM project in that
country. The funders had no role in study design,
data collection and analysis, decision to publish, or
preparation of the manuscript.
Competing Interests: The authors have declared
that no competing interests exist.
completeness of vital events reporting by CHWs varied: we estimated that 30%-90% of
annualized expected births (i.e. the number of births estimated using a FPH) were
documented by CHWs and 22%-91% of annualized expected under-five deaths were
documented by CHWs. Resulting annualized under-five mortality rates based on the CHW vital
events reporting were, on average, under-estimated by 28% in Ethiopia, 32% in Malawi,
and 9% in Mali relative to comparable FPHs. Costs per vital event reported ranged from $21
in Malawi to $149 in Mali.
Our findings in Mali suggest that CHWs can collect complete and high-quality vital events
data useful for monitoring annual changes in under-five mortality rates. Both the supervision
of CHWs in Mali and the rigor of the associated field-based data quality checks were of a
high standard, and the size of the pilot area in Mali was small (comprising of approximately
53,205 residents in 4,200 households). Hence, there are remaining questions about
whether this level of vital events reporting completeness and data quality could be
maintained if the approach was implemented at scale. Our experience in Malawi and Ethiopia
suggests that, in some settings, establishing and maintaining the completeness and quality
of vital events reporting by CHWs over time is challenging. In this sense, our evaluation in
Mali falls closer to that of an efficacy study, whereas our evaluations in Ethiopia and Malawi are more akin to an effectiveness study. Our overall findings suggest that no one-size-fitsall approach will be successful in guaranteeing complete and accurate reporting of vital events by CHWs.
Work to develop plans for monitoring public health and the new Sustainable Development
Goals is at the top of the international public health agenda [
]. As a part of these efforts,
there are increasing demands for measurement of short-term changes in mortality among
children less than five years of age in low- and middle-income countries. In most of these
countries, health information systems are weak, and Ministries of Health and their partners must
rely on household surveys as the primary source of data on vital events and rates . These
surveys are conducted only every three to five years in most countries, and produce rates of
under-five mortality that are three or more years out of date [
The routine collection of vital events data is difficult in rural areas of low-income countries.
Most births and deaths do not occur in health facilities, and thus reliance on facility-based vital
events data often results in systematic under-estimation of under-five mortality rates [
passive nature of traditional vital registration systems in rural settings of low-income countries,
whereby individuals must travel long distances to register an event in areas where
transportation and communication infrastructure is often rudimentary, has also led to problems in the
timing, accuracy and relevance of vital statistics constructed from civil registration systems [
Recently, the United Nations Statistics Division (UNSD) and World Health Organization
(WHO) reported that in sub-Saharan Africa, only 7% of births and 6% of deaths are registered
by official civil registration systems [
]–thus such birth and death registration systems are not
presently viable systems for the monitoring of short-term changes in child mortality rates.
The importance of ‘real-time’ mortality measurement to advance public health has a long
and rich history, dating back to the times of John Graunt in 16th century London [
]. In recent
2 / 15
years, advocates have promoted two principal approaches to improve and scale up civil
registration and vital statistics (CRVS) systems in low-income countries: the development of sample
registration systems (SRSs); and the scaling-up of health and demographic surveillance systems
(HDSSs). Sample registration involves the routine collection of vital events in a geographically
representative sample of the national population. SRSs are currently the primary source of vital
statistics in India [
] and China [
], and have been introduced in Indonesia [
], Zambia, and Vietnam [
]. HDSS sites have remained small, and are not generally
designed to improve CRVS at country level. However, Ye (2012) has advocated for the
introduction of multiple, strategically-located HDSSs throughout a given country as an interim
measure to monitor mortality over time [
]. Mercer et al. (2014) have substantially expanded
this idea by proposing small-area estimation and spatio-temporal smoothing approaches that
integrate HDSS data with available sample surveys to produce estimates of child mortality over
]. Both of these approaches incorporate active, routine collection of data on vital
events, rather than relying on passive systems of vital events reporting by the local population.
However, a notable shortcoming of these approaches is that both the development of SRSs and
the expansion HDSSs require the setting up of new, parallel, data systems that are separate
from existing health systems and the formal CRVS.
Recently in a number of low-income countries, including Ethiopia, Malawi, India, and
Pakistan, the role of CHWs has been substantially expanded from provision of community
awareness and disease prevention to include provision of safe delivery and integrated community
case management of preventable diseases with the highest child mortality burdens (such as
malaria, pneumonia, malnutrition, and diarrhoea) [
]. Thus CHWs create a bridge
between providers of health, social and community services and communities that may have
difficulty in accessing these services . Further, in settings such as Malawi, local
communities have reported favorable impressions of community-based health workers [
]. Hence, in
theory, CHWs are well-placed to proactively track pregnancies as well as document births and
childhood deaths, given that a core focus for CHWs in many low-income countries is maternal,
newborn and child health. In that vein, this study explores the viability of CHW-based
reporting of births and childhood deaths as an interim measure to advance real-time measurement of
under-five mortality and scale-up of CRVS in low-income countries.
In this paper, we synthesize findings from a multi-country study that developed,
implemented and evaluated community-based methods to measure changes in under-five mortality
for recent periods of 12 months or less at small-scale. This study is part of a broader effort to
advance the measurement and accountability agenda for maternal and child health by testing
and developing new methods for estimating under-five mortality in “real-time”, i.e., for recent
periods of 12 months [
]. We studied the completeness and accuracy of vital events reporting
by CHWs in Ethiopia, Ghana, Malawi, and Mali. In particular, we assessed the accuracy of
annualized child mortality rates derived from vital events reporting by these community-based
health workers by comparing them to annualized child mortality rates derived from full
pregnancy or full birth history data collected via household surveys or censuses, and assessed the
running costs of implementing each method. Reporting from Ghana has been delayed due to
concerns about data quality, and will be added to the Collection at a later date if these concerns
are able to be addressed.
Ethical clearance for the project was obtained in the United States from the Johns Hopkins
Bloomberg School of Public Health (JHSPH)'s Institutional Review Board, in Ethiopia from
3 / 15
the Oromia Regional Health Bureau, in Mali from the Ethical Review Committee of the
University of Bamako, and in Malawi from the National Health Sciences Research Committee. For
each household survey, oral informed consent was obtained from each participant. Consent
forms were translated into local languages: Amharic in Ethiopia, Chichewa in Malawi, and
Bambara and French in Mali. The IRB at JHSPH waived the need for written consent from the
study participants given the low literacy of the population under study. Approval letters are
available upon request.
Setting and Selection of ‘Real-time’ Monitoring of Under-five Mortality
We selected the countries that would be invited to participate in the RMM project in March
2008. These countries included Ethiopia, Malawi, and Mali; further details on the selection
process are available elsewhere [
]. Working with the Ministry of Health in each country, we
selected partner institutions. Our in country research partners were Miz Hasab Research
Center and the Alliance for Better Health Services PLC (ABH) in Ethiopia, the National Statistical
Office of Malawi, and CREDOS in Mali.
Ethiopia, Malawi and Mali are low-income countries in sub-Saharan Africa and are amongst
the countries with the poorest health indicators in the world. As shown in Table 1, we
purposefully chose small areas with relatively high under five mortality rates and high total fertility rates.
We conducted cross-sectional qualitative assessments in each study location prior to launching
RMM activities, to understand current practices for recording vital events, actors involved in
the recording of vital events, and barriers and local attitudes towards recording vital events
data. The methods and results of this research are described in related articles [
22, 23, 24
Selection of cadre of community-based workers
We used the results of the formative research in each site as a basis for selecting the community
health worker cadre responsible for RMM reporting. Table 2 summarizes the characteristics of
CHWs across our three study sites. In all three of these settings, vital events reporting was part
of the formal job description of the respective CHWs, although they had other health-related
responsibilities as well. In Ethiopia, we worked exclusively with female CHWs who are paid
government workers with an average of 10 years of formal schooling and resident in their
catchment area. In Malawi, we worked with approximately equal numbers of male and female
CHWs who were each paid health workers with approximately 10 years of schooling, but
sometimes not a resident in their catchment area. In Mali, we worked with community
volunteers who were resident in their local community and had been nominated by it. More than
70% of these volunteers were female and had no formal education.
Size of Districts/
4.4 million (2007 census)
657,075 (2008 census–
both districts combined)
Size of RMM Study Area
568,993 (2009 census)
32,128 (RMM census)
Under-five Mortality Rate
122 (Oromia region, 2005
160 (Balaka) 144 (Salima)
(district estimates from 2006
262 (regional estimate from
Total Fertility Rate (TFR)
6.2 (Oromia region, 2005 DHS)
6.3 (Balaka) 7.1 (Salima)
(district estimates from the
7.1 (regional estimate from
4 / 15
In all three study areas, supervision of the vital events reporting work performed by CHWs
was undertaken on a monthly basis and CHWs were provided with monthly transportation
allowances and cell phone airtime. In Malawi, we undertook a midterm evaluation of the RMM
community-based vital events reporting by CHWs. Based on the results of this assessment, we
instituted more regular data review meetings, introduced Short Message Service (SMS)
reminder and guidance messages, and launched a quarterly award for the best performing
CHW. These post-assessment incentives and their associated effects in Malawi are discussed in
detail in Joos et al. (2015) [
Implementation of community-based vital events reporting
The RMM team in each country prepared for implementation by developing standard
procedures for catchment-area mapping, recording and reporting of vital events and consistent
approaches to supervising their application. The team trained community-based workers and
5 / 15
their supervisors, and developed standard operating procedures for data reporting and
cleaning. We used a trial period of three months to finalize these procedures in each country site.
Each method was implemented for a period of at least fifteen months, with careful
documentation of processes and costs. Intermediate assessments of the procedures and data quality were
conducted at least once in each setting, and used as a basis to reinforce performance and operations.
Validation data. We used current best practices for child mortality measurement in
countries without functioning reliable vital registration system, consisting of household surveys
with full birth or full pregnancy history from women of reproductive age. During full birth
history interviews, women of reproductive health are asked to list all their children ever born with
date of birth and survival status, including age at death for children who have died. For
pregnancy history interviews, women are asked about all pregnancies they ever had. These
approaches are used to estimate child mortality retrospectively for periods of up to 25 years
preceding the survey [
]. In our RMM study areas in Mali and Malawi, we carried out endline
data collection using a full pregnancy history instrument in all households in Mali and in a
random sample of 10,000 households in Malawi [
]. Each of the 160 CHW catchment areas
in Malawi was survey with a sample size of households proportional to the total number of
households in the catchment areas. In Ethiopia, the endline validation survey included a full
birth history of 28,000 households, building on an existing data collection design for an
evaluation study of a child health strategy in the same RMM areas [
]. The survey in Ethiopia
used two stage cluster sampling in the two zones of the RMM study area. The first stage of
sampling involved census enumeration areas that were sampled probability proportional to size.
Households to be interviewed were then sampled at a second stage using a full listing of all
households in each sampled enumeration area. The quality of all the validation data was
assessed using standard demographic data quality assessment methods [
22, 23, 24
We used an array of standard metrics to assess each of the community-based RMM methods
across the country settings. Process metrics focus on the completeness of reporting of vital
events, obtained by comparing the numbers of births and deaths reported through the RMM
method to those estimated from the best practice survey or census. Data quality metrics are
those commonly used in demographic analysis, and include the sex ratio at birth, the ratio of
neonatal to infant deaths, the ratio of infant to under-five deaths, and the distributions of
under-five deaths by age in months and neonatal deaths by age in days. Accuracy metrics
include the concordance between the neonatal, infant and under-five mortality rates produced
by the RMM method and the current best practice method. In comparisons of accuracy across
country settings, we use average annual ratios of under-5, infant, and neonatal mortality rates.
We annualized these ratios to improve comparability across the three country sites.
Throughout the analysis we explicitly consider both the performance of the RMM method and possible
shortcomings in the current best practice method, defined as full pregnancy or full birth
histories collected via household surveys or censuses.
Data on births and under-five deaths reported by CHWs for the periods of January 2012 to
March 2013 in Ethiopia, January 2010 to December 2013 in Malawi, and July 2012 to
September 2013 in Mali were included in the validation analysis, as shown in Fig 1. We analyzed the
data for rolling 12-month periods, with starting periods differing by exactly three months.
Data analysis was conducted in R [
We calculated the number of births and neonatal, infant, and under-five deaths reported by
CHWs for each 12-month period. We did not adjust the CHW vital events data for missing
monthly reports, given that there were only a small number of missing reports in the Ethiopia
6 / 15
Fig 1. Implementation Timeline of RMM Community-based Method by Country Setting.
and Malawi study sites, and none in Mali. We calculated neonatal, infant and under-five
mortality rates for each period by dividing the number of these deaths documented by CHWs in a
given period by the total births documented by CHWs in that same period.
The validation analysis involved two components: (1) an evaluation of the completeness of
births and deaths reporting by CHWs; and (2) a comparison of under-five, infant, and neonatal
mortality rates calculated from the CHW data with those estimated from the FPH validation
data for each 12-month period. We calculated mortality rates from the CHW data and the
validation survey data in the same way to ensure direct comparability.
To evaluate the completeness of births and under-five deaths documented by the CHWs, we
estimated the expected number of births and under-five deaths that should have been collected
by CHWs for each 12-month period. We estimated the crude birth rate and the under-five
mortality rate in each period directly from the validation data. To estimate the expected
number of births that should have been reported by CHWs, we multiplied the crude birth rate for
each 12-month period by the total population size of the RMM catchment area in each district.
The expected number of under-five deaths was calculated similarly, by multiplying the
underfive mortality rate estimated from the validation data by the expected number of births
(calculated as described above). We examined the completeness of births and under-five deaths data
collected by CHWs by calculating the ratio of the total numbers of births and under-five deaths
documented by CHWs to the expected numbers estimated as described above.
Each country team prospectively tracked costs associated with the community-based RMM
methods. We reviewed all costing data centrally, and revised costs in collaboration with the
RMM country teams. We inflated costs using local consumer price indices and converted to
2014 United States Dollars. For each method we calculated total cost, average annual cost per
1,000 population, and cost per vital event reported. We excluded the costs of the validation
surveys from analysis as these would overestimate the true cost of RMM.
Average % of
ACCURACY RELATIVE TO CURRENT BEST
PRACTICE (Average Annual Ratio RMM:Best
Practice Census or Survey)
RUNNING COST PER
The RMM community-based workers submitted their reports regularly, suggesting that the
process of timely vital events reporting by CHWs across diverse settings is feasible. The average
percent of catchment areas for which monthly reports were submitted was over 95% in
Ethiopia and Malawi and 100 percent in Mali.
The completeness of the community-based reports, relative to the best-practice FPHs,
varied widely by country, and to a lesser extent, by type of event within countries. In Ethiopia and
Malawi, as shown in Fig 2, the concordance between community and best practice reports was
higher for births than for under-five deaths. CHWs severely under-reported under-five deaths
in both Ethiopia and Malawi, but not in Mali. There was higher concordance for neonatal
deaths than under-five deaths in Ethiopia and Malawi, and higher proportions of neonatal
deaths documented by CHWs than reported in the best practice FPHs in Mali.
The results on data quality are presented for annualized periods in Fig 3 and Table 4. Fig 3
shows the average sex ratio at birth for the RMM data collected by CHWs, compared with
those from comparable full pregnancy histories. The sex ratios at birth for annualized
12-month periods are, on average, consistent between the two data sources. In Mali, the CHWs
documented a higher proportion of male births to female births than was observed in the
corresponding FPHs, but was still well within 20% of that documented by the FPHs. Table 4
shows mortality ratios (the ratio of neonatal deaths to infant deaths and the ratio of infant
deaths to under-five deaths) for the two data sources. Vital events reporting by CHWs
documented higher proportions of infant deaths within the neonatal period than the FPHs across
all three country settings, with an average difference of 6 percentage points (range 3.9 in Mali
to 9.6 in Malawi). Differences between the two methods were larger for the proportion of infant
deaths to under five deaths, with an average absolute difference of 9 percentage points (range
7.8 in Malawi to 9.4 in Mali), and the difference in ratios in Ethiopia and Mali was positive
while in Malawi was negative. In the FPH data that we used to validate CHW-based reporting
of vital events, we observed digit preference (commonly referred to as “age heaping” in
demography) for ages ending in 0 and 5.
In Ethiopia and Malawi we conducted home visits to confirm that the CHWs recorded the
events they identified correctly and completely in their registers. In Malawi, the proportions of
births and deaths reported by CHWs among a sample of 2,227 (14%) birth events and 646
8 / 15
(59%) death events that were independently verified in village health registers (VHR) were
87% and 89%, respectively; in Ethiopia the proportion of births and deaths reported by CHWs
among a sample of 476 (8%) birth events and 139 (47%) death events that were independently
verified in family folders maintained at local health posts was 93% and 91%, respectively [
]. In both Ethiopia and Malawi, the consistency of recording of event dates was around 90%
for the vital events documented by both CHWs and in VHRs or family folders. In Mali we
attempted to match individual events between the RMM and FPH methods to allow a more
indepth understanding of error patterns, with limited success. .
Fig 4 presents results on the accuracy of vital rate estimates produced by the
communitybased method in each setting, compared to the estimates derived from the FBHs. We present
concordance of the RMM methods tested in these three countries, as reflected in the ratios of
crude birth rates and mortality rates vital events reported by CBWs relative to the
corresponding rates estimated through best practice surveys or censuses, Mali performed the best,
followed by Malawi and Ethiopia. In Mali, the accuracy of the RMM method was above 90% for
the crude birth rate (CBR) and U5MR, but well over 100% for the IMR and the NMR. The
pattern is similar in Ethiopia, but with extreme underreporting of the CBR and U5MR.
The RMM programs in Ethiopia, Malawi, and Mali cost $444,392, $353,782, and $288,766,
and had average annual costs per 1,000 population of $523, $434, and $6,344, respectively.
These amounted to $72, $21, and $149 per vital event reported in Ethiopia, Malawi, and Mali,
respectively. Major cost categories included central office salaries, supervision of CHWs, and
costs of training and equipping CHWs for RMM.
The RMM project has generated important information about the feasibility and accuracy of
CHW-based approaches to measuring changes in under-five mortality within periods of 12–24
Fig 2. Reporting Completeness of Vital Events Information by CHBWs relative to the expected number of events estimated from full pregnancy
histories collected through a sample survey or census
9 / 15
Fig 3. Average sex ratio at birth for RMM data collected by community-based health workers compared with those from comparable full
pregnancy histories for annualized periods
months. The approaches tested varied widely, as did the study settings, and the results must
therefore be generalized with care. The scope of the study also did not permit a full assessment
of the effects of the health system context on the performance of each method. The accuracy of
the results produced by the RMM community-based method was tested against a well-known
and established population-based method for mortality estimation–namely, full pregnancy
histories or full birth histories collected via household surveys or censuses.
Mali demonstrates that complete and accurate reporting of vital events by community
workers is possible, but the investments of resources to support locally-defined incentive
structures and levels of supervision and monitoring that were used in the RMM study [
unlikely to be feasible for implementation at scale and over time.
In Ethiopia and Malawi, the RMM community-based method under-estimated the
annualized under-five mortality rate by more than 20%. Whereas, in Mali the two methods produced
Ratio of average annualized deaths based on RMM
reports to associated death ratios based on best practice
Fig 4. Accuracy of Vital Rate Estimates using RMM community-based method in each setting, compared to estimates derived from full birth
histories collected through a sample survey or census
annualized under-five mortality rate estimates that were statistically equivalent. Our Mali
findings demonstrate that, with relatively strong supervision and strong community engagement
with CHWs, community-based workers were capable of collecting routine vital events reports
that facilitate the accurate tracking of changes in under-5 mortality rates on an annual basis.
Our findings from Ethiopia and Malawi are notably more modest–indicating that task shifting
of CHWs and assignment of CHWs to large geographic areas (as was the case in Ethiopia) as
well as frequent turnover of staff (as was the case in Malawi) can be serious impediments in the
estimation of accurate annualized under-five mortality rates using community-based vital
events reporting by CHWs.
Validating the RMM methods in low-income settings is challenging. The vital registration
systems used as a “gold standard” in some high-income countries have rates of completeness
that meet the 90% standard recommended by the United Nation Statistics Division, and even
the results from these systems are adjusted and refined to account for content and coverage
errors using censuses and surveys [
]. In most low- and middle-income countries, vital
registration systems are weak and incomplete, and population censuses and household surveys are
usually the only available source of data that can be used to estimate vital rates. Although we
have used censuses and surveys as current best practice in the RMM work to validate the
estimates produced by community-based vital event reporting methods, we recognize that these
methods do not provide a “gold standard” and are subject to a number of limitations and
challenges associated with retrospective reporting by the mother rather than direct observation and
enumeration at the time of the event.
We found that both the RMM community-based method and the FPHs produced vital
events data that are generally consistent with known patterns, but with some exceptions. The
observed sex ratios at birth, which are expected to fall between 102 and 107 male births per 100
11 / 15
female births in most populations [
], indicated some underreporting of female births relative
to male births, particularly in Ethiopia and Malawi (Fig 3). We do not know whether these
thresholds are true for the RMM study population. In Mali, the CHWs documented, on
average, higher sex ratios at birth than was documented in the corresponding FPH. We also saw
indications of age heaping for age at death in the FPHs in particular, especially at 12 months of
age. This phenomenon is common in retrospective mortality surveys in low resource settings,
where the mother may not be able to recall the child’s exact age at death with precision.
Notwithstanding these issues, the completeness and the quality of the vital events data collected in
Mali were sufficiently high to allow for a valid test of the performance of the community-based
method in estimating mortality over 12 month periods.
In each of the RMM settings that implemented a community-based vital events reporting
system, we analyzed the ratios of early neonatal to neonatal deaths and neonatal deaths to
infant deaths as one measure of data quality. In all three RMM settings, the ratios of early
neonatal to neonatal deaths and neonatal deaths to infant deaths were larger than expected based
on current best estimates derived from household survey interviews, as shown in Table 4. This
suggests that community-based methods may be more effective than the full birth histories
conducted during household surveys or censuses in capturing neonatal deaths. There is no one
simple reason for this. One factor may be that in the RMM community-based methods tested
here we tracked pregnancies as well as births and deaths, and in Mali, proactively reminded the
CHWs to follow up pregnant women around the expected time of birth. Another factor, at
least in Malawi and Ethiopia, may be that the CHWs also had responsibility for providing
pregnant women with advice on preparations for deliveries. Yet another may be that in settings
where the CHWs were selected with inputs from community members, as in Mali, they were
more closely connected to their communities and therefore more likely to be aware of, or
trusted with information about, early neonatal deaths. However, it is also possible that our
RMM community-based methods were susceptible to misclassifying some stillbirths as
neonatal deaths. Further investigation, using full pregnancy histories and the routine reporting of
pregnancies by community-based health workers, is needed to evaluate whether the higher
proportions of neonatal deaths suggest improved reporting of early deaths or rather potential
misclassification of stillbirths (as neonatal deaths) by CHWs.
The challenges associated with implementing community-based vital reporting systems
varied by setting. In Ethiopia, the CHWs charged with vital event reporting were often called away
from their catchment areas for other duties. In Malawi, there were high rates of turnover
among CHWs, requiring frequent refresher training. We found that the introduction of
customized worker incentives intended to improve the completeness and quality of vital events
reporting in Malawi were insufficient to counter the effects of staff turnover [
most notably there was considerable variation in the size of the resident population per CHW
as shown in Table 2. In Ethiopia, each CHW was responsible on average for 2,799 residents in
their local area, compared with 1,273 residents in Malawi, and the 412 in Mali. This amounts
to a notable variation in workload for individual CHWs that directly affects the completeness
of vital events reporting and therefore the accuracy of resulting child mortality rate estimates
based on CHW routine reports. No one-size-fits-all approach will be successful in guaranteeing
complete and accurate reporting of vital events by community-based health workers.
Costing estimates varied greatly by country. Malawi had the lowest cost per vital event
reported and the lowest average annual cost per 1,000 population. Ethiopia had the most
expensive program but a similar cost per 1,000 population to Malawi. Mali had the highest
average annual cost per 1,000 population and the highest cost per vital event reported, which
was double that of Ethiopia and seven times that of Malawi. An important question is the
extent to which the success of the RMM approach in Mali is attributable to greater investment
12 / 15
and smaller scale relative to the other sites. Further attempts to cost community-based tracking
of vital events should give careful consideration to country and program-specific factors from
an early stage.
The CHW reporting methods evaluated in this study were defined in collaboration with
Ministries of Health and local partners in each setting, based on the existing health system
structure and human resources. There are other approaches for generating vital statistics that
may be more promising in some settings, and warrant careful assessment. For example,
Malqvist and colleagues used a combination of key informant interviews, questionnaires and health
facility records in Vietnam to ascertain neonatal deaths, but similar to our findings they
reported high levels of under-reporting [
]. Advances in information technology may provide
new options for community-based reporting [
], but will require rigorous evaluation.
Furthermore, AbouZahr et al. (2015) have noted that innovations in information technology are
not sufficient to guarantee improvements in CRVS systems, rather a strong integrated program
logic that incorporates innovations in new technology needs to drive system improvements
Our findings from Ethiopia, Malawi and Mali indicate that the completeness and accuracy of
vital events reporting by CHWs is affected by a number of operational factors. Perhaps most
surprisingly, vital events reporting by unpaid community-based volunteers in Mali was more
accurate than that by paid workers in Malawi and Ethiopia. This finding underlines the
importance of community acceptance and engagement with the vital events reporting systems and
acceptance of community health workers, as well as the importance of manageable workloads,
effective supervision and continuous quality control procedures. As the international
community seeks to substantially scale-up civil registration and vital statistics systems as part of the
post-2015 development agenda, the need for customized approaches to improving vital events
reporting systems that build on existing systems should not be overlooked. In contemporary
rural sub-Saharan Africa, relatively few deaths occur in health facilities and thus health facility
records provide an inadequate basis to monitor population-level mortality dynamics [
most of the rural population in low-income countries, community-based health workers are
the first and most common interface with the health system–particularly when it comes to
reproductive health and maternal, newborn and child health. Our results from Mali indicate
that, at least at small scale, RMM community-based methods can generate complete vital
events reporting and accurate annualized child mortality indicators. However, our results from
Ethiopia and Malawi highlight important challenges in field implementation, supervision and
reporting completeness. This suggests that those advocating for drawing on CHWs to advance
vital events reporting and improved child mortality measurement should be cautious and that
further research is needed to understand how to enhance existing CHW capacity, streamline
field processes across diverse low-resource settings, and further unpack the trade-offs between
non-sampling errors (such as recall errors) associated with FPHs and FBHs and CHW-based
‘real-time’ reporting problems (such as disclosure bias) of vital events.
We are grateful to the members of the ‘Real-time’ Monitoring of Under-Five Mortality (RMM)
working group (in alphabetical order): Kenneth Hill, Gareth Jones, Olga Joos, Aklilu Kidanu,
Alain Koffi, Mercy Kanyuka, Larry Mouton, Lois Park, Hamadoun Sangho, Emily Wilson.
Their contributions to the design and implementation of this research reported in this article
were critical. We thank the community health workers, RMM district coordinators, district
13 / 15
health officers and environmental health officers in Balaka and Salima, Malawi, Oromia region
of Ethiopia, and Niono and Baroueli, Mali who participated in the project. We also thank our
collaborators at partner institutions at country level, in particular, Tiope Mleme, Jameson
Ndwala, Willie Kachaka, Roberta Makoko (Malawi National Statistics Office), Mariam Traoré,
Ibrahim Terera, Haoua Keita, Assa Keita, and M Diakité (CREDOS, Mali), and Nolawi Tadesse
(Miz Hasab Research Center, Ethiopia). We are also grateful to the Canadian Department of
Foreign Affairs, Trade, and Development for their generous financial support of the Real time
Results Tracking project.
Conceived and designed the experiments: AA JB MM RB. Performed the experiments: AA
MM. Analyzed the data: RS. Contributed reagents/materials/analysis tools: AM EH CV. Wrote
the paper: RS JB AA.
14 / 15
1. United Nations . Sustainable Development Goals Knowledge Platform . Available at https:// sustainabledevelopment.un.org/topics, accessed 20 October 2015 .
2. Independent Expert Review Group, Commission on Information and Accountability for Women's and Children's Health. ( 2015 ) Final Report . Every Woman, Every Child, Every Adolescent: Achievements and Prospects. Geneva: World Health Organization. IISBN 978 92 4 150928 2.
3. Mathers C , Boerma T ( 2010 ) Mortality Measurement Matters: Improving Data Collection and Estimation Methods for Child and Adult Mortality . PLoS Med 7(4): e1000265. doi: 10.1371/journal.pmed.1000265 PMID: 20405053
4. UNICEF, WHO, World Bank, UN-DESA Population Division Levels and Trends in Child Mortality . Unicef: 2014 .
5. Pedersen J , Liu J ( 2012 ) Child Mortality Estimation: Appropriate time periods for child mortality estimates from full birth histories . PLoS Med 9: e1001303 doi: 10.1371/journal.pmed.1001289 PMID: 22952435
6. Amouzou A , Kachaka W , Banda B , Chimzimu M , Hill K , Bryce J . Monitoring child survival in 'real time' using routine health facility records: results from Malawi . Trop Med Int Health. Oct 2013 ; 18 ( 10 ): 1231 - 1239 .
7. Hill K , Lopez AD , Shibuya K , Jha P , AbouZahr C , Anderson RN , et al. ( 2007 , November). Interim measures for meeting needs for health sector data: births, deaths, and causes of death . Lancet 370 ( 9600 ), 1726 - 35 . PMID: 18029005
8. United Nations . Demographic Yearbook 2013 . United Nations Statistics Division, New York, NY.
9. Slauter W ( 2011 ). "Write up your dead: The bills of mortality and the London plague of 1665." Media History 17 ( 1 ): 1 - 15 .
10. Office of the Registrar General & Census Commissioner , India ( 2012 ). India's Sample Registration System . http://censusindia.gov.in/Vital_Statistics/SRS/Sample_Registration_System.aspx
11. Yang G , Hu J , Rao KQ , Ma J , Rao C , Lopez AD . Mortality registration and surveillance in China: history, current situation and challenges . Popul Health Metr 2005 ; 3 : 3 - doi: 10.1186/ 1478 -7954-3-3 PMID: 15769298 .
12. Pratiwi , Endah Dwi, Kosen, Soewarta Development of an Indonesian sample registration system: a longitudinal study The Lancet , Volume 381 , S118
13. Setel Philip W., Sankoh Osman , Rao Chalapati, Velkoff Victoria A., Mathers Colin , Gonghuan Yang , et al. z Sample registration of vital events with verbal autopsy: a renewed commitment to measuring and monitoring vital statistics Bulletin of the World Health Organization 2005 ; 83 : 611 - 617 . PMID: 16184280
14. Nguyen Phuong Hoa , Rao, Chalapati Hoy, Damian G., Nguyen Duc Hinh, Nguyen Thi Kim Chuc and Duc Anh Ngo ( 2012 ) Mortality measures from sample-based surveillance: evidence of the epidemiological transition in Viet Nam . Bulletin of the World Health Organization, 90 10 : 764 - 772 . doi: 10 .2471/ BLT.11.100750 PMID: 23109744
15. Ye Y , Wamukoya M , Ezeh A , Emina J , Sankoh O ( 2012 , September) . Health and demographic surveillance systems: a step towards full civil registration and vital statistics system in sub-Sahara Africa? BMC public health 12 (1 ), 741 .
16. Clark S , Wakefield J , McCormick T , Ross M. HYAK Mortality Monitoring System: Innovative Sampling and Estimation Methods . Center for Statistics and the Social Sciences , University of Washington. Working Paper No. 118. Sept 2012 .
17. Singh P , Sachs JD . 1 million community health workers in sub-Saharan Africa by 2015 . Lancet, 2013 . Volume 382 , Issue 9889 , 363 - 365 . doi: 10 .1016/S0140- 6736 ( 12 ) 62002 - 9 PMID: 23541538
18. Bhutta ZA , Lassi ZS , Pariyo G , Huicho L . Global experience of community health workers for delivery of health related millennium development goals: a systematic review, country cases studies, and recommendations for integration into national health systems . http://www.who.int/workforcealliance/ knowledge/publications/CHW_FullReport_ 2010 .pdf ( accessed Jan 2 , 2015 )
19. International Labour Organization. International Standard Classification of Occupations , 2008 revision. Geneva, ILO.
20. Kok MC , Muula AS ( 2013 ). Motivation and job satisfaction of Health Surveillance Assistants in Mwanza, Malawi: an explorative study . Malawi Medical Journal , 25 ( 1 ), 5 - 11 . PMID: 23717748
21. Bryce J , RMM Working Group. ' Real-time' monitoring of under-five mortality: A vision tempered by reality . PLoS Medicine , 2015 .
22. Amouzou A , et al. Challenges and Innovations in Using Health Extension Workers and Community Volunteers to Improve Vital Events Registration in Rural Ethiopia . Paper in this Collection. PLoS ONE , 2015 .
23. Munos M , et al. Strengthening community networks for vital event reporting: Community Volunteer Reporting of Vital Events in Rural Mali . Paper in this Collection. PLoS ONE , 2015 .
24. Joos O et al. A Comparative Review of Child Mortality Monitoring Using Village Health Registers, Health Facility Records, and Health Surveillance Assistants in Malawi. Paper in this Collection . PLoS ONE , 2015 .
25. Pullum , Thomas W , Becker S. 2014 . Evidence of Omission and Displacement in DHS Birth Histories . DHS Methodological Reports No. 11 . Rockville , Maryland, USA: ICF International
26. Miller NP , Amouzou A , Tafesse M , Hazel E , Legesse H , Degefie T , et al. ( 2014 ) Integrated Community Case Management of Childhood Illness in Ethiopia: Implementation Strength and Quality of Care . The American Journal of Tropical Medicine and Hygiene , 91 ( 2 ), 424 - 434 . http://doi.org/10.4269/ajtmh.13- 0751 doi: 10.4269/ajtmh.13-0751 PMID: 24799369
27. R Core Team ( 2013 ). R: A language and environment for statistical computing . R Foundation for Statistical Computing , Vienna, Austria. URL http://www.R-project. org/.
28. United Nations , World Mortality Report 2011. Population Division , Department of Economic and Social Affairs, 2012 .
29. Siegel JS , Swanson DA (eds.) , The Methods and Materials of Demography, 2nd edition , San Diego: Elsevier, Academic Press, 2004 ,
30. Målqvist M , Eriksson L , Thu Nga N , Fagerland I , Phuong Hoa L , Wallin D , et al. ( 2008 ) Unreported births and deaths, a severe obstacle for improved neonatal survival in low-income countries; a population based study . BMC International Health and Human Rights , 8 ( 4 ). http://dx.doi.org/10.1186/ 1472 - 698X-8-4
31. Lewis T , Synowiec C , Lagomarsino G , Schweitzer J ( 2012 ) E-health in low- and middle-income countries: findings from the Center for Health Market Innovations Bulletin of the World Health Organization, 90 : 332 - 340 . doi: 10 .2471/BLT.11.099820
32. Piette JD , Lun K , Moura LA , Fraser HS , Mechael PN , Powell J , et al. ( 2012 ). Impacts of e-health on the outcomes of care in low- and middle-income countries: where do we go from here? Bulletin of the World Health Organization, 90 ( 5 ), 365 - 372 . http://doi.org/10.2471/BLT.11.099069 doi: 10.2471/BLT.11. 099069 PMID: 22589570
33. Abouzahr C , de Savigny D , Mikkelsen L , Setel P. , Lozano R. , Nichols E. et al. Civil registration and vital statistics: progress in the data revolution for counting and accountability . Lancet . 2015 ; (published online May 11 ) http://dx.doi.org/10.1016/S0140- 6736 ( 15 ) 60173 - 8 .