Skip to content

Improving Measurement

Data on the quality of facility-based care received by mothers and newborns is often lacking. We were interested in the validity of data sources for quality of care measures, the definitions used for quality indicators, and the methods and indicators needed for respectful maternity care. The rich sources of data generated for tracking progress in Gombe enabled the IDEAS team to carry out research on improved measurement of priority indicators, particularly around effective coverage measurement and respectful maternity care.

Measuring effective coverage for maternal and newborn health

A man is collecting data during a household survey
A household survey in Gombe State, Nigeria.
Our research aimed to improve the measurement of priority indicators for maternal and newborn health, including quality of care measures. A number of problems needed to be addressed: (i) generating greater clarity around what it was possible to measure, and how; (ii) understanding which data should be used to drive decisions; (iii) developing methods for linking household data on access to care with facility data on quality of care; and (iv) creating actionable effective coverage measures for facility-based childbirth care.

In 2010 it was clear that accelerated progress was needed to achieve the Millennium Development Goals 4 and 5 on child and maternal survival. In addition to increasing the access to and uptake of facility-based care, focus had shifted to also prioritise the quality of care provided to families.

The discourse around quality of facility-based care had long been established, but the global health goals, carried forward through the Sustainable Development Goals, prompted new commitment to act. If ‘what gets measured gets done’ then the measurement community needed to contribute actionable solutions to track the coverage of high quality maternal and newborn healthcare at scale. Then, in turn, the proportion of the population in need of quality care who received quality care could be monitored and continuously improved.

To make progress it was important to explicitly recognise the gaps in knowledge. Quality of care indicators were relatively poorly defined and lacked harmonisation between time and place. The need for indicators to guide decision-making at different health system levels – within and beyond health facilities – was not yet fully described.  The data sources available to measure those quality indicators were not always fit for purpose.  And methods for linking data sources to construct quality-adjusted – or effective coverage – measures needed development.

 

 

 

A key contribution of IDEAS was to work together with others to develop solutions: with country partners, university colleagues, and multilateral agencies.  We worked especially closely with the Partnership for Maternal and Newborn Health in Gombe State, Nigeria.  Four main areas of contribution are highlighted here, often reflecting collaboration across multiple stakeholder groups.

 

A Call To Action: Quality of care for mothers and newborns is a complex construct and challenging to measure. The renewed focus on ensuring high quality care for all families provided an opportunity to discuss, engage and reflect on how to address this complexity.

Our research contributed to conceptualising the distinction between essential health care that could already be measured and health care for which measures needed development and investment, particularly measures related to processes and experiences of care.  With a focus on tracking the coverage of high quality care in resource-poor settings, our team called for systematic approaches to effective coverage measurement that decision makers could use to drive change.

Examining available data: 

Actionable data is needed to drive decision-making for quality facility-based care. But not all data sources provide valid measures or are fit for the purpose of decision-making at a given level.

We examined the validity of household and facility-based data sources for maternal and newborn health, finding that a matrix of data sources was needed to estimate the various constructs of high quality facility-based care. We also found that this matrix of data sources  could include routine data sources in settings where there was interest to use routine data for decision-making.

 

Developing methods for linking data sources:

Examples of linked household and facility data have become more prevalent.  But individually linked household and facility data was scarce and the methods applied for “ecological” linking of data sources lacked harmonisation.

Our methodological work demonstrated that linked data should include adjustment for the size or level of health facility, to take account of the population level impact of facility quality in quality-adjusted coverage measures.

 

Constructing effective coverage measures:

Effective coverage measures should estimate the proportion of the population in need of quality care who went on to receive quality care.

For maternal, newborn, child, and adolescent care and nutrition, our research revealed a total lack of harmonisation in the way this complex measure had been defined by researchers. Through our research to understand the most valid and actionable data sources, together with research on methods, we generated an effective coverage measure of childbirth care that could be generated using country-owned data sources, potentially leading to greater use for decision-making.

 

Improving measurement for respectful maternity care

Blood pressure is measured on a woman's arm
A pregnant woman is having her blood pressure measured.
We aimed to study positive and negative facility childbirth experiences and to determine best practices for measuring respectful maternity care. In Gombe State, Nigeria, we did research to understand 1) mistreatment during facility childbirth, 2) the utility women placed on attributes of childbirth care experiences, 3) the validity of measures of childbirth care experiences derived from exit interviews, 4) the validity and acceptability of capturing childbirth care experiences though telephone interviews, 5) the feasibility and acceptability of primary healthcare provider-led phone follow-up with mothers shortly after childbirth.

Respectful maternity care, including effective communication, respect and preservation of dignity and emotional support, is widely acknowledged as a public health issue of global importance. WHO guidelines on positive pregnancy experience, positive childbirth experience and positive postnatal experience all emphasise the need to promote respectful maternity care and to eliminate mistreatment of women during facility childbirth.

However, generating accurate and reliable measures of women’s childbirth care experience is challenging. For example, operational definitions may vary by context, and the optimal recall period for self-reported measures is a subject of debate. There is a dearth of evidence on the validity of self-reported measures of childbirth care experience derived through exit interviews with women. Further, traditional survey methods are resource intensive, limiting their use for continuous monitoring in low-income and middle-income countries. To ensure timely data on women’s childbirth care experience for health facility staff and managers, novel and sustainable methodologies are needed.

Using primary data collected as part of the Gombe Partnership we carried out research on respectful maternity care that responded to local and global needs, and was conducted in collaboration with state actors and community stakeholders in Gombe state, Nigeria, as well as international collaborators. Major contributions are described here.

Understanding mistreatment during institutional delivery

Mistreatment of women during facility childbirth is increasingly seen as one of the leading reasons behind the low utilisation of maternal and newborn health services in settings with poor maternal and newborn health outcomes, such as Gombe state.   We investigated the quality of care relating to the prevalence and manifestations of mistreatment during institutional birth, finding that mistreatment was reported in two-thirds of all institutional deliveries, with reported prevalence varying across the dimensions of care. Qualitative findings highlighted different forms of mistreatment that might take place during institutional births.

The relative value (utility) associated with a range of childbirth care experience attributes

Widespread mistreatment has been reported but there is a limited understanding of the aspects that matter most to women.  We interviewed rural Nigerian women to examine how specific attributes of a hypothetical facility birth experience of care influenced their stated preference for hypothetical place of delivery. We found that poor facility culture, including an unclean birth environment with no privacy, and unclear user fees, negatively impacted choices for facility-based childbirth.

The validity of measures of respectful maternity care derived from exit interviews

Valid methods for assessing respectful maternity care are essential but validation research is sparse, reporting mixed results.  We investigated the validity of eight positive and sixteen negative maternity care experience measures derived from exit interviews. We found that maternity care experiences self-reported in exit interviews by women were consistent with the observation of childbirth for all eight positive maternity care experience indicators and six of the sixteen negative maternity care experience indicators investigated.

The validity of telephone interviews to capture data on childbirth care experiences

Health facility staff and managers need frequent and timely data on the experience of childbirth care. Telephone follow-up interviews may be a low-cost option.  We investigated the validity of eight indicators of positive maternity care experience and 18 indicators of negative maternity care experience derived via telephone interviews. We found that the telephone interviews conducted 14 months after childbirth did not yield results consistent with exit interviews conducted at the time of discharge.

Women’s perceptions of telephone interviews about their childbirth care experiences

The high coverage of mobile phones in low-and middle-income countries  has made telephone interviews a promising alternative or supplement to face-to-face interviews. But there is limited evidence on women’s perceptions and acceptability of telephone interviews about their experiences with facility childbirth care in such settings.  We investigated women’s perceptions of phone interviews about their experiences with facility childbirth care, finding that most women had positive views about the phone interviews.

Feasibility and acceptability of primary healthcare provider-led phone follow-up with recent mothers

An established system of integrating the experience of care measures within routine data capture exercises in health facilities is needed to improve women’s experience of childbirth care.  We investigated the feasibility and acceptability of primary healthcare provider-led phone follow-up with recently delivered women about their maternity care experiences, finding that it was feasible and acceptable.

 

All Improving Measurement related outputs can be found under Resources by selecting that theme from Research Areas