Global estimates of population health are currently needed because of the shortage of adequate quality population-based data. These estimates are complex, because they need to combine relatively complete data from industrialised countries with sometimes very scanty data from developing countries.
Two major sources of such estimates are agencies within the United Nations system and (mostly northern) academic institutions, which differ in their approaches.
Appropriate strategies for ensuring the robustness and transparency of estimates are very important. Long-term strategies must be geared towards improving the quantity and quality of bottom-up data, rather than developing ever more complex estimation methods.
Ultimately, the world must be able to measure population health from reliable individual data rather than relying on estimates.
This article is part of a cluster of five articles on global health estimates.
How Did We Get Here?
Global measurement has long been contentious. Three hundred years ago, the exact size and shape of the world were a matter of scientific controversy and estimation. New ways of measurement were developed, and by the early 18th century that uncertainty ceased . One hundred years ago, Sweden started to account for all deaths in its population, by cause, age, and sex , and annual summary tables based on individual death registration on a national basis have been published ever since. Today, despite increasing globalisation, there is still no similar universal individual registration of vital events in many countries, and consequently we find ourselves in an era of global estimates of population health. These global estimates are complex amalgams of detailed national measures from countries with universal registration and the best available data—which are often scanty—from other settings. Hopefully the long-term aim of the global health community is to move beyond this era of estimates, towards the relative certainty of accounting for individual health globally. Meanwhile, the purpose of this article is to explore issues and tensions around these currently necessary global estimates.
As discussed previously , fundamental links between poverty and data availability mean that global estimates are not trivial to construct. Unlike global estimates of geophysical parameters, which have been revolutionised by remote sensing and satellite surveillance , population health estimates must continue to rely in some way on grass-roots data about individual people. But lack of standards and different approaches are causing contention and confusion.
Why Is There Debate about Current Global Estimates?
Current global estimates mainly come from one of two sources: (1) the United Nations (UN) and its specialised agencies (such as the World Health Organization and the United Nations Children's Fund [UNICEF]) or (2) northern academic institutions. There are important underlying differences between the estimates from these sources, as shown in Figure 1. Why southern academic institutions are not more engaged in the process of developing global estimates, given that the major uncertainties within most estimates centre on southern data, is a further question of interest.
The UN Estimates
To contextualise estimates originating from the UN, it is necessary to understand the nature of the UN system. The UN is fundamentally constructed as a member organisation for most of the world's nation states, and member countries are therefore its constituency. This structure has significant practical implications for the construction of global estimates. The UN system has a direct entrée to member countries at the governmental level and, although unable to exercise compulsion, may be able to access otherwise unpublished national data. After UN experts have processed and modelled available data, member countries are commonly consulted on estimates before publication. UN estimates can then be published under the UN imprimatur , although often staff and external advisors also prepare articles for publication in peer-reviewed journals . This approach has been criticised , perhaps unfairly.
Estimates from Academic Institutions
Academic institutions, by contrast, have to rely largely on public-domain data and previously published studies, often leading to the use of mixed methods involving direct and meta-analyses –. As independent institutions, they have no obligation to consult externally, and countries' first sight of such estimates may well be on publication. While this may lead to interesting post-publication debate and controversy, there can be a problem in that a published paper is still likely to be taken as the definitive version, disregarding any implications of subsequent interactions. Such estimates are, however, normally disseminated via peer-reviewed journals, which should be assumed to assure a paper's quality, as with any other scientific output. However, questions have been raised as to how to effectively peer-review papers describing complex estimates . An interesting development might be the inclusion of a country consultation phase before publication of estimates from academic institutions, making public the issues that thereby arise.
Two Tales of Maternal Mortality
Some of the recent debate and contention around the source of global estimates emerged during 2010 when two separate estimates of global maternal mortality were published. One set of estimates originated from the Institute for Health Metrics and Evaluation (IHME) , and the other was the latest update of the UN inter-agency estimates of maternal mortality . It is impossible to conclude which is the more “correct” set of estimates, because if that were measurable as a matter of fact, the estimates would be redundant anyway. However, it is interesting to compare some key issues in the approaches and conclusions of these different estimates. Headline figures were very similar and well within each other's uncertainty intervals (342,900 and 358,000 maternal deaths worldwide and maternal mortality ratios of 251 and 260 per 100,000, respectively, for 2008). However, there may be important differences at the country level or for specific causes, depending on the data and methods used.
On the critical issue of estimating how many deaths among pregnant women might be associated with HIV/AIDS, very different approaches were used, with widely different conclusions (61,400 and 42,000 deaths, respectively, for 2008). Bottom-up data on deaths associated with both pregnancy and HIV/AIDS—particularly from Africa—are very scanty because, even where verbal autopsies are performed to ascertain cause of death, very often only a single cause is recorded . It is not surprising that where data are critically lacking, different approaches to estimation yield different results. The overlap between pregnancy and HIV/AIDS is an area in which urgent reforms in methods and procedures are needed in order to provide appropriate multiple-cause data, rather than developing more complex estimation methods.
How Can the Robustness and Transparency of Global Estimates Be Ensured?
Because estimates are estimates, and not measurements, it is relatively easy for the proponents of particular estimates to claim high quality and reliability, and for the detractors to question the same, with little scope for objective adjudication. The key factor for robustness is the extent of available data, linked, of course, to sound methods. Transparency involves using all available data of quality and relevance, while usually imposing some explicit framework of rules as to what constitutes usable data. Methodological strategies then also need to be set out in a fully transparent manner. Any epidemiological interpretation depends on an understanding of the provenance and sampling basis of the underlying data, which leads to estimates of uncertainty. However, some of the data used in global estimates are so many stages removed from their origin that associated estimates of uncertainty themselves become very complex and hard to understand. Unfortunately, the gaps and uncertainties around data in many instances drive researchers to ever-increasing levels of methodological complexity in attempts to compensate, and transparency may be obscured by these complexities. This can rapidly lead to an “Emperor's New Clothes” syndrome in which only the cognoscenti truly understand the underlying basis of complex estimates, while the vast majority may be reluctant to admit that the detail is beyond their comprehension.
Where Do We Need to Go from Here?
The undeniable long-term aim must be to foster more and more production of high-quality and complete population data from locations that are as yet devoid of usable material. This implies a bottom-up philosophy emphasising the need to connect with population data at source . If a gradual process of filling in such gaps in global data is realistic, then one would also hope that in parallel with increasing completeness of data there will be reductions in the complexity of appropriate estimation methods. This would lead towards the ideal situation, in which global estimates would become a thing of the past as the world's population actually became measurable.
One potential obstacle to this process is that the world may become so used to the concept of global estimates that insufficient effort will be invested in improving bottom-up data. Even now, there are signs that journal editors can find complex global estimates more enticing to publish than relatively detailed epidemiological descriptions of within-country data. For example, four recent sets of global estimates published in The Lancet have been “fast-tracked” for reasons that are not entirely clear ,,,; since they actually present long-term estimates that have no urgent health care implications, perhaps this is symptomatic of the self-importance that such estimates seem to be acquiring. It is very important that the concept of global estimates does not acquire an undeserved supremacy over conventional analyses of detailed data, if we want to nurture a culture of encouraging and promoting good-quality data at source, with local analyses and interpretation . Strategies to bring all countries to a common standard of high-quality and sustainable health information systems need to be prioritised .
For the foreseeable future, the global health community is likely to be locked in a love–hate relationship with global estimates of population health. We wish we did not need these estimates and could instead rely on objective assessments based on quality data. But we know we do not yet have enough of those data, and so for now we do need estimates as a resource on which to base health policy and planning decisions. However, we should not expect to wait three hundred years, nor even one hundred years, before superseding global estimates with global measurements.
Global Health Estimates: Where Do We Go from Here?
Work towards better availability and quality of data
Enhance country capacity to develop, analyse, and interpret local data
Develop common data standards
Focus on better data rather than more complex estimation methods
Improve robustness of data and methodologies used in estimates
Manage a gradual transition from estimates to measurements
1. WestfallRS 1983 Never at rest: a biography of Isaac Newton Cambridge Cambridge University Press
2. Kungliga Statistika Centralbyrån 1915 Dödorsacker År 1911 Stockholm Kungliga Statistika Centralbyrån
3. ByassP 2009 The unequal world of health data. PLoS Med 6 e1000155 doi:10.1371/journal.pmed.1000155
4. van DonkelaarAMartinRVBrauerMKahnRLevyR 2010 Global estimates of ambient fine particulate matter concentrations from satellite-based aerosol optical depth: development and application. Environ Health Perspect 118 847 855
5. United Nations Children's Fund, World Health Organization, World Bank, United Nations DESA/Population Division 2010 Levels and trends in child mortality—report 2010. Available: http://www.childinfo.org/files/Child_Mortality_Report_2010.pdf. Accessed 17 September 2010
6. YouDJonesGHillKWardlawTChopraM 2010 Levels and trends in child mortality, 1990–2009. Lancet 376 931 933
7. 2010 Soul searching at the UN. Lancet 376 1026
8. BlackRECousensSJohnsonHLLawnJERudanI 2010 Global, regional, and national causes of child mortality in 2008: a systematic analysis. Lancet 375 1969 1987
9. DellicourSTatemAJGuerraCASnowRWter KuileFO 2010 Quantifying the number of pregnancies at risk of malaria in 2007: a demographic study. PLoS Med 7 e1000221 doi:10.1371/journal.pmed.1000221
10. RajaratnamJKMarcusJRFlaxmanADWangHLevin-RectorA 2010 Neonatal, postneonatal, childhood, and under-5 mortality for 187 countries, 1970–2010: a systematic analysis of progress towards Millennium Development Goal 4. Lancet 375 1988 2008
11. GrahamWJBraunholtzDACampbellOM 2010 New modelled estimates of maternal mortality. Lancet 375 1963
12. HoganMCForemanKJNaghaviMAhnSYWangM 2010 Maternal mortality for 181 countries, 1980–2008: a systematic analysis of progress towards Millennium Development Goal 5. Lancet 375 1609 1623
13. World Health Organization, United Nations Children's Fund, United Nations Population Fund, World Bank 2010 Trends in maternal mortality: 1990 to 2008 Geneva World Health Organization Available: http://whqlibdoc.who.int/publications/2010/9789241500265_eng.pdf. Accessed 20 September 2010
14. ByassPKahnKFottrellECollinsonMATollmanSM 2010 Moving from data on deaths to public health policy in Agincourt, South Africa: approaches to analysing and understanding verbal autopsy findings. PLoS Med 7 e1000325 doi:10.1371/journal.pmed.1000325
15. BoermaTAbouZahrC 2005 Monitoring global health: bottom up approach is more likely to be successful. BMJ 330 195 196
16. RajaratnamJKMarcusJRLevin-RectorAChalupkaANWangH 2010 Worldwide mortality in men and women aged 15–59 years from 1970 to 2010: a systematic analysis. Lancet 375 1704 1720
17. RavishankarNGubbinsPCooleyRJLeach-KemonKMichaudCM 2009 Financing of global health: tracking development assistance for health from 1990 to 2007. Lancet 373 2113 2124
18. BanghaMDiagneABawahASankohO 2010 Monitoring the millennium development goals: the potential role of the INDEPTH Network. Glob Health Action 3 5517 doi:10.3402/gha.v3i0.5517
19. ChanMKazatchkineMLob-LevytJObaidTSchweizerJ 2010 Meeting the demand for results and accountability: a call for action on health data from eight global health agencies. PLoS Med 7 e1000223 doi:10.1371/journal.pmed.1000223