You are here

Article Text

Article menu

Download PDFPDF

Research report
Verbal autopsy can consistently measure AIDS mortality: a validation study in Tanzania and Zimbabwe
  1. B Lopman1,
  2. A Cook2,
  3. J Smith1,
  4. G Chawira3,
  5. M Urassa2,
  6. Y Kumogola2,
  7. R Isingo2,
  8. C Ihekweazu4,
  9. J Ruwende4,
  10. M Ndege2,
  11. S Gregson1,3,
  12. B Zaba2,5,
  13. T Boerma6
  1. 1Department of Infectious Disease Epidemiology, Imperial College, London, UK
  2. 2National Institute for Medical Research, Mwanza, Tanzania
  3. 3Biomedical Research and Training Institute, Harare, Zimbabwe
  4. 4Health Protection Agency, London, UK
  5. 5Centre for Population Studies, London School of Hygiene and Tropical Medicine, London, UK
  6. 6World Health Organization, Geneva, Switzerland
  1. Correspondence to Ben Lopman, Imperial College, London, UK; blopman{at}gmail.com

Abstract

Background Verbal autopsy is currently the only option for obtaining cause of death information in most populations with a widespread HIV/AIDS epidemic.

Methods With the use of a data-driven algorithm, a set of criteria for classifying AIDS mortality was trained. Data from two longitudinal community studies in Tanzania and Zimbabwe were used, both of which have collected information on the HIV status of the population over a prolonged period and maintained a demographic surveillance system that collects information on cause of death through verbal autopsy. The algorithm was then tested in different times (two phases of the Zimbabwe study) and different places (Tanzania and Zimbabwe).

Results The trained algorithm, including nine signs and symptoms, performed consistently based on sensitivity and specificity on verbal autopsy data for deaths in 15–44-year-olds from Zimbabwe phase I (sensitivity 79%; specificity 79%), phase II (sensitivity 83%; specificity 75%) and Tanzania (sensitivity 75%; specificity 74%) studies. The sensitivity dropped markedly for classifying deaths in 45–59-year-olds.

Conclusions Verbal autopsy can consistently measure AIDS mortality with a set of nine criteria. Surveillance should focus on deaths that occur in the 15–44-year age group for which the method performs reliably. Addition of a handful of questions related to opportunistic infections would enable other widely used verbal autopsy tools to apply this validated method in areas for which HIV testing and hospital records are unavailable or incomplete.

  • Death certification
  • demography FQ
  • HIV epidemiology
  • HIV infection
  • mortality SI
  • HIV/AIDS
  • mortality
  • sub-Saharan Africa
  • verbal autopsy

https://doi.org/10.1136/jech.2008.081554

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Verbal autopsy, in which care givers to the deceased report information on signs, symptoms and circumstances preceding death,1–3 is currently the best option for obtaining cause of death information in populations without comprehensive civil registration systems. WHO, through the Health Metrics Network, is co-ordinating an effort to develop a common tool for verbal autopsy, since there is an urgent need to validate currently used questions for different populations, settings and causes of mortality.4

    There is particular need to develop methods to measure AIDS mortality, which is the leading cause of death among young adults in virtually all countries with generalised HIV epidemics. Improving measurement of AIDS mortality is crucial since the ultimate goal of interventions—including the scale up of antiretroviral treatment—is to reduce AIDS mortality.5 Monitoring the success of such programmes therefore relies on accurate measurement of AIDS deaths in the community.6 7 Hospital records and vital registration of deaths are especially inadequate for estimating the level of AIDS as a cause of mortality because of bias, underreporting and stigma associated with the disease.8

    Two longitudinal community studies in Tanzania and Zimbabwe provide a unique opportunity to investigate the most sensitive and specific set of questions to ascertain HIV/AIDS-associated mortality. Both studies have collected information on the HIV status of the population over a prolonged period and maintained a demographic surveillance system (DSS) that collects information on cause of death through verbal autopsy. At the time of the verbal autopsy studies, HIV prevalence was approximately 20% in the Zimbabwe cohort and 7% in the Tanzania cohort.9 10 With data from the Zimbabwe study, we developed a computer algorithm to classify AIDS deaths from verbal autopsy data11 12 validated using serological data from deaths that occurred from 1998 to 2003. Before the algorithm can be widely used, it must be demonstrated that it performs similarly and predictably in different settings of high HIV-associated mortality (validation in place) and can perform well at different phases of the epidemic when levels of AIDS mortality differ (validation in time).

    Methods

    All participants of both cohorts were followed as part of demographic surveillance and were tested for HIV at each sero-survey. If an individual died between follow-ups, an attempt was made to perform verbal autopsy. An individual's HIV status at death was assumed to be the same as at his/her most recent test, which was a maximum of 3 years prior.

    The Manicaland Project includes a population-based open cohort study in the rural province of Manicaland in eastern Zimbabwe.9 The study population were residents in two forested small towns, four tea and coffee estates and six rural areas (including four subsistence farming and two roadside trading centres). A baseline survey took place from 1998 to 2000, with two follow-ups occurring 3 and 5 years later (the intersurvey periods are referred to here as phase I and phase II). Of the households, 8376 and 7102 identified in the survey areas at phase I and phase II, respectively, were enumerated. Male and female participation rates in the individual cohort study survey were 78% (4320/5561) and 80% (5134/6419) at phase I and 77% (3047/3958) and 80% (3972/4936) at phase II, respectively. Verbal autopsies were conducted for 94% of all deaths,9 with 53% of the verbal autopsy reports obtained from close relatives (spouse 19%, child 7%, parent 16% and sibling 10%). At each follow-up of the study, testing for the presence of HIV antibody was performed.13

    The Kisesa HIV cohort is located in Magu District, Mwanza Region, in northwestern Tanzania. The cohort was established in 1994 (when baseline studies were conducted) and data collection is based on a biannual DSS that had conducted 14 phases by 2002 and sero-surveys repeated approximately every 3 years, with three testing surveys completed before 2002.10 14–17 The population in the DSS area grew from 19 354 in 1994 to 24 403 by 2002. Participation in the DSS is more than 98%, with proxy reporting accepted for absent household members. The average participation rate in sero-surveys was 72% in the first three surveys. Deaths identified in the DSS are followed up with a verbal autopsy interview between 6 weeks and 6 months later, if a reliable informant can be identified who cared for the deceased during the final illness. Verbal autopsy interviews were completed for 67% (420/629) of the adult male deaths and 64% (424/667) of the adult female deaths recorded in the DSS between 1994 and 2002; 94% of the verbal autopsy reports were obtained from close relatives (spouse 30%, child 28%, parent 21% and sibling 15%).

    The verbal autopsy tool

    The study teams identified deaths through the use of checklists of all individuals interviewed at the previous phase and discussions with village health workers, employers and surviving household members present at follow-up. Data were collected on the signs, symptoms and circumstances preceding death using a structured, closed, interviewer-led questionnaire. The verbal autopsy questionnaire was originally developed in Kisesa and contained specific questions related to symptoms of late-stage HIV and opportunistic infections14 and was used in that site from 1994 to 2002. A nearly identical questionnaire was adapted in Manicaland in both phase I and phase II. The questionnaires were administered in local languages (Shona, the predominant local language in Zimbabwe, and Swahili, in Tanzania). Interviewers were clinical officers/certified nurses who received special training on how to administer the verbal autopsy questionnaire. Verbal autopsy informants were parents, spouse, other relatives or a neighbour (in rare circumstances when close relatives were not available). The interviews were conducted after the recognised mourning period, in a respectful and unhurried manner. In Kisesa, the interviewer gave a small, culturally appropriate gift (a bar of soap) to the person with whom the interview was conducted, as a token of appreciation of the time devoted to answering the lengthy questionnaire. Ethical approval for the DSS and all related procedures (such as the verbal autopsy interviews) was granted by the Tanzanian Medical Research Co-ordinating Committee for the Kisesa study and by the Zimbabwe Medical Research Council for the Manicaland Study.

    Validation procedures

    We developed a computer algorithm that creates a set of criteria for classifying AIDS deaths based on verbal autopsy data.12 Seventy-five per cent of deaths from Manicaland phase I were randomly assigned to a training dataset. From this training dataset, all signs/symptoms with a likelihood ratio >1.92 in univariate analyses were considered as potential identifiers of AIDS death (as defined below). Signs/symptoms were added to a list of criteria one at a time, based on the highest specificity. Verbal autopsy deaths with that sign/symptom were then removed from the dataset, and specificities of the remaining signs/symptoms were recalculated. Sensitivity was plotted against 1–specificity in a receiver operator characteristic (ROC) plot. (The ROC is a tool used to select an optimum cut-off for a diagnostics test based on the trade-off of sensitivity and specificity.) These steps were repeated until the (equally weighted) sensitivity and specificity of the list of symptoms was maximised (ie, the point closest to the top left hand corner of the ROC plot). Deaths were classified as HIV/AIDS associated if the deceased had one or more of the criteria on the list. The sensitivity and specificity of the list was then tested on the Manicaland phase I test data (the remaining 25% of deaths), all phase II and all Kisesa data. AIDS death in the gold standard was defined as an individual who was (a) HIV positive at baseline survey based on antibody testing; (b) was not reported to have suffered major injury from motor vehicle accident, self-inflicted (suicide), or accidentally (accident) or deliberately inflicted by another person (homicide) in the 2 weeks before death; and (c) did not die from direct obstetric causes (death during labour).12 Preliminary analyses highlighted that HIV prevalence among the deceased was markedly lower among the relatively older adults, so analyses were stratified at age 45 years.

    Results

    There were a total of 376 and 219 deaths in phase I (1998–2003) and phase II (2003–2005), respectively, of the Manicaland study and 197 in Kisesa (1994–2002), among 19–59-year-olds for whom there was a verbal autopsy and a conclusive HIV test done within 3 years of death. A minority of deaths occurred in the 45–59-year age group (13%, 17% and 19%) in Manicaland phase I, phase II and Kisesa cohorts, respectively (table 1). In Manicaland, approximately 75% of deaths were caused by AIDS, compared with 51% in Kisesa 15–44-year-olds and 33% among Kisesa 45–59-year-olds. Herpes zoster, acute respiratory tract infections, abscesses and sores, acute diarrhoea and tuberculosis were all less commonly reported in Kisesa than in Manicaland (table 2). For deaths under the age of 45 years, weight loss, jaundice, tumours, respiratory tract infections and tuberculosis were less common among HIV-positive deaths in Kisesa compared with Manicaland.

    View this table:
    Table 1

    Prevalence of AIDS mortality in Kisesa and Manicaland verbal autopsy subjects

    View this table:
    Table 2

    Sensitivity and specificity of individual signs and symptoms for AIDS deaths in Manicaland and Kisesa cohorts, stratified at age 45 years

    In applying the previously developed algorithm to Kisesa data, the sensitivity in classifying AIDS deaths was low (67%), mainly because of poor sensitivity (46%) in the 45–59-year age group. Based on this observation, the algorithm was re-trained on Manicaland phase I data, restricted to 15–44-year-olds. This resulted in a different ordering of symptoms compared with the original algorithm and in the inclusion of a ninth symptom since the addition of diarrhoea gave a slightly higher mean sensitivity and specificity (nine symptoms: 75.1% compared with eight symptoms: 74.7%). Using this new algorithm, a set of nine criteria with a sensitivity of 75.0% and specificity of 75.2% was produced (figure 1).

    Figure 1
    Figure 1

    Receiver operator characteristic (ROC) curve of algorithm on deaths of 15–44-year-olds from Manicaland phase I.

    The newly trained algorithm performed consistently based on sensitivity and specificity of data for 15–44-year-olds from Manicaland phase I (sensitivity 79%; sensitivity 79%), Manicaland phase II (sensitivity 83%; specificity 75%) and Kisesa (sensitivity 75%; specificity 74%) tests. Although a reasonable specificity was maintained on 45–59-year-olds, the sensitivity dropped markedly in Manicaland phase I (sensitivity 73%; specificity 73%), Manicaland phase II (sensitivity 68%; specificity 80%) and Kisesa (sensitivity 54%; specificity 62%) tests in this older age group (figure 2).

    Figure 2
    Figure 2

    Sensitivity and specificity of the trained algorithm. The dashed line represents the performance of the algorithm on the training dataset. Panels A to D use the full set of nine signs and symptoms. Panels E and F use a smaller set of five signs/symptoms available from the current INDEPTH tool.

    The INDEPTH network is an association of health and DSSs in African, Asian and Oceania countries (http://www.indepth-network.org/). The network has developed a widely used verbal autopsy questionnaire, although the tool does not collect information on herpes zoster, abscesses or sores, vaginal tumours or oral candidiasis (table 3).18 We measured the value of using the five signs/symptoms that are available from the INDEPTH questionnaire. In the 15–44-year-old age group, using only these five signs/symptoms resulted in a sensitivity and specificity of 64% and 82%, respectively (figure 2). The reduction in sensitivity was less than expected, using the same five criteria in phase II Manicaland data and earlier Kisesa data sensitivity had decreased to 50% and 44% respectively.

    View this table:
    Table 3

    Signs and symptoms for surveillance of AIDS mortality, and the availability in INDEPTH—another widely used verbal autopsy questionnaire

    Discussion

    We developed a tool that consistently measures AIDS mortality using verbal autopsy. Through slight modification to our previously proposed criteria, the algorithm performs similarly in Zimbabwe and Tanzania—settings with different HIV prevalence (approximately 20% and 7%, respectively9 10) and AIDS mortality and different distribution of other causes of death. The algorithm is robust in that it performs consistently when prevalence is above approximately 5%.

    This method of measuring AIDS mortality produced reliable estimates only in the 15–44-year age group. This is due to increasing levels of other-cause mortality in older ages. In Manicaland, where the proportion of deaths due to HIV in the older age groups remained high,11 the methods worked well. But in Kisesa, where HIV prevalence is lower, AIDS mortality begins to drop off after 35 years of age, especially in women.6 14 19 20 and other causes also increase in the 45–59-year age groups. Tuberculosis, in particular, reduces the specificity marginally and the sensitivity markedly 21 22 because tuberculosis symptoms overlap substantially with HIV symptoms resulting in misclassification.

    Given the sensitivity and specificity of the method from the training data, we would predict that 88% and 53% of deaths of 15–44-year-olds in Manicaland and Kisesa were AIDS deaths, respectively. (The formulae for this calculation are described by Lopman et al.12) This compares with directly measured values of 76% and 51%. The overestimate in Manicaland is a result the algorithm actually performing better in phase II (higher specificity) than it did on the training dataset. To calculate the prevalence of AIDS death, the level of misclassification is corrected; however, the level of misclassification was actually smaller than calculated on the training data. This approach to estimation of AIDS deaths can be applied to other verbal autopsy data for which a gold standard is not available; however, the accuracy of the prediction will not be known. Therefore, further validation of the algorithm is recommended.

    The INDEPTH verbal autopsy is a widely used tool that used a shorter symptom list than ours and does not collect information to identify herpes zoster, oral candidiasis, abscesses/sores or vaginal tumours.21 These symptoms were commonly reported in our studies and (excluding vaginal tumours) were prevalent in AIDS deaths with a sensitivity of approximately 20% for zoster and abscesses/sores and 40% for oral candidiasis. We found that the INDEPTH shortlist would perform less well and have higher levels of misclassification of AIDS deaths.23 The newly released WHO instrument includes signs and symptoms associated with herpes zoster, abscesses/sores and oral candidiasis.4 In general, verbal autopsy have only proved accurate enough to assign cause of death in very broad categories in adults. However, statistical or algorithmic approaches, such as the method used here, have been shown to perform adequately for determining prevalence of a specific cause, such as HIV. Unfortunately, causes of death other than HIV could not be validated, as neither physician assessment nor diagnostics data were routinely available.

    Our analyses show that in areas of generalised epidemics, verbal autopsy can consistently measure AIDS mortality. Both studies had HIV prevalence levels exceeding 5%, which is commonly found in eastern and southern Africa. In settings where prevalence is less than 5%, getting AIDS mortality from verbal autopsy may be difficult and will require further validation studies. Even in higher prevalence settings, analyses should be restricted to age groups in which competing causes of mortality, especially other infectious causes, are relatively low. Especially for the Tanzanian population, the verbal autopsy method was much less accurate for deaths over the age of 45 years. Based on our analyses, we recommend surveillance of deaths in populations with severe HIV epidemics be undertaken in the 15–44-year age group. Addition of a handful of questions related to opportunistic infections would enable other widely used verbal autopsy tools (WHO and INDEPTH) to apply this validated method in areas where HIV testing and hospital records are unavailable or incomplete.

    What is already known on this topic

    Verbal autopsy is currently the best option for obtaining cause of death information in populations without comprehensive civil registration systems. Verbal autopsy is generally used to assign deaths to broad categories and estimate cause-specific mortality.

    What this study adds

    We demonstrate that verbal autopsy can consistently measure AIDS mortality with a simple set of nine criteria. Surveillance should focus on deaths that occur in the 15–44-year age group for which the method performs reliably. Adding just a handful of questions related to opportunistic infections would enable other widely used verbal autopsy tools to apply this validated method to asses AIDS mortality in areas where HIV testing and hospital records are unavailable or incomplete.

    References

    1. ↵
      1. Quigley MA,
      2. Chandramohan D,
      3. Rodrigues LC
      . Diagnostic accuracy of physician review, expert algorithms and data-derived algorithms in adult verbal autopsies. Int J Epidemiol 1999;28:1081–7.
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. Soleman N,
      2. Chandramohan D,
      3. Shibuya K
      . Verbal autopsy: current pratices and challenges. Bull World Health Organ 2006;84:239–45.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Setel PW,
      2. Sankoh O,
      3. Rao C,
      4. et al
      . Sample registration of vital events with verbal autopsy: a renewed commitment to measuring and monitoring vital statistics. Bull World Health Organ 2005;83:611–17.
      OpenUrlPubMedWeb of Science
    4. ↵
      1. Baiden F,
      2. Bawah A,
      3. Biai S,
      4. et al
      . Setting international standards for verbal autopsy. Bull World Health Organ 2007;85:570–1.
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Porter K,
      2. Zaba B
      . The empirical evidence for the impact of HIV on adult mortality in the developing world: data from serological studies. AIDS 2004;18(Suppl 2):S9–S17.
      OpenUrlCrossRef
    6. ↵
      1. Kamali A,
      2. Wagner HU,
      3. Nakiyingi J,
      4. et al
      . Verbal autopsy as a tool for diagnosing HIV-related adult deaths in rural Uganda. Int J Epidemiol 1996;25:679–84.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Diaz T,
      2. Loth G,
      3. Whitworth J,
      4. et al
      . Surveillance methods to monitor the impact of HIV therapy programmes in resource-constrained countries. AIDS 2005;19(Suppl 2):31–7.
      OpenUrl
    8. ↵
      1. Mathers CD,
      2. Fat DM,
      3. Inoue M,
      4. et al
      . Counting the dead and what they died from: an assessment of the global status of cause of death data. Bull World Health Organ 2005;83:171–7.
      OpenUrlPubMedWeb of Science
    9. ↵
      1. Gregson S,
      2. Garnett GP,
      3. Nyamukapa CA,
      4. et al
      . HIV decline associated with behavior change in eastern Zimbabwe. Science 2006;311:664–6.
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Mwaluko G,
      2. Urassa M,
      3. Isingo R,
      4. et al
      . Trends in HIV and sexual behaviour in a longitudinal study in a rural population in Tanzania, 1994-2000. AIDS 2003;17:2645–51.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Lopman BA,
      2. Barnabas R,
      3. Hallett TB,
      4. et al
      . Assessing adult mortality in HIV-1-afflicted Zimbabwe (1998–2003). Bull World Health Organ 2006;84:189–97.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Lopman BA,
      2. Barnabas RV,
      3. Boerma JT,
      4. et al
      . Creating and validating an algorithm to measure AIDS mortality in the adult population using verbal autopsy. PLoS Med 2006;3:e312.
      OpenUrlCrossRefPubMed
    13. ↵
      1. Ray CS,
      2. Mason PR,
      3. Smith H,
      4. et al
      . An evaluation of dipstick-dot immunoassay in the detection of antibodies to HIV-1 and 2 in Zimbabwe. Trop Med Int Health 1997;2:83–8.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Urassa M,
      2. Boerma JT,
      3. Isingo R,
      4. et al
      . The impact of HIV/AIDS on mortality and household mobility in rural Tanzania. AIDS 2001;15:2017–23.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Ngalula J,
      2. Urassa M,
      3. Mwaluko G,
      4. et al
      . Health services utilization and household expenditure during terminal illness due to HIV/AIDS in rural Tanzania. Trop Med Int Health 2002;7:873–7.
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Boerma JT,
      2. Urassa M,
      3. Nnko S,
      4. et al
      . Sociodemographic context of the AIDS epidemic in a rural area in Tanzania with a focus on people's mobility and marriage. Sex Transm Infect 2002;78(Suppl 1):i97–105.
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. Boerma JT,
      2. Urassa M,
      3. Senkoro K,
      4. et al
      . Spread of HIV infection in a rural area of Tanzania. Aids 1999;13:1233–40.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      INDEPTH Standardised VA questionnaire. Accessed May 2009. http://www.indepth-network.org/.
    19. ↵
      1. Tollman SM,
      2. Kahn K,
      3. Garenne M,
      4. et al
      . Reversal in mortality trends: evidence from the Agincourt field site, South Africa, 1992–1995. AIDS 1999;13:1091–7.
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Hosegood V,
      2. Vanneste AM,
      3. Timaeus IM
      . Levels and causes of adult mortality in rural South Africa: the impact of AIDS. AIDS 2004;18:663–71.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Groenewald P,
      2. Nannan N,
      3. Bourne D,
      4. et al
      . Identifying deaths from AIDS in South Africa. AIDS 2005;19:193–201.
      OpenUrlPubMedWeb of Science
    22. ↵
      1. Groenewald P,
      2. Bradshaw D,
      3. Dorrington R,
      4. et al
      . Identifying deaths from AIDS in South Africa: an update. AIDS 2005;19:744–45.
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      World Health Organization. WHO case definitions for AIDS surveillance in adults and adolescents. Wkly Epidemiol Rec 1994;69:273–5.
      OpenUrlPubMed

    Footnotes

    • Funding The comparative study presented here was funded by WHO Health Metrics Network. The Manicaland Study is principally funded by the Wellcome Trust, and the Kisesa study is currently funded by the Global Fund for AIDS, TB and Malaria and by the Wellcome Trust; prior to 2004, it was mainly funded by the Netherlands government through the TANESA programme. Other Funders: Wellcome Trust.

    • Competing interests None.

    • Ethics approval Ethical approval for the DSS and all related procedures (such as the verbal autopsy interviews) was granted by the Tanzanian Medical Research Co-ordinating Committee for the Kisesa study and by the Zimbabwe Medical Research Council for the Manicaland Study.

    • Patient consent Obtained.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.