Comparing methods to predict baseline mortality for excess mortality calculations.
Excess mortality
Prediction
Robustness
Spline regression
Journal
BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545
Informations de publication
Date de publication:
18 10 2023
18 10 2023
Historique:
received:
08
03
2023
accepted:
09
10
2023
medline:
23
10
2023
pubmed:
19
10
2023
entrez:
18
10
2023
Statut:
epublish
Résumé
The World Health Organization (WHO)'s excess mortality estimates presented in May 2022 stirred controversy, due in part to the high estimate provided for Germany, which was later attributed to the spline model used. This paper aims to reproduce the problem using synthetic datasets, thus allowing the investigation of its sensitivity to parameters, both of the mortality curve and of the used method, thereby shedding light on the conditions that gave rise to this error and identifying possible remedies. A negative binomial model was used accounting for long-term change, seasonality, flu seasons, and heat waves. Simulated mortality curves from this model were then analysed using simple methods (mean, linear trend), the WHO method, and the method of Acosta and Irizarry. The performance of the WHO's method with its original parametrization was indeed very poor, however it can be profoundly improved by a better choice of parameters. The Acosta-Irizarry method outperformed the WHO method despite being also based on splines, but it was also dependent on its parameters. Linear extrapolation could produce very good results, but was highly dependent on the choice of the starting year, while the average was the worst in almost all cases. Splines are not inherently unsuitable for predicting baseline mortality, but caution should be taken. In particular, the results suggest that the key issue is that the splines should not be too flexible to avoid overfitting. Even after having investigated a limited number of scenarios, the results suggest that there is not a single method that outperforms the others in all situations. As the WHO method on the German data illustrates, whatever method is chosen, it remains important to visualize the data, the fit, and the predictions before trusting any result. It will be interesting to see whether further research including other scenarios will come to similar conclusions.
Sections du résumé
BACKGROUND
The World Health Organization (WHO)'s excess mortality estimates presented in May 2022 stirred controversy, due in part to the high estimate provided for Germany, which was later attributed to the spline model used. This paper aims to reproduce the problem using synthetic datasets, thus allowing the investigation of its sensitivity to parameters, both of the mortality curve and of the used method, thereby shedding light on the conditions that gave rise to this error and identifying possible remedies.
METHODS
A negative binomial model was used accounting for long-term change, seasonality, flu seasons, and heat waves. Simulated mortality curves from this model were then analysed using simple methods (mean, linear trend), the WHO method, and the method of Acosta and Irizarry.
RESULTS
The performance of the WHO's method with its original parametrization was indeed very poor, however it can be profoundly improved by a better choice of parameters. The Acosta-Irizarry method outperformed the WHO method despite being also based on splines, but it was also dependent on its parameters. Linear extrapolation could produce very good results, but was highly dependent on the choice of the starting year, while the average was the worst in almost all cases.
CONCLUSIONS
Splines are not inherently unsuitable for predicting baseline mortality, but caution should be taken. In particular, the results suggest that the key issue is that the splines should not be too flexible to avoid overfitting. Even after having investigated a limited number of scenarios, the results suggest that there is not a single method that outperforms the others in all situations. As the WHO method on the German data illustrates, whatever method is chosen, it remains important to visualize the data, the fit, and the predictions before trusting any result. It will be interesting to see whether further research including other scenarios will come to similar conclusions.
Identifiants
pubmed: 37853374
doi: 10.1186/s12874-023-02061-w
pii: 10.1186/s12874-023-02061-w
pmc: PMC10585880
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
239Informations de copyright
© 2023. BioMed Central Ltd., part of Springer Nature.
Références
Nat Commun. 2021 Jun 18;12(1):3755
pubmed: 34145260
Popul Dev Rev. 2022 Jun;48(2):279-302
pubmed: 35600716
Nat Commun. 2021 May 12;12(1):2729
pubmed: 33980836
Lancet Planet Health. 2018 Nov;2(11):e478-e488
pubmed: 30318387
Stat Med. 2019 Oct 15;38(23):4545-4554
pubmed: 31321799
Int J Infect Dis. 2019 Nov;88:127-134
pubmed: 31401203
Lancet. 2020 May 2;395(10234):e81
pubmed: 32333839
J R Soc Med. 2020 Sep;113(9):329-334
pubmed: 32910871
J Infect. 2020 Nov;81(5):797-801
pubmed: 32956730
JAMA. 2021 Feb 23;325(8):785-787
pubmed: 33325994
MMWR Morb Mortal Wkly Rep. 2021 Aug 20;70(33):1114-1119
pubmed: 34411075
Lancet. 2020 Jun 13;395(10240):1829
pubmed: 32473099
Int J Epidemiol. 2020 Aug 1;49(4):1059-1062
pubmed: 32601669
Br J Gen Pract. 2020 Nov 26;70(701):e890-e898
pubmed: 33077508
J Epidemiol Community Health. 2017 Oct;71(10):974-980
pubmed: 28830951
Nat Med. 2020 Dec;26(12):1919-1928
pubmed: 33057181
Nature. 2023 Jan;613(7942):130-137
pubmed: 36517599
Am J Public Health. 1997 Dec;87(12):1944-50
pubmed: 9431281
Sci Rep. 2022 Mar 31;12(1):5475
pubmed: 35361826
Eur J Public Health. 2021 Feb 1;31(1):17-22
pubmed: 33169145
Int J Epidemiol. 2021 Jan 23;49(6):1909-1917
pubmed: 33053172
Int J Infect Dis. 2021 Jun;107:101-115
pubmed: 33862214
JAMA. 2022 Jul 5;328(1):74-76
pubmed: 35594035
S Afr Med J. 2021 May 21;111(8):732-740
pubmed: 35227353
BMJ. 2021 May 19;373:n1137
pubmed: 34011491
Isr J Health Policy Res. 2021 Feb 26;10(1):17
pubmed: 33637126
Epidemiol Infect. 2020 Oct 29;148:e264
pubmed: 33115546
Nat Commun. 2022 Jan 25;13(1):482
pubmed: 35079022
Signif (Oxf). 2021 Feb;18(1):16-19
pubmed: 33821160
J Epidemiol Glob Health. 2020 Sep;10(3):209-213
pubmed: 32954711
Nature. 2022 May;605(7909):206
pubmed: 35513455
Lancet. 2020 Sep 12;396(10253):e27-e28
pubmed: 32891216
Elife. 2021 Jun 30;10:
pubmed: 34190045
Nature. 2022 Jun;606(7913):242-244
pubmed: 35650295
Epidemiology. 2022 May 1;33(3):346-353
pubmed: 35383642
BMC Public Health. 2020 Aug 14;20(1):1238
pubmed: 32795276
Lancet. 2022 Apr 16;399(10334):1513-1536
pubmed: 35279232