A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection

Galipaud M, Gillingham MAF, Dechaume-Moncharmont F-X (2017)
METHODS IN ECOLOGY AND EVOLUTION 8(12): 1668-1678.

Download
No fulltext has been uploaded. References only!
Journal Article | Original Article | Published | English

No fulltext has been uploaded

Author
; ;
Abstract
1. In a previous article, we advocated against using the sum of Akaike weights (SW) as a metric to distinguish between genuine and spurious variables in Information Theoretic (IT) statistical analyses. A recent article (Giam & Olden, Methods in Ecology and Evolution, 2016, 7, 388) criticises our finding and instead argues in favour of SW. It points out that (1) we performed a biased data-generation procedure and (2) we erroneously evaluated SW on its capacity to estimate the proportion of variance in the data explained by a variable. We here respond to these points. 2. Giam and Olden's first concern is unfounded. When using the data-generating code they proposed, SW remains very imprecise. To respond to their second concern, we first list the meanings taken by a variable's importance in the context of IT. Although, SW is presented as an estimate of variable relative importance in methodological textbooks (i.e. a variable's rank in importance or its relative contribution to the variance in the data), it is also used as a metric of variable absolute importance (i.e. a variable's absolute effect size or its statistical significance). We then compare SW to alternative metrics on its ability to estimate variable absolute or relative importance. 3. SW values have low repeatability across analyses. As a result, based on SW, it is hard to distinguish between variables with weak and large effects. For estimations of variable absolute importance, experimenters should prefer model-averaged parameter estimates and/or compare nested models based on evidence ratios. Sum of Akaike weights is also a poor metric of variable relative importance. We showed that correct variable ranking in importance was generally more frequent when using model-averaged standardised parameter estimates, than when using SW. 4. To avoid recurrent errors in ecology and evolution, we therefore warn against the use of SW for estimations of variable absolute and relative importance and we propose that experimenters should instead use model-averaged standardised parameter estimates for statistical inferences.
Publishing Year
ISSN
eISSN
PUB-ID

Cite this

Galipaud M, Gillingham MAF, Dechaume-Moncharmont F-X. A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection. METHODS IN ECOLOGY AND EVOLUTION. 2017;8(12):1668-1678.
Galipaud, M., Gillingham, M. A. F., & Dechaume-Moncharmont, F. - X. (2017). A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection. METHODS IN ECOLOGY AND EVOLUTION, 8(12), 1668-1678. doi:10.1111/2041-210X.12835
Galipaud, M., Gillingham, M. A. F., and Dechaume-Moncharmont, F. - X. (2017). A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection. METHODS IN ECOLOGY AND EVOLUTION 8, 1668-1678.
Galipaud, M., Gillingham, M.A.F., & Dechaume-Moncharmont, F.-X., 2017. A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection. METHODS IN ECOLOGY AND EVOLUTION, 8(12), p 1668-1678.
M. Galipaud, M.A.F. Gillingham, and F.-X. Dechaume-Moncharmont, “A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection”, METHODS IN ECOLOGY AND EVOLUTION, vol. 8, 2017, pp. 1668-1678.
Galipaud, M., Gillingham, M.A.F., Dechaume-Moncharmont, F.-X.: A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection. METHODS IN ECOLOGY AND EVOLUTION. 8, 1668-1678 (2017).
Galipaud, Matthias, Gillingham, Mark A. F., and Dechaume-Moncharmont, Francois-Xavier. “A farewell to the sum of Akaike weights: The benefits of alternative metrics for variable importance estimations in model selection”. METHODS IN ECOLOGY AND EVOLUTION 8.12 (2017): 1668-1678.
This data publication is cited in the following publications:
This publication cites the following data publications:

Export

0 Marked Publications

Open Data PUB

Web of Science

View record in Web of Science®

Search this title in

Google Scholar