PUB - Publikationen an der Universität Bielefeld

Supervised machine learning is a popular tool for predictor selection in large data sets. The choice of method is currently either based on theory, subjective preferences, previous general simulation studies, or the best fit of a method to the data, which does not allow evaluating the methods’ performance for a specific empirical application. In this article, we illustrate how to select a method through a simulation study tailored to an empirical application. The simulation study mimics cancer data with the aim of predicting patients’ quality of life. We vary sample sizes, effect sizes, correlation and interaction structures. The methods include seven parametric (OLS and ridge regression, the lasso, the all-pairs lasso, forward, backward, and hybrid stepwise regression) and four non-parametric (regression tree, random forests, bagging, and boosting). Results show that forward stepwise regression, the lasso, the all-pairs lasso, bagging, and boosting outperform the other methods throughout different design conditions. On the empirical data the all-pairs lasso performed best. In sum, this illustration can be used as an orientation for applied researchers investigating supervised machine learning methods in a specific empirical application. We offer code for R, detailed results, and sample data to replicate the analysis discussed here.