Ensemble Methods | Foundations and Algorithms | Taylor & Francis GroupBagging ensemble selection BES is a relatively new ensemble learning strategy. The strategy can be seen as an ensemble of the ensemble selection from libraries of models ES strategy. Previous experimental results on binary classification problems have shown that using random trees as base classifiers, BES-OOB the most successful variant of BES is competitive with and in many cases, superior to other ensemble learning strategies, for instance, the original ES algorithm, stacking with linear regression, random forests or boosting. Motivated by the promising results in classification, this paper examines the predictive performance of the BES-OOB strategy for regression problems. Our results also suggest that the advantage of using a diverse model library becomes clear when the model library size is relatively large. We also present encouraging results indicating that the non-negative least squares algorithm is a viable approach for pruning an ensemble of ensembles.
Scikit Learn Ensemble Learning, Bootstrap Aggregating (Bagging) and Boosting