Desktop Survival Guide
by Graham Williams
A good introduction is available from http://www.idiap.ch/~bengio/lectures/tex_ensemble.png
Bagging is bootstrap aggregation. The underlying idea is that part of the error due to variance in building a model comes from the specific choice of the training dataset. So create many similar training data sets, and for each of them train a new function. The final function will then be the average of each functions output.