One downside of bagging is the tradeoff between prediction accuracy and model interpretability. Although bagging increases prediction accuracy, it does so at the expense of interpretability. We can no longer represent a bagged model as a single tree. It is also no longer clear which variables are most important to the model once bagging has been integrated.

However, there are ways to determine the most important predictors. For regression trees, we can record the total amount the RSS is decreased as a result of the splits for a specific predictor. This value is then averaged over all $B$ trees and the predictor is considered important if that value is large. Likewise, with classification trees, we can record the total amount that the Gini index is decreased as a result of the splits for a specific predictor, and average it over $B$ trees.