Concept

Regression Tree

The predictor space (X1,X2,,XpX_1, X_2, …, X_p) is divided into J non-overlapping regions. Then make the same prediction for each observation that falls into the region RjR_j, and the predicted value is the mean of the response values for the training observations in RjR_j. The goal is to find the matrix region that minimizes the RSS of the model j=1JiRj(yiy^Rj)2\sum_{j=1}^{J} \sum_{i \in R_j} (y_i - \hat{y}_{R_j})^2.

Recursion Binary Splitting: from the top of the tree (top-down), the prediction space is split to two new branches, and the determination of the optimal (best) split is limited to a particular step instead of some future step.

Cost Complexity pruning, as known as Weakest Link Pruning, is able to avoid overfitting. m=1Ti:xiRm(yiy^Rm)2+αT\sum_{m=1}^{|T|} \sum_{i:x_i \in {R_m}} (y_i - \hat{y}_{R_m})^2 + \alpha |T|

1

4

Updated 2020-03-06

Tags

Data Science