https://towardsdatascience.com/decision-tree-classification-de64fc4d5aac

University of Michigan - Ann Arbor

Apply the decision tree method to regression problems. It is quite similar to regression trees except that a qualitative, not quantitive, response is needed. This leads to following differences:
* The criterion for making the binary splits is no longer RSS.
* The most commonly class in the region, rather than the mean value, is used as the prediction result. 
Classification Tree is used to predict qualitative variables instead of quantitative variables. It is very similar to regression tree and use recursive binary splitting to generate the classification tree. But use *Gini index* 
$$ G = \sum_{k=1}^{K} \hat{p}_{mk} (1-\hat{p}_{mk}) $$
and *entropy* as the indicator. 
$$ D = -\sum_{k=1}^{K} \hat{p}_{mk} log \hat{p}_{mk} $$

The small value of purity of node indicated that a node contains observations from almost the same category. *Entropy* is the same as Gini Index, if the $m$th node is pure, the *entropy* is very small. 

Classification trees

Unlike regression trees, RSS cannot be used as a criterion for making binary splits in classification trees since the most commonly classes, rather than mean values, are used as the predictions. Three criteria that can be used for making binary splits in classification trees are

- classification error rate
- Gini index
- entropy

Criteria for making binary splits in a classification tree

Step 1: Change numerical variable into categorical variable
You have two options, 
if it's binary(this will add a new column):
newcolumnname=ifelse(columnname <=threshold,"No","Yes")
newdataset =data.frame(dataset ,newcolumnname) used to merge the column with original dataset

if it's multi-category(this will change original column):
data$column[criterion] <- 'category name1'
data$column[criterion] <- 'category name2'

Step 2: Build a model 

To build a decision tree,we use function tree( )
model<-tree(formula=outputcolumn~., data=dataframe)


Step 3: Predict on test data
tree.pred=predict(model ,testdata,type="class")
table(tree.pred,testlabel)

Step 4:Consider whether prune can lead to improved result
The function cv.tree() performs cross-validation in order to
determine the optimal level of tree complexity; We use the argument FUN=prune.misclass in order to indicate that we want the classiﬁcation error rate to guide the cross-validation and pruning process, rather than the default for the cv.tree() function, which is deviance.

cv.carseats =cv.tree(tree.carseats ,FUN=prune.misclass )





Learn Before

Related