$$5 \leq K \leq 10$$ is commonly used in Macine Learning. A too-small $K$ may lead to high bias, and a too-large $K$ may result in high variance. 

University of Michigan - Ann Arbor

To perform K-fold CV on a dataset with $$n$$ observations, the training set is separated into $$k$$ sets by randomly sampling $$n/k$$ observations from the dataset into each fold. 
The model of interest is then trained on $$k−1$$ folds and validated on the left-out fold, i.e., once the model is trained, we use it to make predictions on the group that was left out of training. This process repeats $n$ times. To get the model performance, we average the results from all of the tests. K-Fold CV proves a good measure of the empirical testing error of a model. 


K-Fold Cross-Validation

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

When selecting $K$ for K-Fold CV, we should consider a bias-variance trade-off. K-Fold CV provides $K$ estimates of the model error. The mean of the errors indicates the model's bias. A lower mean value indicates a higher accuracy of the model. The model variance is calculated based on the standard deviation of the errors. A lower variance indicates a low variation in the model performance. Usually, it is difficult to reduce both model bias and variance.

K-Fold Cross-Validation Bias-Variance Tradeoff

K-Fold CV is usually the method of choice for CV, and functions as a sort of generalization of LOOCV, because:
1. It is much less computationally expensive than LOOCV.
2. It minimizes the **Bias-Variance Tradeoff**.

Learn Before

Related