Notions of Model Complexity
Determining an appropriate measure for model complexity is nuanced and varies across different classes of machine learning models. While complexity is frequently associated with the number of parameters—since models with more parameters can typically fit more arbitrarily assigned labels—this is not always true. For example, kernel methods can operate in spaces with an infinite number of parameters, yet their complexity is controlled through other mechanisms. A more useful notion of complexity often involves the range of values that parameters can take; a model whose parameters can take arbitrary values is considered more complex. Consequently, comparing complexity between substantially different model classes, such as decision trees and neural networks, can be difficult.
0
1
Tags
D2L
Dive into Deep Learning @ D2L
Related
Overfitting a supervised statistical model
Training Error and Test Error
Generalizability of a supervised statistical model
Underfitting a supervised statistical model
Measuring Model Complexity: Rademacher complexity
Bias of Supervised Models in Statistical Learning
Variance of Supervised Models in Statistical Learning
Falsifiability of Machine Learning Models
Notions of Model Complexity
Relationship Between Dataset Size and Model Complexity