Concept

Notions of Model Complexity

Determining an appropriate measure for model complexity is nuanced and varies across different classes of machine learning models. While complexity is frequently associated with the number of parameters—since models with more parameters can typically fit more arbitrarily assigned labels—this is not always true. For example, kernel methods can operate in spaces with an infinite number of parameters, yet their complexity is controlled through other mechanisms. A more useful notion of complexity often involves the range of values that parameters can take; a model whose parameters can take arbitrary values is considered more complex. Consequently, comparing complexity between substantially different model classes, such as decision trees and neural networks, can be difficult.

0

1

Updated 2026-05-03

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L