Learn Before
Local Constancy and Smoothness Priors
Local constancy and smoothness priors are mostly used "priors". they are the methods to ensure the learned functions are locally constant or smooth. They need to be f(x)=f(x+c). For example, k-nearest neighbor copy directly data from nearby while most kernel interpolate between nearby data.
The smoothness works best for function with enough examples and do not have lots of dimensions. However, there are methods to represent and estimate complicated functions.
Complicated functions can be estimated and generalized well- by dividing regions and use underlying assumption to associate each regions. For example, a function with O(2^k) regions can be defined by O(k) examples and use additional assumptions to connect these regions(Chapter 2.5.11).
0
2
Tags
Data Science
Related
Why does regularization prevent overfitting?
Popular Regularization Techniques in Deep Learning
Human Level Performance: Based on the evidence below, which two of the following four options seem the most promising to try?
Local Constancy and Smoothness Priors
Parameter Sharing
Parameter Tying
L1 regularization and L2 regularization
MTL as a Regularizer
Parameter Penalties in Classical Regularization