Concept

Constant Attention Kernel Estimate

When using a constant attention kernel in Nadaraya-Watson regression, the resulting estimate is simply the global average of all training labels, given by the trivial formula f(x)=1niyif(x) = \frac{1}{n} \sum_i y_i. Because it assigns equal weight to every data point regardless of its distance from the query, this trivial estimate produces an unrealistic result that fails to capture local variations in the underlying data.

0

1

Updated 2026-05-14

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L