Kernels are used as a computational “trick” to compute the dot product of two vectors in some new feature space. Kernels can be generally thought of as a measure of distance, and are denoted by $K(\vec{x}_1, \vec{x}_2)$. When we look at the dual formulation of the SVM problem, we see that our feature vectors are now isolated in a dot product. By switching out this dot product for the term $K(\vec{x}^i, \vec{x}^j)$, we can learn non-linear decision boundaries in our original feature space.

For anyone studying math, notice how similar this is to the general notion of an inner-product, however be cautious because the kernel need-not be an inner product.

We can also think of kernels as an implicit feature mapping. This is what is meant by a computational trick (see child node on "Proof of Quadratic Kernel" to see the relationship between one kernel and its explicit feature mapping).