To estimate class probabilities across multiple categories, a classification model requires an output for each class. Using a linear model, this is achieved by defining an affine function for each output. For $$d$$ input features and $$q$$ output categories, the unnormalized outputs (logits) $$\mathbf{o}$$ are computed as $$\mathbf{o} = \mathbf{W}\mathbf{x} + \mathbf{b}$$, where the weight matrix $$\mathbf{W} \in \mathbb{R}^{q 	imes d}$$ contains $$q 	imes d$$ scalars and the bias $$\mathbf{b} \in \mathbb{R}^q$$ contains $$q$$ scalars. Because every output depends on every input feature, this computation represents a fully connected layer in a single-layer neural network.

Linear Model for Multi-Class Classification

Because most classification problems lack a natural ordering among classes, categorical labels are typically represented using a one-hot encoding. A one-hot encoding is a vector with as many components as there are categories, where the element corresponding to the true category is set to $$1$$ and all other elements are set to $$0$$. For example, a label for three classes might be represented as the three-dimensional vector $$(1, 0, 0)$$.

Claude

Supervised learning problems are divided into classification and regression. For classification problems, the predicted value is discrete value; For regression problems, the predicted values are continuous values.

Classification and Regression

Dive into Deep Learning

When the categories in a classification problem possess a natural ordering—such as predicting sequential age groups like baby, toddler, adolescent, and adult—it is appropriate to represent the labels as integers $$y \in \{1, 2, 3, \dots\}$$. This setup can be cast as an ordinal regression problem, which preserves the inherent ranking among the classes rather than treating them as entirely independent categories.

Learn Before

Related

Learn After