Logistic Regression Model

A multinomial logistic model classifies $d$-dimensional real-valued input vectors $x \in \mathbb{R}^d$ into one of $k$ outcomes $c \in \{ 0,\ldots, k-1 \}$ using $k-1$ parameter vectors $\beta_0,\ldots,\beta_{k-2} \in \mathbb{R}^d$:

$$p(c \mid x, \beta) = \left\{ \begin{array}{cl} \frac{\textst... ...{\textstyle 1}{Z_x} & \textrm{if } c = k-1 \end{array} \right.$$ (1)

where the linear predictor is inner product:

$$\beta_c \cdot x = \sum_{i < d} \beta_{c,i} \cdot x_{i}$$ (2)

The normalizing factor in the denominator is the partition function:

$$Z_x = 1 + \sum_{c < k-1} \exp(\beta_c \cdot x)$$ (3)