ML-2 Note: Linear Models for Classification and GLMs

Sat, 28 Mar 2026 02:24:29 +0800

ML-2 Note: Linear Models for Classification and GLMs

1. Why Classification Uses Logistic Regression

In classification problems, the target variable is discrete.

Binary classification example:

$$y \in \left\{ 0,1 \right\}$$

Given input features

$$x \in \mathbb{R}^d$$

we want to model

$$P(y=1|x)$$

Problem with Linear Regression

A linear model predicts

$$f(x) = w^T x$$

but

$$w^T x \in (-\infty, \infty)$$

while probabilities must satisfy

$$P(y=1|x) \in [0,1]$$

Thus we need a function that maps

ML-1 Note: Supervised Learning; Linear Regression

Sun, 22 Mar 2026 03:32:29 +0800

ML-1 Note: Supervised Learning; Linear Regression

1. Basic Model of Linear Regression

Linear Regression is one of the simplest and most fundamental models in supervised learning.
The goal is to model the relationship between input features and a continuous target variable.

Model Form

For a dataset with feature vector $x$:

\[ y = w^T x + b \]

or equivalently

\[ \hat{y} = \theta^T x \]

where:

$x$: input feature vector
$w$: weight vector
$b$: bias term
$\theta$: parameter vector (including bias)
$\hat{y}$: predicted value

Loss Function

The most common loss function for linear regression is Mean Squared Error (MSE).

Machine Learning on Mike's Blog

ML-2 Note: Linear Models for Classification and GLMs

ML-2 Note: Linear Models for Classification and GLMs

1. Why Classification Uses Logistic Regression

Problem with Linear Regression

ML-1 Note: Supervised Learning; Linear Regression

ML-1 Note: Supervised Learning; Linear Regression

1. Basic Model of Linear Regression

Model Form

Loss Function