Machine Learning

ML-2 Note: Linear Models for Classification and GLMs 1. Why Classification Uses Logistic Regression In classification problems, the target variable is discrete. Binary classification example: $$y \in \left\{ 0,1 \right\}$$ Given input features $$x \in \mathbb{R}^d$$ we want to model $$P(y=1|x)$$ Problem with Linear Regression A linear model predicts $$f(x) = w^T x$$ but $$w^T x \in (-\infty, \infty)$$ while probabilities must satisfy $$P(y=1|x) \in [0,1]$$ Thus we need a function that maps ...

ML-1 Note: Supervised Learning; Linear Regression 1. Basic Model of Linear Regression Linear Regression is one of the simplest and most fundamental models in supervised learning. The goal is to model the relationship between input features and a continuous target variable. Model Form For a dataset with feature vector \(x\): \[ y = w^T x + b \] or equivalently \[ \hat{y} = \theta^T x \] where: \(x\): input feature vector \(w\): weight vector \(b\): bias term \(\theta\): parameter vector (including bias) \(\hat{y}\): predicted value Loss Function The most common loss function for linear regression is Mean Squared Error (MSE). ...