Regression

tags: machine learning

Feature engineering

To be possible to get better fitting hypothesis, we can engineer feature, for example: Let’s say we don’t want the hypothesis to be a straight line to get a better fit. We can create a new feature $x_2$ $X_2=X_1^2$ or $X_2=\sqrt{x_1}$

Locally weighted regression

Here we take into consideration the neighboring data, our cost function becomes $\sum w^{(i)}(y^{(i)}-{\theta }^T{x}^{(i)})$ where: $w^{(i)}=\exp (-\frac{(x^{(i)}-x)}{{\tau }^2})$ And $\tau$ here gives us wide is the bell This makes the prediction be highly dependent on the nearest points since they have the largest weights

Logistic Regression

The Sigmoid Function

Squishes everything to be from 0 to 1 and is used in classification algorithms. Also known as logistic function $G(x)=\frac{1}{1+e^{-x}}$ In Logistic regression we use the sigmoid function and set ${\theta }^{T}x$ as the parameter of the function as it is what controls the narrowness of the sigmoid

Terminology

• Parametric learning algorithm: You fit a fixed size parameters to data, eg: ${\theta }_i$ to fit a straight line
• non-Parametric learning algorithm Parameters grows linearly with the size of data