Linear Regression Model

Definition

Linear regression is a model that estimates the relationship (effect) between a scalar response and one or more independent variables.

In the case of just one independent variable (regressor) we call such a model Simple Linear Regression. If there are more than one regressor then we call the model Multiple Linear Regression. This term should not be confused with Multivariate Linear Regression

If the $Y$ is not a scalar, rather a vector then we deal with General Linear Model

One might show a model in the following way:

$$Y=X^{T}B+\epsilon$$ $$Y={\beta }_0+{\beta }_1{x}_1\cdots +{\beta }_n{x}_n+\epsilon$$

where $E\sim NID(0,{\sigma }^2)$.

As far as I understood one can state the model as:

$$Y={\beta }_0+{\beta }_1{\varphi }_1(x_1)\cdots +{\beta }_n{\varphi }_n(x_n)+\epsilon$$

where ${\varphi }_n(x_n)$ can be a non-linear function of $x_n$.

R

We use lm() function in R to create a linear regression model.

results <- lm(Y~X) # returns parameter estimates (b0, b1, sigma)