# Model estimation

Model estimation is the use of statistical analysis techniques to find parameters that most likely explain observed data. Model estimation is a component of Model Calibration and Validation.

## # Notation

Consider a statistical process where an outcome *linear* equation,

where *residual* term

The notation for this equation can be simplified by using a matrix

$$ X = \begin{vmatrix} 1 & x_{11} & x_{21} & \ldots & x_{p1}\ 1 & x_{12} & x_{22} & \ldots & x_{p2}\ \vdots & \vdots & \vdots & & \vdots\ 1 & x_{1n} & x_{2n} & \ldots & x_{pn}\ \end{vmatrix} $$

The linear equation above then becomes *model estimation* is to find estimates of

### # Ordinary Least Squares

Assume we have a linear equation *sum of squared residuals*, or the distance between

If we take the derivative of this sum with respect to

This estimator is referred to as the *Ordinary Least Squares* (OLS) estimator.
If we assume that the residuals

### # Maximum Likelihood Estimation

OLS is powerful and adequate in many situations; however, there may be cases where
the assumptions of OLS modeling (normally distributed *maximum likelihood estimation* (MLE).

In a linear model, we assume that the points follow a normal (Gaussian) probability
distribution, with mean

What we want to find is the parameters *maximize* this
probability for all points

For various reasons, it's easier to use the log of the likelihood function:

$$\log(\mathcal{L}) = \sum_{i = 1}^n-\frac{n}{2}\log(2\pi) -\frac{n}{2}\log(\sigma^2) - \frac{1}{2\sigma^2}(y - X\beta)^2 $$

Most MLE programs work by having a computer attempt to find values of