Method of moments

Definition

If $X_1\dots X_n=\tilde{X}$ is random sample from $f(x;\theta )$ then the $k^{th}$ moment of the sample is given by

$$m_k=E[(X-\mu {)}^k]=\sum _i\frac{x_i^k}{n}$$

Number of moments to calculate

You need $n$ moments if you wish to estimate $n$ parameters.

Now let us walk through an example.

Example

Consider a rv with mean $\mu$ and variance ${\sigma }^2$. Now we say that we have two unknowns $(\mu ,{\sigma }^2)=({\theta }_1,{\theta }_2)$. So we need to consider moments up to second order.

Let us first calculate the sample moments.

\begin{align*} m_{1} &= \sum \frac{x_{i}}{n} \\ m_{2} &= \sum \frac{x_{i}^2}{n}

\end{align*}

$$Nowthetheoreticalmoments$$

\begin{align*} E[X] &= \mu \ E[X^2] &= \sigma^2 + \mu^2 \end{align*}

$$Nowequatethesetwo.$$

\begin{align*} E[X] = m_{1} &\implies \hat{\mu}{MME} = \sum \frac{x{i}}{n} \ E[X^2] = m_{2} &\implies \hat{\sigma}^2 + \hat{\mu}^2 = \sum \frac{x_{i}^2}{n} \ &\implies \hat{\sigma^2} = \sum \frac{x_{i}^2}{n} - \hat{\mu}^2\ &\implies \hat{\sigma^2} = \sum \frac{x_{i}^2}{n} - \left( \sum \frac{x_{i}}{n} \right)^2 \ &\implies \hat{\sigma^2}{MME} = \sum \frac{(x{i} - \bar{x})^2}{n} \end{align*}

For another example see gamma distribution.