Mahalanobis Distance

Definition

Let X be the probability distribution, then

$$d(P,Q;X)=(P-Q{)}^T{\Sigma }_X^{-1}(P-Q)$$

and,

$$d(P;X)=d(P,\bar{x};X)$$

See statistical distance below.

Statistical Distance

Definition

Assume that $x_1\dots x_n$ are independent. In statistical distance we give importance to each axis that is inversely proportional to the standart deviation in the axis.

$$d(p,q)=\sqrt{\sum _{i=1}^n\frac{(p_i-q_i{)}^2}{s_{ii}}}$$

where $s$ is the variance of $i$th observation. Actually such distance can be stated as

$$d(p,q)=\sqrt{\sum _{i=1}^n{\left(\frac{p_i-q_i}{\sqrt{s_{ii}}}\right)}^2}$$

where $\frac{p_i-q_i}{\sqrt{s_{ii}}}$ is the standardized coordinate in that axis.

Now consider that $x_1\dots x_n$ are not independent. Then we rotate the coordinate system $(x_1,x_2)$ with an angle of $\theta$ so that

$$\begin{array}{rl}\widetilde{x_1}& =x_1\cos \theta +x_2\sin \theta \\ -\widetilde{x_2}& =x_1\sin \theta +x_2\cos \theta \end{array}$$

so that

$$d(0,p)=\sqrt{\sum _{i=1}^k\frac{{\widetilde{x_i}}^2}{{\tilde{s}}_{ii}}}$$

where ${\tilde{s}}_{ii}$ is the sample variance computed with respect to ${\tilde{x}}_i$‘s.