I read the concept of statistical distance and understand somewhat. Given the point $P(x_{1},x_{2})$ in 2-D space, the euclidean distance of point P from origin is given by:
$d(O,P) = \sqrt{\frac{x_{1}^2}{s_{11}}+ \frac{x_{2}^2}{s_{22}}}$
where
$s_{11}$ and $s_{22}$ are the variance of points along x1 and x2 direction, assuming the x1 and x2 are independent.
What will be the generalized form for computing statistical distance in the given space using any $l_{p}-norm$ ?
$\endgroup$ 21 Answer
$\begingroup$This is basically using a vector from $z$-scores, with the assumption that the mean of each coordinate is $0$.
In $\mathbb{R}^n$ we get:
$$\vec{z}:=(\frac{x_1}{\sigma_{1}},\frac{x_2}{\sigma_{2}},...,\frac{x_n}{\sigma_{n}})$$
Where $\sigma_i$ is the standard deviation along direction $i$.
In this case, we can re-cast $d(O,P)$ as:
$$d(O,P)=||\vec{z}||_2$$
The generalization to any $L^p$-norm is obvious:
$$d(O,P)=||\vec{z}||_p$$
Extension to Correlated Variables
Note the relationship of the above $L^2$ norm to the quadratic form in the exponent of the multivariate gaussian with uncorrelated components. This suggested that we can model the general $L^2$ statistical distance using this quadratic form. Let $\Sigma$ be the covariance matrix of your variables, $\mu$ be the location of point that you want to calculate the distance to (from $x$) (in your post, you chose the origin $\mu = 0$). The the $L^2$ norm in your post can be rewritten as:
$$d(O,P):= \sqrt{x^T\Sigma^{-1} x}$$
Where $\Sigma$ is a diagonal matrix of the variances. You can incorporate correlated variables using the full covariance matrix.
Not sure if there's a natural extension to $L^p$
$\endgroup$ 3
