11 January, 2019

Relating Random Variables and their Statistics to Inner Products

Let us consider a typical real-valued random variable $Y$ defined on a probability space $(\Omega,\mathcal{F},\mathbb{P})$.  Let $X$ be another such random variable, and assume $X,Y\in L^{2}(\mathcal{F}$).  We emphasize the $\sigma$-algebra $\mathcal{F}$ because this will be the structural element of our probability space subject to frequent change; in particular, the measure and sample space $\Omega$ will remain constant (we ignore the issues that arise with this assumption when additional random variables are added to our universe, and implicitly assume $\Omega$ and $\mathbb{P}$ have been properly extended in a way that is consistent with all other previous random variables in play).

The space $L^{2}(\mathcal{F})$ is a Hilbert space with norm
$$(X,Y)=\mathbb{E}[XY]=\int_{\Omega}X(\omega)Y(\omega)\;d\mathbb{P}(\omega).$$ The integral defining this inner product can be calculated as a Lebesgue integral on the range of the random vector $(X,Y)$ (in particular, an integral over $\mathbb{R}^{2}$) in the usual way by changing variables and using the appropriate push-forward (distribution) measures of $X$ and $Y$ (and densities if the distributions are absolutely continuous with respect to Lebesgue measure).