Taylor expansions for the moments of functions of random variables
(Learn how and when to remove this message) In probability theory, it is possible to approximate the moments of a function f of a random variable X using Taylor expansions, provided that f is sufficiently differentiable and that the moments of X are finite.
First moment
Given
and
, the mean and the variance of
, respectively,[1] a Taylor expansion of the expected value of
can be found via
![{\displaystyle {\begin{aligned}\operatorname {E} \left[f(X)\right]&{}=\operatorname {E} \left[f\left(\mu _{X}+\left(X-\mu _{X}\right)\right)\right]\\&{}\approx \operatorname {E} \left[f(\mu _{X})+f'(\mu _{X})\left(X-\mu _{X}\right)+{\frac {1}{2}}f''(\mu _{X})\left(X-\mu _{X}\right)^{2}\right]\\&{}=f(\mu _{X})+f'(\mu _{X})\operatorname {E} \left[X-\mu _{X}\right]+{\frac {1}{2}}f''(\mu _{X})\operatorname {E} \left[\left(X-\mu _{X}\right)^{2}\right].\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d9453f100161227140288817a070c40ad87c22b2)
Since
the second term vanishes. Also,
is
. Therefore,
.
It is possible to generalize this to functions of more than one variable using multivariate Taylor expansions. For example,
![{\displaystyle \operatorname {E} \left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]}}-{\frac {\operatorname {cov} \left[X,Y\right]}{\operatorname {E} \left[Y\right]^{2}}}+{\frac {\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]^{3}}}\operatorname {var} \left[Y\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/bf8b82f9c15c42b3fa41c397dd3b6a1d67735539)
Second moment
Similarly,[1]
![{\displaystyle \operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}-{\frac {1}{4}}\left(f''(\mu _{X})\right)^{2}\sigma _{X}^{4}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/94fb3c55d082462f42a9a9b9cb7bae4d1e7c3053)
The above is obtained using a second order approximation, following the method used in estimating the first moment. It will be a poor approximation in cases where
is highly non-linear. This is a special case of the delta method.
Indeed, we take
.
With
, we get
. The variance is then computed using the formula
.
An example is,
![{\displaystyle \operatorname {var} \left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {var} \left[X\right]}{\operatorname {E} \left[Y\right]^{2}}}-{\frac {2\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]^{3}}}\operatorname {cov} \left[X,Y\right]+{\frac {\operatorname {E} \left[X\right]^{2}}{\operatorname {E} \left[Y\right]^{4}}}\operatorname {var} \left[Y\right].}](https://wikimedia.org/api/rest_v1/media/math/render/svg/011aff1036d96609635f44161e05afa36d783d19)
The second order approximation, when X follows a normal distribution, is:[2]
![{\displaystyle \operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]+{\frac {\left(f''(\operatorname {E} \left[X\right])\right)^{2}}{2}}\left(\operatorname {var} \left[X\right]\right)^{2}=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}+{\frac {1}{2}}\left(f''(\mu _{X})\right)^{2}\sigma _{X}^{4}+\left(f'(\mu _{X})\right)\left(f'''(\mu _{X})\right)\sigma _{X}^{4}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/14c6f8695bf69e9302602a388257f566ba6f1891)
First product moment
To find a second-order approximation for the covariance of functions of two random variables (with the same function applied to both), one can proceed as follows. First, note that
. Since a second-order expansion for
has already been derived above, it only remains to find
. Treating
as a two-variable function, the second-order Taylor expansion is as follows:
![{\displaystyle {\begin{aligned}f(X)f(Y)&{}\approx f(\mu _{X})f(\mu _{Y})+(X-\mu _{X})f'(\mu _{X})f(\mu _{Y})+(Y-\mu _{Y})f(\mu _{X})f'(\mu _{Y})+{\frac {1}{2}}\left[(X-\mu _{X})^{2}f''(\mu _{X})f(\mu _{Y})+2(X-\mu _{X})(Y-\mu _{Y})f'(\mu _{X})f'(\mu _{Y})+(Y-\mu _{Y})^{2}f(\mu _{X})f''(\mu _{Y})\right]\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f1967664e3eb99df8bca7a26321484a0cbdeb07f)
Taking expectation of the above and simplifying—making use of the identities
and
—leads to
. Hence,
![{\displaystyle {\begin{aligned}\operatorname {cov} \left[f(X),f(Y)\right]&{}\approx f(\mu _{X})f(\mu _{Y})+f'(\mu _{X})f'(\mu _{Y})\operatorname {cov} (X,Y)+{\frac {1}{2}}f''(\mu _{X})f(\mu _{Y})\operatorname {var} (X)+{\frac {1}{2}}f(\mu _{X})f''(\mu _{Y})\operatorname {var} (Y)-\left[f(\mu _{X})+{\frac {1}{2}}f''(\mu _{X})\operatorname {var} (X)\right]\left[f(\mu _{Y})+{\frac {1}{2}}f''(\mu _{Y})\operatorname {var} (Y)\right]\\&{}=f'(\mu _{X})f'(\mu _{Y})\operatorname {cov} (X,Y)-{\frac {1}{4}}f''(\mu _{X})f''(\mu _{Y})\operatorname {var} (X)\operatorname {var} (Y)\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/43a68317901c1d4652df64c01890265e1d49b2d4)
Random vectors
If X is a random vector, the approximations for the mean and variance of
are given by[3]
![{\displaystyle {\begin{aligned}\operatorname {E} (f(X))&=f(\mu _{X})+{\frac {1}{2}}\operatorname {trace} (H_{f}(\mu _{X})\Sigma _{X})\\\operatorname {var} (f(X))&=\nabla f(\mu _{X})^{t}\Sigma _{X}\nabla f(\mu _{X})+{\frac {1}{2}}\operatorname {trace} \left(H_{f}(\mu _{X})\Sigma _{X}H_{f}(\mu _{X})\Sigma _{X}\right).\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/3aca3ccf8ac4079c6105ecf700152efbe074bb18)
Here
and
denote the gradient and the Hessian matrix respectively, and
is the covariance matrix of X.
See also
- Propagation of uncertainty
- WKB approximation
- Delta method
Notes
- ^ a b Haym Benaroya, Seon Mi Han, and Mark Nagurka. Probability Models in Engineering and Science. CRC Press, 2005, p166.
- ^ Hendeby, Gustaf; Gustafsson, Fredrik. "ON NONLINEAR TRANSFORMATIONS OF GAUSSIAN DISTRIBUTIONS" (PDF). Retrieved 5 October 2017.
- ^ Rego, Bruno V.; Weiss, Dar; Bersi, Matthew R.; Humphrey, Jay D. (14 December 2021). "Uncertainty quantification in subject‐specific estimation of local vessel mechanical properties". International Journal for Numerical Methods in Biomedical Engineering. 37 (12): e3535. doi:10.1002/cnm.3535. ISSN 2040-7939. PMC 9019846. PMID 34605615.
Further reading
- Wolter, Kirk M. (1985). "Taylor Series Methods". Introduction to Variance Estimation. New York: Springer. pp. 221–247. ISBN 0-387-96119-4.