Application: Moment Generating Function and the Sum of Random Variables
Contents
Application: Moment Generating Function and the Sum of Random Variables#
This section is based on the work of [Chan, 2021].
Sum of Independent Random Variables#
(Moment Generating Function of the Sum of Two Independent Random Variables)
Let \(X\) and \(Y\) be independent random variables. Let \(Z=X+Y\). Then
Proof. By the definition of MGF, we have that
where (a) is valid because \(X\) and \(Y\) are independent.
\(N\) Independent Random Variables)
(Moment Generating Function of the Sum ofConsider independent random variables \(X_{1}, \ldots, X_{N}\). Let \(Z=\sum_{n=1}^{N} X_{n}\) be the sum of random variables. Then the MGF of \(Z\) is
If these random variables are further assumed to be identically distributed, the MGF is
Proof. This follows immediately from the previous theorem:
If the random variables \(X_{1}, \ldots, X_{N}\) are i.i.d., then the product simplifies to
Sum of Common Distributions via MGFs#
We have seen earlier in Sum of Common Distribution that the sum of two random variables with common distributions is also a random variable with a common distribution. In this section, we will use MGF to prove this fact as well.
(Sum of Poisson Random Variables is Poisson)
Let \(X_{1}, \ldots, X_{N}\) be a sequence of i.i.d. Poisson random variables with parameter \(\lambda\). Let \(Z=X_{1}+\cdots+X_{N}\) be the sum. Then \(Z\) is a Poisson random variable with parameters \(N \lambda\).
Proof. The MGF of a Poisson random variable is
Assume that we have a sum of \(N\) i.i.d. Poisson random variables. Then, by the main theorem, we have that
Therefore, the resulting random variable \(Z\) is a Poisson with parameter \(N \lambda\).
(Sum of Gaussian Random Variables is Gaussian)
Let \(X_{1}, \ldots, X_{N}\) be a sequence of independent Gaussian random variables with parameters \(\left(\mu_{1}, \sigma_{1}^{2}\right), \ldots,\left(\mu_{N}, \sigma_{N}^{2}\right)\). Let \(Z=X_{1}+\cdots+X_{N}\) be the sum. Then \(Z\) is a Gaussian random variable:
Proof. We skip the proof of the MGF of a Gaussian. It can be shown that
When we have a sequence of Gaussian random variables, then
Therefore, the resulting random variable \(Z\) is also a Gaussian. The mean and variance of \(Z\) are \(\sum_{n=1}^{N} \mu_{n}\) and \(\sum_{n=1}^{N} \sigma_{n}^{2}\), respectively.
Further Readings#
Chan, Stanley H. “Chapter 6.1.2. Sum of independent variables via MGF.” In Introduction to Probability for Data Science. Ann Arbor, Michigan: Michigan Publishing Services, 2021.
Pishro-Nik, Hossein. “Chapter 6.1.3. Moment Generating Functions.” In Introduction to Probability, Statistics, and Random Processes. Kappa Research, 2014.