Moments and Variance

The moment and variance of a continuous random variable \(X\) are similar to the moment and variance of a discrete random variable in Moments and Variance, but they are defined using integrals instead of sums.

Notation

Let \(\P\) be a probability function defined over the probability space \(\pspace\).

Let \(X\) be a continuous random variable with sample space \(\S\) and probability density function \(\pdf\).

Moments

Definition 45 (Moments)

The \(k\)-th moment of \(X\) is defined as:

\[ \expectation \lsq X^k \rsq = \int_{-\infty}^{\infty} x^k \cdot \pdf \lsq x \rsq \, dx \]

This definition follows from Property 18 in Expectation.

Essentially, this means that the \(k\)-th moment is the expectation of \(X^k\).

Variance

Definition 46 (Variance)

The variance of \(X\) is defined as:

(14)\[ \var \lsq X \rsq = \expectation \lsq \lpar X - \mu \rpar^2 \rsq = \int_{\S} \lpar x - \mu \rpar^2 \pdf \lpar x \rpar \, dx \]

where \(\mu = \expectation \lsq X \rsq\) is the expectation of \(X\).

We also have an equivalent definition of variance, which is more used in practice.

Definition 47 (Variance (Alternative))

The variance of \(X\) is defined as:

(15)\[ \var \lsq X \rsq = \expectation \lsq X^2 \rsq - \expectation \lsq X \rsq^2 \]

Concept

Concept

• Variance is a measure of how spread out a distribution is. More concretely, it is the expectation of the squared deviation from the expectation of the distribution. One can think of for every data point in the distribution, how far is each data point from the expectation (population mean). The variance is the average of these data points (squared to make it positive).

• Variance is deterministic and is synonymous with Population Variance.

• Sample Variance is the variance of a random sample from the true population, which is a random variable.

Further Readings

• Chan, Stanley H. “Chapter 4.2.3. Momenets and variance.” In Introduction to Probability for Data Science, 184-185. Ann Arbor, Michigan: Michigan Publishing Services, 2021.