Expectation#

Definition#

Definition 21 (Expectation)

Let \(\P\) be a probability function defined over the probability space \(\pspace\).

Let \(X\) be a discrete random variable with \(\S = \lset \xi_1, \xi_2, \ldots \rset\).

Then the expectation of \(X\) is defined as:

\[ \expectation(X) = \sum_{x \in X(\S)} x \cdot \P \lsq X = x \rsq \]

Theorem 7 (Existence of Expectation)

Let \(\P\) be a probability function defined over the probability space \(\pspace\).

A discrete random variable \(X\) with \(\S = \lset \xi_1, \xi_2, \ldots \rset\) has an expectation if and only if it is absolutely summable.

That is,

\[ \expectation \lsq \lvert X \rvert \rsq \overset{\text{def}}{=} \sum_{x \in X(\S)} \lvert x \rvert \cdot \P \lsq X = x \rsq < \infty \]

Let \(\P\) be a probability function defined over the probability space \(\pspace\).

Let \(X\) be a discrete random variable with \(\S = \lset \xi_1, \xi_2, \ldots \rset\).

Then the expectation of \(X\) has the following properties:

Property 1 (The Law of The Unconscious Statistician)

For any function \(g\),

\[ \expectation \lsq g(X) \rsq = \sum_{x \in X(\S)} g(x) \cdot \P \lsq X = x \rsq \]

This is not a trivial result, proof can be found here.

Property 2 (Linearity)

For any constants \(a\) and \(b\),

\[ \expectation \lsq aX + b \rsq = a \cdot \expectation(X) + b \]

Property 3 (Scaling)

For any constant \(c\),

\[ \expectation \lsq cX \rsq = c \cdot \expectation(X) \]

Property 4 (DC Shift)

For any constant \(c\),

\[ \expectation \lsq X + c \rsq = \expectation(X) \]

Property 5 (Stronger Linearity)

It follows that for any random variables \(X_1\), \(X_2\), …, \(X_n\),

\[ \expectation \lsq \sum_{i=1}^n a_i X_i \rsq = \sum_{i=1}^n a_i \cdot \expectation \lsq X_i \rsq \]

Concept

Expectation is a measure of the mean value of a random variable and is deterministic. It is also synonymous with the population mean.
Average is a measure of the average value of a random sample from the true population and is random.
Average of a random sample is a random variable and as sample size increases, the average of a random sample converges to the population mean.