Expectation#

Definition#

Definition 44 (Expectation)

Let \(\P\) be a probability function defined over the probability space \(\pspace\).

Let \(X\) be a continuous random variable with sample space \(\S\) and let \(\pdf\) be its probability density function.

Then the expectation of \(X\) is defined as:

\[ \expectation \lsq X \rsq = \int_{\S} x \cdot \pdf(x) \, dx \]

As seen in the discrete counterpart in Expectation, we have a similar result for continuous random variables.

Theorem 9 (Existence of Expectation)

A continuous random variable \(X\) with sample space \(\S\) has an expectation if and only if it is absolutely integrable.

That is,

\[ \expectation \lsq \lvert X \rvert \rsq \overset{\text{def}}{=} \int_{\S} \lvert x \rvert \cdot \pdf(x) \, dx < \infty \]

Corollary 7

For any continuous random variable \(X\) with sample space \(\S\) and probability density function \(\pdf\),

\[ \lvert \expectation \lsq X \rsq \rvert \leq \expectation \lsq \lvert X \rvert \rsq \]

Almost all properties of the discrete counterpart in Expectation holds here.

Let \(\P\) be a probability function defined over the probability space \(\pspace\).

Let \(X\) be a continous random variable with sample space \(\S\) and probability density function \(\pdf\).

Then the expectation of \(X\) has the following properties:

Property 18 (The Law of The Unconscious Statistician)

For any function \(g: \S \to \R\),

\[ \expectation \lsq g(X) \rsq = \int_{\S} g(x) \cdot \pdf(x) \, dx \]

Property 19 (Linearity)

For any constants \(a\) and \(b\),

\[ \expectation \lsq aX + b \rsq = a \cdot \expectation(X) + b \]

Property 20 (Scaling)

For any constant \(c\),

\[ \expectation \lsq cX \rsq = c \cdot \expectation(X) \]

Property 21 (DC Shift)

For any constant \(c\),

\[ \expectation \lsq X + c \rsq = \expectation(X) \]

Property 22 (Stronger Linearity)

It follows that for any random variables \(X_1\), \(X_2\), …, \(X_n\),

\[ \expectation \lsq \sum_{i=1}^n a_i X_i \rsq = \sum_{i=1}^n a_i \cdot \expectation \lsq X_i \rsq \]

Concept

Expectation is a measure of the mean value of a random variable and is deterministic. It is also synonymous with the population mean.
Average is a measure of the average value of a random sample from the true population and is random.
Average of a random sample is a random variable and as sample size increases, the average of a random sample converges to the population mean.