Probability Mass Function

Definition

Definition 18 (State Space)

The set of all possible states of \(X\) is called the state space of \(X\) and is denoted as \(X(\S)\).

In particular, the state space of a discrete random variable \(X\) is a countable set as per Definition 17.

Example 4 (State Space of Coin Toss)

Let us revisit the example in Example 1 and examine the state space of \(X\).

The state space of \(X\) is the set of all possible values that \(X\) can take. As enumerated in the example, we see that the state space of \(X\) is \(\{0, 1, 2\}\) (i.e. \(X\) takes 3 states 0, 1 and 2).

Example 5 (State Space of Dice Roll)

Let us revisit the example in Example 2 and examine the state space of \(X\).

The state space of \(X\) is the set of all possible values that \(X\) can take. As enumerated in the example, we see that the state space of \(X\) is \(\{1, 2, 3, 4, 5, 6\}\) (i.e. \(X\) takes 6 states 1, 2, 3, 4, 5 and 6).

For two dice rolls, the state space is \(\{(1, 1), (1, 2), \ldots, (6, 6)\}\) where each state is a tuple of two dice rolls.

Definition 19 (Probability Mass Function)

The probability mass function (PMF) of a random variable \(X\) is a function that maps each state \(x\) in the state space \(X(\S)\) to its probability \(\pmf(x) = \P\lsq X = x \rsq\).

We denoted the PMF as

\[\begin{split} \begin{align} \pmf: X(\S) &\to [0, 1] \\ X(\S) \ni x &\mapsto \pmf(x) \end{align} \end{split}\]

Example 6 (PMF of Coin Toss)

Let us revisit Example 1 on coin toss and compute the PMF of \(X\).

Recall the sample space is given by \(\S = \{(HH), (HT), (TH), (TT)\}\) and the state space is given by \(X(\S) = \{0, 1, 2\}\) as enumerated in Example 4.

Thus, our domain of \(\pmf\) is \(X(\S) = \{0, 1, 2\}\) we have 3 mappings to compute:

\[\begin{split} \begin{align} \pmf(0) &= \P\lsq X = 0 \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = 0 \rset \rsq = \P\lsq \{(TT)\} \rsq = \dfrac{1}{4} \\ \pmf(1) &= \P\lsq X = 1 \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = 1 \rset \rsq = \P\lsq \{(HT), (TH)\} \rsq = \dfrac{2}{4} \\ \pmf(2) &= \P\lsq X = 2 \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = 2 \rset \rsq = \P\lsq \{(HH)\} \rsq = \dfrac{1}{4} \end{align} \end{split}\]

Here we have enumerated all the possible states of \(X\) and computed the probability of each state. Thus, the PMF of \(X\) is completely determined by the 3 mappings above.

Example 7 (PMF of Two Dice Rolls)

Let us revisit Example 2 on two dice rolls and compute the PMF of \(X\).

Recall the sample space is given by \(\S = \lset (1, 1), (1, 2), \ldots, (6, 6) \rset\) and the state space is given by \(X(\S) = \lset (1, 1), (1, 2), \ldots, (6, 6) \rset\) as enumerated in Example 5.

Thus, our domain of \(\pmf\) is \(X(\S) = \lset (1, 1), (1, 2), \ldots, (6, 6) \rset\) we have 36 states to compute:

\[\begin{split} \begin{align} \pmf((1, 1)) &= \P\lsq X = (1, 1) \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = (1, 1) \rset \rsq = \P\lsq \{(1, 1)\} \rsq = \dfrac{1}{36} \\ \pmf((1, 2)) &= \P\lsq X = (1, 2) \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = (1, 2) \rset \rsq = \P\lsq \{(1, 2)\} \rsq = \dfrac{1}{36} \\ \vdots \\ \pmf((6, 6)) &= \P\lsq X = (6, 6) \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = (6, 6) \rset \rsq = \P\lsq \{(6, 6)\} \rsq = \dfrac{1}{36} \end{align} \end{split}\]

Normalization

Theorem 4 (Normalization Property of PMF)

A PMF should satisfy the following normalization property:

(4)\[ \sum_{x \in X(\S)} \pmf(x) = 1 \]

Proof. TODO

Sturges’ Rule and Cross Validation

See Introduction to Probability for Data Science section 3.2.5.