Concept
Contents
Concept#
Conditional PMF#
Definition 73 (Conditional PMF)
Let
Remark 12 (Some Intuition)
It is relatively easy to associate the definition of conditional PMF with the definition of conditional probability in the chapter on Conditional Probability.
Indeed,
To see why this perspective makes sense, let us recall the definition of a conditional probability:
In
Remark 13 (Conditional Distribution is a Distribution for a Sub-Population)
Here’s a very important concept mentioned by Professor Chan. in his book. To interpret
the notation
Example 15 (Two Coins)
This example is taken from Professor Chan’s book, [Chan, 2021].
Consider a joint PMF given in the following table. Find the conditional
Before we even look at the solution in the book, we can test our conceptual understanding using
intuition. Since we want to find the distribution of
In that sub-population, the total is
It turns out our intuition is correct.
Theorem 26 (Conditional PMF of an Event
Let
Then the probability
and the probability
Definition 74 (Conditional CDF)
Let
Conditional Independence#
Definition 75 (Conditional Independence)
Two discrete random variables
where
In terms of events
Example 16 (Conditional Independence)
The example is from Dive into Deep Learning, Section 2.6.4.
Interestingly, two variables can be independent in general but become dependent when conditioning on a third. This often occurs when the two random variables and correspond to causes of some third variable . For example, broken bones and lung cancer might be independent in the general population but if we condition on being in the hospital then we might find that broken bones are negatively correlated with lung cancer. That’s because the broken bone explains away why some person is in the hospital and thus lowers the probability that they have lung cancer.
And conversely, two dependent random variables can become independent upon conditioning on a third. This often happens when two otherwise unrelated events have a common cause. Shoe size and reading level are highly correlated among elementary school students, but this correlation disappears if we condition on age.
Conditional PDF#
We now discuss the conditioning of a continuous random variable.
Definition 76 (Conditional PDF)
Let
Even though (32) is in the form of Bayes’ rule, this definition becomes hazy since we cannot treat the conditional PDF of a continuous random variable
the same way we treat the discrete counterpart earlier. That is to say, we cannot just say that the conditional
PDF stems from the Bayes’ rule. This is because the
denominator
To answer this question, we first define the conditional
Definition 77 (Conditional CDF)
Let
Professor Chan probed further by asking why should the conditional CDF of continuous random variable be defined in this way? One way to interpret
Definition 78 (Limiting Perspective)
We can define the conditional CDF as
This should not come as a surprise as we are merely taking the limit for the conditioned variable
With some calculations, we can express (33) in terms of the conditional PDF of
The key here is that the small step size
where (a) follows from the fundamental theorem of calculus (Theorem 1).
Just like the conditional PMF, we can calculate the probabilities using the conditional PDFs. In particular, if we evaluate the probability where
Theorem 27 (Conditional PDF of an Event
Let
Then the probability
and the probability