Week 2

This week, we will briefly discuss topics related to conditional probabilities. Additionally, we will introduce commonly used distribution functions.

Learning Outcomes

First Lecture

• Define sample space and events

• Define probability and rules

• Define random variable, probability mass function, and cumulative density function

Second Lecture

• Describe disjoint events

• Describe a conditional probability

• Define independent events

• Law of Total Probability

• Baye’s Theorem

Important Concepts

First Lecture

Experiment

A process that yields an outcome randomly.

Sample Space

The sample space (S) is all the possible outcomes from an experiment.

Event

An event is a collection of outcomes of an experiment that is any subset of the sample space S (including S).

Random Variable

A random variable (rv) is a function that maps a sample space to the real numbers. A rv is denoted by a capital letter. The observed value is denoted with a lower-case letter.

Probability (Cumulative) Distribution Function

The cumulative distribution function of a random variable \(X\), expressed as \(F_X(x)\), is defined as

\[ F_X(x) = P(X\leq x), \]

providing the probability of observing \(x\) or lower.

Probability Mass Function (Categorical Variable¹)

The probability mass function of a random variable, expressed as \(f_X(x)\), is expressed as

\[ f_X(x)=P(X=x), \]

providing the probability of observing \(x\).

Second Lecture

Disjoint Events

Two events A and B are considered disjoint if \(P(A\cap B)=0\). In general terms, only one event can occur, not both.

Conditional Probability

Let there be 2 events A and B. Given that B has occurred, what is the probability that A occurs? The conditional probability requires there to be at least 2 events and one event must have occurred. Additionally, the events cannot be disjoint. Conditional probabilities are denoted as $P(A|B)$, the probability of A given B has occurred.

• \(P(A|B)=\frac{P(A\cap B)}{P(B)}\)

Independent Events

Events A and B are considered independent if \(P(A\cap B)=P(A)P(B)\). In other words, The occurrence of one event will not affect the occurrence of the other event.

Law of Total Probability

The law of total probability allows you to compute the probability of an event A given that the sets {\(B_1\), …, \(B_n\)} partitions event A. The law of total probability is given as

\[ P(A) = \sum^n_{i=1}P(A\cap B_i) = \sum^n_{i=1}P(A|B_i)P(B_i) \]

Baye’s Theorem

Baye’s theorem computes the probability of an event \(B_i\) given event A

\[ P(B_i|A) = \frac{P(A\cap B_i)}{P(A)}=\frac{P(A|B_i)P(B_i)}{\sum^n_{i=1}P(A|B_i)P(B_i)} \]

Resources

First Lecture

Slides	Videos
Slides	Video 001 Video 002

Second Lecture

Slides	Videos
Slides	Video 001 Video 002

Footnotes

1. We will define this later on in the semester.