Introduction to Probability

Learning Objectives

• Define sample space and experiment

• Define probabilities

• Define random variable and distribution function

Sample Space and Experiments

Experiments and Outcomes

• A random experiment is a process with uncertain results but well-defined possible outcomes.

Examples

• Tossing a coin → outcomes: {Heads, Tails}
  
• Rolling a die → outcomes: {1, 2, 3, 4, 5, 6}
  
• Drawing a card → outcomes: {all 52 cards in a deck}

Sample Space

• The sample space (denoted \(S\)) is the set of all possible outcomes of an experiment.

Examples

• Coin toss: \(S = \{H, T\}\)
• Die roll: \(S = \{1, 2, 3, 4, 5, 6\}\)
  
• Two coin tosses: \(S = \{HH, HT, TH, TT\}\)

Events

• An event is any subset of the sample space.

Examples:
- Event A: “Die shows an even number”
\(A = \{2,4,6\}\)

• Event B: “At least one Head in two tosses”
  \(B = \{HH, HT, TH\}\)

Certain and Impossible Events

• Certain event: The sample space itself \(S\) (probability = 1)

• Impossible event: The empty set \(\varnothing\) (probability = 0)

Event Operations

Since events are subsets of the sample space, we can use set theory:

• Union: \(A \cup B\) → either A or B occurs
  
• Intersection: \(A \cap B\) → both A and B occur
  
• Complement: \(A^c\) → event A does not occur

Basic Probability Theory

Probability Axioms

For any event \(A\):

1. \(0\leq \Pr(A) \leq 1\)

2. \(\Pr(S) = 1\) where \(S\) is the sample space.

Complement Rule

The complement of \(A\) (denoted \(A^c\)) is “not \(A\)”.

\[\Pr(A^c) = 1 - \Pr(A)\]

Example

If the probability of rain is 0.3,
then the probability of no rain is:

Addition Rule

For any two events \(A\) and \(B\):

\[\Pr(A \cup B) = \Pr(A) + \Pr(B) - \Pr(A \cap B)\]

Disjoint Events (mutually exclusive)

If \(A\) and \(B\) cannot happen together,

\[ \Pr(A \cap B) = 0 \]

Then:

\[ \Pr(A \cup B) = \Pr(A) + \Pr(B) \]

Multiplication Rule

For any two events \(A\) and \(B\):

\[ \Pr(A \cap B) = \Pr(A \mid B)\Pr(B) \]

or equivalently,

\[ \Pr(A \cap B) = \Pr(B \mid A)\Pr(A) \]

Conditional Probability

The probability of (A) given that (B) occurred:

\[ \Pr(A \mid B) = \frac{\Pr(A \cap B)}{\Pr(B)}, \quad \Pr(B) > 0 \]

Example

If 30% of students play soccer, 20% play basketball,
and 10% play both:

Summary

• Complement Rule:
  \[ \Pr(A^c) = 1 - \Pr(A) \]

• Addition Rule:
  \[ \Pr(A \cup B) = \Pr(A) + \Pr(B) - \Pr(A \cap B) \]

• Multiplication Rule:
  \[ \Pr(A \cap B) = \Pr(A \mid B)\Pr(B) \]

• Conditional Probability:
  \[ \Pr(A \mid B) = \frac{\Pr(A \cap B)}{\Pr(B)} \]

Example

Example

Suppose we want to understand the efficacy of a test for a certain disease. Consider the following table:

		Disease	Presence	Total
		Yes	No
Test Result	Yes	42	6
	No	17	35
				100

Example

• Find the probability that an individual has a disease

• Find the probability that an individual tests negative for a disease

• Find the probability for someone who has the disease, given that they test positive.

• Find the probability that the test gives an accurate result

• Find the probability that the test gives and inaccurate result