# Week 7

## Learning Outcomes

### Tuesday

Marginal Distributions

Conditional Density Functions

Independence

Expected Value

Covariance

### Thursday

- Sampling Distributions
- Central Limit Theorem

## Important Concepts

### Tuesday

#### Bivariate Distributions

##### Discrete Random Variables

Let \(X_1\) and \(X_2\) be 2 discrete random variables, the joint distribution function of \((X_1, X_2)\) is defined as

\[ p_{X_1,X_2}(X_1, X_2) = P(X_1=x_1, X_2 = x_2). \]

The properties of a bivariate discrete distribution are

\(p_{X_1,X_2}(x_1,x_2)\ge 0\) for all \(x_1,\ x_2\)

\(\sum_{x_1,x_2}p(x_1,x_2)=1\)

##### Continuous Random Variables

Let \(X_1\) and \(X_2\) be 2 continuous random variables, the joint distribution function of \((X_1, X_2)\) is defined as

\[ F_{X_1,X_2}(x_1, x_2) = P(X_1\le x_1, X_2 \le x_2). \]

The properties of a bivariate continuous distribution are

\(f_{X_1,X_2}(x_1,x_2)=\frac{\partial^2F(x_1,x_2)}{\partial x_1\partial x_2}\)

\(f_{X_1,X_2}(x_1, x_2)\ge 0\)

\(\int_{x_1}\int_{x_2}f_{X_1,X_2}(x_1,x_2)dx_2dx_1=1\)

#### Marginal Distributions

##### Discrete Random Variables

Let \(X_1\) and \(X_2\) be 2 discrete random variables, with a joint distribution function of

\[ p_{X_1,X_2}(X_1, X_2) = P(X_1=x_1, X_2 = x_2). \]

The marginal distribution of \(X_1\) is defined as

\[ p_{X_1}(x_1) = \sum_{x_2}p_{X_1,X_2}(x_1,x_2) \]

##### Continuous Random Variables

Let \(X_1\) and \(X_2\) be 2 continuous random variables, with a joint density function of \(f_{X_1,X_2}(x_1,x_2)\). The marginal distribution of \(X_1\) is defined as

\[ f_{X_1}(x_1) = \int_{x_1}f_{X_1,X_2}(x_1,x_2)dx_2 \]

#### Conditional Distributions

##### Discrete Random Variables

Let \(X_1\) and \(X_2\) be 2 discrete random variables, with a joint distribution function of

\[ p_{X_1,X_2}(X_1, X_2) = P(X_1=x_1, X_2 = x_2). \]

The conditional distribution of \(X_1|X_2\) is defined as

\[ p_{X_1|X_2}(x_1) = \frac{p_{X_1,X_2}(x_1,x_2)}{p_{X_2}(x_2)} \]

##### Continuous Random Variables

Let \(X_1\) and \(X_2\) be 2 continuous random variables, with a joint density function of \(f_{X_1,X_2}(x_1,x_2)\). The conditional distribution of \(X_1|X_2\) is defined as

\[ f_{X_1|X_2}(x_1) = \frac{f_{X_1,X_2}(x_1,x_2)}{f_{X_2}(x_2)} \]

#### Independence

##### Discrete Random Variables

Let \(X_1\) and \(X_2\) be 2 discrete random variables, with a joint density function of \(p_{X_1,X_2}(x_1,x_2)\). \(X_1\) is independent of \(X_2\) if and only if

\[ p_{X_1,X_2}(x_1,x_2) = p_{X_1}(x_1)p_{X_2}(x_2) \]

##### Continuous Random Variables

Let \(X_1\) and \(X_2\) be 2 continuous random variables, with a joint density function of \(f_{X_1,X_2}(x_1,x_2)\). \(X_1\) is independent of \(X_2\) if and only if

\[ f_{X_1,X_2}(x_1,x_2) = f_{X_1}(x_1)f_{X_2}(x_2) \]

#### Expected Value

Let \(X_1, X_2, \ldots,X_n\) be a set of random variables, the expectation of a function \(g(X_1,\ldots, X_n)\) is defined as

\[ E\{g(X_1,\ldots, X_n)\} = \sum_{x_1\in X_1}\cdots\sum_{x_n\in X_n}g(X_1,\ldots, X_n)p(x_1,\ldots,x_n) \]

or

\[ E\{g(X_1,\ldots, X_n)\} = \int_{x_1\in X_1}\cdots\int_{x_n\in X_n}g(X_1,\ldots, X_n)f_{X_1,\ldots,X_n}(x_1,\ldots,x_n)dx_n \cdots dx_1 \]

#### Covariance

Let \(X_1\) and \(X_2\) be 2 random variables with mean \(\mu_1\) and \(\mu_2\), respectively. The covariance of \(X_1\) and \(X_2\) is defined as

\[ \begin{eqnarray*} Cov(X_1,X_2) & = & E\{(X_1-\mu_1)(X_2-\mu_2)\}\\ & =& E(X_1X_2)-\mu_1\mu_2 \end{eqnarray*} \]

The correlation of \(X_1\) and \(X_2\) is defined as

\[ \rho = Cor(X_1,X_2) = \frac{Cov(X_1,X_2)}{\sqrt{Var(X_1)Var(X_2)}} \]

If \(X_1\) and \(X_2\) are independent random variables, then

\[ Cov(X_1,X_2)=0 \]

#### Expected Value and Variance of Linear Functions

Let \(X_1,\ldots,X_n\) and \(Y_1,\ldots,Y_m\) be random variables with \(E(X_i)=\mu_i\) and \(E(Y_j)=\tau_j\). Furthermore, let \(U = \sum^n_{i=1}a_iX_i\) and \(V=\sum^m_{j=1}b_jY_j\) where \(\{a_i\}^n_{i=1}\) and \(\{b_j\}_{j=1}^m\) are constants. We have the following properties:

\(E(U)=\sum_{i=1}^na_i\mu_i\)

\(Var(U)=\sum^n_{i=1}a_i^2Var(X_i)+2\underset{i<j}{\sum\sum}a_ia_jCov(X_i,X_j)\)

\(Cov(U,V)=\sum^n_{i=1}\sum^m_{j=1}Cov(X_i,Y_j)\)

#### Conditional Expectations

Let \(X_1\) and \(X_2\) be two random variables, the conditional expectation of \(g(X_1)\), given \(X_2=x_2\), is defined as

\[ E\{g(X_1)|X_2=x_2\}=\sum_{x_1}g(x_1)p(x_1|x_2) \]

or

\[ E\{g(X_1)|X_2=x_2\}=\int_{x_1}g(x_1)f(x_1|x_2)dx_1. \]

Furthermore,

\[ E(X_1)=E_{X_2}\{E_{X_1|X_2}(X_1|X_2)\} \]

and

\[ Var(X_1) = E_{X_2}\{Var_{X_1|X_2}(X_1|X_2)\} + Var_{X_2}\{E_{X_1|X_2}(X_1|X_2)\} \]

### Thursday

#### Sampling Distributions

##### Observing Random Variables

When collecting a sample of \(n\), we tend to observe individual random variables: \(\{X_1, X_2, \cdots,X_n\}\).

##### Sum of Random Variables

Let \(X_i\), for \(i=1, \cdots, n\), be identically and independently distributed (iid) normal distribution with mean \(\mu\) and variance \(\sigma^2\). Let \(T=\sum_{i=1}^nX_i\) follow an normal distribution with mean \(\mu\) and variance \(n\sigma^2\).

#### Central Limit Theorem

Let \(X_1, X_2, \ldots, X_n\) be identical and independent distributed random variables with \(E(X_i)=\mu\) and \(Var(X_i) = \sigma²\). We define

\[ Y_n = \sqrt n \left(\frac{\bar X-\mu}{\sigma}\right) \mathrm{ where }\ \bar X = \frac{1}{n}\sum^n_{i=1}X_i. \]

Then, the distribution of the function \(Y_n\) converges to a standard normal distribution function as \(n\rightarrow \infty\).

#### Other Sampling Distributions

##### \(\chi^2\)-distribution

Let \(Z_1, Z_2,\ldots,Z_n \overset{iid}{\sim}N(0,1)\),

\[ \sum_{i=1}^nZ_i^2\sim\chi^2_n. \]

Let \(X_1, X_2,\ldots,X_n \overset{iid}{\sim}N(\mu,\sigma^2)\), \(S^2 = \frac{1}{n-1}\sum^n_{i=1}(X_i-\bar X)^2\), and \(\bar X \perp S^2\); therefore:

\[ \frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1}. \]

##### t-distribution

Let \(Z\sim N(0,1)\), \(W\sim \chi^2_\nu\), \(Z\perp W\); therefore:

\[ T=\frac{Z}{\sqrt{W/\nu}} \sim t_\nu \]

##### F-distribution

Let \(W_1\sim\chi^2_{\nu_1}\) \(W_2\sim\chi^2_{\nu_2}\), and \(W_1\perp W_2\); therefore:

\[ F = \frac{W_1/\nu_1}{W_2/\nu_2}\sim F_{\nu_1,\nu_2} \]

## Resources

### First Lecture

Slides | Videos |
---|---|

Slides | Video 001 Video 002 |

### Second Lecture

Slides | Videos |
---|---|

Slides | Video 001 Video 002 |