# Week 7

This week, we will discuss joint distributions and their properties. As well as the Central Limit Theorem.
Published

March 3, 2023

## Learning Outcomes

### Tuesday

• Marginal Distributions

• Conditional Density Functions

• Independence

• Expected Value

• Covariance

### Thursday

• Sampling Distributions
• Central Limit Theorem

## Important Concepts

### Tuesday

#### Bivariate Distributions

##### Discrete Random Variables

Let $$X_1$$ and $$X_2$$ be 2 discrete random variables, the joint distribution function of $$(X_1, X_2)$$ is defined as

$p_{X_1,X_2}(X_1, X_2) = P(X_1=x_1, X_2 = x_2).$

The properties of a bivariate discrete distribution are

• $$p_{X_1,X_2}(x_1,x_2)\ge 0$$ for all $$x_1,\ x_2$$

• $$\sum_{x_1,x_2}p(x_1,x_2)=1$$

##### Continuous Random Variables

Let $$X_1$$ and $$X_2$$ be 2 continuous random variables, the joint distribution function of $$(X_1, X_2)$$ is defined as

$F_{X_1,X_2}(x_1, x_2) = P(X_1\le x_1, X_2 \le x_2).$

The properties of a bivariate continuous distribution are

• $$f_{X_1,X_2}(x_1,x_2)=\frac{\partial^2F(x_1,x_2)}{\partial x_1\partial x_2}$$

• $$f_{X_1,X_2}(x_1, x_2)\ge 0$$

• $$\int_{x_1}\int_{x_2}f_{X_1,X_2}(x_1,x_2)dx_2dx_1=1$$

#### Marginal Distributions

##### Discrete Random Variables

Let $$X_1$$ and $$X_2$$ be 2 discrete random variables, with a joint distribution function of

$p_{X_1,X_2}(X_1, X_2) = P(X_1=x_1, X_2 = x_2).$

The marginal distribution of $$X_1$$ is defined as

$p_{X_1}(x_1) = \sum_{x_2}p_{X_1,X_2}(x_1,x_2)$

##### Continuous Random Variables

Let $$X_1$$ and $$X_2$$ be 2 continuous random variables, with a joint density function of $$f_{X_1,X_2}(x_1,x_2)$$. The marginal distribution of $$X_1$$ is defined as

$f_{X_1}(x_1) = \int_{x_1}f_{X_1,X_2}(x_1,x_2)dx_2$

#### Conditional Distributions

##### Discrete Random Variables

Let $$X_1$$ and $$X_2$$ be 2 discrete random variables, with a joint distribution function of

$p_{X_1,X_2}(X_1, X_2) = P(X_1=x_1, X_2 = x_2).$

The conditional distribution of $$X_1|X_2$$ is defined as

$p_{X_1|X_2}(x_1) = \frac{p_{X_1,X_2}(x_1,x_2)}{p_{X_2}(x_2)}$

##### Continuous Random Variables

Let $$X_1$$ and $$X_2$$ be 2 continuous random variables, with a joint density function of $$f_{X_1,X_2}(x_1,x_2)$$. The conditional distribution of $$X_1|X_2$$ is defined as

$f_{X_1|X_2}(x_1) = \frac{f_{X_1,X_2}(x_1,x_2)}{f_{X_2}(x_2)}$

#### Independence

##### Discrete Random Variables

Let $$X_1$$ and $$X_2$$ be 2 discrete random variables, with a joint density function of $$p_{X_1,X_2}(x_1,x_2)$$. $$X_1$$ is independent of $$X_2$$ if and only if

$p_{X_1,X_2}(x_1,x_2) = p_{X_1}(x_1)p_{X_2}(x_2)$

##### Continuous Random Variables

Let $$X_1$$ and $$X_2$$ be 2 continuous random variables, with a joint density function of $$f_{X_1,X_2}(x_1,x_2)$$. $$X_1$$ is independent of $$X_2$$ if and only if

$f_{X_1,X_2}(x_1,x_2) = f_{X_1}(x_1)f_{X_2}(x_2)$

#### Expected Value

Let $$X_1, X_2, \ldots,X_n$$ be a set of random variables, the expectation of a function $$g(X_1,\ldots, X_n)$$ is defined as

$E\{g(X_1,\ldots, X_n)\} = \sum_{x_1\in X_1}\cdots\sum_{x_n\in X_n}g(X_1,\ldots, X_n)p(x_1,\ldots,x_n)$

or

$E\{g(X_1,\ldots, X_n)\} = \int_{x_1\in X_1}\cdots\int_{x_n\in X_n}g(X_1,\ldots, X_n)f_{X_1,\ldots,X_n}(x_1,\ldots,x_n)dx_n \cdots dx_1$

#### Covariance

Let $$X_1$$ and $$X_2$$ be 2 random variables with mean $$\mu_1$$ and $$\mu_2$$, respectively. The covariance of $$X_1$$ and $$X_2$$ is defined as

$\begin{eqnarray*} Cov(X_1,X_2) & = & E\{(X_1-\mu_1)(X_2-\mu_2)\}\\ & =& E(X_1X_2)-\mu_1\mu_2 \end{eqnarray*}$

The correlation of $$X_1$$ and $$X_2$$ is defined as

$\rho = Cor(X_1,X_2) = \frac{Cov(X_1,X_2)}{\sqrt{Var(X_1)Var(X_2)}}$

If $$X_1$$ and $$X_2$$ are independent random variables, then

$Cov(X_1,X_2)=0$

#### Expected Value and Variance of Linear Functions

Let $$X_1,\ldots,X_n$$ and $$Y_1,\ldots,Y_m$$ be random variables with $$E(X_i)=\mu_i$$ and $$E(Y_j)=\tau_j$$. Furthermore, let $$U = \sum^n_{i=1}a_iX_i$$ and $$V=\sum^m_{j=1}b_jY_j$$ where $$\{a_i\}^n_{i=1}$$ and $$\{b_j\}_{j=1}^m$$ are constants. We have the following properties:

• $$E(U)=\sum_{i=1}^na_i\mu_i$$

• $$Var(U)=\sum^n_{i=1}a_i^2Var(X_i)+2\underset{i<j}{\sum\sum}a_ia_jCov(X_i,X_j)$$

• $$Cov(U,V)=\sum^n_{i=1}\sum^m_{j=1}Cov(X_i,Y_j)$$

#### Conditional Expectations

Let $$X_1$$ and $$X_2$$ be two random variables, the conditional expectation of $$g(X_1)$$, given $$X_2=x_2$$, is defined as

$E\{g(X_1)|X_2=x_2\}=\sum_{x_1}g(x_1)p(x_1|x_2)$

or

$E\{g(X_1)|X_2=x_2\}=\int_{x_1}g(x_1)f(x_1|x_2)dx_1.$

Furthermore,

$E(X_1)=E_{X_2}\{E_{X_1|X_2}(X_1|X_2)\}$

and

$Var(X_1) = E_{X_2}\{Var_{X_1|X_2}(X_1|X_2)\} + Var_{X_2}\{E_{X_1|X_2}(X_1|X_2)\}$

### Thursday

#### Sampling Distributions

##### Observing Random Variables

When collecting a sample of $$n$$, we tend to observe individual random variables: $$\{X_1, X_2, \cdots,X_n\}$$.

##### Sum of Random Variables

Let $$X_i$$, for $$i=1, \cdots, n$$, be identically and independently distributed (iid) normal distribution with mean $$\mu$$ and variance $$\sigma^2$$. Let $$T=\sum_{i=1}^nX_i$$ follow an normal distribution with mean $$\mu$$ and variance $$n\sigma^2$$.

#### Central Limit Theorem

Let $$X_1, X_2, \ldots, X_n$$ be identical and independent distributed random variables with $$E(X_i)=\mu$$ and $$Var(X_i) = \sigma²$$. We define

$Y_n = \sqrt n \left(\frac{\bar X-\mu}{\sigma}\right) \mathrm{ where }\ \bar X = \frac{1}{n}\sum^n_{i=1}X_i.$

Then, the distribution of the function $$Y_n$$ converges to a standard normal distribution function as $$n\rightarrow \infty$$.

#### Other Sampling Distributions

##### $$\chi^2$$-distribution

Let $$Z_1, Z_2,\ldots,Z_n \overset{iid}{\sim}N(0,1)$$,

$\sum_{i=1}^nZ_i^2\sim\chi^2_n.$

Let $$X_1, X_2,\ldots,X_n \overset{iid}{\sim}N(\mu,\sigma^2)$$, $$S^2 = \frac{1}{n-1}\sum^n_{i=1}(X_i-\bar X)^2$$, and $$\bar X \perp S^2$$; therefore:

$\frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1}.$

##### t-distribution

Let $$Z\sim N(0,1)$$, $$W\sim \chi^2_\nu$$, $$Z\perp W$$; therefore:

$T=\frac{Z}{\sqrt{W/\nu}} \sim t_\nu$

##### F-distribution

Let $$W_1\sim\chi^2_{\nu_1}$$ $$W_2\sim\chi^2_{\nu_2}$$, and $$W_1\perp W_2$$; therefore:

$F = \frac{W_1/\nu_1}{W_2/\nu_2}\sim F_{\nu_1,\nu_2}$

## Resources

### First Lecture

Slides Videos
Slides Video 001 Video 002

### Second Lecture

Slides Videos
Slides Video 001 Video 002