Week 7
Learning Outcomes
Tuesday
Marginal Distributions
Conditional Density Functions
Independence
Expected Value
Covariance
Thursday
- Sampling Distributions
- Central Limit Theorem
Important Concepts
Tuesday
Bivariate Distributions
Discrete Random Variables
Let \(X_1\) and \(X_2\) be 2 discrete random variables, the joint distribution function of \((X_1, X_2)\) is defined as
\[ p_{X_1,X_2}(X_1, X_2) = P(X_1=x_1, X_2 = x_2). \]
The properties of a bivariate discrete distribution are
\(p_{X_1,X_2}(x_1,x_2)\ge 0\) for all \(x_1,\ x_2\)
\(\sum_{x_1,x_2}p(x_1,x_2)=1\)
Continuous Random Variables
Let \(X_1\) and \(X_2\) be 2 continuous random variables, the joint distribution function of \((X_1, X_2)\) is defined as
\[ F_{X_1,X_2}(x_1, x_2) = P(X_1\le x_1, X_2 \le x_2). \]
The properties of a bivariate continuous distribution are
\(f_{X_1,X_2}(x_1,x_2)=\frac{\partial^2F(x_1,x_2)}{\partial x_1\partial x_2}\)
\(f_{X_1,X_2}(x_1, x_2)\ge 0\)
\(\int_{x_1}\int_{x_2}f_{X_1,X_2}(x_1,x_2)dx_2dx_1=1\)
Marginal Distributions
Discrete Random Variables
Let \(X_1\) and \(X_2\) be 2 discrete random variables, with a joint distribution function of
\[ p_{X_1,X_2}(X_1, X_2) = P(X_1=x_1, X_2 = x_2). \]
The marginal distribution of \(X_1\) is defined as
\[ p_{X_1}(x_1) = \sum_{x_2}p_{X_1,X_2}(x_1,x_2) \]
Continuous Random Variables
Let \(X_1\) and \(X_2\) be 2 continuous random variables, with a joint density function of \(f_{X_1,X_2}(x_1,x_2)\). The marginal distribution of \(X_1\) is defined as
\[ f_{X_1}(x_1) = \int_{x_1}f_{X_1,X_2}(x_1,x_2)dx_2 \]
Conditional Distributions
Discrete Random Variables
Let \(X_1\) and \(X_2\) be 2 discrete random variables, with a joint distribution function of
\[ p_{X_1,X_2}(X_1, X_2) = P(X_1=x_1, X_2 = x_2). \]
The conditional distribution of \(X_1|X_2\) is defined as
\[ p_{X_1|X_2}(x_1) = \frac{p_{X_1,X_2}(x_1,x_2)}{p_{X_2}(x_2)} \]
Continuous Random Variables
Let \(X_1\) and \(X_2\) be 2 continuous random variables, with a joint density function of \(f_{X_1,X_2}(x_1,x_2)\). The conditional distribution of \(X_1|X_2\) is defined as
\[ f_{X_1|X_2}(x_1) = \frac{f_{X_1,X_2}(x_1,x_2)}{f_{X_2}(x_2)} \]
Independence
Discrete Random Variables
Let \(X_1\) and \(X_2\) be 2 discrete random variables, with a joint density function of \(p_{X_1,X_2}(x_1,x_2)\). \(X_1\) is independent of \(X_2\) if and only if
\[ p_{X_1,X_2}(x_1,x_2) = p_{X_1}(x_1)p_{X_2}(x_2) \]
Continuous Random Variables
Let \(X_1\) and \(X_2\) be 2 continuous random variables, with a joint density function of \(f_{X_1,X_2}(x_1,x_2)\). \(X_1\) is independent of \(X_2\) if and only if
\[ f_{X_1,X_2}(x_1,x_2) = f_{X_1}(x_1)f_{X_2}(x_2) \]
Expected Value
Let \(X_1, X_2, \ldots,X_n\) be a set of random variables, the expectation of a function \(g(X_1,\ldots, X_n)\) is defined as
\[ E\{g(X_1,\ldots, X_n)\} = \sum_{x_1\in X_1}\cdots\sum_{x_n\in X_n}g(X_1,\ldots, X_n)p(x_1,\ldots,x_n) \]
or
\[ E\{g(X_1,\ldots, X_n)\} = \int_{x_1\in X_1}\cdots\int_{x_n\in X_n}g(X_1,\ldots, X_n)f_{X_1,\ldots,X_n}(x_1,\ldots,x_n)dx_n \cdots dx_1 \]
Covariance
Let \(X_1\) and \(X_2\) be 2 random variables with mean \(\mu_1\) and \(\mu_2\), respectively. The covariance of \(X_1\) and \(X_2\) is defined as
\[ \begin{eqnarray*} Cov(X_1,X_2) & = & E\{(X_1-\mu_1)(X_2-\mu_2)\}\\ & =& E(X_1X_2)-\mu_1\mu_2 \end{eqnarray*} \]
The correlation of \(X_1\) and \(X_2\) is defined as
\[ \rho = Cor(X_1,X_2) = \frac{Cov(X_1,X_2)}{\sqrt{Var(X_1)Var(X_2)}} \]
If \(X_1\) and \(X_2\) are independent random variables, then
\[ Cov(X_1,X_2)=0 \]
Expected Value and Variance of Linear Functions
Let \(X_1,\ldots,X_n\) and \(Y_1,\ldots,Y_m\) be random variables with \(E(X_i)=\mu_i\) and \(E(Y_j)=\tau_j\). Furthermore, let \(U = \sum^n_{i=1}a_iX_i\) and \(V=\sum^m_{j=1}b_jY_j\) where \(\{a_i\}^n_{i=1}\) and \(\{b_j\}_{j=1}^m\) are constants. We have the following properties:
\(E(U)=\sum_{i=1}^na_i\mu_i\)
\(Var(U)=\sum^n_{i=1}a_i^2Var(X_i)+2\underset{i<j}{\sum\sum}a_ia_jCov(X_i,X_j)\)
\(Cov(U,V)=\sum^n_{i=1}\sum^m_{j=1}Cov(X_i,Y_j)\)
Conditional Expectations
Let \(X_1\) and \(X_2\) be two random variables, the conditional expectation of \(g(X_1)\), given \(X_2=x_2\), is defined as
\[ E\{g(X_1)|X_2=x_2\}=\sum_{x_1}g(x_1)p(x_1|x_2) \]
or
\[ E\{g(X_1)|X_2=x_2\}=\int_{x_1}g(x_1)f(x_1|x_2)dx_1. \]
Furthermore,
\[ E(X_1)=E_{X_2}\{E_{X_1|X_2}(X_1|X_2)\} \]
and
\[ Var(X_1) = E_{X_2}\{Var_{X_1|X_2}(X_1|X_2)\} + Var_{X_2}\{E_{X_1|X_2}(X_1|X_2)\} \]
Thursday
Sampling Distributions
Observing Random Variables
When collecting a sample of \(n\), we tend to observe individual random variables: \(\{X_1, X_2, \cdots,X_n\}\).
Sum of Random Variables
Let \(X_i\), for \(i=1, \cdots, n\), be identically and independently distributed (iid) normal distribution with mean \(\mu\) and variance \(\sigma^2\). Let \(T=\sum_{i=1}^nX_i\) follow an normal distribution with mean \(\mu\) and variance \(n\sigma^2\).
Central Limit Theorem
Let \(X_1, X_2, \ldots, X_n\) be identical and independent distributed random variables with \(E(X_i)=\mu\) and \(Var(X_i) = \sigma²\). We define
\[ Y_n = \sqrt n \left(\frac{\bar X-\mu}{\sigma}\right) \mathrm{ where }\ \bar X = \frac{1}{n}\sum^n_{i=1}X_i. \]
Then, the distribution of the function \(Y_n\) converges to a standard normal distribution function as \(n\rightarrow \infty\).
Other Sampling Distributions
\(\chi^2\)-distribution
Let \(Z_1, Z_2,\ldots,Z_n \overset{iid}{\sim}N(0,1)\),
\[ \sum_{i=1}^nZ_i^2\sim\chi^2_n. \]
Let \(X_1, X_2,\ldots,X_n \overset{iid}{\sim}N(\mu,\sigma^2)\), \(S^2 = \frac{1}{n-1}\sum^n_{i=1}(X_i-\bar X)^2\), and \(\bar X \perp S^2\); therefore:
\[ \frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1}. \]
t-distribution
Let \(Z\sim N(0,1)\), \(W\sim \chi^2_\nu\), \(Z\perp W\); therefore:
\[ T=\frac{Z}{\sqrt{W/\nu}} \sim t_\nu \]
F-distribution
Let \(W_1\sim\chi^2_{\nu_1}\) \(W_2\sim\chi^2_{\nu_2}\), and \(W_1\perp W_2\); therefore:
\[ F = \frac{W_1/\nu_1}{W_2/\nu_2}\sim F_{\nu_1,\nu_2} \]
Resources
First Lecture
| Slides | Videos |
|---|---|
| Slides | Video 001 Video 002 |
Second Lecture
| Slides | Videos |
|---|---|
| Slides | Video 001 Video 002 |