Covariance
Statistics and Inference
Sampling Distributions
Central Limit Theorem
The covariance measures the average dependence between multiple random variables. Let \(W=(^X_Y)\) be a random vector. The variance of \(W\) is defined as
\[ Var(W) = \left(\begin{array}{cc} \sigma^2_X & \sigma_{XY} \\ \sigma_{XY} & \sigma^2_{Y} \end{array}\right) \]
Let \(X_1\) and \(X_2\) be 2 random variables with mean \(\mu_1\) and \(\mu_2\), respectively. The covariance of \(X_1\) and \(X_2\) is defined as
\[ \begin{eqnarray*} Cov(X_1,X_2) & = & E\{(X_1-\mu_1)(X_2-\mu_2)\}\\ & =& E(X_1X_2)-\mu_1\mu_2 \end{eqnarray*} \]
If \(X_1\) and \(X_2\) are independent random variables, then
\[ Cov(X_1,X_2)=0 \]
The correlation of \(X_1\) and \(X_2\) is defined as
\[ \rho = Cor(X_1,X_2) = \frac{Cov(X_1,X_2)}{\sqrt{Var(X_1)Var(X_2)}} \]
Let \(X\) and \(Y\) be independent random variables. Let \(Z = X+Y\), the MGF of Z is
\[ M_Z(t) = M_X(t)M_Y(t) \]
A sampling distribution is the distribution of a statistic. Many known statistics have a known distribution.
Let \(Z\sim N(0,1)\), \(W\sim \chi^2_\nu\), \(Z\perp W\); therefore:
\[ T=\frac{Z}{\sqrt{W/\nu}} \sim t_\nu \]
Let \(W_1\sim\chi^2_{\nu_1}\) \(W_2\sim\chi^2_{\nu_2}\), and \(W_1\perp W_2\); therefore:
\[ F = \frac{W_1/\nu_1}{W_2/\nu_2}\sim F_{\nu_1,\nu_2} \]
Let \(X_1, X_2, \ldots, X_n\overset{iid}{\sim}N(\mu,\sigma^2)\) , show that \(\bar X \sim N(\mu,\sigma^2/n)\). Note: the MGF of \(X_i\) is \(e^{\mu t + \frac{t^2\sigma^2}{2}}\).
Let \(X_1, X_2, \ldots, X_n\) be identical and independent distributed random variables with \(E(X_i)=\mu\) and \(Var(X_i) = \sigma²\). We define
\[ Y_n = \sqrt n \left(\frac{\bar X-\mu}{\sigma}\right) \mathrm{ where }\ \bar X = \frac{1}{n}\sum^n_{i=1}X_i. \]
Then, the distribution of the function \(Y_n\) converges to a standard normal distribution function as \(n\rightarrow \infty\).
\[ \bar X \sim N\left(\mu, \frac{\sigma^2}{n}\right) \]
Let \(X_1, \ldots, X_n \overset{iid}{\sim} \chi^2_p\), the MGF is \(M(t)=(1-2t)^{-p/2}\). Find the distribution of \(\bar X\).