Distribution of Mean
Distribution of Sample Variance
Likelihood Function
If \(X\) and \(Y\) are independent, then the MGF of \(Z = X + Y\) equals the product of their MGFs:
\[ M_{Z}(t) = M_X(t)M_Y(t). \]
Suppose \(X_1, \dots, X_n \sim \text{N}(\mu, \sigma^2)\)
Suppose \(X_1, \dots, X_n \sim \text{N}(\mu, \sigma^2)\)
The likelihood function tells us how plausible a set of parameter values is, given the observed data.
Given a random sample \(X_1, X_2, \dots, X_n \overset{iid}{\sim} F(x;\theta)\), where \(\theta\) are the parameters that shape the distribution function.
Then, the joint density function of the sample is:
\[ f(x_1, x_2, \dots, x_n; \theta) = \prod_{i=1}^n f(x_i; \theta) \]
This expression, viewed as a function of \(\theta\) (for fixed data \(x_i\)), is the likelihood function:
\[ L(\theta \mid x_1, x_2, \dots, x_n) = \prod_{i=1}^n f(x_i; \theta) \]
| Concept | Interpretation |
|---|---|
| Probability | \(P(X = x \mid \theta)\): \(x\) is random, \(\theta\) fixed |
| Likelihood | \(L(\theta \mid x)\): \(x\) fixed, \(\theta\) variable |
Although they share the same mathematical form, their interpretations differ.
For convenience, we often work with the log-likelihood:
\[ \ell(\theta) = \log L(\theta) \]
Suppose \(X_1, \dots, X_n \sim \text{Bernoulli}(p)\).
\[ f(x_i; p) = p^{x_i}(1 - p)^{1 - x_i} \]
Suppose \(X_1, \dots, X_n \sim \text{Pois}(\lambda)\)
\[ f(x_i; \lambda) = \frac{e^{-\lambda}\lambda^{x_i}}{x_i!} \]
Suppose \(X_1, \dots, X_n \sim \text{N}(\mu, \sigma^2)\)
\[ f(x_i; \lambda) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x_i-\mu)^2}{2\sigma^2}} \]
For a given function \(f(x)\), optimization is the process of finding the value of \(x\) where \(f(x)\) is either the maximum or minimum.
To find \(x\),
\[ f^\prime (x) = 0 \]
\[ f^{\prime\prime} (X) <0 \]
\[ f^{\prime\prime} (X) <0 \]
\[ f(x) = x^2 + 5x + 3 \]
\[ f(x) = -4(3x - 9)^2 + 12 \]