## 随机变量的期望

$$\mathbb{E}(X)=\int x d F(x)= \begin{cases}\sum_{x} x f(x) & \text { if } X \text { is discrete } \\ \int x f(x) \mathrm{d}x & \text { if } X \text { is continuous }\end{cases}$$

• $f (x)$ 是概率密度函数
• 积分范围是 $x$ 的域

$k^{\text {th}}$ moment 定义为 $\mathbb {E}(X^k)$ 假定 $\mathbb {E}(|X|^k) < \infty$

$k^{\text {th}}$ central moment 定义为 $\mathbb {E}((X - \mu)^k) = \int_{-\infty}^{+ \infty}(x - \mu)^k f (x) \mathrm {d} x$

$$\mathbb{E}(X)=\mathbb{E} X=\int x d F(x)=\mu=\mu_{X}$$

Lazy Statistician 法则：

$$\mathbb{E}(Y)=\mathbb{E}(r(X))=\int r(x) d F_{X}(x)$$

## 期望的性质

$$\mathbb{E}\left(\sum_{i} a_{i} X_{i}\right)=\sum_{i} a_{i} \mathbb{E}\left(X_{i}\right)$$

$$\mathbb{E}\left(\prod_{i=1}^{n} X_{i}\right)=\prod_{i} \mathbb{E}\left(X_{i}\right)$$

## 方差与协方差

Let $X$ be a random variable with mean $\mu$. The variance of $X$ – denoted by $\sigma^2$ or $\sigma_X^2$ or $\mathbb{V}(X)$ or $\mathbb{V}X$ – is defined by

$$\sigma^2 = \mathbb{E}(X - \mu)^2 = \int (x - \mu)^2\; dF(x)$$

assuming this expectation exists. The standard deviation is $\text{sd}(X) = \sqrt{\mathbb{V}(X)}$ and is also denoted by $\sigma$ and $\sigma_X$.

Theorem 4.14. Assuming the variance is well defined, it has the following properties:

$\mathbb{V}(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2$ 2

If $a$ and $b$ are constants then $\mathbb{V}(aX + b) = a^2 \mathbb{V}(X)$

If $X_1, \dots, X_n$ are independent and $a_1, \dots, a_n$ are constants then

Theorem 4.16. Let $X_1, \dots, X_n$ be IID and let $\mu = \mathbb{E}(X_i)$, $\sigma^2 = \mathbb{V}(X_i)$. Then

$$\mathbb{E}\left(\overline{X}_n\right) = \mu, \quad \mathbb{V}\left(\overline{X}_n\right) = \frac{\sigma^2}{n}, \quad \text{and} \quad \mathbb{E}\left(S_n^2\right) = \sigma^2$$

$$\rho = \rho_{X, Y} = \rho(X, Y) = \frac{\text{Cov}(X, Y)}{\sigma_X \sigma_Y}$$

$$\mathbb{V}(X + Y) = \mathbb{V}(X) + \mathbb{V}(Y) + 2 \text{Cov}(X, Y) \quad \\ \text{ and } \quad \mathbb{V}(X - Y) = \mathbb{V}(X) + \mathbb{V}(Y) - 2 \text{Cov}(X, Y)$$

$$\mathbb{V}\left( \sum_i a_i X_i \right) = \sum_i a_i^2 \mathbb{V}(X_i) + 2 \sum \sum_{i < j} a_i a_j \text{Cov}(X_i, X_j)$$

## 重要随机变量的期望和方差

$$\begin{array}{lll} \text{Distribution} & \text{Mean} & \text{Variance} \\ \hline \text{Point mass at } p & a & 0 \\ \text{Bernoulli}(p) & p & p(1-p) \\ \text{Binomial}(n, p) & np & np(1-p) \\ \text{Geometric}(p) & 1/p & (1 - p)/p^2 \\ \text{Poisson}(\lambda) & \lambda & \lambda \\ \text{Uniform}(a, b) & (a + b) / 2 & (b - a)^2 / 12 \\ \text{Normal}(\mu, \sigma^2) & \mu & \sigma^2 \\ \text{Exponential}(\beta) & \beta & \beta^2 \\ \text{Gamma}(\alpha, \beta) & \alpha \beta & \alpha \beta^2 \\ \text{Beta}(\alpha, \beta) & \alpha / (\alpha + \beta) & \alpha \beta / ((\alpha + \beta)^2 (\alpha + \beta + 1)) \\ t_\nu & 0 \text{ (if } \nu > 1 \text{)} & \nu / (\nu - 2) \text{ (if } \nu > 2 \text{)} \\ \chi^2_p & p & 2p \\ \text{Multinomial}(n, p) & np & \text{see below} \\ \text{Multivariate Nornal}(\mu, \Sigma) & \mu & \Sigma \\ \end{array}$$

MGF

$$$$\begin{array}{ll}\text { Distribution } & \mathrm{MGF} \psi(t) \\ \hline \operatorname{Bernoulli}(p) & p e^{t}+(1-p) \\ \operatorname{Binomial}(n, p) & \left(p e^{t}+(1-p)\right)^{n} \\ \operatorname{Poisson}(\lambda) & e^{\lambda\left(e^{t}-1\right)} \\ \operatorname{Normal}(\mu, \sigma) & \exp \left\{\mu t+\dfrac{\sigma^{2} t^{2}}{2}\right\} \\ \operatorname{Gamma}(\alpha, \beta) & \left(\dfrac{1}{1-\beta t}\right)^{\alpha} \text { for } t<1 / \beta\end{array}$$$$

## 条件期望

### 条件分布列

$$p_{X \mid A}(x)=\mathrm{P}(X=x \mid A)=\dfrac{\mathrm{P}(\{X=x\} \cap A)}{\mathrm{P}(A)} \text {. }$$

$$\mathrm{P}(A)=\sum_{x} \mathrm{P}(\{X=x\} \cap A) .$$

$$\sum_{x} p_{X \mid A}(x)=1,$$

### 条件随机变量

$$p_{X \mid Y}(x \mid y)=\mathrm{P}(X=x \mid Y=y) .$$

$$p_{X \mid Y}(x \mid y)=\dfrac{\mathrm{P}(X=x, Y=y)}{\mathrm{P}(Y=y)}=\dfrac{p_{X, Y}(x, y)}{p_{Y}(y)} .$$

$$\sum_{x} p_{X \mid Y}(x \mid y)=1 .$$

### 条件期望

$$\mathbb{E}(X \mid Y=y)=\left\{\begin{array}{ll} \sum x f_{X \mid Y}(x \mid y) \mathrm {d} x & \text {离散情形} \\ \int x f_{X \mid Y}(x \mid y) \mathrm {d} x & \text {连续情形} \end{array}\right.$$

$$\mathbb{E}(r(X, Y) \mid Y=y)=\left\{\begin{array}{ll} \sum r (x, y) f_{X \mid Y}(x \mid y) \mathrm {d} x & \text {离散情形} \\ \int r (x, y) f_{X \mid Y}(x \mid y) \mathrm {d} x & \text {连续情形} \end{array}\right.$$

### 迭代期望法则

$\dfrac {X}{2}$ 怎么来的：因为相当于截断一个长度为 $X$ 的棍子。相当于转换为一个均匀分布子问题。

$\mathbb{E}(Y) = \mathbb{E}( \mathbb{E}(Y|X) ) = \mathbb{E}(\dfrac{X}{2}) = \dfrac{1}{4}$

$$\mathbb{E}[\mathbb{E}(Y \mid X)]=\mathbb{E}(Y), \\ \mathbb{E}[\mathbb{E}(X \mid Y)]=\mathbb{E}(X)$$

$$\mathbb{E}[\mathbb{E}(r(X, Y) \mid X)]=\mathbb{E}(r(X, Y))$$

\begin{aligned} \mathbb{E}[\mathbb{E}(Y \mid X)] &=\int \mathbb{E}(Y \mid X=x) f_{X}(x) \mathrm{d}x=\iint y f(y \mid x) \mathrm{d}y f(x) \mathrm{d}x \\ &=\iint y f(y \mid x) f(x) \mathrm{d}x \mathrm{d}y=\iint y f(x, y) \mathrm{d}x \mathrm{d}y \\ &=\mathbb{E}(Y) \end{aligned}

$$f_{X, Y}(x, y)= \begin{cases}\frac{1}{3}(x+y), & 0 \leqslant x \leqslant 1,0 \leqslant y \leqslant 2 \\ 0, & \text { else. }\end{cases}$$

$\mathbb{V}(2 X-3 Y+8) = \mathbb{V}(2X - 3Y)$

\begin{aligned} \mathbb{E}(R) &=\iint_{D} r(x, y) f_{X,Y}(x, y) \mathrm{d}y \mathrm{d}x \\ &=\iint_{D}(2 x-3 y) \dfrac{1}{3}(x+y) \mathrm{d}y \mathrm{d}x \\ &=\int_{0}^{1} \mathrm{d}x \int_{0}^{2} \dfrac{1}{3}(2 x-3 y)(x+y) \mathrm{d}y \\ &=\dfrac{1}{3} \int_{0}^{1} \mathrm{d}x \int_{0}^{2} 2 x^{2}-3 y^{2}-x y \mathrm{d}y \\ &=\dfrac{1}{3} \int_{0}^{1} \mathrm{d}x\left(2 x^{2} y-y^{3}-\dfrac{x y^{2}}{2}\right)_{y \in[0,2]} \\ &=\dfrac{1}{3} \int_{0}^{1} 4 x^{2}-2 x-8 \mathrm{d}x \\ &=\dfrac{2}{3} \int_{0}^{1} 2 x^{2}-x-4 \mathrm{d}x \\ &=\dfrac{2}{3}\left(\dfrac{2}{3} x^{3}-\dfrac{x^{2}}{2}-4 x\right) _{x \in[0,1]} \\ &=\dfrac{2}{3}\left(\dfrac{2}{3}-\dfrac{1}{2}-4\right) \\ &=\dfrac{2}{3}\left(\dfrac{4}{6}-\dfrac{3}{6}-\dfrac{24}{6}\right) \\ &=\dfrac{2}{3} \cdot\left(\dfrac{1}{6}-\dfrac{24}{6}\right) \\ &=-\dfrac{23}{9} \end{aligned}

## 习题

♞1 Suppose we play a game where we start with $c$ dollars. On each play of the game you either double your money or half your money, with equal probability. What is your expected fortune after $n$ trials?

$$\mathbb{E}(X_n) = \left(\frac{5}{4}\right)^n c$$

♞2 Show that $\mathbb{V}(X) = 0$ if and only if there is a constant $c$ such that $\mathbb{P}(X = c) = 1$.

$\mathbb{P}(X - \mathbb{E}(X) \neq 0) = 0$

♞3 Let $X_1, \dots, X_n \sim \text{Uniform}(0, 1)$ and let $Y_n = \max \{ X_1, \dots, X_n \}$. Find $\mathbb{E}(Y_n)$.

$$\mathbb{E}(Y_n) = \int_{}^{}y f_Y(y) \mathrm{d}y$$

$Y_n$ 的累计分布函数：

$$F_{Y_n}(y) = \mathbb{P}(Y_n \leq y) = \prod_{i=1}^n \mathbb{P}(X_i \leq y) = y^n$$
• $F_{Y_n}(y) = \mathbb{P}(Y_n \leq y)$ 这一步，是累计分布函数的性质。
• 如何求出这个概率呢？在上一章随机向量那一节讲过，独立同分布的前提下，有： $\mathbb{P}(X_1 \in A_1, \dots, X_n \in A_n) = \prod_{i=1}^n \mathbb{P}(X_i \in A_i)$
• 因此：$\mathbb {P}(Y_n \leq y) = \mathbb {P}\left (\bigcap_{i=1}^{n}\left\{X_{i} < y \right\}\right) = \prod_{i=1}^n \mathbb {P}(X_i \leq y)$
• 而 $\mathbb {P}(X_i \leq y) = y$
• 所以 $\prod_{i=1}^n \mathbb {P}(X_i \leq y) = y^n$

\begin{align} & E\left[\max \left(X_{1}, \ldots, X_{n}\right)\right] \\ &=\int_{0}^{\infty} P\left(\max \left(X_{1}, \ldots, X_{n}\right) \geq y\right) \mathrm{d} y \\ &=\int_{0}^{1}\left(1-y^{n}\right) \mathrm{d} y \\ &=\frac{n}{n+1} \end{align}

♞4 A particle starts at the origin of the real line and moves along the line in jumps of one unit. For each jump the probability is $p$ that the particle will move one unit to the left and the probability is $1 - p$ that the particle will jump one unit to the right. Let $X_n$ be the position of the particle after $n$ units. Find $\mathbb{E}(X_n)$ and $\mathbb{V}(X_n)$. (This is known as a random walk.)

\begin{align} &\mathbb{E}(X_{i+1} \mid_{\mathbb{E}(X_i) = x}) \\&= p(x-1) + (1-p) (x+1) \\&= px - p + x - px + 1- p \\&= 1+x-2p \end{align}

$$\mathbb{E}(X_{i+1}) = 1-2p + \mathbb{E}(X_i)$$

\begin{align} \mathbb{V}(X) &= \mathbb{E}(X^2) - \mu ^2 \\ &= \sum_{}^{}x^2 \mathbb{P}(X=x) - \mu ^{2} \\ \end{align}

\begin{aligned} &\mathbb{V} \left(\frac{Y i+1}{2}\right)=p(p-1) \\ &\Rightarrow \mathbb{V}(Y i)=4 p(p-1) \end{aligned}

\begin{aligned} \mathbb{V}(X) &=\mathbb{V}\left(\sum_{i=1}^{n} Y_{i}\right) \\ &=\sum_{i=1}^{n} \mathbb{V}\left(Y_{i}\right) \\ &=4 n p(p-1) \end{aligned}

$\mathbb{E}(Y^2) = \mathbb{E}(e^{2X}) = M_X(2) = e^2$

1. 一个通俗的证明 在此，建议了解一下，从此再也不会记混平方的位置 ↩︎