Stopping rules. Consider $v(x)$, the optimal stopping problem again.
If $T$ is a random variable taking values in $(0,\infty]$, does the following limit exist? If it exists, what is its value?
Consider the biased random walk on ${0,\ldots,N}$ with $p < 1/2$. Assuming that is a martingale, compute where $T = \inf { n > 0 \colon S_n = 0 \OR N }$. Hint: justify the use of the optional sampling theorem here. <div class=pagebreak-before> </div>
Consider a simple random walk on ${0,1,\ldots}$. This is problem 4.9. $0$ is an absorbing boundary and $f(x) = x^2$ is the payoff.
Let $X_1,X_2,\ldots$ be iid random variables. Consider $M_n = X_1 \cdot X_2 \cdots X_n$ and the filtration $\mathcal{F}_n = \sigma(X_1,\ldots,X_n)$. Under what conditions on $\E[X_i]$ is $M_n$ a martingale?
Consider $S_n = \sum_{i=1}^n X_i$. Compute $\E[ S_n^3 | \mathcal{F}_m]$ for $m < n$. |
<div class=pagebreak-before> </div>
Suppose $v(x)$ is the value function of the optimal strategy $T$ with STOP and CONTINUE sets. Is it possible to have
$v(x) > Pv(x)$ on the CONTINUE set? Why or why not?
Is it possible to have $v(x) < Pv(x)$ on the CONTINUE set? Why or why not?
If $P$ is an irreducible Markov chain, what is $v(x)$? Assume some general payout function $f(x)$ on some state space $S$.
Suppose ${u_i}_{i=0}^{\infty}$ is a collection of super harmonic function satisfying $u_i(x) \geq P u_i(x)$ and $u_i(x) \geq C$ for all $i = 0,1,\ldots$ and some constant $C$. Show that $u(x)$ is superharmonic.
Define the state space $S$ and payout function $f$ as follows. Suppose $P$ is a simple random walk with absorbing barriers at $10$ and $0$.
Find the STOP and CONTINUE sets, and justify your answer.