For a process $A_t$, does $|| \mathbb{E}[A_{t+1}] ||_\infty \leq \delta ||A_t||_\infty$ imply that $A_t \to 0$? I'm currently studying some stochastic approximation-like algorithms and their convergence properties. I'm wondering whether some convergence results from deterministic sequences can be transferred to random-processes.
Consider a (non-random) sequence $(a_t)_t$ which satifies, for constant $\delta \in (0, 1)$:
$$|a_{t+1}| \leq \delta |a_t| \ \forall t$$
Then, $a_t \rightarrow 0$.
Now consider a random process $(A_t)_t$ with:
$$|| \mathbb{E}[A_{t+1}] ||_\infty \leq \delta ||A_t||_\infty$$
Does $A_t \rightarrow 0$ as well? Or are there other requierements that have to be fullfilled?
Thanks in advance!
 A: @PhoemueX gave a counterexample in the comments that showed that this doesn't hold in general.
You asked for other requirements for this to be true. Below is one example of a situation where your assertion holds.
First, since we're talking random variables, we have more than one type of convergence. I'm assuming $A_t \to 0$ means $A_t {\to} 0\; almost\; surely\;(a.s)$.
Assumption 1 $A_t:\Omega \to \mathbb{R^+}$
This allows your condition to be simplified:
$$||E[A_{t+1}]||_{\infty} \leq \delta ||A_t||_{\infty} \implies E[A_{t+1}] \leq \delta A_t$$
Now, note that $E[A_{t+1}]$ is bounded by a random variable, so unless $E[A_t] = 0\;\;\forall t$ we need to assume something about the conditional expectation $E[A_{t+1}|A_t]$. Let's assume that it also follows your bound
Assumption 2: $E[A_{t+1}|A_t] \leq \delta A_t$
Assumptions 1 and 2 imply that $A_t$ is a supermartingale bounded from below.
This allows us to apply Doob's Supermartingale Convergence Theorem:
$$\sup E[A_t^-] =0<\infty \implies \lim_{t\to\infty}A_t(\omega) = A(\omega)<\infty\;\; a.s.$$
$$ \textrm{Where}\; A_t^- = \max(-A_t,0)$$
Therefore, we've shown that $\lim_{t\to\infty}A_t$ is finite and exists $a.s.$
Now, we need to show $A(\omega) = 0\;a.s.$
We can take advantage of our conditional expectation formulation to show that the expected value of $A_t$ converges to $0$:
$$E[A_{t+1}|A_t] \leq \delta A_t \implies E[E[A_{t+1}|A_t]] \leq E[\delta A_t]=\delta E[A_t]$$
Since $E[E[X|Y]] = E[X]$ (see law of total expectation) we have the deterministic sequence:
$$E[E[A_{t+1}|A_t]] = E[A_{t+1}] \leq \delta E[A_t]$$
From here we can get:
$$E[A_t] = \delta^t E[X_0] \implies \lim_{t\to\infty} E[A_t] = 0$$
However, convergence in mean doesn't imply convergence $a.s.$ To get to that, we need to use Markov's Inequality:
$$X\geq 0 \implies P(X\geq a)\leq \frac{E[X]}{a}\;\;\forall a > 0$$
Applying this to our case, we have:
$$A_t \geq 0 \implies P(A_t \geq a) \leq \frac{E[A_t]}{a} = \frac{\delta^t E[X_0]}{a}\;\forall a > 0$$
Since we've shown that $E[A_t] \to 0$ we can conclude that $A_t \to 0\; a.s.$
$$\lim_{t\to\infty} P(A_t\geq a) = 0 \;\;\forall a>0$$
Since this holds for any $a>0$, we can conclude that $A_t$ converges to the degenerate random variable $A=0$ almost surely.
