# Exponential Martingale Intuition

Let $$X_1,X_2,...$$ be a sequence of i.i.d random variables with a moment-generating function $$\phi(\theta) = E[e^{\theta X_i}] < \infty$$ We set ($$S_0 := 0$$) and $$S_n=X_1+...+X_n$$. Let $$(F_n)=(F_n^X)$$, the filtration which is generated by x.

Then $$M_n=\frac{e^{\theta S_n}}{\phi(\theta)^n}$$ is a martingale.

I am able to prove that $$M_n$$ is a martingale by showing that $$E[M_{n+1}|F_n] = M_n$$

However, my question is as follows:

We are given a moment generating function $$\phi(\theta)$$. Therefore, is it obvious or able to be deduced that $$M_n$$ is the above martingale or does it appear that this was given to us. Meaning, how would we be able to construct such a martingale or others given the moment generating function as a part in the expression for the martingale. Secondly, if so, what does are expression representating a martingale tell us. The way I read the expression is that the martingale is equal to the exponential of some parameter $$\theta$$ times the stochastic process up to step $$n$$ divided by the moment generating function to the n-th degree. I guess I am a bit confused as to what this term represents and how it came to construction.

I assume that your MGF exists for arbitrary $$\theta \in \mathbb R$$. (The easiest way to get this is to start with bounded $$X_i$$ then try to relax this condition.)

What you're looking at is part of a general family of techniques called 'exponential tilting'. That is, tilting some random variable's distribution ($$S_n$$'s or $$X_i$$'s if you prefer) from something mediocre into something nice. Outside of martingales it is used e.g. in importance sampling and some direct random walk applications. Part of the idea is that while all random variables have characteristic functions, not all random variables have MGFs. So if the MGF exists, then that implies special structure which we can exploit.

Regarding martingales
(again assuming all $$X_i$$ are real valued and nice enough to have MGFs for all real $$\theta$$ for purposes of this discussion)

The easiest, classical martingale is the

(i) given by sums of iid zero mean random variables. A closely related one is

(ii) the product form of a martingale involving iid random variables with mean one.

now in your problem of summing iid $$X_i$$, if $$E\big[X_i\big] \neq 0$$ it isn't classical or particularly nice. However if you lift these $$X_i$$ through the exponential function, while including a slack parameter $$\theta$$ you can make it nice (this is 'exponential tilting').

The random variable $$\frac{e^{\theta X_i}}{\phi(\theta)}$$ has mean one and this allows us to induce the product form of martingale, i.e. making use of

$$\frac{e^{\theta S_n}}{\phi(\theta)^n} = \prod_{i=1}^n\frac{e^{\theta X_i}}{\phi(\theta)}$$

The fact that you have a mean one, iid random variable martingale created via lift through the exponential function should immediately tell you that you 'almost' have recreated (ii). The MGF is in the denominator to ensure that this has mean one. But, this is an awkward point and it would be nice to 'get rid of' the MGF in the denominator. Since for this problem $$\phi(\theta)$$ is finite for all $$\theta$$, for reasons of convexity and the fact that $$E\big[X_i\big] \neq 0$$,

the function given by $$h(\theta)=\phi\big(\theta\big)-1$$ has two distinct roots-- $$\{0, \theta^*\}$$. The first trivially holds but the second one is non-zero and if you plug it in, you get

$$e^{\theta^* S_n}= \prod_{i=1}^n e^{\theta^* X_i}$$

so now we have recovered classical martingale (ii) -- product form with mean one random variables $$e^{\theta^* X_i}$$ by making a wise selection of $$\theta^*$$ for our slack parameter.

A lot of results related to martingales are hardly 'obvious'. The fact that you can use an MGF to tilt a distribution to something pleasant isn't particularly 'obvious' though its a technique used so often that you get used to it. (A closely related, simpler question, is -- is the Chernoff Bound 'obvious' or something given to us?)

• This was excellently explained. I will definitely share this with my stochastics lecturer. Thank you for this rigorous response! :) Commented Jan 3, 2020 at 23:38