Difference between Backward and Forward differences

In numerical methods we are all familiar with finite difference table where one can identify backward and forward difference within same table e.g. given any entry in finite difference table, one can identify it with both backward and forward differences but with different notations.

Forward difference:

$\Delta y=y_{n+1}-y_{n}$

Backward difference:

$\nabla y=y_{n}-y_{n-1}$

Although the difference between them is visible from definition, but as a any single entry in finite difference table one can call that entry both as backward difference as well as forward difference. So what sort of difference is there between both of these differences?

• What difference does it make anyway? Commented May 6, 2016 at 13:03
• To me, both appears same, but they work differently in procedure of numerical method. Commented May 6, 2016 at 13:30
• Not really sure I understand your question, but since you say in numerical methods, take for example numerical methods to solve ODEs as an example: forward Euler scheme and backward Euler scheme are really different. The former is a so-called explicit method where you don't have to solve linear equations, where as the latter is an implicit method where you need to solve them each step. Commented May 8, 2016 at 13:39

$\newcommand{\Reals}{\mathbf{R}}\newcommand{\Int}{\mathbf{Z}}\newcommand{\Seq}{\mathscr{S}}$Here's one way to organize matters (using notation that in no way pretends to be standard/widely-accepted).

Let $\Seq$ denote the infinite-dimensional vector space of all doubly-infinite real sequences $(y_{n})_{n=-\infty}^{\infty}$. That is, an element of $\Seq$ is a function $y:\Int \to \Reals$.

Let $I$ denote the identity operator on $\Seq$, and define:

• The left shift operator $L:\Seq \to \Seq$ by $(Ly)_{n} = y_{n+1}$.

• The right shift operator $R:\Seq \to \Seq$ by $(Ry)_{n} = y_{n-1}$.

The forward difference operator $\Delta:\Seq \to \Seq$, defined by $(\Delta y)_{n} = y_{n+1} - y_{n}$, satisfies $\Delta = L - I$.

The backward difference operator $\nabla:\Seq \to \Seq$, defined by $(\nabla y)_{n} = y_{n} - y_{n-1}$, satisfies $\nabla = I - R$.

Since $L$ and $R$ are inverses, we have $R\Delta = \nabla$ and $L\nabla = \Delta$. Moreover, $I$, $L$, and $R$ commute, so each commutes with $\Delta$ and $\nabla$, and the difference operators commute with each other. In this setting, there's not much of a technical distinction between forward and backward differences.

If instead we work on the space $\Seq_{\geq 0}$ of "forward sequences" $(y_{n})_{n=0}^{\infty}$, i.e., functions $y:\mathbf{N} \to \Reals$, then $L$ is defined as above, but $$(Ry)_{n} = \begin{cases} y_{n-1} & 1 \leq n, \\ 0 & n = 0. \end{cases}$$ Here, $L$ and $R$ are not invertible: $L$ is not injective, while $R$ is not surjective.

Letting $\Pi_{0}:\Seq \to \Seq$ denote projection to the $n = 0$ term, i.e., $\Pi_{0}y = (y_{0}, 0, 0, \dots)$, we have $LR = I$, but $RL = I - \Pi_{0}$, which sends the initial term ($n = 0$) to zero.

Consequently, the forward and backward difference operators do not commute: \begin{align*} \nabla\Delta &= (I - R)(L - I) = L - I - RL + R, \\ \Delta\nabla &= (L - I)(I - R) = L - I - LR + R = L - 2I + R, \\ [\nabla, \Delta] &= [L, R] = LR - RL = \Pi_{0}. \end{align*}

The question as written is very general, and these two cases may not serve as a definitive answer. They do show, however, that the answer depends substantially on the domain, and should furnish some tools for thinking about the issues in more detail.

Here we can see differences between the forward scheme and the backward scheme directly by equations shown as follows.

Forward Euler scheme: \begin{aligned} y'(t)=f(t,y(t)) \\ \frac{y(t_{j+1})-y(t_{j})}{h} + O(h) = f(t_j,y(t_j)) \\ y_{j+1} = h f(t_j, y_j) + y_j \end{aligned}
where $$j$$ means the last step.

Backward Euler scheme: \begin{aligned} y'(t)=f(t,y(t)) \\ \frac{y(t_{j})-y(t_{j-1})}{h} + O(h) = f(t_j,y(t_j)) \\ y(t_j) = h f(t_j,y_j) + y_{j-1} \end{aligned} which can also be written as: $$y_{j+1} = h f(t_{j+1}, y_{j+1}) + y_{j}$$