Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Is there a way to compute the coefficient of determination $R^2$ in a recursive way?

$R^2$ is defined as following:

$$R^2 \equiv 1 - \frac{SS_{\rm err} }{ SS_{\rm tot}} = 1 - \frac{\sum_i (y_i - f_i)^2}{\sum_i (y_i-\bar{y})^2}$$

where $f_i$ is the estimated value

Regards

share|improve this question

1 Answer 1

I currently have the same problem, and I think I have a solution that converges toward the exact $R^2$.

I'm assuming that you have something to evaluate $f_i$ at each "time" $i$, such as the Recursive Least Squares. Most probably $f_i = x_i*b_i$ where $b_i$ is your best guess of the real coefficient $b$ at time $i$. The thing is that at time $n$, you probably have another estimate $b_n$ of coefficient $b$. So your estimation of $y_i$ would now be $f_i = x_i*b_n$. The trick in my solution is that I'm forgetting this, and I'm only looking at $f_{i,i} = x_i*b_i$ even if I'm at time $n$. I'm calling it $f_{i,i}$ to be clearer and remember that my "real" estimation of $y_i$ at time $n$ would in fact be $f_{i,n}$.

My reasons to do so are:

  • first, without that it would be very difficult to recursively calculate $\sum_i (y_i - f_{i,n})^2$, which would be the real numerator of $R^2$, as $b_n$ would change at every step.
  • second, I think my solution still converges toward the real $R^2$, given some assumptions.
  • third, if you're using the Recursive Least Square with a forgetting factor $\lambda < 1$, it doesn't really make sense to apply the most recent $b_n$ to old $x_i$ values.

To be more accurate on $b_n$, I'm assuming that you have a way to evaluate $b_{n+1}$ using $b_n$ and the observations $y_{n+1}$ and $x_{n+1}$.

With that in mind, let's start. First, let's write $R_n^2 = 1 - \frac{N_n}{D_n}$ and have the denominator $$D_n = {\sum_{i=0}^n (y_i-\bar{y_n})^2}$$

Let's try to calculate $D_{n+1}$. Using $\bar{y_{n+1}} = \bar{y_n} + \frac{1}{n+1}(y_{n+1} - \bar{y_n})$, with several lines of calculations I arrive at this result (any check/confirmation welcome!): $$D_{n+1} = D_n + \frac{n}{n+1}(\bar{y_n}-y_{n+1})^2$$

On the other hand, let's have the numerator $$N_n = {\sum_{i=0}^n (y_i - f_{i,i})^2}$$ (see that here I'm using $f_{i,i}$ and not $f_{i,n}$.) We immediately have: $$N_{n+1} = N_n + (y_{n+1} - f_{n+1,n+1})^2$$

Now if you look at $D_{n+1}$ and $N_{n+1}$, you should have everything you need to do your recursion:

  • $y_{n+1}$ is the new observation,
  • $\bar{y_n}$ is easily calculated with recursion,
  • $f_{n+1,n+1} = x_{n+1}*b_{n+1}$, where $x_{n+1}$ is an observation and I've assumed you can recursively calculate $b_{n+1}$.

With initializations such as $D_0 = 0$, $N_0 = 0$ and $\bar{y_n} = 0$ you can make your recursion work by calculating every element and finally have $$R_{n+1}^2 = 1 - \frac{N_{n+1}}{D_{n+1}}$$

That's it. I've also tried to follow the same path but starting with the formula: $$R^2 \equiv \frac{\sum_i (f_i - \bar{f_i})^2}{\sum_i (y_i-\bar{y})^2}$$ (which should be equal to yours). Calculation of the numerator is a bit longer but it's OK. I've done a few tests and I get slightly different results.

I've also compared the $R^2$s obtained with those methods to the "real" $R^2$ calculated in a classic manner. The results are a bit different but quite close. I'll have to do some more tests and give it some more thoughts to really understand what's going on. I'd be happy to have you thoughts on that.

IMPORTANT EDIT

Actually it's not that difficult to calculate $\sum_i (y_i - f_{i,n})^2$, the "real" numerator. I have: (may it should be noted here that $x_i$ and $b_n$ are vectors): $$N_{n+1} = \sum_{i=1}^{n+1} (y_i - f_{i,n+1})^2 = \sum_{i=1}^{n+1} (y_i - x_i*b_{n+1})^2$$ $$ = \sum_{i=1}^{n+1} (y_i^2 - 2y_i*x_i*b_{n+1} + (x_i*b_{n+1})^2)$$ $$ = \sum_{i=1}^{n+1} y_i^2 - 2(\sum_{i=1}^{n+1}y_i*x_i)*b_{n+1} + b_{n+1}^T*(\sum_{i=1}^{n+1}x_i^T*x_i)*b_{n+1}$$

And every term of this last line can be calculated by recursion.

I've implemented this and it works very well. I'm leaving my first idea in my post because it's not so bad, I still think it converges, and overall it's easier to compute and some folks may be interested in calculation speed.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.