I'm interested in this question because it relates to a bad joke about people in their prime.
It seems to work for the first 20 numbers:
4 is the average of 5 and 3.
6 is the average of 5 and 7.
8 is the average of 5 and 11.
16 is the average of 13 and 19.
It would not be hard to write a program that checks more cases, but I suspect the hypothesis is true. If that is the case, then a more mathematical approach is needed.