I've found the fractional-ordered derivative operator I've been thinking about it since yesterday and have noticed this pattern:
We have, the first order derivative of a function $f(x)$ is:
$$f'(x)=\lim_{h\rightarrow 0}\frac{f(x+h)-f(x)}{h} .......(1)$$
The second order derivative of the same function is:
$$f''(x)=\lim_{h\rightarrow 0}\frac{f'(x+h)-f'(x)}{h}$$
By putting $x=x+h$ in (1), we can have $f'(x+h)$.
So,$$f''(x)=\lim_{h\rightarrow 0}\frac{\lim_{h\rightarrow 0}\frac{f(x+h+h)-f(x+h)}{h}-\lim_{h\rightarrow 0}\frac{f(x+h)-f(x)}{h}}{h}$$
Or , $$f''(x)=\lim_{h\rightarrow 0}\frac{f(x+2h)-2f(x+h)+f(x)}{h^2}.....(2)$$
You can check by L'Hospital's rule that this limit evaluates to $f''(x)$.
Now, the third order derivative of the same function is:
$$f'''(x)=\lim_{h\rightarrow 0}\frac{f''(x+h)-f''(x)}{h}$$
By putting $x=x+h$ in (2), we can get $f''(x+h)$.
So, $$f'''(x)=\lim_{h\rightarrow 0}\frac{\lim_{h\rightarrow 0}\frac{f(x+3h)-2f(x+2h)+f(x+h)}{h^2}-\lim_{h\rightarrow 0}\frac{f(x+2h)-2f(x+h)+f(x)}{h^2}}{h}$$
which gives $$f'''(x)=\lim_{h\rightarrow 0} \frac{f(x+3h)-3f(x+2h)+3f(x+h)-f(x)}{h^3}......(3)$$
Again, by repeating the same process, we can get that:
$$f''''(x)=\lim_{h\rightarrow 0}\frac{f(x+4h)-4f(x+3h)+6f(x+2h)-4f(x+h)+f(x)}{h^4}....(4)$$
So, we observe that the coefficient of $f(x+(n-r)h)$ in the expression of $f^{n}(x)$ ($n^{th}$ derivative of $f(x)$) is actually $(-1)^{r}\cdot {^n}C_r$, same as the coefficient of $x^{n-r}$ in the expansion of $(x-1)^n$.
It can be proved that:
$$f^n(x)=\lim_{h\rightarrow 0}\frac{\sum_{r=0}^n(-1)^{r}\cdot ^{n}C_r\cdot f(x+(n-r)h)}{h^n}$$
where $f^n(x)$ is the $n^{th}$ order derivative of the function $f(x)$.
Now, to generalize this to fractional order derivatives, we just have to generalize the coefficients, which must be similar to the generalization of the expansion of $(x-1)^n$ to fractional exponents.
I'm not very good with binomial theorem, but I guess that it should be:
$$f^n(x)=\lim_{h\rightarrow 0}\frac{f(x+nh)-n\cdot f(x+(n-1)h)+\frac{n(n-1)}{2!}\cdot f(x+(n-2)h)-....}{h^n}$$
, where $n$ can be fractional. Have I done anything wrong?
UPDATE: A lot of people are saying that my method to get the expressions of $f''(x)$, $f'''(x)$, $f''''(x)$, etc are wrong. I've checked by L'Hospital's rule that the results are correct. Could you please write an answer about getting to these results by using a correct method?
UPDATE: On second thoughts, the $n^{th}$ derivative of a function can also proved to be equal to:
$$\lim_{h\rightarrow 0^+}\frac{\sum_{r=0}^{n}(-1)^r\cdot \binom{n}{r}\cdot f(x-(n-r)h)}{(-h)^n}$$
If we try to expand this to fractions, it can turn out to be imaginary because of the fractional power of $-1$ in the denominator.
 A: 
While the following does not address fractional derivatives, the OP has requested in a comment and in an UPDATE in the OP, a way to proceed rigorously to show that $$f''(x)=\lim_{h\to 0}\frac{f(x+h)-2f(x)+f(x-h)}{h^2}$$It is to that end only that we now proceed.


From the extended law of the mean, if $f$ is twice differentiable in the neighborhood of $x$, then for $h$ sufficiently small, there exists a number $\theta\in (0,1)$ such that
$$f(x+h)=f(x)+f'(x)h+\frac12 f''(x+\theta h)h^2 \tag 1$$
Alongside this, the extended law of the mean guarantees that there exists a number $\eta\in(0,1)$ such that
$$f(x-h)=f(x)-f'(x)h+\frac12 f''(x+\eta h)h^2 \tag 2$$
Adding $(1)$ and $(2)$ reveals
$$\frac12(f''(x+\theta h)+f''(x+\eta h))=\frac{f(x+h)-2f(x)+f(x-h)}{h^2} \tag 3$$
If $f''$ is continuous in a neighborhood of $x$, then we have from $(3)$
$$f''(x)=\lim_{h\to 0}\frac12(f''(x+\theta h)+f''(x+\eta h))=\lim_{h\to 0}\frac{f(x+h)-f(x)-f'(x)h}{h^2}$$
And we are done!

To obtain limit expressions for higher order derivatives, we simply use the extended mean value theorem
$$f(x+h)=f(x)+f'(x)h+\frac12f''(x)h^2+\cdots +f^{(n)}(x+\theta h)h^n \tag 4$$
and judiciously select different values combinations of $(4)$ to eliminate all derivative terms.  This is left as an exercise for the reader.

A: As an answer to the 'fractional derivative' part I mention that your derivation was on the way to Gruenwald-Leitnikov derivative. Notice the similarity in the approach and the summation :)
