Finding the Gradient of a Tensor Field Finding the Gradient of a Scalar Field
I understand that you can find the gradient of a scalar field, in an arbitrary number of dimensions like so :
$$grad(f) = \vec{\nabla}f = \left<\frac{\partial f}{\partial x_{1}}, \frac{\partial f}{\partial x_{1}},...,\frac{\partial f}{\partial x_{n}}\right> = \begin{bmatrix}
    \frac{\partial f}{\partial x_{1}}  \\     
    \frac{\partial f}{\partial x_{2}}   \\   
    ... \\
    \frac{\partial f}{\partial x_{n}}       
\end{bmatrix}$$
where $f$ is a scalar function, $f: \mathbb{R^n} \to \mathbb{R}$. And as you can see this generalizes consistently to higher-dimensional scalar fields. All of this is straight out of Multivariable Calculus.
Finding the Gradient of a Vector Field
Furthermore finding the gradient of a Vector Field, is given by a Tensor i.e. given $f$ to be a vector function, $f : \mathbb{R^m} \to \mathbb{R^n}$ :
$$grad(\vec{f}) = \displaystyle
 \nabla \vec{f}  = T =
\begin{bmatrix}
\frac{\partial f_1}{\partial x_1} &
\frac{\partial f_1}{\partial x_2} &
... &
\frac{\partial f_1}{\partial x_m} \\
\frac{\partial f_2}{\partial x_1} &
\frac{\partial f_2}{\partial x_2} &
... &
\frac{\partial f_2}{\partial x_m} \\
... &
... &
... &
... \\
\frac{\partial f_n}{\partial x_1} &
\frac{\partial f_n}{\partial x_2} &
... &
\frac{\partial f_n}{\partial x_m} \\
\end{bmatrix}$$
with $T$ denoting the tensor (a $n\ $x$\ n$ matrix of partial derivatives of $\vec{f}$'s scalar components, i.e. rank-$0$ tensor components, correct me if what I said in these brackets is wrong)  which tells us how the vector field changes in any direction.

How to find the Gradient of a Tensor Field?
But how do you find the gradient of a Tensor Field? I understand that to answer this question we may need to generalize the concept of a tensor field a bit further.
If I understand correctly, a scalar is a tensor of rank-$0$, a vector is a tensor of rank-$1$. Is it then fine to generalize scalar fields as tensor fields of rank-$0$, and vector fields as tensor fields of rank-$1$? If so then it means we've been finding the gradient of tensor fields (albeit of rank-0 being scalar fields) in our Multivariable courses all along, we just didn't know it.
By extending the logic behind the leap between taking the gradient of a Scalar Field, to taking the gradient of a Vector Field, is it then correct to say that :
The gradient of a Tensor field of rank-$n$ is a Tensor field of rank-($n+1$) ?
 A: A tensor is just a multilinear, scalar-valued function. If I write $V \multimap W$, for the collection of linear functions from a vector space $V$ to a vector space $W$, then, over the reals, a rank-$n$ tensor is just $V^{\otimes n}\multimap\mathbb R$ where $V^{\otimes n}$ means the $n$-fold tensor product, e.g. $V^{\otimes 2} = V\otimes V$.  A tensor field is just a smoothly indexed family of tensors.
For simplicity, I'll just talk about the manifold $\mathbb{R}^n$, but anywhere I explicitly write out $\mathbb{R}^n$ (as opposed to $V$), you could just as well use a submanifold of $\mathbb{R}^n$, e.g. a 1-dimensional curve in $\mathbb{R}^n$.  A rank-$k$ tensor field on $\mathbb{R}^n$ is a (suitably smooth) function $\tau : \mathbb{R}^n \to (V^{\otimes k} \multimap \mathbb R)$ where $V$ is itself $\mathbb{R}^n$.  Now say we write the directional derivative of $\tau$ in some direction $v \in V$ at $x \in \mathbb{R}^n$ as $D(\tau)(x; v)$.  The result itself would be a rank-$k$ tensor, i.e. $D(\tau)(x; v) : V^{\otimes k} \multimap \mathbb R$.  So what is the type of $D(\tau)$ itself.  Well we know the type of $\tau$ and we know the type of $V$ and we know $D(\tau)(x; v)$ is linear in $v$ and non-linear in $x$. So we have 
$$D(\tau) : \mathbb{R}^n \to (V \multimap (V^{\otimes k}\multimap\mathbb{R}))$$ but it is easy to show that $(V \multimap (V^{\otimes k}\multimap\mathbb{R})) \cong (V\otimes V^{\otimes k} \multimap \mathbb R) = (V^{\otimes k+1}\multimap \mathbb R)$.  Which is to say $D(\tau)$ as a function of $x$ alone — essentially currying $D(\tau)$ — is a rank-$(k+1)$ tensor field.
So to answer your question, you find the gradient of a tensor field by viewing the directional derivative as a linear function of the direction.  When you have a basis, as you do for $\mathbb{R}^n$, this linear function can be represented by a vector of partial derivatives (which are directional derivatives along the basis directions).  A bit more concretely,
$$D(\tau)(x)_i = \frac{\partial\tau}{\partial x_i}(x)$$
And yes, even high school, single variable calculus was doing this for the $n = 1$ case.  It's just that $\mathbb{R}^{\otimes k} \cong \mathbb{R}$ for any $k$, and $(\mathbb{R}\multimap\mathbb{R}) \cong \mathbb{R}$.
