Is there a notion in mathematics saying that, in a sense, all finite dimensions are actually infinite dimensional? So then every ordered pair or triplet and so on would be actually represented by an infinite sequence of numbers, and what we think of as 3 dimensions would mean that the point has an infinite number of zeros in its sequence starting with the 4th term. Only the first 3 terms of the sequence are not necessarily 0. For instance the point $(1,2,3)$ could be written instead as $(1,2,3,0,0,0...)$
In other words, in n dimensions, the first n terms would correspond to the standard notion of a coordinate in $\mathbb R^n$, and the rest are zero. Does anyone use this, or something similar, in mainstream mathematics? I feel as though I've heard of it somewhere. 
 A: It is possible to force this perspective, but it would be incorrect to say that the mathematics "actually" is this way.
To be more specific, yes, you can choose to consider the inclusions $j_n\colon \mathbb{R}^n\longrightarrow\mathbb{R}^\infty$ defined by
$$j_n(x_1,\ldots,x_n)=(x_1,\ldots,x_n,0,0\ldots)$$
However, the objects $\mathbb{R}^n$ have their own independent existence - in no way are they "really" subspaces of $\mathbb{R}^\infty$, and the maps $j_n$ aren't special.
A: No, just because a finite dimensional space can be embedded in an infinite dimensional space does not make it "actually infinite dimensional". Dimension is well defined, and preserved under embedding.
If you are wondering if any people use standard embeddings of $\Bbb R^n$ into each other (like the conventional $\Bbb N\subset \Bbb Z\subset \Bbb Q\subset\Bbb R\subset\Bbb C$) or into a common infinite dimensional space, the answer is no. For linear algebra purposes making such identifications would only be confusing.
However, there are rather standard injective linear maps from $\Bbb R^n$ into the vector space $\Bbb R[X]$ of polynomials, where coordinates give polynomial coefficients in increasing order. The image of $\Bbb R^n$ is the subspace* $\Bbb R[X]_{<n}$ of polynomials of degree strictly less than$~n$. Now the images of the various $\Bbb R^n$ form a "standard flag", a chain of successively included subspaces of $\Bbb R[X]$, and their union is all of$~\Bbb R[X]$.
Note that $\Bbb R[X]$ is a much smaller space than the space denoted by $\Bbb R^\infty$ or $\Bbb R^\Bbb N$ of unrestricted infinite sequences of real numbers, which has uncountable dimension; the latter space could serve to embed everything in, but it would be quite wasteful.

*By some conspiracy almost all authors that mention it seem to denote this space instead by $\Bbb R_{n-1}[X]$ or by $P_{n-1}(X)$, so as mask the fact that is has dimension$~n$. And none seem to ever need the image of $\Bbb R^0$.
A: First you have to understand that the way you formulated the question is vague enough to permit answers you may find unsatisfactory.
Second, yes, there is something like what you want. For example, take the set of real valued sequences $$\mathbb{R}^\mathbb{N}= \{ (a_n)_{n\in\mathbb{N}} : a_n \in \mathbb{R} \}$$
and give it the standard $\mathbb{R}$ vector space structure. Now, consider for every $k \in \mathbb{N}$ the subspace $$\mathbb{E}_k = \{ (a_n)_{n\in\mathbb{N}} \in \mathbb{R}^\mathbb{N}  : a_n = 0 \; \; \; \forall n > k\}$$
For every  $k \in \mathbb{N}$ you have that $\mathbb{R}^k$ and $\mathbb{E}_k$ are isomorphic as $\mathbb{R}$ vector spaces. This way, you can say in a formal fashion that there is a copy of every finite dimensional $\mathbb{R}$ vector space inside $\mathbb{R}^\mathbb{N}$, which is infinite dimensional.
