The infinite-dimensional limit of sequence of solutions of linear equations when the number of equations goes to infinity Suppose we have an infinite-dimensional real vector $y=(y_1,...)$. Suppose we have an infinite-dimensional real matrix $C=(c_{ij})$, $i,j\in\mathbb{N}$. Let $C^k$ be a submatrix of $C$, $C^k=(c_{ij})_{i,j=1,k}$ and $y_k$ a subvector of $y$, $y^k=(y_1,...,y_k)$. Define 
$$\theta^k=C_k^{-1}y^k$$
My question (which probably is too general) is what conditions should $C$ and $y$ satisfy so that pointwise limits 
$$\lim_{k\to\infty}\theta^k_i$$
exist? I got a feeling that this could be easily solved by applying theory of linear operators, but I cannot figure out how to reformulate the problem. 
To make this question less general we can assume that $y\in\ell_2$ and $C_k$ is symmetric positive-definite matrix for each $k$. 
This question is related to this one I've asked on mathoverflow.  
Update @fedja below produces a possible sketch of a proof. It requires though that $\|C_k^{-1}\|$ is bounded sequence (take the matrix norm $\|\|_2$). If we suppose that $C$ is a linear operator in $\ell_2$, does the property that each submatrix $C_k$ is positive-definite ensures that the sequence $\|C_k^{-1}\|$ is bounded? 
This question can be further rephrased the following way. For symmetric positive-definite matrix $A$ denote its minimal and maximal eigen values by $\lambda_{\min}$ and $\lambda_{\max}$. Then $\|A\|_2=\lambda_{\max}$ and $\|A^{-1}\|_2=\lambda_{\min}^{-1}$. With this in mind the previous question is identical whether $\lambda_{min}(C_k)$ is bounded away from zero.
 A: I'm not sure what exactly you are looking for but if $\|C\|<+\infty$, $\sup_k \|C_k^{-1}\|<+\infty$, and $y\in\ell^2\cap C\ell^2$, then the convergence holds and, moreover, $\theta_k$ converge to the (unique) solution $\theta$ of $C\theta=y$ in $\ell^2$. The reason is very simple. If $P_k$ is the orthogonal projection to the first $k$ coordinates, then $C_k$ is the non-trivial part of $P_kCP_k$ so if $\theta_k$ is extended by zeroes, we have $P_kCP_k\theta_k=P_ky$. On the other hand, $|P_kCP_k\theta-P_ky|\le \|C\|\cdot|P_k\theta-\theta|$. Thus, $|\theta_k-P_k\theta|\le\|C_k^{-1}\|\cdot\|C\|\cdot |P_k\theta-\theta_k|\to 0$ as $k\to\infty$.
A: This is closer to a comment than an answer, but it would be too long for a comment. Basically a few thougths that might or might not help. (Just a things this reminded me of, not necessarily helpfull for your problem.)
Let us rewrite your equation as $y^k=C_k\theta^k$. If I consider $y^k$ and $\theta^k$ as infinite sequences (I add zeroes), then I have
$$z^k=C\theta^k,$$
where the vector $z^k$ has the same values as $y^k$ on the first $k$ coordinates. 
If we additionaly assume that $y^k\to y$ (which implies $z^k\to y$) and $\theta^k\to\theta$ pointwise and that the operator corresponding to $C$ is in some sense continuous, then we get
$$y=C\theta.$$
Hence a necessary condition is that $y$ is in the range of $C$.
My guess is that this condition should be sufficient for the continuity of $C$:
$$\|C\| = \sup_n \sum_k |c_{nk}| < \infty$$

I am not sure that this really helps but the last equation $$y=C\theta$$ is related to summability theory. In fact, matrix methods studied in summability theory are defined in the way that you transform a sequence using a given matrix and then take a limit.
For overview of some known results you can consult e.g. Boos: Classical and modern methods in summability, you can find also some notes here: http://thales.doa.fmph.uniba.sk/sleziak/texty/rozne/trf/boos/
From a functional analytic viewpoint: Morrison's Functional analysis
