How useful are non-square matrices in maths or sciences? I know that a matrix will be either square or rectangular matrix. I know that square matrices are used to solve a system of linear equations. But what's the use of rectangular matrices, why do we study them? Are they used somewhere in Math or science? Please answer my questions.
 A: Rectangular matrices are almost as usable as square matrices, just for different things. For instance, you might be interested in a linear map $\Bbb R^n \to \Bbb R^m$ where $n$ does not equal $m$. Then the map may be represented as a rectangular matrix.
Perhaps the most important everyday application is linear regression where you find the closest almost-solution to an overdetermined system of linear equations (say $900$ equations in $50$ unknowns, for instance).
A: Linear systems can be represented by matrix equations, square or rectangular. Solving them provides you with a set of solutions (potentially empty or infinite, if the system is under- or over-determined).  Mathematics love generalization. 
Square matrices are thus just special cases of rectangular ones, and there are generalized inverses (e.g. Moore-Penrose pseudo-inverses) for rectangular matrices, and many other  tools to work with them. 
They can be used in regression (@Arthur), interpolation and extrapolation, communications and compression (to code, and denote compressed or transmitted data). And many more.
They are used quite often in signal processing and image analysis, especially when you want to enhance blur or noisy data. (Rectangular) linear systems are used to model a type of data degradation. Due to the additional degrees of freedom in rectangular matrices, one can add other hypotheses or limitations (contraints, penalties) that will help you solve (approximately) the whole system with sound, more intuitive solutions. For instance, a blurry image should be crisper or sharper, or simpler (sparser) after the process (called deconvolution, restoration or inversion):

Those rectangular matrices are efficiently coupled with optimization techniques. A quite novel domain is data sampling, and compressive sensing is a typical domain where rectangular matrices, very flat ones, are used.
A: If your augmented matrix of a system in row echelon form can be represented as $Ax=b $ where $A $ is a rectangular matrix, then it just means there are an infinite number of solutions to your system.
However, finding the parameterization of those solutions is still important.
A: Rectangular matrices are used in regular temperament theory as mappings of prime harmonics to temperament generators.
