x + 2y - 3z = 0 3x + 2y + z = 0 5x + 6y - 5z = 0
I was thinking about solving this using Gaussian Elimination, but the result is going to be x = y = z = 0. So could anyone give me a hint how to go about this issue, please?
The determinant of your matrix vanishes. Thus its rank is less than $3$. By inspection, it is also greater than $1$, so it must be $2$. Thus, when you apply Gaussian elimination, you should end up with one row of zeros at the end, which allows you to choose $z=\lambda$ arbitrarily and then express $y$ and $x$ in terms of $\lambda$ using the non-zero rows. That gives you the equation of a line through the origin containing the solutions to the system of equations.