If $3x^2+5xy-2y^2-3x+8y+\lambda$ is a product of linear factors, then the equation $3x^2+5xy-2y^2-3x+8y+\lambda=0$ represents a degenerate conic. This occurs when the associated symmetric matrix is singular, i.e., when $$\det\pmatrix{3&\frac52&-\frac32 \\ \frac52&-2&4\\-\frac32&4&\lambda} = 0.$$ After expanding the determinant, you have a simple linear equation in $\lambda$ to solve.
A related approach is to find the center of the family of conics, which you can do in various ways such as differentiation, then translate to eliminate the linear terms. The resulting equation represents a degenerate conic if the constant term is zero. You can avoid some tedious algebra by taking advantage of the fact that the constant term after this translation is just the value of the original expression evaluated at the center point. For this conic, the center works out to be at $(-4/7,9/7)$. Substituting these values into the original expression yields $\lambda+6$, from which it’s obvious what the value of $\lambda$ should be.