# Maximum likelihood estimation steps

If we have the linear model $$y = \beta^Tx + \epsilon$$ and assuming $$\epsilon \sim N(0, \sigma^2)$$, we can write \begin{align} p(\epsilon) = \frac{1}{\sqrt{2\pi}\sigma}\exp\left({-\frac{(\epsilon)^2}{2\sigma^2}}\right) \end{align}

Since we know $$\epsilon = y - \theta^Tx$$, we can write

\begin{align} p(y - \theta^Tx) = \frac{1}{\sqrt{2\pi}\sigma}\exp\left({-\frac{(y - \theta^Tx)^2}{2\sigma^2}}\right) \end{align}

and apparently the above is equivalent to writing \begin{align} p(y|x; \theta) = \frac{1}{\sqrt{2\pi}\sigma}\exp\left({-\frac{(y - \theta^Tx)^2}{2\sigma^2}}\right) \end{align}

I am confused with going from the 2nd to last to last equation. Why is it that we can turn the marginal probability distribution of $$\epsilon$$ into a conditional distribution of $$y$$ given $$x$$?

Why can't the left hand side of the question be a joint probability, e.g., $$p(y,x; \theta)$$ instead or even $$p(x | y; \theta)$$?

$$\epsilon \sim N(0,\sigma^2) \implies y \sim N(\beta^Tx,\sigma^2)$$ since we generally assume $$x$$ is fixed in linear regression: $$E[y|x] = \beta^Tx$$.
This means that $$y$$ is simply a translated version of $$x$$ which makes it trivial to get the conditional distribution of $$y$$. Note that $$x$$ is generally considered fixed so a joint distribution doesn't make sense.
• So the conditioning on $x$ is a derivative of the linear regression model and not that of any underlying MLE characteristic? Jul 19, 2020 at 4:51
• @David not sure I understood your comment -- you're not doing anything with MLE here. You are proposing a probability model for $y$ given $\beta,x$. No where do you specify the distribution for $x$ and in general it is considered a fixed input. This makes it very easy to derive the conditional distribution of $y$. Imagine if x were also normally distributed with zero mean and some variance -- you'd not get the same symmetry since you'd have to account for the variability of $x$. Jul 19, 2020 at 4:57