# Can regressors be considered as random variables?

In the linear regression model $$y = \beta_1 X_1 + \cdots + \beta_p X_p + \varepsilon \, ,$$ can the regressors $\{X_i\}_{i \in \{1, \ldots, p\}}$ be considered as random variables?

I know that what we really have is $n$ observations of the relationship between the dependent variable and the regressors, i.e., $$y_i = \beta_{i1} x_{i1} + \cdots +\beta_{ip} x_{ip} + \varepsilon_i \, ,$$ but does it make sense to see things like $\Pr[X_1 = 0]$, where $X_1$ is the same $X_1$ as in the first equation? Would that be equal to $\frac{1}{n} \sum_{i = 1}^n [x_{i1} = 0]$ or does it refer to the probability according to a theoretical distribution?

Yes, it is meaningful to imagine the regressors are sampled from a theoretical distribution. The quantity $P(X_1=0)$ would then refer to the theoretical distribution for $X_1$, not to the observed values $x_{1,1},\ldots,x_{1,p}$ of $X_1$. Note that in elementary treatments of regression we assume the regressors are non-random, i.e., they are known constants (equivalently, we are conditioning on the the values of the regressors).

• So, if the theoretical distribution of $X_1$ were known, it would be preferable to use it in order to calculate $\Pr[X_1 = 0]$ than just simply count how many times $x_{i1} = 0$ (for all $i \in \{1, \ldots, n\}$) and divide it by $n$, right? – Cromack Apr 14 '16 at 18:44
• If the theoretical distribution of $X_1$ were known, then you should use it to calculate $P(X_1=0)$. The empirical frequency is only an estimate of the true probability. Then again, I don't know how complicated the theoretical calculation would be, or whether you're really being asked for the theoretical prob vs the empirical frequency. – grand_chat Apr 14 '16 at 19:22
• I am asked about the probability that two given predictors $X_1$ and $X_2$ are collinear, which may be different if calculated by using the theoretical or the empirical distributions. – Cromack Apr 14 '16 at 19:32
• That probability sounds like a theoretical calculation. If you've got one data set of $n$ observations, then the predictors $X_1$ and $X_2$ are either collinear or not. – grand_chat Apr 14 '16 at 19:39