$\mathbf{Y} = \mathbf{X}^T \beta$: dimension and hyperspace Let's say we have that $\mathbf{X}^T$ is $1 \times p$ and $\beta$ is $p \times K$, so that
$$\mathbf{Y} = \mathbf{X}^T \beta$$
means that $\mathbf{Y}$ is $1 \times K$. Since $\mathbf{X}$ has $p$ rows and only a single column, does this mean that the input-output space $(\mathbf{X}, \mathbf{Y})$ is $p$-dimensional, since the length of the standard basis for this input-output (vector) space would be $\{ (1, 0, \dots, 0)_1, (0, 1, 0, \dots, 0)_2, \dots, (0, 0, \dots, 0, 1)_p \}$ ?
And therefore, can we not conclude that, in the $(p + 1)$-dimensional input-output space, $(\mathbf{X}, \mathbf{Y})$ is a hyperplane, since a hyperplane is a subspace whose dimension is one less than that of its ambient space?
 A: Consider the function $f: \mathbb R^p \to \mathbb R^K$ given by
$$f(\mathbf X) = \mathbf X^T \beta.$$
Note that $f$ is a linear function since it is defined by (left-)multiplication by the $p\times K$ matrix $\beta$.
Now define the set $G_f \subset \mathbb R^{p + K}$ by
\begin{align*}
G_f
&=\{(\mathbf{X}, \mathbf Y) \subset \mathbb R^{p + K}: \mathbf{X} \in \mathbb R^p, \mathbf Y = f(\mathbf X)\} \\
&=\{(\mathbf{X}, f(\mathbf X)) \subset \mathbb R^{p + K}: \mathbf{X} \in \mathbb R^p\}
\end{align*}
The set $G_f$ is usually called the graph of $f$ and I assume this is what you mean by input-output-space.
I claim that $G_f$ is a $p$-dimensional linear subspace of $\mathbb R^{p+K}.$ We will first verify that $G_f$ is a linear subspace and then find a basis of $G_f$ with $p$ elements.
To show that $G_f$ is a linear subspace it suffices to verify that it is nonempty and closed under linear combinations. $G_f$ is clearly nonempty because $0 \in \mathbb{R}^{p+K}$ and $0 \in G_f$: This is  since $f(0) = 0$ by linearity of $f$ and so $(0,0) = (0, f(0)) \in G_f$. Now let $a, b \in \mathbb R$ be arbitrary numbers. Then for any $\mathbf X_1, \mathbf X_2 \in \mathbb R^p$, by linearity of $f$:
$$a(\mathbf X_1, f(\mathbf X_1)) + b (\mathbf X_2, f(\mathbf X_2)) = (a\mathbf X_1+b\mathbf X_2,f(a\mathbf X_1 + b\mathbf X_2)) \in G_f.$$
Hence $G_f$ is a linear subspace of $\mathbb R^{p+K}$.
Now let $e_i \in \mathbb R^p$ be the $i$th unit vector of $\mathbb R^p$ and define for $i=1,\dots,p$ the following vectors:
$$v_i = (e_i, f(e_i)) \in \mathbb R^{p+K}.$$
Since the $e_i$ are linearly independent, so are the $v_i$. Moreover, the $v_i$ generate the subspace $G_f$: Indeed, let $(\mathbf X, f(\mathbf X)) \in G_f$. Let $a_i \in \mathbb R$ be coefficients such that $\mathbf X = \sum_{i=1}^p a_i e_i.$ Then
\begin{align*}
(\mathbf X, f(\mathbf X)) 
&= (\sum_{i=1}^p a_i e_i, f(\sum_{i=1}^p a_i e_i)) \\
&= (\sum_{i=1}^p a_i e_i, \sum_{i=1}^p a_i f(e_i)) \\
&= \sum_{i=1}^p a_i (e_i, f(e_i)) \\
&= \sum_{i=1}^p a_i v_i.
\end{align*}
Hence the $v_i$ are a set of linearly independent vectors that generate the subspace $G_f \subset \mathbb R^{p+K}$. That is to say, the set $\{v_1,\dots,v_p\}$ forms a basis of $G_f$.
Hence, $G_f$ is a $p$-dimensional linear subspace of $\mathbb R^{p+K}$.
Now, for $K=1$ it follows that $G_f$ is a hyperplane in $\mathbb R^{p+1}$. Indeed, in that case
\begin{align*}
G_f 
&= \{(\mathbf X,y) \in \mathbb R^{p+1}: y = \mathbf X^T \beta\} \\
&= \{(\mathbf X,y) \in \mathbb R^{p+1}: (-\beta, 1)(\mathbf X, y) = 0\}, \\
\end{align*}
so $G_f$ is a hyperplane through the origin with normal vector $(-\beta, 1)$.
If $K>1$ then $G_f$ is not a hyperplane but simply a linear subspace of dimension $p$ (note that for $K>1$ the dimension $p+K$ of the ambient space $\mathbb R^{p+K}$ is more than one greater than $p$, the dimension of $G_f$).
