Solving $Ax=b$ when $x$ and $b$ are given. I am trying to study least square and linear regression and I understand the solution for $Ax = b$ when x is the unknown and the LS solution is given by $(A^TA)^{-1}A^TA$. Now, I was wondering if something similar exists when A is unknown. I was wondering if I do the corresponding:
$$
Ax=b
$$
$$
x^TA^T = b
$$
Therefore, $A^T = (xx^T)^{-1}xb$ or $A = b^Tx^T ((xx^T)^{-1})^T$
 A: Since $xx^T$ is a rank one matrix, it is not invertible and $(xx^T)^{-1}$ does not exist. Your system is therefore underdetermined.
The best way to obtain a (pseudo-)unique solution is either to replace $(xx^T)^{-1}$ by the Moore-Penrose pseudoinverse of $x^Tx$ or to use a regularization procedure (like the Tikhonov one, where you would replace $(xx^T)^{-1}$ by $(xx^T+\lambda I)^{-1}$ with $\lambda > 0$).
A: You are basically asking for a linear transformation that maps one vector $x$ to another vector $b$. With no other constraints, there are infinitely many answers except in the special case of one dimension.
A linear mapping $A:\mathbb{R}^{n}  \rightarrow \mathbb{R}^{n}$ requires $n$ linearly independent equations in order to be fully specified. This means you would need $n$ equations $Ax_1=b_1,Ax_2=b_2,...,Ax_n=b_n$  (or more simply $AX=B$ where $X$ and $B$ are $n\times n$ matrices) in order to get a unique answer.
A: If $x$ is zero, an $A$ exists (if and) only if $b=0$ and in this case, $A$ can be chosen arbitrarily.
If $x$ is nonzero, then any $A$ such that $Ax=b$ is given by
$$\tag{1}
A=bx^\dagger+Z(I-xx^\dagger),
$$
where $Z$ is any matrix of the same size as $A$ and $x^\dagger=x^*/\|x\|_2^2$. You may verify that
$$
Ax=b(x^\dagger x)+Z(I-xx^\dagger)x=b+Z(x-x(x^\dagger x))=b+Z(x-x)=b
$$
using $x^\dagger x=x^*x/\|x\|_2^2=1$.
The form (1) follows simply from recasting $Ax=b$ as a linear system for the matrix $A$ and from the fact that any solution to $Bz=c$ is given by $z=z_0+w$, where $z_0$ is any solution to $Bz=c$ and $w$ is in the kernel of $B$. 
