Camera position and 3D perspective projection I have 3D points $P_1, P_2, P_3,..., P_N$ and the result of a 3D perspective projection: $p_1, p_2, p_3,..., p_N$. Is there a way to obtain camera position having only that data? Is it possible having camera fov?
 A: Suppose your 3D points and the projections are given in homogeneous coordinates $X_1, X_2, X_3,..., X_N \in \mathbb{R}^4$ and $x_1, x_2, x_3,..., x_N \in \mathbb{R}^3$. Then they have the simple relationship $$x_i=PX_i$$ with the camera matrix $$P= \begin{pmatrix} 
    p_{11} & p_{12} & p_{13} & p_{14}\\ 
    p_{21} & p_{22} & p_{23} & p_{24}\\
    p_{31} & p_{32} & p_{33} & p_{34}\\
  \end{pmatrix} \in \mathbb{R}^{3\times4}\textrm{.}$$
This relationship is linear in the unknowns $p_{11}, p_{12}, \cdots p_{34}$. To put it in the usual form $b=Ax$, where $x$ are the unknowns, use the vector $$p:=\begin{pmatrix} 
    p_{11}\\ 
    p_{12}\\
    p_{13}\\
    \vdots\\
    p_{34}
  \end{pmatrix}.$$ Then
$$x_i=\begin{pmatrix} 
    X_{i1} & X_{i2} & X_{i3} & X_{i4} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 
    0 & 0 & 0 & 0 & X_{i1} & X_{i2} & X_{i3} & X_{i4} & 0 & 0 & 0 & 0\\
    0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & X_{i1} & X_{i2} & X_{i3} & X_{i4}\\
  \end{pmatrix}p$$ where $$X_i:=\begin{pmatrix} 
    X_{i1}\\ 
    X_{i2}\\
    X_{i3}\\
    X_{i4}
  \end{pmatrix}.$$
Each of these correspondences gives us $3$ linear equations that involve the components of $p$ so at least $N\geq 4$ correspondences are required. If $A$ is the matrix of stacked rows involving $X_i$ as shown and $$b:=\begin{pmatrix} 
    x_{1}\\ 
    x_{2}\\
    x_{3}\\
    \vdots\\
    x_{N}
  \end{pmatrix}$$ then $p=A^{-1}b$ if $N=4$ (which makes $A\in \mathbb{R}^{12\times12}$) and otherwise $p=(A^tA)^{-1}A^tb$ gives you the linear least squares estimate.
We have $P=K[R,t]$ where $K$ is the calibration matrix, $R$ the rotation and $t$ the translation. Your camera fov is determined by the components of $K$ - for example the focal length $f$.
