1
$\begingroup$

I'm trying to apply nonlinear refinement to a homography matrix using Levenberg-Marquardt optimization. I'm using what's called the "symmetric transfer error", which is defined here for a set of at least four point correspondences $x_i \leftrightarrow x_i'$.

$\sum_{i} ||x_i - H^{-1}x_i'||^2 + ||x_i' - Hx_i||^2$

To my understanding, I'm trying to find 9 parameters of the 3x3 homography matrix $H$ which minimizes the above error. In order to do this programmatically, I was planning on using scipy.optimize.least_squares but I'm having trouble formulating this as a least squares optimization problem. Mainly because it seems as though I have two distinct sets of residuals, $\{x_i - H^{-1}x_i'\} \forall i$ and $\{x_i' - Hx_i\} \forall i$.

In the documentation for the above function, I need to specify a function $f_i(x, a, b, ...)$ which returns a vector of residuals for parameter $x$ and arguments $a, b$, etc. As well as an error function $rho$ which takes in a vector and returns a scalar. I'm guessing my $rho$ is just $||f_i(x)||^2$ but please correct me if I'm wrong.

So, I guess I'm having trouble representing my loss function using a single $f$ and a single $rho$.

$\endgroup$
5
  • $\begingroup$ It would be good to know if $H $ has ant special properties. For example, it seems that the teo components to minimise could be dealt with separately if $H$ was an isometry. $\endgroup$
    – AnyAD
    Jun 4, 2018 at 5:31
  • $\begingroup$ $H$ is a homography, or a projective transform, defined up to an arbitrary scale. In other words it’s a 8DOF matrix which maps points from one plane to another $\endgroup$
    – Carpetfizz
    Jun 4, 2018 at 5:38
  • $\begingroup$ I may be wrong, but it seems that you would have more than 2 terms to minimise (distinct sets of residuals), since you also have at least four points. So 8 parameters in $H$ and also $4$ of $||f(x_i)||$ and $4$ of $||H^{-1}f(x_i)||$ to minimize. $\endgroup$
    – AnyAD
    Jun 4, 2018 at 15:03
  • $\begingroup$ Do you mind expanding on why? $\endgroup$
    – Carpetfizz
    Jun 4, 2018 at 15:05
  • $\begingroup$ Just from reading online about the L-M method, it seems that the problem to solve needs to be in the form of minimising a $n$ vector of errors, so norm of the type $||f(x) ||^2$, where $f$ is a function from $R^m$ into $R^n$ say. Your $f(x_i)$ seems to be $x_i^{'}-Hx_i$ and there would be 8 vectors of this type (for $i=1,2,3,4$). $\endgroup$
    – AnyAD
    Jun 4, 2018 at 15:13

1 Answer 1

Reset to default
1
$\begingroup$

The 2 separate residuals for each data point can easily be dealt with. If there are $n$ data points, return a vector of residuals of length $2n$. For each data point, there are 2 corresponding residuals in the vector of residuals. It all works out correctly, because the overall objective is the sum of squares of all the residuals (2 per data point).

As for the loss function rho, the default is 'linear', i.e., loss(x) = x, which produces the standard least squares solution. There is no need to change from the default unless you want to use an optional robust loss function instead of least square, in which case follow the instructions.

$\endgroup$
2
  • $\begingroup$ Thanks. Why is the loss linear and not the squared norm of the residual? Also, what do you think of @AnyAD’s comment of optimizing over f as well? $\endgroup$
    – Carpetfizz
    Jun 4, 2018 at 23:31
  • $\begingroup$ I think it's kind of a stupid syntax/convention that function uses for loss function, but the way they define linear "loss" function corresponds to squared norm of the residual, i.e., least squares, i.e., a linear transform is applied to a squared function, which results in the squared function. I think my approach addresses @AnyAD 's cnocerns (doubling of number of residuals), and seems to be the obvious way to implement what whoever came up with "symmetric transfer error" had in mind. iI you want to optimize some other criterion, then I suppose that would not be "symmetric transfer error"/ $\endgroup$
    – Mark L. Stone
    Jun 4, 2018 at 23:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .