# Orientations of pixels of image

I want to get an approximate orientation of each pixel of an image. I'm using C++ in combination of OpenCV (an open MatLab library) for this.

My initial image is this: After some reading I found out that this is possible, following this procedure:

## 1. Applying a Gaussian filter

This is done to reduce the noise in the image. I did this using the following OpenCV function:

GaussianBlur(src, workImg, Size(5, 5), 1.4);


Afterwards it looks like that: ## 2. Find the X- and Y-derivatives of the image

This is done using the Sobel operator like that:

Sobel(workImg, magX, CV_32F, 1, 0, 3);
Sobel(workImg, magY, CV_32F, 0, 1, 3);


The results look like that:

## 3. Get the orientation of the derivatives

This done like that:

$$\theta=arctan2(\partial_y, \partial_x)$$

Or using OpenCV:

phase(magX, magY, direction, true);


This however doesn't quite produce the results I hoped for - for the given example the results can be displayed like that (I used ASCII characters to display the approximate estimated direction of each pixel): This is generated with this code:

for (int y = 0; y < direction.rows; ++y)
{
for (int x = 0; x < direction.cols; ++x)
{
double degrees = direction.at<float>(y,x);
while (degrees < 0)
degrees += 180;
while (degrees > 180)
degrees -= 180;
if (isnan(degrees))
cout << "xx";
if (degrees > 337)
cout << "--";
else if (degrees > 292)
cout << "\\\\";
else if (degrees > 247)
cout << "||";
else if (degrees > 202)
cout << "//";
else if (degrees > 157)
cout << "--";
else if (degrees > 112)
cout << "\\\\";
else if (degrees > 67)
cout << "||";
else if (degrees > 22)
cout << "//";
else
cout << "--";
}
cout << "\n";
}


Obviously the results aren't quite what I hoped for - at least in the right part of the fingerprint image, the directions should be like 20° but are more like -45°.

To show what I mean with the orientation of a pixel, I marked a few directions in my original image (it would be quite a lot of work to draw an arrow into every single pixel): In an optimal case, the orientation could be estimated for every single pixel, however I guess that's not possible, so I only marked the orientations of the distinct areas of the print (the edges). I only marked a few pixels, I guess the rest is self explaining.

Am I applying this approach wrongly or is it generally unsuitable for what I'm trying to achieve? And if it is, how could I get the orientation of each pixel of an image?

• What do you mean by orientation? For instance, could you draw arrows or something on the picture to indicate what data you want to achieve? – Milo Brandt Mar 11 '18 at 16:21
• @MiloBrandt I updated my answer. – MetaColon Mar 11 '18 at 17:36

## 1 Answer

Your approach of taking the orientation of the gradient vector fails at the top of the ridges and bottoms of the valleys, where the gradient becomes zero and its direction given solely by noise.

The solution to this is the structure tensor:

$$S = \overline{(\nabla I)(\nabla I)^T} = \begin{bmatrix} \overline{I_x^2} & \overline{I_x I_y} \\ \overline{I_x I_y} & \overline{I_y^2} \end{bmatrix}$$

where $I_x$ is the partial derivative of the image $I$ in the $x$ direction, and the overline indicates local averaging.

This is basically a trick that allows local averaging of the gradient.

If one eigenvalue is larger than the other, there is a linear structure locally in the image. The larger these eigenvalues, the more meaningful this structure is. The eigenvector corresponding to the largest eigenvalue is perpendicular to the line.

To compute the structure tensor, Gaussian smoothing is typically applied, because it is effective as a smoothing, perfectly isotopic, and cheap to compute (separable). The gradient can be computed as you do, but it is better to compute Gaussian gradients (convolution with the derivative of the Gaussian).

For more details about the structure tensor see Wikipedia, and here. This latter link is to the documentation of DIPlib, which has a function to compute the structure tensor and derive useful information from it. It is C++ and comes with a complete MATLAB toolbox for image analysis. If you don't want to compile yourself, you can get an older version of the MATLAB toolbox here.