How can I calculate the correct rotation output when input needs modification?

I am making a GPS for vehicles in a game and I need them to turn accordingly to the rotation between the ending point and starting point. I have calculated the rotation, and before I can apply the rotation I found out I need to edit the values a bit. Not knowing how to do it but observing the roll/pitch has to be changed somewhere I have come up with the following formula, in which I input the Pitch and Yaw and I get back the almost correct pitch and roll output, but not perfect.

This is the formula:

//values are normalised in range [0,360]
Pair CorrectValues(float Pitch, float Yaw)
{
float
Roll_out = 0.0,
Pitch_out = 0.0;

//if Yaw is bigger than 315 OR Yaw is smaller than 45
if(Yaw > 315.0 || Yaw < 45.0)
Roll_out = -Pitch;

//if YAW is between 45 and 135
if(Yaw > 45.0 && Yaw < 135.0)
Pitch_out = -Pitch;

//if YAW is between 135 and 225
if(Yaw > 135.0 && Yaw < 225.0)
Roll_out = Pitch;

//if YAW is between 225 and 315
if(Yaw > 225.0 && Yaw < 315.0)
Pitch_out = Pitch;

return Pair(Roll_out,Pitch_out);
}


I think I need to do somthing with cos/sin, I have tried many combinations but all of them just make the rotation weird (which mean I just randomly tried applying sin/cos in many combinations to the out values in the form of X_out = sin or cos (Pitch) * Pitch).

Can anyone give me a push in the good direction to solve this? The main issue is that it "yaggy" / "laggy"

Edit: As for request for more detail I'm putting it here:

I Need to set the vehicle's quaternion orientation to the one which you can calculate between two positions, which means there is one rotation we will not calculate = roll.

So I did calculate the Pitch and Yaw between two 3D points (XYZ) with this formula:

const float MATH_PI = 3.14159265359;
void GetPitchYawBetweenCoords(point3D source, point3D target, point2D &output)
{
point delta3D = target-source;//X2-X1, Y2-Y1, Z2-Z1
output.x = atan2( delta3D.y, delta3D.x ) * RAD_TO_DEG;//yaw
output.y = atan2( delta3D.z, sqrt( delta3D.x * delta3D.x + delta3D.y * delta3D.y ) ) * RAD_TO_DEG;//pitch
}


Then I did use a programming library called OpenGL Mathematics to convert these rotations to a Quaternion, and apply it to the vehicle, but this wasn't working all the time so I did the above modifications which approaches the "correct rotations".

But I'm wondering if there is any correct or better way to do this?

Ad an update to my comment:

glm::eulerAngles assumes the input of Pitch,Roll,Yaw in radians which I do, It's weird how this works if the Yaw angle is 0 and the Pitch is set correctly, and if the Yaw angle is 180 degrees then the pitch is reversed.. and at other angles.. well it looks like it's being interpolated? I will upload a screenshot.

a.k.a. : When I ONLY set the Pitch and Yaw without ANY calculations/modifications and calculate it into a quaternion, the vehicle rotation is like being "interpolated" / slowly inversed, look here:

The center of All cars equals (0.0,0.0,0.0) and ALL the cars have the pitch set to a value which equals (XY In front of Car at distance 10.0, CurrentCarZ-10.0). (they all should point DOWN)

Yet you can see this is somehow producing interesting results.

Why is this "interpolating" like this?

Only the car (the closest one to me - ID 495 - pink) which is totally facing UP by some amount of degrees has actually the inverted rotation, all other not what I expected (you can see they are a bit rolled, and the one at heading 0 is totally perfect - ID 499 - green).

(Note: North is 180 degrees)

The "formula" you can see in the picture equals then to this (this is a function which just uses the GLM library):

EulerToQuat(Pitch, 0.0, Yaw, Qw, Qx, Qy, Qz);
//{Qw,Qx,Qy,Qz} = glm::quat(glm::vec3(Pitch,0.0,Yaw)*vector_to_radians)//constructs a quaternion form XYZ euler angles

• This sort of correction shouldn't be necessary. Could you please describe in a little more detail how the pitch and yaw angles are calculated, and how they're applied? I wonder if there's an error in that code. Maybe the rotations are being applied in the wrong order, or something like that. Commented Jul 31, 2013 at 17:41
• I'm having a hard time understanding what your question is. Are you saying that you want a more accurate function than the one you gave? Commented Jul 31, 2013 at 17:41
• I just randomly tried applying sin/cos in many combinations... Usually people put the "flail about wildly with nonsense" problem solving strategy after the "try to reason carefully" strategy :) THis sort of problem should develop a fairly concrete picture which you can start reasoning from. Commented Jul 31, 2013 at 18:23
• @Chris Updated the question Commented Jul 31, 2013 at 18:43
• @Gizmo Okay, you're effectively applying pitch first, then yaw. That makes sense. Which OpenGL function are you calling? Does it assume the same order? Commented Jul 31, 2013 at 18:51

//pitch,roll,yaw = all are degrees