Are all conjugacy classes in $\text{GL}_n(\mathbb R)$ path-connected? Suppose $A$ and $B$ are conjugate invertible real $n \times n$-matrices. Does there always exist a path from $A$ to $B$ inside their conjugacy class?

I thought I had an easy proof for odd $n$ which goes as follows, but it was incorrect as pointed out in this answer. To show where my misunderstanding arised, here is the wrong argument.
Suppose there exists a real matrix $P$ such that $B = PAP^{-1}$. By replacing $P$ with $-P$ if necessary, we can assume that $\det P > 0$ (this is what goes wrong in even dimensions, see this question).
Then we have that $P = e^Q$ for some real matrix $Q$ (since the image of the exponential map is the path-component of the identity in $\text{GL}_n(\mathbb R)$). But now the path $$t \mapsto e^{tQ}Ae^{-tQ}$$
is a path connecting $A$ to $PAP^{-1} = B$.
 A: Other people have given counterexamples, so I would like to demonstrate that counterexamples are somewhat rare. 
 Here is an attempt at a conceptual explanation for both why this is not true in general and also when it is true.  First, we note that if we have a path $P(t)$ in $GL_n(\mathbb R)$ with $P(0)=P_0$ and $P(1)=P_1$, then $P(t)AP(t)^{-1}$ gives us a path inside the conjugacy class of $A$.  Since $GL_n(\mathbb R)$ has two path components (given by sign of determinant), this shows that the conjugacy class of $A$ is the union of two path connected subsets (conjugating by things with positive or negative determinant), and it will be path connected if these two subsets intersect.
If $PAP^{-1}=QAQ^{-1}$ where $\det(P)>0$ and $\det(Q)<0$, we have $A=(P^{-1}Q)A(P^{-1}Q)^{-1}$, so $A$ commutes with a matrix with negative determinant. The converse is also true, so we have the following result.
Lemma: The conjugacy class of $A$ is path connected if and only if $A$ commutes with some matrix with negative determinant.
This gives several conditions that would ensure conjugacy classes are path connected.


*

*If $n$ is odd, since then $\det(-I_n)=-1$

*If $\det(A)<0$ 

*If $\mathbb R^n$ as a direct sum of two $A$-invariant subspaces where one summand is odd dimensional, (e.g, if the Jordan normal form of $A$ has a block of odd size with a real eigenvalue).  

*If $\mathbb R^{n}$ is the direct sum of two $A$-invariant subspaces, and the restriction of $A$ to either subspace has negative determinant.


One can check that Jordan blocks only commute with matrices that are upper triangular and have only a single eigenvalue, and if $n$ is even and that eigenvalue is real, such a matrix could not have negative determinant.  So these give counterexamples.
I suspect one could give a nice characterization of all counterexamples in terms of JNF.  However, I have not worked out the details.
A: No.
Here is a counterexample: Let $n=2$, $A=
\begin{pmatrix}
0 & -1\\
1 & 0
\end{pmatrix}$ and $B=-A=\begin{pmatrix}
0 & 1\\
-1 & 0
\end{pmatrix}$. Then, $A$ and $B$ are conjugate, since $B=C^{-1}AC$ for the
invertible matrix $C=\begin{pmatrix}
1 & 0\\
0 & -1
\end{pmatrix}$. Thus, there exists a conjugacy class $\mathcal{C}$ in
$\operatorname{GL}_{2}\left(  \mathbb{R}\right)  $ that contains
both $A$ and $B$. However, there exists no path from $A$ to $B$ in
$\mathcal{C}$. Why not?
Probably there is a nice conceptual reason [EDIT: yes, and @Wojowu explains it in his answer], but you can just as well
brute-force it: I claim that every matrix in $\mathcal{C}$ has a nonzero
$\left(  1,2\right)  $-th entry. To see this, just notice that any arbitrary
element of $\mathcal{C}$ has the form
\begin{align}
\begin{pmatrix}
a & b\\
c & d
\end{pmatrix} ^{-1}A
\begin{pmatrix}
a & b\\
c & d
\end{pmatrix}
=
\begin{pmatrix}
-\dfrac{ab+cd}{ad-bc} & -\dfrac{b^{2}+d^{2}}{ad-bc}\\
\dfrac{a^{2}+c^{2}}{ad-bc} & \dfrac{ab+cd}{ad-bc}
\end{pmatrix}
\end{align}
for some $\begin{pmatrix}
a & b\\
c & d
\end{pmatrix}
\in\operatorname{GL}_{2}\left(  \mathbb{R}\right)  $, and
thus its $\left(  1,2\right)  $-th entry $-\dfrac{b^{2}+d^{2}}{ad-bc}$ is
nonzero (because $b^{2}+d^{2}$ can only be $0$ if both $b$ and $d$ are $0$,
but then $\begin{pmatrix}
a & b\\
c & d
\end{pmatrix}$ cannot belong to $\operatorname{GL}_{2}\left(
\mathbb{R}\right)  $).
Thus, every matrix in $\mathcal{C}$ has a nonzero $\left(  1,2\right)  $-th
entry. But if there was a path from $A$ to $B$ in $\mathcal{C}$, then some
point on this path would be a matrix in $\mathcal{C}$ with zero $\left(
1,2\right)  $-th entry (since the $\left(  1,2\right)  $-th entries of $A$ and
$B$ have opposite signs). Thus, there cannot be a path from $A$ to $B$ in
$\mathcal{C}$.
A: Notice that the assertion "since the image of the exponential map is the path-component of the identity in $GLn(R)$" is false.
Firstly, the path-component of the identity in $GLn(R)$ is the set of  matrices with $>0$ determinant. Thus (inside this component) there is a path linking your $P$ and $I$ (in fact, there is nothing to prove).
Secondly, $diag(-1,-2)$ has $>0$ determinant and is not in the image of the real matrices by the exponential map.
EDIT.
Let $SC(A)$ be the conjugacy class of $A\in M_n(\mathbb{R})$. 
*In dimension $2$, $SC(A)$ is non-connected in 2 cases
a. The eigenvalues of $A$ are non-zero conjugate complex; then $SC(A)$ is homeomorphic to a hyperboloid of two sheets.
b. $A$ is non-zero and non-diagonalizable; then $SC(A)$ is homeomorphic to a conical surface with the apex cut off.
*In dimension $4$, we consider the matrices $A_1=diag(U,U)$ where $U=\begin{pmatrix}0&1\\0&0\end{pmatrix}$ and 
$A_2=diag(V,V)$ where $V=\begin{pmatrix}0&-1\\1&0\end{pmatrix}$. Then, using the Aaron's test, we can prove that $SC(A_1),SC(A_2)$ are non-connected (it's easy for $A_1$ and more difficult for $A_2$).
*Assume that we randomly choose $A\in M_n(\mathbb{R})$ -the $(a_{i,j})$ follow iid normal laws- We deduce that follows 
$\textbf{Proposition}.$ When $n\rightarrow +\infty$, the probability that $SC(A)$ is connected tends to $1$.
$\textbf{Proof}$. A random matrix $A$ has distinct complex eigenvalues with probabiity $1$. Then, up to a real change of basis, $A=diag(a_1I_2+b_1V,\cdots,a_pI_2+b_pV,\lambda_1,\cdots,\lambda_q)$, where $2p+q=n$, the $(b_i)$ are non-zero and the $(\lambda_j)$ are real distinct.
Thus $SC(A)$ is connected iff $q\not=0$ (Aaron's test). When $n$ tends to $+\infty$, the mean of the number of real zeroes of a polynomial of degree $n$ is in $\Omega(\sqrt{n})$; we can deduce that the probability that $A$ has, at least, one real eigenvalue tends to $1$ when $n$ tends to $+\infty$. $\square$
A: This edit comes a bit late, but the way I see it is a bit different than the other answers so I'll write it anyway.
Again, I use the example from the question that you link to: $A$ is the rotation by $\frac \pi 2$ in the euclidean plane, and $B$ is the rotation by $-\frac \pi 2$.
Now any conjugate of $A$ by a matrix with positive determinant will correspond to a linear map $\varphi$ such that for any non-zero vector $v$, the vectors $v$ and $\varphi (v)$ in this order make a positive basis of the plane.
Conversely, a conjugate of $A$ by a matrix with negative determinant (which is the case of $B$) will correspond to a linear map $\varphi$ such that for any non-zero vector $v$, the vectors $v$ and $\varphi (v)$ in this order make a negative basis of the plane.
A path from $A$ to $B$ has to cross the set of matrices that have real eigenvalues, and such a matrix cannot be conjugate to $A$.
A: We can use the counterexample from your other question to answer this one. Let $A=\begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$. Matrices $\begin{pmatrix} a & b \\ c & d \end{pmatrix}$ must have trace $a+d$ equal to $0$ and determinant $ad-bc=-a^2-bc$ equal to $1$. This implies that such a matrix cannot have $b=0$, since $-a^2\leq 0$. Therefore the set of conjugates of $A$ is disjoint union of open sets defined by $b>0$ and $b<0$. Neither of those sets is empty, since one contains $A$ and the other contains $-A$. Thus the conjugacy class of $A$ is disconnected.
