How to prove these 2 matrix problems? 
*

*I'm reading a book and it gives that $\frac{\partial}{\partial A}Tr(AB)=B^T$, then it shows we can obtain $\frac{\partial}{\partial A}Tr(ABA^T)=A(B+B^T)$. But it seems we should have $\frac{\partial}{\partial A}Tr(ABA^T)=AB^T$ accordingly. Am I wrong or the book?

*The book also gives another expression: $\frac{\partial}{\partial x}ln|A|=Tr(A^{-1})\frac{\partial A}{\partial x}$ , how to prove that?
 A: So we look at the map $\def\Tr{\mathop{\rm Tr}}f \colon A \mapsto \Tr(AB)$. We have
\begin{align*}
  f(A+H) &= \Tr(AB + HB)\\
         &= \Tr(AB) + \Tr(HB)\\
         &= f(A) + \Tr(HB) 
\end{align*}
So $$ f'(A)H = \Tr(HB) $$
(as obviously for linear maps). If we want to calculate the derivatives with respect to the entries of $A$, let $E_{ij}$ denote the $(i,j)$-th unit matrix (the only nonzero entry is $(i,j)$, which is 1). Then 
$$
  \partial_{a_{ij}}f(A) = f'(A)E_{ij} = \Tr(E_{ij}B) =\sum_{k,l=1}^n [E_{ji}]_{k,l}[B]_{l,k} = [B]_{j,i} $$
So if we denote the matrix of these partial derivatives by $\partial_A f$, we have $\partial_A f = B^t$.
Now consider $g(A) = \Tr(ABA^t)$. Note that $g$ is no longer, linear, but quadratic in $A$. Along the same lineas as above,
\begin{align*}
  g(A+H) &= \Tr(ABA^t) + \Tr(HBA^t) + \Tr(ABH^t) + \Tr(HBH^t)\\
         &= g(A) + \Tr(AB^tH^t) + \Tr(ABH^t) + o(\|H\|)\\
\end{align*}
So $$ g'(A)H = \Tr\bigl(A(B^t+B)H\bigr) $$
As above, we have
$$ \partial_{a_{ij}}g(A) = g'(A)E_{ij} = \Tr\bigl(A(B^t+B)E_{ji}\bigr)
      = [A(B^t+B)]_{ij} $$
So $\partial_A g(A) = A(B^t+B)$, so the book is right.
Regarding your last question, what is $x$? What can be said, is:
$$ \det A = \sum_{i=1}^n (-1)^{i+j} a_{ij}\det A_{ij} $$
holds for each $j$, where $A_{ij}$ results from $A$ removing the $i$-row and $j$-th column. Hence with $h(A) := \log\det A$ 
\begin{align*}
  \partial_{a_{ij}}h(A) &= \frac{1}{\det A} \det A_{ij}
\end{align*}
Now recall, that $\det A \cdot (A^{-1})_{ij} = \det A_{ji}$ (by Cramer's formula), hence 
$$   \partial_{a_{ij}}h(A) = \frac{1}{\det A} \det A_{ij} = (A^{-1})_{ji} $$
Hence we have $\partial_A h(A) = (A^{-1})^t$.
A: 1) You can do a quick sanity check using $1\times 1$ matrices, i.e. scalars.  The equation reduces to:  $$\frac {d(a^2 b)}{da} = 2 a b$$ The result in the book is compatible with the scalar version, yours isn't. 
2) Can't explain it any better than this Wikipedia entry.
