$\color{green}{\textbf{Version of 15.05.21.}}$
$\color{brown}{\textbf{Primary transformations.}}$
Substitution in the form of
$$x=\arctan y,\quad \cos^2 x=\dfrac1{1+y^2},\quad \cot x = \dfrac1y,\quad \text dx = \dfrac{\text dy}{1+y^2}\tag1$$
presents the given integral in the form of
$$I=\int\limits_0^{\large\frac\pi2} x \cot x\ln^2(\cos x)\,\text dx
=\dfrac14\int\limits_0^\infty \arctan y \ln^2\left(1+y^2\right)\,\dfrac{\text dy}{y(1+y^2)}\tag2.$$
This allows partitionally use my answer to the similar question.
Is known the integral
$$\int\limits_0^\infty\dfrac{\ln x\,\text dx}{(x+a)(x+b)} =\dfrac{\ln^2a-\ln^2b}{2a-2b},\quad(\Re a>0,\;\Re b>0).\tag3$$
Then
$$\dfrac1y\arctan y\ln(1+y^2) = \dfrac i{2y}(\ln(1-iy)-\ln(1+iy))(\ln(1-iy)+\ln(1+iy))$$
$$ = 2\,\dfrac{\ln^2(1-iy)-\ln^2(1+iy)}{2(1-iy)-2(1+iy)}
= 2\int\limits_0^\infty\dfrac{\ln z\,\text dz}{(z+1-iy)(z+1+iy)},$$
$$\dfrac1y\arctan y\ln(1+y^2)= 2\int\limits_0^\infty\dfrac{\ln z\,\text dz}{(z+1)^2+y^2}.\tag4$$
Besides, for $\;p>0\;$
$$\int\limits_0^\infty \dfrac{\ln(1+y^2)\,\text dy}{p^2+y^2} =\pi\,\dfrac{\ln(p+1)}{p}.\tag5$$
Taking in account $(2)-(5),$ one can get
$$I = \dfrac12\int\limits_0^\infty\int\limits_0^\infty \dfrac{\ln z}{(z+1)^2+y^2} \dfrac{\ln(1+y^2)}{1+y^2}\,\text dz\,\text dy$$
$$= \dfrac12\int\limits_0^\infty\int\limits_0^\infty \left(\dfrac1{1+y^2}-\dfrac1{(z+1)^2+y^2} \right)\ln(1+y^2)\,\dfrac{\ln z}{z^2+2z}\,\text dy\,\text dz,$$
$$I= \dfrac\pi2\int\limits_0^\infty \left(\ln 2-\dfrac{\ln(z+2)}{z+1}\right)\,\dfrac{\ln z}{z^2+2z}\,\text dz.\tag6$$
Considered transformations allowed to simplify the given integral. However, the obtained integral looks non-trivial too.
$\color{brown}{\textbf{Splitting.}}$
Integral $(6)$ can be splitted to the six integrals. Really, taking in account $(3),$
\begin{align}
&\dfrac4\pi\,I= 2\int\limits_0^\infty \dfrac{(z\ln 2-\ln(1+\frac12z))\ln z}{z(1+z)(2+z)}\,\text dz\\[4pt]
&= 2\ln2\int\limits_0^\infty \dfrac{\ln z}{(1+z)(2+z)}\,\text dz
-\int\limits_0^\infty \dfrac{\ln(2z) \ln(1+z)}{z(1+z)(1+2z)}\,\text dz\\[4pt]
&= \ln^3 2
-\int\limits_0^\infty \dfrac{\ln(2) \ln(1+z)}{z(1+z)(1+2z)}\,\text dz
-\int\limits_0^\infty \dfrac{\ln(z) \ln(1+z)}{z(1+z)}\,\text dz
+2\int\limits_0^\infty \dfrac{\ln(z) \ln(1+z)}{(1+z)(1+2z)}\,\text dz\\[4pt]
&= \ln^3 2
-\int\limits_0^\infty \dfrac{\ln(2) \ln(1+z)}{z(1+z)(1+2z)}\,\text dz
-\int\limits_0^\infty \dfrac{\ln(z) \ln(1+z)}{z(1+z)}\,\text dz\\[4pt]
&+\int\limits_0^\infty \dfrac{\ln^2(1+z)}{(1+z)(1+2z)}\,\text dz
+\int\limits_0^\infty \dfrac{\ln^2(z)}{(1+z)(1+2z)}\,\text dz
-\int\limits_0^\infty \dfrac{(\ln(z+1)-\ln z)^2}{(1+z)(1+2z)}\,\text dz.
\end{align}
$\color{brown}{\textbf{Closed forms of the integrals.}}$
Four first interals of the five can be calculated by Wolfram Alpha immediately,
\begin{align}
&I_1 = -\int\limits_0^\infty \dfrac{\ln(2) \ln(1+z)}{z(1+z)(1+2z)}\,\text dz
= -\ln^2 2,\\[4pt]
&I_2 = -\int\limits_0^\infty \dfrac{\ln(z) \ln(1+z)}{z(1+z)}\,\text dz
= -\zeta(3),\\[4pt]
&I_3 = \int\limits_0^\infty \dfrac{\ln^2(1+z)}{(1+z)(1+2z)}\,\text dz
= \dfrac1{12}(21\zeta(3)+4\ln^3 2 -\pi^2 \ln4),\\[4pt]
&I_4 = \int\limits_0^\infty \dfrac{\ln^2(z)}{(1+z)(1+2z)}\,\text dz
= \dfrac13\,\ln2(\pi^2+\ln^2 2).
\end{align}

Besides,
$$I_5 = -\int\limits_0^\infty \dfrac{(\ln(z+1)-\ln z)^2}{(1+z)(1+2z)}\,\text dz
= -\int\limits_0^\infty \dfrac{\ln^2\left(\dfrac1z+1\right)}{z^2\left(1+\dfrac1z\right)\left(2+\dfrac1z\right)}\,\text dz,$$
$$I_5 = -\int\limits_0^\infty \dfrac{\ln^2(z+1)}{(1+z)(2+z)}\,\text dz
= -\frac32\,\zeta(3).$$

Therefore,
$$\dfrac4\pi I = \ln^3 2 - \ln^3 2 - \zeta(3) + \dfrac1{12}(21\zeta(3)+4\ln^3 2 -\pi^2 \ln4) + \dfrac13\,\ln2(\pi^2+\ln^2 2) -\dfrac32\,\zeta(3),$$
$$\color{green}{\mathbf{I = \dfrac\pi{6}\,\ln^3 2 - \dfrac{3\pi}{16}\,\zeta(3) + \dfrac{\pi^3}{24}\ln2.}}\tag7$$