On the definition of adjoint maps on inner product spaces 
Definition: Let $A:V\to V$ be linear, where $V$ is an inner product space over $\mathbb{F}$. Then $A^*:V\to V$ is defined by the equation: $$\langle y,A^*x\rangle = \langle Ay,x\rangle \text{ for all }x,y\in V$$
Here $A^*$ is called the adjoint operator with respect to $A\in\mathcal{L}(V)$.

So, it is clear that $A^*$ is defined on inner product spaces only. However, we see that the conjugate transpose of a matrix is defined regardless of whether $V$ is an inner product space or not (right?), i.e. if $A\in M_n(\mathbb{F})$, we define $A^* = (\bar A)^T$.
I suspect a possible correlation between the conjugate transpose defined for matrices, and the adjoint operator defined on inner product spaces, because of results such as:

Let $\mathcal{B}$ be an orthonormal ordered basis of $V$, then $A:V\to V$ is self adjoint (i.e. $A^* = A$) iff $[A]^*_{\mathcal{B}} = [A]_{\mathcal{B}}$ (matrices of linear maps w.r.t. basis $\mathcal{B}$).

However, it seems weird that $A^*$ (adjoint) in the context of linear maps is defined only on inner product spaces, while the conjugate transpose of a matrix $A\in M_n(\mathbb{F})$ is defined more generally. Am I missing something here? I was hoping for a stronger connection between the two definitions.
Thanks!
P.S.
As mentioned in the comments, it is required to assume $\mathbb{F} = \mathbb{C}$ for $\bar A$ to make sense.
 A: The point is that $\Bbb F^n$ carries a natural inner product, when $\Bbb F=\Bbb R$ or $\Bbb C$, namely the ordinary one:
$$\langle x,y\rangle:=\sum_k\overline{x_k}y_k\,.$$
And one can prove that $A^*={\bar A}^T$ using this inner product, when identifying a matrix $A$ with the linear map $x\mapsto A\cdot x$.

 Let $e_1,\dots,e_n$ be the standard basis, then $\langle e_i,Ae_j\rangle=A_{i,j}$ is the $i,j$th matrix entry, so $$(A^*)_{i,j}\ =\ \langle e_i,A^*e_j\rangle\ =\ \overline{\langle A^*e_j,e_i\rangle}\ =\ \overline{\langle e_j,Ae_i\rangle}\ =\ \overline{A_{j,i}}\,.$$

Note that over $\Bbb R$, conjugation is the identity, we can omit it, so in that case the adjoint of $A$ is simply its transpose $A^T$.
A: If $V$ is an "abstract" finite-dimensional vector space over $\mathbb{C}$, the matrix $A$ corresponding to a linear transformation self-map $L: V \to V$ depends on the ordered basis one uses for $V$. In particular, if $B_1 = (v_1, \ldots v_n)$ is one ordered basis for $V$ and $A$ is the matrix for $L$ in the ordered basis $B_1$, $B_2 = (w_1, \ldots w_n)$ is another ordered basis for $V$ and $B$ is the matrix for $L$ in the ordered basis $B_2$, and $P$ is the transition matrix with $AP = PB$, then $B^* = (P^{-1}AP)^* = P^*A^*(P^{-1})^*$, which is not $P^{-1}A^*P$, the matrix for $B^*$ with respect to the ordered basis $B_2^*$, unless $P$ is unitary. (In fact, the ordered basis $B_2^*$ isn't exactly well-defined at this point; see below.)
This is to say, if one has a linear transformation $L$, one chooses an orthonormal ordered basis $U_1 = (u_1, \ldots, u_n)$ for $V$, sets $R$ to be the matrix for $L$ in the ordered basis $U_1$, chooses $U_2 = (x_1, \ldots, x_n)$ to be another orthonormal ordered basis for $V$, and sets $S$ to be the matrix for $L$ in the orthonormal ordered basis $U_2$, so that $RQ = QS$ for a unitary transition matrix $Q$, then $S^* = Q^{-1}R^*Q$ is the matrix for $L^*$ with respect to the basis $U_2^* = (x_1^*, \ldots, x_n^*)$.
So, one generally needs to have an inner product space for an "abstract" finite-dimensional vector space in order to have a well-defined notion for the matrix of the adjoint of a linear transformation, and, in this setting, the notion is only well-defined in the setting or category of orthonormal ordered bases for the inner product space being used to find the matrix of a linear transformation.
In general, (1) the notion of an eigenvalue and eigenvector is only well-defined for a self-map of a vector space to itself, and (2) the correspondence between the (conjugate) transpose of a matrix and the matrix corresponding the adjoint of the linear transformation defined by the matrix is only well-defined in the category of orthonormal ordered bases ("orthonormal frames") for the inner product spaces.
