Why is isomorphism defined as it is in category theory? In terms of an intuitive understanding of categories, I've been told to think of a category as "objects with some structure" and a  morphism as "a transformation preserving the structure". (This understanding obviously falls apart when considering abstract categories, but seems to me to be unrelated to my problem, which is with its interpretation on concrete categories)
An isomorphism is then meant to indicate that the two objects in question have exactly identical structure. It seems to me that we have no guarantee that this notion is captured in general by our general definition of isomorphism.
If you asked someone who only knew of algebraic structures what an isomorphism on concrete categories ought to be, the answer would obviously be "a bijective morphism", but this fails for topological spaces. 
By the same token, while it's certainly true that "having a morphism with inverse" ensures that two objects have identical structure for all the concrete categories I can think of, how do we know there isn't some kind of structure that violates our current notion of isomorphism in the same way that topological spaces violate the "naive" algebraic notion?
 A: A standard way of trying to explain how an isomorphism between two objects of any category means precisely that they look exactly the same as far as that category is concerned is via the Yoneda embedding. 
In our situation we should observe that two objects $x,y$ in a category $C$ are isomorphic if and only if they represent the same functor, that is, if and only if there is a natural isomorphism between the functors $Hom_C(-,x)$ and $Hom_C(-,y)$. In other words, $x$ and $y$ are isomorphic if and only if they admit isomorphic sets of maps from all other objects of $C$. Since a natural isomorphism is merely a natural transformation whose components are bijections of sets, this actually reduces the general notion of isomorphism to the algebraic concept of "bijective morphism", showing that in this sense every category is a category of algebraic structures. In particular, this is not circular-we don't already need to know what an isomorphism is in general to know what a natural isomorphism is!
So, if we define "$x$ and $y$ have precisely the same structure" as "$x$ and $y$ look the same as codomains to all other objects of $C$", formally, $x$ and $y$ represent naturally isomorphic functors, then the conservativity of the Yoneda embedding proves that our notion of isomorphism is the correct one. 
One possible complaint is that you might wonder why we shouldn't say that $x$ and $y$ are "the same" if they admit the same maps to every other object of $C$, rather than from. Luckily, these are equivalent, since objects are isomorphic in $C$ if and only if they are isomorphic in $C^\mathrm{op}$. The main remaining complaint is that we might somehow want more of $x$ and $y$ than that the other objects of $C$ see them as being the same. But in that case, it seems clear to me that we're no longer talking about any notion of sameness in $C$.
A: This answer addresses (or I hope it does) your concern in the second paragraph, at least for groups. Maybe within category framework this approach becomes "trivial", but I can't say more, as category theory is far beyond my knowledge.
So, a group $G$ shows its structure as soon as we allow the internal operation to fully deploy its effects. Accordingly, we can reasonably state the following:

Definition 1. The structure of a group $G$ is the set $\theta_G:=\{\theta_a, a\in G\}\subseteq \operatorname{Sym}(G)$, where $\theta_a$ is the bijection on $G$ defined by: $g\mapsto \theta_a(g):=ag$.

Here a problem arises, if we want to determine whether two groups, $G$ and $\tilde G$, "have the same structure", since in general $\operatorname{Sym}(G)\cap\operatorname{Sym}(\tilde G)=\emptyset$, and then any attempt to "compare by overlapping" the structures $\theta_G$ and $\tilde\theta_\tilde G$ is doomed to fail. We can overcome this issue "by transporting" the structure of $G$ in $\operatorname{Sym}(\tilde G)$, and see whether we can made the "transported $\theta_G$" to overlap with $\tilde\theta_{\tilde G}$. If we succeed, then we can rightly say that $G$ and $\tilde G$ are isomorphic, since we have been able to bring the structure of one onto precisely that of the other. So, with reference to the following diagram:
$$
\newcommand{\ra}[1]{\kern-1.5ex\xrightarrow{\ \ #1\ \ }\phantom{}\kern-1.5ex}
\newcommand{\ras}[1]{\kern-1.5ex\xrightarrow{\ \ \smash{#1}\ \ }\phantom{}\kern-1.5ex}
\newcommand{\las}[1]{\kern-1.5ex\xleftarrow{\ \ \ \smash{#1}\ \ }\phantom{}\kern-1.5ex}
\newcommand{\da}[1]{\bigg\downarrow\raise.5ex\rlap{\scriptstyle#1}}
\begin{array}{c}
 &  & \\
G & \ras{\space\space\space f \space\space\space}   &   \tilde G \\
\da{\theta} &  &  \da{\tilde\theta}  \\
\operatorname{Sym}(G) & \ras{\varphi^{(f)}} & \operatorname{Sym}(\tilde G) \\
\end{array}
$$
we set forth this other:

Definition 2. Two groups, $G$ and $\tilde G$, are isomorphic if there is a bijection $f\colon G\to \tilde G$ such that the above diagram commutes, namely:
$$\tilde\theta=\varphi^{(f)}\theta f^{-1}\tag 1$$
where $\varphi^{(f)}\colon \operatorname{Sym}(G)\to \operatorname{Sym}(\tilde G)$ is the "structure transporting" bijection defined by $\sigma\mapsto f\sigma f^{-1}$.

Now, as a caracterization of such an "enabling" bijection $f$, the following holds:

Claim. Two groups $G$ and $\tilde G$ are isomorphic (as per Definition 2) if and only if there is a bijection $\psi\colon \tilde G \to G$ such that:
$$\psi(\tilde a\tilde g)=\psi(\tilde a)\psi(\tilde g), \space\space\forall \tilde a,\tilde g\in \tilde G \tag 2$$

Proof.
\begin{alignat}{1}
&\tilde\theta=\varphi^{(f)}\theta f^{-1} &\iff \\
&\tilde\theta_\tilde a(\tilde g)=(\varphi^{(f)}\theta f^{-1})(\tilde a)(\tilde g), \space\space\forall \tilde a,\tilde g\in \tilde G &\iff \\
&\tilde a\tilde g=(\varphi^{(f)}\theta f^{-1})(\tilde a)(\tilde g), \space\space\forall \tilde a,\tilde g\in \tilde G &\iff \\
&\tilde a\tilde g=(\varphi^{(f)}(\theta_{f^{-1}(\tilde a)}))(\tilde g), \space\space\forall \tilde a,\tilde g\in \tilde G &\iff \\
&\tilde a\tilde g=(f\theta_{f^{-1}(\tilde a)}f^{-1})(\tilde g), \space\space\forall \tilde a,\tilde g\in \tilde G &\iff \\
&\tilde a\tilde g=f(\theta_{f^{-1}(\tilde a)}(f^{-1}(\tilde g)), \space\space\forall \tilde a,\tilde g\in \tilde G &\iff \\
&\tilde a\tilde g=f(f^{-1}(\tilde a)f^{-1}(\tilde g)), \space\space\forall \tilde a,\tilde g\in \tilde G &\iff \\
&f^{-1}(\tilde a\tilde g)=f^{-1}(\tilde a)f^{-1}(\tilde g), \space\space\forall \tilde a,\tilde g\in \tilde G\\
\tag 3
\end{alignat}
So, $(1)\Rightarrow (2)$, by setting $\psi:=f^{-1}$, and $(2)\Rightarrow (1)$, by setting $f:=\psi^{-1}$. $\space\space\Box$
Therefore, a bijection between two groups with the property $(2)$ is rightly called isomorphism.
