What if $N$ is not normal in semi-direct product $N\rtimes H$? In classification of groups of order $n$, we always use semi-direct product. And require $N\unlhd G$, $H<G$ and so on. But what if $N$ is not normal? It seems we can still use $(n_1,h_1)(n_2,h_2)=(n_1\Phi_{h_1}(n_2),h_1h_2)$, where $\Phi_{h_1}\in \mathrm{Hom}(H,\mathrm{Aut}(N))$ to define the operation. So, if $N$ is not normal, then it may not be a problem.
 A: I will show that if you have a semidirect product, then $N$ will be normal. Let $G=N \rtimes_\theta H$. Well consider the projection homomorphism $\pi_H:N \rtimes_\theta H \to H$, which is a homomorphism since $$\pi_H \left((n_1,h_1)(n_2,h_2)\right) = \pi_H(n_1 \theta_{h_1}(n_2), h_1h_2) = h_1h_2$$ and $$\pi_H(n_1,h_1) \pi_H(n_2,h_2) =h_1h_2.$$ It is pretty clear that (the identified) $N$ is the kernel of the projection, so $N$ is normal.

The short of it is that the definition of semidirect product implies that $N$ is normal in the semidirect product, so you can't have the situation you are asking about.
A: There is a related construction that generalizes the semidirect product that you might be hitting upon.  The basic idea is that, given finite groups $K,Q$ we wish to find all groups $G$, and injective morphisms $\phi\colon K\to G$, and surjective morphisms $\psi\colon G\to Q$ such that $\operatorname{Im}(\phi)=\ker(\psi)$.  In this case we say that $G$ is an extension of $K$ by $Q$.  We can express this in terms of a short exact sequence, but I'll skip that here (it's just a convenient graphical notation).  
Note that this means that $G$ contains a normal subgroup isomorphic to $K$: $\ker(\psi)$, namely.  
However, the group $G$ need not be a semidirect product. $G=K\rtimes Q$ (after a few notation abuses: I've not supposed that $Q$ is a subgroup of $G$) if and only if there is a homomorphism $\beta\colon Q\to G$ such that $\psi\beta=\operatorname{id}_Q$.  $G$ will be a direct product if there is also a morphism $\alpha\colon G\to K$ with $\alpha\phi=\operatorname{id}_K$.
Now for any such short exact sequence, there is nevertheless a morphism $\theta\colon Q\to\operatorname{Aut}(K)$ when $K$ is abelian--the basic idea being to construct it from a transversal of the cosets of $K$--such that $G$ will be determined by the data $(K,Q,\theta)$.  When $K$ is non-abelian the codomain of $\theta$ is $\operatorname{Aut}(K)/\operatorname{Inn}(K)$; the procedures are generally more complex when $K$ is non-abelian.  Much as with the semidirect product, the morphism $\theta$ effectively describes "how" $K$ is normal in $G$.
It is important to note that not every extension yields a semidirect product.  Indeed, $\mathbb Z_4$ is an extension of $\mathbb Z_2$ by $\mathbb Z_2$, but any semidirect product of those two groups has no elements of order 4 (indeed, such a semidirect product must be a direct product).  The basic reason is that there need not be a choice of coset representatives of $K$ which forms a subgroup of $G$ (isomorphic to $Q$).  The cosets themselves form the quotient group, but it is not always possible to pick coset representatives that give a group; you can do so if and only if there is a morphism $\beta$ as above.
