How is the determinant related to the inverse of matrix? Whenever I needed to find the inverse of a matrix, I was told to check if its determinant is not zero. However, once I directly applied the Gauss-Jordan's method for finding the inverse of matrix whose determinant was zero. The inverse matrix that I got looked pretty normal like any other (if there wasn't a mistake).
I want to know how does the determinant of the matrix is related to inverse of matrix or why is that if determinant is zero then inverse doesn't exist? What exactly is inverse?   
 A: The inverse of a matrix exists if and only if the determinant is non-zero. You probably made a mistake somewhere when you applied Gauss-Jordan's method. 
One of the defining property of the determinant function is that if the rows of a nxn matrix are not linearly independent, then its determinant has to equal zero. In fact, the determinant function is constructed based on several desired properties(one of them being if the rows are dependent, its determinant is zero).
To answer your question what exactly is an inverse, Let's say if A is a nxn matrix, then its inverse is a matrix such that AB = BA = I. They don't necessarily exists though.
A: Since matrices are simply representations of linear maps with respect to a basis of the domain and codomain, the question of whether a matrix is invertible is essentially the same as whether a function from a set to another set is invertible or not (the function being the linear map). A function is invertible if and only if it is surjective and injective (i.e. bijective). It just happens so that when you translate this into linear algebra for linear maps, you get the determinant of a matrix is non-zero. Here’s a sketch of the full explanation. 
To make things simpler, we will only consider linear maps between spaces of equal finite dimension (corresponding to considering only square matrices). Then injectivity alone is sufficient for bijectivity of a linear map. 
We aim to show that a linear map is injectve if and only if its matrix has non-zero determinant. Here are the steps of the argument: 
(1) A linear map is injective $\iff$ the images of basis vectors of the domain are linearly independent.
(2) A set of $n$-vectors sitting in $n$-dimensional space are linearly independent $\iff$ the volume of the parallelopiped they span is non-zero. You can reassure yourself by imagining this in 3 dimensions. 
(3) The determinant of a square matrix measures the (hyper) volume of the parallelopiped spanned by the column vectors in the matrix. So non-zero determinant $\iff$ column vectors span a parallelopiped of non-zero volume. 
(3.5) The column vectors of a matrix representing a linear map are the images of the basis vectors of the domain under that map. This is more-or-less the definition of the matrix of a linear map. 
Following (1) $\iff$ (2) applied to basis vectors $\iff$ (3) and (3.5), you get the equivalence between bijectivity of a linear map and its matrix having non-zero determinant. To conclude, since bijectivity is equivalent to invertibility, a linear map is invertible if and only if its matrix has non-zero determinant. 
EDIT: To answer your question about what is the inverse matrix... 
Suppose $M : V \to W$ is an invertible linear map. Given a basis of $V$ and a basis of $W$, there is a corresponding matrix $M$ representing the linear map $M$. Then the inverse matrix $M^{-1}$ is simply the matrix representing the inverse linear map $M^{-1}$. 
A: You can think a matrix multiplication as a transformation. The determinant of a matrix is geometrically the amount of area of the parallelogram made by the basis in the transformed space. Here are some snapshots from the popular youtube channel 3blue1brown. 

After applying matrix to all vectors(we don't need to think about all vectors just observing where the basis are going is enough, basis in the new space will span the space).

Now if you want the geometric interpretation of inverse of a matrix, you can think it like the reverse transformation, means if you first apply a matrix(apply means multiply) and then apply the inverse of that matrix you will get the same space as it was before. 
If determinant of a matrix is zero, it means that area of the parallelogram in the transformed space is zero. Means if you apply the matrix which has determinant zero, it will squeeze a plane to a single line. Now, as a linear transformation is a map, to exist the reverse map we need a bijection. But, a line can't get mapped to a plane. Hence, the reverse transformation means the inverse of the matrix doesn't exists. 
I think this explanation will help to visualize, if not I will try to explain it and add picture.
A: It holds that $\det(AB)=\det(A)\det(B)$, so that $\det(A)\det(A^{-1})=1$. In other words, an invertible matrix has (multiplicatively) invertible determinant. (If you work over a field, this means just that the determinant is non-zero.)
On the other hand, if the determinant is invertible, then so is the matrix itself because of the relation to its adjugate.
A: Question: "I want to know how does the determinant of the matrix is related to inverse of matrix or why is that if determinant is zero then inverse doesn't exist? What exactly is inverse?"
Answer: You may use the Cayley-Hamilton theorem to prove that an $n\times n$ matrix $A:=(a_{i,j})\in M_n(k)$ with $k$ any commutative unital ring, is invertible iff $det(A)\in k^*$ is a unit.
The Cayley-Hamilton theorem says the following: Let $I(n)$ be the $n\times n$-identity matrix and let
$$p_A(t):=det(tI(n)-A)\in k[t].$$
It follows $p_A(t)$ is a monic polynomial
$$p_A(t)=t^n-tr(A)t^{n-1} + \cdots +(-1)^n det(A)$$
with coefficients in $k$ and $(-1)^ndet(A)$ as "constant term". The Cayley-Hamilton theorem says that
$$p_A(A):=A^n-tr(A)A^{n-1}+\cdots + (-1)^ndet(A)I(n)=$$
$$A^n+a_{n-1}A^{n-1}+\cdots + a_1A +(-1)^n det(A)I(n)=(0)$$
is the zero matrix $(0)$. From this we get an explicit formula for the adjugate matrix, $adj(A)$ in terms of the coefficients $a_i$ of $p_A(t)$ and $A$: We get
$$A(A^{n-1}+a_{n-1}A^{n-1}+\cdots +a_1I(n))=Aadj^*(A)=adj^*(A)A=(-1)^{n+1}det(A)I(n),$$
since $A$ commutes with any power $A^j$ for $j\geq 1$ an integer.
Here we define
$$adj^*(A):=A^{n-1}+a_{n-1}A^{n-1}+\cdots +a_1I(n),$$
and we may define
$$adj(A):=(-1)^{n+1}adj^*(A)$$
where
$$adj^*(A):=A^{n-1}+a_{n-1}A^{n-1}+\cdots +a_1I(n).$$
From this it follows
$$Aadj(A)=adj(A)A=det(A)I(n).$$
The determinant $det(-)$ has the property that $det(AB)=det(A)det(B)$ and we get the following result:
Lemma: A matrix $A\in M_n(k)$ is invertible iff $det(A)\in k^*$ is a unit.
Proof: If $A$ is invertible there is a matrix $B$ with $AB=BA=I(n)$ hence
$$det(AB)=det(A)det(B)=1\in k^*$$
and it follows $det(A)$ is a unit. Conversely if $det(A)\in k^*$ define
$$A^{-1}:=\frac{1}{det(A)}adj(A)\in M_n(k).$$
It follows $AA^{-1}=I(n)=A^{-1}A,$
hence $A$ is invertible. We get an explicit formula for the adjunct matrix $adj(A)$ in terms of the coefficients $a_i$ of $p_A(t)$.
A: An intuitive way of thinking would be as follows:
First, the determinant:-


*

*$detA$ gives us the scaling factor by which the $n \times n$ square matrix $A$, stretches out the $n$-dimensional volume w.r.t the standard basis. 
Example: a $2 \times 2$ matrix scales the $2$-dimensional volume, aka, the area.

*Now, if $A$ is singular, at least one of the dimensions is linearly dependent, as a result of which the $n$-dimensional space collapses to a $(n-1)$-dimensional space. Much like collapsing the sides of a rectangle along one of its edges, thereby obtaining a line.


The inverse:-


*

*If we were to transform a vector $\textbf{b}$ by $A$, we obtain $\textbf{c}$. Now, if we were to transform $\textbf{c}$ back to $\textbf{b}$, we would need to apply the inverse transformation, $A^{-1}$. 

*However, since $A$ is singular, when transforming $\textbf{b}$ to $\textbf{c}$, we lose information along at least one of the dimensions. Think of $A$ as a function that maps two different points to the same point. If we were to recover the original point from the transformed point, we wouldn't know which of the two points to recover to. There is no inverse for $A$.


Thus, we can see why for a singular matrix it's determinant is zero, and there exists no inverse. Hope this gives an intuitive understanding of the relation between the determinant and the inverse.
Note: The above discussion is by no means mathematically rigorous. Practitioners more experienced than I would surely find flaws in it. As such, my intentions are purely to present an intuitive discussion.  
