Groups under Multiplication 
Let $G=GL(2,\mathbb R)$ and $ H =\left\{ 
\left[\begin{array}{ccc|c}
a & 0 \\
0 & b
\end{array} \right]:\mbox {$a$ and $b$ are nonzero integers }\right \}$ under the operation matrix multiplication. Disprove that $H$ is a subgroup of $G=GL(2,\mathbb R)$.

Well I have learned that I have to prove that:

1) Show that e∈H (where e is the identity) 
  2) Assume that a∈H , b∈H 
  3) Show that ab∈H 
  4) Show that $(ab)^{-1}$ (Inverse) 

So I know that It does not hold, But how do I prove that?
When I try to prove that e∈H I only get the Identity matrix and that holds because I get that a and b are nonzero integers.
When I prove that a and b is in the set I get that it is because both a and b are nonzero integers and that is the identity matrix.
Now I proved that a.b∈H as follows:
$$ H = 
\left[ \begin{array}{ccc|c}
a & 0 \\
0 & b
\end{array} \right]
\left[ \begin{array}{ccc|c}
c & 0 \\
0 & d
\end{array} \right] = 
\left[ \begin{array}{ccc|c}
ac & 0 \\
0 & bd
\end{array} \right]
$$
With a = b = c = d = 1 I get the Identity matrix again.
Now I know if a = b = 2 the subgroup will not hold because the inverse will be a set of rational numbers and H is only a subgroup if it contains only Integer.
Is my reasoning correct and if not where did I go wrong? 
 A: You're fourth step is a bit of an overkill, in terms of testing whether a subset is a subgroup. Indeed, we need to check:


*

*$(1)$ the identity  element of $G$, namely $e = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$ is in $H$ and that 

*$(2)$ for two elements $h_1, h_2 \in H$, $h_1\cdot h_2 \in H$, 


But with respect to testing for inclusion of inverses in H (whether for every element $h \in H$, we also have that $h^{-1} \in H)$, we don't need to test for inverses of products of elements, since you've shown $H$ is closed under multiplication.
Indeed, you are correct that for most $h$ with integer entries $a, b$, the inverse $h^{-1} \notin H$ because the corresponding non-zero entries will likely be rational numbers, but not both integers.
So, taking for example $h = \begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix}$, when we compute $h^{-1} = \begin{bmatrix} \frac 12 & 0 \\ 0 & \frac 12\end{bmatrix}$, we've shown that $h^{-1} \notin H$, because $\frac 12 \notin \mathbb Z$. 
Hence, we've shown that $H$ is not closed under taking inverses. (All you need to provide is one counterexample of some element $h \in H$ such that $h^{-1} \notin H$, to show that $H$ is not closed under taking inverses.) And we therefore conclude that $H$, as defined, is NOT a subgroup of $G$. And then you're done.
A: In short:
$$\begin{pmatrix}2&0\\0&1\end{pmatrix}^{-1}=\begin{pmatrix}\frac12&0\\0&1\end{pmatrix}\neq H$$
We, of course, take the inverse from the big group...
A: First, I will show that the traditional $A^{-1}$ is the only general inverse for the matrix $A$. Suppose that $AA^{-1}=AB$, for some some potentially different inverse $B$. Acting on the left by $A^{-1}$ gives $A^{-1}=B$. I will now give a particular example of a matrix in $H$, with non-integer entries down the main diagonal. Consider $\begin{bmatrix} 7&0\\
0&4\end{bmatrix}$. Taking the inverse,$\begin{bmatrix} 7&0\\
0&4\end{bmatrix}^{-1}=\frac{1}{28}\begin{bmatrix} 4&0\\
0&7\end{bmatrix}=\begin{bmatrix} \frac{1}{7}&0\\
0&\frac{1}{4}\end{bmatrix}$. Upon examination, the given matrix does not satisfy the properties of $H$.
