Prove that $H$ is normal subgroup of $G$ I have a following question.
Let $p$ be a prime and let $G$ be a group and $H$ be a subgroup of $G$.
$$
G = \left\{
\begin{bmatrix}
a & b \\
0 & 1 
\end{bmatrix}
: a,b \in \mathbb{Z}_p, a \neq 0
\right\}$$
and 
$$ H = \left\{
\begin{bmatrix}
1 & b \\
0 & 1
\end{bmatrix}
: b \in \mathbb{Z}_p\right\}
$$
prove that $H$ is normal subgroup.
In my textbook, it gives two types of definition for normal subgroup.
The first one is if left coset and right coset are same with the subgroup, the subgroup is normal.
The other definition is let $H$ be a subgroup of $G$. Then $H$ is a normal subgroup of $G$ iff $xhx^{-1} \in H$ for every $h \in H$ and every $x \in G$.
I have tried with the first definition and got 
$$
xH = \left\{ \begin{bmatrix}
a & ac+b \\
0 & 1
\end{bmatrix} \right\}
$$
and for right coset
$$
Hx = \left\{ \begin{bmatrix}
a & b+c \\
0 & 1
\end{bmatrix} \right\}
$$ and I can't see that $xH \subset Hx$ and $Hx \subset xH$.
And I don't know how to apply the second definition.
Need help!
 A: A subgroup $H$ of a group $G$ is a normal subgroup of $G$ if $ghg^{-1}\in H$ whenever $g\in G$ and $h\in H$.
To check if your $H$ is a normal subgroup of your $G$, let 
$
\begin{bmatrix}a&b\\0&1\end{bmatrix}\in G
$
and
$
\begin{bmatrix}1&c\\0&1\end{bmatrix}\in H
$. Then 
\begin{align*}
\begin{bmatrix}a&b\\0&1\end{bmatrix}
\begin{bmatrix}1&c\\0&1\end{bmatrix}
\begin{bmatrix}a&b\\0&1\end{bmatrix}^{-1}
&=
\begin{bmatrix}a&b\\0&1\end{bmatrix}
\begin{bmatrix}1&c\\0&1\end{bmatrix}
\frac{1}{a}
\begin{bmatrix}1 & -b\\ 0&a\end{bmatrix} \\
&= 
\frac{1}{a}
\begin{bmatrix}a & ac+b\\ 0 & 1\end{bmatrix}
\begin{bmatrix}1 & -b\\ 0&a\end{bmatrix} \\
&=
\frac{1}{a}
\begin{bmatrix}
a&a^2c\\0&a
\end{bmatrix} \\
&=\begin{bmatrix}1&ac\\0&1\end{bmatrix} 
\in H
\end{align*}
Hence $H$ is normal in $G$.
Note that $\Bbb Z_p$ is a field (assuming $p$ is prime) so $\frac{1}{a}$ is the modular multiplicative inverse of $a$.
A: You can use your first definition. Suppose $x \in G$ with:
$x = \begin{bmatrix}a&b\\0&1\end{bmatrix}$. Now, let's write:
$H = \left\{\begin{bmatrix}1&h\\0&1\end{bmatrix}: h\in \Bbb Z_p\right\}$
(this is just so we can keep in mind that the $h$ entries came from $H$).
Now, as you discovered, we have:
$xH = \left\{\begin{bmatrix}a&ah+b\\0&1\end{bmatrix}: h \in \Bbb Z_p\right\}$
while:
$Hx = \left\{\begin{bmatrix}a&h+b\\0&1\end{bmatrix}: h \in \Bbb Z_p\right\}$.
So, I will show you how to see that $xH \subseteq Hx$. What we need to show is that:
$ah_1 + b = h_2 + b$, given $h_1$ (in other words we need to find $h_2 \in \Bbb Z_p$ such that this is true). Well, this is true if and only if $ah_1 = h_2$. But since $\Bbb Z_p$ is closed (under multiplication mod $p$), it is certainly true that $ah_1$ is some element of $\Bbb Z_p$, so any element of $xH$ lies in $Hx$.
On the other hand, we also need to show that $Hx \subseteq xH$. In this scenario, we are given $h_2$, and we have to find $h_1 \in \Bbb Z_p$ such that:
$ah_1 = h_2$.
And here is where $a \neq 0$ comes into play: since $a$ is non-zero, it is a unit, which means $a^{-1} \in \Bbb Z_p$. Again, by closure, we have that $a^{-1}h_2 \in \Bbb Z_p$, so if we take $h_1 = a^{-1}h_2$, we find that:
$ah_1 = a(a^{-1}h_2) = (aa^{-1})h_2 = h_2$, as desired.
In short, the mapping, for a given $a \neq 0 \in \Bbb Z_p$, given by:
$f: \Bbb Z_p \to \Bbb Z_p$ where $f(h) = ah$, is a bijection.
A: Those definition are actually equivalent! (why don't you try to prove it?). I'll give you a hint about how to prove it using the second definition.

let 
  $$h=
\begin{bmatrix}
1 & c \\
0 & 1 
\end{bmatrix}\in H$$
  and let
  $$g=
\begin{bmatrix}
a & b \\
0 & 1 
\end{bmatrix}\in G$$ 
  compute $g^{-1}$ and prove that $ghg^{-1}$ lies in $H$.

A: to follow Solid Snake's suggestion it will help if you know that
$$
\begin{bmatrix}
a & b \\
0 & 1 
\end{bmatrix}^{-1} = \begin{bmatrix}
a^{-1} &-ba^{-1}\\
0 & 1 
\end{bmatrix}
$$
