How to answer problems in set theory and other fields of higher mathematics I've recently begun teaching myself some higher mathematics, and am using the textbook Transition to Higher Mathematics, Structure and Proof (2nd Ed.) by Dumas and McCarthy. I've read through the first chapter on set theory and am working on the exercises. The first exercise is below:
Show that the following set is empty:
$$\{ n \in \mathbb{N} \mid \text{$n$ is odd} \land \text{$n = k(k + 1)$ for some $k \in \mathbb{N}$} \}.$$
Being unfamiliar with the proper way to answer such problems, the solution I would provide is as follows:


*

*When $k$ is neither $-1$ nor $0$, $k(k+1)$ is equal to the product of a an odd and an even number, which is always even.

*When $k$ is either $-1$ or $0$, $k(k+1)$ is equal to 0, which is neither odd nor even.

*$k$ is never odd, thus the set is empty.


I would very much appreciate if someone could provide advice on how to answer this and similar problems in a more "mathematical" way.
 A: The argument you present doesn't really make sense to me.  Basically, your Step 1 is the proof, but I think that there is a hole, since you don't actually show that an even number times an odd number is even (if this is a result which has been previously shown, nevermind).  Both Steps 1 and 2 seem to assert that $-1$ is a natural number, which is incorrect—this assertion is confusing.  Finally, in Step 3, I think you mean to say that $n$ is never odd (rather than $k$—this is a typo, not a big deal).
As an aside, there is no need to number your steps (though it is entirely reasonable to number different cases).  Just write sentences in English.
For my own taste, I would give the following proof.  We start by recalling the definition of even and odd, just to be sure.

Definition: A natural number $n$ is said to be even if there is a natural number $m$ such that $n = 2m$.  A natural number $n$ is said to be odd if there is a natural number $m$ such that $n = 2m+1$.

It is worth noting that every natural number is either even or odd, though this is a result which requires some proof (it isn't hard—you should give it try).  We'll use this result below.  And now the proposition we want to prove:

Proposition: The set
  $$\{ n \in \mathbb{N} \mid \text{$n$ is odd} \land \text{$n = k(k + 1)$ for some $k \in \mathbb{N}$} \}$$
  is empty.

Proof:  To prove the proposition, it is sufficient to show that if $k\in\mathbb{N}$, then $n=k(k+1)$ is even.  There are two possibilities:  either (1) $k$ is even, or (2) $k$ is odd.


*

*If $k$ is even, then there is some $m\in\mathbb{N}$ such that $k = 2m$.  Then
$$ n
 = k(k+1)
 = 2m(2m+1)
 = 2 [m(2m+1)], $$
which is the product of $2$ and the natural number $m(2m+1)$, and therefore even.

*If $k$ is odd, then there is some $m\in\mathbb{N}$ such that $k = 2m+1$.  Then
\begin{align} n
 &= k(k+1) \\
 &= (2m+1)[(2m+1)+1] \\
 &= (2m + 1)(2m+2) \\
 &= (2m + 1)[2(m+1)] \\
 &= 2 [(2m+1)(m+1)],
\end{align}
which is the product of $2$ and the natural number $(2m+1)(m+1)$, and therefore even.
In either case, $n$ is even, which is the desired result.
A: Since, as you identified, either $k$ is even or $k+1$ is even, it follows that $n=k(k+1)$ is even, but the set specifies that $n$ is odd; thus the set is indeed empty.
One defines an even number $m$ as any integer that can be written as $m=2r$ for some $r\in\Bbb Z$. 
Exercise: How would one then define a number as odd?
Hint:

 Take the number to be equal to one above an even number.

Can you take it from here?
