Can a vertical line be a valid PDF? Consider a random variable X that always takes a single value, c.
I would think that a valid PDF for X would be
$$f(x) = \begin{cases} 1/c & x = c \\ 0 & \text{Otherwise}\end{cases}$$
However, I know the PDF is supposed to be the derivative of the CDF, and in this case derivative of F(x) would be 0, both when $x<c$ and when $x\geq c$
What am I missing ?
 A: What you're looking for is the Dirac delta "function"; specifically
$$f(x) = \delta(x-c).$$
I've put "function" in scare quotes above, since the Dirac delta is not actually a real function, although it can in many cases be treated as one.
Informally, you can visualize $\delta(x)$ as a function which has an infinitely tall peak at $x=0$ and is zero everywhere else; specifically, the "infinitely tall peak" needs to be just tall enough to have the area under it integrate to $1$.  Of course, no actual real-valued function can have such a peak, but it turns out that, if you simply pretend that such a function exists and don't ask any hard questions about what its value at $x=0$ actually is, many calculations will just work as if nothing odd was going on.
Formally, the Dirac delta can be defined as a generalized function, specifically as a distribution.  (And no, the similarity of the name with "probability distribution" is not a coincidence.)
A: Your $f$ is not a valid PDF, since
$$
\int_{ - \infty }^\infty  {f(x)\,dx}  = 0 \neq 1.
$$
A: A PDF (probability density function) is only for a continuous distribution - more precisely, a distribution that is absolutely continuous with respect to Lebesgue measure.  Your distribution is discrete, not continuous, so it does not have a PDF, it has a PMF (probability mass function).  There are also singular continuous distributions (but nobody talks about them in elementary courses), and mixtures of the three types.
