Poisson Process - non-zero probability of more than one arrival Quoting Bertsekas' Introduction to Probability:

An arrival process is called a Poisson process with rate $\lambda$ if
  it has the following properties:
a) Time homogenity - the probability $P(k,\tau)$ of $k$ arrivals is
  the same for all intervals of the same length $\tau$
b) The number of arrivals during a particular interval is independent
  of the history of arrivals outside this interval.
c) Small interval probabilities - The probabilities $P(k,\tau)$
  satisfy:
$P(0,\tau)=1-\lambda\tau + o(\tau)$
$P(1,\tau)=\lambda\tau + o_1(\tau)$
$P(k,\tau)=o_k(\tau)$ for $k=2,3,...$
Here, $o(\tau)$ and $o_k(\tau)$ are functions of $\tau$ that satisfy
$\mathbb{lim}_{r\to0}\frac{o(\tau)}{\tau}=0$,
  $\mathbb{lim}_{r\to0}\frac{o_k(\tau)}{\tau}=0$

Then we are given the formula:
$$P(k,\tau)=e^{-\lambda\tau}\frac{(\lambda\tau)^k}{k!}$$

Note that a Taylor series expansion of $e^{-\lambda\tau}$ yields:
$P(0,\tau)=e^{-\lambda\tau}=1-\lambda\tau+o(\tau)$
$P(1,\tau)=\lambda\tau
 e^{-\lambda\tau}=\lambda\tau-\lambda^2\tau^2+O(\tau^3)=\lambda\tau+o_1(\tau)$.

First of all, what are $o(\tau)$, $o_1(\tau)$ and $O(\tau)$ in the Taylor expansion? Does it have anything to do with Taylor expansion per se? I thought that $o$ is the little-o notation, but its definition is quite different - $ f(n) = o(g(n))$ if $g(n)$ grows much faster than $f(n)$. In this case, it's quite different. Then what is it? 
Secondly, the author doesn't prove that the $o$ terms above satisfy 

$\mathbb{lim}_{r\to0}\frac{o(\tau)}{\tau}=0$,
  $\mathbb{lim}_{r\to0}\frac{o_k(\tau)}{\tau}=0$

as stated in the definition of Poisson process. How can we prove it?
Most importantly - why do we want it to satisfy the properties described in 'c) Small interval probabilities'? These 3 formulas are not arbitrary, there has to be a good reason for them.
Ideally, if we let $\lambda \to 0$ and it's natural to expect that the probability $P(k,\tau)$ to equal exactly $0$ in the limit, but apparently it's not possible (there will always be that tiny number, $o_k(\tau)$). Or does it equal $0$ in the limit?
 A: $o$ and $o_k$ are the authors functions with the properties that:
$$\lim_\limits{r\to0}\frac{o(\tau)}{\tau}=0$$
$$\lim_\limits{r\to0}\frac{o_k(\tau)}{\tau}=0$$
The actual definition of the functions is left undefined, and the author has said that they behave as above.
As these functions tends to zero, they allow for very minor variations to be treated as negligible. For the purpose of this section of the book, we do not need other properties.
The big-O notation used here:
$$P(1,\tau)=\lambda\tau e^{-\lambda\tau} =\lambda\tau-\lambda^2\tau^2+O(\tau^3) =\lambda\tau+o_1(\tau)$$
means that $|\lambda\tau e^{-\lambda\tau}-\lambda\tau+\lambda^2\tau^2|$ is smaller than $M|\tau^3|$ for some constant $M$ and as $\tau\to0$.
Then we can let:
$$o_1(\tau)=\lambda^2\tau^2+O(\tau^3)$$
because this satisfies the limit property the author has defined.
By the definition of probabilities we have:
$$\sum_k P(k,\tau)=1$$
and if $\lambda,\tau\to0$, when $k=0$ we have $0^0=1$.
The final point is that $o(\tau)$ becomes very trivial in relation to the  other calculations, and can be ignored.
A: This will probably answer a couple or all of your questions:
E.g. this one
"And actually, why do we want the oo terms to satisfy the two properties above? Why exactly these properties, not some others?"
Its one of several definitions of a poisson process and to be a poisson process it needs to satisfy those properties.
(You might need to zoom in to read it or read it here
link)

A: Too long for a comment, too short for the bounty.
Your question

Most importantly - why do we want it to satisfy the properties
  described in 'c) Small interval probabilities'? These 3 formulas are
  not arbitrary, there has to be a good reason for them.

is indeed what makes the work needed to understand Poisson processes worthwhile. The three formulas that define the Poisson process are just what make it useful modeling many different phenomena. 
There are a few applications on the wikipedia page https://en.wikipedia.org/wiki/Poisson_point_process#Early_applications .
Here's a longer list, from http://www.aabri.com/SA12Manuscripts/SA12083.pdf

Whether one observes patients arriving at an emergency room, cars driving up to a gas  station, decaying radioactive atoms,
  bank customers  coming to their bank, or shoppers being  served at a
  cash register, the streams of such even ts typically follow the
  Poisson process. The  underlying assumption is that the events are
  statis tically independent and the rate,  μ , of these  events (the
  expected number of the events per time  unit) is constant. The list of
  applications of  the Poisson distribution is very long. To name just 
  a few more:  
• The number of soldiers of the Prussian army killed 
  accidentally by horse kick per year  (von Bortkewitsch, 1898, p. 25). 
• The number of mutations on a given strand of DNA pe r time unit
  (Wikipedia-Poisson,  2012).  • The number of bankruptcies that are
  filed in a mont h (Jaggia, Kelly, 2012 p.158).   • The number of
  arrivals at a car wash in one hour (A nderson et al., 2012, p. 236).  
• The number of network failures per day (Levine, 201 0, p. 197). \
• The number of file server virus infection at a data  center during a
  24-hour period . The  number of Airbus 330 aircraft engine shutdowns
  per  100,000 flight hours. The number of  asthma patient arrivals in a
  given hour at a walk-i n clinic  (Doane, Seward, 2010, p. 232). 
• The
  number of hungry persons entering McDonald's restaurant. The number
  of work- related accidents over a given production time, The  number
  of birth, deaths, marriages,  divorces,  suicides, and homicides over
  a given per iod of time (Weiers, 2008, p. 187). 
• The number of
  customers who call to complain about  a service problem per month 
  (Donnelly, Jr., 2012, p. 215) .  
• The number of visitors to a Web
  site per minute (Sh arpie, De Veaux, Velleman, 2010, p.  654). 
• The
  number of calls to consumer hot line in a 5-min ute period (Pelosi,
  Sandifer, 2003, p.  D1).  
• The number of telephone calls per minute
  in a small  business. The number of arrivals at a  turnpike tollbooth
  par minute between 3  A.M.  and 4  A.M.  in January on the Kansas 
  Turnpike (Black, 2012, p. 161)

