How did the rule of addition come to be and why does it give the correct answer when compared empirically? I'm still a high school student and very interested in maths but none of my school books describe these kind of things, and only say how to do it not the whys. My question is very simple, for example:
        19
       +25
      = 44

because the one from adding 9 and 5 goes on to add with 1 and two. How did this rule of addition come to be?
Here's a bit of explanation that can be useful(sorry if it is frustrating):
Suppose we are a 3 year old child and no one teaches us how to add and we recognize 1 is our index and 5 all palm fingers held up. Someone gives us the problem add: 1+5 so we hold 'em up, right? and again someone gives us to add 8564+2345 so we can't lift 'em up. So we try to device a rule but we don't recognize 6+4= 10 in which 0 stays and one jumps neither can we say that only the digits from rightmost positions are to be added. This is what i meant.
 A: Here's a sketch of the general theory.
Theorem: Let $b$ be an integer greater than $1$. For every positive integer $N$, there exist


*

*A non-negative integer $n$,

*Integers $a_{n}$, $a_{n-1}$, ..., $a_{2}$, $a_{1}$, $a_{0}$ satisfying $0 \leq a_{k} < b$, and $0 < a_{n}$,
such that
$$
N = a_{n} b^{n} + a_{n-1} b^{n-1} + \dots + a_{1} b + a_{0}
  = \sum_{k=0}^{n} a_{k} b^{k}.
\tag{1}
$$
Moreover, the representation (1) is unique (can be done in precisely one way).
Definition: The integers $(a_{k})_{k=0}^{n}$ are called the (base $b$) digits of $N$. The string of symbols
$$
a_{n}\, a_{n-1} \cdots a_{2}\, a_{1}\, a_{0}
$$
is called the base-$b$ representation of $N$.
The process of carrying arises naturally when you ask how to find the base-$b$ representation of a sum $N + N'$ from the base-$b$ representations of the summands $N$ and $N'$. If
\begin{alignat*}{5}
N &= a_{n} b^{n} &&+ a_{n-1} b^{n-1} &&+ \dots &&+ a_{1} b &&+ a_{0}
  &&= \sum_{k=0}^{n} a_{k} b^{k}, \\
N' &= a_{n}' b^{n} &&+ a_{n-1}' b^{n-1} &&+ \dots &&+ a_{1}' b &&+ a_{0}'
  &&= \sum_{k=0}^{n} a_{k}' b^{k},
\end{alignat*}
then
$$
N + N' = (a_{n} + a_{n}') b^{n} + (a_{n-1} + a_{n-1}') b^{n-1} + \dots + (a_{1} + a_{1}') b + (a_{0} + a_{0}')
  = \sum_{k=0}^{n} (a_{k} + a_{k}') b^{k}.
\tag{2}
$$
If every "coefficient" $a_{k} + a_{k}'$ is smaller than $b$, then (2) immediately gives the base-$b$ representation of $N + N'$ by stringing the coefficients. The snag is, one or more coefficients $a_{k} + a_{k}'$ might be greater than or equal to $b$. In the sum $19 + 25$, for example, the "ones place" $9 + 5$ cannot be represented by a single digit. Instead, we write $9 + 5 = 14 = 10 + 4$; the $4$ stays in the ones place, but the extra $10$ carries to the tens place. Since $10$ units of one is a single unit of ten, we add one to the tens place to accommodate the carry. (The equation $9 + 5 = 14$ itself is a special case.)
Generally, if $b \leq a_{k} + a_{k}'$, write
$$
(a_{k} + a_{k}') b^{k}
  = \bigl[b + (a_{k} + a_{k}' - b)\bigr] b^{k}
  = b^{k+1} + (a_{k} + a_{k}' - b) b^{k}.
\tag{3}
$$
Each of $a_{k}$ and $a_{k}'$ is less than $b$, so $0 \leq a_{k} + a_{k}' - b < b$. In words, $a_{k} + a_{k}' - b$ is a base-$b$ digit. Equation (3) therefore determines the carrying rule: Add the $b^{k}$ coefficients, $a_{k} + a_{k}'$. If the sum of the coefficients is greater than or equal to $b$, subtract $b$ from their sum (obtaining a proper base-$b$ digit) and add $1$ (i.e., $b$ units of $b^{k}$) to the $b^{k+1}$ coefficient, the next column to the left.
To get a clean algorithm for addition, start at the rightmost digit and work leftward, carrying as necessary. (Carrying only affects digits farther to the left. For example, a carry in the hundreds place has no effect on the tens and ones.)
A: Well essentially what you're doing is this:
$$19+25 = (10 + 9) + (20 + 5) = \underbrace{(9 + 5)}_{\text{ones}} + \underbrace{(10 + 20)}_{\text{tens}}$$
Then you'll get:
$$14 + (10 + 20)$$
So for the ones digit you get $4$ so let's subtract that from $14$ to get $10$. For the tens digit, since you still have $10$ left over from adding the ones digits you have to "carry over" $10$ into $(10+20)$:
$$10\,\text{(carried over digit)}+(10+20) = 20+20 = 40$$
Adding the result from the ones digit gives:
$$40 + 4 = 44$$
The "rule" of carrying over from one place value into another is just that, except that you do it vertically.
A: What's the meaning of $19$ ? Obviously not $1+9$, but $10+9$, because the first digit "weights" $10$. Similarly, $25$ is $20+5$. The $2$ is worth $20=2\times10$. Another digit on the left would weight $100$, like $625=6\times100+2\times10+5$.
Now how do we add two decimal numbers ?
$$19+25=(1\times10+9)+(2\times10+5)$$
We can regroup $(1+2)\times 10+(9+5)=3\times10+14$. Fine for the first digit, but no so for the units: $14$ isn't a single digit. To workaround, we decompose $14=1\times10+4$.
Now, $$19+25=(1+2+1)\times10+(4)=44.$$
You can easily generalize and imagine the graphical layout that makes this process well-organized.
A: The reason that this rule of addition came to be is that people count to 10 with their fingers.  Other number systems have developed in other cultures, such as counting by 5's on one hand, or counting to 60 by 5 subgroups of the sub-digits on the other hand.  Computers only use '0' and '1' for positive voltage and negative voltage.  Some people are also trying to get people to start using base 12 since it has many factors for how small of a number it is.  Base 10 is just a standard.  
Standard numbers are represented in the form:
number = place_valebase^1st_digit_value + place_valebase^2nd_digit_value + place_vale*base^3nd_digit_value+...
Where:
0 <= place_value < base
And "place_value", "base", and "nth_digit_value"'s are integer values.  These last two statements are important since it means that for any digit space there is some symbol representing the maximum integer value of that digit space, and there is a minimum of 0 (or any appropriate symbol) for that digit space.  
There needs to be a system that expresses numbers greater than 9 and less than 0 in the number system, as well as decimal values.  The problem is that if we just changed the base, we would need a symbol for every number organized in such a way that each number could be identified upon sight without being seen before.  
That won't work.  Which is why we have digit spaces.  
Each digit space is an improvement upon each previous digit space so that it can pick up the pace where the last digit space left off, so 9+1=10 in base 10, as well as 1+1=10 in base 2, F+1=10 in base 16, etc.  The digit system in place allows the next digit to represent numbers that aren't in the same set as the first digit so that we don't waste symbols representing multiple numbers more than once.  
Except for leading zeroes, which is why there are number systems like this:
https://en.wikipedia.org/wiki/Gray_code.  Gray still has overlapping numbers with different numbers of digits, but this is an example to show that writing out numbers can be done in many ways.  
Speaking of which...
Even though the standard for decimal mathematics is to use big endian notation (where the digit representing the greatest value comes first), it can be useful to write out numbers in little endian notation (where the digit representing the smallest value comes first).  Example:
Big endian notation:
 52
+29
=81

Little endian notation:
 25
+92
=18

Little endian notation can be especially useful for serial transmissions between computers since addition of the first part of the number can cause the carry and the sum can be computed progressively while the message is received, instead of all at once if the message is received in big endian notation.  
There are also mathematical notations such as postfix notation, which I am a fan of since it abolishes the counter-intuitive and perfectly-useless order of operations as well as frees up the parentheses symbols for other uses.  
Or you can just start writing numbers vertically and see how that goes ;)
As for WHY addition is necessary: it's not.  Binary and computers don't actually do addition; it's just a series of logic operations on input values, which could also be mapped into the decimal system as well with some extra effort.  Addition is just a function resulting in a value.  
Consider the equality:
a+b=c+d
There is no such thing as natural equivalence; it has been defined by humans as a way to recognize when two functional outputs give the same result.  This is just a way to say 'map the addition of a and b to the addition of c and d, and vice-versa', AKA
+(a,b)<->+(c,d)
I think I'm done now that I've obliterated mathematical notation and the idea of equivalence altogether.  If you have any remaining questions or feel that this explanation is incomplete, don't hesitate to notify me :)
A: It's not that carrying is a rule, but more that it's a natural result of addition within a number system.
Adding two numbers is a natural process. If I have a big group of things, and I add another big group of things, then I have added the two groups together. When it's with things (rather than tracking the number of them in your head), there's no need to perform any carrying over because the things are simply there. If I want to know the number of them, I can count them.
However, we represent numbers within a base (let's assume we're using base 10 for the moment). A single digit decimal number can only represent 0-9, so even when we're counting (adding 1 at a time) we have to carry to count to 10. That is, when a digit is at the highest value within the base but you still have more to add, you need to use the next digit higher digit. In base 10, to add 1 to 9 I reach 10, effectively having carried the 1.
The necessity of carrying is slightly more apparent with lower bases. For example, in binary (base 2), each digit can be 0 or 1. Instead of decimal where you have a digit for ones, tens, hundreds, etc., in binary you have a digit for ones, twos, fours, eights, etc.
The decimal number 7 is represented in binary as
111

(that's 1*1+1*2+1*4)
If we want to add 1 (binary 1) to 7 (binary 111), we get 8 (binary 1000). It looks like this:
   111
+  001
= 1000

You'll see that, because we're adding to a digit which is already at its maximum value, it carries to the next higher digit. In this case, because each higher digit is also maxed out, it continues carrying up until it reaches an empty digit.

The reason carrying is so explicit in addition is because, with the technique you showed, it's easy to add any numbers. By writing the numbers this way, adding arbitrarily large numbers becomes as easy as adding two 1-digit numbers several times over. 88010343 + 69139740 becomes:
  8|8|0|1|0|3|4|3
+ 6|9|1|3|9|7|4|0

It's only slightly more complicated because these aren't independent operations; we carry up to the next digit as we go along because it's easy to do in-place, rather than having to make a note and do it later (e.g. the millions digit had a result of 17, so once I'm done I'll add 10 million).
A: Suppose you have no money in your pocket, but you remember that
last week you lent Alice $19$ pounds and you lent Bob $25$ pounds.
So you find Alice and Bob and collect the money that they owe you.
But Alice and Bob have only some ten-pound notes and one-pound coins
in their pockets, so they pay you as follows:


*

*Alice: $1$ ten-pound note, $9$ one-pound coins.

*Bob: $2$ ten-pound notes, $5$ one-pound coins.


Now you have $3$ ten-pound notes and $14$ one-pound coins.
How much money do you have?
You don't like carrying around so many coins (they're heavy),
so you go to your bank, give the teller $10$ of your one-pound coins,
and receive back a single ten-pound note. This is a fair exchange, isn't it?
You neither lost nor gained money by this exchange.
You now have $4$ ten-pound notes. You also have $4$ one-pound coins
left after giving $10$ of the original $14$ to the bank.
Is it clear that you have $44$ pounds after adding the $25$ pounds from Bob
to the $19$ pounds from Alice?
When you are adding up numbers in the usual notation
(base-ten place-value numbers),
the "carry" operation is just like going to the bank to exchange
a number of one-pound coins for a smaller number of ten-pound notes.
The main difference is, 
when doing arithmetic with base-ten place-value numbers,
you have to do this, because your "pocket" only has enough "room"
for at most nine coins of notes of any particular denomination;
that is, the only way you can write a number in the standard format
is to write down the number of "ones", 
then the number of "tens" to the left of that,
then the number of "hundreds" to the left of that, etc.,
and you have only the digits $0$ through $9$ to say how many ones,
how many tens, and so forth.
A: 
How did this rule of addition come to be?

From abacus.
A: It's simple.
Say you're working in a certain column.
(It doesn't matter which column .. the first one, fourth column .. whatever.)
Say you get to "ten" in that column.  Well, that's the same as a "1" in the next column!
It's just that simple.  Each "ten" in "that" column is like a "1" in the "next" column.
In fact, you COULD just write a "ten" in that column, if you wanted (!)
So, eight thousand .. 8-0-0-0 ... you could write as 7-ten-0-0 if you wanted .. see ?!!
7-ten-0-0 would mean "seven thousands", "ten hundreds" (which is of course just a thousand) and zero tens and ones.
Of course, the ten in that column is just worth a 1 in the next column!  7 plus 1 is 8, so we just write 7-0-0-0.
Enjoy.
A: This is something that comes from the way we represent numbers with symbols. I'll simplify your example a bit for explanation purposes
     9
   + 3
  = 12

Let's say you have 9 apples in a box. You need to write down how many apples are in the box for when you go to sell them. Now imagine that there is no such thing as letters or numbers than how do you do it?
The easiest way would be to make a line on the box for every apply in it. But then you would have to count the 9 lines instead of apples which gets you nowhere.
What if instead of making 9 separate lines, you grouped the lines together in groups of 5. There is still a little counting involved when reading the lines but it is much quicker.
So then to represent the addition problem it would look like
          ||||/  ||||
       +          |||
 =   ||||/  ||||/  ||

When we do the addition we would start with our 9 lines
||||/  ||||

And in order to add 3 lines we first have to complete our group of 5
||||/  ||||/

Then we "carry" over the last two lines to make 12
||||/  ||||/  ||

When working with numbers the same thing is happening only instead of making groups of 5 we make groups of 10 because we use the 10 symbols of 0-9 to represent our numbers as opposed to using a pattern of 5 lines.
A: We start by separating the ones and the tens:
$$19 + 25 = (10 + 9) + (20 + 5)$$
We will first add the ones, then the tens:
$$(9 + 5) + (10 + 20) = 14 + (10 + 20)$$
Now we extract the ones from 14, and pass along the ten:
$$14 + (10 + 20) = 4 + 10 + (10 + 20) = 4 + (10 + 10 + 20)$$
We can now add the tens:
$$4 + (10 + 10 + 20) = 4 + 40 = 44$$
Now try that for other sums, such as $29 + 43$ and $147 + 84$. In the last one, just separate the ones, the tens and the hundreds.
A: This method works even in mixed radix systems: Say you have $2$ yards, $2$ feet, and $9$ inches of rope, while your friend as $3$ yards, $1$ foot, and $7$ inches. How much do you have together? Well, $9$ inches plus $7$ inches is $16$ inches; but $12$ inches are a foot, so that's $1$ foot and $4$ inches. If we put aside the $4$ inches, we next have $2+1+1=4$ feet; but $3$ feet are a yard, so that's $1$ yard and $1$ foot. If we put aside the $1$ foot, we next have $2+3+1=6$ yards. So in total we have $6$ yards, $1$ foot, $4$ inches.
Of course, with our good old decimal number systems the addition rule becomes less complex as we always have the same ten smaller units in one bigger unit, but the idea behind it is the same. (You may think of counting/adding pebbles, where ten pebbles are bundles into bags, ten bags are bundles into boxes, ten boxes are stuffed into a trunk, and so on.
A: A visual representation:
  19
+ 25
= 44

Let's remove the answer and split the numbers into tens and units:
    10 + 9 {= 19}
+   20 + 5 {= 25}

This give us:
    30 + 14

We can make a new addition out of these numbers:
    14
+   30

Let's split these numbers into tens and units too:
    10 + 4 {= 14}
+   30 + 0 {= 30}

The 10 represents the 1 that was carried from the units column to the tens column. The second number, 30, must end in a zero because we only used digits from the tens column to calculate it. This means that at this stage we will never get a carry from the units column to the tens column, allowing us to progress past this stage, although for larger numbers we could get a carry from the tens column to the hundreds column (not shown). We can now complete the sum:
    10 + 4
+   30 + 0
=   40 + 4 {= 44}

This is why this method works. It is because breaking a number into parts (not digits, $10\ne1$) and adding all the parts up again will always give you the original number. $14=10+4=4+10=7+7=11+3=1+2+3+8$, etc. This is called associativity. When you carry the 1 from 14, you are actually carrying 10, but in the tens column you write 10 as 1.
A: Other people have done all the math-y specifications, and have done so spectactularly. From reading the comments, it seems as if the "why we have to carry exceeds over" hasn't been received by everyone yet. Let me try to explain that with an "analogy" (it's not really an analogy, but.. let's just go with it):
I assume you are familiar with the binary representation (or base-2).
In all numbersystems, there is a highest number any single-digit representation can contain. In base-2, that number is 1. In base-10, that number is 9.
This means that in base-2, it's not possible to represent the number 2 like 2 or 02 -- it has to be 10, because the value 2 is too high to be represented in the first single-digit position. Ergo, you must carry the exceed over to the next position.
Another way of looking at this is, again using base-2, each single-digit representation -- each position -- is a counting of how many of its corresponding values the number should specify, and each position can only correspond to one value. Let's take the base-2 number 10 and start reading from the right:
1st position: 0 -- how many ONEs do we have
2nd position: 1 -- how many TWOs do we have  
Which of course corresponds to the number 2 in base-10.
Another number, 1011101 and its corresponding base-10 values:
1st position: 1 -- number of ONEs
2nd position: 0 -- number of TWOs
3rd position: 1 -- number of FOURs
4th position: 1 -- number of EIGHTs
5th position: 1 -- number of SIXTEENs
6th position: 0 -- number of THIRTYTWOs
7th position: 1 -- number of SIXTYFOURs  
Counting it all up -- I don't speak latex, so I'm sorry for the formatting -- it becomes 64*1 + 32*0 + 16*1 + 8*1 + 4*1 + 2*0 + 1*1 = 93.
Carrying this logic over to base-10, where each position can hold {0, 9} and naturally cannot be double-digit:
1st position: number of ONEs
2nd position: number of TENs
3rd position: number of HUNDREDs
...
nth position: number of 10^(n-1)s
If we have the addition 8 + 4 = 12, and we weren't able to carry exceeds, the first position on the right-hand side would contain the double-digit number 12 (and the second position would contain 0), which is illegal. The first position can only contain the number of ones, and it is impossible in this representation to have more than nine ones. The moment you get 10 ONEs, you don't actually have 10 ONEs -- you have one TEN.
Thusly, we either carry the 1 (meaning one TEN in this example) over to the next position where it belongs (with the rest of the TENs), leaving 2 in the position for ONEs, giving us the number 1*10 + 2*1 = 12 -- or the numbersystem breaks down.
In the everyday, this representation is still used explicitly to some extent:
"One-thousand two-hundred" = 1200:
1st position: 0 -- number of ONEs
2nd position: 0 -- number of TENs
3rd position: 2 -- number of HUNDREDs
4th position: 1 -- number of THOUSANDs  
Finally, imagine if you weren't able to carry exceeds: you have literally no way of knowing what any number you're presented with means. The number 1200 could mean "one-thousand two-hundred" or it could mean "one-thousand and twenty", or the number 10200 could mean "one-thousand two-hundred" or "ten-thousand twenty" or "one-thousand twenty", depending on how you align the addition. 
Re: myself earlier, the numbersystem is 100% useless if you do not carry.
A: Every natural number $N$ in the decimal system is represented by $n + 1$ digits
$a_n \dots a_1a_0$
for which each stands for the number of powers of $10$ that add up to the given number.
$N = a_n\cdot 10^n + \dots + a_1 \cdot 10^1 + a_0 \cdot 10^0 $
Each digit $a_n$ has 10 possible "states" form $0$ to $9$ indicating how many of the powers contribute to $N$.
If you now add two numbers $A + B$
$A = a_n \cdot 10^n + \dots +a_1 \cdot 10 + a_0$
$B = b_n \cdot 10^n + \dots +b_1 \cdot 10 + b_0$
$A+B = (a_n + b_n) \cdot 10^n + \dots + (a_1 + b_1) \cdot 10 + (a_0+b_0)$
it can happen that these $(a_n + b_n)$ exceed the possible 10 "states" that are representable by 10 digits. Thus, we have to remove the exceed $c$ from $a_n + b_n$: $c = a_n + b_n - 10$ and add this exceed to the next higher power (and repeat if we get again another exceed).
Note that in other number systems (like hexadecimal) the procedure works the same except we have more (or less) states to represent with each digit.
A: All addition is based on a basic assumption about numbers that fit our observations of reality:  numbers are consistent.  5 is always going to be five.  If it's five on monday, then it's going to be five on tuesday.  And if we pick it up, get on an airplane and take it to France and turn it upside down it will still be 5.
So...
FACT 1: Numbers exist 
FACT 2: a = a.  If b = a then a and b are the same number and if a and be are different numbers a $\ne$ b.
Now if we combine two numbers we will get a third and as numbers are consistent if we add the same two numbers we will always get the same third number.
So ...
FACT 3: Addition is a binary operation.  For any two numbers, a and b, we can add them together to get a + b = c,, a third number.
Now, again, numbers are consistent.  So it doesn't matter how we group the numbers when we add them or what order we add them in.  They will add to the same amount.
So...
FACT 4:  Addition is associative:  a + (b + c) = (a + b) +c
FACT 5: Addition is commutative: a + b = b + a.
Numbers are consistent so we can group them into bunches.  We can count these groups of numbers as though they were discrete objects.
FACT 6: A  multiplication exists for integers where  na = a + a + ... + a (a added n times).
And numbers are consistent, he says yet again for a final time, so no matter how you bunch them up the still add to the same number.
So 
FACT 7:  Multiplication is distributive over addition: n(a + b) = na + nb.
And as long as those facts are observed .... addition works.  And we can do addition by observing those facts.
So now the real question is how do we represent numbers and he do the representations allow as to add?
We could do what the romans did and use seven symbols to represent values from 1 to 1000.
Then: XIX is two tens and a one but the one before the ten means it is one less: two tens less a one.  XXV is two tens and a five.
So XIX + XXV = (X + X -I) + (X + X +V) but we can ungroup these and move them around because addition is associative and commutative: X + X + X + X + V - I = (X + X + X + X) + (IV).  We have IV of those Xes  so we can group those: IV(X) + (IV) = V*X - I*X + (IV) = L - X + IV = XLIV.
Or we can do what no culture has ever done in the history of the world and write numbers by their prime factorization.
So $19 + 5^2$ is... well that's really hard.  We have to replace numbers with other numbers.  $19 = 2^2*5 - 1$ So $19 + 5^2 = (2^2*5 -1) + 5^2 = (2^2*5 + 5^2) - 1 = 5(2^2 + 5) - 1 = 5*(3^2) - 1 = (2^2 + 1)3^2 - 1 = (2^2*3^2) + (3^2 - 1) = 2^2*3^2 + 2^3 = 2^2(3^2 + 2) = 2^2*11$.  That was all substituting and grouping and substituting and regrouping.
$19 + 5^2 = 2^2*11$
Or we can use arabic decimal numbers where the number $abc = a*10^2 + b*10 + c$.
So $19 + 25$ can be spread out to $(1*10 + 9) + (2*10 + 5)$ which we can ungroup and shift around to get $1*10 + 2*10 + 9 + 5 = (1*10 + 2*10) + (9 + 5)$ which we can undistribute $(1 + 2)*10 + (14) = 3*10 + 14$ and spread out $3*10 + 10 + 4$ and regroup and rebunch $(3 + 1)*10 + 4 = 4*10 + 4$ and ... compress: $44$.
That's why addition works.
A: Though my answer is not quite what I want as well, this is what I could conclude seeing all the views here and it's not an algebric verification. If it will be possible I'll come back here and post an answer delving deeper into the history:
