So I am not looking for a proof that some are recursively enumerable or others are not, but I have trouble understanding why this "middle set" of total recursive functions has not a property that the "outer sets" have!?
That in itself shouldn't be surprising. Every "difficult" set sits between the empty set and the set of all strings/numbers, even though those to sets are the easiest there are. It shouldn't be remarkable, then, that we can also sandwich a difficult set such as the set of (Turing machines that implement) total recursive functions, between two more interesting, but still easier, sets.
Also, I didn't found a proof that the set of (Gödel numbers of) primitive recursive functions are not recursive? Is it non-sense?
That depends on your encoding. It is possible to define a Gödel numbering such that every primitive recursive function has a number, and such that the set of numbers that encode primitive recursive functions is recursive.
The cost of that is that the numbering can't then include indices for all total recursive functions, at least not if the encoding is effective -- that is, there is an algorithm to get from the index of a function and a concrete function argument, to the value of the function at that argument.
If we have an effective (and otherwise well-behaved in a certain minimal sense) encoding that represents every total recursive function, then the set of numbers that happen to represent primitive recursive functions under that encoding is necessarily undecidable.
And, what about elementary functions? Is it a recursive set? Recursively enumerable set?
"Elementary functions" are usually taken to be certain functions $\mathbb R\to \mathbb R$, which live in a somewhat different world than the functions $\mathbb N\to\mathbb N$ (or equivalently $\{0,1\}^*\to\{0,1\}^*$) that we speak about in recursion theory.
Again, it is certainly possible to choose an encoding that can only encode elementary function but does encode all of them -- in which case set of indices of elementary functions is "boringly" recursive. But that doesn't help us much if what we were after was something we could give an arbitrary real function and ask whether it is elementary or not.