What are the consequences if Axiom of Infinity is negated? What mathematics can be built with standard ZFC with Axiom of Infinity replaced with its negation?
Can the analysis be built? 
Is there special name for "ZFC without Infinity" set theory?
I also assume that using symbols like $+\infty$, $-\infty$ when dealing with properties of functions and their limits would still be possible even with Axim of Infinity negated (correct me if I am wrong).
UPDATE
In light of the answer by Andres Caicedo which suggested Peano arithmetic, I want to point out that Wikipedia says about Peano arithmetic "Axioms 1, 6, 7 and 8 imply that the set of natural numbers is infinite, because it contains at least the infinite subset". I do not know how this can be interpreted as having axiom of infinity but I am interested what if Peano arithmetic modified the following way:


*

*Added an axiom 5a:


There is a natural number $\infty$ which has no successor, for any natural number n $S(\infty)=n$ is false.


*

*Axiom 6 modified:


For every natural number n except $\infty$, S(n) is a natural number
 A: I'm not certain why this isn't in the previous answer, but it seems worth mentioning that in first-order PA you cannot restrict the models to the specific one you want (the natural numbers 0, 1, 2, ...).  Put another way, the same axioms of PA are satisfied by, for one example, a model where you have 0, 1, 2, ..., a, b, c, d, ..., where the latter set of "numbers" are an infinite distance from the former (standard) set.  First-order PA is essentially too weak to define "natural number" in the restrictive manner that is typically intended.
The point I wanted to make is that your axiomatization does not change this fact.  I'm not certain off-hand whether it is consistent with the definitions for + and *, but if it is, it still allows for (can be modeled by) an infinite number of objects (e.g. 0, 1, 2, ..., a, b, c, inf.).
A: ZFCfin, ZFC with infinity replaced by its negation is biinterpretable with PA, Peano Arithmetic. This result goes back to Ackermann,


*

*W. Ackermann. Die Widerspruchsfreiheit der allgemeinen Mengenlehre, 
Math. Ann. 114 (1937), 305–315. 


You may want to read about the early history of this theory here:


*

*Ch. Parsons. Developing Arithmetic in Set Theory without Inﬁnity: some 
historical remarks, History and Philosophy of Logic 8 (1987), 201–213.


That this is a reasonable setting for finitary mathematics has been argued extensively, see for example:


*

*G. Kreisel. Ordinal logics and the characterization of informal notions of 
proof, in Proceedings of the International Congress of Mathematicians. Edinburgh, 14–21 August 1958, J.A. Todd, ed., Cambridge 
University Press (1960), 289–299. 

*G. Kreisel. Principles of proof and ordinals implicit in given concepts, in 
Intuitionism and proof theory, A. Kino, J. Myhill, R. E. Veseley, eds., 
North-Holland (1970), 489–516. 
That the theories are biinterpretable essentially means that they are the same, only expressed in different ways. More formally, we have a recursive procedure that assigns to each formula $\phi$ in the language of PA a formula $\phi^t$ in the language of ZFCfin and conversely, there is another recursive procedure that to each $\psi$ in the language of ZFCfin assigns a $\psi^{t'}$ in the language of PA. These procedures have the property that $\phi$ is provable in PA iff $\phi^t$ is provable in ZFCfin, and $\psi$ is provable in ZFCfin iff $\psi^{t'}$ is provable in PA, and moreover PA proves for each $\phi$ that it is equivalent to $(\phi^t)^{t'}$ and similarly ZFCfin proves of  each $\psi$ that it is equivalent to $(\psi^{t'})^t$. 
The translations are actually not too involved. In one direction, we can define in ZFCfin the class of natural numbers (the ordinals) and define its arithmetic operations the usual way, so that the resulting class satisfies PA. In the other direction, there is a nice argument: We can define in PA the relation $nEm$ iff there is a 1 in the $n$th place from the right in the binary representation of $m$. It turns out that the resulting structure consisting of the numbers seeing as the universe, with $E$ being membership is a model of ZFCfin. If we apply these procedures to the standard model $\omega$, we obtain $V_\omega$, and viceversa. 

As for your question of whether one can do analysis in this setting, not quite. One can code some infinite sets in PA but not enough for all the arguments one needs in elementary analysis. A bit more formally, the subsystem of second order arithmetic known as $ACA_0$ proves the same arithmetic statements as PA, they are equiconsistent (provably in PA) and it is understood that it captures what we usually call predicativism. In particular, arguments requiring non-predicative definitions cannot be proved here. See:


*

*Feferman, S. Systems of predicative analysis, i. The Journal of Symbolic Logic 29, 1 
(1964), 1–30. 

*Feferman, S. Systems of predicative analysis, ii. The Journal of Symbolic Logic 33 
(1968), 193–220. 
The standard reference for systems of second order arithmetic is the very nice book:


*

*Simpson, S. Subsystems of second order arithmetic. 2nd edn, Cambridge University Press, 2009.


You will find there many examples of theorems, results, and techniques in basic analysis that go beyond what $ACA_0$ can do. For example, the basic theory of Borel sets cannot be developed in $ACA_0$.
