You can see :
We prefer, however, to axiomatize not "set" but "function". The latter notion
certainly includes the former. [...] We consider two domains of objects, that of " arguments" and that of" functions". (Both words are, of course, to be taken in a purely formal way, as if they had no meaning.) The two domains are not identical, but they partly overlap. (There are "argument-functions", which belong to both domains.)
Now a two-variable operation $[x,y]$ (read "the value of the function $x$ for the argument $y$"), whose first variable $x$ must always be a "function" and whose second variable $y$ must always be an "argument", is defined in these domains. What is formed by means of it is always an "argument", $[x,y]$.
The operation $[x,y]$ corresponds to a procedure that is that is encountered everywhere in mathematics, namely, the formation, from a function $f$ (which must be carefully distinguished from its values $f(x)$!) and an argument $x$, of the value $f(x)$ of the function $f$ for the argument $x$. Instead of $f(x)$ we write $[f,x]$.
You can see also :
The lambda calculus is a type free theory about functions as rules, rather
than as graphs. "Functions as rules" is the old fashioned notion of function and refers to the process of going from argument to value, a process coded by a definition. The idea, usually attributed to Dirichlet, that functions could also be considered as graphs, that is, as sets of pairs of argument and value, was an important mathematical contribution. Nevertheless the $\lambda$-calculus regards functions again as rules in order to stress their computational aspects.
For Category theory, you can see :
What is category theory? As a first approximation, one could say that category theory is the mathematical study of (abstract) algebras of functions.
We begin by considering functions between sets. I am not going to say here what
a function is, anymore than what a set is. Instead, we will assume a working
knowledge of these terms. They can in fact be defined using category theory, but
that is not our purpose here.