Can you define categories in terms of functors first? I've heard the properties of an object are fully determined by the morphisms to and from it.
So if functors map between categories then it ought to be possible to define what a category is in terms of functors?
Instead of "a functor maps between categories" "a category is what a functor maps to and from".
 A: It can be done and it's not at all useless to think about how this can be done. The fact that it can be done is due to the same principle that a category can be defined by specifying its morphisms together with a partial binary operation satisfying some properties. Then, the objects can be defined in terms of the morphisms as those morphisms with some special property. Now, suppose that you want to define the category of sets in an object free manner. Well, there is a well-known axiomatisation due to Lawvere of the category of sets. So, take these axioms and state them in the object free manner (this can certainly be done, since it's just a syntactical difference between the language with objects and without objects). So, we can describe the category of sets and functions without saying what sets are.
Now, you can repeat the trick for the category of categories. But why stop here. Functors are the things the natural transformations translate between, so let's define the whole 2-category of categories purely in terms of its $2$-cells. And why stop here, let's define the $3$-category of $2$-categories purely in terms of its $3$-cells. Well, this never stops. There is a slick way to define strict $n$-categories; simply the category of all categories enriched in strict $(n-1)$-categories. I suspect that this slick mechanism gives a similarly slick way to mechanically climb up the ladder of defining strict $n$-categories purely in terms of its highest level cells.
Now, let's improve. Strict $n$-categories are too strict. We really want weak $n$-categories. And we really really want weak $\infty $-categories. So, if we can take the syntactic approach to defining strict categories and methodically weaken it sufficiently, we might get a workable definition of weak categories. There are many approach to weak categories and some are syntactic (e.g., Batanin's definition via a suitable monad on globular sets).
A: I don't know whether it can be done, but I have also had this question recently.
To show that it could be done, we would need to state axioms in terms of "categories, functors, (other primitive notions)" and show that those were bi-interpretable with the typical axioms in terms of "categories, objects, morphisms".
Obviously categories can be interpreted as categories in both directions, and we know how to interpret functors from primitive notions of objects and morphisms. So what we need is some way to interpret objects and morphisms from primitive notions of functors.
For the sake notation, define (in the typical axiomatization)
$$ \mathbf{1} = \{ \ast \} \,, $$
the category with one object (the identity morphism is meant to be implicit above), and
$$ \mathbf{2} = \{\ast \to \ast \} \,,$$
(sometimes called the "interval category", cf. here or here) the category with two objects and one non-identity morphism.
Then we know in the standard axiomatization that objects of any category $\mathcal{C}$ can be identified with functors $\mathbf{1} \to \mathcal{C}$ and likewise morphisms of any category $\mathcal{C}$ can be identified with functors $\mathbf{2} \to \mathcal{C}$.
So if it is possible, I would guess that it likely requires not just having "category" and "functor" as primitive notions, but also "$\mathbf{1}$" and $"\mathbf{2}"$ as primitive notions. Then in this case objects would be interpreted/defined as functors $\mathbf{1} \to \mathcal{C}$, and likewise morphisms would be interpreted/defined as functors $\mathbf{2} \to \mathcal{C}$.
Using $\mathbf{1}$ as a primitive notion shouldn't be difficult, because that would just require an axiom saying something to the effect that it is the terminal object in the "category of all categories" $Cat$ (cf. here or here for rigorous definitions). E.g. something to the effect that,

There exists a category $\mathbf{1}$ such that for every category $\mathcal{C}$, there exists a unique functor $\mathcal{C} \to \mathbf{1}$.

I would guess that $\mathbf{2}$ also has a universal property that could be axiomatized solely in terms of functors, but I'm not sure. Because $Cat$ is apparently a (2-)topos (apparently the "archetypical example"), I would guess that $\mathbf{2}$ is "probably" (possibly) a subobject classifier for $Cat$ (a "Boolean subobject classifier"?). Anyway, if it has a universal property that can be stated solely in terms of categories and functors, and thus serve as an axiom, we are probably in good shape.
That being said, we would then either need to give axioms for the behavior of functors without referencing $\mathbf{1}$ or $\mathbf{2}$, or cheat and basically encode the standard axioms for objects and morphisms using/in terms of functors from $\mathbf{1}$ and $\mathbf{2}$. So whether such a project is possible (in a meaningful way) would seem to depend on how necessary it is to cheat when axiomatizing functoriality.
At this point it seems like what we might be looking for might be an axiomatization for $Cat$. This is definitely not my area of expertise, but it seems like such problems have been studied, and progress is possibly incomplete. In any case, see the following links for more reading that might give a better answer:

*

*(nLab) ET2CC: an elementary theory of the 2-category of categories

*(nLab) formal category theory

*(MathOverflow) category of categories as a foundation for mathematics
None of these seem to directly give axiomatizations in terms of categories and functors, per se, but the idea at least seems related. Again, hopefully someone gives a more clarifying/edifying answer one day.
