If you are programming or using categories or universal algebra then in some sense the type of the objects does not matter. In particular there is no really reason to conceptualize a notion of "number" you just describe operations on your objects. For example, in functional programming the concept of a flat-map (or map-reduce) is regarded by some as a multipliction metaphor. That is made precise by extracting the abstraction of a monad functor F with a join $F\circ F\to F$. E.g. a List of Lists of integers is easily "joined" together into a single List of integers. For all mathematical purposes that is in a product, even if you wouldn't regard lists as numbers.
As for division then, if division to you means $a\div b$, i.e. a function
def div(a:A,b:A):A then its can be as general as any multiplication metaphor, e.g. operating on sets or lists, or trees, elephants. You might then ask what makes it division verses a multiplication, and for that I would suggest that you have it in the company of some other operation you already called multiplication and which is in a meaningful sense reciprocal to your division. E.g. if you have $a\times b$ and $a\div b$ then $(a\times b)\div b=a$ or some such identity.
But as for caring if $a$ and $b$ are "numbers", there is no need for that on a mathematics level at least. Maybe a philosopher would object.