Welcome to MSE!
What "exactly" is a truth value is a tricky question to answer -- it depends on what you're doing. For everyday mathematics, a truth value is just true or false, and tells you something about your proposition/conjecture/etc.
In mathematical logic, we view logic itself as a mathematical object which we can study, and that means we can consider variants and see what happens. A lot of people get bogged down in questions like "well what are sets really" or "what is truth really", and (at least to me) these questions are analogous to learning that groups other than $\mathbb{Z}$ exist, then asking "but what is $\mathbb{Z}$ really"?
So, with my NB out of the way, let's talk about some ways that people have played with truth values. This (roughly) corresponds to the different kinds of semantics that people have used to interpret logical formulas.
First and foremost, truth values could be $\{ T, F \}$. Now if you see a formula like $p \lor q$, and you know whether $p$ and $q$ are true or false (that is, if you fix a model) you can bootstrap your way up to get the truth value of $p \lor q$, or any other (propositional) logical formula you're interested in.
Of course, there's a next kind of "obvious" step if you're abstraction-minded. We aren't using many properties of $\{ T, F \}$ here. All we really need is a way to interpret $\lor$, $\land$, and $\lnot$. There are also some rules we expect these symbols to satisfy, and it turns out we can do this with any boolean algebra! (There are actually ways to get by with only a heyting algebra, but you need to change your logic, and the semantics aren't different enough to warrant their own paragraph).
Concretely, maybe your lattice is the subsets of $\{a,b\}$, and we interpret $\lor$ by union, $\land$ by intersection, and $\lnot$ by complementation. Then now we have $4$ "truth values",
and if we think of $p$ as getting the truth value $\{a\}$ and $q$ as getting the truth value $\{b\}$, then $p \lor q$ must get the truth value $\{a,b\}$, $p \land q$ gets $\emptyset$, etc.
These semantics are useful in understanding forcing, and also have some application in computer science.
But possible answers get even wilder. This next section is a pretty big spike in complexity, so take care ye who enter here.
If we let $\mathcal{C}$ be a topos, then it has an internal logic whose truth values are the elements of $\Omega$, its subobject classifier. I want to emphasize that even though we're working in some abstract setting, we really haven't changed too much. If you unpack all the category theoretic language, it turns out the elements of $\Omega$ form a heyting algebra! Then we're really doing the truth values from the last paragraph, but the way we get those truth values is a bit more complicated (and lets us do more complicated things).
So again, when we as mathematicians talk daily about "truth values", we mean True or False. Logicians like to explore alternate systems, in which "truth values" are more interesting than that, but that doesn't mean truth values are "really" something else. The rabbit hole doesn't go that deep, there are just other rabbit holes to explore.
I hope this helps ^_^