What does it mean to say "something over set"? For example, when it comes to strings and languages, we say that an alphabet is any finite set and a string over alphabet is a finite sequence of symbols from the alphabet.
What does, "string over alphabet," mean?
Does it mean that just that, "a finite sequence of symbols from the alphabet"?
Maybe I am confused by subsets, which we know to be unordered. Is this concept similar, but a sequence of values from a set, therefore ordered?
 A: A string is a sequence of symbols. Depending on the context, this might mean exclusively finite sequences or infinite sequences might be permitted.
An $X$ over a set is an $X$ whose "elements" are taken from that set ... what exactly the elements are depends on the context.
Let $\mathbb{N}$ be the numbers $1, 2, 3 \cdots$.
If $\Sigma$ is an alphabet and $\Sigma^*$ is the set of strings over $\Sigma$, then each element of $\Sigma^*$ is a total function from $\{x : x \in \mathbb{N} \;\text{and}\; 1 \le x \le n \}$ to $\Sigma$, where $n \in \mathbb{N} \cup \{\infty\}$ and the string in question has length $n$.
Your next question raises a subtle point.
A language $L$ is typically defined as a subset of $\Sigma^*$. The reason why $L$ is a subset in general is that it lets you rule out ill-formed strings. For example $((((x+$ is ill-formed in the language of arithmetic, but $(x+1)$ is not.
$L$ sometimes has a fairly natural ordering called the lexicographic ordering. If $\Sigma$ itself is ordered, then we can order any two strings relative to each other by comparing their elements one by one and arbitrarily decreeing that the absence of an element is less than any given element, so $ab$ would come before $aba$ in lexicographic order. However, the lexicographic ordering does not give you a sequence in general, for example these strings are in lexicographic order, but are not order-isomorphic to $\mathbb{N}$.
$$ a, aa, aaa, a^4, a^5, \cdots, b $$
Because of issues like this, it's convenient not to insist up front that all the strings of a given language be possible to arrange into a sequence.
