A "set" is usually understood to be a collection of things that all share some common property, such as the set of all red fruits, the set of all animals that have kidneys, or the set of all numbers that are prime. A language is a set of strings, nothing more or less.
A number can be written as a numeral in any of several natural ways. For example, the number 135 might be written as
135, or in base 2 as
10000111, or in "unary" as
Such a string can be written onto the input tape of a Turing machine, and then you can run the machine and see if the machine accepts the input.
You can then ask the question of whether a particular set of numbers is decidable by asking the analogous question about the language consisting of the strings that represent those numbers. For example, is there a Turing machine that accepts the strings
1788, … and rejects the strings
1789, …? Yes, there is, and so we say that the set of even numbers is decidable.
In questions of decidability it does not matter whether we represent numbers with base-10 numerals, or base-2 numerals, or "unary" numerals, since a Turing machine can easily convert between these representations. So we allow ourself to forget about the strings that represent the numbers, and just focus on the numbers themselves. The language consisting of strings that represent even numbers is decidable, for any reasonable way of representing numbers as strings, so we just say that the set of even numbers is decidable, or we say that the property of evenness is decidable.
Your question is asking about the set of prime numbers. Is there a Turing machine which will accept the strings
1013,… and reject the others?