You understand the concept of a probability distribution, so let's start there.
A probability distribution has a cumulative distribution function that gives us the probability that a variable is less than or equal to a given value. In the discrete case, this CDF is the sum of values at discrete points of the probability mass function; in the continuous case, it is the integral over the real line of a probability distribution function.
In either case, the pmf/pdf is non-zero and consequently its sum/integral is monotonically non-decreasing to 1.
From the pmf/pdf, we can obtain distribution moments in the typical way: expected value, variance, and higher-order moments, using the standard formulas which need not repeat here. One way of looking at this is that you can characterize a distribution in terms of its moments. A Gaussian distribution is parameterized by its mean and variance; a Poisson distribution is parameterized by its process intensity. You still need to know the shape of the distribution function, but if you do know that, all you need is a handful of parameters.
(Actually, there are other ways that we can address this when you don't know the distribution!)
Now, let's look at a data set. In the real world, we really don't like dealing with a continuum. If you measure the voltage on a widget, it would be a lot easier if 5.000001 volts was effectively the same as 5.000002. Even when the physics underlying our data set dictate that the output belongs to a continuum, we want to discretize it some way.
Typically, we do this using a histogram. There are plenty of resources on how to intelligently set the bin size for a histogram, but ultimately there is no perfect, natural, context-free way to do so. As you know, a histogram counts events at discrete points.
In this way, a histogram is very much like a probability mass function: we cannot have a negative number of events in a bin, and if you add the total events from left to right, you end up counting the total number of events. Although the histogram won't sum to unity, if you just divide every count by the total number of events, you end up exactly with something that looks like a pmf.
Furthermore, you can compute statistics on the data. Mean, variance, kurtosis, etc. are all statistical moments that can be computed in a straightforward manner. In fact, there are many different types of moments that you can compute, but if you compare the formula for doing so to the canonical way of computing different, say, expected values on a pmf, they are very similar (if not identical)!
So you can take your data and turn it into something that looks like a pmf. You can even perform the same steps on the data to get statistical moments. The only thing that's really difficult to do is to find the shape of the distribution. Is it Gaussian? Binomial? Poisson? Weibull?
There are tests for showing how well your data fits any given theoretical distribution, but unless you have infinity samples, you can never say for 100% sure. Furthermore, your moment computations aren't exactly the same. Your theoretical distribution might demand discrete values at exactly 1 volt, 2 volts, 3 volts, etc., but you compute your actual sample mean using measured values; .956 volts, 2.14 volts, 2.98 volts, etc.
So in the end, a distribution of data is the characterization of the statistical moments of the data along with a comparison of the data to a theoretical distribution. Saying that data has mean and variance of X and Y doesn't give you the full picture. But saying that the data has mean X and variance Y and passes a goodness of fit test for a normal distribution does mean something, because we have tools for organizing the data in a way that has a natural analog to purely theoretical probability definitions.
