I have two data sets $M$ and $N$ that are equally large. They have different averages and medians and I want to prove that, with 5% significance, the values don't differ solely out of chance.
I'm trying to understand what the two-tailed paired t-test really tells me. I've performed such a test using Google Sheets built-in function TTEST() and it returns the value 0.03. Does this mean that it's a 3% chance that the values differed as they did solely out of chance and that I've successfully proved (at a 5% significance level) that they in fact differ?