Statistical significance: Test for different (small) sample sizes? I recently finished my simulations which I now need to statistically analyse. I am comparing mean firing rates (FR) of neurons in the same population (Pop1) before and after I change the connectivity probability (Pconn).
I have n=140 observations/simulations for the network with the natural connectivity state, but I have only n=5 observations/simulations for the network with varied connectivity.
(I am arbitrarily changing the Pconn, while measuring the FR of a population before and after the change. n=140 observations correspond to BEFORE and n=5 observations correspond to AFTER the change)
I am quite unsure in which statistical test should I use to determine the significance of my results, since the sizes of samples vary so much. Can anyone suggest a solution?
Thanks in advance, Tea
 A: Welcome to MSE, Tea. First of all I'd like to suggest this post gets moved to https://stats.stackexchange.com/ as that is the Statistics Stack Exchange and your question is strictly about Statistics.
Now, as a Statistics undergrad, here is my opinion:
Ideally you should get more observations corresponding to AFTER, since you have 140 observations corresponding to BEFORE. If effect, you really have only n = 5 observations because your goal is to compare before vs after and you only have 5 observations related to after. If getting more after observations is not possible, then you should use some small-sample significante test, such as the t-test already mentioned. However, notice two things:


*

*If you use the t-test, then you should use the paired version of the t-test because your observations correspond to before some treatment, and after some treatment.

*The t-test is only appropriate if you assume that your sample data follows some t distribution (or, equivalently, approximately normal). So in order to use a significance t-test you'd first need to test your data to verify it follows a t distribution in the first place. If the latter test fails, you should use a non-parametric (distribution-free) paired test.
