I'm not sure if this is the right place to publish this question, but I am trying to understand something about hypothesis testing. recently in a lecture on the subject a professor emphasized how people can "cheat" at hypothesis testing to get the results they want by adjusting an alpha value or by continuing to take samples until their data matched the results they wanted.. I obviously want to avoid doing that.
i am working on designing a computer program that simulates a simple card game where the player draws three cards, if one of the cards is a 10, or he draws all odd numbered cards, he wins.
My question is this.. i can calculate the probability of the player winning on paper to be .3833 but how can i test that against the simulation? Lets say the program runs some large number of times, and i calculate the mean of the probability of the player winning to be some number x. If i run a two sided hypothesis test with alpha being .05, and the null gets rejected, why is it wrong to then run the test again with alpha = .1 to see what certainty level the test would fail to reject at? in other words, the program won't spit out the exact .3833 every time, so i want to prove the correctness of my program using a hypothesis test to see how close or far off it might be.