What does the p-value really mean? Suppose I fit a linear model m1 to the data produced below:
  #true parameters
    x = rnorm(100,5,1)
    b = 0.5
    e = rnorm(100,0,3)
    beta_0= 2.5
    beta_1= 0.5
    y = beta_0 + beta_1*x + e
    plot(x,y)
   #linear fit
    m1 = lm(y~x)
    abline(m1)
    summary(m1)

The p-value I get is 0.66. When I sample x again (rerun the code), and fit the regression the p-value is 0.05. Why do I get different p-values for different samples, and how do you reconcile these differences? 
 A: Because the slope is too small reliably to distinguish it from 0, in view 
of the relatively large error SD, with only 100 observations. 
Change $n = 100$ t0 $n = 1000$ and you'll get P-values less than 1%.
Call:
lm(formula = y ~ x)

Residuals:
    Min      1Q  Median      3Q     Max 
-8.2432 -2.0408  0.0304  1.9965 11.0491 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  2.23279    0.49033   4.554 5.92e-06 ***
x            0.54850    0.09601   5.713 1.47e-08 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 3 on 998 degrees of freedom
Multiple R-squared:  0.03167,   Adjusted R-squared:  0.0307 
F-statistic: 32.64 on 1 and 998 DF,  p-value: 1.465e-08


In half a dozen runs, my largest P-value was about 0.0003.
Note: I'm not sure you're running the intended simulation because I don't 
see where b = 0.5 is used.
A: First, a p-value is a $statistic$, so it's value will change based on the data. The real benefit of the p-value is that it effectively transforms all hypothesis tests into a standard uniform random variable.
Given test statistic $T$ and data $X$, we have a function $H$ that calculates a p-value (p) from $T$ and $X$, so:
$$p(X)=H(T(X),X)\sim U(0,1)$$
Now, inferences about the original test are equivalent to inferences about $p(X)$. In other words, the following rejection rules have the same Type I error rate (but not the same Type II error rate!!):


*

*Reject when $p(X)<0.05$

*Reject when $p(X)>0.95$

*Reject when $0.025 > p(X)$ or $p(X) < 0.975$


Or any other crazy interval you want, so long as the rejection region contains 5%,
A: Since you are simulating the data, each estimate is a random variable and so is every p-value. Here is a probability histogram of the p-values in question, for the parameters that you chose. It's not a uniform distribution because the null hypothesis "slope = 0" is in fact false.
 
