Large sample confidence interval Can someone please be so kind to check if my answer is correct

 A: See more details in this answer for the definition of a Clopper Pearson confidence interval, and problems with approximations in confidence intervals (the approach you have used). I've considered the same principles here.
Clopper Pearson confidence interval: with $s=2$ (defects), $n=500$ (boards), $\alpha=0.05$ and the Beta quantile function from R, that is qbeta:
$$[\text{qbeta}(\alpha/2,s,n-s+1), \text{qbeta}(1-\alpha/2,s+1,n-s)]=[0.00049,0.01435].$$
Therefore, the 95% Clopper Pearson confidence interval for the true frequency is $[0.00049,0.01435]$ in this case.
A: The "exact" Clopper-Pearson CI may not be
so good for only 2 successes in 500 trials. The version of the Clopper-Pearson CI given in @bluemaster's answer (+1) uses a continuous beta approximation to the discrete binomial. (For some details, you can google 'Clopper-Pearson' and look at the Wikipedia article, among others.)
A Bayesian-based approach, also using beta distributions, might be be more accurate, but the answer does not differ by much for your problem.
Suppose we use the 'flat' prior distribution $\mathsf{Unif}(0,1) \equiv
\mathsf{Beta}(1,1),$ which has density function $f(p) = p^{1-1}(1-p)^{1-1} = 1,$
for $0 < p < 1.$ 
The likelihood function corresponding to your data is 
$$f(x\mid p) = {500 \choose 2}p^2(1-p)^{488} \propto p^2(1-p)^{488},$$
where the symbol $\propto$ (read "proportional to") indicates that the
expression on the right omits the constant of the function and shows only
the 'kernel'.
The posterior distribution is
$$f(p\mid x) = f(p)\times f(x\mid p) \propto p^{1-1}(1-p)^{1-1} \times p^2(1-p)^{488} \\
\propto p^2(1-p)^{488} = p^{3-1}(1-p)^{499-1},$$
where we recognize the final expression as the kernel of
$\mathsf{Beta}(3, 499).$ Then a 95% Bayesian probability interval for $p$ is 
$(0.0012, 0.0143),$ which can be
found by cutting 2.5% from each tail of the posterior distribution.
The computation using R statistical software is shown below.
qbeta(c(.025, .975), 3, 499)
## 0.001236581 0.014345537

Notes: (a) Bayesian statisticians interpret 'probability intervals' differently from
frequentist 'confidence intervals', but frequentists often use Bayesian
interval estimates as confidence intervals. (b) The prior distribution can
influence the Bayesian interval estimate, but the 'flat' prior used here
provides minimal information. If prior information is available, it should
be reflected in the prior distribution (c) When there are almost no
successes or almost no tails, experience and simulation experiments
have shown that a flat-prior Bayesian interval often comes closer to
providing the promised 95% coverage than do traditional intervals
based on a normal approximation.
A: The problem with the Clopper-Pearson interval is that, being an exact interval, it guarantees a coverage probability of at least $100(1-\alpha)\%$, but due to the discrete nature of the binomial distribution, the actual coverage probability can be substantially higher, especially when $p$ or $1-p$ is close to $0$, or $n$ is small.  The resulting interval is in some sense "too wide" or "too conservative," because not all of the $\alpha$ can be spent.
For large samples, the Wald interval (i.e. arising from a normal approximation to the binomial) is good when $p$ and $1-p$ is not close to $0$.  But here, this is not the case:  the number of observed events in the sample is very small.
An alternative to a Bayesian approach that remains in the frequentist regime is the Wilson score interval:
$$\frac{1}{1+z_{\alpha/2}^2/n} \left( \hat p + \frac{z_{\alpha/2}^2}{2n} \pm z_{\alpha/2} \sqrt{\frac{\hat p (1 - \hat p)}{n} + \frac{z_{\alpha/2}^2}{4n^2}} \right),$$ where $\hat p = x/n$ is the sample proportion, $n$ is the sample size, and $z_{\alpha/2}$ is the upper $\alpha/2$ quantile of the standard normal distribution (so for example, when $\alpha = 0.05$, $z_{.025} \approx 1.96$).  This interval has good coverage properties even for small sample sizes or extreme sample proportions.  I leave it to you as an exercise to compute this interval for your question. 
