Is a proof also "evidence"? Can I use the terms proof and evidence synonymously or is there a difference? You usually see mathematicians writing about "proof" while other sciences instead discuss "evidence" - is there a practical difference or do they mean the same, an understanding of the basis for the scientific result and it is that basis that we either call proof or evidence?
 A: If you're a 20th-century sort of scientist, you might describe "evidence" for a hypothesis as a valiant but unsuccessful attempt to falsify it—to prove it wrong by showing that it implies something false.
For example, say it's the early 1800s, and you want to test the hypothesis H that Fresnel's wave model accurately describes the behavior of light. You notice that Fresnel's model has a really surprising consequence: it implies that, under certain conditions, the shadow of an opaque disk should have a bright spot at the center! Let's call that surprising consequence I. If the hypothesis I turns out to be wrong—which seems pretty likely—then H must be wrong as well, because H implies I.
Your colleague François Arago sets up an opaque disk under just the right lighting conditions. He sees the bright spot! Against all expectation, the hypothesis I holds. Your valiant attempt to falsify H has failed. Arago's experiment goes down in history as strong evidence in favor of H. The bright spot, to your chagrin, is sometimes named after you.

When mathematicians talk about "evidence," they're talking about the same thing. If you want to test a mathematical conjecture C, you look at its implications. If you're lucky, you discover that C has a really surprising consequence I which is feasible to check. If you check I, and find that it's true, you've found evidence for C.
The generalized Riemann hypothesis GRH, as @BillCook mentions, is an excellent example of a conjecture with lots of supporting evidence, but no known proof.
The generalized Riemann hypothesis is a conjecture about certain functions from the integers to the complex numbers, called Dirichlet characters. I found its statement very intimidating at first glance, but it's not terribly hard to understand if you break it down into bite-size pieces and become very comfortable with complex analysis.
The conjecture GRH has many surprising implications, and quite a few of them have been proven true. Here are some examples, cribbed from an excellent MathOverflow answer by @KConrad. The dagger † marks things I got more or less directly from @KConrad's answer.

PRIMES_P. There's a fast way to check whether a given whole number is prime.

It's notoriously hard to factorize a whole number quickly. Since checking whether a number is prime is the same as checking whether it has more than two factors, it would be very surprising if PRIMES_P were true.
In the 1970s, a graduate student named Miller showed that GRH implies PRIMES_P.† About 25 years later, a series of undergraduate research projects by Pandey, Bhattacharjee, Kayal, and Saxena culminated in a proof, by Agrawal and the newly graduated Kayal and Saxena, that PRIMES_P is true.

WG (weak Goldbach). Every odd number larger than five is the sum of three primes.

This conjecture is a consequence of the infamous Goldbach conjecture, which claims that every even number larger than two is the sum of two primes. The Goldbach conjecture has resisted proof for hundreds of years, so even just finding evidence for it by proving WG would be surprising.
In the 1920s, however, the famous mathematical duo Hardy and Littlewood found evidence that GRH implies WG.† In the late 1990s, Deshouillers, Effinger, te Riele, and Zinoviev confirmed that GRH implies WG.† About 15 years later, Helfgott proved WG.†

SN (Skewes' number). As $n$ grows, the number of primes from $1$ through $n$ will exceed
  $$\int_0^n \frac{dt}{\log t}$$
  before $n$ reaches
  $$10^{10^{10^{964}}}.$$

Littlewood proved in the 1910s that the number of primes from $1$ through $n$ will exceed that integral eventually, but nobody has ever seen it happen. Saying that it has to happen before a certain value of $n$ is pretty bold—like predicting the apocalypse!
Nonetheless, Skewes proved in the 1930s that GRH implies SN.† Then, in the 1950s, Skewes proved SN.†

CN (class number). There are only finitely many imaginary quadratic fields with class number one.

The numbers you can build from the integers and $\sqrt{-43}$ by adding, subtracting, multiplying, and dividing form a consistent number system, called an "imaginary quadratic field." If you start from $\sqrt{-1}$ or $\sqrt{-19}$ or some other square root of a negative integer, you get a different imaginary quadratic field. Every imaginary quadratic field is labeled by a number called its "class number." Imaginary quadratic fields with class number one are special, and they seem to be very rare. The conjecture CN says they're incredibly rare: among the infinite variety of imaginary quadratic fields, there are only finitely many with class number one.
In the 1910s, Gronwall proved that GRH implies CN.† Specifically, what Gronwall proved is that GRH implies a slightly weaker conjecture, WGRH, which implies CN.† Later, Deuring, Mordell, and Heilbronn proved CN.† They did it in a bizarre way: they proved that the negation of WGRH also implies CN!†
A: Hypothesis: $n^2-n+41$ is prime, for all natural $n$.
Evidence: True for $n=1, 2, 3,\ldots, 40$.
That seems persuasive, but for $n=41$ the hypothesis is false.
In general, science typically uses inductive reasoning, i.e. noticing a pattern and claiming it continues forever.  An improvement is the scientific method, i.e. noticing a pattern, making a prediction, and checking if the prediction is true  the next time.  Unfortunately, neither of these is considered a mathematical proof.
A: I'd like to add an aspect to your question, that hasn't been touched by any of the other answers thus far:
Evidence plays a significant role in current day mathematics.
Vectornaut already mentioned one example of this. Let me provide another, that has a different kind of flavour to it.
I'd like to talk about large cardinals, but before I do so, let me provide a definition that is suitable for our purposes. Let us say that $\kappa$ is a large cardinal iff there is a set theoretical formula $\phi$ such that 
$$\operatorname{ZFC} + \phi(\kappa) \vdash "\text{there is some weakly inaccessible cardinal } \mu \le \kappa"$$
(Please note that the term large cardinal doesn't have a widespread definition in the literature and there are properties that people (including me) would regard as large cardinal properties but don't satisfy the definition above.)
Using my definition and assuming that $\operatorname{ZFC}$ is consistent, we cannot prove that there is a large cardinal property $\phi$ and a cardinal $\kappa$ such that $\phi(\kappa)$ holds. In fact, the assumption that any large cardinal exists is strictly stronger than the assumption that $\operatorname{ZFC}$ has a model and it seems reasonable to believe that large cardinals have to lead to contradictions. In the past there have been some examples of large cardinal properties that turned out to be inconsistent (see for example Reinhardt cardinals).
However, over the last 40+ years, people intensely studied large cardinals and where able to derive surprising and beautiful theorems from their existence. (For example, the existence of $n$ Woodin cardinals and a measurable above all of them implies that $\bf{\Pi}^{1}_{n+1}$-determinacy holds, for any given $n < \omega$. And conversely, for any given $n < \omega$, $\bf{\Pi}_{n+1}^{1}$ implies the existence of an inner model with $n$ Woodin cardinals.
Since we cannot prove that a given large cardinal assumption is consistent, relative to $\operatorname{ZFC}$, we have to rely on evidence, rather than proof. One way to 'strengthen' this approach is, for a given large cardinal property $\phi$, to study larger large cardinals $\lambda$ such that the existence of $\lambda$ implies the existence of many $\kappa < \lambda$ such that $\phi(\kappa)$ holds. If we can't prove, in this setting, that the existence of $\lambda$ is inconsistent, this is stronger evidence for the consistency of the existence of $\kappa$ such that $\phi(\kappa)$ holds, than just the fact that we weren't able to prove the inconsistency of such $\kappa$. Obviously, this isn't rigorous in the sense that we can be sure that a certain large cardinal assumption is consistent, but the same holds true for $\operatorname{ZFC}$ itself. By pushing its boundaries one merely gathers evidence and - over time - gains some reasonable confidence in its consistency.
A: In mathematics, "evidence" is weaker than "proof".
Mathematicians use the words "proof" and "evidence" differently from the sciences. When we speak of a "proof" that something is true, we mean an irrefutable line of logical implications. When we speak of "evidence" that something is true, we typically mean indications (like many worked out examples or related theorems). 
For example, as far as I know, no one has a "proof" that the Riemann Hypothesis is true. However, there is a ton of evidence that it probably is true (examples: there are infinitely many zeros on the critical line, all of the zeros must lie at least "close" to it, many of the theorems that it implies has been proven true). Again even with all of this evidence, this still does not give us "proof" that it is true.
So I guess in a kind of perverse way you could think of a "proof" as "evidence". But "evidence" is not enough to be a "proof".
Yes, science sees these terms as equals. Mathematics holds us to a higher standard. Unlike scientific results, mathematics results are true (not approximately sort of true for today).
A: Proposition: Every real number is less than $1,000,000$.
Evidence: The proposition seems to hold for all numbers in my test set $\{1,2,3,\ldots,100\}$.
