Is it problem of Mathematica or my own? The following is a plot comparing Exp[Derivative[1,0][Zeta][0,x]+1/2Log[2 Pi]] and Gamma[x]:

In theory the blue and the red graphs should be identical, giving the Gamma function. But in reality we have the blue plot (the plot of Exp[Derivative[1,0][Zeta][0,x]+1/2Log[2 Pi]]) representing the absolute value of the Gamma function instead. 
The question is: is this my fault (bad formula); and if so, how then do I correct it? Or it is a Mathematica bug?
 A: To expand on the comments I made (and Anixx has confirmed that it works):
The problem, as noted in the Mathematica documentation for Zeta[] and HurwitzZeta[] is that those two functions are defined slightly differently:
$$\mathrm{Zeta}(s,a):=\sum_{j=0}^\infty\frac1{((j+a)^2)^{\frac{s}{2}}}$$
$$\zeta(s,a):=\sum_{j=0}^\infty\frac1{(j+a)^s}$$
Though the two expressions are equivalent for $\Re a > 0$, they behave rather differently in the left half-plane:
N[Zeta[1/3, -1/5], 20]
0.99113858338730120895

N[HurwitzZeta[1/3, -1/5], 20]
0.1361506100489527143 - 1.4808826096823642385 I

and this manifests itself when you use Zeta[] instead of HurwitzZeta[] for your gamma function expression in the left half-plane.
Not knowing how they implemented HurwitzZeta[] internally, however, (it seems to have only been implemented starting with version 7 of Mathematica), I cannot say what, exactly, are the differences in the behavior of those two functions in the left half of the complex plane.
As an aside, a similar issue arises with the Lerch transcendent $\Phi(z,s,a)$, the trivariate generalization of the zeta function and the related polylogarithms; Mathematica from version 7 onwards provides for two different functions, LerchPhi[] and HurwitzLerchPhi[] which also agree for $\Re a > 0$ but behave differently otherwise.
A: This seems to work:
dzeta[x_] := Derivative[1, 0][Zeta][y, x]
Plot[{Exp[dzeta[0, x] + 1/2 Log[2 Pi]] , Gamma[x]}, {x, -3, 3}]
The derivative of a function returns a function. I think that in your version, maybe Mathematica binds x to specific numerical values during the plot and then tries to take the derivative. The version above computes the derivative once and for all and then does the plot.
EDIT:
Oops, Annix is right. This is not the first time I have been undone by mistaking the graph color for two graphs instead of one. Anyway, I'm sorry for the false starts, but the following code properly displays both functions and the graphs coincide.
Clear[x, y, dzeta]
dzeta[y_, x_] := Derivative[1, 0][HurwitzZeta][y, x]
Plot[{Exp[dzeta[0, x] + 1/2 Log[2 Pi]], Gamma[x]}, {x, -3, 3}]
Changing "Zeta" to "HurwitzZeta" in your original expression also works:
Plot[{Exp[
   Derivative[1, 0][HurwitzZeta][0, x] + 1/2 Log[2 Pi]]}, {x, -3, 3}]
