The answer is yes in both cases. If B is an elementary extension of A, then everything that is expressible using $L$ and all the elements of A must also hold in B, and vice versa. The important part is "... and all the elements of A". It's not sufficient for all closed formulas $\phi$ to hold in A exactly if they hold in $B$! You also need that for any $\phi$ with free variables $x_1,\ldots,x_n$ and all tuples $a_1,\ldots,a_n \in A$, $$
A \models \phi(a_1,\ldots,a_n) \text{ iff } B \models \phi(a_1,\ldots,a_n)
$$
Thus, additional constants don't really change anything - you can just replace the constant with a free variable $x_{n+1}$ to get a formula in $L$ instead of $L(\bar{c})$. You then know that for all possible $(n+1)$-tuples that new formula holds in $A$ exactly if it holds in $B$, which in particular includes those tuples with $a_{n+1} = \bar{c}$.