function inversion and the horizontal shift

I am currently doing inverse functions and graphing radical equations of the form $y=a\sqrt{x-h}+k$ with my algebra class and one of my students asked me the following question.

"Why is it that we shift left when $\sqrt{x+h}$ and into the negative x-values but we shift up when $\sqrt{x}+k$ and into the positive y-values?"

I explained to them that part of it can be seen as a result of the inversion process of some functions/relations. So I recalled the vertex form of a parabola, something they studied at the beginning of the year and I put it on the board in a specific function, as $$y=x^2+3$$. Then I graphed it and showed them that it was a parabola shifted up three units. Then I inverted it by switching $y$ and $x$ and solving for $y$. So $$x=y^2+3 \Rightarrow x-3=y^2 \Rightarrow y=\pm\sqrt{x-3}$$ Now this is the inverse of our original function (although in its current form it is NOT a function) and notice since they are inverses they are symmetrical along the $y=x$ line. Now look at the graphs and notice that even though our argument under the radical is $(x-3)$ we shift right into the positive reals.

I don't feel like this is reason enough to justify the why. I'm also not sure that I satisfied the asker. They saw both functions, they saw the symmetry on the $y=x$ line, so after learning about inverses, I think they sort of understood the relationship, but hopefullly someone can provide me with a bit of better insight as to why. Please remember this question is ultimately for me and I do understand math well. If I understand, then I can better teach.

EDIT: I'm specifically looking for teachers who have taught this material before who have maybe had to explain this.

I explain the "horizontal shift" this way: when we graph $\ y = f(x+h) \ ,$ we are composing $\ f(x) \$ on the function $\ x + h \ .$ This is to say that we are first adding $\ h \$ to $\ x \ ,$ evaluating the function $\ f \$ at $\ x + h \ ,$ and then plotting the result at $\ x \ .$ This has the effect of reading off values of the function that lie to the right of $\ x \$ and moving them back to the left by $\ h \$ units. Similarly, $\ f(x - h ) \$ moves the graph of $\ f(x) \$ to the right by $\ h \$ units, as we are reading off values of the function that lie to the left of $\ x \ ,$ in order to plot them at $\ x \ .$
Horizontal shifts are definitely the more puzzling of the two. I rarely see anyone have difficulties understanding what adding or subtracting a value directly to $\ f(x) \$ is going to do to the graph. I also don't find anything "wrong" with your explanation in terms of inverse functions (a sort of "diagonal mirror" argument), but I think a student would typically need more experience with graphing functions (like so many things, a skill even university students don't have nearly enough practice with) to appreciate your description.