What does Timothy Gowers mean by 'semantics' and 'syntax' in math? Source: How should logarithms be taught? All bolds are mine. 

Those who criticized this view tended to think that I was advocating pure rote learning rather than understanding. Actually, I was suggesting that a true understanding of a sophisticated concept such as the exponential function involves letting go of the intuitive meaning (once it has served its purpose of telling you the rules you want the function to satisfy) and using the defining properties instead.
Behind that suggestion is a more general claim, which is that mathematicians greatly underestimate the extent to which they think syntactically rather than semantically. [...]

Do the Linguistic definitions below relate to these terms' meanings in math?  
Source: 
An Introduction to Language (10 ed., 2013. But $\exists$ 11 ed.)

[p 578:] semantics 
  The study of the linguistic meanings of morphemes, words, phrases, and
  sentences.
[p 582:] syntax 
  The rules of sentence formation; the component of the mental grammar that
  represents speakers’ knowledge of the structure of phrases and sentences.

 A: The linguistic meaning of syntax and semantics roughly corresponds to the mathematical understanding.
Syntax is the symbol-shuffling operations in mathematics. This includes substitution. In the context of logarithms, there's the rule that $a^y = x \implies y = \log_a(x)$. If we use this rule by substituting different numbers into roles of $a, x, y$, we're thinking of logarithms syntactically.
Semantics is the underlying meaning of the equations. This includes object representations. So, when we're thinking about logarithms, we think about the exponentiation number. Like, how $\log_{10}(100) = 2$, which means we exponentiated $10$ two times to get to $100$. The point being, rather than defining a logarithm as an inverse, or using rules, we consider it meaning-first.
These words have much more specific definitions in mathematical logic, where the objects studied are truth values rather than numbers. Syntax is like proof rules, and semantics is the actual truth value, interpreted under a model.
