0
$\begingroup$

I'm currently taking an online course where the professor has a habit of writing norms like this:

$||a^{[l](C)} - a^{[l](G)} ||^2$

Since I don't have a great amount of experience in math or the concepts of deep learning, I was often confused whether the 2 simply meant, in conjunction with the double bars, "apply the L2 norm to the terms within, i.e. square each of them and then sum the result" or if the 2 was, itself, a squaring of whatever the double brackets meant on their own.

So I googled for confirmation of the notation and it seems that the 2 is normally written as a subscript, not a superscript. For example you can see the notation in Wikipedia is written this way: https://en.wikipedia.org/wiki/Norm_(mathematics)#Notation

So is the superscript notation wrong? Or is this just one of those unfortunate cases where there is no standard?

$\endgroup$
  • 1
    $\begingroup$ So if the $2$ appears in the exponent on a quantity, it's meant as a square. As a subscript, it indicates that it is the $L^2$ norm most likely. However you will see both $L^2$ and $L_2$ in use. I prefer the former, but some prefer the latter. $\endgroup$ – Cameron Williams Mar 14 '18 at 1:32
3
$\begingroup$

Since I don't have a great amount of experience in math or the concepts of deep learning, I was often confused whether the 2 simply meant, in conjunction with the double bars, "apply the L2 norm to the terms within, i.e. square each of them and then sum the result" or if the 2 was, itself, a squaring of whatever the double brackets meant on their own.

I have never seen an author disambiguate the norm delimiters $\lVert\quad\rVert$ through the use of a superscript. In analysis, such notation would be incredibly confusing, since we frequently need to establish inequalities among norms of vectors raised to some power.

Also, an $L^2$ norm of a vector is the square root of the sum of the absolute squares of its components: $$\lVert x\rVert_2=\sqrt{\sum_{i=1}^n\lvert x_i\rvert^2}\text{;}$$ consequently, $$\lVert x\rVert^2_2=\sum_{i=1}^n\lvert x_i\rvert^2\text{.}$$

$\endgroup$
  • $\begingroup$ I too would like to assert that I have never seen that notation ever $\endgroup$ – Stella Biderman Mar 14 '18 at 2:08
  • $\begingroup$ Thanks, it is very helpful to have this cleared up. $\endgroup$ – Stephen Mar 14 '18 at 3:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.