3
$\begingroup$

Consider the calculation of the (uncorrected) sample covariance as an example. Given a set of $N$ vector observations $\mathbf { X } = \{ \mathbf { x } _ 1, \dots, \mathbf { x } _ N \}$, the sample covariance is calculated as follows:

  • Step 1: $ ~~~~~~~ \displaystyle \boldsymbol { \mu } = \frac{1}{N} \sum _ {n = 1} ^ N \mathbf { x } _ n $

  • Step 2. $ ~~~~~~~ \displaystyle \boldsymbol { \Sigma } = \frac{1}{N} \sum _ {n = 1} ^ N ~( \mathbf { x } _ n - \boldsymbol { \mu } ) ( \mathbf { x } _ n - \boldsymbol { \mu } )^ { \mathrm T } $

There exists in many scientific programming languages the idea of "broadcasting", i.e. expanding singleton dimensions of a matrix or array, to perform elementwise operations with another array of higher dimensionality.

Let's notate such a "broadcasted subtraction" operation between a matrix and a vector as "$\circleddash$", such that: $$ \mathbf { X } \circleddash \boldsymbol { \mu } ~~\equiv~~ \Bigl (~ \mathbf { x } _ 1 \! - \! \boldsymbol { \mu } ~~~~~~ \mathbf { x } _ 2 \! - \! \boldsymbol { \mu } ~~~~~~\dots ~~~~~~ \mathbf { x } _ n \! - \! \boldsymbol { \mu } ~ \Bigr ) $$

With this notation, we could then rewrite step 2 above simply as:

  • Step 2. $ ~~~~~~~ \displaystyle \boldsymbol { \Sigma } = \frac{1}{N} ~( \mathbf { X } \circleddash \boldsymbol { \mu } ) ( \mathbf { X } \circleddash \boldsymbol { \mu } )^ { \mathrm T } $


Does such notation for broadcasting exist? Or would people simply:

  • use the normal subtraction "$-$" symbol and infer broadcasting from context
  • rely on collecting into intermediate variables via some sort of "generator syntax" to bypass the need for such an operator altogether, e.g. $$ \mathbf { D } \equiv \Bigl ( \mathbf { d }_1 ~~ \mathbf { d }_2 ~~ \dots ~~ \mathbf { d }_N \Bigr ) ~~:~~ \mathbf { d } _ n = \mathbf { x } _ n - \boldsymbol { \mu }, ~~~n \in \{1, \dots, N\} $$ (and therefore $\boldsymbol { \Sigma } = \frac{1}{N} D D ^ { \mathrm T } $ )
  • define their own broadcasted operators as (I've done above)

etc?

$\endgroup$
2
$\begingroup$

I would avoid introducing extra symbols, and simply write something like:

Slightly abusing notation, let $\mathbf{X} - \boldsymbol{\mu}$ denote the matrix $(\mathbf{x}_1 - \boldsymbol{\mu}, \dotsm, \mathbf{x}_n - \boldsymbol{\mu})$. Then the covariance can be written as $\displaystyle \boldsymbol{\Sigma} = \frac{1}{N} ~(\mathbf{X} - \boldsymbol{\mu}) (\mathbf{X} - \boldsymbol{\mu})^{ \mathrm T}$.

$\endgroup$
  • $\begingroup$ +1 Nice. With regard to the main question, do I take it that there is no special notation denoting a broadcasting operation then as far as you're aware (either in general or for subtraction specifically)? $\endgroup$ – Tasos Papastylianou Jun 13 '17 at 12:10
  • $\begingroup$ I am not aware of a special notation for broadcasting operations in mathematics but such operators exist in some programming languages, like APL. $\endgroup$ – J.-E. Pin Jun 14 '17 at 9:11
  • $\begingroup$ Yes, indeed, that's my motivation for asking in the first place (I mention it in the question somewhere). In any case, thank you, I will accept your answer, it seems no explicit notation exists and your suggestion does indeed seem more straightforward than explicitly defining a new operator. $\endgroup$ – Tasos Papastylianou Jun 14 '17 at 14:50
  • $\begingroup$ May I just suggest a small correction for the sake of future readers though: I would replace the word "vector" with "matrix", as $ ( \mathbf { x } _1 - \boldsymbol { \mu }, \dots, \mathbf { x } _ n - \boldsymbol { \mu } ) $ is a horizontal arrangement of vertical vectors, i.e. a matrix. $\endgroup$ – Tasos Papastylianou Jun 14 '17 at 14:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.