If we want to multiply a weight to a product of unequal matrices, may someone explain why we have to write the notation in all three dimensions? For example, I read the example in Jensen's inequality weight ($\pi_{ijk}$) of NMF. Despite it sums over $k$, the $\pi_{ijk}$ has all $ijk$ notation. Is this important?


To apply Jensen's inequality, we introduce $\sum_k \pi_{ijk} = 1$ (on product $W_{ik}H_{kj}$)

\begin{equation*} d_{\mathrm{KL}} (\mathbf{V}\ \vert\vert \mathbf{WH}) = \sum_{ij}-V_{ij}\log\sum_{k}\pi_{ijk}\frac{W_{ik}H_{kj}}{\pi_{ijk}} + \sum_{ij}\sum_{k}{W_{ik}H_{kj}} \end{equation*}

  • $\begingroup$ Well, it lets you weigh different pairs of elements contributing to $(WH)_{ij}$ differently. Is that useful? Maybe, depends what you're doing. $\endgroup$
    – Ian
    Aug 17 '17 at 12:50
  • $\begingroup$ @Ian You mean every $(WH)_{ij}$ has its $\pi_{ij}$? So why $\sum_{k}$ is needed? The first $\sum_{ij}$ seems to pair the weight and product of $WH$, what is the role of $k$ in $\sum_{k} \pi_{ijk}$? $\endgroup$
    – Jan
    Aug 17 '17 at 13:43
  • 1
    $\begingroup$ I mean that if the weight $\pi$ cannot depend on k then every $(WH)_{ij}$ has just one weight. $\endgroup$
    – Ian
    Aug 17 '17 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.