let $f_i(e_i) = \sum_j^N a_{ij}x_j$ be a linear combination of N variables $x_1 \dots x_n$.

Let $f(x_1, \cdots, x_n) = \prod_i^N f_i(e_i)$ be a product of linear combinations.

To derive $f$ I used the following steps:

$\prod_i^N f_i(e_i) = exp(log(\prod_i^N f_i(e_i))) = exp(\sum_i^N log(f_i(e_i))) $

$\frac{\partial}{\partial x_1 \cdots \partial x_n} f(x_1, \cdots, x_n) = \left( \frac{\partial}{\partial x_1 \cdots \partial x_n} \sum_i^N log(f_i(e_i))) \right) \cdot exp(\sum_i^N log(f_i(e_i))) \\= \left(\sum_i^N \frac{\frac{\partial}{\partial x_1 \cdots \partial x_n}f_i(e_i)}{f_i(e_i)}\right) \cdot \prod_i^N f_i(e_i) $

As far as I know these steps are all legal and are similary found in derivatives for linear models. The usecase is, that I have a cumulative desnity functions over linearcombinations of $x_1, \dots, x_n$ and i want to get from the integral form of the product to the density. Sadly, when i put in a very simple linear combination and a uniform distribution I dont get what I expected.

Let $e_1 = x_1 = \mathcal{U}(0,1); e_2 = x_2 = \mathcal{U}(0,1)$, then

$f(x_1, x_2) = \prod_i^2 F_i(e_i) = \left\{ \begin{array}{lr} x_1 \cdot x_2 & \text{if } x_1, x_2 \in [0, 1]\\ 0 & \text{else} \end{array} \right\}\\$

and the derivative

$f'(x_1, x_2) = (\frac{1}{x_1} + \frac{1}{x_2}) \cdot x_1 \cdot x_2$.

But for a 2D uniform distribution over the unit space I would expect a density of 1. Where am I wrong?


1 Answer 1


You made a mistake at this point : $$ \frac{\partial^n \ln f_i}{\partial x_1 \ldots \partial x_n} \neq \frac{1}{f_i}\frac{\partial^n f_i}{\partial x_1 \ldots \partial x_n} \verb+ +\mathrm{when}\verb+ + n\ge2 $$ Indeed, let's take a counter-example with $f_i(x_1,x_2) = x_1+x_2$; one has then $$ \frac{\partial^2 \ln f_i}{\partial x_1 \partial x_2} = \frac{\partial^2}{\partial x_1 \partial x_2}\ln(x_1+x_2) = \frac{\partial}{\partial x_1}\frac{1}{x_1+x_2} = -\frac{1}{(x_1+x_2)^2} $$ while $$ \frac{1}{f_i}\frac{\partial^2 f_i}{\partial x_1 \partial x_2} = \frac{1}{x_1+x_2}\frac{\partial^2}{\partial x_1 \partial x_2}(x_1+x_2) = 0 $$

Moreover, you seem to interpret $f$ as the density function of a multidimensional uniform distribution; in that case, beware of the fact that its derivative should be taken as a distributional derivative. In consequence, in your example with $f(x_1,x_2) = x_1x_2 \chi_{[0,1]}(x_1)\chi_{[0,1]}(x_2)$, where $\chi_{[0,1]}$ is the characteristic map over the given interval, we have : $$ \nabla f(x_1,x_2) = \begin{pmatrix} \partial f/\partial x_1 \\ \partial f/\partial x_2 \end{pmatrix} $$ $-$ because $f$ has several variables $-$, with $$ \frac{\partial f}{\partial x_1} = x_2 \chi_{[0,1]}(x_1)\chi_{[0,1]}(x_2) + x_1x_2 \chi_{[0,1]}'(x_1)\chi_{[0,1]}(x_2) $$ where $$ \chi_{[0,1]}'(x_1) = (H(x_1) - H(x_1-1))' = \delta(x_1)-\delta(x_1-1), $$ $H$ being the Heaviside function and $\delta = H'$ the Dirac delta function, hence $$ \begin{array}{rcl} \displaystyle \frac{\partial f}{\partial x_1} &=& \displaystyle x_2 \chi_{[0,1]}(x_1)\chi_{[0,1]}(x_2) + x_1x_2(\delta(x_1)-\delta(x_1-1))\chi_{[0,1]}(x_2) \\ &\equiv& \displaystyle (1-\delta(x_1-1))x_2\chi_{[0,1]}(x_2) \end{array} $$ and similarly for $\frac{\partial f}{\partial x_2}$.

  • 1
    $\begingroup$ Thanks you for the quick answer! So is there no general way to derive a product of N linear combinations of N variables? You are right that I want to get from the CDF representation to the PDF representation. Is there perhaps a better way if i have densities fitted independently on linear combinations of the original dimensions? $\endgroup$ Dec 31, 2022 at 13:37
  • $\begingroup$ @TomSchierenbeck It really depends on the considered case. Products of random variables generally behave quite differently from the individual variables (e.g. a squared normal law follows a $\chi^2$ distribution). Linear combinations can be handled with the help of characteristic functions (cf. en.wikipedia.org/wiki/…); unfortunately, uniform distributions are not stable, which means that their linear combinations are not uniform anymore. $\endgroup$
    – Abezhiko
    Dec 31, 2022 at 13:49
  • $\begingroup$ Hm so the original use case was that I had a distribution that is linear dependent. Therefore I transform the data such that it is not dependent anymore via the ICA. On the new "transformed axis" I then fit the empirical CDFs and to get the probability I just want to assume independence of the transfromed axis. Hence the formula from the question. Do you have a resource where I can read about the mathematics behind it? $\endgroup$ Dec 31, 2022 at 14:03
  • $\begingroup$ @TomSchierenbeck I'm a bit confused : if you want to study the product of random variables, such as $Z = XY$, then it cannot be done as you wish, but in the case where you just want to "aggregate" your data through multiplication of the density functions, then yes, your model will assume that the variables are independent. I guess that you are in the second case, thus your method is valid, since the ICA algorithm has decorrelated your data. $\endgroup$
    – Abezhiko
    Dec 31, 2022 at 15:05
  • $\begingroup$ @TomSchierenbeck Unfortunately, I'm a theoretical physicist, that is why I have extensive ressources in data analysis, sorry. However, you cannot simplify the formula of the logarithm derivatives as you did in the first place. $\endgroup$
    – Abezhiko
    Dec 31, 2022 at 15:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .