# Derivative of a product of linear combinations

let $$f_i(e_i) = \sum_j^N a_{ij}x_j$$ be a linear combination of N variables $$x_1 \dots x_n$$.

Let $$f(x_1, \cdots, x_n) = \prod_i^N f_i(e_i)$$ be a product of linear combinations.

To derive $$f$$ I used the following steps:

$$\prod_i^N f_i(e_i) = exp(log(\prod_i^N f_i(e_i))) = exp(\sum_i^N log(f_i(e_i)))$$

$$\frac{\partial}{\partial x_1 \cdots \partial x_n} f(x_1, \cdots, x_n) = \left( \frac{\partial}{\partial x_1 \cdots \partial x_n} \sum_i^N log(f_i(e_i))) \right) \cdot exp(\sum_i^N log(f_i(e_i))) \\= \left(\sum_i^N \frac{\frac{\partial}{\partial x_1 \cdots \partial x_n}f_i(e_i)}{f_i(e_i)}\right) \cdot \prod_i^N f_i(e_i)$$

As far as I know these steps are all legal and are similary found in derivatives for linear models. The usecase is, that I have a cumulative desnity functions over linearcombinations of $$x_1, \dots, x_n$$ and i want to get from the integral form of the product to the density. Sadly, when i put in a very simple linear combination and a uniform distribution I dont get what I expected.

Let $$e_1 = x_1 = \mathcal{U}(0,1); e_2 = x_2 = \mathcal{U}(0,1)$$, then

$$f(x_1, x_2) = \prod_i^2 F_i(e_i) = \left\{ \begin{array}{lr} x_1 \cdot x_2 & \text{if } x_1, x_2 \in [0, 1]\\ 0 & \text{else} \end{array} \right\}\\$$

and the derivative

$$f'(x_1, x_2) = (\frac{1}{x_1} + \frac{1}{x_2}) \cdot x_1 \cdot x_2$$.

But for a 2D uniform distribution over the unit space I would expect a density of 1. Where am I wrong?

You made a mistake at this point : $$\frac{\partial^n \ln f_i}{\partial x_1 \ldots \partial x_n} \neq \frac{1}{f_i}\frac{\partial^n f_i}{\partial x_1 \ldots \partial x_n} \verb+ +\mathrm{when}\verb+ + n\ge2$$ Indeed, let's take a counter-example with $$f_i(x_1,x_2) = x_1+x_2$$; one has then $$\frac{\partial^2 \ln f_i}{\partial x_1 \partial x_2} = \frac{\partial^2}{\partial x_1 \partial x_2}\ln(x_1+x_2) = \frac{\partial}{\partial x_1}\frac{1}{x_1+x_2} = -\frac{1}{(x_1+x_2)^2}$$ while $$\frac{1}{f_i}\frac{\partial^2 f_i}{\partial x_1 \partial x_2} = \frac{1}{x_1+x_2}\frac{\partial^2}{\partial x_1 \partial x_2}(x_1+x_2) = 0$$
Moreover, you seem to interpret $$f$$ as the density function of a multidimensional uniform distribution; in that case, beware of the fact that its derivative should be taken as a distributional derivative. In consequence, in your example with $$f(x_1,x_2) = x_1x_2 \chi_{[0,1]}(x_1)\chi_{[0,1]}(x_2)$$, where $$\chi_{[0,1]}$$ is the characteristic map over the given interval, we have : $$\nabla f(x_1,x_2) = \begin{pmatrix} \partial f/\partial x_1 \\ \partial f/\partial x_2 \end{pmatrix}$$ $$-$$ because $$f$$ has several variables $$-$$, with $$\frac{\partial f}{\partial x_1} = x_2 \chi_{[0,1]}(x_1)\chi_{[0,1]}(x_2) + x_1x_2 \chi_{[0,1]}'(x_1)\chi_{[0,1]}(x_2)$$ where $$\chi_{[0,1]}'(x_1) = (H(x_1) - H(x_1-1))' = \delta(x_1)-\delta(x_1-1),$$ $$H$$ being the Heaviside function and $$\delta = H'$$ the Dirac delta function, hence $$\begin{array}{rcl} \displaystyle \frac{\partial f}{\partial x_1} &=& \displaystyle x_2 \chi_{[0,1]}(x_1)\chi_{[0,1]}(x_2) + x_1x_2(\delta(x_1)-\delta(x_1-1))\chi_{[0,1]}(x_2) \\ &\equiv& \displaystyle (1-\delta(x_1-1))x_2\chi_{[0,1]}(x_2) \end{array}$$ and similarly for $$\frac{\partial f}{\partial x_2}$$.
• @TomSchierenbeck It really depends on the considered case. Products of random variables generally behave quite differently from the individual variables (e.g. a squared normal law follows a $\chi^2$ distribution). Linear combinations can be handled with the help of characteristic functions (cf. en.wikipedia.org/wiki/…); unfortunately, uniform distributions are not stable, which means that their linear combinations are not uniform anymore. Dec 31, 2022 at 13:49
• @TomSchierenbeck I'm a bit confused : if you want to study the product of random variables, such as $Z = XY$, then it cannot be done as you wish, but in the case where you just want to "aggregate" your data through multiplication of the density functions, then yes, your model will assume that the variables are independent. I guess that you are in the second case, thus your method is valid, since the ICA algorithm has decorrelated your data. Dec 31, 2022 at 15:05