Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

We are learning matrix calculus in class. This is really new for me and I am trying to get extra practice by going through old assignments. One of the questions asks to find the derivative of the function below with respect to $\beta$ and if possible to find the value of the elements of $\beta$ that maximize the function. After taking logs to simplify the original function, I took the partial with respect to $\beta$. I thought my derivative was correct, but when I set the partial equal to zero, I am unable to solve for $\beta$. So either my derivative is wrong or I don't know enough to solve for $\beta$, or it's a trick question and there is no analytical expression for $\beta$. A more advanced student suggested that it can probably only be solved by numerical methods but I was wondering if anyone here can tell me if my derivative is correct and whether or not it's possible to solve for $\beta$?

$$ x_{1} = \begin{pmatrix} a\\ b\\ \end{pmatrix}\quad x_{2} = \begin{pmatrix} c\\ d\\ \end{pmatrix}\quad \beta = \begin{pmatrix} \beta_{1}\\ \beta_{2}\\ \end{pmatrix}\quad Y = \begin{pmatrix} y_{1}\\ y_{2}\\ \end{pmatrix}\quad $$

$$ f(x_{1},x_{2},y_{1},y_{2},\beta) = \frac{1}{x_{1}'\beta}\exp\left\{\frac{-y_{1}}{x_{1}'\beta}\right\}\frac{1}{x_{2}'\beta}\exp\left\{\frac{-y_{2}}{x_{2}'\beta}\right\} $$

My answer:

$$ \frac{\partial \ln f}{\partial\beta}=-\frac{1}{x_{1}'\beta}x_{1}-\frac{1}{x_{2}'\beta}x_{2}+\frac{x_{1}y_{1}}{x_{1}'\beta.x_{1}'\beta}+\frac{x_{2}y_{2}}{x_{2}'\beta.x_{2}'\beta}=0 $$

Is this correct and is it possible to solve for $\beta$? If so, how? Thanks!

share|improve this question
    
It seems that in the last equation you've already divided out common factors, so this isn't in fact $\partial f/\partial\beta$? –  joriki Jan 17 '12 at 20:40
    
Well I'm not sure. That's why I'm asking for help. Taking logs of the original function eliminates the exp's - I'm under the impression this is OK since it's just a monotone transformation and so the Beta that maximizes the original function or its log will be the same. I just took the partial from there and got what I posted. I didn't try fiddling with it or try to cancel things out until I tried solving for Beta. So maybe my derivative is wrong? –  Sean Jan 17 '12 at 20:59
    
I'm not sure I understand correctly. Are you saying that what you wrote as "$\partial f/\partial\beta$" is in fact your result for $\partial\log f/\partial\beta$? –  joriki Jan 17 '12 at 21:08
    
Oh sorry, it was my fault I didn't indicate that I was taking the derivative of the transformed function as you pointed out. I made a correction in my original post. Thank you. –  Sean Jan 17 '12 at 21:16
    
Two more small mistakes: You've got $y_1$ twice in $f$ where the second one should presumably be $y_2$, and the last two terms in the derivative have the wrong sign. –  joriki Jan 17 '12 at 21:39
show 4 more comments

1 Answer

This function doesn't have a global maximum, at least not if $x_1$ and $x_2$ are linearly independent: You can choose $x_1'\beta$ such that the first exponential goes to $\infty$, and you can independently choose the sign of $x_2'\beta$ such that the product is positive and thus goes to $\infty$.

However, you can still look for the stationary points and see whether they are local maxima. Taking the logarithm before taking the derivative is a good approach, and the resulting equation can be solved with nothing but linear algebra.

If $x_1$ and $x_2$ are linearly independent, their coefficients in this equation have to vanish individually. That gives you one linear equation for $x_1'\beta$ and one for $x_2'\beta$. You can solve these, and then knowing the scalar products of $\beta$ with two linearly independent vectors, you can reconstruct $\beta$.

If $x_1$ and $x_2$ aren't linearly independent, you can substitute $x_2=\lambda x_1$ into the equation, and setting the coefficient of $x_1$ in the result to $0$ gives you a linear equation for $x_1'\beta$. In this case the stationarity condition only restricts $\beta$ in one direction, since the function is independent of the component of $\beta$ orthogonal to $x_1$ and $x_2$.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.