I'm starting to attempt to learn how regularized multi-class logistic regression classifiers work, but I'm stuck at the very beginning. The negative log-likelihood function of logistic regression for $m$ classes and its gradients are given by:

enter image description here

If I have a feature matrix $\bf X$ that is size 800K x 50, and a $\bf Y$ that is size 800K x 1 with 30K unique values, what are the dimensions of $\bf W$, $\bf w_{k}$, $\bf x_{j}$, and what do $n$ and $m$ equal?

I thought that $m=50$, $n=800K$, $\bf W$ is also a 800K x 50 matrix, $\bf w_{k}$ is a column of $\bf W$ of size 800K x 1, and $\bf x_{j}$ is a row of $\bf X$ of size 50 x 1. However obviously I'm wrong because I can't take the dot product $\bf w_{k}^{T} \bf x_{j}$ if these vectors have unequal lengths. What part(s) am I misunderstanding?


$n=800K$ and $m = 300K$. (Recall $m$ is the number of classes.) So, $\mathbf{W}$ is $50 \times 300K$ with columns $\mathbf{w}_k$. Note that $k \in \{1,\ldots,m\}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.