Taking Dr. Andrew Ng's Machine Learning course on Coursera and I'm trying to come to grips with the regularized logistic regression formula in the course. Notation is not my strong suit. Formula in question:

$J(\theta) = 1/m \sum_{i=i}^m[y^{(i)}log(h_{\theta}(x^{(i)}) + (1 - y^{(i)}) log(1 - h_{\theta}(x^{(i)}))] + \lambda/2m\sum_{j=1}^n\theta_j^2$

$\theta, y, x$ are all column vectors as far as I know. So it is strange to me when I see $x^{(i)}$ which I would assume to be referencing a column as opposed to $\theta_j$ which references a row in the column vector. If $x$ is a column vector there will only be one. It would make more sense to me if it were $X^{(i)}$.

I'm even more confused now that we've started talking about deep learning where he references items in a particular row and column i.e. $x_{12}$ which would refer to the item in the first row second column. Why the need then for $x^{(i)}$?

I'm also sure I my assumptions could be 100% incorrect.

| cite | improve this question | | | | |
  • $\begingroup$ Is $x^{(i)}$ a column vector ? $\endgroup$ – Smilia Jan 4 at 20:58
  • $\begingroup$ I understand $x^{(i)}$ as the $i^{th}$ iterate of $x$ (or the $i^{th}$ derivative, but this is highly unlikely). Not as a component of a row or column (could be $x^i$ in tensor notation, but $x_i$ otherwise). $\endgroup$ – Yves Daoust Jan 4 at 21:00
  • $\begingroup$ @Smilia $x^{(i)}$ is indeed a column vector. Or at least I'm assuming it is very highly likely to be so given previous context. $\endgroup$ – Breedly Jan 5 at 0:42

OK, so reading more material Dr. Ng presents another formula like so:

$$ X = \begin{bmatrix} x^{(1)T} \\ x^{(2)T} \\ ... \\ x^{(n)T}\\ \end{bmatrix} $$

Which confirms my suspicions that its the "ith" instance in the training set. If anyone else wants to provide a less crude explanation I will take that over mine :). I think this is what Yves Daoust was telling me, but I'm not entirely sure.

$x$ is a column vector, $X$ is another column vector of $x^T$, and $x^{(i)}$ is the "ith" $x$ in $X$.

| cite | improve this answer | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.