Neil G
Reputation
781
Top tag
Next privilege 1,000 Rep.
Create tags
 Oct31 comment Computing Gradient and Hessian of a vector function @SeeBees: Take a look at Pearlmutter and Schraudolph's work on the R technique that approximates exactly that in linear time. Sep3 comment Intuitive explanation of a definition of the Fisher information Is there a derivative missing in your definition of the score? Apr10 comment Beta function derivation @EricAuld: Bayes rule is much easier to think about without the denominator, which is just a constant given your observations. So: $P(q\mid a) \propto P(a\mid q) P(q)$. (These terms are called posterior, likelihood, and prior respectively.) Then, you can fill it in: $\int_0^x f(q \mid a) dq \propto \int_0^x f(q)f(a \mid q)dq \implies f(q \mid a) \propto f(q)f(a \mid q)$. And, the likelihood $f(a \mid q)$ is just $q$ for heads and $1-q$ for tails. Feel free to ask a question on stats.stackexchange. There are a lot of very helpful people there. Apr9 comment Beta function derivation @EricAuld: Suppose I give you a coin whose bias $p$ you don't know. You flip the coin $n$ times and get heads $a$ times. Then the likelihood induced on $p$ is a Beta distribution. You can work out the Beta distribution up to normalization by writing out the product of the Bernoulli mass function $p^x(1-p)^{1-x}$ for each realization $x_i \in \{0,1\}$ (where $\sum_i x_i = a$ and $\sum_i 1-x_i = n-a$). Dec1 comment What is the purpose of the first test in an inductive proof? IH: All sets of $n \ge 4$ lines on the plane intersect at a single point. If it holds true for $n\ge 4$ points, then it holds true for $n+1$ points since the first $n\ge 3$ points intersect at a point and so too do the last $n$ lines and the two intersection points must be the same point. Therefore, all lines on the plane intersect at a single point. Aug14 comment Reasoning that $\sin2x=2 \sin x \cos x$ +1: This method also gives all of the double-angle (and triple-angle, etc.) formulas. Aug11 comment An interesting puzzle I think a rough sketch might be that in order to create any significant probability mass in $\vert X-Y \vert$ between 1 and 2, you have to also create mass between 0 and 1. So, $P{|X−Y|≤1}$ is no less than half the LHS? Apr15 comment Log likelihood of a realization of a Poisson process? Hi David, do you have a citation for the likelihood? I arrived at the same likelihood by reason directly from the entropy of a Poisson process given by McFadden. Mar21 comment Graphically, what is positive semidefinite-ness? Thanks, I think I see it now. Essentially, Newton's method is looking for points with zero slope, and decides for each eigenvector whether to go towards a local maximum or minimum based on the sign of the eigenvalue. Is that right? Mar21 comment Graphically, what is positive semidefinite-ness? Thanks for taking the time to answer. You're right that Newton's method will only give a local minimum (and will stop at a local maximum) since it's looking for a point where the gradient is zero. I am still having trouble visualizing the second part of your answer. Feb4 comment Probability problem of dice game Feb4 comment Probability problem of dice game @joriki: Yes, it's solving a system of linear equations. You could have written it as a dynamic program that memoizes the transition probabilities and the probability of return for every state, which would be more efficient for a sparse transition matrix. Feb3 comment Probability problem of dice game @joriki: I think you could turn all of the loops into self-loops first, and then you would have a traversal order. Anyway, I calculated the transition matrix. Perhaps you can finish it from there? Feb3 comment Probability problem of dice game @joriki: I was thinking of doing something like this: stats.stackexchange.com/questions/48396/… I see your point that you can get the transition matrix and decompose it, which might give an easier solution. I have some time now to give it a shot. Feb3 comment Probability problem of dice game It's not hard to use dynamic programming to find an exact solution. The state space is the last number rolled cross the run length (1 or 2). So, there are 22 states plus 2 finish states. The start space can be $(12, 1)$. What happens if they both simultaneously win? Feb2 comment Trace of a matrix times outer product Thanks for your answer so far. Is there an intuitive explanation of why someone would prefer to write the first expression? Does it reveal some mathematical structure? The second one seems clearer to me. Nov24 comment Convex function plus $v e^{-x}$ @coffeemath: I meant $w$, but I don't think it makes any difference? Nov24 comment Convex function plus $v e^{-x}$ Yes, thanks for the correction. Aug2 comment What are the practical uses of $e$? Interesting. Note that the "information contribution" is often called the "surprisal" or "self-information". Jul13 comment General nonatomic measure that cannot be expressed as an integral Thanks! (Now, I wonder if there are any real-world Poisson processes whose intensity varies as the cantor set ;) )