# Questions tagged [computer-arithmetic]

For questions concerning finite precision arithmetic in computers and other related concepts.

112 questions
Filter by
Sorted by
Tagged with
181 views

1 vote
57 views

### How to avoid overflows and numerical issues when computing derivatives?

I'm trying to compute the derivative of this quantity w.r.t to $\mu$ and $k$: $$\mathcal{L}(t) = - \eta_n\log\left(1-P(t)\right)$$ Where $\eta_n$ is just a weighting factor and P(t) is the cdf of ...
1 vote
83 views

• 723
1 vote
40 views

### Is there a way to quantify how well I solved a matrix? (Gauss-Jordan elimination efficiency)

I'm just starting to learn linear algebra and I'm finishing up the section of Gauss-Jordan elimination. Something that piqued my interest was the variety of different ways I could go about solving ...
• 177
210 views

### Proof of a different (Russian peasant) method of multiplication [duplicate]

I got this problem in the book Topics in the Theory of Numbers by Paul Erdos and Janos Suranyi and I found it very interesting. I have no idea how to prove this. (Although I believe it can be proved ...
• 1,318
105 views

### Can one do Conjugate Gradient for non-floating point arithmetics?

So the last few years I have used Krylov subspace methods (mostly Conjugate Gradient) for solving various kinds of problems in science and engineering, but in all of these applications I have only ...
• 26k
106 views

### Fast computation of the direction of a vector

I need to compute the direction of 2D vectors that are provided as a pair of 16 bit signed integers, and return an integer in range $[0,256)$. This can be achieved by the expression \frac{128}\pi\...