Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rounding-error]

The tag has no usage guidance.

1
vote
1answer
70 views

What's the minimum step size that can be used in Euler's method before it becomes unreliable?

In particular, if Euler's method is implemented on a computer, what's the minimum step size that can be used before rounding errors cause the Euler approximations to become completely unreliable? I ...
2
votes
0answers
26 views

How does rounding affect subsequent calculations? [duplicate]

When we are doing calculations in mathematics, we often express exact values, like $\sqrt 2$ or $\arctan (1)$, as decimals and round these to a finite number of decimal places/significant figures ...
6
votes
3answers
115 views

How to prove that expessions like $\sqrt{93+63\sqrt{85}} - \sqrt{143} \notin \Bbb{Z}$?

The Problem: There are multiple "rooty" equations that can be simplified to a whole number, for example: $$\sqrt{19 + 6\sqrt{2}} - \sqrt{18} = 1$$ Because: $$\sqrt{19 + 6\sqrt{2}} - \sqrt{18} = \...
0
votes
0answers
20 views

Thresholding to minimize round-off errors (hubbard)

I'm working through Hubbard's Vector Calculus text, and I'm a little fuzzy concerning a detail of one example: He later asks you to analyze where the errors occur in formula 2.1.12, and I see that $$ ...
1
vote
0answers
42 views

Is the function $\sinh(x)/x$ fractal at small values of $x,y$ or am I seeing rounding errors in computation?

I asked Wolfram Alpha to give me a solution to an integral function https://www.wolframalpha.com/input/?i=(integral+exp(-mx)+dx+between+x-a+and+x%2Ba+)%2F(2+a+exp(-mx)) and it gave me an expression ...
0
votes
1answer
19 views

Rounding to nearest integer

If you round to the nearest integer, why do we look at the number it was before instead of the number which it is at that moment? For example: 17.495 5 or higher goes up which makes it 17.50 which ...
1
vote
0answers
19 views

How to determine rounding precision of intermediate value to avoid loss of precision?

I have a formula that can be represented as A * B where A = 1 / x and B is a fixed value ...
0
votes
0answers
13 views

How to calculate $\sum_{i=0}^N\lfloor ai\rfloor^2$ for $0<a<1$ and $N\approx10^{16}$?

I have an irrational number $0<a<1$ for which I need to calculate the sum $$\sum_{i=0}^N\lfloor ai\rfloor^2\mbox{ mod }m.$$ Here $m$ is an $8$-digit number and $N$ is a $15$-digit number. The ...
0
votes
2answers
39 views

Why is the error of $O(h^2)$ when using Taylor expansion and centered approximation for the first derivative

We know that the approximation of the first derivative by centered approximation is given by $$ f'(x) = \frac{f(t+h) - f(x-h)}{2h} + O(h^2)$$ The quality of the above approximation is determined by ...
0
votes
1answer
41 views

Rounding in physics Vs Rounding in math [closed]

Candice is toppling dominoes. In order for one of these dominos to topple the next one, the next domino must be exactly 1.5 times the height of the previous one. She measures the first domino and ...
0
votes
1answer
32 views

Representing rounding algebraically [closed]

Is there a standard way to deal with rounding in algebra? For example: y = x + round(x/2) Would give 2 when x = [1, 3), 3 when x = [3, 5), etc. This of course ...
0
votes
1answer
42 views

Finding error in a an approximation

We want to see the total error in approximating $$ f'(x) \approx \frac{ f(x+h)-f(x) }{h} $$ where $f: R \to R$ is differentiable. We can find $\theta \in [x,x+h]$ by Taylor's to that ...
0
votes
1answer
20 views

Round off to decimals

I m not sure about this problem. Pl help. 1. Roundoff this number to tenths place 87.952 2. Round off this number to hundredths place 75.195 As per me answer should be 88.0 and 75.2 for ...
0
votes
1answer
41 views

How to avoid cumulative rounding errors when calculating a result to a specific number of significant figures?

I have two operands and I want to calculate the result of an arithmetic operation (add, sub, mul, div, pow, sqrt, ln...) to S significant figures. How many ...
1
vote
1answer
58 views

How to generalize upper and lower bounds rules

I want to create a rule on finding the upper and lower bound of a number. Examples: For $29.8$ (3 s.f.) we write $1$ in the place of the $8$ and zero in every other position, giving us $0.01$. ...
0
votes
0answers
11 views

How to turn line around point of cross with other line?

my question seems to be very easy, but I can't figure it out. Maybe someone could help. I know rounding the line around the point $(P_x,P_y)$ is very easy, just like that: $f(x) = R(x-P_x)+P_y$ ...
2
votes
1answer
28 views

How to tell when a fraction does not end? [duplicate]

Is there a way in math / programming to tell if a fraction (reciprocal in particular) does not end? For example, 1/3 is 0.33 repeated, but 1/2 is just 0.5 Is there a way to find if 1/n for any ...
0
votes
1answer
11 views

How to round the mean number of people

When you are calculating the average of the number of people who do something, and you get a value with decimal points, should I still round up when the decimal point is > or = 5, and round down when ...
0
votes
0answers
53 views

Which is the correct one?

Here is an excerpt from the book 'Applied Numerical Linear Algebra' by James W. Demmel from SIAM But I have done it slightly different taking $\hat{l}_{ij}$ and $\hat{u}_{ij}$ and ended up getting $|{...
0
votes
2answers
124 views

A dollar amount $\${-}67.9{-}$ is divisible by 72. What is this number?

A client buys 72 turkeys. On the receipt, 2 digits are missing, the first and the last. So, the total price is $\${-}67.9{-}$. Find a way to make the price of each turkey round at the second digit ...
0
votes
0answers
60 views

Chebyshev polynomial: recursive formula error estimate

I am trying to solve Problem 3 & 4 from Numerical methods of Bakhvalov, Zhidkov, Kobelkov from section 2.8 on Chebyshev polynomials. If in the recursive formula $$ T_n(x) = 2xT_{n-1}(x)-T_{n-2}(x)...
0
votes
1answer
23 views

Multiplication after rounding, missing information of order of 1000, how to find if multiplication had multiple values for each data couple?

Imagine a financial report in a certain currency, indicating the following numbers (and below them, indicating the amount after conversion to USD, from the same report). 1,394,278 (USD46,743) ...
0
votes
1answer
77 views

How to compute rounding error bound?

I am using the IEEE754 half-precision floating point format, which has 11 significand bits. My input is drawn randomly with values between 1.0 and 2.0. I would like to approximate the maximum ...
2
votes
1answer
61 views

Should I use powers of $2$ when possible in computations?

For example, if I want the most accuracy and efficiency when performing millions of iterations wouldn't it be better to use $2^k$ -- as opposed to some other nearby number -- whenever possible, since ...
1
vote
0answers
131 views

Inverse function of round; solving equations involving round function

I am trying to solve the equation with which I came up from the accounting task: Round((0.844+delta)*1; 2)=0.89 or generally: ...
2
votes
1answer
46 views

What's the flaw in “collapsed rounding”?

A couple of years ago, I asked my high school math teacher the following question and she couldn't give me an answer. I had since forgotten about it, but now I'm curious to find the answer again. Let'...
0
votes
1answer
17 views

Interpolation with almost constant slope E

I have a curve $(\sigma_t,\varepsilon_t)$ described parametrically: Data here for (implicit) $t_i=(0,1,...n)$ I have two data series $\sigma_{t_i}=\sigma_1, \sigma_2, ... \sigma_n$ $\varepsilon_{...
0
votes
0answers
21 views

What are methods for correction of experimental measurements?

I have experimental data series collected from sensors, and I have some problems because the data is obviously digitally distorted. I don't know if it is a problem of rounding, or quantization, or ...
0
votes
2answers
638 views

rounding up to two decimal points

I just have a question regarding my answers in my exam paper, my teacher said to give the answer to two decimal points, I got the correct answer (2.89) but I rounded it up to 2.90. Now my teacher did ...
3
votes
0answers
114 views

Error bound for floating-point interval dot product

In Handbook of Floating-Point Arithmetic (Birkhäuser, 2010, Chapter 6) Muller et al. presented the following absolute forward error bound for the floating-point recursive dot product: $$ \left|...
-2
votes
1answer
28 views

Maclaurin polynomial error bounds [closed]

I need some help with my Calculus II Maclaurin polynomial error bounds. $Mn(x)$ is the $n^{th}$ Maclaurin polynomial for $f(x) = e^x$. I need to use the error bound formula to determine a value of $...
1
vote
3answers
78 views

Find an alternative expression for $y :=cos(x+\delta)-cos(x)$

Explain the difficulty of computing $y :=cos(x+\delta)-cos(x)$ for small values of $\delta$. Find an alternative expression of $y$ that does not exhibit these difficulties. So far I have figured ...
1
vote
0answers
18 views

Find the number of significant figure in the approximate number $0.39865$ for the given relative error of it as $0.2\times 10^{-2}$.

Find the number of significant figure in the approximate number $0.39865$ for the given relative error of it as $0.2\times 10^{-2}$. Adding $0.2\times 10^{-2}$ to $0.39865$ gives $0.40065$ and it ...
1
vote
1answer
370 views

Chopping a number

Iv'e been trying to understand this really really simple concept of number chopping. Let's say that I have a system which is able to save decimal numbers with 3 significant figures and uses the ...
1
vote
0answers
52 views

How do I fight loss of significance and/or improve convergence for this recursive algorithm?

While trying to answer this question I used the series approach and obtained a recursive algorithm. While checking it numerically, I found it suffering from "catastrophic cancellation", i.e. loss of ...
0
votes
1answer
34 views

The error when we rounding down and sum in $\sum_i \lfloor n/i \rfloor$ vs. $\sum_i n/i$.

For some $x \in \mathbb R$ denote by $\lfloor x \rfloor := \sup\{ k \in \mathbb Z \mid k \le x \}$ the biggest integer smaller than $x$, the so called floor function. Then in my textbook it is ...
2
votes
1answer
25 views

Rounding fractions with less digits

The problem I'm trying to round a fraction $\frac{a}{b}$ taking away digits from one or both integers. Let's say a or b have 3 digits, and I want a new $\frac{a'}{b'}$ with as much as 2 digits for ...
0
votes
1answer
167 views

Rounding down values taken from an exponential distribution

Suppose that $Y_1,Y_2,...,Y_n$ are i.i.d. with distribution $exp(\lambda)$, and we define another set of random variables $Z_1,Z_2,...,Z_n$ where $Z_j=\delta \lfloor \frac{Y_j}{\delta} \rfloor$. I ...
0
votes
1answer
36 views

If $x$ is an approximate number, is $2x=x+x$ true?

Suppose I have a mass and I measured it. I found that it weighs $1.001 kg$. Since there may be error in my measurement process, the mass could be a few milligrams heavier or lighter. In grams, its ...
2
votes
2answers
74 views

0.5×2=1 but 0.5+0.5=1.0. Explain? [closed]

So when we multiply $0.5$ (approximate number) by $2$ (exact number), we get $1$, since our product must contain as many significant figures as $0.5$. When we add $0.5$ to $0.5$ (both approximate), ...
3
votes
3answers
59 views

How precise should an absolute error be?

While adding approximate numbers, our result must be as precise as the least precise number that was given to us. For instance, $101+1.001+1.0≈103$ because the number with the least precision viz., $...
0
votes
1answer
130 views

Recursive definition of the error

Given the definition of $I_n:=\frac{1}{e}\int_0^1 x^ne^x$ for $n=0,1,2,...$ there is a recurrence relation: $I_n=1-nI_{n-1} $ for $n=1,2,...$ and $I_0=\frac{e-1}{e}$ I've got to show ...
1
vote
1answer
625 views

Accuracy in multiplication; precision in addition

Why is it that while multiplying approximate numbers we are concerned with significant figures (accuracy) and while adding we are concerned with decimal places (precision)? Any explanation would be ...
1
vote
0answers
24 views

Worst case scenarios for companion matrix method

In what case is the companion matrix method for computing the roots of a polynomial expected to suffer accuracy loss. Are nearby roots the main issue?
2
votes
1answer
171 views

How to approximate relative error further?

Background and original problem: I want to minimize the difference (error) of a numerical derivative approximation of a function and its true derivative. Let $f(x) = \sin(x) $ be the anti-derivative ...
3
votes
2answers
313 views

How to reduce numerical error of computing $\ln(x) - \ln(y)$ when $x \approx y$?

$\newcommand{\Cond}{\operatorname{Cond}}$There will be a huge cancellation error computing $\ln(x)-\ln(y)$ for $x \approx y$. One method for solving this issue is rewriting $\ln(x)-\ln(y)$ as $\log(\...
1
vote
1answer
229 views

How do I calculate standard deviation and $\bar{x}$ if radius of circle is given?

It is given radius of circle $r = 5cm$ and standard deviation $\sigma = 1mm$. How do I calculate $\bar{x}$ and standard deviation of calculated circle area? I know these formulas but I dont know how ...
0
votes
1answer
45 views

Rounding(A+B) == Rounding(A) + Rounding(B)?

Could I say Rounding(A+B, n) == Rounding(A, n) + Rounding(B, n) ? For example: ...
0
votes
1answer
616 views

Computing Absolute and Relative Error Using Double Precision

Relevant definitions: If $\tilde a$ is an approximation of $a$, then the absolute error of $a$ is $$ |\tilde a - a| $$ and the relative error of $a$ is $$ \frac{| \tilde a - a|}{a} .$$ Statement of ...
0
votes
1answer
47 views

Reversibility of Integer Scaling

So this problem originates from thinking about scaling an pixel coordinate from one resolution to another and back again. Is that operation truly reversible, or is it possible for the pixel coordinate ...