Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rounding-error]

The tag has no usage guidance.

1
vote
0answers
36 views

Is the function $\sinh(x)/x$ fractal at small values of $x,y$ or am I seeing rounding errors in computation?

I asked Wolfram Alpha to give me a solution to an integral function https://www.wolframalpha.com/input/?i=(integral+exp(-mx)+dx+between+x-a+and+x%2Ba+)%2F(2+a+exp(-mx)) and it gave me an expression ...
0
votes
1answer
17 views

Rounding to nearest integer

If you round to the nearest integer, why do we look at the number it was before instead of the number which it is at that moment? For example: 17.495 5 or higher goes up which makes it 17.50 which ...
0
votes
0answers
16 views

How to determine rounding precision of intermediate value to avoid loss of precision?

I have a formula that can be represented as A * B where A = 1 / x and B is a fixed value ...
0
votes
0answers
12 views

How to calculate $\sum_{i=0}^N\lfloor ai\rfloor^2$ for $0<a<1$ and $N\approx10^{16}$?

I have an irrational number $0<a<1$ for which I need to calculate the sum $$\sum_{i=0}^N\lfloor ai\rfloor^2\mbox{ mod }m.$$ Here $m$ is an $8$-digit number and $N$ is a $15$-digit number. The ...
0
votes
2answers
39 views

Why is the error of $O(h^2)$ when using Taylor expansion and centered approximation for the first derivative

We know that the approximation of the first derivative by centered approximation is given by $$ f'(x) = \frac{f(t+h) - f(x-h)}{2h} + O(h^2)$$ The quality of the above approximation is determined by ...
0
votes
1answer
40 views

Rounding in physics Vs Rounding in math [closed]

Candice is toppling dominoes. In order for one of these dominos to topple the next one, the next domino must be exactly 1.5 times the height of the previous one. She measures the first domino and ...
0
votes
1answer
28 views

Representing rounding algebraically [closed]

Is there a standard way to deal with rounding in algebra? For example: y = x + round(x/2) Would give 2 when x = [1, 3), 3 when x = [3, 5), etc. This of course ...
0
votes
1answer
40 views

Finding error in a an approximation

We want to see the total error in approximating $$ f'(x) \approx \frac{ f(x+h)-f(x) }{h} $$ where $f: R \to R$ is differentiable. We can find $\theta \in [x,x+h]$ by Taylor's to that ...
0
votes
1answer
17 views

Round off to decimals

I m not sure about this problem. Pl help. 1. Roundoff this number to tenths place 87.952 2. Round off this number to hundredths place 75.195 As per me answer should be 88.0 and 75.2 for ...
0
votes
1answer
30 views

How to avoid cumulative rounding errors when calculating a result to a specific number of significant figures?

I have two operands and I want to calculate the result of an arithmetic operation (add, sub, mul, div, pow, sqrt, ln...) to S significant figures. How many ...
1
vote
1answer
48 views

How to generalize upper and lower bounds rules

I want to create a rule on finding the upper and lower bound of a number. Examples: For $29.8$ (3 s.f.) we write $1$ in the place of the $8$ and zero in every other position, giving us $0.01$. ...
0
votes
0answers
10 views

How to turn line around point of cross with other line?

my question seems to be very easy, but I can't figure it out. Maybe someone could help. I know rounding the line around the point $(P_x,P_y)$ is very easy, just like that: $f(x) = R(x-P_x)+P_y$ ...
2
votes
1answer
26 views

How to tell when a fraction does not end? [duplicate]

Is there a way in math / programming to tell if a fraction (reciprocal in particular) does not end? For example, 1/3 is 0.33 repeated, but 1/2 is just 0.5 Is there a way to find if 1/n for any ...
0
votes
1answer
9 views

How to round the mean number of people

When you are calculating the average of the number of people who do something, and you get a value with decimal points, should I still round up when the decimal point is > or = 5, and round down when ...
0
votes
0answers
52 views

Which is the correct one?

Here is an excerpt from the book 'Applied Numerical Linear Algebra' by James W. Demmel from SIAM But I have done it slightly different taking $\hat{l}_{ij}$ and $\hat{u}_{ij}$ and ended up getting $|{...
0
votes
2answers
115 views

A dollar amount $\${-}67.9{-}$ is divisible by 72. What is this number?

A client buys 72 turkeys. On the receipt, 2 digits are missing, the first and the last. So, the total price is $\${-}67.9{-}$. Find a way to make the price of each turkey round at the second digit ...
0
votes
0answers
51 views

Chebyshev polynomial: recursive formula error estimate

I am trying to solve Problem 3 & 4 from Numerical methods of Bakhvalov, Zhidkov, Kobelkov from section 2.8 on Chebyshev polynomials. If in the recursive formula $$ T_n(x) = 2xT_{n-1}(x)-T_{n-2}(x)...
0
votes
1answer
23 views

Multiplication after rounding, missing information of order of 1000, how to find if multiplication had multiple values for each data couple?

Imagine a financial report in a certain currency, indicating the following numbers (and below them, indicating the amount after conversion to USD, from the same report). 1,394,278 (USD46,743) ...
0
votes
1answer
57 views

How to compute rounding error bound?

I am using the IEEE754 half-precision floating point format, which has 11 significand bits. My input is drawn randomly with values between 1.0 and 2.0. I would like to approximate the maximum ...
2
votes
1answer
61 views

Should I use powers of $2$ when possible in computations?

For example, if I want the most accuracy and efficiency when performing millions of iterations wouldn't it be better to use $2^k$ -- as opposed to some other nearby number -- whenever possible, since ...
1
vote
0answers
105 views

Inverse function of round; solving equations involving round function

I am trying to solve the equation with which I came up from the accounting task: Round((0.844+delta)*1; 2)=0.89 or generally: ...
1
vote
1answer
38 views

What's the flaw in “collapsed rounding”?

A couple of years ago, I asked my high school math teacher the following question and she couldn't give me an answer. I had since forgotten about it, but now I'm curious to find the answer again. Let'...
0
votes
1answer
16 views

Interpolation with almost constant slope E

I have a curve $(\sigma_t,\varepsilon_t)$ described parametrically: Data here for (implicit) $t_i=(0,1,...n)$ I have two data series $\sigma_{t_i}=\sigma_1, \sigma_2, ... \sigma_n$ $\varepsilon_{...
0
votes
0answers
21 views

What are methods for correction of experimental measurements?

I have experimental data series collected from sensors, and I have some problems because the data is obviously digitally distorted. I don't know if it is a problem of rounding, or quantization, or ...
0
votes
0answers
40 views

Discretization Error amplification in a numerical scheme written in Python

I am using python for my numerical method projects and at this point I am facing a problem that can be related to the way I am translating the numerical problem with python. So I wrote a code on ...
0
votes
2answers
552 views

rounding up to two decimal points

I just have a question regarding my answers in my exam paper, my teacher said to give the answer to two decimal points, I got the correct answer (2.89) but I rounded it up to 2.90. Now my teacher did ...
3
votes
0answers
108 views

Error bound for floating-point interval dot product

In Handbook of Floating-Point Arithmetic (Birkhäuser, 2010, Chapter 6) Muller et al. presented the following absolute forward error bound for the floating-point recursive dot product: $$ \left|...
-2
votes
1answer
26 views

Maclaurin polynomial error bounds [closed]

I need some help with my Calculus II Maclaurin polynomial error bounds. $Mn(x)$ is the $n^{th}$ Maclaurin polynomial for $f(x) = e^x$. I need to use the error bound formula to determine a value of $...
1
vote
3answers
70 views

Find an alternative expression for $y :=cos(x+\delta)-cos(x)$

Explain the difficulty of computing $y :=cos(x+\delta)-cos(x)$ for small values of $\delta$. Find an alternative expression of $y$ that does not exhibit these difficulties. So far I have figured ...
1
vote
0answers
18 views

Find the number of significant figure in the approximate number $0.39865$ for the given relative error of it as $0.2\times 10^{-2}$.

Find the number of significant figure in the approximate number $0.39865$ for the given relative error of it as $0.2\times 10^{-2}$. Adding $0.2\times 10^{-2}$ to $0.39865$ gives $0.40065$ and it ...
1
vote
1answer
265 views

Chopping a number

Iv'e been trying to understand this really really simple concept of number chopping. Let's say that I have a system which is able to save decimal numbers with 3 significant figures and uses the ...
1
vote
0answers
50 views

How do I fight loss of significance and/or improve convergence for this recursive algorithm?

While trying to answer this question I used the series approach and obtained a recursive algorithm. While checking it numerically, I found it suffering from "catastrophic cancellation", i.e. loss of ...
0
votes
1answer
33 views

The error when we rounding down and sum in $\sum_i \lfloor n/i \rfloor$ vs. $\sum_i n/i$.

For some $x \in \mathbb R$ denote by $\lfloor x \rfloor := \sup\{ k \in \mathbb Z \mid k \le x \}$ the biggest integer smaller than $x$, the so called floor function. Then in my textbook it is ...
2
votes
1answer
22 views

Rounding fractions with less digits

The problem I'm trying to round a fraction $\frac{a}{b}$ taking away digits from one or both integers. Let's say a or b have 3 digits, and I want a new $\frac{a'}{b'}$ with as much as 2 digits for ...
0
votes
1answer
158 views

Rounding down values taken from an exponential distribution

Suppose that $Y_1,Y_2,...,Y_n$ are i.i.d. with distribution $exp(\lambda)$, and we define another set of random variables $Z_1,Z_2,...,Z_n$ where $Z_j=\delta \lfloor \frac{Y_j}{\delta} \rfloor$. I ...
0
votes
1answer
36 views

If $x$ is an approximate number, is $2x=x+x$ true?

Suppose I have a mass and I measured it. I found that it weighs $1.001 kg$. Since there may be error in my measurement process, the mass could be a few milligrams heavier or lighter. In grams, its ...
2
votes
2answers
71 views

0.5×2=1 but 0.5+0.5=1.0. Explain? [closed]

So when we multiply $0.5$ (approximate number) by $2$ (exact number), we get $1$, since our product must contain as many significant figures as $0.5$. When we add $0.5$ to $0.5$ (both approximate), ...
3
votes
3answers
59 views

How precise should an absolute error be?

While adding approximate numbers, our result must be as precise as the least precise number that was given to us. For instance, $101+1.001+1.0≈103$ because the number with the least precision viz., $...
0
votes
1answer
127 views

Recursive definition of the error

Given the definition of $I_n:=\frac{1}{e}\int_0^1 x^ne^x$ for $n=0,1,2,...$ there is a recurrence relation: $I_n=1-nI_{n-1} $ for $n=1,2,...$ and $I_0=\frac{e-1}{e}$ I've got to show ...
1
vote
1answer
507 views

Accuracy in multiplication; precision in addition

Why is it that while multiplying approximate numbers we are concerned with significant figures (accuracy) and while adding we are concerned with decimal places (precision)? Any explanation would be ...
1
vote
0answers
22 views

Worst case scenarios for companion matrix method

In what case is the companion matrix method for computing the roots of a polynomial expected to suffer accuracy loss. Are nearby roots the main issue?
2
votes
1answer
128 views

How to approximate relative error further?

Background and original problem: I want to minimize the difference (error) of a numerical derivative approximation of a function and its true derivative. Let $f(x) = \sin(x) $ be the anti-derivative ...
1
vote
1answer
200 views

How do I calculate standard deviation and $\bar{x}$ if radius of circle is given?

It is given radius of circle $r = 5cm$ and standard deviation $\sigma = 1mm$. How do I calculate $\bar{x}$ and standard deviation of calculated circle area? I know these formulas but I dont know how ...
0
votes
1answer
43 views

Rounding(A+B) == Rounding(A) + Rounding(B)?

Could I say Rounding(A+B, n) == Rounding(A, n) + Rounding(B, n) ? For example: ...
0
votes
1answer
592 views

Computing Absolute and Relative Error Using Double Precision

Relevant definitions: If $\tilde a$ is an approximation of $a$, then the absolute error of $a$ is $$ |\tilde a - a| $$ and the relative error of $a$ is $$ \frac{| \tilde a - a|}{a} .$$ Statement of ...
0
votes
1answer
46 views

Reversibility of Integer Scaling

So this problem originates from thinking about scaling an pixel coordinate from one resolution to another and back again. Is that operation truly reversible, or is it possible for the pixel coordinate ...
0
votes
0answers
303 views

How to round numbers when performing linear interpolation?

Say I have to find the number of people above a certain mass. I can do linear interpolation to get a value that will give me the number of people below this mass and I can simply subtract this from ...
1
vote
1answer
46 views

Where $x \in [1,2]$, find an upper bound on $|1-fl(x*fl(1/x))|$

Where $x \in [1,2]$, find an upper bound on $|1-fl(x*fl(1/x))|$ Where $fl(x)$ rounds a number to the nearest IEEE 64-bit approximate encoded value. My Work 64-bit IEEE has a 52-bit fraction with an ...
3
votes
1answer
331 views

Homework Problem on Absolute + Relative Error [closed]

Let $f \in C[a,b]$ be a function whose derivative exists on $(a,b)$. Suppose $f$ is to be evaluated at $x_0$ in $(a,b)$, but instead of computing the actual value $f(x_0)$, the approximate value, $\...
0
votes
0answers
51 views

floating point system

Need to check my answer which I'm not confident about my solution. The question based on Floating point systems, Roundoff Error. Please correct me if you think my answers are wrong, my question is on ...