Questions tagged [floating-point]

Mathematical questions concerning floating point numbers, a finite approximation of the real numbers used in computing.

Filter by
Sorted by
Tagged with
-2 votes
0 answers
38 views

How can I tell which is closer to 0.72 without converting the binary numbers to decimal? [closed]

If I have two binary floats: $0.10111000_2$ and $0.10111001_2$ How can I tell which is closer to 0.72 without converting the binary numbers to decimal (base 10)?
user avatar
0 votes
0 answers
36 views

Polynomial Curve Fit without floating point

big math dummy here hoping to get some advice. I'm working on a closed loop servo system that requires a curve fit on some feedback. The controller for this system is $16$-bit. With the help of excel ...
user avatar
0 votes
0 answers
50 views

The gap size of floating point

In Trefethen Bau Numerical Algebra, floating point set F is defined by $\textbf{F} = \left\{\pm(m/\beta^{t})\beta ^{e}| 1 \leq m \leq \beta^{t}, e \in \mathbb{Z} \right\}$ Equivalently, by making $m$ ...
user avatar
  • 11
13 votes
1 answer
2k views

What's special about the number $1.000000015047466$E+$30$?

I'm a programmer by trade by I've run into a weirdly special number and need some help deciphering its significance. I was writing some machine learning code that compiles into GPU kernel code and the ...
user avatar
0 votes
0 answers
14 views

What is the smallest positive number represented in the floating point system F(5,3,4,4) in decimal base?

I know this is easy but I have a feeling something is not right We have the base 5. 3 is the number of significant digits and the 5 varies from -4 to 4. As the exercise asks for the smallest positive, ...
user avatar
  • 313
1 vote
0 answers
26 views

Math behind the floating-point 2e-15 and 5e-323 in python unittest function rassertAlmostEqual [closed]

In python, there is a function rAssertAlmostEqual to check whether two floating-point numbers are almost equal. https://github.com/python/cpython/blob/main/Lib/test/...
user avatar
0 votes
0 answers
55 views

Condition number for the sum of two numbers

The condition number for the sum of two numbers $a$ and $b$ is $K=\frac{|a|+|b|}{|a+b|}$. But if I have $a=10$ and $b=-10$, I know that their difference is exactly $0$, so I have in practice no error, ...
user avatar
1 vote
1 answer
31 views

Rounding error of matrix multiplication when one of the matrices is orthogonal

I am studying Scientific computing from Biswa Nath Datta's Numerical Linear Algebra and Applications and there is a corollary after explaining matrix multiplication rounding error described below. if $...
user avatar
0 votes
2 answers
44 views

Exact number representation to represent time from $1$ nanosecond (ns) to $1000$ seconds (sec)

I need help figuring out the exact number representation to represent time from $1$ nanosecond (ns) to $1000$ seconds (sec) with an accuracy of $1\%$. I know a floating-point is preferred over a fixed ...
user avatar
  • 341
0 votes
0 answers
71 views

Stability and Backward stability of arithmetic and series: Numerical Linear Algebra (Trefethen and Bau) Exercise 15.1

I can't post image with below-10 reputation. The image of the cited definition/problem description in the book is uploaded to Imgur and labeled as [Image:$\dots$]. Sorry for the inconvenience. Context ...
user avatar
0 votes
0 answers
23 views

How can IEEE 754 denormalized numbers result in "significant error"?

This Microsofts manual page for Excel says following: Cases in which we do not adhere to IEEE 754 Denormalized numbers: A denormalized number is indicated by an exponent of 0. In that case, the ...
user avatar
0 votes
2 answers
43 views

How to deal with $e^{-x}$ when $x$ is too big ($x \gg 0$) and therefore $e^{-x}$ gives us a "prohibitive" value (computer context)?

I don't know how to formulate my question correctly but I'm going to try: Suppose the following: $\textbf{a} = e^{-101}$ $\textbf{b} = e^{-98}$ $\textbf{c} = e^{-97}$ $\textbf{W} = (\textbf{w}_0,\...
user avatar
0 votes
0 answers
72 views

Number of FLOPs for the inner and outer vector product?

I'm trying to educate myself on the relative cost of the cross and dot products relative to the number of floating point operations (FLOPs) each one requires. My understanding based on this paper (see ...
user avatar
0 votes
0 answers
37 views

How are results guaranteed in algorithms that include intermediate approximation?

Let me elucidate this vague question with an example. Consider for example the following Gauss sum of roots of unity $N=(e^{2\pi i/5} + e^{2\pi i/4} e^{4\pi i/5} + e^{4\pi i/4} e^{8\pi i/5} + e^{6\pi ...
user avatar
  • 1,482
3 votes
2 answers
90 views

Cancellation Problem in Borwein's quartic Algorithm for computing $\pi$

In the center of Borwein's Algorithm from 1985 occurs a term $$T_n = 1-\sqrt[4]{1-y_n^4}$$ with $y_n \to 0_+$. For small $y_n$ you need 4 times the precision of result $T_n$ due to cancellation. With ...
user avatar
18 votes
3 answers
955 views

Polynomial for very large number of roots

Update: Please also see this solution here provided by MattL. I have the roots for a very large order polynomial (>100), and from those alone wish to recreate the polynomial and run into numerical ...
user avatar
0 votes
0 answers
62 views

How to Add the IEEE 754 single-precision floating-point numbers from hexadecimal?

Question: Add the following IEEE 754 single-precision floating-point numbers. (a) C0123456 + 81C564B7 (c) 5EF10324 + 5E039020 I know I first need to convert to binary, then add and later change into ...
user avatar
0 votes
1 answer
50 views

Calculating the flops for triangular system

My lecture notes state that for a lower triangular system, $\left(a_{i j}=0\right.$ if $\left.j>i\right) n^{2}$ flops are required to solve it, through the process below. $$ \begin{aligned} x_{1} &...
user avatar
  • 109
0 votes
1 answer
54 views

What is the meaning of $1$ in a relative error?

If we measure a length and is measured as $12.5$ meters long, accurate to $0.1$ of a meter this means the absolute error is $0.05$m. The relative error is: $\frac{0.05}{12.5} = 0.004$. This means that ...
user avatar
  • 1,539
2 votes
1 answer
217 views

Correct comparison of real number for n digits precision (absolute vs relative difference)

To compare if $2$ real numbers are equal, we define a desirable precision e.g. $n$ digits and then check if the following condition holds: $-\frac{1}{10^n} \lt x - y \lt \frac{1}{10^n}$ Now I was ...
user avatar
  • 1,539
0 votes
1 answer
65 views

Why increase the mantissa while converting float to hex in Python

This is essentially a coding question, but I'll try my best to convert the code to logic here I'm trying to implement float to hex by referring the Python's inbuilt float.hex Here is a brief of how it'...
user avatar
3 votes
1 answer
111 views

Solveablity of Diophantine equation over "computer numbers"

Hilbert's tenth problem asks whether there is an algorithm to determine if a given solution set to a Diophantine equation is non-empty. There is no such algorithm. In practice for many engineering ...
user avatar
0 votes
1 answer
94 views

Comparing finite-precision floating point rounding errors of $e^{-x}$

Given the following 2 series approximations for $e^{-x}$, which is more prone to finite precision f-p rounding errors? \begin{equation} e^{-x}=1-x+\frac{1}{2}x^2-\frac{1}{6}x^3+...\\ e^{-x}=\frac{1}{1+...
user avatar
  • 377
1 vote
0 answers
36 views

Sequence in floating point arithmetic.

I'm working in some topics of Numerical Analysis and I founded a problem that sounds so interesting but I don't really know how to do it. The problem: Let $a=(+.b_1b_2\dots)\times \beta^{p}$, $0<...
user avatar
3 votes
0 answers
76 views

Limit of Euler's number failing due to precision errors - A surprising case. Why does it happen?

It is a known fact that floating point precision errors are bound to happen when one forces a computer to deal with very large or very small numbers, especially when both things are done at the same ...
user avatar
0 votes
0 answers
45 views

In a floating-point system, is the unit roundoff $\epsilon_{mach}$ necessarily a machine number?

I have to answer the following questions: (a)In a floating-point system, is the unit roundoff $\epsilon_{mach}$ necessarily a machine number? (Explain your answer or give a counterexample). (b) Is ...
user avatar
1 vote
0 answers
48 views

Floating point arithmetic error propagation

I have a function $$y=ln(\frac{x_1}{x_2})$$, $$x_1, x_2 > 1$$ and two ways to calculate it: $$v = \frac{x_1}{x_2} => y_1=ln(v)$$ $$v_1 = ln(x_1)$$ $$v_2 = ln(x_2)$$ $$y_2 = v_1 - v_2$$ How ...
user avatar
3 votes
0 answers
89 views

Relative error when chopping is used of a 32 bit floating point number in IEEE 754 format

Find the binary representation of 85.125 using IEEE 754 standard 32-bit floating point number presentation. Find the relative error if chopping is used. I calculated the binary representation of 85....
user avatar
  • 57
1 vote
1 answer
76 views

What is the ULP variance of the common implementation of lerp?

Sort of a spiritual successor to Accurate floating-point linear interpolation. Using $\oplus$, $\ominus$, and $\otimes$ to represent IEEE-754 addition, subtraction, and multiplication respectively, ...
user avatar
  • 121
0 votes
0 answers
32 views

Floating point numbers multiplication and error

I do not understand how $(1+ε')(1+ε'')(1+ε)$ is (almost) equal to $1+|ε'+ε''+ε|$.
user avatar
1 vote
3 answers
82 views

Complex floating point types

I'm implementing my own programming language for fun, and have been playing around with the implementation of various floating point numeric types. I'd like to think about implementing a numeric type ...
user avatar
1 vote
1 answer
105 views

Is there an icon latin, glyph or symbol that means rounding?

Is there an icon that means rounding? I'm not talking about notation but specifically an icon, glyph or symbol. For example, "pi" has the icon $\pi$. I would also accept an icon that means ...
user avatar
2 votes
1 answer
127 views

Avoiding catastrophic cancellation in $1-\operatorname{sinc}x$

When I try to calculate the function $f(x)=1-\operatorname{sinc}x$ for small values of $x$ I get large relative errors due to catastrophic cancellation. I want an accurate way to calculate $f(x)$ ...
user avatar
1 vote
1 answer
74 views

Numerical method to solve difference equation

I am solving the following problem: Write a program to generate the first 60 terms in the sequence given by the difference equation: $$x_{k+1}=2.25x_{k}-0.5x_{k-1}$$ with starting values $x_1=\frac{1}{...
user avatar
  • 688
0 votes
1 answer
143 views

Cancellation in the exponential function

I have the following issue about numerical analysis: Well, part a) is the one I did, in the following code in Octave ...
user avatar
  • 688
1 vote
1 answer
34 views

Using the rounding function to compute $x=0.21045$ should the output be $0.210$ or $0.211$ within a certain set of floating points?

I'm self-learning LA through an online book. And wanted to clear up my confusion regarding the Rounding Function. I'm finding it hard to describe the problem, so I first started off by describing how ...
user avatar
  • 35
2 votes
3 answers
83 views

Floating point division resulting in a value exceeding 1 but should be equal to 1...maybe!

I am computing the apparent magnitude of comets and minor planets using data from the Minor Planet Center. The formula I use has a division and that result is passed to the arccosine function (in ...
user avatar
2 votes
4 answers
173 views

Computing continued fraction expansions

My question concerns the numerical accuracy of a continued fraction expansion. A typical algorithm for computing a continued fraction can be written in Python as : ...
user avatar
  • 143
0 votes
1 answer
34 views

Approach to address errors from float precision limits by its storage limit

Background In computer, the limited float precision due to the storage limit e.g. 64 bit can cause problems. Trying to understand what approaches are available and being used to cope with or overcome ...
user avatar
  • 235
5 votes
2 answers
288 views

Mathematical constants and approximations of irrational numbers

I found two examples where various constants have some surprising properties, related to the approximations of real numbers (you can convince yourself with Wolfram Alpha): The real number $\pi^{\pi^{...
user avatar
2 votes
0 answers
78 views

Backward Stability solving linear system with unitary matrices

Let $Q \in \mathbb{C}^{n\times n}$ be a unitary matrix that can be exactly stored in floating point arithmetic. Suppose we want to solve the following linear system: \begin{equation} Qx=b \end{...
user avatar
1 vote
1 answer
174 views

0.1+0.2=0.30000000000000004 in floating point IEEE representations arithmetic addition manual proof.

I am trying to prove the 0.1+0.2=0.30000000000000004 by adding the numbers 0.1, and 0.2 in IEEE floating-point representation. I have added the operand values in IEEE format, with the general addition ...
user avatar
2 votes
2 answers
132 views

How can I simplify this lerp arithmetic to avoid floating point precision errors?

I am making a "cinematic camera" that pans around a scene. Each step has a position for the camera and a point that the camera should be looking at. For instance: ...
user avatar
  • 43
1 vote
0 answers
67 views

Proof of accuracy of converting decimal to binary floating-point number to decimal

As mentioned in the book, "we may think of its value not as exact but as exact within a factor of $1+\epsilon$. Thus for example, IEEE single format numbers are accurate to whitin a factor of ...
user avatar
  • 308
2 votes
1 answer
102 views

How to improve the precision of this approximation

I have to approximate $\pi$ by using $\pi=\lim_{n\to\infty}z_n$ with $z_2=2, z_{n+1}=2^{n-1/2}\sqrt{1-\sqrt{1-4^{1-n}z_{n}^2}}, n=2,3,...$ How can I improve the accuracy of this approximation when ...
user avatar
4 votes
1 answer
333 views

Bisection method with geometric mean

The bisection method is a well-known method for root-finding. Given a continuous function $f$ and an interval $[a,b]$ where $f(a)$ and $f(b)$ have opposite signs, a root can be guaranteed to be in $(a,...
user avatar
-2 votes
1 answer
33 views

A question from floating point number representation.

The numbers in a floating-point system are defined by a base B, a mantissa length t, and an exponent range [L, U]. A nonzero floating-point number x has the form x = +/-(o.b1b2.....bt)B^e ---1 then ...
user avatar
0 votes
1 answer
504 views

machine epsilon value for IEEE double precision standard alternative proof using relative error

From the textbook, I know that the machine epsilon number for IEEE double precision standard $F(\beta=2, t = 53, L = -1022, U = 1023)$ is: $$ \epsilon_{M} = 2 \mu $$ where $\mu$ is the unit round-off ...
user avatar
  • 481
0 votes
1 answer
30 views

With what exponent rules does $-(2^{-55} + .4\times2^{-56})+(2^{-54})$ become $.1\times2^{-52}$?

I have a math class focusing on numerical analysis so I'm working with very small numbers. My professor has set $0.4 − 2^{−55} − 0.4×2^{−56} + 2^{−54} = 0.4 + 0.1×2^{-52}$ but shown no steps in ...
user avatar
0 votes
1 answer
51 views

Converting decimal number to a floating point system

I am trying to convert $1.5\times10^{8}$ to a normalized floating point system defined as $0.d_1d_2d_3...d_t\times B^c$ for $t=20, B=2$ I don't know how I can change my base 10 to a base 2 without ...
user avatar

1
2 3 4 5
9